ANAB: QMS and EMS Outputs Matter!

The ANSI-ASQ National Accreditation Board (ANAB) recently issued a notice titled, “Outputs Matter!” Since ANAB is the U.S. accreditation body for ISO 9001 and ISO 14001 registrars (certification bodies), we need to pay attention to their advice.

The ANAB notice said for clients with a certified Quality Management System (QMS), the expected outputs should be trends of improving:

  • customer satisfaction,
  • product and/or service conformance, and
  • continual improvement.

For clients with a certified Environmental Management System (EMS), the expected outputs should be trends of improving:

  • environmental performance,
  • legal compliance,
  • pollution prevention, and
  • continual improvement.

The purpose of their notice was to encourage an increased focus by certification bodies (CBs) during audits on the outputs (results) of certified clients. Early in every audit of a client, the audit team should review information regarding the trends of the expected outputs. When the information indicates no improvement or, even worse, a negative trend, the audit team should determine what the client is doing to identify why its management system is failing.

Users of certified organizations are saying with increasing emphasis that these firms are not delivering the expected outputs. These users are even beginning to question the value of accredited certification. As a result, ANAB accreditation auditors (during their office and witness audits of CBs) will increase their focus on what is being done to ensure the CB auditors have a consistent and appropriate focus on outputs during their audits of certified clients.

The ANAB notice included the EMS and QMS white papers below that elaborate on the parts of the ISO standards, ISO guides, and IAF guidance that support this focus on system outputs.

Outputs matter in EMS Audits


When ISO 14001 was introduced in 1996, a key concern of non-governmental organizations and regulators was the lack of performance requirements in the standard. Compliance with law was (and is) not required for registration. Specific performance requirements for air, water, and waste management were also not prescribed.

With the introduction of ISO 9001:2000, the focus shifted from simple conformity to the effectiveness of the management system. The EMS focus also shifted. There is now an expectation that the output of the EMS includes an improved level of compliance with legal requirements and improvement in the level of environmental performance.


The expectations of an effectively implemented EMS are stated in the introduction to ISO 14001:2004: “[An EMS] enables an organization to … take action as needed to improve its performance …” The standard requires top management to commit to continual improvement, prevention of pollution, and compliance with legal and other requirements (subclause 4.2). The standard also requires an organization to evaluate and prioritize its environmental aspects and impacts, establish objectives and targets, and define and implement the means for their achievement (subclauses 4.3.1 and 4.3.3).

The organization must evaluate its compliance with legal and other requirements (subclause 4.5.2) and take corrective and preventive actions (subclause 4.5.3). The organization must review the results of evaluations of compliance with legal and other requirements, its environmental performance, its progress in meeting objectives and targets, the status of corrective and preventive actions, and opportunities for improvement as part of management reviews (subclause 4.6).

While EMS conformity assessment requirements do not discuss specific EMS performance expectations, the importance of an organization’s demonstrated commitment to compliance and the need to consider the organization’s environmental performance and compliance are underscored in ISO Guide 66 and the associated IAF guidance.

ISO Guide 66 and the IAF guidance provide several performance-focused criteria for the selection and qualification of audit teams (ISO Guide 66 subclauses a, c and d; IAF guidance G.4.2.11 and G.4.2.15). The IAF guidance discusses expectations regarding the information gathered during audits and surveillance that provides a basis for initial and continued certification, including information relating to environmental performance, compliance with legal and other requirements, continual improvement, and overall effectiveness of the organization’s EMS (IAF Guidance G5.3.21, G5.3.22, and G5.6.5 a, d and e).

ISO Guide 66 and the IAF guidance have been careful to avoid the EMS conformity assessment process developing into “compliance” auditing. However, the importance of considering compliance information as a basis for initial and continuing certification is recognized in ISO Guide 66 subclause and IAF guidance G.4.1.6 and G5.6.5 d, which emphasize the need for the CB to verify that a system is in place to assure compliance.

Expectations of CB Audit Teams

CB audit teams are in a position to drive a focus on outputs by a certified organization. The CB audit team should focus on environmental performance measures and compliance at the earliest opportunity during the audit.

The audit team should review environmental performance measures to determine if the organization shows improvement in all the measures based on the objectives and targets of the EMS. If the organization does not show improvement on any of the performance measures, the audit team should investigate to determine what element of the EMS might be allowing this finding to occur. The audit team should use the initial tour of the facility and subsequent tours to look for compliance issues. If any are discovered, the audit team should ensure management is alerted to the non-compliance.

The audit team should, as in the previous situation, investigate to determine where a failure in the EMS might be allowing the non-compliance to occur. The EMS elements to be investigated could include 4.3.2 Legal and Other Requirements; 4.3.3 Objectives, Targets and Programs; 4.4.1 Resources, Roles, Responsibility and Authority; 4.4.2 Competence, Training and Awareness; 4.4.6 Operational Control; 4.5.1 Monitoring and Measurement; 4.5.2 Evaluation of Compliance; 4.5.3 Nonconformity, Corrective Action and Preventive Action; and 4.6 Management Review. The nonconformance should be written against the appropriate requirement of the standard.


Criteria and guidance for the certification process emphasize the need for the CB to consider an organization’s compliance with legal and other requirements and its environmental performance as factors essential to demonstrating that an EMS is effectively implemented.

Outputs Matter in QMS Audits


The ISO 9001:2000 revision brought a major shift in emphasis on management commitment to and participation in an organization’s QMS. The revised standard contains specific requirements for establishing objectives and criteria for QMS processes for management activities, provision of resources, product realization, and monitoring and measurement of products, processes, and the QMS itself for achieving customer satisfaction and continual improvement.

If supply chain customers and other interested parties are to rely upon and receive value from the third-party certification system, it is imperative that these QMS processes provide outputs that meet customer, regulatory, and the organization’s own requirements. In plain words, outputs matter! To achieve these outputs, the inputs to processes must be complete and appropriate, and the processes must be robust. Feedback to ANAB, through witness audits, complaints, and their involvement at ISO and IAF indicates the objectives of this shift in emphasis of ISO 9001:2000 have not been consistently achieved.


Clauses 4 through 8 of ISO 9001:2000 require top management  to commit to its QMS by communicating to the organization the importance of meeting established requirements, establishing a quality policy and measurable quality objectives, conducting management reviews, allocating sufficient resources, and ensuring that customer requirements are determined and met, with the aim of enhancing customer satisfaction. Responsibilities and authorities are to be defined and communicated within the organization. Resources are to be provided, including competent personnel and a supportive infrastructure and work environment.

Product realization must be planned and carried out effectively and efficiently. Products, processes, the QMS itself, and customer satisfaction must be monitored and measured. Data must be analyzed to demonstrate the suitability and effectiveness of the QMS, including customer satisfaction and product conformity to established requirements. Decisions related to conformity of product and continual improvement of the QMS must be made and actions taken when results are not achieved. The organization is required to conduct internal audits that determine if the QMS is effectively implemented and maintained and conforms to the organization’s planned arrangements and to the ISO 9001 standard.

Expectations of CB Auditors

Top management of a certified organization is ultimately responsible for establishing and sustaining a customer-focused, process-based QMS in conformance with ISO 9001:2000. CB auditors are in an excellent position to instill through their auditing practices a driving focus by the organization on the intent and requirements of ISO 9001:2000 (and the concept of this paper) that output does matter. CB auditors should focus on the suitability and effectiveness of the organization’s QMS by probing the organization’s ability to determine necessary inputs, implement effective processes, and produce outputs that consistently deliver products that meet customer and regulatory requirements.

CB auditors should thoroughly audit the interaction and communication of the QMS processes both within the organization and with customers. Successful implementation of the QMS should lead to the establishment of appropriate planning, realization, and monitoring of processes and products to assure product conformance and continual improvement of the QMS. Where the goals and objectives of the organization and customer and regulatory requirements have not been met or the trends are negative, CB auditors should identify the underlying processes that are deficient. Where requirements of the standard have not been met, nonconformities should be written.

CB auditors should similarly focus on the processes of the organization’s internal audit function, corrective action, and preventive action to maintain the effectiveness of the QMS. While the CB auditor’s periodic surveillance gives an independent assessment of QMS status, it is ultimately the scope and depth of the organization’s ongoing efforts between such audits that assure long-term effectiveness. Internal auditors need to look at process inputs, outputs, and interactions with the same diligence as CB auditors.

CB auditors should probe the effectiveness of root cause analyses and corrective action applied by the organization to internally identified nonconformities of products, processes, and the QMS. Similarly, the effectiveness of handling customer returns and complaints should be probed to validate data on perceived customer satisfaction.


The consistent application of criteria in auditing an organization’s QMS with regard to achieving customer satisfaction, delivering conforming product, and continually improving the QMS is imperative. This is especially important to the continued reliance of the customer supply chain and other interested parties on the added value of the third-party certification system. To achieve the intended purposes of ISO 9001:2000, certified organizations must be held accountable. Outputs do matter!