How Do You Audit the Analysis of Data?

How do you audit conformity to ISO 9001:2000, clause 8.4, Analysis of Data? Or, do you audit it? Unfortunately, some auditors say they don’t, at least not directly.

We should start by recognizing the importance of the requirement; it is one of the eight principles of quality management. ISO 9000, Clause 0.2.g, lists the 7th principle as the “Factual Approach to Decision Making” and explains that effective decisions are based on the analysis of data.

Analyzing data is an essential activity for improving your system and its processes, as well as, your products and services. Data collection has no purpose if the data isn’t examined, evaluated, analyzed, and converted into proposals for decision making.

To judge conformity to the audit criteria, you must clearly understand the requirement.

Clause 8.4 begins, “The organization shall determine, collect, and analyze appropriate data to demonstrate the suitability and effectiveness of the quality management system and to evaluate where continual improvement of the effectiveness of the quality management system can be made.” 

The Guidance on Terminology resource at the ISO web site defines the term “analysis” as the breaking up of something complex into its various simple elements.

The reason to separate something into its elements is to determine either their nature (qualitative analysis) or their proportions (quantitative analysis).

Therefore, we analyze the data to show the quality management system is effective (achieving planned results) and to spot where improvements can be made. It is not enough to just collect the data, we must analyze it and reach some conclusions.

Also, Clause 8.4 links back to Clauses 4.1.e and 4.1.f, which state, “The organization shall monitor, measure, and analyze these processes, and implement actions necessary to achieve planned results and continual improvement of these processes.”

Clause 8.4 continues, “This shall include data generated as a result of monitoring and measurement and from other relevant sources.”

This requirement emphasizes that the data we collect, especially numerical facts, are to be included in the analysis.

As a result of the monitoring and measurement activities called for by ISO 9001:2000, Clauses 8.2.3 and 8.2.4, you will have collected a lot of data, which can be analyzed to indicate trends. Any trends you may find could suggest where there are problems in your quality management system and indicate areas where improvements are needed.

Clause 8.4 also links back to Clause 8.1, which states, “The organization shall plan and implement the monitoring, measurement, analysis, and improvement processes needed to demonstrate conformity of the product, ensure conformity of the quality management system, and continually improve the effectiveness of the quality management system.”

Clause 8.1 also states, “This shall include determination of applicable methods, including statistical techniques, and the extent of their use.” So, you must consider and identify the methods to be used for analysis. And, you may find that statistical techniques are useful tools for the analysis process.

Clause 8.4 concludes, “The analysis of data shall provide information relating to:

a) customer satisfaction (see 8.2.1),
b) conformity to product requirements (see 7.2.1),
c) characteristics and trends of processes and products including opportunities for preventive action, and
d) suppliers.”

The analysis of the “data” is to provide “information”, which means to convey the knowledge derived from the analysis. The results can be used to assess customer satisfaction, provide evidence of meeting product requirements, make decisions on corrective and preventive actions, and evaluate supplier eligibility and performance.

Analysis of data is also mentioned in Clause 8.5.1, Continual Improvement. It states that “The organization shall continually improve the effectiveness of the quality management system through the use of the quality policy, quality objectives, audit results, analysis of data, corrective and preventive actions and management review.”

Turning our attention to the corresponding Clause 8.4 of the ISO 9004 guidelines standard, it says that decisions should be based on an analysis of measurement data and collected information. You should analyze data to:

  • Assess performance against plans, objectives, and other defined goals
  • Identify areas for improvement and possible benefits to interested parties

It states that decisions based on facts require effective and efficient actions, such as:

  • Valid analysis methods
  • Appropriate statistical techniques
  • Making decisions and taking actions based on results of logical analyses, as balanced with experience and intuition

Analysis of data can help to determine the root cause of existing or potential problems, and thereby guide decisions about corrective and preventive actions need for improvement.

For an effective evaluation by management of the total performance of an organization, data from all parts of the organization should be integrated and analyzed.

The overall performance should be presented in a format that is suitable for different levels of the organization. The results of this analysis can be used to determine:

  • Trends; customer satisfaction (and satisfaction of other interested parties)
  • Process effectiveness and efficiency; supplier contribution
  • Success of performance improvement objectives
  • Economics of quality, financial, and market-related performance
  • Performance benchmarking; competitiveness

Back to our original question: How do you audit the analysis of data? Well, it can be hard to assess because ISO 9001:2000 doesn’t say what data should be collected or how to analyze the data. The requirement calls for analysis of the “appropriate” data.

Begin by looking at the results of the data analysis implied by other clauses:

  • Are they evaluating performance against their quality objectives (5.4.1)?
  • Do they know their current level of customer satisfaction (8.2.1)?
  • Are they determining the effectiveness of their key processes (8.2.2)?
  • Are they determining their level of conformity to product requirements (8.2.4)?
  • Do they track the performance of their existing suppliers (7.4.1)?
  • Are they examining trends to identify preventive actions? (8.2.3)

If any of this information is missing or incomplete, see if the data is being collected and not analyzed, or if the data is just not being gathered. Writing a more specific finding will help the organization focus on the appropriate data and its analysis.