Software Quality Costs

In decades past, the focus of quality was merely finding problems at the end of an assembly line and removing the defects before shipping to the consumer. If the product didn’t meet specs, it was either reworked or scrapped; both expensive options. This approach is prone to human error and rarely finds all defects. And, this quality control approach only identified the defects found through random sampling, but actually did nothing to determine the root cause of the problem for resolution.

If preventive quality measures and rework are deferred until the testing phase, the cost of change may be 40 to 100 times greater than if the defect was fixed when it was created. The testing stage has the least recovery time for show-stopper problems or unexpectedly large amounts of rework. This unpredictability becomes a large contributing factor to why projects miss their schedules.

Quality Message

In the post-World War II reconstruction years, Dr. W. Edwards Deming introduced a quality program that simultaneously controlled the production and quality processes. Unfortunately, the United States did not adopt these principles until the 1980s with the introduction of the Total Quality Management System.

Deming’s core message-that we should stop inspecting defects out of products and start building quality in-has remained. The common thread of various quality methodologies is that the project team will build quality into the system design and will address quality continually throughout the life cycle. The goal is to identify problems up front and early, allowing corrective action and quality prevention to take place to reduce the number of critical defects found at the end of the assembly line.

This goal can be met through software quality surveillance, which includes walkthroughs, peer reviews, inspections, and testing, as well as, any method that identifies quality problems, risks, and operational capability weaknesses as early as possible. Approaching quality in this manner provides early corrective action and promotes lower quality costs upfront and early, thereby reducing end-of-program cost overruns.

Risk Management

Today, we have gone a step further by identifying risks which may have the potential to change engineering requirements, operational capabilities, and the quality of the product. In the spirit of risk management, software developers can help prevent one of the most common causes of defects-ambiguous requirements-by writing comprehensive acceptance tests when recording each requirement. And, automating these tests, and running them as part of frequent integration builds, will help detect defects when they happen.

While common sense says that preventing defects or finding them when they are cheapest to fix is preferable to finding them at the end when they are many times more expensive, many software development projects fail to write tests upfront, do inspections, or perform frequent integration-despite the benefits.

Implementing quality processes is tedious, time-consuming work in most environments. And, time is money. There is document inspection and writing early tests for critical requirements at the beginning of a project. It is hard to keep the tests up-to-date as the requirements change and even harder when you realize that you have to inspect the tests. These strategies increase the cost of implementing quality and the return on investment is not always predictable with a high degree of reliability, especially when the requirements and design have not been frozen. Thus, it is mind-wrenching work to determine which of all the possible strategies for implementation will bring the best value to the project.

We’re far too focused on product delivery, not process capability. We’re too busy trying to get the product out the door. Granted, this is a market-driven phenomenon, but we’ll have to change that deadline-driven attitude to one of good processes. If you get the process right, the product will have a far better chance at success. Unfortunately, many IT professionals still don’t quite understand the concept of process management.

We do far too much pretending in software. We pretend we know who our users are, we know what their needs are, that we won’t have staff turnover problems, that we can solve all technical problems that arise, that our estimates are achievable, and that nothing unexpected will happen. Risk management is about discarding the rose-colored glasses and confronting the very real potential of undesirable events conspiring to throw our project off track.

Risk identification requires a look into the future as to the potential success of the program. The challenge lies in the identification of risk versus current problems. Current problems require attention and action even if the immediate remedy is to defer corrective action until later. Risks realized may require actions that lead a project team to proceed in a different direction altogether or canceling the project entirely. Project teams must accept some risks due to other requirements, conditions, assumptions, or constraints; however, if a project team chooses to completely ignore risk, they greatly increase the probability of project failure.

Economic Status 

A company’s economic status is a significant factor when deciding what process to implement to track quality cost. The profitable nature of the business can make it more difficult to convince management of the need to track COQ. For example, having more engineers and fewer quality assurance people on a project can be great for a company’s short-term financial success. However, if project staff members do not build in quality from the start, a greater reliance on product rework results.

Quality Problems

The organization will eventually pay for the inadequate quality as customers identify problems with the product or service. Engineering changes must take place before the customer deems the product usable. These engineering changes late in the development process may result in a product or service that does not quite meet the original intent on the capabilities delivery; this, in turn, can lead to lost business. If this happens on a recurring basis, the company may experience competitive and financial difficulties. If so, a company may be more open to performing an assessment in an attempt to get back on track.

Then, after collecting and analyzing data that reveals a quality problem, the company finally decides to track quality costs. This may also be the time when the company experiences total failure. They know something has to be done, but don’t have a well thought-out plan. They make knee-jerk decisions, such as simply canceling the project and not addressing the underlying quality problems in their processes; that, in turn, causes unintended conflict within the organization.

Cost of Quality

Not having a clear understanding of the actual value of Cost of Quality also hinders the adoption of quality processes. There has been a persistent misconception in the business community that the COQ is a cost over and above that of developing and producing a project to meet a specific and required outcome and schedule. The COQ, regardless if it is software or hardware, is the price of not creating a quality product or service.

If the development process was perfect with no problems and there was no possibility of substandard service, failure of products, or defects in their manufacture, then organizations would have no COQ expenditures. The Cost of Quality is the sum of costs incurred in maintaining acceptable quality levels plus the cost of failure to maintain that level (cost of poor quality), and typically ranges from 15-25 percent of total cost.

Quality is Free

Philip B. Crosby’s “Quality Is Free” concept, identified two main categories of quality costs: conformance costs (cost of good quality), and nonconformance costs (cost of poor quality).

Conformance costs include prevention and appraisal costs; nonconformance costs include internal failures, as well as, external failures. A defect found early in the project prior to customer delivery is termed an internal failure. A defect identified after the product has been deployed to the customer is an external failure. External failures can also include incompatibility of the software with legacy software installed in the field, or a lack of commonality between redundant systems.

Quality Knowledge

Beyond not clearly understanding COQ concepts, key decision makers in an organization may lack knowledge in determining quality costs and the principles for collecting quality costs. Without knowing what quality principles are, an individual or organization may have no idea where to place their focus to obtain quality costs. The organization can remedy this either by ensuring that a quality curriculum is included in the training for project staff and senior leadership, or by hiring a quality consultant to guide the organization.

Conclusion

At first glance, an individual might be prone to think that collecting quality costs is expensive, adding unnecessary costs to the product or project. Quality is not free, in that you have to make an up-front investment in time, money, and effort. However, if performed properly over the full life cycle of the project, you can recoup the resources expended for quality processes by avoiding rework later in the project life cycle.

By communicating the quality story in terms of dollars, you can enlist the help of senior management to infuse quality processes throughout the project life cycle and contain project costs for the long haul. Collecting quality costs is like project planning; it is cheaper to properly plan than it is to plan a little and fail a lot.

Note: This article was based on a November 2008 article in Crosstalk. For more on how to put a COQ system in place, see the article by George Webb and Nanette Patton,“Quality and Cost – It’s Not Either/Or: Making the Case With Cost of Quality”.