Photo by ThisisEngineering RAEng on Unsplash

A healthy software organization thrives on many items but none may be as important as Quality. Software Quality contributes to happy end-user experiences and provides greater psychological safety for product teams. Poor quality in software results in exponential cost factors and, sometimes, catastrophic outcomes, such as loss of users and revenue, engineering apathy, and reduced employee morale.

While the concept of testing dominates the conversation around Quality in software, it is only one part of a comprehensive Quality approach. Quality-driven companies bake behaviors into the entire software lifecycle to promote well-made products. It’s not just about unit testing or manual QA testing; it’s about instilling a culture of Quality within every person’s mind across the organization. When product teams have the right behaviors, desirable outcomes abound, such as:

• Increased user confidence in the software, leading to increased engagement
• Psychological safety for the product teams, knowing they can safely deploy new features, even on Friday
• Increased number of deployments, leading to more user engagement on new features and more learning
• A culture of Experimenting to Learn on product teams, leading to more innovative solutions

Steering a product organization to a culture of Quality requires solid change management. Leadership buys in and drives the process; the organization’s vision emphasizes Quality; teams onboard slowly at first, but more rapidly as people learn and adapt; the people doing the work drive improvement.

Leadership Sets the Vision

Rather than impose a “we’re going to do more testing now” message to product teams, leadership needs to emphasize a new focus on Quality, driven by feedback and data. This vision highlights how Quality is everyone’s job and sets forth the key results for the teams' goals. Example:

Our teams deploy quality software frequently with minimal change failures, experimenting to learn, and adapting our products to meet the user both today and in the future.

The above statement makes frequently deploying quality software “Job #1”, but not at the expense of Quality (“minimal change failures”), and it elevates Experimenting to Learn into a first-class citizen. Product teams use this vision as a “north star,” tracking metrics that cascade from the vision’s intent, such as:

• Number of production deployments over a period of time (month –> week –> day)
• Change failure rate (the number of times a fix for new deployments is necessary) < 10%
• A/B Testing campaigns for new, data-proven user features with tangible results

These metrics become Objectives and Key Results (OKRs) for the team that support the vision. In this example, Key Performance Indicators (KPIs) cascade from the OKRs down to the components of product development, such as:

• Automated test coverage
• Deployment time (how long does it take to deploy new software)
• Mean Time to Restore
• Adoption of an A/B testing platform
• Canary or Blue/Green deployments
• Smaller deployment batches

The KPIs drive realistic but ambitious goals set by the team. The impact of the team is clearly measured and understood, and can be traced all the way back to the company ( vision.

Focus on these metrics also bolsters positive team behaviors such as frequent user testing, Test-Driven Development (TDD), pair programming and code reviews, and automated static analysis of source code. Teams begin to appreciate the need to design software for high testability, which helps drive these metrics.

Onboarding Teams

As with any organizational change, the rollout is methodical. The first team starts with tools, frameworks, and metrics. As they evolve with the new process, standards can emerge, such as:

• Testing frameworks
• Development tools
• Testing policy (coverage, test time, etc.)
• Automated deployment

These standards form the beginnings of a platform for the company. Subsequent teams onboard by using these learnings, further enhancing the standards and evolving governance around product quality. Collaboration is vital in this stage, and using approaches like “I do, we do, you do” as one team onboards the next creates a kind of Von Neumann process that can replicate “in the wild.”

Starting new products and onboarding new team members is simpler as the platform has removed a barrier to entry and focus is on meeting user needs instead of setting up environments.

When discussing testing, for example, there are many ways to test, each promoting different outcomes. For example:

• Unit testing/TDD drives component design and correctness
• Integration testing automates testing from the user perspective
• Load testing focuses on scalability and resilience
• Penetration testing promotes security
• Fuzz testing searches for holes in the software
• A/B testing enables experimentation and learning

Leadership and Quality champions build a perspective for the organization’s test strategy, addressing each test method and evangelizing the new Quality gospel. However, don’t fall into the trap of letting testing dramatically slow down deployment. A code freeze for weeks so pen testing can happen works against everything in modern product development.

Teams Drive Improvement

As the culture of quality blazes through the organization, the people doing the work learn how to meet their goals more effectively. Since their goals cascade from the vision, they are motivated to share improvements, making local learnings into global knowledge.

Setting the KPIs for teams means their focus is not solely on cranking out features. They are measured on quality and outcome: low defect rate, frequency of deployment, low change failure rate, happy users and higher engagement, etc. As such, teams want to improve how they work, not just what they create. When a KPI starts to slip, empower the teams to pull the Andon cord, swarm the issue, fix it, and go. Now, instead of going through the motion of best practices, the teams can see what behavior actually meets their stated goals.

Conclusion

Product development is a journey without end. There are waypoints along the route, but the destination is always moving. Focusing on Quality is a focus on the journey. If the team travels well, the rest of the trip takes care of itself.

Finally, some concepts advance quality product development effectively for any product team. Examples are:

• Automation - the vast majority of testing (and other repeatable tasks) is automated.
• Work in Small Batches - Big bang deployments create significant problems that are hard to track down. Therefore, strive for incremental, smaller deployments.
• Minimize Deployment Pain - Software that is hard to deploy will reduce deployments, smother opportunities to learn, and erode employee morale.

Quality is not solely a culture of testing; it touches every aspect of product development. It’s not a revolution; it’s an evolution. Set a Quality-focused vision, start small, and grow your culture through behavior and improvement.