by Dov Rosenberg –
Consumers in this Instagram/Amazon/Google age have many different expectations of the applications that they use in every aspect of their lives than they did a few years ago. Digital transactions have become the lifeblood of everyone’s daily activities. Digital commerce has replaced the traditional telephone dial tone as the metaphor for “always on, always ready when you need it”.
Twenty years ago (2000 – seriously?) software quality was focused on making sure the application didn’t cause a blue screen of death on your Windows 2000 laptop with 2 gigs of RAM. The standard of excellence was low – people were happy that they were able to save their important documents before their word processor became unresponsive and had to be rebooted and lose all of their unsaved changes. Web applications sucked a little less because the back end applications were very limited in the types of transactions that could be performed. We never worried about data security because we would never have dreamed about using our credit card to purchase something on a website. Instead, we would call an 800 number and give our payment information to a random person over the phone.
Fast forward to the present day – every aspect of our lives has been taken over by technology. Ranging from our Ring™ doorbells to our NEST™ thermostats, Alexa enabled speakers to smartphones, cars, refrigerators, and more. The last time I checked my wireless router at home to see how many devices were using it, I was shocked – 27!! It seems like a lot for 2 people and 3 dogs. Right now, I have an Apple Watch, laptop, smartphone, tablet, and Kindle e-reader in my backpack. My son took me out for a drive in his Tesla Model 3 recently and showed off the autopilot functionality while we were driving down a major highway full of traffic and road construction – worked like a charm.
My web/virtual life has taken over my real-world experience as well. I had to start using a password manager application to track my password information because I could not keep track of nearly 300 different accounts for the various online services that I use regularly. Services like Amazon, Netflix, Hulu, and Apple iTunes have replaced visits to the mall, department stores, movie theaters, and grocery stores. My millennial-aged kids haven’t done any Christmas shopping in a brick and mortar store in several years now. I know of several couples in our social circle that met each other over various dating apps and are now married. I haven’t balanced my checkbook in over 10 years, my bank balance is updated in near real-time, and I get phone calls and text messages if something is purchased that is outside of my regular purchasing habits.
Bottom line – quality has a much different meaning in 2020 for the average consumer than it ever has had in the past. Our expectations from the products and services that we use are very high. If it doesn’t work and make our lives easier, better, faster, or cheaper – we’ll replace it with something that will. Today – Quality is measured by the value that a product or service brings in exchange for the level of effort required to use it. Everyone expects things to work. Period.
Today, the definition of quality in terms of Consumer-facing products has blurred the lines between various IT-related disciplines. Security, Product Management, and Infrastructure are no longer independent of the Quality organization. There are several aspects to Consumer quality that are important:
Product Quality seems to be reasonably obvious – everything should work as intended without unintended consequences. For Consumer Products and Services, product quality goes well beyond the classical definitions. The quality of customer support is critically important. If a consumer can help themselves solve problems efficiently and accurately – the reputation of the company is higher, and their products are perceived to be higher quality.
Product quality is also perceived based on the ease of use of a product or service. Having products that work well but that is hard to use provides a negative impression of the overall product quality. Steve Jobs famously said “Simple can be harder than complex; you have to work hard to get your thinking clean to make it simple.”. Apple’s products are consistently rated as high quality based on their user-centric designs.
Our online lives depend on the security of the data that we generate and provide. Without the perceived security of payment services like PayPal and Apple Pay – online commerce would essentially cease to exist. Modern applications are no longer monolithic tightly integrated applications – they are multi-tiered service-oriented architectures running in cloud-based data centers. Every boundary between applications and services is a possible point of intrusion or security breaches. It is no longer possible to only secure the customer-facing endpoints of an application and assume that nothing wrong will happen inside the infrastructure.
Data has to be secured while in transit, while at rest, and while it is being transformed. Network infrastructures need to be kept up to date with security patches, and configurations need to be well documented and secure, log files need to avoid writing sensitive data to unsecured files. Encryption keys need to be well managed to avoid being compromised or lost.
Consumer quality programs MUST include robust data security testing capability. A single security lapse could cripple a company from both a financial and reputation perspective.
Consumers have lost their patience with slow-performing services and applications. In 2018, Google released the results of a study on web site performance that was very telling:
“The average time it takes to load the average mobile landing page fully is 22 seconds. However, research also indicates 53% of people will leave a mobile page if it takes longer than 3 seconds to load.”1.
Popular digital services have to be able to support millions of concurrent users at a time. This requires a robust architecture, a complex infrastructure, and great attention to detail because even the smallest performance issue can be significantly magnified when it is overloaded. Poor performance is a leading cause of customer dissatisfaction with products and services.
Testing large scale applications can be expensive and complicated. The hardware requirements needed to support 1M concurrent users could require upwards of 20+ servers and other networking hardware. Load testing software is notoriously expensive as well. In addition to testing under peak load, it is essential to test under sustained loads. Memory leaks and CPU utilization are critical metrics to understand to support large scale loads.
Mobile applications add an additional burden on quality from a performance viewpoint. Applications that drain a battery or consume too much bandwidth quickly get flagged as low quality and can add another strain on the overall product quality.
This is similar to product quality in that it is a measure of the quality of the product roadmap that a company is pursuing. Apple has become very successful by designing products and services that integrate well together across devices and services. I can take a picture on my iPhone, and it will automatically sync up to my iCloud account, and I can embed it in a presentation I am working on with my laptop in a matter of minutes without any additional effort on my part.
This high level of integration requires a strong commitment to building comprehensive test automation capabilities that allow seamless testing across devices, platforms, and services. Having a great product roadmap, but a buggy implementation, is the fastest way to alienate customers and waste a lot of financial resources chasing a dream that will never be achievable.
The expectations that consumers have on product quality for the services that they use in their private lives carries over to the products that they use in their work lives. These expectations get pushed onto enterprise applications in their workplaces and their interactions with their clients. The standards for success are very high, as well as the impacts of not meeting those expectations.
1. think with Google "Find Out How You Stack Up to New Industry Benchmarks for Mobile Page Speed" Retrieved from https://think.storage.googleapis.com/docs/mobile-page-speed-new-industry-benchmarks.pdf