Research Shows E-Commerce Would Benefit From Greater Testing Maturity

Does your company rely on E-commerce applications to reach your customer base, your partners or suppliers, or to otherwise deliver important business services? If you answered ‘yes’, you should know that this characteristic identifies you as an ‘at-risk’ candidate for less effective change management and testing. How’s that you say?

A recent study on testing maturity for applications and operating systems conducted by StackSafe and Research Edge reveals that applications heavily influence the approach companies adopt for change management and testing. This is in part due to differences in application complexity, change lead time, and downtime costs. It is also determined by the unique nature of the application and the manner of organizational operation.

From our earlier blog post three application profiles have emerged from the manner in which applications influence change management and testing.

One profile, the High Stress Environment, is characterized in part by companies who’s IT operations departments must support E-commerce applications.

E-commerce Applications and Downtime

Let’s take a closer look at the impact of E-commerce applications on change management and testing. One important characteristic from the study jumps out: companies that rely on E-commerce typically experience a higher percentage of emergency changes than companies running other application types.

As we’ve discussed previously in regard to testing maturity, companies experience more problems in production when they require IT operations to address a higher number of unscheduled (e.g. emergency) changes. True to form, our most recent study shows that over 25 percent of companies supporting E-commerce applications test less than 50 percent of all changes. As web applications continue to grow in both users and complexity, more outages (a la Yahoo’s Cyber Monday snafu) may become commonplace. Don’t forget that:

“With an ecommerce website or an advertisement supported website, this downtime means lost revenue and worse yet, lost customers.”

Because of the nature of E-commerce applications, IT operations appears to face more pressure in making changes to multi-tier applications. More than half (52 percent) of all E-commerce companies support in-house software development, so expectations are likely higher for short development-to-deployment cycles. The customer-facing nature of E-commerce and the constant change associated with web-based strategies and technology combine to create the emergency change scenario discussed above. The study shows that E-commerce changes are introduced without sufficient lead time or planning when compared to other application types.

Companies report that 29 percent of E-commerce changes are considered emergencies, which naturally causes delays and increases tensions within the IT department. As the preceding graphic demonstrates, IT is not at all confident of changes to E-commerce production 16 percent of the time. Small wonder, seeing that 28 percent of changes cause problems to E-commerce production systems.

At StackSafe, we believe that it doesn’t matter if an application exists in a e-commerce environment. IT requires the capability to establish representative staging environments for IT testing, to test every change, and to test changes against the entire infrastructure, no matter how tight the timeframe or how high the change priority.

When IT cuts corners with testing, particularly with E-commerce application testing, “always on” may end up being “sometimes on” or “maybe on in a few hours.” In business-critical situations, such as preparing for high spikes in traffic around peak online shopping times the diligence given to testing changes will make or break the online shopping experience.

Download the Report:
You can download a full copy of the IT Operations Research Report: Testing Maturity Part II – Applications and Operating Systems here.

Popularity: 12% [?]

Filed Under: Change Impact Analysis, Downtime, IT Operations Research, Testing