Takeaways from Testing Webinar

Posted February 8th, 2008 by Dennis Powell

StackSafe and Ecora Software joined forces again earlier this week to deliver a webinar entitled “Improving Change Management Through Better Testing for IT Operations”. The StackSafe segment of the webinar discussed our latest research report on testing maturity. As our presenter, I discussed the three categories of customer testing profiles – “Test Leaders”, “Test Neophytes”, and the “Unfortunate Middle” – that we’ve identified through a rating for testing maturity in our research study. A replay of the Webinar will be posted shortly on our Change Impact Analysis Knowledge Center.

During the webinar I alluded to some statistics from our research study that prompted questions. For example, I explained that our research participants told us that they average between 106 and 113 changes per year to their product infrastructures. I immediately received a question asking me to qualify this number, because the person asking the question stated that his company introduces “over 1000 changes” to production “every month”.

We have heard this from other sources and, quite frankly, it confirms our emphasis on the importance of change management and testing maturity. When we conducted our research, we talked with several companies that generated this volume of change activity to production. We did not identify any correlation between extremely high numbers of changes made and specific vertical market segment or vertical application implementation. However, this warrants further investigation to determine whether there is a certain profile of customer that is prone to introduce high change volume into production. The obvious difficulty here is finding a common definition for “change.”

I’d make the normal assumption that these are global companies with very large disperse and heterogeneous enterprises running highly componentized applications that contain lots of custom code. It would very likely use an e-commerce business model with multiple systems that require a high degree of online interaction with a high volume of customers on a 24X7 basis. This is a hypothesis that we may test as we continue our research of IT Operations teams.

A second question from an attendee asked me to identify all of the requirements that constitute effective testing and staging practices. Not knowing exactly what type of infrastructure or testing requirements this person has, I could not provide a response with the level of detail I would have liked. However, as I mentioned on the Webinar, the best practice guidelines provide an excellent starting point to understand the nuances and requirements of effective change management (and release management) processing. ITIL Change Management and Release Management standards and CobiT measurements will provide the best practices and the control model upon which to establish an IT change management program.