Configuration Management Maturity Models
In this month’s CMsights we begin to explore the value of a Configuration Management Assessment using configuration management maturity models developed specifically for the practice of CM. This follows our April 2020 article on Configuration Management Assessments. In that post, we presented an example configuration management checklist of questions that when answered can expose problems which signal CM processes are poor or altogether absent.
Afterwards, we received a question from a reader as to whether maturity models existed for configuration management that can help to synthesize the responses to our checklist questions for the purpose of constructing an overall performance evaluation. To help answer that query we invited a representative, Bas Berk, from the European CM consultants at ADSE to be a guest subject-matter expert author. Bas is a Senior Consultant in CM and QM with ADSE who has been trained and certified by CMPIC and IPX. From his 15 years of working on industry projects Bas was instrumental in specifying, developing, testing, and employing the customized configuration management maturity model used by ADSE with their clients.
Why Managers Care About Configuration Management
As discussed in the April CMsights article, we at ADSE in Europe agree with our peers at CMstat in the U.S. that rarely do business executives or product managers explicitly ask for improvements to their organization’s performance of the Configuration Management (CM) function. This is true even when they understand the important role of CM in quality management or regulatory compliance.
However, we do receive inquiries as to how managers can ascertain whether they even have a problem, deficiency, or risk with respect to the CM function that would necessitate an improvement. This is often prompted by their organizations seeking budgetary funding to invest in yet another generation of enterprise PLM, engineering PDM or industry-specific CM tools. It is not uncommon that CM is used – and occasionally misused if not abused – as the central justification in an ROI analysis for deploying such new software solutions.
Management’s questions about Configuration Management are also driven by customer complaints, missed performance requirements, faulty documentation, or laborious efforts to validate regulatory compliance. Other commonly heard questions are:
“How much exposure and risk – technical, financial, regulatory, legal, and reputation wise – do we have if something goes wrong or fails?”
“How much time and money do we actually spend on correcting hardware products or software components along with their associated data and documentation?”
When the above questions arise, it is usually a sign that CM has been insufficiently implemented across the organization. As a result, processes within their supply chain, aftermarket services, and MRO partners are often being contaminated with faulty configuration data.
To Be or Not to Be
Every desired CM-related improvement requires two reference points: awareness of where I am, and awareness of where I want or need to be. In general, “where I want to be” is a matter of your internal vision and ambition. “Where I need to be” is often driven by external requirements following from your ambition. For example, requirements from customers or regulatory bodies, and influenced by comparisons to your competitors and peers.
First, how does one determine where they are in their CM maturity journey? Typical questions that arise first appear rather basic to address, although in practice they rarely are easy when you ask the same question of different personnel throughout an entire organization. These core questions include:
Just how proficient is my organization in CM?
Where are the specific gaps in performance that need improvement?
How do we perform compared to others in our industry?
Where do we start; what should we fix first?
Who best to lead us in this effort?
What metrics do we use to measure and monitor our progress?
These questions can best be answered, and the results analysed, by use of a Configuration Management Maturity Assessment Model. This is sometimes referred to as a Capability Assessment Model or Capability Maturity Model. Such a model and resulting analysis enables an organization to identify the current CM capability maturity level of their organization. It can also be extended to cover their supply chain and its interaction with other business processes so you can plan a way forward. In essence, it answers the questions how do we find CM issues, fix them, maintain control, and prosper as we grow.
For those not familiar with the concept, Capability Maturity Models (CMM) were first developed in the United States during the 1980s from research funded by the Department of Defense to assess the performance of government contractors. In the 1990s a more robust framework was developed by the Software Engineering Institute at Carnegie Mellon University which would lead to the Capability Maturity Model Integration (CMMI) project.
While originally focused on measuring the ability and optimization of software development processes, CMMI became a popular tool to use for benchmarking other business processes across five levels of performance: ad-hoc, repeatable, defined, managed, and optimized.
More references about CMMI can be found on Wiki HERE as well as in a 2013 paper by Niknam and Ovtcharova at the Karlsruhe Institute for Technology titled “Towards Higher Configuration Management Maturity” available HERE.
The specific processes – or spider chart axes – that are evaluated by maturity models will vary by discipline and industry. As a result, consulting organizations with deep subject matter expertise in both a discipline, like configuration management, and an industry, like aerospace & defense, are often at the forefront of developing custom CMMs for use with their clients.
An Example Configuration Management Maturity Model
ADSE Consulting & Engineering has created its own CM maturity model based on its aircraft OEM origins and nearly 25 years’ experience in the aerospace, rail, and defense industries. We at ADSE did this because we found that realizing robust improvements in CM requires a consideration of the full scope of individual CM processes. For example, service partners cannot perform proper change management if the OEM did not properly define Configuration Items (CIs) at the onset. It is also difficult to perform reporting and make the right decisions when data is untraceable and thus unreliable. An orchestration of topics and harmonization of CM processes must be aligned in order to be successful.
The ADSE CM Maturity Assessment Model describes a five-level evolutionary path of increasingly organized and systematically more mature CM processes as depicted below.
At the lowest maturity Level 1 an organization is focused on executing the (corrective) tasks required for managing documentation. Level 2 resembles becoming more oriented towards execution and control of processes for managing configurations. At this level more formal procedures and processes are introduced that are increasingly optimized to reach the next level of maturity required to have effective Configuration Management. This is done by continual process improvement and introduction of adequate enabling IT software tools.
Beyond Level 2 Configuration Management becomes more and more a core business process at the enterprise level while increasingly involving suppliers and customers. The focus shifts towards making CM also an efficient repeatable process by using CM software tools such as CMstat’s EPOCH CM with which we at ADSE are familiar and impressed by its CM-focused capabilities.
A key to the use of maturity models in general is that it is very hard, and often counterproductive, to skip levels. For example, how can you ever effectively automate processes – if that is what is required – when you are surrounded by undocumented processes, and each employee having their own terminology and definitions? Most likely your organization will be more productive, but in the wrong thing; generating more chaos! You have to learn to walk before you can run. Each level forms a necessary foundation from which to reach the next level.
The Focus Should Be On Processes
The ADSE Configuration Management Capability Model is based on six main CM process areas for effective implementation of Configuration Management. They are: CM Organization, Configuration Definition, Change Request, Change Implementation, Configuration Disclosure, and IT Systems. We at ADSE have found these six axes to be the most appropriate given our industry focus and customers.
Each of the six process areas illustrated below are subdivided into subprocess areas which allow for accurate identification and remediation of issues. Characteristics of each CM maturity level were developed for these subprocess areas to describe what should be in place at each level of maturity of an organization. These characteristics were based on experience in a variety of industries of the CM consultants working at ADSE who employ standards such as EIA-649.
The characteristics are used as a benchmark to determine and score 1 – 5 the level of CM maturity of an organization. The review of processes take place from at-desk documentation reviews and on-site interviews with relevant stakeholders. The results are exemplified per CM process area in “spider diagrams” for easy understanding.
How are Results Used?
Once we know what the current CM maturity level is per (sub) process area, what’s next? First of all, the scores allow one to determine the “performance gap” between where you are and where you want or need to be. Be aware that a level 5 score is not always required. It depends on your ambition and what is required from the organization context.
Once the desired level is defined, the gaps between the required level and the current level become clear and actions can be defined to close these gaps. They are the basis for an improvement plan and serve as a roadmap. The model helps to set priorities for these actions and also helps to understand the required sequence of these steps. Intermediate steps in maturity levels might be required if the gap between the required and current levels are large. Remember, you need to be able to walk before you can run a marathon.
The CM maturity model can next be used to measure the results of actions by periodically repeating the assessment and comparing the results with earlier assessments. Because progress becomes visible in a simple way this can be of great value for motivation and perseverance of the team and the management. An ethos of continuous improvement then becomes part of the organizational culture. This result is one of the most important benefits of using configuration management maturity models.
Next Up: Lessons Learned
In a future CMsights article we will provide additional lessons learned from performing CM assessments. We will also share the most common mistakes and avoidable detours we have seen others make in interpreting results and acting upon the conclusions. To follow our discussion about CM maturity assessments, to subscribe to CMsights CLICK HERE. To learn more about CMstat’s experience with performing CM Assessments contact us at information@cmstat.com.
About ADSE
ADSE is an independent consulting and engineering company located in The Netherlands that serves the European Aerospace, Defense, and Rail Transport industries. From initial advice to project execution ADSE develops solutions in collaboration with their customers. Visit http://www.adse.eu for their full range of capabilities or for information about specific CM services contact Bas Berk by email at Bas.berk@adse.eu or connect with him on LinkedIn HERE.
Receive CMsights
Subscribe to CMsights News for the latest updates from CMstat on Configuration Management, Data Management, EPOCH CM, and EPOCH DM.
Request a Demo
See how EPOCH CM and EPOCH DM support industry standards and best practices in Configuration Management and Data Management.
