How to assess and improve the quality of your application portfolio

Posted by Jochem Schulenklopper on Jan 4, 2018 11:35:18 AM

Xebia IT Architects share improvements on Gartner’s TIME model for application portfolio management

Application Portfolio Management is about visualizing and managing applications within in an IT landscape on a regular and correct basis. It also makes it possible to analyse certain KPI’s to improve the quality of your IT portfolio. The more practical and easy-to-understand a model is, the more supportive it can be for IT managers in this respect. Read here some enhancements to the Gartner TIME model.

This article is a translation from the Dutch IT magazine AG Connect. Click here for the original article.

It's evident that the support of business processes by IT, and consequently, an organization's dependence on IT, has increased over the past few decades. Over a significant amount of time, companies have made high and far-reaching investments in IT.

As a result of these developments, a (highly) heterogeneous IT landscape emerged in many companies. Heterogeneous in software technologies used, application types and application domains, but also in suppliers, user groups and processes of application development and maintenance.

Of course, a continued responsibility for a company is to effectively and efficiently use existing applications, and to create new applications were necessary. This is the reason that deciding about the future of an organization's application portfolio in a regular, correct and fair fashion is a top priority for IT managers.

This article reviews this responsibility in a number of theses. 

Thesis 1: Application portfolio management is crucial for effective and efficient IT

Making decisions about a company-wide application portfolio is not a simple task. IT landscapes with 40-50 applications are more often a rule than an exception. IT managers must balance complex organisations and business processes, limited budgets, an existing IT landscape, an ambitious strategy and, of course, various stakeholders and affected parties. This leads to questions like: how do you effectively decide about the future of IT? How do you decide about innovation, renovation and elimination of applications? What do you maintain, reuse, modify or phase out?

 Viewed from a higher level, the requirement is on how to make an honest and correct assessment of the applications in a portfolio. How do you make decisions that are consistent with the organization strategy and its information needs? How do you ensure that these decisions are actually implemented? This is the essence of application portfolio management.

Thesis 2: TIME must be refined to facilitate practical adoption

'All models are wrong, but some are useful' - George E. P. Box

To support decision-making within application portfolio management processes, various models to assess an application portfolio have been proposed. The best-known model, TIME, was described by Gartner in 2009. The advantages of this model are that it is simple, understandable, and useful, and that's why it is applied regularly. However, in our experience the model requires some refinements, particularly concerning tangible and practical guidelines for measuring IT quality.

Thesis 3: Nowadays IT quality is all about agility

An essential refinement of the TIME model concerns measuring IT quality. Specific, useful explanations of this remain lacking in Gartner's description.

At least two challenges are to be overcome in measuring IT quality. First, we need a concise definition of IT quality. Secondly, based on that: what is a metric that can be used to reliably determine IT quality for all (types of) applications? In other words, what metric is applicable for all applications within an organisation: developed in-house, developed by external parties, standard software packages and customisations, as well as contracted SaaS services.

What is the definition of “IT application quality”? There are lots of perspectives on quality. In the context of portfolio management, we would like to focus on a single one. For companies that want to be agile - and therefore want to quickly anticipate to threats and opportunities in their business operations and IT - the answer is self-evident. Quality in this sense is a quick implementation of new functionality, and a swift resolution of defects, into production. In other words: IT quality is the speed at which you are able to implement any required changes correctly, while the applications continue to meet any other specified functional as well as non-functional requirements. This ability to change the IT environment can be expressed, for example, as the average time that elapses between the registration of ideas and issues, up until the time that the required changes are deployed in production.

This metric has a few important advantages. Quality is reasonably easy to measure for all (types of) applications within the landscape. It is specific, but also technology-independent. Furthermore, it is easy to compare quality of different applications. The biggest advantage of this metric, however, is that it not only expresses the quality of the architecture, software and documentation... but also about the organisation as a whole and all processes involved in implementing the change. As such, it considers IT quality in a much broader perspective.

Thesis 4: You improve an application (portfolio) by reducing the elapsed time 'from idea to production'

The (refined) TIME model automatically suggests a number of actions to improve the quality of the entire portfolio:

  • improve IT quality: an IT renovation;
  • increase business value by a more effective application of IT for (new) business operations: an IT-driven transformation;
  • reduce the size of IT applications to enable the organisation to implement future improvements more focussed, cheaper and faster;
  • phase out applications that represent too little value for the company.

Because the IT quality metric we suggested is directly related to the elapsed time required to implement changes, improving quality means reducing the time required to implement changes. Of course, you don't apply changes in all applications, but first and foremost in the applications with a high business value but a (too) low IT quality. You improve the portfolio by improving the 'right' applications... and by doing this the 'right' way.

Thesis 5: IT quality is more than just software quality

It should be clear that the elapsed time 'from idea to production' is determined by much more than software architecture, application technology and code. Examples of other factors that affect elapsed time include:

  • the organisation involved in determining requirements, including its inertia and indecisiveness;
  • the IT analysis process, including under-capacity, complexity of the domain and its tendency to over-analyse;
  • IT implementation, including slowness in design, testing and acceptance;
  • the capacity, knowledge and productivity of development teams.

Therefore, it is also evident that shortening the elapsed time involves much more than improving the architecture and program code. A more decisive organisation, a faster requirements analysis process, more autonomous development teams, smoother continuous integration and development (CI/CD) processes and on-demand available infrastructure all contribute to a shorter elapsed time.

To improve IT quality it is therefore important to determine the activities that require the most time and/or involve the highest costs. The Value Stream Mapping technique is very useful for this purpose.

Next, once the activities with the highest elapsed time have been identified, further analysis is required to identify any slowdowns – see above for several examples of potential causes. Once it is clear where the biggest gains with the least effort can be achieved, specific improvement projects for the integrated IT road map can be defined to reduce the elapsed time 'from idea to production', and consequently increase IT quality.

In summary: Practical IT Application Portfolio Management

Application portfolio management consists of regularly and correctly assessing the applications in an IT landscape, analysing the measures required to improve the quality of the IT portfolio, followed by planning and actually implementing the decisions taken in this respect. The more practical and specific the model for supporting IT managers in this regard, the better. 

Refined Application Portfolio Management Model

In 2009 Gartner published an IT application portfolio management model called TIME.

The TIME recipe is that applications are assessed along two scales: Business Value and IT Quality. Depending on the position of the applications in a 2x2 matrix, you decide one the future of each application: Tolerate, Invest, Migrate or Eliminate.

An illustration of an application portfolio in accordance with the TIME model

Figure 1: An illustration of an application portfolio in accordance with the TIME model (with the axes reversed for comparability with Figure 2).

The TIME model requires several refinements to make it more tangible, specific and practically useful for application portfolio management purposes.

In TIME, Gartner describes a number of aspects that form part of quality; however, unfortunately no guideline is provided to assist in specifically assessing IT quality.

In addition, it does not propose a clear metric for determining the size of the applications, the diameter of the circles in the illustration of the portfolio. Although this omission is not crucial, it seems like a good idea to relate the size of the applications in the illustration to the effort of, i.e. the inherent resistance to, changing an application.

See Figure 2 where this is included in the visualization. This can be expressed technology-neutral as a number of function points, the size of the data collections, the number of interfaces with other systems, the size of the user group or the size of the development or application management team.

Refined Application Portfolio Management Model with improved metrics, axes orientation and terminology

Figure 2: A better illustration of the applications in an application portfolio: in accordance with clearer metrics, a more useful orientation of the axes and better decision-making terminology

TIME also identifies aspects or areas for attention as a means of expressing the business value of IT applications. However, determining the interrelated business value of applications with different stakeholders with different interests is a difficult group exercise. It would have been useful if TIME had included a process describing how to effectively do this.

Finally, the names assigned to the portfolio decisions might elicit the wrong associations. We are therefore suggesting that the terminology be improved by making the following changes:

  • 'Tolerate' will be 'Accept' – this is more positive and more respectful of the past;
  • 'Invest' will be 'Maintain' – the question remains whether you need to invest in something in order to maintain its position;
  • 'Migrate' will be 'Improve' – migrate implies a relocation, while you can also improve the current application;
  • 'Eliminate' will be 'Phase-out' – phasing out suggests a planned gradual elimination, in contrast to an abrupt elimination.

Topics: DevOps & Continuous Delivery