29.10.2019 - Read in 11 min.
Technical debt: What it means and how to deal with it
29.10.2019 - Read in 11 min.
A full of knowledge article by Paweł Jarosz about technical debt in a software house. He will present ideas to realistically define technological debt, the threats it involves, as well as offer ways to recognize it, manage it, and take advantage of it.
Technical debt is a term that often gets tossed around by programmers, project managers or business owners. Unfortunately, it is often underestimated, swept under the rug or ignored altogether. Some companies do not even realize that they have technological debt threatening their business, limiting activities, exposing them to additional costs and blocking access to potential customers. At times, companies consciously run into tech debt hoping to deal with it at some point in the future.
In this article, I would like to discuss what we mean by tech debt and what are its consequences for a development team or company. I will present ideas to realistically define technological debt, the threats it involves, as well as offer ways to recognize it, manage it, and take advantage of it.
Table of content
- Technological Debt – Definition
- Types of technical debt
- Managing technical debt – Is it possible?
- The effects of having technical debt
- The main causes of technological debt
- How to recognize technical debt?
- Avoiding and minimizing technical debt
Technical Debt – Definition
Let’s start from the beginning, that is, from the very origins of the term “technological debt”. This term was introduced back in 1992 by Ward Cunningham, the creator of the first wiki and pioneer in the field of design patterns and extreme programming. Ward coined the term to convey the delicate balance between hasty software development practices leading to provide functioning code, and later improvement of the code in places where quality suffered as a result of haste.
Since we started creating software, we’ve been struggling with this problem under other names: software maintenance, software evolution, software aging, code decay, code reengineering, etc.
The term “technical debt” therefore refers to a situation where programmers write software that violates good practices related to architecture, code structure, design patterns, standards for creating certain mechanisms, etc.
When explaining technological debt, Ward Cunningham used a metaphor in which he compared speedy software development to the rapid growth of a company which financed its expansion with credit. The credit allows to achieve goals faster, but the long-term strategy of the company must factor in the repayment of the debt as long delays will entail costly interest charges. Not spending enough time restructuring existing code will make it more and more difficult to maintain and develop in the future.
Interest is accrued on debt, which further increases the liability. The faster we pay off such debt, the lower interest we pay. The situation is similar with technical debt. In this case, interest takes the form of a longer time required to implement the changes. There are various causes for technical debt. Sometimes it is introduced consciously through the implementation of temporary solutions or due to significant external circumstances. As a rule, however, it is a consequence of business pressure, incorrect design decisions, postponing refactoring indefinitely, updating dependencies or simply the lack of experience of the developer.
Types of technical debt
Since we’ve learned what technological debt is, let’s now distinguish its different causes. The ability to categorize tech debt is extremely important for its accurate analysis, because the estimation of technical debt depends on its nature (and the time it was incurred). Martin Fowler distinguished four types of tech debt (technical debt quadrant).
Going further, we can identify each quarter by assigning a color that matches its purpose, where red-orange means warning colors, and green-blue indicates desirable colors.
1. Accidental technical debt
Let’s start with red – reckless and accidental debt. It is the least desirable type as the team has no choice and either did not recognize the moment when the debt was incurred or there is no way to eliminate the existing debt. This quarter is red because in an ideal scenario the team should stop and self-improve (through training, employing a competent person or other available means) so that they can at least see the moment and recognize the type of incurred debt. At this point, there is space for deciding whether to further increase technological debt or pay it off.
2. Deliberate technical debt
Orange is a reckless and deliberate debt. This type of debt occurs when the team has the appropriate knowledge to implement the task, yet consciously decides to go with a quick and poor quality solution, usually for the benefit of quick implementation. This type of debt mainly relates to long-term behavior, where the decision to incur debt was not caused at all by a short deadline.
3. Prudent technical debt
Green represents prudent and deliberate debt. It involves cutting some corners in terms of quality as a part of decisions taken by the team due to the looming deadline or simply no need to use a better quality solution. The team recognizes the problem and its consequences, but must currently provide the functionality in time and there is no way to focus on quality at this point or the team sees that the implementation of the issue in an ideal way is not a value in this case, so calmly and consciously gives up.
4. Unintentional technical debt
Unintentional debt is represented by the blue colour – it happens despite a careful approach to design and, as it might seem – care for quality. It represents the team’s actual learning process while working on the project. Often, it is only after the functionality is implemented or the project is done when the team realizes that if they had designed the individual components of the application differently, they would probably end up with a better quality solution. We can treat unintentional technical debt as accidental as the team did not incur it on purpose.
Managing technical debt – Is it possible?
From the types of technological debt presented, it is clear that the most common cause of debt is rush, time pressure to meet business needs as soon as possible. In such situations, we often cut corners and reach for suboptimal technical solutions that in the long run make it difficult or even impossible to achieve the company’s future goals. Further development is possible only after paying off the previously incurred debt.
On the other hand, there are situations in which technological debt is incurred consciously and deliberately. In that case, is it possible to control and manage technical debt?
It turns out that so that conscious and proactive technical debt management helps companies bring projects to completion, is one of the most important points determining the implementation of projects on time but at the same time provides organizations with the ability to control the costs of technological decisions, which, as experience shows – do not always have to focus on best quality and perfect solution.
Over the past ten years, many companies whose sources of revenue are dependent on software have realized that technical debt under one name or another is a real problem and weakens their ability to satisfy customer desires and translates into negative financial effects. On the other hand, deliberate and prudent indebtedness in specific situations may not be worth paying back if the interest is low enough – for example, if the tech debt incurred involves a rarely modified and less significant part of the code.
If you got the short end of the stick and are dealing with such bad debt – characterized by orange and red colors – usually the best way forward is to do what we usually do with financial debts – paying back the capital gradually. This approach will require more time on the part of the development team, but by removing this piece of debt, we reduce the costs of future changes in this area of application. The great advantage of such gradual improvement is that naturally more time is devoted to eliminating debt in the areas which we often modify – more precisely where the application code most needs minimizing technical debt.
The effects of having technical debt
We have already said that unwanted technical debt usually results in a financial loss for the company, but this is not the only problem.
Technological debt can lead to other operational issues, such as failures, incorrect calculations, loss of productivity due to dropping performance, security breaches, and ultimately bring the project to a halt. Technical debt has a direct impact on the technical side and the problems it causes can be grouped according to which area of the project has the greatest impact:
- reliability – application stability and likelihood of new defects appearing during its modification,
- efficiency – application response time,
- security – the application’s ability to prevent unauthorized intrusion,
- portability – the ease with which a new team can understand the application and quickly start working productively on it,
- agility – the ability to easily and quickly modify the application.
Growing technical debt is a sign that the development team is getting slower. We spend more and more time “struggling” with the code, trying to coexist with the incurred technological debt. It won’t go unnoticed that a team that previously implemented 10 requirements in a single iteration now commits to implementing just 5, and spends the rest of the time on fighting technical debt. Unfortunately – the improvement in code quality is never seen in the short term. The more debt we have, the longer it will take us to introduce new functionalities. Technological debt reduces the ability to quickly implement new features. In some cases, when the debt is really big, adding new features within a reasonable time and budget is impracticable. This situation can bring software development to a halt and, consequently, cause big problems.
The main causes of technological debt
- Imprecise or evolving requirements during the project,
- Pressure to deliver the project to the Customer earlier than expected,
- Business pressure – usually business expects new functionalities to be delivered as soon as possible, often before all project-related activities are carried out even at the lowest acceptable level,
- Lack of knowledge and understanding on the business side, business not realizing the technical debt makes decisions that have later disastrous effects for the entire project,
- Delivering the product to the client before it’s finalized,
- Too rigid solutions preventing easy adaptation to additional needs (lack of modularity, too many interdependencies),
- Excessive focus on quick profit instead of deferring revenues in the early stages and reaping possible benefits later,
- Lack of adaptation to standards and good practices in the initial stages. Adaptation to standards in later stages may be significantly difficult or even impossible,
- No refactoring in the initial stages (subsequent refactoring will be more problematic),
- Incompetence of the people involved in the project and cutting corners,
- Problems with the solutions architecture, functionalities are not properly designed and separated. They are not flexible enough to easily adapt to the changing business environment and requirements,
- Lack of tests or their insufficient number significantly increases the risk of production incidents, and errors located in production may give unexpected side effects,
- Lack of cooperation in a team, when knowledge from experienced team members does not flow sufficiently to less experienced staff and / or business people,
- Insufficient code refactoring or delaying the execution of it,
- Difficulties in writing valuable code when team members don’t know how to write elegant code and then manage it effectively.
How to recognize technical debt?
- Fragments of code and functions that cover excessive areas and seem to do everything
- lack of consistency in implemented and used patterns / architecture,
- the presence of a large amount of dead – unused code
- “Spaghetti code” – code that’s difficult to use, and complex logic that cannot be easily changed or developed,
- unreadable code,
- no tests,
- duplicate code, breaking the DRY rule,
- slow tools or no tools at all,
- no test environment,
- inefficient test environment with a lack of full functionality of the production environment,
- loss of productivity, including loss of motivation and decrease in team morale,
- an increase in the amount of work associated with tests, especially manual tests,
- postponed implementations,
- increase in the number of errors and bugs,
- high level of stress in the team associated with the upcoming implementation,
- fear of any changes in the code,
- falling velocity of the team
- reports on tools for measuring technical debt.
Avoiding and minimizing technical debt
- Dissemination of awareness – the more programmers and clients are aware of the effects of technical debt, the higher priority should be given to debt elimination,
- compliance with good code writing practices,
- code review
- avoiding technical debt from the moment of writing the first line of code – using thoughtful solutions,
- software architect in the team. Is responsible for managing the team through technical decisions, especially at the architecture level,
- business awareness and the value of the company you work for,
- refactoring as an effective tool to fight technical debt,
- regression tests – help detect when errors are entered into the code, allowing them to be immediately removed or undone
- automatic code analysis and technological debt measurement,
- reducing technical debt should be part of your work ethic or the culture of the company you work for,
- sometimes accept a small technological debt,
- technological debt should not be incurred in relation to the most important areas and processes in the enterprise – it is only allowed for less important topics,
- it is worth taking it only when it is necessary (i.e. economically viable) – for a short, predetermined period, in a planned and predictable way, treating the introduced solutions as temporary,
- before incurring a technological debt, determine its size, impact on our business and the analysis of the risk associated with it,
- regular investments in technologies (e.g. ongoing software updates) are associated with a significantly lower total investment cost and less organizational effort than in the case of one-off repayment of significant technical debt,
- pair programming,
- caring for (personal) education and learning the techniques that help maintain high quality code,
- maintaining high readability of the code (measured for example by understandability for other team members) as one of the important quality criteria.
Technical debt is a basic component of the cost of owning an application and sometimes its incurrence can be profitable, but it is the responsibility of the modern developer to at least maintain the debt level at the current level without increasing it. Raising awareness about technical debt on both the business side and the software development company is able to significantly improve technical debt management and related costs. It is necessary for the business side (our clients) to clearly understand the effects of technical debt.
For programmers, the important information is that our code is never perfect and technological debt is incurred along with writing the first line of code – what is currently acceptable in a year or more may turn out to be an outdated and unused solution. The most important thing is that our code is good enough when it is created.
Understanding the concept of technological debt through the metaphor of “debt” as a loan works great on both the client and developers, because, like in real life, most people have some kind of financial debt on a more or less constant basis. One could say that it is not terrible as long as we can pay interest and until the debt is out of control.
Technical debt does not have to be a negative phenomenon. On the contrary, like in the example, when planned and undertaken with caution it can bring quick business benefits for the company at a relatively low cost. However, unlike a financial loan (which sometimes achieves a favorable leverage effect), technical debt is not profitable in the long run. We will always have to pay back more than we incurred – and the larger the debt, the more “interest” to pay.
However, if it occurs, technological debt should be properly managed – so as to minimize the associated risk.