Technical Debt in Software Development

Demystifying the "jargon"

The concept of this technical jargon is widely introduced in the software and technology industry when a team of software engineers, programmers or managers work around the development and integration of a piece of software, functionality, feature or a project using the limited and easy solution instead of better approaches available to intentionally expedite the process of time to delivery with intended deliverables discussed at the inception of the project.

In layman's terms, technical debt, also known as code debt is a metric that determines the extra refactoring work required in software development practices. A certain degree of technical debt accumulates over the development timeline when the working team constantly chooses quick solutions over the right solutions. These processes of bad programming decisions eventually compile down to form up an unprecedented weight of code refactoring that cannot be averted and hence requires rework at a certain point.

Recent Activities

In recent times when software advancement has been ever-growing so rapidly, the idea of technical debt has gained more traction in the workplaces of software research and development.

In addition, with the growing popularity of Extreme Programming (XP) and Agile methodologies in most software companies, project managers/ product engineers enforce a very tight schedule of completing their time-boxed code sprints for their developers, expecting little to no rework. So at this very time of debacle, the mounting to-do lists of backlogs on their scrum boards lead the developers to find more and more workarounds than the better fit solution to their coding problems. The compounding urge to release faster than their competitors to beat them to the market acquisition produces even less assured software quality with more shortcuts degrading the scalability, performance and overall health of the software being built. A research paper even suggests that in addition to such technical and economical consequences, it might also degrade the developer’s morale and psychological state.^[1] However, this kind of indicative nature of influence has clearly no empirical observations or ongoing research that may emphasize the dwindling morale within the developers’ mindset.

An exploratory field study and several data collected from software professionals actively working in varied industry domains via interviews and online survey forms profess that technical debt mainly has a negative influence over the developer’s morale and affects their perception of achieving future goals. In essence, the technical debt hinders the developer's efficiency in completing tasks and accordingly, proper management increases the developer’s sense of progress.

Since the dawn of the internet bubble, technical debt has been used as the new term for the jargon dictionary in the software development and engineering industry. The ever-growing need for highly skilled manpower in the software engineering discipline has brought a new trend for topics of interest in these socio-technical fields. People are investigating more on the economical and organizational aspects of work performance and effectiveness. This has led to the scarcity of research and findings towards the mental and psychological aspects of the working manpower glued to their big screens tireless. The burden of code rework on the individual developer’s shoulders has yet to be discussed. More importantly, the obstacles faced due to the consequences of TD are not spoken widely enough in the industry, which has added more negative results in itself. Due to the lack of significant investigation and rigorous studies, the core relationship between the developer’s TD and their psychological consequences has received minimal attention in recent years.

Developers' Teams

Software development is a socio-technical aspect, and its progress depends on both social and technical factors. Most software developers work in collective teams with one or multiple supervisors, project leaders and engineering managers. Teamwork and collaboration for the software creation process have been sought out to be highly effective in the software development life cycles and considered as the right way to tackle any kind of project from inception to production and even refactoring processes. Numerous studies in recent times indicate that the happiness and motivational factors in the developers' morale directly relate to their productivity and increase the overall software quality.^[2] In addition, the research emerging from the management science fields even suggests a more critical link exists between an employer’s morale and their productivity in work.^[3]

Considering the lack of specific points in previous studies and research papers about team collaboration to manage technical debt, our focus has been on providing novel contributions to this exact topic. In this study, we are trying to find an optimal correlation between team collaboration and its actual effects on the management of technical debt. We conducted a field study and shared multiple copies of an online survey form with proper guidelines and privacy policy within a circle of active software developers working remotely from various parts of the globe to understand the advantages and disadvantages of managing software developers in a team to reduce the effort of rework in their codebase. Based on our own findings, we propose that the collaboration of developers within a team and between other teams reduces the effort to manage technical debt. The dismantled group of individual developers find it comparatively hard to manage the cost of refactoring technical debt in the long run.

Mitigating Technical Debt

Some of the kinds of TD are Architecture TD, Testing TD and Source Code TD, depending on the artefacts and level of abstraction. There are many ways to eradicate TD and mitigate its consequences in an existing project or software team. Management of TD includes a set of activities like identification, measurement, prioritization, prevention, monitoring, repayment, documentation, representation and communication. The principal of the technical debt is the cost of refactoring TD. Piling up technical debt over the development time may give a sense of gain in the short term but ultimately weighs more than the gain achieved when perceived in a substantial timeframe, even leading to a development crisis.

Results based on the several qualitative interviews with the active software development teams, a research paper even found that the majority of the interviewees think that they are hardly criticized by their peers from the team or the managerial teams.^[1] Being criticized for introducing TD to their peers or pointing out in the team might be a bad feeling. Moreover, the majority of interviewees also emphasized on a different perspective that the collective efforts to manage the existing or the foreseeable technical debt of the team members, including developers and engineering managers, are also appreciated and encouraged within the software team. Hence, a proper way of communication within the software team to point out TD is crucial in managing and reducing the anticipated consequences. On the other hand, when a software product requires the involvement of developers' teams to work on various code modules separately, a TD of a module in a team can also impact another developer in a different team at some point, usually during the merge and integration steps. Managing TD in software teams leads to a boost in productivity and progress among the software developers. Identification and communication of TD at the right time about its management within the software teams accumulate a sense of contribution towards the team's future goals, and such positive feelings help the developers to successfully complete tasks.

Based on the collective perspective of developers, a good level of cohesive understanding persists in the software team about the reasons behind the occurrence of TD. Developers believe that teamwork is the key to managing a team's TD. The introduction of TD in an existing team cannot be put on the previous works done by the former developers. Most software developers believe that the TD introduced must be solved and mitigated within the current team by the present team members as a responsibility towards the overall team’s goals and the consistently improving quality of software artefacts. To be more evidently promising on the topic, a software team collaboration in “repaying the TD can be motivational and good for the team’s morale”.^[4]

The primary goal of the software development team should be to create quality software and embrace tools and technologies for the process. People attached to the software team in managerial roles also care about the software quality but mainly focus on the costs, burndown and scheduling factors. Specifically, software developers and their engineering managers seem to be more motivated to manage their technical debt. So we firmly believe that technical debt should not only be related to the technical aspects of the software product but also to the management and business decisions that would help the whole team involved to push the release to market faster.

Perspectives

A different perspective is that TD can do both good and harm to the software project and its team. An intentionally incurred TD to achieve short-term benefit can be fruitful if the cost of TD is known well under control. A software team might want to take TD intentionally to develop new features quickly to gain some business value and step ahead in the competition to the market. Such acts of intentional TD can reap fruitful goals in teams.  However, accumulating TD unintentionally without knowing its existence to the core team members and project managers for a longer time frame can have adverse effects resulting in missing deadlines and uncalled maintenance.

TD can be introduced by the stakeholders and product owners in a software team. A paper puts forward the notion that TD induced by the stakeholders in a software project can be in the form of strict deadlines at the cost of quality or the cascading effects of decisions made on correlated projects on which a given project depends. This kind of TD induced by stakeholders is hard to manage for the team and mainly beyond the team's control given the stake of decisions made in the project.^[5] The paper even argues that unintentional TD caused by acquisition, realignments and drastic changes to the market ecosystem is more difficult to manage in the team. These scenarios can lead to the failure of software teams to convince the business stakeholders of the project to invest in fixing the technical shortcuts taken. Hence the changes to the assumed environmental factors in a software engineering process can induce unintentional TD that may be uncontrollable for the development team.

1. 2017 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement
2. D. Graziotin, X. Wang and P. Abrahamsson, "Happy software developers solve problems better: psychological measurements in empirical software engineering," PeerJ. 2: e289, 2014.
3. J. Abbott, "Does employee satisfaction matter? A study to determine whether low employee morale affects customer satisfaction and profits in the business-to-business sector," Journal of Communication Management 7(4): 333-339, 2003.
4. S. McConnell, “Technical Debt”, 2007, http://www.construx.com/10x_Software_Development/Technical_Debt/.
5. T. Klinger, P. Tarr, P. Wagstrom, C. Williams, An enterprise perspective on technical debt in Proceedings of the 2nd International Workshop on Managing Technical Debt (MTD’11), ACM, Waikiki, Honolulu, HI, USA, 2011, pp. 35–38.

Thank you for reading till the end. This article is inspired by true events.
If you've any feedback or want to start a discussion thread, please use the comments below.