TDD

Archived Posts from this Category

Executable Specifications – Presentation from AgilePalooza

Posted by on 06 Aug 2009 | Tagged as: Acceptance Testing, Agile, Architecture, DotNet, General, Java, Open Source, Scrum, Software Architecture, TDD, User Stories, XP

Earlier this year I did a presentation on Executable Specficiations for AgilePalooza conference. There is information about working with legacy code, commercial off-the-shelf (COTS) systems, and Acceptance Test-Driven Development (ATDD) using automated acceptance testing tools. Also, the presentation lists types of automated acceptance testing tools out there along with actual names of tools and what they are best used for on projects. Hope it is interesting to you.

Designing Through Programmer Tests (TDD)

Posted by on 27 May 2009 | Tagged as: Agile, General, Java, Scrum, TDD, Uncategorized, XP

To reduce duplication and rigidity of the programmer test relationship to implementation code, move away from class and methods as the definition of a “unit” in your unit tests. Instead, use the following question to drive your next constraint on the software:

What should the software do next for the user?

The following coding session will provide an example of applying this question. The fictitious application is a micro-blogging tool named “Jitter”. This is a Seattle-based fictitious company that focuses on enabling coffee injected folks write short messages and have common online messaging shorthand to be expanded for easy reading. The user story we are working on is:

So that it is easier to keep up with my kid’s messages, Mothers want to automatically expand their kid’s shorthand

The acceptance criteria for this user story are:

  • LOL, AFAIK, and TTYL are expandable
  • Able to expand lower and upper case versions of shorthand

The existing code already includes a JitterSession class that users obtain when they authenticate into Jitter to see messages from other people they are following. Mothers can follow their children in this application and so will see their messages in the list of new messages. The client application will automatically expand all of the messages written in shorthand.

The following programmer test expects to expand LOL to “laughing out loud” inside the next message in the JitterSession.

public class WhenUsersWantToExpandMessagesThatContainShorthandTest {

    @Test
    public void shouldExpandLOLToLaughingOutLoud() {
        JitterSession session = mock(JitterSession.class);
        when(session.getNextMessage()).thenReturn("Expand LOL please");
        MessageExpander expander = new MessageExpander(session);
        assertThat(expander.getNextMessage(), equalTo("Expand laughing out loud please"));
    }

}

The MessageExpander class did not exist so along the way I created a skeleton of this class to make the code compile. Once the assertion is failing, I then make the test pass with the following implementation code inside the MessageExpander class:

public String getNextMessage() {
    String msg = session.getNextMessage();
    return msg.replaceAll("LOL", "laughing out loud");
}

This is the most basic message expansion I could do for only one instance of shorthand text. I notice that there are different variations of the message that I want to handle. What if LOL is written in lower case? What if it is written as “Lol”? Should it be expanded? Also, what if some variation of LOL is inside a word? It probably should not expand the shorthand in that case except if the characters surrounding it are symbols, not letters. I write all of this down in the programmer test as comments so I don’t forget about all of these.

// shouldExpandLOLIfLowerCase
// shouldNotExpandLOLIfMixedCase
// shouldNotExpandLOLIfInsideWord
// shouldExpandIfSurroundingCharactersAreNotLetters

I then start working through this list of test cases to enhance the message expansion capabilities in Jitter.

@Test
public void shouldExpandLOLIfLowerCase() {
    when(session.getNextMessage()).thenReturn("Expand lol please");
    MessageExpander expander = new MessageExpander(session);
    assertThat(expander.getNextMessage(), equalTo("Expand laughing out loud please"));
}

This forced me to use the java.util.regex.Pattern class to handle case insensitivity.

public String getNextMessage() {
    String msg = session.getNextMessage();
    return Pattern.compile("LOL", Pattern.CASE_INSENSITIVE).matcher(msg).replaceAll("laughing out loud");
}

Now make it so mixed case versions of LOL are not expanded.

@Test
public void shouldNotExpandLOLIfMixedCase() {
    String msg = "Do not expand Lol please";
    when(session.getNextMessage()).thenReturn(msg);
    MessageExpander expander = new MessageExpander(session);
    assertThat(expander.getNextMessage(), equalTo(msg));
}

This forced me to stop using the Pattern.CASE_INSENSITIVE flag in the pattern compilation. Instead I tell it to match only “LOL” or “lol” for replacement.

public String getNextMessage() {
    String msg = session.getNextMessage();
    return Pattern.compile("LOL|lol").matcher(msg).replaceAll("laughing out loud");
}

Next we’ll make sure that if LOL is inside a word it is not expanded.

@Test
public void shouldNotExpandLOLIfInsideWord() {
    String msg = "Do not expand PLOL or LOLP or PLOLP please";
    when(session.getNextMessage()).thenReturn(msg);
    MessageExpander expander = new MessageExpander(session);
    assertThat(expander.getNextMessage(), equalTo(msg));
}

The pattern matching is now modified to use spaces around each variation of valid LOL shorthand.

return Pattern.compile("\\sLOL\\s|\\slol\\s").matcher(msg).replaceAll("laughing out loud");

Finally, it is important that if the characters around LOL are not letters it still expands.

@Test
public void shouldExpandIfSurroundingCharactersAreNotLetters() {
    when(session.getNextMessage()).thenReturn("Expand .lol! please");
    MessageExpander expander = new MessageExpander(session);
    assertThat(expander.getNextMessage(), equalTo("Expand .laughing out loud! please"));
}

The final implementation of the pattern matching code looks as follows.

return Pattern.compile("\\bLOL\\b|\\blol\\b").matcher(msg).replaceAll("laughing out loud");

I will defer refactoring this implementation until I have to expand additional instances of shorthand text. It just so happens that our acceptance criterion for the user story asks that AFAIK and TTYL are expanded, as well. I won’t show the code for the other shorthand variations in the acceptance criteria. However, I do want to discuss how the focus on “what should the software do next” drove the design of this small component.

Driving the software development using TDD focusing on what the software should do next helps guide us to only implement what is needed and with 100% programmer test coverage for all lines of code. For those who have some experience with object-oriented programming will implement the code with high cohesion, modules focused on specific responsibilities, and low coupling, modules that make few assumptions about other module they interact with will do. This is supported by the disciplined application of TDD. The failing programmer test represents something that the software does not do yet. We focus on modifying the software with the simplest implementation that will make the programmer test pass. Then we focus on enhancing the software’s design with the refactoring step. It has been my experience that refactoring refactoring represents most of the effort expended when doing TDD effectively.

AgilePalooza in San Francisco May 29th

Posted by on 19 May 2009 | Tagged as: Acceptance Testing, Agile, Architecture, General, Leadership, Open Source, Product Owner, Scrum, Software Architecture, TDD, Travel, User Stories, XP

AgilePalooza is a one day Agile conference on Friday May 29th at the San Francisco State University downtown campus. There will be two tracks: Learning Agility and Advancing Agility.

“Learning Agility” will be presentation style whereas “Advancing Agility” will use the open space format.

Speakers include David Hussman (DevJam), Chris Sterling (SolutionsIQ), Luke Hohmann (Enthiosys), Lee Henson (VersionOne Services) with special guests Ainsley Nies (open space co-facilitator) and Ian Culling (VersionOne CTO). When not presenting for the “Learning Agility” track speakers will participate in the open space.

Space is limited and the cost is low so please register soon if you would like to attend. For more information or to register please visit www.AgilePalooza.com.

Where?

San Francisco State University Downtown Campus, 835 Market Street, San Francisco. (Powell street BART station in the Westfield Center) – Map/Directions

When?

9am-5pm (check-in and continental breakfast starts at 8am)

Cost?

$69

Register here: www.AgilePalooza.com

My Talk @ SD West 2009 on “Managing Software Debt”

Posted by on 27 Mar 2009 | Tagged as: Acceptance Testing, Agile, Architecture, Distributed Computing, DotNet, General, IASA, Java, Jini/JavaSpaces, Leadership, Maven, Open Source, Podcasts, Product Owner, Ruby, Scrum, Software Architecture, TDD, User Stories, XP

I have uploaded the talk I did at SD West 2009 on Yahoo! Video and here it is:

It’s BeyondAgile: people making software that works

Posted by on 25 Mar 2009 | Tagged as: Acceptance Testing, Agile, Architecture, DotNet, General, Leadership, Product Owner, Scrum, TDD, User Stories, XP

The hopefully-not-anticlimatic second event of the still-pretty-new BeyondAgile is happening on Thursday:

“Agile Challenges Clinic/Swap Meet”
Thursday, 26 March 2009
6:30 to 8:30 p.m.
Locations on the Eastside and Seattle!
Read on for agenda and location information, or visit our Google Group at
http://groups.google.com/group/beyondagile

The Blurb
At the first event, everyone (over forty people) created a backlog of ideas, suggestions, and work items. Among them were two suggestions that gave rise to this event’s focus: “a bring-a-problem” session where everyone helps everyone with their challenges. We’ve extended the idea of a clinic, where you bring a problem to an expert and get help, to a swap meet (or potluck), were everyone brings something and gets something. So come if you think you’re brimming over with answers and expertise, and come if you’re wanting other perspectives or advice on how to tangle with the problems that bedevil your days.

We’ll also form a program team, a group that’ll work proactively to plan interesting and exciting events in sufficient time for the blurb-writer–that’s me–to write and distribute the blurb a few weeks before the event. If you’re wondering who’d be good at doing that, go look in a mirror—it’s you we need!

Second Event Agenda

  • Welcome and Announcements (10 min.)
  • Formation of Program Team (10 min.)
  • “Agile Clinic/Swap Meet (85 min.)
  • Giveaway Drawing (5 min.)
  • Meeting Retrospective (10 min.)
  • Socializing and Breakout/Breakup/Beer

Event Locations

Eastside: SolutionsIQ
First Floor Training Center
10785 Willows Road NE, Suite 250
Redmond, WA
NOTE: directions and picture of building at http://www.solutionsiq.com/about-us/contact-us.php?Look for the tent signs and the blue BeyondAgile logo

Seattle: Amdocs Digital Commerce Division (formerly QPass)
Greece Room
2211 Elliott Ave Suite 400
Seattle, WA

Questions?
Contact us at beyondagile@taoproductions.com, or visit our Google Group, or contribute to the nascent Wiki at beyondagile.org.

Why come?
Here’s your chance to be in on the beginning of an exciting new collaboration of the Puget-Sound area (and beyond!) agile-interested. And if that’s not enough, we’ll hold a drawing to win exciting and valuable prizes!

What’s BeyondAgile?
It’s the combination and follow-on to a number of the agile-oriented user and interest groups that have operated in the Puget Sound area. Representatives of the Seattle XP User Group and Seattle Scrum came together in December to try and combine the efforts of all of us who care about and use agile methods.

Why does BeyondAgile exist?
o Find best practices among all agile processes
o Build a bigger agile community
o Take advantage of overlaps between existing groups
o Expand beyond meetings and provide a place to collaborate
o Align software development and business
o Explore cutting-edge ideas and techniques

What is BeyondAgile like?
That, in large part, is up to you. A primary reason for consolidating our efforts is to broaden our base of support, capability, and leadership. We envision a more active, multi-faceted organization that does more than just host talking heads. We’ll gather at least once a month on the 4th Thursday of the month, and probably more often, once you figure out what other events you’d like.

Where do BeyondAgile events happen?
Our goal is to have many answers to this. As a result, we’ve worked to remove the impediment offered by bridges and commuting: for our monthly meetings, we hold our events in both Eastside and Seattle locations, through the semi-magic of videocasting. We’re experimenting: we’ve thought of trying to alternate the “real” physical meeting between sides of the lake. At this point, we’re only at the stage of using a semi-okayish video link between the two locations, and try *very* hard to make the meeting balanced between locations. That’s harder than it seems; come and help us work it out! We’re still dreaming of enabling people anywhere to attend through streaming video, even after the meeting’s already happened, but we need more knowledge, resources, and volunteers before that’s going to happen.

I have question for you.
Great! Visit the Google Group (BeyondAgile) or send a message to beyondagile@taoproductions.com

What is Architecture?

Posted by on 19 Jan 2009 | Tagged as: Agile, Architecture, General, IASA, Leadership, Product Owner, Scrum, Software Architecture, TDD, XP

Talk about a loaded term. Even the term itself, “architecture”, when used in the Agile community can start a heated discussion. When I was coordinator of the International Association of Software Architects Puget Sound chapter, the discussions about “what is architecture” caused passionate debate. I am sure this entry will get some interesting comments, as well.

I don’t believe that this entry will solve the question but I hope to at least give some focus for teams who are dealing with the structural integrity of their applications. First off, a definition of “architecture”:

architecture: noun – “the art or practice of designing and constructing…” or “the complex or carefully designed structure of something” or “the conceptual structure and logical organization of a computer or computer-based system”

Each of these definitions interests me but to put them to practice is quite difficult. They are left to interpretation about what is design, construction, structure, conceptual, and logical. Earlier interpretations of this term interpreted these to mean the high-level structure, architectural style, and finally the stuff that is too expensive to get wrong.

It seems to me that my interpretation is a bit different. Working with tools, practices, and processes over the past 15 years has guided me to a different conclusion. I believe that architecture is how it is SAID:

Structure – how the pieces (components) create a solution (application)
Alignment – the degree to which the application or enterprise structures align to current business objectives and strategy
Integrity – the components that provide stability to the structure (application or enterprise) (ie. automated tests and builds, service-level agreements, network infrastructure, etc…)
Design – a way to conceptually communicate the planned or implemented structure of a component, application, or enterprise (ie. system of names, XP practice of Metaphor, information radiators, etc…)

Using these as the basic building blocks for teams to focus their efforts can be helpful.

The structure is the reality of the solution’s construction. If the structure is too brittle or complex to support future needs then the structure is not sound. If you have been in the software development industry for even short period of time you have probably seen applications that are too brittle or complex.

If the application or enterprise structures are not aligned with current business needs then the value of those structures have deteriorated. We sometimes keep a specific architecture and force-fit new business needs into it because it once was the right architecture to have or we paid a bunch of money for it. Continual restructuring of our architecture to meet today’s business needs is important.

Providing supports to our structure allows for it withstand changes in the environment such as new business needs and updates in technology. Automated tests and builds help us keep the structural integrity of our applications intact while these changes are introduced into our applications and enterprise.

Communicating the conceptual structure of a component, application, or enterprise is important because it is common for new people to work on them and for those structures to interact with other components and applications. Getting someone up to speed on a component or application involves verbal, tactile, written, and visual examples. Much of what is needed can be kept in or close to the codebase along with conversations with existing developers. It is important to understand how components and applications are currently structure conceptually so we can discuss their interactions with other components and applications. For instance, should we connect via library, SOAP, RPC, or REST?

When I think of application architecture I want to focus on just a few principles:

  • Application architecture is about changeability, not longevity
  • Application architecture decisions should be made closest to where they are implemented as possible
  • Application architecture design is not only about our ability to design a solution but also knowing what components, applications, and solutions already exist
  • The value for change in a component’s, application’s, or enterprise’s architecture is directly proportional to the cost of not addressing

If the structure is not able to change as the needs of our business change then the solution will become a liability. If someone other than those who are constructing components and applications are deciding on architecture decisions then there will be important information lost in translation between the two. If we keep using the same design hammer (ie. always using 3-tier or IoC for everything) then we are not allowing the strongest solutions to emerge. The value of architecture can usually be described by the cost incurred if it is not taken care of.

Well, let the onslaught of comments commence. I have put out my ideas on architecture. There are many more out there to discuss and I am sure I will hear some of them. I will finish this entry with a quote from Martin Fowler’s “Who Needs an Architect?” article:

“I define architecture as a word we use when we want to talk about design but want to puff it up to make it sound important.”

See You at SD West 2009

Posted by on 15 Jan 2009 | Tagged as: Acceptance Testing, Agile, Architecture, General, Leadership, Open Source, Product Owner, Scrum, Software Architecture, TDD, Travel, User Stories, XP

I’d like to invite you to join me at …

SD West 2009 Conference & Expo
March 9–13
Santa Clara Convention Center, Santa Clara, CA
http://www.SDExpo.com/

I’m pleased to announce that I’ll be teaching the following session at SD West 2009:

“Managing Software Debt: Continuing to Deliver High Value as Systems Age” on Friday, March 13th from 3:30-5:00pm

SD West is where the software development community gathers to learn about the latest business-critical technologies, network with peers, connect with innovative vendors and get inspiration from industry visionaries. The comprehensive conference program covers today’s most important topics including cloud computing, concurrent programming, dynamic languages, agile processes, security, testing and much more.

Super early bird conference discounts of up to $400 expire Friday, January 16.  As a speaker, I can also offer you an additional $100 off the VIP Pass. Simply register at http://www.SDExpo.com with the code 9ESPK to get your discount.

Check out all the excellent educational opportunities SD West 2009 has to offer at http://www.SDExpo.com.

I look forward to seeing you in Santa Clara!

Executable Design — A New Name for TDD?

Posted by on 13 Dec 2008 | Tagged as: Acceptance Testing, Agile, Architecture, General, Java, Scrum, Software Architecture, TDD, XP

For multiple years now I have thrown around the name “Executable Design” to describe Test-Driven Development (TDD) and how it is used for design rather than a test-centric tool. The name itself causes problems for those who are initially introduced to the technique. As a coach I was looking for a way to introduce it without stereotyping it as extra tests inhibiting more code getting delivered.

From my readings of multiple books, articles, and blog postings along with my own experiences with TDD the content of what I am about to distill is not new. This post is entirely about explaining the technique in a way that garners interest quickly. There are multiple pieces to “Executable Design” beyond the basic process of:

  • Red, Green, Refactor or
  • Write Test, Write Code, Refactor

These statements and the technique is the basis for practicing Executable Design but are not sufficient for describing the value and nuance of the practice. Not that I will be able to present it sufficiently in a single blog post but I want to present the basic principles.

While in a meeting with a team recently we were presented with a question I have heard often:

“Why should we use TDD?”

There are many reasons but generic reasoning alone is not sufficient. We discussed the safety net that good code coverage creates. We discussed the reason system tests do not take the place of unit tests. Then we started to touch on design and this is where it got interesting (and usually it does about this time for me). Before I can describe the rest of this discussion I want to present what lead up to this meeting.

A coach that I highly respect seemed a bit preoccupied one day when he wandered into my team’s area. I asked him what was going on and he told me that some of his issues with the current team he was coaching. He wondered why they were not consistently using TDD in their day-to-day development. The team had allowed a card saying “We do TDD” onto their Working Agreement and were not adhering to it.

I happened to know a bit about the background of this project that our company has been working on for over 2 1/2 years. There is a significant legacy codebase developed over many more years with poor design, multiple open source libraries included, and heavy logic built into relational database stored procedures. Also, just recently management on the client’s side had changed significantly and caused quite a shake up in terms of their participation and guidance of release deliverables. Yet the team was supposed to deliver on a date with certain features that were not well defined. This lead me to discuss the following situations that a coach can find their way into:

  1. You could come into a team that has limited pressure on features and schedule and has considered the impact of learning a new technique such as Executable Design. Also, they have asked for a coach to help them implement Executable Design effectively. This is a highly successful situation for a coach to enter.
  2. You could come into a team that has deadline pressures but has some leeway on features or vise versa and has considered the impact of learning a new technique such as Executable Design within their current release. Also, they have asked for a coach to help them implement Executable Design effectively. This is somewhat successful but pressures of the release rise and fall in this situation and may impact the effectiveness of the coaching.
  3. You could come into a team that has deadline pressures and has not considered implementing Executable Design seriously as a team. Also, they have NOT asked for a coach and yet they have gotten one. The coach and the techniques they are attempting to help the team implement may seem like a distraction to the team’s real work of delivering a release. This is usually not successful and please let me know if you are a person who is somewhat successful in this situation because we could hire you.

The current team situation seemed to be more like #3 above and therefore the lack of success in helping the team adopt TDD did not surprise me. Also, I started to play devil’s advocate and provide a list of reasons for this team NOT to do TDD:

  • At current velocity the team is just barely going to make their release date with the minimum feature set
  • Not enough people on the team know how to do TDD well enough to continue it’s use without the coach
  • The architecture of the system is poor since most logic is captured in Java Server Pages (JSP) and stored procedures
  • The code base is large and contains only about 5-10% test coverage at this time
  • It sometimes takes 10 times longer to do TDD than just add functionality desired by customer

This is not the full list but you get the picture. Don’t get me wrong, the list above begs to me the need for Executable Design but if the team does not have significant experience to implement it effectively it could seem overhead with little benefit to show for it.

After discussing this and more stuff that I won’t go into he told me about a couple of things that he can do to help the team. One of them was to work on minimizing the reasons for not doing Executable Design by discussing them with their ScrumMaster and actioning them on the impediments list. Some of those actions would go to upper management who get together each day and resolve impediments at an organizational level. One of the actions was to get our CTO and myself into a room with the team so they can ask the question “why should we do TDD?”.

Now we are in the room and most of the team members had been exposed to TDD through pairing sessions. Some of them had some ideas about where TDD was useful and why they thought it was not on this project. During the discussion one of the team members brought up a great discussion point:

“One of the problems with our use of TDD is that we are not using it for improving the design. If we just create unit tests to test the way the code is structured right now it will not do any good. In fact, it seems like we are wasting time putting in unit tests and system tests since they are not helping us implement new functionality faster.”

This team member had just said in the first sentence what I instinctually think when approaching a code base. The reason to do TDD is not just to create code coverage but to force design improvement as the code is being written. This is why I call TDD and its best known principles and practices of applying it Executable Design. If you are not improving the design of the application then you are not doing Executable Design. You might be just adding tests.

Some of the principles I have found to help me in applying Executable Design effectively are (and most, if not all, of these are not something I came up with):

  • Don’t write implementation code for your application without a failing unit test
  • Separate unit tests from system and persistence tests. (as described in this previous blog entry)
  • Create interfaces with integration points in a need-driven way (as described in this previous blog entry)
  • Always start implementing from the outside in (such as in Behavior-Driven Development and as described in this previous blog entry)
  • Mercilessly refactor the code you are working on to an acceptable design (the limits of which are described in this previous blog entry)
  • Execute your full “unit test” suite as often as possible (as described in this previous blog entry)
  • Use the “campground rules” of working in the code: “Leave the site in better shape than when you arrived”
  • Create a working agreement that the whole team is willing to adhere to, not just what the coach or a few think is the “right” agreements to have.

Try these out on your own or with your team and see how they work for you. Modify as necessary and always look for improvements. There are many thought leaders in the Agile community that have written down important principles that may work for you and your team.

And finally, now that I have filled an entire blog post with “Executable Design” what do people think about the name? It has worked for me in the past to explain the basic nature of TDD so I will use it either way unless others have better names that I can steal?

Managing Software Debt presentation @ Agile Vancouver

Posted by on 10 Nov 2008 | Tagged as: Agile, Architecture, DotNet, General, Java, Leadership, Product Owner, Scrum, Software Architecture, TDD, XP

On November 6th I presented an updated version of the Managing Software Debt talk at Agile Vancouver “Much Ado About Agile” conference. This is a link to the presentation deck:

Managing Software Debt – Agile Vancouver (PDF)

I was honored to present at this local conference and had a great time meeting up with old friends and getting to know some new ones. I hope that I can do this again soon. If you are interested in more information and other presentations at Agile Vancouver you can go to their home page.

Managing Software Debt – article

Posted by on 20 Oct 2008 | Tagged as: Acceptance Testing, Agile, Architecture, General, Leadership, Product Owner, Scrum, Software Architecture, TDD, XP

Managing Software Debt
Continued Delivery of High Values as Systems Age

Many software developers have to deal with legacy code at some point during their careers.  Seemingly simple changes are turned into frustrating endeavors.  Code that is hard to read and unnecessarily complex. Test scripts and requirements are lacking, and at the same time are out of sync with the existing system. The build is cryptic, minimally sufficient, and difficult to successfully configure and execute. It is almost impossible to find the proper place to make a requested change without breaking unexpected portions of the application.  The people who originally worked on the application are long gone.

How did the software get like this?  It is almost certain the people who developed this application did not intend to create such a mess.  The following article will explore the multitude of factors involved in the development of software with debt.

What Contributes to Software Debt?

Software debt accumulates when focus remains on immediate completion while neglecting changeability of the system over time. The accumulation of debt does not impact software delivery immediately, and may even create a sense of increased feature delivery. Business’ responds well to the pace of delivered functionality and the illusion of earlier returns on investment. Team members may complain about the quality of delivered functionality while debt is accumulating, but do not force the issue due to enthusiastic acceptance and false expectations they have set with the business. Debt usually shows itself when the team works on stabilizing the software functionality later in the release cycle. Integration, testing, and bug fixing is unpredictable and does not get resolved adequately before the release.

The following sources constitute what I call software debt:

  • Technical Debt[1]: those things that you choose not to do now and will impede future development if left undone
  • Quality Debt: diminishing ability to verify functional and technical quality of entire system
  • Configuration Management Debt: integration and release management become more risky, complex, and error-prone
  • Design Debt: cost of adding average sized features is increasing to more than writing from scratch
  • Platform Experience Debt: availability and cost of people to work on system features are becoming limited

Software Debt Creeps In

Figure 1.1: A relatively new system with little debt accrued.

Figure 1.1 displays a system that has minimal amount of software debt accrued. A few low priority defects have been logged against the system and the build process may involve some manual configuration. The debt is not enough to significantly prolong implementation of upcoming features.

Business owners expect a sustained pace of feature development and the team attempts to combine both features and bugs into daily activities, which accelerates the accrual of debt. Software debt is accruing faster than it is being removed. This may become apparent with an increase in the number of issues logged against the system.

Figure 1.2: An aging software system slowly incurs significant debt in multiple functional areas.

As a system ages, small increments of software debt are allowed to stay so the team can sustain their velocity of feature delivery. The team may be complaining about insufficient time to fix defects. Figure 1.2 shows a system that has incurred software debt across all functional areas and components.

At this point, delivery slows down noticeably.  The team asks for more resources to maintain their delivery momentum, which will increase the costs of delivery without increasing the value delivered. Return on investment (ROI) is affected negatively, and management attempts to minimize this by not adding as many resources as the team asks for, if any.

Even if business owners covered the costs of extra resources, it would only reduce the rate of debt accrual and not the overall software debt in the system. Feature development by the team produced artifacts, code, and tests that complicate software debt removal. The cost of fixing the software debt increases exponentially as the system ages and the code-base grows.

Figure 1.3: The aging system has accrued significant debt in all functional areas and components.

Software debt in the system continues to accrue over time, as shown in figure 1.3. At this point, new feature implementation is affected significantly. Business owners may start to reduce feature development and put the system into “maintenance” mode. These systems usually stay in use until business users complain that the system no longer meets their needs.

Managing Software Debt

There are no magic potions for managing software debt. Software can accrue debt through unforeseen circumstances and shortsighted planning. There are some basic principles that help minimize software debt over the lifespan of the product:

  • Maintain one list of work
  • Emphasize quality
  • Evolve tools and infrastructure continually
  • Always improve system design
  • Share knowledge across the organization
  • And most importantly, hire the right people to work on your software!

Maintain One List of Work

One certain way to increase software debt is to have multiple lists of work. Clear direction is difficult to maintain with separate lists of defects, desired features, and technical infrastructure enhancements. Which list should a team member choose from? If the bug tracker includes high priority bugs, it seems like an obvious choice. However, influential stakeholders want new features so they can show progress to their management and customers. Also, if organizations don’t enhance their technical infrastructure, future software delivery will be affected.

Deployed software considered valuable to its users is a business asset, and modifications to a business asset should be driven from business needs. Bugs, features, and infrastructure desires for software should be prioritized together on one list. Focus on one prioritized list of work will minimize confusion on direction of product and context-switching of team members.

Emphasize Quality

An emphasis on quality is not only the prevention, detection, and fixing of defects. It also includes the ability of software to incorporate change as it ages at all levels. An example is the ability of a Web application to scale. Added traffic to the web site makes performance sluggish, and becomes a high priority feature request.  Failed attempts to scale the application result in a realization that the system’s design is insufficient to meet the new request. Inability of the application to adapt to new needs may hinder future plans.

Evolve Tools and Infrastructure Continually

Ignoring the potential for incremental improvements in existing software assets leads to the assets becoming liabilities. Maintenance efforts in most organizations lack budget and necessary attention. The International Organization for Standardization (ISO) standardizes on four basic categories of software maintenance in ISO/IEC 14764[2]:

  • Corrective maintenance – Reactive modification of a software product performed after delivery to correct discovered problems
  • Adaptive maintenance – Modification of a software product performed after delivery to keep a software product usable in a changed or changing environment
  • Perfective maintenance – Modification of a software product after delivery to improve performance or maintainability
  • Preventive maintenance – Modification of a software product after delivery to detect and correct latent faults in the software product before they become effective faults

Most maintenance efforts seek to prolong the life of the system rather than increase its maintainability. Maintenance efforts tend to be reactive to end user requests while business evolves and the technology decays.

To prevent this, attention must be given to all four categories of software maintenance. Budgets for software projects frequently ignore costs for adaptive, perfective, and preventive maintenance. Understanding that corrective maintenance is only part of the full maintenance picture can help an organization manage their software assets over it’s lifespan.

Improve System Design Always

Manage visibility of system design issues with the entire team. Create a common etiquette regarding modification of system design attributes. Support the survival of good system design through supportive mentoring, proactive system evolution thinking, and listening to team member ideas. In the end, a whole team being thoughtful of system design issues throughout development will be more effective than an individual driving it top down.

Share Knowledge Across the Organization

On some software systems there is a single person in the organization who owned development for 5 years or more. Some of these developers may find opportunities to join other companies or are getting close to retirement. The amount of risk these organizations bear due to lack of sharing knowledge on these systems is substantial.

Although that situation may be an extreme case of knowledge silos, a more prevalent occurrence in IT organizations is specialization. Many specialized roles have emerged in the software industry for skills such as usability, data management, and configuration management.  The people in these roles are referred to as “shared resources” because they use their specialized skills with multiple teams.

Agile software development teams inherit team members with specialized roles, which initially is a hindrance to the team’s self-organization around the work priorities. Teams who adhere to agile software development values and principles begin to share specialized knowledge across the team, which allows teams to be more flexible in developing software based on priorities set by business. Sharing knowledge also reduces the risk of critical work stoppage from unavailable team members who are temporarily on leave.

Hire the Right People!

It is important to have the team involved in the hiring process for potential team members. Teams will provide the most relevant skills they are looking for, thus, allowing them to review and edit the job description is essential. Traditional interview sessions that smother candidates with difficult questions are insufficient in determining if the candidate will be a great fit. Augmenting the interview questions with a process for working with the candidate during a 1 to 2 hour session involving multiple team members in a real-world situation adds significant value to the interview process. Before hiring a candidate, teams members should be unanimous in the decision.  This will increase the rate of success for incorporation of a new team member since the team is accepting of their addition.

Another significant hiring focus for organizations and teams is placing more emphasis on soft skills than technical expertise. I am not advocating ignoring technical experience. However, it is critical in an agile software development organization or team to have people who can collaborate and communicate effectively. Soft skills are more difficult to learn than most technical skills. Look for people who have alignment with the hiring team’s values and culture.

In Summary

As systems age they can become more difficult to work with. Software assets become liabilities when software debt creeps into systems through technical debt, quality debt, configuration management debt, design debt, and platform experience debt.

Applying the six principles in this article will lead to small changes that over time will add up to significant positive change for teams and organizations.  The goal of managing software debt is to optimize the value of software assets for our business and increase the satisfaction of our customers in the resulting software they use.

References

  1. Ward Cunningham – “Technical Debt” – http://c2.com/cgi/wiki?TechnicalDebt
  2. “ISO/IEC 14764: Software Engineering — Software Life Cycle Processes — Maintenance” – International Organization for Standardization (ISO), revised 2006

Next Page »