The Essential Infrastructure
Source-code control, unit testing, daily builds and bug tracking—using these support pieces daily can help you deliver robust code on a reasonable schedule.
I am convinced that only inertia and sloth prevent the universal adoption
of these tools; the technical difficulties are no longer valid excuses.
—Frederick P. Brooks, Jr.,
The Mythical Man-Month
Sure, you know how to write perfect code but what good does that do you
when you can't get it to market? In this column, I'll discuss the scaffolding
that makes it possible to go from code to product, including source code
control, unit testing, daily builds and bug tracking. If you're not using
these support pieces in your daily work, it's likely that you could be
doing a better job of delivering robust code on a reasonable schedule.
Knuckles off the Ground, Please
Brooks was writing about high-level languages and interactive programming,
both radical advances at the time his book, The
Mythical Man-Month: Essays on Software Engineering (Addison-Wesley),
was published 21 years ago. But the sentiment applies equally well to
the sorts of tools that go along with today's code production. There seems
to be a sentiment among some coders that "harder is better" or, perhaps,
"real coders don't need that sissy stuff." Real coders, in this view,
do everything from the command line, don't need an IDE, and wouldn't be
caught dead using unit tests. For all I know, they also walk around with
their knuckles scraping the ground making grunting noises at one another.
If you're the sort of software developer who doesn't want to be bothered
with any process at all outside of the editor, you have to ask yourself
one simple question: Do you want to write code or do you want to
ship code? The former is a whole lot easier than the latter, as
very smart people have proved time and again. Sadly, they've mostly proved
this by failing to ship products. Even Microsoft (where, whatever else
you think about the company, it ships a lot of code) has had projects
mire and die. Entire versions of Word and Access never made it from "neat
code" to "shipping product."
Don't get me wrong. Putting the perfect process and perfect tools in
place will not guarantee shipping code. (Developer and author Steve McConnell
calls it "cargo cult software engineering"; see http://www.stevemcconnell.com/ieeesoftware/eic10.htm.)
But not having some sort of process in place will make it much harder
than it has to be to actually ship code. Fortunately, tools have progressed
to the point where it's pretty easy to put together a working process
for small projects.
Source Code Control
Start with source code control. To write a decent-sized application,
you need to be able to experiment and change things in your code. But
what do you do when your changes lead you down the rabbit hole? I've seen
developers keep a separate "before" copy of their project in another directory,
but really, this is exactly what source code control is good for. Put
the project under source code control, check out modules to work on them,
check them back in when you're done. Then if something does go wrong,
just roll back to a known good version.
If you're using Visual Studio, you've already got Visual SourceSafe available.
There are many other alternatives out there, too, from CVS
to Perforce. Depending on your
working habits and whether your team is local or distributed, you may
need to evaluate a few alternatives to find one that works for you. But
it's worth the effort. Once you don't feel like you're in danger of blowing
up your entire project, it's a lot easier to explore when you need to
One word of advice: Keep the time between check-ins short. Checking out
your entire project and then keeping everything checked out all the time
will prevent you from doing effective, granular rollbacks.
Unit testing is one of the practices that very few developers I
know follow. That's because most of us think that continuously testing
every little thing is a waste of time. However, having recently tried
using "test-first development" on a couple of projects, I have to say
that it works for me. Here's the rough outline of this process:
- When you're ready to add a new feature to your product, first write
the test for the feature.
- Verify that the test fails before you write the code.
- Write the code.
- Verify that the test works—and that all of your other tests still
Sounds crazy, doesn't it? But with the right tools, it's amazingly fast;
and the productivity improvement can be dramatic. The key is to have something
that makes it easy to write tests, and easy to run them. For Visual Studio
.NET work, I'm using NUnit,
which fits the bill for me—writing a new test is just adding a new
method to a class.
Yes, you can end up with a lot of tests this way. Right now I'm working
on a project with about 700 lines of code and (currently) 353 unit tests.
That's 353 things that I know can go wrong, and 353 things that I know
are going right. And it's amazing how much safer I feel about working
on this code than I do on other projects where I don't have a good test
The Build Process
You need to have a repeatable way to go from source code to shipping
bits. This seems obvious—that's what Build, Rebuild Solution is for,
right? Well, for all but the simplest projects, that won't do. For example,
on one current project my build process has these steps:
- Make sure everything is checked in to source code control.
- Build the solution using the Release configuration.
- Build the solution using the Lite configuration.
- Build the unit test project.
- Run NUnit to execute the unit tests.
- Run NDoc to create developer
- Build the setup project and put it on a share for testing.
How much work does it take to do these steps? In my case, clicking one
button. That's because I've set up an automated build process. The process
performs each step, saves any error messages, aborts the build if the
errors are serious, and saves an XML build log for me to inspect. I'm
for this job, but you could equally use makefiles or NAnt
or many other systems. The key is to make sure that you automate your
entire build process, with as few manual steps as possible (ideally, none).
That way, everything happens on each build, and you'll never be faced
with explaining "oh, I just forgot to replace the placeholder file with
the real one" when you ship the wrong bits to a customer.
Finally, you need to have a bug-tracking system in place if you
write code with bugs (as we all do). At a minimum, you need to keep track
of what bugs are reported and whether they're fixed. Beyond that, there
are many things that are nice to have: links to the source code control
system, e-mail notification if you're working in a team, and more. Here
I don't have a particular recommendation; I'm mostly using a home-rolled
Access database for bug tracking at the moment (though I recently gave
FogBugz a spin, and it
looks good for distributed team development).
But whatever means you use, track those bugs! And as you fix them, write
the unit tests that would have caught them in the first place. We all
have bugs in our code; that's normal. But you should never have the same
bug show up again after it's been fixed.
So that's my four-part program for getting your code out the door: control
the source, control the tests, control the build, control the bugs. Any
reasonably disciplined developer should be able to manage this. Where
it gets interesting is when you carry these processes to extremes. As
you might expect, extreme programming (XP) does just that, with a notion
called "continuous integration."
What continuous integrations does is combine the check-in, build and
test processes into one big process. An automated process monitors the
source code control repository, and when it spots a changed file, it kicks
off the build process and then the test process. You can read a good explanation
of how this works in practice on Martin Fowler's Web site at http://www.martinfowler.com/articles/continuousIntegration.html.
I haven't yet tried this on any project, but it's certainly worth thinking
about. By reducing the manual steps even more, continuous integration
should contribute to the goal of having working executables built from
the latest source code at all times. When I try it, I'll report back.