Archive for April, 2008

Redefining Test Terminology

Sunday, April 27th, 2008

So, earlier I mentioned Mock Objects — The TDD variant that requires isolation of the tested component. The Mock Objects technique requires a hierarchy of tests:

  1. At the lowest level, objects are tested in isolation against mocks.
  2. Higher up, objects are tested in collaboration with a few other real objects.
  3. Higher up, the entire system is tested.
  4. At the top level, test define if the system is complete.

That’s a good hierarchy. My only problem is with the names that they use.

Mock Object developers use these names:

  1. Unit
  2. Integration
  3. System
  4. Acceptance

Long before that, I head people use these names:

  1. “Why are you testing at such a low level? Don’t waste your time.”
  2. Unit
  3. “We probably ought to run the whole thing on some sample data.”
  4. Acceptance

On top of which, there are two other things called “Integration Tests”, which long predate Mock Objects:

QA Integration Tests: We’ve upgraded to the latest version of of GUI library. Let’s run some integration tests to make sure nothing broke. (Usually a full regression sweep.)

(Pre-Agile) Developer Integration Tests: We’re merging three new feature branches on Tuesday. We’d better do some testing to make sure they integrate properly.

At this stage of the game, it’s just not a good idea to re-use existing terms for different things. Especially when telling the difference between those things requires you to read the mind of the speaker.

Families of Test Driven Development

Thursday, April 24th, 2008

So I’ve been looking in at Test Driven Development more lately, and I’m trying to separate the several things that call themselves TDD. Here’s what I’ve got so far:

Classic TDD: as described in Kent Beck’s Test Driven Development by Example, this technique tests and writes code a a very low level. No line of code is written without a failing test. Design comes from refactoring to remove duplication.

Mock Objects: Steve Freeman and co. promote this version of TDD. Its distinguishing feature is a requirement for near-complete isolation of objects under test. Design happens during the writing of the test when new collaborators are discovered.

The Un-named TDD: Maybe we should call it “Test-First”. Medium size tests are written, often close to the level of a use case or scenario. Corner cases are usually not tested. (Disk is full, so you can’t write the file, etc…) Just enough testing is done to verify that the system works. Design is done via refactoring, but the decision of how to refactor is based on the developers taste, not just on removing duplication.

The Un-named TDD (with Mocks): Some people combine the convenient mock generation frameworks of Mock Objects with the unnamed TDD. However, they use them for conveniently removing parts of the app that are annoying to configure, not for isolating the system under test.

Presenter First: Maybe just a sub-species of Mock Objects, this strategy is notable for recommending that the application be built outside-in.

Behavior Driven Development: I know this is related, but I don’t know enough about it to say anything.

I used to do the unnamed TDD, and have only recently discovered that I wasn’t doing Classic TDD. I’ve been trying out Classic TDD, and it’s very intriguing. I learned TDD from a combination of Wiki, XP Explained, Refactoring, and XP Installed. (Sort of.) I wonder if those older sources don’t contain as good a description of it as the TDD book, or if I just missed it.

Flashback: How To Develop Software Faster

Friday, April 18th, 2008

In 2006 I wrote an article with Matt Heusser about how to develop
software faster. It’s available here: How to develop software faster

Mad props to my Peeps

Tuesday, April 15th, 2008

No man is an island. If I have any good ideas, it’s because I have good people to bounce them off of. I have to put out a major shout-out here to my some-time partner in crime, Matt Heusser. We’ve worked together on a few projects, and it’s always come out great. He’s a sharp dude.

Also, you should apply to go to his Workshop on Technical Debt. I can’t go because I have a prior engagement, but it definitely looks like a good deal.

Unit Test Abuses — Who Needs Mocks?

Friday, April 11th, 2008

So, when I fix a bug, I like to wrap a test around it to catch regressions. When I wrote the test for the bug discussed under Debugging adventures with DBD::Sybase, I had to make an unusual work-around…

eval {
    # ick. ick. ick. You didn't see me do this.
    no warnings 'redefine';
    local *App::is_schedule_found = sub { pass("called bogus func");  return (1, 'ok'); };
    ($ok, $msg) = App::process_row($dbh, $questionable_row);
    ok($ok, "processing row: $msg");
unlike($@, qr/^Panic: Can't have multiple statement handles on a single database handle when AutoCommit is OFF/,
        "Sybase Error Handling busted");

I’ve changed the package name to “App” to protect the guilty. process_row calls is_schedule_found, and bails out if it’s not found. Since this bug only show up when testing against the real database, I can’t fake out the data source. So I insert the appropriate prerequisite data, and then run the procedure. But I don’t want to build up the whole structure just to adjust one table. (The database doesn’t have integrity constraints… Not my preference, but that’s how it is today.)

So I replace the function that checks for the schedule with one that always returns true. I don’t need to dependency inject a mock with an interface, I just adjust the global symbol table.

Debugging adventures with DBD::Sybase

Tuesday, April 8th, 2008

So, I recently debugged an error in some code I inherited. It was a relatively bizarre corner case that I thought might be interested.

The code is Perl running on windows, accessing a sybase database using DBI and DBD::Sybase.

This was the basic structure of the code:

 sub process_record {
	my $record = shift;

	if (is_record_in_db($record)) {
		if (mark_existing_record_deleted($record)) {
	} else {

There was some other stuff in there too, but it doesn’t matter for this story.

This code had been in production for about a year, and it’s been working fine. Suddenly we start getting this error:

Panic: Can't have multiple statement handles on a single database handle when AutoCommit is OFF

It also gives a line number from the bowels of DBD::Sybase. I try to get a full stack trace by running it under -MCarp=verbose . But, DBD::Sybase doesn’t croak, it dies, so I can’t see which call of mine is killing it. I don’t feel like hitting the debugger yet, so I think some more.

Now, this error comes only after we’ve processed 5000 or so records, so I take a look to see what’s different about the record where it fails, and I notice that it takes the path which calls mark_existing_record_deleted, while none of the others do. (Inserts heavily outnumber updates on this application.)

(Note: all of the called routines access the database.) DBD::Sybase doesn’t allow you to have multiple statement handles. So I look in the code and see that, sure enough, mark_existing_record_deleted doesn’t have a call to $sth->finish . But neither does is_record_in_db! Now I’m wondering, “how does this thing work at all?”

The answer is, that DBD::Sybase automatically finishes the handle for you when you read past the last row. So when there’s no data in the database, the query in is_record_in_db returns no rows. One fetch is “past the end,” so the handle gets closed. But when there is data, the query handle is open, causing the application to die when you try to open another handle in mark_existing_record_deleted.

So, watch out for that when you write code against Sybase.

Bacon Driven Coding — But Seriously

Friday, April 4th, 2008

OK, so bacon driven coding isn’t real. (What, you couldn’t tell?)

Some time last year, Matt and I were having a conversation about how to do development, and it was noted that the word “driven” had become quite popular in the software development world:

  • Test Driven Development,
  • Responsibility Driven Design,
  • Behavior Driven Development,
  • Domain Driven Design
  • Context Driven Testing

The list was notable. A Comment was made that you could sell anything as long as it was structured as “X Driven Y”. A quick search was made for the most ridiculous X and Y, and bacon driven coding was the result.

In a fit of giggles, I registered the domain name, and the rest is history.

Bacon Driven Coding FAQ

Wednesday, April 2nd, 2008

Q: Should I eat all my bacon at once, or should I space it out during the day?
A: Different programmers have different reactions; Some people require quite a lot of bacon to get the initial bacon rush, but once they get it, it lasts all day. Others get a short rush from a few bites of bacon. Experiment to see what works best for you.

Q: I’m a vegetarian. Can I use vegetarian bacon?
A: No. Bacon driven coding only works on real bacon. It’s not called tofu driven coding.

Q: What about sausage?
A: Some people are experimenting with sausage to see if they can produce similar results. If you have any success with this, let us know.

Q: Don’t the two answers above kind of contradict eachother?
A: Yes.

Q: Can you use bacon driven coding in a non-object oriented framework?
A: Yes! Unlike so many competing methodologies, which only work with a single software development paradigm, bacon driven coding can be used with any kind of technology: From ajax javascript to mainframe cobol and even assembly language, bacon driven coding will help any technology.

Q: Can I use bacon driven coding with legacy code?
A: Yes! Unlike other strategies, you can gain the benefits of bacon driven coding on your existing code base. No more clunky test retrofits and refactoring, or building UML Models to match the existing system — Just add bacon and go.

Q: What about the health risks?
A: Eating large quantities of bacon can pose health risks. Weight gain is the most likely, but other problems have been reported. Overall, however, we believe that the stress reduction gained by all the time saved through bacon driven coding should mitigate the risk for most people. After all, when you finish a week’s worth of work on Monday afternoon, you have lots of time left in the week to spend exercising to reduce your health risks.

Q: Does bacon driven coding work well with pancake driven design, orange juice driven requirements, or toast driven testing?
A: Some practitioners have reported success with this combination, leading them to laud the virtues of a whole-lifecycle process of “breakfast-driven development”. While we are excited about the possibilities of this development, we don’t want to overpromise, so we stick to the reliable formula that so many have had success with: bacon leads to better code.

Bacon Driven Coding

Tuesday, April 1st, 2008

Bacon driven coding is a software development technique where source code is written based on the burst of skill and productivity that a programmer gets when under the influence of bacon.

To practice bacon driven coding, follow these three simple steps:

  1. Eat bacon
  2. Write code
  3. Repeat

We call this the bacon-code-repeat cycle.

Please see the bacon driven coding FAQ for more information.