I spent last week extracting some features from CrowdVine into a stand alone simple CMS service I’m calling big.ly. I gave myself a fixed time period–one week starting on Monday and ending Sunday.

In engineering management terms, a project like this is called time boxed development. It comes up all the time, especially in Agile development methodologies. The idea is that you commit to a time period and production-ready code–but are allowed to adjust the scope of your work as you go. So for last week’s example, a successful result could have been as simple as a page describing the project and a contact-me-when-you-launch form. If, instead, I had built all of my dream features but a bug prevented me from launching–that would be failure.

Working in a time-boxed way is actually a skill and something I’ve practiced a bit. I like having the time box because it makes it much easier to make decisions. Those decisions almost always come down to: take the simpler option. You spend less time going down rabbit holes and more time reacting to actual feedback on live features.

The first time I heard about time boxed development was at a users conference for an XML database company. They used six month time boxes. We used Scrum at Odeo, which is an iterative approach where work is pulled from a feature backlog and then performed in “sprints” that are usually two or four weeks long. I did some agile project management training and got to see how this works in the corporate world (good for engineers, bad for antsy product managers). Then I tried two-day projects in order to refine my own development practices (IHeartQuotes was probably the most successful project to come out of that).

Along the way, I’ve learned a couple lessons.

Activity is not accomplishment
The hardest part of being a first time Scrum manager was not knowing how to help people adjust the scope of features they were working on. I spent a lot of time telling people that they’d bitten off more than they could chew, that they weren’t good enough to finish on time, that they might as well give up, etc. Nobody would ever listen to me. The engineers were cocky and ambitious and were shooting for the moon.

I think they also thought I was being a jerk by not believing in them. They wanted to be challenged and I was doing the opposite by trying to get them to scale back their ambitions.

I think this is a failure of Scrum. It was set up to live in a world where you have to commit to features. So it uses time boxing to help give some visibility to the development process, but at the end of the day everyone knows they’re judged on how many features they check off. So that’s what ambitious programmers shoot for–do as much as possible, even at the risk of not having releasable code at the end of an iteration.

But that activity, checking features off or having code in a half-working state, doesn’t have any inherent value. Luckily, there’s been a huge move toward metrics driven development. That gives a better way to challenge yourself–instead of how many features you can build, you can judge yourself on how much value you’ve created. Did your change effect the conversion rate? Did it change results from customer surveys? Did it increase retention?

If I was judging my CMS project by features, I’d be ticked by some of the things I didn’t finish (no contact forms and subscriptions aren’t properly wired to a merchant account). I might even have risked the entire iteration by chasing for those missing features. Instead, I worked through a series of accomplishments. The first was being able to move tonystubblebine.com over. Check. The second was to have something that I could pass to the rest of the CrowdVine team to move our marketing pages over. Check. The third was having something I could send to a few people for feedback. Check. Those are all accomplishments, and more valuable than having reams of code that nobody uses.

Release half-way through your iteration (or earlier!)
There’s a classic pattern where you bite off more than you can chew and leave yourself a ton of work at the end. The problem isn’t just having an unrealistic amount of work on the final day, it’s that running into a single problem on the last day will sabotage the entire iteration.

Instead of doing this, you need to scope a version of the work that is releasable as early as possible in the iteration. That way you have plenty of time for gold plating.

Even better, releasing early leaves you time for A/B testing.

Every time I’ve been involved in somebody else’s time boxed development there’s been this huge rush to finish as many features as possible at the end. Scrum calls its iterations sprints, and that’s exactly what it feels like. Beside feeling bad (suck it up, pansy), this doesn’t leave any time for people to catch their breath and actually check if any of the features were creating any value.

Releasing late in the time box almost always means you’re stuck in a pattern of release and abandon. My favorite personal example is the first version of CrowdVine’s private messaging feature. It’s now basically one of the most popular features on the site and gets used a couple of times by every member. However, the first version of the feature was live for six months and was only used three times total. It was in an awkward place and was trying to do too much. I built it, released it, and moved on to other features. I’d mistaken activity for accomplishment.

Minimum Viable Product vs. Half-a-product
Even though this post is sparked by this product I built last week, I’m not really talking about the product much. That’s because what that product needs right now is not a ton of users–it needs a little bit of feedback. And I got that by just sending it off to a few people. By all means, try it if you want, but it’s what I would call a minimum viable product. It’s a product that’s good enough to get useful feedback on but not so polished that it’s worth building a marketing campaign around.

In comparison, 37signals promotes the idea of building half a product that kicks ass, rather than a half assed product. However, like writing a short essay, a great half-product takes a lot of work.

Time boxing can get you there, but usually not in one go round. When you scope the product down and get it in front of users early you find out what features were necessary and which weren’t. You also find out which features need to be improved and which are already working well.

If you are trying to build a high quality product inside of a single time box, then you need to be very careful to polish one piece before moving on to the next. My friend Luke gave me a phrase for naming your definition of polished called, “Done, done.” That way when someone tells you they’re done, you can respond by asking “Are you done, done?”

Two weeks is good
I’ve done development iterations of two days, one week, two weeks, three weeks, and four weeks.

Four weeks is too long. You end up working on a bunch of unrelated things. Two days is too short and I’ve never felt like I could really do anything of significance in that time. I just use two day projects if I’m having trouble carving out time.

Three weeks weirds everyone out because it doesn’t line up to anything regular on the calendar.

Last week was the first time I’ve done a one week project. I thought the project was a perfect size, but I also found it almost impossible to balance it with other work responsibilities. I had phone calls. I spent a day at the Game Developers Conference. I organized a big party for the weekend.

My ideal would be to pick a project that looked like it had about 40 hours of work and that had a legitimate accomplishment that could be completed in the first 8-10 hours. I’d take that project and give it two weeks of calendar time. There’s not a lot of harm in finishing early because it’s not like there’s ever a shortage of programming work. If the project finishes early then make tweaks based on usage measurements, do some A/B testing, move on to code cleanup and test coverage, etc.

Changes for next time
I really liked making the time for this and I’m definitely going to do it again. The next product I have lined up is an extension to CrowdVine. Here are some of the things I’m going to change.

I’m going to move to a two week time box but keep the new feature work to the first week. That way I can still call it product-in-a-week, but I have some flexibility to keep the rest of my business running.

I’m also going to spend more time up front defining what accomplishments we’re shooting for. I think I didn’t do a good enough job of that when I designed my simple CMS service. I was definitely at risk of falling into the feature trap because I really wanted a product the whole world could use (built in just a week!), but I’d be better served if I had more of my sites already ported over.

For example, instead of porting over the CrowdVine pages, I spent time on a subscription billing system. Unfortunately, since I didn’t finish the subscriptions by wiring them to a payment system, that was work without value.

Probably the biggest change is that I’m going to involve more of the CrowdVine crew. I did most of this work solo, but the next project more naturally fits a bigger team.

I think that’s it for changes. Does anyone else have strong feeling about time boxing?

(WARNING: This is another bloglines link dump surrounded by poorly disguised context!)

Traditional software development usually calls for dedicated QA. When I went to Odeo I knew that the engineers there wanted to work in an agile environment. But Odeo had also just hired a traditional QA person coming from a big-software QA group. I’d never heard how people integrated dedicated QA into an agile group so I went hunting for relevant blogs. One that I liked was Agile Testing by Grig Gheorghiu.

Grig’s writing is skewed toward Python but there’s still plenty of gems for the rest of us. Here were the links I had bookmarked in blogines:

Useful tools for writing Selenium tests
Discussion of tools that make it much easier to work with Selenium, a program for writing GUI-based testing for web apps.

Extreme Programming and QA With Kent Beck
A pointer to this QA Podcast interview.

Xooglers – A Blog By Ex-Googlers
Not testing related, but sometimes it’s good to see how and how far a person strays from their blog subject. These are ex-google guys commenting on google on a google-owned blogging platform.

I’m still not entirely sure what the role of dedicated testing is for an agile group. When you’re iterating the product design quickly I think the testing you want is all automated, so that you know quickly if you’ve broken something important (quickly being within 5 minutes). But after you’ve gone through several months of quick product iteration I think you want a thorough QA pass to mark all the rough edges. So either the engineeers the write automated tests as they go and then you bring in contract QA at the end or you hire someone who can do both.

This might be a good time to mention that Rabble, formerly of Odeo, is writing a Testing Rails blog (as well as a testing rails book for O’Reilly).

Odeo just finished week one of Scrum, our new engineering and product development methodology. In Scrum, development is focused around month-long Sprints with the goal of delivering as much production-ready functionality as possible. It’s a management dream, engineers are constantly sprinting all-out and releasing massive amounts of features. No more worrying about motivation or snickering about herding cats. Engineering at full throttle. Hurray for management.

The top question from Odeo-ers hearing about Scrum was, “How much rest do we get between Sprints?” Normal Scrum methodology finishes the Sprint with a wrapup meeting, then the next Sprint starts with an all-day planning meeting, then it’s back to coding. In our case we’ll also have a weekend between since we’re doing Scrum on a 28 day cycle. So that’s four days of rest (non-coding) one of which is an all day meeting.

I guess that does sound intense.

I’d been thinking of Scrum as a marathon, as a way to keep focus between mile markers. You focus on one goal for four weeks then, take a refill of requirements, then settle in for the next four weeks.

Two problems with that. First, it’s called a Sprint so you’ve got to expect people to get pretty intense about effort and expectations. Second, it leaves engineering almost no time for context switching.

The latter is pretty important to me. We’re a small company. We rely on engineers making smart decisions and contributing feature ideas. Back-to-back sprints clearly doesn’t allow for this. What to do?