Read Fullscreen

Engineering is not the bottleneck

I'm Peter, a Rails developer and entrepreneur. I'm an ex-computer vision researcher, the former CEO of Clickpass and a YC alum. I have built and sold a lot of different software and am now the CTO of Brojure.com - which lets you quickly create stunning sales pitches

Most slow software companies believe that the engineering department is what slows them down.

I’ve always found this odd though. The companies I’ve heard complain about it were indeed slow and features did indeed take two or three months to create, however engineering didn’t actually account for very much of that. Often no more than a week or so.

image

Not only was the engineering time relatively brief but so were the other stages. UX and UI would only take three or four days between them and QA was maybe a day at the end. Despite a total development time of only two weeks, features still took two or more months to become a reality. Why was this?

It happened because features spent most of their time simply waiting around in queues. The product department had a backlog of requests from biz-dev, the design team had wireframes waiting to be made beautiful, engineering had specs waiting to be made whole. Every step involved a queue and an associated delay.

Product gets delayed by queues not by engineering

It was the best of intentions which created these queues. The first was the fear that engineering would “run dry” and the second was an understandable but misguided belief that backlogging specs and designs helps things move faster.

It was primarily the fear of an unoccupied engineering team that underlay things though. Engineers are expensive and hard to recruit and their salaries go out of the door like blood leaking from a severed artery. With all of that investment, the last thing you want is for such expensive machinery to sit idle on the shop floor.

Both managers and product managers felt that if engineering wasn’t humming then they hadn’t been doing their jobs. CEOs and department heads only tended to exacerbate things. I was once told in no uncertain terms that, “if engineering isn’t working on something then someone will have to answer for it”. Which doesn’t seem unreasonable until you see the consequences.

The reality of product development is that it’s less about production and more about reduction. Reduction of the myriad ideas that don’t work and exploration of the few that do. Product development is less of a design process and more of an evolution process. And a critical part of the evolution process is the rate at which generations die or to be more accurate, the rate at which they reproduce.

Startups live and die by Cycle Time not by engineering throughput

When geneticists study heredity and evolution they often do so using fruit flies. One of the main reasons for this is that fruit flies reproduce very quickly so the the observer can record evolutionary effects in real time.

If you ever hear a great founder speak, they don’t usually talk about a small number of brilliant ideas, they talk about a large number of very sensible ideas. They tell good stories. Good teams don’t build saviour features they expect will change everything, they build sensible features that will probably change something. Like geneticists, they’re looking to maximise their learning rate. Like geneticists, they seek a fast cycle rate.

If it takes you two months to take a feature from inception to release then you have an end-to-end cycle time of two months. That gives you a mere six chances to iterate on a feature each year. If each release gives you a 10% improvement then by the end of that year you can expect to be about 77% better (assuming you focus on the same feature).

Drosphila Startupis FTW

If you move instead to a two-week, end-to-end cycle, you give yourself not six but twenty four iterations. If you improve 10% each time this compounds to an eye-watering 880% improvement over the year. Perhaps even more importantly, when you reach the inevitable point where where incremental improvements start to wane you can quickly cut your losses and move your attention elsewhere.

If startup progress were linear then that would surely be compelling enough but startups are chaotic with both a small c and a large C and small changes in initial conditions can result in qualitatively different final states. Being better faster means disproportionately more press and attention, users and funding. Maximising the number of small positive changes may just be the difference between being Instagram and PicPlz. Get those little butterfly wings flapping.

Engineering iteration time is not the same as cycle time

Cycle time is the time that elapses between deciding to build a feature and being able to measure the success of the feature. It is an end-to-end time. You might have a nominal, two-week “agile engineering process” while still having a two month cycle time. Engineering time is only one part of it.

Most teams optimise for capacity utilisation

Pressure from managers and a misunderstanding of how product inventory depreciates means that instead of optimising for cycle rate, most teams optimise for capacity utilisation.

Managers make sure that engineers are “always busy” which inevitably means queueing more work up for them. Getting more features specced, more features designed, more UX wireframed. The bigger the queue, the lower the risk they’ll run out of stuff to do.

Unfortunately, as we’ve seen, queues are bad and lead inevitably to slow cycle rates. Queues are actually even worse than you imagine since they build up quickly, cause non-linear delays and take insufferably long to work off.

There was no way engineering could either see or avoid the jetwash which caused the product stall

You might not like your engineers, they might not like you but they are not responsible for your slow releases. For that you need to look elsewhere, you need to understand why queues arise and you need to understand what’s necessary to get rid of them.

In my next essay (which would have just been this one but it was way too long) I’ll take a look at how you can kill them (the queues, not your engineers) and live fast. Subscriber s’il vous plait to make sure you catch it.

NB this wasn’t just a ruse to get you to subscribe, it’s a way for me to help you avoid slogging through 2,000 word essays. But, since we’re here, come on in, the water’s lovely!

Discussion on Hacker News


1 Notes

  1. kano89 reblogged this from peternixey
  2. peternixey posted this