Peter Nixey

I'm Peter, a Rails developer and entrepreneur. I'm an ex-computer vision researcher, the former CEO of Clickpass and a YC alum. I have built and sold a lot of different software and am now the CTO of Brojure.com - which lets you quickly create stunning sales pitches

Beware your project’s Schwarzschild radius

A good physicist knows that every gravitational mass has an associated escape velocity. The escape velocity is the speed which an object needs to have to get away from the mass’ gravitational field. In layman’s terms this simply means that it takes a bunch of energy to fire a rocket off a planet’s surface and get it out into space and the stronger the gravity, the more energy you need.

The heavier a planet is, the more rocket fuel you need to carry to get up and into space. However, the more rocket fuel you carry, the more you need to carry just to lift the extra fuel you’re carrying. Unfortunately there comes a point when, like an overfed Christmas turkey, your rocket needs so much fuel to get away from the planet that it’s too heavy to lift its own bloated self. BTW if this all makes sense then feel good because it really is rocket science.

The same phenomenon is true for projects. The larger your project is, the harder it is for you to actually release a product from it. The more features in your backlog, the more of a struggle it is to wrestle the final product to freedom. This you already know. What you may not know though is that there’s a point where a product becomes so bloated that it may never escape the project’s gravity.

Every project has “churn”. This is the rate at which the project and the requirements change. No sooner have you written down a list of features than some portion of them will change. Priorities, requirements, previously undiscovered dependencies - all of these cause churn.

Despite its name, churn seldom means change and more often means growth. The rate at which this happens depends on the size of the project. The bigger the project, the more things there are to attract change and growth.

Unfortunately while your project churns faster and faster, the rate at which you can actually get it done is constant. You can only do so much in the day and if you’re not careful your forward progress gets drowned by the rate of change. Like a bird flying into a strong wind you will slow down and may even start to move backwards.

actual progress = velocity - churn

Teams assume that their velocity is equal to the number of points the engineering team work through each week. Wrong. Engineering velocity is just your airspeed and airspeed is not the same thing as ground speed. What matters is not how fast you’re flying into the wind but how fast you’re actually travelling towards your destination.

A team’s actual velocity is equal to the engineering velocity (jobs done per week) minus the amount of new jobs added (churn). Churn is approximately proportional to the size of the project- the bigger it is, the more that will change.

Project duration = Project size / (velocity - (churn rate x project size))

So for faster projects you can either reduce churn (bad)

The old school way to accelerate your ground speed was by reducing churn and having a feature-freeze. Feature freezes remove churn but at the cost of flexibility and reactiveness. This is not what you want. Building a product tells you what that product actually is. Seeing people use it tells you what needs to be changed. Adaptation is good and you need the flexibility to change so simply locking down features is going to grind you into some rigid hulk that no-one wants.

…or you can reduce project size (good)

Rather than reducing the rate at which you change things, you can instead reduce the overall mass of things available to be changed. The smaller the project, the less scope there is to churn.

“But we need to build lots of things!” you say. Fine but only ever build small, entirely self-contained versions of the big story. Don’t be afraid to change stuff or to make things that you’ll later have to dismantle. Make each sprint, each release, self contained and effectual. Always have a usable product and have each release add a new chunk of self-contained product to it.

Some teams get confused with this and assume that a massive project squeezed out like a sausage and crimped off in two week chunks is the same thing as a series of two-week projects. It’s not. Large projects will beget large amounts of change, they will beget discussions in meetings, changing requirements and constantly updated (and outdated) specs.

Truly small projects are self contained and work as-is without needing anything else that’s still in the pipeline. Small projects don’t even have stuff in the backlog. You write that when the time comes.

Stay away from a project’s Schwartzschild radius

If you keep chucking stuff at a planet it eventually turns into a star. If you keep chucking stuff at a star then it eventually gets so big that it collapses in on its own gravity and forms a black hole. A black hole is so called because nothing can move fast enough to escape its gravity. Not even light. The distance away from the black hole where this phenomenon begins (the edge of the black) is called the Schwartzschild radius.

Some projects have a Schwartzschild radius of their own. They are so big and so doomed that there’s a critical distance within which people and resources will never be seen again. Steer clear of these, they’re bad news and Accenture’s doomed multi-billion pound rewriting of the British, NHS computer system was one of them.

Conclusion: for faster, more pleasurable projects, stay small

Most teams think that they’re throttled by their engineering teams but in truth, engineering is seldom the bottleneck and the degree to which you can control engineering velocity is extremely limited.

Since you can’t do much about engineering and you don’t want to freeze features, the only way to speed up your project is to make it smaller. Not globally smaller but just divided into genuinely self-contained small digestible chunks. Tear up your backlog, keep an entirely separate wishlist and stay speedy.


(Disclaimer: the bit about Accenture may just have been an excuse to use the term Schwartzchild radius in a blog title).

Absolute knowledge

I’m the type of person who needs to understand why I feel what I feel. I want to understand what drives me and what drives the people around me. I’m emotional but I like to be aware of what’s behind those emotions.

That’s why I found myself confused by last week’s NSA revelations. I found them uncomfortable but I could explain why. My gut said that a loss of privacy was bad but my brain couldn’t verbalise the consequences.

I read and re-read Daniel Solove’s essay, Why Privacy Matters Even if You Have Nothing to Hide. I also read a terrifying account of a citizen of one of the Arab Spring countries. However the post-privacy, dystopian future that Daniel painted is already the rather pleasant reality of my today and I’ve already given away almost enough information to be as vulnerable as that Arabic citizen.

Solove references the Orwellian vision of the future where our location, activities thoughts and relationships are tracked constantly. I’ve largely already opted into that world. My relationships, thoughts, activities and location are tracked via Facebook, my blog and Twitter. I’ve already opted in to being surveilled. My Facebook details may be only available to my friends but when there’s 900 of those, the distinction is academic.

I’ve personally written the dossier for a Kafkaesque trial where information about me is used to convict me without me knowing what’s within. I put my CV on LinkedIn, my philosophies on my blog and my affiliations on Facebook.

So I find myself confused. Sharing per-se does not seem to be bad. CIA dossiers of the 1950s were probably slimmer than Google’s dossier on me today and yet having that information in public has been of great personal and professional benefit. So why did the idea of PRISM bother me? Was I simply being a hypocritical reactionist? Why did I feel that Edward Snowden had done something patriotic and admirable and worth the tremendous cost he will have to pay for it?

The more I thought about it, the more I realised that it wasn’t the loss of privacy which was was bothering me, it was the loss of my right to privacy and perhaps even more than that, the loss of privacy of those who represent and protect me.

PRISM is most frightening because it is secretive and forces us to assume that all of our online activities are recorded. If I know how the government is hoovering up data then I can modify my behaviour accordingly. If I don’t know where they’re gathering data then it’s only sensible to assume that everything is available to them. If I tweet it’s public and I expect that my public thoughts may be taken down and used in evidence against me. While there was some talk of Twitter’s noticeable absence from the PRISM slide it’s probably less to do with their moral fibre and more to do with their API. I expect and you should too that our public thoughts will become public record

The idea of losing my privacy online is uncomfortable. However I can always switch off and become private again. But while going offline today is inconvenient, going offline in 20 years may be debilitating. The rate at which our lives are becoming digitally assisted is breathtaking. That means that not only will we be more digitally-dependent but switching off will be more conspicuous.

If your baggage boards a plane but you don’t follow your absence is not only conspicuous it’s consequential. The plane will be delayed and your baggage removed. In years to come our online absence may prove to be just as conspicuous. Simply by falling off the radar, by switching off our location monitoring, step-tracking, heart-monitoring and sleep monitoring we may become just as conspicuous as that empty seat in 22c.

Today PRISM removes privacy in the part of our lives which are online. When online and offline become inseparable and switching off isn’t any more an option that removing the number plates on my car, PRISM will have removed my right to privacy completely.

So losing online privacy may ultimately mean losing the my entire notion of privacy but I still have little more than an ideological cost to show for that. It sounds bad but how does it hurt me? Is it perhaps a price worth paying in the multi-billion dollar fight against the spectre of terrorism?

While troubling, the loss of my personal privacy may not actually be so consequential, I’m not of immediate interest to the government. They’re much more concerned with politicians they’re debating with with, contractors they’re negotiating with, lawyers or judges who might at some day need to impeach them. What happens though when their lives cease to be private? If an impeached president can read a judge’s notes using his NSA search engine can he really ever be tried?

Our information is our achilles heel, it’s leverage. A disease we haven’t revealed to our employer could compromise our jobs. An STD we kept private might publicly shame us. The fact that maybe our true sexuality is not the one we publicly portray, the fact that maybe the people we love are not the ones we live with. The fact that maybe we weren’t sick on that day we called into work and were instead skiing. All of these are things that can be used to manipulate and maneuver us.

Everyone has an Achilles’ heel buried somewhere in their lives and if a government has unfettered access to everyone’s information then they have access to everyone’s Achilles’ heel. The right to access our lives is the right to access the lives of the people who protect us and the right to do that is the right to rule unchallenged. All the good that Obama did with Obamacare will fade to black if he inadvertently negates the safety nets which protect us from bad leadership. Absolute knowledge is absolute power and absolute power corrupts absolutely.

All democratic nations were built on a system of checks and balances. All of them were built to be resilient to the occasional bad or corrupt leader. If you give that leader control over his opposition though, if you give him an arsenal of weaponry that no politician, no lawyer and no judge can oppose then you hand him the keys to do whatever he wants both domestically and internationally.

So when the average man or woman on the street says they don’t mind the NSA having access to their Gmail they’re probably not being naive. Ask them instead though whether the people that protect them should have privacy, whether the judges, the lawyers, the journalists and the opposition leaders should have the freedom from government. Ask them whether the people who protect them should have freedom to do their jobs unencumbered and you may not get the same answer.

When I first thought about it I couldn’t understand what was tangibly bad about giving government unfettered access to its our digital lives. The more I think about it though, the more I fear that with our ever more digitally entwined lives, a government with absolute access to that digital life has absolute power over us and absolute power to neutralise those we elect to protect us.

First world demons

image

An increasingly fashionable position on Hacker News is to invoke the “First World Problems” objection to a new startup. The First World Problem objection says that the founders of a particular startup are nought but huckster hipsters because they’ve addressed the tertiary problem of photo sharing rather than a core issue like medical-image sharing.

There’s an old saying which says that we make our own demons. It means that the things that haunt us and that we despise in others are in fact the things we fear in ourselves. We deal with our internal demons by externalising them and belittling them in others. A classic example is Chris Cooper’s portrayal of the angry homophobe in American Beauty. Outraged by the prospect of his son’s apparent homosexuality he rages against Spacey until events finally and violently expose his own inner conflicts.

Whenever I hear someone say “I hate this person” or “that group are idiots” my immediate reaction is to ask what it is that bothers that person. This isn’t pretentiously harmonious - it’s more often phrased as “that guy’s an angry idiot” but my assumption is that whatever they hate is simply the same thing they fear in themselves.

The First World Problem objection is a classic of these. Why should it bother anyone that someone should build another photo sharing service? Is it really such an offensive thing to do? Of course not. Unless that is, you’re in a job you don’t enjoy and the reason you’ve coached yourself not to leave it is that you haven’t found anything “worthy enough” to start yourself. When you’re trapped in this world the idea of someone having the courage to jump ship without waiting for something “worthy enough” to land in forces you back hard against your logic. That is painful.

Many, many people dream of something that frees them from their working lives and the presence of entrepreneurs and startups can be inspirational but also jarring and frightening. If you yearn to do something but have trained yourself you can’t, you have to give that commitment teeth. The most common teeth are the assertion that you will fail. For some though that is not enough and they need to believe that not only they will fail but so will everyone else. Those who don’t were either lucky or perhaps “in the club”.

The credit, as we know, belongs to the man in the ring but maybe it’s empathy we should reserve for the First World Objectionist for he is perhaps only sheltering from his own fears. And to those of you solving the First World Problem of photo sharing - forge on, I would love to see a First World Solution.

Comments on Hacker News

Dear Apple, let’s talk about photos

We’ve been managing our photos together for almost a decade now. Things were nice and simple at the start and we both knew what to expect from each other - I pulled my photos off my camera on the computer, imported them into iPhoto and arranged them. Life was good.

image

But then you came and gave me the great iPhone camera and I started taking more photos like that of this lovely cow. Also you give me a wonderful wireless sync facility so I didn’t ever need to plug my phone into my laptop so I didn’t. So then the photos on my phone and the decade of photos in my iPhoto library started to diverge because I didn’t get round to syncing them.

Then you gave me a beautiful iPad which would looked so perfect for curating my iPhoto library. I couldn’t wait to edit and organise photos from the comfort of my sofa except that you didn’t give me any way to save things back to the library. So I couldn’t. So things just kept piling up on my camera roll.

Then I thought you’d heard my pain and figured it out - you gave me “photo streams” which put every photo on every device. Unfortunately the only way to get photos into iPhoto was still through my desktop so the fact they were available on the iPad was academic because I still couldn’t save them to my library. Also when I pulled them off the photostream they still existed in the camera roll. Two copies (three if I edited them) and you also occasionally told me to delete some because I ran out of space.

To be honest, I got a little confused at this point. Have I transferred them into iPhoto? I’m not sure I’ll check the photostream - ack, no, I remember you told me to delete photos off my photostream because there were too many. Hmmm, check the camera roll again. Hang on a minute - where are my videos?! Turns out photostreams don’t stream videos. Man, these things are not cool.

Just so as you know by the way and don’t freak out but, I’d like to sort my photos when I’m sat on the loo. Or in the bus, or anywhere else I want to kill time with my phone. I don’t want to edit them when I’m sat at my desktop - that’s work time. It really pains me that I can’t do that and so my photos just pile up in a big heap while I waste time reading things I don’t care about on Twitter.

Finally, to add insult to injury, when we moved to your lovely little MacBook Air you kyboshed my disk drive. All of a sudden, after 10 years of taking high-res photos and videos I couldn’t even fit them onto my laptop. Not only that but you insist on keeping a cache on my Macbook of my entire decade of photos at just the right size for each iDevice. Did you know that I’m using 7% of my entire Macbook disk drive just caching photos for my iPhone and iPad? That’s crazy! I paid a lot of money to you for that flash drive - I don’t want to waste it on a photo cache.


image

So Apple, I think you’ve got a bit confused. Don’t worry about sharing, we don’t need you for that. Your job is to take photos, organise them and make sure they don’t get lost. So let’s talk about how you can do that.

What I would like from Apple to manage my photos

1. I want the canonical copy of my iPhoto library in the cloud. One iPhoto library in the cloud, many devices with access to it. I want to edit, organise and delete photos on any device and see the same changes on all other devices. No master/slave setup - just straight cloud access.

2. You can charge me for this. I suggest $5/month. Maybe that’s a bit more than it costs you at the moment but that’s what I’m prepared to pay and we both know that you’ll do very well out of this in the long run. However for that I want unlimited space including for all of my videos. FYI that’s not what really I’m paying you for. I’m really paying you for the peace of mind that you’ve got my memories safe-guarded. I’m technically paying you for insurance. The utility this offers just the carrot that gets me over the hump of paying you.

3. Get rid of photo streams. Make the camera roll a single photo stream that shows up in iPhoto (on all devices). I want a single camera roll that all devices feed into. I want to take photos, queue them in my camera roll then pull them out as I organise and sort them into my library. Let me explain: photos and videos have two phases 1. on the camera roll 2. in my photo library. Nowhere else.

4. I’d like you to create API for my iCloud camera roll so any camera I own could hook into it - I want to be able to buy an SLR with wireless capabilities and simply connect it as a new source to my camera roll.

5. Make iPhoto on the iPad and the iPhone work well. Make them do clever things to give me fast access to my photos from the cloud. You did it for iTunes, let’s bring the magic a second time. Let me harness power of that splendid little device direct from the loo.

This opportunity won’t last forever by the way. Most people haven’t got a good backup strategy and don’t spend any money on backup but by the time they do they won’t be prepared to give you yet more dollar to take care of their photos. When the inevitable time comes that they’re paying Dropbox or Google $10/month for unlimited backup they won’t cough up another $5 for photos. However they’re not doing that now and you’ve got a window to bring some Apple magic.

I know Steve’s not around to crack the whip but you need to get your act together Apple. This could be a nice little $2Bn/year recurring revenue stream but the window will close. If it closes before you’ve been bothered to sort this mess out we’re all rodgered so chop chop!

Comments on Hacker News

Fort Mason, San Francisco

Photo credits - my iPhone

Why does “I pencil” keep falling through Hacker News?

I Pencil is by any standards a deeply interesting essay. It looks at how incredibly specialised our jobs and roles have become and reflects on the astonishing and powerful fact that no single person in the world today holds enough knowledge even to make a pencil.

The essay is a reflection on the importance of trade and specialisation. It’s a reminder of the massive sophistication of the society that is required to produce even our most simple products and it’s a wonderful portrayal of the dangers of patents and the power of free markets and capitalism. It is also a good reminder of just how big the shoulders on which we stand actually are.

image

And yet despite having been submitted twelve times in the (search-documented) history of Hacker News, I Pencil only made the front page once, three years ago.

I also don’t think it’s because the HN crowd don’t find it interesting, I have no doubt that it would spawn a load of debate about software stacks, open source and patents. It did certainly do well three years ago. Since then it’s been resubmitted about nine times and seldom gets any upvotes.

It may be because there’s de-duping going on in Hacker News which would be fair enough (although still a bit of a shame - classics will come around again). However since it had been submitted twice before even catching the frontpage three years ago, I suspect it’s not that.

I suspect that “I pencil” never makes its way onto the front page because if you come at it cold, it’s not completely clear why it’s significant or just how well reputed it is. It takes a little time to digest and it’s you ideally want someone to reassure you that it’s really worth reading and thinking about. It’s the sort of article that you’d take the time to read and appreciate if you saw that a lot of people on Hacker News upvoted it.

However, it’s not the sort of essay that does well if you come to it from a standing start, you don’t know that anyone else finds it important and you read it quickly without thinking about it. In short it’s not the sort of article that does well on the “Newest” section of Hacker News.

Hacker News does an excellent job of migrating stories up and down the front page according to their level of interest but the newest section has a more capricious atmosphere. A story has between half an hour and an hour before disappearing off the bottom and if it doesn’t garner three to four upvotes during that time then it won’t make it to the two front pages and to the traffic and upvotes that follow.

The challenge is that upvotes are throttled by the number of visitors to the story. Your upvotes will only ever be a tiny fraction of your visitors. However, with only a title to judge the story by, the “Newest” section is often a beauty parade of titles. Good titles get lots of clicks, a fraction of which convert into votes, poor titles get few clicks and fewer votes still. An article may be twice as interesting and convert visitors into upvotes twice as well as the next but if its title attracts a tenth of the traffic (which from personal experience on both sides is perfectly possible) then it’s dead in the water.

A “deeply interesting”, slow burner like “I pencil” is just not cut out to survive it and is always going to lose against a well written 800 word piece with a new angle on Self Quantification While Programming and a link back to Hacker News comments.

Both as a reader and a writer, Hacker News brings a great deal of value to me. As a writer it’s my greatest source of traffic. As a reader I go there to be entertained but mostly to learn from the community - both what’s hot and what I should be aware of in general. I’m very happy to read long, solid, thought-provoking pieces (especially with Pocket) and I’d love to see even more of them.

I might be completely off-target with this and it may just be that “I pencil” is boring or has been hellbanned. If I’m not though then it would be great to dampen the the title-parade effect. If all new articles got the same amount of traffic then there wouldn’t be quite the same positive feedback loop that happens when catchy-titles beget traffic which begets upvotes and yet more traffic.

It would be really interested to run two experiments on HN. The first would be to remove the display of upvote count on the newest page and see whether it reduces positive-feedback. The second would be to take normalise the weight of upvotes by the traffic going to the story (disclaimer this may already happen). If I had the reigns I’d be super interested to put in a redirect on story links so I could measure clickthroughs and then normalise upvotes against that traffic.

I normally shoot for a clever last line to an article but I have none for this except that ideas are easy, execution is hard. As you were.

Comments on Hacker News (because I’m not Leonard E. Read)

Where lean ends

And so we come to the end of Eric’s talk and a gaggle of entrepreneurs are shepherded onstage to discover why they haven’t yet fulfilled their recurring-revenue destiny.

image

One sheepish fellow tells us his company is struggling to get signups and he doesn’t know what to do next. Eric turns to the audience and smiles knowingly, telling us that this is clearly a case of doing what we in the Lean Community refer to as “Reverse Order Planning”. He turns back to the entrepreneur and tells him what he should do. The crowd roars, the entrepreneur raises his palms. I’m not completely sure this is how it ended but in my mind’s eye, Eric walked up to the other two entrepreneurs, placed his hands on their foreheads and as they burst into tears he pushed their collapsing bodies back into the raging audience. Lean had most definitely arrived.

In 2007 (4BL), I’d just pitched my then company, Clickpass, to Mike Maples. They say that if you want advice then ask for investment and Mike duly responded to my pitch with a story about IMVU. IMVU was a Virtual World company who had formalised their product development and engineering process and Mike’s uncloaked suggestion was that I should consider the same. I read the case study and was blown away. One of the subjects of the study, Eric Ries, kindly agreed to meet me and my co-founder, Immad at IMVU’s office in Palo Alto so we grabbed a zipcar and headed down from San Francisco to meet him. I found Eric inspirational not only in the humility of his presentation but in the sheer rationality of the approach he presented. In the structureless madness of Silicon Valley, it was the type of actionable startup framework I’d been craving.

In the years since then, thousands more people have been touched by Lean. Just as it affected me, so it has affected them. However, Lean’s converts have begun to take on an almost religious zeal. Entrepreneurs wear their MVP like a crucifix and demand to see yours before they will deign to interact with you. “Don’t have a product out there clocking up signups? Why not?! What’s wrong with you? What’s the point in building another feature until you know your existing ones are useful?”… but I’ve only written a login system… “Doesn’t matter, how can you know it’s not enough until you test it? Release, RELEASE!”.

In the midst of the excitement of Lean, something’s got a bit weird, something’s been forgotten and trampled underfoot. In the heat of excitement we’ve forgotten that Lean is by its own admission not the same thing as cheap and that testing a vision may mean building something beyond what is envisionable today. We’ve forgotten that testing a hypothesis efficiently and running an experiment quickly doesn’t mean the experiment itself is trivial or inexpensive.

What does the MVP for the iPad look like? Is it a colour multi-touch device which lasts at least five hours, that you can surf the net, watch video on and read books. Is it that? Because the Blackberry Playbook does that and that doesn’t feel like an iPad to me.

What was the iPad’s disprovable hypothesis? At which point would you have closed the door because the iPad just seemed “a bit too heavy” or “a bit too sluggish”. When would you have told Steve the project was flat-lining and that he should forget about the last 100g of weight loss or the last hour extra of battery life?

When are you wrong and when are you just not “right enough”?

The MVP for the iPad was something that matched Steve’s vision of what it should be. The failed hypothesis was him building it and realising he didn’t like it. Steve Jobs is of course the definitive outlier but he’s not the only person to revolutionise things by by going much further out on a limb than Lean might sensibly suggest.

What would your MVP have looked like for Dropbox? Would it have included sync across OSX, Windows and Linux? Would it include versioning as well as just backup? Because I’ve got account #140 on Dropbox and I had all of those from the start. The lean movement frequently cites Dropbox as a “lean company” but while impressive, it wasn’t what we would now think of as lean. Dropbox was built to a single vision rather than a cascade of hypotheses. Drew and Arash built out what felt right to them, what they wanted. Would it have worked with less? We’ll never know but we can guess that either way it probably wouldn’t have worked for Drew.

Lean teaches us to relate constantly to the market and to focus on the minimum viable product that tests our hypothesis. If what we’re building is high-fat, low-protein, salty, internet snacking then we may be able to simply release, test and iterate our way to that from the start. Anything truly worthwhile will demand more though. We will have to build something of real value before we can see whether it even works, never mind whether we can onboard efficiently. It may need to be rethought even before that point. It may not fit our own vision of what it should be and we may have to rethink our approach before we release to the market at large. That doesn’t mean that we won’t release the minimal form of it, only that the minimal form may still require a lot of functionality.

Lean is a clinical, scientific methodology that has changed the startup community almost entirely for the better. It’s a methodology that ensures you progress efficiently, formalise what you are testing and keep your focus on doing so. It is a process of systematic, incremental experimentation and it is brilliant.

What Lean doesn’t do though is tell you is how minimal your viable product actually is. Creating a faster horse will probably mean doing a lot more than simply A/B testing what grain you feed it.

image

Engineering is not the bottleneck

Most slow software companies believe that the engineering department is what slows them down.

I’ve always found this odd though. The companies I’ve heard complain about it were indeed slow and features did indeed take two or three months to create, however engineering didn’t actually account for very much of that. Often no more than a week or so.

image

Not only was the engineering time relatively brief but so were the other stages. UX and UI would only take three or four days between them and QA was maybe a day at the end. Despite a total development time of only two weeks, features still took two or more months to become a reality. Why was this?

It happened because features spent most of their time simply waiting around in queues. The product department had a backlog of requests from biz-dev, the design team had wireframes waiting to be made beautiful, engineering had specs waiting to be made whole. Every step involved a queue and an associated delay.

Product gets delayed by queues not by engineering

It was the best of intentions which created these queues. The first was the fear that engineering would “run dry” and the second was an understandable but misguided belief that backlogging specs and designs helps things move faster.

It was primarily the fear of an unoccupied engineering team that underlay things though. Engineers are expensive and hard to recruit and their salaries go out of the door like blood leaking from a severed artery. With all of that investment, the last thing you want is for such expensive machinery to sit idle on the shop floor.

Both managers and product managers felt that if engineering wasn’t humming then they hadn’t been doing their jobs. CEOs and department heads only tended to exacerbate things. I was once told in no uncertain terms that, “if engineering isn’t working on something then someone will have to answer for it”. Which doesn’t seem unreasonable until you see the consequences.

The reality of product development is that it’s less about production and more about reduction. Reduction of the myriad ideas that don’t work and exploration of the few that do. Product development is less of a design process and more of an evolution process. And a critical part of the evolution process is the rate at which generations die or to be more accurate, the rate at which they reproduce.

Startups live and die by Cycle Time not by engineering throughput

When geneticists study heredity and evolution they often do so using fruit flies. One of the main reasons for this is that fruit flies reproduce very quickly so the the observer can record evolutionary effects in real time.

If you ever hear a great founder speak, they don’t usually talk about a small number of brilliant ideas, they talk about a large number of very sensible ideas. They tell good stories. Good teams don’t build saviour features they expect will change everything, they build sensible features that will probably change something. Like geneticists, they’re looking to maximise their learning rate. Like geneticists, they seek a fast cycle rate.

If it takes you two months to take a feature from inception to release then you have an end-to-end cycle time of two months. That gives you a mere six chances to iterate on a feature each year. If each release gives you a 10% improvement then by the end of that year you can expect to be about 77% better (assuming you focus on the same feature).

Drosphila Startupis FTW

If you move instead to a two-week, end-to-end cycle, you give yourself not six but twenty four iterations. If you improve 10% each time this compounds to an eye-watering 880% improvement over the year. Perhaps even more importantly, when you reach the inevitable point where where incremental improvements start to wane you can quickly cut your losses and move your attention elsewhere.

If startup progress were linear then that would surely be compelling enough but startups are chaotic with both a small c and a large C and small changes in initial conditions can result in qualitatively different final states. Being better faster means disproportionately more press and attention, users and funding. Maximising the number of small positive changes may just be the difference between being Instagram and PicPlz. Get those little butterfly wings flapping.

Engineering iteration time is not the same as cycle time

Cycle time is the time that elapses between deciding to build a feature and being able to measure the success of the feature. It is an end-to-end time. You might have a nominal, two-week “agile engineering process” while still having a two month cycle time. Engineering time is only one part of it.

Most teams optimise for capacity utilisation

Pressure from managers and a misunderstanding of how product inventory depreciates means that instead of optimising for cycle rate, most teams optimise for capacity utilisation.

Managers make sure that engineers are “always busy” which inevitably means queueing more work up for them. Getting more features specced, more features designed, more UX wireframed. The bigger the queue, the lower the risk they’ll run out of stuff to do.

Unfortunately, as we’ve seen, queues are bad and lead inevitably to slow cycle rates. Queues are actually even worse than you imagine since they build up quickly, cause non-linear delays and take insufferably long to work off.

There was no way engineering could either see or avoid the jetwash which caused the product stall

You might not like your engineers, they might not like you but they are not responsible for your slow releases. For that you need to look elsewhere, you need to understand why queues arise and you need to understand what’s necessary to get rid of them.

In my next essay (which would have just been this one but it was way too long) I’ll take a look at how you can kill them (the queues, not your engineers) and live fast. Subscriber s’il vous plait to make sure you catch it.

NB this wasn’t just a ruse to get you to subscribe, it’s a way for me to help you avoid slogging through 2,000 word essays. But, since we’re here, come on in, the water’s lovely!

Discussion on Hacker News

Don’t forget to live

I have a friend, a designer, who’s convinced she’s not very good. She doesn’t believe her work is anything special. She doesn’t even believe her discipline is anything special and prefers to assign credit to her environment or to the pressure of others’ expectations.

This belief is very deeply ingrained in her. It comes naturally but has been amplified by her reaction to other people’s feedback. You’d think then that they given her so little reassurance that she loses her self belief; that they erode her self confidence?

That’s not what people tell her though. They tell her that she’s good, that she delivers work which is very good. That she works very hard. That the combination of the three is unusual; that she is unusually good. I don’t know what people told her before I met her, when she was young, but now there’s no doubt. She is exceptional.

She prefers not to see it that way. And she prefers not to see it that way because she wants to be better still. Because she wants to be as good as she aspires to be. And that means ever better.

She has created a reality that forces her forward. Her need is always to be better. To be stronger at what she does. She values what others think of her but what she really cares about is what she thinks of herself. What she thinks is that she needs to be better. To achieve aggressive goals and then to push them further. Usually to push them on before she even arrives at the old ones; forcing herself to fail inevitably and improve relentlessly.

She must never feel comfortable, she must never sit down. She creates a belief set for herself that ingrains a sense of inadequacy. With that in hand, she has no option but to march forward, to make herself better and stronger. She cannot exist in a world where excellence is a prerequisite and where she is awful and so by grading herself down she drives herself ever forward, becoming ever better.

“The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man”
 - George Bernard Shaw

So too ourselves. The unreasonable man adapts himself to fit his own vision of himself.

But adaptation is hard and distortion is painful and we can’t find peace while twisted. We have to believe that in the end our adaptation will bring us something: respite, fulfilment, happiness. We have to find a peace and an equilibrium or we are our own Sisyphean captor.

Because as we hold ourselves hostage to our dreams, as we push the rock back up the hill, life happens. It happens and as we get older it happens faster. Life accelerates around us until months disappear like sleepers under a train. As we focus on rolling our rock better, further, faster, the year that it took us to do so passed. It’s goes. It won’t come back again and it can’t be traded with a one from the next decade. Years aren’t fungible, being young passes.

This post started off as a message to my friend but it ends as a note to self. I think there is perhaps more of my friend the designer in me than I care to admit. I wrote this to implore her not to throw off comfort and contentment. That these are the things to to treasure and not to demonize.

We avoid accepting praise because we fear we will start to believe it and cease to improve. We avoid creating a home because we fear it will make us too content to move, to lay our head on the rolled up sweater of the next opportunity. We avoid taking a job because we fear we will get addicted to the money and never start a company.

In the Stanford Marshmallow Experiment, young children were promised two marshmallows by a researcher. The researcher presented them with the first one straight away and told them they’d get the second as long as they didn’t eat the first before the researcher returned. The researcher then left the room for 15 minutes. A few kids scarfed the marshmallow immediately while a minority held out for the second.

The children who held out did so by forcing themselves. They turned around and covered their eyes, they tugged at their pigtails, they kicked the table. Some even “stroked the marshmallow like a tiny stuffed animal”. Long term studies went on to show that the children who were able to do this, to delay gratification and wait for the second marshmallow also turned out to achieve more in life.

The conclusion was that the ability to delay gratification correlated with more success in the long run.

However I think there’s a second conclusion. It took 15 minutes for the researcher to eventually reappear and for the remaining children to receive their second marshmallow. The girl who held out, spent 15 minutes of her life tugging her pigtails, waiting anxiously for a marshmallow. Meanwhile, one of her other friends got a marshmallow immediately, left the experiment 15 minutes earlier and got back home in time for dinner and playtime.

Sometimes we have to force ourselves to stop living with our eyes covered and our backs turned, stroking our dreams like stuffed animals while we wait for a sorry marshmallow. We need to just eat the one that’s in front of us and enjoy it.

10 steps to deleting someone’s business off Heroku

Heroku provides a wonderful service. It’s a service that’s solid enough that you can really grow big on it. There are some businesses that have grown very big on Heroku, had many millions of dollars invested in them and in some cases been sold for many millions of dollars too. I love using Heroku.

It’s a bit disconcerting though that if you go to SXSW, you hang out with some eager young founder whose business is on Heroku and you borrow their phone; you can delete their website before they finish their drink.

This is a bit of a flippant post and intentionally so as I’m hoping to draw attention to the problem for Heroku and other services too. I have mentioned it to them before but it hasn’t yet got any love. It’s not only a problem with Heroku it’s just that’s where my business is so that’s what I’m concerned about.

10 steps to delete someone’s business off Heroku

1. Put down your drink

2. Ask to borrow their phone (or watch their pin and take it)

3. Go to Heroku’s reset password link

4. Enter their email and reset their password

5. Open the mail app on the phone

6. Click on the email from Heroku

7. Enter a new password

8. Go to the app > their business > settings

9. Click delete

10. Finish your drink

It’s not clear to me whether backups created using Heroku’s PGBackups service will also be synchronously deleted but if they are then what just happened was Really Bad.

2-factor doesn’t really help

This isn’t something that 2-factor authentication is going to fix. 2-factor auth is great at preventing a man-in-the-middle attack but when the attacker has your phone, they probably also have the second auth channel (Google Authenticator / SMS / keys with your RSA fob ).

What might help

Here are a few simple things that would make an attack harder:

1. Don’t delete or even deactivate an app when someone asks, set a job to do it in a few days
2. Don’t delete any database backups for longer still
3. Send that person an email every day in between telling them their app is going to be deleted
4. Let them set a second password to delete the account
5. Do multi-channel auth but do it 72 hours later, once they’ve either got their phone back or noted that it’s gone.
6. Allow a “second signatory” on the account - someone else who has to concur before you push the button

In the case of Heroku, all of the above measures would ideally need to apply to other abilities that could damage the service such as removing addons, workers, dynos etc. too.

This isn’t just Heroku’s problem, it may apply to your service

In the past you could be fairly certain that short of getting phished etc. your email was reasonably secure - if only by obscurity. You used a computer and you left it at home or at work. The computer itself probably had a login. You probably didn’t have the computer with you when you were drunk at parties.

However our email is no longer inaccessible, it’s in our pockets and we don’t log out of it, we just put a 4-digit pin on it. We give it to someone when we show them a photo or leave our phones an iPod dock at a party.

Since almost every account has a password-reset via email every account is accessible via our phone. It would be great to have some different auth patterns but in the meantime it would be good to lock down those things which matter most. One of those things for me is Heroku.

Discussion on HN

Software, fear and witchcraft

A throwaway comment I left on HN today resonated enough to make me rethink its significance.

“Evidently the corollary to Arthur C Clarke’s famous quote on technology and magic is that those who create it are witches and wizards. You like the magic and you need a few practitioners but when things start getting weird, it’s pitchfork o’clock.”

It was really only meant to be cute but it got so many upvotes (31 83) that it made me realise that it was accidentally more true than I realised.

The quote from ACK is of course that “any sufficiently advanced technology is indistinguishable from magic”. Back in the day magic was (to my uneducated understanding) largely chemistry, slight of hand and showmanship.

Magic led to both fear and respect. Witches and wizards feature frequently at the right hand of leaders but seldom seemed to make their deathbeds in a single piece.

As technologists, we wield our powers for the advancement of ourselves, our world, and just for fun. We can do pretty crazy things and create wealth and power out of seemingly nothing in a way that witches and warlocks of yore would have been awe of. We are perhaps too glib about the comprehension and understanding this takes from the rest of the world.

Take the legend of a chosen redhead. Even as a child he cast a spell so powerful it blanketed the entire world. His spell gave him the power to hang the thoughts, fears and dreams of a billion people in front of him like the silvery strands of the pensieve. It was magic so strong that neither the young nor the old could resist. All were seduced. Finally, unsatisfied at merely holding the thoughts of the world in his hands, the talented magician went further. Within a year he had emptied the purses of the richest merchants in the land to the tune of the combined value of the greatest shipping and trading companies of his day.

As crazy (and implausible) a tale as this is, this is the world we live in. This is the magic we practice and the power that it gives us.

In the wake of the terrible news of Aaron Swartz there have been several followup stories of information crusaders, crucified by the establishment. It’s maybe easy to overlook the fact that those who mete out such punishments are not just reacting to the action. They are presumably also reacting to their own fear.

Nobody with any real understanding of the facts and technologies involved would have offered up 35 years in jail as the appropriate punishment for Aaron. It’s perhaps not a coincidence that witchhunt is the word that we reach for in these circumstances.

When power destroys things we presumed safe, awe turns to fear and fear to a call to arms. 35 years is better than being burned at the stake but it’s similar enough to stop and think. Why would facilitating the distribution of legally available academic research papers carry a greater punishment than stabbing and beating someone to within an inch of their life which (in the UK at least) carries a maximum sentence of 18 years?

In abstract it seems crazy. The trolly problem is a bona fide moral dilemma. If however you were offered the alternative of either a) stabbing someone to the point of their lifetime paralysis or b) making some already-public scientific papers more available it wouldn’t even qualify as a dilemma, merely a question of how quickly the papers could be set free.

And yet in the eyes of the law, freeing the papers carries twice the sentence of stabbing someone. There has to be more to such irrationality than simply an unbalanced emphasis on intellectual property.

Magic: “the art of producing a desired effect or result through the use of incantation or various other techniques that presumably assure human control of supernatural agencies or the forces of nature”

Technology at its best, as Arthur C Clarke said, is indistinguishable from magic. But throughout history, magic has come hand in hand with fear.

In dealing with others’ perceptions of magic we should remember that it is fear that whistles through the pitchforks and fans the torches. Arguing that the mother would have died during childbirth regardless and that the healing spell was not what killed her won’t allay the villagers’ fears. They need to know that the spell can’t kill their wives and mothers.

When faced stories such as Ahmed or Aaron’s, we technologists focus simply on the rights and wrongs of the act itself. Consciously or otherwise though, non-technologists are reacting both to the act and the magic. Perhaps sometimes we underestimate the importance and significance of simply demystifying ourselves.


Discussion on Hacker News