The March of Nines
I love this quote from Karpathy in the podcast with Dwarkesh Patel where he talks about why self-driving cars are taking so long to develop:
What takes the long amount of time and the way to think about it is that it’s a march of nines. Every single nine is a constant amount of work. Every single nine is the same amount of work. When you get a demo and something works 90% of the time, that’s just the first nine. Then you need the second nine, a third nine, a fourth nine, a fifth nine. While I was at Tesla for five years or so, we went through maybe three nines or two nines. I don’t know what it is, but multiple nines of iteration. There are still more nines to go.
I’m going to refer to this as the “March of Nines”, where the smaller gains take an equal amount of time.
I’ve seen this constantly in AI native software. Getting an LLM feature to work impressively in a demo takes a few days. Getting it to work reliably enough that you’d actually ship it to users takes months. Same amount of work for each nine.
