Is Time Estimation in Software Engineering a System 1 or 2 Task?

Software engineers are notoriously bad at time estimation. When they receive a new bug report or product feature to work on, engineers are often asked by their project manager to guess how long it will take. It’s a very reasonable request. For example, if your website goes down, public relations needs to know how long it will take to put together a response. If you’re working on the next version of your app, product management needs to know what features are possible before the ship date.

Unfortunately, engineers are bad at time estimation. A “one line fix” can become a rabbit hole when that one little change has massive implications across the rest of the code. A big project might be very simple to complete if a lot of the pieces were already lying around. This mismatch between apparent and actual complexity makes estimation. However, engineers often don’t even know what they will find until they are deep in the code. And by then, they will have already committed to a bad estimate.

Engineers have plenty of tricks to avoid blame. They make a real estimate then immediately double it, just in case. “Under-promise, over-deliver” is a popular strategy. However, both of those hide  the original problem of making a good estimate. These tweaks only work if you’re original estimate within an order of magnitude. If your estimate is wildly off, then your fluffed estimate will still be wildly off.

If you’re looking for a solution, I don’t have it. However, I realize that there’s tension in how have been explaining and using time estimation on my team. That tension is rooted in the difference between Daniel Kahneman’s “System 1” and “System 2” in how humans think.

System 1 and System 2

If you haven’t read Thinking, Fast and Slow, I highly recommend it: it is an interesting yet academically honest book of popular psychology. The wikipedia article will explain this better, but a central idea is that humans think in 2 different ways. System 1 is fast, automatic, effortless, and instinctual. I use it to cruise through my quiet neighborhood in my very trendy Toyota Corolla and to make the same steel cut oatmeal I have been eating for years. System 2 is slow, deliberate, and structured. I use it to plan out my shopping list for the week and to decide who to draft in my fantasy football league.

System 1 gets a bad rap in pop psychology because we obsess about biases and heuristics. We have many misleading instincts and often don’t make rational decisions without deeper thought. However, without System 1, we would be paralyzed by every minor decision. Many things are just not worth thinking too hard about.

So how do you know if a particular problem or task is more appropriate for System 1 or System 2? That answer varies because you can learn to use System 1 for some types of problems. For example, most people need to think very hard about what to do in a game of chess. When I (infrequently) play, I think through a many sequences of future moves to pick the best one: it’s a slow, System 2 process. However, chess grandmasters can look at a chess board and almost instantly understand (and even recall) the details. They may make odd , unexplainable choices that later payoff and seem like deep wisdom or black magic. Through thousands of games, they have developed useful instincts to use System 1.

However, other problems are not subject to rules that are conducive to System 1 thinking. Kahneman presents the stock market as an example of a System 1 failure. Many people have spent careers in finance and develop some instincts, but those instincts are no better than picking stocks at random.

Kahneman suggests that the difference between financial markets and chess is regularity, repetition, and attribution. Chess has strict rules without randomness, and through experience, you can develop a holistic understanding of the game with clear attribution of good and bad choices. Financial markets are influenced by an immense number of individual factors, and in one’s lifetime, you won’t witness more than a handful of recessions to integrate wisdom over time. When you guess right or wrong, it’s hard to know if you really saw something or were just lucky.

With that understanding, is time estimation better suited for System 1 or System 2? Depending on which one it is, what’s the best way to create estimates?

So Which is Time Estimation?

If it is possible to use System 1 for time estimation, then engineers should make real estimates with their gut but not rely on it immediately. By carefully documenting how long a task actually takes, engineers can compare their estimate against the actual time required. No matter how close or far off the estimate is, they become wiser having seen another example, and next time, their estimate will be better. Time estimation will naturally improve through aggregated experience, and eventually, engineers become graybeards who magically seems to know the unknowable.

If you can’t use System 1, engineers need to go through rigorous planning to create a estimate. The project needs to be broken down into small tasks with each one is laid out into a timeline with individual estimates. That entire timeline is added up, and it produces the estimate for that project. This method is clear and calculated, and it works as a guide for tracking progress during the project.

I’m not certain how to judge software engineering within Kahneman’s guidelines (regularity, repetition, and attribution). Codebases can be complex, and no two are alike, but engineers do have many conventions and guidelines to reduce the cognitive load. Most situations are consistent and deterministic. Attribution is logically clear but can be difficult to understand given the complexity of systems. Overall, it seems harder than chess but easier than understanding the global economy. However, I’m not sure where in-between those two points either software engineering or the threshold are.

This problem reduces down to another question: is it possible to predict the unexpected (even if you can’t quite articulate it)? Perhaps some engineers can see why a project gets hard half-way through. If they can budget that extra month without an obvious reason, then it’s a System 1 problem. If you need the rigor and structure to fully understand the task at hand, then it’s a System 2 problem.

Of course, engineers can use both: come up with a guess but also break down the task. Start early in one’s career with timelines and slowly rely on instinct over time. Independent of the time estimate, the timeline planning and reporting is useful to other team members.

Also, the difficulty of estimation depends on the specific task. “Fix this typo” and “build the next social media platform for teens” are very different problems that both require time estimates from engineers. However, if we could break down and understand the characteristics that make a problem easy or hard, that would also be equivalent to the question above.

Again, I have no answers on how to do time estimation right, but this System 1 and System 2 distinction has been useful to me. It has helped me understand why I give certain advice in developing skills as a software engineer. Maybe one day, my peers and I will be wise enough to be honest and correct about our estimates.