Why Software Project Estimation Is Difficult

In my experience, the following factors make software project effort/cost estimation difficult:

1. It is much easier to estimate for the happy path which leaves gaps in specifications for handling edge cases and exceptions.
2. It is common to overlook non-functional requirements like performance, security, scalability. They often become the reason a project fails at scale, even if it succeeds at first. 
3. Not cleaning up code during development.
4. Frequent changes in libraries and frameworks necessitating unplanned rework.

Every if, switch, or loop condition can introduce new paths through the code. If a function has n independent Boolean conditions, the number of possible paths is up to 2^n. If a function takes multiple parameters, the combinations of edge values increase multiplicatively. 1 input has usually at least 3 edge cases, n inputs would have at least 3^n edge case combinations. If the function depends on or alters internal state, interacts with external systems, or uses asynchronous logic, you must test for edge timing, race conditions, resource exhaustion, etc. Edge cases scale faster than linearly with added logic, often closer to exponential or combinatorial growth depending on interaction depth.

Handling error cases and edge conditions often takes much more effort than the happy path due to the need for additional logic, testing, and debugging. If edge cases are not considered early, addressing them later can introduce costly redesign efforts. Unfortunately, the number and complexity of edge cases grow as the project progresses, especially if new scenarios are discovered during development or testing. Handling one error case might introduce or expose others, creating a chain of additional considerations that were not part of the original estimate.

Neglecting regular code cleanup leads to the accumulation of technical debt, which increases the time and effort required to implement similar features later in the project. New developers joining the team may struggle to understand and contribute to the codebase, further slowing development. Additionally, the effort required to fix bugs, integrate new systems, or perform upgrades can grow exponentially over time.

APIs, libraries and frameworks are typically updated every six months, and programming languages undergo significant changes every few years. While these updates bring improvements, they can impact project timelines.

All these factors can make your initial effort/time estimation 10 times less than the actual cost in the end. A rule of thumb you can use is to estimate effort/time for the happy path and multiply that by 10 to get a realistic number.