Why Software Project Estimation Is Difficult

In my experience, there are three factors that make software project effort/cost estimation difficult:

1. It is much easier to estimate for the happy path which leaves gaps in specifications for handling edge cases and exceptions.
2. Not cleaning up code during development.
3. Frequent changes in libraries and frameworks necessitating unplanned rework.

Handling error cases and edge conditions often takes much more effort than the happy path due to the need for additional logic, testing, and debugging. If edge cases are not considered early, addressing them later can introduce costly redesign efforts. Unfortunately, the number and complexity of edge cases grow as the project progresses, especially if new scenarios are discovered during development or testing. Handling one error case might introduce or expose others, creating a chain of additional considerations that were not part of the original estimate.

Neglecting regular code cleanup leads to the accumulation of technical debt, which increases the time and effort required to implement similar features later in the project. New developers joining the team may struggle to understand and contribute to the codebase, further slowing development. Additionally, the effort required to fix bugs, integrate new systems, or perform upgrades can grow exponentially over time.

APIs, libraries and frameworks are typically updated every six months, and programming languages undergo significant changes every few years. While these updates bring improvements, they can impact project timelines.

Interpreted vs compiled programming languages

With an interpreted language like Python or PHP, if you notice a bug in the web app API logic or the UI script, you can fix it and test the change immediately. This is much faster than stopping to recompile and deploy every time, as would be required in a compiled language.

Without compiled binaries, the same source code can be deployed on multiple platforms without adjustments, as long as the runtime environment is consistent.

Many modern web frameworks (e.g., Flask, Laravel Vite) have features like "hot reloading," which automatically detect changes in code and reload the application without restarting. This is much easier to implement in interpreted environments.

While these advantages are significant, they may come at the cost of runtime performance and error detection, as compilation often catches errors early. However, for most web applications, the increased development speed and flexibility usually outweigh these trade-offs.

Music: The Cardigans - My Favorite Game