You can improve but not perfect
Software development time estimates are notoriously difficult to get right. In fact, it's probably impossible to get 100% accurate estimates, but that doesn't mean your team shouldn't try! Just like everything else your team does, you should continuously improve and iterate your time estimate methods. With that in mind, let's dive in to a few easy places for improvement.
Definition of "done"
This one is critical. If your team doesn't have a written (even if it's one sentence) definition of done then you must agree on one before you do anything else. Lack of clarity on this topic can wildly swing time estimates. A developer's default estimate is usually "I solved the problem and it works in local development". They will often leave out time needed for writing tests, conducting UAT, deployment, etc. Naturally, this frustrates business stakeholders because their default definition of done is "deployed to production and serving customers".
If you do nothing else, get this nailed down!
Accounting for risk
This is where the "unknown unknowns" come in. Maybe the library your developers have chosen doesn't have all the functionality they thought it did, or the design team left something out. Like in investing, your team should be "pricing" their estimates to include uncertainty.
Anything unfamiliar to your team creates uncertainty. Here are a few examples:
1. Business decisions that haven't been made yet
2. Functionality that isn't yet designed
3. New technologies or languages
4. Data sources that haven't been selected or vetted
5. Undefined testing or deployment strategy
6. Varying developer skill levels and familiarity with the code base
7. Scope changes and context switching (see below)
Anchoring and peer pressure
In order to be meaningful, time estimates must avoid anchoring and peer pressure. The easiest way to accomplish this is via planning poker. When each team member submits an estimate without knowing what others' estimates are you get an honest accounting of the amount of work to be done (including risk!). After estimates are given then estimators can defend their positions and ask questions until the team reaches consensus.
Deal with tasks that are too big
Sometimes during planning poker sessions there may be large estimates or tasks that are seemingly impossible to estimate. Large tasks should be broken down in to their atomic parts — that is, bite-sized chunks that are easier to estimate. Be careful, though! Don't simply add up all the smaller task estimates and say that's the overall estimate. There is always a bit of extra effort involved to tie everything together.
If a task is too difficult to estimate then the team should identify what it needs to know in order to estimate the task properly. The task then becomes discovery, which should be easier to estimate, and the original task should be moved to the backlog. Once the team has gathered the new information it needs they can make an informed estimate and proceed with the original task.
Scope changes and context switching
Changes in scope are usually the result of miscommunication. When a stakeholder reviews work and functionality doesn't match expectations then a scope change typically follows. This is an indication that your team needs to improve communication and/or better define requirements.
Context switching's costs are well-documented, so I won't belabor them too much. Suffice it to say that if your developers are working on multiple projects, have other responsibilities, or a lot of meetings, then you should be increasing estimates to compensate.
Need a hand?
Want to go more in-depth?
This article is pretty high-level. If you want more details and tactics on how to deal with developers, sign up to get 50% off my upcoming book, Dealing with Developers.