The Why of Agile

If you ask people what agile is, they'll usually refer to characteristics. Iterations, flexibility, collaboration.

But a lot of the time they miss the point, and they don't have a real sense of what agile is.


Agile is process improvement for software development.

That's it. All the processes and ceremonies should be understood in that context to make them effective.


The following process improvement principles provide context for agile processes and ceremonies.

a. Do What Works

Outcomes over process: If process doesn't work, change it. General principles can be useful, but should not be applied at the expense of outcomes.

b. Aim for the Target

Activities should be aligned towards end goals, for a more efficient and more effective result. Aligning to secondary goals, or established processes, or general principles, will sub-optimise.

c. Customer Quality

A solution can be beautifully engineered in a way that doesn't matter to the customer.

Instead of saying "What do customers want", say "I'm a customer. What do I want?" (Customer method acting). But there will always be a gap if you don't observe and talk to customers!

d. Gemba

In process improvement, you go where the work happens (gemba means workplace). Sit with or directly observe users, try the product as a customer, answer support calls.

e. Self Organise

Instead of a steamship where the captain rings a bell and the workers stoke the boilers, arrange the organisation as a swarm of clever agents working in parallel for maximum value and maximum efficiency. Avoid heirarchical control and Management By Objective.

f. Pull, not Push

This principle from kanban has deeper psychological implications. Instead of pushing tasks on workers and controlling them to complete within an agreed timeframe, workers pull tasks when they want to at the fastest rate they can manage.

g. Continuous Improvement

The PDCA cycle (Shewhart cycle) is about observing, measuring, creating experiments. Use data where possible, and avoid confirmation bias.

In kaizen, everyone has a "job 2": to improve their primary job.


In light of the principles above, following are the processes and ceremonies in agile, why they exist, and the spirit in which they should be implemented.

1. Collaboration

Find optimal solutions by matching a range of desirable outcomes to a range of possible solutions, and developing better outcomes and solutions through this process.

The ideal is direct interaction between end users who have real needs, and developers who know available solutions.

Third parties in this process should be facilitators who reduce friction, not buffers who slow interactions and prevent optimal solutions.

2. Pigs and Chickens

This is from a scrum fable about a pig and a chicken who open a bacon and egg restaurant. The chicken says "I'm really committed to this project". The pig says "You're just involved. I'm committed."

Agile seeks to involve internal and external customers who have a vested interest to improve the outcome for end users.

3. Flexibility

Flexibility is just a side-effect of a continuous focus on outcome over process. Flexibility and adaptability are not the main aim of agile!

4. User Stories

As an X I want Y so Z.

The point is to:

  • Focus on customer quality by identifying the customer
  • Identify who to talk to about the user story
  • Focus on the reason (outcome)
  • Facilitate customer method acting

5. Points

It's simpler to estimate in hours and days. But we estimate in points because:

a. We are bad at estimating time. Points allow us to baseline and calibrate. (We are good at pairwise comparison, and this can be used before points are allocated).

b. Development is problem solving. It's inherently unknown and time variable, so we don't promise hours and days. Points insulate developers from the need to promise delivery dates.

Points are not to establish a baseline contract for developer performance. They are to measure and continuously improve velocity.

Side note, we often use the Fibonacci sequence because powers of 2 escalate too quickly.

Graph of Fibonacci vs Powers of 2

6. Parkinson's Law

Tasks expand to fill the available time. When given a defined timebox, developers will

  • Work in the direction of the goal without taking the most direct path (engage in process instead of delivering only outcome)
  • Do unplanned refactoring or over-processing
  • Delay delivery to meet the agreed timeline and avoid surprising stakeholders
  • Delay delivery to keep expectations low, and maintain a buffer for future estimates
  • Delay delivery as a form of self-determination

All these factors are significantly worse in bureaucratic, control-focused environments with poor psychological safety.

Scrum is particularly prone to enforcing delivery within agreed timeframes.

7. Working Software

In the spirit of "outcome over process" and "aim for the target", and to facilitate continuous collaboration and improvement, working software is the desired outcome and measure of success.

8. Self Organisation

People doing a job are best placed to efficiently organise it.

9. A toolbox, Not a Bible

Agile contains a set of approaches and ceremonies. But they have to be selected/adapted to work in each situation. Do what works.

10. Sustainable Pace

Agile culture is about respect for and full engagement of individuals. A happy team will tend to be more productive, but team happiness is an end in itself. Sustainable pace also equates to the kaizen principle of "smooth and even flow" to maximise efficiency.

11. Removing Blockers

Scrum uses the unfortunate term "scrum master" to refer to an agile delivery facilitator. The role should be less about control and more about enablement and removing blockers. There should be a recognition that many activities of an agile facilitator might be worthwhile, but they are not "the work".

12. Iterations

  • Shewhart Cycle
  • Baseline productivity to improve velocity
  • Broad stroke timelines

13. Retrospectives

Retrospectives are to identify improvements. A list of problems without proposed solutions is not a good retro.


(Will be hidden)

Restricted HTML

  • Allowed HTML tags: <a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type> <li> <dl> <dt> <dd> <h2 id> <h3 id> <h4 id> <h5 id> <h6 id>
  • Lines and paragraphs break automatically.
  • Web page addresses and email addresses turn into links automatically.
2 + 2 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.
Are you human?