The Why of Agile

If you ask people what agile is, they'll usually refer to characteristics. Iterations, flexibility, collaboration.

But a lot of the time they miss the point, and they don't have a real sense of what agile is.

THE DEFINITION

Agile is process improvement for software development.

That's it. All the processes and ceremonies should be understood in that context to make them effective.

THE PRINCIPLES

The following process improvement principles provide context for agile processes and ceremonies.

a. Do What Works

Outcomes over process: If process doesn't work, change it. General principles can be useful, but should not be applied at the expense of outcomes.

b. Aim for the Target

Activities should be aligned towards end goals, for a more efficient and more effective result. Aligning to secondary goals, or established processes, or general principles, will sub-optimise.

c. Customer Quality

A solution can be beautifully engineered in a way that doesn't matter to the customer.

Instead of saying "What do customers want", say "I'm a customer. What do I want?" (Customer method acting). But there will always be a gap if you don't observe and talk to customers!

d. Gemba

In process improvement, you go where the work happens (gemba means workplace). Sit with or directly observe users, try the product as a customer, answer support calls.

e. Self Organise

Instead of a steamship where the captain rings a bell and the workers stoke the boilers, arrange the organisation as a swarm of clever agents working in parallel for maximum value and maximum efficiency. Avoid heirarchical control and Management By Objective.

f. Pull, not Push

This principle from kanban has deeper psychological implications. Instead of pushing tasks on workers and controlling them to complete within an agreed timeframe, workers pull tasks when they want to at the fastest rate they can manage.

g. Continuous Improvement

The PDCA cycle (Shewhart cycle) is about observing, measuring, creating experiments. Use data where possible, and avoid confirmation bias.

In kaizen, everyone has a "job 2": to improve their primary job.

AGILE PROCESSES AND CEREMONIES

In light of the principles above, following are the processes and ceremonies in agile, why they exist, and the spirit in which they should be implemented.

1. Collaboration

Tight end-to-end collaboration comes from process improvement.

The ideal is direct interaction between end users who have a range of real needs, and developers who know a range of available solutions. Matching them identifes optimal solutions, to maximise customer value and reduce build time.

Third parties in this process should be facilitators who reduce friction, not buffers who slow interactions and prevent optimal solutions.

Free collaboration also enables immediate removal of blockers.

2. Stand-Ups

If stand-ups are used to monitor or encourage performance, you have moved from a pull model of trust and enablement, to a push model of control. This is the worst case scenario you fall back on when the good way has failed.

Stand-ups can be used to provide visibility across the team, and to identify opportunities for collaboration and help. But team members shouldn't feel obligated to prepare a list of sub-tasks to recite as proof of work.

3. Pigs and Chickens

This is from a scrum fable about a pig and a chicken who open a bacon and egg restaurant. The chicken says "I'm really committed to this project". The pig says "You're just involved. I'm committed."

Agile seeks to involve internal and external customers who have a vested interest to improve the outcome for end users.

4. Flexibility

Flexibility is just a side-effect of a continuous focus on outcome over process. Flexibility and adaptability are not the main aim of agile!

5. User Stories

As an X I want Y so Z.

The point is to:

  • Focus on customer quality by identifying the customer
  • Identify who to talk to about the user story
  • Focus on the reason (outcome)
  • Facilitate customer method acting

6. Points

It's simpler to estimate in hours and days. But we estimate in points because:

a. We are bad at estimating time. Points allow us to baseline and calibrate. (We are good at pairwise comparison, and this can be used before points are allocated).

b. Development is problem solving. It's inherently unknown and time variable, so we don't promise hours and days. Points insulate developers from the need to promise delivery dates.

Points are not to establish a baseline contract for developer performance. They are to measure and continuously improve velocity.

Side note, we often use the Fibonacci sequence because powers of 2 escalate too quickly.

Graph of Fibonacci vs Powers of 2

7. Parkinson's Law

Tasks expand to fill the available time. When given a defined timebox, developers will

  • Work in the direction of the goal without taking the most direct path (engage in process instead of delivering only outcome)
  • Do unplanned refactoring or over-processing
  • Delay delivery to meet the agreed timeline and avoid surprising stakeholders
  • Delay delivery to keep expectations low, and maintain a buffer for future estimates
  • Delay delivery as a form of self-determination

All these factors are significantly worse in bureaucratic, control-focused environments with poor psychological safety.

Scrum is particularly prone to enforcing delivery within agreed timeframes.

8. Working Software

In the spirit of "outcome over process" and "aim for the target", and to facilitate continuous collaboration and improvement, working software is the desired outcome and measure of success.

9. Self Organisation

People doing a job are best placed to efficiently organise it.

10. A toolbox, Not a Bible

Agile contains a set of approaches and ceremonies. But they have to be selected/adapted to work in each situation. Do what works.

11. Sustainable Pace

Agile culture is about respect for and full engagement of individuals. A happy team will tend to be more productive, but team happiness is an end in itself. Sustainable pace also equates to the kaizen principle of "smooth and even flow" to maximise efficiency.

12. Removing Blockers

Scrum uses the unfortunate term "scrum master" to refer to an agile delivery facilitator. The role should be less about control and more about enablement and removing blockers. There should be a recognition that many activities of an agile facilitator might be worthwhile, but they are not "the work".

13. Backlog Grooming

The Product Owner grooms the backlog to ensure stories are prioritised and Ready for developers. This is equivalent to Kanban principles of Just in Time and continuous flow, ensuring that upstream processes supply downstream processes in time.

14. Sprint Planning (Backlog Prioritisation)

This is to prioritise early delivery of greater customer value, and to orchestrate the flow to reduce waste.

15. Limited Work in Progress

Completing one user story at a time reduces context switching cost (the overhead of readjustment from one task to another). Problems in production should immediately be addressed to unblock the flow. These principles are both from kanban.

16. Continuous Integration/Continuous Deployment

  • Continuous Integration (CI): Developers frequently merge their code changes into a central repository, after which automated builds and tests are run.
  • Continuous Deployment (CD): Automatically releasing a developer's changes from the repository to production, ensuring the software is delivered incrementally.

This Just in Time and smooth and even flow approach is also from kaizen.

17. Iterations

Iterations equate to the Shewhart Cycle (Plan, Do, Check, Act), with a retrospective after each cycle.

They also baseline productivity to continuously improve velocity.

Iterations also enable broad stroke timelines, with a recognition that much of development is problem solving which is inherently variable in duration.

18. Retrospectives

Retrospectives are to identify improvements. A list of problems without proposed solutions is not a good retro.

Comments

(Will be hidden)

Restricted HTML

  • Allowed HTML tags: <a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type> <li> <dl> <dt> <dd> <h2 id> <h3 id> <h4 id> <h5 id> <h6 id>
  • Lines and paragraphs break automatically.
  • Web page addresses and email addresses turn into links automatically.
CAPTCHA
5 + 12 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.
Are you human?