Agile, everyone talks about it, but is anyone actually doing it?

No, not behind the bike sheds… Almost every company say’s “we’re agile”, and in many cases, like the peer pressured teenager, they say what they think makes them cool. Having a Jira board doesn’t mean you’re agile, neither does working in sprints. Perhaps agile development is confined to a small corner of the company, with sand in the gears everywhere the team interfaces with other teams or departments in the business. Enter the mega-waterfall of dependencies, hello 18 months lead project time, goodbye agile. So, while some companies clearly “do Agile” (whatever that means), in the world of business and tech there is clearly a lot more talk about Agile than there is action!

Agile stems from software engineering in 2001 when the agile manifesto was born out of a now famous Extreme Programming meeting in the Utah mountains in 2001, and has subsequently taken the software engineering storm. In the world of data, many data engineering / platform team and data science teams might “do agile”, but for analyst teams, both the principles are processes are much less commonly applied. If Agile is so great for software engineering teams, why shouldn’t analysts get a piece of the action?

At Gousto we operate a tribe and squad model of cross-functional teams dedicated to a particular part of our overall proposition, with some, but not all analysts “embedded” into a given cross functional squad with an interesting vegetable name (turnips, kales, jalapeños etc.). These squads have significant autonomy to determine their tactics and manage their systems as they see fit, which should lend itself well to Agile working, whatever that means…

The Agile Manifesto and the case for Agile Analytics

The Agile Manifesto is alarmingly simple, and almost fits into a single tweet:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

Alright, there are also 12 Agile Principles that underpin it, but the spirit is captured in the Manifesto.

Basically, the takeaways are, communicate with your stakeholders lots; accept that there will be change; devolve decision making as much as possible; get working stuff out there, iterate and continuously develop.

And there you have it, strip away all the Jira and jargon which is probably best associated with Agile working today, and the objective to release better “products” and adapt to change now sounds eminently applicable to Analytics as well as software engineering!

We could debate what a “product” means for analysts or data teams, but the short answer is it ranges from a quick piece of analysis to support a decision, bigger one-off projects, continuous work streams to optimise a product or feature or an actual analytical product. For the Gousto analytics team it is all of the above. Regardless of the maturity of your team, the principles however still hold true.

Consider the following situation every analyst has encountered at least once, if not on a regular basis: Important stakeholder X asks for some really important analysis, gives minimal context or brief. 4 weeks (or months!?) later, the response to the fancy slide deck presented by the analyst is “thanks, this is all very interesting but doesn’t really answer my question”. It’s frustrating for stakeholders, demoralising for the analyst and a big ol’ waste of time.

This is effectively the equivalent problem that the extreme programmers in 2001 were trying to avoid so it stands to reason that analytics teams should benefit from Agile in the same way that software engineers have.

Back to where it all started?

So, given analysts face similar problems to software engineers for which the Agile manifesto was developed, why don’t analysts “do agile”?

A short, and perhaps biased opinion is that all the technical jargon and agile processes developed in the years since 2001 has obscured the original vision of Agile, which, ironically breaks the first principle of the manifesto (Individuals and interactions over processes and tools)! Clearly, different teams have different needs, meaning more or less process is required. So, true to the spirit of Agile, start with something that seems sensible and adapt as circumstances require.

As a data company (that loves food), Gousto has a comparatively large tech team relative to it’s size, with a modern tech stack and some well defined Agile processes for software engineering, so this was the logical starting point for me to adapt them to them to meet the needs of analysts.

It’s not what you do it’s how you do it

To help keep us true to the spirit of Agile, we have the following process and tooling in place:

• Lightweight roadmap: Minimum 1 quarter of epics (bigger, harder, important stuff) planned and agreed with PMs via our quarterly OKR process.
• Visualise work on boards: Does what is says on the tin.
• Definition of done: A watered down version of our engineering definition of done; code reviewed and in GitHub; analysis shared/communicated with/to stakeholder; outputs linked in Confluence if relevant.
• Work in small batches: I cannot emphasise enough how important this is. Analyst time is easily wasted, and breaking jobs down as much as possible is the best method of de-risking our work and ensuring our time is spent on things that matter.
• Shitty first draft: This is an “analytics only” addition, borrowed from Taylor Brownlow’s great article on Agile for Analytics. It’s similar, but not necessarily the same as reducing batch size. Share early, share frequently, invite feedback from fellow analysts and the stakeholders you work closely with (PMs etc). Clearly this might not work with all stakeholders, but seniority needn’t (or shouldn’t?) be a reason. More likely it’s people who “get” data vs those who don’t.
• Hold [some] standups: Daily standups might be overkill, or maybe your standups are not standup-y enough?
• Do some retrospectives: Since we don’t work in sprints (and the manifesto doesn’t say we need to), and our work is less well planned and groomed than many engineering teams, we don’t have a retro after each sprint. We do run retros from time to time, normally focussed on projects that were challenging, had issues etc.
• Limit work in progress: Focus on finishing rather than starting.
• Effort and points: We don’t bust out the Fibonacci sequence to estimate story points and effort. Since the aim is to work in small batches, we make a reasonable effort to estimate epics, but less to individual tickets, especially ad-hoc requests.
• Measure cycle time and throughput: Given we don’t fixate on effort or story points, the idea here is to track cycle time and ageing work in progress to ensure we are a) working in small batches and b) focusing on finishing rather than starting.

And there you have it. A back to basics, no frills approach to Agile. The original and perhaps still the best?

Again, for every team your tooling and process will be a bit different, but that is literally the whole point of Agile! So, embrace the Agile Manifesto, start with some simple process and tooling and get a piece of the agile action!

Check out more stories from Gousto and follow us here