Using Data To Become An Agile Analytics Team

How we used Actionable Agile to improve our workflow by 30%

Back in March this year (2021), I set out to understand whether my team’s JIRA board was as efficient as it could have been. I reached out to one of the Agile coaches at work for help (Hi Zara). What followed was a journey down a road to a complete shift in mindset and ways of working that I had not imagined.

During our first session, Zara mentioned that the state of our JIRA board was in pretty good shape and in line with Agile best practices. Being an analytics team, we tried to“adopt” the Kanban methodology and mindset to help manage and visualise our work. Stakeholders would raise tickets that would sit in our backlog and working with them we’d prioritise accordingly. Unfortunately, that’s where our adoption of Kanban ended. We weren’t doing anything that even remotely represented the core philosophy of Agile. We were not an Agile team!

Kanban Board Setup

The screenshot below shows our current Kanban board. If you use the Kanban method, your board might look something like this:

Product Analytics Team JIRA Board

The columns we use to manage our workflow are backlog, this week, in progress, in review, done and on hold (hopefully, the names are self-explanatory). And we use swimlanes to represent the different teams we support. There is also a swimlane dedicated to functional work —  personal development, improvements to data quality, hiring, etc…

Actionable Agile (AA)

After we reviewed the board setup, Zara asked if I had come across Actionable Agile. I had not. But as soon as Zara loaded this scatterplot from the demo account to show me AA, I was hooked. I didn’t know what I was looking at, I just knew that whatever it was, I wanted it — I am an analyst after all. Screenshot of the “Cycle Time Scatterplot” with demo data below:

Actionable Agile Demo Data

What is Actionable Agile?

Actionable Agile is a way to help you understand and visualise how your team’s work moves through the process (columns). There are a bunch of charts to help you analyse key workflow metrics such as:

  • Cycle time —  how long it takes a ticket to get to the “done” column once it’s out of “backlog”
  • Throughput —  the number of tickets finished in any period

You can also run Monte Carlo simulations to predict how long a piece of work is going to take. Below is a list of all the available things you can do with AA. I’ve marked the ones we used every week to help us with our process.

Learning about Actionable Agile

Despite my obvious enthusiasm for the charts, I wasn’t naive enough to change the team’s ways of working until I fully understood what I was looking at and how we might benefit from it. So I put a bunch of sessions in with Zara to learn more about Agile ways of working and Actionable Agile.

Below is a summary of the 3-week crash course I did with Zara and the outcomes:

Actionable Agile allowed me to analyse my teams average cycle time (see graphs below) at a macro-level and understand when tickets were at risk at a micro-level. Using the data in AA, I established what my team’s historical baseline numbers looked like and identified the actions we should take to improve them. I also made some changes to the JIRA board so that we could use AA more effectively. Having learned enough to know this was something we could benefit from and feeling confident in what I was talking about, I shared AA with the team — They were equally excited about the graphs. Once they were all bought in we created an OKR for ourselves.

Setting an Objective

The objective was to reduce variability in our data and become more predictable as a team. The key result was to reduce the 80th percentile cycle time by 30% for Q2.

Results

Note: Each point represents one or more JIRA tickets and the y-axis is the cycle-time, i.e. how long it took the ticket to move to “done”. The x-axis is the day the tickets were moved to “done”. All coming together to create a time series of our work.

Before

Pre-Agile graph

After

Post-Agile graph

At first glance, it might not be easy to see the impact but if you look closely, you’ll observe the following:

  • the data points on the scatter graphs are squeezed together a lot more and we have significantly fewer outliers (reduced variability).
  • The 80th percentile line has gone from 30 days to 19 days (37% improvement). OKR achieved!
  • The 70th and 50th percentile lines are down by 30% and 33% respectively.

How did we achieve this?

Having access to our historical data was useful as it allowed us to see how we were doing as a team, but we needed to change our ways of working if we wanted to see an improvement in our cycle time.

1) Standups

The most basic change we made was the introduction of standups! We had team meetings every Monday which covered a whole host of topics, including what the team were working on, but rarely did we look at the JIRA board. So we made a decision to spend the first 30 minutes going through every single active ticket in each swimlane and column. We also added a shorter 15-minute standup on Wednesday mornings.

The simple act of looking at the board frequently played a huge part in this. Maybe it was just the dopamine rush you get from moving a ticket to “done”. Whatever the reason, there was definitely a positive psychological impact of doing standups. I’m even certain that some proportion of our reduced cycle-time was due to the fact that we were moving and closing tickets in a timely fashion.

2) Reducing Work in Progress (WIP)

Every Monday I would send an update to all our stakeholders on the things we were working on but by mid-week, it was clear that the team weren’t going to finish half of what they had committed to. It wasn’t their fault! The constant barrage of meetings, data issues, feedback loops, emails, slack messages and “small” requests meant that finding solid blocks of time to focus on actual work was impossible.

So along with standups, the next change was reducing the amount of work we were committing to each week. We took on less, so we could deliver more. Once again, the psychological impact of having fewer things in progress, meant that the team could focus more and deliver frequently.

3) Reviewing Aging Work in Progress

The second chart that we used every meeting was the “Aging Working in Progress” chart. This chart allowed us to see the age of each active ticket and spot anything that was at risk. Sometimes there is so much going on that it’s easy to lose track of time. Having an easy way to identifying risks, meant that we could act quickly. This could be updating stakeholders, reprioritising work or chasing dependencies. (Yes I know there is a ticket floating in the red zone. We’re only human).

4) Better Scoping of Bigger Tasks

We haven’t perfected this one, but we’re a lot more mindful of it. There was one piece of analysis that took two months to complete and we should have broken it down into smaller tasks in hindsight. In fact, this analysis was the catalyst that led to me reaching out to Zara. The outcome has been a better scoping of work, agreeing on clear outcomes with stakeholders and the use of epics when needed.

5) Controlling the Controllable

All the above has made us realise that whilst we’re vulnerable to many factors outside of our control, there is so much that we as analysts have within our control. Saying “no” or “not yet” when someone asks us to do something that was not on the priorities is one of those things. To this day, I know the team are still saying yes to the “small favours” but they’re saying it less frequently. I’ve also observed a change in the team’s attitude. They’re ringfencing their time more by not dwelling on tasks. When SQL queries keep timing out, they move on. When they’re awaiting stakeholder feedback, they move on.

Lessons Learned

We’ve seen a huge improvement in our workflow metrics and I truly believe that what we’ve learned can be applied to any analytics team, but there are some final lessons that I wanted to share if you do embark on this journey yourself.

  1. Create a safe space for your team — This is not a naming and shaming exercise. They should be able to discuss openly and honestly why things are taking longer than they hoped without feeling like they’re under a microscope.
  2. Remain disciplined — don’t skip the standups under any circumstance.
  3. Continue adapting and iterating — we still haven’t got it perfect. Keep reviewing your process.
  4. Be honest — are the numbers improving because you’ve improved your process or are you engineering the numbers?
  5. Adopt good JIRA practices — AA is not perfect so get into the habit of using features in JIRA to help you remove the noise in the data.

Final words

I want to say a big thank you to Zara for opening my eyes to a whole new world of data, but more importantly, having the patience to help me understand how to use it properly. If you’re lucky enough to have Agile coaches at your company, I suggest reaching out to them. I’ve learned so much over the last 5 months.

You can follow me on Twitter or with minimal effort you can follow me and CRAP Talks on Medium.

Add a Comment

Your email address will not be published.