The downside of frequent releases

Nowadays we tell people to release often, to do continuous delivery or to do continuous deployment. Facebook for example does a release at least once a day (for a presentation about the Facebook release process, see here), Netflix does so at least once a week. The advantages are well-known and obvious, one of the most famous ones being reduced risk. But there are also disadvantages, which are rarely talked about. Since continuous delivery and deployment mostly focus on what advantages it has for us, as the developers, we tend to forget to check if customers do like it. And I have to say: As a customer I don't always like it.

I don't mind CD (continuous d...) on websites like Facebook or Netflix, because as a customer mostly I don't even realize that there's been a release. But they drive me crazy when it comes to apps on my mobile devices. I know that I can activate automatic updates, but I don't do this mostly due to security reasons. I have often removed apps because at some point they wanted to have additional rights I did concur with (example: When an app suddenly wants to have access to my contacts but is not contacts-related at all).

Since I had a constant feeling of "update-spam" regarding my apps, I decided to track the data for almost 2 months (March 16 - May 7). I did so for my 2 mobile devices, 1 iPhone 5 running iOS 9, 1 Nexus 7 tablet running Android 6, writing down the number of updates, installed Apps and OS version usually every two days (I did not manage to do so every two days). The Nexus 7 had 94 apps installed the whole time, the iPhone 5 57, with 58 apps for a few days. These are the results:

• In average there were 2 app updates on iOS and 3 on Android per day
• In regard to the apps installed that made up 3% of the apps installed per day
• In total there were 104 app updates on iOS and 164 on Android
• In total almost every app had 2 updates in 2 months

You can find the spreadsheet with the full collected data here.

You could argue that 2 updates for every app in 2 months is not so much and you would be right. I have not collected data on this, but the amount of updates is not spread across all apps, as some of my installed apps' last updates date back to December 2015 or even back to March 2015.

I have a similar experience with the program FileZilla. I use it once in a while (maybe once every 1-2 weeeks) and almost everytime I open it, it asks me if I wish to install the latest update.

For me, as a customer, having 2-3 so many visible(!) updates is really unbelievably annoying. Therefore my advice would be: If you plan on releasing often, also take into consideration what kind of product you have. Last but not least, try to make your updates as silent as possible.

The effects of team size

When I started at Infineon back in May 2015, one of my teams consisted of roughly 8 developers - a team size which in my opinion is already too large. The Scrum Guide proposes the following:

Fewer than three Development Team members decrease interaction and results in smaller productivity gains.  Having more than nine members requires too much coordination.

My own experience is that starting from around 6 people you already have too much coordination. For my team people in the company were complaining that the output of the team was so low. One of my suggestions was clear: Split the team. A suggestion which my predecessor also already tried to change for a year before I started. It took me around 1/2 year to finally get the team to split into two teams of 4. This is what happened to the velocity (the graph shows roughly one year):

I have asked the team in the retrospective directly following the team split (Sprint 100) aswell as after Sprint 100 why they think the velocity has gone up almost 100% and how they felt. This is what they replied:

• Higher identification with stories due to smaller teams
• More focused daily
• More pair programming
• No more "unpopular" stories
• Less time wasted (e.g. in meetings you cannot really contribute)

Now I'm certain this might not happen to every time. It is useful to know that in my case the stories in the initial team were quite diverse and spanned over several products that have to be integrated into the SDK they are building.

Release planning using the Agile Strategy Map

The situation

One of my teams provides the SDK for a range of hardware products, of which some are still in development. The hardware product development is working with milestones, at which certain new features of the product are available or changes to the hardware layout are made. The SDK is expected to support these features and layout changes with the release of the milestone. Although many features are known upfront, requirements do change in the course of the milestone completion. Example: The hardware layout has a flaw and needs to be changed.

The SDK does get a major version update (e.g. from 2.5.0 to 2.6.0) with a milestone release. The team was struggling to plan the releases. The scope of the features was unknown. Features that were not needed for the release were started to be built in, only to be stopped one sprint later, because a major feature request was forgotton until the last sprint before the milestone release. The team has 2 product owners, of which one is more of a consulting product owner, and at the moment one main stakeholder (apart from a lot more).

The approach

My approach was to use a hierarchical backlog to plan the contents of a release. There are quite some ways to achieve that, like User Story Maps, Impact Mapping or, my preferred approach in this case, the Agile Strategy Map. I helped the product owners and the one main stakeholder creating the Map using sticky notes of different sizes and colors. Our Agile Strategy Map consists of the following elements:

• Release goal (large sticky note): The goal of the release in one sentence
• Critical Success Factor (red sticky notes): A CSF is a feature/item/etc. that has to be done in order to reach the release goal
• Possible Success Factor (yellow sticky notes): A PSF is a nice-to-have. It does not have to be done in order to reach the release goal
• Necessary Condition (orange sticky note): A sub work-item of a PSF/CSF. In order to complete a PSF/CSF, every NC has to be completed.

Our first map draft looked like this:

The Agile Strategy Map at the start of the release planning

In order to create the map, we took the following steps in a series of sessions:

Collect contents for the release

Everyone writes down what they think and/or know has to be in the release

Weight contents using Business Value Poker

Similar to Planning Poker, everyone defines the business value of the contents defined. This will nicely show in a later step that business value does not automatically mean higher priority.

Define the release goal

Once the contents and the business values were clear, define the release goal. This is one sentence that describes what will be achieved with this release

Identify CSF and PSF

Once the release goal is clear, identity the critical items that have to be done in order to reach the release goal. Everything else will be a PSF.

Prioritize

Prioritize the PSFs and CSFs based on the available information likes Business Value, PSF, CSF, etc. We only used small sticky notes with the priorities on them combined with a discussion, but any other means of priorization such as dot-voting, "buy a feature" or else is fine aswell. This will show that high business value does not automatically mean high priority. On the picture above you can see that the item with the highest business value 3000 only had priority 5.

Identify the NCs

Identify all the requirements and work items a CSF or PSF has. You can group NCs below another NC.

The result

The Agile Strategy Map near the end of the release

Creating the Agile Strategy Map for release planning had several benefits. First off, since we put up the map on a whiteboard accessible for everyone, everyone including stakeholders could see what we were up to. In this case it actually lead to a stakeholder causing a priority shift, since she saw that an item that was a very important customer request only had priority 8. It also helped us to remove certain subparts (NCs) of a feature (CSF/PSF) we thought had to be in the release, the most common reason being that we didn't have enough time to complete all the NCs for a feature and they weren't crucial for the release anyhow (which we didn't know beforehand). Having a hierarchical backlog helped us to only remove certain parts of a feature easily instead of dropping the whole feature. Last but not least we could track throughput of PSFs, CSFs and NCs and because of that we were able to make pretty good predictions on how many items will fit in the next release (example: For the next release we only prioritized until 10, knowing that anything below most likely won't make it anyhow).

P.S. One side note: There is no rule how many user stories come out of a PSF/CSF or a NC. In our case we had PSFs that were only one user story and NCs that were 3 user stories.