Hello and welcome to this, the third article in the series ‘How Not to be Agile’.
‘How Not to be Agile’ may seem a strange title for blogs about how good Agile is! What I intend to do over this series of articles is to share with you the misinterpretations, omissions, and mistakes that people make that significantly reduce the potential benefits when an organisation, or a part of it, embark on an Agile Transformation.
Last time, I discussed the potential pitfalls of not having a suitable Business Case for whatever it is that we are trying to achieve, be it a new product, a major change to an existing product or the Agile transition itself.
This time, I will cover some of the misunderstandings and malpractices that I have come across with respect to the next major item to consider in any Agile framework, the requirements. The most often used term for the requirements list is the Product Backlog from Scrum; I will use this and other Scrum terminology in this article but you can use whatever names you like; it is the content and management of the requirements list that is important.
I will start with a description of the Product Backlog process and then give examples of what has and could go wrong if not follows the Product Backlog tips.
The Definition of Scrum Product Backlog
Having achieved an agreed Vision and Objectives and decided that the Business Case for the product development is viable, the business stakeholders and development team need to discover a list of ‘business requirements’, at the same level of detail, so that the development team can develop the product in a series of short development timeboxes.
These ‘requirements’ become Product Backlog Items (PBI).
What is a ‘Requirement’?
Functional Requirements:
There are 3 commonly used ‘structures’ used to express ‘Agile functional requirements’:
Elementary Business Processes (EBP) – the names of EBP, usually extracted from the results of the Business Process Modelling exercise concerned with the area of the business that the product is for; for example:
‘Capture Customer Details’
Use Cases– from the Unified Modelling Language (UML). Only the names of the Use Cases, again, usually extracted from the results of Business Process Modelling exercise and the example is the same as for the EBP above User Stories – a technique from eXtreme Programming which states the user of a business process, the name of the process and why the process is needed; for example: ‘As a sales person I need to Capture the Customer Details So that payment can be taken’
For all the 3 ‘requirements’ types, it is only the names of the business process that are needed (with the addition of the other 2 clauses in User Stories); there should be no lower level details in the PBI. The level of detail for all of the above can be summarised as: ‘The name of one job, being performed by one person, at one time’
Non-Functional ‘Requirements’:
In addition to the Functional Requirements, the ‘What’, the business will also express the needs of ‘How’ the product will perform such as:
All screens must refresh within 5 seconds
The system must be available 24/7/365
The system must conform to the company marketing graphic language and so on and so forth.
These Non-Functional ‘requirements’ can be added to the Product Backlog in a separate section from the Functional Requirements or in a separate ‘document’.
What A PBI Is Not
There was a fashion, that still exists in some places, to have ‘Technical User Stories’; ‘User Stories’ those name technical activities that need to be done; for example: As a database designer I need to create data access scripts So that the product can store and retrieve the data that it needs to use
Such User Stories would just be stating elements of a team member’s work and do not add anything to the understanding of the business needs which the Product Backlog is meant to represent.
It is my strong recommendation that so-called ‘Technical User Stories’ are NOT used because the Product Backlog becomes large, unwieldy, almost impossible to order by business value and prioritisation; the estimated effort for these technical needs are included in the estimation of the ‘business’ PBI; more of which later.
It is true that there may be a need for some activities to set-up the technical environment before the development team can start developing the product, some of which are out of the control of the development team; these activities should take place in parallel with creating the initial Product Backlog.
Creating a Product Backlog
Ordering the Product Backlog The whole point of Agile is to deliver a product incrementally so that the business can accrue the best value in the shortest possible time.
It, therefore, follows that the product needs to be developed by developing the PBI in the business value order; the highest business value PBI first.
Therefore, the PBI needs to be ordered by business value. I will not go into the techniques for ordering PBI, they are well documented elsewhere but I will give an example, later, of what happened when one team did not use Product Backlog ordering.
Estimating the PBI We create the initial Product Backlog just after agreeing on the Vision and Objectives and ensuring development viability via the Business CASE but before development begins; it needs to be estimated in some way to determine how long it may take to develop the product and consequently how much it may cost to develop.
It is a tenet of all Agile frameworks that only the development team is allowed to estimate the PBI. Once done, these estimates will ‘inform’ the Business Case and analysis can be made to see if the original ‘high-level’ estimates in the initial Business Case are of the same order; a decision can be made as to whether the development is still viable.
Types of PBI estimating There are 2 generally used types of PBI estimating:
1. Absolute Estimating: Whereby each PBI is estimated by how much time it would take go from just its name to ‘potentially shippable’ functionality. Of course, the tasks include analysis, design, construction and testing and what normally happens is that each ‘skill’ will estimate the time it would take for their part in the process and the times summed to get the overall estimate.
2. Relative Estimating: Whereby the team chooses the PBI which is likely to be ‘the smallest’.
“The smallest in what respect?” you ask. The ‘smallest’ does not directly consider time; the PBI criteria that are considered are size and complexity.
However, opinions of size and complexity will vary amongst all team members depending on their skill and experience; what may seem ‘simple’ to an experienced developer may seem complex to the less experienced person. Similarly, although the analysis and construction may seem simple, the testing may be complex.
For these reasons, it is highly recommended that Relative Estimating is done using ‘Planning Poker’ which is well described elsewhere (see ‘Planning Poker: An Agile Estimating and Planning Technique’); suffice to say here, that once the team have decided which is the ‘smallest’ PBI, they give it a ‘Story Point’ of 1 and all other PBI are estimated with respect to their individual relative size and complexity compared to the ‘smallest’.
So which estimating technique to use? 1. Absolute Estimating can take a long time; it is akin to the traditional way of estimating for the Project Plan. Also, it has been established that, generally, we are not very good at estimating in time; the larger the thing we are trying to estimate, the worse our estimates get. However, the technique is familiar to most people and does not require a change of mindset to produce estimates.
2. Relative Estimating requires a totally different mindset amongst the development team members because, initially, not only are they using unfamiliar criteria to estimate the PBI but also, they are estimating in collaboration with other team members with different skill sets and experience.
Despite there being ample evidence that Relative Estimating takes less time overall and allows the stakeholders and development team to focus on business benefit being delivered, because of the ‘difficulties’ of introducing Relative Estimating, many teams revert to time estimation and tracking time. Also, as we will see in one of the Case Studies below, an organisation’s choice of requirements management tool can make the use of Relative Estimating seem a waste of time for team members.
Another Potential Problem with Relative Estimating So, having carried out ‘Planning Poker’, the development team now know the total number of ‘Story Points’ that the development represents; not much good to the business stakeholders who want to know how long it is going to take and how much it is going to cost.
On completion of the initial Product Backlog and before development begins, we need to ‘translate’ this total ‘Story Points’ into time.
An Agile Team’s progress is measured by the number of ‘Story Points’ that they deliver in one Sprint; known as the team’s ‘Velocity’.
So we need a knowledge of the team’s Velocity in order to establish an estimate of how long the development is likely to take; for example, if the total number of ‘Story Points’ is 200 and the team’s velocity for a 2-week Sprints is 10, then the estimated time to complete all the PBI is:
200 ‘Story Points’ / 10 ‘Story Points per Sprint x 2 (the number of weeks per Sprint) = 40 weeks
However, at the start of an Agile transition, we do not know what the team’s Velocity is or will be. There are 2 methods to determine what the Velocity may be:
1. The team takes one of the ‘smallest’ PBI (with a Story Point of 1) and does a traditional break down to the tasks involved and estimate those tasks in time; 1 ‘Story Point’ is equivalent to the time represented by the sum of the times estimated for all the tasks The team’s Velocity may be estimated by: ‘The number of working days in the Sprint / the estimated days for 1 Story Point’ For example, if the ‘time estimate’ for a PBI of 1 ‘Story Point’ is 1 day: 10 working days in 2-week Sprint / 1 day = Velocity of 10 This velocity can then be used in the formula above to determine the estimate for the potential total time for the development.
2. The development team is allowed to start development for 2 or 3 Sprints taking PBI from the top of the Product Backlog. At the end of the ‘experiment’ period, the total number of ‘Story Points’ for the completed PBI divided by the number of Sprints completed becomes the Velocity to use in the formula above to determine the estimate for the potential total time for the development.
The potential problem with this method is that management have to release funds for this experimental period without knowing the values to feed into the Business Case to determine whether the product development is viable before development begins.
Once teams have completed 1 product development, we recommend that the team members are kept together for other product developments even though the business area or technology may be different. Having a team that works well together is more important than creating teams in an ad-hoc manner. A team’s Velocity varies insignificantly over different product developments and so can be used directly in the formula above to determine the estimate for the potential total time for the development.\
Scoping the Product Backlog Generally, Agile product developments have strict deadlines so that the major benefits can be accrued; benefits such as competitive advantage, meeting marketing needs, meeting customer expectations or complying with regulations.
Also, Agile teams comprise all the skills necessary to take the PBI from ‘names’ to a potentially shippable product; not all team members are full-time but the team is fixed.
Therefore, generally, the costs for a fixed time product development are also fixed. Given the classic ‘Iron Triangle’ of development variables:
If we fix time and cost, then the only thing that can vary is the Requirements that are delivered in the fixed time for the fixed cost.
It is quite probable that the initial Product Backlog will contain more PBI than could reasonably be developed in the fixed time available; because the stakeholders/Product Owner has ordered the Backlog by business value, it is relatively straightforward to ‘draw a line’ underneath the lowest PBI in the list that will probably be developed. We do not remove the lower value PBI because it has been often found during development that a high-value PBI is actually dependant on one of the lower value ones and the ordering has to be amended.
Stakeholders need to understand that even though some requirements (PBI) have been requested, it is probable that they will not be delivered.
But the Product Backlog scoping does not stop there. The ‘story Points’ for the PBI are estimates and, by definition, they are not accurate. In the past, to account for ‘inaccurate’ estimates, organisations have added contingency in time and costs to the estimates.
However, in Agile environments where the time and costs are effectively fixed, the development needs some ‘wriggle room’ for when estimates are proved significantly wrong and when requirements change during the development. For an effective Product Backlog, you need to follow Scrum Product Backlog tips.
The Minimum Viable Product (MVP) The development team must agree with the Product Owner/Stakeholders which PBI represent the Minimum Viable Product, ie what is the minimum to be delivered in the fixed time in order to accrue the minimum benefit required.
There are ‘rules’ how big this MVP can be. It has been found, from hard-won experience that for a ‘comfortable’ working environment:
1. When a team first starts an Agile transformation they should accept PBI in the MVP which represent around 50% to 55% of the estimated effort; ie if there are 200 ‘Story Points’ in the probable PBI, then the MVP should only contain the highest value PBI whose ‘Story Points add up to between 100 and 110.
2.When the team and the organisation are reasonably Agile-mature, between 9 months and a year of transformation, the team can accept PBI representing up to 65% of the total estimated effort in the MVP.
3.When the team and organisation are fully Agile-mature, PBI representing up to 70% by estimated effort may be accepted in the MVP. It has been found that no matter how good the teams and organisation are, the variables that come along with all product development make promising more than 70% by estimated effort fraught with problems; the teams must strive not to give stakeholders over-expectations.
Looking to become a project management expert? Our PRINCE2 Foundation certification offers the perfect opportunity to achieve mastery. With flexible online training and comprehensive materials, you'll have the support you need to succeed. Enroll now and start building your path to success!
The MoSCoW Rules
Some Agile Frameworks recommend using a technique called MoSCoW to ‘categorise’ the PBI. The acronym stands for:
Must Haves: Those PBI that is essential to the product; equivalent to the MVP above.
Should Haves: Those PBI that is important but there is a workaround for them if they not completed in the fixed time
Could Haves: Those PBI that might be developed if all the Musts and Shoulds have been completed before the end of the fixed time
Won’t Haves: Those PBI that was initially discovered but was not likely to be completed based on the estimates for the more valuable PBI
“The ‘o’s are just for fun”!
Using the MoSCoW rules allows the Business Case to be prepared with 3 scenarios:
The ‘Best Case’ scenario where all the probable PBI are completed; giving the best benefit in the fixed time
The ‘Expected Case’ scenario where all the Musts and Shoulds are completed giving the majority of the expected benefits in the fixed time
The ‘Worst Case’ scenario where only the Musts are delivered giving only the essential expected benefits in the fixed time.
Case Study 1: I was working as a business analyst and ‘just-in-time’ Agile trainer/coach for an outsourced team building a replacement system. The team had 6 full-time business representatives to specify the Agile Requirements (UML Use Cases in this instance).
Creation of the Product Backlog went smoothly; a good ‘To-Be’ Business Process Model was created and Use Cases to be included in the new system were identified; there were 92.
Unfortunately, I was due to go on 3 weeks holiday, so I briefed the team on how to order the Product Backlog and we ran through some Planning Poker with a few of the Use Cases.
When I came back from holiday, I discovered that the team business representatives had had problems in ordering the Product Backlog and had abandoned the attempt because they considered that all the PBI were essential. Also, the whole team had had problems with Planning Poker because they didn’t know enough about the low-level details.
It had been decided to prototype all the Use Cases before estimation and the developers were split into 3 sub-teams for prototyping and the business representatives split into 3 pairs.
I had taught the team that the prototyping of any PBI should only go through 3 cycles:
1. Investigationwhere a very rough ‘first draft’ is specified, built and reviewed to make sure that the developers are ‘on the right lines’
2.Refinement where the majority of the functionality is specified, built and reviewed
3.Consolidation where any ‘tidying-up’ was specified, completed and the complete prototype demonstrated to the wider stakeholder community
If it takes more than the 3 cycles, then something is wrong with the detailed process being followed.
The time taken to produce a prototype should be no more than 2 days but it can be as short as 2 to 3 hours if done in a workshop.
Unfortunately, it was decided to pick Use Cases to prototype based on their perceived ‘easiness’ and ‘rotate’ the business representative pairs around the prototype builders such that one pair specified the Investigation, another pair reviewed the Investigation and specified the Refinement and the third pair reviewed the Refinement and specified the Consolidation.
At the ‘final’ review, with all 6 business representatives present, the ‘final’ prototypes were significantly different from what was originally specified for the Investigation and ‘recriminations’ took place. So the prototype developers started again.
It had taken 3 weeks to develop the prototypes for 6 PBI!
I asked the project manager what his estimate was for completing the prototypes for all the PBI; he calculated that the time that would be necessary was 1 month longer than the specified length of the project.
Suffice to say that I worked with the customer representatives to convince them that it would be improbable to complete all the PBI in the fixed time/fixed cost development contract and helped them order the Product Backlog and decide on an MVP. It is worth noting this product development had a Vision, there were no stated objectives nor a visible Business Case.
I worked with the developers and business representatives (who were full team members) to ‘play’ Planning Poker.
Actual development work started one week later; PBI were completed in the fixed time that equated to approximately 70% of the expected business benefit; a further short project was commissioned to bring the expected business benefit up to 90%.
Lessons: 1. Do not go on holiday at a crucial time during the product development lifecycle!
2. If the Product Backlog is not ordered by business value, significant time will be wasted prototyping PBI of low business value
3. If prototyping is to be used, then the whole team should do it together in a workshop; all views will be accessible to all team members and a consensus reached in the minimal amount of time
4. If you are an Agile Project Manager/Scrum Master it is your responsibility to keep the team ‘on track’ and to question decisions that may cause a risk to the project
Case Study 2: I had been asked to do some Agile coaching for some teams in a global organisation that was undertaking the Agile transformation of one of their divisions to use SAFe®/Scrum; the transformation had been running for 6 months. There were 4 teams distributed around the globe doing support work to enable 24-hour coverage for this global organisation.
The reasons for SAFe®/Scrum for this transition I covered in my first article.
When I arrived Product Backlog requests were arriving daily to the team members from systems users and the individual team member decided whether the requests were important enough to interrupt the work that they were already doing; the Product Owners were not involved in the decision-making process at all. As a consequence, there were many requests that had been started and abandoned in favour of ‘questionable’ higher priorities and I discovered that some requests had been sent to inappropriate team members, the work would have been better carried out by members of the other global teams.
Furthermore, although the teams were doing some Relative Estimating at Sprint Planning (not before), they were focusing on time estimates because that was what the requirements management tool required and what management was tracking progress by. I shall cover the pitfalls of Sprint Planning in my next article.
I started resolving the Product Backlog issues by speaking to all the Product Owners, mostly via video-conferencing, to see if they understood the problems that the current Product Backlog processes were causing. They did have some understanding but explained that that was the way they worked before the Agile transition and believed that changes to the process were out of their control.
I helped them devise a workable process whereby all requests would be sent to the Product Owners at a collective address. They would take turns during a 24-hour period to determine, with Product Managers, the relative priority of new requests against those already on the Backlog. In addition, once a week, they would ‘get together’ to decide which team would handle any unassigned requests; ‘show-stopper’ requests would be handled as a matter of urgency and handled by whichever team was immediately available and could resolve the issue before their end-of-day.
By this process, the Product Owners would gain control of the Product Backlog, there would be less questionable priority decisions and there would be fewer ‘disgruntled’ customers.
The process was approved by the Product Managers and all team members apprised of the reasons for the new process and their responsibilities within it.
It quickly became obvious at daily Scrums that not all team members were passing all requests that came to them directly from customers to the Product Owners; the process was reiterated to these team members but then the productivity of some of these members decreased significantly. The cause was that they were still working on their ‘favourite’ customers requests but not reporting it to the rest of the team.
The Product Owners then had to analyse the central Change Request system for items that had been completed by their teams but had not appeared on the Product Backlog; the Change Request system was not integrated with the Product Backlogs.
Eventually, the ‘lesson’ was learned by ‘miscreant’ team members.
Lessons
1. Without control of the Product Backlog by the relevant authority (Product Owners), it is impossible to police work to give the best business value to the organisation
2. Without control of the Product Backlog, the resolution to some requests may be started and then abandoned in favour of other later requests leaving the customer frustrated
3.Estimating by time at the task level wastes a great deal of time; more of which in my next article.
Conclusion The Product Backlog is arguably the most important artifact for product development:
The PBI must be ordered by business value
The PBI must be estimated, preferably by using Planning Poker
Its content over time must be controlled by the Product Owner or equivalent
Team members must be closely coached and mentored during the early instigation of process change to ensure their understanding of the need for the change and their compliance
Steve Ash
Blog Author
Steve Ash has been working with ‘Agile’ since 1993 when it was known as ‘Managed RAD’. He was an early adopter of the DSDM Framework in 1995 becoming a DSDM Board Member in 2002 and was a DSDM examiner. He is a DSDM Advanced Practitioner and Accredited Trainer/Coach. Steve has since embraced Scrum, Kanban, the techniques advocated by XP, Lean Software Development and Lean Startup. He joined the Agile Alliance in 2002 and is a Certified Scrum Master (CSM), Certified Scrum Product Owner (CSPO), SAFe® Certified Consultant (SPC4) and certified by APMG International to teach Agile Project Management and Agile Business Analysis courses. Since 1996, Steve has trained, mentored and coached hundreds of people in many public and commercial organisations in 11 countries from the USA, through Europe and India to Asia/PAC.
Share This Article
Ready to Master the Skills that Drive Your Career?
