A Walk Through Agile Hardware: Post 2:
By:Kevin Thompson, Ph.D. – Chief Scientist, cPrime
Scrum Team Definitions in Agile Hardware and Software Environments
The organizational ‘building block’ of our approach to Agile development of hardware and mixed hardware-software products is the Scrum Team. Each Scrum Team plans and executes its development work according to the Scrum framework.
A Scrum Team consists of people fulfilling the three Scrum Roles: Scrum Master, Product Owner, and Team member. The number of Team members must be in the range from three to nine people, which means that any product requiring more than nine people for development thus requires multiple Scrum Teams to do the work. The question then arises as to how best to divide the people involved across an effective set of Scrum Teams.
Structure of Multiple-Teams in an Agile Hardware and Software Environment
Let’s begin by thinking about some fundamental concepts and numeric limits involved in large-Scale Agile development with Scrum.
One common goal in a multi-team world is to avoid sharing one person across two or more teams, or across external organizations. We want each person fully dedicated to the work of a single team, both for the sake of maximum productivity, and to avoid the stress of having one person being pulled in multiple directions. It is not always possible to achieve this goal, but we should approach it as closely as possible.
The above rule does not apply to Scrum Masters and Product Owners. A person who fulfills one of these roles typically can do so for two, or perhaps three, Teams in parallel, but no more. So it is quite common for a particular Scrum Master or Product Owner to fill that role for more than one team. Indeed, this scenario is generally desirable, as it enables an important degree of context-sharing and holistic oversite beyond the bounds of a single team. This kind of oversite is extremely important, and represents the first level of what I call “Agile Program Management.”
I define Agile Program Management as the set of practices and behavior that enable a set of teams to collaborate to build large products effectively. The need is a common one, although the solutions are not as standardized as is Scrum itself. The approach I describe here is based on the RAGE framework I’ve developed at cPrime.
It follows that the lowest level of Agile Program Management is achieved by having a Scrum Master-Product Owner pair working with the same two or three teams concurrently. Once we go beyond the three-team limit, though, we require some additional structure.
At the Program Leve, therefore, we’ll add two more roles: Program Manager and Area Product Owner. These roles do for a set of teams what the Scrum Master and Product Owners do for a single team.
- The Program Manager’s primary responsibility is to “make everything work out” for the set of teams for which this person has responsibility. Much as for Scrum Masters, this role contains a substantial amount of one-off random tasks, but some topics do recur frequently. The biggest specific responsibility of the Program Manager is to ensure that cross-team dependencies are identified, recorded, tracked, and managed. The Program Manager must exercise care to ensure that these dependencies are satisfied successfully; or, if that is not possible, to work with the various teams to identify and implement the best fallback strategies.
- The Area Product Owner is responsible for overall direction of the product and development work. This person does not write Stories for use by teams, but does define the priorities and direction for the overall effort (and may indeed write Epics). This role is also responsible for identifying changes in direction, and ensuring that these changes are communicated to the team level, and acted up effectively.
Neither of these roles acts in a vacuum. Program Managers work closely with Scrum Masters, and Area Product Owners work with the team-level Product Owners, to accomplish their respective responsibilities.
Now that the roles have been defined, we can identify the basic scaling parameters, as shown in the table.
|Scrum Master||Supports up to 3 Teams|
|Team Product Owner||Supports up to 3 Teams|
|Program Manager||Collaborates with up to 5 Scrum Masters|
|Area Product Owner||Collaborates with up to 5 Team Product Owners|
A quick look at the numbers shows that this structure can accommodate more than one hundred team members across as many as 15 teams. Larger organizations would create additional structures of this form.
The details of the Agile Program Management framework and practices cannot easily be summarized here. Instead, I recommend reading the RAGE paper, and taking our Agile Program Management training.
Team-Definition Strategies in an Agile Hardware and Software Environment
With the structure understood, we can now look at the basic question of team formation. We need some guidance for how to define the focus of each team, and how to divide the population into Scrum Teams.
We can start by asking a simple question: What optimization should we pursue when dividing people into teams? I believe the proper optimization is to minimize the cost of collaboration; i.e., minimize the time and effort expended for people to be able to work together.
Consider a Scrum Team containing six Team members. These people collaborate to complete their work. They interact frequently, as needed, throughout the course of each day. This collaboration takes place in a context that is shared and understood by everyone on the Team. The shared context includes the Sprint goals, Sprint plan (Sprint Backlog and end date), and technical underpinnings of the product and their work on it. Precisely because the Team members share so much context, their communications with each other do not need to explain most of the context. Likewise, collaboration can happen mostly in an unstructured, as-needed basis, with relatively little need for scheduling and room reservations.
In short, communication and collaboration are quick and easy, and therefore inexpensive with respect to effort.
Now imagine a second team, working within its own context, and having to collaborate with the first team at times. The two teams do not share nearly as much context with each other that they do within their own team bounds. The teams’ schedules, goals, and area of focus differ. Dependencies that cross the team’s mutual interface require planning on both sides, and slippage on a predecessor Story will likely require re-planning on both sides.
In short, communication and collaboration across the interface between teams is substantially more expensive that within teams.
This logic leads to the conclusion that we should optimize team definitions to minimize the frequency with which cross-team interactions must occur.
One implication of this concept is that each team should have a specific focus, and work on deliverables relevant for that focus. While the idea that “any team can do anything” is attractive from a resource-management perspective, it is almost always impractical. If that idea is impractical, then we should define a focus for each team in a way that clarifies which type of work it should be asked to do, and within which the members can develop deep expertise.
I use three different patterns for team definition, depending on the nature of the organization’s work.
Feature Teams in an Agile Hardware and Software Environment
The Feature-Team concept is the simplest strategy for team organization, and has long been popular in the field of software development. We consider the product we are building, and divide the overall functionality of the product into a number of functional areas.
For example, we might have twenty developers and QA personnel, and choose to divide this set of people into three Scrum Teams with seven, seven, and six Team members. Associated with these three teams would likely be one or two Scrum Masters and Product Owners.
Let’s say we divide the product’s functionality into three areas, to match the three teams. These areas might be a Billing module, a Catalog module, and a Subscription module, as we expect roughly comparable workloads in these three areas. We then have a Billing Team, a Catalog Team, and a Subscription Team. Each Team has a cross-functional membership, meaning that different people on each Team may have different skills, and each Team contains the right mix of skills to develop all product capabilities in its area of focus.
What we do not want to do align teams with skill areas. For example, we might require skills in Java development, Database development, and Quality Assurance. If we choose these technical skills as each team’s focus, then adding a new Billing capability would require collaboration across three teams. In fact, most capabilities we add to the product would require multiple skills, and therefore cross-team collaboration. The cost of such an intense pattern of cross-team interaction would then be very high. The overhead of managing the cross-team dependencies would be large, the teams would move slowly, and the team members would likely feel frustrated and be under constant pressure to perform.
In contrast, if the Billing Team contains all of the skills needed to add a capability to the Billing module, then the only cross-Team interactions are those that reflect a need to do synchronized work across modules. This scenario is much less common in a Feature-Team context, and therefore represents a much lower cost of interaction than does the skill-oriented team definition.
The Feature-Team approach to team definition is the easiest to manage, and generates the least amount of program-management overhead, of the three approaches. Why, then, would we do anything else? Because there are situations in which the Feature-Team model does not work well.
Client-Server Teams in an Agile Hardware and Software Environment
Now consider the situation where we have multiple client applications (e.g., Android, iOS, Web) and a single back-end server process.
We could use the same Feature-Team approach as before, and perhaps define Billing, Catalog, and Subscription Teams. In that case, each Team would require people who could develop capabilities for all three client environments. This is not such a bad situation if a single person can do all of this user-interface support, but it becomes problematic if we need multiple people. In the latter case, we might end up with three client-support engineers on each of the three Scrum Teams. It is likely that each of these people would be under-utilized, and our salary costs would be too high for the value we are getting.
The Client-Server Team model allocates one Scrum Team for each client, and one (or more) for the back-end server process. This model is often, if not always, less expensive and easier to manage than the Feature Team approach.
Compared to the Feature-Team model, the cost of managing cross-Team dependencies is higher in the Client-Server model, and the burden on Program Management is therefore larger. This drawback must be weighed against the drawback of under-utilized resources when making the Team definition decision.
I recommend considering both models when developing functionality for multi-client solutions, and selecting the one with the least-serious drawbacks.
Component Teams in an Agile Hardware and Software Environment
The Component-Team model is generally discouraged in the context of software development, and for good reason: It imposes the largest burden on Program Management of all three models, as the effort to maintain synchronization of work across teams, and manage the cross-team dependencies, is substantial. That said, the Component-Team model does have one great advantage: If you can handle the Program Management issues, the model can always be applied.
The sweet spot for Component Teams is for organizations that develop components of solutions, rather than products. A solution then consists of an appropriate set of components, set up and configured to inter-operate correctly, to fulfill the customer’s needs.
One obvious category is that of telecommunications solutions. I might purchase a solution that enables a fire department, police department, or (at much larger scales) an army in the field to provide communications capability between any two individuals, to broadcast to a subset of the people, to provide an open channel for any group, and so forth. These solutions are hugely complex, and contain a large number of components:
- Handheld radios
- Portable satellite communication stations
- Wireless routers
- Administration stations
- Relay stations
- Centralized communication hubs
Each of these components contains both hardware and software, and communicates with other devices and control systems. Many of the components have no human users as such, and only interact with other components. The set-up, configuration, and management of such a system can require a great deal of work.
Other examples might include networked patient-monitoring systems in hospitals, cell phone networks, in-office networks, remotely-operated surgical robots, and so forth.
In component-dominated systems, Component Teams make a great deal of sense. For each category of device (say, a network router), we define one or more Scrum Teams (such as the Network Router Team). Each Team or set of Teams is responsible for the development of components in a specific category, including hardware and embedded software. (Note that monitoring or administration software, which are pure software products, also qualify as components in this model.)
A component-dominated world requires a clear overall architecture and robust interface standards. Each component has to be able to ‘trust’ another component to respond appropriately to their shared interface, as it is not possible for any one component to ‘know’ the internals of all of the others.
Concluding Thoughts or Agile Hardware and Software Environments
Perfect solutions for organizing people into Scrum Teams are rare. In most cases, multiple solutions are possible, and all have drawbacks. We normally end up choosing the one with the least serious drawbacks, and managing the latter as needed. If you can find a Team Definition solution that works well for 80% of the likely scenarios, consider yourself fortunate.
In the world of hardware products, or hardware-software products, I believe the Component Team model will be the most common. Program management, architecture, and interface definition become critical factors in success, and must be managed carefully. For pure software products, the Feature Team and Client-Server Teams suffice most of the time, although the Component-Team model may be appropriate for sufficiently complex software environments as well.
Provide all of the above links here, and an