Contents
Teams in Charge
Extreme Self-Organizing
The work that introduced the term Scrum to business, “The New New Product Development Game,” says that in development efforts it covered, “The project team begins to operate like a start-up company—it takes initiatives and risks, and develops an independent agenda. At some point, the team begins to create its own concept.” The article likens top managers to venture capitalists, quoting one as saying, “‘We open up our purse but keep our mouth closed.’”[1]
At some point when converting a group to Full Scale agile, I will surprise them by saying I do not consider myself an Agile expert. Yes, by most standards, I am one. My point is that I came to agility through my earlier and ongoing interest in leaderless teams. These “self-directed” or “self-managing” teams have consistently been shown to outperform standard business teams when properly set up. Ironically, though, it doesn’t work to just say, “Poof, you’re a self-directed team. Go outperform!” A framework of best practices like this site’s “Self-Directed agile” chapter helps the team pull this off efficiently. After that the team can make whatever changes it wants to improve performance.
Because of that background, seeing these two Manifesto principles had the biggest impact in converting me to Agile:
- “Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.”
- “The best architectures, requirements, and designs emerge from self-organizing teams.”
To my mind, employee empowerment was the only concept important enough to the founders to rate two principles! Combined with some other principles, I thought that meant Agile practitioners saw the value of what I was doing in a way many traditionally run companies did not. Most do not, I came to realize, as described in “Good Intentions Turned Bad.”
“Nature abhors a vacuum,” so when someone is hired for a full-time gig that is not full-time work, they find ways to justify being full-time… and thus become a de facto team leader. When I told a newly formed team of subcontracted programmers on Long Island they would choose their own part-time SM, they got visibly excited. One said, “That’s a real difference, because every Scrum Master I have worked with has acted like they were the team leader.” Everyone nodded their heads. A team I was working with in India two weeks earlier chose to rotate the position, based on the strong opinion of one member who said otherwise that person ends up getting treated as the boss. I found their histories with Agile depressing.
Self-organizing means the team runs itself as a group. The organization has an obligation to provide clear requirements. But no one, including the PO and Facilitator, tells Scrum team members how or when to deliver those requirements. Instead, the members including the PO and Facilitator decide together and tell the organization how and when, on an iterative basis.
The Power of Stability
Agile advocates generally recommend the creation of teams with stable membership. An in-depth study of Agile team success factors in six Brazilian companies using structured interviews, team document reviews, and direct observation for six months found lower “staff turnover emerged as critical for team productivity.” Although Agile processes like those in this site do much to reduce the negative impact, the researchers found it “surprising that, even adopting such practices, the teams were visibly affected by medium turnover.”[2]
Three professors analyzed data from more than 300 projects over 20 months in one Indian company. This company re-formed teams for each project, and tracked a lot of relevant data like who had worked with whom in the past; success metrics of each project; and various factors that could impact performance. How often project team members had worked together was more strongly linked to team performance than the members’ job experience. What scientists call “team familiarity” was linked to fewer code defects and greater likelihood the project was completed within the labor estimates, or labor and schedule estimates combined. Higher familiarity related to fewer problems in almost every area. So the best predictor of success on most measures was how well people knew each other, not their years on the job or other factors.[3]
Imagine how much more powerful that impact would have been if the company did not break apart the teams! The sad thing is, this information should not be news to anyone with business experience. Check out this quote: “Service troops are trained in working units and will prove most efficient when the unit is kept intact. Men of each squad, for example, usually work best under their own leader, each man knowing from past experience his particular assignment in the group.” This comes from a U.S. Army manual published in 1945![4]
According to an analysis of data from more than 10,000 teams that used Rally as reported at the Agile 2014 Conference, “Stable teams resulted in up to:
- “60% better Productivity.”
- “40% better Predictability.”
Also, “Teams made up of people who only work on that one team have double the Productivity.”[5]
Studies have confirmed each member learns a lot from “past experience” with the group that benefits the group, and takes months or years to re-create in a new team. As I wrote for one client, “In group psychology terms, this is because the following traits of high-performance teams become possible or are enhanced:
- Team cohesion—The sense of identity with a group of people to whom members feel mutually accountable.
- Team potency—Members’ belief in the general effectiveness of the team.
- Team efficacy—Members’ belief in the ability to achieve a specific goal.
- Shared mental models—A common understanding of how work is accomplished.
- Team memory—Understanding of who knows what on a team, and how things worked or didn’t in the past.
- Familiarity—Understanding of each other’s skills, communication styles, etc.
- Trust—Belief that fellow team members will do what they say they will do.”[6]
Don’t move people to the work, most Agile coaches say; move the work to the team. A major objection raised is that the old team may not have the needed skill sets. First I note that the same was true for every waterfall project team I ever managed—they always needed help from some other team, often ones not formally part of the project.
The answer is to let the team figure out how to handle the work for which it does not have the skills. Options I have seen succeed include cross-training with internal experts; formal training from external experts; getting help from other teams; or in the worst case, hiring outside expertise in the form of contract workers. For all these reasons, the creation of stable, cross-functional teams is listed as one of the desired agreements for implementing FuSca.
Full-Stack Capability
Agile software development teams “usually include members with a broad skill set, such as analysis, programming, design, database architecture and administration, systems engineering, and project management,” wrote three information systems professors. “In fact, agile teams share the premise that everyone who the team needs to fill all the roles necessary to complete the project must participate in the project team. This situation differs from the traditional waterfall/structured environments, where teams often specialize according to function (e.g., analysis, design, development, testing).”[7] The emphasis on cross-functionality also is consistent with studies showing diversity of work background and tenure within an industry improve team outcomes.
In software these are referred to as “full stack” teams, a software engineering term I will redefine as “containing all skill sets necessary for delivering customer-usable functionality, if not entire products or services.” For an example, consider a company I worked with that developed and manufactured utility meters, those boxes on the back of your house and office that measure how much electricity is used. Modern versions are usually “smart” meters capable of talking wirelessly to the utility, meaning they have mini-computers and cell phones inside in addition to the measurement functions. One of the teams there was rapidly developing the company’s next-generation meter. The team included hardware, radio frequency, firmware, functional software, interface software, and test engineers. That’s about as “full-stack” as it gets! (This term is why the first version of this website was called “Full Stack Scrum™.”)
The single easiest way to eliminate interface conflicts between functions is to eliminate the interfaces. People writing requirements cannot gripe about “the development teams” not delivering what they asked for, because they are on those teams. Developers cannot “toss over the wall” to testers problematic code, because there is no wall. The testers are on the team. The same is true for each function represented. Another advantage is that the impact of each function’s decisions on other functions is understood before they are implemented, a cause of many problems when component teams are used.
Workload balancing and resource planning becomes easier in this setup. Phases in typical waterfall projects tend to use certain skill sets more than others. When people are not available to take one phase’s output, the project is slowed; when the people are available but the output isn’t, the project is slowed and the organization loses money as people are paid for doing nothing. In Agile generally and FuSca specifically, after a relatively short planning cycle to initiate the project, everyone stays fully engaged to the end.
A curious side effect was revealed in the data mentioned in the last section based on thousands of teams. “Interestingly,” the presentation[8] said, “teams that self-identify as having no testers have:
- “The best Productivity.”
- “Almost as good Quality.”
That didn’t mean they did not test. It meant the developers did testing, and testers did developing, and members tested each other’s work, with very positive results.
FuSca allows exceptions to this full-stack concept. For instance, the typical ratio of technical writers to developers means you have to choose between making the writers their own team or splitting each member’s time across teams. In other words, you have to choose between violating stability or violating full-stack capability. From the standpoint of group psychology, err on the side of stability by letting highly specialized functions whose members each support many other teams stand alone in specialist teams. This should obviously be the rare exception, though.
Direct Daily Communication
Teamwork studies have consistently shown that “collocated” teams—those able to meet together physically every workday—outperform otherwise similar teams spread across different locations. In the study in Brazil cited earlier, “projects reported significant productivity gains through improvements in communication, teamwork spirit (cohesion), planning, and requirements negotiation when collocated.”[9] There are several reasons for this. The value of face-to-face communication in terms of nonverbals and speed was discussed under “The Principles of Agile.” Self-organizing requires a high quantity, not just quality, of contact among members as a group and as individuals. Trust cannot be “created,” despite the claim of many teambuilders—it develops organically through repeated positive interactions with another person.
Fortunately, members need not be in the same room, which may make collocation easier for some organizations. In fact, for maximum productivity and job satisfaction, they should not be in “open workspaces” despite the popularity of the concept. See “Does the team need a team room?” for the scientific evidence against it. Nor do they need to be together all the time. Given the documented benefits of flexible work hours, the best option is to maintain core hours each week when everyone is together, but allow people to work where and when they want the rest of the time. (Having advised this approach since the early 2000s, I noted with satisfaction and frustration that this was a consistent recommendation of workplace experts for reopening offices after the covid pandemic.)
If not in the same building, they should sit in the same time zone. That Rally big-data analysis reported: “Teams distributed within the same time zone have up to 25% better productivity.”[10] Certainly they must be close enough to meet regularly as a group, within three time zones of each other at worst. And there is significant evidence it pays off in productivity and reduced conflict to invest in bringing virtual teams together physically at least once a year.
In other words, from the social psychology perspective, there is no such thing as a “global team.” Though I found many proposed ways to overcome the problems of “temporal distance” in the scientific literature on Agile, not a single study provided evidence a solution had been found. Technology cannot fix the “problems” of people needing to sleep at night, and night occurring at different times around the world. Because of that, in widely separated teams, some members are going to miss meetings regularly and/or have less job satisfaction because meetings are held outside normal working hours in their time zone. I refuse, based on the science, to a call a work unit that does not have the capability of regular, easily scheduled, all-member meetings a “team.” It is just a “work group” of individuals, each creating their own output.
Unlike many in the Agile Industrial Complex, I am not going to tell managers in global companies what they want to hear just to sell them on my system. FuSca provides means for teams around the globe to work on the same program, but each of those teams must be composed of members close enough in time zone to practice self-organization through participation in all of the prescribed Scrum ceremonies. Otherwise, you are not “being Agile.”
Agile Architecture
Architecture as a Team Effort
In a sense, architecture in hardware or software design fills the same role it does in buildings, or in my early career of crafting scenery for plays: It provides the blueprint for how the parts of the deliverable fit together to create the whole. In waterfall, this is supposed to be fully specified before any development teams are formed, so those teams don’t waste labor time figuring out the design or having to change direction radically.
Speaking, I remind you, as a certified project manager, I am confident in saying system architecture design has often been a primary cause of delays in waterfall projects. Usually one or more issues like these occur:
- The architectural specification takes longer than expected, such that development teams are formed but cannot start serious development.
- Contributing to the above, requirements keep changing after the specification is fairly mature, triggering delays, wasted development time, and/or significant changes in direction.
- The architecture does not fit the needs of the teams, but architects resist changing it—usually for understandable reasons beyond their control, like they have moved on to the next project, or the Change Management System is burdensome.
- Better ways of doing things discovered by the teams are disallowed merely because they do not fit the blueprints.
Recall this Agile Manifesto principle: “The best architectures, requirements, and designs emerge from self-organizing teams.” This anticipates that architecture will be a joint responsibility of the system architects and teams.
By extension, it will also be developed according to Agile principles, for the same reasons the code or product is:
- The architect will be a full-time member of the program, working closely with the developers throughout each project.
- The architecture will be co-developed iteratively throughout the program by the teams and architect.
- The program team as a whole will have final say over its designs.
- Little time will be wasted on pilots or “proof of concept” deliverables, instead of which the teams will:
- Try one approach to the architecture.
- Keep what works.
- Scrap what doesn’t and try another approach for that.
Direction without Specification
The expertise of a talented architect prevents teams from wandering long in the wilderness, and reduces conflicts over approach that can delay the product. A certain amount of upfront planning regarding broad technology decisions, design patterns, and components or services is valuable in any project, as pointed out by senior architect James Madison in IEEE Software. “Although much of this is similar to the activities of a nonagile approach, up-front architectural work with agile development includes a subtle but important difference,” he wrote. “The architectural direction should include a range of options rather than a specific solution. A looser set of architectural possibilities is acceptable based on the agile assumption that the empirical knowledge gathered by all participants while building the system will make better options more evident.”[11]
So under Full Scale agile, we do ask for an initial plan to get everyone pulling in the same general direction. I have seen this called an “architectural roadmap” by some experts on agile architecture, but I prefer the term “architectural runway.” Like that for an airplane, a runway provides an initial direction as the plane/project gets up to speed—yet only that much. Once the project is flying, it can turn in whatever direction it needs to go.
Like a airplane’s pilot or navigator, we do keep re-plotting the direction in the form of updated architecture. However, in agile we only do enough to keep the teams going. Those updates are part of planning the next release or sprint, since our direction may have changed by the release/sprint after that. I call this “just enough, just in time” documentation. Per the Manifesto statement downplaying “comprehensive documentation,” this does not take the form of a formal specification document. Diagrams and bullet lists are the majority of the content in agile architecture.
The runway needed to start a project can be as minimal as the top level and one level of decomposition. Question marks are not only acceptable; they are expected. The teams and architect will answer those questions together. I do not recommend going past the second level of decomposition before development for the same reason we do not groom user stories more than one iteration at a time: Change is inevitable.
However, the airplane you are flying has to be chosen before you take off! I refer here to the hardware and software to be used in the development work, the “stack.” Madison’s warning is on point: “Plan carefully; hardware and software changes are inherently nonagile.” In both waterfall and agile projects, I would expect a good architect to consult with the developers on making those decisions, though in established companies most of these are “pre-made” in the sense that you are going to re-use existing resources. Regardless, include these decisions in the runway documentation.
Hardware’s Longer Runway
An airliner needs a longer runway than a single-engine propeller plane, and the same is true for projects to build an airliner. Much more of a hardware product must be designed before you spend money on a tangible prototype. Even plastic mockups can be expensive, despite 3D printing. Often, too, a company wants a preliminary bill of materials (BOM) specifying likely parts and their costs before approving a hardware project.
The same agile architecture principles apply, however. The iterations just take longer than they would in software projects:
- Write requirements and stories for specific parts of the architecture, creating it iteratively.
- Conduct reviews with the engineers who will use the architecture documentation each sprint, or at least engineers with similar skill sets.
Note: Those folks can still have most of their time dedicated to other projects, but treat them like a “customer focus group” to increases the likelihood of meeting the chosen group’s needs. - Incorporate their feedback into your next iteration.
- Start the project team as soon as those folks say the architecture is mature enough, instead of waiting for it to be “finished.”
- Co-develop the design from that point forward.
This approach, like the rest of this “System” chapter, prescribes a huge amount of culture change for many organizations. Making these changes will cost a lot of money in the form of people’s time, not to mention frustration or consternation from many. No valid business decision is made solely on the basis of costs, however. We must compare those to the potential benefits as we “Decide to Change.”
Benefits of Agile and Scrum
The Business Need to Meet Expectations
As mentioned, in the Planning Ceremony on Day 1 of the sprint, you will create a list of stories you expect to complete in the sprint. In FuSca, a Sprint Plan is not a wish list of the stories you hope to accomplish over the next few weeks. It is a promise. The verbal contract you make with your customer(s) is this: “If you agree to wait until the end of the sprint to change our workload, we agree to deliver that workload almost every sprint.” My training for Agile certification called this “commitment-driven iteration planning.” In marketing terms, I would call it “under-promise and over-deliver,” a simple principle for making your customers happy.
Customers and internal stakeholders want to know what is coming in the near term because there are things they need to do to prepare. In a retail store, it takes time to update the cash register and inventory systems, set aside space on the shelves, train sales associates, and prepare announcements of the new product. When a vendor gives the store a date for an item, and doesn’t deliver that exact item on time, you can cause them financial harm, not to mention embarrassment.
The same is true if you deliver that item and the customer finds something wrong with it. That retail store starts getting returns from angry customers. Not only does the store incur the costs of returning those goods, its reputation is harmed. The teenager who bought a shirt that came apart at the seam on the first wearing doesn’t care that the clothing store didn’t make the shirt. They are going to be mad at the store. After repeated instances of poor quality, people will stop going to that store. In turn, the store is going to stop buying your goods.
With a little imagination, you can create an allegory between a retail store and any other company, for example by thinking of the product manager as the store manager and his customers as the shoppers visiting the store.
All this may seem a strange analogy when you hear that FuSca never promises a specific delivery for a longer period than a sprint, for the simple reasons that humans cannot predict the future. It has achieved four-month predictability as high as 90%. More importantly, the method refuses to lie to the customer about what it can predict. A combination of higher transparency and higher customer involvement ensures the “store” and its customers aren’t taken by surprise by either a delivery date or poor quality.
Project management is an organized attempt to help customers by meeting clearly defined “customer expectations” about what the customer will get, by when, in a customer-pleasing condition. The problem with waterfall is it doesn’t easily update those expectations. Full Scale agile guarantees you will meet or beat those expectations most of the time, even when the delivery date is a few months away, by proactively managing the expectations.
Proven Benefits for (Almost) All
There is plenty of hard evidence on the Web and in books proving the measurable value of Agile to companies and most employees, some of which I have cited already. These include significant bottom-line financial benefits. I won’t bother repeating them here, since you can easily look them up—and probably already have, since you are (still) reading this site!
In general, agility and Scrum capture the reality of research and development work better than waterfall. I have personally witnessed a number of benefits from Scrum since my introduction to it at a small Agile startup in 2008. Specifics include:
- Quicker delivery of value to the customer—Instead of having to wait months to see some return for their money, the customer can see requirements move from discussion to reality in a matter of weeks.
- The advantages of self-management—As noted earlier, a huge amount of research going back decades shows that teams with no daily leadership outperform similar managed teams, producing more and better work at lower costs and stress. FuSca efficiently maximizes self-organization.
- A team structure—That same body of research proves that teams taking time to structure themselves by defining roles clearly, identifying team rules and procedures, documenting processes, and planning their work are more effective than those who “wing it.” Scrum gets the team much of the way to an effective structure in a few weeks.
- Better decision-making—Also clear from those studies is that cross-functional teams make better decisions on average over time than a single expert on the given topic. Agile hinges on this kind of decision-making while providing boundaries to prevent it from bogging down.
- Acceptance of change—People naturally resist changes in direction to their work. Agile addresses this by time-boxing change into small increments, balancing the financial and emotional costs of frequent changes against the reality that requirements do change. It also shows benefits to the individual relatively quickly compared to other, less systematic approaches to process change, reducing ongoing resistance.
- Higher mid-term predictability—Most traditional projects fail to meet their schedule, budget, scope and/or quality commitments. By focusing on consistent rather than maximum delivery, FuSca teams arrive at a point where they can promise delivery of all or most stories every sprint, and thus more accurately forecast deliveries over several sprints than a waterfall plan done months earlier.
- Reduced team stress—Because workload commitments are based on historical data, teams stop promising to do more than they realistically can. They also get hard data to support limiting those commitments, or to prove the need for cross-training and/or increasing the number of people on the team. This reduces chronic overtime with its well-documented personal and financial costs, and eases pressure to do more without more resources.
- Reduced manager stress—Managers reduce their stress, too, because they get better visibility into team progress yet no longer have to manage the team on a daily basis. Plus, their short-term forecasts to customers and other stakeholders become far more accurate and they, too, get hard data to support those forecasts.
- Better value—Each story is developed to the Customer’s acceptance criteria, ensuring it meets their needs.
- Better quality—Since quality is emphasized at each step and complete testing is part of the delivery, the chance of “escaped bugs” getting through to the customer or end users is greatly reduced. This in turn increases customer satisfaction and lowers rework costs.
- Higher job satisfaction—Drawing on research about the factors that increase job satisfaction going back to the 1980s, three scientists surveyed software developers and found the use of Agile project management practices was strongly correlated with job satisfaction, primarily due to the sense of job autonomy and regular feedback on performance.[12] A scientific survey of more than 1,000 people within one large corporation after it converted to Agile found that only 10% would go back to waterfall given the option.[13]
This graphic summarizes the benefits for most people from their organization’s switch to Agile:
One financial benefit to the company the Agile Industrial Complex tries to hide is that fewer manager roles are needed, hence the “almost” in the section title. This has always been a barrier to the adoption of self-organizing teams, which is why most consultants won’t discuss it openly. I will shortly, including ways the people in those roles may still be able to add value.
The Right Thing to Do
I may make some of you nervous with my final “benefit,” but bear with me. I am not injecting religious beliefs into this discussion. Rather, I would like to raise a sociological argument. Who said this?:
Do not do to others what you would not have them do to you.
People raised in Christianity or Islam might believe Jesus was the originator of this saying. But notice the word “not” that appears twice. That word does not appear in the quotation in The Bible. The Chinese philosopher Confucius wrote that line around 600 years before Jesus was born.[14] However, another version of it appears in writings dating much further back. Sometime after Confucius quoted it in his Analects, it appeared in a Hindu scripture in India. In 70 B.C.E, a Jewish writer says it. Then Jesus.
The fact this line that some call the “Golden Rule” shows up in so many different cultures suggests it is a rule every human will agree on, even if we debate the specifics of how to apply it. When I was young I embarked on a study of the world’s religions and philosophies, and I constructed a list of behavioral guidelines that were common to every one of them. I think all of those can be tied back to the “Golden Rule.”
One can quibble with the rule—some troubled people hurt themselves, so the rule when strictly applied by them would mean they could hurt others. Certainly people apply the rule differently, because we disagree about what is right and wrong to do to ourselves and others. Yet I think when we get upset with someone, the root cause is our belief that the other person did something violating the Golden Rule. “I would not do that to them,” we feel, “so they should not have done that to me.”
Think about something physical you buy online or over the phone that is important to you: electronics; clothes; in my case, sailboat parts. Whenever you order that thing, you have to choose a shipping method. Those methods always come with a range, like “5 to 7 business days.” Unless you’re rich or a shopaholic, you debate among several different items you can get with the limited amount of money you are able to spend right now. You make your choice, and get excited that within a week, it will show up.
If you have been paying attention over the years, you have noticed that most of the time, it shows up toward the front end of that range. Almost everything I expect to receive within “5 to 7 business days” shows up in four or five. Now what happens if you get to Day 8, and the thing hasn’t shown up yet?
You get upset, right? Even if you are a patient person, you notice the problem, and if the delay continues, you eventually are on the phone demanding to know where your item is. When people give you a date by which you can expect something, you get upset if it doesn’t show up by that date.
Whether on time or not, let’s say you eagerly open the box and find… something other than what you ordered. Or it’s the wrong size. Or it’s broken. Or it doesn’t work the way the Web site said it would. You get upset, right?
You know where I am going with this. With every Sprint Plan, you tell your stakeholders that you are going to deliver these five or 10 items within 10 or 15 or 20 business days, and here’s how each will work, and they will work correctly. When you don’t deliver 100% of those items, and any of them have problems (defects) that the customer notices, you are treating the customer the way you don’t want to be treated when you are a customer. You are violating an arguably universal rule of fair treatment.
↑ Agile 101 |← Scrum Overview | → Self-Directed agile
Full references are in the Agility Bibliography.
[1] Takeuchi & Nonaka 1986.
[2] Melo, Cruzes, Kon & Conradi 2013.
[3] Huckman, Staats & Upton 2009.
[4] FM10-22, War Department Field Manual, October 1945, titled, “Quartermaster Depot Company, Supply.”
[5] Maccherone 2014.
[6] Morgan 2016.
[7] Tripp, Riemenschneider & Thatcher 2016.
[8] Maccherone 2014.
[9] Melo, Cruzes, Kon & Conradi 2013.
[10] Maccherone 2014.
[11] Madison 2010.
[12] Tripp, Riemenschneider & Thatcher 2016.
[13] Laanti, Salo & Abrahamsson 2011; another 30% saw no difference, leaving 60% who did see a difference and would not go back to waterfall.
[14] Waley 1938.