A Process for Agile Transformation

Contents

Lot of Claims, Little Evidence

Silhouette of a person saying something represented by a balloon, and a separate hand puncturing the balloon with a pinDespite the growing number of books and blogs about Agile transformations, there was virtually no objective evidence on how to do them successfully when I wrote a research paper on the topic. As of Spring 2018, only one scientific article reviewing full-enterprise change had been published.[1] And the authors admitted the articles they found were all written by the people who led the projects (and thus were biased); who were “pro-Agile” (again, biased); and mostly reported success stories (providing no comparison to failures, to see what made the difference).

However, there was at least 70 years’ worth of evidence about organizational change (OC) in general and efforts similar to an Agile transformation, like adoption of Total Quality Management. Sadly, around two-thirds of OC efforts fail.[2] Common reasons include lack of top management support, lack of two-way communication, middle manager and worker fears, inertia, and pressures for companies to use the same structures and processes (“isomorphism”). Mistakes by change agents can play a role, too, such as treating all resistance as negative even though it might arise from legitimate concerns. By leaning on the OC research, we can increase the odds of success in Agile. (If you are an Agile coach, consider getting training and certification in organizational change to help understand the complexities and pitfalls.)

The most cited approach is that published by a Harvard professor, John Kotter, in his 1996 book Leading Change. Its eight steps have received scientific support,[3] and align with the smaller set from the Association of Change Management Professionals[4]:

  • “Evaluate Change Impact and Organizational Readiness.”
  • “Formulate the Change Management Strategy.”
  • “Develop the Change Management Plan.”
  • “Execute the Change Management Plan.”
  • “Complete the Change Management Effort.”

Note that “change management” in this sense refers to organizational change, not to managing change within projects.

This section of the site leverages my review of these and nearly 80 other sources related to OC (see the paper[5]), plus my OC certification and experience, to make an “educated best guess” at what creates successful Agile transformations. The result is a set of 11 epics based on the evidence from OC research. They go beyond the steps elsewhere on the site, which are focused on my Full Scale agile™ system, to increase the odds of a successful overall transformation regardless of which Agile system you choose. That is, the epics and their child user stories cover the complete journey from waterfall (or no) workflow management to agility, and not necessarily to FuSca™. (In fact, with minor wording changes, they should apply to any OC effort!) Because this section is system-agnostic, I mostly stick with the conventional understanding of the word “Agile” in it, rather than the FuSca variations.

The FuSca step sets form a subset of what needs to be done if you choose FuSca as your system. If you do, think of those steps as operational changes within your overall transformation project, mostly made via user stories in Epic 10.

Sources of Resistance and Failure

Silhouette of protestorsResistance to change (RTC) is a key factor in OC failures. It can come from any person in or related to your organization, and for a large number of reasons. Two researchers concluded that while “formulating” a change, these sources include:

  • “lack of a creative response,” due to the speed and complexity of the forces for change; resignation to the change; or lack of top management strategy and commitment.
  • “low motivation for change” thanks to the direct or opportunity costs; hiding of problem costs; prior failures; and different levels of interest between managers and employees.
  • “wrong initial perception,” including short-term thinking; denial of the problems; refusal to adapt thinking to current circumstances; and subconscious assumptions.

In the “implementation” phase, they found: “political and cultural deadlocks to change” such as “departmental politics” or a disconnect between organizational and change values; leader fear; “embedded routines”; and lack of skills to make the change.[6]

After their review of the literature, two other scientists reported 20 similar reasons.[7] Organizational factors they found included several leadership issues (lack of trust, management RTC), dysfunction, top-down imposition of the change, resource conflicts, and again, politics. Two factors were specific to the change itself: whether the chosen change was appropriate to the organization, and bad planning of the change.

Those scientists also described personal factors in RTC, including fear of failure, disruptions, extra workload, “Lack of reward,” and, “Perceived loss of control, security, or status.” Fear or anxiety shows up regularly in the RTC literature. “Unresolved anxiety in organizations can lead to a range of dysfunctional behaviors including bullying, depersonalisation, ritualized behavior, techniques for blame shifting or diffusion, approaches that reduce the chance to learn from failures and… resistance to change,” another review said.[8]

Surely it is no coincidence these factors overlap with the reasons identified in the literature for OC failures. Resistance can be both a mechanism for failure and a symptom of problems with the change effort. A significant driver behind the epics and stories in this section is the prevention and reduction of negative (irrational) RTC and utilization of positive (fact-based) RTC.

Transformation as an Agile Project

For example, running the effort with Agile epics and tracking it in an Agile tool will not only provide all the benefits of Agile to the transformation project, but will also create transparency that reduces resistance. You can download the Agile transformation epics and their initial child stories, and either use them in the spreadsheet provided or import them into your Agile tool.

Critically, the epics and stories may be started in any order that makes sense for your organization, and multiple epics will be in-progress after the project is rolling. That is the nature of Agile project management. However, the underlying logic and OC models suggest that in most projects, the epics will start and finish in something close to the order shown. In other words, epics 4 through 7 might all be in progress at the same time, but Epic 4 probably would have started before Epic 6 (if not Epic 5) and would likely end first.

Consider following the basic principles of Scrum to run the transformation project, as specified under Epic 7, treating the “change sponsor” (see Epic 2) as your “customer” and having that person accept all stories and epics. Although this next point violates an important principle of FuSca in most cases, this kind of project is one of the rare exceptions where it is okay for the change leader to fill both the Product Owner and Facilitator (formerly “Scrum Master”) roles. That’s because that person also does most of the tasks, and the Change Team that will be formed in Epic 7 is only an advisory group.

The “change leader” is the person who starts and guides the change effort. Assuming that is you, dear reader, you might pass the title off later, especially if you hire an Agile coach, or if as a top leader you can delegate it. But for now, when you see that term or “you” on this page, it means you! Following the organizational change literature, I refer to people helping to drive the change (regardless of job title) as “change agents.” When no one is specified as the person to do a task, I expect the change leader, or possibly a change agent who volunteers, will do it.

The epics assume you are someone below the executive level who believes your organization should become agile. A CEO already committed to agility can probably skip some steps, though assuming people will fully support the implementation just because you want it is a mistake, as mentioned under Epic 1.

Since advisory groups are made up of people serving on them in addition to their regular teams, I recommend a one-month cycle. It generally takes that long to accomplish significant chunks of work when coordinating with a bunch of part-timers, and you want to limit the number of extra meetings. I have sized the stories to fit that cycle, and use the word “sprint” here with one month in mind. (Those not using Scrum can simply substitute “month” for “sprint.”) If you choose a shorter schedule, you will need to break up stories into smaller chunks, bearing in mind the need to deliver from each story something tangible at the best possible quality given current knowledge.

Read “Choose a Tracker” for suggestions on where to house the backlog and how to track the work. Note that there are free trackers online, and some vendors of paid tools offer free limited-feature access for small teams. You could easily create accounts for yourself and the few others needing editing rights, plus a few guest accounts for people wanting visibility (discussed under Epic 5).

If you are sure the goals of an epic have been met already, you can shorten it to a single-sprint story for communicating the results, or perhaps delete it. For example, if your company is in an obvious crisis that all workers are talking about, Epic 1 (“Define the Why”) may require only a concise statement of that crisis. And if you have chosen FuSca as your system, Epic 8 is done! In general, though, one of the lessons of the OC literature is that skipping steps tends to increase failures.

The pages starting with “Epic 1” linked below will go through the justifications of each epic and child story in detail. Rather than clutter up the downloadable spreadsheet with a lot of text, I put it here and link each story in the spreadsheet back to its relevant page section. You can read through them in sequence using the navigation at the bottom of the pages.

In each of the story descriptions I include a potential list of specific tasks for completing the story. Treat the lists as suggestions, and revise according to your needs. Do create a list, however, and use the tasks as “to-dos.” You are more likely to make forward progress that way, and the satisfaction of marking tasks “Done” can be a fun motivator!

Discipline Brings Success

Agile transformations require complex, disciplined, time-consuming efforts to identify and address multiple barriers in and outside of the organization. There is no shortcut or guaranteed path to success. As I said at the top of the page, these epics comprise a “best educated guess” for Agile change leaders wishing to increase their odds.

Given that two-thirds of OC efforts fail, and repeatable success factors have been identified, it seems likely many problems with Agile transformations are avoidable if we leverage the lessons of OC research. The only choice is between a disciplined, long-term, intensive effort like that described on these Web pages… and failure. You choose.

↑ Agile Management | ← Manufacturing and More | → Epic 1

Full references are in the Agile Transformation bibliography.

[1] Dikert, Paasivaara, & Lassenius 2016.

[2] Smith 2002b. A 2011 review of the 70% figure was correct that its sources presented no empirical evidence (objective experimental data), but missed this empirical source, and presented no such evidence to the contrary (Hughes, Mark, ‘Do 70 Per Cent of All Organizational Change Initiatives Really Fail?’, Journal of Change Management, 11.4 (2011), 451–64 <https://doi.org/10.1080/14697017.2011.630506>).

[3] Appelbaum, et al. 2012.

[4] ACMP 2014.

[5] Available for free from the Social Science Research Network: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3306206.

[6] Pardo del Val & Martínez Fuentes 2003.

[7] Rosenberg & Mosca 2011.

[8] Edwards & Saltman 2007.

Share this page