Winning with Scrum Under a Phase-Based Program

Which one of these guys is agile?  Which one is waterfall?

OK, that's not what this is about.

Scrum teams may become connected to a program delivering a new suite of products.  The program and its other projects may follow a different delivery method than the Scrum teams, and it is up to the people in both the program and teams to inspect and adapt their models—bridging the impedance mismatch brought by incongruent methods.

Program and architecture governance may require specific steps, approvals, and engagement with specific partners.  These program requirements may not lock in step with the project start and development cycle of the Scrum team.  The reverse is the same—the Scrum team cadence may not match the start and end of a phased based project.

While Agile and Waterfall or phase-based methods need to coexist (just the state of the times in your organization, for example), we need to make the best of it.  Both processes can operate fine independently, but the Agile team has an interest in making it work where the two methods meet.

Agile teams need to ride the Waterfall to make it work

One of the top reasons for failed Agile projects 

is external pressure to follow a traditional waterfall process

PROCESS MISMATCH
The mismatch starts with the differences in process:  When waterfall has completed planning, Scrum is setting up a development environment.  When waterfall is gathering requirements, Scrum is testing working software.  When waterfall has completed requirements and architecture reviews, Scrum is adapting the plan and getting more requirements.  A waterfall, or phase-based program's specific steps, approvals, and engagement requirements with specific partners may not match the Scrum team's cadence and practices.

Waterfall and Scrum have a process mismatch

INTEGRATING THE SCRUM TEAM, STAKEHOLDERS, AND PARTNERS

Mixed in with these planning and delivery mismatches, partners and the people themselves are potentially mismatched based on mindset, experience, and motivation.  Phase-based team members are possibly not comfortable with limited information.  Scrum team members want to get started with little up front information so that they can get feedback.  Mismatches involving people’s understanding of the process, techniques, values, and principles should be assessed prior to the project start and resolved throughout the project by continuous coaching (at the team level and then at the individual level as well).

Some specific challenges to resolve through coaching are:  The change in roles and responsibilities; making inappropriate comparisons; hesitancy about or hindering delivery by blocking, or building in waste to a Scrum team's processes or practices; controlling team commitments or focusing on lower priorities; misunderstanding the value of an informative workspace (wanting everything electronic); locking down scope, limiting scope flexibility, using change control; incorrectly measuring progress; wanting a Scrum team to use phase-based methods (estimates, requirements gathering, etc.); unaligned business and customer partners, and Scrum team communication and reporting with program stakeholders.

Before starting a project, train, coach, and involve people.  Ensure the delivery team and stakeholders are brought along and value the Agile mindset.

The delivery team and stakeholders should train before the project

INTEGRATING THE SCRUM TEAM AND ARCHITECTURE FUNCTION

Another mismatch occurs while planning the first release of the project.  The Scrum team may recommend a feature-driven development approach.  Architects may recommend a component-driven approach.  The feature-driven approach uses all layers (possibly tiers) of the architecture and produces value to the customer in the form of a potential production-ready feature.  The component-driven approach shows the architects progress on the architecture up front.

Specific architecture domains are also put in the same situation.  In large architecture functions and programs, each domain may have its own governance.  Business, application, information, integration, infrastructure, and security architecture domains may require up front, approved solutions before functional development begins.  These architecture domains may raise issues or identify risks to the program if as User Stories are placed into a Sprint before, for example, the data model is finalized and approved.  Architects or technical Scrum team members may try to encourage the Product Owner to place component-driven backlog items into the Sprint backlog, or to put documentation-driven work ahead of working software (valuing full business context, or non-functional requirements documentation over a risk adjusted backlog of User Stories).

A solution to this mismatch is to ensure the architecture is defined using the correct process for the product, project, and people involved.  Typically, candidate architectures are crafted from the architecturally significant requirements.  Any development framework or methodology can determine the architecturally significant requirements.  As those requirements are identified, the mismatch can be solved.  In this example, the Scrum team can map User Stories to architectural significances and progress of the architecture can be measured against completion of those User Stories.  Since Product Owners consider architecture significance and risk in prioritizing, these User Stories should be completed early in the project.  The Scrum team will follow and implement architecture guidelines and standards during feature development and update necessary documentation and repositories incrementally. 

To support this solution, an embedded solution or delegate architect can perform periodic or ongoing reviews of the architecture to ensure the visibility of architecturally significant decisions and leveragable work.

The product can be driven by features, while prioritizing the architecture components for those features

INTEGRATING PROJECT AND PROGRAM MANAGEMENT WITH AGILE DELIVERY

1.  Charter and Funding

A first mismatch occurs during the program’s project charter and funding.  The Scrum team may recommend a small funding of the determined team for a small number of sprints in order to gain a better understanding of the project, and produce some working software from which immediate feedback can be gained from the customer.  The program and architects may recommend a full requirements and architecture phase to determine the complete scope up front, and then request approval and funding for the entire project.

A solution to this mismatch is to sell the Scrum team’s recommendation to the program and architects.  Having successful delivery in the past will help the Scrum team with the sell, but if a new Scrum team is assembled, the arguments for assembling the Scrum team can be directly used to the program for a small initial funding of the project.

2.  Release Approval & Estimates

Another mismatch also occurs while planning the first release of the project.  The funding and approval process of the program may take significant time relative to the Sprint cadence of the Scrum team.  Two or more Sprints can go by while the next release is socialized with the program governance.  In the first release, the governance may need to review the Scrum team’s velocity in order to gauge completion of later releases.  However, if the Scrum team has not been provided sufficient time to stabilize a velocity, the approval for the next release may not occur based on accurate information. 

A solution to this mismatch is to allow appropriate time to gain accurate information.  If the first release is only four sprints, the approval for the following release may occur after two sprints—not sufficient time for a stabilized velocity.  The Scrum team may recommend a longer first release or a shortened approval cycle.  An alternate solution is to complete the full release (all sprints) and hold the team while the approval is gained for a next release.

Stabilize velocity, backlog, and architecture before seeking long-term approvals 

Estimating with Scrum typically uses story points or ideal days at times when the entire architecture and design of the product is unknown.  Program stakeholders may find a higher level of comfort with the phase-based approach of defining the entire scope up front, and estimating that entire scope based on components.

A solution to this mismatch is to coach the stakeholders on the problems with estimating an entire scope and how using points or ideal days allows the team to start working quickly on refining the estimate by producing and basing measurements on the construction of a working product.  Allow stakeholders to compare the Scrum team’s estimate with any estimating model they can produce in parallel to show how the estimate of the Scrum team is within a similar magnitude.

During the project, a dependent team or customer may ask the Scrum team for feature estimates in financial terms.  A point or ideal day estimate would not satisfy the request.  A solution to this estimate mismatch is to the estimate in the following terms: 

• Duration:  Number of Sprints (based on velocity and the Scrum team’s estimate of the effort in points or ideal days)
• Cost:  Burn rate of the Scrum team (or a portion thereof) for that number of Sprints

Integrate cost planning with other teams by using portions of your velocity to determine costs of features

3.  Planning, Monitoring, and Controlling

Another mismatch occurs in monitoring and controlling project execution.  Program and project managers may naturally want a project plan containing all activities, durations, and start dates.  The Scrum team may recommend Sprint and release burndown charts to track progress.  However, the program manager can point to another dependent team that needs a specific feature or component completed by a specific date.

A solution to this mismatch is to categorize those stories needed to fulfill that dependency.  Then, similar to a Sprint or release burndown, the Scrum team will burn down that story category, and track its completion to the date.  Stories from this category can be planned into sprints as necessary (starting backward from the delivery date).

Categorize work to meet dependencies and burn down that work each Sprint

A Scrum team previously not involved in a program could potentially not have experience with complex delivery dependencies.  Backlog items for developing the product may not incorporate post-development activities such as deployment, operational readiness, etc.  Project management supported by the program can plan these activities, and the Scrum team can look to the program’s expertise for this portion of the delivery.

A phased based project plan typically uses contingency time equating to some percentage of the estimate to account for unknowns.  A Scrum team avoids contingency using an empirical process, customer collaboration, and allowing the people doing the work to do the estimating.  If the program requires the project manager to add contingency to the delivery plan, a percentage of the release points could be added and a suitable duration extension based on velocity could be added to the project plan.  The Scrum team members are not affected by this contingency, but the ScrumMaster must ensure the Product Owner or customer remains involved with the current progress and scope so that transparency remains.

Utilize complex delivery expertise of project and program management

From a project management perspective, communicating updates to scope and release schedule may occur more frequently than if Scrum were not employed.  Project managers can view this as intentionally over communicating to program leadership, but also a necessary part of the process.

A ScrumMaster could protect the team by letting the project manager interface with external requests to the team.  Dependency coordination can be handled by the project manager, while the Scrum team focuses on the current release.  The project manager would have to ensure the Scrum process is followed by involving the Product Owner for dependency information affecting priority.

CONCLUSION

To win the mismatch, both sides need to evolve

To fit Scrum and phase-based programs together, both sides need to evolve.  Gains toward a lean organization that sends prioritized project work to agile teams focused on their product are desirable.  To make gains, champion Scrum and agile values and principles; and demonstrate to project managers a high confidence level throughout the project, a team building the right product, and competitive advantage delivered to the customer.