18F Forest Service: Open Forest Transition Plan

Overview

Since August 2016, the U.S. Forest Service has partnered with 18F to design and build an online permitting application to accommodate several permit types derived from five pilot forests. After nearly three years of partnership in demonstrating the efficacy of human centered design, modular contracting, and agile and open practices, 18F is handing the keys to the platform over to the USFS. This roadmap lays out the success scenarios, success criteria and tasks to ensure a successful transition.

The vision

The 18F team has departed the project, and the USFS team fully owns the management and development of the Open Forest platform. The team is fully empowered and capable of maintaining the app so that it continues to meet user needs in an iterative, open, and human-centered manner. This includes all aspects of the program, including design and research; software development and devOps; integration of other systems; and program communications and customer support.

What does success look like?

The USFS Open Forest team members (including contractors) regularly attend sprint ceremonies with their vendor(s); provide product, technical, and design guidance to the vendor team(s); regularly demo progress to a wide group of stakeholders; are capable of scaling the application to onboard new forests and permit types; and plan for and research future procurements. The FS or vendor team has changed the application in several significant ways in response to user feedback and usability testing. More forests have begun to use the app, including for additional permitting use cases.

What might failure look like?

After 18F left the project, the FS team did not continually improve the application. Additional FS talent was not identified or fully committed to the project, and as a result, contractors didn’t receive adequate feedback, so the product they delivered did not meet user needs. Additionally, operations were not conducive to pushing changes to production because of security and compliance concerns. As a result, momentum was lost and eventually, the application stalled and ceased to be used.

Transition roadmap: ## Success scenarios, success criteria, and tasks

This roadmap lays out the success scenarios, success criteria and tasks to ensure a successful transition, in priority order. These scenarios represent the type of situations the Open Forest team will face after 18F departs the project and what needs to be in place to ensure the team’s success. You can learn more about how arrived at this plan here.

1. Success scenario

Several users suggest a new reporting feature that would require we collect additional information that would classify as personally identifiable information (PII). The Open Forest team recognizes the PII implications, determines that a System of Record Notice (SORN) would be required, and presents tradeoffs to the Product Owner to make a decision.

● Success criteria

⃞ The Open Forest team is empowered to critically assess recommended design changes and tradeoffs

⃞ The Privacy Impact Analysis (PIA) is up to date

⃞ The Open Forest team is familiar with the current PIA and how changes might impact the PIA

● Tasks

⃞ 18F delivers protecting PII training to vendor team

✓ Ongoing collaboration with current PO

⃞ Open Forest team documents a process for regularly revisiting the PIA and making any updates

⃞ The Open Forest team reads and understands the PIA

⃞ Assign an EBSS ISSO to Open Forest

2. Success scenario

A user reports a bug that the team triages as critical. A member of the vendor team files a pull request (PR) to resolve it. They are able to quickly get the appropriate reviews from Forest Service team members and push to production within 2 business days using an automated deployment process and without breaking the app.

● Success criteria

⃞ There is a shared understanding of the Product Owner’s prioritization.

⃞ At least two members of the Open Forest team can review and merge Pull Requests without 18F’s help

⃞ The site release process is documented

⃞ Users have the ability to report bugs and get them into the queue

✓ The app is in production

⃞ The Open Forest dev team uses Continuous Development/Continuous Integration pipeline (e.g. github -> circleCi -> cloudFoundryCli -> cloud.gov)

● Tasks

✓ Verify release process documentation is up to date.

✓ Continue to adhere to sprint ceremonies (planning, backlog refinement and grooming)

⃞ Ensure that Shadat can view and merge pull requests without help

⃞ Ongoing collaboration to establish a strong feedback loop with current PO

⃞ Ensure that a bug reporting and response feedback loop is added it to the product roadmap

⃞ Ensure that our applications and services are using appropriate authentication credentials

⃞ Fill open engineering role at EBSS

⃞ Ensure that our CI/CD pipelines are documented, well understood, and 2+ people have access

3. Success scenario

The Open Forest team is designing a new print-at-home permit for a new permit type. The new permit has a complex hierarchy, several graphical elements, and is intended to be printed with a landscape orientation. The team’s accessibility testing process includes automated pa11y testing followed by lightweight manual testing. The team manually tests the permit using a screen reader and identifies graphical elements that need to be converted to accessible text and adjusts the read order to ensure that a screen reader will move through the information in a logical sequence.

● Success criteria

✓ The Open Forest team has automated accessibility testing tooling

⃞ The Open Forest team conducts manual accessibility checks on every new feature

● Tasks

⃞ Investigate additional tooling and processes

✓ 18F provides training on manual accessibility testing

✓ Share GSA/TTS guidance on automated accessibility testing tools with Open Forest team

⃞ Establish and document a regular cadence for accessibility testing

4. Success scenario

In the course of onboarding a new permit type, the Open Forest team encounters a need to change the Open Forest middle layer. The team is able to redesign the current architecture and build more of the permit generation capability within Open Forest.

● Success criteria

⃞ The Open Forest team is comfortable working in Node JS

⃞ The Open Forest team knows how to create a new permit type and how to handle changes to the SUDS API

● Tasks

⃞ Documentation re: who at NRM the Open Forest team should contact for support

⃞ Fill back end engineering role

✓ Detailed, accurate instructions for adding a new permit type and required changes to the middle layer are findable on GitHub

⃞ Improve existing README documentation on how to set up the environment.

5. Success scenario

A senior stakeholder proposes adding a new feature to the site. Open Forest designers work with the stakeholder to form a testable hypothesis. The lead then scopes the minimum amount of design work needed to test the feature. After prototyping it, the team conducts user research to validate whether it serves user needs, and refines the feature based on that research.

● Success criteria

⃞ Open Forest staff provide meaningful design and research feedback to vendor without 18F involvement

● Tasks

✓ 18F continues to mentor researchers and designers in sprint ceremonies and regular design pairing and refinement sessions

⃞ EBSS hires mid or senior level design researcher who can dedicate .25 FTE to Open Forest

6. Success scenario

Several stakeholders attending a product demo point out that new product features will have a significant, yet unknown impact on frontliners’ workflow. The Open Forest team prioritizes a discovery research spike. Open Forest researchers/designers use contextual inquiry as a method of interviewing and observing the way frontliners currently work. The team then documents insights in a research brief and creates any new user stories to ensure that frontliners needs are accounted for in subsequent design and development work.

● Success criteria

⃞ Open Forest staff possess a solid understanding of basic user research concepts

⃞ Open Forest team has conducted discovery research

⃞ Open Forest team recognizes where additional discovery research may be needed in order to inform design and strategic programmatic decisions

● Tasks

✓ Ongoing training and mentorship by 18F

⃞ Vendor team plans, executes, and documents discovery research with help from 18F

⃞ Ongoing training and mentorship by 18F

⃞ The Open Forest team writes up a usability test plan to validate some of the team's recent work and publishes the plan to the GitHub wiki

7. Success scenario

The team moves away from cloud.gov to a new infrastructure provider. The Open Forest team is able to reconstitute the app from the configuration as code (CAC).

● Success criteria

✓ The team is able to migrate configurations in the release pipeline, alongside application code

● Tasks

⃞ Confirm that CAC documentation is up to date and findable on GitHub

8. Success scenario

Open Forest becomes wildly popular and the current plan for one of our managed services in Cloud.gov is insufficient. The Open Forest team is able to upgrade the brokered service plan in Cloud.gov with minimal to no downtime.

● Success criteria

⃞ The team can make changes to environment variables

⃞ The team understands and can make changes to our deployment configuration

● Tasks

⃞ Add sharing of credentials to developer onboarding

⃞ The team has appropriate permissions and credentials to configure Cloud.gov

⃞ Create an automated task for updating environment variables in Cloud.gov

⃞ The team has an understanding of any resource quotas in our contract with Cloud.gov

9. Success scenario

The Open Forest team launches the special uses pilot. Through regular usability testing and user feedback they receive, the team incorporates learnings from the pilot to incrementally improve the product.

● Success criteria

⃞ The application is live

⃞ The team has a process for integrating user research into its sprint cycles

⃞ The application has a customer feedback mechanism

⃞ The Open Forest team has successfully run an agile development sprint without 18F’s help

● Tasks

⃞ 18F does not attend sprint ceremonies for the sprint following the end of this engagement, but reserves a few hours to attend sprint demo and retro and provide feedback

⃞ Forest Service implements Login.gov

✓ 18F provides training on integrating design research into agile sprints

✓ Add the design and implementation of a customer feedback intercept to the product roadmap including a backend process for ingesting and responding to customer feedback

⃞ 18F delivers agile 101 training

⃞ The EBSS team conducts a round of usability testing

10. Success scenario

Team research uncovers that new features will require an additional “super” administrative user to administer the existing administrators. The Open Forest team determines the desired user experience and the changes are implemented carefully and thoughtfully to ensure maintainability.

● Success criteria

⃞ The Open Forest team discusses the user and platform impact of feature additions and changes

⃞ Features with a higher level of risk, such as authentication and third party integrations, are given more scrutiny and are fully testable

● Tasks

⃞ Hire/onboard a technical leader with the motivation and technical skills to implement feature additions and changes in a way that prioritizes the long-term health of the Open Forest platform

11. Success scenario

A team member wants to make a change to the current application architecture and wonders what went in to the initial decision. They find and access, documentation from earlier phases of the project that informs their decision to make a different sort of change and they update the documentation accordingly (Google drive, GitHub wiki, etc.)

● Success criteria

⃞ Project information is up-to-date

⃞ Project information is findable by FS staff and contractors

⃞ At least two members of the Forest Open team have the permissions and knowledge to update GitHub

● Tasks

⃞ 18F and FS review and ensure that all documentation is up to date

⃞ 18F and FS revise Google Drive architecture

⃞ The Open Forest team sets up the appropriate permissions in Box and the FS GitHub

⃞ Establish and document a practice of auditing and updating project documentation on a quarterly basis

⃞ Migrate content to FS Box drive

⃞ Revise information architecture in the GitHub wiki

⃞ Migrate content to FS GitHubAccount

⃞ Ensure that a history of all issues is maintained in the content migration

12. Success scenario

A vulnerability is disclosed for underlying dependency. The Open Forest team pushes that change to production in under 48 hours.

● Success criteria

⃞ The Open Forest dev team uses Continuous Development/Continuous Integration pipeline (e.g. github -> circleCi -> cloudFoundryCli -> cloud.gov)

✓ The Open Forest operations team has both automated and manual measures for identifying potential system vulnerabilities

⃞ The Open Forest team understands what communication with integrated services will be needed in the event that scaling up pushes up against our limits.

⃞ The team understand the criticality of different vulnerabilities and the required remediation per our security plan

● Tasks

⃞ The Open Forest team is subscribed to all integrated service listservs (cloud.gov, pay.gov, login.gov, new relic--or its alternative--alerts)

⃞ Ensure that our applications are using a FS user's Snyk account

⃞ 18F provides training on the different services, current limits, and potential overage situations

⃞ The vendor team reads and understands the System Security Plan (SSP)

⃞ 18F documents third party service limits and what kind of communication will be needed in the event the application nears limits

⃞ Review documentation on how our process meets change control

⃞ Shadat and 18F security peers go through change management plan focusing on how it is an approved process

How we arrived at this plan

18F staff began by drafting a list of 20 common scenarios that the team has encountered over the course of our two and a half year long collaboration with the Forest Service. We reviewed and revised these scenarios with input from the broader Open Forest team including Forest Service stakeholders and the vendor team. 18F staff and Forest Service stakeholders then conducted a gap analysis following this outline: Each participant independently assessed the priority and team readiness for each of the scenarios using the following ratings: Priority

● P1 = project will likely fail without this

● P2 = project may fail without this

● P3 = project will get by without this

Readiness

● 1 = insufficient readiness

● 2 = satisfactory readiness

● 3 = high readiness

We then selected the above 12 scenarios because participants identified them as high priority, yet participants scored the team’s readiness to handle these scenarios as either insufficient or just satisfactory. This let’s us focus our transition efforts on building the highest impact skills and capacities in the coming weeks.