Agile Delivery Pilot: Non-Blocking Change Request - Phase 1 New Features

Expectations for the Agile Delivery Pilot

Agile Delivery Pilot Prerequisites

A CSO must meet the following prerequisites before applying to participate in the pilot:

• Does the CSO have one or more new services or features they plan to deploy during the pilot period that will allow for an opt-in capability for agencies?
• Does the CSO have mature, automated, and well-documented configuration and change management plans? CSO implementation of NIST Special Publication (SP) 800-218, Secure Software Development Framework (SSDF) Version 1.1: Recommendations for Mitigating the Risk of Software Vulnerabilities and/or similar secure software development frameworks is encouraged.
• Does the CSO have mature configuration and change management procedures that implement automated mechanisms for deploying services and features and verifying their secure implementation? CSO use of non-manual configuration implementations such as infrastructure as code is encouraged.
• Does the CSO incorporate vulnerability scanning and developer testing as part of their change and deployment processes?
• Is the CSO willing to share their mature processes and plans (including security artifacts) as examples to inform the FedRAMP knowledge base?
• Does the CSO demonstrate strong compliance with no open corrective action plans (CAPs) in the past 6 months due to issues with configuration management or incidents of failing to follow existing SCR processes?
• Does the list of participants reflect a diverse representation of CSOs, including to the extent possible, small and large businesses and different cloud service deployment models (IaaS, PaaS, SaaS)?
• Does the CSO demonstrate unique or innovative approaches to change or configuration management?
• Are agency customers who leverage the CSO willing to participate in the pilot?

Pilot Application Process

To apply for the initial round, a CSO must have completed the Agile Delivery Pilot: Non-Blocking Change Request - Phase 1 Features Application Sign Up Form (applications closed) with their contact information.

Additional pilot participants may be accepted and added to the pilot on a case by case basis. If you would like to be considered for the pilot but did not initially apply, please reach out to info@fedramp.gov for more information.

The CSO must also make the following available in its “Agile Delivery Pilot” folder:

• The security impact analysis for each change, in advance of the change
• Scan artifacts such as SAST or SCA scan outputs and OS or container vulnerability scans conducted in a staging environment
• If applicable, threat analysis, red team exercise results, contingency, privacy and supply chain risk analysis, and updates to control implementations
• Signed approval by the system owner or designee for each change

Please note: for systems that use the Connect repository, FedRAMP will create an “Agile Delivery Pilot” folder in their CSO submissions folder. CSOs with their own secure repository will need to create an “Agile Delivery Pilot” folder.

Pilot Selection Process

CSOs will be selected for the pilot based on each of the criteria described in the above section. In particular, FedRAMP will focus on CSOs with strong compliance and automated change control processes. Below is an example of a mature CI/CD pipeline that incorporates automation and robust testing.

Example of a Mature CI/CD Pipeline

FedRAMP will follow up with identified CSOs to discuss the requirements and their application to make a determination to include the CSOs in the pilot. FedRAMP will then engage with each included agency reviewers and seek AO concurrence. FedRAMP must receive AO concurrence for all CSOs involved in the pilot. FedRAMP will also notify any CSOs that were not selected. There will be a limited number of open spaces for the pilot.

Pilot Kickoff Process

FedRAMP will notify CSOs who applied of their acceptance status via email. From there, FedRAMP will schedule a 30-minute kickoff meeting with each pilot participant, along with participating agency reviewers. During this kickoff, the ground rules and expected outcomes of the pilot will be reviewed and agreed upon. Proposed changes for the pilot will be described at a high level and stakeholder expectations will be clarified based on each CSO. Additionally, a recurring cadence of meetings will be scheduled for check ins with FedRAMP and agency reviewers.

Pilot Evaluation Process

The evaluation of the pilot will include the following considerations:

• FedRAMP will review collected cost data and report to stakeholders the value proposition of implementing the automated SCR approval processes.
• FedRAMP will compare each CSO’s security impact analysis (SIA) against those performed by an independent assessor and report to each CSO their successes and areas of improvement in the security impact analysis (SIA) processes.
• FedRAMP will examine the continuous monitoring deliverables of each participating CSO to identify significant vulnerabilities, and analyze their causes and possible mitigations to prevent significant vulnerabilities from being introduced by future service or feature onboarding.
• FedRAMP will analyze pilot participants to identify innovative technology or processes that increased the effectiveness of CSOs’ change management programs.

Potential Next Steps

FedRAMP may take additional action based on lessons learned and successes achieved during the pilot. These may include:

• FedRAMP will evaluate the program, including characteristics of successful change management programs, and make updates to guidance and training for CSOs.
• FedRAMP, based on the evaluation of the pilot, may decide to extend the pilot timeline or expand the pilot scope.
• FedRAMP may identify other pilots for significant changes that would continue to identify areas of improvement and cost and resource savings.

Pilot New Feature Change Process for CSO Participants

CSOs participating in the Agile Delivery pilot are expected to follow the process below for each new service or feature they add as part of this pilot:

• Pre-Deployment
• The CSP should follow their existing change control processes, however certain actions must be taken for changes introduced during the pilot. CSPs must determine whether a change is significant and meets the pilot criteria by performing the following:
• First, determine whether the change is standard or significant (for example, by comparing the change to the list of example significant changes in NIST SP 800-37 and 800-128). This step may be automated based on data submitted in a ticket for the change request.
• Next, determine the type of significant change (i.e. feature/service, interconnection, etc). Only changes related to introducing a new feature or service are in scope for this pilot. For all other significant changes, the standard SCR process must still be followed.
• Next, define the security thresholds for introducing the change (i.e. based on the FedRAMP Performance Management Guide).
• Finally, conduct a Security Impact Analysis (SIA) that includes all impacted controls, and upload the SIA to the FedRAMP repository.
• The SIA may use a template based on organization preference, but it must include:
• An overview of the new feature or service functionality
• A timeline including key milestones
• Impacted controls and planned control implementation, including consideration of supply chain risk, contingency and incident response planning, and privacy impact
• Description of tests to be performed (including failure thresholds), including:
• SAST/SCA scans
• Vulnerability scans (OS, network, DB, web, and container) targeting a staging environment
• Compliance checks (hardening, configuration baseline validation)
• If warranted, threat modeling and/or a penetration test may be performed
• For this, FedRAMP encourages CSPs to leverage automated triage of vulnerability findings (i.e. CISA VEX for false positives, Vulnrichment for risk adjustments) where this is applicable
• Planned customer responsibilities for the new feature or service. There must be a clear path for customers to opt-in of changes if they do wish to accept the risk.
• Approval by System Owner (or designee) attesting to review and approval of the change. This may be part of an automated CI/CD gate.
• Once the change is approved, the CSP must ensure that all relevant documentation is also updated (i.e. diagrams, System Security Plan, Customer Responsibility Matrix, etc.)
• OSCAL component-based machine readable deliverables are encouraged.
• FedRAMP recommends that CSOs consider using SBOMs generated at build time as artifacts to inform the SIA.
• Production Deployment
• Implement the new service or feature within the production environment
• Perform production testing and validation of the new components. This must include vulnerability and configuration scans.
• Review and update any operational requirements that extend to the new service or feature, including validating that any mitigations are in place.
• Gather all artifacts that validate the secure implementation of the service or feature.
• Gather all artifacts that demonstrate the secure implementation of the service or feature. CSOs must provide evidence of:
• The secure implementation of all services or features that are onboarded and that no unmitigated critical or high vulnerabilities exist.
• That appropriate scans, testing, and threat modeling was performed for the new components, including those that reflect no vulnerabilities.
• All identified issues are accurately captured within the POA&M.
• Record all vulnerabilities identified as part of the change on their POA&M, indicating the new service or feature name and Agile Delivery pilot in the Service Name field.
• FedRAMP and the authorizing agency will analyze evidence and POA&M during the pilot to ensure compliance with the FedRAMP requirements and the CSO’s configuration and change management processes.
• Post-Deployment
• Notify FedRAMP and the AO that the change has been completed and upload updated inventory, artifacts, scans, and POA&M to their documentation repository.

Throughout the pilot period, CSOs must submit their vulnerability scans, inventory, and POA&M monthly. These should be uploaded to their “Vulnerability Scans” folder in either their Connect repository or in their own secure repository. The vulnerability scans must be in a machine-readable format, and the inventory and POA&M must be in the latest format.

CSOs must finalize all changes made under the pilot by December 31, 2024.

Each CSO must have a third party assessment organization (3PAO) perform testing of all services and features that were deployed as part of the pilot. This may be part of an annual assessment that happens to occur at the appropriate time, or it may be one or more separate specific assessments (preferably one). Each CSO may also perform its own testing during the pilot period. The CSO or 3PAO must submit all assessment reports, including SAP and SAR, and all artifacts by the end of the pilot closeout period. Any findings found internally by the CSO or the 3PAO must be included and tracked in the POA&M and must be remediated in accordance with the FedRAMP ConMon Performance Management Guide remediation timelines.

Failure to follow the process outlined above may result in being removed from the pilot. If there are issues with the process, FedRAMP will be available to assist the CSO in clarifying requirements and expected outcomes. It is important that the participants engage with FedRAMP to ensure that the pilot identifies the successful elements as well as any opportunities to improve the process. The goal is to identify requirements and best practices that allow CSOs to use their configuration and change management processes to make changes without intervention or approval from the AO, and to consolidate testing into annual assessments to reduce time and cost of doing individual assessments for each change.

CSOs participating in the pilot should collect data throughout the pilot on cost savings or financial impacts of utilizing the pilot processes for new services and features. This data should include the following:

• Projected cost-savings per change in independent assessment costs
• Projected time savings
• Projected number of changes per year (if the new process is adopted)