No, the 3PAO assessor has to perform penetration testing as part of the assessment. The pen test results, along with vulnerability scanning and control testing results, determine the findings and risk exposures that assessors identify on the Security Assessment Report (SAR) and are the basis on which the assessors recommend whether the cloud service be granted an authorization or not. Because the pen test is so important to the overall assessment, 3PAOs need to have confidence that the pen test was performed by competent and independent assessors and was adequately scoped to provide the proper coverage. Therefore, 3PAOs need to perform their own pen testing activities during the assessment process.
Furthermore, the scope of a typical penetration test will not provide adequate coverage of the attack scenarios and risks that FedRAMP requires. A FedRAMP pen test has a unique scope that corresponds to guidance provided by the FedRAMP PMO, including the following testing scenarios:
- Non-credentialed external attack on Internet-facing systems and components, including any routable IPs, URLs, and login pages that are accessible to the general public.
- Credentialed attack on the cloud system as an authorized user, including traditional OWASP testing for SaaS systems for browser-based systems, mobile transactions, and APIs.
- Tenant-to-tenant attacks within the target system, including specially crafted attacks to access data pertaining to another cloud tenant. Because cloud systems are defined by their multitenancy features, 3PAO penetration tests focus on this attack path because it specifically addresses a distinctive element of risk within cloud systems.
- Credentialed attack from the cloud system to the management system. The point of this scenario is to see if one tenant can leverage management system components as a stepping-stone to launch attacks on other tenants. Many components (e.g., backup systems, vulnerability scanning systems, AV systems, etc.) within the management system have access to multiple tenant environments and therefore could provide a stepping-stone for one tenant to reach other tenants or even all tenants.
- Attack on the content stored on a mobile device by mobile-based applications. The purpose of this scenario is to assess the risk to the system if a mobile device that is used to access the application is lost or stolen.
- Phishing attack on cloud system administrators. The intent of the phishing attack is to determine the likelihood that corporate workstations used by cloud system administrators could be compromised through social engineering.
- Simulated attack from a presumed remotely-owned corporate workstation to the cloud management system. The point of this scenario is to assess the likelihood that a breach of a corporate workstation could lead to a compromise of the cloud management system, and then ultimately the target cloud system. The assessor has to test a representative corporate workstation, but the rest of the test is usually simulated via a tabletop walkthrough with the cloud provider. The intent of this scenario is to obtain insight into the risk that an attack on a corporate asset could pivot to and therefore endanger the cloud system without conducting actual pen testing activities on the CSP’s corporate environment, which is technically out of scope for FedRAMP.
CSPs are usually surprised to learn about the last two testing scenarios because they involve corporate systems that are not within the FedRAMP authorization boundary. What may not be clear at first is that these two scenarios, when combined, represent a back-door threat to the cloud system. Adversaries that learn that Federal agencies are using a cloud system may target administrative users in the cloud provider’s corporate environment as the first step of an attack path ultimately to reach the target cloud environment. During the phishing and simulated corporate workstation scenarios, the 3PAO pen testers attempt to determine the likelihood of success that such an alternative, back-door attack path could impact the cloud system. They also identify control improvements to eliminate that back-door threat without having to engage with much of the CSP’s corporate environment. Seen in this light, those scenarios are a reasonable middle-ground approach for assessing and reducing risk to the cloud system.
Cloud providers that are seeking a FedRAMP authorization are advised to have a FedRAMP-like penetration test performed on their cloud system as part of their readiness efforts to make sure that their system can withstand the scrutiny they will face when the 3PAO performs their assessment pen test. CSPs often engage another 3PAO to perform advisory services and gap assessments (including pen testing) to enhance their chances of a successful outcome on their official assessment. Using a 3PAO pen tester ensures that the scope of the gap assessment pen test includes all of the attack scenarios outlined in the PMO’s guidance.