AWS Container Competency

Applies to: SaaS | Customer Deployed

The AWS Partner solution meets the definition of the category specified in the application, and all four customer case studies (see prerequisite section 2.0) also align to this specific category. See the AWS Container Definition and Competency Categories for category definitions.

For solutions in the Security category, the solution must meet all requirements defined for at least one of the security subcategories defined below: CVE Scanning and Remediation, Network Security, Security Event Monitoring.

Applies to: SaaS | Customer Deployed

The AWS Partner will provide a live demonstration via screen sharing to an AWS Solutions Architect during the technical validation process. This demonstration will include the initial setup and integration steps customers are required to complete in order to use the solution with AWS. All other functional requirements described in this document may also be requested to be demonstrated during this process.

Applies to: SaaS | Customer Deployed

The solution can be run as either an agent (Deployment/DaemonSet/StatefulSet) or a side car. Alternatively, solutions can provide a library embeded in an application to gain insight into container level traces and metrics.

AWS Partner must use one of the above for the demo and provide overview of the Dockerfile and task definition/manifest.

Applies to: SaaS | Customer Deployed

The solution's official documentation includes instructions for configuring the solution using either the agent or side car deployment model.

Applies to: SaaS | Customer Deployed

The AWS Partner solution provides generally available support for one of the following AWS container services:

• Amazon Elastic Container Service (Amazon ECS)
• Amazon Elastic Kubernetes Service (Amazon EKS)
• AWS Fargate
• AWS AppMesh
• AWS AppRunner

Support must be publicly documented and fully endorsed for production use cases.

Applies to: SaaS | Customer Deployed

The solution supports both Linux and Windows containers and hosts.

Solutions in the Security Event Monitoring subcategory are currently exempted from this requirement, however this exemption is likely to be removed in the future.

Solution must support the following authentication and authorization mechanisms:

• Centralized identity and access management
• Support for identity federation via SAML 2.0, OIDC, or direct LDAP/Active Directory integration.
• Role Based Access Control (RBAC)
• Access controls per namespaces and finer grained resources including clusters and workloads
• Audit logs for changes to authentication or authorization configuration
• User access logs

Please provide the following as evidence:

• Link to documentation or description of how the partner integrates with SSO and/or IdP providers for access control to the underlying solution.
• Link to documentation or description of how RBAC can be leveraged in the platform. An example would include the ability to create users and groups with defined permission boundaries for those users and groups within a cluster.
• Description of how the solution applies access controls for clusters, namespaces, and workloads
• Description of how solution provides auditing mechanisms to ensure that an administrator has the ability to look at audit logs for compliance reviews for example. Solution should also have the ability to designate an organizational administrator that have full read access to audit logs.

Applies to: SaaS | Customer Deployed

The solution is capable of running multiple containers on a single host.

Applies to: SaaS | Customer Deployed

The solution provides facilities for managing the entire lifecycle of a container. It should provide support for capabilies such as portability, service discovery, load balancing, security, performance, and elasticity of containers and host infrastructure.

Applies to: SaaS | Customer Deployed

The solution allows customers to implement role based access control for provisioning and management actions.

Applies to: SaaS | Customer Deployed

The solution integrates with at least one of the following AWS services:

• Amazon Virtual Private Cloud
• Elastic Load Balancing
• AWS CodeDeploy
• AWS CodePipeline
• Amazon Elastic Container Service (Amazon ECS)
• Amazon Elastic Kubernetes Service (Amazon EKS)
• Amazon Fargate

Applies to: SaaS | Customer Deployed

The solution provides baseline, peak and trend related metrics at the infrastructure layer around cluster health, nodes, and critical system components such as coredns, kube-proxy, CNI, etc. The solution must support observability controls at the container and host level, including auto scaling based off metrics or container deployment strategies. The solution must provide granular metrics at the container and host level and allow users to filter to specific containers on individual hosts through a dashboard. The solution should use OpenTelemetry (OTel) (use of AWS Distro for OpenTelemetry (ADOT) is preferred) for cluster monitoring. If the solution is using Prometheus and/or Grafana, then such solutions should demonstrate integration capabilities with Amazon Managed Service for Prometheus and Amazon Managed Grafana respectively

AWS Partner must demonstrate the ability to capture metrics during both task/service/pod level scaling as well as instance scaling events.

Applies to: SaaS | Customer Deployed

The solution provides a single pane of glass view to provide end-to-end monitoring of compute resources, orchestration, containers and application code in real time. The solution must provide metrics at the service level, container level, and host level, as well as a view into the Docker logs for each container and aggregated at a service level.

Applies to: SaaS | Customer Deployed

The solution provides monitoring of components and resources running on-premises and in AWS. This includes support for environments that span multiple AWS Regions and/or accounts. The solution should use OpenTelemetry (OTel) (use of AWS Distro for OpenTelemetry (ADOT) is preferred) for cross-environment monitoring. If the solution is using Prometheus and/or Grafana, then such solutions should demonstrate integration capabilities with Amazon Managed Service for Prometheus and Amazon Managed Grafana respectively

Applies to: SaaS | Customer Deployed

The solution can collect logs from control plane, worker nodes and applications.

For logging solutions, the solution must support one of the following three modes of integration:

• Ingesting logs from Fluent Bit (optionally via Fluentd)
• Ingest logs using ADOT or OTel
• Demonstrate the usage of using Amazon CloudWatch as a log destination is mandatory. Usage of ADOT or OTel preferred in this demonstration

Applies to: SaaS | Customer Deployed

The provides tracing capabilities for collecting detailed information about components and services involved in the request-response flow of an application.

For tracing solutions, the solution must demonstrate tracing with one of the following three modes:

• Demonstrate and support tracing capabilities with AWS X-Ray Daemon
• Demonstrate and support tracing capabilities with ADOT or OTel
• Demonstrate the usage of using AWS X-Ray as a trace destination is mandatory. Usage of ADOT or OTel preferred in this demonstration

Solutions that support on other self-managed Kubernetes or Container orchestration platforms on on-premises or other cloud platforms must also support Amazon EKS Anywhere and/or Amazon Elastic Container Service (ECS) Anywhere.

Please provide the following as evidence:

• Description or links to documentation that guide customers through the setup and management of observability on AWS EKS Anywhere, Amazon EKS Distro or Amazon ECS Anywhere clusters in environments outside of AWS (on-prem and/or other cloud providers).

Applies to: SaaS | Customer Deployed

Solution must provide capabilities to provision container infrastructure for the following dimensions:

Solutions for managing on AWS

1. Multi-cluster, multi-region, and multi-account EKS deployments
2. Edge locations (on-premise data centers, on-premise edge locations/sites)
   • Validation will focus only on AWS as a regular IaaS provider for provisioning of non-AWS managed container solutions (e.g. managed or self-managed kubernetes)
   • AWS Outposts is a valid edge location

Solutions for managing outside AWS

1. Multiple cloud providers (GCP, Azure, Alibaba Cloud, IBM Cloud)
   • Ability to provision EKS Distro based clusters
   • Ability to provision and manage EKS-A clusters
2. Edge locations (on-premise data centers, on-premise edge locations/sites)
   • Validation will focus only on AWS as a regular IaaS provider for provisioning of non-AWS managed container solutions (e.g. managed or self-managed kubernetes)
   • AWS Outposts is a valid edge location

Please provide the following as evidence:

1. (If applies) Link to documentation that walks a customer through how to provision EKS clusters across multiple AWS locations/regions. This may also be evaluated through a live demo with an AWS PSA.
2. (If applies) Demo or link to documentation that demonstrates the ability to deploy EKS-Anywhere and/or EKS-Distro based clusters across different environments outside of AWS.
3. (If applies) Demo or link to documentation that demonstrates the ability to provision EKS-A and/or EKS-D clusters at different edge locations. AWS Outposts is considered a valid edge location

Applies to: SaaS | Customer Deployed

Solution must provide higher level logical groupings (namespaces) of resources that facilitate multi-tenancy for customers. For example it could be a group of resources that represent a scope of access for development or infrastructure team. Such constructs typically represent organizational units (OU), environments, projects. The “soft” tenant groupings are required to have clear object level access controls and define permission boundaries for the users (e.g. ability to add/remove users within a namespace, ability to create/drop workloads within a namespace).

Please provide the following as evidence:

1. Description of how the solution supports mapping of enterprise organizational hierarchy to the abstractions provided by the solution. This may be in the form of documentation or doing a live demonstration with the respective AWS PSA running the validation.
2. Description of how a customer would leverage the namespacing controls provided by the partner to define permission boundaries for users within a logical namespace. This may be in the form of documentation or doing a live demonstration with the respective AWS PSA running the validation.

Applies to: SaaS | Customer Deployed

Solution must provide observability controls including logging and monitoring either directly or through packaged integrations with other solutions (e.g. CloudWatch, Prometheus/Grafana, Partner solution). Observability controls must provide the ability to observe state of the cluster, nodes, workloads as well as apply the monitoring, logging and tracing data for troubleshooting. This may be achieved using Partner and/or open source solutions.

Notes: For multi-tenant SaaS, solutions are expected to provide full tenant isolation for observability. Similar requirements apply to the namespace level isolation controls as it pertains to observability and monitoring.

Please provide the following as evidence:

Evidence should be in the form of a technology demonstration and 2 current customer Case Studies of solutions which have been in production for at least 6 months.

Case 1: Demonstrate that individual tenants have access to the logs and monitoring data (as well as any higher order data such service reliability reports, extra intelligence collected through monitoring with real-time or post-analysis) which can be accessed directly by the tenant within the scope of access boundaries defined for the tenant (only data that belongs to the tenant). This may be in the form of documentation or doing a live demonstration with the respective AWS PSA running the validation.

Case 2: Demonstrate that members of namespaces can access logging and monitoring data within the scope of their access boundaries. For example, a member of a development team can access logs and monitoring data for the application workloads to which this member (or development team) is authorized to access. This may be in the form of documentation or doing a live demonstration with the respective AWS PSA running the validation.

For internal verification it is recommended that an AWS PSA be able to perform these actions using a test environment/account provided by the partner

Applies to: SaaS | Customer Deployed

Solution must demonstrate security controls that address each of the following categories:

1. Centralized identity and access management
2. IdP integration with common identity providers (e.g. SAML 2.0 and OIDC providers)
   • Integration with AWS SSO and/or packaged integration for Amazon Cognito for identities is a plus
   • Direct LDAP/Active Directory integrations can also be part of the validation provided all communications are encrypted (TLS)
3. Role Based Access Control (RBAC)
4. Access controls per namespaces and finer grained resources including clusters and workloads
5. Audit controls

Please provide the following as evidence:

1. Description of how the partner integrates with SSO and/or IdP providers for access control to the underlying solution.
2. Description of how RBAC can be leveraged in the platform. An example would include the ability to create users and groups with defined permission boundaries for those users and groups within a cluster.
3. Description of how the solution applies access controls for clusters, namespaces, and workloads
4. Description of how solution provides auditing mechanisms to ensure that an administrator has the ability to look at audit logs for compliance reviews for example. Solution should also have the ability to designate an organizational administrator that have full read access to audit logs.

Applies to: SaaS | Customer Deployed

Solution provides at least two of the specified governance controls:

1. Central policy management: ability to define governance policies centrally and apply them consistently across multiple container infrastructures
2. Ability to prevent workloads/deployments that bypass governance controls (either prevention, e.g. suppress workloads from running or reporting capability to identify violations). Validation may be achieved by applying a third-party solution such as OPA Gatekeeper. This assumes that provisioned cluster use a blue-print with the required policy software installed.
3. Ability to provision and maintain clusters that adhere to the governance “blueprint”. For example, demonstrate how the following governance policy can be enforced by the solution: ensure that each provisioned and operating cluster runs container runtime and network security solutions as well demonstrate controls that prevent removal of such solutions. Example negative scenario: developer/admin gets access to the cluster directly and removes the container runtime security or network security product. Is this “drift” detected? Can it be remediated.

Please provide the following as evidence:

1. Description of how the partners solution provides the customer the ability to define governance policies consistent within and across clusters. This may be in the form of a third-party integration such as OPA Gatekeeper or Kyverno.
2. Description of how the partner approaches providing consistent guardrails and permission boundaries across clusters to prevent the following scenario: A malicious user (internal) gains admin access to the cluster and removes all security controls within the cluster. Description and demonstration of customer experience in such cases should cover 1) detection controls for such events 2) remediation controls (may include processes explicitly activated by the customer to remediate).

Applies to: SaaS | Customer Deployed

Integration with at least two services from each of the following service groups (through API, CloudFormation and UX integration):

Group 1 (Amazon Container Services)

1. Amazon EKS (and/or EKS Fargate)
2. Amazon ECS (and/or ECS Fargate)
3. Amazon ECR
4. AWS App Mesh

Group 2 (Amazon Infrastructure Services)

1. Amazon CloudWatch (Monitoring, Logs, Alarms), Container Insights, AWS CloudTrail (e.g. for anomaly detection)
2. AWS VPC (ability to define and deploy cluster infrastructure to private/public subnets and availability zones)
3. Load balancers: CLB, NLB, ALB provisioning with workloads
4. ACM integration: ability to use public and private ACM certificates for TLS protecting ingress to workloads

Please provide the following as evidence:

1. Description of an integration with at least two of the Amazon Container services including links to documentation that reference how a customer would utilize Amazon Container services
2. Description of how customers would leverage Amazon infrastructure services to support the provisioning and management of Amazon Container services. An example would include being able to provision and manage an Application Load Balancer with an Amazon EKS cluster.

Applies to: SaaS | Customer Deployed

Ability to provision and support EKS-Anywhere (EKS-A), EKS Distro clusters or ECS-Anywhere (ECS-A) deployments in Hybrid and Multi-cloud environments. Partners that support other self-managed Kubernetes on multiple cloud platforms and on-prem are expected to support EKS-A, EKS-D or ECS-A based clusters to help AWS customers maintain a homogenous setup across cloud provider boundaries.

Note: This section is evaluated for partners supporting Hybrid and Multi-cloud container management solutions

Please provide the following as evidence:

1. Description or links to documentation that guide customers through how to provision and manage EKS, EKS-D or ECS-A clusters in environments outside of AWS (on-prem and/or other cloud providers).
2. Description or links to documentation that cover the support model provided for customers running hybrid workloads using EKS-A, EKS-D or ECS-A.

Applies to: SaaS | Customer Deployed

Solution must demonstrate availability of well-defined APIs (governed by an API specification) and CLI to facilitate CI/CD processes as well as actual examples of integration with CI/CD tooling (e.g. AWS CodePipeline) for development and operations teams to deploy and operate workloads. Solution must support following scenarios:

1. Scenario 1: using CI system to build and upload container image to ECR and perform deployment of the workload in EKS/ECS (sync or async)
2. Scenario 2: using CD system to deploy a workload with subsequent rollback
3. Scenario 3: using CD system deploy a workload supporting the rolling, blue/green and canary deployment methods

Notes: actual examples of CI/CD pipelines with the solution will be evaluated. It is advisable to have integration with imperative or declarative infrastructure provisioning and workload provisioning and demonstrate IaC and GitOps deployment models.

Please provide the following as evidence:

1. Description of CI and CD solutions that are available for customers to leverage for automation of updates/deployment to infrastructure and applications. Links to documentation will also be evaluated.
2. Description of integration with infrastructure-as-code (IaC) tools to achieve GitOps deployment models. Links to documentation will also be evaluated.

Applies to: SaaS | Customer Deployed

The solution can scan container images in a registry as well as running container images for CVEs.

Applies to: SaaS | Customer Deployed

The solution can prevent container images with identified security findings from being deployed.

Applies to: SaaS | Customer Deployed

The solution provides a user extensible rules engine that allows customers to define custom policies. The solution provides alerting based on these policies and allows for manual or automated remediation.

Applies to: SaaS | Customer Deployed

The solution notifies users when a new CVEs are available and allows them to automatically update to the latest version or select when to perform the update.

Applies to: SaaS | Customer Deployed

The solution allows customers to deine policies that enforce network traffic patterns and segmentation.

Applies to: SaaS | Customer Deployed

The solution provides a facility for capturing and monitoring network traffic.

Applies to: SaaS | Customer Deployed

The solution provides the ability to transparently encrypt network traffic between containers.

Applies to: SaaS | Customer Deployed

The solution monitors running containers for suspicious activity. The solution must be able to detect at least the following events:

• Shell logins
• Modifications to critical files including the root file system and other critical application and configuration files
• User ID changes for a process

Applies to: SaaS | Customer Deployed

The solution monitors activity at the orchestration layer and detects suspicious events such as unexpected container starts.

Applies to: SaaS | Customer Deployed

The solution provides an interface for users to search and filter identified events.

Applies to: SaaS | Customer Deployed

The solution integrates with AWS Security Hub or 3rd party SEIM solutions.

Applies to: SaaS | Customer Deployed

AWS Support provides a mix of tools and technology, people, and programs designed to proactively help you optimize performance, lower costs, and innovate faster. AWS Business Support provides additional benefits including access to AWS Trusted Advisor and AWS Personal Health Dashboard and faster response times.

For SaaS solutions, subscribing to AWS Business Support or greater (including AWS Partner-Led Support) for all production accounts is a requirement to successfully complete the validation for Competency Program. For customer deployed solutions, the Partner must subscribe to premium support for the AWS accounts which produce the solution.

Applies to: SaaS | Customer Deployed

AWS Partner offers product support via web chat, phone, or email support to customers.

Evidence must be in the form of description of support offered to customers for their product or solution.

Applies to: SaaS | Customer Deployed

AWS Partner must submit architectural diagrams depicting the overall design and deployment of the solution on AWS as well as any other relevant details of the solution.

The provided diagrams are intended to provide context to the AWS Solutions Architect conducting the technical validation. These diagrams will be referenced during the validation call. It is critical to provide clear diagrams with an appropriate level of detail in order to enable an efficient conversation about the product architecture with respect to the other technical requirements described in this document.

Each architecture diagram must show:

• The major elements of the architecture, and how they combine to provide the AWS Partner Solution to customers
• All of the AWS services used, using the appropriate AWS service icons.
• How the AWS services are deployed, including, VPCs, Availability Zones (AZs), subnets, and connections to systems outside of AWS.
• Includes elements deployed outside of AWS, e.g. on-premises components, or hardware devices.

Applies to: Customer Deployed

The deployment guide must provide best practices for deploying the AWS Partner Solution on AWS, and include all of the sections outlined in the AWS Foundational Technical Review Guide

Applies to: SaaS | Customer Deployed

All customer facing documentation and artifacts to deploy, operate and maintain the solution must meet the following standards:

• All references to AWS services must use the correct product names. (Please reference the official AWS product landing pages for the correct spelling and styling of product names)
• Documentation must be generally free of typos and grammatical errors.

Evidence: Provide a link to, or attach a copy of, your customer facing documentation.

Applies to: SaaS | Customer Deployed

AWS Partner has field-ready documentation and a seller toolkit that enable their field sellers to articulate a clear product value proposition to AWS customers.

Evidence must be in the form of sales collateral including a customer presentation, one-pager, and customer success stories.

Applies to: SaaS | Customer Deployed

If the solution is available via AWS Marketplace, AWS Partner must provide a link to the AWS Marketplace listing in their application.

An AWS Marketplace listing is not mandatory to achieve the AWS Competency.

Applies to: SaaS | Customer Deployed

Static AWS Access Keys are not used, except in the following cases:

1. Used by humans to access AWS services, and stored securely on a device controlled by that human.
2. Used by a service to access AWS services, but only in cases where all of the following are true:
   • It is not feasible to use an Amazon EC2 instance role, ECS Task role or similar mechanism
   • The AWS Access Keys are rotated at least weekly
   • Associated IAM Policies are tightly scoped such that it only allows access to specific actions and resources.
   • An IAM policy is in place that denies access to all actions except when requests originate from an IP address owned by the AWS Partner (see Example IAM Identity-Based Policies).

This requirement is applicable to internal processes running in the AWS Partner's AWS accounts as well as any components of the solution running in the customer's account. Customers must never be required to create IAM users in order to use any feature or capability of the solution. Partner-hosted or SaaS solutions must use cross-account IAM roles with external IDs to gain access to the customer's account. Customer deployed solutions must natively support using IAM roles associated with AWS compute resources.

Applies to: SaaS

All events from AWS Health with the service type "RISK" are handled within defined SLAs. APN Partner must implement all of the following:

• Define an SLA for security operations staff to evaluate and mitigate AWS Health RISK events.
• Implement automated integration with the primary internal issue tracking system such that a new issue is created for all events published by the AWS Health service with detail type "AWS Health Event" and service equal to "RISK". These events include "AWS_RISK_CREDENTIALS_COMPROMISED" and "AWS_RISK_CREDENTIALS_EXPOSED".
• Implement alerting and notification to security operations staff for both new issues created and SLA breeches.

Applies to: SaaS

All SaaS solutions must authenticate paid API or UI requests. The solution has specific technologies and mechanisms in place to mitigate the attacks described in the OWASP Top 10 Web Application Security Attacks

Applies to: SaaS

A hardened configuration based on CIS or equivalent benchmark has been defined for operating systems and containers used to host the solution. There are automated mechanisms in place to ensure this hardened configuration is applied to all compute resources (e.g., persisting the configuration to an immutable Amazon Machine Image [AMI]).

Applies to: SaaS

Protect your compute resources (e.g., instances, containers, functions, etc.) from unauthorized activity. This may include detecting any changes to the OS or application files and storing these changes in a durable location to allow for future forensic investigation. The mechanism employed for this purpose must at least:

• Detect any changes to the OS or application files in the compute resources used in the solution.
• Store data recording these changes in a durable location, external to the compute resources.

Applies to: SaaS | Customer Deployed

All cryptographic keys are encrypted at rest and in transit, and access to use the keys is controlled using an AWS solution such as AWS Key Management Service (AWS KMS) or an AWS Partner solution such as HashiCorp Vault.

Applies to: SaaS | Customer Deployed

A purpose-built ticketing system or similar tool is used by the security team to manage and track the ownership and resolution of any reported or detected security issues with the solution. This system should at a minimum track externally or internally reported security concerns about the solution. Additionally, for partner hosted/SaaS solutions, this system should track security operations tasks such as host patching as well as the analysis and remediation of any alerts raised by other detective measures.

This can be the same system used to fulfill OPE-005.

Applies to: SaaS

AWS CloudTrail logs are actively monitored using automated tools (e.g. Amazon GuardDuty) to detect suspicious or anomalous behavior, and alerting is configured to notify security operations staff in the event such behaviors are detected. Alerting must be integrated with the security ticketing system referenced in SECOPS-006.

Applies to: SaaS

Network activity for production VPCs is actively monitored using automated tools (e.g. Amazon GuardDuty) to detect suspicious or anomalous behavior, and alerting is configured to notify security operations staff in the event such behaviors are detected. Alerting must be integrated with the security ticketing system referenced in SECOPS-006.

Applies to: SaaS

The AWS Partner has defined a set of configuration policies for their AWS infrastructure (e.g. all Amazon Elastic Block Store (EBS) volumes must be encrypted), and these policies are enforced or monitored using automated tooling such as AWS Config or other comparable tools. This does not require automated remediation of policy violations, but detected violations must be automatically logged in a centralized ticketing system to ensure they are properly resolved by human operators.

Applies to: SaaS

Automated mechanisms are in place to prevent certificates from expiring unexpectedly.

Applies to: SaaS

Any components that are deployed to infrastructure shared across multiple tenants must model tenant identity in the software layer. The code for the component implements controls to ensure that tenants are isolated from one another and data is not leaked across tenant boundaries.

Applies to: SaaS

Any components which are not explicitly designed to isolate multiple tenants at the software layer are deployed to tenant-specific compute resources in tenant-specific VPCs. (In this context "tenant" refers to tenants of the AWS Partner solution.)

Supporting existing, legacy customers who may already be deployed to shared VPCs is acceptable as long as this model is completely deprecated and all new customers are deployed using a VPC-per-tenant model. There must be multiple active customers in production using the VPC-per-tenant model.

Applies to: SaaS

All single-tenant components must use dedicated IAM roles per tenant. These roles must limit access to only that tenant's resources using Resource and Condition statements within the attached policies.

Evidence must be provided in the form of a description of the roles used per tenant and an example of the IAM policies attached to those roles which provide explicit resource isolation.

Applies to: SaaS | Customer Deployed

The solution uses an official AWS SDK for all calls made to AWS API endpoints, or alternatively the solution implements retries with exponential backoff and jitter for all AWS API calls.

Applies to: SaaS | Customer Deployed

Measures are in place to rate limit calls to AWS APIs in the customer's account in order to prevent API throttling.

Applies to: SaaS | Customer Deployed

The solution uses methods such as ingesting AWS CloudTrail logs, consuming Amazon EventBridge events, or integrating with AWS Config to avoid polling AWS APIs wherever possible.

AWS Partner must describe what AWS resources state the solution tracks (if any), and how that resource state is discovered. In cases where there is no alternative to polling, AWS Partner must provide a detailed description of how they avoid causing API throttling issues.

Applies to: Customer Deployed

All components of the solution that are hosted in the customer's account support the ability for the customer to disable Instance Metadata Service Version 1 (IMDSv1). The solution must support the use of IAM roles associated with Amazon EC2 instances in conjunction with IMDSv2 for getting credentials to make calls to AWS APIs. Legacy support for static IAM access keys must be clearly deprecated.

AWS Partner must provide the specific version of the official AWS SDK used by the latest generally available version of the solution. The version of the AWS SDK in use must have been released after November 20th 2019.

If the solution accesses AWS APIs without using an official AWS SDK, AWS Partner must describe the steps taken to ensure IMDSv2 is supported.

Applies to: SaaS

Network connectivity to the solution must be highly available. If using VPN or Direct Connect to connect to customer networks, the solution must support redundant connections, even if the customers do not always implement this.

Applies to: SaaS

Logs and metrics for each component of the system are actively monitored. Critical metrics and thresholds are defined for each components and alarms are configured to notify operators in the event those thresholds are breached.

Evidence must be provided in the form of a list of the metrics and thresholds defined for each system component and a description of the tools used to surface alarms to operators.

Applies to: SaaS

All log information from the application, and from the AWS infrastructure, must be consolidated into a single system.

Applies to: SaaS

The application and the AWS infrastructure must be monitored centrally, with alarms generated and sent to the appropriate operations staff.

Applies to: SaaS

The solution must use an automated tool such as AWS CloudFormation or Terraform to provision and manage the AWS infrastructure. The AWS Management Console must not be used to make routine changes to the production AWS infrastructure.

Applies to: SaaS | Customer Deployed

Key Performance Indicators (KPIs) that indicate whether the product is performing as expected from the perspective of the customer have been defined. The definition of these KPIs must include how to measure the indicator as well as the acceptable threshold.

For example, p99 roundtrip response latency for API methods X, Y, and Z must be less than 200ms as measured by the TargetResponseTime AWS CloudWatch metric.

Partner must provide a list of KPIs and the defined acceptable thresholds as evidence.

Applies to: SaaS | Customer Deployed

Performance test suites (either automated or manual) are run before major releases or production deployments in order to ensure new versions meet defined performance targets.

Applies to: SaaS

The solution must use an automated method of deploying code and configuration to the AWS infrastructure. Interactive SSH or RDP sessions must not be used to deploy updates in the AWS infrastructure.

Applies to: SaaS

Runbooks must be developed to define the standard procedures used in response to different application and AWS events. An escalation process must be defined to deal with alerts and alarms generated by the system, and to respond to customer-reported incidents. The escalation process must also include escalating to AWS Support where appropriate.

Applies to: SaaS | Customer Deployed

Code is peer reviewed before being released or deployed to production.

Applies to: SaaS | Customer Deployed

Automated functional test suites exist and are automatically run before software is released or deployed to production.

Applies to: SaaS | Customer Deployed

A purpose-built issue tracking tool or set of tools is used to manage product backlogs, feature requests, defects and identified operational issues.

Applies to: SaaS | Customer Deployed

A process is established to track and remediate all customer reported product bugs as well as operational incidents that cause customer impact.