AWS SaaS Competency

AWS Partner has a company overview presentation that sets the stage for customer conversations about their AWS SaaS capabilities and showcases AWS Partner’s demonstration capabilities.

Presentation contains information about the AWS Partner’s AWS SaaS capabilities, including AWS specific differentiators, e.g., what is unique about the AWS Partner’s practice that can only be accomplished leveraging AWS.

Overview presentations contain:

• Company history
• Office locations
• Number of employees
• Customer profile, including number, size, and industries of customers
• Overview of SaaS practice
• Notable AWS projects

Please provide the following as evidence:

• Delivery of presentation by a business development executive at the beginning of the validation session. This should be limited to 15 minutes.

AWS Partner has internal mechanisms for maintaining their consultants' expertise on SaaS-related AWS services and tools.

Please provide the following as evidence:

• List of internal and/or external AWS-focused education events lead by AWS Partner staff (e.g. formal training, lunch and learns, meetups, user groups, etc.) in last 12 months.
• Resources provided by AWS Partner to staff for ongoing AWS skills development

AWS Partner must describe how SaaS opportunities are identified, how their sellers are trained to identify and sell those opportunities, and specific demand generation/lead generation efforts associated to their AWS SaaS practice.

Please provide the following as evidence:

• A description on how the AWS Partner engages with customers, their internal sellers, and AWS sellers if applicable.

AWS Partner must describe how and when they engage with AWS sellers and AWS Solutions Architects.

Please provide the following as evidence:

• A verbal description for how and when they engage AWS sellers or AWS Solutions Architects on an opportunity or in the form of a demonstration of the AWS Opportunity Management tool in AWS Partner Central with sales qualified opportunities submitted (sales qualified = budget, authority, need, timeline, and competition fields completed).

AWS Partner must have a process to ensure that there are sufficient SaaS trained personnel to effectively support customers.

Please provide the following as evidence:

• An established training plan including on-boarding processes that identify job roles (sellers, solutions architects, project managers) and required training paths
• A verbal description of methods used to allocate required resources to SaaS projects

AWS Partner has processes for working with customers to determine and define expected outcomes associated with the projects.

Please provide the following as evidence:

• Project deliverable templates or other resources used for project scoping and definition

AWS Partner has processes to determine scope of work with specific criteria defining customer project with expected deliverables.

Please provide the following as evidence:

• Project templates or other resources(e.g. RACI Matrix) used for project scoping and definition

AWS Partner has standard Statement of Work (SOW) templates for SaaS projects that can be customized to customer needs.

Please provide the following as evidence:

• Default SOW template

AWS Partner assigns Project Manager to each project to ensure project remains on time and within budget.

Please provide the following as evidence:

• Documentation to show that Project Managers were assigned to each of the 4 customer example projects.

AWS Partner has processes to document, manage, and respond to requests for changes to the project scope.

Please provide the following as evidence:

• Documentation of change management practices

AWS Partner has a customer acceptance process.

Please provide the following as evidence:

• Example customer training documents
• SOW language describing handoff responsibilities and acceptance criteria

AWS Partner implements customer satisfaction checkpoints as part of the project plan.

Please provide the following as evidence:

• Project plan and customer satisfaction results for milestone-defined checkpoints

AWS Partners must complete the SaaS Business Practice Plan Spreadsheet.

AWS Partners must have a SaaS Business Practice Plan to showcase their business capabilities and plans, including sales motions, Go-to-Market, and marketing activities. The objective of this requirement is to ensure alignment on priorities, motions, and investments between the AWS Partners and AWS. The goals in the spreadsheet will be used to plan joint activities, once the Partner is approved for AWS SaaS Competency. The goals are owned by the AWS Partner and supported by the AWS team.

AWS customers seeking builder or design services view AWS SaaS competency partners as the go-to experts in the field. Potential customers often ask for examples of solutions built for other customers when choosing a Partner and want confidence that consultants are up to date on AWS services. The following are a list of requirements for each corresponding SaaS competency category.

Category 1 – Builders

Partners must share at least 3 technical artifacts per customer example with AWS for software development projects implemented on Public Cloud within the last 18 months. Examples Include:

• Project Requirements Document
• Project Roadmap
• Backlog creation
• Project Status Updates
• Testing Documentation
• Technical Solution Documentation (API documentation)
• Product Documentation
• Project Plans
• Design Documents
• Architecture Diagrams (UML, Infrastructure)
• Retrospective, and Subsequent Planning Updates

Category 2 – Design Services Partners must share at least 3 technical artifacts per customer example with AWS for software development projects implemented on Public Cloud within the last 18 months. Examples Include:

• Project Requirements Document
• Project Roadmap
• Backlog creation
• Project Status Updates
• Testing Documentation
• Technical Solution Documentation (API documentation)
• Product Documentation
• Project Plans
• Design Documents
• Architecture Diagrams (UML, Infrastructure)
• Infrastructure Templates (CloudFormation, Terraform, Puppet, Chef)
• Retrospective, and Subsequent Planning Updates

AWS Partner must submit architecture diagrams depicting the overall design and deployment of its AWS Partner solution on AWS as well as any other relevant details of the solution for the specific customer in question.

The submitted diagrams are intended to provide context to the AWS Solutions Architect conducting the Technical Validation. It is critical to provide clear diagrams with an appropriate level of detail that enable the AWS Solutions Architect to validate the other requirements listed below.

Each architecture diagram must show:

• All of the AWS services used
• How the AWS services are deployed, including virtual private clouds (VPCs), availability zones, subnets, and connections to systems outside of AWS.
• Elements deployed outside of AWS, e.g. on-premises components, or hardware devices.
• how design scales automatically - Solution adapts to changes in demand. The architecture uses services that automatically scale such as Amazon S3, Amazon CloudFront, AWS Auto Scaling, and AWS Lambda.
• how design has high availability with multi-AZ or multi-region deployment. When intentional tradeoffs have been made (e.g. to optimize cost in favor of high availability), please explain the customer's requirements.

Please provide the following as evidence (required for all provided customer examples):

• An architecture diagram depicting the overall design and deployment of your solution on AWS.
• Explanation of how the major solutions elements will keep running in case of failure.
• Description of how the major solutions elements scale up automatically.

AWS expects all Services Partners to be prepared to create AWS accounts and implement basic security best practices. Even if most of your customer engagements do not require this, you should be prepared in the event you work with a customer who needs you to create new accounts for them.

Establish internal processes regarding how to create AWS accounts on behalf of customers when needed, including:

• When to use root account for workload activities
• Enable MFA on root
• Set the contact information to corporate email address or phone number
• Enable CloudTrail logs in all region and protect CloudTrail logs from accidental deletion with a dedicated S3 bucket

Please provide the following as evidence:

• Documents describing Security engagement SOPs which met all the 4 criteria defined above. Acceptable evidence types are security training documents, internal wikis, or standard operating procedures documents.
• Description of how Secure AWS Account Governance is implemented in one (1) of the submitted customer examples.

Define standard approach to access customer-owned AWS accounts, including:

• Both AWS Management Console access and programmatic access using the AWS Command Line Interface or other custom tools.
• When and how to use temporary credentials such as IAM roles
• Leverage customer's existing enterprise user identities and their credentials to access AWS services through Identity Federation or migrating to AWS Managed Active Directory

Establish best practices around AWS Identity and Access Management (IAM) and other identity and access management systems, including:

• IAM principals are only granted the minimum privileges necessary. Wildcards in Action and Resource elements should be avoided as much as possible.
• Every AWS Partner individual who accesses an AWS account must do so using dedicated credentials

Please provide the following as evidence:

• Security engagement Standard Operation Procedure (SOP) which met all the 2 criteria defined above. Acceptable evidence types are: security training documents, internal wikis, standard operating procedures documents. Written descriptions in the self-assessment excel is not acceptable.
• Description of how IAM best practices are implemented in one (1) of the submitted customer examples.

AWS Partner has defined metrics for determining the health of each component of the workload and provided the customer with guidance on how to detect operational events based on these metrics.

Establish the capability to run, monitor and improve operational procedure by:

• Defining, collecting and analyzing workload health metrics w/AWS services or 3rd Party tool
• Exporting standard application logs that capture errors and aid in troubleshooting and response to operational events.
• Defining threshold of operational metrics to generate alert for any issues

Please provide the following as evidence:

• Standardized documents or guidance on how to develop customer workload health KPIs with the three components above
• Description of how workload health KPIs are implemented in (1) of the submitted customer examples.

Create a runbook to document routine activities and guide issue resolution process with a list of operational tasks and troubleshooting scenarios covered that specifically addresses the KPI metrics defined in OPE-001.

Please provide the following as evidence:

• Standardized documents or runbook met the criteria defined above.

Deployments are tested or otherwise validated before being applied to the production environment. For example, DevOps pipelines used for the project for provisioning resources or releasing software and applications.

Use a consistent approach to deploy to customers including:

• A well-defined testing process before launching in production environment
• Automated testing components

Please provide the following as evidence:

• A deployment checklist example or written descriptions met all the criteria defined above.

Establish internal processes regarding how to secure traffic within VPC, including:

• Security Groups to restrict traffic between Internet and Amazon VPC
• Security Groups to restrict traffic within the Amazon VPC
• Network ACL to restrict inbound and outbound traffic
• Other AWS security services to protect network security

Please provide the following as evidence:

• Written descriptions/documents on network security best practices met the criteria defined above.
• Description of how network security is implementation in one (1) of the submitted customer examples.

Establish internal processes regarding a data encryption policy used across all customer projects

• Summary of any endpoints exposed to the Internet and how traffic is encrypted
• Summary of processes that make requests to external endpoints over the Internet and how traffic is encrypted
• Enforcing encryption at rest. By default you should enable the native encryption features in an AWS service that stores data unless there is a reason not to.

All cryptographic keys are stored and managed using a dedicated key management solution

Please provide the following as evidence:

• Data encryption and key management policy met the criteria defined above.
• Description of how data encryption is implementation in one (1) of the submitted customer examples.

Changes to infrastructure are automated for customer implementation

• Tools like AWS CloudFormation, the AWS CLI, or other scripting tools were used for automation.
• Changes to the production environment were not done using the AWS Management Console.

Please provide the following as evidence:

• Written description of deployment automation and an example template (e.g., CloudFormation templates, architecture diagram for CI/CD pipeline) met the criteria defined above.

Incorporate resilience discussion and advise a RTO&PRO target when engaging with customer. Customer acceptance and adoption on RTO/RPO is not required.

• Establish a process to establish workload resilience including:
• RTO & RPO target
• Explanation of the recovery process for the core components of the architecture
• Customer awareness and communication on this topic

Please provide the following as evidence:

• Descriptions or documents on workload resilience guidance met the three criteria defined above
• Description of how resilience is implementation in one (1) of the submitted customer examples including reasons for exception when RTO&RPO is not defined

Determine solution costs using right sizing and right pricing for both technical and business justification.

Conducted TCO analysis or other form of cost modelling to provide the customer with an understanding of the ongoing costs including all the following 3 areas:

• Description of the inputs used to estimate the cost of the solution
• Summary of the estimates or cost model provided to the customer before implementation
• Business value analysis or value stream mapping of AWS solution

Please provide the following as evidence:

• Description of how to develop cost analysis or modeling with the critical components defined above
• Cost analysis example in one (1) of the submitted customer examples. Acceptable evidence types are: price calculator link, reports or presentations on business values analysis

In order to demonstrate SaaS Competency partner proficiency within the corresponding category, Consulting Partners are required to provide the following technical artifacts.

All Categories

• Partners must share ≥ 2 materials on approach to practice management (such as proof of training, certification, institution accreditation, training materials, business process management documentation) related to the corresponding SaaS competency category to be shared under NDA with the AWS SaaS Competency review team.

Note: It is recommended that partners provide evidence of AWS Well Architected Reviews, conducted with their customers, for each of the submitted case studies. Once partner is approved for SaaS Competency, it is recommended that partner will complete (or facilitate) a Well Architected Review along with SaaS Lens for the AWS Well-Architected Framework for all customers engaged through the AWS SaaS competency to verify that AWS best practices have been sufficiently followed, or outlined for improvement.

Customer example provided must include the required context for which type of project (build or design) was delivered. customer example materials must indicate the type of work completed to allow an appropriate classification and competency evaluation for one of the corresponding categories.

Provide the project type and the use case each project addressed. Providing these project types will also allow the AWS field organization to sufficiently evangelize a SaaS partner's expertise specifically.

Project types and use cases:

Build

• Re-factor - Improve the readability, reusability, functionality, integration, or structure of an existing solution by consulting, re-writing, or modifying the application code.
• Integrate - Introduce a new critical application framework, dependency, or AWS Service within an existing solution by consulting, (re-)writing, and integrating the new solution within the application code.
• Re-design - A significant change to the majority of the components or services within an architecture which fundamentally changes the way the solution code is developed and deployed on AWS.
• Greenfield - The development of a net new cloud-native solution on AWS that will be sold as a product to customers.

Design

• Optimize - Improve the efficiency, performance, or cost of an existing solution by consulting, re-writing, or modifying the solutions architecture.
• Enhance - Enhance an existing architecture by integrating one or more major Infrastructure components within the production solution-design.
• Re-platform - A significant change to an existing solution by (consulting to a customer on) migrating and integrating two or more AWS services within the architecture or application code.
• Migrate - A migration of an existing product to a SaaS delivery model, or a migration of an existing SaaS solution to AWS.

Category 1 - Builders

• Partners must provide a detailed description indicating a minimum of 3 technical resources supporting each customer example.

Category 2 - Design Services

• Partners must provide a detailed description indicating a minimum of 3 technical resources supporting each customer example.

SaaS competency partners within the builders and design services categories are seen as the go-to experts within the field to design and build technology as a service (XaaS) solutions for customers. SaaS competency partners have a wealth of expertise in enabling vendors by assisting them to develop the appropriate technology as a service (XaaS) option within the customers vertical or horizontal segment.

All Categories

• SaaS competency partners for both categories must submit the self-service supportive documentation indicating the SaaS Technical Architecture patterns leveraged for the corresponding customer example provided.

Note: Supportive documentation is in the form of 1 or more sentence response to each question listed within the customer example SaaS Technical Approach found in section below.

The solution has security at every layer of the architecture to ensure an adequate tenant isolation boundary.

Security should be considered at all levels of an application's architecture. Understanding how each layer of the XaaS offering is secured will help to ensure that tenant isolation policies are followed as intended.

The solution offers the capability to scope access to features or product functionality.

SaaS Leveraging role-based access control (RBAC) enables XaaS developers to manage and limit access to various products, features, andfunctions within and across applications. Support for this model is often achieved via a centralized mechanism that manages and enforces these access policies. More advanced and more flexible offerings will support a richer set of options associating one or more roles with a user. Third-party solutions are also commonly used to simply and depth to a XaaS solutions RBAC capabilities.

The solution has measures in place to prevent cross-tenant access to system resources (infrastructure, data, etc.).

SaaS solutions can implement tenant isolation models in various ways. A silo model can be implemented by separating tenant resources inside their own AWS accounts or VPCs. A pooled model can be implemented by having multiple tenants share resources. Different models have various places where tenant data is processed, persisted, and transmitted (Compute, Storage, Database, Streams/Queues, Gateways/Proxies, Networks) within a XaaS application. The confidentiality, integrity, and availability (CIA) of a customer's data is of paramount importance.

AWS SaaS Competency requires that both pooled and silo-based tenant isolation models are represented as part of the four case studies. Each case study should identify the isolation models and clearly demonstrate the solution to enforce isolation.

The solution has measures in place to track, audit, and alert on cross-tenant events.

In order for XaaS solutions to meet the regulatory and compliance of the trusted authorities (such as SOC2), an organization must have the ability to log, audit, monitor, and alert on potential cross-tenant events that take place to indicate compliance with the regulatory standards.

The solution APIs have adequate security (authentication, authorization), and mitigation strategies for distributed denial of service (DDoS) attacks. XaaS applications typically provide Public and Private APIs for customers to perform operations for specific use cases. Understanding how a XaaS provider is exposing, and protecting its APIs to a tenant is important to assess the security posture of the XaaS solution.

The solution is meeting the required compliance standards for its customers.

Businesses must conform to the industry standards and policies of compliance organizations, and perform regular audits to retain compliance. Customers (tenants) of the XaaS solution must have guarantees in place to protect these businesses from the regulatory hurdles that they may encounter (subject to the customer agreements)

The solution is securing Tenant access to AWS Resources/Services, through either custom application logic, IAM cross account roles, Per-resource policies (S3 Bucket Policies), or STS credentials.

SaaS providers can provide access to resources, and services in AWS (such as DynamoDB, S3, and EC2) by leveraging fine grained permissions to AWS resources using various techniques.

The solution stores, manages, and version’s Tenant (customer) configuration, such that the SaaS provider (or the Tenant) can rollback changes when required.

Tenants may require infrastructure tuning or customization that is unique to their environment. This is common in situations where tenants are deployed with a siloed infrastructure. Supporting this model and maintaining agility requires a commitment to automating the management and deployment of these configuration options.

The solution has the ability to effectively and automatically respond to, and manage spikes in tenant activity.

A shared platform XaaS can easily be impacted by a single customer (tenant) event. This reality means that we must have robust set of mitigation policies in place to anticipate and react to these spikes in tenant activity. This scaling strategies can vary based on the architecture employed by your system. More robust solutions may support some notion of per-tenant throttling.

The solution has adequate monitoring in place, such that the SaaS provider may evaluate, and optimize individual tenant performance.

It is often useful to be able to monitor, and introduce optimizations that enhance the experience of a tenant without requiring this optimization to be applied to all tenants. These optimizations can be used as part of a tiering strategy, or as a general mechanism for selectively tuning the performance of the system and addressing tenant load profiles.

The solution has sufficient strategies and techniques in place to scale the database and mitigate database performance issues.

There are a variety of strategies used to scale databases used in a XaaS System. Leveraging a scalable approach for database architecture, will allow the SaaS provider to on-board and maintain considerable number of tenants without the risk of considerable performance degradation. (Examples include read replication, multi-master, multi-source, sharding, multiple clusters, Database caching, CQRS, Serverless DB).

The solution has sufficient measures in place to ensure adequate performance for all tenants of the system.

In a multi-tenant system, there are shared components which introduce the potential of performance degradation through the heavy consumers of the application. Having measures in place to react to heavy consumers (noisy neighbors) is important to ensure every customer has the service level they are promised.

The solution has techniques to define, and monitor tenant-specific limits. Understanding what service limits to define for tenants in a multi-tenant system is crucial to being able to scale the infrastructure, provide tier-based consumption, and volume-based discounts. These limits also correlate directly to the requirements defined in one's architecture to ensure an adequate customer experience.

The solution has sufficient techniques in place to manage the lifecycle of active and inactive tenants.

"as a Service" solutions should implement policies to manage onboarding and off-boarding of a tenant. The solution should have a predictable process to introduce new tenants into the system. The environments should also have specific strategies for managing tenants that are inactive. This usually includes some defined window of time where tenant data (application, configuration, users, etc.) and resources are maintained. These policies should also consider what happens to data for tenants that have been inactive for a prolonged period of time.

The solution has a well-defined process for assessing, triaging, and escalating tenant specific issues.

XaaS support requires policies and mechanisms that allow organizations to capture and escalate issues with a specific tenant context. With a shared environment, every attempt should be made to automate and streamline these processes. It's also valuable to have integration with AWS support, which may have added information and insight into customer issues.

The solution has sufficient strategies to mitigate scheduled downtime for deployment of new application features/versions.

Successful XaaS organizations are expected to provide customers with strictly upheld service level agreements (SLA) with zero planned downtime for the services provided. XaaS organizations should have no noticeable impact to planned changes to the application or infrastructure.

The solution is resilient to failure, and has the ability to recover without a noticeable customer impact.

Successful XaaS organizations are expected to have high availability architectures in place to ensure resiliency of applications to ensure an adequate service level agreement (SLA) for customers.

The solution has Continuous Integration and Deployment integrated within its Development or Infrastructure Deployment Pipeline.

Successful XaaS organizations rely heavily on their ability to continually release new features. Having an architecture and automation in place that supports this deployment automation enables organizations to react quickly to market dynamics and competitive pressure.

The solution is being delivered in a Business-to-Business (B2B) SaaS delivery model.

Organizations can deliver their SaaS solutions either in a Business-to-Business (B2B) or Business-to-Customer (B2C) SaaS model. AWS SaaS Competency requires that at least two case studies should be delivered as B2B SaaS. For B2B SaaS, describe the target customer or tenant profile to whom the solution is being primarily targeted towards, with approximate number of tenants utilizing the system.