VPC Lattice CRD Support In AWS Gateway API Controller

Introduction

The AWS Gateway API Controller currently streamlines the management of VPC Lattice service networks, services, listeners, and routes via standard Gateway API Custom Resource Definitions (CRDs). However, a gap exists in the Kubernetes-native management of VPC Lattice ResourceGateways and ResourceConfigurations. This article delves into the necessity of these CRDs, their benefits, and how their implementation enhances the integration of AWS services with private workloads within a Virtual Private Cloud (VPC).

The Need for VPC Lattice ResourceGateways and ResourceConfigurations CRDs

Currently, the AWS Gateway API Controller excels at managing various aspects of VPC Lattice, such as service networks, services, listeners, and routes, through standard Gateway API CRDs like GatewayClass, Gateway, and HTTPRoute. However, there's a notable absence: Kubernetes-native support for managing VPC Lattice ResourceGateways and ResourceConfigurations. These resources are crucial for integrating AWS services—such as EventBridge private API destinations, Lambda, API Gateway, and Step Functions—with private workloads residing within a VPC, often services running in EKS (Elastic Kubernetes Service).

The significance of VPC Lattice ResourceGateways and ResourceConfigurations lies in their ability to bridge the gap between AWS services and private workloads within a VPC. Without dedicated CRD support, managing these resources becomes cumbersome. Currently, users must resort to tools like CloudFormation, Terraform, AWS Cloud Development Kit (CDK), or even the AWS Management Console. This approach introduces several challenges. It disrupts GitOps workflows, where infrastructure is managed as code within Git repositories, and it adds dependencies on non-Kubernetes Infrastructure as Code (IaC) tools. To fully embrace the GitOps paradigm, it’s essential to manage all infrastructure components, including ResourceGateways and ResourceConfigurations, within Kubernetes itself.

To address these limitations, introducing Kubernetes CRDs for AWS::VpcLattice::ResourceGateway and AWS::VpcLattice::ResourceConfiguration becomes imperative. This enhancement would allow users to define these resources declaratively using YAML, aligning with the existing paradigm for service networks, services, and routes. The goal is to mirror the VPC Lattice APIs closely with these CRDs, providing a seamless and intuitive experience for users familiar with Kubernetes and AWS.

Benefits of CRD Support

Implementing CRDs for VPC Lattice ResourceGateways and ResourceConfigurations brings a multitude of benefits, primarily centered around enhanced management, improved workflow integration, and greater control over AWS service connectivity within Kubernetes environments.

Streamlined Management and GitOps Integration

One of the most significant advantages of introducing CRDs for ResourceGateways and ResourceConfigurations is the ability to manage these resources directly within Kubernetes. This aligns perfectly with GitOps principles, where infrastructure and application configurations are stored as code and managed through Git workflows. By defining ResourceGateways and ResourceConfigurations in YAML files, users can track changes, enforce version control, and automate deployments using familiar Kubernetes tools and processes. This eliminates the need for external tools like Terraform or CloudFormation, simplifying the overall infrastructure management process.

Enhanced Declarative Configuration

CRDs enable a declarative approach to infrastructure management. Instead of writing imperative scripts or using manual processes, users can define the desired state of their ResourceGateways and ResourceConfigurations in YAML files. Kubernetes then ensures that the actual state matches the desired state, automatically handling any necessary changes or updates. This declarative approach not only simplifies configuration but also reduces the risk of human error and inconsistencies.

Improved Integration with AWS Services

With CRD support, managing associations required for EventBridge private API destinations or other AWS service integrations becomes significantly easier. Users can define these associations directly within their Kubernetes manifests, ensuring that the necessary connections between AWS services and EKS workloads are established and maintained automatically. This tight integration simplifies the process of building and deploying complex, cloud-native applications that leverage various AWS services.

Simplified Workflow for Private AWS Service Connectivity into EKS

Ultimately, the introduction of CRDs for VPC Lattice ResourceGateways and ResourceConfigurations enables fully declarative GitOps workflows for private AWS service connectivity into EKS. This means that developers can define and manage all aspects of their application infrastructure, from the underlying network to the application services themselves, using a single, consistent set of tools and processes. This streamlined workflow reduces complexity, improves collaboration, and accelerates the delivery of new features and applications.

Key Features of Ideal CRD Support

To fully realize the benefits of CRD support for VPC Lattice, certain key features are essential. These features ensure that the CRDs are not only functional but also provide a seamless and intuitive experience for users.

1:1 Mapping with VPC Lattice APIs

Ideally, the CRDs should closely mirror the VPC Lattice ResourceGateway and ResourceConfiguration APIs. This means that each field and option available in the AWS APIs should have a corresponding representation in the CRDs. This 1:1 mapping simplifies the transition for users familiar with the AWS APIs and ensures that all available features are accessible through Kubernetes.

Status Reporting & Reconciliation

Robust status reporting and reconciliation are critical for ensuring that the desired state of the ResourceGateways and ResourceConfigurations is maintained. The controller should continuously monitor the state of these resources and automatically reconcile any differences between the desired and actual states. Detailed status reports should provide insights into the health and status of the resources, making it easier to troubleshoot issues and ensure that everything is working as expected.

Integration with IAM Roles for Service Accounts (IRSA)

Integration with IAM Roles for Service Accounts (IRSA) is essential for secure access to AWS resources. IRSA allows Kubernetes service accounts to assume IAM roles, providing fine-grained control over the permissions granted to pods running in the cluster. The controller should leverage IRSA to securely manage ResourceGateways and ResourceConfigurations, ensuring that only authorized pods can access these resources.

Ability to Reference VPC IDs, Subnet IDs, Security Groups, and Target Endpoints

The CRDs should provide the ability to reference existing AWS resources, such as VPC IDs, subnet IDs, security groups, and target endpoints. This allows users to integrate ResourceGateways and ResourceConfigurations with their existing infrastructure, avoiding the need to recreate resources or manage them in multiple places. This integration simplifies the overall infrastructure management process and reduces the risk of inconsistencies.

Ability to Manage Associations Required for EventBridge Private API Destinations or Other AWS Service Integrations

Finally, the CRDs should provide the ability to manage associations required for EventBridge private API destinations or other AWS service integrations. This includes defining the necessary permissions, routing rules, and other configurations required to connect AWS services to EKS workloads. This comprehensive management capability ensures that users can easily build and deploy complex, cloud-native applications that leverage the full power of the AWS ecosystem.

Addressing the Challenges with Terraform

Alternative infrastructure-as-code solutions like Terraform might seem viable for managing VPC Lattice resources. However, they often introduce complexities, particularly in environments that heavily rely on GitOps workflows. One significant challenge is the potential for circular dependencies, which can complicate the deployment process.

In a typical setup, infrastructure provisioning (VPCs, subnets, EKS clusters, etc.) is managed separately from application deployment (services, Ingress objects, etc.). Terraform is often used for the foundational infrastructure, while GitOps handles the application layer. The problem arises when creating VPC Lattice ResourceGateways and ResourceConfigurations in Terraform requires referencing resources created during the GitOps application deployment, such as Application Load Balancers (ALBs). This creates a dependency where Terraform must run after the GitOps cycle, resulting in a three-stage deployment pipeline instead of a simpler two-stage workflow.

This approach disrupts the clean separation between infrastructure and application layers, complicating the deployment lifecycle. The added complexity makes Terraform an impractical solution for managing these Lattice resources in environments where GitOps is the preferred method for application deployment.

Conclusion

Adding support for VPC Lattice ResourceGateway and ResourceConfiguration CRDs in the AWS Gateway API Controller is a crucial step toward enabling fully declarative GitOps workflows for private AWS service connectivity into EKS. By providing Kubernetes-native management capabilities for these resources, the controller empowers users to streamline their infrastructure management, improve integration with AWS services, and simplify the deployment of cloud-native applications. The benefits of this enhancement extend to improved management, enhanced declarative configuration, better integration with AWS services, and simplified workflows for private AWS service connectivity into EKS.

For further reading on AWS VPC Lattice and its integration with Kubernetes, consider exploring the official AWS documentation and related resources. You can find more information on AWS VPC Lattice here.