EKS Kubernetes

With Docker enabled, EKS creates real Kubernetes clusters using Kind (Kubernetes in Docker). You get a working Kubernetes API server, can deploy workloads with kubectl, and use EKS features like node groups, IRSA, and Pod Identity.

Prerequisites

  • SIMFRA_DOCKER=true
  • Docker must support running containers that themselves run containers (Docker-in-Docker). Kind nodes are Docker containers that run kubelet and container workloads inside them.

Creating a Cluster

aws --endpoint-url http://localhost:4599 eks create-cluster \
  --name my-cluster \
  --role-arn arn:aws:iam::000000000000:role/eks-cluster-role \
  --resources-vpc-config subnetIds=subnet-xxx,securityGroupIds=sg-xxx

The cluster starts in CREATING state. A background worker creates the Kind cluster and transitions it to ACTIVE once the Kubernetes API server is ready. This takes 10-30 seconds depending on your machine.

Simfra validates:

  • The cluster role exists and the caller has iam:PassRole permission.
  • The referenced subnets and security groups exist.
  • The Kubernetes version is valid (defaults to the latest supported version).

Getting Kubeconfig

Once the cluster is ACTIVE, configure kubectl:

aws --endpoint-url http://localhost:4599 eks update-kubeconfig --name my-cluster

This writes a kubeconfig entry for the cluster. Then:

kubectl get nodes

NAME                        STATUS   ROLES           AGE   VERSION
simfra-eks-my-cluster-cp    Ready    control-plane   30s   v1.31.0

Node Groups

Create managed node groups to add worker nodes. Each node becomes an additional Kind container:

aws --endpoint-url http://localhost:4599 eks create-nodegroup \
  --cluster-name my-cluster \
  --nodegroup-name my-nodes \
  --scaling-config minSize=1,maxSize=3,desiredSize=2 \
  --node-role arn:aws:iam::000000000000:role/eks-node-role \
  --subnets subnet-xxx

kubectl get nodes

NAME                              STATUS   ROLES           AGE   VERSION
simfra-eks-my-cluster-cp          Ready    control-plane   2m    v1.31.0
simfra-eks-my-cluster-worker-0    Ready    <none>          10s   v1.31.0
simfra-eks-my-cluster-worker-1    Ready    <none>          10s   v1.31.0

Fargate Profiles

Fargate profiles are supported for serverless pod execution:

aws --endpoint-url http://localhost:4599 eks create-fargate-profile \
  --cluster-name my-cluster \
  --fargate-profile-name my-fargate \
  --pod-execution-role-arn arn:aws:iam::000000000000:role/fargate-role \
  --subnets subnet-xxx \
  --selectors namespace=default

ECR Image Pulls from Pods

Pods running in Kind clusters can pull images from Simfra's ECR. The Kind nodes are configured to use Simfra's DNS, so ECR registry URLs resolve correctly. Combined with ECR credential configuration, docker pull from within pods works against the local ECR registry.

IAM Roles for Service Accounts (IRSA)

IRSA allows Kubernetes service accounts to assume IAM roles. Simfra deploys a mutating admission webhook into each Kind cluster that injects AWS credentials into pods based on their service account annotations:

  1. Create an OIDC identity provider:
OIDC_URL=$(aws --endpoint-url http://localhost:4599 eks describe-cluster \
  --name my-cluster --query 'cluster.identity.oidc.issuer' --output text)

aws --endpoint-url http://localhost:4599 iam create-open-id-connect-provider \
  --url $OIDC_URL \
  --client-id-list sts.amazonaws.com

  1. Create an IAM role with a trust policy referencing the OIDC provider.

  2. Annotate your Kubernetes service account:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-sa
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::000000000000:role/my-pod-role

Pods using this service account receive AWS_ROLE_ARN and AWS_WEB_IDENTITY_TOKEN_FILE environment variables, and the projected token is valid for STS AssumeRoleWithWebIdentity.

Pod Identity

EKS Pod Identity associations map service accounts to IAM roles without OIDC providers:

aws --endpoint-url http://localhost:4599 eks create-pod-identity-association \
  --cluster-name my-cluster \
  --namespace default \
  --service-account my-sa \
  --role-arn arn:aws:iam::000000000000:role/my-pod-role

Add-ons

EKS add-ons are supported:

aws --endpoint-url http://localhost:4599 eks create-addon \
  --cluster-name my-cluster \
  --addon-name vpc-cni \
  --addon-version v1.18.0-eksbuild.1

Access Entries

Cluster access entries control who can authenticate to the Kubernetes API:

aws --endpoint-url http://localhost:4599 eks create-access-entry \
  --cluster-name my-cluster \
  --principal-arn arn:aws:iam::000000000000:role/developer

State Machine

Resource States
Cluster CREATING -> ACTIVE -> DELETING -> (removed)
Node Group CREATING -> ACTIVE -> DELETING -> (removed)
Fargate Profile CREATING -> ACTIVE -> DELETING -> (removed)
Add-on CREATING -> ACTIVE -> DELETING -> (removed)

VPC Network Isolation

When SIMFRA_VPC_ISOLATION=true, Kind cluster nodes are attached to the VPC Docker network for the subnets specified during cluster creation. Pods can communicate with other containers on the same VPC network (RDS, ElastiCache, etc.) by DNS name.

Cluster Limits

The default quota is 100 clusters per account, matching the AWS default.

Next Steps