Docker Issues

Docker-backed services (Lambda, RDS, ECS, EC2, ELBv2, ElastiCache, CloudFront, etc.) require SIMFRA_DOCKER=true and access to the Docker daemon.

Docker socket not mounted

Symptom: SIMFRA_DOCKER=true is set but Docker operations fail with "cannot connect to Docker daemon" or similar errors.

Solutions:

  1. When running Simfra in Docker, mount the Docker socket:

    docker run -v /var/run/docker.sock:/var/run/docker.sock \
      -e SIMFRA_DOCKER=true \
      ghcr.io/simfra-dev/simfra:latest
    
  2. On macOS with Docker Desktop, the socket is at /var/run/docker.sock by default. Verify it exists:

    ls -la /var/run/docker.sock
    
  3. On Linux, ensure the user running Simfra has permission to access the Docker socket (is in the docker group or running as root).


Image pull failures

Symptom: Container creation fails with image pull errors.

Solutions:

  1. Network access - Verify the Docker daemon can reach the image registry. Test with:

    docker pull amazonlinux:2023
    
  2. Private registry - If using a private registry, set SIMFRA_DOCKER_IMAGE_REGISTRY to the registry prefix:

    export SIMFRA_DOCKER_IMAGE_REGISTRY=ghcr.io/myorg
    
  3. Lambda images - Lambda runtime images are pulled from public.ecr.aws/lambda by default. Override with SIMFRA_LAMBDA_IMAGE_REGISTRY if your environment cannot reach ECR Public.

  4. Image tag - Override the default image tag for Simfra service containers with SIMFRA_DOCKER_IMAGE_TAG.

  5. Service-specific images - Individual service images can be overridden:

    • SIMFRA_RDS_IMAGE_MYSQL, SIMFRA_RDS_IMAGE_POSTGRES, SIMFRA_RDS_IMAGE_MARIADB
    • SIMFRA_ELASTICACHE_IMAGE_REDIS, SIMFRA_ELASTICACHE_IMAGE_VALKEY, SIMFRA_ELASTICACHE_IMAGE_MEMCACHED
    • SIMFRA_EC2_IMAGE_AMAZONLINUX2023, SIMFRA_EC2_IMAGE_UBUNTU
    • SIMFRA_BEDROCK_OLLAMA_IMAGE

Container startup failures

Symptom: Docker containers are created but fail to start or crash immediately.

Solutions:

  1. Check container logs:

    # Via admin API
    curl http://localhost:4599/_simfra/docker/containers
    
    # Get logs for a specific container
    curl http://localhost:4599/_simfra/docker/containers/{id}/logs
    
  2. Check Simfra logs - Set SIMFRA_LOG_LEVEL=debug for detailed container lifecycle logging.

  3. Verify the image works standalone:

    docker run --rm postgres:16 postgres --version
    
  4. Check resource limits - Docker may have insufficient memory or CPU configured. On Docker Desktop, check Settings > Resources.


Port conflicts

Symptom: Container creation fails with "port is already allocated" or "bind: address already in use".

Each Docker-backed service allocates host ports from a configurable range. Conflicts occur when multiple services or external applications use the same ports.

Solutions:

  1. Customize port ranges - Move conflicting services to different ranges:

    export SIMFRA_RDS_PORT_RANGE=20400-20499
    export SIMFRA_ELBV2_PORT_RANGE=20200-20299
    

    See Port Ranges for the full list of defaults.

  2. Check what is using the port:

    lsof -i :10400
    
  3. Widen the range if you need more concurrent containers:

    export SIMFRA_RDS_PORT_RANGE=10400-10599
    
  4. Use random ports by setting the range to 0:

    export SIMFRA_RDS_PORT_RANGE=0
    

    Note: random ports are harder to predict and may complicate client configuration.


Docker network address pool exhausted

Symptom: VPC creation or container startup fails with "all predefined address pools have been fully subnetted" or "could not find an available, non-overlapping IPv4 address pool among the defaults to assign to the network".

Cause: Docker Desktop's default address pool only supports approximately 30 bridge networks (using /24 subnets from the 172.17-31.x.x range). Simfra creates one Docker bridge network per VPC. When bootstrapping with SIMFRA_BOOTSTRAP=standard, default VPCs are created across 16 regions per account. With two bootstrapped accounts, that is 32 VPC networks before any user-created VPCs, EKS Kind clusters, or other Docker networks. This exhausts the default pool.

Fix: Configure Docker Desktop with a wider default-address-pools in ~/.docker/daemon.json:

{
  "default-address-pools": [
    { "base": "10.200.0.0/14", "size": 24 },
    { "base": "10.204.0.0/14", "size": 24 }
  ]
}

This provides approximately 2000 /24 subnets using the 10.200.0.0-10.207.255.255 range, which does not overlap with Docker's default bridge network (172.17.0.0/16) or common VPC CIDR blocks.

After editing daemon.json, restart Docker Desktop for the change to take effect.

This is primarily a Docker Desktop issue on macOS and Windows. Linux Docker installations with custom daemon configuration, or alternative Docker runtimes like Colima, may already have wider address pools configured.


VPC network issues

Symptom: Containers cannot communicate with each other or with Simfra. Private resources are unreachable.

Solutions:

  1. Check Docker network exists - Simfra creates Docker bridge networks named simfra-vpc-{accountId}-{vpcId} for VPC isolation. List them with:

    curl http://localhost:4599/_simfra/docker/networks
    
  2. Verify container connectivity - Containers on the same VPC network should be able to reach each other. Check with:

    docker exec <container_id> ping <target_ip>
    
  3. VPC isolation mode - When SIMFRA_VPC_ISOLATION=true (the default when Docker is enabled), private containers do not publish ports to the host. Use port forwarding to access them:

    curl -X POST http://localhost:4599/_simfra/port-forwards \
      -d '{"targetArn": "arn:aws:rds:us-east-1:000000000000:db:mydb"}'
    
  4. DNS resolution - Containers use Simfra's DNS server for name resolution. If DNS is not working, check the DNS container is running:

    curl http://localhost:4599/_simfra/dns/000000000000
    

Containers not starting

Symptom: Simfra acknowledges resource creation but the backing container never starts.

Solutions:

  1. Ensure Docker has enough resources - Check available disk space, memory, and CPU. Docker Desktop on macOS defaults to limited resources.

  2. Check disk space - Docker images and containers consume disk. Clean unused resources:

    docker system prune
    
  3. Cleanup stale containers - Previous Simfra sessions may have left containers. Clean up via the admin API:

    curl -X POST http://localhost:4599/_simfra/docker/cleanup
    

    Or set SIMFRA_DOCKER_CLEANUP_ON_START=true (the default) to clean up automatically on startup.

  4. Docker daemon health - Verify Docker is responsive:

    docker info