Tips and Tricks – Use AWS Lambda Layers for Shared Dependencies

After deploying hundreds of Lambda functions in production, I’ve learned that Lambda Layers are the difference between maintainable serverless architecture and dependency hell. This guide shows how to use Lambda Layers effectively for shared dependencies, reducing deployment package sizes and improving code reusability.

1. The Lambda Dependency Problem

Without Lambda Layers, every function packages its own dependencies:

• Large deployment packages: 50MB+ for functions with heavy dependencies
• Slow deployments: Upload 50MB for every code change
• Duplicate dependencies: 50 functions = 50 copies of same libraries
• Version consistency nightmare: Each function might use different library versions

2. What Are Lambda Layers?

Lambda Layers are ZIP archives containing libraries, custom runtimes, or other dependencies. Functions can reference up to 5 layers.

2.1 Layer Structure

# Correct layer structure for Python
my-layer/
└── python/
    └── lib/
        └── python3.11/
            └── site-packages/
                ├── requests/
                ├── boto3/
                └── pandas/

# For Node.js
nodejs/
└── node_modules/
    ├── axios/
    └── lodash/

3. Creating Lambda Layers

3.1 Python Layer with Docker

# Build dependencies in Lambda-compatible environment
docker run --rm -v $(pwd):/var/task   public.ecr.aws/lambda/python:3.11   pip install -r requirements.txt -t python/lib/python3.11/site-packages/

# Package layer
zip -r layer.zip python/

# Publish to AWS
aws lambda publish-layer-version   --layer-name my-dependencies   --zip-file fileb://layer.zip   --compatible-runtimes python3.11   --description "Common dependencies: requests, boto3, pandas"

3.2 Terraform for Layer Management

# lambda-layer.tf
resource "aws_lambda_layer_version" "dependencies" {
  filename            = "layer.zip"
  layer_name          = "common-dependencies"
  compatible_runtimes = ["python3.11"]
  source_code_hash    = filebase64sha256("layer.zip")
  
  description = "Common dependencies: requests==2.31.0, boto3==1.29.0"
}

# Grant access to layer
resource "aws_lambda_layer_version_permission" "layer_permission" {
  layer_name     = aws_lambda_layer_version.dependencies.layer_name
  version_number = aws_lambda_layer_version.dependencies.version
  statement_id   = "AllowAccountAccess"
  action         = "lambda:GetLayerVersion"
  principal      = "*"
  organization_id = "o-xxxxx"  # Restrict to your org
}

# Use in Lambda function
resource "aws_lambda_function" "my_function" {
  filename      = "function.zip"
  function_name = "my-function"
  role          = aws_iam_role.lambda_role.arn
  handler       = "index.handler"
  runtime       = "python3.11"
  
  layers = [aws_lambda_layer_version.dependencies.arn]
}

4. Production Patterns

4.1 Versioned Layers

# Semantic versioning for layers
aws lambda publish-layer-version   --layer-name pandas-layer   --zip-file fileb://pandas-2.1.4.zip   --compatible-runtimes python3.11   --description "pandas==2.1.4"

# Functions reference specific layer versions
# ARN: arn:aws:lambda:us-east-1:123456789:layer:pandas-layer:5

4.2 Multi-Layer Strategy

• Base layer: Common utilities (requests, boto3)
• Framework layer: FastAPI, Flask
• Data layer: pandas, numpy
• ML layer: scikit-learn, transformers
• Custom layer: Internal libraries

5. Best Practices

• Pin dependency versions: Reproducible builds
• Separate layers by update frequency: Stable dependencies in one layer, frequently updated in another
• Size limit aware: 250MB unzipped limit (all layers + function)
• Use Docker for builds: Ensures Lambda compatibility
• Version control layer code: Treat infrastructure as code

6. CI/CD Integration

# .github/workflows/deploy-layer.yml
name: Deploy Lambda Layer

on:
  push:
    paths:
      - 'layers/**'

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Build layer
        run: |
          docker run --rm -v $(pwd)/layers:/var/task             public.ecr.aws/lambda/python:3.11             pip install -r requirements.txt -t python/lib/python3.11/site-packages/
          
          cd layers && zip -r ../layer.zip python/
      
      - name: Publish layer
        run: |
          aws lambda publish-layer-version             --layer-name production-dependencies             --zip-file fileb://layer.zip             --compatible-runtimes python3.11

7. Conclusion

Lambda Layers reduce deployment sizes by 60-90% and ensure dependency consistency across functions. Essential for production serverless architectures.

Written for serverless developers and DevOps engineers deploying Lambda at scale.