Testing Serverless Applications: A Practical Strategy Guide

The Testing Challenge

Serverless applications compress the deploy cycle to minutes, which changes the economics of testing: the bug that used to be caught by a long release process now reaches production before a human reviews it. The testing strategy has to catch what the deploy cadence no longer will. At the same time, serverless architecture itself breaks local-first testing; Lambda cold starts, IAM permissions, event schemas, and the boundaries between managed services all behave differently locally than in a live account.

This post covers a testing strategy for serverless applications on AWS. It covers the layered approach (unit, integration with LocalStack versus a live account, contract, end-to-end), the specific failure modes that mocks hide, and the CI patterns that keep the test suite proportional to the deploy speed.

The false confidence problem: Tests pass locally with mocked AWS SDK calls, then production fails due to IAM permissions or incorrect event structures.

The slow feedback loop: Waiting several minutes for CloudFormation deployments just to test a simple Lambda change.

The LocalStack gap: Tests pass against LocalStack but fail in real AWS due to API differences.

The async complexity: EventBridge and Step Functions introduce asynchronous behavior that's difficult to test reliably.

This post shares practical patterns for testing serverless applications that balance fast feedback with production confidence.

The Serverless Testing Pyramid

The traditional testing pyramid applies to serverless, but with adjusted proportions and techniques:

Unit tests (70%): Fast, no AWS calls, test business logic in isolation. Run on every commit.

Integration tests (20%): Test service integrations with real or local AWS services. Run on pull requests.

E2E tests (10%): Full workflow testing in cloud environment. Run on main branch deployments.

Here's what works in practice:

When to Use Each Testing Level

Unit tests catch:

Business logic errors
Input validation issues
Data transformation bugs
Error handling gaps

Integration tests catch:

IAM permission issues
Service configuration problems
Event structure mismatches
Timeout scenarios

E2E tests catch:

Multi-service orchestration issues
Cross-account routing problems
Production configuration drift
Real-world performance issues

Unit Testing Lambda Functions

The key to effective unit testing is separating your handler from business logic:

typescript

// Bad: Everything in the handlerexport const handler = async (event: APIGatewayProxyEvent) => {  // Business logic mixed with handler code  const body = JSON.parse(event.body || '{}');  const discount = body.amount > 100 ? 0.1 : 0;  const total = body.amount * (1 - discount);
  return {    statusCode: 200,    body: JSON.stringify({ total })  };};
// Good: Separate concernsexport const calculateTotal = (amount: number): number => {  const discount = amount > 100 ? 0.1 : 0;  return amount * (1 - discount);};
export const handler = async (event: APIGatewayProxyEvent) => {  const body = JSON.parse(event.body || '{}');  const total = calculateTotal(body.amount);
  return {    statusCode: 200,    body: JSON.stringify({ total })  };};

// Bad: Everything in the handlerexport const handler = async (event: APIGatewayProxyEvent) => {  // Business logic mixed with handler code  const body = JSON.parse(event.body || '{}');  const discount = body.amount > 100 ? 0.1 : 0;  const total = body.amount * (1 - discount);
  return {    statusCode: 200,    body: JSON.stringify({ total })  };};
// Good: Separate concernsexport const calculateTotal = (amount: number): number => {  const discount = amount > 100 ? 0.1 : 0;  return amount * (1 - discount);};
export const handler = async (event: APIGatewayProxyEvent) => {  const body = JSON.parse(event.body || '{}');  const total = calculateTotal(body.amount);
  return {    statusCode: 200,    body: JSON.stringify({ total })  };};

Now you can test calculateTotal without mocking API Gateway events.

Testing with AWS SDK v3

Here's a practical example testing a Lambda that reads from DynamoDB:

typescript

// user-handler.tsimport { DynamoDBDocumentClient, GetCommand } from '@aws-sdk/lib-dynamodb';import { DynamoDBClient } from '@aws-sdk/client-dynamodb';import { APIGatewayProxyEvent, APIGatewayProxyResult } from 'aws-lambda';
const client = new DynamoDBClient({});const ddb = DynamoDBDocumentClient.from(client);
export const getUser = async (userId: string) => {  const result = await ddb.send(new GetCommand({    TableName: process.env.TABLE_NAME,    Key: { PK: `USER#${userId}` }  }));
  if (!result.Item) {    throw new Error('User not found');  }
  return result.Item;};
export const handler = async (event: APIGatewayProxyEvent): Promise<APIGatewayProxyResult> => {  try {    const userId = event.pathParameters?.id;    if (!userId) {      return { statusCode: 400, body: JSON.stringify({ error: 'Missing user ID' }) };    }
    const user = await getUser(userId);
    return {      statusCode: 200,      body: JSON.stringify(user)    };  } catch (error) {    return {      statusCode: 404,      body: JSON.stringify({ error: (error as Error).message })    };  }};

// user-handler.tsimport { DynamoDBDocumentClient, GetCommand } from '@aws-sdk/lib-dynamodb';import { DynamoDBClient } from '@aws-sdk/client-dynamodb';import { APIGatewayProxyEvent, APIGatewayProxyResult } from 'aws-lambda';
const client = new DynamoDBClient({});const ddb = DynamoDBDocumentClient.from(client);
export const getUser = async (userId: string) => {  const result = await ddb.send(new GetCommand({    TableName: process.env.TABLE_NAME,    Key: { PK: `USER#${userId}` }  }));
  if (!result.Item) {    throw new Error('User not found');  }
  return result.Item;};
export const handler = async (event: APIGatewayProxyEvent): Promise<APIGatewayProxyResult> => {  try {    const userId = event.pathParameters?.id;    if (!userId) {      return { statusCode: 400, body: JSON.stringify({ error: 'Missing user ID' }) };    }
    const user = await getUser(userId);
    return {      statusCode: 200,      body: JSON.stringify(user)    };  } catch (error) {    return {      statusCode: 404,      body: JSON.stringify({ error: (error as Error).message })    };  }};

Test this with aws-sdk-client-mock:

typescript

// user-handler.test.tsimport { mockClient } from 'aws-sdk-client-mock';import { DynamoDBDocumentClient, GetCommand } from '@aws-sdk/lib-dynamodb';import { APIGatewayProxyEvent } from 'aws-lambda';import { handler, getUser } from './user-handler';
const ddbMock = mockClient(DynamoDBDocumentClient);
beforeEach(() => {  ddbMock.reset();  process.env.TABLE_NAME = 'users';});
describe('getUser', () => {  it('returns user when found', async () => {    ddbMock.on(GetCommand).resolves({      Item: { PK: 'USER#123', name: 'John', email: '[email protected]' }    });
    const user = await getUser('123');
    expect(user).toEqual({      PK: 'USER#123',      name: 'John',      email: '[email protected]'    });  });
  it('throws when user not found', async () => {    ddbMock.on(GetCommand).resolves({ Item: undefined });
    await expect(getUser('999')).rejects.toThrow('User not found');  });
  it('calls DynamoDB with correct parameters', async () => {    ddbMock.on(GetCommand).resolves({ Item: { PK: 'USER#123' } });
    await getUser('123');
    expect(ddbMock.calls()[0].args[0].input).toEqual({      TableName: 'users',      Key: { PK: 'USER#123' }    });  });});
describe('handler', () => {  it('returns 400 when userId is missing', async () => {    const event = createApiGatewayEvent({ pathParameters: null });
    const result = await handler(event);
    expect(result.statusCode).toBe(400);    expect(JSON.parse(result.body)).toEqual({ error: 'Missing user ID' });  });
  it('returns user when found', async () => {    ddbMock.on(GetCommand).resolves({      Item: { PK: 'USER#123', name: 'John' }    });    const event = createApiGatewayEvent({ pathParameters: { id: '123' } });
    const result = await handler(event);
    expect(result.statusCode).toBe(200);    expect(JSON.parse(result.body)).toEqual({ PK: 'USER#123', name: 'John' });  });});
// Helper function for creating test eventsfunction createApiGatewayEvent(overrides?: Partial<APIGatewayProxyEvent>): APIGatewayProxyEvent {  return {    body: null,    headers: {},    multiValueHeaders: {},    httpMethod: 'GET',    isBase64Encoded: false,    path: '/users',    pathParameters: null,    queryStringParameters: null,    multiValueQueryStringParameters: null,    stageVariables: null,    requestContext: {      accountId: '123456789012',      apiId: 'test-api',      protocol: 'HTTP/1.1',      httpMethod: 'GET',      path: '/users',      stage: 'test',      requestId: 'test-request',      requestTimeEpoch: Date.now(),      resourceId: 'test-resource',      resourcePath: '/users',      identity: {        sourceIp: '127.0.0.1',        userAgent: 'test-agent',        accessKey: null,        accountId: null,        apiKey: null,        apiKeyId: null,        caller: null,        clientCert: null,        cognitoAuthenticationProvider: null,        cognitoAuthenticationType: null,        cognitoIdentityId: null,        cognitoIdentityPoolId: null,        principalOrgId: null,        user: null,        userArn: null      },      authorizer: null    },    resource: '/users',    ...overrides  } as APIGatewayProxyEvent;}

// user-handler.test.tsimport { mockClient } from 'aws-sdk-client-mock';import { DynamoDBDocumentClient, GetCommand } from '@aws-sdk/lib-dynamodb';import { APIGatewayProxyEvent } from 'aws-lambda';import { handler, getUser } from './user-handler';
const ddbMock = mockClient(DynamoDBDocumentClient);
beforeEach(() => {  ddbMock.reset();  process.env.TABLE_NAME = 'users';});
describe('getUser', () => {  it('returns user when found', async () => {    ddbMock.on(GetCommand).resolves({      Item: { PK: 'USER#123', name: 'John', email: '[email protected]' }    });
    const user = await getUser('123');
    expect(user).toEqual({      PK: 'USER#123',      name: 'John',      email: '[email protected]'    });  });
  it('throws when user not found', async () => {    ddbMock.on(GetCommand).resolves({ Item: undefined });
    await expect(getUser('999')).rejects.toThrow('User not found');  });
  it('calls DynamoDB with correct parameters', async () => {    ddbMock.on(GetCommand).resolves({ Item: { PK: 'USER#123' } });
    await getUser('123');
    expect(ddbMock.calls()[0].args[0].input).toEqual({      TableName: 'users',      Key: { PK: 'USER#123' }    });  });});
describe('handler', () => {  it('returns 400 when userId is missing', async () => {    const event = createApiGatewayEvent({ pathParameters: null });
    const result = await handler(event);
    expect(result.statusCode).toBe(400);    expect(JSON.parse(result.body)).toEqual({ error: 'Missing user ID' });  });
  it('returns user when found', async () => {    ddbMock.on(GetCommand).resolves({      Item: { PK: 'USER#123', name: 'John' }    });    const event = createApiGatewayEvent({ pathParameters: { id: '123' } });
    const result = await handler(event);
    expect(result.statusCode).toBe(200);    expect(JSON.parse(result.body)).toEqual({ PK: 'USER#123', name: 'John' });  });});
// Helper function for creating test eventsfunction createApiGatewayEvent(overrides?: Partial<APIGatewayProxyEvent>): APIGatewayProxyEvent {  return {    body: null,    headers: {},    multiValueHeaders: {},    httpMethod: 'GET',    isBase64Encoded: false,    path: '/users',    pathParameters: null,    queryStringParameters: null,    multiValueQueryStringParameters: null,    stageVariables: null,    requestContext: {      accountId: '123456789012',      apiId: 'test-api',      protocol: 'HTTP/1.1',      httpMethod: 'GET',      path: '/users',      stage: 'test',      requestId: 'test-request',      requestTimeEpoch: Date.now(),      resourceId: 'test-resource',      resourcePath: '/users',      identity: {        sourceIp: '127.0.0.1',        userAgent: 'test-agent',        accessKey: null,        accountId: null,        apiKey: null,        apiKeyId: null,        caller: null,        clientCert: null,        cognitoAuthenticationProvider: null,        cognitoAuthenticationType: null,        cognitoIdentityId: null,        cognitoIdentityPoolId: null,        principalOrgId: null,        user: null,        userArn: null      },      authorizer: null    },    resource: '/users',    ...overrides  } as APIGatewayProxyEvent;}

Key patterns here:

Mock only external dependencies (DynamoDB), not business logic
Test the business function (getUser) separately from the handler
Verify actual AWS SDK call parameters, not just return values
Create helper functions for test event construction
Reset mocks in beforeEach to avoid test pollution

Integration Testing Strategies

Unit tests give fast feedback but can't catch integration issues. Here's when integration testing becomes critical.

Testing with Real DynamoDB

For important operations, test against real DynamoDB:

typescript

// user-repository.integration.test.tsimport { DynamoDBClient } from '@aws-sdk/client-dynamodb';import { DynamoDBDocumentClient, PutCommand, GetCommand, QueryCommand, DeleteCommand } from '@aws-sdk/lib-dynamodb';import { randomUUID } from 'crypto';
const client = new DynamoDBClient({  region: process.env.AWS_REGION || 'us-east-1'});const ddb = DynamoDBDocumentClient.from(client);
// Use unique table name per test run to allow parallel executionconst TABLE_NAME = `users-test-${Date.now()}`;const testUserIds: string[] = [];
beforeAll(async () => {  // In real setup, create table with AWS CDK or CloudFormation  // Here we assume table exists});
afterEach(async () => {  // Clean up test data  await Promise.all(    testUserIds.map(id =>      ddb.send(new DeleteCommand({        TableName: TABLE_NAME,        Key: { PK: `USER#${id}` }      }))    )  );  testUserIds.length = 0;});
describe('DynamoDB Integration Tests', () => {  it('creates and retrieves user', async () => {    const userId = randomUUID();    testUserIds.push(userId);
    // Create user    await ddb.send(new PutCommand({      TableName: TABLE_NAME,      Item: {        PK: `USER#${userId}`,        name: 'John Doe',        email: '[email protected]',        createdAt: new Date().toISOString()      }    }));
    // Retrieve user    const result = await ddb.send(new GetCommand({      TableName: TABLE_NAME,      Key: { PK: `USER#${userId}` }    }));
    expect(result.Item).toMatchObject({      PK: `USER#${userId}`,      name: 'John Doe',      email: '[email protected]'    });  });
  it('queries users by email GSI', async () => {    const userId = randomUUID();    testUserIds.push(userId);    const email = `test-${userId}@example.com`;
    await ddb.send(new PutCommand({      TableName: TABLE_NAME,      Item: {        PK: `USER#${userId}`,        email: email,        name: 'Test User'      }    }));
    // Query by email using GSI    const result = await ddb.send(new QueryCommand({      TableName: TABLE_NAME,      IndexName: 'EmailIndex',      KeyConditionExpression: 'email = :email',      ExpressionAttributeValues: { ':email': email }    }));
    expect(result.Items).toHaveLength(1);    expect(result.Items?.[0].PK).toBe(`USER#${userId}`);  });});

// user-repository.integration.test.tsimport { DynamoDBClient } from '@aws-sdk/client-dynamodb';import { DynamoDBDocumentClient, PutCommand, GetCommand, QueryCommand, DeleteCommand } from '@aws-sdk/lib-dynamodb';import { randomUUID } from 'crypto';
const client = new DynamoDBClient({  region: process.env.AWS_REGION || 'us-east-1'});const ddb = DynamoDBDocumentClient.from(client);
// Use unique table name per test run to allow parallel executionconst TABLE_NAME = `users-test-${Date.now()}`;const testUserIds: string[] = [];
beforeAll(async () => {  // In real setup, create table with AWS CDK or CloudFormation  // Here we assume table exists});
afterEach(async () => {  // Clean up test data  await Promise.all(    testUserIds.map(id =>      ddb.send(new DeleteCommand({        TableName: TABLE_NAME,        Key: { PK: `USER#${id}` }      }))    )  );  testUserIds.length = 0;});
describe('DynamoDB Integration Tests', () => {  it('creates and retrieves user', async () => {    const userId = randomUUID();    testUserIds.push(userId);
    // Create user    await ddb.send(new PutCommand({      TableName: TABLE_NAME,      Item: {        PK: `USER#${userId}`,        name: 'John Doe',        email: '[email protected]',        createdAt: new Date().toISOString()      }    }));
    // Retrieve user    const result = await ddb.send(new GetCommand({      TableName: TABLE_NAME,      Key: { PK: `USER#${userId}` }    }));
    expect(result.Item).toMatchObject({      PK: `USER#${userId}`,      name: 'John Doe',      email: '[email protected]'    });  });
  it('queries users by email GSI', async () => {    const userId = randomUUID();    testUserIds.push(userId);    const email = `test-${userId}@example.com`;
    await ddb.send(new PutCommand({      TableName: TABLE_NAME,      Item: {        PK: `USER#${userId}`,        email: email,        name: 'Test User'      }    }));
    // Query by email using GSI    const result = await ddb.send(new QueryCommand({      TableName: TABLE_NAME,      IndexName: 'EmailIndex',      KeyConditionExpression: 'email = :email',      ExpressionAttributeValues: { ':email': email }    }));
    expect(result.Items).toHaveLength(1);    expect(result.Items?.[0].PK).toBe(`USER#${userId}`);  });});

Why this matters: This test catches real issues like:

IAM permissions (if your test role lacks permissions, this fails)
GSI configuration problems
Conditional write conflicts
DynamoDB API changes

Tradeoffs: Slower than unit tests (2-5 seconds vs 10ms), costs a few cents per month.

LocalStack vs Real AWS: When to Use Each

Note: LocalStack has known issues with Step Functions and EventBridge integration. For testing workflows involving these services, use real AWS or be prepared for behavior differences.

Here's a decision framework I use:

Use LocalStack for:

Simple DynamoDB operations (GET, PUT, QUERY)
Basic S3 operations
SQS message handling
Rapid iteration during development

Use Real AWS for:

IAM permission validation
Step Functions orchestration
EventBridge event routing
Complex DynamoDB streams
Pre-deployment validation

Testing API Gateway Integrations

API Gateway events have complex structures. Here's how to test them properly:

typescript

// api-integration.test.tsimport { APIGatewayProxyEvent } from 'aws-lambda';import { handler } from './api-handler';
describe('API Gateway Integration', () => {  it('handles POST request with JSON body', async () => {    const event: APIGatewayProxyEvent = {      httpMethod: 'POST',      path: '/users',      headers: {        'content-type': 'application/json'      },      body: JSON.stringify({        name: 'John Doe',        email: '[email protected]'      }),      // ... other required fields    } as APIGatewayProxyEvent;
    const result = await handler(event);
    expect(result.statusCode).toBe(201);    expect(result.headers).toMatchObject({      'content-type': 'application/json',      'access-control-allow-origin': '*' // CORS    });  });
  it('validates CORS headers', async () => {    const event = createApiGatewayEvent({      httpMethod: 'OPTIONS',      headers: {        'origin': 'https://example.com',        'access-control-request-method': 'POST'      }    });
    const result = await handler(event);
    expect(result.headers).toMatchObject({      'access-control-allow-origin': '*',      'access-control-allow-methods': 'GET,POST,PUT,DELETE',      'access-control-allow-headers': 'Content-Type,Authorization'    });  });
  it('returns 400 for invalid JSON', async () => {    const event = createApiGatewayEvent({      httpMethod: 'POST',      body: 'invalid json{'    });
    const result = await handler(event);
    expect(result.statusCode).toBe(400);    expect(JSON.parse(result.body)).toMatchObject({      error: 'Invalid JSON'    });  });});

// api-integration.test.tsimport { APIGatewayProxyEvent } from 'aws-lambda';import { handler } from './api-handler';
describe('API Gateway Integration', () => {  it('handles POST request with JSON body', async () => {    const event: APIGatewayProxyEvent = {      httpMethod: 'POST',      path: '/users',      headers: {        'content-type': 'application/json'      },      body: JSON.stringify({        name: 'John Doe',        email: '[email protected]'      }),      // ... other required fields    } as APIGatewayProxyEvent;
    const result = await handler(event);
    expect(result.statusCode).toBe(201);    expect(result.headers).toMatchObject({      'content-type': 'application/json',      'access-control-allow-origin': '*' // CORS    });  });
  it('validates CORS headers', async () => {    const event = createApiGatewayEvent({      httpMethod: 'OPTIONS',      headers: {        'origin': 'https://example.com',        'access-control-request-method': 'POST'      }    });
    const result = await handler(event);
    expect(result.headers).toMatchObject({      'access-control-allow-origin': '*',      'access-control-allow-methods': 'GET,POST,PUT,DELETE',      'access-control-allow-headers': 'Content-Type,Authorization'    });  });
  it('returns 400 for invalid JSON', async () => {    const event = createApiGatewayEvent({      httpMethod: 'POST',      body: 'invalid json{'    });
    const result = await handler(event);
    expect(result.statusCode).toBe(400);    expect(JSON.parse(result.body)).toMatchObject({      error: 'Invalid JSON'    });  });});

Testing with AWS SAM Local

For higher-fidelity testing, use SAM CLI:

bash

# Start local API Gatewaysam local start-api --port 3000
# In another terminal, test with curlcurl -X POST http://localhost:3000/users \  -H "Content-Type: application/json" \  -d '{"name":"John","email":"[email protected]"}'
# Or invoke function directly with test eventsam local invoke UserFunction --event events/create-user.json

# Start local API Gatewaysam local start-api --port 3000
# In another terminal, test with curlcurl -X POST http://localhost:3000/users \  -H "Content-Type: application/json" \  -d '{"name":"John","email":"[email protected]"}'
# Or invoke function directly with test eventsam local invoke UserFunction --event events/create-user.json

events/create-user.json:

json

{  "httpMethod": "POST",  "path": "/users",  "headers": {    "content-type": "application/json"  },  "body": "{\"name\":\"John Doe\",\"email\":\"[email protected]\"}"}

{  "httpMethod": "POST",  "path": "/users",  "headers": {    "content-type": "application/json"  },  "body": "{\"name\":\"John Doe\",\"email\":\"[email protected]\"}"}

This catches issues like:

Lambda timeout configuration
Memory limit problems
Environment variable configuration
Cold start behavior

Testing Step Functions

Step Functions orchestrate multiple services. Here's how to test them:

Note: AWS Step Functions Local is currently unsupported by AWS and may have compatibility issues. For reliable testing, use real AWS Step Functions with test state machines.

Level 1: State Machine Definition Validation

typescript

// state-machine.test.tsimport * as fs from 'fs';import * as path from 'path';
describe('State Machine Definition', () => {  it('has valid JSON syntax', () => {    const definitionPath = path.join(__dirname, 'state-machine.asl.json');    const definition = JSON.parse(fs.readFileSync(definitionPath, 'utf8'));
    expect(definition).toHaveProperty('StartAt');    expect(definition).toHaveProperty('States');  });
  it('has error handling for each state', () => {    const definition = JSON.parse(fs.readFileSync('state-machine.asl.json', 'utf8'));
    Object.entries(definition.States).forEach(([name, state]: [string, any]) => {      if (state.Type === 'Task') {        expect(state).toHaveProperty('Catch',          `State ${name} should have error handling`        );      }    });  });});

// state-machine.test.tsimport * as fs from 'fs';import * as path from 'path';
describe('State Machine Definition', () => {  it('has valid JSON syntax', () => {    const definitionPath = path.join(__dirname, 'state-machine.asl.json');    const definition = JSON.parse(fs.readFileSync(definitionPath, 'utf8'));
    expect(definition).toHaveProperty('StartAt');    expect(definition).toHaveProperty('States');  });
  it('has error handling for each state', () => {    const definition = JSON.parse(fs.readFileSync('state-machine.asl.json', 'utf8'));
    Object.entries(definition.States).forEach(([name, state]: [string, any]) => {      if (state.Type === 'Task') {        expect(state).toHaveProperty('Catch',          `State ${name} should have error handling`        );      }    });  });});

Level 2: Integration Testing with Real Executions

typescript

// step-functions-integration.test.tsimport {  SFNClient,  StartExecutionCommand,  DescribeExecutionCommand} from '@aws-sdk/client-sfn';
const sfn = new SFNClient({ region: 'us-east-1' });const STATE_MACHINE_ARN = process.env.STATE_MACHINE_ARN;
describe('Order Processing State Machine', () => {  it('processes order successfully', async () => {    const executionName = `test-${Date.now()}`;
    // Start execution    const { executionArn } = await sfn.send(new StartExecutionCommand({      stateMachineArn: STATE_MACHINE_ARN,      name: executionName,      input: JSON.stringify({        orderId: '12345',        items: [{ id: 'item1', quantity: 2 }]      })    }));
    // Poll for completion (with timeout)    const result = await waitForExecution(executionArn!, 30000);
    expect(result.status).toBe('SUCCEEDED');
    const output = JSON.parse(result.output!);    expect(output).toMatchObject({      orderId: '12345',      status: 'COMPLETED'    });  });
  it('handles validation errors', async () => {    const executionName = `test-error-${Date.now()}`;
    const { executionArn } = await sfn.send(new StartExecutionCommand({      stateMachineArn: STATE_MACHINE_ARN,      name: executionName,      input: JSON.stringify({        orderId: '', // Invalid        items: []      })    }));
    const result = await waitForExecution(executionArn!, 30000);
    expect(result.status).toBe('FAILED');  });});
async function waitForExecution(executionArn: string, timeoutMs: number) {  const startTime = Date.now();
  while (Date.now() - startTime < timeoutMs) {    const { status, output } = await sfn.send(new DescribeExecutionCommand({      executionArn    }));
    if (status === 'SUCCEEDED' || status === 'FAILED') {      return { status, output };    }
    await new Promise(resolve => setTimeout(resolve, 1000));  }
  throw new Error('Execution timeout');}

// step-functions-integration.test.tsimport {  SFNClient,  StartExecutionCommand,  DescribeExecutionCommand} from '@aws-sdk/client-sfn';
const sfn = new SFNClient({ region: 'us-east-1' });const STATE_MACHINE_ARN = process.env.STATE_MACHINE_ARN;
describe('Order Processing State Machine', () => {  it('processes order successfully', async () => {    const executionName = `test-${Date.now()}`;
    // Start execution    const { executionArn } = await sfn.send(new StartExecutionCommand({      stateMachineArn: STATE_MACHINE_ARN,      name: executionName,      input: JSON.stringify({        orderId: '12345',        items: [{ id: 'item1', quantity: 2 }]      })    }));
    // Poll for completion (with timeout)    const result = await waitForExecution(executionArn!, 30000);
    expect(result.status).toBe('SUCCEEDED');
    const output = JSON.parse(result.output!);    expect(output).toMatchObject({      orderId: '12345',      status: 'COMPLETED'    });  });
  it('handles validation errors', async () => {    const executionName = `test-error-${Date.now()}`;
    const { executionArn } = await sfn.send(new StartExecutionCommand({      stateMachineArn: STATE_MACHINE_ARN,      name: executionName,      input: JSON.stringify({        orderId: '', // Invalid        items: []      })    }));
    const result = await waitForExecution(executionArn!, 30000);
    expect(result.status).toBe('FAILED');  });});
async function waitForExecution(executionArn: string, timeoutMs: number) {  const startTime = Date.now();
  while (Date.now() - startTime < timeoutMs) {    const { status, output } = await sfn.send(new DescribeExecutionCommand({      executionArn    }));
    if (status === 'SUCCEEDED' || status === 'FAILED') {      return { status, output };    }
    await new Promise(resolve => setTimeout(resolve, 1000));  }
  throw new Error('Execution timeout');}

Testing EventBridge

EventBridge testing is tricky due to asynchronous delivery. Here's a reliable pattern:

typescript

// eventbridge-integration.test.tsimport { EventBridgeClient, PutEventsCommand } from '@aws-sdk/client-eventbridge';import { DynamoDBDocumentClient, QueryCommand } from '@aws-sdk/lib-dynamodb';
const eventBridge = new EventBridgeClient({ region: 'us-east-1' });const ddb = DynamoDBDocumentClient.from(new DynamoDBClient({}));
describe('EventBridge Event Processing', () => {  it('processes user.created event', async () => {    const testId = `test-${Date.now()}`;
    // Publish event    await eventBridge.send(new PutEventsCommand({      Entries: [{        Source: 'user-service',        DetailType: 'user.created',        Detail: JSON.stringify({          userId: testId,          email: '[email protected]',          testMarker: testId // For identifying test events        })      }]    }));
    // Wait for processing (EventBridge + Lambda execution)    await waitForEventProcessing(testId, 10000);
    // Verify side effects in DynamoDB    const result = await ddb.send(new QueryCommand({      TableName: 'event-log',      KeyConditionExpression: 'testMarker = :marker',      ExpressionAttributeValues: { ':marker': testId }    }));
    expect(result.Items).toHaveLength(1);    expect(result.Items?.[0]).toMatchObject({      eventType: 'user.created',      processed: true    });  });});
async function waitForEventProcessing(testId: string, timeoutMs: number) {  const startTime = Date.now();
  while (Date.now() - startTime < timeoutMs) {    const result = await ddb.send(new QueryCommand({      TableName: 'event-log',      KeyConditionExpression: 'testMarker = :marker',      ExpressionAttributeValues: { ':marker': testId }    }));
    if (result.Items && result.Items.length > 0) {      return;    }
    await new Promise(resolve => setTimeout(resolve, 500));  }
  throw new Error('Event processing timeout');}

// eventbridge-integration.test.tsimport { EventBridgeClient, PutEventsCommand } from '@aws-sdk/client-eventbridge';import { DynamoDBDocumentClient, QueryCommand } from '@aws-sdk/lib-dynamodb';
const eventBridge = new EventBridgeClient({ region: 'us-east-1' });const ddb = DynamoDBDocumentClient.from(new DynamoDBClient({}));
describe('EventBridge Event Processing', () => {  it('processes user.created event', async () => {    const testId = `test-${Date.now()}`;
    // Publish event    await eventBridge.send(new PutEventsCommand({      Entries: [{        Source: 'user-service',        DetailType: 'user.created',        Detail: JSON.stringify({          userId: testId,          email: '[email protected]',          testMarker: testId // For identifying test events        })      }]    }));
    // Wait for processing (EventBridge + Lambda execution)    await waitForEventProcessing(testId, 10000);
    // Verify side effects in DynamoDB    const result = await ddb.send(new QueryCommand({      TableName: 'event-log',      KeyConditionExpression: 'testMarker = :marker',      ExpressionAttributeValues: { ':marker': testId }    }));
    expect(result.Items).toHaveLength(1);    expect(result.Items?.[0]).toMatchObject({      eventType: 'user.created',      processed: true    });  });});
async function waitForEventProcessing(testId: string, timeoutMs: number) {  const startTime = Date.now();
  while (Date.now() - startTime < timeoutMs) {    const result = await ddb.send(new QueryCommand({      TableName: 'event-log',      KeyConditionExpression: 'testMarker = :marker',      ExpressionAttributeValues: { ':marker': testId }    }));
    if (result.Items && result.Items.length > 0) {      return;    }
    await new Promise(resolve => setTimeout(resolve, 500));  }
  throw new Error('Event processing timeout');}

Key pattern: Use a unique test marker and poll for side effects instead of trying to intercept async events directly.

CI/CD Pipeline Integration

Here's a GitHub Actions workflow that implements the testing pyramid:

yaml

# .github/workflows/test.ymlname: Serverless Tests
on:  push:    branches: [main, develop]  pull_request:    branches: [main]
jobs:  unit-tests:    name: Unit Tests    runs-on: ubuntu-latest    steps:      - uses: actions/checkout@v4      - uses: actions/setup-node@v4        with:          node-version: '20'          cache: 'npm'
      - name: Install dependencies        run: npm ci
      - name: Run unit tests        run: npm run test:unit -- --coverage
      - name: Upload coverage        uses: codecov/codecov-action@v3
    # Fast: 30-60 seconds, Cost: $0
  integration-tests:    name: Integration Tests (LocalStack)    runs-on: ubuntu-latest    needs: unit-tests    steps:      - uses: actions/checkout@v4      - uses: actions/setup-node@v4        with:          node-version: '20'          cache: 'npm'
      - name: Start LocalStack        run: |          docker run -d \            -p 4566:4566 \            -e SERVICES=dynamodb,s3,sqs \            localstack/localstack:latest
          # Wait for LocalStack to be ready          timeout 60 bash -c 'until curl -s http://localhost:4566/_localstack/health | grep -q "\"dynamodb\": \"available\""; do sleep 1; done'
      - name: Install dependencies        run: npm ci
      - name: Run integration tests        run: npm run test:integration        env:          AWS_ENDPOINT: http://localhost:4566
    # Medium: 2-5 minutes, Cost: $0
  e2e-tests:    name: E2E Tests (Real AWS)    runs-on: ubuntu-latest    needs: integration-tests    if: github.ref == 'refs/heads/main'
    permissions:      id-token: write      contents: read
    steps:      - uses: actions/checkout@v4      - uses: actions/setup-node@v4        with:          node-version: '20'          cache: 'npm'
      - name: Configure AWS credentials        uses: aws-actions/configure-aws-credentials@v4        with:          role-to-assume: ${{ secrets.AWS_ROLE_ARN }}          aws-region: us-east-1
      - name: Install dependencies        run: npm ci
      - name: Deploy test stack        run: |          npx cdk deploy TestStack \            --require-approval never \            --outputs-file outputs.json
      - name: Run E2E tests        run: npm run test:e2e
      - name: Destroy test stack        if: always()        run: npx cdk destroy TestStack --force
    # Slow: 5-15 minutes, Cost: ~$0.50 per run

# .github/workflows/test.ymlname: Serverless Tests
on:  push:    branches: [main, develop]  pull_request:    branches: [main]
jobs:  unit-tests:    name: Unit Tests    runs-on: ubuntu-latest    steps:      - uses: actions/checkout@v4      - uses: actions/setup-node@v4        with:          node-version: '20'          cache: 'npm'
      - name: Install dependencies        run: npm ci
      - name: Run unit tests        run: npm run test:unit -- --coverage
      - name: Upload coverage        uses: codecov/codecov-action@v3
    # Fast: 30-60 seconds, Cost: $0
  integration-tests:    name: Integration Tests (LocalStack)    runs-on: ubuntu-latest    needs: unit-tests    steps:      - uses: actions/checkout@v4      - uses: actions/setup-node@v4        with:          node-version: '20'          cache: 'npm'
      - name: Start LocalStack        run: |          docker run -d \            -p 4566:4566 \            -e SERVICES=dynamodb,s3,sqs \            localstack/localstack:latest
          # Wait for LocalStack to be ready          timeout 60 bash -c 'until curl -s http://localhost:4566/_localstack/health | grep -q "\"dynamodb\": \"available\""; do sleep 1; done'
      - name: Install dependencies        run: npm ci
      - name: Run integration tests        run: npm run test:integration        env:          AWS_ENDPOINT: http://localhost:4566
    # Medium: 2-5 minutes, Cost: $0
  e2e-tests:    name: E2E Tests (Real AWS)    runs-on: ubuntu-latest    needs: integration-tests    if: github.ref == 'refs/heads/main'
    permissions:      id-token: write      contents: read
    steps:      - uses: actions/checkout@v4      - uses: actions/setup-node@v4        with:          node-version: '20'          cache: 'npm'
      - name: Configure AWS credentials        uses: aws-actions/configure-aws-credentials@v4        with:          role-to-assume: ${{ secrets.AWS_ROLE_ARN }}          aws-region: us-east-1
      - name: Install dependencies        run: npm ci
      - name: Deploy test stack        run: |          npx cdk deploy TestStack \            --require-approval never \            --outputs-file outputs.json
      - name: Run E2E tests        run: npm run test:e2e
      - name: Destroy test stack        if: always()        run: npx cdk destroy TestStack --force
    # Slow: 5-15 minutes, Cost: ~$0.50 per run

package.json test scripts:

json

{  "scripts": {    "test:unit": "jest --testPathPattern=\\.test\\.ts$",    "test:integration": "jest --testPathPattern=\\.integration\\.test\\.ts$",    "test:e2e": "jest --testPathPattern=\\.e2e\\.test\\.ts$ --runInBand"  }}

{  "scripts": {    "test:unit": "jest --testPathPattern=\\.test\\.ts$",    "test:integration": "jest --testPathPattern=\\.integration\\.test\\.ts$",    "test:e2e": "jest --testPathPattern=\\.e2e\\.test\\.ts$ --runInBand"  }}

Common Pitfalls and Solutions

Pitfall 1: Over-Mocking

Problem: Mocking AWS SDK calls but missing IAM permission issues.

typescript

// BAD: Passes test, fails in productionddbMock.on(PutCommand).resolves({});// Deploy → "AccessDenied: User is not authorized to perform: dynamodb:PutItem"
// GOOD: Integration test with real DynamoDB catches thisconst result = await ddb.send(new PutCommand({...}));// Test fails if IAM permissions are wrong

// BAD: Passes test, fails in productionddbMock.on(PutCommand).resolves({});// Deploy → "AccessDenied: User is not authorized to perform: dynamodb:PutItem"
// GOOD: Integration test with real DynamoDB catches thisconst result = await ddb.send(new PutCommand({...}));// Test fails if IAM permissions are wrong

Solution: Use integration tests with real AWS for critical paths.

Pitfall 2: Event Structure Mismatches

Problem: Test events missing required fields.

typescript

// BAD: Incomplete test eventconst event = { body: JSON.stringify({ id: 1 }) };
// GOOD: Use @types/aws-lambdaimport { APIGatewayProxyEvent } from 'aws-lambda';const event: APIGatewayProxyEvent = {  body: JSON.stringify({ id: 1 }),  headers: {},  httpMethod: 'POST',  isBase64Encoded: false,  path: '/users',  pathParameters: null,  queryStringParameters: null,  multiValueHeaders: {},  multiValueQueryStringParameters: null,  stageVariables: null,  requestContext: {    accountId: '123456789012',    apiId: 'test',    // ... complete context  },  resource: '/users'};

// BAD: Incomplete test eventconst event = { body: JSON.stringify({ id: 1 }) };
// GOOD: Use @types/aws-lambdaimport { APIGatewayProxyEvent } from 'aws-lambda';const event: APIGatewayProxyEvent = {  body: JSON.stringify({ id: 1 }),  headers: {},  httpMethod: 'POST',  isBase64Encoded: false,  path: '/users',  pathParameters: null,  queryStringParameters: null,  multiValueHeaders: {},  multiValueQueryStringParameters: null,  stageVariables: null,  requestContext: {    accountId: '123456789012',    apiId: 'test',    // ... complete context  },  resource: '/users'};

Solution: Create event builder functions or use saved real events.

Pitfall 3: Ignoring Async Behavior

Problem: Testing async EventBridge without waiting for processing.

typescript

// BAD: Event not yet processedawait eventBridge.putEvents({ Entries: [event] });const result = await queryResults(); // Empty!
// GOOD: Wait for processingawait eventBridge.putEvents({ Entries: [event] });await waitForEventProcessing(); // Poll or use Step Functionsconst result = await queryResults(); // Has data

// BAD: Event not yet processedawait eventBridge.putEvents({ Entries: [event] });const result = await queryResults(); // Empty!
// GOOD: Wait for processingawait eventBridge.putEvents({ Entries: [event] });await waitForEventProcessing(); // Poll or use Step Functionsconst result = await queryResults(); // Has data

Solution: Implement polling or use Step Functions to track event processing.

Pitfall 4: Test Environment Pollution

Problem: Parallel tests interfering with each other.

typescript

// BAD: Shared resourcesconst TABLE_NAME = 'users-test'; // Conflicts in parallel tests
// GOOD: Unique resource namesconst TABLE_NAME = `users-test-${Date.now()}-${Math.random()}`;
// Or use beforeEach/afterEach cleanupafterEach(async () => {  await Promise.all(    testItems.map(id => ddb.delete({ Key: { id } }))  );});

// BAD: Shared resourcesconst TABLE_NAME = 'users-test'; // Conflicts in parallel tests
// GOOD: Unique resource namesconst TABLE_NAME = `users-test-${Date.now()}-${Math.random()}`;
// Or use beforeEach/afterEach cleanupafterEach(async () => {  await Promise.all(    testItems.map(id => ddb.delete({ Key: { id } }))  );});

Solution: Use unique resource names and implement cleanup hooks.

Key Takeaways

Here's what works in practice:

1. Follow the testing pyramid: 70% unit tests for fast feedback, 20% integration tests for confidence, 10% E2E tests for production validation.

2. Separate business logic from handlers: This makes unit testing easier and faster. Test your business logic thoroughly, then do lighter testing on the handler wiring.

3. Use LocalStack for rapid iteration, real AWS for validation: LocalStack is great for development speed, but always validate against real AWS before deploying.

4. Test IAM permissions explicitly: The most common "works in test, fails in prod" issue is IAM permissions. Integration tests with real AWS catch these.

5. Build event builder utilities: Create helper functions for constructing realistic test events. Use @types/aws-lambda types to ensure completeness.

6. Implement proper cleanup: Use afterEach/afterAll hooks and unique resource names to prevent test pollution. This also saves on AWS costs.

7. Handle async testing properly: EventBridge and Step Functions are asynchronous. Implement polling or use Step Functions executions to validate event processing.

8. Optimize CI/CD pipeline costs: Run unit tests on every commit, integration tests on PRs, E2E tests only on main branch. Use ephemeral stacks with auto-deletion.

9. Track test metrics: Monitor execution time, flakiness rate, and AWS costs. Optimize based on data, not assumptions.

10. Start simple: Begin with basic unit tests and add integration/E2E tests as your application matures. Perfect is the enemy of good.

Working with serverless taught me that testing strategy matters as much as the tests themselves. Fast feedback catches most bugs early, while strategic integration testing catches the issues that only appear when services actually interact. The key is finding the right balance for your team and application.

References

docs.aws.amazon.com - Lambda functions: execution model and scaling.
jestjs.io - Jest testing framework documentation.
docs.aws.amazon.com - AWS Lambda Developer Guide.
serverless.com - Serverless learning resources (patterns and operations).
docs.github.com - GitHub Actions documentation.
typescriptlang.org - TypeScript Handbook and language reference.
github.com - TypeScript project wiki (FAQ and design notes).
docs.aws.amazon.com - Amazon DynamoDB Developer Guide.
docs.aws.amazon.com - Amazon EventBridge User Guide.
docs.aws.amazon.com - Amazon API Gateway Developer Guide.
docs.aws.amazon.com - AWS Overview (official whitepaper).
cloud.google.com - Google Cloud documentation.

The Testing Challenge#

The Serverless Testing Pyramid#

When to Use Each Testing Level#

Unit Testing Lambda Functions#

Testing with AWS SDK v3#

Integration Testing Strategies#

Testing with Real DynamoDB#

LocalStack vs Real AWS: When to Use Each#

Testing API Gateway Integrations#

Testing with AWS SAM Local#

Testing Step Functions#

Level 1: State Machine Definition Validation#

Level 2: Integration Testing with Real Executions#

Testing EventBridge#

CI/CD Pipeline Integration#

Common Pitfalls and Solutions#

Pitfall 1: Over-Mocking#

Pitfall 2: Event Structure Mismatches#

Pitfall 3: Ignoring Async Behavior#

Pitfall 4: Test Environment Pollution#

Key Takeaways#

References#

Related Posts

The Testing Challenge

The Serverless Testing Pyramid

When to Use Each Testing Level

Unit Testing Lambda Functions

Testing with AWS SDK v3

Integration Testing Strategies

Testing with Real DynamoDB

LocalStack vs Real AWS: When to Use Each

Testing API Gateway Integrations

Testing with AWS SAM Local

Testing Step Functions

Level 1: State Machine Definition Validation

Level 2: Integration Testing with Real Executions

Testing EventBridge

CI/CD Pipeline Integration

Common Pitfalls and Solutions

Pitfall 1: Over-Mocking

Pitfall 2: Event Structure Mismatches

Pitfall 3: Ignoring Async Behavior

Pitfall 4: Test Environment Pollution

Key Takeaways

References