Add Lambda OpenTelemetry demo with AWS ADOT Layer auto-instrumentation

[Copilot]

Demonstrates end-to-end distributed tracing across Lambda microservices using OpenTelemetry without CloudWatch/X-Ray. Implements e-commerce order flow with instrumented service-to-service calls and failure scenarios.

Architecture

Three Lambda services with realistic workflow:

• Order Service: Orchestrates inventory check → payment processing
• Inventory Service: Stock validation with configurable levels (items 003, 005 are out-of-stock)
• Payment Service: Payment processing with test failure patterns (specific card numbers trigger declines/insufficient funds)

Implementation

Auto-instrumentation via AWS ADOT Layer

• Uses AWS_LAMBDA_EXEC_WRAPPER=/opt/otel-handler for zero-code instrumentation
• Automatic HTTP/HTTPS/AWS SDK tracing
• W3C trace context propagation across service boundaries
• Node.js 20.x runtime with Layer ARN per region

Trace context propagation

// Automatic via ADOT Layer - axios requests include traceparent headers
await callService(inventoryUrl, { orderId, items });

// Manual spans for business logic
return withSpan('process-order', async (span) => {
  addSpanAttributes({
    'order.id': orderId,
    'customer.id': customerId,
    'order.items.count': items.length
  });
  // orchestration logic
});

OTLP export configuration

Environment:
  Variables:
    OTEL_EXPORTER_OTLP_ENDPOINT: http://collector:4318
    OTEL_SERVICE_NAME: order-service
    OTEL_TRACES_SAMPLER: AlwaysOn
    OTEL_PROPAGATORS: tracecontext

Test Scenarios

• Success path: Items in stock, valid payment → HTTP 200
• Out-of-stock: itemId item-003 or item-005 → HTTP 409
• Payment failures:
  • Card 4111111111111111 → declined
  • Card 4222222222222222 → insufficient funds
  • 10% random failure rate for other cards

Deployment

SAM template includes:

• ADOT Layer attachment with regional ARN configuration
• API Gateway with explicit resource naming (avoids circular dependency warnings)
• Service URL injection via environment variables for cross-service calls

Minimal dependencies: only axios (1.13.5) in application code. OpenTelemetry SDK provided by layer reduces package size and cold start impact.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• aws-otel.github.io
  • Triggering command: /home/REDACTED/work/_temp/ghcca-node/node/bin/node /home/REDACTED/work/_temp/ghcca-node/node/bin/node --enable-source-maps /home/REDACTED/work/_temp/copilot-developer-action-main/dist/index.js (dns block)
• aws-serverless-tools-telemetry.us-west-2.amazonaws.com
  • Triggering command: /usr/local/aws-sam-cli/1.154.0/dist/sam sam build bin/WALinuxAgent-2.15.1.3-py3.12.egg -collect-logs (dns block)
  • Triggering command: /usr/local/aws-sam-cli/1.154.0/dist/sam sam build (dns block)
  • Triggering command: /usr/local/aws-sam-cli/1.154.0/dist/sam sam validate (dns block)
• opentelemetry.io
  • Triggering command: /home/REDACTED/work/_temp/ghcca-node/node/bin/node /home/REDACTED/work/_temp/ghcca-node/node/bin/node --enable-source-maps /home/REDACTED/work/_temp/copilot-developer-action-main/dist/index.js (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Original prompt

Create a sample AWS Lambda Project project using NodeJS with demonstrate how to integrate Opentelemetry with AWS Lambda without using AWS CloudWatch and XRAY to get end to end tracing. Create 3 different lambda functions as 3 different services so the trace can show how request flows between 3 services. it can be Order service which accept an order, Inventory service check inventory and Payment service process payment. Introduce out of stock and Failed payment scenario for testing.

---

🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.

[amitava82]

@ansh-devs https://aws-otel.github.io/docs/getting-started/lambda

[amitava82]

looks like it didn't use auto instrumentation. I see manual tracer code