Skip to content

Comments

Introducing Harness Testing#1103

Open
darklight3it wants to merge 13 commits intomainfrom
dmelfi/add-dockerized-test-infrastructure
Open

Introducing Harness Testing#1103
darklight3it wants to merge 13 commits intomainfrom
dmelfi/add-dockerized-test-infrastructure

Conversation

@darklight3it
Copy link

@darklight3it darklight3it commented Feb 20, 2026

📬 Issue #, if available:

N/A

✍️ Description of changes:

This PR adds Docker-based testing infrastructure using AWS's containerized-test-runner-for-aws-lambda, enabling automated testing of Lambda functions in a containerized environment that closely mirrors the AWS Lambda execution environment.

The PR introduces a test-dockerized Makefile target that runs test suites defined in test/dockerized/*.json files. These test suites specify handlers to test (from the examples directory), request payloads, and expected response assertions with optional jq transforms for validation.

The infrastructure reuses Lambda binaries from the /examples folder as test handlers, demonstrating the concept with an initial test case for basic-lambda. Additional tests and multi-concurrency scenarios can be added by creating new test suite JSON files.

A GitHub Actions workflow provides CI/CD integration for automated testing on pull requests.

Testing

Run dockerized tests locally:

make test-dockerized

Run RIE tests:

make test-rie
HANDLERS_TO_BUILD="basic-lambda basic-sqs" make test-rie

Build specific examples:

EXAMPLES="basic-lambda basic-lambda-concurrent" make build-examples

🔏 By submitting this pull request

  • I confirm that I've ran cargo +nightly fmt.
  • I confirm that I've ran cargo clippy --fix.
  • I confirm that I've made a best effort attempt to update all relevant documentation.
  • I confirm that my contribution is made under the terms of the Apache 2.0 license.on.
  • I confirm that my contribution is made under the terms of the Apache 2.0 license.

@darklight3it darklight3it marked this pull request as draft February 20, 2026 01:30
@darklight3it darklight3it force-pushed the dmelfi/add-dockerized-test-infrastructure branch 2 times, most recently from 0fb86a2 to b414e80 Compare February 20, 2026 01:41
@darklight3it darklight3it force-pushed the dmelfi/add-dockerized-test-infrastructure branch from b414e80 to 2b40b80 Compare February 20, 2026 01:44
@darklight3it darklight3it self-assigned this Feb 20, 2026
run: docker build . -t local/test -f Dockerfile.rie

- name: Run tests
uses: aws/containerized-test-runner-for-aws-lambda@v1
Copy link
Collaborator

@jlizen jlizen Feb 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this action exist? The ci is failing to find it, and I don't see references elsewhere on the Internet.

I wonder if the existing test-rie workflow is close to what you want here, and could be adapted?

Seems like the main thing missing would be a small script to translate the request mappings into curl requests that assert against the response? Along with some parameterizing to feed in the proper lambda binary and test assets to the dockerfile?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I like what you are trying to do here with the testing harness though, seems super handy... If the referenced action is in development / not yet public... Excited to see it!)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry 😢 for this you miss some context here, and I had no time to write this in the appropriate channel. This stuff is agreed with @maxday.

That action is already used in the ruby repo and Maxime is actively working in making it public. The idea is to use the same harness test framework in all lambda runtimes.

I created a CR on that repo and this one is dependent on that to be working.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW I think the existing test-rie way of dealing with those tests will be removed in favor how the harness test standard that easily allow you to make assertions on the lambda response.

@darklight3it darklight3it marked this pull request as ready for review February 20, 2026 11:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants