Feat/use durable functions emulator image#8708
Conversation
…tead of building image using binary
# Conflicts: # samcli/local/docker/durable_functions_emulator_container.py
… fails to start up
| f"Durable Functions Emulator container failed to become ready within {timeout} seconds. " | ||
| "You may set the DURABLE_EXECUTIONS_EMULATOR_IMAGE_TAG env variable to a specific image " | ||
| "to ensure that you are using a compatible version. " | ||
| "Check https://gallery.ecr.aws/o4w4w0v6/aws-durable-execution-emulator. " |
There was a problem hiding this comment.
minor, but can you use self._get_emulator_image() here too?
maybe
f"Check https://${self._get_emulator_image().replace('public.ecr', 'gallery.ecr')}. "
?
or have another method that uses the same variables? Basically so if we change the location of the image, we have a consistent messaging too
There was a problem hiding this comment.
ah yeah, I can definitely do that
| "--store-path", | ||
| "/tmp/.durable-executions-local/durable-executions.db", |
There was a problem hiding this comment.
I don't see this being used before. Is this a new thing needed?
There was a problem hiding this comment.
Similar question about the other properties --log-level, --lambda-endpoint..
There was a problem hiding this comment.
yeah we removed a "layer" to the emulator, these were previously being set by default but not transparently to the SAM CLI code
| "--store-type", | ||
| self._get_emulator_store_type(), |
There was a problem hiding this comment.
In line 187 this is already added to the environment variables, so we probably don't need to add it here again.
"EXECUTION_STORE_TYPE": self._get_emulator_store_type(),
There was a problem hiding this comment.
ah, EXECUTION_STORE_TYPE is part of the emulator "layer" that was removed. So it's not being used anymore. I can remove it from the environment variables.
There was a problem hiding this comment.
conversely, I can also use the "new" environment variables instead:
There was a problem hiding this comment.
Ah. I see. I think both alternatives work, as long as it's clear. I think it makes sense to keep it here and remove it from the env vars.
There was a problem hiding this comment.
"HOST": "0.0.0.0", # ❌ UNUSED - hardcoded, passed via --host
"PORT": str(self.port), # ❌ UNUSED - passed via --port (command wins)
"LOG_LEVEL": "DEBUG", # ❌ UNUSED - hardcoded, passed via --log-level
"EXECUTION_STORE_TYPE": self._get_emulator_store_type(), # ❌ UNUSED - passed via --store-type (command wins)
"EXECUTION_TIME_SCALE": self._get_emulator_time_scale(), # ❌ UNUSED - not even a CLI arg!
so only
self.port - Can be overridden by user
self._get_emulator_store_type() - Can be overridden by user
so maybe:
command=[
"dex-local-runner", "start-server",
"--host", "0.0.0.0",
"--port", str(self.port),
"--log-level", "DEBUG",
"--lambda-endpoint", "http://host.docker.internal:3001",
"--store-type", self._get_emulator_store_type(),
"--store-path", "/tmp/.durable-executions-local/durable-executions.db",
],
environment={
"AWS_ACCESS_KEY_ID": "foo",
"AWS_SECRET_ACCESS_KEY": "bar",
"AWS_DEFAULT_REGION": "us-east-1",
}
| "start-server", | ||
| "--host", | ||
| "0.0.0.0", | ||
| "--port", |
There was a problem hiding this comment.
I guess not related to this PR, but do we have plan to assign this port dynamically. Currently it would just use 9014 no matter it's taken or not
There was a problem hiding this comment.
oh really, I thought the code already dynamically assigned the port according to the env variable.
| _RAPID_SOURCE_PATH = Path(__file__).parent.joinpath("..", "rapid").resolve() | ||
| _EMULATOR_IMAGE = "public.ecr.aws/ubuntu/ubuntu:24.04" | ||
| _EMULATOR_IMAGE_PREFIX = "samcli/durable-execution-emulator" | ||
| _EMULATOR_IMAGE = "public.ecr.aws/o4w4w0v6/aws-durable-execution-emulator:" |
There was a problem hiding this comment.
This is not the final URL right
| _RAPID_SOURCE_PATH = Path(__file__).parent.joinpath("..", "rapid").resolve() | ||
| _EMULATOR_IMAGE = "public.ecr.aws/ubuntu/ubuntu:24.04" | ||
| _EMULATOR_IMAGE_PREFIX = "samcli/durable-execution-emulator" | ||
| _EMULATOR_IMAGE = "public.ecr.aws/o4w4w0v6/aws-durable-execution-emulator:" |
There was a problem hiding this comment.
nit (since it needs a tag - and concatenate the : when we combine them):
| _EMULATOR_IMAGE = "public.ecr.aws/o4w4w0v6/aws-durable-execution-emulator:" | |
| _EMULATOR_IMAGE_PREFIX = "public.ecr.aws/o4w4w0v6/aws-durable-execution-emulator" |
|
|
||
| def _get_emulator_image(self): | ||
| """Get the full emulator image name with tag.""" | ||
| return self._EMULATOR_IMAGE + self._get_emulator_image_tag() |
There was a problem hiding this comment.
| return self._EMULATOR_IMAGE + self._get_emulator_image_tag() | |
| return f'{self._EMULATOR_IMAGE}:{self._get_emulator_image_tag()}' |
There was a problem hiding this comment.
ahh yeah this would be better, thanks
| """ | ||
| Allow pinning to a specific emulator image tag/version | ||
| """ | ||
| ENV_EMULATOR_IMAGE_TAG = "DURABLE_EXECUTIONS_EMULATOR_IMAGE_TAG" |
There was a problem hiding this comment.
sam local invoke has a --invoke-image parameter, where customers can pass the location of a specific image to be used as the execution base image instead of the default Lambda base image.
could explore using this established pattern rather than adding ENV_EMULATOR_IMAGE_TAG?
Which issue(s) does this change fix?
#8488
Why is this change necessary?
This introduces a fix for the Homebrew linkage issue with the Durable Functions emulator. We are also introducing this so that SAM CLI can receive Durable Functions emulator updates uncoupled with SAM CLI release.
How does it address the issue?
We are automatically vending the emulator via an image published to ECR. See aws/aws-durable-execution-sdk-python-testing#196. Since the arm64 emulator is no longer included as an executable directly in the SAM CLI, this also fixed the issue where Homebrew users of SAM CLI could not emulate durable functions locally without doing some workaround.
What side effects does this change have?
As a bonus, this reduces the SAM CLI size.
Mandatory Checklist
PRs will only be reviewed after checklist is complete
make prpassesmake update-reproducible-reqsif dependencies were changedBy submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.