Build profiles

[itowlson]

Incomplete draft for discussion. This diverges from Till's draft SIP in some ways, mostly cosmetic and open to change; but also somewhat in preferring to constrain variation between profiles.

Example:

[component.build-profile-test]
source = "target/wasm32-wasip1/release/build_profile_test.wasm"
profile.debug.source = "target/wasm32-wasip1/debug/build_profile_test.wasm"
allowed_outbound_hosts = []
[component.build-profile-test.build]
command = "cargo build --target wasm32-wasip1 --release"
profile.debug.command = "cargo build --target wasm32-wasip1"
watch = ["src/**/*.rs", "Cargo.toml"]

This proposal uses a partial approach to the possible "am I running the same profile I built" problem. When you do a build, it records the built profile in a "last profile built" file. When you do a run, it checks if the profile being run matches the last built one, and prints a warning if not. This is certainly far from reliable: if you build both debug and release profiles, and then run both of them, you'll get a spurious warning.

If we're okay with this as a basis for further development, we need to look at what other fields should be allowed to vary by profile, e.g.:

• allowed_outbound_hosts - you should be able to specify additional hosts. I'd be inclined not to say you can specify an entirely separate collection, because the 90% case is going to "I need to do sockets to a debugger." (Things like "I want to talk to the dev database not the prod database" should be handled by variables.)
• dependencies - should be able to replace a dependency, e.g. to use its debug build or a stub/mock. If I squint I can imagine use cases for extra dependencies, in case a component has debug-only imports, not sure
• environment - should be able to replace an EV or set additional EVs
• KV/SQLite/LLM - my inclination that varying these by profile is probably an anti-pattern but I am open to being talked round if someone has a concrete use case - otherwise propose deferring to later
• variables - not clear why you would, I guess there could be a knob to control profile-specific behaviour? Propose defer
• files - not clear why you would, but I guess I could imagine a special build needing some additional asset? Again my inclination would be to defer until someone needs it

Possible further features:

• whole components being conditional on the profile - e.g. include KV explorer only in certain build configs
• this implies conditional triggers and I am not sure how much I love where this is going

Anyway here it is for discussion, please be gentle with me

[lann]

This diverges from Till's draft SIP in some ways, mostly cosmetic and open to change; but also somewhat in preferring to constrain variation between profiles.

I think I prefer the SIP's TOML structure with everything for component X / profile Y going under component.X.profile.Y. I agree that we should constrain variation between profiles regardless of the structure.

[lann]

I need to do sockets to a debugger.

I think we should do this as a separate feature: #3153

[itowlson]

Updated with reworked manifest:

[component.build-profile-test]
source = "target/wasm32-wasip1/release/build_profile_test.wasm"
allowed_outbound_hosts = []
[component.build-profile-test.build]
command = "cargo build --target wasm32-wasip1 --release"
watch = ["src/**/*.rs", "Cargo.toml"]
[component.build-profile-test.profile.debug]
source = "target/wasm32-wasip1/debug/build_profile_test.wasm"
build.command = "cargo build --target wasm32-wasip1"

I dislike the verbosity and repetition of the way TOML does this. It takes a lot to make me yearn for JSON but man this comes close. But the clarify benefit of the grouping overrides the verbosity issue.

(again, will do squashes once we have agreement on the approach)

[itowlson]

Done env vars and deps, holding off on allowed hosts: if we can do the special-case thing for debug connections, then I reckon we can defer allowed hosts in general until someone comes up with a use case.

[lann]

This brings to mind a famous quote, by Abraham Lincoln I believe...

#deep-inside-joke

[itowlson]

Yes: the trouble is that we've already rather violated President Lincoln's decree. We use serialisation types throughout Spin's application logic, and now the choice is major rework of multiple subsystems for no functional benefit, or papering over where we've coupled to a serialisation format that no longer gives us the correctness guarantees we seek. Making dangerous fields as private as possible and providing safer accessors seems like the lowest risk...

I did explore keeping Component as POD and doing something at the application level so that the components collection returned pre-profiled components, which would avoid a lot of this nonsense in favour of a single piece of once-and-for-all nonsense. It didn't really seem to work out, but I can revisit it if you feel that's a better strategy.

There, that will teach you to make light inside jokes.

[lann]

It didn't really seem to work out, but I can revisit it if you feel that's a better strategy.

Speak of the devil: https://github.com/spinframework/spin/pull/3246/files/ea595fcfd5df993eb55155409fb56eaf37f4d715#r2320229292

[lann]

Another "I wonder if it would be better" here:

Would it be reasonable to handle profiles in a new normalize_manifest pass (i.e. normalize_manifest(manifest, profile))? It would have the benefit and/or fatal flaw of hiding most of the profile logic from the rest of the downstream code.

[itowlson]

I explored this but had trouble getting it to work safely. This may point to a deeper problem of normalisation not being sufficiently embedded in the loader logic, but again this is a risk of world+dog being able to go "eh I'll just deserialise from TOML." The loader pipeline goes from TOML to LockedApp but there are things in the dev tooling that care about AppManifests and they tend to bypass the correct load sequence because it's not readily accessible. But like I say this may be an opportunity to improve the load sequence and make better guarantees, it will just likely be more work.

[lann]

Could you say more about the troubles here? I can see a few potential problems along those lines but it isn't clear to me where things would be worse by normalizing in spin-manifest rather than spin-loader.

[itowlson]

If I remember rightly, the trouble was that some places were deserialising manifests rather than asking a loader to do it for them (because the loader produced a LockedApp and they wanted a Manifest). And this is hard to stop because deserialisability is a public feature. You need to make the DTO private and force consumers to go through a loader which can then enforce normalisation. (But some things do have a legit interest in the raw manifest. So, choices, choices.)

Let me look again. I didn't spend much time on it and my memory of what I was trying is not great.

[lann]

I'm reasonably confident this is equivalent (and less-not-inarguably better):

Suggested change

	let is_match = match (profile, latest_build) {
	(None, None) => true,
	(Some(_), None) | (None, Some(_)) => false,
	(Some(p), Some(latest)) => p == latest,
	};
	if !is_match {
	if profile != latest_build {

[itowlson]

FACEPALM

Thank you!

[lann]

Not inarguably better, but another option:

Suggested change

	if last_build_str == LAST_BUILD_ANON_VALUE {
	None
	} else {
	Some(last_build_str)
	}
	(last_build_str != LAST_BUILD_ANON_VALUE).then_some(last_build_str)

[itowlson]

Maybe it's too much Go programming but I find this a bit obscure and would prefer to keep spelling it out. Appreciate the education though - just give me a couple of years to internalise it!

[lann]

🚲🏠

Suggested change

	const LAST_BUILD_ANON_VALUE: &str = "<anonymous>";
	const LAST_BUILD_NONE_VALUE: &str = "<none>";

[itowlson]

It's not necessarily "none" (which to me implies no last build). It's usually "the anonymous profile."

[lann]

Ah ok that makes sense.

[lann]

The only comment here rising to the level of "maybe-blocking" is the one about logging errors. The rest are minor nits/suggestions.

[karthik2804]

I am in agreement with @tschneidereit's idea of having an inherits field making things clearer and easier to follow. It also makes sense to me to call it "default" instead of "anonymous" for the unnamed profile.

No automatic release build before registry push.

This I feel quite strongly is important to do: I'm almost entirely certain that I myself would get this wrong and push debug builds at some point in the future.

I do not feel as strongly here with deploy/push taking into consideration the profile arg and also profile names being arbitrary strings means we would have to special case things.

Allows override of environment and dependencies

I feel like allowing outbound hosts would be useful thinking about it from the perspective of the JS debugger. I see the suggested approach is #3153 but to me that feels like more things for the user to setup every time when they need to debug a component. They would need to setup the debugger port as well as also the profile to choose the binary built with debugging enabled.

I see an argument as to why we may want to enable overriding variables in profiles in case where we want different defaults. This does not seem super important because the variables can be overriden with environment variables anyway but I think it is a nice to have.

[tschneidereit]

profile names being arbitrary strings means we would have to special case things.

Can you say more about what we'd need to special-case? I'm imagining something like spin deploy --profile=gofastest, where the default profile is used if --profile isn't specified. This would then internally run spin build --profile=gofastest before actually deploying.

If we end up doing this, we should definitely make it possible to skip, with something like --skip-build.

[karthik2804]

I may have misunderstood the intent here. I interpreted the "automatic release build" to imply that there would be a specific release profile (which is what would required special-casing). I do not have a strong opinion on if we should require build before deploy but that seems like a problem that can be punted as it does not change existing behavior where deploy does not build.

[tschneidereit]

that seems like a problem that can be punted as it does not change existing behavior where deploy does not build.

I don't think it can be, because the introduction of profiles changes the picture. Yes, it's currently possible to accidentally deploy a stale build. I think that's not ideal and we probably should already be doing an automatic build. But with profiles the failure mode is very different: the developer might be ensuring very carefully that they do a build, but accidentally use the wrong profile to do so.

One thing I'm just realizing though is that profiles (can, at least) specify different locations for their outputs. It seems likely that they would in practice, too. That means the main risk is that the likelihood of deploying stale builds increases, not that a debug build might accidentally be deployed—which would be much worse.

I still think an automatic build would be very highly desirable, but this realization at least allays my biggest fears.

[karthik2804]

One thing I'm just realizing though is that profiles (can, at least) specify different locations for their outputs. It seems likely that they would in practice, too. That means the main risk is that the likelihood of deploying stale builds increases, not that a debug build might accidentally be deployed—which would be much worse.

This is what I assumed and hence the lack of strong opinion on my end. I think automatic builds would be good though yes but personally don't feel strong enough to block on it.

[itowlson]

inherits feels like a significant increase in complexity (topologically sorting profiles, cycle detection, chaining overrides). Could we articulate what functional benefit justifies that complexity? Could we consider deferring it until someone needs it?

[karthik2804]

To me, it feels like a clean way to describe if a profile is fully isolated or if it is just overriding the default profile.

I do agree on the point of the increase in complexity. Could a middle ground be that inherit supports only boolean values and only allows inheriting from the "default" profile.

As I write this now, I realize that this is describing new behavior of profiles being completely isolated and not just overriding the default 😅.

[tschneidereit]

I think deferring inherits is fine, as long as without it each profile is a blank slate, fwiw. Implicitly inheriting anything at all from somewhere would mean a later introduction of inherits is a breaking change.

Could a middle ground be that inherit supports only boolean values and only allows inheriting from the "default" profile.

If we were to special-case the default profile as a source of inheritance, I think we shouldn't do it with a boolean, but with the string "default" being the only permitted value to begin with.

[karthik2804]

I finally played around with this once again and now I have changed my mind about the inherits. The current state of things seem ideal. I like the idea of every profile option being an optional override.

The only feedback I have here is that I would really prefer to be able to override the allowed_outbound_hosts specifically coming from the JS debugger usecase. I see that we want to address it seperately but I am not too convinced about the value from that. To me personally it seems cleaner for it to be enabled by profiles. @lann Can you expand on why we do not want to do it here?

This looks great and I would love for it to land!

[lann]

Debugging just seems orthogonal to app configuration to me. I've run debuggers in production a fair number of times to track down particularly wiley bugs.

[karthik2804]

I could potentially dream up a usecase where we have build profiles that enable building with different features where said one of the optional features is exporting telemetry data. I want to only allow list the host in the case where the app is built with the telemetry feature. And enabling debugging just happens to be a side benefit. Or is there a strong reason for us to not do this?

[lann]

exporting telemetry data

You mean like wasi-otel, which is configured outside of the manifest? 🙂

[lann]

To be clear I'm not opposed to using profiles to enable debugging; I just think #3153 would still be useful.

One incremental change we could make here would be to add support for an allowed_outbound_hosts "no-op" value, e.g. allowed_outbound_hosts = [""], which you could then use as your default debug_host variable. Technically you could get this effect today by using an unroutable IPv6 "black hole" address like 100::1, though that might be more confusing.

[itowlson]

The reason for not doing AOH is that we primarily want this variation to be driven by variables, so that the component talks to the host given in the variable instead of needing to have separate logic.

But debugging is a great example of when that doesn't work - it happens below the component level.

I'd be wary of making AOH overridable for this, because if means that either:

1. there's a lot of repetition if there are multiple hosts; or
2. the profile value needs to merge instead of replace, which is different behaviour from other fields.

I somewhat like Lann's suggestion of variable values which resolve to 'ignore this', so you could have a single list of AOH in the base component, with {{ debug_host }} resolving to the "ignore" string or localhost per profile. I'm not sure empty string works for this, though, because I'm not sure if we can reliably tell this apart from "variable is not set."

I suspect there is also some fiddliness around application variable default values not currently being profileable, at least I don't think they are, which is what you'd want to make this transparent. (I may be confusing myself though.)

[karthik2804]

Lann's suggestion is interesting. Even without modifying what is acceptable in thefield, I think we can make it work by using if clauses using the liquid templating engine we already have. I'll try verify that and if it works, I'm happy to punt on allowed outbound hosts.

[itowlson]

I don't think AOH uses Liquid templating.

[itowlson]

(force push was just a rebase, and profilifying target env signatures which didn't exist when I started this PR)

[karthik2804]

I don't think AOH uses Liquid templating.
You are right I was misremembering.

I like the approach @lann suggested as well.

[itowlson]

I like the approach @lann suggested as well.

There are a couple of Lann suggestions floating around - could you clarify which one you are proposing?

[karthik2804]

@itowlson I like the one where we allow allowed_outbound_host to have a nullable value which would mean we can punt on making that field profileable.

[itowlson]

@itowlson I like the one where we allow allowed_outbound_host to have a nullable value which would mean we can punt on making that field profileable.

AOH uses variables from the application, not the component, and application-level stuff is not currently profileable. So we would need to either:

1. Have users pass in the debugger host value via -v (e.g. spin up --profile debug -v debug_host="tcp://127.0.0.1:*"); or
2. Support profiling application variables (so that the value can be defaulted differently).

The first one is easy and mostly-safe from a design point of view (the only work needed is to make AOH ignore empty strings, which I am suspicious of but could go along with). But it's a bit of a faff for users!

The second is more convenient for users, but requires another area where we have to consult the profiles system. I am wary of doing this for the specific purpose of JS debugging, especially when it only provides us with a rather kludgy solution (and one where it becomes tied to the profile which, as Lann notes, is not necessarily desirable).

My current preference is 3. let's not hold the PR up on JS debugging. We can add AOH empty string forgiveness and/or app variable profiling as follow-up work if we want them: they would be non-breaking. But if we feel this is something we need to solve before we can land the PR, then I'm happy to entertain options!

[karthik2804]

@itowlson I am in agreement, let us on punt on the design for enabling debugger to a future PR.

[itowlson]

What is required to land this?