Add a new wasm32-wasip2 target

[rylev]

This is the initial implementation of the MCP rust-lang/compiler-team#694 creating a new tier 3 target wasm32-wasip2. That MCP has been seconded and will most likely be approved in a little over a week from now. For more information on the need for this target, please read the MCP.

There is one aspect of this PR that will become insta-stable once these changes reach a stable compiler:

• A new target_family named wasi is introduced. This target family incorporates all wasi targets including wasm32-wasi and its derivative wasm32-wasi-preview1-threads. The difference between target_family = wasi and target_os = wasi will become much clearer when wasm32-wasi is renamed to wasm32-wasi-preview1 and the target_os becomes wasm32-wasi-preview1. You can read about this target rename in this MCP which has also been seconded and will hopefully be officially approved soon.

Additional technical details include:

• Both std::sys::wasi_preview2 and std::os::wasi_preview2 have been created and mostly use #[path] annotations on their submodules to reach into the existing wasi (soon to be wasi_preview1) modules. Over time the differences between wasi_preview1 and wasi_preview2 will grow and most like all #[path] based module aliases will fall away.
• Building wasi-preview2 relies on a wasi-sdk in the same way that wasi-preview1 does (one must include a wasi-root path in the Config.toml pointing to sysroot included in the wasi-sdk). The target should build against wasi-sdk v21 without modifications. However, the wasi-sdk itself is growing preview2 support so this might shift rapidly. We will be following along quickly to make sure that building the target remains possible as the wasi-sdk changes.
• This requires a patch to libc that we'll need to land in conjunction with this change. Until that patch lands the target won't actually build.

[rustbot]

r? @m-ou-se

(rustbot has picked a reviewer for you, use r? to override)

[rustbot]

These commits modify compiler targets.
(See the Target Tier Policy.)

These commits modify the Cargo.lock file. Unintentional changes to Cargo.lock can be introduced when switching branches and rebasing PRs.

If this was unintentional then you should revert the changes before this PR is merged.
Otherwise, you can ignore this comment.

[CryZe]

The difference between target_family = wasi and target_os = wasi will become much clearer when wasm32-wasi is renamed to wasm32-wasi-preview1 and the target_os becomes wasm32-wasi-preview1

Nothing in that MCP mentions that the target_os is being renamed. Can the preview1 / preview2 not just be a target_env or target_abi?

[rylev]

Leaving target_os as wasi for all wasi related targets is probably an oversight of the MCP. The wasi preview 1 and preview 2 logical OSes have very significant differences that go beyond just the ABI level. I don't believe we would want to rely on target_abi since that setting is not stable. That leaves us just with target_env which according to the reference is " disambiguating information about the target platform". Perhaps target_env is more appropriate.

Preview 2 and preview 1 differ from each other in the following ways:

• Different ABI
• Different platform API (i.e., the functions exposed from the host environment significantly differ in shape and semantics)
• Different binary artifact (i.e., preview 1 emits core Wasm modules while preview 2 emits Wasm components which are an extension of modules)

Does target_env capture those differences?

I tried looking at how Windows versions are handled and surprisingly it seems that target_vendor is often used for different versions like uwp and win7.

cc @alexcrichton.

[alexcrichton]

Part of the motivation for adding an entirely new target is that existing target_os = "wasi" code will not work indefinitely on the new target. Existing target_os = "wasi" code is probably using items from the wasi_snapshot_preview1 module which is not defined in terms of either the component model or the latest WASI standard. This necessarily means that any code using target_os = "wasi" is highly likely to need an update (e.g. usage of std::os::wasi or the wasi crate on crates.io).

I bring this up in that this is motivation for not only a new target but additionally a new target_os = "..." setting. The MCP could have gone into more depth about this (but let's be real any description of anything could go into more depth about anything) and currently only states:

This could be configured with target_vendor = "preview2" or target_os = "wasi_preview2" for example (TBD).

This PR is sort of where that TBD is being resolved. Given the motivation for adding a brand new target to rustc this is what additionally motivates the addition of a new target_os entry.

I would ideally like to rename the target_os entry of the wasm32-wasi target as part of the rename to wasm32-wasi-preview1 but that's probably too much breakage to take on so it likely won't happen at this time.

[CryZe]

I do think that using target_env here is a whole lot cleaner. Funnily enough I have a perfectly timed example for as to why I think this is. Before I knew about this PR, I was actually preparing a PR for serde that adds a cfg(target_os = "wasi") to its OsStr handling. This is not code that does any direct calls to any low level WASI functions. I also did a quick search on GitHub for code that uses wasi as its target_os and it seems like almost all code is like this. And that's not very surprising either. Low level crates such as std already in general are concerned about OS versions, where certain Windows API functions or so are only available on Windows 11 for example. However almost all crates above are doing perfectly fine by checking for the general operating system, not any version. This seems to be the same for WASI.

In the case of serde, my PR would've silently been broken by the target_os being changed. From what I'm seeing in the GitHub search, this also is true for a lot of the code I'm seeing. Most of the remaining code would fail to compile... but it had no reason to change, because it never calls into any low level code either. So there's only a tiny fraction of code that would benefit from a changed target_os whereas the rest either unnecessarily stops compiling or even silently misbehaves. Also even if a crate would still accidentally use preview1 functions, isn't there some backwards compatibility in the runtimes? So overall I'm very unconvinced and even concerned if the target_os changes (because of the silent breakage in e.g. serde) and would heavily prefer target_env or target_vendor.

[alexcrichton]

I think the breakage and what to do can go both ways, so at least in my mind there's not a clear-cut answer of what the best route here is. As you've pointed out a con of changing target_os is that all existing code will need to be updated. This includes code which doesn't interact with wasi_snapshot_preview1 imports directly, such as path-related functionality that you're talking about. A pro, however, is that a new string acknowledges that wasi_snapshot_preview1 is no longer natively supported and is now required to go through an "adapter" and it's best to use component definitions directly.

To me one way to go about solving this is to ignore historical precedent (e.g. the preexisting wasm32-wasi target), decide on a solution, and then see if it can be worked into today's systems. In that sense the preview1 and preview2 targets represent entirely different "OS" definitions (the term "OS" here is a bit loose granted). They're entirely separate embedding APIs which overlap in functionality but to me it's similar-ish to Windows vs Unix. Both have the ability to open a file and such but have very different ways of going about it. Similarly preview1 and preview2 can both open files but behave pretty differently in terms of the fine details (e.g. something that's ignored from the I'm-using-libstd perspective but matters a lot if you're programming directly against the interfaces).

Given all that I at least personally think it makes sense to have different target_os strings. Not to say there aren't similarities to the two targets, however. As you've pointed out the structure of paths being byte-based things with / separators is shared in common across the two targets. This I think is solid motivation for target_family or some similar cfg. That way code can be written to span both.

If you'll continue to walk this hypothetical with me, the next question is that if this state of affairs is desirable does it make sense to retrofit onto the existing ecosystem. Personally I think so. The new component-based target is "new enough" that I'm not sure it's worth it to keep everything working seamlessly as much as possible. This is a good opportunity to rethink what WASI support means and how best to support it. For example libc support is changing, the standard library's platform-specific extensions are changing, the capabilities of the platform are different, etc. All existing code will continue to work for preview1 but code will need to be explicitly ported to preview2.

---

That's at least my thinking, although I don't know how best to resolve this still myself. I personally think it's best to go with target_os, but I'm certainly not the only stakeholder here!

[rylev]

I don't have strong feelings on how to differentiate between preview1 and preview2, but I do have strong opinions that there needs to be some way of doing it. I agree with @alexcrichton that preview2 is different enough from preview1 that it can argued that preview2 is an entirely different OS, but it can also be argued that preview2 is just a newer version of the same logical operating system (albeit with very deep and significant changes!).

Looking at how the question of new versions of operating systems are already handled in rustc, there's unfortunately not good, clear precedent for what to do. Windows based targets that want to target older versions that what x86_64-pc-windows-msvc does, use target_vendor to do so, but this seems like a strange choice to me. On the Linux side, newer kernel and libc versions are typically targeted through target_env.

What all of this seems to indicate is that there is likely no obviously correct answer, and so we should pick something that:

• allows one to easily differentiate between preview1 and preview2
• limits the amount of churn in existing crates ecosystem as there will already be quite a bit of churn
• seems like a reasonable and easy to understand choice

Talking with a few others, it does seem like perhaps target_env is the best choice to accomplish the goals above. Unless folks feel strongly some other way, I can make the change over to target_env.

[ranile]

One more thing worth noting is that target_os = wasi is used to check if the WASM compilation is for non-browser targets. Here we use wasm-bindgen or tokio depending upon where the WASM output is supposed to run: https://github.com/yewstack/prokio/blob/master/src%2Flib.rs#L59-L64

Not breaking this use case is very important so target_env is the best way to go

[alexcrichton]

I was talking with Ryan a bit off-thread as well and I agree now too that target_env is the way to go 👍

[m-ou-se]

👍 for /library/*, but this needs another reviewer for the other parts:

r? compiler

[petrochenkov]

The rustc_target infra looks good to me, but I'm not familiar with wasi specifics.
Unless some more wasi-specific review is required, r=me after rebase and the target_env changes (#119616 (comment)).
@rustbot author

[alexcrichton]

For historical reference and if anyone's interested, here's the notes for the last meeting where the naming of "p2" vs "preview2" was discussed. The salient points in the notes, to copy/paste here, are:

Pat Hickey: We’ve received feedback from our community that the word Preview throws them off because they don’t know what it means. We are already using semver. We use it in the WIT. Most developers already know semver. When we say WASI is at 0.2 right now that can get across better than Preview 2 is that we’re not at 1.0. And 0.2.2, 0.2.3, etc means that we’re not breaking anything. Let’s communicate better by using terms most folks are already familiar with. WASI 0.2 is recommended for most audiences. This is what you’ll see in wit docs and import names.

...

Pat Hickey: I’d like to salute the go community for seeing this problem coming. Go decided to use wasm32-wasip1 and wasm32-wasip2. Folks in the rust community saw that and also has chosen to adopt this triple.

This was discussed a bit more in-depth in that meeting that the notes don't capture super well, but the consensus was that the "p2" naming is the best going forward as it means "preview 2" in the standards context, "point two" to folks who just want to know what it is, it aligns with what others are already leaning towards (Go), and it's easier to type.

[jeffparsons]

@petrochenkov or @m-ou-se would one of y'all be ok re-r+-ing this to go back into the queue?

Okay, now it looks like it's ready for Bors. 😊

[m-ou-se]

@bors r=petrochenkov,m-ou-se

[bors]

📌 Commit 5e9bed7 has been approved by petrochenkov,m-ou-se

It is now in the queue for this repository.

[bors]

⌛ Testing commit 5e9bed7 with merge ef32456...

[bors]

☀️ Test successful - checks-actions
Approved by: petrochenkov,m-ou-se
Pushing ef32456 to master...

[rust-timer]

Finished benchmarking commit (ef32456): comparison URL.

Overall result: ✅ improvements - no action needed

@rustbot label: -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.1%	[-0.1%, -0.1%]	1
All ❌✅ (primary)	-	-	0

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-5.1%	[-5.1%, -5.1%]	1
All ❌✅ (primary)	-	-	0

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 651.173s -> 651.239s (0.01%)
Artifact size: 311.15 MiB -> 311.17 MiB (0.01%)

[RalfJung]

Would be nice to see the PR name and description updated to match what actually landed (in particular regarding the target name). This just confused be quite a bit.^^