Venus/VirtIO over a network treating WASM as gues OS and far away server as Host

[devshgraphicsprogramming]

The Venus host device shall always run with basic validation and every form of robustness you can enable. Issue vk_device_lost and break connection with guest upon first validation error. We can experiment with syncval and GPU AV in the future but that's for shipping things and subject to no performance bottlenecks.

Unfortunately IPC via sockets sucks
https://github.com/goldsborough/ipc-bench

Sure the machine sucks by today's standards but it's not like you'll get deviations by an order of magnitude.

In the context of 60fps rendering this means "only" 1000 messages with Websockets and you can only hope for 4-50x improvements with native messaging
https://codereview.stackexchange.com/questions/299482/native-messaging-performance-test
https://gist.github.com/TotallyNotChase/8c9e79ed967ae911869aec82545b21ae
https://github.com/TotallyNotChase/lib-native-messaging/tree/master?tab=readme-ov-file#connectionful-messaging-1

Given that your average Vulkan app does bazillions of calls per frame, we need to buffer up and delay certain calls so they can be batched and there's less small message spam. Check if Venus already does this.

The primary example of things which can be buffered up is command buffer recording, you should only send it over once end().
has been called. Or more extremely if it hasn't been sent since last end().call and it's used in a submit, although that incurs latency.

If you rename vulkan objects and give them different handles on the guest (which you should do for security), then you can escape having to send a message for things like image, buffer, etc creation right away. Map and unmap are also things which don't need serialising because you get your own copy of the range in wasm.

The only things you can't buffer up and defer are sync primitive operations, submits, device memory allocation, mapped range flushes and invalidations. Everything else I can see flushing in a deferred manner whenever these former operations run.

Default behaviour should be to not advertise any Coherent memory.

However because most Vulkan code in the wild won't work without coherent and we want Crysis in a browser, we need an override which allows coherent and deals with it by copying every mapped range (what Renderdoc used to do for coherent and persistent mappings in GL and VK) to the host upon a submit, and back when a fence or timeline sema is awaited on.

The architecture should allow range tracking (in page size increments of 16kb) to be implemented in the future so we can narrow down what to copy. Ranges bound to buffers with BDA must always be considered in use.

Without being able to track pages accessed by the guest (also with networked impl, when guest taps a mapped page it's already too late to start sending) every time a client performs a domain operation from device to host (vulkan host not host host I terms of Venus) you need to preemptively start a transfer of all mappings which could be dirty. This means tracking if dirty resources would have to be per queue, and every time you'd wait in a signal you'd have to do an analysis of what to pop not only from the signalling queues' dirty range list but also other queues because the signalling queue may have had a dependency in the past with them.

For now just send everything, well figure this out later if perf is needed.

P.S. for non-COHERENT memory we should report a pretty large nonCoherentAtomSize

[devshgraphicsprogramming]

Basically every coherent range mapped between last submit and current, intersected with every range still bound to a resource existing at current submit time should be sent right away upon a submit.

Stuff that was mapped but is not viewable through any resource can be sent later, but still needs to be tracked so not forgotten about.

E.g. if a buffer or image view (image too cause copies and blitz but you can track that) is later made over such a range and another submit happens we need to remember to flush that range.

[devshgraphicsprogramming]

For the download, every coherent range stil mapped at syncval primitive wait needs.to be considered.

But from that only subranges that had resources bound to them during previous submits we haven't transitively "seen yet".

Again as a starting point we can drop.the resource binding analysis.

[devshgraphicsprogramming]

Bonus I don't thing Vulkan spec mandates that unmap be a domain operation.

So while a map of a coherent memory requires you immediately send back the whole range because you need to populate a local view.

When you unmap, you need a queue submit or host timeline sema signal to perform domain ops, so I think it's 100% legit and valid to make writes happening between the last one of those UB and unobservable to the GPU until the next submit.
This obviously would mean that the mapping array would need to stay alive if you unmap and have unflushed writes, until those writes are flushed.

One caveat is that if you unmap and then map.again the CPU should be able to see those writes so map should also send any old unflushed writes along with mapping request before receiving data.

[devshgraphicsprogramming]

Btw because of the mapping overhead, we need to lie a bit
https://vulkan.gpuinfo.org/displayreport.php?id=39716#memory

We need to not advertise a non CACHED memory (mappings will always be host CACHED).

We also need to prevent a systems from advertising all of their very large heaps (System RAM, and ReBAR) as COHERENT memory types, our Venus server should have a setting how many MB of DEVICE_LOCAL and non DEVICE_LOCAL it will allow to be used as a heap.

Since there are still non ReBAR systems out there in the wild even for the latest GPUs because of bad Mobo and CPU combos, apps should be well behaved when we limit the size of the COHERENT heap to less than 220MB.

Basically we need to split those heaps in two (which reduces the size of the non COHERENT part of the heap) or lie and add an extra heap (which makes you appear to have a heap few extra MB so were you to allocate everything you'd run out of space)

Note that Vulkan spec says that implementation must only have HOST_VISIBLE and COHERENT, we can disable reporting COHERENT on any DEVICE_LOCAL if non-device local memory type exists.

[devshgraphicsprogramming]

Given that we have about 2GB/s per thread from a single websocket (we should probably use a separate one for flush and invalidate), the amount of COHERENT memory should be limited to

64MB for 30 Hz
32MB for 60 Hz
16MB for 120 Hz

Another way to get around this issue is to return VK_ERROR_MEMORY_MAP_FAILED when there are too many ranges getting mapped, but most apps/engines probably don't handle this gracefully.

However because we'll only transfer the portions of mapped ranges that are bound to any resource, and only when there are submits that could access them (resource is BDA or extracted as a descriptor + there are draw or dispatch commands). The issue is that you can have more memory mapped than what actually needs to be sent but you can only limit based on whats mapped (Because if you allow mapping first and then memory is bound, the total amount to send increases).