multiple states and faster ipc

[imyxh]

So, two related things:

1. Sometimes a loop might want to have multiple vectors/matrices/etc.; for example, maybe device A wants to output two things which are then read by devices B and C, or maybe device D wants to send something to device F, but without replacing the main pipeline state because device E needs it too.
2. The current paradigm for doing IPC between anyloop and e.g. a plotting program in Julia is to use a named pipe. However, this has two disadvantages:
   1. named pipes have a limited buffer size, so Julia will eventually start blocking anyloop
      • this is good for applications where we don't want to miss data, but not for e.g. a realtime view of a matrix
   2. named pipes involve copying to kernel space and back, which can be slow.

My current proposal is to fix both of these issues by using proper mmaped shared memory. There are a few considerations for this, however:

1. We can't use pointers in shared memory.
2. Care will have to be taken with regards to allocating GSL objects to make their underlying data reside in shared memory.
3. We probably want mutexes too.
4. Julia will need a package or something.

[imyxh]

I guess the multiple states part is trivial to implement, since the GSL objects don't store data themselves.

But coordinating the underlying storage for each state (each block of the state? nomenclature TBD ...) to lie on the shm without copying is much more annoying, since part of what makes coding for anyloop so elegant is that the state is just a pointer to the device's own data. In fact, many devices don't even have control over where the memory they use is allocated. So in this case you always have to copy at least once if you want IPC. Maybe we just deal with copying and implement a anyloop:copy_to_shm device?

I don't think it's worth tackling these more niche problems until we have an actual use case for them. But I wanted to write out my thoughts for when it does come up.

[imyxh]

For now, the only actual annoyance we've experienced is the Julia blocking issue, which could be fixed by:

1. adding a anyloop:copy_to_shm device
2. using header.status as a mutex
3. reading from shm with Julia, obeying the mutex.

And in the future when multiple states (banks? blocks?) are implemented then copy_to_shm can just take a parameter to select the source.

[imyxh]

I guess the device can just be anyloop:shm, and can take care of creating the shm object too. Then a source parameter can tell it whether to copy from the shm instead of to it.