A note on persistence efficiency

[Alex North]

My most recent change loads all waves from storage into memory when the
server starts up. This is clearly not a long term solution, but a short term
way to get the system to behave similarly to how it does now. I'm going to
leave it in this state for now. There are at least two things that need
fixing before we can allow some waves to exist on disk only, not in RAM.

- The WaveServer and WaveMap classes need to pass requests to list all wave
ids directly through to storage, so that the store can check the filesystem
(or whatever) without necessarily loading waves entirely into RAM in order
to enumerate them
- The index wave needs rework so as not to require initialisation from every
wave at start-up

This latter most likely requires secondary storage for the index, which is
something Tad and I have been investigating.

I'm not planning to fix these right now, but wanted to leave this note for
whenever someone gets round to it.

[JeanV]

Is this change available in the source code? Can we have access to
this functionality? I've updated the source code but this
functionality does not seem to be present.

Post by Alex North
My most recent change loads all waves from storage into memory when the
server starts up. This is clearly not a long term solution, but a short term
way to get the system to behave similarly to how it does now. I'm going to
leave it in this state for now. There are at least two things that need
fixing before we can allow some waves to exist on disk only, not in RAM.
- The WaveServer and WaveMap classes need to pass requests to list all wave
ids directly through to storage, so that the store can check the filesystem
(or whatever) without necessarily loading waves entirely into RAM in order
to enumerate them
- The index wave needs rework so as not to require initialisation from every
wave at start-up
This latter most likely requires secondary storage for the index, which is
something Tad and I have been investigating.
I'm not planning to fix these right now, but wanted to leave this note for
whenever someone gets round to it.