This is document describes the Design Goals and Refactoring Strategies of our ongoing efforts to improve the architecture, and with it the stability and performance of Plume.
Let's look at the current architecture's problems, and the attempts we've made at resolving them.
This data structure was introduced by yours truly with the intent to please Clippy, reduce the number of arguments passed around (the main thing Clippy complained about). We also tried reduce the amount of bytes being passed around, by using references.
At the time, this seemed like a good idea. Right now, it's the main source of our problems.
DbConn, which makes it difficult migrate Plume to
Passing around a giant
async world, means that different owners are waiting for it to be
.awaited, so they can access them.
This makes working with such a
struct very unwieldy, if not impossible sometimes.
DbConn, Searcher and Post
Searcher are deeply bundled via
Searcher is called every time a
Post is added/updated/deleted.
It needs access to
DbConn to fill the data that
Post does not have.
While removing one or the other from
PlumeRocket is possible, complications still arise with
AsObject APIs are called every time a Post is added/updated/deleted.
It builds on
PlumeRocket as main
Context, and so we'd have to touch every API if we split either
Searcher out of
Solution Attempts and their Problems
in the past couple of weeks, we've made the following attepts to at least partially dissolve
- plume-model: refactor Searcher to have its own DbPool
- WIP: Experiment: extract Searcher into an Actor
- extract DbConn from PlumeRocket
As soon as we attempted to delete out one of the members from
PlumeRocket, compiles would fail all over the place, meaning we'd have to touch almost every single function's signature that uses
This then means we'd have to touch every single function that in turn use those functions!
That is a lot of broken code, before we've even started refactoring.
Despite ambitions to use an Actor System (Riker), it is not magnitude of the ambitions, but the size of the steps we've taken.
So, given past failures we devise a strategy.
We want to replace each of the systems in
PlumeRocket with an
Actor, accessed by a single reference to the
This way we don't have to touch every single function's parameters that
PlumeRocket flows thru, while the code is still in motion.
DbPool instead of DbConn
In our previous attempts at this code, we've realized that
DbPool, being wrapped in an
Arc is very cheap to
This means that every
Actor that needs a
DbConn, could get its own
We also realized that
DbPool acts like an
Actor in its own right:
- when that message is sent, it responds with a
- if the pool is exhausted, it does not!
Thus, we conclude:
We want to
DbPool for every Actor that we extract from
PlumeRocket that needs it.
In #799, we've identified the following
Here is the list of things the worker is used for:
- sending activities to other instances (just needs the activity and a list of recipients)
- rotating user keypair when they delete a post (after 10 minutes), which requires the DB
- fetching posts for a blog/users, either because it is new or because it has not been updated in 24 hours (depends on the DB too, and on the searcher)
For the first type of worker, we'll have to make repeated network requests. There's a Riker issue asking how to best do that (with an answer.)
The two workers that need access to the database, should get their own
Being part of the same
ActorSystem, the last type of worker will be able to access
Searcher thru messages.
Request Path vs DbConn vs async
For the Rocket Request path, we want to wrap
DbPool in an
Actor of its own:
Then, in the
RequestGuard we'd ask for a
This brings us on-par with Rocket's current
#[database] approach, that puts database connection access into
ask() responds with a Future, which would mean that in
async functions we could simply
The long road
Once we've refactored the main systems in
PlumeRocket into their own Actors, and once we're down to only the
ActorSystem being the main component of
PlumeRocket, we can finally shed the husk.
That means that we do go and touch all the functions that take
PlumeRocket and only give them access to the things they need.
This is also a good chance to look at functions that are too long, or are doing to much, and mark them for refactoring.