Federated blogging application, thanks to ActivityPub https://joinplu.me
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 
 
 

5.5 KiB

Refactoring

This is document describes the Design Goals and Refactoring Strategies of our ongoing efforts to improve the architecture, and with it the stability and performance of Plume.

Current Issues

Let’s look at the current architecture’s problems, and the attempts we’ve made at resolving them.

PlumeRocket

This data structure was introduced by yours truly with the intent to please Clippy, reduce the number of arguments passed around (the main thing Clippy complained about). We also tried reduce the amount of bytes being passed around, by using references.

At the time, this seemed like a good idea. Right now, it’s the main source of our problems.

This struct bundles DbConn, which makes it difficult migrate Plume to async Rocket:

Passing around a giant struct as ref in async world, means that different owners are waiting for it to be .awaited, so they can access them. This makes working with such a struct very unwieldy, if not impossible sometimes.

DbConn, Searcher and Post

DbConn and Searcher are deeply bundled via Post. Searcher is called every time a Post is added/updated/deleted. It needs access to DbConn to fill the data that Post does not have.

While removing one or the other from PlumeRocket is possible, complications still arise with AsObject:

Posts’s AsObject APIs are called every time a Post is added/updated/deleted. It builds on PlumeRocket as main Context, and so we’d have to touch every API if we split either DbConn or Searcher out of PlumeRocket

Solution Attempts and their Problems

in the past couple of weeks, we’ve made the following attepts to at least partially dissolve PlumeRocket

As soon as we attempted to delete out one of the members from PlumeRocket, compiles would fail all over the place, meaning we’d have to touch almost every single function’s signature that uses PlumeRocket. This then means we’d have to touch every single function that in turn use those functions! That is a lot of broken code, before we’ve even started refactoring.

Strategy

Despite ambitions to use an Actor System (Riker), it is not magnitude of the ambitions, but the size of the steps we’ve taken. So, given past failures we devise a strategy. We want to replace each of the systems in PlumeRocket with an Actor, accessed by a single reference to the ActorSystem. This way we don’t have to touch every single function’s parameters that PlumeRocket flows thru, while the code is still in motion.

Actors

in #807, we’ve already made our first attempt at extracting Searcher into a Riker Actor. Here, just like in #809, we’ve already given Searcher its own DbPool.

Why?

DbPool instead of DbConn

In our previous attempts at this code, we’ve realized that DbPool, being wrapped in an Arc is very cheap to .clone(). This means that every Actor that needs a DbConn, could get its own DbPool. We also realized that DbPool acts like an Actor in its own right:

  • DbPool has a .get() message
  • when that message is sent, it responds with a DbConn
  • if the pool is exhausted, it does not!

Thus, we conclude: We want to .clone() a DbPool for every Actor that we extract from PlumeRocket that needs it.

Workers

In #799, we’ve identified the following workers:

Here is the list of things the worker is used for:

  • sending activities to other instances (just needs the activity and a list of recipients)
  • rotating user keypair when they delete a post (after 10 minutes), which requires the DB
  • fetching posts for a blog/users, either because it is new or because it has not been updated in 24 hours (depends on the DB too, and on the searcher)

For the first type of worker, we’ll have to make repeated network requests. There’s a Riker issue asking how to best do that (with an answer.)

The two workers that need access to the database, should get their own DbPool. Being part of the same ActorSystem, the last type of worker will be able to access Searcher thru messages.

Request Path vs DbConn vs async

For the Rocket Request path, we want to wrap DbPool in an Actor of its own:

Then, in the RequestGuard we’d ask for a DbConn. This brings us on-par with Rocket’s current #[database] approach, that puts database connection access into spawn_blocking(). However, unlike #[database], Riker’s ask() responds with a Future, which would mean that in async functions we could simply .await it!

The long road

Once we’ve refactored the main systems in PlumeRocket into their own Actors, and once we’re down to only the ActorSystem being the main component of PlumeRocket, we can finally shed the husk. That means that we do go and touch all the functions that take PlumeRocket and only give them access to the things they need.

This is also a good chance to look at functions that are too long, or are doing to much, and mark them for refactoring.