#799 [Refactoring] send messages to dedicated actor instead of directly accessing objects

Відкрито
4 місяці тому відкрито igalic · 7 коментарів
igalic прокоментував(ла) 4 місяці тому

Instead of passing around references to directly access worker, and searcher, we could instead send messages to an actor!

This would also help split responsibility of these functions, as they would no longer invoke these actions themselves, only sending messages to invoke them.

This would allow to remove searcher and worker from our PlumeRocket. Together with #797, this could help completely dissolve the struct.

/// Common context needed by most routes and operations on models
pub struct PlumeRocket {
    pub conn: DbConn,
    pub user: Option<users::User>,
    pub searcher: Arc<search::Searcher>,
    pub worker: Arc<ScheduledThreadPool>,
}

How?

We could use an actor system, like riker.
Using multiple messages we could send different messages to different actors that then do what needs to be done

Instead of passing around references to directly access `worker`, and `searcher`, we could instead send messages to an actor! This would also help split responsibility of these functions, as they would no longer invoke these actions themselves, only sending messages to invoke them. This would allow to remove searcher and worker from our PlumeRocket. Together with #797, this could help completely dissolve the struct. ```rust /// Common context needed by most routes and operations on models pub struct PlumeRocket { pub conn: DbConn, pub user: Option<users::User>, pub searcher: Arc<search::Searcher>, pub worker: Arc<ScheduledThreadPool>, } ``` ### How? We could use an actor system, like [riker](https://riker.rs/). Using [multiple messages](https://riker.rs/messaging/) we could send different messages to different actors that then do what needs to be done
kiwii згадано цю проблему в коміті 4 місяці тому
kiwii прокоментував(ла) 4 місяці тому
Власник

Honestly, it makes me wonder if actix wouldn’t be more appropriate for our use case… we would get actors and a web framework out of the box. I think we already tried to migrate once and it didn’t went very far, right? What was blocking?

But I’m okay with refactoring our code with Riker and keeping Rocket too.

Honestly, it makes me wonder if actix wouldn't be more appropriate for our use case… we would get actors and a web framework out of the box. I think we already tried to migrate once and it didn't went very far, right? What was blocking? But I'm okay with refactoring our code with Riker and keeping Rocket too.
igalic прокоментував(ла) 4 місяці тому
Власник

the main reasons i abandoned the actix experiment were:

  • it’s effectively a rewrite
  • we’re actually one of the very few (two?) ActivityPub servers not written with Actix!

(the other one isn’t even public yet, so that’s actually quite nice :P)

the main reasons i abandoned the [actix experiment](https://github.com/Plume-org/plume-async) were: - it's effectively a rewrite - we're actually one of the very few (two?) ActivityPub servers not written with Actix! (the other one isn't even public yet, so that's actually quite nice :P)
igalic прокоментував(ла) 4 місяці тому
Власник

I’ve created a basic example with riker and multi-type messages for how to replace Searcher: https://gist.github.com/igalic/16ec36a4c9277b3b46f6bea032226189

in a similar fashion, we can replace Worker, which currently mostly sents broadcast messages to all subscribers.

I've created a basic example with riker and multi-type messages for how to replace Searcher: https://gist.github.com/igalic/16ec36a4c9277b3b46f6bea032226189 in a similar fashion, we can replace Worker, which currently mostly sents broadcast messages to all subscribers.
igalic прокоментував(ла) 4 місяці тому
Власник

here’s what I’ve found out in an experiment attempting to extract Searcher into an Actor:

  • Searcher is intimately bundled with Post
  • despite that, Searcher is also intimately bundled with DbConn, because Post doesn’t actually have all the info Searcher needs

given this asymmetry, i thought it’s best to only pass Post.id

but we still have to give it access to a DbConn

perhaps on creation of a Searcher Actor, we could give it its own DbConn?

here's what I've found out in an experiment attempting to extract `Searcher` into an `Actor`: - `Searcher` is intimately bundled with `Post` - despite that, `Searcher` is also intimately bundled with `DbConn`, because `Post` doesn't actually have all the info `Searcher` needs given this asymmetry, i thought it's best to only pass `Post.id` but we still have to give it access to a `DbConn` perhaps on creation of a `Searcher` `Actor`, we could give it its own `DbConn`?
igalic прокоментував(ла) 4 місяці тому
Власник

Some observations, which lead to the confidence with which we’ve started on #807:

DbPool is an Arc, it can be easily shared via .clone().
This would allow us to create a wrapper object around Searcher which can then have its own DbConn instead of passing one from a — hopefully — short-lived Request, potentially infinitely blocking on it.

The general framework lends itself nicely to be solved with an actor framework, but if that doesn’t pan out, we can use any old object, and send it any old message!
We’ve already done good work there in extracting the essence.

It bears repeating why the refactoring to break up PlumeRocket is so important:

We pass PlumeRocket around as ref.
It’s a giant struct with lots of maybe-related data, giving access to different parts of the system.
Passing it, by ref thru Rocket will become an issue as soon as we try to go async, because then we have a borrowed object sitting somewhere, waiting to be .awaited.
This is all a big mess, and best of all avoided, but it’s also a good opportunity to see which parts of the system we can tear apart from there tight coupling so they will work better.

Some observations, which lead to the confidence with which we've started on #807: `DbPool` is an `Arc`, it can be easily shared via `.clone()`. This would allow us to create a wrapper object around `Searcher` which can then have its own `DbConn` instead of passing one from a — hopefully — short-lived `Request`, potentially infinitely blocking on it. The general framework lends itself nicely to be solved with an actor framework, but if that doesn't pan out, we can use any old object, and send it any old message! We've already done good work there in extracting the essence. It bears repeating why the refactoring to break up PlumeRocket is so important: We pass PlumeRocket around as `ref`. It's a giant struct with lots of maybe-related data, giving access to different parts of the system. Passing it, by `ref` thru Rocket will become an issue as soon as we try to go `async`, because then we have a borrowed object sitting somewhere, waiting to be `.await`ed. This is all a big mess, and best of all avoided, but it's also a good opportunity to see which parts of the system we can tear apart from there tight coupling so they will work better.
kiwii прокоментував(ла) 4 місяці тому
Власник

Here is the list of things the worker is used for:

  • sending activities to other instances (just needs the activity and a list of recipients)
  • rotating user keypair when they delete a post (after 10 minutes), which requires the DB
  • fetching posts for a blog/users, either because it is new or because it has not been updated in 24 hours (depends on the DB too, and on the searcher)
Here is the list of things the worker is used for: - sending activities to other instances (just needs the activity and a list of recipients) - rotating user keypair when they delete a post (after 10 minutes), which requires the DB - fetching posts for a blog/users, either because it is new or because it has not been updated in 24 hours (depends on the DB too, and on the searcher)
igalic прокоментував(ла) 4 місяці тому
Власник

for the first case, i already have opened a question: https://github.com/riker-rs/riker/issues/130
and received an answer

for the first case, i already have opened a question: https://github.com/riker-rs/riker/issues/130 and received an answer
Підпишіться щоб приєднатися до обговорення.
Етап відсутній
Немає виконавеця
2 учасників
Сповіщення
Дата завершення

Термін виконання не встановлений.

Залежності

Ця проблема в даний час не має залежностей.

Завантаження…
Тут ще немає жодного змісту.