WIP: Extract DbConn, Searcher & Worker into Riker Actors
#813
Open
igalic
wants to merge 18 commits from igalic/Plume:refactor/extract-actors
into main
pull from: igalic/Plume:refactor/extract-actors
merge into: Plume:main
Plume:DearRude/force-lang
Plume:RAOF/fix-arm64-build
Plume:blog-title
Plume:drone-ci
Plume:epsilon-phase/authorized-fetch
Plume:feature/ldap
Plume:fix-mobile-margin
Plume:floreal/translations-update
Plume:go/async
Plume:igalic/feat/custom-fairing-domains
Plume:igalic/go/async-all-mut
Plume:improve-the-editor-once-again
Plume:ldap-non-anon
Plume:main
Plume:missing-docs
Plume:remove-dup-images
Plume:signature
Plume:test/dotenv_error
Plume:timeline-cli
Plume:upgrade
Reviewers
Request review
No reviewers
Labels
Related to the REST API A: Backend
Code running on the server A: Federation
Stuff related to Federation A: Front-End
Related to the front-end A: I18N
Translations, and related code A: Meta
More about project management or code than the project itself A: Security Build
The building, or installation process of Plume C: Bug
Something isn't working C: Discussion
We need to talk C: Enhancement
New feature or request C: Feature
This is a new feature Compatibility
Compatibility with different browsers, readers and OS Dependency
Related to an external package that Plume uses Design
UI/UX related issues and PRs Documentation Good first issue
Good for newcomers Help welcome
Extra attention is needed Mobile
Issues affecting only mobile UX Rendering
How elements're rendered out for the end user S: Blocked
Something else needs to be fixed first S: Duplicate
This issue or pull request already exists S: Incomplete
This PR is not complete yet S: Instance specific
Issues concern a limited number of instances S: Invalid
This doesn't seem right S: Needs Voting/Discussion
Need to be discussed by the community (on Loomio) S: Ready for review
This PR is ready to be reviewed Suggestion
Proposed ideas worth considering S: Voted on Loomio
This is issue has been created after a vote on Loomio S: Wontfix
This will not be worked on
Apply labels
Clear labels
A: API
Related to the REST API A: Backend
Code running on the server A: Federation
Stuff related to Federation A: Front-End
Related to the front-end A: I18N
Translations, and related code A: Meta
More about project management or code than the project itself A: Security Build
The building, or installation process of Plume C: Bug
Something isn't working C: Discussion
We need to talk C: Enhancement
New feature or request C: Feature
This is a new feature Compatibility
Compatibility with different browsers, readers and OS Dependency
Related to an external package that Plume uses Design
UI/UX related issues and PRs Documentation Good first issue
Good for newcomers Help welcome
Extra attention is needed Mobile
Issues affecting only mobile UX Rendering
How elements're rendered out for the end user S: Blocked
Something else needs to be fixed first S: Duplicate
This issue or pull request already exists S: Incomplete
This PR is not complete yet S: Instance specific
Issues concern a limited number of instances S: Invalid
This doesn't seem right S: Needs Voting/Discussion
Need to be discussed by the community (on Loomio) S: Ready for review
This PR is ready to be reviewed Suggestion
Proposed ideas worth considering S: Voted on Loomio
This is issue has been created after a vote on Loomio S: Wontfix
This will not be worked on
No Label
A: API
A: Backend
A: Federation
A: Front-End
A: I18N
A: Meta
A: Security
Build
C: Bug
C: Discussion
C: Enhancement
C: Feature
Compatibility
Dependency
Design
Documentation
Good first issue
Help welcome
Mobile
Rendering
S: Blocked
S: Duplicate
S: Incomplete
S: Instance specific
S: Invalid
S: Needs Voting/Discussion
S: Ready for review
Suggestion
S: Voted on Loomio
S: Wontfix
Milestone
Set milestone
Clear milestone
No items
No Milestone
Assignees
Assign users
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.
No due date set.
Dependencies
This pull request currently doesn't have any dependencies.
Reference in new issue
There is no content yet.
Delete Branch 'igalic/Plume:refactor/extract-actors'
Deleting a branch is permanent. It CANNOT be undone. Continue?
No
Yes
This PR addresses #799 and #797.
In #805 and #809 we have learned that
Searcher
andDbConn
are intimately connected, especially inPost
s.We want to replace
PlumeRocket
as the main context being passed around with a Riker ActorSystem, so we'll be able to access all the systems managed by the actor system. This includes the Context of our federationAsObject
s.Can you compile this branch? I couldn't...
Note: SearcherActor cannot commit indices due to
EnterError
(https://github.com/tantivy-search/tantivy/issues/898). This issue might be related: https://github.com/rust-lang/futures-rs/issues/2090.This occurs because perhaps both Riker and Tantivy use
futures::excutor::LocalPool
and Tantivy'sblock_on
is called in Riker'sLocalPool
.Possible options (based on my poor understanding of async) are:
Note that we cannot search new posts until their documents are commited not just added.
I respect your great research and implementation, and completely agree with introducing Riker. In addition, I make a suggestion: use Riker's Channels as a global variable. There are two points:
What is solved?
At current attempting (its strategy and current implementation:
8aa4ae4302
), we need touch searcher more than ever before. Also, we need flow, some agents (like searcher actor or post ids) through multiple structs.See updating search document procedure. When a user updates a post, a request handler does two (or more) things: it
UpdateDocument
message toSearcherActor
Developers need always do them together. This is needed for not only request handlers but
AsObject
andFromId
implementations for models. As such, developers need write more code than now. And they might have forgotten sending message to the actor.To make matters worse, this introduces more complexity. Assume you're deleting a user.
routes::user::delete()
request handler is called. It needs to know all blogs and posts that the deleting user owns in order to sendDeleteDocument
messeges corresponding to deleted posts to searcher actor. How?One way is:
Post::delete()
returns itself toBlog
,Blog::delete()
returns deletedPost
s toUser
,User::delete()
returns deletedBlog
s' posts to the handler, and fanally the handler sends messeges to the actor. If some posts could not be deleted, the handler need make sure to sendDeleteDocument
for each only deleted post. That's complex.The another is: the handler passes the searcher actor to
User::delete()
as an argument,User::delete()
passes it toBlog::delete()
,Blog::delete()
callsPost::delete()
and send aDeleteDocument
message to the actor several times. This is essentially same to above.Introducing a global variable solves this. Using it,
Post::insert()
,update()
anddelete()
need to neither accept a searcher as an agument nor return back itself (or its ID) to the caller. What it needs to do is getting global searcher (or so) and sending a messege that it is deleted. That's it.There are options what struct should be global like a searcher actor, an actor system or a channel. I suggest Riker's channel because...
More benefit
Use of channel introduces pub/sub model. In the case above,
Post
publishes "delete" event to a channel andSearcherActor
subscribes to the event and deletes the search index corresponding to the post. More than that, we can add other actors which concern post's deletion without touching any code of request handlers and models. Examples are:A proof of concept is here: https://git.joinplu.me/KitaitiMakoto/Plume/compare/igalic/refactor/extract-actors...channel. Pay your attention to
Rebasing
What do you think? I you aggree with me, I'm thiking it might be better to restart from current 0.6.0+ code (of course, I will do it). Any questions and discussions are welcome!
that's a good idea, i hope I'll have time to do that soon.
but i do feel like every time i attempt this i have learned something in between, and so it becomes a little bit easier
Thanks! I will start later.