Better handling of background jobs #622
Labels
No labels
A: API
A: Backend
A: Federation
A: Front-End
A: I18N
A: Meta
A: Security
Build
C: Bug
C: Discussion
C: Enhancement
C: Feature
Compatibility
Dependency
Design
Documentation
Good first issue
Help welcome
Mobile
Rendering
S: Blocked
S: Duplicate
S: Incomplete
S: Instance specific
S: Invalid
S: Needs Voting/Discussion
S: Ready for review
Suggestion
S: Voted on Loomio
S: Wontfix
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: Plume/Plume#622
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Is your feature request related to a problem? Please describe.
For the moment, there is no way to retry a failed job. And if the app is stopped, all queued jobs are lost.
Describe the solution you'd like
Having a
jobs
table that stores queued jobs, and use a worker pool to regularly pull them and run them. Rustodon has a good implementation of background jobs that does that IIRC, and we could probably get inspiration from here (maybe it was even published as a crate that we could just pull?).Describe alternatives you've considered
Storing these jobs elsewhere like in Redis or RabbitMQ as it is more usually done (and there are probably crates to handle most of the basic logic for that), but I don't really like adding another dependency, so going for a database table seems better (and also, installing Redis when you want to run a very small instance is… meh).
Additional contextWhat particular jobs would be useful to support?
For the moment, I can think of:
at the moment, background threads are used for federation (broadcast and fetch), and key rotation upon deleting comments