Migration 2019-04-28-201506_create_tantivy_index is failing to run
#702
Closed
opened 5 years ago by rhaamo
·
3 comments
No Branch/Tag Specified
paginate-search-init
main
s3
fix-delete-user
timeline-cli
blog-title
signature
remove-dup-images
ldap-non-anon
drone-ci
DearRude/force-lang
igalic/go/async-all-mut
go/async
floreal/translations-update
missing-docs
RAOF/fix-arm64-build
epsilon-phase/authorized-fetch
upgrade
improve-the-editor-once-again
igalic/feat/custom-fairing-domains
feature/ldap
test/dotenv_error
fix-mobile-margin
0.7.2
0.7.0
0.2.0-alpha-1
0.3.0-alpha-2
0.4.0-alpha-4
0.5.0
0.6.0
0.7.1
Labels
Clear labels
Related to the REST API
Code running on the server
Stuff related to Federation
Related to the front-end
Translations, and related code
More about project management or code than the project itself
The building, or installation process of Plume
Something isn't working
We need to talk
New feature or request
This is a new feature
Compatibility with different browsers, readers and OS
Related to an external package that Plume uses
UI/UX related issues and PRs
Good for newcomers
Extra attention is needed
Issues affecting only mobile UX
How elements're rendered out for the end user
Something else needs to be fixed first
This issue or pull request already exists
This PR is not complete yet
Issues concern a limited number of instances
This doesn't seem right
Need to be discussed by the community (on Loomio)
This PR is ready to be reviewed
Proposed ideas worth considering
This is issue has been created after a vote on Loomio
This will not be worked on
Apply labels
A: API
Related to the REST API
A: Backend
Code running on the server
A: Federation
Stuff related to Federation
A: Front-End
Related to the front-end
A: I18N
Translations, and related code
A: Meta
More about project management or code than the project itself
A: Security
Build
The building, or installation process of Plume
C: Bug
Something isn't working
C: Discussion
We need to talk
C: Enhancement
New feature or request
C: Feature
This is a new feature
Compatibility
Compatibility with different browsers, readers and OS
Dependency
Related to an external package that Plume uses
Design
UI/UX related issues and PRs
Documentation
Good first issue
Good for newcomers
Help welcome
Extra attention is needed
Mobile
Issues affecting only mobile UX
Rendering
How elements're rendered out for the end user
S: Blocked
Something else needs to be fixed first
S: Duplicate
This issue or pull request already exists
S: Incomplete
This PR is not complete yet
S: Instance specific
Issues concern a limited number of instances
S: Invalid
This doesn't seem right
S: Needs Voting/Discussion
Need to be discussed by the community (on Loomio)
S: Ready for review
This PR is ready to be reviewed
Suggestion
Proposed ideas worth considering
S: Voted on Loomio
This is issue has been created after a vote on Loomio
S: Wontfix
This will not be worked on
No Label
A: API
A: Backend
A: Federation
A: Front-End
A: I18N
A: Meta
A: Security
Build
C: Bug
C: Discussion
C: Enhancement
C: Feature
Compatibility
Dependency
Design
Documentation
Good first issue
Help welcome
Mobile
Rendering
S: Blocked
S: Duplicate
S: Incomplete
S: Instance specific
S: Invalid
S: Needs Voting/Discussion
S: Ready for review
Suggestion
S: Voted on Loomio
S: Wontfix
Milestone
Set milestone
Clear milestone
No items
No Milestone
Assignees
Assign users
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.
No due date set.
Dependencies
No dependencies set.
Reference: Plume/Plume#702
Reference in New Issue
There is no content yet.
Delete Branch '%!s(<nil>)'
Deleting a branch is permanent. It CANNOT be undone. Continue?
No
Yes
Updated to latest git from an old plume and the migration
2019-04-28-201506_create_tantivy_index
doesn't run (postgresql).this is probably due to the migration using the latest Database Model, but with an old database, because
users.role
is created in migration2019-06-18-152700_moderator_role
later.I can't reproduce but I'm pretty sure I know what the issue is. Could you run the same command with
RUST_BACKTRACE=1
? I would expectSearcher::add_document
to appear probably as the first non std call in the backtrace.The following is my understanding from your report, but would require a bit more testing to make sure it's actually the issue.
Short story:
#555 introduced arbitrary rust in migration, that is totally broken.
Long story
At compile time, the schema used in rust structures is always the one of latest migration, and it is assumed that migrations will be run before trying to query, which works pretty much always. But in this case, some rust code is run before end of migrations (it is a migration), and so it might query the database (again, it is a migration), but will expect the model to be as it will be after the last migration will be run. This is a big design flaw, and I think the best way to handle it is to remove such migrations for now, and search for better ideas or some other way to do. While that seemed like a good idea at the time, I don't see any clean way of handling this very non rare situation in the future.
Here is the full backtrace:
Yeah I've thought of the same thing, unfortunately there is no magic solution for data migrations mixed with schema ones.
I had issues with data migrations in my schema ones on my projects and either I was able to use the ORM without binding a schema by querying directly the right things, or I ended up moving all the data migrations in separate CLI commands.
The goal of such migrations was to reduce such separate commands (the exact migration failing for you is strictly equivalent to
plm search init
, although running it won't mark the migration as run).The fix will probably be to remove this migration, reintroduce it's command as an installation step, and remove one other, and try to rewrite it in sql