Plume
/
Plume
8
16
Fork 22
Code Issues 174 Pull Requests 8 Releases 8 Wiki Activity

Labels Milestones

Count items in database as much as possible #344

Merged
elegaanz merged 6 commits from count-in-db into master 5 years ago
Conversation 5 Commits 6 Files Changed 19
elegaanz commented 5 years ago (Migrated from github.com)
There is no content yet.
👍 1
codecov[bot] commented 5 years ago (Migrated from github.com)

Codecov Report

Merging #344 into master will increase coverage by 0.14%.
The diff coverage is 21.62%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #344      +/-   ##
==========================================
+ Coverage   28.56%   28.71%   +0.14%     
==========================================
  Files          62       62              
  Lines        5601     6165     +564     
==========================================
+ Hits         1600     1770     +170     
- Misses       4001     4395     +394
Impacted Files Coverage Δ
src/routes/user.rs 0% <0%> (ø) ⬆️
plume-models/src/posts.rs 12.01% <0%> (-0.13%) ⬇️
plume-models/src/notifications.rs 0% <0%> (ø) ⬆️
src/routes/blogs.rs 0% <0%> (ø) ⬆️
plume-models/src/db_conn.rs 0% <0%> (ø) ⬆️
src/main.rs 0% <0%> (ø) ⬆️
plume-models/src/comments.rs 0% <0%> (ø) ⬆️
src/routes/posts.rs 0% <0%> (ø) ⬆️
src/routes/comments.rs 0% <0%> (ø) ⬆️
src/routes/instance.rs 0% <0%> (ø) ⬆️
... and 18 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 61b6cee...d418093. Read the comment docs.

# [Codecov](https://codecov.io/gh/Plume-org/Plume/pull/344?src=pr&el=h1) Report > Merging [#344](https://codecov.io/gh/Plume-org/Plume/pull/344?src=pr&el=desc) into [master](https://codecov.io/gh/Plume-org/Plume/commit/61b6ceed92ef838a085da6eb66f87ab1de65a520?src=pr&el=desc) will **increase** coverage by `0.14%`. > The diff coverage is `21.62%`. [![Impacted file tree graph](https://codecov.io/gh/Plume-org/Plume/pull/344/graphs/tree.svg?width=650&token=sHtxmDuZ06&height=150&src=pr)](https://codecov.io/gh/Plume-org/Plume/pull/344?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master #344 +/- ## ========================================== + Coverage 28.56% 28.71% +0.14% ========================================== Files 62 62 Lines 5601 6165 +564 ========================================== + Hits 1600 1770 +170 - Misses 4001 4395 +394 ``` | [Impacted Files](https://codecov.io/gh/Plume-org/Plume/pull/344?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [src/routes/user.rs](https://codecov.io/gh/Plume-org/Plume/pull/344/diff?src=pr&el=tree#diff-c3JjL3JvdXRlcy91c2VyLnJz) | `0% <0%> (ø)` | :arrow_up: | | [plume-models/src/posts.rs](https://codecov.io/gh/Plume-org/Plume/pull/344/diff?src=pr&el=tree#diff-cGx1bWUtbW9kZWxzL3NyYy9wb3N0cy5ycw==) | `12.01% <0%> (-0.13%)` | :arrow_down: | | [plume-models/src/notifications.rs](https://codecov.io/gh/Plume-org/Plume/pull/344/diff?src=pr&el=tree#diff-cGx1bWUtbW9kZWxzL3NyYy9ub3RpZmljYXRpb25zLnJz) | `0% <0%> (ø)` | :arrow_up: | | [src/routes/blogs.rs](https://codecov.io/gh/Plume-org/Plume/pull/344/diff?src=pr&el=tree#diff-c3JjL3JvdXRlcy9ibG9ncy5ycw==) | `0% <0%> (ø)` | :arrow_up: | | [plume-models/src/db\_conn.rs](https://codecov.io/gh/Plume-org/Plume/pull/344/diff?src=pr&el=tree#diff-cGx1bWUtbW9kZWxzL3NyYy9kYl9jb25uLnJz) | `0% <0%> (ø)` | :arrow_up: | | [src/main.rs](https://codecov.io/gh/Plume-org/Plume/pull/344/diff?src=pr&el=tree#diff-c3JjL21haW4ucnM=) | `0% <0%> (ø)` | :arrow_up: | | [plume-models/src/comments.rs](https://codecov.io/gh/Plume-org/Plume/pull/344/diff?src=pr&el=tree#diff-cGx1bWUtbW9kZWxzL3NyYy9jb21tZW50cy5ycw==) | `0% <0%> (ø)` | :arrow_up: | | [src/routes/posts.rs](https://codecov.io/gh/Plume-org/Plume/pull/344/diff?src=pr&el=tree#diff-c3JjL3JvdXRlcy9wb3N0cy5ycw==) | `0% <0%> (ø)` | :arrow_up: | | [src/routes/comments.rs](https://codecov.io/gh/Plume-org/Plume/pull/344/diff?src=pr&el=tree#diff-c3JjL3JvdXRlcy9jb21tZW50cy5ycw==) | `0% <0%> (ø)` | :arrow_up: | | [src/routes/instance.rs](https://codecov.io/gh/Plume-org/Plume/pull/344/diff?src=pr&el=tree#diff-c3JjL3JvdXRlcy9pbnN0YW5jZS5ycw==) | `0% <0%> (ø)` | :arrow_up: | | ... and [18 more](https://codecov.io/gh/Plume-org/Plume/pull/344/diff?src=pr&el=tree-more) | | ------ [Continue to review full report at Codecov](https://codecov.io/gh/Plume-org/Plume/pull/344?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/Plume-org/Plume/pull/344?src=pr&el=footer). Last update [61b6cee...d418093](https://codecov.io/gh/Plume-org/Plume/pull/344?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
trinity-1686a approved these changes 5 years ago
trinity-1686a left a comment
Owner

Things I commented are further optimizations possible, however they're not strictly needed to be implemented for this to be merged. Actually I don't even know if it would give better performance as optimizations I proposed can be proved equivalent to current implementation with what DBMSs know about our model, so they might do the optimization themselves

Things I commented are further optimizations possible, however they're not strictly needed to be implemented for this to be merged. Actually I don't even know if it would give better performance as optimizations I proposed can be proved equivalent to current implementation with what DBMSs know about our model, so they might do the optimization themselves
plume-models/src/blogs.rs
trinity-1686a commented 5 years ago
Owner

I wonder if the second query is useful, I think count()ing on the first one is sufficient, except in case where :

  • multiple users have the same id, which is guaranteed to never happen
  • an author_id exist in blog_authors but not in users, which is guaranteed to never happen too
  • more than one couple (blog,author) with the same blog&author exist (but different entry id), which is guaranteed to never happen since the constraint blog_author_unique was added (only apply if .select(blog_authors::author_id) is removed too, if it is not then multiple (id_X,blog_1,author_1) would be mapped to only one (author_1) anyway. Hope I'm clear enough)
I wonder if the second query is useful, I think `count()`ing on the first one is sufficient, except in case where : - multiple users have the same id, which is guaranteed to never happen - an author_id exist in blog_authors but not in users, which is guaranteed to never happen too - more than one couple (blog,author) with the same blog&author exist (but different entry id), which is guaranteed to never happen since the constraint blog_author_unique was added (only apply if `.select(blog_authors::author_id)` is removed too, if it is not then multiple (id_X,blog_1,author_1) would be mapped to only one (author_1) anyway. Hope I'm clear enough)
plume-models/src/notifications.rs
trinity-1686a commented 5 years ago
Owner

ordering then counting is useless, I guess PostgreSQL would see it and optimize that away, but I also think SQLite is too dumb to do the same, so it might be shorter and faster to remove that

ordering then counting is useless, I guess PostgreSQL would see it and optimize that away, but I also think SQLite is too dumb to do the same, so it might be shorter and faster to remove that
elegaanz (Migrated from github.com) reviewed 5 years ago
plume-models/src/notifications.rs
elegaanz (Migrated from github.com) commented 5 years ago

Yes, I just copy/pasted another function. I'll change that.

Yes, I just copy/pasted another function. I'll change that.
elegaanz (Migrated from github.com) reviewed 5 years ago
plume-models/src/blogs.rs
elegaanz (Migrated from github.com) commented 5 years ago

OK, I'll change it (and yes you were clear enough 🙂)

OK, I'll change it (and yes you were clear enough :slightly_smiling_face:)
trinity-1686a approved these changes 5 years ago
trinity-1686a commented 5 years ago
Owner

Love it when an issue is only reproducible on one db backend. It might be because sqlite don't enforce some constraints by default, we could enabled PRAGMA foreign_keys = ON;, but I wonder what changes to make this test fail

Love it when an issue is only reproducible on one db backend. It might be because sqlite don't enforce some constraints by default, we could enabled `PRAGMA foreign_keys = ON;`, but I wonder what changes to make this test fail
igalic commented 5 years ago (Migrated from github.com)

It might be because sqlite don't enforce some constraints by default, we could enabled PRAGMA foreign_keys = ON;

we have that enabled — on the users table — not sure if/how we enable it by default.

, but I wonder what changes to make this test fail

maybe we're finally using that constraint? 😅

> It might be because sqlite don't enforce some constraints by default, we could enabled `PRAGMA foreign_keys = ON;` we have that enabled — on the `users` table — not sure if/how we enable it by default. > , but I wonder what changes to make this test fail maybe we're finally using that constraint? 😅
trinity-1686a commented 5 years ago
Owner

@igalic I think we only set it in a migration, this property is not stored so we need to set it somehow at each Plume launch, and this is a global var, so it was enabled for all tables (but only until the migration end)

@igalic I think we only set it in a migration, this property is not stored so we need to set it somehow at each Plume launch, and this is a global var, so it was enabled for all tables (but only until the migration end)
igalic commented 5 years ago (Migrated from github.com)

this property is not stored

unlike (some) other pragma instructions, it's not storable… it's, as you say, per connection or per transaction.

So perhaps the diesel sqlite driver has something we can turn on by default

i checked, it can't
but maybe we can wrap the query functions with a begin transaction; pragma foreign_keys = on;

> this property is not stored unlike (some) other `pragma` instructions, it's not storable… it's, as you say, per connection or per transaction. So perhaps the diesel sqlite driver has something we can turn on by default ------ i checked, it can't but maybe ***we*** can wrap the query functions with a `begin transaction; pragma foreign_keys = on;`
trinity-1686a reviewed 5 years ago
plume-models/src/db_conn.rs
@ -41,0 +48,4 @@
sql_query("PRAGMA foreign_keys = on;").execute(conn)
.map(|_| ())
.map_err(|_| ConnError::ConnectionError(ConnectionError::BadConnection(String::from("PRAGMA foreign_keys = on failed"))))
}
trinity-1686a commented 5 years ago
Owner

oh, I was searching for exactly that but did not find it

oh, I was searching for exactly that but did not find it
elegaanz (Migrated from github.com) reviewed 5 years ago
plume-models/src/db_conn.rs
@ -41,0 +48,4 @@
sql_query("PRAGMA foreign_keys = on;").execute(conn)
.map(|_| ())
.map_err(|_| ConnError::ConnectionError(ConnectionError::BadConnection(String::from("PRAGMA foreign_keys = on failed"))))
}
elegaanz (Migrated from github.com) commented 5 years ago

Looks like it is not working at all 🙃 nvm I think I found the issue

~~Looks like it is not working at all~~ :upside_down_face: nvm I think I found the issue
trinity-1686a reviewed 5 years ago
plume-models/src/db_conn.rs
@ -41,0 +48,4 @@
sql_query("PRAGMA foreign_keys = on;").execute(conn)
.map(|_| ())
.map_err(|_| ConnError::ConnectionError(ConnectionError::BadConnection(String::from("PRAGMA foreign_keys = on failed"))))
}
trinity-1686a commented 5 years ago
Owner

I think your changes are only affecting the web interface, which is at the same time the most important part, and the one not tested by travis 🙃
I've tested doing similar changes in plume_model::lib:tests::db (adding sql_query("PRAGMA foreign_keys = on;").execute(&conn).unwrap();) which seems to work fine

I think your changes are only affecting the web interface, which is at the same time the most important part, and the one not tested by travis :upside_down_face: I've tested doing similar changes in plume_model::lib:tests::db (adding `sql_query("PRAGMA foreign_keys = on;").execute(&conn).unwrap();`) which seems to work fine
trinity-1686a reviewed 5 years ago
plume-models/src/blogs.rs
trinity-1686a commented 5 years ago
Owner

"bfore", "after bfore", "after after", then is it "after after bfore" or "after after after"?
(joking about the comments, but they should be removed actually, I guess you forgot to)

"bfore", "after bfore", "after after", then is it "after after bfore" or "after after after"? (joking about the comments, but they should be removed actually, I guess you forgot to)
elegaanz (Migrated from github.com) reviewed 5 years ago
plume-models/src/blogs.rs
elegaanz (Migrated from github.com) commented 5 years ago

Oops 😬

Oops :grimacing:

Reviewers

trinity-1686a approved these changes 5 years ago
The pull request has been merged as 38302203f4.
You can also view command line instructions.

Step 1:

From your project repository, check out a new branch and test the changes.
git checkout -b count-in-db master
git pull origin count-in-db

Step 2:

Merge the changes and update on Forgejo.
git checkout master
git merge --no-ff count-in-db
git push origin master
Sign in to join this conversation.
Reviewers
Request review
No reviewers
trinity-1686a
Labels
Apply labels
Clear labels   
A: API

Related to the REST API   
A: Backend

Code running on the server   
A: Federation

Stuff related to Federation   
A: Front-End

Related to the front-end   
A: I18N

Translations, and related code   
A: Meta

More about project management or code than the project itself   
A: Security
   
Build

The building, or installation process of Plume   
C: Bug

Something isn't working   
C: Discussion

We need to talk   
C: Enhancement

New feature or request   
C: Feature

This is a new feature   
Compatibility

Compatibility with different browsers, readers and OS   
Dependency

Related to an external package that Plume uses   
Design

UI/UX related issues and PRs   
Documentation
   
Good first issue

Good for newcomers   
Help welcome

Extra attention is needed   
Mobile

Issues affecting only mobile UX   
Rendering

How elements're rendered out for the end user   
S: Blocked

Something else needs to be fixed first   
S: Duplicate

This issue or pull request already exists   
S: Incomplete

This PR is not complete yet   
S: Instance specific

Issues concern a limited number of instances   
S: Invalid

This doesn't seem right   
S: Needs Voting/Discussion

Need to be discussed by the community (on Loomio)   
S: Ready for review

This PR is ready to be reviewed   
Suggestion

Proposed ideas worth considering   
S: Voted on Loomio

This is issue has been created after a vote on Loomio   
S: Wontfix

This will not be worked on
No Label
A: API
A: Backend
A: Federation
A: Front-End
A: I18N
A: Meta
A: Security
Build
C: Bug
C: Discussion
C: Enhancement
C: Feature
Compatibility
Dependency
Design
Documentation
Good first issue
Help welcome
Mobile
Rendering
S: Blocked
S: Duplicate
S: Incomplete
S: Instance specific
S: Invalid
S: Needs Voting/Discussion
S: Ready for review
Suggestion
S: Voted on Loomio
S: Wontfix
Milestone
Set milestone
Clear milestone
No items
No Milestone
Assignees
Assign users
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: Plume/Plume#344
Write Preview
Loading…
Cancel
Save
There is no content yet.
Delete Branch 'count-in-db'

Deleting a branch is permanent. It CANNOT be undone. Continue?

No
Yes