Plume/Plume

Deriva 24

Allow for media cover deletion #387

Unito

Plume_migration_agent ha unito 2 commit da delete-cover a master

2019-01-05 21:11:54 +00:00

trinity-1686a

2018-12-25 16:57:31 +00:00

ha commentato

Proprietario

Fix #356

igalic (Migrato da github.com) revisionato 2018-12-25 22:10:32 +00:00

igalic (Migrato da github.com) lascia un commento

sssiiiighqlite 🙀

_sssiiiighqlite_ 🙀

migrations/sqlite/2018-12-25-164502_media-cover-deletion/down.sql

igalic (Migrato da github.com)

2018-12-25 22:09:11 +00:00

ha commentato

unfortunately, you cannot drop or add constraints on sqlite tables without doing the create temporary table dance

trinity-1686a revisionato 2018-12-25 23:38:40 +00:00

migrations/sqlite/2018-12-25-164502_media-cover-deletion/down.sql

trinity-1686a

2018-12-25 23:38:40 +00:00

ha commentato

Autore

Proprietario

Travis made me notice it. I knew for columns, I through it worked at least for constraints (it's only drop, add work just fine if I remember well)

elegaanz

2018-12-29 13:33:53 +00:00

ha commentato (Migrato da github.com)

Maybe this change should be integrated in the migrations that introduced constraints, because when you run migrations on an instance with illustrated articles, you get this error:

Running migration 2018-12-08-175515_constraints
update or delete on table "medias" violates foreign key constraint "posts_cover_id_fkey" on table "posts"

(After seeing this error on baptiste.gelez.xyz I tried to manually fix it, but it just ended in an instance reset 😬)

(I don't think it is a good idea to change previous migrations, but we don't really have choice…)

Maybe this change should be integrated in the migrations that introduced constraints, because when you run migrations on an instance with illustrated articles, you get this error: ``` Running migration 2018-12-08-175515_constraints update or delete on table "medias" violates foreign key constraint "posts_cover_id_fkey" on table "posts" ``` (After seeing this error on baptiste.gelez.xyz I tried to manually fix it, but it just ended in an instance reset :grimacing:) (I don't think it is a good idea to change previous migrations, but we don't really have choice…)

trinity-1686a

2018-12-29 14:14:26 +00:00

ha commentato

Autore

Proprietario

@BaptisteGelez I don't understand, what error did you run into? Is it when an article from before 2018-12-08-175515_constraints reference a no longer existing media?

@BaptisteGelez I don't understand, what error did you run into? Is it when an article from before `2018-12-08-175515_constraints` reference a no longer existing media?

elegaanz

2018-12-29 17:02:19 +00:00

ha commentato (Migrato da github.com)

Edit: actually, the media doesn't need to be used as avatar. If it owned by a duplicate user, the query will fail.

Edit 2: the user doesn't even need to be duplicated. For instance, some user have an empty followers_endpoint property, because they were created at a time this property didn't exist.

It seem to happen when a media is used both as avatar and post cover. This query deletes it: https://github.com/Plume-org/Plume/blob/master/migrations/postgres/2018-12-08-175515_constraints/up.sql#L51 but it fails because it is used on a post. Edit: actually, the media doesn't need to be used as avatar. If it owned by a duplicate user, the query will fail. Edit 2: the user doesn't even need to be duplicated. For instance, some user have an empty `followers_endpoint` property, because they were created at a time this property didn't exist.

codecov[bot]

2019-01-05 18:35:19 +00:00

ha commentato (Migrato da github.com)

Codecov Report

Merging #387 into master will increase coverage by 1.42%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #387      +/-   ##
==========================================
+ Coverage   27.86%   29.29%   +1.42%     
==========================================
  Files          63       63              
  Lines        6280     7671    +1391     
==========================================
+ Hits         1750     2247     +497     
- Misses       4530     5424     +894

# [Codecov](https://codecov.io/gh/Plume-org/Plume/pull/387?src=pr&el=h1) Report > Merging [#387](https://codecov.io/gh/Plume-org/Plume/pull/387?src=pr&el=desc) into [master](https://codecov.io/gh/Plume-org/Plume/commit/9b3b79ef9c7a77a60c62c8c3682ed86723e611ba?src=pr&el=desc) will **increase** coverage by `1.42%`. > The diff coverage is `n/a`. ```diff @@ Coverage Diff @@ ## master #387 +/- ## ========================================== + Coverage 27.86% 29.29% +1.42% ========================================== Files 63 63 Lines 6280 7671 +1391 ========================================== + Hits 1750 2247 +497 - Misses 4530 5424 +894 ```

elegaanz (Migrato da github.com) hanno approvato queste modifiche 2019-01-05 20:59:18 +00:00

elegaanz (Migrato da github.com) lascia un commento

Travis is happy, I think we can safely merge that. Thanks!

EliotBerriot

2019-02-22 13:46:53 +00:00

ha commentato (Migrato da github.com)

I'm facing the same issue described by @BaptisteGelez while applying this migration, and I cannot afford to reset my instance. Is there anything to do to manually fix the issue? I can run SQL by hand if needed :)

elegaanz

2019-02-22 16:41:19 +00:00

ha commentato (Migrato da github.com)

@EliotBerriot try to find user with empty followers_endpoint with SELECT id, username FROM users WHERE followers_endpoint = ''; and if anything is returned, edit the follower endpoint so that it has a value (it can be anything as long as it is not null, nor already used by another user). If I remember correctly, that's how I fixed it on fediverse.blog. Sorry for the bug… 😕

If it still doesn't work, try to check users.ap_url, user.inbox_url and user.outbox_url too.

@EliotBerriot try to find user with empty `followers_endpoint` with `SELECT id, username FROM users WHERE followers_endpoint = '';` and if anything is returned, edit the follower endpoint so that it has a value (it can be anything as long as it is not null, nor already used by another user). If I remember correctly, that's how I fixed it on fediverse.blog. Sorry for the bug… :confused: If it still doesn't work, try to check `users.ap_url`, `user.inbox_url` and `user.outbox_url` too.

EliotBerriot

2019-02-25 08:37:17 +00:00

ha commentato (Migrato da github.com)

@BaptisteGelez SELECT id, username FROM users WHERE followers_endpoint = ''; returned a bunch of users, and I updated it w ith UPDATE users SET followers_endpoint='noop' WHERE followers_endpoint = '';`

This is the current returns for all the attributes you mentions:

plume=# SELECT id, username FROM users WHERE ap_url = '';
 id | username 
----+----------
(0 rows)

plume=# SELECT id, username FROM users WHERE inbox_url = '';
 id | username 
----+----------
(0 rows)

plume=# SELECT id, username FROM users WHERE outbox_url = '';
 id | username 
----+----------
(0 rows)

plume=# SELECT id, username FROM users WHERE followers_endpoint = '';
 id | username 
----+----------
(0 rows)

Unfortunately, applying the migrations return the same error again:

eliotberriot@peeters:/srv/Plume$ docker-compose run --rm plume diesel migration run --migration-dir migrations/postgres/
Starting plume_postgres_1 ... done
Running migration 2018-12-08-175515_constraints
update or delete on table "medias" violates foreign key constraint "posts_cover_id_fkey" on table "posts"

The issues seems to lie within the medias/posts relation, so I'm not sure what it has to do with users?

Anyway, don't be sorry for this: I know the software was still in development and I could face this kind of bug, that was part of the deal :)

@BaptisteGelez `SELECT id, username FROM users WHERE followers_endpoint = ''; returned a bunch of users, and I updated it w ith `UPDATE users SET followers_endpoint='noop' WHERE followers_endpoint = '';` This is the current returns for all the attributes you mentions: ``` plume=# SELECT id, username FROM users WHERE ap_url = ''; id | username ----+---------- (0 rows) plume=# SELECT id, username FROM users WHERE inbox_url = ''; id | username ----+---------- (0 rows) plume=# SELECT id, username FROM users WHERE outbox_url = ''; id | username ----+---------- (0 rows) plume=# SELECT id, username FROM users WHERE followers_endpoint = ''; id | username ----+---------- (0 rows) ``` Unfortunately, applying the migrations return the same error again: ``` eliotberriot@peeters:/srv/Plume$ docker-compose run --rm plume diesel migration run --migration-dir migrations/postgres/ Starting plume_postgres_1 ... done Running migration 2018-12-08-175515_constraints update or delete on table "medias" violates foreign key constraint "posts_cover_id_fkey" on table "posts" ``` The issues seems to lie within the medias/posts relation, so I'm not sure what it has to do with users? Anyway, don't be sorry for this: I know the software was still in development and I could face this kind of bug, that was part of the deal :)

elegaanz

2019-02-25 11:21:46 +00:00

ha commentato (Migrato da github.com)

Okay, so maybe try to reset the post covers (it will not impact your articles as you didn't used them for Funkwhale's blog), with UPDATE posts SET cover_id = null;. I don't know if it will fix the issue, but it is worth a try.

Okay, so maybe try to reset the post covers (it will not impact your articles as you didn't used them for Funkwhale's blog), with `UPDATE posts SET cover_id = null;`. I don't know if it will fix the issue, but it is worth a try.

👍 1

EliotBerriot

2019-02-25 11:57:13 +00:00

ha commentato (Migrato da github.com)

@BaptisteGelez at least with the cover_id thing I get another error:

eliotberriot@peeters:/srv/Plume$ docker-compose run --rm plume diesel migration run --migration-dir migrations/postgres/
Starting plume_postgres_1 ... done
Running migration 2018-12-08-175515_constraints
update or delete on table "comments" violates foreign key constraint "comments_in_response_to_id_fkey" on table "comments"

The issues is withing the comments_in_response_to_id_fkey constraint now

@BaptisteGelez at least with the `cover_id` thing I get another error: ``` eliotberriot@peeters:/srv/Plume$ docker-compose run --rm plume diesel migration run --migration-dir migrations/postgres/ Starting plume_postgres_1 ... done Running migration 2018-12-08-175515_constraints update or delete on table "comments" violates foreign key constraint "comments_in_response_to_id_fkey" on table "comments" ``` The issues is withing the `comments_in_response_to_id_fkey` constraint now

elegaanz

2019-02-25 13:27:02 +00:00

ha commentato (Migrato da github.com)

I made this script that should delete comments with invalid ActivityPub URLs, and their replies.

CREATE OR REPLACE FUNCTION delete_comment (id int) RETURNS comments
AS $$
DECLARE
  r int;
BEGIN
  FOR r IN (SELECT id FROM comments WHERE in_response_to_id = id) LOOP
    SELECT delete_comment(r);
  END LOOP;
  DELETE FROM comments WHERE id = id;
END $$ LANGUAGE plpgsql;

SELECT delete_comment(id) FROM comments WHERE ap_url = '';

I didn't tested it, but it should only delete broken comments (and hopefully fix the migrations).

I made this script that should delete comments with invalid ActivityPub URLs, and their replies. ```sql CREATE OR REPLACE FUNCTION delete_comment (id int) RETURNS comments AS $$ DECLARE r int; BEGIN FOR r IN (SELECT id FROM comments WHERE in_response_to_id = id) LOOP SELECT delete_comment(r); END LOOP; DELETE FROM comments WHERE id = id; END $$ LANGUAGE plpgsql; SELECT delete_comment(id) FROM comments WHERE ap_url = ''; ``` I didn't tested it, but it should only delete broken comments (and hopefully fix the migrations).

EliotBerriot

2019-02-25 14:25:34 +00:00

ha commentato (Migrato da github.com)

@BaptisteGelez unfortunately, this doesn't fix it (for me):

eliotberriot@peeters:/srv/Plume$ docker-compose exec postgres psql -U plume
psql (10.5 (Debian 10.5-2.pgdg90+1))
Type "help" for help.

plume=# 
plume=# 
plume=# CREATE OR REPLACE FUNCTION delete_comment (id int) RETURNS comments
plume-# AS $$
plume$# DECLARE
plume$#   r int;
plume$# BEGIN
plume$#   FOR r IN (SELECT id FROM comments WHERE in_response_to_id = id) LOOP
plume$#     SELECT delete_comment(r);
plume$#   END LOOP;
plume$#   DELETE FROM comments WHERE id = id;
plume$# END $$ LANGUAGE plpgsql;
CREATE FUNCTION
plume=# 
plume=# SELECT delete_comment(id) FROM comments WHERE ap_url = '';
 delete_comment 
----------------
(0 rows)

plume=# \q

eliotberriot@peeters:/srv/Plume$ docker-compose run --rm plume diesel migration run --migration-dir migrations/postgres/
Starting plume_postgres_1 ... done
Running migration 2018-12-08-175515_constraints
update or delete on table "comments" violates foreign key constraint "comments_in_response_to_id_fkey" on table "comments"

@BaptisteGelez unfortunately, this doesn't fix it (for me): ``` eliotberriot@peeters:/srv/Plume$ docker-compose exec postgres psql -U plume psql (10.5 (Debian 10.5-2.pgdg90+1)) Type "help" for help. plume=# plume=# plume=# CREATE OR REPLACE FUNCTION delete_comment (id int) RETURNS comments plume-# AS $$ plume$# DECLARE plume$# r int; plume$# BEGIN plume$# FOR r IN (SELECT id FROM comments WHERE in_response_to_id = id) LOOP plume$# SELECT delete_comment(r); plume$# END LOOP; plume$# DELETE FROM comments WHERE id = id; plume$# END $$ LANGUAGE plpgsql; CREATE FUNCTION plume=# plume=# SELECT delete_comment(id) FROM comments WHERE ap_url = ''; delete_comment ---------------- (0 rows) plume=# \q eliotberriot@peeters:/srv/Plume$ docker-compose run --rm plume diesel migration run --migration-dir migrations/postgres/ Starting plume_postgres_1 ... done Running migration 2018-12-08-175515_constraints update or delete on table "comments" violates foreign key constraint "comments_in_response_to_id_fkey" on table "comments" ```

Effettua l'accesso per partecipare alla conversazione.

Nessun revisore

elegaanz

Nessuna pietra miliare

Nessun progetto

Nessun assegnatario

2 partecipanti

Notifiche

Scadenza

La scadenza non è valida o fuori intervallo. Si prega di utilizzare il formato "aaaa-mm-dd".

Nessuna scadenza impostata.

Dipendenze

Nessuna dipendenza impostata.

Riferimento: Plume/Plume#387

Descrizione non fornita.

Rows
Columns