Adjust/fix pruning after import : missing tracks

kalon33

Hello all,

Is there a way to adjust tracks pruning after indexation in a library? A lot of tracks that are seen during indexation (and transiently listed in my library) are pruned (marked ignored because they already exist in a library), even if they have no duplicates in database. So they are not playable.

I noticed that my first library import from an in place directory on my server timed out, but tried it again and it finished properly. Might it be linked to missdetection of duplicates?

Thanks for your help.

petitminion

I don't understand : the metadata of the tracks where uploaded but not the audio file ?

kalon33

petitminion During indexation from my in-place music folder, I see tracks appearing in my Funkwhale library that disappear at the end of the process (during pruning step it seems). Unfortunately, an important number of the ones that are removed are not duplicates (but they are considered as such and marked as "ignored"), so I miss a good amount of tracks in Funkwhale that are on my hard drive.

petitminion

Strange, what command did you used exactly ?

kalon33

petitminion I used the web interface "import" section of my library to add the tracks.

petitminion

do you have a step after Pruning dangling tracks, albums and artists from library… ? can you provide the full log?

kalon33

petitminion

Yes, here is the message after that step (journalctl on the celery task) and the pruning seems really huge...

May 24 06:45:58 caius-bonux2 celery[3402781]: [2022-05-24 06:45:58,136: WARNING/ForkPoolWorker-2] For details, please refer to import reference '2022-05-24T08:41:33+02:00' or URL https://funkwhale.badab>
May 24 06:45:58 caius-bonux2 celery[3402781]: [2022-05-24 06:45:58,136: WARNING/ForkPoolWorker-2] Pruning dangling tracks, albums and artists from library…
May 24 06:45:58 caius-bonux2 celery[3402781]: [2022-05-24 06:45:58,143: WARNING/ForkPoolWorker-2] Dry-run off, *pruning for real*
May 24 06:45:58 caius-bonux2 celery[3402781]: [2022-05-24 06:45:58,143: WARNING/ForkPoolWorker-2]
May 24 06:45:58 caius-bonux2 celery[3402781]: [2022-05-24 06:45:58,179: WARNING/ForkPoolWorker-2] Deleting 5269/6017 tracks…
May 24 06:46:00 caius-bonux2 celery[3402781]: [2022-05-24 06:46:00,269: WARNING/ForkPoolWorker-2] Deleting 360/1298 albums…
May 24 06:46:00 caius-bonux2 celery[3402781]: [2022-05-24 06:46:00,300: WARNING/ForkPoolWorker-2] Deleting 236/865 artists…
May 24 06:46:00 caius-bonux2 celery[3402781]: [2022-05-24 06:46:00,328: WARNING/ForkPoolWorker-2]
May 24 06:46:00 caius-bonux2 celery[3402781]: [2022-05-24 06:46:00,328: WARNING/ForkPoolWorker-2] Pruning completed!
May 24 06:46:00 caius-bonux2 celery[3402781]: [2022-05-24 06:46:00,328: WARNING/ForkPoolWorker-2]
May 24 06:46:00 caius-bonux2 celery[3402781]: [2022-05-24 06:46:00,894: INFO/ForkPoolWorker-2] Task music.fs_import[b1665b5a-02d4-4a88-b6a3-d580fc0debb8] succeeded in 264.41741112899035s: None

Why does it deletes so much stuff? Not even duplicated/redundant?

Thanks for your help.

petitminion

idk I need to investigate, can you open an isssue in our gitlab plz ?
Has a work around you can try with the cli : https://docs.funkwhale.audio/admin/importing-music.html

kalon33

petitminion thanks for your help. I opened the issue here: https://dev.funkwhale.audio/funkwhale/funkwhale/-/issues/1795

gcrk

petitminion The CLI is broken atm, I wouldn't advertise it until its fixed