Acoustid implementation

petitminion

phw
Thx for your detailed answer ! 🙂

I will just say my pov is the same : fw is not picard. Implementing human control will take time and people who wants their library to be tagged well can use picard. Personally I would :

match audio files to release automatically even if there is a probability of error and wait for a well tagged file to be uploaded and replace the old one. Or would not match file to release if we have not enough metadata.
give user the possibility to disable acoustid.

I understand this is not ideal but personally I don't want to work on a tagging system in funkwhale when it can be done elsewhere. But maybe other people wants to do this on funkwhale ! o/ lets here them out 😃

phw

petitminion match audio files to release automatically even if there is a probability of error and wait for a well tagged file to be uploaded and replace the old one. Or would not match file to release if we have not enough metadata.

I guess that is the important part here: Files must be matched against both recording and release. That'll avoid animaldaydream's main concern I think.

And there should be some kind of threshold under which the matching is simply not done.

[deleted]

petitminion I will just say my pov is the same : fw is not picard. Implementing human control will take time and people who wants their library to be tagged well can use picard.

I think the problem here is that expecting Funkwhale to do an optimal job based on AcoustID alone is just not possible. If you were to match them to a recording that'd be one thing, but you want to match them to all releases with that recording and that's gonna be a mess for all the reasons listed above. I do not approve that there should be a system in place for doing this.

And you probably expect the tracks to be matched with an album or single released by the artist. However, half of the time they're gonna be matched with compilations or bootlegs that no user is gonna be looking for.

[deleted]

petitminion I don't want to match a audio file to all release

but you said

petitminion match audio files to release automatically even if there is a probability of error and wait for a well tagged file to be uploaded and replace the old one.

If there's no release MBID you shouldn't match it to any release at all.

petitminion

[deleted] I don't want to match a audio file to all release... I repeatly said we will use the acoustid AND the metadata

@phw I totally agree

phw

phw AcoustID takes the fingerprint of the first 30 seconds of an recording plus the duration of of the recording for identification

Re-reading my comment I see I must correct this: it's of course the first 120 seconds, not 30. But the general idea stays the same.

mayhem

I've been musing about how to bring recommendations to Funkwhale without requiring perfectly tagged collections. And the answer is: A content resolver. Here is how that could work:

Create a funkwhale plugin (if possible) that fetches all the metadata for each of the tracks and builds an index from it. (An index resistant to typos and other common BS -- we use this concept for our MBID mapper and our Spotify content resolver)
Add troi to the plugin.
Adapt troi patches to download the collaborative filtering recs from ListenBrainz and then resolve the content to the local node using the index in step 1.
Continue with the rest of the troi patch (e.g. daily jams) to generate a playlist for the local funkwale node with each track properly resolved.

The key is to make the index and the content resolver, both of which we've done for LB, twice over. So, we know how to do this.

Is there anyone from the funkwhale community who would like to help me write this plugin? If you can provide the FW knowledge for how to create a skeleton plugin, how to fetch all the textual metadata for the tracks and how to write a playlist back to FW, I can provide the rest of the guts. A couple of days of my time should do the trick.

Anyone interested?

Wanna discuss at FOSDEM?

petitminion

mayhem
Hello again 🙂 I can totally help you to generated fw playlist and fetch fw metadata. Idk about the pluging system but I will gladly learn haha I will not be at FOSDEM but we could start this after you've meet some of the team 😃

mayhem

I'm not hung up on it being a plugin -- whatever you see fit. But, great, good to know that I have someone so I can be more effective.

petitminion

mayhem We should follow our feature implementation steps : the first one is to create a dedicated forum post so we can discuss the feature and write a proper specification o/ When we have a proper view of how it will work we can see if its better to implement it has a pluging or not. Also I have various questions but I think it will be better to speak in another forum post 🙂

mayhem

Oh wow. I think I've just lost interest in writing this feature. This sounds terrible.

petitminion

mayhem I can do the writing ^^ But in any case we need to design this, we can make a call or we can work directly on a proof of concept.
Because there is various people implicated and we need to make sure everybody understand how this work and can give their though. But feel free to create a mr and I can manage the specification and communication when I see how it works

mayhem

Well, the proof of concept is the MBID Mapper that is in production on ListenBrainz. I am not proposing to write any amount of new code -- just slapping together bits that already exist, so proof of concept is pretty much the working thing.
Let me lay out the moving parts:

Typesense index -- this would require running a typesense container: https://typesense.org/
An indexer that builds an initial index of all the metadata and receives updates of tracks being added, removed or renamed from the funkwhale DB.
A troi component of some sort. Let's assume that we want to implement daily-jams in funkwhale. This component would need to include Troi and be called on a nightly basis to generate the playlist.
A modified daily-jams patch that downloads the recs from LB -- it would do the normal daily-jams stuff, but it would add an extra step of narrowing the recommended tracks down to only tracks available locally.

That's about it. This then makes troi a part of FW -- and troi patch should be able to run and generate playlists now. I might need to think about how the FW aspects of troi could be included in troi patches without having a make custom patches, but that is a minor design detail.

Does this make sense so far?

[deleted]

mayhem Well, the proof of concept is the MBID Mapper that is in production on ListenBrainz.

I am interested in learning more about this ^^

Do you have a link?

[deleted]

I really don't want to be a bother, but before implementing the feature we probably should define what its meant to do and what its behavior should be.

At this point in time we were in the middle of discussing the behavior of AcoustID itself; the conclusion was that it was unreliable for distinguishing a recording and we needed to link every track to a release entity as well as the recording entity. As @mayhem said, only the first 120 seconds of the track are considered, so the conclusion of the analysis of each audio file is not final.

@petitminion however insists on using the AcoustID to link an audio file to every known release with that recording ID. I insist we should not. We're at a stalemate here, and I'm bothered nobody else has come to talk about it yet you're moving on to figuring out how to implement the feature already.

EDIT: I also want to point out this would be a massive dealbreaker if such a feature is hard-coded. I can definitely see users walking away when the software is becoming opinionated on how the library should behave, when witnessing automatic tagging. Which nobody with a large library can accept to see to their carefully tagged to the standard, or deliberately mistagged to their own purposes, library. I would know.

mayhem

@petitminion : Can you please ping me in #metabrainz?

petitminion

mayhem I reponded here https://forum.funkwhale.audio/d/286-troi-recommendation-system
tried to ping you on irc

petitminion

Considering that the process cannot be completely automated because the acoustid matching will need a manual review before we can accurately tag track, the steering committee has decided this implementation could be replace by the mbid matching mechanism we are implementing now (see https://forum.funkwhale.audio/d/286-troi-recommendation-system/79)

The alternative solution does not resolve all the issue, but combined with the option to only allow mb tagged content to a pod it should be enough for admin to manage content metadata at a pod level. (see https://forum.funkwhale.audio/d/243-new-feature-only-allow-musicbrainz-tagged-file-to-be-uploaded/16)

Any thought about this ?

If everyone agrees I will close the related issue and forum post

mayhem

petitminion Considering that the process cannot be completely automated because the acoustid matching will need a manual review before we can accurately tag track, the steering committee has decided this implementation could be replace by the mbid matching mechanism we are implementing now (see https://forum.funkwhale.audio/d/286-troi-recommendation-system/79)

I'm glad the steering committee finally sees my point -- took only a few months to catch up to what I was originally suggesting, but we got there in the end.

So, yes, lets close this and focus on the content resolution approach. In fact, there will be blog posts posted soon that shows exactly how to do this. Stay tuned.

« Previous Page