Troi recommendation system

mayhem

This will be used in lb-content-resolver and in listenbrainz-server and I expect other projects will pick it up as well, so yes, I plan to maintain it.

No rush on getting this working -- we've got lots on too!

petitminion

hello 🙂
We've been asking ourselve if its better to use postgre or redis to store matches. The main question is about how much time do you think we should keep a match in db before deleting it in case of metadata change ?

mayhem

It would be ideal to invalidate the match when the metadata changes, but that could be a lot more work. But, using redis and then discarding all keys in redis when any metadata changes would be a start.

petitminion

mayhem indeed o/ But if metadata change in musicbrainz side ?

mayhem

If you want to be 100% correct, then install a MB instance and get notified when the metadata changes. That's even more work!

Realistically, I wouldn't bother storing the matches, honestly. Each lookup is already expensive and I would expect that not that many playlist resolutions are going to happen per hour/minute. And given that the greatest expense in looking up a match is loading all the data in the first place, if you repeat a few lookups that otherwise would've been cache hits, won't be noticeable in the overall performance.

petitminion

Each lookup is already expensive

lookup of what ?

the greatest expense in looking up a match is loading all the data in the first place,

loading what data ?

sorry I didn't understood ^^ But I get your point is to not cache the matches and repeat the matching process at each playlist generation. I will test that o/

mayhem

Sorry, I forgot that you decided to go with Typesense -- the old approach needed to have all the metadata loaded to build the index, which is what I was referring to.

But in the typesense case, why not update the metadata in typesense when the metadata changes? E.g. add a hook to update typesense when the metadata for a track changes in the DB? That would be the easiest, I would think.

petitminion

mayhem yes ofc this is easy because its about updating fw metadata but it do not resolve the question of how long do e want the matches to persist on disk in case mb metadata change. I implemented redis, I will test how it works and if there is performance issues I will use postgres.

mayhem

Hi!

Sorry for the long silence, but life last quarter got very hectic. But once that all settled down, I started coding. I pretty much skipped xmas (so happy for that!) and spent nearly two weeks coding. And the results are rather quite impressive -- take a look at this branch I am getting ready to merge:

https://github.com/metabrainz/listenbrainz-content-resolver/tree/add-artist-element

The list of features has grown quite long!

mayhem

Now cleaned up and merged: https://github.com/metabrainz/listenbrainz-content-resolver

ppom

I may be totally out off-topic, but what do you think of using Typesense for Funkwhale search functionality? If we already have all the tracks, albums, artists synced into Typesense, that may be a very cool search engine, with little extra work. Related to #2236

mayhem

ppom Not off-topic -- this is why @petitminion add typesense to Funkwhale in the first place. The matching solution that is present in listenbrainz-content-resolver is meant to server as a non-server solution. (And works quite well for that job).

gcrk

ppom Thats actually something I was considering, however search isn't our huge focus at the moment. If you are willing to work on this, let me know!

petitminion

Amazing work @mayhem. I'm looking forward to implement this into funkwhale. Great idea to use the subsonic api to make troi aware of the local collection. We need to discuss some technical detail with the funkwhale team before starting the implementation but maybe I will make some experiments before o/ Will let you know 🙂

phw

petitminion Great idea to use the subsonic api to make troi aware of the local collection.

Just as a side note if someone wants to try the listenbrainz-content-resolver with Funkwhale: For this to work FW currently needs my patches from https://dev.funkwhale.audio/funkwhale/funkwhale/-/merge_requests/2695 for OpenSubsonic support.

It works with the current release versions of Navidrome and LMS.

« Previous Page