What's wrong with libraries, and a path to fix them

mjourdan

TL;DR

Dear Funkwhale community,
There are issues with libraries, we have ideas to solve them, and we could use your input.

Once upon a time…

Back in the early days of Funkwhale, library was a monolithic storage for a whole Funkwhale instance, so pod administrators could upload their music. Since then, per-user libraries were introduced, bringing people with an account the possibility to upload their own content, decide which libraries to share, with whom (instance, followers, or keep private). That was four years ago already, and arguably this is one of the greatest feature of Funkwhale. So everything is for the best, right? Not quite.

As soon as Funkwhale 0.17 got released, libraries became a central part of the user experience. There are many things one can do with a library: create it, give it a name and a description, give access to it, upload content to it, browse it, follow it… Unfortunately, all those actions have flaws, more or less bothersome taken separately, but when combined altogether result in a cumbersome experience. So we've been asking the following question: what's wrong with libraries?

An ambivalent word

Library as a word is confusing. As stated earlier, it used to be the unique storage place a pod could hold. Since then, its definition shifted. Nowadays, it refers to very orthogonal ideas depending on what part of the UI we are looking at:

in the sidebar, the library contains the whole set of music attached to peoples' account, including some remote content (favourites).
on the /content/libraries page, libraries are the global storage people have to manage (feed in, purge…)
on the /content/libraries page, libraries are containers for people to organise subsets of their music
on the /content/libraries page, libraries are a mean to share subsets of their music
on other people's page, libraries have a follow button to widen the catalog of content to browse
on artist page, they are for discovering related contents

This results in a fuzzy concept, and we think this fuzziness is at the root of some of the biggest issues we have with libraries. Namely uploading and organising content. We need clearer definitions.

Digging the history

We've been digging into Funkwhale community's past discussions to shed some light on libraries current state and their initial goals. Fortunately we are not left blind here.

At the time of Funkwhale 0.17, Agate shared her reasoning behind the current implementation of library of library. In an other message, she draws the big picture where libraries were taking place. The orientation the project was aiming for was to connect people to each other, allowing them to recommend music and discover new things based on common tastes.

In November 2019 on Matrix, where Agate was helping us to understand what libraries were, and to distinguish them from radios. I quote:

library is a collection of audio files (as: this file is stored in this library)
library is for storage / sharing

Goals

use separate wordings to convey distinguishable ideas
avoid hefty management of current libraries (upload, deduplication, and purge)
make organisation of music a breeze
make interactions with other people's "libraries" consistent and predictable
give a clear visibility over what is shared with whom
give a clear visibility over what's in a library

What others do

Let see if there is inspiration to take from other projects.

The gaming world

Steam allows to organise players game library into collections. It also sports dynamic collections, to automatically organise games based on tags and filters. The announcement on their blog shows what it looks like.

GoG also has some kind of dynamic collections: they allow to save filters as bookmarks on the one hand, and on the other hand they allow to add tags to games, so that they could be retrieved through bookmarks. A blog post [talks about it][https://www.gog.com/news/6_cool_things_you_can_do_in_gog_galaxy_20_that_you_might_have_missed).

Media players

Amongst media centers, Plex has both collections and libraries. The first one are meant for organising content of a same kind, the second are more like top-level containers related to media types (music, TV shows, movies…).

Music streaming platforms like Spotify, Youtube Music or Apple Music all have a similar concept of library. We could define it as the whole set of music hosted on the platform and attached to a user account.

Proposal

Breaking it up

In order to make things easy to grasp for users, and maybe help with the underlying technical issues that make use of library so odd, I propose to solve the "library" ambiguity by breaking it up into the pieces below.

Library (idea #1). This is the concept people are likely already familiar with. We keep this wording in the sidebar, and in the few places where it is relevant.

Storage (idea #2). It would be attached to a user account, instead of being attached to a library. This way content could be uploaded once, and used many times. Deduplication and purge mechanisms would not get in the way each time someone tries to organize something. Bandwidth, time and nerves would be preserved.

Collections (ideas #3 #4). This would be the new name we give to the containers that allow simultaneously to organise and to share content.

Adding remote collections to library

Wherever the pod they are hosted, followed collections should appear in the library. For more clarity, we could even change the label of the "follow" button to "add to library".

Adding content to collections

By allowing people to organize content and share it altogether, collections can be seen as part of a human-based, non algorithmic recommendation system. Going this road, Funkwhale UI should encourage
creating multiple smaller libraries of somehow related content, rather than into a few rag-bags of audio files. Adding music to (and removing music from) collections should be easy and straightforward.

People should have the possibility to add any album, playlist or track to a (new or existing) collection of their own, and doing so would make the content part of their library, even when the files are hosted on a remote pod.

Dynamic collections

Pre-defined collections could help make obvious where music in the library comes from. Like Uploads (for all content uploaded to a given library), Followed (for all content fetched from any followed collection), and Favourites (for all favourites, whether the files were uploaded or fetched from remote collections).

Conceptually this overlaps very much with radios. Just it would be presented like albums, playlists and tracks to browse, instead of being an automatically feeded queue of tracks like radios are.

Previous discussions

Back in March 2021, we used to talk about the differences between channels and libraries in the perspective of artists.

In July 2021, we concluded that if we were to rename the libraries for collections, we would have to rename API endpoints. So it has been instead proposed to keep the libraries with their API, and simply rename "My Library" to "My Collection".

August 2022, we have an opportunity to bring new API endpoints if we were to rename libraries.

Request for comments

Thanks for reading! Do not hesitate to drop your questions and share your thoughts.

gcrk

Thank you @mjourdan for bringing this up again! Its exactly the right time and I really appreciate you keep the topic going even if previous discussions in different circumstances probably have not been satisfactory.

So where to start? Well, I am involved in the software, so thats probably the area where I can have some useful input. At first, this totally changes the concepts we use. So this will be quite some work and it will probably have some side effects we are not yet aware of. So we need to think deeply about how to design this technically and how to migrate existing content, or if we even want to do this.

From my perspective it would be the easiest to throw away what we have and built something new from the ground up. I am not exactly sure if this is even possible here, but APIv2 development is a huge opportunity to do this. But all of this is something we can do later on.

As a user... What I am currently doing when uploading music is to think about "What should the visibility level of this music be?". Depending on this I am uploading the music to libraries. Later on I use or misuse radios to built subsets of the music because its too hard to do this within libraries. It has the disadvantage I cannot really decide upon the order and maintaining those radios is kinda painful. So I would totally appreciate to upload content once and put it into whatever collection I'd like to. So I want to have this!

From my perspective there is no reason this might be controversial at all. The only downside would be the work we have to spend implementing this, but I guess its worth the effort and can actually be a lot of fun.

To bring this topic forward, I think we should streamline our process, iirc @Sporiff already proposed something in some pad I cannot find right now. Maybe this can be a great case to try his proposal to work on a new feature (ofc we can adjust this while going forward if needed)

Again, thanks a lot for spending so much time and thoughts on all those fundamental issues. Lets see how we can make this happen!

mjourdan

gcrk

we need to think deeply about how to design this technically and how to migrate existing content, or if we even want to do this.

Yes, personally I could use some explanations from someone who knows the code about how many deduplication mechanisms Funkwhale currently has, and how they work (between libraries of a single user / of different user on the same pod / accross federation). If I understand

As a user... What I am currently doing when uploading music is to think about "What should the visibility level of this music be?". Depending on this I am uploading the music to libraries. Later on I use or misuse radios to built subsets of the music because its too hard to do this within libraries. It has the disadvantage I cannot really decide upon the order and maintaining those radios is kinda painful.

Interesting! Could you tell us more about the kind of subsets you are trying to organise?

petitminion

When if we need a concept for the front to define all the content a user follow or owns we could use the concept "Content".

That would basically encompass everything send by people, including music, podcasts, and even messages. We need to be a minimum specific.

Sporiff

On the point of migration, I don't think we necessarily need to move existing content. A lot of this can be worked around by splitting things up visually in the frontend for existing libraries (e.g. just labelling old libraries as "collections").

Besides how storage management currently works, maybe we would like to make it possible to have separate storages or "data sources" per user. So that people the library would aggregate:

the music they uploaded to the pod
the Funkwhale-hosted things they added to their collections
the music shared with them through their nextcloud account
the music they bought on Bandcamp

petitminion

This just make me understood that the proposition of @mjourdan is to replace the libraries with collections (and not with strorage as I understood in my last comment).

I propose both at the same time, actually.

But we could have separate storages for the audiobooks picked from there, the podcasts episodes saved, etc.

I think it would be better to keep libraries for rights access managemennt because :
We keep our current logic

That's what I propose not to do, because that's where the fundamental flaws lie. However, I think it's fair to say what the suggested changes would allow to keep (if we do things right), is the initial intent of the libraries feature.

what if I want to share only one track of a collection and keep the rest private ? With libraries I can just add the track to my public library and keep it in under any collection(s) I want.

It should be possible to share items like tracks, albums or playlists by link at least. Or by creating a new collection from them, holding just them. Doing either of these could make those items appear in a dynamic collection called something like "public" or "sharing". None of this would remove the items from the collections they were already part of (unless they were in a dynamic collection called "private").

keunes

I was trying to get Funkwhale up and running with my music stored on a NAS today, and came across this topic. It resonated, because of questions I had around libraries. With the risk that my comments are (no longer) useful, given that I haven't read the whole conversation:

gcrk As a user... What I am currently doing when uploading music is to think about "What should the visibility level of this music be?". Depending on this I am uploading the music to libraries

mjourdan Interesting! Could you tell us more about the kind of subsets you are trying to organise?

I conceptualise libraries the same way. I have two sets of music on the NAS: CDs (which were mostly bought by my parents and can be shared the close family on my pod) and Creative Commons music (which I downloaded from the internet and which is safe to share with anyone on the world). Just for practical file organisation-reasons, I keep these two sets of music in separate folders. So for me the concepts #1 'Library' and #2 'Storage' are closely tied.

#3 'Collections' would be something rather different. I see it as a collection of stuff I put together, regardless the library, the source - it's like a like a notebook that says 'I love this film on Peertube; I have this CD with the soundtrack on my pod; my friend published a podcast episode on his pod'. But an interactive notebook that allows me to immediately consume the content. And if I share my notebook, others can consume it as well as long as they have the right. (as described already in a way:

petitminion If a user add a foreign track to his collection that is public, he is not copying the audio file. he is just saying "I want to have access to this file and add the metadata to my collection". No need to copy the audio file.

(I initially thought that Libraries and Data sources might be linked, but keeping them separate would feel logical as well. I could have 2 libraries as described above, but 3 Data sources - one for my CDs, one for my parents' CDs, one for my CC music.)

petitminion

Thx a lot this is very great development of the library problem o/

I agree this will bring a lot of changes :s In the backend lots of things are built with the library concept as a storage and right management concept. So I totally agree with your propositions but I would just keep the concept of library has its original mean (instead of changing it to "storage", or maybe only change the front but this could be very confusing). I agree we could use API v2 to change the endpoints but its not only about the endpoints, its about a big part of the logic in the backend code and I fear only changing the endpoints will just create more confusion, we would need to change all the backend wording and it seams like a lot of work. When if we need a concept for the front to define all the content a user follow or owns we could use the concept "Content".
This way we only change some wording in the front, we keep our current concept and we implement the collection feature.

Sporiff

I think the pad in question is this one. It just outlines a basic flow of how we can work with new features.

This is a really thorough and detailed writeup, and I'm really glad you've brought attention back to this. The jargon around libraries has confused users from the start and has necessitated some really cumbersome explanations in documentation and UX copy. I agree that an entirely new UX flow and data breakup is necessary to address this.

On the point of migration, I don't think we necessarily need to move existing content. A lot of this can be worked around by splitting things up visually in the frontend for existing libraries (e.g. just labelling old libraries as "collections"). However, I'm the sort of person who really likes to have everything up-to-date and organized, so I would personally want to migrate my library content.

I think we can probably split this work up into a couple of epics:

Create a new library management system
Add migration process for existing library content

For each we need to lay out:

The API architecture
The frontend experience, including designs and UX flow
How we communicate this to users

petitminion

Sporiff
The new request flow is greeeat : D

Sporiff On the point of migration, I don't think we necessarily need to move existing content. A lot of this can be worked around by splitting things up visually in the frontend for existing libraries (e.g. just labelling old libraries as "collections")

This just make me understood that the proposition of @mjourdan is to replace the libraries with collections (and not with strorage as I understood in my last comment). I thought we would keep some kind of libraries to manage right access.

I think it would be better to keep libraries for rights access managemennt because :

We keep our current logic
If collection are meant for right management and music organization it could still be messy and a complicated to understand : what if I want to share only one track of a collection and keep the rest private ? With libraries I can just add the track to my public library and keep it in under any collection(s) I want. If I want to share a collection publicly, and it contains privates tracks, we can prompt a warning and propose to make theses tracks public : we just have to automatically add the tracks in the public library with the backend. Or not and the user will have access to the collection but will not see private tracks that are within.

petitminion

mjourdan

mjourdan That would basically encompass everything send by people, including music, podcasts, and even messages. We need to be a minimum specific.

My music ?

mjourdan It should be possible to share items [...] by creating a new collection from them, holding just them. Doing either of these could make those items appear in a dynamic collection called something like "public" or "sharing"

This is why I think it would be confusing : because collection would be used for right management and music organization. And they will have two right management access : one for the tracks it contains, one for the collection itself. (are the tracks in this collection public and is this collection public)
We will have to handle conflict : if I put a song that's in my "public collection" to "my private sleep collection" we will have to ask if the user wants the tracks to be set from public to private. If we don't when the user will add a track from "my public collection" to "my private collection" the track will be public, but it will be in the "private collection"

It would make sens that audiobook, podcast etc are stored in libraries that allow access management rights. And then you could add them to any collection.
Maybe this way content management will be easier : if you want to follow only one kind of content (like only music) of an artist or a user you go to its libraries and follow the public music library.

gcrk

mjourdan Yes, personally I could use some explanations from someone who knows the code about how many deduplication mechanisms Funkwhale currently has, and how they work (between libraries of a single user / of different user on the same pod / accross federation).

Currently we don't have any deduplication in place as far as I am aware.

mjourdan Interesting! Could you tell us more about the kind of subsets you are trying to organise?

For example when I work on my university papers I can't listen to music with text, so I basically have a radio which contains only instrumental music. Then again I do have some radio with all jazz and one with all german rap music, so I have one-button actions for different scenarios when I want to listen to music.

petitminion Yes we have channels for that 🙂 But technically channels are just public libraries that broadcast themselves automatically to all the network.

This is a major design flaw on the technical level and should be resolved by splitting those two things properly into different classes/domains.

mjourdan When asking at what level we should distinguish between different content types, my question wasn't clear at all. My question was which of the following hierarchy should we have?

I usually don't want to have them mixed up. There is time for music, there is time for podcasts and times for audio books. I don't want to launch some music and have a random part of an audio book mixed in. I also have different needs for queuing those different kinds of content, random and mixed playback of music makes sense, while audiobooks should be played in order.

I've read everything, but I might have missed some points. My take on the storage/sharing problem is the following, maybe too technical but probably this should be taken into account anyway.

So someone uploads a piece of audio. Funkwhale will create an Upload which basically adds some technical metadata and stores the actual audio file or at least a reference to it. I don't think we should change anything here. Funkwhale doesn't care if this is a song, a part of an audio book or an podcast episode. Its just naked data. This one is always private. Its also not playable and is never directly exposed.

In the next step right now a track is assigned. We should add differentiation here already, since it might be a podcast episode or a song. And we want to know this. You can imagine by just adding a paperclip to the actual upload with some additional data. Now this track, episode or chapter is again private, never linked to but can be played back (by requesting the information from the Upload actually).

Now those items can be parts of collections. As @mjourdan this should be pretty easy and available all around the UI. You can add whatever item to whatever collection, maybe differentiated by content type, reasons for this are listen above. Those collections can be shared or sharable (be federated).

To resolve the above concerns and considering those collections are actually pretty cheap in implementation but not in the UI, we can have something like a hidden, implicit collection. This basically allows us to click on a track, click "Share by URL". In the background Funkwhale will create a new collection, with a random ID but without a name. This collection will be shown nowhere, it will only lead to a hint on the track "This is shared by URL". This way you can share whatever you want with different groups without cluttering your interface with thousands of collections you need to manage. Its also pretty easy to implement, if we want to go the collection path anyways.

A quick note: If we go this path, we can actually have a clean migration from libraries to collections, since they share a common ground (The uploads) we wont touch. This can be used to built up the new thing allowing people to migrate slowly, as soon as they are ready. We can even hide the collections behind feature flags in the UI until we are confident they are stable.

I totally ignored channels here for the moment, but I think this breaks the scope anyway and should be another discussion anyways.

mjourdan

petitminion

My music ?

Maybe that's me nitpicking but I consider the use of possessive to be often intrusive, infantilising, or manipulative, when it is used to speak in the place of people. So Music would be left. Showing this in the sidebar instead of Library could be ok, but then we would have to:

drop the Explore / Music / Podcasts triptic we were about to implement.
drop the Channels section in the sidebar and replace it with only podcasts.

So that's possible, but it has implications too.

This is why I think it would be confusing : because collection would be used for right management and music organization. And they will have two right management access : one for the tracks it contains, one for the collection itself. (are the tracks in this collection public and is this collection public)
We will have to handle conflict : if I put a song that's in my "public collection" to "my private sleep collection" we will have to ask if the user wants the tracks to be set from public to private. If we don't when the user will add a track from "my public collection" to "my private collection" the track will be public, but it will be in the "private collection"

Mmh. Fair questions! Why couldn't we have a single management access at the collection level to avoid such conflicts? Sharing could either require:

attaching the tracks to any collection whose visibility is set to "pod" or "public" or shared by link
creating such collection if needed
creating a sharing link

Then unsharing would require:

closing access to the attached collections
detaching the tracks from any collection whose visibility is set to "pod" or "public", or shared by link (empty collections would be automatically deleted)
deleting the sharing link

What do you think of this?

It would make sens that audiobook, podcast etc are stored in libraries that allow access management rights. And then you could add them to any collection.

For files hosted on Funkwhale, yes, definetely. I am a little less sure about putting RSS subscriptions into libraries though. Haven't we channels for that? Other question: shall we distinguish between audiobooks, podcasts and music inside a given library, or should we use independent top-level libraries for that?

Maybe this way content management will be easier : if you want to follow only one kind of content (like only music) of an artist or a user you go to its libraries and follow the public music library.

I fully agree collection should never mix content of different nature.

petitminion

mjourdan Maybe that's me nitpicking but I consider the use of possessive to be often intrusive, infantilising, or manipulative, when it is used to speak in the place of people. So Music would be left. Showing this in the sidebar instead of Library could be ok, but then we would have to:
drop the Explore / Music / Podcasts triptic we were about to implement.
drop the Channels section in the sidebar and replace it with only podcasts.

So that's possible, but it has implications too.

In the design you propose you use "Music" I think its a good idea. This way the concept of "library" is free to be used for other things.

mjourdan What do you think of this?

I think this do not resolve the problems I raised. We still have two kinds of acces (collections access and track access) to manage in the same conceptual space...

mjourdan . I am a little less sure about putting RSS subscriptions into libraries though. Haven't we channels for that?

Yes we have channels for that 🙂 But technically channels are just public libraries that broadcast themselves automatically to all the network.

mjourdan shall we distinguish between audiobooks, podcasts and music inside a given library, or should we use independent top-level libraries for that?

I'm not sure I understand the question.
If we want to use libraries to manage access rights I think it should only contains one kind of audio-content to make the following logic easy to manage.

mjourdan

petitminion

Library

In the design you propose you use "Music" I think its a good idea. This way the concept of "library" is free to be used for other things.

I'm not sure what design you are referring to. Anyway, I doubt showing "Music" in the UI in the sidebar would keep the "library concept" free. Because we would still do have the idea of the-single-place-where-all-content-is-to-be-found (eventually aggregated from different sources).

I'm not sure I understand the question.
If we want to use libraries to manage access rights I think it should only contains one kind of audio-content to make the following logic easy to manage.

When asking at what level we should distinguish between different content types, my question wasn't clear at all. My question was which of the following hierarchy should we have?

Scenario 1: - single all-purpose library

Library
** M. Artists
** M. Albums
** M. Playlists
** M. Collections
** M. Favourites
** A. Books
** A. Authors
** A. Readers
** A. Collections

Scenario 2 - separate top-level libraries

Music library
** Artists
** Albums
** Playlists
** Collections
** Favourites
Audiobooks library

Sharing

I think this do not resolve the problems I raised. We still have two kinds of acces (collections access and track access) to manage in the same conceptual space...

It's true that could end up being too complex. Maybe adding sharing management at the content level, despite being often requested, could actually turn into a very bad thing.

On the other hand I am not sure dropping track access would solve any such thing as a conflict like in the concern you raised. Just to make sure it didn't go unnoticed: in the collection system I describe, any given album could appear simultaneously in (let's say) Collection A which is private, Collection B which is federated, and Collection C which is shared by link, and Collection D which is also shared by link to someone else.

For the sake of completeness I get back at an earlier quote of yours:

With libraries I can just add the track to my public library and keep it in under any collection(s) I want. If I want to share a collection publicly, and it contains privates tracks, we can prompt a warning and propose to make theses tracks public : we just have to automatically add the tracks in the public library with the backend. Or not and the user will have access to the collection but will not see private tracks that are within.

I would avoid this kind of exclusivity. All content uploaded would be private until explicitely shared by it's owner. Attaching a track to a public collection would be an explicit statement this track must be publicly accessible. Attaching a track to a (second) private collection would not be a statement that suddenly the track should disappear from the first one. It just means the second collection as a collection is not meant to be shared and that's ok.

Libraries vs collections vs channels

Yes we have channels for that 🙂 But technically channels are just public libraries that broadcast themselves automatically to all the network.

Currently the difference between channels and libraries is a source of confusion. It comes up from time to time from user feedback. I believe part of the reason of this is the channels never appeared in the sidebar like they were meant to, to notify subscriber of new content. Anyway, I worry adding rss feeds to libraries and collections would make things even more confusing.

As a complement to the distinction you point out, I would also add this one from @Sporiff (I've just found it in my notes!):

Libraries are sharable only between Funkwhale pods. If they are open to all, then anybody on another pod can paste in the address and follow the library.
Channels can be followed from other Fediverse software such as Mastodon. These are always available to other Fediverse software unless the pod is not federating with them for some reason.

Edit:
Just thinking out loud here… I wonder if making collections a third kind of channels would be a good idea?
Pros:

allowing people to share part of their library on the Fediverse
having a single way to share (channels), for different kind of purpose (collections or stuff you make)

Note:

we would probably want to avoid having some channels inside a library, while other channels would not be

petitminion

Library

mjourdan I'm not sure what design you are referring to. Anyway, I doubt showing "Music" in the UI in the sidebar would keep the "library concept" free. Because we would still do have the idea of the-single-place-where-all-content-is-to-be-found (eventually aggregated from different sources).

I'm refering to the penpot design where there is three categories in the side bar : "music", "explore" and "podcats". In this case we would not use the library concept in the front. Could be less confusing if we keep it on the back.

Sharing

mjourdan in the collection system I describe, any given album could appear simultaneously in (let's say) Collection A which is private, Collection B which is federated, and Collection C which is shared by link, and Collection D which is also shared by link to someone else.

Yes I understand this, but doing so the access right of a track is based on the more flexible access right of the collection it's in. It means you could share tracks without wanting. But this is actually how it works now. If you add the same track in a public library it ill be reachable.

So we can choose to keep our system like this and just change the concept of library with the concept of collection and add new features to it (dynamically pass a track, album, artist from a collection to another). But I still wouldn't change all the back-end terminology.
And if we drop libraries for collections, i would keep some rules like : only one audio content type per collection allowed. Or create a specific tag for collections so we don't follow audio-content we don't want to.

My proposition was to use the library concept in a user management space to handle tracks rights. But this is yet another thing to do and I think we can go with your proposition.

mjourdan I would avoid this kind of exclusivity. All content uploaded would be private until explicitely shared by it's owner.

this is the point yes. Allow a clear management of right access. But personaly I would prefer to keep things has they are now.

Channels

mjourdan Anyway, I worry adding rss feeds to libraries and collections would make things even more confusing.

yes we should never do that !

mjourdan I wonder if making collections a third kind of channels would be a good idea?

channels are not meant for that. Collection could only be public...so better not haha

For me the confusion about channel come from the fact that there is podcast channels and music channels. And when you go to /subscriptions clicking on "channels" you got a mix of artist and podcast channels...Its confusing. Instead of "channels" in the ui, we should have "podcast" (like we have actually in the explore section) and the music channels should be seemlessly integrated into the music content.

petitminion

We had a vocal discussion with @mjourdan and we writed our conclusions in this pad : https://pad.funkwhale.audio/zs-SKbVgROapjv56lsWWtw?both

abstract :

Collection would be the new name for our current libraries
How we should call default collections intented for content management ?
should we keep radios ?
how to handle the upload process ?

petitminion

gcrk Currently we don't have any deduplication in place as far as I am aware.

I think means the duplication of the content when your reupload the same file.

gcrk This is a major design flaw on the technical level and should be resolved by splitting those two things properly into different classes/domains.

Channel class inherit from the Library class. I don't see the issue but maybe we should speak about this when we work on the technical aspect of this design.

gcrk In the background Funkwhale will create a new collection, with a random ID but without a name. This collection will be shown nowhere, it will only lead to a hint on the track

Yeah I totally agree on this method. In our report we create the concept of "default or special collection" that could also be used to implement the use case. Without having to create a new collection entry in the db. Also we could keep the "special" collection called "libraries" in the backend since they do what our libraries are meant to : right and share management.
But maybe this is too much technical and we should focus on the design first, agree on it, and then starting the different working groups ?

mjourdan

Deduplication

gcrk

Currently we don't have any deduplication in place as far as I am aware.
[...]
So someone uploads a piece of audio. Funkwhale will create an Upload which basically adds some technical metadata and stores the actual audio file or at least a reference to it. I don't think we should change anything here.

Uploading a piece of audio to finally actually store only a reference to it is what I would call deduplication. Not sure there is nothing to change here, because I suspect this causes these annoying odd things. That is to say, not finding content where it is expected, and technical maintenance to "purge" the uploaded content. If I'm not mistaken, the upload process doesn't only add references to already uploaded files, it also removes them which breaks the libraries where the content was first added to.

User story

For example when I work on my university papers I can't listen to music with text

Thank you for sharing!

Sharing

This basically allows us to click on a track, click "Share by URL".

Yes, sharing by url should be allowed for playlists and albums too. Same for the embedded widget.

Channels

I totally ignored channels here for the moment, but I think this breaks the scope anyway and should be another discussion anyways.

I think so.

Playlists

Is there a reason to have both playlists and collections? What sets them apart? Don't they have the same feature set anyways? Wouldn't it be better to merge them into one thing?

They don't have the same goals, so I don't think their feature sets overlap much.

Playlists are simple list of tracks aimed to be played contiguously. They can take the form of .m3u files that most (if not all) software music player would handle.

Collections on the other hand are about sorting out your ever growing music library. They can be seen also as mimicking what happens when you share your discs with your friends: you would rather share a handful of discs rather than a single one. And amongst those discs, maybe there is even a couple of playlists you burned.

petitminion

maybe we could merge them anyway by making collections ordered
If we merge them the design will be easier : we still have a playlist menu in the collection, this menu contain all the tracks of the collection ordered by the date they where added but you can change the order in that menu..

Let's say we do this. Questions:

someone has several playlists and wants to share them all at once. Would we create playlists of playlists for that?
how to handle nested collections in the playlist menu? How deep would we go?
for people who have several playlists, would we turn them all into as many collections? Why?

gcrk

My comments on the current spec draft:

Collections would be a way to organise any kind of music item, includings single tracks, albums, whole artist discographys, or playlists. Adding an item to a collection would be as easy as adding it to a playlist: open the 3-dots menu, select “organise and share” in the popover, and (un)tick as many collection checkboxes as you would like.

Is there a reason to have both playlists and collections? What sets them apart? Don't they have the same feature set anyways? Wouldn't it be better to merge them into one thing?

I will proceed tomorrow, I am noticing I am too tired to understand the whole thing now 😃

petitminion

gcrk Is there a reason to have both playlists and collections? What sets them apart? Don't they have the same feature set anyways? Wouldn't it be better to merge them into one thing?

Playlists are ordered while collections are not 🙂 but maybe we could merge them anyway by making collections ordered
If we merge them the design will be easier : we still have a playlist menu in the collection, this menu contain all the tracks of the collection ordered by the date they where added but you can change the order in that menu.. We don't have to implement playlist federation or management, it will be included in the collection management yeeaaah
I think its a cool idea 🙂 (#lazyness_is_efficiency)

petitminion

mjourdan how to handle nested collections in the playlist menu? How deep would we go?

Yeah the main issue, I wouldn't create nested collection/playlist. I would only allow one playlist by collection.
If we want various playlist in one collection we should keep separated concepts

gcrk

Follow-up from Mastodon: @Sporiff created a new draft after we had a call about technical details and possibilities. Here it is: https://pad.funkwhale.audio/8Qvwu_jZRuGPMvgNvjIhAg?view

Questions from @petitminion

why not use the user page instead of creating a "user library" page ?
why create a "collections" pages when we have the "explore" page. I think a "collections" page will be confusing, while "explore" or "federated content" is more clear. The content should be presented has playlist, feeds etc and not only as a list of collections.

If we want a list of all accessibles collections we could create it but in a sub-menu of the explore section with a very clear name like "explore unknown collections"

Questions by @wvffle

How about the Favorites automatic collection?
Can a collection be added to another collection? That way the library might itself be implemented as a collection which may simplify things.
I assume that a playlist can also be added to a collection, right? What would be a difference between a playlist and a collection in terms of playback? Can user change the order of tracks in a collection? If so, wouldn't that make playlists obsolete?
Whether the default collection is Private content or Public content should be changable in the user settings in my opinion.
Wouldn't an ability to upload to a specific collection be simpler? Just go to the desired collection page, click the upload button and the uploaded content is automatically added to the said collection.

Answers by @Sporiff

I'm not sure that it's simpler to have favorites as a collection or in its current form. It may be better as a collection given that it would give us the option to include other types (albums/artists)
This would be a question for @georgkrause, but the library isn't really a container. The uploads can float freely without needing a container, so it's conceptually not that complex.
This depends on 2. From our discussions, a playlist is just a collection that is extended to include track positions
Possibly, but the safest assumption (and sanest default) is for the content to be private. This avoids any possibility of content being made public by mistake
This is something I would have in addition. For example, users can go to a specific collection and choose to upload music there. But the happy path should be that we don't introduce unnecessary steps for uploading content if the user doesn't want to create collections or just wants to add music to their library then sort it later.

gcrk

@wvffle How about the Favorites automatic collection?
@Sporiff I'm not sure that it's simpler to have favorites as a collection or in its current form. It may be better as a collection given that it would give us the option to include other types (albums/artists)

I don't agree. The current implementation of favorites is abstracted enough to favorite whatever you want, its just not in the API and this is called stable. So its a dilemma we can solve pretty easy with V2.

Also, favorites are a concept of activitypub which can be applied to everything, eg users. But users cannot be in a collection, so it doesn't make any sense to actually store the favorite as a collection.

If we want to have a automatically generated collection which contains all the favorites, we can add one, but for me the dynamic collections are technically different from static collections, since they are unable to manage access properly.

@wvffle Can a collection be added to another collection? That way the library might itself be implemented as a collection which may simplify things.
@Sporiff This would be a question for @georgkrause, but the library isn't really a container. The uploads can float freely without needing a container, so it's conceptually not that complex.

Well, library as a term for all playable tracks isn't really something which has an implementation, its more a concept.

@wvffle I assume that a playlist can also be added to a collection, right? What would be a difference between a playlist and a collection in terms of playback? Can user change the order of tracks in a collection? If so, wouldn't that make playlists obsolete?

I can't see why we would want to add collections to collections? What is this relation displaying? Can't you just create another collection next to the first, eg by "Duplicate this collection" which takes over all properties and content and you can easily remove tracks from it?

Playlists are not obsolete. Playlists to have a order. Also, playlists don't manage access to uploads and are therefore able to implement something like shared ownership.

@Sporiff This depends on 2. From our discussions, a playlist is just a collection that is extended to include track positions

We had this in the first place, thats right. But later on we noticed we need to have something which allows us to implement shared owner ship, so playlists would be a different concept.

Possibly, but the safest assumption (and sanest default) is for the content to be private. This avoids any possibility of content being made public by mistake

Agree. Convention over Configuration, and making public the default is a nogo.