Quality filter proposal

Sporiff

Hi everyone o/

One of the issues we're keen to tackle in upcoming releases is the quality of content that is shown on the landing page of a Funkwhale pod. This is the first impression new users get of the pod and its content, so seeing a collection of poorly tagged content without album art may not be desirable. Additionally, low quality audio or poorly supported formats can give a really bad impression of the service as a whole.

To address this, we propose adding a set of quality filters to our endpoints. These filters will enable instance admins to apply a set of filters to the API calls made for Explore page results so that only the best quality content is shown on the initial landing page. All other pages (e.g. Album/Artist search, your library overview) will not be affected by this change.

The quality filters we have in mind are as follows:

A minimum bitrate
A list of requested formats
Mandatory Album art
Mandatory Release dates
Mandatory Genre tags

Each of these would be configurable by the instance admin so that they can tailor the experience for their pod.

Check out the proposed spec here: https://dev.funkwhale.audio/funkwhale/funkwhale/-/wikis/specs/Funkwhale-quality-filter

jooola

Great idea!

If I understand this correctly the admin would define some defaults for the filter params but those values can be changed from the UI ? This isn't about creating a new "quality" property that will be used to always filter the content ?

I think the formats and bitrate should be more abstract, or configurable as mapping.

A bitrate can represent a lot of different quality depending on the format/bit depth/sample rate..., so I feel like it doesn't make sens to use it alone. To keep the filter simple, we could provide a function that transform abstract qualities (e.g. 1 to 10) to a set of formats/bitrates/... (mp3, >=256kbps), (ogg, >=192kbps), flac (24bit) ...

Or we use a mapping using as key the format, and as value a list of conditions ? And make sure this can be extended to use other audio properties.

Maybe this is adding too much complexity, in that case I would rather go with the abstraction and wait until everything is implemented to allow a public usage of this "complex" filtering.

The has_art filter seem only to apply for albums right ? It could also be applied to artists, and we could require the artist to has some artwork.

I think the feature should be opt in, but for the default settings I would suggest:

bitrate: As per above I don't know how to fill this value properly
format: flac,ogg,vorbis,mp3,aac (no strong opinion)
has_tags: true
has_art: true
has_release_date: false

Sporiff

jooola

If I understand this correctly the admin would define some defaults for the filter params but those values can be changed from the UI ? This isn't about creating a new "quality" property that will be used to always filter the content ?

That's correct. The plan is not to limit the quality of uploads etc. since people should be able to upload any content they want to. This is just to give a good first impression for the landing page by specifying some minimum quality parameters. There's no plan at the moment to extend this to other parts of the app such as search screens, although once the filters exist we would be able to consider it if there's call for it.

A bitrate can represent a lot of different quality depending on the format/bit depth/sample rate..., so I feel like it doesn't make sens to use it alone. To keep the filter simple, we could provide a function that transform abstract qualities (e.g. 1 to 10) to a set of formats/bitrates/... (mp3, >=256kbps), (ogg, >=192kbps), flac (24bit) ...

Interesting idea. This would require us to come up with a sensible set of defaults for each filetype as well as document each one and communicate the implications of the quality scale properly in the frontend. If done correctly, an abstract scale would be much more user friendly!

The has_art filter seem only to apply for albums right ? It could also be applied to artists, and we could require the artist to has some artwork.

Yes, there's no reason it can't be extended to artists. At the moment I don't think we show artists on the front page, but there's no harm building it in for future use.

has_release_date: false

Could you explain why you wouldn't set this one by default? If we're assuming an admin turns on the quality filter to give the best impression to new users, wouldn't having proper release dates (at least years) be desirable as a default?

Sporiff

petitminion Yes, I'm going to add the MBID switch (has_mbid) into the quality filter spec in GitLab. This should be a fairly trivial thing to implement, and could then easily be added to the frontend too 👍️

petitminion As for this, it opens up a bit of complexity and requires us to do some working out for each of our supported file formats as mentioned by jooola. If we can work out the thresholds for each, I still think it would be good from the admin's side to have a much more basic selection of:

No filter (all quality returned)
Low (all quality above a low threshold returned)
Medium (all quality above a medium threshold returned)
High (only high quality content returned)

Rather than them needing to specifically select the bitrate (or whatever measure we're using) for each filetype that might be on their pod or come across federation.

Sporiff

Based on these discussions, @gcrk and I have decided we'll do the following:

Remove the bitrate filter for now and do some research on exactly how each file format's quality works
Implement the metadata quality filters and filetype selection in the endpoints
Add the frontend controls for the aforementioned filters

At a later date, we can create a selection of audio quality for each supported file format and map this to a High, Medium, and Low quality filter which can enable admins to broadly select the quality of all audio formats they want to use. We'll communicate clearly what these filters actually map to when we've worked that out.

petitminion

I think the bitrate is too important to be set appart or after. We could do a simple mapping like this :

FLAC (Free Lossless Audio Codec): Offers high-quality audio without any loss of information. Bitrate can vary based on the complexity of the audio (but it will alway be much higer tham compressed formats)
MP3 (MPEG Audio Layer 3):
128 kbps: Acceptable quality for most listeners, suitable for portable devices and online streaming.
192 kbps: Good quality, often considered the sweet spot for balancing quality and file size.
256 kbps: Very good quality, especially for casual listening.
320 kbps: Excellent quality, providing high fidelity and minimal perceivable loss.
AAC (Advanced Audio Coding) and OGG (Ogg Vorbis):
128 kbps: Equivalent to or slightly better than MP3 at the same bitrate.
192 kbps: Similar to 256 kbps MP3 in terms of quality.
256 kbps: Very good quality, comparable to 320 kbps MP3.
320 kbps: Excellent quality, offering transparency for most listeners.

note that this will never be acurate because it depends on the compression algorithm but its better than nothing

petitminion

Another thing I think I would like to have is a mbid quality filter. it could be use for this has a complement of : https://forum.funkwhale.audio/d/243-new-feature-only-allow-musicbrainz-tagged-file-to-be-uploaded/22

To let the user choose if he wants only mbid content to be displayed in frontend (while the linked issue is more about pod politics, administration)

petitminion

Sporiff I couln't find the list of our supported formats :s do you know where it is ? I alread made a proposiion for 4 audio format I can look into the other ones

Sporiff

petitminion

flac
ogg
mp3
opus
aac
m4a
aiff
aif

petitminion

proposition :


|                         |FLAC  |AIF   |AIFF  |Opus|AAC  |OGG  |MP3  |
|-------------------------|------|------|------|----|-----|-----|-----|
|Low  <=                   |never |never |never |96  |96   |96   |192  |
|Medium       <=           |never |never |never |128 |128  |192  |256  |
|High         <=          |always|always|always|160 |288  |256  |320  |
|Ultra       >=             |always|always|always|510 |never|never|never|

m4a can contain AAC, MP3 or others audio format so...its complicated ^^ I would not alow m4a audio file in fw because its more intended for video than audio

Sporiff

petitminion Since it's currently allowed, we have to assume that people have previously uploaded m4a content. Removing support for the filetype would be an undesirable break in behavior for the sake of a quality check in my opinion.

Do you think it's possible to sniff out the underlying codec name and use that to determine quality somehow? I know you can do this with ffprobe like so:

$ ffprobe -v error -select_streams a:0 -show_entries stream=codec_name -of default=noprint_wrappers=1:nokey=1 myfile.m4a

petitminion

Sporiff I suppose its possible to extract the audio format of the m4a file but what bother me is that this audio format might not be supported by funkwhale.. (like MPEG-4 Part 3) Actually I don't know how we handle m4a, maybe if the audio format is not supported the m4a file is not uploaded..

petitminion

I lookup a little bit into this. Typically m4a use AAC audio format so we could use this.

from mutagen import File
file_path = 'path/to/your/file.m4a'
# Load the audio file using mutagen
audio = File(file_path)
audio.info.codec
Out[13]: 'mp4a.40.2'
audio.info.codec_description
Out[14]: 'AAC LC'

I think ISO/IEC 14496-3:2009 give the last m4a codec formats spec but I dont want to pay 200 dollars for this to get a list of codecs codes. So we could parse the codec_description. If there is a supported audio format inside we use this audio format quality table, if not we raise an error.

for format in $list_of_supported_audio_format :
if format in audio.info.codec_description:
    return format
else : 
   raise "Format not support {audio.info.codec_description}"

petitminion

@Sporiff any new on this ?
has_mbid isn't in the spec

Sporiff

petitminion Yes, looks like I missed this. I'll add it presently.