Unable to upload ~150MB mp3

ElFDA

Hello, we have problem uploading big file
I configured NGINX_MAX_BODY_SIZE=999M on .env file and client_max_body_size 999M; on the nginx that proxypass the requests but we got 413 (Payload Too Large) on the browser console after file is uploaded.
I also checked on the docker side:
the NGINX_MAX_BODY_SIZE var is correctly configured also on docker nginx conf is client_max_body_size 999M;

Thank you
fda

petitminion

Do you have enough space allowed for your funkwhale user ? Could you share backends and console logs ?

gcrk

@petitminion We already did some basic evaluation in the chat yesterday, I don't think this is related to the storage, I was suspicious of some limitation on the http level...

ElFDA

gcrk backend logs are clean and the users have a very high quota
Thanks 🙂

petitminion

So you have the nginx docker container running on a vm, that get internet from a reverse proxy ? Could you share the nginx config of the reverse proxy ?

gcrk I was suspicious of some limitation on the http level…

Hum..I'm not aware of such limitations, you might have some hints ? In my old instance I add a similar configuration, and was able to upload files of 90mb but maybe the 100mb is a threshold

petitminion

also where is your reverse proxy hosted ? Is the reverse proxy and the vm in the same machine ? Maybe there is limitations from your isp or some firewall ?

ElFDA

petitminion I have a my own proxmox server with several VMs so no isp limitations
This is nginx proxy revers for funkwhale:

server_name  stereo.kenobit.it;
        client_max_body_size 200M;
    location / {
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_set_header X-NginX-Proxy true;
        proxy_pass http://10.10.10.12:80;
        proxy_redirect off;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }

I downsized max body size to 200 for tests

gcrk

@ElFDA I created an account on your instance called gcrkrause, mind giving me some quota to test this? Because at the moment I only have 1MB

ElFDA

gcrk yep, gimme a sec
just gave you 200mb upload quota

petitminion

ElFDA proxmox machine is on a datacenter ? can you send large files to your other services ?

ElFDA

petitminion it's in my house 😃
I have other services that can receive big files (even a Castopod istance with the same podcast mp3 that we are trying to upload to Funkwhale)

petitminion

ElFDA so this is related to funkwhale, can you send the nginx config of the container ? what funkwhale version do you run ? Could you update big file in the past ? Sorry I think these are the last questions ^^

gcrk

petitminion Sorry I think these are the last questions ^^

This sounds like you got an idea already, so I will leave the case to you. Feel free to ping me if my help is required again 🙂

ElFDA

Hello! Sorry for the late anwer, has been busy days :/
I run the latest version (1.40) and this is the first installation so I cannot tell you if with there was the same problem in the past release.
Here it is the nginx docker configuration:

 # nginx -T
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
# configuration file /etc/nginx/nginx.conf:

user  nginx;
worker_processes  auto;

error_log  /var/log/nginx/error.log notice;
pid        /var/run/nginx/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;
}

# configuration file /etc/nginx/mime.types:

types {
    text/html                                        html htm shtml;
    text/css                                         css;
    text/xml                                         xml;
    image/gif                                        gif;
    image/jpeg                                       jpeg jpg;
    application/javascript                           js;
    application/atom+xml                             atom;
    application/rss+xml                              rss;

    text/mathml                                      mml;
    text/plain                                       txt;
    text/vnd.sun.j2me.app-descriptor                 jad;
    text/vnd.wap.wml                                 wml;
    text/x-component                                 htc;

    image/avif                                       avif;
    image/png                                        png;
    image/svg+xml                                    svg svgz;
    image/tiff                                       tif tiff;
    image/vnd.wap.wbmp                               wbmp;
    image/webp                                       webp;
    image/x-icon                                     ico;
    image/x-jng                                      jng;
    image/x-ms-bmp                                   bmp;

    font/woff                                        woff;
    font/woff2                                       woff2;

    application/java-archive                         jar war ear;
    application/json                                 json;
    application/mac-binhex40                         hqx;
    application/msword                               doc;
    application/pdf                                  pdf;
    application/postscript                           ps eps ai;
    application/rtf                                  rtf;
    application/vnd.apple.mpegurl                    m3u8;
    application/vnd.google-earth.kml+xml             kml;
    application/vnd.google-earth.kmz                 kmz;
    application/vnd.ms-excel                         xls;
    application/vnd.ms-fontobject                    eot;
    application/vnd.ms-powerpoint                    ppt;
    application/vnd.oasis.opendocument.graphics      odg;
    application/vnd.oasis.opendocument.presentation  odp;
    application/vnd.oasis.opendocument.spreadsheet   ods;
    application/vnd.oasis.opendocument.text          odt;
    application/vnd.openxmlformats-officedocument.presentationml.presentation
                                                     pptx;
    application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
                                                     xlsx;
    application/vnd.openxmlformats-officedocument.wordprocessingml.document
                                                     docx;
    application/vnd.wap.wmlc                         wmlc;
    application/wasm                                 wasm;
    application/x-7z-compressed                      7z;
    application/x-cocoa                              cco;
    application/x-java-archive-diff                  jardiff;
    application/x-java-jnlp-file                     jnlp;
    application/x-makeself                           run;
    application/x-perl                               pl pm;
    application/x-pilot                              prc pdb;
    application/x-rar-compressed                     rar;
    application/x-redhat-package-manager             rpm;
    application/x-sea                                sea;
    application/x-shockwave-flash                    swf;
    application/x-stuffit                            sit;
    application/x-tcl                                tcl tk;
    application/x-x509-ca-cert                       der pem crt;
    application/x-xpinstall                          xpi;
    application/xhtml+xml                            xhtml;
    application/xspf+xml                             xspf;
    application/zip                                  zip;

    application/octet-stream                         bin exe dll;
    application/octet-stream                         deb;
    application/octet-stream                         dmg;
    application/octet-stream                         iso img;
    application/octet-stream                         msi msp msm;

    audio/midi                                       mid midi kar;
    audio/mpeg                                       mp3;
    audio/ogg                                        ogg;
    audio/x-m4a                                      m4a;
    audio/x-realaudio                                ra;

    video/3gpp                                       3gpp 3gp;
    video/mp2t                                       ts;
    video/mp4                                        mp4;
    video/mpeg                                       mpeg mpg;
    video/quicktime                                  mov;
    video/webm                                       webm;
    video/x-flv                                      flv;
    video/x-m4v                                      m4v;
    video/x-mng                                      mng;
    video/x-ms-asf                                   asx asf;
    video/x-ms-wmv                                   wmv;
    video/x-msvideo                                  avi;
}

# configuration file /etc/nginx/conf.d/default.conf:
upstream funkwhale-api {
    # depending on your setup, you may want to update this
    server api:5000;
}

# Required for websocket support.
map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}


server {
    listen      80;
    listen [::]:80;

    server_name _;

    # General configs
    root /usr/share/nginx/html;
    client_max_body_size 200M;
    charset utf-8;

    # compression settings
    gzip on;
    gzip_comp_level    5;
    gzip_min_length    256;
    gzip_proxied       any;
    gzip_vary          on;
    gzip_types
        application/javascript
        application/vnd.geo+json
        application/vnd.ms-fontobject
        application/x-font-ttf
        application/x-web-app-manifest+json
        font/opentype
        image/bmp
        image/svg+xml
        image/x-icon
        text/cache-manifest
        text/css
        text/plain
        text/vcard
        text/vnd.rim.location.xloc
        text/vtt
        text/x-component
        text/x-cross-domain-policy;
    # end of compression settings

    # headers
    add_header Content-Security-Policy "default-src 'self'; connect-src https: wss: http: ws: 'self' 'unsafe-eval'; script-src 'self' 'wasm-unsafe-eval'; style-src https: http: 'self' 'unsafe-inline'; img-src https: http: 'self' data:; font-src https: http: 'self' data:; media-src https: http: 'self' data:; object-src 'none'";
    add_header Referrer-Policy "strict-origin-when-cross-origin";
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header Service-Worker-Allowed "/";

    location /api/ {
        include /etc/nginx/funkwhale_proxy.conf;
        # This is needed if you have file import via upload enabled.
        client_max_body_size 200M;
        proxy_pass   http://funkwhale-api;
    }

    location ~ ^/library/(albums|tracks|artists|playlists)/ {
        include /etc/nginx/funkwhale_proxy.conf;
        proxy_pass   http://funkwhale-api;
    }

    location /channels/ {
        include /etc/nginx/funkwhale_proxy.conf;
        proxy_pass   http://funkwhale-api;
    }

    location ~ ^/@(vite-plugin-pwa|vite|id)/ {
        include /etc/nginx/funkwhale_proxy.conf;
        alias /usr/share/nginx/html/;
        try_files $uri $uri/ /index.html;
    }

    location /@ {
        include /etc/nginx/funkwhale_proxy.conf;
        proxy_pass   http://funkwhale-api;
    }

    location / {
        expires 1d;
        include /etc/nginx/funkwhale_proxy.conf;
        alias /usr/share/nginx/html/;
        try_files $uri $uri/ /index.html;
    }

    location ~ "/(front/)?embed.html" {
        alias /usr/share/nginx/html/embed.html;
        add_header Content-Security-Policy "connect-src https: http: 'self'; default-src 'self'; script-src 'self' unpkg.com 'unsafe-inline' 'unsafe-eval'; style-src https: http: 'self' 'unsafe-inline'; img-src https: http: 'self' data:; font-src https: http: 'self' data:; object-src 'none'; media-src https: http: 'self' data:";
        add_header Referrer-Policy "strict-origin-when-cross-origin";

        expires 1d;
    }

    location /federation/ {
        include /etc/nginx/funkwhale_proxy.conf;
        proxy_pass   http://funkwhale-api;
    }

    # You can comment this if you do not plan to use the Subsonic API.
    location /rest/ {
        include /etc/nginx/funkwhale_proxy.conf;
        proxy_pass   http://funkwhale-api/api/subsonic/rest/;
    }

    location /.well-known/ {
        include /etc/nginx/funkwhale_proxy.conf;
        proxy_pass   http://funkwhale-api;
    }

    # Allow direct access to only specific subdirectories in /media
    location /media/__sized__/ {
        alias /srv/funkwhale/data/media/__sized__/;
        add_header Access-Control-Allow-Origin '*';
    }

    # Allow direct access to only specific subdirectories in /media
    location /media/attachments/ {
        alias /srv/funkwhale/data/media/attachments/;
        add_header Access-Control-Allow-Origin '*';
    }

    # Allow direct access to only specific subdirectories in /media
    location /media/dynamic_preferences/ {
        alias /srv/funkwhale/data/media/dynamic_preferences/;
        add_header Access-Control-Allow-Origin '*';
    }

    # This is an internal location that is used to serve
    # media (uploaded) files once correct permission / authentication
    # has been checked on API side.
    # Comment the "NON-S3" commented lines and uncomment "S3" commented lines
    # if you're storing media files in a S3 bucket.
    location ~ /_protected/media/(.+) {
        internal;
        alias   /srv/funkwhale/data/media/$1;                                           # NON-S3
        # Needed to ensure DSub auth isn't forwarded to S3/Minio, see #932.
#       proxy_set_header Authorization "";                                  # S3
#       proxy_pass $1;                                                      # S3
        add_header Access-Control-Allow-Origin '*';
    }

    location /_protected/music/ {
        # This is an internal location that is used to serve
        # local music files once correct permission / authentication
        # has been checked on API side.
        # Set this to the same value as your MUSIC_DIRECTORY_PATH setting.
        internal;
        alias   /srv/funkwhale/data/music/;
        add_header Access-Control-Allow-Origin '*';
    }

    location /manifest.json {
        # If the reverse proxy is terminating SSL, nginx gets confused and redirects to http, hence the full URL
        return 302 https://stereo.kenobit.it/api/v1/instance/spa-manifest.json;
    }

}

# configuration file /etc/nginx/funkwhale_proxy.conf:
# use this one if you put the nginx container behind another proxy
# you will have to set some headers on this proxy as well to ensure
# everything works correctly, you can use the ones from the funkwhale_proxy.conf file
# at https://dev.funkwhale.audio/funkwhale/funkwhale/blob/develop/deploy/funkwhale_proxy.conf
# your proxy will also need to support websockets

proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $http_x_forwarded_proto;
proxy_set_header X-Forwarded-Host $http_x_forwarded_host;
proxy_set_header X-Forwarded-Port $http_x_forwarded_port;
proxy_redirect off;

# websocket support
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;

Thank you very much for your interest 🙂