Issue with updating to 1.3.1

Skro

Hello,
I'm updating Funkwhale 1.2.10 ==> 1.3.1 using the Quick Install script (https://docs.funkwhale.audio/administrator/upgrade/quick-install.html - side note, there is a "." at the end of the code that prevents the copy/pasting of the script to run. No big deal, but still), and the script fails with a nasty message:
TASK [funkwhale : Collect static files] *************************************************************************************************************************************************** fatal: [127.0.0.1]: FAILED! => {"changed": true, "cmd": ["/srv/funkwhale/virtualenv/bin/python", "api/manage.py", "collectstatic", "--no-input"], "delta": "0:00:01.267212", "end": "2023-07-08 08:37:10.557166", "msg": "non-zero return code", "rc": 1, "start": "2023-07-08 08:37:09.289954", "stderr": "/srv/funkwhale/api/manage.py:7: DeprecationWarning: the './manage.py' script has been deprecated, please use the 'funkwhale-manage' entrypoint instead (e.g. 'funkwhale-manage migrate')\n warnings.warn(\n2023-07-08 08:37:10,187 funkwhale_api.config INFO Loaded env file at /srv/funkwhale/config/.env\n2023-07-08 08:37:10,187 funkwhale_api.config INFO Running with the following plugins enabled: funkwhale_api.contrib.scrobbler, funkwhale_api.contrib.listenbrainz, funkwhale_api.contrib.maloja\nTraceback (most recent call last):\n File \"/srv/funkwhale/api/manage.py\", line 15, in <module>\n raise SystemExit(main())\n File \"/srv/funkwhale/api/funkwhale_api/main.py\", line 30, in main\n execute_from_command_line(sys.argv)\n File \"/srv/funkwhale/virtualenv/lib/python3.10/site-packages/django/core/management/__init__.py\", line 419, in execute_from_command_line\n utility.execute()\n File \"/srv/funkwhale/virtualenv/lib/python3.10/site-packages/django/core/management/__init__.py\", line 395, in execute\n django.setup()\n File \"/srv/funkwhale/virtualenv/lib/python3.10/site-packages/django/__init__.py\", line 24, in setup\n apps.populate(settings.INSTALLED_APPS)\n File \"/srv/funkwhale/virtualenv/lib/python3.10/site-packages/django/apps/registry.py\", line 91, in populate\n app_config = AppConfig.create(entry)\n File \"/srv/funkwhale/virtualenv/lib/python3.10/site-packages/django/apps/config.py\", line 124, in create\n mod = import_module(mod_path)\n File \"/usr/lib/python3.10/importlib/__init__.py\", line 126, in import_module\n return _bootstrap._gcd_import(name[level:], package, level)\n File \"<frozen importlib._bootstrap>\", line 1050, in _gcd_import\n File \"<frozen importlib._bootstrap>\", line 1027, in _find_and_load\n File \"<frozen importlib._bootstrap>\", line 1006, in _find_and_load_unlocked\n File \"<frozen importlib._bootstrap>\", line 688, in _load_unlocked\n File \"<frozen importlib._bootstrap_external>\", line 883, in exec_module\n File \"<frozen importlib._bootstrap>\", line 241, in _call_with_frames_removed\n File \"/srv/funkwhale/virtualenv/lib/python3.10/site-packages/daphne/apps.py\", line 6, in <module>\n import daphne.server # noqa: F401\n File \"/srv/funkwhale/virtualenv/lib/python3.10/site-packages/daphne/server.py\", line 7, in <module>\n from twisted.internet import asyncioreactor # isort:skip\n File \"/srv/funkwhale/virtualenv/lib/python3.10/site-packages/twisted/internet/asyncioreactor.py\", line 19, in <module>\n from twisted.internet.posixbase import (\n File \"/srv/funkwhale/virtualenv/lib/python3.10/site-packages/twisted/internet/posixbase.py\", line 19, in <module>\n from twisted.internet import error, tcp, udp\n File \"/srv/funkwhale/virtualenv/lib/python3.10/site-packages/twisted/internet/tcp.py\", line 38, in <module>\n from twisted.internet._newtls import (\n File \"/srv/funkwhale/virtualenv/lib/python3.10/site-packages/twisted/internet/_newtls.py\", line 18, in <module>\n from twisted.protocols.tls import TLSMemoryBIOFactory, TLSMemoryBIOProtocol\n File \"/srv/funkwhale/virtualenv/lib/python3.10/site-packages/twisted/protocols/tls.py\", line 40, in <module>\n from OpenSSL.SSL import (\n File \"/srv/funkwhale/virtualenv/lib/python3.10/site-packages/OpenSSL/__init__.py\", line 8, in <module>\n from OpenSSL import crypto, SSL\n File \"/srv/funkwhale/virtualenv/lib/python3.10/site-packages/OpenSSL/crypto.py\", line 1556, in <module>\n class X509StoreFlags(object):\n File \"/srv/funkwhale/virtualenv/lib/python3.10/site-packages/OpenSSL/crypto.py\", line 1577, in X509StoreFlags\n CB_ISSUER_CHECK = _lib.X509_V_FLAG_CB_ISSUER_CHECK\nAttributeError: module 'lib' has no attribute 'X509_V_FLAG_CB_ISSUER_CHECK'", "stderr_lines": ["/srv/funkwhale/api/manage.py:7: DeprecationWarning: the './manage.py' script has been deprecated, please use the 'funkwhale-manage' entrypoint instead (e.g. 'funkwhale-manage migrate')", " warnings.warn(", "2023-07-08 08:37:10,187 funkwhale_api.config INFO Loaded env file at /srv/funkwhale/config/.env", "2023-07-08 08:37:10,187 funkwhale_api.config INFO Running with the following plugins enabled: funkwhale_api.contrib.scrobbler, funkwhale_api.contrib.listenbrainz, funkwhale_api.contrib.maloja", "Traceback (most recent call last):", " File \"/srv/funkwhale/api/manage.py\", line 15, in <module>", " raise SystemExit(main())", " File \"/srv/funkwhale/api/funkwhale_api/main.py\", line 30, in main", " execute_from_command_line(sys.argv)", " File \"/srv/funkwhale/virtualenv/lib/python3.10/site-packages/django/core/management/__init__.py\", line 419, in execute_from_command_line", " utility.execute()", " File \"/srv/funkwhale/virtualenv/lib/python3.10/site-packages/django/core/management/__init__.py\", line 395, in execute", " django.setup()", " File \"/srv/funkwhale/virtualenv/lib/python3.10/site-packages/django/__init__.py\", line 24, in setup", " apps.populate(settings.INSTALLED_APPS)", " File \"/srv/funkwhale/virtualenv/lib/python3.10/site-packages/django/apps/registry.py\", line 91, in populate", " app_config = AppConfig.create(entry)", " File \"/srv/funkwhale/virtualenv/lib/python3.10/site-packages/django/apps/config.py\", line 124, in create", " mod = import_module(mod_path)", " File \"/usr/lib/python3.10/importlib/__init__.py\", line 126, in import_module", " return _bootstrap._gcd_import(name[level:], package, level)", " File \"<frozen importlib._bootstrap>\", line 1050, in _gcd_import", " File \"<frozen importlib._bootstrap>\", line 1027, in _find_and_load", " File \"<frozen importlib._bootstrap>\", line 1006, in _find_and_load_unlocked", " File \"<frozen importlib._bootstrap>\", line 688, in _load_unlocked", " File \"<frozen importlib._bootstrap_external>\", line 883, in exec_module", " File \"<frozen importlib._bootstrap>\", line 241, in _call_with_frames_removed", " File \"/srv/funkwhale/virtualenv/lib/python3.10/site-packages/daphne/apps.py\", line 6, in <module>", " import daphne.server # noqa: F401", " File \"/srv/funkwhale/virtualenv/lib/python3.10/site-packages/daphne/server.py\", line 7, in <module>", " from twisted.internet import asyncioreactor # isort:skip", " File \"/srv/funkwhale/virtualenv/lib/python3.10/site-packages/twisted/internet/asyncioreactor.py\", line 19, in <module>", " from twisted.internet.posixbase import (", " File \"/srv/funkwhale/virtualenv/lib/python3.10/site-packages/twisted/internet/posixbase.py\", line 19, in <module>", " from twisted.internet import error, tcp, udp", " File \"/srv/funkwhale/virtualenv/lib/python3.10/site-packages/twisted/internet/tcp.py\", line 38, in <module>", " from twisted.internet._newtls import (", " File \"/srv/funkwhale/virtualenv/lib/python3.10/site-packages/twisted/internet/_newtls.py\", line 18, in <module>", " from twisted.protocols.tls import TLSMemoryBIOFactory, TLSMemoryBIOProtocol", " File \"/srv/funkwhale/virtualenv/lib/python3.10/site-packages/twisted/protocols/tls.py\", line 40, in <module>", " from OpenSSL.SSL import (", " File \"/srv/funkwhale/virtualenv/lib/python3.10/site-packages/OpenSSL/__init__.py\", line 8, in <module>", " from OpenSSL import crypto, SSL", " File \"/srv/funkwhale/virtualenv/lib/python3.10/site-packages/OpenSSL/crypto.py\", line 1556, in <module>", " class X509StoreFlags(object):", " File \"/srv/funkwhale/virtualenv/lib/python3.10/site-packages/OpenSSL/crypto.py\", line 1577, in X509StoreFlags", " CB_ISSUER_CHECK = _lib.X509_V_FLAG_CB_ISSUER_CHECK", "AttributeError: module 'lib' has no attribute 'X509_V_FLAG_CB_ISSUER_CHECK'"], "stdout": "", "stdout_lines": []}

The Frontend : https://sound.zocradio.info/ stays on "loading Funkwhale
but the API seems to still run fine (used here: https://www.zocradio.info/radio/radio.html )

Did I miss a component upgrade?

petitminion

Skro maybe try to collect the static file manually ?

Skro

petitminion The error message (as far as I understand it) says that it has been deprecated, so it's not available anymore, am I right?
(btw, I use the install script because I'm very bad at Unix command line 😃 )

Skro

Ran it manually (got a couple of other issues on the machine in the meantime), and it still fails
/srv/funkwhale# virtualenv/bin/funkwhale-manage collectstatic --no-input 2023-07-16 19:17:21,050 funkwhale_api.config INFO Loaded env file at /srv/funkwhale/config/.env 2023-07-16 19:17:21,051 funkwhale_api.config INFO Running with the following plugins enabled: funkwhale_api.contrib.scrobbler, funkwhale_api.contrib.listenbrainz, funkwhale_api.contrib.maloja Traceback (most recent call last): File "/srv/funkwhale/virtualenv/bin/funkwhale-manage", line 8, in <module> sys.exit(main()) File "/srv/funkwhale/api/funkwhale_api/main.py", line 30, in main execute_from_command_line(sys.argv) File "/srv/funkwhale/virtualenv/lib/python3.10/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line utility.execute() File "/srv/funkwhale/virtualenv/lib/python3.10/site-packages/django/core/management/__init__.py", line 395, in execute django.setup() File "/srv/funkwhale/virtualenv/lib/python3.10/site-packages/django/__init__.py", line 24, in setup apps.populate(settings.INSTALLED_APPS) File "/srv/funkwhale/virtualenv/lib/python3.10/site-packages/django/apps/registry.py", line 91, in populate app_config = AppConfig.create(entry) File "/srv/funkwhale/virtualenv/lib/python3.10/site-packages/django/apps/config.py", line 124, in create mod = import_module(mod_path) File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1050, in _gcd_import File "<frozen importlib._bootstrap>", line 1027, in _find_and_load File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 688, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 883, in exec_module File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed File "/srv/funkwhale/virtualenv/lib/python3.10/site-packages/daphne/apps.py", line 6, in <module> import daphne.server # noqa: F401 File "/srv/funkwhale/virtualenv/lib/python3.10/site-packages/daphne/server.py", line 7, in <module> from twisted.internet import asyncioreactor # isort:skip File "/srv/funkwhale/virtualenv/lib/python3.10/site-packages/twisted/internet/asyncioreactor.py", line 19, in <module> from twisted.internet.posixbase import ( File "/srv/funkwhale/virtualenv/lib/python3.10/site-packages/twisted/internet/posixbase.py", line 19, in <module> from twisted.internet import error, tcp, udp File "/srv/funkwhale/virtualenv/lib/python3.10/site-packages/twisted/internet/tcp.py", line 38, in <module> from twisted.internet._newtls import ( File "/srv/funkwhale/virtualenv/lib/python3.10/site-packages/twisted/internet/_newtls.py", line 18, in <module> from twisted.protocols.tls import TLSMemoryBIOFactory, TLSMemoryBIOProtocol File "/srv/funkwhale/virtualenv/lib/python3.10/site-packages/twisted/protocols/tls.py", line 40, in <module> from OpenSSL.SSL import ( File "/srv/funkwhale/virtualenv/lib/python3.10/site-packages/OpenSSL/__init__.py", line 8, in <module> from OpenSSL import crypto, SSL File "/srv/funkwhale/virtualenv/lib/python3.10/site-packages/OpenSSL/crypto.py", line 1556, in <module> class X509StoreFlags(object): File "/srv/funkwhale/virtualenv/lib/python3.10/site-packages/OpenSSL/crypto.py", line 1577, in X509StoreFlags CB_ISSUER_CHECK = _lib.X509_V_FLAG_CB_ISSUER_CHECK AttributeError: module 'lib' has no attribute 'X509_V_FLAG_CB_ISSUER_CHECK'
(I updated pip and pyOpenSSL to the latest version)

petitminion

maybe its a depedency error, you can update your packages in your python virtualenv following instructions here :https://docs.funkwhale.audio/administrator/upgrade/debian.html

gcrk

petitminion I don't think so, they said (very hidden at the end of the post 😉 ):

(I updated pip and pyOpenSSL to the latest version)

It looks like the same issue that was reported here: https://dev.funkwhale.audio/funkwhale/funkwhale/-/issues/2168

Skro

As a side note, I don't have a /venv directory, but a /virtualenv one
Do you need me to file another bug atop the issue?

gcrk

Skro No, thats expected for the quick install.

Skro

OK, I made the upgrade with all the manual commands, and managed to go through the whole upgrade process, so now I'm on the same spot that everyone else (the big "Loading Funkwhale" screen on the home page: https://sound.zocradio.info/ )
API works fine (My player: https://www.zocradio.info/radio/radio.html accesses the tracks without any issue)
I'm using Nginx as my reverse proxy (if that helps, as others are using different reverse proxies)

petitminion

Skro do you use apache or nginx ? did you update nginx ? Can you link the model of the nginx / apache template you use ?

Skro

I'm using nginx

# Ansible managed
server {
    listen 80;
    listen [::]:80;
    server_name sound.zocradio.info;
    location / { return 301 https://$host$request_uri; }
}
server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name sound.zocradio.info;
    ssl_certificate /etc/letsencrypt/live/sound.zocradio.info/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/sound.zocradio.info/privkey.pem;
    # from https://cipherli.st/
    ssl_prefer_server_ciphers on;
    ssl_ciphers EECDH+AESGCM:EDH+AESGCM;
    ssl_ecdh_curve secp384r1;
    ssl_session_timeout  10m;
    ssl_session_cache shared:SSL:10m;
    ssl_session_tickets off;
    ssl_stapling on;
    ssl_stapling_verify on;
    add_header Strict-Transport-Security "max-age=63072000;includeSubDomains; preload";
    # Security-related headers
    add_header Content-Security-Policy "default-src 'self'; script-src 'self'; style-src 'self' 'unsafe-inline'; img-src 'self' data:; font-src 'self' >
    add_header Access-Control-Allow-Origin *;
    root /srv/funkwhale/front/dist;

    # compression settings
    gzip on;
    gzip_comp_level    5;
    gzip_min_length    256;
    gzip_proxied       any;
    gzip_vary          on;

    gzip_types
        application/javascript
        application/vnd.geo+json
        application/vnd.ms-fontobject
        application/x-font-ttf
        application/x-web-app-manifest+json
        font/opentype
        image/bmp
        image/svg+xml
        image/x-icon
       text/cache-manifest
        text/css
        text/plain
        text/vcard
        text/vnd.rim.location.xloc
        text/vtt
        text/x-component
        text/x-cross-domain-policy;
    # end of compression settings

    location / {
        include /etc/nginx/funkwhale_proxy.conf;
        # this is needed if you have file import via upload enabled
        client_max_body_size 100M;
        proxy_pass   http://127.0.0.1:5000/;
    }

    location /front/ {
        alias /srv/funkwhale/front/dist/;
        expires 30d;
        add_header Pragma public;
        add_header Cache-Control "public, must-revalidate, proxy-revalidate";
        add_header Service-Worker-Allowed "/";
    }

    location /federation/ {
        include /etc/nginx/funkwhale_proxy.conf;
        proxy_pass   http://127.0.0.1:5000/federation/;
    }

    # You can comment this if you do not plan to use the Subsonic API
    location /rest/ {
        include /etc/nginx/funkwhale_proxy.conf;
        proxy_pass   http://127.0.0.1:5000/api/subsonic/rest/;
    }

    location /.well-known/ {
        include /etc/nginx/funkwhale_proxy.conf;
        proxy_pass   http://127.0.0.1:5000/.well-known/;
    }

    location /media/ {
        alias /srv/funkwhale/data/media/;
    }
    location /_protected/media {
        # this is an internal location that is used to serve
        # audio files once correct permission / authentication
        # has been checked on API side
        internal;
        alias   /srv/funkwhale/data/media;
    }

    location /_protected/music {
        # this is an internal location that is used to serve
        # audio files once correct permission / authentication
        # has been checked on API side
        # Set this to the same value as your MUSIC_DIRECTORY_PATH setting
        internal;
        alias   /srv/funkwhale/data/music;
    }

    location /staticfiles/ {
        # django static files
        alias /srv/funkwhale/data/static/;
    }

}

gcrk

Skro

    location / {
        include /etc/nginx/funkwhale_proxy.conf;
        # this is needed if you have file import via upload enabled
        client_max_body_size 100M;
        proxy_pass   http://127.0.0.1:5000/;
    }

Thats the problem here, this shouldn't be a proxy_pass

petitminion

gcrk this mean it ansible that has a wrong model, and don't know apache either but we should find someone to update the apache config

@Skro this what should be this nginx location :

    location / {
        alias ${FUNKWHALE_FRONTEND_PATH}/;
        expires 1d;
        try_files $uri $uri/ /index.html;
    }

${FUNKWHALE_FRONTEND_PATH} should be something like /srv/funkwhale/front/dist/
let us know if it resolve your issue 🙂

Skro

We are progressing, but it's still not ok 😃
I can access the main page with library and so on, but everything is empty, I don't even have my user name anymore...
And it broke the API access.

Reverting to my previous conf restores the API; but breaks the front end.

And, with the old conf, going to https://sound.zocradio.info/front/ brings me to the buggy front-end I have by typing https://sound.zocradio.info with the new conf

petitminion

this is probably because you don't have the following entry

    location /api/ {
        include /etc/nginx/funkwhale_proxy.conf;
        # This is needed if you have file import via upload enabled.
        client_max_body_size ${NGINX_MAX_BODY_SIZE};
        proxy_pass   http://funkwhale-api;
    }

plz have a look a : https://dev.funkwhale.audio/funkwhale/funkwhale/raw/develop/deploy/nginx.template

gcrk

@Skro I am already working on fixing the quick install, I guess its the best to wait for it. Updating the configuration is not trivial, which is also why fixing it takes some time for me, you'd need to compare your config with the template linked by @petitminion and fix all single differences.

Skro

Upgraded my Nginx config to the latest template.
The site is back online. (note for the other ones: clear your cookies to be able to log out from your front page, then log back in 😃 )
The API seems to fetch the data. However, the playback of the tracks still fails (I have to check this again from my side, as it looks like the issue I went into when testing my app locally. It somehow ran on the server, and I suspect it's linked to the refferer Policy² - my player and Funkwhale are on different servers)

²note: IIRC: add_header Referrer-Policy "strict-origin-when-cross-origin"; was the culprit.
If anyone has hints about how to bypass this, i'd be happy to hear about them, but it's a different subject 😃

Skro

After further investigation, it appears that my last issue was due to the fact that the URL of the track in the API is now sent by the track request as '/api/v1/listen/...' when it was 'api/v1/listen/...' in the previous version, on which my code was built.
I removed the '/' that I had added in my code back then and everything works fine now.

Thank you very much for your help!