Sadly your pip install --user --upgrade pip is unlikely to have any impact at all to the default pip (you could verify by running pip --version and seeing that it is still the old version). It is believable that running ~/.local/lib/python3.?/site-packages/bin/pip instead might work, but it isn’t a given due to the way Python has been configured to pickup the various site-package directories.
If you really want to be able to influence your environment properly, I’d personally recommend using virtual environments. I posted about using them reasonably effectively (though at the cost of having some ugly setup code at the beginning of each notebook) in Avoiding the use of ``--user`` and ``.local`` for pip installations. Given the sluggish home filesystem though it can take a while to first install a sizeable virtual environment (after that it is much quicker and cached in your $HOME/python/environments directory).
Note: I’m not a SWAN maintainer and there may be simpler approaches to solving this problem which don’t involve the above suggestion (I didn’t look too hard).
Cheers!