| |
@@ -22,10 +22,11 @@
|
| |
This has already been https://fedoraproject.org/wiki/Changes/Making_sudo_pip_safe[happening since Fedora Linux 27]. However the way this is achieved has been significantly re-implemented in Fedora Linux 36 and that has created several minor differences.
|
| |
|
| |
The `sysconfig` Python module from the standard library defines several *installation schemes*.
|
| |
- By default, the installation scheme used on Fedora 36 when installing Python packages using root privileges (for example via `sudo`) is `{prefix}/local/lib(64)/python3.10/site-packages/` (where `{prefix}` is defined as `/usr` by default).
|
| |
- When Python itself runs from a Python virtual environment or when building RPM packages, the installation scheme is `{prefix}/lib(64)/python3.10/site-packages/` (it does not include `/local/`).
|
| |
+ By default, the installation scheme used on Fedora 36 when installing Python packages using root privileges (for example via `sudo`) is `{prefix}/lib(64)/python3.10/site-packages/`, where `{prefix}` is defined as `/usr/local` by default.
|
| |
+ When building RPM packages, `{prefix}` is defined as `/usr` by default.
|
| |
+ When Python itself runs from a Python virtual environment, `{prefix}` is defined as the path for the virtual environment.
|
| |
|
| |
- Previously, the `/local/` part was only artificially added when installing packages, now it is part of the installation scheme.
|
| |
+ Previously, the `/local/` part was only artificially added when installing packages via `distutils`, now it is also defined in `sysconfig`.
|
| |
This was changed to be more consistent with what other Python distributors are doing, so that the scheme is more likely to be accepted in upstream Python and to work well with upstream tools like setuptools or pip, which we cannot modify when they are installed or upgraded using pip directly from the https://pypi.org/[Python Package Index].
|
| |
|
| |
Here are the differences that might be observed with the new approach:
|
| |
@@ -33,7 +34,7 @@
|
| |
|
| |
=== `sysconfig.get_path(key)` returns paths with `/local/`
|
| |
|
| |
- Previously, on Fedora Linux 35:
|
| |
+ Previously, on Fedora Linux 35 outside of a virtual environment:
|
| |
|
| |
[source,python]
|
| |
----
|
| |
@@ -77,12 +78,12 @@
|
| |
- https://github.com/rhinstaller/anaconda/pull/3646[anaconda/dracut: Don't assume Python modules are in sysconfig.get_path('purelib')]
|
| |
- https://github.com/rpm-software-management/dnf/pull/1782[dnf: Determine the default plugin path at configure time, rather than runtime]
|
| |
|
| |
- If you need to restore the previous behavior of `sysconfig.get_path(key)`, you may explicitly select the `rpm_prefix` installation scheme:
|
| |
+ If you need to restore the previous behavior of `sysconfig.get_path(key)`, you may explicitly set the `{base}` and `{platbase}` variables to `/usr`:
|
| |
|
| |
[source,python]
|
| |
----
|
| |
>>> for key in sysconfig.get_path_names():
|
| |
- ... print(f'{key} = {sysconfig.get_path(key, scheme="rpm_prefix")}')
|
| |
+ ... print(f'{key} = {sysconfig.get_path(key, vars={"base": "/usr", "platbase": "/usr"})}')
|
| |
...
|
| |
stdlib = /usr/lib64/python3.10
|
| |
platstdlib = /usr/lib64/python3.10
|
| |
@@ -93,61 +94,14 @@
|
| |
data = /usr
|
| |
----
|
| |
|
| |
- However this installation scheme is entirely Fedora Linux 36+ specific and such code will not work on other operating systems (or even older Fedora releases).
|
| |
-
|
| |
-
|
| |
- === pip/setup.py installation with `--prefix`
|
| |
-
|
| |
- When pip or `python setup.py` installation is invoked with the `--prefix` option, the `/usr` part of the standard installation path is replaced with the given `--prefix` value. Note that **`/local/` is not a part of the prefix** but a part of the installation scheme. Hence, https://bugzilla.redhat.com/show_bug.cgi?id=2026979[despite some quite reasonable expectations], the following invocation:
|
| |
-
|
| |
- ----
|
| |
- $ sudo pip install --prefix /usr Pello
|
| |
- ----
|
| |
-
|
| |
- Will *still* install the `Pello` package to `/usr/local/lib/python3.10/site-packages/`.
|
| |
-
|
| |
- And this:
|
| |
-
|
| |
- ----
|
| |
- $ sudo pip install --prefix /usr/local Pello
|
| |
- ----
|
| |
-
|
| |
- Will even install the `Pello` package to `/usr/local/local/lib/python3.10/site-packages/`.
|
| |
-
|
| |
- The only supported way to explicitly install a Python package directly to `/usr/lib(64)/python3.10/site-packages/` is to build an RPM package with it and install it.
|
| |
- Python checks the `$RPM_BUILD_ROOT` environment variable and selects the `rpm_prefix` installation scheme when it is set.
|
| |
-
|
| |
- Strictly for testing purposes, it is possible to set the variable outside of RPM build environment to simulate installation from an RPM package.
|
| |
- **Beware, this usage might have unexpected consequences** on a production system, including an entirely unrecoverable breakage.
|
| |
-
|
| |
- ----
|
| |
- $ sudo env RPM_BUILD_ROOT=/ pip install Pello
|
| |
- ----
|
| |
-
|
| |
- ==== Installation with `--prefix` and `--root`
|
| |
-
|
| |
- The change in behavior also applies when a custom `--root` value is passed together with `--prefix`.
|
| |
- The following command will install `Pello` to `~/myroot/usr/local/lib/python3.10/site-packages/`:
|
| |
-
|
| |
- ----
|
| |
- $ pip install --prefix /usr --root ~/myroot Pello
|
| |
- ----
|
| |
-
|
| |
- To install it to `~/myroot/usr/lib/python3.10/site-packages/`, the simulated RPM environment can be used:
|
| |
-
|
| |
- ----
|
| |
- $ RPM_BUID_ROOT=~/myroot pip install --prefix /usr --root ~/myroot Pello
|
| |
- ----
|
| |
-
|
| |
-
|
| |
=== RPM build–related caveats
|
| |
|
| |
- When Python runs during RPM build, it selects the `rpm_prefix` installation scheme.
|
| |
+ When Python runs during RPM build, it sets the defaults for `{base}` and `{platbase}` variables to `/usr`.
|
| |
This behavior is triggered when the `$RPM_BUILD_ROOT` environment variable is set. That has several caveats:
|
| |
|
| |
==== Executing Python in Python's subprocess
|
| |
|
| |
- If the Python code that runs in RPM build (for example in `%check`) executes another Python instance via a subprocess, it is relatively easy to inadvertently unset all environment variables. When this happens, the *inner* Python will not know it runs within RPM build and will return paths with the `/local/` infix.
|
| |
+ If the Python code that runs in RPM build (for example in `%check`) executes another Python instance via a subprocess, it is relatively easy to inadvertently unset all environment variables. When this happens, the *inner* Python will not know it runs within RPM build and will return paths with the `/usr/local/` prefix.
|
| |
|
| |
In the most trivial example, the surrounding environment variables are implicitly passed to subprocess and everything works as expected:
|
| |
|
| |
@@ -215,6 +169,7 @@
|
| |
----
|
| |
|
| |
The `venv` scheme is currently Fedora specific, but other Python distributors (such as https://launchpad.net/~deadsnakes/+archive/ubuntu/ppa[Ubuntu Deadsnakes]) may define it as well.
|
| |
+ The `venv` scheme is generally available starting with Python 3.11.
|
| |
Due to checking if the `venv` install scheme exists the code is functional on other operating systems as well, as it falls back to a backwards compatible behavior.
|
| |
|
| |
Example fixes from affected open source projects:
|
| |
churchyard
commented 2 years ago
An update is coming to Fedora 36 that removes most of the Python installation
scheme caveats documented in the release notes.
See https://src.fedoraproject.org/rpms/python3.10/c/38f9dd15905
and https://bugzilla.redhat.com/show_bug.cgi?id=2026979
and https://bugzilla.redhat.com/show_bug.cgi?id=2097183