[BUG] Cannot find the cmake module in a wheel build venv when the cmake executable is installed in another venv
#586
Comments
|
You should conditionally use cmake if it is not in your path and never put it into your However, I believe this is a bug, though I think this might be a bug with pip. |
|
I think this might be the same root issue as in scikit-build/scikit-build-core#955, which also looks like an issue with pip messing up the environment when calling a subprocess only when doing isolated installs. |
Yes. It should be an upstream bug for how |
|
I opened an issue on the |
|
This only affects scripts that use an entry point to wrap them. Some of those directly place the executable inside the scripts directory, which would work. CMake has helper files (the directory that contains CMake modules, like |
|
Did you check using |
FYI, |
|
A temporary workaround for pypa/pip#13222: - self.spawn([cmake, '-S', str(ext.source_dir), '-B', str(build_temp), *cmake_args])
- if not self.dry_run:
- self.spawn([cmake, '--build', str(build_temp), *build_args])
+ python_path = None
+ try:
+ # pip's build environment pseudo-isolation sets `PYTHONPATH` and may break console scripts
+ python_path = os.environ.pop('PYTHONPATH', None) # unset `PYTHONPATH`
+ self.spawn([cmake, '-S', str(ext.source_dir), '-B', str(build_temp), *cmake_args])
+ if not self.dry_run:
+ self.spawn([cmake, '--build', str(build_temp), *build_args])
+ finally:
+ if python_path is not None:
+ os.environ['PYTHONPATH'] = python_path |
|
Working around this might be a bit ugly, but I think it's possible. We'd have to replace the entry point with a custom script that we install, then in the custom script we could use the same workaround that is present in https://github.com/astral-sh/ruff/pull/13591/files. (I'm pretty sure this is breaking down inside the script pip generates, and not inside |
I think we will need to update the distutils' console template. The error arises during importing the |
|
Do you know what we'd need to add to the template? I have a replacement for it set up, but not sure how to safely check for and fix this case. #!python
import os
import sys
# Scripts can be broken by pip's build isolation hacks
# See: https://github.com/pypa/pip/blob/102d8187a1f5a4cd5de7a549fd8a9af34e89a54f/src/pip/_internal/build_env.py#L87
# ?
from cmake import ${program}
if __name__ == '__main__':
if sys.argv[0].endswith('-script.pyw'):
sys.argv[0] = sys.argv[0][: -11]
elif sys.argv[0].endswith('.exe'):
sys.argv[0] = sys.argv[0][: -4]
sys.exit(${program}()) |
I think the replacement should be: #!python
import os
import sys
try:
from cmake import ${program}
except ImportError:
python_path = os.environ.pop('PYTHONPATH', None)
from cmake import ${program}
if python_path is not None:
os.environ['PYTHONPATH'] = python_path
if __name__ == '__main__':
if sys.argv[0].endswith('-script.pyw'):
sys.argv[0] = sys.argv[0][: -11]
elif sys.argv[0].endswith('.exe'):
sys.argv[0] = sys.argv[0][: -4]
sys.exit(${program}())
Note that we should put execute_process(
COMMAND "${Python_EXECUTABLE}" -m pybind11 --cmakedir
OUTPUT_VARIABLE pybind11_DIR
) |
|
Changing the |
Good point. I forgot that.
That might not work. The
Maybe we can do it in a new subprocess: #!python
import os
import subprocess
import sys
try:
from cmake import ${program}
except ImportError:
old_sys_path = sys.path[:]
sys.path[:] = (
subprocess.check_output(
[sys.executable, '-c', 'import os, sys; print(os.pathsep.join(sys.path))'],
env={'PYTHONPATH': ''},
text=True,
)
.strip()
.split(os.pathsep)
)
from cmake import ${program}
sys.path[:] = old_sys_path
if __name__ == '__main__':
if sys.argv[0].endswith('-script.pyw'):
sys.argv[0] = sys.argv[0][: -11]
elif sys.argv[0].endswith('.exe'):
sys.argv[0] = sys.argv[0][: -4]
sys.exit(${program}()) |
Expected Behavior
cmakeshould work with an arbitrary runtime environment when calling thecmakeexecutable with an absolution path.Current Behavior
When a
venv(A) is installed with thecmakePyPI package, avenv/bin/cmakeexecutable will present inPATHif thevenv(A) is activated. While install a PyPI package from source, a buildvenv(B) will be created. In the buildvenv(B), runcmakeinstalled in thevenv(A) with absolute path will raise an error that cannot find thecmakemodule (installed invenv(A)).After adding some debug script to the
cmakeexecutable:#!${PROJECT}/venvA/bin/python3.13 # -*- coding: utf-8 -*- import re import sys + import site + from pprint import pformat + + with open('debug.txt', 'w') as f: + print(f'site-packages: {pformat(site.getsitepackages())}', file=f) + print(f'sys.path: {pformat(sys.path)}', file=f) + from cmake import cmake if __name__ == '__main__': sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0]) sys.exit(cmake())I got:
where the
site-packagesofvenv(A) is not present insys.path.Reproduce Script
Additional Context
Related:
optreetries to installcmakefrom PyPI unconditionally metaopt/optree#187The above issue and PR changed to use the
cmakeexecutable inPATHinstead of adding thecmakePyPI package tobuild-system.requiresinpyproject.toml.cc @mgorny
The text was updated successfully, but these errors were encountered: