2.18.2: pytest fails in tests/cli/test_build.py::test_build_with_no_isolation unit

[kloczek]

• I have searched the issue tracker and believe that this is not a duplicate.

Steps to reproduce

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

• python3 -sBm build -w --no-isolation
• because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
• install .whl file in </install/prefix> using installer module
• run pytest with $PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>
• build is performed in env which is cut off from access to the public network (pytest is executed with -m "not network")

Actual behavior

Here is pytest output:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-pdm-2.18.2-2.fc37.x86_64/usr/lib64/python3.10/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-pdm-2.18.2-2.fc37.x86_64/usr/lib/python3.10/site-packages
+ /usr/bin/pytest -ra -m 'not network' --deselect tests/test_project.py::test_access_index_with_auth --deselect tests/test_formats.py::test_convert_setup_py_project --deselect 'tests/test_project.py::test_create_venv_in_project[venv-True]'
============================= test session starts ==============================
platform linux -- Python 3.10.14, pytest-8.2.2, pluggy-1.5.0
rootdir: /home/tkloczko/rpmbuild/BUILD/pdm-2.18.2
configfile: pyproject.toml
testpaths: tests/
plugins: anyio-4.4.0, mock-3.14.0, pytest_httpserver-1.1.0, flaky-3.8.1
collected 947 items / 7 deselected / 940 selected

tests/cli/test_add.py .................................                  [  3%]
tests/cli/test_build.py ..........F.                                     [  4%]
tests/cli/test_cache.py ............                                     [  6%]
tests/cli/test_config.py ..................                              [  7%]
tests/cli/test_fix.py .....                                              [  8%]
tests/cli/test_hooks.py ................................................ [ 13%]
...........................................                              [ 18%]
tests/cli/test_init.py ..........                                        [ 19%]
tests/cli/test_install.py .............................................. [ 24%]
.........                                                                [ 25%]
tests/cli/test_list.py ...............................                   [ 28%]
tests/cli/test_lock.py ..............................................    [ 33%]
tests/cli/test_others.py ......................                          [ 35%]
tests/cli/test_outdated.py .                                             [ 35%]
tests/cli/test_publish.py ............                                   [ 37%]
tests/cli/test_python.py ........                                        [ 37%]
tests/cli/test_remove.py .............                                   [ 39%]
tests/cli/test_run.py .................................................. [ 44%]
..............                                                           [ 46%]
tests/cli/test_self_command.py ........                                  [ 46%]
tests/cli/test_template.py ....                                          [ 47%]
tests/cli/test_update.py .....................                           [ 49%]
tests/cli/test_use.py .......                                            [ 50%]
tests/cli/test_utils.py .                                                [ 50%]
tests/cli/test_venv.py ...............................                   [ 53%]
tests/models/test_backends.py ...........                                [ 54%]
tests/models/test_candidates.py .................................        [ 58%]
tests/models/test_marker.py ...................                          [ 60%]
tests/models/test_requirements.py ....................                   [ 62%]
tests/models/test_setup_parsing.py ..........                            [ 63%]
tests/models/test_specifiers.py ........................................ [ 67%]
..                                                                       [ 68%]
tests/models/test_versions.py ...................                        [ 70%]
tests/resolver/test_resolve.py ....................................      [ 73%]
tests/test_formats.py ................                                   [ 75%]
tests/test_installer.py ..................................               [ 79%]
tests/test_integration.py .                                              [ 79%]
tests/test_plugin.py .....                                               [ 79%]
tests/test_project.py .................................................. [ 85%]
..........                                                               [ 86%]
tests/test_signals.py ..                                                 [ 86%]
tests/test_utils.py ...............................s.................... [ 92%]
........................................................................ [ 99%]
...                                                                      [100%]

=================================== FAILURES ===================================
_________________________ test_build_with_no_isolation _________________________

pdm = <function pdm.<locals>.caller at 0x7ffb4fc857e0>
project = <Project '/tmp/pytest-of-tkloczko/pytest-410/test_build_with_no_isolation0'>

    @pytest.mark.usefixtures("local_finder")
    def test_build_with_no_isolation(pdm, project):
        result = pdm(["build", "--no-isolation"], obj=project)
>       assert result.exit_code == 1
E       AssertionError: assert 0 == 1
E        +  where 0 = RunResult(exit_code=0, stdout='Building sdist...\nBuilt sdist at /tmp/pytest-of-tkloczko/pytest-410/test_build_with_no...loczko/pytest-410/test_build_with_no_isolation0/dist/test_project-0.0.0-py3-none-any.whl\n', stderr='', exception=None).exit_code

tests/cli/test_build.py:164: AssertionError
---------------------------- Captured stdout setup -----------------------------
Changes are written to pyproject.toml.
Changes are written to pyproject.toml.
------------------------------ Captured log call -------------------------------
INFO     pdm.termui:base.py:209 Preparing environment(Non-isolated mode) for PEP 517 build...
INFO     pdm.termui:base.py:71 /usr/lib/python3.10/site-packages/pdm/backend/sdist.py:56: PDMWarning: No license files are matched with glob patterns ['LICENSES/*', 'LICEN[CS]E*', 'COPYING*', 'NOTICE*', 'AUTHORS*'].
INFO     pdm.termui:base.py:71   *self.find_license_files(),
INFO     pdm.termui:base.py:71  - Adding .pdm-build/pyproject.toml -> pyproject.toml
INFO     pdm.termui:base.py:71  - Adding PKG-INFO -> PKG-INFO
INFO     pdm.termui:base.py:209 Preparing environment(Non-isolated mode) for PEP 517 build...
INFO     pdm.termui:base.py:71 /usr/lib/python3.10/site-packages/pdm/backend/wheel.py:243: PDMWarning: No license files are matched with glob patterns ['LICENSES/*', 'LICEN[CS]E*', 'COPYING*', 'NOTICE*', 'AUTHORS*'].
INFO     pdm.termui:base.py:71   for file in self.find_license_files():
INFO     pdm.termui:base.py:71  - Adding .pdm-build/test_project-0.0.0.dist-info/METADATA -> test_project-0.0.0.dist-info/METADATA
INFO     pdm.termui:base.py:71  - Adding .pdm-build/test_project-0.0.0.dist-info/WHEEL -> test_project-0.0.0.dist-info/WHEEL
INFO     pdm.termui:base.py:71  - Adding test_project-0.0.0.dist-info/RECORD -> test_project-0.0.0.dist-info/RECORD
=========================== short test summary info ============================
SKIPPED [1] tests/test_utils.py:171: Windows test
FAILED tests/cli/test_build.py::test_build_with_no_isolation - AssertionError: assert 0 == 1
 +  where 0 = RunResult(exit_code=0, stdout='Building sdist...\nBuilt sdist at /tmp/pytest-of-tkloczko/pytest-410/test_build_with_no...loczko/pytest-410/test_build_with_no_isolation0/dist/test_project-0.0.0-py3-none-any.whl\n', stderr='', exception=None).exit_code
====== 1 failed, 938 passed, 1 skipped, 7 deselected in 161.10s (0:02:41) ======

Expected behavior

pytest should not fail.

Environment Information

WARNING: Project requires a python version of >=3.8, The virtualenv is being created for you as it cannot be matched to the right version.
INFO: python.use_venv is on, creating a virtualenv for this project...
Virtualenv is created successfully at /home/tkloczko/rpmbuild/BUILD/pdm-2.18.2/.venv
PDM version:
  2.16.1
Python Interpreter:
  /home/tkloczko/rpmbuild/BUILD/pdm-2.18.2/.venv/bin/python (3.10)
Project Root:
  /home/tkloczko/rpmbuild/BUILD/pdm-2.18.2
Local Packages:

INFO: PDM 2.16.1 is installed, while 2.18.2 is available.
Please run `pdm self update` to upgrade.
Run `pdm config check_update false` to disable the check.
{
  "implementation_name": "cpython",
  "implementation_version": "3.10.14",
  "os_name": "posix",
  "platform_machine": "x86_64",
  "platform_release": "6.8.4-200.fc39.x86_64",
  "platform_system": "Linux",
  "platform_version": "#1 SMP PREEMPT_DYNAMIC Thu Apr  4 20:45:21 UTC 2024",
  "python_full_version": "3.10.14",
  "platform_python_implementation": "CPython",
  "python_version": "3.10",
  "sys_platform": "linux"
}
INFO: PDM 2.16.1 is installed, while 2.18.2 is available.
Please run `pdm self update` to upgrade.
Run `pdm config check_update false` to disable the check.

[kloczek]

On top of that unit sill fails that unit which I've reported in #2015
pytest should not execute pip to install or upgrade anything in test suite because pdm it is not python modules management module but pep517 backend + pytest ext.

Here is pytest output with failing those two units:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-pdm-2.18.2-2.fc37.x86_64/usr/lib64/python3.10/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-pdm-2.18.2-2.fc37.x86_64/usr/lib/python3.10/site-packages
+ /usr/bin/pytest -ra -m 'not network'
==================================================================================== test session starts ====================================================================================
platform linux -- Python 3.10.14, pytest-8.2.2, pluggy-1.5.0
rootdir: /home/tkloczko/rpmbuild/BUILD/pdm-2.18.2
configfile: pyproject.toml
testpaths: tests/
plugins: anyio-4.4.0, mock-3.14.0, pytest_httpserver-1.1.0, flaky-3.8.1
collected 947 items / 4 deselected / 943 selected

tests/cli/test_add.py .................................                                                                                                                               [  3%]
tests/cli/test_build.py ..........F.                                                                                                                                                  [  4%]
tests/cli/test_cache.py ............                                                                                                                                                  [  6%]
tests/cli/test_config.py ..................                                                                                                                                           [  7%]
tests/cli/test_fix.py .....                                                                                                                                                           [  8%]
tests/cli/test_hooks.py ...........................................................................................                                                                   [ 18%]
tests/cli/test_init.py ..........                                                                                                                                                     [ 19%]
tests/cli/test_install.py .......................................................                                                                                                     [ 25%]
tests/cli/test_list.py ...............................                                                                                                                                [ 28%]
tests/cli/test_lock.py ..............................................                                                                                                                 [ 33%]
tests/cli/test_others.py ......................                                                                                                                                       [ 35%]
tests/cli/test_outdated.py .                                                                                                                                                          [ 35%]
tests/cli/test_publish.py ............                                                                                                                                                [ 36%]
tests/cli/test_python.py ........                                                                                                                                                     [ 37%]
tests/cli/test_remove.py .............                                                                                                                                                [ 39%]
tests/cli/test_run.py ................................................................                                                                                                [ 45%]
tests/cli/test_self_command.py ........                                                                                                                                               [ 46%]
tests/cli/test_template.py ....                                                                                                                                                       [ 47%]
tests/cli/test_update.py .....................                                                                                                                                        [ 49%]
tests/cli/test_use.py .......                                                                                                                                                         [ 50%]
tests/cli/test_utils.py .                                                                                                                                                             [ 50%]
tests/cli/test_venv.py ...............................                                                                                                                                [ 53%]
tests/models/test_backends.py ...........                                                                                                                                             [ 54%]
tests/models/test_candidates.py .................................                                                                                                                     [ 58%]
tests/models/test_marker.py ...................                                                                                                                                       [ 60%]
tests/models/test_requirements.py ....................                                                                                                                                [ 62%]
tests/models/test_setup_parsing.py ..........                                                                                                                                         [ 63%]
tests/models/test_specifiers.py ..........................................                                                                                                            [ 67%]
tests/models/test_versions.py ...................                                                                                                                                     [ 69%]
tests/resolver/test_resolve.py ....................................                                                                                                                   [ 73%]
tests/test_formats.py ...............F.                                                                                                                                               [ 75%]
tests/test_installer.py ..................................                                                                                                                            [ 79%]
tests/test_integration.py .                                                                                                                                                           [ 79%]
tests/test_plugin.py .....                                                                                                                                                            [ 79%]
tests/test_project.py ..............................................................                                                                                                  [ 86%]
tests/test_signals.py ..                                                                                                                                                              [ 86%]
tests/test_utils.py ...............................s...............................................................................................                                   [100%]

========================================================================================= FAILURES ==========================================================================================
_______________________________________________________________________________ test_build_with_no_isolation ________________________________________________________________________________

pdm = <function pdm.<locals>.caller at 0x7ff7051bb1c0>, project = <Project '/tmp/pytest-of-tkloczko/pytest-2/test_build_with_no_isolation0'>

    @pytest.mark.usefixtures("local_finder")
    def test_build_with_no_isolation(pdm, project):
        result = pdm(["build", "--no-isolation"], obj=project)
>       assert result.exit_code == 1
E       AssertionError: assert 0 == 1
E        +  where 0 = RunResult(exit_code=0, stdout='Building sdist...\nBuilt sdist at /tmp/pytest-of-tkloczko/pytest-2/test_build_with_no_i...tkloczko/pytest-2/test_build_with_no_isolation0/dist/test_project-0.0.0-py3-none-any.whl\n', stderr='', exception=None).exit_code

tests/cli/test_build.py:164: AssertionError
----------------------------------------------------------------------------------- Captured stdout setup -----------------------------------------------------------------------------------
Changes are written to pyproject.toml.
Changes are written to pyproject.toml.
------------------------------------------------------------------------------------- Captured log call -------------------------------------------------------------------------------------
INFO     pdm.termui:base.py:209 Preparing environment(Non-isolated mode) for PEP 517 build...
INFO     pdm.termui:base.py:71 /usr/lib/python3.10/site-packages/pdm/backend/sdist.py:56: PDMWarning: No license files are matched with glob patterns ['LICENSES/*', 'LICEN[CS]E*', 'COPYING*', 'NOTICE*', 'AUTHORS*'].
INFO     pdm.termui:base.py:71   *self.find_license_files(),
INFO     pdm.termui:base.py:71  - Adding .pdm-build/pyproject.toml -> pyproject.toml
INFO     pdm.termui:base.py:71  - Adding PKG-INFO -> PKG-INFO
INFO     pdm.termui:base.py:209 Preparing environment(Non-isolated mode) for PEP 517 build...
INFO     pdm.termui:base.py:71 /usr/lib/python3.10/site-packages/pdm/backend/wheel.py:243: PDMWarning: No license files are matched with glob patterns ['LICENSES/*', 'LICEN[CS]E*', 'COPYING*', 'NOTICE*', 'AUTHORS*'].
INFO     pdm.termui:base.py:71   for file in self.find_license_files():
INFO     pdm.termui:base.py:71  - Adding .pdm-build/test_project-0.0.0.dist-info/METADATA -> test_project-0.0.0.dist-info/METADATA
INFO     pdm.termui:base.py:71  - Adding .pdm-build/test_project-0.0.0.dist-info/WHEEL -> test_project-0.0.0.dist-info/WHEEL
INFO     pdm.termui:base.py:71  - Adding test_project-0.0.0.dist-info/RECORD -> test_project-0.0.0.dist-info/RECORD
_______________________________________________________________________________ test_convert_setup_py_project _______________________________________________________________________________

project = <Project '/tmp/pytest-of-tkloczko/pytest-2/test_convert_setup_py_project0'>

    def test_convert_setup_py_project(project):
        golden_file = FIXTURES / "projects/test-setuptools/setup.py"
        assert setup_py.check_fingerprint(project, golden_file)
>       result, settings = setup_py.convert(project, golden_file, ns())

tests/test_formats.py:222:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../BUILDROOT/python-pdm-2.18.2-2.fc37.x86_64/usr/lib/python3.10/site-packages/pdm/formats/setup_py.py:20: in convert
    parsed = parse_setup_py(str(project.environment.interpreter.executable), os.path.dirname(filename))
../../BUILDROOT/python-pdm-2.18.2-2.fc37.x86_64/usr/lib/python3.10/site-packages/pdm/models/in_process/__init__.py:42: in parse_setup_py
    subprocess.check_call(cmd)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

popenargs = (['/usr/bin/python', '-Es', '/home/tkloczko/rpmbuild/BUILDROOT/python-pdm-2.18.2-2.fc37.x86_64/usr/lib/python3.10/site...tup.py', '/home/tkloczko/rpmbuild/BUILD/pdm-2.18.2/tests/fixtures/projects/test-setuptools', '/tmp/tmpyus29adz.json'],)
kwargs = {}, retcode = 1
cmd = ['/usr/bin/python', '-Es', '/home/tkloczko/rpmbuild/BUILDROOT/python-pdm-2.18.2-2.fc37.x86_64/usr/lib/python3.10/site-...setup.py', '/home/tkloczko/rpmbuild/BUILD/pdm-2.18.2/tests/fixtures/projects/test-setuptools', '/tmp/tmpyus29adz.json']

    def check_call(*popenargs, **kwargs):
        """Run command with arguments.  Wait for command to complete.  If
        the exit code was zero then return, otherwise raise
        CalledProcessError.  The CalledProcessError object will have the
        return code in the returncode attribute.

        The arguments are the same as for the call function.  Example:

        check_call(["ls", "-l"])
        """
        retcode = call(*popenargs, **kwargs)
        if retcode:
            cmd = kwargs.get("args")
            if cmd is None:
                cmd = popenargs[0]
>           raise CalledProcessError(retcode, cmd)
E           subprocess.CalledProcessError: Command '['/usr/bin/python', '-Es', '/home/tkloczko/rpmbuild/BUILDROOT/python-pdm-2.18.2-2.fc37.x86_64/usr/lib/python3.10/site-packages/pdm/models/in_process/parse_setup.py', '/home/tkloczko/rpmbuild/BUILD/pdm-2.18.2/tests/fixtures/projects/test-setuptools', '/tmp/tmpyus29adz.json']' returned non-zero exit status 1.

/usr/lib64/python3.10/subprocess.py:369: CalledProcessError
----------------------------------------------------------------------------------- Captured stdout setup -----------------------------------------------------------------------------------
Changes are written to pyproject.toml.
----------------------------------------------------------------------------------- Captured stderr call ------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tkloczko/rpmbuild/BUILDROOT/python-pdm-2.18.2-2.fc37.x86_64/usr/lib/python3.10/site-packages/pdm/models/in_process/parse_setup.py", line 219, in <module>
    json.dump(parse_setup(sys.argv[1]), f, default=json_default)
  File "/home/tkloczko/rpmbuild/BUILDROOT/python-pdm-2.18.2-2.fc37.x86_64/usr/lib/python3.10/site-packages/pdm/models/in_process/parse_setup.py", line 181, in parse_setup
    raise RuntimeError(
RuntimeError: setuptools is required to convert setup.py, install it by `pdm add setuptools`
================================================================================== short test summary info ==================================================================================
SKIPPED [1] tests/test_utils.py:171: Windows test
FAILED tests/cli/test_build.py::test_build_with_no_isolation - AssertionError: assert 0 == 1
 +  where 0 = RunResult(exit_code=0, stdout='Building sdist...\nBuilt sdist at /tmp/pytest-of-tkloczko/pytest-2/test_build_with_no_i...tkloczko/pytest-2/test_build_with_no_isolation0/dist/test_project-0.0.0-py3-none-any.whl\n', stderr='', exception=None).exit_code
FAILED tests/test_formats.py::test_convert_setup_py_project - subprocess.CalledProcessError: Command '['/usr/bin/python', '-Es', '/home/tkloczko/rpmbuild/BUILDROOT/python-pdm-2.18.2-2.fc37.x86_64/usr/lib/python3.10/site-packages/pdm/models/in_process/parse_setup.py', '/home/tkloczko/rpmbuild/BUILD/pdm-2.18.2/tests/fixtures/projects/test-setuptools', '/tmp/tmpyus29adz.json']' returned non-zero exit status 1.
============================================================ 2 failed, 940 passed, 1 skipped, 4 deselected in 165.49s (0:02:45) =============================================================

[frostming]

RuntimeError: setuptools is required to convert setup.py, install it by `pdm add setuptools`

setuptools is required for that case, you need to install it into /usr/bin/python environment

[frostming]

    @pytest.mark.usefixtures("local_finder")
    def test_build_with_no_isolation(pdm, project):
        result = pdm(["build", "--no-isolation"], obj=project)
>       assert result.exit_code == 1
E       AssertionError: assert 0 == 1

While this is an error handling case to verify it should fail when build dependencies are not installed, but your test setup break this assumption, you may have to exclude this test yourself.