Macro to smoke-test-import a Python module in %check

[Miro Hrončok]

Hello Python RPM packagers,

based on some discussion in the "F35 Change: Python Packaging Guidelines 
overhaul (System-Wide Change proposal)" thread [0], I've drafted a macro that 
can help to test-import a Python module in %check when no other tests exist or 
are when they cannot be executed during build [1].

The semantics is quite simple:

%check
     %py3_check_import mymodule mymodule.submodule

When all listed modules are successfully imported, "nothing happens", when at 
lest one fails to import, the entire build fails. The above line is translated 
very-roughly to `python3 -c 'import mymodule, mymodule.submodule'` (see the 
implementation [0] for more accurate representation). Given the Python 
semantics, it is possible to use commas as module separators as well (but no 
parentheses).

The %buildroot's %pythn3_site{lib,arch} is added to PYTHONPATH.

The runtime dependencies are obviously needed for this to work, so they need to 
be manually BuildRequired or even better, generated in %generate_buildrequires 
via `%pyproject_buildrequires -r` to use this macro.

The macro can be used repeatedly when importing multiple modules at once is 
undesired (e.g. when only one module can be imported from the same Python 
interpreter):

%check
     %py3_check_import mymodule.either
     %py3_check_import mymodule.or

Before I merge this and backport to all Fedoras and EPELs, I'd like to know:

- Do you have better ideas for the macro name?
  - Do you have better ideas for the macro semantics?

[0] 
https://lists.fedoraproject.org/archives/list/[e-mail]/...
[1] https://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/99

[Felix Schwarz]

Am 28.06.21 um 21:44 schrieb Miro Hrončok:
> The semantics is quite simple:
>
> %check
>      %py3_check_import mymodule mymodule.submodule

Looks great! Thank you.

Please let us know when we should start adding that to our Python packages. :-)

Felix

[Miro Hrončok]

On 29. 06. 21 8:28, Felix Schwarz wrote:
> Am 28.06.21 um 21:44 schrieb Miro Hrončok:
> > The semantics is quite simple:
> >
> > %check
> >      %py3_check_import mymodule mymodule.submodule
>
> Looks great! Thank you.
>
> Please let us know when we should start adding that to our Python packages. :-)

Feel free to test it out in rawhide.

If there is no more feedback, I'll initiate the backports next week.

[Miro Hrončok]

On 08. 07. 21 16:48, Miro Hrončok wrote:
> On 29. 06. 21 8:28, Felix Schwarz wrote:
> > Am 28.06.21 um 21:44 schrieb Miro Hrončok:
> > > The semantics is quite simple:
> > >
> > > %check
> > >      %py3_check_import mymodule mymodule.submodule
> >
> > Looks great! Thank you.
> >
> > Please let us know when we should start adding that to our Python packages. :-)
>
> Feel free to test it out in rawhide.
>
> If there is no more feedback, I'll initiate the backports next week.

Fedora 34: https://bodhi.fedoraproject.org/updates/FEDORA-2021-98d8c2b4fd

Fedora 33: https://bodhi.fedoraproject.org/updates/FEDORA-2021-74b84549a3

CentOS Stream 9: 
https://gitlab.com/redhat/centos-stream/rpms/python-rpm-macros/-/merge_reque...

EPEL 8: https://src.fedoraproject.org/rpms/epel-rpm-macros/pull-request/31

EPEL 7: https://src.fedoraproject.org/rpms/epel-rpm-macros/pull-request/32

[Michel Alexandre Salim]

On Wed, Jul 14, 2021 at 07:27:52PM +0200, Miro Hrončok wrote:
> On 08. 07. 21 16:48, Miro Hrončok wrote:
> > On 29. 06. 21 8:28, Felix Schwarz wrote:
> > > Am 28.06.21 um 21:44 schrieb Miro Hrončok:
> > > > The semantics is quite simple:
> > > >
> > > > %check
> > > >      %py3_check_import mymodule mymodule.submodule
> > >
> > > Looks great! Thank you.
> > >
> > > Please let us know when we should start adding that to our Python packages. :-)
> >
> > Feel free to test it out in rawhide.
> >
> > If there is no more feedback, I'll initiate the backports next week.

<snip>
> EPEL 8: https://src.fedoraproject.org/rpms/epel-rpm-macros/pull-request/31
>
> EPEL 7: https://src.fedoraproject.org/rpms/epel-rpm-macros/pull-request/32

Tested the EPEL updates:
EPEL 8 - https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2021-dfd462a782
EPEL 7 - https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2021-e3b1cc2b6e

with this `python-dialog` PR:
https://src.fedoraproject.org/rpms/python-dialog/pull-request/1

Works fine. If someone wants to help test, the needed `epel-rpm-macros`
is in buildroot override, and here's a good way to find EPEL specs that
do not have checks yet:

wget -cN https://src.fedoraproject.org/lookaside/rpm-specs-latest.tar.xz
   tar xf rpm-specs-latest.tar.xz
   cd rpm-specs
   grep -l epel $(grep -L '%check' python*.spec)

currently resulting in:

python-bitarray.spec
python-dialog.spec
python-docker.spec
python-epel-rpm-macros.spec
python-fedora.spec
python-flask-login.spec
python-graphitesend.spec
python-pretend.spec
python-progress.spec
python-pvc.spec
python-pysb.spec
python-pyvirtualize.spec
python-qt5.spec
python-requests-oauthlib.spec
python-sphinx-bootstrap-theme.spec
python-sphinx_lv2_theme.spec
python-stuf.spec
python-unidecode.spec
python-urlobject.spec
python-vevents.spec
python-vpoller.spec
python-wrapt.spec
python-wtforms.spec
python-xcffib.spec

Best regards,

[Richard Shaw]

Trying this with python-pyside2...

+ /usr/bin/python3 -c 'import PySide2'
PySide2/__init__.py: Unable to import shiboken2 from ,
/builddir/build/BUILDROOT/python-pyside2-5.15.2-4.fc35.x86_64/usr/lib64/python3.10/site-packages,
/builddir/build/BUILDROOT/python-pyside2-5.15.2-4.fc35.x86_64/usr/lib/python3.10/site-packages,
/usr/lib64/python310.zip, /usr/lib64/python3.10,
/usr/lib64/python3.10/lib-dynload, /usr/lib64/python3.10/site-packages,
/usr/lib/python3.10/site-packages
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File
"/builddir/build/BUILDROOT/python-pyside2-5.15.2-4.fc35.x86_64/usr/lib64/python3.10/site-packages/PySide2/__init__.py",
line 107, in <module>
    _setupQtDirectories()
  File
"/builddir/build/BUILDROOT/python-pyside2-5.15.2-4.fc35.x86_64/usr/lib64/python3.10/site-packages/PySide2/__init__.py",
line 58, in _setupQtDirectories
    import shiboken2
  File
"/builddir/build/BUILDROOT/python-pyside2-5.15.2-4.fc35.x86_64/usr/lib64/python3.10/site-packages/shiboken2/__init__.py",
line 27, in <module>
    from .shiboken2 import *
ImportError: libshiboken2.cpython-310-x86_64-linux-gnu.so.5.15: cannot open
shared object file: No such file or directory

Worked fine once installed. The 5.15 file is a symlink to 5.15.2 which is
the actual library file name, but I don't see why it would follow symlinks
once installed but not here?

Here's a scratch build I just kicked off if that helps:

https://koji.fedoraproject.org/koji/taskinfo?taskID=71716212

Thanks,
Richard

[Miro Hrončok]

On 11. 07. 21 21:05, Richard Shaw wrote:
> Trying this with python-pyside2...
> /usr/bin/python3 -c 'import PySide2'
>
> PySide2/__init__.py: Unable to import shiboken2 from , 
> /builddir/build/BUILDROOT/python-pyside2-5.15.2-4.fc35.x86_64/usr/lib64/python3.10/site-packages, 
> /builddir/build/BUILDROOT/python-pyside2-5.15.2-4.fc35.x86_64/usr/lib/python3.10/site-packages, 
> /usr/lib64/python310.zip, /usr/lib64/python3.10, 
> /usr/lib64/python3.10/lib-dynload, /usr/lib64/python3.10/site-packages, 
> /usr/lib/python3.10/site-packages
> Traceback (most recent call last):
>    File "<string>", line 1, in <module>
>    File 
> "/builddir/build/BUILDROOT/python-pyside2-5.15.2-4.fc35.x86_64/usr/lib64/python3.10/site-packages/PySide2/__init__.py", 
> line 107, in <module>
>      _setupQtDirectories()
>    File 
> "/builddir/build/BUILDROOT/python-pyside2-5.15.2-4.fc35.x86_64/usr/lib64/python3.10/site-packages/PySide2/__init__.py", 
> line 58, in _setupQtDirectories
>      import shiboken2
>    File 
> "/builddir/build/BUILDROOT/python-pyside2-5.15.2-4.fc35.x86_64/usr/lib64/python3.10/site-packages/shiboken2/__init__.py", 
> line 27, in <module>
>      from .shiboken2 import *
> ImportError: libshiboken2.cpython-310-x86_64-linux-gnu.so.5.15: cannot open 
> shared object file: No such file or directory
>
> Worked fine once installed. The 5.15 file is a symlink to 5.15.2 which is the 
> actual library file name, but I don't see why it would follow symlinks once 
> installed but not here?

Is it possible that the symbolic link is absolute and hence not resolvable 
during the build (because it does not lead to the buildroot)?

Not sure how the macro could compensate for that.