----- Original Message -----
From: "Neal Gompa" ngompa13@gmail.com To: "Fedora Python SIG" python-devel@lists.fedoraproject.org Cc: "David Gray" dagray@redhat.com Sent: Saturday, October 26, 2019 5:01:52 PM Subject: Re: Building the python executable statically with libpython*.a for better performance
On Sat, Sep 21, 2019 at 5:16 PM Miro Hrončok mhroncok@redhat.com wrote:
Hello,
we've been recently approached by a colleague from Red Hat working on performance (CCed).
According to their testing, Fedora Python performance could be improved by ~15% by building /usr/bin/python* statically with libpython*.a. That sounds like a worthy thing to do.
https://bugzilla.redhat.com/show_bug.cgi?id=1749479
Since Python 3.8 Python extension modules are no longer linked to libpython.so and we can do the following:
- build /usr/bin/python3(.8) statically with libpython*.a
- build and ship libpython3.8.so.1.0 for packages that "embed" Python
The change in the python3 package is trivial:
https://src.fedoraproject.org/rpms/python3/pull-request/133
However it can have serious impact on Python extension modules that are linked to libpython3.8.so.1.0 by various "nonstandard" build mechanisms or by compiling code for Python extension module and code that embeds Python into one file.
We will likely propose a Fedora 32 Change for this, however I'm opening this topic for discussion before we do so.
Testing the proposed Pull Request with your code is also helpful. Let me know how can we make that easier (e.g. if you want a Copr or a Fedora 30/31 python38 package with this change).
WDYT?
One major concern: We currently rely on the libpython shared object linkage for non-standard builds to ensure we get everything for moving to new Python versions. I've personally experienced quirks with trying to force linking via the shared object when the Python interpreter itself is not. For example, perl-Inline-Python determines how to build itself by asking the interpreter. In Fedora, this works correctly because the interpreter returns the shared object. In other distributions where I've built the module, it doesn't and uses the static link object, which makes it difficult to automatically generate the link dependency to the system Python we are using.
In Mageia, we actually reverted Python's change to not link libpython with extension modules so that we still have the dependency link for binary objects: https://svnweb.mageia.org/packages/cauldron/python3/current/SOURCES/link-C-m...
Is there no other way to get better performance? Do we want to potentially give up the easily trackable dependency web for that? Is it worth breaking non-standard build mechanisms that interrogate the interpreter for how to link to it?
Hi Neal and thanks for voicing that concern,
Through our latest benchmarks the speedup could be up to 27% so I would say it's definitely worth it, and we plan to work with the affected packages to see how to best resolve the issues, if they arise, on a case by case basis. Currently on rawhide there are 113 packages requiring libpython.
Ubuntu and Debian are doing this for years so I would expect that at least the packages that are also included there, would either be working or require slight modifications.
The change will be posted soon for F32 on the devel list, so you can check all the details and we can continue this conversation by then.
-- 真実はいつも一つ!/ Always, there's only one truth! _______________________________________________ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproje...