On Wed, Aug 12, 2020 at 10:23 AM Petr Viktorin pviktori@redhat.com wrote:
I'll move some discussion here, where it doesn't become part of the document:
On 2020-08-11 14:19, Petr Viktorin wrote:
These Guidelines represent a major rewrite and paradigm shift, and not all packages are updated to reflect this. Older guidelines are still being kept up to date, and existing packages **MAY** use them instead of this document:
- 201x-era [Python packaging
guidelines](https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/) (for packages that use e.g. `%py3_install` or `setup.py install`)
- [Python 2
appendix](https://docs.fedoraproject.org/en-US/packaging-guidelines/Python_Appendix/#_...) (for e.g. `%py2_install`) (Python 2 packages require a FESCo exception.)
@Conan-Kudo:
Is there something you've done here to make the new macros unequivocally better? Do you have automated flavor builds, for example? Right now there is no effective difference.
The new macros allow other build tools than just setuptools. They also use upstream metadata for BuildRequires & file lists.
Of course, where possible the changes were backported to the existing macros. I don't necessarily see the macros as the main thing that changes. But when you look at a specfile, you can usually tell what conventions it uses by what macros you see.
Those are all nice things, for sure. I think it'd be important to spell that out somewhere.
## PyPI parity
Every Python package in Fedora **SHOULD** also be available on [the Python Package Index](https://pypi.org) (PyPI).
The command `pip install PROJECTNAME` **MUST** install the same package (possibly in a different version), install nothing, or fail with a reasonable error message.
If this is not the case, the packager **MUST** contact upstream about this. The goal is to get the project name registered or blocked on PyPI, or to otherwise ensure the rule is followed.
To block a project name on PyPI, email [admin@pypi.org](mailto:admin@pypi.org), giving the project name and explaining the situation (for example: the package cannot currently be installed via `pip`). You can ask questions and discuss the process at the [Python Discourse](https://discuss.python.org/t/block-names/4045).
NOTE: Project names that were in Fedora but not on PyPI when these guidelines were proposed are *blocked* from being uploaded to PyPI. This prevents potential trolls from taking them, but it also blocks legitimate owners. If your package is affected, contact the Python SIG or [file a PyPA
issue](https://github.com/pypa/pypi-support/issues/new?labels=PEP+541&template=...)
and mention `@encukou`.
This is necessary to protect users, avoid confusion, and enable automation as Fedora and upstream ecosystems grow more integrated.
As always, [specific exceptions can be granted by the Packaging Committee](https://docs.fedoraproject.org/en-US/packaging-guidelines/#_general_exceptio...).
@Conan-Kudo:
This is not reasonable to ask of packagers to do. Such a check is difficult to do, and there is no particularly compelling reason for making everything written in Python using Python build system available in PyPI. Unless you plan to provide an automated system to inform PyPI of these things, this is not only unreasonable, it is unenforceable.
A lot of stuff in the guidelines is unenforceable, so let's focus on the "unreasonable" part.
There is no reason to have everything available on PyPI, but I do believe it's reasonable to *reserve the name* in such cases.
Here's a reason why I want this:
The issue here is that Python tools have access to project names. For example, I can get the version of Requests using:
from importlib.metadata import version version('requests')
'2.22.0'
Things like this are only useful if we use a common namespace. Upstream, that namespace *is* PyPI; there's little we can do about that. If project names mean something else in Fedora than in the wider ecosystem, we'll get user confusion at best.
(If you use a private package index, like they do at CERN or some closed-source shops, there can be some collisions -- but in that case there's a limited number of software authors and users, and a lot more control over the package set. Conflicts in global repositories of free/open-source software are much harder to manage.)
Lately, I think about another way to handle this: packages that aren't on PyPI could not ship the .dist-info at all, and so, they would not have things like `python3dist(...)` provides. They'd only be usable with Fedora tooling, not in the wider Python ecosystem. It's a major case to think out and test, but maybe it would be worth it?
I think omitting the Provides is unfair.
What I think we should do is figure out a system to make such reservations more automated, like making such requests and linking them to package review bugs, packaging git repos, and a koji build to indicate their validity. PyPI upstream can then automatically verify the validity of the request and reserve them, and we could have a bug filed to have the maintainer reach out to upstream (or bug filed to upstream directly if the metadata has info on that) to submit stuff to PyPI.