ello everyone, I would like to continue with the discussion about Python3 as a default and what it means for python packaging guidelines (f23+). This mail derives from [0][1]. Before I proceed to open a tickets I would like to hear your opinions first.
*INTRO* (Not so) Recently the following change to the packaging guidelines was accepted[1]: "The unversioned macros, %{__python}, %{python_sitelib}, and %{python_sitearch} are generic macros that will always point to the default implementation. You may only use them with applications that need to choose to use the system's default version of python. (Currently this is the python2 interpreter.)" with the conjunction of another accepted and ongoing change targeted for f23 "Python 3 as default"[2] it means that %{__python} macro (and its derivatives) will/should change to point to python3 related paths.
*USED TERMS* For further reference we need to *distinguish* between python *modules* and python *applications*. Simply said, application is not a module :). Application foo is not meant to be used within others python libraries via `import foo` and Both python3 and python2 versions of foo provides same functionality and therefore only one version is needed. This also includes scripts. DevAssistant is an *application* - We invoke DA and we don't care if it is python2 and python3 based, both will fulfill our task. pip is not a application, even though it is not used via import statement both python3 and python2 versions provides different functionality (python-pip installs python2 packages and python3-pip installs python3 packages), therefore it is a *module*.
versioned macros - %{__python2}, %{__python3}, %{python3_sitearch} etc unversioned macros - %{__python}, %{python_sitelib}, %{python_sitearch}
What needs to be done, assuming that {__python} will represent default python - python3.
*MODULES*
M1. First of all, all *modules* which aren't using versioned macros must be fixed to use them. This can be done right away as this is already part of packaging guidelines and all packages should comply with guidelines. * Note: There is around of 1000 packages using unversioned macros [3]
M2. We should add provides for python2-foo modules. So python-foo would provide python2-foo. Fix all the modules to (Build)Require python2-bar instead of python-bar (python should also provide python2). Also if module foo ships bin file `baz` it should have `baz` and `baz2` bin file inside `python-foo` and `baz3` file inside `python3-foo`. This comply with upstream recommendation to keep /usr/bin/python point to python2, when (and if ever) upstream starts to recommend that /usr/bin/python should point to python3 we will follow.
M3. All modules should be build with option --executable='/usr/bin/python(2,3)'. This could be resolved in [4]
*APPLICATIONS*
A1. All application must use the default python (of course only if upstream supports it). Applications can continue using {__python} macros and it derivatives. We should add a macro for (Build)Requires:
%global py_default_major 3 # this could be part of f23 buildroot macros BuildRequires: python%{?py_default_major}-foo
This way would maintainer have same specfile for both fedora and epel and also if the default python will change in the future the only thing that would need a change is the `py_default_major` value or we could make the value to be resolved by %{__python} macro.
A2. Same as M3 (=should be resolved by [4]).
*{__python} VS /usr/bin/python CONFUSION* Why is value of {__python} being changed and /usr/bin/python (along with python-foo being python2) is untached? I see this as two different situations or two different point of views.
/usr/bin/python is a *user view*, as a user I would expect when I type python that it would fire up python2 interpreter as this is the default behaviour for all(-ArchLinux) distros and also python.org recommendation. Similarly when I type `sudo dnf install python-foo` I would expect to receive python2 version of foo package. This is why we stay with /usr/bin/python pointing to python2 and python-foo to provide python2 version of package. As a user I don't care for macros and their values, they are hidden from me => I am not confused, I get what I expect.
{__python} is a *packager view*, as a packager, I follow the guidelines and I follow the changes. I understand that there are two major versions of interpreter and we are switching to the python3 to be the default one. For me, python-foo is just a name of the package. I operate with python(2/3)-foo as build(requires) and versioned macros within my specfile if the package is the module. I understand why python-foo provides python2 version of package, yet I operate only with versioned packages/macros => I am not confused, its just *python2* or *python3* for me. If my package is an application, I use only default python macros because I ship only one version of an application for one version of an interpreter => I am not confused its just *python* for me and *python* is the default distro python.
*FUTURE* These suggestions (M1-M3, A1-A2) I've listed here are minimal changes needed for the Python3 as a default change. There is of course much more to deal with but for f23 timeframe it should be enough as it doesn't seem that /usr/bin/python will point to python3 any time soon [citation needed].
If those changes get accepted I would like to start applying them right away (for F23+ branches) because they should work even with __python macro still pointing to /usr/bin/python(2): 1. Fix all the modules specfiles and rebuild them because of new provides python2-foo. 2. Fix all the applications specfiles. (Rebuild is not needed.) 3. Change the default macro to point to /usr/bin/python3 (when anaconda is py3 ready) 4. Rebuild applications 5. Fix those which fails to build 6. Profit
-- Robert Kuska {rkuska}
[0] https://etherpad.mozilla.org/ep/pad/view/2Uqk0ikCll/vFEmg9YT2h [1] https://fedorahosted.org/fpc/ticket/498 [2] https://fedoraproject.org/wiki/Changes/Python_3_as_Default [3] https://pastebin.mozilla.org/8829944 (silly script used may miss some may contain redudant) [4] https://fedorahosted.org/fpc/ticket/513
-- Robert Kuska {rkuska}
[3] https://pastebin.mozilla.org/8829944 (silly script used may miss some may contain redudant)
[3] https://pastebin.mozilla.org/8830199
(I somehow pasted wrong link)
[4] https://fedorahosted.org/fpc/ticket/513
-- Robert Kuska {rkuska} _______________________________________________ python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
+1 from me on the suggested changes.
On 15 April 2015 at 02:56, Robert Kuska rkuska@redhat.com wrote:
*{__python} VS /usr/bin/python CONFUSION* Why is value of {__python} being changed and /usr/bin/python (along with python-foo being python2) is untached? I see this as two different situations or two different point of views.
/usr/bin/python is a *user view*, as a user I would expect when I type python that it would fire up python2 interpreter as this is the default behaviour for all(-ArchLinux) distros and also python.org recommendation. Similarly when I type `sudo dnf install python-foo` I would expect to receive python2 version of foo package. This is why we stay with /usr/bin/python pointing to python2 and python-foo to provide python2 version of package. As a user I don't care for macros and their values, they are hidden from me => I am not confused, I get what I expect.
We discussed the future fate of the /usr/bin/python symlink at the language summit (and a little further at the sprints), and it currently seems likely that we're going to go down the path of the Python Launcher for Windows model, and make "python" refer to a configurable shim that:
1. Launches an administrator configured Python version by default 2. Can be switched to run a different Python via command line options
This is essentially the rubypick model, but the CLI and configuration file format will be defined by CPython upstream rather than by the distro.
Cheers, Nick.
----- Original Message -----
+1 from me on the suggested changes.
On 15 April 2015 at 02:56, Robert Kuska rkuska@redhat.com wrote:
*{__python} VS /usr/bin/python CONFUSION* Why is value of {__python} being changed and /usr/bin/python (along with python-foo being python2) is untached? I see this as two different situations or two different point of views.
/usr/bin/python is a *user view*, as a user I would expect when I type python that it would fire up python2 interpreter as this is the default behaviour for all(-ArchLinux) distros and also python.org recommendation. Similarly when I type `sudo dnf install python-foo` I would expect to receive python2 version of foo package. This is why we stay with /usr/bin/python pointing to python2 and python-foo to provide python2 version of package. As a user I don't care for macros and their values, they are hidden from me => I am not confused, I get what I expect.
We discussed the future fate of the /usr/bin/python symlink at the language summit (and a little further at the sprints), and it currently seems likely that we're going to go down the path of the Python Launcher for Windows model, and make "python" refer to a configurable shim that:
- Launches an administrator configured Python version by default
- Can be switched to run a different Python via command line options
This is essentially the rubypick model, but the CLI and configuration file format will be defined by CPython upstream rather than by the distro.
It seems that Debian people are already onto something. I just saw this on debian-python ML [1] (where I'm lurking to catch things like this :)) - Robert, perhaps you should get in touch with Geoffrey and talk to him about this? IMO in the end this should end up being a PEP or at least a cross-distro agreed standard. Either way, I think we want to be part of this from the very beginning.
Slavek
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
[1] https://ldpreload.com/blog/usr-bin-python-23
----- Original Message -----
From: "Bohuslav Kabrda" slavek@redhat.com To: "Fedora Python SIG" python-devel@lists.fedoraproject.org Sent: Monday, April 20, 2015 9:02:35 AM Subject: Re: Python3 as default
----- Original Message -----
+1 from me on the suggested changes.
On 15 April 2015 at 02:56, Robert Kuska rkuska@redhat.com wrote:
*{__python} VS /usr/bin/python CONFUSION* Why is value of {__python} being changed and /usr/bin/python (along with python-foo being python2) is untached? I see this as two different situations or two different point of views.
/usr/bin/python is a *user view*, as a user I would expect when I type python that it would fire up python2 interpreter as this is the default behaviour for all(-ArchLinux) distros and also python.org recommendation. Similarly when I type `sudo dnf install python-foo` I would expect to receive python2 version of foo package. This is why we stay with /usr/bin/python pointing to python2 and python-foo to provide python2 version of package. As a user I don't care for macros and their values, they are hidden from me => I am not confused, I get what I expect.
We discussed the future fate of the /usr/bin/python symlink at the language summit (and a little further at the sprints), and it currently seems likely that we're going to go down the path of the Python Launcher for Windows model, and make "python" refer to a configurable shim that:
- Launches an administrator configured Python version by default
- Can be switched to run a different Python via command line options
This is essentially the rubypick model, but the CLI and configuration file format will be defined by CPython upstream rather than by the distro.
It seems that Debian people are already onto something. I just saw this on debian-python ML [1] (where I'm lurking to catch things like this :)) - Robert, perhaps you should get in touch with Geoffrey and talk to him about this? IMO in the end this should end up being a PEP or at least a cross-distro agreed standard. Either way, I think we want to be part of this from the very beginning.
Will do, I am currently catching on with their mailing list which I haven't checked for some time. I am already prowling on #debian-python at irc.debian.org.
Slavek
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
[1] https://ldpreload.com/blog/usr-bin-python-23
-- Regards, Slavek Kabrda _______________________________________________ python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
We discussed the future fate of the /usr/bin/python symlink at the language summit (and a little further at the sprints), and it currently seems likely that we're going to go down the path of the Python Launcher for Windows model, and make "python" refer to a configurable shim that:
- Launches an administrator configured Python version by default
- Can be switched to run a different Python via command line options
This is essentially the rubypick model, but the CLI and configuration file format will be defined by CPython upstream rather than by the distro.
It seems that Debian people are already onto something. I just saw
there are still many Debian developers that will do their best to make sure /usr/bin/python points to /usr/bin/python2.7 at least until 2025 or so (assuming we'll still have this package in the archive after 2020)
That said, we'll do our best to remove this package from most/all installations.
Hi, BTW. I'm lurking here since a while :)
----- Original Message -----
From: "Nick Coghlan" ncoghlan@gmail.com To: "Fedora Python SIG" python-devel@lists.fedoraproject.org Sent: Friday, April 17, 2015 7:50:54 AM Subject: Re: Python3 as default
+1 from me on the suggested changes.
On 15 April 2015 at 02:56, Robert Kuska rkuska@redhat.com wrote:
*{__python} VS /usr/bin/python CONFUSION* Why is value of {__python} being changed and /usr/bin/python (along with python-foo being python2) is untached? I see this as two different situations or two different point of views.
/usr/bin/python is a *user view*, as a user I would expect when I type python that it would fire up python2 interpreter as this is the default behaviour for all(-ArchLinux) distros and also python.org recommendation. Similarly when I type `sudo dnf install python-foo` I would expect to receive python2 version of foo package. This is why we stay with /usr/bin/python pointing to python2 and python-foo to provide python2 version of package. As a user I don't care for macros and their values, they are hidden from me => I am not confused, I get what I expect.
We discussed the future fate of the /usr/bin/python symlink at the language summit (and a little further at the sprints), and it currently seems likely that we're going to go down the path of the Python Launcher for Windows model, and make "python" refer to a configurable shim that:
- Launches an administrator configured Python version by default
- Can be switched to run a different Python via command line options
This is essentially the rubypick model, but the CLI and configuration file format will be defined by CPython upstream rather than by the distro.
I like the idea of letting the user to decide whether the /usr/bin/python should point to python2 or python3, even better, to make it configuarable for both users and admins of the machine.
Admin choose the default. User may change it for himself.
(I don't know if it works the same way on Windows)
This of course means that we have to make sure to don't leave any shebang with /usr/bin/python in fedora packages.
The question is, (with user choice for /usr/bin/python in mind which makes python to be different interpret per user machine) what should python-foo provides? Should we provide python2 package and change it later (or never) to python3 (?) or make python-foo a dummy package (containing no files, is it even possible in rpm?) and force users (packagers will have to) to use python2-foo python3-foo?
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia _______________________________________________ python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
On Apr 15, 2015 2:57 AM, "Robert Kuska" rkuska@redhat.com wrote:
pip is not a application, even though it is not used via import statement
both python3 and python2
versions provides different functionality (python-pip installs python2
packages and python3-pip
installs python3 packages), therefore it is a *module*.
This should probably be phrased differently in the final draft. Pip is an application. But because we need it to provide both a python 2 and python 3 cli tool it follows the same guidelines as dual-python-version modules rather than applications. This category might even deserve its own subsection as there's a couple other specific things to do with these types of packages.
*MODULES*
M1. First of all, all *modules* which aren't using versioned macros must
be fixed to use them.
This can be done right away as this is already part of packaging
guidelines and all packages
should comply with guidelines.
- Note: There is around of 1000 packages using unversioned macros [3]
M2. We should add provides for python2-foo modules. So python-foo would
provide python2-foo.
I'd make the following its own should bullet as the first part of M2 is more important. the python-foo package names aren't going away so if we get into a time crunch for f22, this second portion isn't as critical as the first.
Fix all the modules to (Build)Require python2-bar instead of python-bar
(python should
also provide python2).
Also if module foo ships bin file `baz` it should have `baz` and `baz2` bin file inside `python-foo` and `baz3` file inside `python3-foo`.
I disagree with this but I think it's probably just an omission of some information. We need to make clear here that this is only for bin files where it is necessary to shop a version that runs on py2 and a version that runs on py3. Most packages should just ship one version of the bin scripts for the default python version. (Note, I don't think we can wrap this choice into a convenient macro. It'll probably need a spec file conditional if packagers want to have a single spec for multiple branches.)
M3. All modules should be build with option
--executable='/usr/bin/python(2,3)'. This could be
resolved in [4]
I'm not sure if this is true. Pure modules don't have a shebang line so I think the choice of which python interpreter runs them and determines the path they install into is sufficient.
From a message from ncoghlan a long time ago I think things in bin should
use /usr/bin/python(2,3) in their shebang as long as the setup.py is invoked with the versioned path. So --executable is extraneous for these purposes. (But if [4] is the -s guideline update, we would want to use --executable for that purpose for packages providing things in bin).
*APPLICATIONS*
A1. All application must use the default python (of course only if
upstream supports it).
Applications can continue using {__python} macros and it derivatives. We
should add a macro
for (Build)Requires:
%global py_default_major 3 # this could be part of f23 buildroot macros BuildRequires: python%{?py_default_major}-foo
This way would maintainer have same specfile for both fedora and epel and
also if the default
python will change in the future the only thing that would need a change
is the `py_default_major`
value or we could make the value to be resolved by %{__python} macro.
A2. Same as M3 (=should be resolved by [4]).
*{__python} VS /usr/bin/python CONFUSION* Why is value of {__python} being changed and /usr/bin/python (along with
python-foo being python2)
is untached? I see this as two different situations or two different
point of views.
/usr/bin/python is a *user view*, as a user I would expect when I type
python that it would fire
up python2 interpreter as this is the default behaviour for
all(-ArchLinux) distros and also
python.org recommendation. Similarly when I type `sudo dnf install
python-foo` I would expect
to receive python2 version of foo package. This is why we stay with
/usr/bin/python pointing
to python2 and python-foo to provide python2 version of package. As a
user I don't care for macros
and their values, they are hidden from me => I am not confused, I get
what I expect.
{__python} is a *packager view*, as a packager, I follow the guidelines
and I follow the changes.
I understand that there are two major versions of interpreter and we are
switching to the python3
to be the default one. For me, python-foo is just a name of the package. I operate with
python(2/3)-foo as build(requires)
and versioned macros within my specfile if the package is the module. I
understand why python-foo
provides python2 version of package, yet I operate only with versioned
packages/macros => I
am not confused, its just *python2* or *python3* for me. If my package is an application, I use only default python macros because
I ship only one version
of an application for one version of an interpreter => I am not confused
its just *python* for me and
*python* is the default distro python.
*FUTURE* These suggestions (M1-M3, A1-A2) I've listed here are minimal changes
needed for the Python3 as a
default change. There is of course much more to deal with but for f23
timeframe it should be
enough as it doesn't seem that /usr/bin/python will point to python3 any
time soon [citation needed].
If those changes get accepted I would like to start applying them right
away (for F23+ branches)
because they should work even with __python macro still pointing to
/usr/bin/python(2):
- Fix all the modules specfiles and rebuild them because of new provides
python2-foo.
- Fix all the applications specfiles. (Rebuild is not needed.)
- Change the default macro to point to /usr/bin/python3 (when anaconda
is py3 ready)
- Rebuild applications
- Fix those which fails to build
- Profit
Couple thoughts here:
(1) We should have a package cleanup to implement these like we had for removing the python-setuptools-devel buildrequires and the python-pillow update. Since this could be a rather more invasive change to spec files we should start this sooner to allow package maintainers to update their spec files on their own but still allow us plenty of time to fix all the specs.
(2) Conditionalizing specs for multiple distributions has been something of an anti goal in the past as it obfuscated the spec file. We have a guideline not to do this for other distributions. Since our packagers work on both fedora and epel, it's reasonable to want to conditionalize so they can keep a single spec file if they desire. However,I think we should also allow people to keep simpler specs and simply maintain diverged spec files between python 3 by default and python 2 by default distro versions. So I think we should allow people to hardcore things like "buildrequires: python3-foo" and maintaining two specs if they wish.
If so, then the guideline draft would need to make clear which portion of this is the policy and goals and which is implementation.
We'd also want to mention whether people who participate in cleaning up the packages for other maintainers will use a specific style or can use either method just so there's no surprises later. (I think the style being up to the person porting is fine as long as that expectation is properly set.)
-Toshio
-- Robert Kuska {rkuska}
[0] https://etherpad.mozilla.org/ep/pad/view/2Uqk0ikCll/vFEmg9YT2h [1] https://fedorahosted.org/fpc/ticket/498 [2] https://fedoraproject.org/wiki/Changes/Python_3_as_Default [3] https://pastebin.mozilla.org/8829944 (silly script used may miss some
may contain redudant)
[4] https://fedorahosted.org/fpc/ticket/513
-- Robert Kuska {rkuska} _______________________________________________ python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
----- Original Message -----
From: "Toshio Kuratomi" a.badger@gmail.com To: "Fedora Python SIG" python-devel@lists.fedoraproject.org Sent: Friday, April 17, 2015 2:14:20 PM Subject: Re: Python3 as default
On Apr 15, 2015 2:57 AM, "Robert Kuska" < rkuska@redhat.com > wrote:
pip is not a application, even though it is not used via import statement both python3 and python2 versions provides different functionality (python-pip installs python2 packages and python3-pip installs python3 packages), therefore it is a *module*.
This should probably be phrased differently in the final draft. Pip is an application. But because we need it to provide both a python 2 and python 3 cli tool it follows the same guidelines as dual-python-version modules rather than applications. This category might even deserve its own subsection as there's a couple other specific things to do with these types of packages.
Yes, I agree that this needs better wording for the guidelines draft.
*MODULES*
M1. First of all, all *modules* which aren't using versioned macros must be fixed to use them. This can be done right away as this is already part of packaging guidelines and all packages should comply with guidelines.
- Note: There is around of 1000 packages using unversioned macros [3]
M2. We should add provides for python2-foo modules. So python-foo would provide python2-foo.
I'd make the following its own should bullet as the first part of M2 is more important. the python-foo package names aren't going away so if we get into a time crunch for f22, this second portion isn't as critical as the first.
That's a good point as it will save us from the figuring out the rebuild dependency chain.
Fix all the modules to (Build)Require python2-bar instead of python-bar (python should also provide python2).
Also if module foo ships bin file `baz` it should have `baz` and `baz2` bin file inside `python-foo` and `baz3` file inside `python3-foo`.
I disagree with this but I think it's probably just an omission of some information. We need to make clear here that this is only for bin files where it is necessary to shop a version that runs on py2 and a version that runs on py3. Most packages should just ship one version of the bin scripts for the default python version. (Note, I don't think we can wrap this choice into a convenient macro. It'll probably need a spec file conditional if packagers want to have a single spec for multiple branches.)
That's meant to be only for the applications like a modules/modules like a applications (pip, pytest and similiar).
I agree that that /usr/bin/foo is enough for an (pure) application module. No need for a versioned one.
M3. All modules should be build with option --executable='/usr/bin/python(2,3)'. This could be resolved in [4]
I'm not sure if this is true. Pure modules don't have a shebang line so I think the choice of which python interpreter runs them and determines the path they install into is sufficient.
Again, this point was constructed with an assumption of pip and pytest being (kind of) modules and also with a possibility of creating macros for easier packaging which contain `--executable` in the draft.
From a message from ncoghlan a long time ago I think things in bin should use /usr/bin/python(2,3) in their shebang as long as the setup.py is invoked with the versioned path. So --executable is extraneous for these purposes. (But if [4] is the -s guideline update, we would want to use --executable for that purpose for packages providing things in bin).
*APPLICATIONS*
A1. All application must use the default python (of course only if upstream supports it). Applications can continue using {__python} macros and it derivatives. We should add a macro for (Build)Requires:
%global py_default_major 3 # this could be part of f23 buildroot macros BuildRequires: python%{?py_default_major}-foo
This way would maintainer have same specfile for both fedora and epel and also if the default python will change in the future the only thing that would need a change is the `py_default_major` value or we could make the value to be resolved by %{__python} macro.
A2. Same as M3 (=should be resolved by [4]).
*{__python} VS /usr/bin/python CONFUSION* Why is value of {__python} being changed and /usr/bin/python (along with python-foo being python2) is untached? I see this as two different situations or two different point of views.
/usr/bin/python is a *user view*, as a user I would expect when I type python that it would fire up python2 interpreter as this is the default behaviour for all(-ArchLinux) distros and also python.org recommendation. Similarly when I type `sudo dnf install python-foo` I would expect to receive python2 version of foo package. This is why we stay with /usr/bin/python pointing to python2 and python-foo to provide python2 version of package. As a user I don't care for macros and their values, they are hidden from me => I am not confused, I get what I expect.
{__python} is a *packager view*, as a packager, I follow the guidelines and I follow the changes. I understand that there are two major versions of interpreter and we are switching to the python3 to be the default one. For me, python-foo is just a name of the package. I operate with python(2/3)-foo as build(requires) and versioned macros within my specfile if the package is the module. I understand why python-foo provides python2 version of package, yet I operate only with versioned packages/macros => I am not confused, its just *python2* or *python3* for me. If my package is an application, I use only default python macros because I ship only one version of an application for one version of an interpreter => I am not confused its just *python* for me and *python* is the default distro python.
*FUTURE* These suggestions (M1-M3, A1-A2) I've listed here are minimal changes needed for the Python3 as a default change. There is of course much more to deal with but for f23 timeframe it should be enough as it doesn't seem that /usr/bin/python will point to python3 any time soon [citation needed].
If those changes get accepted I would like to start applying them right away (for F23+ branches) because they should work even with __python macro still pointing to /usr/bin/python(2):
- Fix all the modules specfiles and rebuild them because of new provides
python2-foo. 2. Fix all the applications specfiles. (Rebuild is not needed.) 3. Change the default macro to point to /usr/bin/python3 (when anaconda is py3 ready) 4. Rebuild applications 5. Fix those which fails to build 6. Profit
Couple thoughts here:
(1) We should have a package cleanup to implement these like we had for removing the python-setuptools-devel buildrequires and the python-pillow update. Since this could be a rather more invasive change to spec files we should start this sooner to allow package maintainers to update their spec files on their own but still allow us plenty of time to fix all the specs.
(2) Conditionalizing specs for multiple distributions has been something of an anti goal in the past as it obfuscated the spec file. We have a guideline not to do this for other distributions. Since our packagers work on both fedora and epel, it's reasonable to want to conditionalize so they can keep a single spec file if they desire. However,I think we should also allow people to keep simpler specs and simply maintain diverged spec files between python 3 by default and python 2 by default distro versions. So I think we should allow people to hardcore things like "buildrequires: python3-foo" and maintaining two specs if they wish.
If so, then the guideline draft would need to make clear which portion of this is the policy and goals and which is implementation.
We'd also want to mention whether people who participate in cleaning up the packages for other maintainers will use a specific style or can use either method just so there's no surprises later. (I think the style being up to the person porting is fine as long as that expectation is properly set.)
-Toshio
-- Robert Kuska {rkuska}
[0] https://etherpad.mozilla.org/ep/pad/view/2Uqk0ikCll/vFEmg9YT2h [1] https://fedorahosted.org/fpc/ticket/498 [2] https://fedoraproject.org/wiki/Changes/Python_3_as_Default [3] https://pastebin.mozilla.org/8829944 (silly script used may miss some may contain redudant) [4] https://fedorahosted.org/fpc/ticket/513
-- Robert Kuska {rkuska} _______________________________________________ python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
(I've miss hitted enter and sent the email earlier)
----- Original Message -----
From: "Robert Kuska" rkuska@redhat.com To: "Fedora Python SIG" python-devel@lists.fedoraproject.org Sent: Monday, April 20, 2015 10:05:44 AM Subject: Re: Python3 as default
----- Original Message -----
From: "Toshio Kuratomi" a.badger@gmail.com To: "Fedora Python SIG" python-devel@lists.fedoraproject.org Sent: Friday, April 17, 2015 2:14:20 PM Subject: Re: Python3 as default
On Apr 15, 2015 2:57 AM, "Robert Kuska" < rkuska@redhat.com > wrote:
pip is not a application, even though it is not used via import statement both python3 and python2 versions provides different functionality (python-pip installs python2 packages and python3-pip installs python3 packages), therefore it is a *module*.
This should probably be phrased differently in the final draft. Pip is an application. But because we need it to provide both a python 2 and python 3 cli tool it follows the same guidelines as dual-python-version modules rather than applications. This category might even deserve its own subsection as there's a couple other specific things to do with these types of packages.
Yes, I agree that this needs better wording for the guidelines draft.
*MODULES*
M1. First of all, all *modules* which aren't using versioned macros must be fixed to use them. This can be done right away as this is already part of packaging guidelines and all packages should comply with guidelines.
- Note: There is around of 1000 packages using unversioned macros [3]
M2. We should add provides for python2-foo modules. So python-foo would provide python2-foo.
I'd make the following its own should bullet as the first part of M2 is more important. the python-foo package names aren't going away so if we get into a time crunch for f22, this second portion isn't as critical as the first.
That's a good point as it will save us from the figuring out the rebuild dependency chain.
Fix all the modules to (Build)Require python2-bar instead of python-bar (python should also provide python2).
Also if module foo ships bin file `baz` it should have `baz` and `baz2` bin file inside `python-foo` and `baz3` file inside `python3-foo`.
I disagree with this but I think it's probably just an omission of some information. We need to make clear here that this is only for bin files where it is necessary to shop a version that runs on py2 and a version that runs on py3. Most packages should just ship one version of the bin scripts for the default python version. (Note, I don't think we can wrap this choice into a convenient macro. It'll probably need a spec file conditional if packagers want to have a single spec for multiple branches.)
That's meant to be only for the applications like a modules/modules like a applications (pip, pytest and similiar).
I agree that that /usr/bin/foo is enough for an (pure) application module. No need for a versioned one.
M3. All modules should be build with option --executable='/usr/bin/python(2,3)'. This could be resolved in [4]
I'm not sure if this is true. Pure modules don't have a shebang line so I think the choice of which python interpreter runs them and determines the path they install into is sufficient.
Again, this point was constructed with an assumption of pip and pytest being (kind of) modules and also with a possibility of creating macros for easier packaging which contain `--executable` in the draft.
https://fedorahosted.org/fpc/ticket/281#comment:19
From a message from ncoghlan a long time ago I think things in bin should use /usr/bin/python(2,3) in their shebang as long as the setup.py is invoked with the versioned path. So --executable is extraneous for these purposes. (But if [4] is the -s guideline update, we would want to use --executable for that purpose for packages providing things in bin).
*APPLICATIONS*
A1. All application must use the default python (of course only if upstream supports it). Applications can continue using {__python} macros and it derivatives. We should add a macro for (Build)Requires:
%global py_default_major 3 # this could be part of f23 buildroot macros BuildRequires: python%{?py_default_major}-foo
This way would maintainer have same specfile for both fedora and epel and also if the default python will change in the future the only thing that would need a change is the `py_default_major` value or we could make the value to be resolved by %{__python} macro.
A2. Same as M3 (=should be resolved by [4]).
*{__python} VS /usr/bin/python CONFUSION* Why is value of {__python} being changed and /usr/bin/python (along with python-foo being python2) is untached? I see this as two different situations or two different point of views.
/usr/bin/python is a *user view*, as a user I would expect when I type python that it would fire up python2 interpreter as this is the default behaviour for all(-ArchLinux) distros and also python.org recommendation. Similarly when I type `sudo dnf install python-foo` I would expect to receive python2 version of foo package. This is why we stay with /usr/bin/python pointing to python2 and python-foo to provide python2 version of package. As a user I don't care for macros and their values, they are hidden from me => I am not confused, I get what I expect.
{__python} is a *packager view*, as a packager, I follow the guidelines and I follow the changes. I understand that there are two major versions of interpreter and we are switching to the python3 to be the default one. For me, python-foo is just a name of the package. I operate with python(2/3)-foo as build(requires) and versioned macros within my specfile if the package is the module. I understand why python-foo provides python2 version of package, yet I operate only with versioned packages/macros => I am not confused, its just *python2* or *python3* for me. If my package is an application, I use only default python macros because I ship only one version of an application for one version of an interpreter => I am not confused its just *python* for me and *python* is the default distro python.
*FUTURE* These suggestions (M1-M3, A1-A2) I've listed here are minimal changes needed for the Python3 as a default change. There is of course much more to deal with but for f23 timeframe it should be enough as it doesn't seem that /usr/bin/python will point to python3 any time soon [citation needed].
If those changes get accepted I would like to start applying them right away (for F23+ branches) because they should work even with __python macro still pointing to /usr/bin/python(2):
- Fix all the modules specfiles and rebuild them because of new provides
python2-foo. 2. Fix all the applications specfiles. (Rebuild is not needed.) 3. Change the default macro to point to /usr/bin/python3 (when anaconda is py3 ready) 4. Rebuild applications 5. Fix those which fails to build 6. Profit
Couple thoughts here:
(1) We should have a package cleanup to implement these like we had for removing the python-setuptools-devel buildrequires and the python-pillow update. Since this could be a rather more invasive change to spec files we should start this sooner to allow package maintainers to update their spec files on their own but still allow us plenty of time to fix all the specs.
Yes, I want to start right away after the guidelines get accepted.
(2) Conditionalizing specs for multiple distributions has been something of an anti goal in the past as it obfuscated the spec file. We have a guideline not to do this for other distributions. Since our packagers work on both fedora and epel, it's reasonable to want to conditionalize so they can keep a single spec file if they desire. However,I think we should also allow people to keep simpler specs and simply maintain diverged spec files between python 3 by default and python 2 by default distro versions. So I think we should allow people to hardcore things like "buildrequires: python3-foo" and maintaining two specs if they wish.
That's a reason why I want to allow packagers to use unversioned macros within applications specfiles and introduce the %{py_default_major} macro so they can keep one specfile for both epel and fedora.
BuildRequires: python%{?py_default_major}-foo -> EPEL: python-foo Fedora: python3-foo
%install %{__python} setup.py install -> EPEL: /usr/bin/python setup.py install Fedora: /usr/bin/python3 setup.py install
and same for %{python_site*} macros.
Also thats the reason why I proposed to create also unversioned macros. https://fedorahosted.org/fpc/ticket/513#comment:15
Of course if someone wish to, they can have different specfiles. It is not 'must to have' just 'nice to have'.
If so, then the guideline draft would need to make clear which portion of this is the policy and goals and which is implementation.
We'd also want to mention whether people who participate in cleaning up the packages for other maintainers will use a specific style or can use either method just so there's no surprises later. (I think the style being up to the person porting is fine as long as that expectation is properly set.)
-Toshio
-- Robert Kuska {rkuska}
[0] https://etherpad.mozilla.org/ep/pad/view/2Uqk0ikCll/vFEmg9YT2h [1] https://fedorahosted.org/fpc/ticket/498 [2] https://fedoraproject.org/wiki/Changes/Python_3_as_Default [3] https://pastebin.mozilla.org/8829944 (silly script used may miss some may contain redudant) [4] https://fedorahosted.org/fpc/ticket/513
-- Robert Kuska {rkuska} _______________________________________________ python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
python-devel@lists.fedoraproject.org
-
Bohuslav Kabrda -
Nick Coghlan -
Piotr Ozarowski -
Robert Kuska -
Toshio Kuratomi