krop and python3
by Raphael Groner
Hi,
I need some help to fix a bug ¹ with python3 and site-packages.
> cd /usr/lib/python3.4/site-packages/krop/
> python2 -c 'import krop ; krop.main()'
→ dialog
> python3 -c 'import krop ; krop.main()'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/lib/python3.4/site-packages/krop/krop.py", line 25, in main
formatter_class=RawTextHelpFormatter)
TypeError: __init__() got an unexpected keyword argument 'version'
Besides that error, the start script in %{_bindir}/krop is b0rken and should just work for users clicking on the desktop icon.
¹ https://bugzilla.redhat.com/show_bug.cgi?id=1321376
7 years, 11 months
Alternatives for python versions
by Avram Lubkin
I wasn't able to find much recent discussion on this, but I'm a bit
frustrated that there is so much manual work being done creating symlinks
when a system already exists for that.
It seem many projects which support both python2 and python3 are depending
on a hardcoded shebang, install order, or symlink creation in the spec file
to set which version of python to use. It also seems the py*_build macros
are hardcoded to use a versioned python interpreter for shebang lines.
Frankly I feel like I should be able to use /usr/bin/python or /usr/bin/env
python and that should point to a non version-specifc python interpreter.
If I have definite dependencies on a specific interpreter, then it is my
responsibility to set my shebang appropriately. Otherwise, if my project
has an unversioned executable, I have to change it based on my will and the
Fedora version, rather than the will of the end user. Maybe they prefer
python2, python3, or some other distribution of python becomes popular
later (pypy?). If I don't have hard dependencies, why should I force a
version? And why should I need to have conditionals on the Fedora version
or maintain two spec files?
Assuming an end user never touches the default configuration, dependencies
should already be taken care of with python_provides since unversioned
dependencies should default to the release default.
Whether /usr/bin/python is symlinked using alternatives or another method,
the py*_build macros should have a mechanism to overwrite the executable
value.
Avram
7 years, 12 months
Problems with scripts in a common spec file
by John Dennis
With reference to the guidelines for Python packaging found here:
https://fedoraproject.org/wiki/Packaging:Python
Specifically the material concerning executables in /usr/bin.
In the "Example common spec file" section is this comment.
# Must do the python2 install first because the scripts in /usr/bin are
# overwritten with every setup.py install, and in general we want the
# python3 version to be the default.
But this does not work if you only install the python2-XXX package
instead of both the python2-XXX and python3-XXX packages. Here is why.
distutils will adjust the shbang interpreter line in installed scripts
to be interpreter specific, e.g. you get one of:
#!/usr/bin/python2
or
#!/usr/bin/python3
But the guidelines require the py3 version of the script in the py2
package. Thus if you install just the python2-XXX package you'll end up
with a script that runs the /usr/bin/python3 interpreter whose sys.path
only includes library locations for py3! Thus the script cannot execute
because it's missing its libraries (because the libraries were installed
in the py2 locations).
How is this supposed to be addressed?
One possible workaround is to add a requirement on the py3 package in
the py2 package. But that is silly because it completely negates
everything in the py2 package (everything will run as if it were py3
after installing the py2 package and the py2 files will never be
referenced).
Perhaps the issue is better rephrased as "How does one package Python
when there are no version specific differences/requirements?" If this is
covered in the guidelines it's not obvious and needs clarification. I
assume this means using the generic %{_python} macros instead of dual
use of the version specific macros so that everything corresponds to the
system default version of Python. Correct?
--
John
7 years, 12 months
Applying for Python SIG
by Ardian Haxha
Hi there,
I am Ardian a student from Kosovo. I have been contributing to fedora
for quite some time as an Ambassador, I am almost done with my studies
so there is more free time, I would like to contribute in the
coding/packaging different parts of the project. Python being one of my
first and favorite programming languages is what I would like to get
started. I have already build an RPM with the help of Robert Scheck (I
am very thankful). I would like to join the python interest group.
Kindly,
Ardian.
8 years
Python-SIG introduction
by Charalampos Stratakis
Hello,
I am a member of Red Hat's Python Maintenance team and I have applied for the FAS group as well as the mailing list.
I am interested in Python in Fedora, upstream and well, everything python related.
Regards,
Charalampos Stratakis
8 years
Introduction (cydrobolt)
by Chaoyi Zha
Hey python-devel,
I'm Chaoyi, a Fedora user and contributor. I've been using Python for a
couple years now, and it has been a great experience. I write my apps in
Python whenever I can because of its readability and versatility. I found
out about the python-sig through a Trac ticket.
I love Python and would love to be involved in the Python community at
Fedora. I mostly contribute to Fedora on the infrastructure team, but I
also work with the ambassadors and web teams, among others.
My FAS and IRC username is "cydrobolt"
Thanks,
Chaoyi
8 years
Problems with script installation in RPM's
by John Dennis
I've been following the guidelines for Python packaging found here:
https://fedoraproject.org/wiki/Packaging:Python
in particular the cookbook for supporting both Py2 and Py3.
I've discovered two places where things fail to work as expected with
respect to script installation. Both of these problems took an
inordinately long time to fully diagnose. The first problem I believe is
a bug in the python rpm macros and needs to be fixed (filed as bug
#1335203). The second problem can be fixed via a workaound in the spec
file, but the cookbook recipe will need to be updated to call attention
to it.
Problem 1:
----------
If the script includes an interpreter argument in it's shebang line the
installed script cannot execute and aborts. For example if the source
script has a shebang line like this:
#!/usr/bin/python -E
The installed script will have a shebang line like this (using Py2 as an
example)
#!/usr/bin/python2 -s -E
The problem is how the shebang line is parsed to build parameters for
exec. Unfortunately there is no standard. Some operating systems split
at whitespace and treat the first part as the path to the interpreter
and the rest as individual arguments. Some operating systems split at
the first whitespace and treat the front part as the path to the
interpeter and the rest as a single argument (this is what happens in
Linux).
Thus when the Python interpreter is run it sees argv[1] as "-s -E" which
it doesn't understand as a single argument and the result is:
Unknown option: -
usage: /usr/bin/python [option] ... [-c cmd | -m mod | file | -] [arg] ...
Try `python -h' for more information.
The cause of the problem is in /usr/lib/rpm/macros.d/macros.python2 and
/usr/lib/rpm/macros.d/macros.python3. I'll show the Py2 version but the
Py3 version is equivalent.
%py2_shbang_opts -s
%py2_build() %{expand:\
CFLAGS="%{optflags}" %{__python2} %{py_setup} %{?py_setup_args} build
--executable="%{__python2} %{py2_shbang_opts}" %{?1}\
}
Then in distutils/command/build_scripts.py it uses the first_line_re
regular expression to split the shebang line into 2 parts, the
interpreter and the interpreter arguments. It then replaces the shebang
line with the value of --executable followed by a space and interpreter
arguments the regexp found. But in our case the interpreter is *not* the
path to an interpreter, rather it's the path to the interpreter *plus*
the %py2_shbang_opts. Thus you end up with multiple arguments to the
interpreter, but the interpreter doesn't see it as multiple arguments,
instead it sees 1 argument, a single string containing "-s -E".
I think the fundamental problem is that macros.python{2,3} is abusing
the --executable argument, it's meant to be the path to an executable
*only*. It clearly does not expect extra arguments.
If build_scripts.py had logic to merge arguments together, e.g. -s -E
would become -sE then all would be good, but it doesn't.
Problem 2:
----------
The Py2 version of the script (which forces /usr/bin/python2 in it's
shebang line) is installed in the python3 package. Thus if you install
python3-NVR and it has a script in /usr/bin you'll end up executing
python2 instead of python3.
The cookbook shows this snippet:
%install
# Must do the python2 install first because the scripts in /usr/bin are
# overwritten with every setup.py install, and in general we want the
# python3 version to be the default.
%py2_install
%py3_install
The problem is this is exactly the *opposite* of what actually happens
if builds are fast. %py2_install runs first and copies the Py2 version
of the script into $BUILDROOT/usr/bin. Next %py3_install runs and an
attempt is made to install the Py3 version into the exact same location
$BUILDROOT/usr/bin. However distutils has logic which avoids copying
over an existing installed file unless the src is newer. This occurs in
distutils/file_util.py which has this code in copy_file():
if update and not newer(src, dst):
if verbose >= 1:
log.debug("not copying %s (output up-to-date)", src)
return dst, 0
The documentation for newer() in distutils/dep_util.py acknowledges the
fundamental problem:
Note that this test is not very accurate: files created in the
same second will have the same "age".
This is exactly what was happening. %py2_build and %py3_build each
created the script with the same timestamp and the logic in copy_file()
concluded it shouldn't replace the Py2 script with the Py3 script.
The result is the python3-package ended up with a script with a Py2
interpreter shebang line, clearly this is wrong.
The workaround I came up with is to delay the execution of %py3_build by
at least 1 second by inserting a sleep in-between the %py2_build and
%py3_build macros in the spec file, like this:
%py2_build
sleep 1
%py3_build
Actually I think sleeping 2 seconds might be safer given the 1 second
resolution on the timestamps.
BTW, the fact the file was not updated is silent in the output of the
setup.py command. Note the message is contained in a log.debug, not a
log.info. I could find no way to enable debug log output using the
command line arguments. I had to resort to adding the following in my
setup.py script in order to see the message:
import distutils.log
distutils.log.set_verbosity(distutils.log.DEBUG) # Set DEBUG level
--
John
8 years
