Some heads-ups for Rawhide users: python3-ptyprocess and systemd issues

[Adam Williamson]

Hey folks! Just wanted to send out a heads-up for Rawhide users about
some issues that have shown up in openQA testing.

First, you may have found that trying to update to today's Rawhide
fails like this:

Error: Transaction test error:
  file /usr/lib/python3.11/site-packages/ptyprocess-0.7.0-py3.11.egg-info from install of python3-ptyprocess-0.7.0-2.fc38.noarch conflicts with file from package python3-ptyprocess-0.7.0-1.fc38.noarch

That's https://bugzilla.redhat.com/show_bug.cgi?id=2164207 , we figured
out the problem (thanks to Panu for getting to the root cause of it)
and it is fixed in python-ptyprocess-0.7.0-3.fc38 , which should be in
the next Rawhide compose. If you really can't wait you can update
directly to that version
from https://koji.fedoraproject.org/koji/buildinfo?buildID=2138649 ,
but you'd better watch out for issue #2...

The second issue is that systems that update to systemd-253~rc1-1.fc38
seem to get stuck on boot. With Plymouth enabled you just see the
splash screen. With it disabled (or by pressing esc) it seems to be
stuck at "Stopped initrd-switch-root.service - Switch Root.". I'm still
looking into this one, but it's happened to a lot of openQA tests and I
was able to confirm it first try in a local VM, by installing from the
20230123.n.0 compose then updating systemd and rebooting. Fresh
installs with the newer systemd seem to be OK, at least most openQA
tests for the new compose passed - it seems to be only updating an
existing install that has the problem, at least so far.

[Adam Williamson]

On Wed, 2023-01-25 at 10:30 -0800, Adam Williamson wrote:
> The second issue is that systems that update to systemd-253~rc1-1.fc38
> seem to get stuck on boot. With Plymouth enabled you just see the
> splash screen. With it disabled (or by pressing esc) it seems to be
> stuck at "Stopped initrd-switch-root.service - Switch Root.". I'm still
> looking into this one, but it's happened to a lot of openQA tests and I
> was able to confirm it first try in a local VM, by installing from the
> 20230123.n.0 compose then updating systemd and rebooting. Fresh
> installs with the newer systemd seem to be OK, at least most openQA
> tests for the new compose passed - it seems to be only updating an
> existing install that has the problem, at least so far.

I've now filed this as
https://bugzilla.redhat.com/show_bug.cgi?id=2164594 .

[Adam Williamson]

On Wed, 2023-01-25 at 10:30 -0800, Adam Williamson wrote:
> Hey folks! Just wanted to send out a heads-up for Rawhide users about
> some issues that have shown up in openQA testing.
>
> First, you may have found that trying to update to today's Rawhide
> fails like this:
>
> Error: Transaction test error:
>   file /usr/lib/python3.11/site-packages/ptyprocess-0.7.0-py3.11.egg-info from install of python3-ptyprocess-0.7.0-2.fc38.noarch conflicts with file from package python3-ptyprocess-0.7.0-1.fc38.noarch
>
> That's https://bugzilla.redhat.com/show_bug.cgi?id=2164207 , we figured
> out the problem (thanks to Panu for getting to the root cause of it)
> and it is fixed in python-ptyprocess-0.7.0-3.fc38 , which should be in
> the next Rawhide compose. If you really can't wait you can update
> directly to that version
> from https://koji.fedoraproject.org/koji/buildinfo?buildID=2138649 ,
> but you'd better watch out for issue #2...
>
> The second issue is that systems that update to systemd-253~rc1-1.fc38
> seem to get stuck on boot. With Plymouth enabled you just see the
> splash screen. With it disabled (or by pressing esc) it seems to be
> stuck at "Stopped initrd-switch-root.service - Switch Root.". I'm still
> looking into this one, but it's happened to a lot of openQA tests and I
> was able to confirm it first try in a local VM, by installing from the
> 20230123.n.0 compose then updating systemd and rebooting. Fresh
> installs with the newer systemd seem to be OK, at least most openQA
> tests for the new compose passed - it seems to be only updating an
> existing install that has the problem, at least so far.

And to add to the fun, I think either a mesa update or a glibc update
has broken GNOME and KDE: in current openQA tests, with an updated base
image to avoid the systemd and ptyprocess issues, GNOME and KDE tests
boot fine initially, run an update that includes glibc and mesa, then
reboot and fail to reach the desktop. I'm gonna look into this in more
detail when I get home, but here's the X traceback:

Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE) open /dev/fb0: Permission denied
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (WW) VGA arbiter: cannot open kernel arbiter, no multi-card support
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (II) modeset(0): Creating default Display subsection in Screen section
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]:         "Default Screen Section" for depth/fbbpp 24/32
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (==) modeset(0): Depth 24, (==) framebuffer bpp 32
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (==) modeset(0): RGB weight 888
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (==) modeset(0): Default visual is TrueColor
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (II) Loading sub module "glamoregl"
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (II) LoadModule: "glamoregl"
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (II) Loading /usr/lib64/xorg/modules/libglamoregl.so
Jan 25 13:38:47 fedora abrtd[622]: Skipping /var/spool/abrt/ccpp-2023-01-25-16:38:47.526247-1012.new: directory locked. Is a backtrace being generated?
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (II) Module glamoregl: vendor="X.Org Foundation"
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]:         compiled for 1.20.14, module version = 1.0.1
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]:         ABI class: X.Org ANSI C Emulation, version 0.4
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: free(): double free detected in tcache 2
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE)
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE) Backtrace:
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]:         ABI class: X.Org ANSI C Emulation, version 0.4
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: free(): double free detected in tcache 2
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE)
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE) Backtrace:
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE) 0: /usr/libexec/Xorg (ErrorFSigSafe+0xcc) [0x556438ff496c]
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE) 1: /lib64/libc.so.6 (__sigaction+0x50) [0x7fbae3c3bba0]
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE) 2: /lib64/libc.so.6 (__pthread_kill_implementation+0x114) [0x7fbae3c8cad4]
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE) 3: /lib64/libc.so.6 (gsignal+0x1e) [0x7fbae3c3baee]
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE) 4: /lib64/libc.so.6 (abort+0xdf) [0x7fbae3c2487f]
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE) 5: /lib64/libc.so.6 (__libc_message.cold+0x5) [0x7fbae3c2560f]
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE) 6: /lib64/libc.so.6 (malloc_printerr+0x15) [0x7fbae3c96a05]
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE) 7: /lib64/libc.so.6 (_int_free+0x9e5) [0x7fbae3c98de5]
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE) 8: /lib64/libc.so.6 (__libc_free+0x7e) [0x7fbae3c9b42e]
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE) 9: /usr/lib64/dri/zink_dri.so (__driDriverGetExtensions_zink+0x9e70) [0x7fbad82b8180]
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE) 10: /lib64/libgbm.so.1 (gbm_format_get_name+0xe81) [0x7fbae3229361]
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE) 11: /lib64/libgbm.so.1 (gbm_format_get_name+0x1018) [0x7fbae32294f8]
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE) 12: /lib64/libgbm.so.1 (gbm_format_get_name+0x121a) [0x7fbae32296fa]
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE) unw_get_proc_name failed: no unwind info found [-10]
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE) 13: /lib64/libgbm.so.1 (?+0x0) [0x7fbae3227c5c]
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE) 14: /lib64/libgbm.so.1 (gbm_create_device+0x4e) [0x7fbae3227dae]
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE) 15: /usr/lib64/xorg/modules/libglamoregl.so (glamor_egl_init+0x58) [0x7fbae324f818]
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE) unw_get_proc_name failed: no unwind info found [-10]
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE) 16: /usr/lib64/xorg/modules/drivers/modesetting_drv.so (?+0x0) [0x7fbae328b65b]
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE) 17: /usr/libexec/Xorg (InitOutput+0x121b) [0x556438ed2e4b]
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE) 18: /usr/libexec/Xorg (miPutImage+0xb6d) [0x556438e7449c]
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE) 19: /lib64/libc.so.6 (__libc_start_call_main+0x7a) [0x7fbae3c25b4a]
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE) 20: /lib64/libc.so.6 (__libc_start_main+0x8b) [0x7fbae3c25c0b]
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE) 21: /usr/libexec/Xorg (_start+0x25) [0x556438e752a5]
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE)
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE)
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: Fatal server error:
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE) Caught signal 6 (Aborted). Server aborting
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE)
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE)
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: Please consult the The X.Org Foundation support
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]:          at http://wiki.x.org
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]:  for help.
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE) Please also check the log file at "/var/lib/gdm/.local/share/xorg/Xorg.0.log" for additional information.
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE)
Jan 25 13:38:47 fedora /usr/libexec/gdm-x-session[1040]: (EE) Server terminated with error (1). Closing log file.

so, yeah, today is a great day to go outside, enjoy the
sunshine/snow/rain (delete as appropriate), and *not* update your
Rawhide installation. :D