Python code error in F39, but not in F38

[Ranjan Maitra]

Hi,

I was running the code here:
https://github.com/maitra/Visual-Information-Fidelity---Python3

However, the code runs in F38 (python 3.11) without error, but not in F39 (python 3.12)
where it
ends with a segmentation fault (after doing the calculations).

Both, however, give an answer that are not the same after the 14th
decimal place. Not sure if I should be bothered about that.

Looking at valgrind, I can not tell if there is an error in the python call itself.  Here
is what I get (note that the code calls shared objects created by the gcc compiler).

==278781== Invalid read of size 8
==278781==    at 0x4A50B5B: UnknownInlinedFun (pycore_pystate.h:118)
==278781==    by 0x4A50B5B: UnknownInlinedFun (obmalloc.c:866)
==278781==    by 0x4A50B5B: _PyObject_Free (obmalloc.c:1850)
==278781==    by 0x6526E92E: UnknownInlinedFun (siplib.c:2241)
==278781==    by 0x6526E92E: UnknownInlinedFun (objmap.c:69)
==278781==    by 0x6526E92E: finalise (siplib.c:2143)
==278781==    by 0x49AC4D3: UnknownInlinedFun (pylifecycle.c:3018)
==278781==    by 0x49AC4D3: Py_FinalizeEx.cold (pylifecycle.c:1977)
==278781==    by 0x4B22336: Py_RunMain (main.c:691)
==278781==    by 0x4ADD85B: Py_BytesMain (main.c:743)
==278781==    by 0x4EE1149: (below main) (libc_start_call_main.h:58)
==278781==  Address 0x10 is not stack'd, malloc'd or (recently) free'd
==278781==
==278781==
==278781== Process terminating with default action of signal 11 (SIGSEGV): dumping core
==278781==  Access not within mapped region at address 0x10
==278781==    at 0x4A50B5B: UnknownInlinedFun (pycore_pystate.h:118)
==278781==    by 0x4A50B5B: UnknownInlinedFun (obmalloc.c:866)
==278781==    by 0x4A50B5B: _PyObject_Free (obmalloc.c:1850)
==278781==    by 0x6526E92E: finalise (siplib.c:2143)
==278781==    by 0x49AC4D3: UnknownInlinedFun (pylifecycle.c:3018)
==278781==    by 0x49AC4D3: Py_FinalizeEx.cold (pylifecycle.c:1977)
==278781==    by 0x4B22336: Py_RunMain (main.c:691)
==278781==    by 0x4ADD85B: Py_BytesMain (main.c:743)
==278781==    by 0x4EE1149: (below main) (libc_start_call_main.h:58)
==278781==  If you believe this happened as a result of a stack
==278781==  overflow in your program's main thread (unlikely but
==278781==  possible), you can try to increase the size of the
==278781==  main thread stack using the --main-stacksize= flag.
==278781==  The main thread stack size used in this run was 8388608.
==278781==
==278781== HEAP SUMMARY:
==278781==     in use at exit: 18,046,811 bytes in 92,137 blocks
==278781==   total heap usage: 2,212,074 allocs, 2,119,937 frees, 1,983,813,202 bytes
allocated
==278781==
==278781== LEAK SUMMARY:
==278781==    definitely lost: 2,160,027 bytes in 41,915 blocks
==278781==    indirectly lost: 112 bytes in 2 blocks
==278781==      possibly lost: 5,021,632 bytes in 34,430 blocks
==278781==    still reachable: 10,863,024 bytes in 15,769 blocks
==278781==         suppressed: 0 bytes in 0 blocks



It is possible that there is a bug in the code itself, but nothing above
points to my created code.

Any suggestions? Or is this a bug?

Many thanks and best wishes,
Ranjan

[Barry]

>  On 19 Nov 2023, at 16:26, Ranjan Maitra via users
> <users(a)lists.fedoraproject.org&gt; wrote:
>  
>  Hi,
>  
>  I was running the code here:
> https://github.com/maitra/Visual-Information-Fidelity---Python3
>  
>  However, the code runs in F38 (python 3.11) without error, but not in F39 (python 3.12)
> where it
>  ends with a segmentation fault (after doing the calculations).
>  
>  Both, however, give an answer that are not the same after the 14th
>  decimal place. Not sure if I should be bothered about that.
>  
>  Looking at valgrind, I can not tell if there is an error in the python call itself.  Here
> is what I get (note that the code calls shared objects created by the gcc compiler).
>  
>  ==278781== Invalid read of size 8
>  ==278781==    at 0x4A50B5B: UnknownInlinedFun (pycore_pystate.h:118)
>  ==278781==    by 0x4A50B5B: UnknownInlinedFun (obmalloc.c:866)
>  ==278781==    by 0x4A50B5B: _PyObject_Free (obmalloc.c:1850)
>  ==278781==    by 0x6526E92E: UnknownInlinedFun (siplib.c:2241)
>  ==278781==    by 0x6526E92E: UnknownInlinedFun (objmap.c:69)
>  ==278781==    by 0x6526E92E: finalise (siplib.c:2143)
>  ==278781==    by 0x49AC4D3: UnknownInlinedFun (pylifecycle.c:3018)
>  ==278781==    by 0x49AC4D3: Py_FinalizeEx.cold (pylifecycle.c:1977)
>  ==278781==    by 0x4B22336: Py_RunMain (main.c:691)
>  ==278781==    by 0x4ADD85B: Py_BytesMain (main.c:743)
>  ==278781==    by 0x4EE1149: (below main) (libc_start_call_main.h:58)
>  ==278781==  Address 0x10 is not stack'd, malloc'd or (recently) free'd
>  ==278781==
>  ==278781==
>  ==278781== Process terminating with default action of signal 11 (SIGSEGV): dumping core
>  ==278781==  Access not within mapped region at address 0x10
>  ==278781==    at 0x4A50B5B: UnknownInlinedFun (pycore_pystate.h:118)
>  ==278781==    by 0x4A50B5B: UnknownInlinedFun (obmalloc.c:866)
>  ==278781==    by 0x4A50B5B: _PyObject_Free (obmalloc.c:1850)
>  ==278781==    by 0x6526E92E: finalise (siplib.c:2143)
>  ==278781==    by 0x49AC4D3: UnknownInlinedFun (pylifecycle.c:3018)
>  ==278781==    by 0x49AC4D3: Py_FinalizeEx.cold (pylifecycle.c:1977)
>  ==278781==    by 0x4B22336: Py_RunMain (main.c:691)
>  ==278781==    by 0x4ADD85B: Py_BytesMain (main.c:743)
>  ==278781==    by 0x4EE1149: (below main) (libc_start_call_main.h:58)
>  ==278781==  If you believe this happened as a result of a stack
>  ==278781==  overflow in your program's main thread (unlikely but
>  ==278781==  possible), you can try to increase the size of the
>  ==278781==  main thread stack using the --main-stacksize= flag.
>  ==278781==  The main thread stack size used in this run was 8388608.
>  ==278781==
>  ==278781== HEAP SUMMARY:
>  ==278781==     in use at exit: 18,046,811 bytes in 92,137 blocks
>  ==278781==   total heap usage: 2,212,074 allocs, 2,119,937 frees, 1,983,813,202 bytes
> allocated
>  ==278781==
>  ==278781== LEAK SUMMARY:
>  ==278781==    definitely lost: 2,160,027 bytes in 41,915 blocks
>  ==278781==    indirectly lost: 112 bytes in 2 blocks
>  ==278781==      possibly lost: 5,021,632 bytes in 34,430 blocks
>  ==278781==    still reachable: 10,863,024 bytes in 15,769 blocks
>  ==278781==         suppressed: 0 bytes in 0 blocks
>  
>  
>  
>  It is possible that there is a bug in the code itself, but nothing above
>  points to my created code.
>  
>  Any suggestions? Or is this a bug? 
Are you using C extensions in your code? If so I would suspect them for corrupting python.
I think you can get into gdb inside the core file with coredumpctl debug.
What does “info shared” tell you about what code has been loaded into memory?
It is also worth trying to find the type of the object that is being deleted,
_PyObject_Free will have pointer to the object.
Also not sure where siplib.c is from is it python or an extension?

I am assuming here that you are familiar with debugging with gdb.

Barry

>  
>  Many thanks and best wishes,
>  Ranjan
>  
>  --
>  _______________________________________________
>  users mailing list -- users(a)lists.fedoraproject.org
>  To unsubscribe send an email to users-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]
>  Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue

[Ranjan Maitra]

On Sun Nov19'23 05:35:19PM, Barry wrote:
>  From: Barry <barry(a)barrys-emacs.org&gt;
>  Date: Sun, 19 Nov 2023 17:35:19 +0000
>  To: Community support for Fedora users <users(a)lists.fedoraproject.org&gt;
>
>  >
>  >
>  > It is possible that there is a bug in the code itself, but nothing above
>  > points to my created code.
>  >
>  > Any suggestions? Or is this a bug?
>
>  Are you using C extensions in your code? If so I would suspect them for corrupting
> python. I think you can get into gdb inside the core file with coredumpctl debug. 
The python code calls a shared object library that is written in C. But
why does this core dump only happen in F39 (python 3.12) and not with
F38 (python 3.11)?

>  What does “info shared” tell you about what code has been loaded into
> memory? 
This would be under gdb, I presume.


>  It is also worth trying to find the type of the object that is being
> deleted, _PyObject_Free will have pointer to the object.
>  Also not sure where siplib.c is from is it python or an extension? 
Thanks, I don't know either, but there is no siplib.c anywhere on either
of my two systems.

>  I am assuming here that you are familiar with debugging with gdb.
Thanks, I am familiar with using gdb for a C program, never used it with
python code calling a C shared object.

I tried:

$ gdb python "./test.py ultadanga.tiff ultadanga-64-rgb.png 1 7"
GNU gdb (Fedora Linux) 13.2-10.fc39
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>;.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>;.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from python...

This GDB supports auto-downloading debuginfo from the following URLs:
  <https://debuginfod.fedoraproject.org/>
Enable debuginfod for this session? (y or [n]) y
Debuginfod has been enabled.
To make this setting permanent, add 'set debuginfod enabled on' to .gdbinit.
Reading symbols from /home/localuser/.cache/debuginfod_client/07023eb4f297ae1d4591e--Type
<RET> for more, q to quit, c to continue without paging--
c808ab20b2788c542be/debuginfo...
./test.py ultadanga.tiff ultadanga-64-rgb.png 1 7: No such file or directory.
(gdb)


Not quite sure how to proceed here.

Many thanks again, and best wishes,
Ranjan

[Neal Becker]

Start gdb python.  Set a breakpoint in the shared module.  continue.

On Sun, Nov 19, 2023 at 1:02 PM Ranjan Maitra via users <
users(a)lists.fedoraproject.org&gt; wrote:

>  On Sun Nov19'23 05:35:19PM, Barry wrote:
>  > From: Barry <barry(a)barrys-emacs.org&gt;
>  > Date: Sun, 19 Nov 2023 17:35:19 +0000
>  > To: Community support for Fedora users <users(a)lists.fedoraproject.org&gt;
>   >
>  > >
>  > >
>  > > It is possible that there is a bug in the code itself, but nothing
>  above
>  > > points to my created code.
>  > >
>  > > Any suggestions? Or is this a bug?
>  >
>  > Are you using C extensions in your code? If so I would suspect them for
>  corrupting python. I think you can get into gdb inside the core file with
>  coredumpctl debug.
>
>  The python code calls a shared object library that is written in C. But
>  why does this core dump only happen in F39 (python 3.12) and not with
>  F38 (python 3.11)?
>
>  > What does “info shared” tell you about what code has been loaded into
>  memory?
>
>  This would be under gdb, I presume.
>
>
>  > It is also worth trying to find the type of the object that is being
>  deleted, _PyObject_Free will have pointer to the object.
>  > Also not sure where siplib.c is from is it python or an extension?
>
>  Thanks, I don't know either, but there is no siplib.c anywhere on either
>  of my two systems.
>
>  > I am assuming here that you are familiar with debugging with gdb.
>
>  Thanks, I am familiar with using gdb for a C program, never used it with
>  python code calling a C shared object.
>
>  I tried:
>
>  $ gdb python "./test.py ultadanga.tiff ultadanga-64-rgb.png 1 7"
>  GNU gdb (Fedora Linux) 13.2-10.fc39
>  Copyright (C) 2023 Free Software Foundation, Inc.
>  License GPLv3+: GNU GPL version 3 or later <
>  http://gnu.org/licenses/gpl.html>
>  This is free software: you are free to change and redistribute it.
>  There is NO WARRANTY, to the extent permitted by law.
>  Type "show copying" and "show warranty" for details.
>  This GDB was configured as "x86_64-redhat-linux-gnu".
>  Type "show configuration" for configuration details.
>  For bug reporting instructions, please see:
>  <https://www.gnu.org/software/gdb/bugs/>;.
>  Find the GDB manual and other documentation resources online at:
>      <http://www.gnu.org/software/gdb/documentation/>;.
>
>  For help, type "help".
>  Type "apropos word" to search for commands related to "word"...
>  Reading symbols from python...
>
>  This GDB supports auto-downloading debuginfo from the following URLs:
>    <https://debuginfod.fedoraproject.org/>
>  Enable debuginfod for this session? (y or [n]) y
>  Debuginfod has been enabled.
>  To make this setting permanent, add 'set debuginfod enabled on' to
>  .gdbinit.
>  Reading symbols from
>  /home/localuser/.cache/debuginfod_client/07023eb4f297ae1d4591e--Type <RET>
>  for more, q to quit, c to continue without paging--
>  c808ab20b2788c542be/debuginfo...
>  ./test.py ultadanga.tiff ultadanga-64-rgb.png 1 7: No such file or
>  directory.
>  (gdb)
>
>
>  Not quite sure how to proceed here.
>
>  Many thanks again, and best wishes,
>  Ranjan
>
>  --
>  _______________________________________________
>  users mailing list -- users(a)lists.fedoraproject.org
>  To unsubscribe send an email to users-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>  Do not reply to spam, report it:
>  https://pagure.io/fedora-infrastructure/new_issue
>  

-- 
*Those who don't understand recursion are doomed to repeat it*

[Samuel Sieb]

On 11/19/23 10:01, Ranjan Maitra via users wrote:
>  I tried:
>  
>  $ gdb python "./test.py ultadanga.tiff ultadanga-64-rgb.png 1 7"
>  (gdb) 
$ gdb python
(gdb) run ./test.py ultadanga.tiff ultadanga-64-rgb.png 1 7

[Ranjan Maitra]

On Tue Nov21'23 05:25:35PM, Samuel Sieb wrote:
>  From: Samuel Sieb <samuel(a)sieb.net&gt;
>  Date: Tue, 21 Nov 2023 17:25:35 -0800
>  To: users(a)lists.fedoraproject.org
>  Reply-To: Community support for Fedora users <users(a)lists.fedoraproject.org&gt;
>  Subject: Re: Python code error in F39, but not in F38
>
>  On 11/19/23 10:01, Ranjan Maitra via users wrote:
>  > I tried:
>  >
>  > $ gdb python "./test.py ultadanga.tiff ultadanga-64-rgb.png 1 7"
>  > (gdb)
>
>  $ gdb python
>  (gdb) run ./test.py ultadanga.tiff ultadanga-64-rgb.png 1 7
>  -- 
Thanks very much for this for explaining how to run gdb on a python script! On Python 3.12
(Fedora 39), I get, at the very end:

 Thread 1 "python" received signal SIGSEGV, Segmentation fault.
 0x00007ffff79e4b5b in _PyInterpreterState_GET ()
     at
/usr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Include/internal/pycore_pystate.h:118
 118         return tstate->interp;
 (gdb)

On Python 3.11 (Fedora 38), I get, at the very end:

 [Inferior 1 (process 199650) exited normally]
 (gdb)

Any suggestions as to what is going wrong here?

Many thanks and best wishes,
Ranjan








> _______________________________________________
> users mailing list -- users(a)lists.fedoraproject.org
> To unsubscribe send an email to users-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]
> Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue

[Barry Scott]

>  On 22 Nov 2023, at 20:16, Ranjan Maitra via users
> <users(a)lists.fedoraproject.org&gt; wrote:
>  
>  Thanks very much for this for explaining how to run gdb on a python script! On Python
> 3.12 (Fedora 39), I get, at the very end:
>  
>  Thread 1 "python" received signal SIGSEGV, Segmentation fault.
>  0x00007ffff79e4b5b in _PyInterpreterState_GET ()
>      at
> /usr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Include/internal/pycore_pystate.h:118
>  118         return tstate->interp;
>  (gdb)
>  
>  On Python 3.11 (Fedora 38), I get, at the very end:
>  
>  [Inferior 1 (process 199650) exited normally]
>  (gdb)
>  
>  Any suggestions as to what is going wrong here? 
At this point you need get more information.

What do the folllowing GDB commands print?

p tstate
p tstate->interp
thread allow all bt

Barry

[Barry Scott]

>  On 22 Nov 2023, at 21:32, Barry Scott <barry(a)barrys-emacs.org&gt;
> wrote:
>  
> > Any suggestions as to what is going wrong here?
>  
>  At this point you need get more information.
>  
>  What do the folllowing GDB commands print?
>  
>  p tstate
>  p tstate->interp
>  thread allow all bt 
Doh...

	thread apply all bt

I always type:
	t a a bt

and typo'ed the expansion.

Barry

[Ranjan Maitra]

On Wed Nov22'23 09:49:49PM, Barry Scott wrote:
>  From: Barry Scott <barry(a)barrys-emacs.org&gt;
>  Date: Wed, 22 Nov 2023 21:49:49 +0000
>  To: Community support for Fedora users <users(a)lists.fedoraproject.org&gt;
>  CC: Ranjan Maitra <mlmaitra(a)gmx.com&gt;
>  Reply-To: Community support for Fedora users <users(a)lists.fedoraproject.org&gt;
>  Subject: Re: Python code error in F39, but not in F38
>
>
>
>  > On 22 Nov 2023, at 21:32, Barry Scott <barry(a)barrys-emacs.org&gt; wrote:
>  >
>  >> Any suggestions as to what is going wrong here?
>  >
>  > At this point you need get more information.
>  >
>  > What do the folllowing GDB commands print?
>  >
>  > p tstate
>  > p tstate->interp
>  > thread allow all bt
>  Doh...
>
>  	thread apply all bt
>
>  I always type:
>  	t a a bt
>
>  and typo'ed the expansion. 
Thanks, here is what comes out:

 (gdb)  p tstate->interp

 value has been optimized out
 (gdb) thread apply all bt


 Thread 1 (Thread 0x7ffff7ec7740 (LWP 387946) "python"):
 #0  0x00007ffff79e4b5b in _PyInterpreterState_GET () at
/usr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Include/internal/pycore_pystate.h:118
 #1  get_state () at
/usr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Objects/obmalloc.c:866
 #2  _PyObject_Free (ctx=<optimized out>, p=0x555556d898a0) at
/usr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Objects/obmalloc.c:1850
 #3  0x00007fff9e5e292f in sip_api_free (mem=<optimized out>) at
/usr/src/debug/sip-4.19.25-9.fc39.x86_64/siplib/siplib.c:2239
 #4  sipOMFinalise (om=0x7fff9e5ff9a0 <cppPyMap.lto_priv.0>) at
/usr/src/debug/sip-4.19.25-9.fc39.x86_64/susr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Python/pylifecycle.c:1977
 #8  0x00007ffff7ab6337 in Py_RunMain () at
/usr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Modules/main.c:691
 #9  0x00007ffff7a7185c in Py_BytesMain (argc=<optimized out>, argv=<optimized
out>) at /usr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Modules/main.c:743
 #10 0x00007ffff764614a in __libc_start_call_main (main=main@entry=0x555555555160
 --Type <RET> for more, q to quit, c to continue without paging--

 <main>, argc=argc@entry=4, argv=argv@entry=0x7fffffffdfd8) at
../sysdeps/nptl/libc_start_call_main.h:58
 #11 0x00007ffff764620b in __libc_start_main_impl (main=0x555555555160 <main>,
argc=4, argv=0x7fffffffdfd8, init=<optimized out>, fini=<optimized out>,
rtld_fini=<optimized out>, stack_end=0x7fffffffdfc8) at ../csu/libc-start.c:360
 #12 0x0000555555555095 in _start ()
(gdb)

Many thanks and best wishes,
Ranjan

[Samuel Sieb]

On 11/22/23 14:20, Ranjan Maitra via users wrote:
>  On Wed Nov22'23 09:49:49PM, Barry Scott wrote:
> > From: Barry Scott <barry(a)barrys-emacs.org&gt;
> > Date: Wed, 22 Nov 2023 21:49:49 +0000
> >> At this point you need get more information.
> >>
> >> What do the folllowing GDB commands print?
> >>
> >> p tstate
> >> p tstate->interp
> >> thread allow all bt
> > Doh...
> >
> > 	thread apply all bt
> >
> > I always type:
> > 	t a a bt
> >
> > and typo'ed the expansion.
>  
>  Thanks, here is what comes out:
>  
>    (gdb)  p tstate->interp
>  
>    value has been optimized out
>    (gdb) thread apply all bt 
You missed the first one: "p tstate".

[Ranjan Maitra]

On Wed Nov22'23 02:36:35PM, Samuel Sieb wrote:
>  From: Samuel Sieb <samuel(a)sieb.net&gt;
>  Date: Wed, 22 Nov 2023 14:36:35 -0800
>  To: users(a)lists.fedoraproject.org
>  Reply-To: Community support for Fedora users <users(a)lists.fedoraproject.org&gt;
>  Subject: Re: Python code error in F39, but not in F38
>
>  On 11/22/23 14:20, Ranjan Maitra via users wrote:
>  > On Wed Nov22'23 09:49:49PM, Barry Scott wrote:
>  > > From: Barry Scott <barry(a)barrys-emacs.org&gt;
>  > > Date: Wed, 22 Nov 2023 21:49:49 +0000
>  > > > At this point you need get more information.
>  > > >
>  > > > What do the folllowing GDB commands print?
>  > > >
>  > > > p tstate
>  > > > p tstate->interp
>  > > > thread allow all bt
>  > > Doh...
>  > >
>  > > 	thread apply all bt
>  > >
>  > > I always type:
>  > > 	t a a bt
>  > >
>  > > and typo'ed the expansion.
>  >
>  > Thanks, here is what comes out:
>  >
>  >   (gdb)  p tstate->interp
>  >
>  >   value has been optimized out
>  >   (gdb) thread apply all bt
>
>  You missed the first one: "p tstate".
>  
My apologies, I did not include this in my cut-and-paste:

(gdb) p tstate

$1 = <optimized out>

Sorry about that. All this in F39 (python 3.12, of course).

Ranjan

[Barry]

>  On 22 Nov 2023, at 22:21, Ranjan Maitra via users
> <users(a)lists.fedoraproject.org&gt; wrote:
>  
>  #4  sipOMFinalise (om=0x7fff9e5ff9a0 <cppPyMap.lto_priv.0>) at /usr/src/debug/
It is a bug in the sip code is my strong expectation.
I would also guess that tstate is likely 0 and causing a SEGV.
What did sip do to damage the tstate (thread state I think).

Barry

[Ranjan Maitra]

On Thu Nov23'23 08:28:36AM, Barry wrote:
>  From: Barry <barry(a)barrys-emacs.org&gt;
>  Date: Thu, 23 Nov 2023 08:28:36 +0000
>  To: Community support for Fedora users <users(a)lists.fedoraproject.org&gt;
>  CC: Ranjan Maitra <mlmaitra(a)gmx.com&gt;
>  Reply-To: Community support for Fedora users <users(a)lists.fedoraproject.org&gt;
>  Subject: Re: Python code error in F39, but not in F38
>
>
>
>  > On 22 Nov 2023, at 22:21, Ranjan Maitra via users
> <users(a)lists.fedoraproject.org&gt; wrote:
>  >
>  > #4  sipOMFinalise (om=0x7fff9e5ff9a0 <cppPyMap.lto_priv.0>) at
> /usr/src/debug/
>
>  It is a bug in the sip code is my strong expectation.
>  I would also guess that tstate is likely 0 and causing a SEGV.
>  What did sip do to damage the tstate (thread state I think).
>
>  Barry 
From another thread:

(gdb) p tstate

$1 = <optimized out>

Sorry, I forgot this (also posted on another sub-thread).

And here are the complete valgrind errors from

valgrind --leak-check=full  --track-origins=yes --show-leak-kinds=all python ./test.py
~/Datasets/Images/tiffs/ultadanga.tiff ../results/ultadanga-64-rgb.png 1 7

at:

https://raw.githubusercontent.com/maitra/Visual-Information-Fidelity---Py...

One aspect to note is that the C code (not by me) is from 1988, though last updated in
1997. It is not clear to me if that is a plus (because programmers had the time to be more
careful in those days) or a minus (because more sophisticated environments are available
now). I do compile with -Wall -pedantic and there are no complaints. However, that does
not mean that there can not be some memory leak or something else that has stayed dormant
all these years and have surfaced with the changes to Python 3.12.

Many thanks and best wishes,
Ranjan

>
> --
> _______________________________________________
> users mailing list -- users(a)lists.fedoraproject.org
> To unsubscribe send an email to users-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]
> Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue

[Barry]

>  On 23 Nov 2023, at 14:18, Ranjan Maitra via users
> <users(a)lists.fedoraproject.org&gt; wrote:
>  
>  One aspect to note is that the C code (not by me) is from 1988, though last updated in
> 1997. It is not clear to me if that is a plus (because programmers had the time to be more
> careful in those days) or a minus (because more sophisticated environments are available
> now). I do compile with -Wall -pedantic and there are no complaints. However, that does
> not mean that there can not be some memory leak or something else that has stayed dormant
> all these years and have surfaced with the changes to Python 3.12. 
Old code is bug free is it? News to me :-)
Schedule pressure was a thing in 1997 as well as every year!

It is pointless to speculate, just debug what you have in front of you.

Barry

[Roger Heflin]

I have seen the bug in old code that a new compiler
optimization/library fix exposed.

I have seen code unload a loadable library and then turn around and
call a function in the unloaded library (it worked since the unload
was a NOOP), but broke when the vendor removed/fixed  the NOOP and
made the unload work like it should have always  worked.   Developers
argued with the vendor for *MONTHS* that their code had worked for
years and on 3 different unixes (all being lazy and NOOPing).     It
is not clear to me how anyone is stupid enough to argue that the calls
they asked to be unloaded should work after the unload, but they did
argue that for months.

I have seen fortran code that "worked" for years.  The Intel compiler
broke it by putting fortran const's in a read-only segment and they
were passing a const (in fortran it is not allowed to be changed) into
a subroutine and changing it.    The professor was yelling at my
company for similar reasons (the code has worked for years it cannot
be the code, it is this crappy compiler you sold us).

I also previously worked for a company that compiled their code with 3
different bought linux compilers, and a SGI and Sun compiler and
examined the warnings from all and resolved them and found that
warnings on one or the other compiler would flag something and upon
examination of the code they often could not understand why the code
"appeared" work well enough even with what was clearly a bug in the
code.

Just because your code "appears" to work in your test case says little
or nothing about the code being "perfect".   I have found significant
bugs that made me question how the code was returning correct results
too many times.

Never assume that old code is right.  there are many ways for it to be
wrong and still return correct results until someone else fixes their
external code that was allowing your bad code to work.

On Thu, Nov 23, 2023 at 5:15 PM Barry <barry(a)barrys-emacs.org&gt; wrote:
>
>
>
> > On 23 Nov 2023, at 14:18, Ranjan Maitra via users
<users(a)lists.fedoraproject.org&gt; wrote:
> >
> > One aspect to note is that the C code (not by me) is from 1988, though last
updated in 1997. It is not clear to me if that is a plus (because programmers had the time
to be more careful in those days) or a minus (because more sophisticated environments are
available now). I do compile with -Wall -pedantic and there are no complaints. However,
that does not mean that there can not be some memory leak or something else that has
stayed dormant all these years and have surfaced with the changes to Python 3.12.
>
> Old code is bug free is it? News to me :-)
> Schedule pressure was a thing in 1997 as well as every year!
>
> It is pointless to speculate, just debug what you have in front of you.
>
> Barry
>
> --
> _______________________________________________
> users mailing list -- users(a)lists.fedoraproject.org
> To unsubscribe send an email to users-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]
> Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue

[George N. White III]

On Fri, Nov 24, 2023 at 5:01 PM Roger Heflin <rogerheflin(a)gmail.com&gt; wrote:

>  I have seen the bug in old code that a new compiler
>  optimization/library fix exposed.
>  [...]
>  I also previously worked for a company that compiled their code with 3
>  different bought linux compilers, and a SGI and Sun compiler and
>  examined the warnings from all and resolved them and found that
>  warnings on one or the other compiler would flag something and upon
>  examination of the code they often could not understand why the code
>  "appeared" work well enough even with what was clearly a bug in the
>  code.
>
>  Just because your code "appears" to work in your test case says little
>  or nothing about the code being "perfect".   I have found significant
>  bugs that made me question how the code was returning correct results
>  too many times.
>
>  Never assume that old code is right.  there are many ways for it to be
>  wrong and still return correct results until someone else fixes their
>  external code that was allowing your bad code to work. 

There are some codes where there are ways to cross-check a computation,
but others where you have to rely on everything being done correctly.
For 25 years my job included maintaining and porting a Fortran code that
started on a CDC Cyber and did not have ways to cross-check the results, so
I relied on test data sets.  We were able to get test data to produce
bit-for-bit
identical results across linux, Windows, and macOS, but, especially in the
early
years, often required some effort to install toolchains that didn't have
bugs,
including building GCC with upstream bug fixes that hadn't yet made it into
the versions available on user systems. In recent years most of the work
has
been finding replacements for libraries that have been dropped from linux
distros.

-- 
George N. White III

[Ranjan Maitra]

Hi,

Sorry to bring this back up, but I can not find any error yet in the legacy C code.

gdb with python 3.11 gives no error (on F38, program exited normally) but gdb with python
3.12 segfaults.

On Thu Nov23'23 08:18:05AM, Community Support for Fedora Users wrote:
>  From: Ranjan Maitra via users <users(a)lists.fedoraproject.org&gt;
>  Date: Thu, 23 Nov 2023 08:18:05 -0600
>  To: Community support for Fedora users <users(a)lists.fedoraproject.org&gt;
>  Cc: Ranjan Maitra <mlmaitra(a)gmx.com&gt;
>  Reply-To: Community support for Fedora users <users(a)lists.fedoraproject.org&gt;
>  Subject: Re: Python code error in F39, but not in F38
>
>  On Thu Nov23'23 08:28:36AM, Barry wrote:
>  > From: Barry <barry(a)barrys-emacs.org&gt;
>  > Date: Thu, 23 Nov 2023 08:28:36 +0000
>  > To: Community support for Fedora users <users(a)lists.fedoraproject.org&gt;
>  > CC: Ranjan Maitra <mlmaitra(a)gmx.com&gt;
>  > Reply-To: Community support for Fedora users <users(a)lists.fedoraproject.org&gt;
>  > Subject: Re: Python code error in F39, but not in F38
>  >
>  >
>  >
>  > > On 22 Nov 2023, at 22:21, Ranjan Maitra via users
> <users(a)lists.fedoraproject.org&gt; wrote:
>  > >
>  > > #4  sipOMFinalise (om=0x7fff9e5ff9a0 <cppPyMap.lto_priv.0>) at
> /usr/src/debug/
>  >
>  > It is a bug in the sip code is my strong expectation. 
So, it seems, so is this a bug that needs to be reported? Where to? I am not even sure how
to build a small reproducible example because I do not understand what part of the C or
the python code is resulting in this bug. All the files referenced in the valgrind output
are in the system(?) python files. The code files do not even appear to show up.

>  > I would also guess that tstate is likely 0 and causing a SEGV.
The segfault is here (per gdb):

    at
/usr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Include/internal/pycore_pystate.h:118
118         return tstate->interp;

>  > What did sip do to damage the tstate (thread state I think).
I don't know what sip is, and how it comes into that code.

Any suggestions on what I should do?

Many thanks and best wishes,
Ranjan


> >
> > Barry
>
> From another thread:
>
> (gdb) p tstate
>
> $1 = <optimized out>
>
> Sorry, I forgot this (also posted on another sub-thread).
>
> And here are the complete valgrind errors from
>
> valgrind --leak-check=full  --track-origins=yes --show-leak-kinds=all python
./test.py ~/Datasets/Images/tiffs/ultadanga.tiff ../results/ultadanga-64-rgb.png 1 7
>
> at:
>
>
https://raw.githubusercontent.com/maitra/Visual-Information-Fidelity---Py...
>
> One aspect to note is that the C code (not by me) is from 1988, though last updated
in 1997. It is not clear to me if that is a plus (because programmers had the time to be
more careful in those days) or a minus (because more sophisticated environments are
available now). I do compile with -Wall -pedantic and there are no complaints. However,
that does not mean that there can not be some memory leak or something else that has
stayed dormant all these years and have surfaced with the changes to Python 3.12.
>
> Many thanks and best wishes,
> Ranjan
>
> >
> > --
> > _______________________________________________
> > users mailing list -- users(a)lists.fedoraproject.org
> > To unsubscribe send an email to users-leave(a)lists.fedoraproject.org
> > Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]
> > Do not reply to spam, report it:
https://pagure.io/fedora-infrastructure/new_issue
> --
> _______________________________________________
> users mailing list -- users(a)lists.fedoraproject.org
> To unsubscribe send an email to users-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]
> Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue

[Barry Scott]

>  On 1 Dec 2023, at 03:44, Ranjan Maitra via users
> <users(a)lists.fedoraproject.org&gt; wrote:
>  
>  So, it seems, so is this a bug that needs to be reported? 
Start by reporting on fedora bugzilla against the sip package with all the info you have
collected in this email thread.

Barry

[Ranjan Maitra]

On Sat Dec02'23 09:34:08AM, Barry Scott wrote:
>  From: Barry Scott <barry(a)barrys-emacs.org&gt;
>  Date: Sat, 2 Dec 2023 09:34:08 +0000
>  To: Community support for Fedora users <users(a)lists.fedoraproject.org&gt;
>  CC: Ranjan Maitra <mlmaitra(a)gmx.com&gt;
>  Reply-To: Community support for Fedora users <users(a)lists.fedoraproject.org&gt;
>  Subject: Re: Python code error in F39, but not in F38
>
>
>
>  > On 1 Dec 2023, at 03:44, Ranjan Maitra via users
> <users(a)lists.fedoraproject.org&gt; wrote:
>  >
>  > So, it seems, so is this a bug that needs to be reported?
>
>  Start by reporting on fedora bugzilla against the sip package with all the info you have
> collected in this email thread.
>  
Thank you! Interestingly enough, the sip or sip6 packages were not even installed on my
system, (though I should add that installing either made no difference).

However, the following:
/usr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Include/internal/pycore_pystate.h:118 file
is where the segfault is, and that is in python3-devel. Perhaps  I just report it there
and see where that gets me.

Thanks very much again!

Best wishes,
Ranjan

[Barry Scott]

>  On 2 Dec 2023, at 15:24, Ranjan Maitra via users
> <users(a)lists.fedoraproject.org&gt; wrote:
>  
>  However, the following:
> /usr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Include/internal/pycore_pystate.h:118 file
> is where the segfault is, and that is in python3-devel. Perhaps  I just report it there
> and see where that gets me. 
That the victim not the cause.

Its in here that you need to report the problem

#3  0x00007fff9e5e292f in sip_api_free (mem=<optimized out>) at
/usr/src/debug/sip-4.19.25-9.fc39.x86_64/siplib/siplib.c:2239
#4  sipOMFinalise (om=0x7fff9e5ff9a0 <cppPyMap.lto_priv.0>) at
/usr/src/debug/sip-4.19.25-9.fc39.x86_64/susr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Python/pylifecycle.c:1977

It's this "sip" that you need to report against.

Not sure what you mean by it was not installed, its in the back trace.

Barry

[Ranjan Maitra]

On Sun Dec03'23 01:47:46PM, Barry Scott wrote:
>  From: Barry Scott <barry(a)barrys-emacs.org&gt;
>  Date: Sun, 3 Dec 2023 13:47:46 +0000
>  To: Community support for Fedora users <users(a)lists.fedoraproject.org&gt;
>  CC: Ranjan Maitra <mlmaitra(a)gmx.com&gt;
>  Reply-To: Community support for Fedora users <users(a)lists.fedoraproject.org&gt;
>  Subject: Re: Python code error in F39, but not in F38
>
>
>
>  > On 2 Dec 2023, at 15:24, Ranjan Maitra via users
> <users(a)lists.fedoraproject.org&gt; wrote:
>  >
>  > However, the following:
> /usr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Include/internal/pycore_pystate.h:118 file
> is where the segfault is, and that is in python3-devel. Perhaps  I just report it there
> and see where that gets me.
>
>  That the victim not the cause.
>
>  Its in here that you need to report the problem
>
>  #3  0x00007fff9e5e292f in sip_api_free (mem=<optimized out>) at
> /usr/src/debug/sip-4.19.25-9.fc39.x86_64/siplib/siplib.c:2239
>  #4  sipOMFinalise (om=0x7fff9e5ff9a0 <cppPyMap.lto_priv.0>) at
> /usr/src/debug/sip-4.19.25-9.fc39.x86_64/susr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Python/pylifecycle.c:1977
>
>  It's this "sip" that you need to report against.
>
>  Not sure what you mean by it was not installed, its in the back trace. 
Thanks, I never installed it earlier (and now I installed sip6 -- see other subthread),
The issue is still the same, though. That is what I meant.


>  Barry
>
>  
>  --
>  _______________________________________________
>  users mailing list -- users(a)lists.fedoraproject.org
>  To unsubscribe send an email to users-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]
>  Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue

[Jeffrey Walton]

On Wed, Nov 22, 2023 at 5:21 PM Ranjan Maitra via users
<users(a)lists.fedoraproject.org&gt; wrote:
>  On Wed Nov22'23 09:49:49PM, Barry Scott wrote:
>  > [...]
>  > > On 22 Nov 2023, at 21:32, Barry Scott <barry(a)barrys-emacs.org&gt; wrote:
>  > >
>  > >> Any suggestions as to what is going wrong here?
>  > >
>  > > At this point you need get more information.
>  > >
>  > > What do the folllowing GDB commands print?
>  > >
>  > > p tstate
>  > > p tstate->interp
>  > > thread allow all bt
>  > Doh...
>  >
>  >       thread apply all bt
>  >
>  > I always type:
>  >       t a a bt
>  >
>  > and typo'ed the expansion.
>
>  Thanks, here is what comes out:
>
>   (gdb)  p tstate->interp
>
>   value has been optimized out
>   (gdb) thread apply all bt
>
>   Thread 1 (Thread 0x7ffff7ec7740 (LWP 387946) "python"):
>   #0  0x00007ffff79e4b5b in _PyInterpreterState_GET () at
> /usr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Include/internal/pycore_pystate.h:118
>   #1  get_state () at
> /usr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Objects/obmalloc.c:866
>   #2  _PyObject_Free (ctx=<optimized out>, p=0x555556d898a0) at
> /usr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Objects/obmalloc.c:1850
>   #3  0x00007fff9e5e292f in sip_api_free (mem=<optimized out>) at
> /usr/src/debug/sip-4.19.25-9.fc39.x86_64/siplib/siplib.c:2239
>   #4  sipOMFinalise (om=0x7fff9e5ff9a0 <cppPyMap.lto_priv.0>) at
> /usr/src/debug/sip-4.19.25-9.fc39.x86_64/susr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Python/pylifecycle.c:1977
sip-4.19 is from 2018. The Python folks will probably tell you to
update to something newer, like sip-6.7. Also see
<https://pypi.org/project/sip/#history>;.

>   #8  0x00007ffff7ab6337 in Py_RunMain () at
> /usr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Modules/main.c:691
>   #9  0x00007ffff7a7185c in Py_BytesMain (argc=<optimized out>, argv=<optimized
> out>) at /usr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Modules/main.c:743
>   #10 0x00007ffff764614a in __libc_start_call_main (main=main@entry=0x555555555160 
Interesting... You are missing stack frames 5, 6 and 7. You have 0-4,
and 8-10. That may indicate an issue with GDB.

Jeff

[Ranjan Maitra]

On Sun Dec03'23 09:38:20AM, Jeffrey Walton wrote:
>  From: Jeffrey Walton <noloader(a)gmail.com&gt;
>  Date: Sun, 3 Dec 2023 09:38:20 -0500
>  To: Community support for Fedora users <users(a)lists.fedoraproject.org&gt;
>  CC: Ranjan Maitra <mlmaitra(a)gmx.com&gt;
>  Reply-To: noloader(a)gmail.com, Community support for Fedora users
>   <users(a)lists.fedoraproject.org&gt;
>  Subject: Re: Python code error in F39, but not in F38
>
>  On Wed, Nov 22, 2023 at 5:21 PM Ranjan Maitra via users
>  <users(a)lists.fedoraproject.org&gt; wrote:
>  >
>  > On Wed Nov22'23 09:49:49PM, Barry Scott wrote:
>  > > [...]
>  > > > On 22 Nov 2023, at 21:32, Barry Scott <barry(a)barrys-emacs.org&gt;
> wrote:
>  > > >
>  > > >> Any suggestions as to what is going wrong here?
>  > > >
>  > > > At this point you need get more information.
>  > > >
>  > > > What do the folllowing GDB commands print?
>  > > >
>  > > > p tstate
>  > > > p tstate->interp
>  > > > thread allow all bt
>  > > Doh...
>  > >
>  > >       thread apply all bt
>  > >
>  > > I always type:
>  > >       t a a bt
>  > >
>  > > and typo'ed the expansion.
>  >
>  > Thanks, here is what comes out:
>  >
>  >  (gdb)  p tstate->interp
>  >
>  >  value has been optimized out
>  >  (gdb) thread apply all bt
>  >
>  >  Thread 1 (Thread 0x7ffff7ec7740 (LWP 387946) "python"):
>  >  #0  0x00007ffff79e4b5b in _PyInterpreterState_GET () at
> /usr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Include/internal/pycore_pystate.h:118
>  >  #1  get_state () at
> /usr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Objects/obmalloc.c:866
>  >  #2  _PyObject_Free (ctx=<optimized out>, p=0x555556d898a0) at
> /usr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Objects/obmalloc.c:1850
>  >  #3  0x00007fff9e5e292f in sip_api_free (mem=<optimized out>) at
> /usr/src/debug/sip-4.19.25-9.fc39.x86_64/siplib/siplib.c:2239
>  >  #4  sipOMFinalise (om=0x7fff9e5ff9a0 <cppPyMap.lto_priv.0>) at
> /usr/src/debug/sip-4.19.25-9.fc39.x86_64/susr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Python/pylifecycle.c:1977
>
>  sip-4.19 is from 2018. The Python folks will probably tell you to
>  update to something newer, like sip-6.7. Also see
>  <https://pypi.org/project/sip/#history>;. 
Thank you for this! I have no idea where sip gets pulled from, because
it was not installed in the system.

So I went ahead and installed sip6 via:

sudo dnf install sip6
 # brings in sip-6.7.12.

I get:

(gdb) run ./test.py statlab.tiff statlab-8-rgb.png 1 5
 Starting program: /usr/bin/python ./test.py statlab.tiff statlab-8-rgb.png 1 5
 Downloading separate debug info for system-supplied DSO at 0x7ffff7fc7000
 [Thread debugging using libthread_db enabled]
 Using host libthread_db library "/lib64/libthread_db.so.1".
 [New Thread 0x7fff9c39b6c0 (LWP 6832)]                            <2-t_��>ý
v:lua.cmp.utils.feedkeys.call.run(12)
,
 [New Thread 0x7fff9bb9a6c0 (LWP 6833)]
 [New Thread 0x7fff9b3996c0 (LWP 6834)]
 [New Thread 0x7fff91d856c0 (LWP 6835)]
 [New Thread 0x7fff915846c0 (LWP 6836)]
 [New Thread 0x7fff90d836c0 (LWP 6837)]
 [New Thread 0x7fff905826c0 (LWP 6838)]
 0.5003370299685227

 Thread 1 "python" received signal SIGSEGV, Segmentation fault.
 0x00007ffff79e4b5b in _PyInterpreterState_GET ()
     at
/usr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Include/internal/pycore_pystate.h:118
 118         return tstate->interp;

(gdb) p tstate
 $1 = <optimized out>

(gdb) p tstate->interp
 value has been optimized out

I get:

(gdb) thread apply all bt

 Thread 8 (Thread 0x7fff905826c0 (LWP 6838) "python"):
 #0  futex_wait (val=128, addr=0x555555d99b84) at
../../../libgomp/config/linux/x86/futex.h:97
 #1  do_wait (val=128, addr=<optimized out>) at
../../../libgomp/config/linux/wait.h:67
 #2  gomp_barrier_wait_end (bar=0x555555d99b80, state=128) at
../../../libgomp/config/linux/bar.c:48
 #3  0x00007fffe95cdd28 in gomp_simple_barrier_wait (bar=0x555555d99b80) at
../../../libgomp/config/posix/simple-bar.h:60
 #4  gomp_thread_start (xdata=<optimized out>) at ../../../libgomp/team.c:133
 #5  0x00007ffff76ac897 in start_thread (arg=<optimized out>) at
pthread_create.c:444
 #6  0x00007ffff77336bc in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

 Thread 7 (Thread 0x7fff90d836c0 (LWP 6837) "python"):
 #0  futex_wait (val=128, addr=0x555555d99b84) at
../../../libgomp/config/linux/x86/futex.h:97
 #1  do_wait (val=128, addr=<optimized out>) at
../../../libgomp/config/linux/wait.h:67
 #2  gomp_barrier_wait_end (bar=0x555555d99b80, state=128) at
../../../libgomp/config/linux/bar.c:48
 #3  0x00007fffe95cdd28 in gomp_simple_barrier_wait (bar=0x555555d99b80) at
../../../libgomp/config/posix/simple-bar.h:60
 #4  gomp_thread_start (xdata=<optimized out>) at ../../../libgomp/team.c:133
 #5  0x00007ffff76ac897 in start_thread (arg=<optimized out>) at
pthread_create.c:444
 #6  0x00007ffff77336bc in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

 Thread 6 (Thread 0x7fff915846c0 (LWP 6836) "python"):
 #0  futex_wait (val=128, addr=0x555555d99b84) at
../../../libgomp/config/linux/x86/futex.h:97
 #1  do_wait (val=128, addr=<optimized out>) at
../../../libgomp/config/linux/wait.h:67
 #2  gomp_barrier_wait_end (bar=0x555555d99b80, state=128) at
../../../libgomp/config/linux/bar.c:48
 #3  0x00007fffe95cdd28 in gomp_simple_barrier_wait (bar=0x555555d99b80) at
../../../libgomp/config/posix/simple-bar.h:60
 #4  gomp_thread_start (xdata=<optimized out>) at ../../../libgomp/team.c:133
 #5  0x00007ffff76ac897 in start_thread (arg=<optimized out>) at
pthread_create.c:444
 #6  0x00007ffff77336bc in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

 Thread 5 (Thread 0x7fff91d856c0 (LWP 6835) "python"):
 #0  futex_wait (val=128, addr=0x555555d99b84) at
../../../libgomp/config/linux/x86/futex.h:97
 #1  do_wait (val=128, addr=<optimized out>) at
../../../libgomp/config/linux/wait.h:67
 #2  gomp_barrier_wait_end (bar=0x555555d99b80, state=128) at
../../../libgomp/config/linux/bar.c:48
 #3  0x00007fffe95cdd28 in gomp_simple_barrier_wait (bar=0x555555d99b80) at
../../../libgomp/config/posix/simple-bar.h:60
 #4  gomp_thread_start (xdata=<optimized out>) at ../../../libgomp/team.c:133
 #5  0x00007ffff76ac897 in start_thread (arg=<optimized out>) at
pthread_create.c:444
 #6  0x00007ffff77336bc in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

 Thread 4 (Thread 0x7fff9b3996c0 (LWP 6834) "python"):
 #0  futex_wait (val=128, addr=0x555555d99b84) at
../../../libgomp/config/linux/x86/futex.h:97
 #1  do_wait (val=128, addr=<optimized out>) at
../../../libgomp/config/linux/wait.h:67
 #2  gomp_barrier_wait_end (bar=0x555555d99b80, state=128) at
../../../libgomp/config/linux/bar.c:48
 #3  0x00007fffe95cdd28 in gomp_simple_barrier_wait (bar=0x555555d99b80) at
../../../libgomp/config/posix/simple-bar.h:60
 #4  gomp_thread_start (xdata=<optimized out>) at ../../../libgomp/team.c:133
 #5  0x00007ffff76ac897 in start_thread (arg=<optimized out>) at
pthread_create.c:444
 #6  0x00007ffff77336bc in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

 Thread 3 (Thread 0x7fff9bb9a6c0 (LWP 6833) "python"):
 #0  futex_wait (val=128, addr=0x555555d99b84) at
../../../libgomp/config/linux/x86/futex.h:97
 #1  do_wait (val=128, addr=<optimized out>) at
../../../libgomp/config/linux/wait.h:67
 #2  gomp_barrier_wait_end (bar=0x555555d99b80, state=128) at
../../../libgomp/config/linux/bar.c:48
 #3  0x00007fffe95cdd28 in gomp_simple_barrier_wait (bar=0x555555d99b80) at
../../../libgomp/config/posix/simple-bar.h:60
 #4  gomp_thread_start (xdata=<optimized out>) at ../../../libgomp/team.c:133
 #5  0x00007ffff76ac897 in start_thread (arg=<optimized out>) at
pthread_create.c:444
 #6  0x00007ffff77336bc in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

 Thread 2 (Thread 0x7fff9c39b6c0 (LWP 6832) "python"):
 #0  futex_wait (val=128, addr=0x555555d99b84) at
../../../libgomp/config/linux/x86/futex.h:97
 #1  do_wait (val=128, addr=<optimized out>) at
../../../libgomp/config/linux/wait.h:67
 #2  gomp_barrier_wait_end (bar=0x555555d99b80, state=128) at
../../../libgomp/config/linux/bar.c:48
 #3  0x00007fffe95cdd28 in gomp_simple_barrier_wait (bar=0x555555d99b80) at
../../../libgomp/config/posix/simple-bar.h:60
 #4  gomp_thread_start (xdata=<optimized out>) at ../../../libgomp/team.c:133
 #5  0x00007ffff76ac897 in start_thread (arg=<optimized out>) at
pthread_create.c:444
 #6  0x00007ffff77336bc in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

 Thread 1 (Thread 0x7ffff7ec7740 (LWP 6828) "python"):
 #0  0x00007ffff79e4b5b in _PyInterpreterState_GET () at
/usr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Include/internal/pycore_pystate.h:118
 #1  get_state () at
/usr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Objects/obmalloc.c:866
 #2  _PyObject_Free (ctx=<optimized out>, p=0x555556d85180) at
/usr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Objects/obmalloc.c:1850
 #3  0x00007fff9e5e292f in sip_api_free (mem=<optimized out>) at
/usr/src/debug/sip-4.19.25-9.fc39.x86_64/siplib/siplib.c:2239
 #4  sipOMFinalise (om=0x7fff9e5ff9a0 <cppPyMap.lto_priv.0>) at
/usr/src/debug/sip-4.19.25-9.fc39.x86_64/siplib/objmap.c:69
 #5  finalise() () at /usr/src/debug/sip-4.19.25-9.fc39.x86_64/siplib/siplib.c:2143
 #6  0x00007ffff79404d4 in call_ll_exitfuncs (runtime=0x7ffff7ddaa60 <_PyRuntime>)
at /usr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Python/pylifecycle.c:3018
 #7  Py_FinalizeEx () at
/usr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Python/pylifecycle.c:1977
 #8  0x00007ffff7ab6337 in Py_RunMain () at
/usr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Modules/main.c:691
 #9  0x00007ffff7a7185c in Py_BytesMain (argc=<optimized out>, argv=<optimized
out>) at /usr/src/debug/python3.12-3.12.0-1.fc39.x86_64/Modules/main.c:743
 #10 0x00007ffff764614a in __libc_start_call_main (main=main@entry=0x555555555160
<main>, argc=argc@entry=6, argv=argv@entry=0x7fffffffe0d8) at
../sysdeps/nptl/libc_start_call_main.h:58
 #11 0x00007ffff764620b in __libc_start_main_impl (main=0x555555555160 <main>,
argc=6, argv=0x7fffffffe0d8, init=<optimized out>, fini=<optimized out>,
rtld_fini=<optimized out>, stack_end=0x7fffffffe0c8) at ../csu/libc-start.c:360
 #12 0x0000555555555095 in _start ()
(gdb)

Interestingly, it still refers to sip-4.19, however:

$ sudo dnf list sip\*
 Last metadata expiration check: 2:48:01 ago on Sun 03 Dec 2023 08:21:56 AM CST.
 Installed Packages
 sip6.x86_64                           6.7.12-1.fc39                     @updates
 Available Packages
 sip.x86_64                            4.19.25-9.fc39                    fedora
 sip-doc.noarch                        4.19.25-9.fc39                    fedora
 sip-redirect.noarch                   0.2.0-20.fc39                     fedora
 sipcalc.x86_64                        1.1.6-24.fc39                     fedora
 sipp.x86_64                           3.7.2-2.fc39                      updates
 sipsak.x86_64                         0.9.8.1-8.fc39                    fedora
 sipvicious.noarch                     0.3.3-9.fc39                      fedora


Any suggestions? I still get the segfault.

Best wishes,
Ranjan

[George N. White III]

On Sun, Nov 19, 2023 at 2:02 PM Ranjan Maitra via users <
users(a)lists.fedoraproject.org&gt; wrote:

>  On Sun Nov19'23 05:35:19PM, Barry wrote:
>  > From: Barry <barry(a)barrys-emacs.org
> >  
>  >
> Date: Sun, 19 Nov 2023 17:35:19 +0000
>  > To: Community support for Fedora users <users(a)lists.fedoraproject.org
> >  
> >  
>  >
> >  
>  >
> >  
>  > > It is possible that there is a bug in the code
> itself, but nothing
>  above
>  > > points to my created code.
>  >
> >  
>  > > Any suggestions? Or is this a bug?
> >  
>  > Are you using C extensions in your code? If so I would
> suspect them for
>  corrupting python. I think you can get into gdb inside the core file with
>  coredumpctl debug.
>
>  The python code calls a shared object library that is written in C. But
>  why does this core dump only happen in F39 (python 3.12) and not with
>  F38 (python 3.11)?
>  
There were some big changes in 3.12 (driven by the need to support Fortran
on
Windows and macOS by replacing very old gfortran version with flang-new).
I havrn't
had time to look at what is being used in linux, but you could get a quick
overview from
ldd.  See: <
https://discuss.python.org/t/why-scipy-builds-for-python-3-12-on-windows-...
>  
There is also new build system, so a lot of work getting everything working
in 3.12.  Meanwhile
you can run 3.11 stuff using python environments.


-- 
George N. White III

[Ranjan Maitra]

On Wed Nov22'23 08:12:58AM, George N. White III wrote:
>  From: "George N. White III" <gnwiii(a)gmail.com&gt;
>  Date: Wed, 22 Nov 2023 08:12:58 -0400
>  To: Community support for Fedora users <users(a)lists.fedoraproject.org&gt;
>  Reply-To: Community support for Fedora users <users(a)lists.fedoraproject.org&gt;
>  Subject: Re: Python code error in F39, but not in F38
>
>  On Sun, Nov 19, 2023 at 2:02 PM Ranjan Maitra via users <
>  users(a)lists.fedoraproject.org&gt; wrote:
>
>  > On Sun Nov19'23 05:35:19PM, Barry wrote:
>  > > From: Barry <barry(a)barrys-emacs.org&gt;
>  > > Date: Sun, 19 Nov 2023 17:35:19 +0000
>  > > To: Community support for Fedora users <users(a)lists.fedoraproject.org&gt;
>  >  >
>  > > >
>  > > >
>  > > > It is possible that there is a bug in the code itself, but nothing
>  > above
>  > > > points to my created code.
>  > > >
>  > > > Any suggestions? Or is this a bug?
>  > >
>  > > Are you using C extensions in your code? If so I would suspect them for
>  > corrupting python. I think you can get into gdb inside the core file with
>  > coredumpctl debug.
>  >
>  > The python code calls a shared object library that is written in C. But
>  > why does this core dump only happen in F39 (python 3.12) and not with
>  > F38 (python 3.11)?
>  >
>
>  There were some big changes in 3.12 (driven by the need to support Fortran
>  on
>  Windows and macOS by replacing very old gfortran version with flang-new).
>  I havrn't
>  had time to look at what is being used in linux, but you could get a quick
>  overview from
>  ldd.  See: <
>
> https://discuss.python.org/t/why-scipy-builds-for-python-3-12-on-windows-...
>  >
>
>  There is also new build system, so a lot of work getting everything working
>  in 3.12.  Meanwhile
>  you can run 3.11 stuff using python environments.
>  
Thanks! Interesting, I wonder if that is also what has caused tensorflow to not go up to
3.12 for now. From what I read somewhere (I think on an archived tensorflow mailing list
or something) the linux builds are up, but not the MacO$ and Windoze builds. Both of
which, if I had my way, should never have been allowed even indirectly into any
professional environment.

Ranjan

[Barry]

>  On 22 Nov 2023, at 16:14, Ranjan Maitra via users
> <users(a)lists.fedoraproject.org&gt; wrote:
>  
>  Thanks! Interesting, I wonder if that is also what has caused tensorflow to not go up to
> 3.12 for now. 
Complex packages usually take a while to port to a new python is released.
All the python packages that fedora ships have been ported to python 3.12.

Why a crash in 3.12 and not 3.11? Because bugs can be tripped up on compiler
changes. FYI the fortran issues do not effect python 3.12 itself, they effected
packages in the scientific python world like numpy, pandas etc.
But due to heroic folk they all made it to 3.12.

Barry

[Samuel Sieb]

On 11/19/23 08:25, Ranjan Maitra via users wrote:
>  I was running the code here:
> https://github.com/maitra/Visual-Information-Fidelity---Python3
>  
>  However, the code runs in F38 (python 3.11) without error, but not in F39 (python 3.12)
> where it
>  ends with a segmentation fault (after doing the calculations). 
I just upgraded my system to F39, so I tested this code and it worked 
fine with no crashes.  Did you rebuild the shared library after upgrading?

[Ranjan Maitra]

On Sun Dec03'23 01:51:24PM, Samuel Sieb wrote:
>  From: Samuel Sieb <samuel(a)sieb.net&gt;
>  Date: Sun, 3 Dec 2023 13:51:24 -0800
>  To: users(a)lists.fedoraproject.org
>  Reply-To: Community support for Fedora users <users(a)lists.fedoraproject.org&gt;
>  Subject: Re: Python code error in F39, but not in F38
>
>  On 11/19/23 08:25, Ranjan Maitra via users wrote:
>  > I was running the code here:
> https://github.com/maitra/Visual-Information-Fidelity---Python3
>  >
>  > However, the code runs in F38 (python 3.11) without error, but not in F39 (python
> 3.12) where it
>  > ends with a segmentation fault (after doing the calculations).
>
>  I just upgraded my system to F39, so I tested this code and it worked fine
>  with no crashes.  Did you rebuild the shared library after upgrading? 
Wow, that is weird! But good to know. As to your question, yes I did rebuild the shared
object (library) several times, but that should not matter, right? I guess this is local
to my F39 machines, but all (three) of them appear to be dumping core.

Thanks again, I will try and get to the bottom of it. Perhaps by trying the code on a
clean F39 installation.

Best wishes,
Ranjan



> --
> _______________________________________________
> users mailing list -- users(a)lists.fedoraproject.org
> To unsubscribe send an email to users-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]
> Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue