[Bug 1054061] New: Segfault in Perl_gv_fetchpvn_flags when trying to initialize back_perl openldap backend

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=1054061

Bug ID: 1054061
           Summary: Segfault in Perl_gv_fetchpvn_flags when trying to
                    initialize back_perl openldap backend
           Product: Red Hat Enterprise Linux 7
           Version: 7.0
         Component: perl
          Assignee: [e-mail]
          Reporter: [e-mail]
        QA Contact: [e-mail]
                CC: [e-mail], [e-mail],
                    [e-mail], [e-mail], [e-mail],
                    [e-mail], [e-mail],
                    [e-mail], [e-mail],
                    [e-mail], [e-mail],
                    [e-mail]
        Depends On: 967719
             Group: devel
   External Bug ID: OpenLDAP ITS 7573

+++ This bug was initially created as a clone of Bug #967719 +++

Description of problem:
When trying to initialize back_perl, a segfault occurs deep in perl itself.

Version-Release number of selected component (if applicable):
perl-5.16.3-264.fc19.x86_64
openldap-2.4.35-4.fc19.x86_64

How reproducible:
Almost always.

Steps to Reproduce:
1. Install fresh F19
2. If you try the reproducer here, all goes well
3. yum install perl-A* (I have no idea why I needed to do this to get it to
segfault)
4. Try reproducer
5. Observe the segfault
6. From now on, reproducer works *without* producing any segfaults. I had to
reboot the machine to be able to reproduce the issue again.

Note on how to use the reproducer:
1. Unpack
2. Run go.sh (warning: it will wipe your /var/lib/ldap/* and your
/etc/openldap/*, so don't run if you use openldap in production)
   This will run slapd in debug mode, so you will need another console to run
the rest.
3. Run try.sh
4. If you want to repeat, go to 2.

You can modify go.sh to run slapd through a debugger. However, you will
probably need to set LD_PRELOAD=/usr/lib64/perl5/CORE/libperl.so to be able to
run try.sh.

Actual results:
Perl segfaults.

Expected results:
The back_perl gets initialized without any problems.

Additional info:
This also happens on my production F18, but it happens always. It looks like it
might have something to do with Bug 960048.

--- Additional comment from Jan Synacek on 2013-05-28 08:53:11 CEST ---

Note that in the frame #3, the my_perl variable changes from a (probably valid)
pointer to 0x0.

--- Additional comment from Petr Pisar on 2013-07-03 14:38:09 CEST ---

What's expected output on the client site?

I have patched perl so that it links all XS modules to libperl.so explicitly in
my private F20 machine, and I do not get any segfaults. Instead I just get an
error message from the second command:

$ sh try.sh
SASL/EXTERNAL authentication started
SASL username: gidNumber=500+uidNumber=500,cn=peercred,cn=external,cn=auth
SASL SSF: 0
adding new entry "cn=module,cn=config"
ldap_add: Insufficient access (50)

SASL/EXTERNAL authentication started
SASL username: gidNumber=500+uidNumber=500,cn=peercred,cn=external,cn=auth
SASL SSF: 0
adding new entry "olcDatabase=perl,cn=config"
ldap_add: Invalid syntax (21)
        additional info: objectClass: value #1 invalid per syntax

I guess this is how to should work.

If this is the real fix, than this issue will be fixed in F20.

--- Additional comment from Jan Synacek on 2013-07-03 14:57:38 CEST ---

All LDAP operations should succeed without any errors. You don't have
sufficient access rights to modify the LDAP database. Running the script as
root should do it.

You should see something like this:
# ./try.sh 
SASL/EXTERNAL authentication started
SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth
SASL SSF: 0
adding new entry "cn=module,cn=config"

SASL/EXTERNAL authentication started
SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth
SASL SSF: 0
adding new entry "olcDatabase=perl,cn=config"
ldap_result: Can't contact LDAP server (-1)

The last line indicates the termination (you can see that slapd segfaulted in
the console where you ran go.sh). Expected output would look like the one above
minus the 'ldap_result: ...' line. Plus there may be one additional 'adding new
entry' line.

--- Additional comment from Petr Pisar on 2013-07-03 16:07:01 CEST ---

I run it as a non-root. I get much better results as a root:

# ./try.sh
SASL/EXTERNAL authentication started
SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth
SASL SSF: 0
adding new entry "cn=module,cn=config"

SASL/EXTERNAL authentication started
SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth
SASL SSF: 0
adding new entry "olcDatabase=perl,cn=config"

adding new entry "dc=perl-example,dc=com"

However the server sometimes segfaults as you reported.

--- Additional comment from Jan Synacek on 2013-07-04 07:54:23 CEST ---

(In reply to Petr Pisar from comment #4)
> I run it as a non-root. I get much better results as a root:

Yes, that's how the output is supposed to look.
> However the server sometimes segfaults as you reported.

Ok, it's strange that it segfaults only sometimes. Does this mean that you can
reproduce it even with all the perl modules linked to libperl.so explicitly?

--- Additional comment from Petr Pisar on 2013-07-04 09:07:56 CEST ---

I recompiled only modules in perl binary package and openldap for the test. Not
yet all modules that exist. I also reverted the RTLD_GLOBAL patch and checked
slapd does not crashes on linkage error. So the test does not use any
not-yet-rebuild modules.

I conclude this issue is independent on the linking bug #960048. But we will
see more after rebuilding everything.

--- Additional comment from Fedora Admin XMLRPC Client on 2013-08-12 13:50:07
CEST ---

This package has changed ownership in the Fedora Package Database.  Reassigning
to the new owner of this component.

--- Additional comment from Howard Chu on 2013-10-13 02:20:16 CEST ---

Please also followup to OpenLDAP ITS#7573 with any conclusions you reach,
thanks.

--- Additional comment from Jan Synacek on 2013-10-16 08:13:38 CEST ---

This problem still occurs in F20.

Referenced Bugs:

https://bugzilla.redhat.com/show_bug.cgi?id=967719
[Bug 967719] Segfault in Perl_gv_fetchpvn_flags when trying to initialize
back_perl openldap backend

[Bug 1054061] Segfault in Perl_gv_fetchpvn_flags when trying to initialize back_perl openldap backend

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=1054061

Petr Pisar  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
         Depends On|967719                      |

Referenced Bugs:

https://bugzilla.redhat.com/show_bug.cgi?id=967719
[Bug 967719] Segfault in Perl_gv_fetchpvn_flags when trying to initialize
back_perl openldap backend

[Bug 1054061] Segfault in Perl_gv_fetchpvn_flags when trying to initialize back_perl openldap backend

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=1054061

Marcela Mašláňová  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]
              Flags|                            |needinfo?(jsynacek@redhat.c
                   |                            |om)

--- Comment #1 from Marcela Mašláňová  ---
It's not reproducible on F20, so we can't fix it even on RHEL-7. If you can
provide to us better reproducer, we can try to fix it.

[Bug 1054061] Segfault in Perl_gv_fetchpvn_flags when trying to initialize back_perl openldap backend

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=1054061

Jan Synacek  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
              Flags|needinfo?(jsynacek@redhat.c |
                   |om)                         |

--- Comment #2 from Jan Synacek  ---
The reproducer is still the same and you can even simplify it a bit. And yes,
it's still reproducible on F20 as well as RHEL7 (Beta 1).

Reproducer:
0) install RHEL-7-Beta-1 minimal installation
   yum install openldap-{servers,clients}
1) unpack reproducer pack
2) in one terminal window, run go.sh
3) in another terminal window, run try.sh
   the output will look like this:

# ./try.sh 
  SASL/EXTERNAL authentication started
  SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth
  SASL SSF: 0
  adding new entry "cn=module,cn=config"

SASL/EXTERNAL authentication started
  SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth
  SASL SSF: 0
  adding new entry "olcDatabase=perl,cn=config"
  ldap_result: Can't contact LDAP server (-1)

the last line indicates that the server has been terminated

4) see the terminal window in which you have run the go.sh script and you will
see a segfault there
5) if you want to repeat the crash, reboot and go to 2)

I had to reboot the machine if I wanted to repeat the crash, because any
subsequent attempts to reproduce the issue were not successful (the try.sh
script finished without any errors).

If try.sh finishes without any errors, its output looks like so:
  # ./try.sh 
  SASL/EXTERNAL authentication started
  SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth
  SASL SSF: 0
  adding new entry "cn=module,cn=config"

SASL/EXTERNAL authentication started
  SASL username: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth
  SASL SSF: 0
  adding new entry "olcDatabase=perl,cn=config"

adding new entry "dc=perl-example,dc=com"

[Bug 1054061] Segfault in Perl_gv_fetchpvn_flags when trying to initialize back_perl openldap backend

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=1054061

--- Comment #3 from Jan Synacek  ---
Created attachment 851575
  --> https://bugzilla.redhat.com/attachment.cgi?id=851575&action=edit
reproducer

[Bug 1054061] Segfault in Perl_gv_fetchpvn_flags when trying to initialize back_perl openldap backend

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=1054061

--- Comment #4 from RHEL Product and Program Management  ---
This request was not resolved in time for the current release.
Red Hat invites you to ask your support representative to
propose this request, if still desired, for consideration in
the next release of Red Hat Enterprise Linux.

[Bug 1054061] Segfault in Perl_gv_fetchpvn_flags when trying to initialize back_perl openldap backend

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=1054061

Petr Pisar  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
              Flags|                            |needinfo?(jsynacek@redhat.c
                   |                            |om)

--- Comment #5 from Petr Pisar  ---
I'm sorry but the reproducer involves a lot of openldap code. Please provide
perl-only reproducer. Otherwise I must conclude the bug is in the openldap
because we've never seen such a failure elsewhere (e.g. mod_perl is a dlopened
DSO which embeds the perl interpreter).

[Bug 1054061] Segfault in Perl_gv_fetchpvn_flags when trying to initialize back_perl openldap backend

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=1054061

--- Comment #6 from Howard Chu  ---
(In reply to Petr Pisar from comment #5)
> I'm sorry but the reproducer involves a lot of openldap code. Please provide
> perl-only reproducer. Otherwise I must conclude the bug is in the openldap
> because we've never seen such a failure elsewhere (e.g. mod_perl is a
> dlopened DSO which embeds the perl interpreter).

Typically mod_perl doesn't run with thread support though, so not directly
comparable.

[Bug 1054061] Segfault in Perl_gv_fetchpvn_flags when trying to initialize back_perl openldap backend

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=1054061

Joe Orton  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

[Bug 1054061] Segfault in Perl_gv_fetchpvn_flags when trying to initialize back_perl openldap backend

[Red Hat Bugzilla]

https://bugzilla.redhat.com/show_bug.cgi?id=1054061

Martin Kyral  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]