Certs expired, CA Unreachable

[Evan G]

Good afternoon.

We currently have FreeIPA v4.6.8 running on CentOS7. We have tried many of the solutions posted on this mailer however none have helped us bring the environment back online. Our current situation is as follows:

- We have a single master / single CA with a total of 4 FreeIPA (2 in each site) servers in production.
- Replication is not working between the master and secondaries.
- The FreeIPA admin account password is working and we are able to kinit as admin
- We can bring the IPA services online by rolling the clock back to before the HTTP cert expired, however the CA refuses to sign any of our cert requests -- giving a Kerberos authentication error when CURL'd
- We are able to login to the HTTP interface with the services up and date rolled back, however we are unable to issue a new cert, we receive a 500 error in reaching the CA

Happy to provide any other requested info but we've been troubleshooting this for 3 days straight and we're coming up empty on every avenue.

[Rob Crittenden]

Evan G via FreeIPA-users wrote:
> Good afternoon.
>
> We currently have FreeIPA v4.6.8 running on CentOS7. We have tried many of the solutions posted on this mailer however none have helped us bring the environment back online. Our current situation is as follows:
> We have a single master / single CA with a total of 4 FreeIPA (2 in each site) servers in production.

ipa config-show will tell you which one is the renewal master. All
renewals need to start there.
> Replication is not working between the master and secondaries.

Are all certs expired or just some?

`getcert list` will tell us.
> The FreeIPA admin account password is working and we are able to kinit as admin
> We can bring the IPA services online by rolling the clock back to before the HTTP cert expired, however the CA refuses to sign any of our cert requests -- giving a Kerberos authentication error when CURL'd

I'm not sure what you are using CURL for.
> We are able to login to the HTTP interface with the services up and date rolled back, however we are unable to issue a new cert, we receive a 500 error in reaching the CA
>
> Happy to provide any other requested info but we've been troubleshooting this for 3 days straight and we're coming up empty on every avenue.

You'll want to look at /var/log/pki/pki-tomcat/ca debug after a start.
Read from the top down looking for start-up errors. Reading from the
bottom up from the log usually leads to red herrings.

There is also a selfsign.log near that directory and it will tell you if
start-up failed due to inconsistencies.

rob

[Evan G]

Hi Rob,

When we start tomcat with the date rolled back, we are not seeing any errors at all. All of the ipa services start up without issue. The problem is in actually renewing the certs, when we do so we have seen many different errors as we've been troubleshooting -- mostly this one: `ca-error: Server at https://<HOSTNAME>/ipa/xml failed request, will retry: 4035 (RPC failed at server.  Request failed with status 500: Non-2xx response from CA REST API: 500. String index out of range: -1).[02/Aug/2023:00:00:31][ajp-bio-127.0.0.1-8009-exec-2]: EnrollProfile: populate: begins`

When I restart certmonger after all services up, these are the errors that I am seeing in the tomcat debug logs:
```
[02/Aug/2023:00:00:31][ajp-bio-127.0.0.1-8009-exec-2]: BasicProfile: populate: policy setid =serverCertSet
[02/Aug/2023:00:00:31][ajp-bio-127.0.0.1-8009-exec-2]: EnrollDefault: populate: SubjectNameDefault: start
java.lang.StringIndexOutOfBoundsException: String index out of range: -1
    at java.lang.String.substring(String.java:1967)
    at com.netscape.certsrv.pattern.Pattern.substitute2(Pattern.java:132)
    at com.netscape.cms.profile.def.EnrollDefault.mapPattern(EnrollDefault.java:815)
    at com.netscape.cms.profile.def.SubjectNameDefault.populate(SubjectNameDefault.java:160)
    at com.netscape.cms.profile.def.EnrollDefault.populate(EnrollDefault.java:226)
    at com.netscape.cms.profile.common.BasicProfile.populate(BasicProfile.java:1114)
    at com.netscape.cms.profile.common.EnrollProfile.populate(EnrollProfile.java:2626)
    at com.netscape.cms.servlet.cert.CertProcessor.populateRequests(CertProcessor.java:379)
    at com.netscape.cms.servlet.cert.EnrollmentProcessor.processEnrollment(EnrollmentProcessor.java:188)
    at com.netscape.cms.servlet.cert.EnrollmentProcessor.processEnrollment(EnrollmentProcessor.java:96)
    at com.netscape.cms.servlet.cert.CertRequestDAO.submitRequest(CertRequestDAO.java:197)
    at org.dogtagpki.server.ca.rest.CertRequestService.enrollCert(CertRequestService.java:155)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:137)
    at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:280)
    at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:234)
    at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:221)
    at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:356)
    at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:179)
    at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:220)
    at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
    at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)
    at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
    at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)
    at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:175)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:297)
    at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:55)
    at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:191)
    at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:187)
    at java.security.AccessController.doPrivileged(Native Method)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:186)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)
    at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
    at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)
    at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:260)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:237)
    at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:55)
    at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:191)
    at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:187)
    at java.security.AccessController.doPrivileged(Native Method)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:186)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:218)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:498)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)
    at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:190)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:750)
```

This is what we see when we run `getcert list` and `ipa-getcert list` respectively:

```
Number of certificates and requests being tracked: 9.
Request ID '20190920201259':
    status: CA_UNREACHABLE
    ca-error: Server at https://<HOSTNAME>/ipa/xml failed request, will retry: 4035 (RPC failed at server.  Request failed with status 500: Non-2xx response from CA REST API: 500. String index out of range: -1).
    stuck: no
    key pair storage: type=FILE,location='/var/kerberos/krb5kdc/kdc.key'
    certificate: type=FILE,location='/var/kerberos/krb5kdc/kdc.crt'
    CA: IPA
    issuer: CN=Certificate Authority,O=<OU>
    subject: CN=<HOSTNAME>,O=<OU>
    expires: 2023-08-25 18:05:07 UTC
    principal name: krbtgt/<OU>@<OU>
    key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
    eku: id-kp-serverAuth,id-pkinit-KPKdc
    pre-save command:
    post-save command: /usr/libexec/ipa/certmonger/renew_kdc_cert
    track: yes
    auto-renew: yes
Request ID '20210908000050':
    status: MONITORING
    stuck: no
    key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
    certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
    CA: dogtag-ipa-ca-renew-agent
    issuer: CN=Certificate Authority,O=<OU>
    subject: CN=CA Audit,O=<OU>
    expires: 2025-07-21 02:36:57 UTC
    key usage: digitalSignature,keyEncipherment,dataEncipherment
    eku: id-kp-serverAuth,id-kp-clientAuth
    pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
    post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "auditSigningCert cert-pki-ca"
    track: yes
    auto-renew: yes
Request ID '20210908000051':
    status: MONITORING
    stuck: no
    key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
    certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
    CA: dogtag-ipa-ca-renew-agent
    issuer: CN=Certificate Authority,O=<OU>
    subject: CN=OCSP Subsystem,O=<OU>
    expires: 2025-07-21 02:36:17 UTC
    key usage: digitalSignature,keyEncipherment,dataEncipherment
    eku: id-kp-serverAuth,id-kp-clientAuth
    pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
    post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert cert-pki-ca"
    track: yes
    auto-renew: yes
Request ID '20210908000052':
    status: MONITORING
    stuck: no
    key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
    certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
    CA: dogtag-ipa-ca-renew-agent
    issuer: CN=Certificate Authority,O=<OU>
    subject: CN=CA Subsystem,O=<OU>
    expires: 2025-07-21 02:37:17 UTC
    key usage: digitalSignature,keyEncipherment,dataEncipherment
    eku: id-kp-serverAuth,id-kp-clientAuth
    pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
    post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert cert-pki-ca"
    track: yes
    auto-renew: yes
Request ID '20210908000053':
    status: MONITORING
    stuck: no
    key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
    certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
    CA: dogtag-ipa-ca-renew-agent
    issuer: CN=Certificate Authority,O=<OU>
    subject: CN=Certificate Authority,O=<OU>
    expires: 2039-09-20 20:11:25 UTC
    key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
    pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
    post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert cert-pki-ca"
    track: yes
    auto-renew: yes
Request ID '20210908000054':
    status: MONITORING
    stuck: no
    key pair storage: type=FILE,location='/var/lib/ipa/ra-agent.key'
    certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem'
    CA: dogtag-ipa-ca-renew-agent
    issuer: CN=Certificate Authority,O=<OU>
    subject: CN=IPA RA,O=<OU>
    expires: 2025-06-26 02:36:15 UTC
    key usage: digitalSignature,keyEncipherment,dataEncipherment
    eku: id-kp-serverAuth,id-kp-clientAuth
    pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre
    post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert
    track: yes
    auto-renew: yes
Request ID '20210908000055':
    status: MONITORING
    stuck: no
    key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
    certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
    CA: dogtag-ipa-ca-renew-agent
    issuer: CN=Certificate Authority,O=<OU>
    subject: CN=<HOSTNAME>,O=<OU>
    expires: 2025-07-21 02:36:37 UTC
    dns: <HOSTNAME>
    key usage: digitalSignature,keyEncipherment,dataEncipherment
    eku: id-kp-serverAuth,id-kp-clientAuth
    pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
    post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "Server-Cert cert-pki-ca"
    track: yes
    auto-renew: yes
Request ID '20210908000056':
    status: CA_UNREACHABLE
    ca-error: Server at https://<HOSTNAME>/ipa/xml failed request, will retry: 4035 (RPC failed at server.  Request failed with status 500: Non-2xx response from CA REST API: 500. String index out of range: -1).
    stuck: no
    key pair storage: type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS FIPS 140-2 Certificate DB',pinfile='/etc/dirsrv/slapd-<OU>/pwdfile.txt'
    certificate: type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS FIPS 140-2 Certificate DB'
    CA: IPA
    issuer: CN=Certificate Authority,O=<OU>
    subject: CN=<HOSTNAME>,O=<OU>
    expires: 2023-09-03 18:30:48 UTC
    dns: <HOSTNAME>
    key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
    eku: id-kp-serverAuth,id-kp-clientAuth
    pre-save command:
    post-save command: /usr/libexec/ipa/certmonger/restart_dirsrv <OU>
    track: yes
    auto-renew: yes
Request ID '20210908000057':
    status: CA_UNREACHABLE
    ca-error: Server at https://<HOSTNAME>/ipa/xml failed request, will retry: 4035 (RPC failed at server.  Request failed with status 500: Non-2xx response from CA REST API: 500. String index out of range: -1).
    stuck: no
    key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS FIPS 140-2 Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
    certificate: type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS FIPS 140-2 Certificate DB'
    CA: IPA
    issuer: CN=Certificate Authority,O=<OU>
    subject: CN=<HOSTNAME>,O=<OU>
    expires: 2023-09-03 18:30:48 UTC
    dns: <HOSTNAME>
    key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
    eku: id-kp-serverAuth,id-kp-clientAuth
    pre-save command:
    post-save command: /usr/libexec/ipa/certmonger/restart_httpd
    track: yes
    auto-renew: yes
```

```
Number of certificates and requests being tracked: 9.
Request ID '20190920201259':
    status: CA_UNREACHABLE
    ca-error: Server at https://<HOSTNAME>/ipa/xml failed request, will retry: 4035 (RPC failed at server.  Request failed with status 500: Non-2xx response from CA REST API: 500. String index out of range: -1).
    stuck: no
    key pair storage: type=FILE,location='/var/kerberos/krb5kdc/kdc.key'
    certificate: type=FILE,location='/var/kerberos/krb5kdc/kdc.crt'
    CA: IPA
    issuer: CN=Certificate Authority,O=<OU>
    subject: CN=<HOSTNAME>,O=<OU>
    expires: 2023-08-25 18:05:07 UTC
    principal name: krbtgt/<OU>@<OU>
    key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
    eku: id-kp-serverAuth,id-pkinit-KPKdc
    pre-save command:
    post-save command: /usr/libexec/ipa/certmonger/renew_kdc_cert
    track: yes
    auto-renew: yes
Request ID '20210908000056':
    status: CA_UNREACHABLE
    ca-error: Server at https://<HOSTNAME>/ipa/xml failed request, will retry: 4035 (RPC failed at server.  Request failed with status 500: Non-2xx response from CA REST API: 500. String index out of range: -1).
    stuck: no
    key pair storage: type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS FIPS 140-2 Certificate DB',pinfile='/etc/dirsrv/slapd-<OU>/pwdfile.txt'
    certificate: type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS FIPS 140-2 Certificate DB'
    CA: IPA
    issuer: CN=Certificate Authority,O=<OU>
    subject: CN=<HOSTNAME>,O=<OU>
    expires: 2023-09-03 18:30:48 UTC
    dns: <HOSTNAME>
    key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
    eku: id-kp-serverAuth,id-kp-clientAuth
    pre-save command:
    post-save command: /usr/libexec/ipa/certmonger/restart_dirsrv <OU>
    track: yes
    auto-renew: yes
Request ID '20210908000057':
    status: CA_UNREACHABLE
    ca-error: Server at https://<HOSTNAME>/ipa/xml failed request, will retry: 4035 (RPC failed at server.  Request failed with status 500: Non-2xx response from CA REST API: 500. String index out of range: -1).
    stuck: no
    key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS FIPS 140-2 Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
    certificate: type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS FIPS 140-2 Certificate DB'
    CA: IPA
    issuer: CN=Certificate Authority,O=<OU>
    subject: CN=<HOSTNAME>,O=<OU>
    expires: 2023-09-03 18:30:48 UTC
    dns: <HOSTNAME>
    key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
    eku: id-kp-serverAuth,id-kp-clientAuth
    pre-save command:
    post-save command: /usr/libexec/ipa/certmonger/restart_httpd
    track: yes
    auto-renew: yes
```

[Florence Blanc-Renaud]

Hi,
it seems that PKI is not happy with the subject name of the certificates.
The failing certs are for KDC, dirsrv and httpd and they all use the same
subject name constraint in their profile.

1. Was any certificate profile modified (caIPAserviceCert or
KDCs_PKINIT_Certs)? You can use
ipa certprofile-show <name> --out /dev/stdout
And then check the part related to Subject Name Constraint. In my default
installation, I have
policyset.serverCertSet.1.constraint.class_id=subjectNameConstraintImpl
policyset.serverCertSet.1.constraint.name=Subject Name Constraint
policyset.serverCertSet.1.constraint.params.accept=true
policyset.serverCertSet.1.constraint.params.pattern=CN=[^,]+,.+
policyset.serverCertSet.1.default.class_id=subjectNameDefaultImpl
policyset.serverCertSet.1.default.name=Subject Name Default
policyset.serverCertSet.1.default.params.name=CN=$
request.req_subject_name.cn$, O=IPA.TEST

which means that the subject name should match CN= followed by (anything
except a comma) multiple times then a comma and any char multiple times.

2. If the profile wasn't changed, can you check in
/var/log/pki/pki-tomcat/ca/debug.$DATE.log the received certificate
request? Does its subject match the pattern? The error message
java.lang.StringIndexOutOfBoundsException: String index out of range: -1
hints that an expected pattern was not found.

flo

On Thu, Sep 14, 2023 at 4:11 PM Evan G via FreeIPA-users <
[e-mail]> wrote:
> Hi Rob,
>
> When we start tomcat with the date rolled back, we are not seeing any
> errors at all. All of the ipa services start up without issue. The problem
> is in actually renewing the certs, when we do so we have seen many
> different errors as we've been troubleshooting -- mostly this one:
> `ca-error: Server at https://<HOSTNAME>/ipa/xml failed request, will
> retry: 4035 (RPC failed at server.  Request failed with status 500: Non-2xx
> response from CA REST API: 500. String index out of range:
> -1).[02/Aug/2023:00:00:31][ajp-bio-127.0.0.1-8009-exec-2]: EnrollProfile:
> populate: begins`
>
> When I restart certmonger after all services up, these are the errors that
> I am seeing in the tomcat debug logs:
> [02/Aug/2023:00:00:31][ajp-bio-127.0.0.1-8009-exec-2]: BasicProfile:
> populate: policy setid =serverCertSet
> [02/Aug/2023:00:00:31][ajp-bio-127.0.0.1-8009-exec-2]: EnrollDefault:
> populate: SubjectNameDefault: start
> java.lang.StringIndexOutOfBoundsException: String index out of range: -1
>         at java.lang.String.substring(String.java:1967)
>         at
> com.netscape.certsrv.pattern.Pattern.substitute2(Pattern.java:132)
>         at
> com.netscape.cms.profile.def.EnrollDefault.mapPattern(EnrollDefault.java:815)
>         at
> com.netscape.cms.profile.def.SubjectNameDefault.populate(SubjectNameDefault.java:160)
>         at
> com.netscape.cms.profile.def.EnrollDefault.populate(EnrollDefault.java:226)
>         at
> com.netscape.cms.profile.common.BasicProfile.populate(BasicProfile.java:1114)
>         at
> com.netscape.cms.profile.common.EnrollProfile.populate(EnrollProfile.java:2626)
>         at
> com.netscape.cms.servlet.cert.CertProcessor.populateRequests(CertProcessor.java:379)
>         at
> com.netscape.cms.servlet.cert.EnrollmentProcessor.processEnrollment(EnrollmentProcessor.java:188)
>         at
> com.netscape.cms.servlet.cert.EnrollmentProcessor.processEnrollment(EnrollmentProcessor.java:96)
>         at
> com.netscape.cms.servlet.cert.CertRequestDAO.submitRequest(CertRequestDAO.java:197)
>         at
> org.dogtagpki.server.ca.rest.CertRequestService.enrollCert(CertRequestService.java:155)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:498)
>         at
> org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:137)
>         at
> org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:280)
>         at
> org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:234)
>         at
> org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:221)
>         at
> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:356)
>         at
> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:179)
>         at
> org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:220)
>         at
> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
>         at
> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:498)
>         at
> org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)
>         at
> org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
>         at
> org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)
>         at
> org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:175)
>         at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:297)
>         at
> org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:55)
>         at
> org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:191)
>         at
> org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:187)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:186)
>         at
> org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:498)
>         at
> org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)
>         at
> org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
>         at
> org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)
>         at
> org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:260)
>         at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:237)
>         at
> org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:55)
>         at
> org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:191)
>         at
> org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:187)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:186)
>         at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:218)
>         at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
>         at
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:498)
>         at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
>         at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
>         at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962)
>         at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
>         at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)
>         at
> org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:190)
>         at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
>         at org.apache.tomcat.util.net
> .JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
>         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>         at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
>         at java.lang.Thread.run(Thread.java:750)
>
> This is what we see when we run `getcert list` and `ipa-getcert list`
> respectively:
> Number of certificates and requests being tracked: 9.
> Request ID '20190920201259':
>         status: CA_UNREACHABLE
>         ca-error: Server at https://<HOSTNAME>/ipa/xml failed request,
> will retry: 4035 (RPC failed at server.  Request failed with status 500:
> Non-2xx response from CA REST API: 500. String index out of range: -1).
>         stuck: no
>         key pair storage:
> type=FILE,location='/var/kerberos/krb5kdc/kdc.key'
>         certificate: type=FILE,location='/var/kerberos/krb5kdc/kdc.crt'
>         CA: IPA
>         issuer: CN=Certificate Authority,O=<OU>
>         subject: CN=<HOSTNAME>,O=<OU>
>         expires: 2023-08-25 18:05:07 UTC
>         principal name: krbtgt/<OU>@<OU>
>         key usage:
> digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>         eku: id-kp-serverAuth,id-pkinit-KPKdc
>         pre-save command:
>         post-save command: /usr/libexec/ipa/certmonger/renew_kdc_cert
>         track: yes
>         auto-renew: yes
> Request ID '20210908000050':
>         status: MONITORING
>         stuck: no
>         key pair storage:
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
> cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
>         certificate:
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
> cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
>         CA: dogtag-ipa-ca-renew-agent
>         issuer: CN=Certificate Authority,O=<OU>
>         subject: CN=CA Audit,O=<OU>
>         expires: 2025-07-21 02:36:57 UTC
>         key usage: digitalSignature,keyEncipherment,dataEncipherment
>         eku: id-kp-serverAuth,id-kp-clientAuth
>         pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
>         post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
> "auditSigningCert cert-pki-ca"
>         track: yes
>         auto-renew: yes
> Request ID '20210908000051':
>         status: MONITORING
>         stuck: no
>         key pair storage:
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
> cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
>         certificate:
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
> cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
>         CA: dogtag-ipa-ca-renew-agent
>         issuer: CN=Certificate Authority,O=<OU>
>         subject: CN=OCSP Subsystem,O=<OU>
>         expires: 2025-07-21 02:36:17 UTC
>         key usage: digitalSignature,keyEncipherment,dataEncipherment
>         eku: id-kp-serverAuth,id-kp-clientAuth
>         pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
>         post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
> "ocspSigningCert cert-pki-ca"
>         track: yes
>         auto-renew: yes
> Request ID '20210908000052':
>         status: MONITORING
>         stuck: no
>         key pair storage:
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
> cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
>         certificate:
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
> cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
>         CA: dogtag-ipa-ca-renew-agent
>         issuer: CN=Certificate Authority,O=<OU>
>         subject: CN=CA Subsystem,O=<OU>
>         expires: 2025-07-21 02:37:17 UTC
>         key usage: digitalSignature,keyEncipherment,dataEncipherment
>         eku: id-kp-serverAuth,id-kp-clientAuth
>         pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
>         post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
> "subsystemCert cert-pki-ca"
>         track: yes
>         auto-renew: yes
> Request ID '20210908000053':
>         status: MONITORING
>         stuck: no
>         key pair storage:
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
> cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
>         certificate:
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
> cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
>         CA: dogtag-ipa-ca-renew-agent
>         issuer: CN=Certificate Authority,O=<OU>
>         subject: CN=Certificate Authority,O=<OU>
>         expires: 2039-09-20 20:11:25 UTC
>         key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
>         pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
>         post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
> "caSigningCert cert-pki-ca"
>         track: yes
>         auto-renew: yes
> Request ID '20210908000054':
>         status: MONITORING
>         stuck: no
>         key pair storage: type=FILE,location='/var/lib/ipa/ra-agent.key'
>         certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem'
>         CA: dogtag-ipa-ca-renew-agent
>         issuer: CN=Certificate Authority,O=<OU>
>         subject: CN=IPA RA,O=<OU>
>         expires: 2025-06-26 02:36:15 UTC
>         key usage: digitalSignature,keyEncipherment,dataEncipherment
>         eku: id-kp-serverAuth,id-kp-clientAuth
>         pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre
>         post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert
>         track: yes
>         auto-renew: yes
> Request ID '20210908000055':
>         status: MONITORING
>         stuck: no
>         key pair storage:
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
> cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
>         certificate:
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
> cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
>         CA: dogtag-ipa-ca-renew-agent
>         issuer: CN=Certificate Authority,O=<OU>
>         subject: CN=<HOSTNAME>,O=<OU>
>         expires: 2025-07-21 02:36:37 UTC
>         dns: <HOSTNAME>
>         key usage: digitalSignature,keyEncipherment,dataEncipherment
>         eku: id-kp-serverAuth,id-kp-clientAuth
>         pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
>         post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
> "Server-Cert cert-pki-ca"
>         track: yes
>         auto-renew: yes
> Request ID '20210908000056':
>         status: CA_UNREACHABLE
>         ca-error: Server at https://<HOSTNAME>/ipa/xml failed request,
> will retry: 4035 (RPC failed at server.  Request failed with status 500:
> Non-2xx response from CA REST API: 500. String index out of range: -1).
>         stuck: no
>         key pair storage:
> type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> FIPS 140-2 Certificate DB',pinfile='/etc/dirsrv/slapd-<OU>/pwdfile.txt'
>         certificate:
> type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> FIPS 140-2 Certificate DB'
>         CA: IPA
>         issuer: CN=Certificate Authority,O=<OU>
>         subject: CN=<HOSTNAME>,O=<OU>
>         expires: 2023-09-03 18:30:48 UTC
>         dns: <HOSTNAME>
>         key usage:
> digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>         eku: id-kp-serverAuth,id-kp-clientAuth
>         pre-save command:
>         post-save command: /usr/libexec/ipa/certmonger/restart_dirsrv <OU>
>         track: yes
>         auto-renew: yes
> Request ID '20210908000057':
>         status: CA_UNREACHABLE
>         ca-error: Server at https://<HOSTNAME>/ipa/xml failed request,
> will retry: 4035 (RPC failed at server.  Request failed with status 500:
> Non-2xx response from CA REST API: 500. String index out of range: -1).
>         stuck: no
>         key pair storage:
> type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> FIPS 140-2 Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
>         certificate:
> type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> FIPS 140-2 Certificate DB'
>         CA: IPA
>         issuer: CN=Certificate Authority,O=<OU>
>         subject: CN=<HOSTNAME>,O=<OU>
>         expires: 2023-09-03 18:30:48 UTC
>         dns: <HOSTNAME>
>         key usage:
> digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>         eku: id-kp-serverAuth,id-kp-clientAuth
>         pre-save command:
>         post-save command: /usr/libexec/ipa/certmonger/restart_httpd
>         track: yes
>         auto-renew: yes
> Number of certificates and requests being tracked: 9.
> Request ID '20190920201259':
>         status: CA_UNREACHABLE
>         ca-error: Server at https://<HOSTNAME>/ipa/xml failed request,
> will retry: 4035 (RPC failed at server.  Request failed with status 500:
> Non-2xx response from CA REST API: 500. String index out of range: -1).
>         stuck: no
>         key pair storage:
> type=FILE,location='/var/kerberos/krb5kdc/kdc.key'
>         certificate: type=FILE,location='/var/kerberos/krb5kdc/kdc.crt'
>         CA: IPA
>         issuer: CN=Certificate Authority,O=<OU>
>         subject: CN=<HOSTNAME>,O=<OU>
>         expires: 2023-08-25 18:05:07 UTC
>         principal name: krbtgt/<OU>@<OU>
>         key usage:
> digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>         eku: id-kp-serverAuth,id-pkinit-KPKdc
>         pre-save command:
>         post-save command: /usr/libexec/ipa/certmonger/renew_kdc_cert
>         track: yes
>         auto-renew: yes
> Request ID '20210908000056':
>         status: CA_UNREACHABLE
>         ca-error: Server at https://<HOSTNAME>/ipa/xml failed request,
> will retry: 4035 (RPC failed at server.  Request failed with status 500:
> Non-2xx response from CA REST API: 500. String index out of range: -1).
>         stuck: no
>         key pair storage:
> type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> FIPS 140-2 Certificate DB',pinfile='/etc/dirsrv/slapd-<OU>/pwdfile.txt'
>         certificate:
> type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> FIPS 140-2 Certificate DB'
>         CA: IPA
>         issuer: CN=Certificate Authority,O=<OU>
>         subject: CN=<HOSTNAME>,O=<OU>
>         expires: 2023-09-03 18:30:48 UTC
>         dns: <HOSTNAME>
>         key usage:
> digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>         eku: id-kp-serverAuth,id-kp-clientAuth
>         pre-save command:
>         post-save command: /usr/libexec/ipa/certmonger/restart_dirsrv <OU>
>         track: yes
>         auto-renew: yes
> Request ID '20210908000057':
>         status: CA_UNREACHABLE
>         ca-error: Server at https://<HOSTNAME>/ipa/xml failed request,
> will retry: 4035 (RPC failed at server.  Request failed with status 500:
> Non-2xx response from CA REST API: 500. String index out of range: -1).
>         stuck: no
>         key pair storage:
> type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> FIPS 140-2 Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
>         certificate:
> type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> FIPS 140-2 Certificate DB'
>         CA: IPA
>         issuer: CN=Certificate Authority,O=<OU>
>         subject: CN=<HOSTNAME>,O=<OU>
>         expires: 2023-09-03 18:30:48 UTC
>         dns: <HOSTNAME>
>         key usage:
> digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>         eku: id-kp-serverAuth,id-kp-clientAuth
>         pre-save command:
>         post-save command: /usr/libexec/ipa/certmonger/restart_httpd
>         track: yes
>         auto-renew: yes
>
> FreeIPA-users mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedorahosted.org/archives/list/[e-mail]...
> Do not reply to spam, report it:
> https://pagure.io/fedora-infrastructure/new_issue

[IT Guy]

Hi Florence,

Thank you for your response. What does it mean if I run the ipa
certprofile-show command as outlined above and it just hangs? I don't think
there is any other way to see the settings you mentioned unless this
command is able to run right?

Many thanks,

Evan

On Fri, Sep 15, 2023 at 3:19 AM Florence Blanc-Renaud 
wrote:
> Hi,
> it seems that PKI is not happy with the subject name of the certificates.
> The failing certs are for KDC, dirsrv and httpd and they all use the same
> subject name constraint in their profile.
> Was any certificate profile modified (caIPAserviceCert or
>
> KDCs_PKINIT_Certs)? You can use
> ipa certprofile-show <name> --out /dev/stdout
> And then check the part related to Subject Name Constraint. In my default
> installation, I have
> policyset.serverCertSet.1.constraint.class_id=subjectNameConstraintImpl
> policyset.serverCertSet.1.constraint.name=Subject Name Constraint
> policyset.serverCertSet.1.constraint.params.accept=true
> policyset.serverCertSet.1.constraint.params.pattern=CN=[^,]+,.+
> policyset.serverCertSet.1.default.class_id=subjectNameDefaultImpl
> policyset.serverCertSet.1.default.name=Subject Name Default
> policyset.serverCertSet.1.default.params.name=CN=$
> request.req_subject_name.cn$, O=IPA.TEST
>
> which means that the subject name should match CN= followed by (anything
> except a comma) multiple times then a comma and any char multiple times.
> If the profile wasn't changed, can you check in
>
> /var/log/pki/pki-tomcat/ca/debug.$DATE.log the received certificate
> request? Does its subject match the pattern? The error message
> java.lang.StringIndexOutOfBoundsException: String index out of range: -1
> hints that an expected pattern was not found.
>
> flo
>
> On Thu, Sep 14, 2023 at 4:11 PM Evan G via FreeIPA-users <
> [e-mail]> wrote:
> > Hi Rob,
> >
> > When we start tomcat with the date rolled back, we are not seeing any
> > errors at all. All of the ipa services start up without issue. The problem
> > is in actually renewing the certs, when we do so we have seen many
> > different errors as we've been troubleshooting -- mostly this one:
> > `ca-error: Server at https://<HOSTNAME>/ipa/xml failed request, will
> > retry: 4035 (RPC failed at server.  Request failed with status 500: Non-2xx
> > response from CA REST API: 500. String index out of range:
> > -1).[02/Aug/2023:00:00:31][ajp-bio-127.0.0.1-8009-exec-2]: EnrollProfile:
> > populate: begins`
> >
> > When I restart certmonger after all services up, these are the errors
> > that I am seeing in the tomcat debug logs:
> > [02/Aug/2023:00:00:31][ajp-bio-127.0.0.1-8009-exec-2]: BasicProfile:
> > populate: policy setid =serverCertSet
> > [02/Aug/2023:00:00:31][ajp-bio-127.0.0.1-8009-exec-2]: EnrollDefault:
> > populate: SubjectNameDefault: start
> > java.lang.StringIndexOutOfBoundsException: String index out of range: -1
> >         at java.lang.String.substring(String.java:1967)
> >         at
> > com.netscape.certsrv.pattern.Pattern.substitute2(Pattern.java:132)
> >         at
> > com.netscape.cms.profile.def.EnrollDefault.mapPattern(EnrollDefault.java:815)
> >         at
> > com.netscape.cms.profile.def.SubjectNameDefault.populate(SubjectNameDefault.java:160)
> >         at
> > com.netscape.cms.profile.def.EnrollDefault.populate(EnrollDefault.java:226)
> >         at
> > com.netscape.cms.profile.common.BasicProfile.populate(BasicProfile.java:1114)
> >         at
> > com.netscape.cms.profile.common.EnrollProfile.populate(EnrollProfile.java:2626)
> >         at
> > com.netscape.cms.servlet.cert.CertProcessor.populateRequests(CertProcessor.java:379)
> >         at
> > com.netscape.cms.servlet.cert.EnrollmentProcessor.processEnrollment(EnrollmentProcessor.java:188)
> >         at
> > com.netscape.cms.servlet.cert.EnrollmentProcessor.processEnrollment(EnrollmentProcessor.java:96)
> >         at
> > com.netscape.cms.servlet.cert.CertRequestDAO.submitRequest(CertRequestDAO.java:197)
> >         at
> > org.dogtagpki.server.ca.rest.CertRequestService.enrollCert(CertRequestService.java:155)
> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >         at
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> >         at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >         at java.lang.reflect.Method.invoke(Method.java:498)
> >         at
> > org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:137)
> >         at
> > org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:280)
> >         at
> > org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:234)
> >         at
> > org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:221)
> >         at
> > org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:356)
> >         at
> > org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:179)
> >         at
> > org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:220)
> >         at
> > org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
> >         at
> > org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
> >         at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >         at
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> >         at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >         at java.lang.reflect.Method.invoke(Method.java:498)
> >         at
> > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)
> >         at
> > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
> >         at java.security.AccessController.doPrivileged(Native Method)
> >         at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
> >         at
> > org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)
> >         at
> > org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:175)
> >         at
> > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:297)
> >         at
> > org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:55)
> >         at
> > org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:191)
> >         at
> > org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:187)
> >         at java.security.AccessController.doPrivileged(Native Method)
> >         at
> > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:186)
> >         at
> > org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >         at
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> >         at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >         at java.lang.reflect.Method.invoke(Method.java:498)
> >         at
> > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)
> >         at
> > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
> >         at java.security.AccessController.doPrivileged(Native Method)
> >         at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
> >         at
> > org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)
> >         at
> > org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:260)
> >         at
> > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:237)
> >         at
> > org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:55)
> >         at
> > org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:191)
> >         at
> > org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:187)
> >         at java.security.AccessController.doPrivileged(Native Method)
> >         at
> > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:186)
> >         at
> > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:218)
> >         at
> > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
> >         at
> > org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:498)
> >         at
> > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
> >         at
> > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
> >         at
> > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962)
> >         at
> > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
> >         at
> > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)
> >         at
> > org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:190)
> >         at
> > org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
> >         at org.apache.tomcat.util.net
> > .JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
> >         at
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> >         at
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> >         at
> > org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> >         at java.lang.Thread.run(Thread.java:750)
> >
> > This is what we see when we run `getcert list` and `ipa-getcert list`
> > respectively:
> > Number of certificates and requests being tracked: 9.
> > Request ID '20190920201259':
> >         status: CA_UNREACHABLE
> >         ca-error: Server at https://<HOSTNAME>/ipa/xml failed request,
> > will retry: 4035 (RPC failed at server.  Request failed with status 500:
> > Non-2xx response from CA REST API: 500. String index out of range: -1).
> >         stuck: no
> >         key pair storage:
> > type=FILE,location='/var/kerberos/krb5kdc/kdc.key'
> >         certificate: type=FILE,location='/var/kerberos/krb5kdc/kdc.crt'
> >         CA: IPA
> >         issuer: CN=Certificate Authority,O=<OU>
> >         subject: CN=<HOSTNAME>,O=<OU>
> >         expires: 2023-08-25 18:05:07 UTC
> >         principal name: krbtgt/<OU>@<OU>
> >         key usage:
> > digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> >         eku: id-kp-serverAuth,id-pkinit-KPKdc
> >         pre-save command:
> >         post-save command: /usr/libexec/ipa/certmonger/renew_kdc_cert
> >         track: yes
> >         auto-renew: yes
> > Request ID '20210908000050':
> >         status: MONITORING
> >         stuck: no
> >         key pair storage:
> > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
> > cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
> >         certificate:
> > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
> > cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
> >         CA: dogtag-ipa-ca-renew-agent
> >         issuer: CN=Certificate Authority,O=<OU>
> >         subject: CN=CA Audit,O=<OU>
> >         expires: 2025-07-21 02:36:57 UTC
> >         key usage: digitalSignature,keyEncipherment,dataEncipherment
> >         eku: id-kp-serverAuth,id-kp-clientAuth
> >         pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
> >         post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
> > "auditSigningCert cert-pki-ca"
> >         track: yes
> >         auto-renew: yes
> > Request ID '20210908000051':
> >         status: MONITORING
> >         stuck: no
> >         key pair storage:
> > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
> > cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
> >         certificate:
> > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
> > cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
> >         CA: dogtag-ipa-ca-renew-agent
> >         issuer: CN=Certificate Authority,O=<OU>
> >         subject: CN=OCSP Subsystem,O=<OU>
> >         expires: 2025-07-21 02:36:17 UTC
> >         key usage: digitalSignature,keyEncipherment,dataEncipherment
> >         eku: id-kp-serverAuth,id-kp-clientAuth
> >         pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
> >         post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
> > "ocspSigningCert cert-pki-ca"
> >         track: yes
> >         auto-renew: yes
> > Request ID '20210908000052':
> >         status: MONITORING
> >         stuck: no
> >         key pair storage:
> > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
> > cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
> >         certificate:
> > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
> > cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
> >         CA: dogtag-ipa-ca-renew-agent
> >         issuer: CN=Certificate Authority,O=<OU>
> >         subject: CN=CA Subsystem,O=<OU>
> >         expires: 2025-07-21 02:37:17 UTC
> >         key usage: digitalSignature,keyEncipherment,dataEncipherment
> >         eku: id-kp-serverAuth,id-kp-clientAuth
> >         pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
> >         post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
> > "subsystemCert cert-pki-ca"
> >         track: yes
> >         auto-renew: yes
> > Request ID '20210908000053':
> >         status: MONITORING
> >         stuck: no
> >         key pair storage:
> > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
> > cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
> >         certificate:
> > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
> > cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
> >         CA: dogtag-ipa-ca-renew-agent
> >         issuer: CN=Certificate Authority,O=<OU>
> >         subject: CN=Certificate Authority,O=<OU>
> >         expires: 2039-09-20 20:11:25 UTC
> >         key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
> >         pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
> >         post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
> > "caSigningCert cert-pki-ca"
> >         track: yes
> >         auto-renew: yes
> > Request ID '20210908000054':
> >         status: MONITORING
> >         stuck: no
> >         key pair storage: type=FILE,location='/var/lib/ipa/ra-agent.key'
> >         certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem'
> >         CA: dogtag-ipa-ca-renew-agent
> >         issuer: CN=Certificate Authority,O=<OU>
> >         subject: CN=IPA RA,O=<OU>
> >         expires: 2025-06-26 02:36:15 UTC
> >         key usage: digitalSignature,keyEncipherment,dataEncipherment
> >         eku: id-kp-serverAuth,id-kp-clientAuth
> >         pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre
> >         post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert
> >         track: yes
> >         auto-renew: yes
> > Request ID '20210908000055':
> >         status: MONITORING
> >         stuck: no
> >         key pair storage:
> > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
> > cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
> >         certificate:
> > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
> > cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
> >         CA: dogtag-ipa-ca-renew-agent
> >         issuer: CN=Certificate Authority,O=<OU>
> >         subject: CN=<HOSTNAME>,O=<OU>
> >         expires: 2025-07-21 02:36:37 UTC
> >         dns: <HOSTNAME>
> >         key usage: digitalSignature,keyEncipherment,dataEncipherment
> >         eku: id-kp-serverAuth,id-kp-clientAuth
> >         pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
> >         post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
> > "Server-Cert cert-pki-ca"
> >         track: yes
> >         auto-renew: yes
> > Request ID '20210908000056':
> >         status: CA_UNREACHABLE
> >         ca-error: Server at https://<HOSTNAME>/ipa/xml failed request,
> > will retry: 4035 (RPC failed at server.  Request failed with status 500:
> > Non-2xx response from CA REST API: 500. String index out of range: -1).
> >         stuck: no
> >         key pair storage:
> > type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> > FIPS 140-2 Certificate DB',pinfile='/etc/dirsrv/slapd-<OU>/pwdfile.txt'
> >         certificate:
> > type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> > FIPS 140-2 Certificate DB'
> >         CA: IPA
> >         issuer: CN=Certificate Authority,O=<OU>
> >         subject: CN=<HOSTNAME>,O=<OU>
> >         expires: 2023-09-03 18:30:48 UTC
> >         dns: <HOSTNAME>
> >         key usage:
> > digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> >         eku: id-kp-serverAuth,id-kp-clientAuth
> >         pre-save command:
> >         post-save command: /usr/libexec/ipa/certmonger/restart_dirsrv <OU>
> >         track: yes
> >         auto-renew: yes
> > Request ID '20210908000057':
> >         status: CA_UNREACHABLE
> >         ca-error: Server at https://<HOSTNAME>/ipa/xml failed request,
> > will retry: 4035 (RPC failed at server.  Request failed with status 500:
> > Non-2xx response from CA REST API: 500. String index out of range: -1).
> >         stuck: no
> >         key pair storage:
> > type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> > FIPS 140-2 Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
> >         certificate:
> > type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> > FIPS 140-2 Certificate DB'
> >         CA: IPA
> >         issuer: CN=Certificate Authority,O=<OU>
> >         subject: CN=<HOSTNAME>,O=<OU>
> >         expires: 2023-09-03 18:30:48 UTC
> >         dns: <HOSTNAME>
> >         key usage:
> > digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> >         eku: id-kp-serverAuth,id-kp-clientAuth
> >         pre-save command:
> >         post-save command: /usr/libexec/ipa/certmonger/restart_httpd
> >         track: yes
> >         auto-renew: yes
> > Number of certificates and requests being tracked: 9.
> > Request ID '20190920201259':
> >         status: CA_UNREACHABLE
> >         ca-error: Server at https://<HOSTNAME>/ipa/xml failed request,
> > will retry: 4035 (RPC failed at server.  Request failed with status 500:
> > Non-2xx response from CA REST API: 500. String index out of range: -1).
> >         stuck: no
> >         key pair storage:
> > type=FILE,location='/var/kerberos/krb5kdc/kdc.key'
> >         certificate: type=FILE,location='/var/kerberos/krb5kdc/kdc.crt'
> >         CA: IPA
> >         issuer: CN=Certificate Authority,O=<OU>
> >         subject: CN=<HOSTNAME>,O=<OU>
> >         expires: 2023-08-25 18:05:07 UTC
> >         principal name: krbtgt/<OU>@<OU>
> >         key usage:
> > digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> >         eku: id-kp-serverAuth,id-pkinit-KPKdc
> >         pre-save command:
> >         post-save command: /usr/libexec/ipa/certmonger/renew_kdc_cert
> >         track: yes
> >         auto-renew: yes
> > Request ID '20210908000056':
> >         status: CA_UNREACHABLE
> >         ca-error: Server at https://<HOSTNAME>/ipa/xml failed request,
> > will retry: 4035 (RPC failed at server.  Request failed with status 500:
> > Non-2xx response from CA REST API: 500. String index out of range: -1).
> >         stuck: no
> >         key pair storage:
> > type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> > FIPS 140-2 Certificate DB',pinfile='/etc/dirsrv/slapd-<OU>/pwdfile.txt'
> >         certificate:
> > type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> > FIPS 140-2 Certificate DB'
> >         CA: IPA
> >         issuer: CN=Certificate Authority,O=<OU>
> >         subject: CN=<HOSTNAME>,O=<OU>
> >         expires: 2023-09-03 18:30:48 UTC
> >         dns: <HOSTNAME>
> >         key usage:
> > digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> >         eku: id-kp-serverAuth,id-kp-clientAuth
> >         pre-save command:
> >         post-save command: /usr/libexec/ipa/certmonger/restart_dirsrv <OU>
> >         track: yes
> >         auto-renew: yes
> > Request ID '20210908000057':
> >         status: CA_UNREACHABLE
> >         ca-error: Server at https://<HOSTNAME>/ipa/xml failed request,
> > will retry: 4035 (RPC failed at server.  Request failed with status 500:
> > Non-2xx response from CA REST API: 500. String index out of range: -1).
> >         stuck: no
> >         key pair storage:
> > type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> > FIPS 140-2 Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
> >         certificate:
> > type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> > FIPS 140-2 Certificate DB'
> >         CA: IPA
> >         issuer: CN=Certificate Authority,O=<OU>
> >         subject: CN=<HOSTNAME>,O=<OU>
> >         expires: 2023-09-03 18:30:48 UTC
> >         dns: <HOSTNAME>
> >         key usage:
> > digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> >         eku: id-kp-serverAuth,id-kp-clientAuth
> >         pre-save command:
> >         post-save command: /usr/libexec/ipa/certmonger/restart_httpd
> >         track: yes
> >         auto-renew: yes
> >
> > FreeIPA-users mailing list -- [e-mail]
> > To unsubscribe send an email to
> > [e-mail]
> > Fedora Code of Conduct:
> > https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
> > https://lists.fedorahosted.org/archives/list/[e-mail]...
> > Do not reply to spam, report it:
> > https://pagure.io/fedora-infrastructure/new_issue

[Rob Crittenden]

IT Guy via FreeIPA-users wrote:
> Hi Florence,
>
> Thank you for your response. What does it mean if I run the ipa
> certprofile-show command as outlined above and it just hangs? I don't
> think there is any other way to see the settings you mentioned unless
> this command is able to run right?

I can't explain why it would hang but you can get the profile directly
from LDAP:

$ ldapsearch -LLL -o ldif-wrap=no -x -D 'cn=directory manager' -W -b
cn=caIPAserviceCert,ou=certificateProfiles,ou=ca,o=ipaca
certProfileConfig > /tmp/profile

Edit this file and remove the dn value and 'certProfileConfig:: ' then
base64-decode the result.

The final really huge string should look something like:

YXV0aC5pbnN0YW5jZV9pZ...=

I used the coreutils base64 program to decode it:

$ base64 -d /tmp/profile

rob
> Many thanks,
>
> Evan
>
> On Fri, Sep 15, 2023 at 3:19 AM Florence Blanc-Renaud <[e-mail]
> > wrote:
> Hi,
> it seems that PKI is not happy with the subject name of the
> certificates.
> The failing certs are for KDC, dirsrv and httpd and they all use the
> same subject name constraint in their profile.
>
> 1. Was any certificate profile modified (caIPAserviceCert or
> KDCs_PKINIT_Certs)? You can use
> ipa certprofile-show <name> --out /dev/stdout
> And then check the part related to Subject Name Constraint. In my
> default installation, I have
> policyset.serverCertSet.1.constraint.class_id=subjectNameConstraintImpl
> policyset.serverCertSet.1.constraint.name
> <http://policyset.serverCertSet.1.constraint.name>=Subject Name
> Constraint
> policyset.serverCertSet.1.constraint.params.accept=true
> policyset.serverCertSet.1.constraint.params.pattern=CN=[^,]+,.+
> policyset.serverCertSet.1.default.class_id=subjectNameDefaultImpl
> policyset.serverCertSet.1.default.name
> <http://policyset.serverCertSet.1.default.name>=Subject Name Default
> policyset.serverCertSet.1.default.params.name
> <http://policyset.serverCertSet.1.default.params.name>=CN=$request.req_subject_name.cn
> <http://request.req_subject_name.cn>$, O=IPA.TEST
>
> which means that the subject name should match CN= followed by
> (anything except a comma) multiple times then a comma and any char
> multiple times.
>
> 2. If the profile wasn't changed, can you check in
> /var/log/pki/pki-tomcat/ca/debug.$DATE.log the received certificate
> request? Does its subject match the pattern? The error
> messagejava.lang.StringIndexOutOfBoundsException: String index out
> of range: -1 hints that an expected pattern was not found.
>
> flo
>
> On Thu, Sep 14, 2023 at 4:11 PM Evan G via FreeIPA-users
> <[e-mail]
> <mailto:[e-mail]>> wrote:
>
>     Hi Rob,
>
>     When we start tomcat with the date rolled back, we are not
>     seeing any errors at all. All of the ipa services start up
>     without issue. The problem is in actually renewing the certs,
>     when we do so we have seen many different errors as we've been
>     troubleshooting -- mostly this one: `ca-error: Server at
>     https://<HOSTNAME>/ipa/xml failed request, will retry: 4035 (RPC
>     failed at server.  Request failed with status 500: Non-2xx
>     response from CA REST API: 500. String index out of range:
>     -1).[02/Aug/2023:00:00:31][ajp-bio-127.0.0.1-8009-exec-2]:
>     EnrollProfile: populate: begins`
>
>     When I restart certmonger after all services up, these are the
>     errors that I am seeing in the tomcat debug logs:
>     ```
>     [02/Aug/2023:00:00:31][ajp-bio-127.0.0.1-8009-exec-2]:
>     BasicProfile: populate: policy setid =serverCertSet
>     [02/Aug/2023:00:00:31][ajp-bio-127.0.0.1-8009-exec-2]:
>     EnrollDefault: populate: SubjectNameDefault: start
>     java.lang.StringIndexOutOfBoundsException: String index out of
>     range: -1
>             at java.lang.String.substring(String.java:1967)
>             at
>     com.netscape.certsrv.pattern.Pattern.substitute2(Pattern.java:132)
>             at
>     com.netscape.cms.profile.def.EnrollDefault.mapPattern(EnrollDefault.java:815)
>             at
>     com.netscape.cms.profile.def.SubjectNameDefault.populate(SubjectNameDefault.java:160)
>             at
>     com.netscape.cms.profile.def.EnrollDefault.populate(EnrollDefault.java:226)
>             at
>     com.netscape.cms.profile.common.BasicProfile.populate(BasicProfile.java:1114)
>             at
>     com.netscape.cms.profile.common.EnrollProfile.populate(EnrollProfile.java:2626)
>             at
>     com.netscape.cms.servlet.cert.CertProcessor.populateRequests(CertProcessor.java:379)
>             at
>     com.netscape.cms.servlet.cert.EnrollmentProcessor.processEnrollment(EnrollmentProcessor.java:188)
>             at
>     com.netscape.cms.servlet.cert.EnrollmentProcessor.processEnrollment(EnrollmentProcessor.java:96)
>             at
>     com.netscape.cms.servlet.cert.CertRequestDAO.submitRequest(CertRequestDAO.java:197)
>             at
>     org.dogtagpki.server.ca.rest.CertRequestService.enrollCert(CertRequestService.java:155)
>             at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>     Method)
>             at
>     sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>             at
>     sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>             at java.lang.reflect.Method.invoke(Method.java:498)
>             at
>     org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:137)
>             at
>     org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:280)
>             at
>     org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:234)
>             at
>     org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:221)
>             at
>     org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:356)
>             at
>     org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:179)
>             at
>     org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:220)
>             at
>     org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
>             at
>     org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
>             at
>     javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
>             at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>     Method)
>             at
>     sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>             at
>     sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>             at java.lang.reflect.Method.invoke(Method.java:498)
>             at
>     org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)
>             at
>     org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
>             at java.security.AccessController.doPrivileged(Native
>     Method)
>             at
>     javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
>             at
>     org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)
>             at
>     org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:175)
>             at
>     org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:297)
>             at
>     org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:55)
>             at
>     org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:191)
>             at
>     org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:187)
>             at java.security.AccessController.doPrivileged(Native
>     Method)
>             at
>     org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:186)
>             at
>     org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
>             at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>     Method)
>             at
>     sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>             at
>     sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>             at java.lang.reflect.Method.invoke(Method.java:498)
>             at
>     org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)
>             at
>     org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
>             at java.security.AccessController.doPrivileged(Native
>     Method)
>             at
>     javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
>             at
>     org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)
>             at
>     org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:260)
>             at
>     org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:237)
>             at
>     org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:55)
>             at
>     org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:191)
>             at
>     org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:187)
>             at java.security.AccessController.doPrivileged(Native
>     Method)
>             at
>     org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:186)
>             at
>     org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:218)
>             at
>     org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
>             at
>     org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:498)
>             at
>     org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
>             at
>     org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
>             at
>     org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962)
>             at
>     org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
>             at
>     org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)
>             at
>     org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:190)
>             at
>     org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
>             at org.apache.tomcat.util.net
>     <http://org.apache.tomcat.util.net>.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
>             at
>     java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>             at
>     java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>             at
>     org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
>             at java.lang.Thread.run(Thread.java:750)
>     ```
>
>     This is what we see when we run `getcert list` and `ipa-getcert
>     list` respectively:
>
>     ```
>     Number of certificates and requests being tracked: 9.
>     Request ID '20190920201259':
>             status: CA_UNREACHABLE
>             ca-error: Server at https://<HOSTNAME>/ipa/xml failed
>     request, will retry: 4035 (RPC failed at server.  Request failed
>     with status 500: Non-2xx response from CA REST API: 500. String
>     index out of range: -1).
>             stuck: no
>             key pair storage:
>     type=FILE,location='/var/kerberos/krb5kdc/kdc.key'
>             certificate:
>     type=FILE,location='/var/kerberos/krb5kdc/kdc.crt'
>             CA: IPA
>             issuer: CN=Certificate Authority,O=<OU>
>             subject: CN=<HOSTNAME>,O=<OU>
>             expires: 2023-08-25 18:05:07 UTC
>             principal name: krbtgt/<OU>@<OU>
>             key usage:
>     digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>             eku: id-kp-serverAuth,id-pkinit-KPKdc
>             pre-save command:
>             post-save command:
>     /usr/libexec/ipa/certmonger/renew_kdc_cert
>             track: yes
>             auto-renew: yes
>     Request ID '20210908000050':
>             status: MONITORING
>             stuck: no
>             key pair storage:
>     type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
>     cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
>             certificate:
>     type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
>     cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
>             CA: dogtag-ipa-ca-renew-agent
>             issuer: CN=Certificate Authority,O=<OU>
>             subject: CN=CA Audit,O=<OU>
>             expires: 2025-07-21 02:36:57 UTC
>             key usage: digitalSignature,keyEncipherment,dataEncipherment
>             eku: id-kp-serverAuth,id-kp-clientAuth
>             pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
>             post-save command:
>     /usr/libexec/ipa/certmonger/renew_ca_cert "auditSigningCert
>     cert-pki-ca"
>             track: yes
>             auto-renew: yes
>     Request ID '20210908000051':
>             status: MONITORING
>             stuck: no
>             key pair storage:
>     type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
>     cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
>             certificate:
>     type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
>     cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
>             CA: dogtag-ipa-ca-renew-agent
>             issuer: CN=Certificate Authority,O=<OU>
>             subject: CN=OCSP Subsystem,O=<OU>
>             expires: 2025-07-21 02:36:17 UTC
>             key usage: digitalSignature,keyEncipherment,dataEncipherment
>             eku: id-kp-serverAuth,id-kp-clientAuth
>             pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
>             post-save command:
>     /usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert
>     cert-pki-ca"
>             track: yes
>             auto-renew: yes
>     Request ID '20210908000052':
>             status: MONITORING
>             stuck: no
>             key pair storage:
>     type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
>     cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
>             certificate:
>     type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
>     cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
>             CA: dogtag-ipa-ca-renew-agent
>             issuer: CN=Certificate Authority,O=<OU>
>             subject: CN=CA Subsystem,O=<OU>
>             expires: 2025-07-21 02:37:17 UTC
>             key usage: digitalSignature,keyEncipherment,dataEncipherment
>             eku: id-kp-serverAuth,id-kp-clientAuth
>             pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
>             post-save command:
>     /usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert
>     cert-pki-ca"
>             track: yes
>             auto-renew: yes
>     Request ID '20210908000053':
>             status: MONITORING
>             stuck: no
>             key pair storage:
>     type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
>     cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
>             certificate:
>     type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
>     cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
>             CA: dogtag-ipa-ca-renew-agent
>             issuer: CN=Certificate Authority,O=<OU>
>             subject: CN=Certificate Authority,O=<OU>
>             expires: 2039-09-20 20:11:25 UTC
>             key usage:
>     digitalSignature,nonRepudiation,keyCertSign,cRLSign
>             pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
>             post-save command:
>     /usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert
>     cert-pki-ca"
>             track: yes
>             auto-renew: yes
>     Request ID '20210908000054':
>             status: MONITORING
>             stuck: no
>             key pair storage:
>     type=FILE,location='/var/lib/ipa/ra-agent.key'
>             certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem'
>             CA: dogtag-ipa-ca-renew-agent
>             issuer: CN=Certificate Authority,O=<OU>
>             subject: CN=IPA RA,O=<OU>
>             expires: 2025-06-26 02:36:15 UTC
>             key usage: digitalSignature,keyEncipherment,dataEncipherment
>             eku: id-kp-serverAuth,id-kp-clientAuth
>             pre-save command:
>     /usr/libexec/ipa/certmonger/renew_ra_cert_pre
>             post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert
>             track: yes
>             auto-renew: yes
>     Request ID '20210908000055':
>             status: MONITORING
>             stuck: no
>             key pair storage:
>     type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
>     cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
>             certificate:
>     type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
>     cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
>             CA: dogtag-ipa-ca-renew-agent
>             issuer: CN=Certificate Authority,O=<OU>
>             subject: CN=<HOSTNAME>,O=<OU>
>             expires: 2025-07-21 02:36:37 UTC
>             dns: <HOSTNAME>
>             key usage: digitalSignature,keyEncipherment,dataEncipherment
>             eku: id-kp-serverAuth,id-kp-clientAuth
>             pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
>             post-save command:
>     /usr/libexec/ipa/certmonger/renew_ca_cert "Server-Cert cert-pki-ca"
>             track: yes
>             auto-renew: yes
>     Request ID '20210908000056':
>             status: CA_UNREACHABLE
>             ca-error: Server at https://<HOSTNAME>/ipa/xml failed
>     request, will retry: 4035 (RPC failed at server.  Request failed
>     with status 500: Non-2xx response from CA REST API: 500. String
>     index out of range: -1).
>             stuck: no
>             key pair storage:
>     type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
>     FIPS 140-2 Certificate
>     DB',pinfile='/etc/dirsrv/slapd-<OU>/pwdfile.txt'
>             certificate:
>     type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
>     FIPS 140-2 Certificate DB'
>             CA: IPA
>             issuer: CN=Certificate Authority,O=<OU>
>             subject: CN=<HOSTNAME>,O=<OU>
>             expires: 2023-09-03 18:30:48 UTC
>             dns: <HOSTNAME>
>             key usage:
>     digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>             eku: id-kp-serverAuth,id-kp-clientAuth
>             pre-save command:
>             post-save command:
>     /usr/libexec/ipa/certmonger/restart_dirsrv <OU>
>             track: yes
>             auto-renew: yes
>     Request ID '20210908000057':
>             status: CA_UNREACHABLE
>             ca-error: Server at https://<HOSTNAME>/ipa/xml failed
>     request, will retry: 4035 (RPC failed at server.  Request failed
>     with status 500: Non-2xx response from CA REST API: 500. String
>     index out of range: -1).
>             stuck: no
>             key pair storage:
>     type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
>     FIPS 140-2 Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
>             certificate:
>     type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
>     FIPS 140-2 Certificate DB'
>             CA: IPA
>             issuer: CN=Certificate Authority,O=<OU>
>             subject: CN=<HOSTNAME>,O=<OU>
>             expires: 2023-09-03 18:30:48 UTC
>             dns: <HOSTNAME>
>             key usage:
>     digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>             eku: id-kp-serverAuth,id-kp-clientAuth
>             pre-save command:
>             post-save command: /usr/libexec/ipa/certmonger/restart_httpd
>             track: yes
>             auto-renew: yes
>     ```
>
>     ```
>     Number of certificates and requests being tracked: 9.
>     Request ID '20190920201259':
>             status: CA_UNREACHABLE
>             ca-error: Server at https://<HOSTNAME>/ipa/xml failed
>     request, will retry: 4035 (RPC failed at server.  Request failed
>     with status 500: Non-2xx response from CA REST API: 500. String
>     index out of range: -1).
>             stuck: no
>             key pair storage:
>     type=FILE,location='/var/kerberos/krb5kdc/kdc.key'
>             certificate:
>     type=FILE,location='/var/kerberos/krb5kdc/kdc.crt'
>             CA: IPA
>             issuer: CN=Certificate Authority,O=<OU>
>             subject: CN=<HOSTNAME>,O=<OU>
>             expires: 2023-08-25 18:05:07 UTC
>             principal name: krbtgt/<OU>@<OU>
>             key usage:
>     digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>             eku: id-kp-serverAuth,id-pkinit-KPKdc
>             pre-save command:
>             post-save command:
>     /usr/libexec/ipa/certmonger/renew_kdc_cert
>             track: yes
>             auto-renew: yes
>     Request ID '20210908000056':
>             status: CA_UNREACHABLE
>             ca-error: Server at https://<HOSTNAME>/ipa/xml failed
>     request, will retry: 4035 (RPC failed at server.  Request failed
>     with status 500: Non-2xx response from CA REST API: 500. String
>     index out of range: -1).
>             stuck: no
>             key pair storage:
>     type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
>     FIPS 140-2 Certificate
>     DB',pinfile='/etc/dirsrv/slapd-<OU>/pwdfile.txt'
>             certificate:
>     type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
>     FIPS 140-2 Certificate DB'
>             CA: IPA
>             issuer: CN=Certificate Authority,O=<OU>
>             subject: CN=<HOSTNAME>,O=<OU>
>             expires: 2023-09-03 18:30:48 UTC
>             dns: <HOSTNAME>
>             key usage:
>     digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>             eku: id-kp-serverAuth,id-kp-clientAuth
>             pre-save command:
>             post-save command:
>     /usr/libexec/ipa/certmonger/restart_dirsrv <OU>
>             track: yes
>             auto-renew: yes
>     Request ID '20210908000057':
>             status: CA_UNREACHABLE
>             ca-error: Server at https://<HOSTNAME>/ipa/xml failed
>     request, will retry: 4035 (RPC failed at server.  Request failed
>     with status 500: Non-2xx response from CA REST API: 500. String
>     index out of range: -1).
>             stuck: no
>             key pair storage:
>     type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
>     FIPS 140-2 Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
>             certificate:
>     type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
>     FIPS 140-2 Certificate DB'
>             CA: IPA
>             issuer: CN=Certificate Authority,O=<OU>
>             subject: CN=<HOSTNAME>,O=<OU>
>             expires: 2023-09-03 18:30:48 UTC
>             dns: <HOSTNAME>
>             key usage:
>     digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>             eku: id-kp-serverAuth,id-kp-clientAuth
>             pre-save command:
>             post-save command: /usr/libexec/ipa/certmonger/restart_httpd
>             track: yes
>             auto-renew: yes
>     ```
>     _______________________________________________
>     FreeIPA-users mailing list --
>     [e-mail]
>     <mailto:[e-mail]>
>     To unsubscribe send an email to
>     [e-mail]
>     <mailto:[e-mail]>
>     Fedora Code of Conduct:
>     https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>     List Guidelines:
>     https://fedoraproject.org/wiki/Mailing_list_guidelines
>     List Archives:
>     https://lists.fedorahosted.org/archives/list/[e-mail]
>     Do not reply to spam, report it:
>     https://pagure.io/fedora-infrastructure/new_issue
>
>
> FreeIPA-users mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedorahosted.org/archives/list/[e-mail]...
> Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue

[IT Guy]

Wow that worked Rob, thank you! If I compare the values that Florence sent
to what I have in this file, the only difference is this line:

policyset.serverCertSet.1.default.params.name=CN=$$
request.req_subject_name.cn$$, $SUBJECT_DN_O

Here's the full snippet for reference:

policyset.serverCertSet.1.constraint.class_id=subjectNameConstraintImpl
policyset.serverCertSet.1.constraint.name=Subject Name Constraint
policyset.serverCertSet.1.constraint.params.accept=true
policyset.serverCertSet.1.constraint.params.pattern=CN=[^,]+,.+
policyset.serverCertSet.1.default.class_id=subjectNameDefaultImpl
policyset.serverCertSet.1.default.name=Subject Name Default
policyset.serverCertSet.1.default.params.name=CN=$$
request.req_subject_name.cn$$, $SUBJECT_DN_O

One other thing I wanted to call out is that I have a good snapshot of this
server that I have restored a couple of times to try different things and
the one that got me the farthest was when I changed the name of the cert
from our custom name back to Server-Cert. Even when I had the config this
way I still could not renew but maybe modifying something in the above
config plus changing back to Server-Cert could alleviate the issue?

Many thanks,

Evan

On Fri, Sep 15, 2023 at 9:47 AM Rob Crittenden  wrote:
> IT Guy via FreeIPA-users wrote:
> > Hi Florence,
> >
> > Thank you for your response. What does it mean if I run the ipa
> > certprofile-show command as outlined above and it just hangs? I don't
> > think there is any other way to see the settings you mentioned unless
> > this command is able to run right?
>
> I can't explain why it would hang but you can get the profile directly
> from LDAP:
>
> $ ldapsearch -LLL -o ldif-wrap=no -x -D 'cn=directory manager' -W -b
> cn=caIPAserviceCert,ou=certificateProfiles,ou=ca,o=ipaca
> certProfileConfig > /tmp/profile
>
> Edit this file and remove the dn value and 'certProfileConfig:: ' then
> base64-decode the result.
>
> The final really huge string should look something like:
>
> YXV0aC5pbnN0YW5jZV9pZ...=
>
> I used the coreutils base64 program to decode it:
>
> $ base64 -d /tmp/profile
>
> rob
> > Many thanks,
> >
> > Evan
> >
> > On Fri, Sep 15, 2023 at 3:19 AM Florence Blanc-Renaud <[e-mail]
> > > wrote:
> > Hi,
> > it seems that PKI is not happy with the subject name of the
> > certificates.
> > The failing certs are for KDC, dirsrv and httpd and they all use the
> > same subject name constraint in their profile.
> >
> > 1. Was any certificate profile modified (caIPAserviceCert or
> > KDCs_PKINIT_Certs)? You can use
> > ipa certprofile-show <name> --out /dev/stdout
> > And then check the part related to Subject Name Constraint. In my
> > default installation, I have
>
> policyset.serverCertSet.1.constraint.class_id=subjectNameConstraintImpl
> > policyset.serverCertSet.1.constraint.name
> > <http://policyset.serverCertSet.1.constraint.name>=Subject Name
> > Constraint
> > policyset.serverCertSet.1.constraint.params.accept=true
> > policyset.serverCertSet.1.constraint.params.pattern=CN=[^,]+,.+
> > policyset.serverCertSet.1.default.class_id=subjectNameDefaultImpl
> > policyset.serverCertSet.1.default.name
> > <http://policyset.serverCertSet.1.default.name>=Subject Name Default
> > policyset.serverCertSet.1.default.params.name
> > <http://policyset.serverCertSet.1.default.params.name>=CN=$
>
> request.req_subject_name.cn
> > <http://request.req_subject_name.cn>$, O=IPA.TEST
> >
> > which means that the subject name should match CN= followed by
> > (anything except a comma) multiple times then a comma and any char
> > multiple times.
> >
> > 2. If the profile wasn't changed, can you check in
> > /var/log/pki/pki-tomcat/ca/debug.$DATE.log the received certificate
> > request? Does its subject match the pattern? The error
> > messagejava.lang.StringIndexOutOfBoundsException: String index out
> > of range: -1 hints that an expected pattern was not found.
> >
> > flo
> >
> > On Thu, Sep 14, 2023 at 4:11 PM Evan G via FreeIPA-users
> > <[e-mail]
> > <mailto:[e-mail]>> wrote:
> >
> >     Hi Rob,
> >
> >     When we start tomcat with the date rolled back, we are not
> >     seeing any errors at all. All of the ipa services start up
> >     without issue. The problem is in actually renewing the certs,
> >     when we do so we have seen many different errors as we've been
> >     troubleshooting -- mostly this one: `ca-error: Server at
> >     https://<HOSTNAME>/ipa/xml failed request, will retry: 4035 (RPC
> >     failed at server.  Request failed with status 500: Non-2xx
> >     response from CA REST API: 500. String index out of range:
> >     -1).[02/Aug/2023:00:00:31][ajp-bio-127.0.0.1-8009-exec-2]:
> >     EnrollProfile: populate: begins`
> >
> >     When I restart certmonger after all services up, these are the
> >     errors that I am seeing in the tomcat debug logs:
> >     ```
> >     [02/Aug/2023:00:00:31][ajp-bio-127.0.0.1-8009-exec-2]:
> >     BasicProfile: populate: policy setid =serverCertSet
> >     [02/Aug/2023:00:00:31][ajp-bio-127.0.0.1-8009-exec-2]:
> >     EnrollDefault: populate: SubjectNameDefault: start
> >     java.lang.StringIndexOutOfBoundsException: String index out of
> >     range: -1
> >             at java.lang.String.substring(String.java:1967)
> >             at
>
> com.netscape.certsrv.pattern.Pattern.substitute2(Pattern.java:132)
> >             at
>
> com.netscape.cms.profile.def.EnrollDefault.mapPattern(EnrollDefault.java:815)
> >             at
>
> com.netscape.cms.profile.def.SubjectNameDefault.populate(SubjectNameDefault.java:160)
> >             at
>
> com.netscape.cms.profile.def.EnrollDefault.populate(EnrollDefault.java:226)
> >             at
>
> com.netscape.cms.profile.common.BasicProfile.populate(BasicProfile.java:1114)
> >             at
>
> com.netscape.cms.profile.common.EnrollProfile.populate(EnrollProfile.java:2626)
> >             at
>
> com.netscape.cms.servlet.cert.CertProcessor.populateRequests(CertProcessor.java:379)
> >             at
>
> com.netscape.cms.servlet.cert.EnrollmentProcessor.processEnrollment(EnrollmentProcessor.java:188)
> >             at
>
> com.netscape.cms.servlet.cert.EnrollmentProcessor.processEnrollment(EnrollmentProcessor.java:96)
> >             at
>
> com.netscape.cms.servlet.cert.CertRequestDAO.submitRequest(CertRequestDAO.java:197)
> >             at
>
> org.dogtagpki.server.ca.rest.CertRequestService.enrollCert(CertRequestService.java:155)
> >             at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> >     Method)
> >             at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> >             at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >             at java.lang.reflect.Method.invoke(Method.java:498)
> >             at
>
> org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:137)
> >             at
>
> org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:280)
> >             at
>
> org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:234)
> >             at
>
> org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:221)
> >             at
>
> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:356)
> >             at
>
> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:179)
> >             at
>
> org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:220)
> >             at
>
> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
> >             at
>
> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
> >             at
> >     javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
> >             at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> >     Method)
> >             at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> >             at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >             at java.lang.reflect.Method.invoke(Method.java:498)
> >             at
>
> org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)
> >             at
>
> org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
> >             at java.security.AccessController.doPrivileged(Native
> >     Method)
> >             at
> >     javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
> >             at
>
> org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)
> >             at
>
> org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:175)
> >             at
>
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:297)
> >             at
>
> org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:55)
> >             at
>
> org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:191)
> >             at
>
> org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:187)
> >             at java.security.AccessController.doPrivileged(Native
> >     Method)
> >             at
>
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:186)
> >             at
>
> org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
> >             at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> >     Method)
> >             at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> >             at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >             at java.lang.reflect.Method.invoke(Method.java:498)
> >             at
>
> org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)
> >             at
>
> org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
> >             at java.security.AccessController.doPrivileged(Native
> >     Method)
> >             at
> >     javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
> >             at
>
> org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)
> >             at
>
> org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:260)
> >             at
>
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:237)
> >             at
>
> org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:55)
> >             at
>
> org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:191)
> >             at
>
> org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:187)
> >             at java.security.AccessController.doPrivileged(Native
> >     Method)
> >             at
>
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:186)
> >             at
>
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:218)
> >             at
>
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
> >             at
>
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:498)
> >             at
>
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
> >             at
>
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
> >             at
>
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962)
> >             at
>
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
> >             at
>
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)
> >             at
> >     org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:190)
> >             at
>
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
> >             at org.apache.tomcat.util.net
> >     <http://org.apache.tomcat.util.net
> >
> > .JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
> >                 at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> >             at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> >             at
>
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> >             at java.lang.Thread.run(Thread.java:750)
> >     ```
> >
> >     This is what we see when we run `getcert list` and `ipa-getcert
> >     list` respectively:
> >
> >     ```
> >     Number of certificates and requests being tracked: 9.
> >     Request ID '20190920201259':
> >             status: CA_UNREACHABLE
> >             ca-error: Server at https://<HOSTNAME>/ipa/xml failed
> >     request, will retry: 4035 (RPC failed at server.  Request failed
> >     with status 500: Non-2xx response from CA REST API: 500. String
> >     index out of range: -1).
> >             stuck: no
> >             key pair storage:
> >     type=FILE,location='/var/kerberos/krb5kdc/kdc.key'
> >             certificate:
> >     type=FILE,location='/var/kerberos/krb5kdc/kdc.crt'
> >             CA: IPA
> >             issuer: CN=Certificate Authority,O=<OU>
> >             subject: CN=<HOSTNAME>,O=<OU>
> >             expires: 2023-08-25 18:05:07 UTC
> >             principal name: krbtgt/<OU>@<OU>
> >             key usage:
> >     digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> >             eku: id-kp-serverAuth,id-pkinit-KPKdc
> >             pre-save command:
> >             post-save command:
> >     /usr/libexec/ipa/certmonger/renew_kdc_cert
> >             track: yes
> >             auto-renew: yes
> >     Request ID '20210908000050':
> >             status: MONITORING
> >             stuck: no
> >             key pair storage:
>
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
> >     cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
> >             certificate:
>
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
> >     cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
> >             CA: dogtag-ipa-ca-renew-agent
> >             issuer: CN=Certificate Authority,O=<OU>
> >             subject: CN=CA Audit,O=<OU>
> >             expires: 2025-07-21 02:36:57 UTC
> >             key usage:
>
> digitalSignature,keyEncipherment,dataEncipherment
> >             eku: id-kp-serverAuth,id-kp-clientAuth
> >             pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
> >             post-save command:
> >     /usr/libexec/ipa/certmonger/renew_ca_cert "auditSigningCert
> >     cert-pki-ca"
> >             track: yes
> >             auto-renew: yes
> >     Request ID '20210908000051':
> >             status: MONITORING
> >             stuck: no
> >             key pair storage:
>
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
> >     cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
> >             certificate:
>
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
> >     cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
> >             CA: dogtag-ipa-ca-renew-agent
> >             issuer: CN=Certificate Authority,O=<OU>
> >             subject: CN=OCSP Subsystem,O=<OU>
> >             expires: 2025-07-21 02:36:17 UTC
> >             key usage:
>
> digitalSignature,keyEncipherment,dataEncipherment
> >             eku: id-kp-serverAuth,id-kp-clientAuth
> >             pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
> >             post-save command:
> >     /usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert
> >     cert-pki-ca"
> >             track: yes
> >             auto-renew: yes
> >     Request ID '20210908000052':
> >             status: MONITORING
> >             stuck: no
> >             key pair storage:
>
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
> >     cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
> >             certificate:
>
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
> >     cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
> >             CA: dogtag-ipa-ca-renew-agent
> >             issuer: CN=Certificate Authority,O=<OU>
> >             subject: CN=CA Subsystem,O=<OU>
> >             expires: 2025-07-21 02:37:17 UTC
> >             key usage:
>
> digitalSignature,keyEncipherment,dataEncipherment
> >             eku: id-kp-serverAuth,id-kp-clientAuth
> >             pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
> >             post-save command:
> >     /usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert
> >     cert-pki-ca"
> >             track: yes
> >             auto-renew: yes
> >     Request ID '20210908000053':
> >             status: MONITORING
> >             stuck: no
> >             key pair storage:
>
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
> >     cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
> >             certificate:
>
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
> >     cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
> >             CA: dogtag-ipa-ca-renew-agent
> >             issuer: CN=Certificate Authority,O=<OU>
> >             subject: CN=Certificate Authority,O=<OU>
> >             expires: 2039-09-20 20:11:25 UTC
> >             key usage:
> >     digitalSignature,nonRepudiation,keyCertSign,cRLSign
> >             pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
> >             post-save command:
> >     /usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert
> >     cert-pki-ca"
> >             track: yes
> >             auto-renew: yes
> >     Request ID '20210908000054':
> >             status: MONITORING
> >             stuck: no
> >             key pair storage:
> >     type=FILE,location='/var/lib/ipa/ra-agent.key'
> >             certificate:
>
> type=FILE,location='/var/lib/ipa/ra-agent.pem'
> >             CA: dogtag-ipa-ca-renew-agent
> >             issuer: CN=Certificate Authority,O=<OU>
> >             subject: CN=IPA RA,O=<OU>
> >             expires: 2025-06-26 02:36:15 UTC
> >             key usage:
>
> digitalSignature,keyEncipherment,dataEncipherment
> >             eku: id-kp-serverAuth,id-kp-clientAuth
> >             pre-save command:
> >     /usr/libexec/ipa/certmonger/renew_ra_cert_pre
> >             post-save command:
>
> /usr/libexec/ipa/certmonger/renew_ra_cert
> >             track: yes
> >             auto-renew: yes
> >     Request ID '20210908000055':
> >             status: MONITORING
> >             stuck: no
> >             key pair storage:
>
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
> >     cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
> >             certificate:
>
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
> >     cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
> >             CA: dogtag-ipa-ca-renew-agent
> >             issuer: CN=Certificate Authority,O=<OU>
> >             subject: CN=<HOSTNAME>,O=<OU>
> >             expires: 2025-07-21 02:36:37 UTC
> >             dns: <HOSTNAME>
> >             key usage:
>
> digitalSignature,keyEncipherment,dataEncipherment
> >             eku: id-kp-serverAuth,id-kp-clientAuth
> >             pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
> >             post-save command:
> >     /usr/libexec/ipa/certmonger/renew_ca_cert "Server-Cert
>
> cert-pki-ca"
> >             track: yes
> >             auto-renew: yes
> >     Request ID '20210908000056':
> >             status: CA_UNREACHABLE
> >             ca-error: Server at https://<HOSTNAME>/ipa/xml failed
> >     request, will retry: 4035 (RPC failed at server.  Request failed
> >     with status 500: Non-2xx response from CA REST API: 500. String
> >     index out of range: -1).
> >             stuck: no
> >             key pair storage:
>
> type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> >     FIPS 140-2 Certificate
> >     DB',pinfile='/etc/dirsrv/slapd-<OU>/pwdfile.txt'
> >             certificate:
>
> type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> >     FIPS 140-2 Certificate DB'
> >             CA: IPA
> >             issuer: CN=Certificate Authority,O=<OU>
> >             subject: CN=<HOSTNAME>,O=<OU>
> >             expires: 2023-09-03 18:30:48 UTC
> >             dns: <HOSTNAME>
> >             key usage:
> >     digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> >             eku: id-kp-serverAuth,id-kp-clientAuth
> >             pre-save command:
> >             post-save command:
> >     /usr/libexec/ipa/certmonger/restart_dirsrv <OU>
> >             track: yes
> >             auto-renew: yes
> >     Request ID '20210908000057':
> >             status: CA_UNREACHABLE
> >             ca-error: Server at https://<HOSTNAME>/ipa/xml failed
> >     request, will retry: 4035 (RPC failed at server.  Request failed
> >     with status 500: Non-2xx response from CA REST API: 500. String
> >     index out of range: -1).
> >             stuck: no
> >             key pair storage:
>
> type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> >     FIPS 140-2 Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
> >             certificate:
>
> type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> >     FIPS 140-2 Certificate DB'
> >             CA: IPA
> >             issuer: CN=Certificate Authority,O=<OU>
> >             subject: CN=<HOSTNAME>,O=<OU>
> >             expires: 2023-09-03 18:30:48 UTC
> >             dns: <HOSTNAME>
> >             key usage:
> >     digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> >             eku: id-kp-serverAuth,id-kp-clientAuth
> >             pre-save command:
> >             post-save command:
>
> /usr/libexec/ipa/certmonger/restart_httpd
> >             track: yes
> >             auto-renew: yes
> >     ```
> >
> >     ```
> >     Number of certificates and requests being tracked: 9.
> >     Request ID '20190920201259':
> >             status: CA_UNREACHABLE
> >             ca-error: Server at https://<HOSTNAME>/ipa/xml failed
> >     request, will retry: 4035 (RPC failed at server.  Request failed
> >     with status 500: Non-2xx response from CA REST API: 500. String
> >     index out of range: -1).
> >             stuck: no
> >             key pair storage:
> >     type=FILE,location='/var/kerberos/krb5kdc/kdc.key'
> >             certificate:
> >     type=FILE,location='/var/kerberos/krb5kdc/kdc.crt'
> >             CA: IPA
> >             issuer: CN=Certificate Authority,O=<OU>
> >             subject: CN=<HOSTNAME>,O=<OU>
> >             expires: 2023-08-25 18:05:07 UTC
> >             principal name: krbtgt/<OU>@<OU>
> >             key usage:
> >     digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> >             eku: id-kp-serverAuth,id-pkinit-KPKdc
> >             pre-save command:
> >             post-save command:
> >     /usr/libexec/ipa/certmonger/renew_kdc_cert
> >             track: yes
> >             auto-renew: yes
> >     Request ID '20210908000056':
> >             status: CA_UNREACHABLE
> >             ca-error: Server at https://<HOSTNAME>/ipa/xml failed
> >     request, will retry: 4035 (RPC failed at server.  Request failed
> >     with status 500: Non-2xx response from CA REST API: 500. String
> >     index out of range: -1).
> >             stuck: no
> >             key pair storage:
>
> type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> >     FIPS 140-2 Certificate
> >     DB',pinfile='/etc/dirsrv/slapd-<OU>/pwdfile.txt'
> >             certificate:
>
> type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> >     FIPS 140-2 Certificate DB'
> >             CA: IPA
> >             issuer: CN=Certificate Authority,O=<OU>
> >             subject: CN=<HOSTNAME>,O=<OU>
> >             expires: 2023-09-03 18:30:48 UTC
> >             dns: <HOSTNAME>
> >             key usage:
> >     digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> >             eku: id-kp-serverAuth,id-kp-clientAuth
> >             pre-save command:
> >             post-save command:
> >     /usr/libexec/ipa/certmonger/restart_dirsrv <OU>
> >             track: yes
> >             auto-renew: yes
> >     Request ID '20210908000057':
> >             status: CA_UNREACHABLE
> >             ca-error: Server at https://<HOSTNAME>/ipa/xml failed
> >     request, will retry: 4035 (RPC failed at server.  Request failed
> >     with status 500: Non-2xx response from CA REST API: 500. String
> >     index out of range: -1).
> >             stuck: no
> >             key pair storage:
>
> type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> >     FIPS 140-2 Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
> >             certificate:
>
> type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> >     FIPS 140-2 Certificate DB'
> >             CA: IPA
> >             issuer: CN=Certificate Authority,O=<OU>
> >             subject: CN=<HOSTNAME>,O=<OU>
> >             expires: 2023-09-03 18:30:48 UTC
> >             dns: <HOSTNAME>
> >             key usage:
> >     digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> >             eku: id-kp-serverAuth,id-kp-clientAuth
> >             pre-save command:
> >             post-save command:
>
> /usr/libexec/ipa/certmonger/restart_httpd
> >             track: yes
> >             auto-renew: yes
> >     ```
> >     _______________________________________________
> >     FreeIPA-users mailing list --
> >     [e-mail]
> >     <mailto:[e-mail]>
> >     To unsubscribe send an email to
> >     [e-mail]
> >     <mailto:[e-mail]>
> >     Fedora Code of Conduct:
> >     https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> >     List Guidelines:
> >     https://fedoraproject.org/wiki/Mailing_list_guidelines
> >     List Archives:
>
> https://lists.fedorahosted.org/archives/list/[e-mail]...
> >     Do not reply to spam, report it:
> >     https://pagure.io/fedora-infrastructure/new_issue
> >
> >
> > FreeIPA-users mailing list -- [e-mail]
> > To unsubscribe send an email to
>
> [e-mail]
> > Fedora Code of Conduct:
>
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
>
> https://lists.fedorahosted.org/archives/list/[e-mail]...
> > Do not reply to spam, report it:
>
> https://pagure.io/fedora-infrastructure/new_issue
> >

[IT Guy]

OK just one more thing to add, I had run across this link during
troubleshooting and it seems that my co-worker had updated some of the
lines in this configuration according to the steps outlined in this forum
post: https://pagure.io/freeipa/issue/7267

However I can say that this was a last ditch effort to try and get the
renewals working, we had already been troubleshooting for 3+ days at the
point that this was changed.

On Fri, Sep 15, 2023 at 9:58 AM IT Guy 
wrote:
> Wow that worked Rob, thank you! If I compare the values that Florence sent
> to what I have in this file, the only difference is this line:
>
> policyset.serverCertSet.1.default.params.name=CN=$$
> request.req_subject_name.cn$$, $SUBJECT_DN_O
>
> Here's the full snippet for reference:
>
> policyset.serverCertSet.1.constraint.class_id=subjectNameConstraintImpl
> policyset.serverCertSet.1.constraint.name=Subject Name Constraint
> policyset.serverCertSet.1.constraint.params.accept=true
> policyset.serverCertSet.1.constraint.params.pattern=CN=[^,]+,.+
> policyset.serverCertSet.1.default.class_id=subjectNameDefaultImpl
> policyset.serverCertSet.1.default.name=Subject Name Default
> policyset.serverCertSet.1.default.params.name=CN=$$
> request.req_subject_name.cn$$, $SUBJECT_DN_O
>
> One other thing I wanted to call out is that I have a good snapshot of
> this server that I have restored a couple of times to try different things
> and the one that got me the farthest was when I changed the name of the
> cert from our custom name back to Server-Cert. Even when I had the config
> this way I still could not renew but maybe modifying something in the above
> config plus changing back to Server-Cert could alleviate the issue?
>
> Many thanks,
>
> Evan
>
> On Fri, Sep 15, 2023 at 9:47 AM Rob Crittenden 
> wrote:
> > IT Guy via FreeIPA-users wrote:
> > > Hi Florence,
> > >
> > > Thank you for your response. What does it mean if I run the ipa
> > > certprofile-show command as outlined above and it just hangs? I don't
> > > think there is any other way to see the settings you mentioned unless
> > > this command is able to run right?
> >
> > I can't explain why it would hang but you can get the profile directly
> > from LDAP:
> >
> > $ ldapsearch -LLL -o ldif-wrap=no -x -D 'cn=directory manager' -W -b
> > cn=caIPAserviceCert,ou=certificateProfiles,ou=ca,o=ipaca
> > certProfileConfig > /tmp/profile
> >
> > Edit this file and remove the dn value and 'certProfileConfig:: ' then
> > base64-decode the result.
> >
> > The final really huge string should look something like:
> >
> > YXV0aC5pbnN0YW5jZV9pZ...=
> >
> > I used the coreutils base64 program to decode it:
> >
> > $ base64 -d /tmp/profile
> >
> > rob
> > > Many thanks,
> > >
> > > Evan
> > >
> > > On Fri, Sep 15, 2023 at 3:19 AM Florence Blanc-Renaud <[e-mail]
> > > > wrote:
> > > Hi,
> > > it seems that PKI is not happy with the subject name of the
> > > certificates.
> > > The failing certs are for KDC, dirsrv and httpd and they all use the
> > > same subject name constraint in their profile.
> > >
> > > 1. Was any certificate profile modified (caIPAserviceCert or
> > > KDCs_PKINIT_Certs)? You can use
> > > ipa certprofile-show <name> --out /dev/stdout
> > > And then check the part related to Subject Name Constraint. In my
> > > default installation, I have
> >
> > policyset.serverCertSet.1.constraint.class_id=subjectNameConstraintImpl
> > > policyset.serverCertSet.1.constraint.name
> > > <http://policyset.serverCertSet.1.constraint.name>=Subject Name
> > > Constraint
> > > policyset.serverCertSet.1.constraint.params.accept=true
> > > policyset.serverCertSet.1.constraint.params.pattern=CN=[^,]+,.+
> > > policyset.serverCertSet.1.default.class_id=subjectNameDefaultImpl
> > > policyset.serverCertSet.1.default.name
> > > <http://policyset.serverCertSet.1.default.name>=Subject Name
> >
> > Default
> > > policyset.serverCertSet.1.default.params.name
> > > <http://policyset.serverCertSet.1.default.params.name>=CN=$
> >
> > request.req_subject_name.cn
> > > <http://request.req_subject_name.cn>$, O=IPA.TEST
> > >
> > > which means that the subject name should match CN= followed by
> > > (anything except a comma) multiple times then a comma and any char
> > > multiple times.
> > >
> > > 2. If the profile wasn't changed, can you check in
> > > /var/log/pki/pki-tomcat/ca/debug.$DATE.log the received certificate
> > > request? Does its subject match the pattern? The error
> > > messagejava.lang.StringIndexOutOfBoundsException: String index out
> > > of range: -1 hints that an expected pattern was not found.
> > >
> > > flo
> > >
> > > On Thu, Sep 14, 2023 at 4:11 PM Evan G via FreeIPA-users
> > > <[e-mail]
> > > <mailto:[e-mail]>> wrote:
> > >
> > >     Hi Rob,
> > >
> > >     When we start tomcat with the date rolled back, we are not
> > >     seeing any errors at all. All of the ipa services start up
> > >     without issue. The problem is in actually renewing the certs,
> > >     when we do so we have seen many different errors as we've been
> > >     troubleshooting -- mostly this one: `ca-error: Server at
> > >     https://<HOSTNAME>/ipa/xml failed request, will retry: 4035
> >
> > (RPC
> > >     failed at server.  Request failed with status 500: Non-2xx
> > >     response from CA REST API: 500. String index out of range:
> > >     -1).[02/Aug/2023:00:00:31][ajp-bio-127.0.0.1-8009-exec-2]:
> > >     EnrollProfile: populate: begins`
> > >
> > >     When I restart certmonger after all services up, these are the
> > >     errors that I am seeing in the tomcat debug logs:
> > >     ```
> > >     [02/Aug/2023:00:00:31][ajp-bio-127.0.0.1-8009-exec-2]:
> > >     BasicProfile: populate: policy setid =serverCertSet
> > >     [02/Aug/2023:00:00:31][ajp-bio-127.0.0.1-8009-exec-2]:
> > >     EnrollDefault: populate: SubjectNameDefault: start
> > >     java.lang.StringIndexOutOfBoundsException: String index out of
> > >     range: -1
> > >             at java.lang.String.substring(String.java:1967)
> > >             at
> >
> > com.netscape.certsrv.pattern.Pattern.substitute2(Pattern.java:132)
> > >             at
> >
> > com.netscape.cms.profile.def.EnrollDefault.mapPattern(EnrollDefault.java:815)
> > >             at
> >
> > com.netscape.cms.profile.def.SubjectNameDefault.populate(SubjectNameDefault.java:160)
> > >             at
> >
> > com.netscape.cms.profile.def.EnrollDefault.populate(EnrollDefault.java:226)
> > >             at
> >
> > com.netscape.cms.profile.common.BasicProfile.populate(BasicProfile.java:1114)
> > >             at
> >
> > com.netscape.cms.profile.common.EnrollProfile.populate(EnrollProfile.java:2626)
> > >             at
> >
> > com.netscape.cms.servlet.cert.CertProcessor.populateRequests(CertProcessor.java:379)
> > >             at
> >
> > com.netscape.cms.servlet.cert.EnrollmentProcessor.processEnrollment(EnrollmentProcessor.java:188)
> > >             at
> >
> > com.netscape.cms.servlet.cert.EnrollmentProcessor.processEnrollment(EnrollmentProcessor.java:96)
> > >             at
> >
> > com.netscape.cms.servlet.cert.CertRequestDAO.submitRequest(CertRequestDAO.java:197)
> > >             at
> >
> > org.dogtagpki.server.ca.rest.CertRequestService.enrollCert(CertRequestService.java:155)
> > >             at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> > >     Method)
> > >             at
> >
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > >             at
> >
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > >             at java.lang.reflect.Method.invoke(Method.java:498)
> > >             at
> >
> > org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:137)
> > >             at
> >
> > org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:280)
> > >             at
> >
> > org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:234)
> > >             at
> >
> > org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:221)
> > >             at
> >
> > org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:356)
> > >             at
> >
> > org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:179)
> > >             at
> >
> > org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:220)
> > >             at
> >
> > org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
> > >             at
> >
> > org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
> > >             at
> > >     javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
> > >             at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> > >     Method)
> > >             at
> >
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > >             at
> >
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > >             at java.lang.reflect.Method.invoke(Method.java:498)
> > >             at
> >
> > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)
> > >             at
> >
> > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
> > >             at java.security.AccessController.doPrivileged(Native
> > >     Method)
> > >             at
> > >     javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
> > >             at
> >
> > org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)
> > >             at
> >
> > org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:175)
> > >             at
> >
> > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:297)
> > >             at
> >
> > org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:55)
> > >             at
> >
> > org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:191)
> > >             at
> >
> > org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:187)
> > >             at java.security.AccessController.doPrivileged(Native
> > >     Method)
> > >             at
> >
> > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:186)
> > >             at
> >
> > org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
> > >             at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> > >     Method)
> > >             at
> >
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > >             at
> >
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > >             at java.lang.reflect.Method.invoke(Method.java:498)
> > >             at
> >
> > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)
> > >             at
> >
> > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
> > >             at java.security.AccessController.doPrivileged(Native
> > >     Method)
> > >             at
> > >     javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
> > >             at
> >
> > org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)
> > >             at
> >
> > org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:260)
> > >             at
> >
> > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:237)
> > >             at
> >
> > org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:55)
> > >             at
> >
> > org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:191)
> > >             at
> >
> > org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:187)
> > >             at java.security.AccessController.doPrivileged(Native
> > >     Method)
> > >             at
> >
> > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:186)
> > >             at
> >
> > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:218)
> > >             at
> >
> > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
> > >             at
> >
> > org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:498)
> > >             at
> >
> > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
> > >             at
> >
> > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
> > >             at
> >
> > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962)
> > >             at
> >
> > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
> > >             at
> >
> > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)
> > >             at
> >
> > org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:190)
> > >             at
> >
> > org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
> > >             at org.apache.tomcat.util.net
> > >     <http://org.apache.tomcat.util.net
> > >
> > > .JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
> > >                 at
> >
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > >             at
> >
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > >             at
> >
> > org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> > >             at java.lang.Thread.run(Thread.java:750)
> > >     ```
> > >
> > >     This is what we see when we run `getcert list` and `ipa-getcert
> > >     list` respectively:
> > >
> > >     ```
> > >     Number of certificates and requests being tracked: 9.
> > >     Request ID '20190920201259':
> > >             status: CA_UNREACHABLE
> > >             ca-error: Server at https://<HOSTNAME>/ipa/xml failed
> > >     request, will retry: 4035 (RPC failed at server.  Request failed
> > >     with status 500: Non-2xx response from CA REST API: 500. String
> > >     index out of range: -1).
> > >             stuck: no
> > >             key pair storage:
> > >     type=FILE,location='/var/kerberos/krb5kdc/kdc.key'
> > >             certificate:
> > >     type=FILE,location='/var/kerberos/krb5kdc/kdc.crt'
> > >             CA: IPA
> > >             issuer: CN=Certificate Authority,O=<OU>
> > >             subject: CN=<HOSTNAME>,O=<OU>
> > >             expires: 2023-08-25 18:05:07 UTC
> > >             principal name: krbtgt/<OU>@<OU>
> > >             key usage:
> > >     digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> > >             eku: id-kp-serverAuth,id-pkinit-KPKdc
> > >             pre-save command:
> > >             post-save command:
> > >     /usr/libexec/ipa/certmonger/renew_kdc_cert
> > >             track: yes
> > >             auto-renew: yes
> > >     Request ID '20210908000050':
> > >             status: MONITORING
> > >             stuck: no
> > >             key pair storage:
> >
> > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
> > >     cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
> > >             certificate:
> >
> > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
> > >     cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
> > >             CA: dogtag-ipa-ca-renew-agent
> > >             issuer: CN=Certificate Authority,O=<OU>
> > >             subject: CN=CA Audit,O=<OU>
> > >             expires: 2025-07-21 02:36:57 UTC
> > >             key usage:
> >
> > digitalSignature,keyEncipherment,dataEncipherment
> > >             eku: id-kp-serverAuth,id-kp-clientAuth
> > >             pre-save command:
> >
> > /usr/libexec/ipa/certmonger/stop_pkicad
> > >             post-save command:
> > >     /usr/libexec/ipa/certmonger/renew_ca_cert "auditSigningCert
> > >     cert-pki-ca"
> > >             track: yes
> > >             auto-renew: yes
> > >     Request ID '20210908000051':
> > >             status: MONITORING
> > >             stuck: no
> > >             key pair storage:
> >
> > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
> > >     cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
> > >             certificate:
> >
> > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
> > >     cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
> > >             CA: dogtag-ipa-ca-renew-agent
> > >             issuer: CN=Certificate Authority,O=<OU>
> > >             subject: CN=OCSP Subsystem,O=<OU>
> > >             expires: 2025-07-21 02:36:17 UTC
> > >             key usage:
> >
> > digitalSignature,keyEncipherment,dataEncipherment
> > >             eku: id-kp-serverAuth,id-kp-clientAuth
> > >             pre-save command:
> >
> > /usr/libexec/ipa/certmonger/stop_pkicad
> > >             post-save command:
> > >     /usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert
> > >     cert-pki-ca"
> > >             track: yes
> > >             auto-renew: yes
> > >     Request ID '20210908000052':
> > >             status: MONITORING
> > >             stuck: no
> > >             key pair storage:
> >
> > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
> > >     cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
> > >             certificate:
> >
> > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
> > >     cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
> > >             CA: dogtag-ipa-ca-renew-agent
> > >             issuer: CN=Certificate Authority,O=<OU>
> > >             subject: CN=CA Subsystem,O=<OU>
> > >             expires: 2025-07-21 02:37:17 UTC
> > >             key usage:
> >
> > digitalSignature,keyEncipherment,dataEncipherment
> > >             eku: id-kp-serverAuth,id-kp-clientAuth
> > >             pre-save command:
> >
> > /usr/libexec/ipa/certmonger/stop_pkicad
> > >             post-save command:
> > >     /usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert
> > >     cert-pki-ca"
> > >             track: yes
> > >             auto-renew: yes
> > >     Request ID '20210908000053':
> > >             status: MONITORING
> > >             stuck: no
> > >             key pair storage:
> >
> > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
> > >     cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
> > >             certificate:
> >
> > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
> > >     cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
> > >             CA: dogtag-ipa-ca-renew-agent
> > >             issuer: CN=Certificate Authority,O=<OU>
> > >             subject: CN=Certificate Authority,O=<OU>
> > >             expires: 2039-09-20 20:11:25 UTC
> > >             key usage:
> > >     digitalSignature,nonRepudiation,keyCertSign,cRLSign
> > >             pre-save command:
> >
> > /usr/libexec/ipa/certmonger/stop_pkicad
> > >             post-save command:
> > >     /usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert
> > >     cert-pki-ca"
> > >             track: yes
> > >             auto-renew: yes
> > >     Request ID '20210908000054':
> > >             status: MONITORING
> > >             stuck: no
> > >             key pair storage:
> > >     type=FILE,location='/var/lib/ipa/ra-agent.key'
> > >             certificate:
> >
> > type=FILE,location='/var/lib/ipa/ra-agent.pem'
> > >             CA: dogtag-ipa-ca-renew-agent
> > >             issuer: CN=Certificate Authority,O=<OU>
> > >             subject: CN=IPA RA,O=<OU>
> > >             expires: 2025-06-26 02:36:15 UTC
> > >             key usage:
> >
> > digitalSignature,keyEncipherment,dataEncipherment
> > >             eku: id-kp-serverAuth,id-kp-clientAuth
> > >             pre-save command:
> > >     /usr/libexec/ipa/certmonger/renew_ra_cert_pre
> > >             post-save command:
> >
> > /usr/libexec/ipa/certmonger/renew_ra_cert
> > >             track: yes
> > >             auto-renew: yes
> > >     Request ID '20210908000055':
> > >             status: MONITORING
> > >             stuck: no
> > >             key pair storage:
> >
> > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
> > >     cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
> > >             certificate:
> >
> > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
> > >     cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
> > >             CA: dogtag-ipa-ca-renew-agent
> > >             issuer: CN=Certificate Authority,O=<OU>
> > >             subject: CN=<HOSTNAME>,O=<OU>
> > >             expires: 2025-07-21 02:36:37 UTC
> > >             dns: <HOSTNAME>
> > >             key usage:
> >
> > digitalSignature,keyEncipherment,dataEncipherment
> > >             eku: id-kp-serverAuth,id-kp-clientAuth
> > >             pre-save command:
> >
> > /usr/libexec/ipa/certmonger/stop_pkicad
> > >             post-save command:
> > >     /usr/libexec/ipa/certmonger/renew_ca_cert "Server-Cert
> >
> > cert-pki-ca"
> > >             track: yes
> > >             auto-renew: yes
> > >     Request ID '20210908000056':
> > >             status: CA_UNREACHABLE
> > >             ca-error: Server at https://<HOSTNAME>/ipa/xml failed
> > >     request, will retry: 4035 (RPC failed at server.  Request failed
> > >     with status 500: Non-2xx response from CA REST API: 500. String
> > >     index out of range: -1).
> > >             stuck: no
> > >             key pair storage:
> >
> > type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> > >     FIPS 140-2 Certificate
> > >     DB',pinfile='/etc/dirsrv/slapd-<OU>/pwdfile.txt'
> > >             certificate:
> >
> > type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> > >     FIPS 140-2 Certificate DB'
> > >             CA: IPA
> > >             issuer: CN=Certificate Authority,O=<OU>
> > >             subject: CN=<HOSTNAME>,O=<OU>
> > >             expires: 2023-09-03 18:30:48 UTC
> > >             dns: <HOSTNAME>
> > >             key usage:
> > >     digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> > >             eku: id-kp-serverAuth,id-kp-clientAuth
> > >             pre-save command:
> > >             post-save command:
> > >     /usr/libexec/ipa/certmonger/restart_dirsrv <OU>
> > >             track: yes
> > >             auto-renew: yes
> > >     Request ID '20210908000057':
> > >             status: CA_UNREACHABLE
> > >             ca-error: Server at https://<HOSTNAME>/ipa/xml failed
> > >     request, will retry: 4035 (RPC failed at server.  Request failed
> > >     with status 500: Non-2xx response from CA REST API: 500. String
> > >     index out of range: -1).
> > >             stuck: no
> > >             key pair storage:
> >
> > type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> > >     FIPS 140-2 Certificate
> >
> > DB',pinfile='/etc/httpd/alias/pwdfile.txt'
> > >             certificate:
> >
> > type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> > >     FIPS 140-2 Certificate DB'
> > >             CA: IPA
> > >             issuer: CN=Certificate Authority,O=<OU>
> > >             subject: CN=<HOSTNAME>,O=<OU>
> > >             expires: 2023-09-03 18:30:48 UTC
> > >             dns: <HOSTNAME>
> > >             key usage:
> > >     digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> > >             eku: id-kp-serverAuth,id-kp-clientAuth
> > >             pre-save command:
> > >             post-save command:
> >
> > /usr/libexec/ipa/certmonger/restart_httpd
> > >             track: yes
> > >             auto-renew: yes
> > >     ```
> > >
> > >     ```
> > >     Number of certificates and requests being tracked: 9.
> > >     Request ID '20190920201259':
> > >             status: CA_UNREACHABLE
> > >             ca-error: Server at https://<HOSTNAME>/ipa/xml failed
> > >     request, will retry: 4035 (RPC failed at server.  Request failed
> > >     with status 500: Non-2xx response from CA REST API: 500. String
> > >     index out of range: -1).
> > >             stuck: no
> > >             key pair storage:
> > >     type=FILE,location='/var/kerberos/krb5kdc/kdc.key'
> > >             certificate:
> > >     type=FILE,location='/var/kerberos/krb5kdc/kdc.crt'
> > >             CA: IPA
> > >             issuer: CN=Certificate Authority,O=<OU>
> > >             subject: CN=<HOSTNAME>,O=<OU>
> > >             expires: 2023-08-25 18:05:07 UTC
> > >             principal name: krbtgt/<OU>@<OU>
> > >             key usage:
> > >     digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> > >             eku: id-kp-serverAuth,id-pkinit-KPKdc
> > >             pre-save command:
> > >             post-save command:
> > >     /usr/libexec/ipa/certmonger/renew_kdc_cert
> > >             track: yes
> > >             auto-renew: yes
> > >     Request ID '20210908000056':
> > >             status: CA_UNREACHABLE
> > >             ca-error: Server at https://<HOSTNAME>/ipa/xml failed
> > >     request, will retry: 4035 (RPC failed at server.  Request failed
> > >     with status 500: Non-2xx response from CA REST API: 500. String
> > >     index out of range: -1).
> > >             stuck: no
> > >             key pair storage:
> >
> > type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> > >     FIPS 140-2 Certificate
> > >     DB',pinfile='/etc/dirsrv/slapd-<OU>/pwdfile.txt'
> > >             certificate:
> >
> > type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> > >     FIPS 140-2 Certificate DB'
> > >             CA: IPA
> > >             issuer: CN=Certificate Authority,O=<OU>
> > >             subject: CN=<HOSTNAME>,O=<OU>
> > >             expires: 2023-09-03 18:30:48 UTC
> > >             dns: <HOSTNAME>
> > >             key usage:
> > >     digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> > >             eku: id-kp-serverAuth,id-kp-clientAuth
> > >             pre-save command:
> > >             post-save command:
> > >     /usr/libexec/ipa/certmonger/restart_dirsrv <OU>
> > >             track: yes
> > >             auto-renew: yes
> > >     Request ID '20210908000057':
> > >             status: CA_UNREACHABLE
> > >             ca-error: Server at https://<HOSTNAME>/ipa/xml failed
> > >     request, will retry: 4035 (RPC failed at server.  Request failed
> > >     with status 500: Non-2xx response from CA REST API: 500. String
> > >     index out of range: -1).
> > >             stuck: no
> > >             key pair storage:
> >
> > type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> > >     FIPS 140-2 Certificate
> >
> > DB',pinfile='/etc/httpd/alias/pwdfile.txt'
> > >             certificate:
> >
> > type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> > >     FIPS 140-2 Certificate DB'
> > >             CA: IPA
> > >             issuer: CN=Certificate Authority,O=<OU>
> > >             subject: CN=<HOSTNAME>,O=<OU>
> > >             expires: 2023-09-03 18:30:48 UTC
> > >             dns: <HOSTNAME>
> > >             key usage:
> > >     digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> > >             eku: id-kp-serverAuth,id-kp-clientAuth
> > >             pre-save command:
> > >             post-save command:
> >
> > /usr/libexec/ipa/certmonger/restart_httpd
> > >             track: yes
> > >             auto-renew: yes
> > >     ```
> > >     _______________________________________________
> > >     FreeIPA-users mailing list --
> > >     [e-mail]
> > >     <mailto:[e-mail]>
> > >     To unsubscribe send an email to
> > >     [e-mail]
> > >     <mailto:[e-mail]>
> > >     Fedora Code of Conduct:
> > >     https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > >     List Guidelines:
> > >     https://fedoraproject.org/wiki/Mailing_list_guidelines
> > >     List Archives:
> >
> > https://lists.fedorahosted.org/archives/list/[e-mail]...
> > >     Do not reply to spam, report it:
> > >     https://pagure.io/fedora-infrastructure/new_issue
> > >
> > >
> > > FreeIPA-users mailing list -- [e-mail]
> > > To unsubscribe send an email to
> >
> > [e-mail]
> > > Fedora Code of Conduct:
> >
> > https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > > List Archives:
> >
> > https://lists.fedorahosted.org/archives/list/[e-mail]...
> > > Do not reply to spam, report it:
> >
> > https://pagure.io/fedora-infrastructure/new_issue
> > >

[Rob Crittenden]

IT Guy wrote:
> OK just one more thing to add, I had run across this link during
> troubleshooting and it seems that my co-worker had updated some of the
> lines in this configuration according to the steps outlined in this
> forum post: https://pagure.io/freeipa/issue/7267
>
> However I can say that this was a last ditch effort to try and get the
> renewals working, we had already been troubleshooting for 3+ days at the
> point that this was changed.

Looks like this was not correctly applied: "Especially note the
replacement of occurrences of $$ with $."

Your profile has $$ and it should be $, according to Fraser.

rob
> On Fri, Sep 15, 2023 at 9:58 AM IT Guy <[e-mail]
> > wrote:
> Wow that worked Rob, thank you! If I compare the values that
> Florence sent to what I have in this file, the only difference is
> this line: 
>
> policyset.serverCertSet.1.default.params.name
> <http://policyset.serverCertSet.1.default.params.name>=CN=$$request.req_subject_name.cn
> <http://request.req_subject_name.cn>$$, $SUBJECT_DN_O
>
> Here's the full snippet for reference:
>
> policyset.serverCertSet.1.constraint.class_id=subjectNameConstraintImpl
> policyset.serverCertSet.1.constraint.name
> <http://policyset.serverCertSet.1.constraint.name>=Subject Name
> Constraint
> policyset.serverCertSet.1.constraint.params.accept=true
> policyset.serverCertSet.1.constraint.params.pattern=CN=[^,]+,.+
> policyset.serverCertSet.1.default.class_id=subjectNameDefaultImpl
> policyset.serverCertSet.1.default.name
> <http://policyset.serverCertSet.1.default.name>=Subject Name Default
> policyset.serverCertSet.1.default.params.name
> <http://policyset.serverCertSet.1.default.params.name>=CN=$$request.req_subject_name.cn
> <http://request.req_subject_name.cn>$$, $SUBJECT_DN_O
>
>
> One other thing I wanted to call out is that I have a good snapshot
> of this server that I have restored a couple of times to try
> different things and the one that got me the farthest was when I
> changed the name of the cert from our custom name back to
> Server-Cert. Even when I had the config this way I still could not
> renew but maybe modifying something in the above config plus
> changing back to Server-Cert could alleviate the issue?
>
> Many thanks,
>
> Evan
>
> On Fri, Sep 15, 2023 at 9:47 AM Rob Crittenden <[e-mail]
> <mailto:[e-mail]>> wrote:
>
>     IT Guy via FreeIPA-users wrote:
>     > Hi Florence,
>     >
>     > Thank you for your response. What does it mean if I run the ipa
>     > certprofile-show command as outlined above and it just hangs?
>     I don't
>     > think there is any other way to see the settings you mentioned
>     unless
>     > this command is able to run right?
>
>     I can't explain why it would hang but you can get the profile
>     directly
>     from LDAP:
>
>     $ ldapsearch -LLL -o ldif-wrap=no -x -D 'cn=directory manager' -W -b
>     cn=caIPAserviceCert,ou=certificateProfiles,ou=ca,o=ipaca
>     certProfileConfig > /tmp/profile
>
>     Edit this file and remove the dn value and 'certProfileConfig::
>     ' then
>     base64-decode the result.
>
>     The final really huge string should look something like:
>
>     YXV0aC5pbnN0YW5jZV9pZ...=
>
>     I used the coreutils base64 program to decode it:
>
>     $ base64 -d /tmp/profile
>
>     rob
>     >
>     > Many thanks,
>     >
>     > Evan
>     >
>     > On Fri, Sep 15, 2023 at 3:19 AM Florence Blanc-Renaud
>     <[e-mail] <mailto:[e-mail]>
>     > <mailto:[e-mail] <mailto:[e-mail]>>> wrote:
>     >
>     >     Hi,
>     >     it seems that PKI is not happy with the subject name of the
>     >     certificates.
>     >     The failing certs are for KDC, dirsrv and httpd and they
>     all use the
>     >     same subject name constraint in their profile.
>     >
>     >     1. Was any certificate profile modified (caIPAserviceCert or
>     >     KDCs_PKINIT_Certs)? You can use
>     >     ipa certprofile-show <name> --out /dev/stdout
>     >     And then check the part related to Subject Name
>     Constraint. In my
>     >     default installation, I have
>     >   
>      policyset.serverCertSet.1.constraint.class_id=subjectNameConstraintImpl
>     >     policyset.serverCertSet.1.constraint.name
>     <http://policyset.serverCertSet.1.constraint.name>
>     >     <http://policyset.serverCertSet.1.constraint.name>=Subject
>     Name
>     >     Constraint
>     >     policyset.serverCertSet.1.constraint.params.accept=true
>     >   
>      policyset.serverCertSet.1.constraint.params.pattern=CN=[^,]+,.+
>     >   
>      policyset.serverCertSet.1.default.class_id=subjectNameDefaultImpl
>     >     policyset.serverCertSet.1.default.name
>     <http://policyset.serverCertSet.1.default.name>
>     >     <http://policyset.serverCertSet.1.default.name>=Subject
>     Name Default
>     >     policyset.serverCertSet.1.default.params.name
>     <http://policyset.serverCertSet.1.default.params.name>
>     >   
>      <http://policyset.serverCertSet.1.default.params.name>=CN=$request.req_subject_name.cn
>     <http://request.req_subject_name.cn>
>     >     <http://request.req_subject_name.cn>$, O=IPA.TEST
>     >
>     >     which means that the subject name should match CN= followed by
>     >     (anything except a comma) multiple times then a comma and
>     any char
>     >     multiple times.
>     >
>     >     2. If the profile wasn't changed, can you check in
>     >     /var/log/pki/pki-tomcat/ca/debug.$DATE.log the received
>     certificate
>     >     request? Does its subject match the pattern? The error
>     >     messagejava.lang.StringIndexOutOfBoundsException: String
>     index out
>     >     of range: -1 hints that an expected pattern was not found.
>     >
>     >     flo
>     >
>     >     On Thu, Sep 14, 2023 at 4:11 PM Evan G via FreeIPA-users
>     >     <[e-mail]
>     <mailto:[e-mail]>
>     >     <mailto:[e-mail]
>     <mailto:[e-mail]>>> wrote:
>     >
>     >         Hi Rob,
>     >
>     >         When we start tomcat with the date rolled back, we are not
>     >         seeing any errors at all. All of the ipa services start up
>     >         without issue. The problem is in actually renewing the
>     certs,
>     >         when we do so we have seen many different errors as
>     we've been
>     >         troubleshooting -- mostly this one: `ca-error: Server at
>     >         https://<HOSTNAME>/ipa/xml failed request, will retry:
>     4035 (RPC
>     >         failed at server.  Request failed with status 500: Non-2xx
>     >         response from CA REST API: 500. String index out of range:
>     >         -1).[02/Aug/2023:00:00:31][ajp-bio-127.0.0.1-8009-exec-2]:
>     >         EnrollProfile: populate: begins`
>     >
>     >         When I restart certmonger after all services up, these
>     are the
>     >         errors that I am seeing in the tomcat debug logs:
>     >         ```
>     >         [02/Aug/2023:00:00:31][ajp-bio-127.0.0.1-8009-exec-2]:
>     >         BasicProfile: populate: policy setid =serverCertSet
>     >         [02/Aug/2023:00:00:31][ajp-bio-127.0.0.1-8009-exec-2]:
>     >         EnrollDefault: populate: SubjectNameDefault: start
>     >         java.lang.StringIndexOutOfBoundsException: String
>     index out of
>     >         range: -1
>     >                 at java.lang.String.substring(String.java:1967)
>     >                 at
>     >       
>      com.netscape.certsrv.pattern.Pattern.substitute2(Pattern.java:132)
>     >                 at
>     >       
>      com.netscape.cms.profile.def.EnrollDefault.mapPattern(EnrollDefault.java:815)
>     >                 at
>     >       
>      com.netscape.cms.profile.def.SubjectNameDefault.populate(SubjectNameDefault.java:160)
>     >                 at
>     >       
>      com.netscape.cms.profile.def.EnrollDefault.populate(EnrollDefault.java:226)
>     >                 at
>     >       
>      com.netscape.cms.profile.common.BasicProfile.populate(BasicProfile.java:1114)
>     >                 at
>     >       
>      com.netscape.cms.profile.common.EnrollProfile.populate(EnrollProfile.java:2626)
>     >                 at
>     >       
>      com.netscape.cms.servlet.cert.CertProcessor.populateRequests(CertProcessor.java:379)
>     >                 at
>     >       
>      com.netscape.cms.servlet.cert.EnrollmentProcessor.processEnrollment(EnrollmentProcessor.java:188)
>     >                 at
>     >       
>      com.netscape.cms.servlet.cert.EnrollmentProcessor.processEnrollment(EnrollmentProcessor.java:96)
>     >                 at
>     >       
>      com.netscape.cms.servlet.cert.CertRequestDAO.submitRequest(CertRequestDAO.java:197)
>     >                 at
>     >       
>      org.dogtagpki.server.ca.rest.CertRequestService.enrollCert(CertRequestService.java:155)
>     >                 at
>     sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>     >         Method)
>     >                 at
>     >       
>      sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>     >                 at
>     >       
>      sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     >                 at
>     java.lang.reflect.Method.invoke(Method.java:498)
>     >                 at
>     >       
>      org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:137)
>     >                 at
>     >       
>      org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:280)
>     >                 at
>     >       
>      org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:234)
>     >                 at
>     >       
>      org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:221)
>     >                 at
>     >       
>      org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:356)
>     >                 at
>     >       
>      org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:179)
>     >                 at
>     >       
>      org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:220)
>     >                 at
>     >       
>      org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
>     >                 at
>     >       
>      org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
>     >                 at
>     >       
>      javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
>     >                 at
>     sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>     >         Method)
>     >                 at
>     >       
>      sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>     >                 at
>     >       
>      sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     >                 at
>     java.lang.reflect.Method.invoke(Method.java:498)
>     >                 at
>     >       
>      org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)
>     >                 at
>     >       
>      org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
>     >                 at
>     java.security.AccessController.doPrivileged(Native
>     >         Method)
>     >                 at
>     >       
>      javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
>     >                 at
>     >       
>      org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)
>     >                 at
>     >       
>      org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:175)
>     >                 at
>     >       
>      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:297)
>     >                 at
>     >       
>      org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:55)
>     >                 at
>     >       
>      org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:191)
>     >                 at
>     >       
>      org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:187)
>     >                 at
>     java.security.AccessController.doPrivileged(Native
>     >         Method)
>     >                 at
>     >       
>      org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:186)
>     >                 at
>     >       
>      org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
>     >                 at
>     sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>     >         Method)
>     >                 at
>     >       
>      sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>     >                 at
>     >       
>      sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     >                 at
>     java.lang.reflect.Method.invoke(Method.java:498)
>     >                 at
>     >       
>      org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)
>     >                 at
>     >       
>      org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
>     >                 at
>     java.security.AccessController.doPrivileged(Native
>     >         Method)
>     >                 at
>     >       
>      javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
>     >                 at
>     >       
>      org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)
>     >                 at
>     >       
>      org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:260)
>     >                 at
>     >       
>      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:237)
>     >                 at
>     >       
>      org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:55)
>     >                 at
>     >       
>      org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:191)
>     >                 at
>     >       
>      org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:187)
>     >                 at
>     java.security.AccessController.doPrivileged(Native
>     >         Method)
>     >                 at
>     >       
>      org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:186)
>     >                 at
>     >       
>      org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:218)
>     >                 at
>     >       
>      org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
>     >                 at
>     >       
>      org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:498)
>     >                 at
>     >       
>      org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
>     >                 at
>     >       
>      org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
>     >                 at
>     >       
>      org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962)
>     >                 at
>     >       
>      org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
>     >                 at
>     >       
>      org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)
>     >                 at
>     >       
>      org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:190)
>     >                 at
>     >       
>      org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
>     >                 at org.apache.tomcat.util.net
>     <http://org.apache.tomcat.util.net>
>     >       
>      <http://org.apache.tomcat.util.net>.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
>     >                 at
>     >       
>      java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>     >                 at
>     >       
>      java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>     >                 at
>     >       
>      org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
>     >                 at java.lang.Thread.run(Thread.java:750)
>     >         ```
>     >
>     >         This is what we see when we run `getcert list` and
>     `ipa-getcert
>     >         list` respectively:
>     >
>     >         ```
>     >         Number of certificates and requests being tracked: 9.
>     >         Request ID '20190920201259':
>     >                 status: CA_UNREACHABLE
>     >                 ca-error: Server at https://<HOSTNAME>/ipa/xml
>     failed
>     >         request, will retry: 4035 (RPC failed at server. 
>     Request failed
>     >         with status 500: Non-2xx response from CA REST API:
>     500. String
>     >         index out of range: -1).
>     >                 stuck: no
>     >                 key pair storage:
>     >         type=FILE,location='/var/kerberos/krb5kdc/kdc.key'
>     >                 certificate:
>     >         type=FILE,location='/var/kerberos/krb5kdc/kdc.crt'
>     >                 CA: IPA
>     >                 issuer: CN=Certificate Authority,O=<OU>
>     >                 subject: CN=<HOSTNAME>,O=<OU>
>     >                 expires: 2023-08-25 18:05:07 UTC
>     >                 principal name: krbtgt/<OU>@<OU>
>     >                 key usage:
>     >       
>      digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>     >                 eku: id-kp-serverAuth,id-pkinit-KPKdc
>     >                 pre-save command:
>     >                 post-save command:
>     >         /usr/libexec/ipa/certmonger/renew_kdc_cert
>     >                 track: yes
>     >                 auto-renew: yes
>     >         Request ID '20210908000050':
>     >                 status: MONITORING
>     >                 stuck: no
>     >                 key pair storage:
>     >       
>      type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
>     >         cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
>     >                 certificate:
>     >       
>      type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
>     >         cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
>     >                 CA: dogtag-ipa-ca-renew-agent
>     >                 issuer: CN=Certificate Authority,O=<OU>
>     >                 subject: CN=CA Audit,O=<OU>
>     >                 expires: 2025-07-21 02:36:57 UTC
>     >                 key usage:
>     digitalSignature,keyEncipherment,dataEncipherment
>     >                 eku: id-kp-serverAuth,id-kp-clientAuth
>     >                 pre-save command:
>     /usr/libexec/ipa/certmonger/stop_pkicad
>     >                 post-save command:
>     >         /usr/libexec/ipa/certmonger/renew_ca_cert
>     "auditSigningCert
>     >         cert-pki-ca"
>     >                 track: yes
>     >                 auto-renew: yes
>     >         Request ID '20210908000051':
>     >                 status: MONITORING
>     >                 stuck: no
>     >                 key pair storage:
>     >       
>      type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
>     >         cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
>     >                 certificate:
>     >       
>      type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
>     >         cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
>     >                 CA: dogtag-ipa-ca-renew-agent
>     >                 issuer: CN=Certificate Authority,O=<OU>
>     >                 subject: CN=OCSP Subsystem,O=<OU>
>     >                 expires: 2025-07-21 02:36:17 UTC
>     >                 key usage:
>     digitalSignature,keyEncipherment,dataEncipherment
>     >                 eku: id-kp-serverAuth,id-kp-clientAuth
>     >                 pre-save command:
>     /usr/libexec/ipa/certmonger/stop_pkicad
>     >                 post-save command:
>     >         /usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert
>     >         cert-pki-ca"
>     >                 track: yes
>     >                 auto-renew: yes
>     >         Request ID '20210908000052':
>     >                 status: MONITORING
>     >                 stuck: no
>     >                 key pair storage:
>     >       
>      type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
>     >         cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
>     >                 certificate:
>     >       
>      type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
>     >         cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
>     >                 CA: dogtag-ipa-ca-renew-agent
>     >                 issuer: CN=Certificate Authority,O=<OU>
>     >                 subject: CN=CA Subsystem,O=<OU>
>     >                 expires: 2025-07-21 02:37:17 UTC
>     >                 key usage:
>     digitalSignature,keyEncipherment,dataEncipherment
>     >                 eku: id-kp-serverAuth,id-kp-clientAuth
>     >                 pre-save command:
>     /usr/libexec/ipa/certmonger/stop_pkicad
>     >                 post-save command:
>     >         /usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert
>     >         cert-pki-ca"
>     >                 track: yes
>     >                 auto-renew: yes
>     >         Request ID '20210908000053':
>     >                 status: MONITORING
>     >                 stuck: no
>     >                 key pair storage:
>     >       
>      type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
>     >         cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
>     >                 certificate:
>     >       
>      type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
>     >         cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
>     >                 CA: dogtag-ipa-ca-renew-agent
>     >                 issuer: CN=Certificate Authority,O=<OU>
>     >                 subject: CN=Certificate Authority,O=<OU>
>     >                 expires: 2039-09-20 20:11:25 UTC
>     >                 key usage:
>     >         digitalSignature,nonRepudiation,keyCertSign,cRLSign
>     >                 pre-save command:
>     /usr/libexec/ipa/certmonger/stop_pkicad
>     >                 post-save command:
>     >         /usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCert
>     >         cert-pki-ca"
>     >                 track: yes
>     >                 auto-renew: yes
>     >         Request ID '20210908000054':
>     >                 status: MONITORING
>     >                 stuck: no
>     >                 key pair storage:
>     >         type=FILE,location='/var/lib/ipa/ra-agent.key'
>     >                 certificate:
>     type=FILE,location='/var/lib/ipa/ra-agent.pem'
>     >                 CA: dogtag-ipa-ca-renew-agent
>     >                 issuer: CN=Certificate Authority,O=<OU>
>     >                 subject: CN=IPA RA,O=<OU>
>     >                 expires: 2025-06-26 02:36:15 UTC
>     >                 key usage:
>     digitalSignature,keyEncipherment,dataEncipherment
>     >                 eku: id-kp-serverAuth,id-kp-clientAuth
>     >                 pre-save command:
>     >         /usr/libexec/ipa/certmonger/renew_ra_cert_pre
>     >                 post-save command:
>     /usr/libexec/ipa/certmonger/renew_ra_cert
>     >                 track: yes
>     >                 auto-renew: yes
>     >         Request ID '20210908000055':
>     >                 status: MONITORING
>     >                 stuck: no
>     >                 key pair storage:
>     >       
>      type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
>     >         cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin set
>     >                 certificate:
>     >       
>      type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
>     >         cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
>     >                 CA: dogtag-ipa-ca-renew-agent
>     >                 issuer: CN=Certificate Authority,O=<OU>
>     >                 subject: CN=<HOSTNAME>,O=<OU>
>     >                 expires: 2025-07-21 02:36:37 UTC
>     >                 dns: <HOSTNAME>
>     >                 key usage:
>     digitalSignature,keyEncipherment,dataEncipherment
>     >                 eku: id-kp-serverAuth,id-kp-clientAuth
>     >                 pre-save command:
>     /usr/libexec/ipa/certmonger/stop_pkicad
>     >                 post-save command:
>     >         /usr/libexec/ipa/certmonger/renew_ca_cert "Server-Cert
>     cert-pki-ca"
>     >                 track: yes
>     >                 auto-renew: yes
>     >         Request ID '20210908000056':
>     >                 status: CA_UNREACHABLE
>     >                 ca-error: Server at https://<HOSTNAME>/ipa/xml
>     failed
>     >         request, will retry: 4035 (RPC failed at server. 
>     Request failed
>     >         with status 500: Non-2xx response from CA REST API:
>     500. String
>     >         index out of range: -1).
>     >                 stuck: no
>     >                 key pair storage:
>     >       
>      type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
>     >         FIPS 140-2 Certificate
>     >         DB',pinfile='/etc/dirsrv/slapd-<OU>/pwdfile.txt'
>     >                 certificate:
>     >       
>      type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
>     >         FIPS 140-2 Certificate DB'
>     >                 CA: IPA
>     >                 issuer: CN=Certificate Authority,O=<OU>
>     >                 subject: CN=<HOSTNAME>,O=<OU>
>     >                 expires: 2023-09-03 18:30:48 UTC
>     >                 dns: <HOSTNAME>
>     >                 key usage:
>     >       
>      digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>     >                 eku: id-kp-serverAuth,id-kp-clientAuth
>     >                 pre-save command:
>     >                 post-save command:
>     >         /usr/libexec/ipa/certmonger/restart_dirsrv <OU>
>     >                 track: yes
>     >                 auto-renew: yes
>     >         Request ID '20210908000057':
>     >                 status: CA_UNREACHABLE
>     >                 ca-error: Server at https://<HOSTNAME>/ipa/xml
>     failed
>     >         request, will retry: 4035 (RPC failed at server. 
>     Request failed
>     >         with status 500: Non-2xx response from CA REST API:
>     500. String
>     >         index out of range: -1).
>     >                 stuck: no
>     >                 key pair storage:
>     >       
>      type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
>     >         FIPS 140-2 Certificate
>     DB',pinfile='/etc/httpd/alias/pwdfile.txt'
>     >                 certificate:
>     >       
>      type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
>     >         FIPS 140-2 Certificate DB'
>     >                 CA: IPA
>     >                 issuer: CN=Certificate Authority,O=<OU>
>     >                 subject: CN=<HOSTNAME>,O=<OU>
>     >                 expires: 2023-09-03 18:30:48 UTC
>     >                 dns: <HOSTNAME>
>     >                 key usage:
>     >       
>      digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>     >                 eku: id-kp-serverAuth,id-kp-clientAuth
>     >                 pre-save command:
>     >                 post-save command:
>     /usr/libexec/ipa/certmonger/restart_httpd
>     >                 track: yes
>     >                 auto-renew: yes
>     >         ```
>     >
>     >        ��```
>     >         Number of certificates and requests being tracked: 9.
>     >         Request ID '20190920201259':
>     >                 status: CA_UNREACHABLE
>     >                 ca-error: Server at https://<HOSTNAME>/ipa/xml
>     failed
>     >         request, will retry: 4035 (RPC failed at server. 
>     Request failed
>     >         with status 500: Non-2xx response from CA REST API:
>     500. String
>     >         index out of range: -1).
>     >                 stuck: no
>     >                 key pair storage:
>     >         type=FILE,location='/var/kerberos/krb5kdc/kdc.key'
>     >                 certificate:
>     >         type=FILE,location='/var/kerberos/krb5kdc/kdc.crt'
>     >                 CA: IPA
>     >                 issuer: CN=Certificate Authority,O=<OU>
>     >                 subject: CN=<HOSTNAME>,O=<OU>
>     >                 expires: 2023-08-25 18:05:07 UTC
>     >                 principal name: krbtgt/<OU>@<OU>
>     >                 key usage:
>     >       
>      digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>     >                 eku: id-kp-serverAuth,id-pkinit-KPKdc
>     >                 pre-save command:
>     >                 post-save command:
>     >         /usr/libexec/ipa/certmonger/renew_kdc_cert
>     >                 track: yes
>     >                 auto-renew: yes
>     >         Request ID '20210908000056':
>     >                 status: CA_UNREACHABLE
>     >                 ca-error: Server at https://<HOSTNAME>/ipa/xml
>     failed
>     >         request, will retry: 4035 (RPC failed at server. 
>     Request failed
>     >         with status 500: Non-2xx response from CA REST API:
>     500. String
>     >         index out of range: -1).
>     >                 stuck: no
>     >                 key pair storage:
>     >       
>      type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
>     >         FIPS 140-2 Certificate
>     >         DB',pinfile='/etc/dirsrv/slapd-<OU>/pwdfile.txt'
>     >                 certificate:
>     >       
>      type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
>     >         FIPS 140-2 Certificate DB'
>     >                 CA: IPA
>     >                 issuer: CN=Certificate Authority,O=<OU>
>     >                 subject: CN=<HOSTNAME>,O=<OU>
>     >                 expires: 2023-09-03 18:30:48 UTC
>     >                 dns: <HOSTNAME>
>     >                 key usage:
>     >       
>      digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>     >                 eku: id-kp-serverAuth,id-kp-clientAuth
>     >                 pre-save command:
>     >                 post-save command:
>     >         /usr/libexec/ipa/certmonger/restart_dirsrv <OU>
>     >                 track: yes
>     >                 auto-renew: yes
>     >         Request ID '20210908000057':
>     >                 status: CA_UNREACHABLE
>     >                 ca-error: Server at https://<HOSTNAME>/ipa/xml
>     failed
>     >         request, will retry: 4035 (RPC failed at server. 
>     Request failed
>     >         with status 500: Non-2xx response from CA REST API:
>     500. String
>     >         index out of range: -1).
>     >                 stuck: no
>     >                 key pair storage:
>     >       
>      type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
>     >         FIPS 140-2 Certificate
>     DB',pinfile='/etc/httpd/alias/pwdfile.txt'
>     >                 certificate:
>     >       
>      type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
>     >         FIPS 140-2 Certificate DB'
>     >                 CA: IPA
>     >                 issuer: CN=Certificate Authority,O=<OU>
>     >                 subject: CN=<HOSTNAME>,O=<OU>
>     >                 expires: 2023-09-03 18:30:48 UTC
>     >                 dns: <HOSTNAME>
>     >                 key usage:
>     >       
>      digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>     >                 eku: id-kp-serverAuth,id-kp-clientAuth
>     >                 pre-save command:
>     >                 post-save command:
>     /usr/libexec/ipa/certmonger/restart_httpd
>     >                 track: yes
>     >                 auto-renew: yes
>     >         ```
>     >         _______________________________________________
>     >         FreeIPA-users mailing list --
>     >         [e-mail]
>     <mailto:[e-mail]>
>     >         <mailto:[e-mail]
>     <mailto:[e-mail]>>
>     >         To unsubscribe send an email to
>     >         [e-mail]
>     <mailto:[e-mail]>
>     >         <mailto:[e-mail]
>     <mailto:[e-mail]>>
>     >         Fedora Code of Conduct:
>     >       
>      https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>     >         List Guidelines:
>     >         https://fedoraproject.org/wiki/Mailing_list_guidelines
>     >         List Archives:
>     >       
>      https://lists.fedorahosted.org/archives/list/[e-mail]
>     >         Do not reply to spam, report it:
>     >         https://pagure.io/fedora-infrastructure/new_issue
>     >
>     >
>     > _______________________________________________
>     > FreeIPA-users mailing list --
>     [e-mail]
>     <mailto:[e-mail]>
>     > To unsubscribe send an email to
>     [e-mail]
>     <mailto:[e-mail]>
>     > Fedora Code of Conduct:
>     https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>     > List Guidelines:
>     https://fedoraproject.org/wiki/Mailing_list_guidelines
>     > List Archives:
>     https://lists.fedorahosted.org/archives/list/[e-mail]
>     > Do not reply to spam, report it:
>     https://pagure.io/fedora-infrastructure/new_issue
>     >

[IT Guy]

Rob,

Thank you. So it looks like what I shared as the current config is actually
what was there when the snapshot was taken. The changes outlined in that
post were made on a machine which has since been deleted. So what I am
saying is that the config I shared does not include any of the changes my
co-worker had made. When I make the changes to match what Florence shared
as a default config and attempt to renew the certs, I am now getting the
following error:

ca-error: Server at https://<HOSTNAME>/ipa/xml failed request, will retry:
4016 (RPC failed at server.  Failed to authenticate to CA REST API).

How can I make sure that the credentials that are attempting to be used are
valid for this operation?

Many thanks,

Evan

On Fri, Sep 15, 2023 at 10:25 AM Rob Crittenden  wrote:
> IT Guy wrote:
> > OK just one more thing to add, I had run across this link during
> > troubleshooting and it seems that my co-worker had updated some of the
> > lines in this configuration according to the steps outlined in this
> > forum post: https://pagure.io/freeipa/issue/7267
> >
> > However I can say that this was a last ditch effort to try and get the
> > renewals working, we had already been troubleshooting for 3+ days at the
> > point that this was changed.
>
> Looks like this was not correctly applied: "Especially note the
> replacement of occurrences of $$ with $."
>
> Your profile has $$ and it should be $, according to Fraser.
>
> rob
> > On Fri, Sep 15, 2023 at 9:58 AM IT Guy <[e-mail]
> > > wrote:
> > Wow that worked Rob, thank you! If I compare the values that
> > Florence sent to what I have in this file, the only difference is
> > this line:
> >
> > policyset.serverCertSet.1.default.params.name
> > <http://policyset.serverCertSet.1.default.params.name>=CN=$$
>
> request.req_subject_name.cn
> > <http://request.req_subject_name.cn>$$, $SUBJECT_DN_O
> >
> > Here's the full snippet for reference:
>
> policyset.serverCertSet.1.constraint.class_id=subjectNameConstraintImpl
> > policyset.serverCertSet.1.constraint.name
> > <http://policyset.serverCertSet.1.constraint.name>=Subject Name
> > Constraint
> > policyset.serverCertSet.1.constraint.params.accept=true
> > policyset.serverCertSet.1.constraint.params.pattern=CN=[^,]+,.+
> > policyset.serverCertSet.1.default.class_id=subjectNameDefaultImpl
> > policyset.serverCertSet.1.default.name
> > <http://policyset.serverCertSet.1.default.name>=Subject Name Default
> > policyset.serverCertSet.1.default.params.name
> > <http://policyset.serverCertSet.1.default.params.name>=CN=$$
>
> request.req_subject_name.cn
> > <http://request.req_subject_name.cn>$$, $SUBJECT_DN_O
> >
> >
> > One other thing I wanted to call out is that I have a good snapshot
> > of this server that I have restored a couple of times to try
> > different things and the one that got me the farthest was when I
> > changed the name of the cert from our custom name back to
> > Server-Cert. Even when I had the config this way I still could not
> > renew but maybe modifying something in the above config plus
> > changing back to Server-Cert could alleviate the issue?
> >
> > Many thanks,
> >
> > Evan
> >
> > On Fri, Sep 15, 2023 at 9:47 AM Rob Crittenden <[e-mail]
> > <mailto:[e-mail]>> wrote:
> >
> >     IT Guy via FreeIPA-users wrote:
> >     > Hi Florence,
> >     >
> >     > Thank you for your response. What does it mean if I run the ipa
> >     > certprofile-show command as outlined above and it just hangs?
> >     I don't
> >     > think there is any other way to see the settings you mentioned
> >     unless
> >     > this command is able to run right?
> >
> >     I can't explain why it would hang but you can get the profile
> >     directly
> >     from LDAP:
> >
> >     $ ldapsearch -LLL -o ldif-wrap=no -x -D 'cn=directory manager'
>
> -W -b
> >     cn=caIPAserviceCert,ou=certificateProfiles,ou=ca,o=ipaca
> >     certProfileConfig > /tmp/profile
> >
> >     Edit this file and remove the dn value and 'certProfileConfig::
> >     ' then
> >     base64-decode the result.
> >
> >     The final really huge string should look something like:
> >
> >     YXV0aC5pbnN0YW5jZV9pZ...=
> >
> >     I used the coreutils base64 program to decode it:
> >
> >     $ base64 -d /tmp/profile
> >
> >     rob
> >     >
> >     > Many thanks,
> >     >
> >     > Evan
> >     >
> >     > On Fri, Sep 15, 2023 at 3:19 AM Florence Blanc-Renaud
> >     <[e-mail] <mailto:[e-mail]>
> >     > <mailto:[e-mail] <mailto:[e-mail]>>> wrote:
> >     >
> >     >     Hi,
> >     >     it seems that PKI is not happy with the subject name of the
> >     >     certificates.
> >     >     The failing certs are for KDC, dirsrv and httpd and they
> >     all use the
> >     >     same subject name constraint in their profile.
> >     >
> >     >     1. Was any certificate profile modified (caIPAserviceCert
>
> or
> >     >     KDCs_PKINIT_Certs)? You can use
> >     >     ipa certprofile-show <name> --out /dev/stdout
> >     >     And then check the part related to Subject Name
> >     Constraint. In my
> >     >     default installation, I have
> >     >
>
> policyset.serverCertSet.1.constraint.class_id=subjectNameConstraintImpl
> >     >     policyset.serverCertSet.1.constraint.name
> >     <http://policyset.serverCertSet.1.constraint.name>
> >     >     <http://policyset.serverCertSet.1.constraint.name>=Subject
> >     Name
> >     >     Constraint
> >     >     policyset.serverCertSet.1.constraint.params.accept=true
> >     >
> >      policyset.serverCertSet.1.constraint.params.pattern=CN=[^,]+,.+
> >     >
>
> policyset.serverCertSet.1.default.class_id=subjectNameDefaultImpl
> >     >     policyset.serverCertSet.1.default.name
> >     <http://policyset.serverCertSet.1.default.name>
> >     >     <http://policyset.serverCertSet.1.default.name>=Subject
> >     Name Default
> >     >     policyset.serverCertSet.1.default.params.name
> >     <http://policyset.serverCertSet.1.default.params.name>
> >     >
> >      <http://policyset.serverCertSet.1.default.params.name>=CN=$
>
> request.req_subject_name.cn
> >     <http://request.req_subject_name.cn>
> >     >     <http://request.req_subject_name.cn>$, O=IPA.TEST
> >     >
> >     >     which means that the subject name should match CN=
>
> followed by
> >     >     (anything except a comma) multiple times then a comma and
> >     any char
> >     >     multiple times.
> >     >
> >     >     2. If the profile wasn't changed, can you check in
> >     >     /var/log/pki/pki-tomcat/ca/debug.$DATE.log the received
> >     certificate
> >     >     request? Does its subject match the pattern? The error
> >     >     messagejava.lang.StringIndexOutOfBoundsException: String
> >     index out
> >     >     of range: -1 hints that an expected pattern was not found.
> >     >
> >     >     flo
> >     >
> >     >     On Thu, Sep 14, 2023 at 4:11 PM Evan G via FreeIPA-users
> >     >     <[e-mail]
> >     <mailto:[e-mail]>
> >     >     <mailto:[e-mail]
> >     <mailto:[e-mail]>>> wrote:
> >     >
> >     >         Hi Rob,
> >     >
> >     >         When we start tomcat with the date rolled back, we are
>
> not
> >     >         seeing any errors at all. All of the ipa services
>
> start up
> >     >         without issue. The problem is in actually renewing the
> >     certs,
> >     >         when we do so we have seen many different errors as
> >     we've been
> >     >         troubleshooting -- mostly this one: `ca-error: Server
>
> at
> >     >         https://<HOSTNAME>/ipa/xml failed request, will retry:
> >     4035 (RPC
> >     >         failed at server.  Request failed with status 500:
>
> Non-2xx
> >     >         response from CA REST API: 500. String index out of
>
> range:
> >     >
>
> -1).[02/Aug/2023:00:00:31][ajp-bio-127.0.0.1-8009-exec-2]:
> >     >         EnrollProfile: populate: begins`
> >     >
> >     >         When I restart certmonger after all services up, these
> >     are the
> >     >         errors that I am seeing in the tomcat debug logs:
> >     >         ```
> >     >         [02/Aug/2023:00:00:31][ajp-bio-127.0.0.1-8009-exec-2]:
> >     >         BasicProfile: populate: policy setid =serverCertSet
> >     >         [02/Aug/2023:00:00:31][ajp-bio-127.0.0.1-8009-exec-2]:
> >     >         EnrollDefault: populate: SubjectNameDefault: start
> >     >         java.lang.StringIndexOutOfBoundsException: String
> >     index out of
> >     >         range: -1
> >     >                 at java.lang.String.substring(String.java:1967)
> >     >                 at
> >     >
>
> com.netscape.certsrv.pattern.Pattern.substitute2(Pattern.java:132)
> >     >                 at
> >     >
>
> com.netscape.cms.profile.def.EnrollDefault.mapPattern(EnrollDefault.java:815)
> >     >                 at
> >     >
>
> com.netscape.cms.profile.def.SubjectNameDefault.populate(SubjectNameDefault.java:160)
> >     >                 at
> >     >
>
> com.netscape.cms.profile.def.EnrollDefault.populate(EnrollDefault.java:226)
> >     >                 at
> >     >
>
> com.netscape.cms.profile.common.BasicProfile.populate(BasicProfile.java:1114)
> >     >                 at
> >     >
>
> com.netscape.cms.profile.common.EnrollProfile.populate(EnrollProfile.java:2626)
> >     >                 at
> >     >
>
> com.netscape.cms.servlet.cert.CertProcessor.populateRequests(CertProcessor.java:379)
> >     >                 at
> >     >
>
> com.netscape.cms.servlet.cert.EnrollmentProcessor.processEnrollment(EnrollmentProcessor.java:188)
> >     >                 at
> >     >
>
> com.netscape.cms.servlet.cert.EnrollmentProcessor.processEnrollment(EnrollmentProcessor.java:96)
> >     >                 at
> >     >
>
> com.netscape.cms.servlet.cert.CertRequestDAO.submitRequest(CertRequestDAO.java:197)
> >     >                 at
> >     >
>
> org.dogtagpki.server.ca.rest.CertRequestService.enrollCert(CertRequestService.java:155)
> >     >                 at
> >     sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> >     >         Method)
> >     >                 at
> >     >
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> >     >                 at
> >     >
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >     >                 at
> >     java.lang.reflect.Method.invoke(Method.java:498)
> >     >                 at
> >     >
>
> org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:137)
> >     >                 at
> >     >
>
> org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:280)
> >     >                 at
> >     >
>
> org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:234)
> >     >                 at
> >     >
>
> org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:221)
> >     >                 at
> >     >
>
> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:356)
> >     >                 at
> >     >
>
> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:179)
> >     >                 at
> >     >
>
> org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:220)
> >     >                 at
> >     >
>
> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
> >     >                 at
> >     >
>
> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
> >     >                 at
> >     >
> >      javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
> >     >                 at
> >     sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> >     >         Method)
> >     >                 at
> >     >
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> >     >                 at
> >     >
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >     >                 at
> >     java.lang.reflect.Method.invoke(Method.java:498)
> >     >                 at
> >     >
>
> org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)
> >     >                 at
> >     >
>
> org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
> >     >                 at
> >     java.security.AccessController.doPrivileged(Native
> >     >         Method)
> >     >                 at
> >     >
> >      javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
> >     >                 at
> >     >
>
> org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)
> >     >                 at
> >     >
>
> org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:175)
> >     >                 at
> >     >
>
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:297)
> >     >                 at
> >     >
>
> org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:55)
> >     >                 at
> >     >
>
> org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:191)
> >     >                 at
> >     >
>
> org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:187)
> >     >                 at
> >     java.security.AccessController.doPrivileged(Native
> >     >         Method)
> >     >                 at
> >     >
>
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:186)
> >     >                 at
> >     >
>
> org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
> >     >                 at
> >     sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> >     >         Method)
> >     >                 at
> >     >
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> >     >                 at
> >     >
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >     >                 at
> >     java.lang.reflect.Method.invoke(Method.java:498)
> >     >                 at
> >     >
>
> org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)
> >     >                 at
> >     >
>
> org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
> >     >                 at
> >     java.security.AccessController.doPrivileged(Native
> >     >         Method)
> >     >                 at
> >     >
> >      javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
> >     >                 at
> >     >
>
> org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)
> >     >                 at
> >     >
>
> org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:260)
> >     >                 at
> >     >
>
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:237)
> >     >                 at
> >     >
>
> org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:55)
> >     >                 at
> >     >
>
> org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:191)
> >     >                 at
> >     >
>
> org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:187)
> >     >                 at
> >     java.security.AccessController.doPrivileged(Native
> >     >         Method)
> >     >                 at
> >     >
>
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:186)
> >     >                 at
> >     >
>
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:218)
> >     >                 at
> >     >
>
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
> >     >                 at
> >     >
>
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:498)
> >     >                 at
> >     >
>
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
> >     >                 at
> >     >
>
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
> >     >                 at
> >     >
>
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962)
> >     >                 at
> >     >
>
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
> >     >                 at
> >     >
>
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)
> >     >                 at
> >     >
>
> org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:190)
> >     >                 at
> >     >
>
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
> >     >                 at org.apache.tomcat.util.net
> >     <http://org.apache.tomcat.util.net>
> >     >
> >      <http://org.apache.tomcat.util.net
> >
> > .JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
> >         >                 at
> >         >
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> >     >                 at
> >     >
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> >     >                 at
> >     >
>
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> >     >                 at java.lang.Thread.run(Thread.java:750)
> >     >         ```
> >     >
> >     >         This is what we see when we run `getcert list` and
> >     `ipa-getcert
> >     >         list` respectively:
> >     >
> >     >         ```
> >     >         Number of certificates and requests being tracked: 9.
> >     >         Request ID '20190920201259':
> >     >                 status: CA_UNREACHABLE
> >     >                 ca-error: Server at https://<HOSTNAME>/ipa/xml
> >     failed
> >     >         request, will retry: 4035 (RPC failed at server.
> >     Request failed
> >     >         with status 500: Non-2xx response from CA REST API:
> >     500. String
> >     >         index out of range: -1).
> >     >                 stuck: no
> >     >                 key pair storage:
> >     >         type=FILE,location='/var/kerberos/krb5kdc/kdc.key'
> >     >                 certificate:
> >     >         type=FILE,location='/var/kerberos/krb5kdc/kdc.crt'
> >     >                 CA: IPA
> >     >                 issuer: CN=Certificate Authority,O=<OU>
> >     >                 subject: CN=<HOSTNAME>,O=<OU>
> >     >                 expires: 2023-08-25 18:05:07 UTC
> >     >                 principal name: krbtgt/<OU>@<OU>
> >     >                 key usage:
> >     >
> >      digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> >     >                 eku: id-kp-serverAuth,id-pkinit-KPKdc
> >     >                 pre-save command:
> >     >                 post-save command:
> >     >         /usr/libexec/ipa/certmonger/renew_kdc_cert
> >     >                 track: yes
> >     >                 auto-renew: yes
> >     >         Request ID '20210908000050':
> >     >                 status: MONITORING
> >     >                 stuck: no
> >     >                 key pair storage:
> >     >
>
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
> >     >         cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin
>
> set
> >     >                 certificate:
> >     >
>
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
> >     >         cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
> >     >                 CA: dogtag-ipa-ca-renew-agent
> >     >                 issuer: CN=Certificate Authority,O=<OU>
> >     >                 subject: CN=CA Audit,O=<OU>
> >     >                 expires: 2025-07-21 02:36:57 UTC
> >     >                 key usage:
> >     digitalSignature,keyEncipherment,dataEncipherment
> >     >                 eku: id-kp-serverAuth,id-kp-clientAuth
> >     >                 pre-save command:
> >     /usr/libexec/ipa/certmonger/stop_pkicad
> >     >                 post-save command:
> >     >         /usr/libexec/ipa/certmonger/renew_ca_cert
> >     "auditSigningCert
> >     >         cert-pki-ca"
> >     >                 track: yes
> >     >                 auto-renew: yes
> >     >         Request ID '20210908000051':
> >     >                 status: MONITORING
> >     >                 stuck: no
> >     >                 key pair storage:
> >     >
>
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
> >     >         cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin
>
> set
> >     >                 certificate:
> >     >
>
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
> >     >         cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
> >     >                 CA: dogtag-ipa-ca-renew-agent
> >     >                 issuer: CN=Certificate Authority,O=<OU>
> >     >                 subject: CN=OCSP Subsystem,O=<OU>
> >     >                 expires: 2025-07-21 02:36:17 UTC
> >     >                 key usage:
> >     digitalSignature,keyEncipherment,dataEncipherment
> >     >                 eku: id-kp-serverAuth,id-kp-clientAuth
> >     >                 pre-save command:
> >     /usr/libexec/ipa/certmonger/stop_pkicad
> >     >                 post-save command:
> >     >         /usr/libexec/ipa/certmonger/renew_ca_cert
>
> "ocspSigningCert
> >     >         cert-pki-ca"
> >     >                 track: yes
> >     >                 auto-renew: yes
> >     >         Request ID '20210908000052':
> >     >                 status: MONITORING
> >     >                 stuck: no
> >     >                 key pair storage:
> >     >
>
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
> >     >         cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin
>
> set
> >     >                 certificate:
> >     >
>
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
> >     >         cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
> >     >                 CA: dogtag-ipa-ca-renew-agent
> >     >                 issuer: CN=Certificate Authority,O=<OU>
> >     >                 subject: CN=CA Subsystem,O=<OU>
> >     >                 expires: 2025-07-21 02:37:17 UTC
> >     >                 key usage:
> >     digitalSignature,keyEncipherment,dataEncipherment
> >     >                 eku: id-kp-serverAuth,id-kp-clientAuth
> >     >                 pre-save command:
> >     /usr/libexec/ipa/certmonger/stop_pkicad
> >     >                 post-save command:
> >     >         /usr/libexec/ipa/certmonger/renew_ca_cert
>
> "subsystemCert
> >     >         cert-pki-ca"
> >     >                 track: yes
> >     >                 auto-renew: yes
> >     >         Request ID '20210908000053':
> >     >                 status: MONITORING
> >     >                 stuck: no
> >     >                 key pair storage:
> >     >
>
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
> >     >         cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin
>
> set
> >     >                 certificate:
> >     >
>
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
> >     >         cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
> >     >                 CA: dogtag-ipa-ca-renew-agent
> >     >                 issuer: CN=Certificate Authority,O=<OU>
> >     >                 subject: CN=Certificate Authority,O=<OU>
> >     >                 expires: 2039-09-20 20:11:25 UTC
> >     >                 key usage:
> >     >         digitalSignature,nonRepudiation,keyCertSign,cRLSign
> >     >                 pre-save command:
> >     /usr/libexec/ipa/certmonger/stop_pkicad
> >     >                 post-save command:
> >     >         /usr/libexec/ipa/certmonger/renew_ca_cert
>
> "caSigningCert
> >     >         cert-pki-ca"
> >     >                 track: yes
> >     >                 auto-renew: yes
> >     >         Request ID '20210908000054':
> >     >                 status: MONITORING
> >     >                 stuck: no
> >     >                 key pair storage:
> >     >         type=FILE,location='/var/lib/ipa/ra-agent.key'
> >     >                 certificate:
> >     type=FILE,location='/var/lib/ipa/ra-agent.pem'
> >     >                 CA: dogtag-ipa-ca-renew-agent
> >     >                 issuer: CN=Certificate Authority,O=<OU>
> >     >                 subject: CN=IPA RA,O=<OU>
> >     >                 expires: 2025-06-26 02:36:15 UTC
> >     >                 key usage:
> >     digitalSignature,keyEncipherment,dataEncipherment
> >     >                 eku: id-kp-serverAuth,id-kp-clientAuth
> >     >                 pre-save command:
> >     >         /usr/libexec/ipa/certmonger/renew_ra_cert_pre
> >     >                 post-save command:
> >     /usr/libexec/ipa/certmonger/renew_ra_cert
> >     >                 track: yes
> >     >                 auto-renew: yes
> >     >         Request ID '20210908000055':
> >     >                 status: MONITORING
> >     >                 stuck: no
> >     >                 key pair storage:
> >     >
>
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
> >     >         cert-pki-ca',token='NSS FIPS 140-2 Certificate DB',pin
>
> set
> >     >                 certificate:
> >     >
>
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
> >     >         cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
> >     >                 CA: dogtag-ipa-ca-renew-agent
> >     >                 issuer: CN=Certificate Authority,O=<OU>
> >     >                 subject: CN=<HOSTNAME>,O=<OU>
> >     >                 expires: 2025-07-21 02:36:37 UTC
> >     >                 dns: <HOSTNAME>
> >     >                 key usage:
> >     digitalSignature,keyEncipherment,dataEncipherment
> >     >                 eku: id-kp-serverAuth,id-kp-clientAuth
> >     >                 pre-save command:
> >     /usr/libexec/ipa/certmonger/stop_pkicad
> >     >                 post-save command:
> >     >         /usr/libexec/ipa/certmonger/renew_ca_cert "Server-Cert
> >     cert-pki-ca"
> >     >                 track: yes
> >     >                 auto-renew: yes
> >     >         Request ID '20210908000056':
> >     >                 status: CA_UNREACHABLE
> >     >                 ca-error: Server at https://<HOSTNAME>/ipa/xml
> >     failed
> >     >         request, will retry: 4035 (RPC failed at server.
> >     Request failed
> >     >         with status 500: Non-2xx response from CA REST API:
> >     500. String
> >     >         index out of range: -1).
> >     >                 stuck: no
> >     >                 key pair storage:
> >     >
>
> type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> >     >         FIPS 140-2 Certificate
> >     >         DB',pinfile='/etc/dirsrv/slapd-<OU>/pwdfile.txt'
> >     >                 certificate:
> >     >
>
> type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> >     >         FIPS 140-2 Certificate DB'
> >     >                 CA: IPA
> >     >                 issuer: CN=Certificate Authority,O=<OU>
> >     >                 subject: CN=<HOSTNAME>,O=<OU>
> >     >                 expires: 2023-09-03 18:30:48 UTC
> >     >                 dns: <HOSTNAME>
> >     >                 key usage:
> >     >
> >      digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> >     >                 eku: id-kp-serverAuth,id-kp-clientAuth
> >     >                 pre-save command:
> >     >                 post-save command:
> >     >         /usr/libexec/ipa/certmonger/restart_dirsrv <OU>
> >     >                 track: yes
> >     >                 auto-renew: yes
> >     >         Request ID '20210908000057':
> >     >                 status: CA_UNREACHABLE
> >     >                 ca-error: Server at https://<HOSTNAME>/ipa/xml
> >     failed
> >     >         request, will retry: 4035 (RPC failed at server.
> >     Request failed
> >     >         with status 500: Non-2xx response from CA REST API:
> >     500. String
> >     >         index out of range: -1).
> >     >                 stuck: no
> >     >                 key pair storage:
> >     >
>
> type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> >     >         FIPS 140-2 Certificate
> >     DB',pinfile='/etc/httpd/alias/pwdfile.txt'
> >     >                 certificate:
> >     >
>
> type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> >     >         FIPS 140-2 Certificate DB'
> >     >                 CA: IPA
> >     >                 issuer: CN=Certificate Authority,O=<OU>
> >     >                 subject: CN=<HOSTNAME>,O=<OU>
> >     >                 expires: 2023-09-03 18:30:48 UTC
> >     >                 dns: <HOSTNAME>
> >     >                 key usage:
> >     >
> >      digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> >     >                 eku: id-kp-serverAuth,id-kp-clientAuth
> >     >                 pre-save command:
> >     >                 post-save command:
> >     /usr/libexec/ipa/certmonger/restart_httpd
> >     >                 track: yes
> >     >                 auto-renew: yes
> >     >         ```
> >     >
> >     >        ��```
> >     >         Number of certificates and requests being tracked: 9.
> >     >         Request ID '20190920201259':
> >     >                 status: CA_UNREACHABLE
> >     >                 ca-error: Server at https://<HOSTNAME>/ipa/xml
> >     failed
> >     >         request, will retry: 4035 (RPC failed at server.
> >     Request failed
> >     >         with status 500: Non-2xx response from CA REST API:
> >     500. String
> >     >         index out of range: -1).
> >     >                 stuck: no
> >     >                 key pair storage:
> >     >         type=FILE,location='/var/kerberos/krb5kdc/kdc.key'
> >     >                 certificate:
> >     >         type=FILE,location='/var/kerberos/krb5kdc/kdc.crt'
> >     >                 CA: IPA
> >     >                 issuer: CN=Certificate Authority,O=<OU>
> >     >                 subject: CN=<HOSTNAME>,O=<OU>
> >     >                 expires: 2023-08-25 18:05:07 UTC
> >     >                 principal name: krbtgt/<OU>@<OU>
> >     >                 key usage:
> >     >
> >      digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> >     >                 eku: id-kp-serverAuth,id-pkinit-KPKdc
> >     >                 pre-save command:
> >     >                 post-save command:
> >     >         /usr/libexec/ipa/certmonger/renew_kdc_cert
> >     >                 track: yes
> >     >                 auto-renew: yes
> >     >         Request ID '20210908000056':
> >     >                 status: CA_UNREACHABLE
> >     >                 ca-error: Server at https://<HOSTNAME>/ipa/xml
> >     failed
> >     >         request, will retry: 4035 (RPC failed at server.
> >     Request failed
> >     >         with status 500: Non-2xx response from CA REST API:
> >     500. String
> >     >         index out of range: -1).
> >     >                 stuck: no
> >     >                 key pair storage:
> >     >
>
> type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> >     >         FIPS 140-2 Certificate
> >     >         DB',pinfile='/etc/dirsrv/slapd-<OU>/pwdfile.txt'
> >     >                 certificate:
> >     >
>
> type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> >     >         FIPS 140-2 Certificate DB'
> >     >                 CA: IPA
> >     >                 issuer: CN=Certificate Authority,O=<OU>
> >     >                 subject: CN=<HOSTNAME>,O=<OU>
> >     >                 expires: 2023-09-03 18:30:48 UTC
> >     >                 dns: <HOSTNAME>
> >     >                 key usage:
> >     >
> >      digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> >     >                 eku: id-kp-serverAuth,id-kp-clientAuth
> >     >                 pre-save command:
> >     >                 post-save command:
> >     >         /usr/libexec/ipa/certmonger/restart_dirsrv <OU>
> >     >                 track: yes
> >     >                 auto-renew: yes
> >     >         Request ID '20210908000057':
> >     >                 status: CA_UNREACHABLE
> >     >                 ca-error: Server at https://<HOSTNAME>/ipa/xml
> >     failed
> >     >         request, will retry: 4035 (RPC failed at server.
> >     Request failed
> >     >         with status 500: Non-2xx response from CA REST API:
> >     500. String
> >     >         index out of range: -1).
> >     >                 stuck: no
> >     >                 key pair storage:
> >     >
>
> type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> >     >         FIPS 140-2 Certificate
> >     DB',pinfile='/etc/httpd/alias/pwdfile.txt'
> >     >                 certificate:
> >     >
>
> type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> >     >         FIPS 140-2 Certificate DB'
> >     >                 CA: IPA
> >     >                 issuer: CN=Certificate Authority,O=<OU>
> >     >                 subject: CN=<HOSTNAME>,O=<OU>
> >     >                 expires: 2023-09-03 18:30:48 UTC
> >     >                 dns: <HOSTNAME>
> >     >                 key usage:
> >     >
> >      digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> >     >                 eku: id-kp-serverAuth,id-kp-clientAuth
> >     >                 pre-save command:
> >     >                 post-save command:
> >     /usr/libexec/ipa/certmonger/restart_httpd
> >     >                 track: yes
> >     >                 auto-renew: yes
> >     >         ```
> >     >         _______________________________________________
> >     >         FreeIPA-users mailing list --
> >     >         [e-mail]
> >     <mailto:[e-mail]>
> >     >         <mailto:[e-mail]
> >     <mailto:[e-mail]>>
> >     >         To unsubscribe send an email to
> >     >         [e-mail]
> >     <mailto:[e-mail]>
> >     >         <mailto:[e-mail]
> >     <mailto:[e-mail]>>
> >     >         Fedora Code of Conduct:
> >     >
> >      https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> >     >         List Guidelines:
> >     >         https://fedoraproject.org/wiki/Mailing_list_guidelines
> >     >         List Archives:
> >     >
>
> https://lists.fedorahosted.org/archives/list/[e-mail]...
> >     >         Do not reply to spam, report it:
> >     >         https://pagure.io/fedora-infrastructure/new_issue
> >     >
> >     >
> >     > _______________________________________________
> >     > FreeIPA-users mailing list --
> >     [e-mail]
> >     <mailto:[e-mail]>
> >     > To unsubscribe send an email to
> >     [e-mail]
> >     <mailto:[e-mail]>
> >     > Fedora Code of Conduct:
> >     https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> >     > List Guidelines:
> >     https://fedoraproject.org/wiki/Mailing_list_guidelines
> >     > List Archives:
>
> https://lists.fedorahosted.org/archives/list/[e-mail]...
> >     > Do not reply to spam, report it:
> >     https://pagure.io/fedora-infrastructure/new_issue
> >     >

[IT Guy]

I think the failed to authenticate was a red herring. I am still getting
the string index out of range even after making changes to
the /usr/share/ipa/profiles/caIPAserviceCert.cfg

On Fri, Sep 15, 2023 at 10:40 AM IT Guy 
wrote:
> Rob,
>
> Thank you. So it looks like what I shared as the current config is
> actually what was there when the snapshot was taken. The changes outlined
> in that post were made on a machine which has since been deleted. So what I
> am saying is that the config I shared does not include any of the changes
> my co-worker had made. When I make the changes to match what Florence
> shared as a default config and attempt to renew the certs, I am now getting
> the following error:
>
> ca-error: Server at https://<HOSTNAME>/ipa/xml failed request, will
> retry: 4016 (RPC failed at server.  Failed to authenticate to CA REST API).
>
> How can I make sure that the credentials that are attempting to be used
> are valid for this operation?
>
> Many thanks,
>
> Evan
>
> On Fri, Sep 15, 2023 at 10:25 AM Rob Crittenden 
> wrote:
> > IT Guy wrote:
> > > OK just one more thing to add, I had run across this link during
> > > troubleshooting and it seems that my co-worker had updated some of the
> > > lines in this configuration according to the steps outlined in this
> > > forum post: https://pagure.io/freeipa/issue/7267
> > >
> > > However I can say that this was a last ditch effort to try and get the
> > > renewals working, we had already been troubleshooting for 3+ days at the
> > > point that this was changed.
> >
> > Looks like this was not correctly applied: "Especially note the
> > replacement of occurrences of $$ with $."
> >
> > Your profile has $$ and it should be $, according to Fraser.
> >
> > rob
> > > On Fri, Sep 15, 2023 at 9:58 AM IT Guy <[e-mail]
> > > > wrote:
> > > Wow that worked Rob, thank you! If I compare the values that
> > > Florence sent to what I have in this file, the only difference is
> > > this line:
> > >
> > > policyset.serverCertSet.1.default.params.name
> > > <http://policyset.serverCertSet.1.default.params.name>=CN=$$
> >
> > request.req_subject_name.cn
> > > <http://request.req_subject_name.cn>$$, $SUBJECT_DN_O
> > >
> > > Here's the full snippet for reference:
> >
> > policyset.serverCertSet.1.constraint.class_id=subjectNameConstraintImpl
> > > policyset.serverCertSet.1.constraint.name
> > > <http://policyset.serverCertSet.1.constraint.name>=Subject Name
> > > Constraint
> > > policyset.serverCertSet.1.constraint.params.accept=true
> > > policyset.serverCertSet.1.constraint.params.pattern=CN=[^,]+,.+
> > > policyset.serverCertSet.1.default.class_id=subjectNameDefaultImpl
> > > policyset.serverCertSet.1.default.name
> > > <http://policyset.serverCertSet.1.default.name>=Subject Name
> >
> > Default
> > > policyset.serverCertSet.1.default.params.name
> > > <http://policyset.serverCertSet.1.default.params.name>=CN=$$
> >
> > request.req_subject_name.cn
> > > <http://request.req_subject_name.cn>$$, $SUBJECT_DN_O
> > >
> > >
> > > One other thing I wanted to call out is that I have a good snapshot
> > > of this server that I have restored a couple of times to try
> > > different things and the one that got me the farthest was when I
> > > changed the name of the cert from our custom name back to
> > > Server-Cert. Even when I had the config this way I still could not
> > > renew but maybe modifying something in the above config plus
> > > changing back to Server-Cert could alleviate the issue?
> > >
> > > Many thanks,
> > >
> > > Evan
> > >
> > > On Fri, Sep 15, 2023 at 9:47 AM Rob Crittenden <[e-mail]
> > > <mailto:[e-mail]>> wrote:
> > >
> > >     IT Guy via FreeIPA-users wrote:
> > >     > Hi Florence,
> > >     >
> > >     > Thank you for your response. What does it mean if I run the
> >
> > ipa
> > >     > certprofile-show command as outlined above and it just hangs?
> > >     I don't
> > >     > think there is any other way to see the settings you mentioned
> > >     unless
> > >     > this command is able to run right?
> > >
> > >     I can't explain why it would hang but you can get the profile
> > >     directly
> > >     from LDAP:
> > >
> > >     $ ldapsearch -LLL -o ldif-wrap=no -x -D 'cn=directory manager'
> >
> > -W -b
> > >     cn=caIPAserviceCert,ou=certificateProfiles,ou=ca,o=ipaca
> > >     certProfileConfig > /tmp/profile
> > >
> > >     Edit this file and remove the dn value and 'certProfileConfig::
> > >     ' then
> > >     base64-decode the result.
> > >
> > >     The final really huge string should look something like:
> > >
> > >     YXV0aC5pbnN0YW5jZV9pZ...=
> > >
> > >     I used the coreutils base64 program to decode it:
> > >
> > >     $ base64 -d /tmp/profile
> > >
> > >     rob
> > >     >
> > >     > Many thanks,
> > >     >
> > >     > Evan
> > >     >
> > >     > On Fri, Sep 15, 2023 at 3:19 AM Florence Blanc-Renaud
> > >     <[e-mail] <mailto:[e-mail]>
> > >     > <mailto:[e-mail] <mailto:[e-mail]>>> wrote:
> > >     >
> > >     >     Hi,
> > >     >     it seems that PKI is not happy with the subject name of
> >
> > the
> > >     >     certificates.
> > >     >     The failing certs are for KDC, dirsrv and httpd and they
> > >     all use the
> > >     >     same subject name constraint in their profile.
> > >     >
> > >     >     1. Was any certificate profile modified (caIPAserviceCert
> >
> > or
> > >     >     KDCs_PKINIT_Certs)? You can use
> > >     >     ipa certprofile-show <name> --out /dev/stdout
> > >     >     And then check the part related to Subject Name
> > >     Constraint. In my
> > >     >     default installation, I have
> > >     >
> >
> > policyset.serverCertSet.1.constraint.class_id=subjectNameConstraintImpl
> > >     >     policyset.serverCertSet.1.constraint.name
> > >     <http://policyset.serverCertSet.1.constraint.name>
> > >     >     <http://policyset.serverCertSet.1.constraint.name
> > >
> > > =Subject
> > >         Name
> > >         >     Constraint
> > >         >     policyset.serverCertSet.1.constraint.params.accept=true
> > >         >
> > >          policyset.serverCertSet.1.constraint.params.pattern=CN=[^,]+,.+
> > >         >
> >
> > policyset.serverCertSet.1.default.class_id=subjectNameDefaultImpl
> > >     >     policyset.serverCertSet.1.default.name
> > >     <http://policyset.serverCertSet.1.default.name>
> > >     >     <http://policyset.serverCertSet.1.default.name>=Subject
> > >     Name Default
> > >     >     policyset.serverCertSet.1.default.params.name
> > >     <http://policyset.serverCertSet.1.default.params.name>
> > >     >
> > >      <http://policyset.serverCertSet.1.default.params.name>=CN=$
> >
> > request.req_subject_name.cn
> > >     <http://request.req_subject_name.cn>
> > >     >     <http://request.req_subject_name.cn>$, O=IPA.TEST
> > >     >
> > >     >     which means that the subject name should match CN=
> >
> > followed by
> > >     >     (anything except a comma) multiple times then a comma and
> > >     any char
> > >     >     multiple times.
> > >     >
> > >     >     2. If the profile wasn't changed, can you check in
> > >     >     /var/log/pki/pki-tomcat/ca/debug.$DATE.log the received
> > >     certificate
> > >     >     request? Does its subject match the pattern? The error
> > >     >     messagejava.lang.StringIndexOutOfBoundsException: String
> > >     index out
> > >     >     of range: -1 hints that an expected pattern was not found.
> > >     >
> > >     >     flo
> > >     >
> > >     >     On Thu, Sep 14, 2023 at 4:11 PM Evan G via FreeIPA-users
> > >     >     <[e-mail]
> > >     <mailto:[e-mail]>
> > >     >     <mailto:[e-mail]
> > >     <mailto:[e-mail]>>> wrote:
> > >     >
> > >     >         Hi Rob,
> > >     >
> > >     >         When we start tomcat with the date rolled back, we
> >
> > are not
> > >     >         seeing any errors at all. All of the ipa services
> >
> > start up
> > >     >         without issue. The problem is in actually renewing the
> > >     certs,
> > >     >         when we do so we have seen many different errors as
> > >     we've been
> > >     >         troubleshooting -- mostly this one: `ca-error: Server
> >
> > at
> > >     >         https://<HOSTNAME>/ipa/xml failed request, will
> >
> > retry:
> > >     4035 (RPC
> > >     >         failed at server.  Request failed with status 500:
> >
> > Non-2xx
> > >     >         response from CA REST API: 500. String index out of
> >
> > range:
> > >     >
> >
> > -1).[02/Aug/2023:00:00:31][ajp-bio-127.0.0.1-8009-exec-2]:
> > >     >         EnrollProfile: populate: begins`
> > >     >
> > >     >         When I restart certmonger after all services up, these
> > >     are the
> > >     >         errors that I am seeing in the tomcat debug logs:
> > >     >         ```
> > >     >         [02/Aug/2023:00:00:31][ajp-bio-127.0.0.1-8009-exec-2]:
> > >     >         BasicProfile: populate: policy setid =serverCertSet
> > >     >         [02/Aug/2023:00:00:31][ajp-bio-127.0.0.1-8009-exec-2]:
> > >     >         EnrollDefault: populate: SubjectNameDefault: start
> > >     >         java.lang.StringIndexOutOfBoundsException: String
> > >     index out of
> > >     >         range: -1
> > >     >                 at
> >
> > java.lang.String.substring(String.java:1967)
> > >     >                 at
> > >     >
> >
> > com.netscape.certsrv.pattern.Pattern.substitute2(Pattern.java:132)
> > >     >                 at
> > >     >
> >
> > com.netscape.cms.profile.def.EnrollDefault.mapPattern(EnrollDefault.java:815)
> > >     >                 at
> > >     >
> >
> > com.netscape.cms.profile.def.SubjectNameDefault.populate(SubjectNameDefault.java:160)
> > >     >                 at
> > >     >
> >
> > com.netscape.cms.profile.def.EnrollDefault.populate(EnrollDefault.java:226)
> > >     >                 at
> > >     >
> >
> > com.netscape.cms.profile.common.BasicProfile.populate(BasicProfile.java:1114)
> > >     >                 at
> > >     >
> >
> > com.netscape.cms.profile.common.EnrollProfile.populate(EnrollProfile.java:2626)
> > >     >                 at
> > >     >
> >
> > com.netscape.cms.servlet.cert.CertProcessor.populateRequests(CertProcessor.java:379)
> > >     >                 at
> > >     >
> >
> > com.netscape.cms.servlet.cert.EnrollmentProcessor.processEnrollment(EnrollmentProcessor.java:188)
> > >     >                 at
> > >     >
> >
> > com.netscape.cms.servlet.cert.EnrollmentProcessor.processEnrollment(EnrollmentProcessor.java:96)
> > >     >                 at
> > >     >
> >
> > com.netscape.cms.servlet.cert.CertRequestDAO.submitRequest(CertRequestDAO.java:197)
> > >     >                 at
> > >     >
> >
> > org.dogtagpki.server.ca.rest.CertRequestService.enrollCert(CertRequestService.java:155)
> > >     >                 at
> > >     sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> > >     >         Method)
> > >     >                 at
> > >     >
> >
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > >     >                 at
> > >     >
> >
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > >     >                 at
> > >     java.lang.reflect.Method.invoke(Method.java:498)
> > >     >                 at
> > >     >
> >
> > org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:137)
> > >     >                 at
> > >     >
> >
> > org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:280)
> > >     >                 at
> > >     >
> >
> > org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:234)
> > >     >                 at
> > >     >
> >
> > org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:221)
> > >     >                 at
> > >     >
> >
> > org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:356)
> > >     >                 at
> > >     >
> >
> > org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:179)
> > >     >                 at
> > >     >
> >
> > org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:220)
> > >     >                 at
> > >     >
> >
> > org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
> > >     >                 at
> > >     >
> >
> > org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
> > >     >                 at
> > >     >
> > >      javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
> > >     >                 at
> > >     sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> > >     >         Method)
> > >     >                 at
> > >     >
> >
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > >     >                 at
> > >     >
> >
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > >     >                 at
> > >     java.lang.reflect.Method.invoke(Method.java:498)
> > >     >                 at
> > >     >
> >
> > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)
> > >     >                 at
> > >     >
> >
> > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
> > >     >                 at
> > >     java.security.AccessController.doPrivileged(Native
> > >     >         Method)
> > >     >                 at
> > >     >
> > >      javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
> > >     >                 at
> > >     >
> >
> > org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)
> > >     >                 at
> > >     >
> >
> > org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:175)
> > >     >                 at
> > >     >
> >
> > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:297)
> > >     >                 at
> > >     >
> >
> > org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:55)
> > >     >                 at
> > >     >
> >
> > org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:191)
> > >     >                 at
> > >     >
> >
> > org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:187)
> > >     >                 at
> > >     java.security.AccessController.doPrivileged(Native
> > >     >         Method)
> > >     >                 at
> > >     >
> >
> > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:186)
> > >     >                 at
> > >     >
> >
> > org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
> > >     >                 at
> > >     sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> > >     >         Method)
> > >     >                 at
> > >     >
> >
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > >     >                 at
> > >     >
> >
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > >     >                 at
> > >     java.lang.reflect.Method.invoke(Method.java:498)
> > >     >                 at
> > >     >
> >
> > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)
> > >     >                 at
> > >     >
> >
> > org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
> > >     >                 at
> > >     java.security.AccessController.doPrivileged(Native
> > >     >         Method)
> > >     >                 at
> > >     >
> > >      javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
> > >     >                 at
> > >     >
> >
> > org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)
> > >     >                 at
> > >     >
> >
> > org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:260)
> > >     >                 at
> > >     >
> >
> > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:237)
> > >     >                 at
> > >     >
> >
> > org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:55)
> > >     >                 at
> > >     >
> >
> > org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:191)
> > >     >                 at
> > >     >
> >
> > org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:187)
> > >     >                 at
> > >     java.security.AccessController.doPrivileged(Native
> > >     >         Method)
> > >     >                 at
> > >     >
> >
> > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:186)
> > >     >                 at
> > >     >
> >
> > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:218)
> > >     >                 at
> > >     >
> >
> > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
> > >     >                 at
> > >     >
> >
> > org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:498)
> > >     >                 at
> > >     >
> >
> > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
> > >     >                 at
> > >     >
> >
> > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
> > >     >                 at
> > >     >
> >
> > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962)
> > >     >                 at
> > >     >
> >
> > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
> > >     >                 at
> > >     >
> >
> > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)
> > >     >                 at
> > >     >
> >
> > org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:190)
> > >     >                 at
> > >     >
> >
> > org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
> > >     >                 at org.apache.tomcat.util.net
> > >     <http://org.apache.tomcat.util.net>
> > >     >
> > >      <http://org.apache.tomcat.util.net
> > >
> > > .JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
> > >         >                 at
> > >         >
> >
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > >     >                 at
> > >     >
> >
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > >     >                 at
> > >     >
> >
> > org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> > >     >                 at java.lang.Thread.run(Thread.java:750)
> > >     >         ```
> > >     >
> > >     >         This is what we see when we run `getcert list` and
> > >     `ipa-getcert
> > >     >         list` respectively:
> > >     >
> > >     >         ```
> > >     >         Number of certificates and requests being tracked: 9.
> > >     >         Request ID '20190920201259':
> > >     >                 status: CA_UNREACHABLE
> > >     >                 ca-error: Server at https://
> >
> > <HOSTNAME>/ipa/xml
> > >     failed
> > >     >         request, will retry: 4035 (RPC failed at server.
> > >     Request failed
> > >     >         with status 500: Non-2xx response from CA REST API:
> > >     500. String
> > >     >         index out of range: -1).
> > >     >                 stuck: no
> > >     >                 key pair storage:
> > >     >         type=FILE,location='/var/kerberos/krb5kdc/kdc.key'
> > >     >                 certificate:
> > >     >         type=FILE,location='/var/kerberos/krb5kdc/kdc.crt'
> > >     >                 CA: IPA
> > >     >                 issuer: CN=Certificate Authority,O=<OU>
> > >     >                 subject: CN=<HOSTNAME>,O=<OU>
> > >     >                 expires: 2023-08-25 18:05:07 UTC
> > >     >                 principal name: krbtgt/<OU>@<OU>
> > >     >                 key usage:
> > >     >
> >
> > digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> > >     >                 eku: id-kp-serverAuth,id-pkinit-KPKdc
> > >     >                 pre-save command:
> > >     >                 post-save command:
> > >     >         /usr/libexec/ipa/certmonger/renew_kdc_cert
> > >     >                 track: yes
> > >     >                 auto-renew: yes
> > >     >         Request ID '20210908000050':
> > >     >                 status: MONITORING
> > >     >                 stuck: no
> > >     >                 key pair storage:
> > >     >
> >
> > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
> > >     >         cert-pki-ca',token='NSS FIPS 140-2 Certificate
> >
> > DB',pin set
> > >     >                 certificate:
> > >     >
> >
> > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
> > >     >         cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
> > >     >                 CA: dogtag-ipa-ca-renew-agent
> > >     >                 issuer: CN=Certificate Authority,O=<OU>
> > >     >                 subject: CN=CA Audit,O=<OU>
> > >     >                 expires: 2025-07-21 02:36:57 UTC
> > >     >                 key usage:
> > >     digitalSignature,keyEncipherment,dataEncipherment
> > >     >                 eku: id-kp-serverAuth,id-kp-clientAuth
> > >     >                 pre-save command:
> > >     /usr/libexec/ipa/certmonger/stop_pkicad
> > >     >                 post-save command:
> > >     >         /usr/libexec/ipa/certmonger/renew_ca_cert
> > >     "auditSigningCert
> > >     >         cert-pki-ca"
> > >     >                 track: yes
> > >     >                 auto-renew: yes
> > >     >         Request ID '20210908000051':
> > >     >                 status: MONITORING
> > >     >                 stuck: no
> > >     >                 key pair storage:
> > >     >
> >
> > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
> > >     >         cert-pki-ca',token='NSS FIPS 140-2 Certificate
> >
> > DB',pin set
> > >     >                 certificate:
> > >     >
> >
> > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
> > >     >         cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
> > >     >                 CA: dogtag-ipa-ca-renew-agent
> > >     >                 issuer: CN=Certificate Authority,O=<OU>
> > >     >                 subject: CN=OCSP Subsystem,O=<OU>
> > >     >                 expires: 2025-07-21 02:36:17 UTC
> > >     >                 key usage:
> > >     digitalSignature,keyEncipherment,dataEncipherment
> > >     >                 eku: id-kp-serverAuth,id-kp-clientAuth
> > >     >                 pre-save command:
> > >     /usr/libexec/ipa/certmonger/stop_pkicad
> > >     >                 post-save command:
> > >     >         /usr/libexec/ipa/certmonger/renew_ca_cert
> >
> > "ocspSigningCert
> > >     >         cert-pki-ca"
> > >     >                 track: yes
> > >     >                 auto-renew: yes
> > >     >         Request ID '20210908000052':
> > >     >                 status: MONITORING
> > >     >                 stuck: no
> > >     >                 key pair storage:
> > >     >
> >
> > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
> > >     >         cert-pki-ca',token='NSS FIPS 140-2 Certificate
> >
> > DB',pin set
> > >     >                 certificate:
> > >     >
> >
> > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
> > >     >         cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
> > >     >                 CA: dogtag-ipa-ca-renew-agent
> > >     >                 issuer: CN=Certificate Authority,O=<OU>
> > >     >                 subject: CN=CA Subsystem,O=<OU>
> > >     >                 expires: 2025-07-21 02:37:17 UTC
> > >     >                 key usage:
> > >     digitalSignature,keyEncipherment,dataEncipherment
> > >     >                 eku: id-kp-serverAuth,id-kp-clientAuth
> > >     >                 pre-save command:
> > >     /usr/libexec/ipa/certmonger/stop_pkicad
> > >     >                 post-save command:
> > >     >         /usr/libexec/ipa/certmonger/renew_ca_cert
> >
> > "subsystemCert
> > >     >         cert-pki-ca"
> > >     >                 track: yes
> > >     >                 auto-renew: yes
> > >     >         Request ID '20210908000053':
> > >     >                 status: MONITORING
> > >     >                 stuck: no
> > >     >                 key pair storage:
> > >     >
> >
> > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
> > >     >         cert-pki-ca',token='NSS FIPS 140-2 Certificate
> >
> > DB',pin set
> > >     >                 certificate:
> > >     >
> >
> > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
> > >     >         cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
> > >     >                 CA: dogtag-ipa-ca-renew-agent
> > >     >                 issuer: CN=Certificate Authority,O=<OU>
> > >     >                 subject: CN=Certificate Authority,O=<OU>
> > >     >                 expires: 2039-09-20 20:11:25 UTC
> > >     >                 key usage:
> > >     >         digitalSignature,nonRepudiation,keyCertSign,cRLSign
> > >     >                 pre-save command:
> > >     /usr/libexec/ipa/certmonger/stop_pkicad
> > >     >                 post-save command:
> > >     >         /usr/libexec/ipa/certmonger/renew_ca_cert
> >
> > "caSigningCert
> > >     >         cert-pki-ca"
> > >     >                 track: yes
> > >     >                 auto-renew: yes
> > >     >         Request ID '20210908000054':
> > >     >                 status: MONITORING
> > >     >                 stuck: no
> > >     >                 key pair storage:
> > >     >         type=FILE,location='/var/lib/ipa/ra-agent.key'
> > >     >                 certificate:
> > >     type=FILE,location='/var/lib/ipa/ra-agent.pem'
> > >     >                 CA: dogtag-ipa-ca-renew-agent
> > >     >                 issuer: CN=Certificate Authority,O=<OU>
> > >     >                 subject: CN=IPA RA,O=<OU>
> > >     >                 expires: 2025-06-26 02:36:15 UTC
> > >     >                 key usage:
> > >     digitalSignature,keyEncipherment,dataEncipherment
> > >     >                 eku: id-kp-serverAuth,id-kp-clientAuth
> > >     >                 pre-save command:
> > >     >         /usr/libexec/ipa/certmonger/renew_ra_cert_pre
> > >     >                 post-save command:
> > >     /usr/libexec/ipa/certmonger/renew_ra_cert
> > >     >                 track: yes
> > >     >                 auto-renew: yes
> > >     >         Request ID '20210908000055':
> > >     >                 status: MONITORING
> > >     >                 stuck: no
> > >     >                 key pair storage:
> > >     >
> >
> > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
> > >     >         cert-pki-ca',token='NSS FIPS 140-2 Certificate
> >
> > DB',pin set
> > >     >                 certificate:
> > >     >
> >
> > type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
> > >     >         cert-pki-ca',token='NSS FIPS 140-2 Certificate DB'
> > >     >                 CA: dogtag-ipa-ca-renew-agent
> > >     >                 issuer: CN=Certificate Authority,O=<OU>
> > >     >                 subject: CN=<HOSTNAME>,O=<OU>
> > >     >                 expires: 2025-07-21 02:36:37 UTC
> > >     >                 dns: <HOSTNAME>
> > >     >                 key usage:
> > >     digitalSignature,keyEncipherment,dataEncipherment
> > >     >                 eku: id-kp-serverAuth,id-kp-clientAuth
> > >     >                 pre-save command:
> > >     /usr/libexec/ipa/certmonger/stop_pkicad
> > >     >                 post-save command:
> > >     >         /usr/libexec/ipa/certmonger/renew_ca_cert "Server-Cert
> > >     cert-pki-ca"
> > >     >                 track: yes
> > >     >                 auto-renew: yes
> > >     >         Request ID '20210908000056':
> > >     >                 status: CA_UNREACHABLE
> > >     >                 ca-error: Server at https://
> >
> > <HOSTNAME>/ipa/xml
> > >     failed
> > >     >         request, will retry: 4035 (RPC failed at server.
> > >     Request failed
> > >     >         with status 500: Non-2xx response from CA REST API:
> > >     500. String
> > >     >         index out of range: -1).
> > >     >                 stuck: no
> > >     >                 key pair storage:
> > >     >
> >
> > type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> > >     >         FIPS 140-2 Certificate
> > >     >         DB',pinfile='/etc/dirsrv/slapd-<OU>/pwdfile.txt'
> > >     >                 certificate:
> > >     >
> >
> > type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> > >     >         FIPS 140-2 Certificate DB'
> > >     >                 CA: IPA
> > >     >                 issuer: CN=Certificate Authority,O=<OU>
> > >     >                 subject: CN=<HOSTNAME>,O=<OU>
> > >     >                 expires: 2023-09-03 18:30:48 UTC
> > >     >                 dns: <HOSTNAME>
> > >     >                 key usage:
> > >     >
> >
> > digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> > >     >                 eku: id-kp-serverAuth,id-kp-clientAuth
> > >     >                 pre-save command:
> > >     >                 post-save command:
> > >     >         /usr/libexec/ipa/certmonger/restart_dirsrv <OU>
> > >     >                 track: yes
> > >     >                 auto-renew: yes
> > >     >         Request ID '20210908000057':
> > >     >                 status: CA_UNREACHABLE
> > >     >                 ca-error: Server at https://
> >
> > <HOSTNAME>/ipa/xml
> > >     failed
> > >     >         request, will retry: 4035 (RPC failed at server.
> > >     Request failed
> > >     >         with status 500: Non-2xx response from CA REST API:
> > >     500. String
> > >     >         index out of range: -1).
> > >     >                 stuck: no
> > >     >                 key pair storage:
> > >     >
> >
> > type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> > >     >         FIPS 140-2 Certificate
> > >     DB',pinfile='/etc/httpd/alias/pwdfile.txt'
> > >     >                 certificate:
> > >     >
> >
> > type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> > >     >         FIPS 140-2 Certificate DB'
> > >     >                 CA: IPA
> > >     >                 issuer: CN=Certificate Authority,O=<OU>
> > >     >                 subject: CN=<HOSTNAME>,O=<OU>
> > >     >                 expires: 2023-09-03 18:30:48 UTC
> > >     >                 dns: <HOSTNAME>
> > >     >                 key usage:
> > >     >
> >
> > digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> > >     >                 eku: id-kp-serverAuth,id-kp-clientAuth
> > >     >                 pre-save command:
> > >     >                 post-save command:
> > >     /usr/libexec/ipa/certmonger/restart_httpd
> > >     >                 track: yes
> > >     >                 auto-renew: yes
> > >     >         ```
> > >     >
> > >     >        ��```
> > >     >         Number of certificates and requests being tracked: 9.
> > >     >         Request ID '20190920201259':
> > >     >                 status: CA_UNREACHABLE
> > >     >                 ca-error: Server at https://
> >
> > <HOSTNAME>/ipa/xml
> > >     failed
> > >     >         request, will retry: 4035 (RPC failed at server.
> > >     Request failed
> > >     >         with status 500: Non-2xx response from CA REST API:
> > >     500. String
> > >     >         index out of range: -1).
> > >     >                 stuck: no
> > >     >                 key pair storage:
> > >     >         type=FILE,location='/var/kerberos/krb5kdc/kdc.key'
> > >     >                 certificate:
> > >     >         type=FILE,location='/var/kerberos/krb5kdc/kdc.crt'
> > >     >                 CA: IPA
> > >     >                 issuer: CN=Certificate Authority,O=<OU>
> > >     >                 subject: CN=<HOSTNAME>,O=<OU>
> > >     >                 expires: 2023-08-25 18:05:07 UTC
> > >     >                 principal name: krbtgt/<OU>@<OU>
> > >     >                 key usage:
> > >     >
> >
> > digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> > >     >                 eku: id-kp-serverAuth,id-pkinit-KPKdc
> > >     >                 pre-save command:
> > >     >                 post-save command:
> > >     >         /usr/libexec/ipa/certmonger/renew_kdc_cert
> > >     >                 track: yes
> > >     >                 auto-renew: yes
> > >     >         Request ID '20210908000056':
> > >     >                 status: CA_UNREACHABLE
> > >     >                 ca-error: Server at https://
> >
> > <HOSTNAME>/ipa/xml
> > >     failed
> > >     >         request, will retry: 4035 (RPC failed at server.
> > >     Request failed
> > >     >         with status 500: Non-2xx response from CA REST API:
> > >     500. String
> > >     >         index out of range: -1).
> > >     >                 stuck: no
> > >     >                 key pair storage:
> > >     >
> >
> > type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> > >     >         FIPS 140-2 Certificate
> > >     >         DB',pinfile='/etc/dirsrv/slapd-<OU>/pwdfile.txt'
> > >     >                 certificate:
> > >     >
> >
> > type=NSSDB,location='/etc/dirsrv/slapd-<OU>',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> > >     >         FIPS 140-2 Certificate DB'
> > >     >                 CA: IPA
> > >     >                 issuer: CN=Certificate Authority,O=<OU>
> > >     >                 subject: CN=<HOSTNAME>,O=<OU>
> > >     >                 expires: 2023-09-03 18:30:48 UTC
> > >     >                 dns: <HOSTNAME>
> > >     >                 key usage:
> > >     >
> >
> > digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> > >     >                 eku: id-kp-serverAuth,id-kp-clientAuth
> > >     >                 pre-save command:
> > >     >                 post-save command:
> > >     >         /usr/libexec/ipa/certmonger/restart_dirsrv <OU>
> > >     >                 track: yes
> > >     >                 auto-renew: yes
> > >     >         Request ID '20210908000057':
> > >     >                 status: CA_UNREACHABLE
> > >     >                 ca-error: Server at https://
> >
> > <HOSTNAME>/ipa/xml
> > >     failed
> > >     >         request, will retry: 4035 (RPC failed at server.
> > >     Request failed
> > >     >         with status 500: Non-2xx response from CA REST API:
> > >     500. String
> > >     >         index out of range: -1).
> > >     >                 stuck: no
> > >     >                 key pair storage:
> > >     >
> >
> > type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> > >     >         FIPS 140-2 Certificate
> > >     DB',pinfile='/etc/httpd/alias/pwdfile.txt'
> > >     >                 certificate:
> > >     >
> >
> > type=NSSDB,location='/etc/httpd/alias',nickname='CN=<HOSTNAME>,O=<OU>',token='NSS
> > >     >         FIPS 140-2 Certificate DB'
> > >     >                 CA: IPA
> > >     >                 issuer: CN=Certificate Authority,O=<OU>
> > >     >                 subject: CN=<HOSTNAME>,O=<OU>
> > >     >                 expires: 2023-09-03 18:30:48 UTC
> > >     >                 dns: <HOSTNAME>
> > >     >                 key usage:
> > >     >
> >
> > digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> > >     >                 eku: id-kp-serverAuth,id-kp-clientAuth
> > >     >                 pre-save command:
> > >     >                 post-save command:
> > >     /usr/libexec/ipa/certmonger/restart_httpd
> > >     >                 track: yes
> > >     >                 auto-renew: yes
> > >     >         ```
> > >     >         _______________________________________________
> > >     >         FreeIPA-users mailing list --
> > >     >         [e-mail]
> > >     <mailto:[e-mail]>
> > >     >         <mailto:[e-mail]
> > >     <mailto:[e-mail]>>
> > >     >         To unsubscribe send an email to
> > >     >         [e-mail]
> > >     <mailto:[e-mail]>
> > >     >         <mailto:[e-mail]
> > >     <mailto:[e-mail]>>
> > >     >         Fedora Code of Conduct:
> > >     >
> > >      https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > >     >         List Guidelines:
> > >     >
> >
> > https://fedoraproject.org/wiki/Mailing_list_guidelines
> > >     >         List Archives:
> > >     >
> >
> > https://lists.fedorahosted.org/archives/list/[e-mail]...
> > >     >         Do not reply to spam, report it:
> > >     >         https://pagure.io/fedora-infrastructure/new_issue
> > >     >
> > >     >
> > >     > _______________________________________________
> > >     > FreeIPA-users mailing list --
> > >     [e-mail]
> > >     <mailto:[e-mail]>
> > >     > To unsubscribe send an email to
> > >     [e-mail]
> > >     <mailto:[e-mail]>
> > >     > Fedora Code of Conduct:
> > >     https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > >     > List Guidelines:
> > >     https://fedoraproject.org/wiki/Mailing_list_guidelines
> > >     > List Archives:
> >
> > https://lists.fedorahosted.org/archives/list/[e-mail]...
> > >     > Do not reply to spam, report it:
> > >     https://pagure.io/fedora-infrastructure/new_issue
> > >     >