Cristian Le via FreeIPA-users wrote:
Ok, let me walk through some of the specific errors, and I will also
censor out some of the output since this is going to the public
mail-list as well.
Starting from the beginning.
- I have set the date to `1 month` before certificate expired with `sudo
date`
- I ran `ipactl restart --force` and `pki-tomcatd` failed
If tomcat fails to start then you need to stop and figure out why. Any
further poking at certificates won't yield anything useful without a
working CA.
- I try to run `ipa-cacert-manage renew` with `--external-ca` and
`--external-cert-file`, with the CSR signed with `openssl x509 -req
-copy_extensions=copyall`, but this one failed because of `ERROR: CA
certificate chain in is incomplete: missing certificate with subject`,
even though I passed both the root and new CA
What failed?
Then I have tried to manually add the new CA, but then, that did not
work. So moving on to the next layer, when I run to `sudo getcert list`,
the certificates I get:
Manually add how?
```
Request ID '20220810070559':
status: CA_UNREACHABLE
ca-error: Error setting up ccache for "host" service on client
using default keytab: Cannot contact any KDC for requested realm.
The KDC isn't running.
stuck: no
key pair storage: type=FILE,location='/var/kerberos/krb5kdc/kdc.key'
certificate: type=FILE,location='/var/kerberos/krb5kdc/kdc.crt'
CA: IPA
issuer: CN=FreeIPA Root Certificate Authority,O=My Lab
subject: CN=ipa1.my.lab,O=My Lab
issued: 2023-09-20 14:21:40 UTC
expires: 2025-09-20 14:21:40 UTC
Request ID '20230920203122':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=FreeIPA Root Certificate Authority,O=My Lab
subject: CN=CA Audit,O=My Lab
issued: 2023-09-20 14:21:35 UTC
expires: 2025-09-09 14:21:35 UTC
Request ID '20230920203123':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
cert-pki-ca',token='NSS
Certificate DB',pin set
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
cert-pki-ca',token='NSS
Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=FreeIPA Root Certificate Authority,O=My Lab
subject: CN=OCSP Subsystem,O=My Lab
issued: 2023-09-20 14:21:33 UTC
expires: 2025-09-09 14:21:33 UTC
Request ID '20230920203125':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=FreeIPA Root Certificate Authority,O=My Lab
subject: CN=CA Subsystem,O=My Lab
issued: 2023-09-20 14:21:31 UTC
expires: 2025-09-09 14:21:31 UTC
Request ID '20230920203128':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=FreeIPA Root Certificate Authority,O=My Lab
subject: CN=ipa1.my.lab,O=My Lab
issued: 2023-09-20 11:29:21 UTC
expires: 2023-12-20 11:29:21 UTC
Request ID '20230920203130':
status: MONITORING
stuck: no
key pair storage: type=FILE,location='/var/lib/ipa/ra-agent.key'
certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=FreeIPA Root Certificate Authority,O=My Lab
subject: CN=IPA RA,O=My Lab
issued: 2023-09-20 14:21:36 UTC
expires: 2025-09-09 14:21:36 UTC
Request ID '20230920203131':
status: CA_UNREACHABLE
ca-error: Error setting up ccache for "host" service on client
using default keytab: Cannot contact any KDC for requested realm.
stuck: no
key pair storage:
type=NSSDB,location='/etc/dirsrv/slapd-MY-LAB',nickname='Server-Cert',token='NSS
Certificate DB',pinfile='/etc/dirsrv/slapd-MY-LAB/pwdfile.txt'
certificate:
type=NSSDB,location='/etc/dirsrv/slapd-MY-LAB',nickname='Server-Cert',token='NSS
Certificate DB'
CA: IPA
issuer: CN=FreeIPA Root Certificate Authority,O=My Lab
subject: CN=ipa1.my.lab,O=My Lab
issued: 2023-09-20 14:21:39 UTC
expires: 2025-09-20 14:21:39 UTC
Request ID '20230920203134':
status: CA_UNREACHABLE
stuck: no
key pair storage:
type=FILE,location='/var/lib/ipa/private/httpd.key',pinfile='/var/lib/ipa/passwds/ipa1.my.lab-443-RSA'
certificate: type=FILE,location='/var/lib/ipa/certs/httpd.crt'
CA: IPA
issuer: CN=My Lab Root CA,O=My Lab
subject: CN=ipa1.my.lab,O=My Lab
issued: 2023-07-10 04:31:16 UTC
expires: 2023-08-09 04:31:16 UTC
Request ID '20230510042436':
status: MONITORING
ca-error: Updated certificate not available
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=My Lab Root CA,O=My Lab
subject: CN=FreeIPA Root Certificate Authority,O=My Lab
issued: 2022-08-10 04:29:31 UTC
expires: 2023-08-10 04:29:31 UTC
```
Note that even the `MONITORING` ones are unusable due to the
non-overlapping date. So when I do something like `ipa-certupdate` I get:
`The ipa-certupdate command failed, exception: DatabaseError: Connect
error: error:0A000086:SSL routines::certificate verify failed (unable to
get issuer certificate)` when trying to use `ipaldap.py`.
It is a mess, that's why I think it is better to have a way to simply
re-bootstrap all certificates.
There isn't really a good way to start over.
rob
On 2023/09/22 15:57, Florence Blanc-Renaud wrote:
> Hi,
>
> On Fri, Sep 22, 2023 at 12:36 PM Cristian Le <fedora(a)lecris.me> wrote:
>
> Hi Florence,
>
> Thanks for the feedback, let me clarify the situation on the
> certificates:
> - External CA is still valid and it is a self-signed certificate
> that we use for other services. So we can manually sign any
> service certificates to get them back up and running
> - IPA CA is expired, let's say Aug/10
> - I have managed to import a renewed IPA CA and ran `ipa-cert-fix`
> (and also seemed to have run `ipa-certupdate`) on a current date,
> let's say Sep/20. But not all services were recovered and now
> there is no overlap between earliest date in service certificates
> and the original IPA CA
>
> Which are the not-recovered services? Can you provide the output of
> "getcert list" at the current date?
> flo
>
> - I have run a backup, but also did some system upgrades to get
> the `ipa-cacert-manage prune` command, but when I've tried to
> recover it, I've found that the backup was not there.
>
> > you can still find the original certificates in the LDAP
> database (below ou=certificateRepository,ou=ca,o=ipaca) but it
> requires a bit of searching. You would need to restore the expired
> certificates, go back in time and force the renewal.
>
> I suspect we cannot do this all within LDAP right? If we get back
> the expired certificates, how do we restore them in each service?
> `httpd` is straightforward, and I guess `nssdb` should be doable,
> assuming the same key is used, but is there another database type
> where the certificates are located? Are all the certificates
> tracked by `getcert list`? Is it safe to assume that after running
> something like `ipa-cert-fix`, they are using the same private key?
>
> Some symptoms in the current setup:
> - When we are forward in time, `pki-tomcatd` is able to run, but
> then I can't do any `ipa-cert-fix` or `ipa-cacert-manage renew`.
> From what I've read, all of these commands (or at least
> `ipa-cacert-manage renew`) must be done backwards in time.
> - When we are backwards in time `pki-tomcatd` is unable to run,
> failing to access `:8080/ca/admin/ca/getStatus`. This then blocks
> various other services to be run. But about `ipactl restart`, only
> `pki-tomcatd` service is actually failing (and ipa service itself
> of course).
>
> I have navigated to `ou=certificateRepository,ou=ca,o=ipaca` and
> indeed there are still a bunch of certificates there in linear
> order. What are the services I should look for in there? I am
> using Apache Directory Studio and I can download the
> `userCertificate`. Should I just run `certutil -A` with those
> values with corresponding `subjectName`?
>
> BTW, I want to document this process on the website, should I make
> a PR on the github repo or is there somewhere else?
>
> Kind regards,
> Cristian
>
> On 2023/09/22 9:00, Florence Blanc-Renaud wrote:
>> Hi,
>>
>> On Thu, Sep 21, 2023 at 5:04 PM Cristian Le via FreeIPA-users
>> <freeipa-users(a)lists.fedorahosted.org> wrote:
>>
>> I have tried my luck around with all the helpers: `pki-server
>> cert-fix`, `ipa-cacert-manage`, `ipa-certupdate`, etc. but
>> each one is failing on me for multiple reasons.
>> - `ipa-cacert-manage` Cannot update the CA with
>> `--external-cert-file` because the root ca is not detected to
>> be in the trust list
>>
>> This command is useful if you need to trust a new external CA or
>> renew IPA CA. Is your IPA CA expired?
>>
>> - `ipa-cert-fix` Was run without overlapping validity time,
>> and the certificate were re-created, so now it is not
>> recoverable, neither back in time, nor in current time
>>
>> It is recommended to do a backup before running ipa-cert-fix. If
>> you didn't, and want to try the back-in-time method, you can
>> still find the original certificates in the LDAP database (below
>> ou=certificateRepository,ou=ca,o=ipaca) but it requires a bit of
>> searching. You would need to restore the expired certificates, go
>> back in time and force the renewal.
>>
>> - `pki-tomcat` is failing
>>
>> What is the current situation? Which certs are expired (getcert
>> list)? If you start the services with "ipactl start
>> --ignore-service-failures", is pki-tomcat the only service failing?
>> flo
>>
>> It is quite a mess and I would like to ask for some guidance
>> on how one could recover manually from such dependency issues:
>> - Is it possible to do a `ipa-server-install` and keep the
>> user data?
>>
>> - If I sign all of the service's certificates manually, what
>> are all of the manual steps needed to get the services back
>> up so that the helpers can be run.
>> - I've tried to install the CA certificate in the nssdb
>> database, ldap, and /etc/ipa/ca.crt. Are there other locations?
>> - I've recreated an httpd certificate signed by the root,
>> but I can't figure how to do the same with the ones located
>> in the nssdb database, i.e. to recreate a csr with the same
>> data as one of the certificates there
>> - What is the order of services that should be updated. My
>> understanding is CA -> `certutil`'s CA -> httpd + slapd +
>> pki-tomcat (not sure where the last one is or how to edit it)
>> -> `ipa-certupdate`
>> _______________________________________________
>> FreeIPA-users mailing list --
>> freeipa-users(a)lists.fedorahosted.org
>> To unsubscribe send an email to
>> freeipa-users-leave(a)lists.fedorahosted.org
>> Fedora Code of Conduct:
>>
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>> List Guidelines:
>>
https://fedoraproject.org/wiki/Mailing_list_guidelines
>> List Archives:
>>
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
>> Do not reply to spam, report it:
>>
https://pagure.io/fedora-infrastructure/new_issue
>>
_______________________________________________
FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
Do not reply to spam, report it:
https://pagure.io/fedora-infrastructure/new_issue