10:04 a.m.
I have tried my luck around with all the helpers: `pki-server cert-fix`,
`ipa-cacert-manage`, `ipa-certupdate`, etc. but each one is failing on me for multiple
reasons.
- `ipa-cacert-manage` Cannot update the CA with `--external-cert-file` because the root ca
is not detected to be in the trust list
- `ipa-cert-fix` Was run without overlapping validity time, and the certificate were
re-created, so now it is not recoverable, neither back in time, nor in current time
- `pki-tomcat` is failing
It is quite a mess and I would like to ask for some guidance on how one could recover
manually from such dependency issues:
- Is it possible to do a `ipa-server-install` and keep the user data?
- If I sign all of the service's certificates manually, what are all of the manual
steps needed to get the services back up so that the helpers can be run.
- I've tried to install the CA certificate in the nssdb database, ldap, and
/etc/ipa/ca.crt. Are there other locations?
- I've recreated an httpd certificate signed by the root, but I can't figure how
to do the same with the ones located in the nssdb database, i.e. to recreate a csr with
the same data as one of the certificates there
- What is the order of services that should be updated. My understanding is CA ->
`certutil`'s CA -> httpd + slapd + pki-tomcat (not sure where the last one is or
how to edit it) -> `ipa-certupdate`
2 a.m.
Hi,
On Thu, Sep 21, 2023 at 5:04 PM Cristian Le via FreeIPA-users <
freeipa-users(a)lists.fedorahosted.org> wrote:
I have tried my luck around with all the helpers: `pki-server
cert-fix`,
`ipa-cacert-manage`, `ipa-certupdate`, etc. but each one is failing on me
for multiple reasons.
- `ipa-cacert-manage` Cannot update the CA with `--external-cert-file`
because the root ca is not detected to be in the trust list
This command is useful if you need to trust a new external CA or renew IPA
CA. Is your IPA CA expired?
- `ipa-cert-fix` Was run without overlapping validity time, and the
certificate were re-created, so now it is not recoverable, neither
back in
time, nor in current time
It is recommended to do a backup before running ipa-cert-fix. If you
didn't, and want to try the back-in-time method, you can still find the
original certificates in the LDAP database (below
ou=certificateRepository,ou=ca,o=ipaca) but it requires a bit of searching.
You would need to restore the expired certificates, go back in time and
force the renewal.
- `pki-tomcat` is failing
What is the current situation? Which certs are expired (getcert list)? If
you start
the services with "ipactl start --ignore-service-failures", is
pki-tomcat the only service failing?
flo
It is quite a mess and I would like to ask for some guidance on how one
could recover manually from such dependency issues:
- Is it possible to do a `ipa-server-install` and keep the user data?
- If I sign all of the service's certificates manually, what are all of the
manual steps needed to get the services back up so that the helpers
can be
run.
- I've tried to install the CA certificate in the nssdb database, ldap,
and /etc/ipa/ca.crt. Are there other locations?
- I've recreated an httpd certificate signed by the root, but I can't
figure how to do the same with the ones located in the nssdb database, i.e.
to recreate a csr with the same data as one of the certificates there
- What is the order of services that should be updated. My understanding
is CA -> `certutil`'s CA -> httpd + slapd + pki-tomcat (not sure where the
last one is or how to edit it) -> `ipa-certupdate`
_______________________________________________
FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
Do not reply to spam, report it:
https://pagure.io/fedora-infrastructure/new_issue
5:36 a.m.
Hi Florence,
Thanks for the feedback, let me clarify the situation on the certificates:
- External CA is still valid and it is a self-signed certificate that we
use for other services. So we can manually sign any service certificates
to get them back up and running
- IPA CA is expired, let's say Aug/10
- I have managed to import a renewed IPA CA and ran `ipa-cert-fix` (and
also seemed to have run `ipa-certupdate`) on a current date, let's say
Sep/20. But not all services were recovered and now there is no overlap
between earliest date in service certificates and the original IPA CA
- I have run a backup, but also did some system upgrades to get the
`ipa-cacert-manage prune` command, but when I've tried to recover it,
I've found that the backup was not there.
> you can still find the original certificates in the LDAP database
(below ou=certificateRepository,ou=ca,o=ipaca) but it requires a bit of
searching. You would need to restore the expired certificates, go back
in time and force the renewal.
I suspect we cannot do this all within LDAP right? If we get back the
expired certificates, how do we restore them in each service? `httpd` is
straightforward, and I guess `nssdb` should be doable, assuming the same
key is used, but is there another database type where the certificates
are located? Are all the certificates tracked by `getcert list`? Is it
safe to assume that after running something like `ipa-cert-fix`, they
are using the same private key?
Some symptoms in the current setup:
- When we are forward in time, `pki-tomcatd` is able to run, but then I
can't do any `ipa-cert-fix` or `ipa-cacert-manage renew`. From what I've
read, all of these commands (or at least `ipa-cacert-manage renew`) must
be done backwards in time.
- When we are backwards in time `pki-tomcatd` is unable to run, failing
to access `:8080/ca/admin/ca/getStatus`. This then blocks various other
services to be run. But about `ipactl restart`, only `pki-tomcatd`
service is actually failing (and ipa service itself of course).
I have navigated to `ou=certificateRepository,ou=ca,o=ipaca` and indeed
there are still a bunch of certificates there in linear order. What are
the services I should look for in there? I am using Apache Directory
Studio and I can download the `userCertificate`. Should I just run
`certutil -A` with those values with corresponding `subjectName`?
BTW, I want to document this process on the website, should I make a PR
on the github repo or is there somewhere else?
Kind regards,
Cristian
On 2023/09/22 9:00, Florence Blanc-Renaud wrote:
Hi,
On Thu, Sep 21, 2023 at 5:04 PM Cristian Le via FreeIPA-users
<freeipa-users(a)lists.fedorahosted.org> wrote:
I have tried my luck around with all the helpers: `pki-server
cert-fix`, `ipa-cacert-manage`, `ipa-certupdate`, etc. but each
one is failing on me for multiple reasons.
- `ipa-cacert-manage` Cannot update the CA with
`--external-cert-file` because the root ca is not detected to be
in the trust list
This command is useful if you need to trust a new external CA or renew
IPA CA. Is your IPA CA expired?
- `ipa-cert-fix` Was run without overlapping validity time, and
the certificate were re-created, so now it is not recoverable,
neither back in time, nor in current time
It is recommended to do a backup before running ipa-cert-fix. If you
didn't, and want to try the back-in-time method, you can still find
the original certificates in the LDAP database (below
ou=certificateRepository,ou=ca,o=ipaca) but it requires a bit of
searching. You would need to restore the expired certificates, go back
in time and force the renewal.
- `pki-tomcat` is failing
What is the current situation? Which certs are expired (getcert list)?
If you start the services with "ipactl start
--ignore-service-failures", is pki-tomcat the only service failing?
flo
It is quite a mess and I would like to ask for some guidance on
how one could recover manually from such dependency issues:
- Is it possible to do a `ipa-server-install` and keep the user data?
- If I sign all of the service's certificates manually, what are
all of the manual steps needed to get the services back up so that
the helpers can be run.
- I've tried to install the CA certificate in the nssdb
database, ldap, and /etc/ipa/ca.crt. Are there other locations?
- I've recreated an httpd certificate signed by the root, but I
can't figure how to do the same with the ones located in the nssdb
database, i.e. to recreate a csr with the same data as one of the
certificates there
- What is the order of services that should be updated. My
understanding is CA -> `certutil`'s CA -> httpd + slapd +
pki-tomcat (not sure where the last one is or how to edit it) ->
`ipa-certupdate`
_______________________________________________
FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
To unsubscribe send an email to
freeipa-users-leave(a)lists.fedorahosted.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
Do not reply to spam, report it:
https://pagure.io/fedora-infrastructure/new_issue
8:57 a.m.
Hi,
On Fri, Sep 22, 2023 at 12:36 PM Cristian Le <fedora(a)lecris.me> wrote:
Hi Florence,
Thanks for the feedback, let me clarify the situation on the certificates:
- External CA is still valid and it is a self-signed certificate that we
use for other services. So we can manually sign any service certificates to
get them back up and running
- IPA CA is expired, let's say Aug/10
- I have managed to import a renewed IPA CA and ran `ipa-cert-fix` (and
also seemed to have run `ipa-certupdate`) on a current date, let's say
Sep/20. But not all services were recovered and now there is no overlap
between earliest date in service certificates and the original IPA CA
Which are the not-recovered services? Can you provide the output of
"getcert list" at the current date?
flo
- I have run a backup, but also did some system upgrades to get the
`ipa-cacert-manage prune` command, but when I've tried to recover it, I've
found that the backup was not there.
> you can still find the original certificates in the LDAP database
(below ou=certificateRepository,ou=ca,o=ipaca) but it requires a bit of
searching. You would need to restore the expired certificates, go back in
time and force the renewal.
I suspect we cannot do this all within LDAP right? If we get back the
expired certificates, how do we restore them in each service? `httpd` is
straightforward, and I guess `nssdb` should be doable, assuming the same
key is used, but is there another database type where the certificates are
located? Are all the certificates tracked by `getcert list`? Is it safe to
assume that after running something like `ipa-cert-fix`, they are using the
same private key?
Some symptoms in the current setup:
- When we are forward in time, `pki-tomcatd` is able to run, but then I
can't do any `ipa-cert-fix` or `ipa-cacert-manage renew`. From what I've
read, all of these commands (or at least `ipa-cacert-manage renew`) must be
done backwards in time.
- When we are backwards in time `pki-tomcatd` is unable to run, failing to
access `:8080/ca/admin/ca/getStatus`. This then blocks various other
services to be run. But about `ipactl restart`, only `pki-tomcatd` service
is actually failing (and ipa service itself of course).
I have navigated to `ou=certificateRepository,ou=ca,o=ipaca` and indeed
there are still a bunch of certificates there in linear order. What are the
services I should look for in there? I am using Apache Directory Studio and
I can download the `userCertificate`. Should I just run `certutil -A` with
those values with corresponding `subjectName`?
BTW, I want to document this process on the website, should I make a PR on
the github repo or is there somewhere else?
Kind regards,
Cristian
On 2023/09/22 9:00, Florence Blanc-Renaud wrote:
Hi,
On Thu, Sep 21, 2023 at 5:04 PM Cristian Le via FreeIPA-users <
freeipa-users(a)lists.fedorahosted.org> wrote:
> I have tried my luck around with all the helpers: `pki-server cert-fix`,
> `ipa-cacert-manage`, `ipa-certupdate`, etc. but each one is failing on me
> for multiple reasons.
> - `ipa-cacert-manage` Cannot update the CA with `--external-cert-file`
> because the root ca is not detected to be in the trust list
>
This command is useful if you need to trust a new external CA or renew IPA
CA. Is your IPA CA expired?
- `ipa-cert-fix` Was run without overlapping validity time, and the
> certificate were re-created, so now it is not recoverable, neither back in
> time, nor in current time
>
It is recommended to do a backup before running ipa-cert-fix. If you
didn't, and want to try the back-in-time method, you can still find the
original certificates in the LDAP database (below
ou=certificateRepository,ou=ca,o=ipaca) but it requires a bit of searching.
You would need to restore the expired certificates, go back in time and
force the renewal.
- `pki-tomcat` is failing
>
> What is the current situation? Which certs are expired (getcert list)? If
you start the services with "ipactl start --ignore-service-failures", is
pki-tomcat the only service failing?
flo
It is quite a mess and I would like to ask for some guidance on how one
> could recover manually from such dependency issues:
> - Is it possible to do a `ipa-server-install` and keep the user data?
>
- If I sign all of the service's certificates manually, what are all of
> the manual steps needed to get the services back up so that the helpers can
> be run.
> - I've tried to install the CA certificate in the nssdb database, ldap,
> and /etc/ipa/ca.crt. Are there other locations?
> - I've recreated an httpd certificate signed by the root, but I can't
> figure how to do the same with the ones located in the nssdb database, i.e.
> to recreate a csr with the same data as one of the certificates there
> - What is the order of services that should be updated. My understanding
> is CA -> `certutil`'s CA -> httpd + slapd + pki-tomcat (not sure where the
> last one is or how to edit it) -> `ipa-certupdate`
> _______________________________________________
> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
> To unsubscribe send an email to
> freeipa-users-leave(a)lists.fedorahosted.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
> Do not reply to spam, report it:
> https://pagure.io/fedora-infrastructure/new_issue
>
10:57 a.m.
Ok, let me walk through some of the specific errors, and I will also
censor out some of the output since this is going to the public
mail-list as well.
Starting from the beginning.
- I have set the date to `1 month` before certificate expired with `sudo
date`
- I ran `ipactl restart --force` and `pki-tomcatd` failed
- I try to run `ipa-cacert-manage renew` with `--external-ca` and
`--external-cert-file`, with the CSR signed with `openssl x509 -req
-copy_extensions=copyall`, but this one failed because of `ERROR: CA
certificate chain in is incomplete: missing certificate with subject`,
even though I passed both the root and new CA
Then I have tried to manually add the new CA, but then, that did not
work. So moving on to the next layer, when I run to `sudo getcert list`,
the certificates I get:
```
Request ID '20220810070559':
status: CA_UNREACHABLE
ca-error: Error setting up ccache for "host" service on client
using default keytab: Cannot contact any KDC for requested realm.
stuck: no
key pair storage:
type=FILE,location='/var/kerberos/krb5kdc/kdc.key'
certificate: type=FILE,location='/var/kerberos/krb5kdc/kdc.crt'
CA: IPA
issuer: CN=FreeIPA Root Certificate Authority,O=My Lab
subject: CN=ipa1.my.lab,O=My Lab
issued: 2023-09-20 14:21:40 UTC
expires: 2025-09-20 14:21:40 UTC
Request ID '20230920203122':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=FreeIPA Root Certificate Authority,O=My Lab
subject: CN=CA Audit,O=My Lab
issued: 2023-09-20 14:21:35 UTC
expires: 2025-09-09 14:21:35 UTC
Request ID '20230920203123':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=FreeIPA Root Certificate Authority,O=My Lab
subject: CN=OCSP Subsystem,O=My Lab
issued: 2023-09-20 14:21:33 UTC
expires: 2025-09-09 14:21:33 UTC
Request ID '20230920203125':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=FreeIPA Root Certificate Authority,O=My Lab
subject: CN=CA Subsystem,O=My Lab
issued: 2023-09-20 14:21:31 UTC
expires: 2025-09-09 14:21:31 UTC
Request ID '20230920203128':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=FreeIPA Root Certificate Authority,O=My Lab
subject: CN=ipa1.my.lab,O=My Lab
issued: 2023-09-20 11:29:21 UTC
expires: 2023-12-20 11:29:21 UTC
Request ID '20230920203130':
status: MONITORING
stuck: no
key pair storage: type=FILE,location='/var/lib/ipa/ra-agent.key'
certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=FreeIPA Root Certificate Authority,O=My Lab
subject: CN=IPA RA,O=My Lab
issued: 2023-09-20 14:21:36 UTC
expires: 2025-09-09 14:21:36 UTC
Request ID '20230920203131':
status: CA_UNREACHABLE
ca-error: Error setting up ccache for "host" service on client
using default keytab: Cannot contact any KDC for requested realm.
stuck: no
key pair storage:
type=NSSDB,location='/etc/dirsrv/slapd-MY-LAB',nickname='Server-Cert',token='NSS
Certificate DB',pinfile='/etc/dirsrv/slapd-MY-LAB/pwdfile.txt'
certificate:
type=NSSDB,location='/etc/dirsrv/slapd-MY-LAB',nickname='Server-Cert',token='NSS
Certificate DB'
CA: IPA
issuer: CN=FreeIPA Root Certificate Authority,O=My Lab
subject: CN=ipa1.my.lab,O=My Lab
issued: 2023-09-20 14:21:39 UTC
expires: 2025-09-20 14:21:39 UTC
Request ID '20230920203134':
status: CA_UNREACHABLE
stuck: no
key pair storage:
type=FILE,location='/var/lib/ipa/private/httpd.key',pinfile='/var/lib/ipa/passwds/ipa1.my.lab-443-RSA'
certificate: type=FILE,location='/var/lib/ipa/certs/httpd.crt'
CA: IPA
issuer: CN=My Lab Root CA,O=My Lab
subject: CN=ipa1.my.lab,O=My Lab
issued: 2023-07-10 04:31:16 UTC
expires: 2023-08-09 04:31:16 UTC
Request ID '20230510042436':
status: MONITORING
ca-error: Updated certificate not available
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=My Lab Root CA,O=My Lab
subject: CN=FreeIPA Root Certificate Authority,O=My Lab
issued: 2022-08-10 04:29:31 UTC
expires: 2023-08-10 04:29:31 UTC
```
Note that even the `MONITORING` ones are unusable due to the
non-overlapping date. So when I do something like `ipa-certupdate` I get:
`The ipa-certupdate command failed, exception: DatabaseError: Connect
error: error:0A000086:SSL routines::certificate verify failed (unable to
get issuer certificate)` when trying to use `ipaldap.py`.
It is a mess, that's why I think it is better to have a way to simply
re-bootstrap all certificates.
On 2023/09/22 15:57, Florence Blanc-Renaud wrote:
Hi,
On Fri, Sep 22, 2023 at 12:36 PM Cristian Le <fedora(a)lecris.me> wrote:
Hi Florence,
Thanks for the feedback, let me clarify the situation on the
certificates:
- External CA is still valid and it is a self-signed certificate
that we use for other services. So we can manually sign any
service certificates to get them back up and running
- IPA CA is expired, let's say Aug/10
- I have managed to import a renewed IPA CA and ran `ipa-cert-fix`
(and also seemed to have run `ipa-certupdate`) on a current date,
let's say Sep/20. But not all services were recovered and now
there is no overlap between earliest date in service certificates
and the original IPA CA
Which are the not-recovered services? Can you provide the output of
"getcert list" at the current date?
flo
- I have run a backup, but also did some system upgrades to get
the `ipa-cacert-manage prune` command, but when I've tried to
recover it, I've found that the backup was not there.
> you can still find the original certificates in the LDAP
database (below ou=certificateRepository,ou=ca,o=ipaca) but it
requires a bit of searching. You would need to restore the expired
certificates, go back in time and force the renewal.
I suspect we cannot do this all within LDAP right? If we get back
the expired certificates, how do we restore them in each service?
`httpd` is straightforward, and I guess `nssdb` should be doable,
assuming the same key is used, but is there another database type
where the certificates are located? Are all the certificates
tracked by `getcert list`? Is it safe to assume that after running
something like `ipa-cert-fix`, they are using the same private key?
Some symptoms in the current setup:
- When we are forward in time, `pki-tomcatd` is able to run, but
then I can't do any `ipa-cert-fix` or `ipa-cacert-manage renew`.
From what I've read, all of these commands (or at least
`ipa-cacert-manage renew`) must be done backwards in time.
- When we are backwards in time `pki-tomcatd` is unable to run,
failing to access `:8080/ca/admin/ca/getStatus`. This then blocks
various other services to be run. But about `ipactl restart`, only
`pki-tomcatd` service is actually failing (and ipa service itself
of course).
I have navigated to `ou=certificateRepository,ou=ca,o=ipaca` and
indeed there are still a bunch of certificates there in linear
order. What are the services I should look for in there? I am
using Apache Directory Studio and I can download the
`userCertificate`. Should I just run `certutil -A` with those
values with corresponding `subjectName`?
BTW, I want to document this process on the website, should I make
a PR on the github repo or is there somewhere else?
Kind regards,
Cristian
On 2023/09/22 9:00, Florence Blanc-Renaud wrote:
> Hi,
>
> On Thu, Sep 21, 2023 at 5:04 PM Cristian Le via FreeIPA-users
> <freeipa-users(a)lists.fedorahosted.org> wrote:
>
> I have tried my luck around with all the helpers: `pki-server
> cert-fix`, `ipa-cacert-manage`, `ipa-certupdate`, etc. but
> each one is failing on me for multiple reasons.
> - `ipa-cacert-manage` Cannot update the CA with
> `--external-cert-file` because the root ca is not detected to
> be in the trust list
>
> This command is useful if you need to trust a new external CA or
> renew IPA CA. Is your IPA CA expired?
>
> - `ipa-cert-fix` Was run without overlapping validity time,
> and the certificate were re-created, so now it is not
> recoverable, neither back in time, nor in current time
>
> It is recommended to do a backup before running ipa-cert-fix. If
> you didn't, and want to try the back-in-time method, you can
> still find the original certificates in the LDAP database (below
> ou=certificateRepository,ou=ca,o=ipaca) but it requires a bit of
> searching. You would need to restore the expired certificates, go
> back in time and force the renewal.
>
> - `pki-tomcat` is failing
>
> What is the current situation? Which certs are expired (getcert
> list)? If you start the services with "ipactl start
> --ignore-service-failures", is pki-tomcat the only service failing?
> flo
>
> It is quite a mess and I would like to ask for some guidance
> on how one could recover manually from such dependency issues:
> - Is it possible to do a `ipa-server-install` and keep the
> user data?
>
> - If I sign all of the service's certificates manually, what
> are all of the manual steps needed to get the services back
> up so that the helpers can be run.
> - I've tried to install the CA certificate in the nssdb
> database, ldap, and /etc/ipa/ca.crt. Are there other locations?
> - I've recreated an httpd certificate signed by the root,
> but I can't figure how to do the same with the ones located
> in the nssdb database, i.e. to recreate a csr with the same
> data as one of the certificates there
> - What is the order of services that should be updated. My
> understanding is CA -> `certutil`'s CA -> httpd + slapd +
> pki-tomcat (not sure where the last one is or how to edit it)
> -> `ipa-certupdate`
> _______________________________________________
> FreeIPA-users mailing list --
> freeipa-users(a)lists.fedorahosted.org
> To unsubscribe send an email to
> freeipa-users-leave(a)lists.fedorahosted.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines:
> https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
>
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
> Do not reply to spam, report it:
> https://pagure.io/fedora-infrastructure/new_issue
>
2:06 p.m.
Cristian Le via FreeIPA-users wrote:
Ok, let me walk through some of the specific errors, and I will also
censor out some of the output since this is going to the public
mail-list as well.
Starting from the beginning.
- I have set the date to `1 month` before certificate expired with `sudo
date`
- I ran `ipactl restart --force` and `pki-tomcatd` failed
If tomcat fails to start then you need to stop and figure out why. Any
further poking at certificates won't yield anything useful without a
working CA.
- I try to run `ipa-cacert-manage renew` with `--external-ca` and
`--external-cert-file`, with the CSR signed with `openssl x509 -req
-copy_extensions=copyall`, but this one failed because of `ERROR: CA
certificate chain in is incomplete: missing certificate with subject`,
even though I passed both the root and new CA
What failed?
Then I have tried to manually add the new CA, but then, that did not
work. So moving on to the next layer, when I run to `sudo getcert list`,
the certificates I get:
Manually add how?
```
Request ID '20220810070559':
status: CA_UNREACHABLE
ca-error: Error setting up ccache for "host" service on client
using default keytab: Cannot contact any KDC for requested realm.
The KDC isn't running.
stuck: no
key pair storage: type=FILE,location='/var/kerberos/krb5kdc/kdc.key'
certificate: type=FILE,location='/var/kerberos/krb5kdc/kdc.crt'
CA: IPA
issuer: CN=FreeIPA Root Certificate Authority,O=My Lab
subject: CN=ipa1.my.lab,O=My Lab
issued: 2023-09-20 14:21:40 UTC
expires: 2025-09-20 14:21:40 UTC
Request ID '20230920203122':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=FreeIPA Root Certificate Authority,O=My Lab
subject: CN=CA Audit,O=My Lab
issued: 2023-09-20 14:21:35 UTC
expires: 2025-09-09 14:21:35 UTC
Request ID '20230920203123':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
cert-pki-ca',token='NSS
Certificate DB',pin set
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
cert-pki-ca',token='NSS
Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=FreeIPA Root Certificate Authority,O=My Lab
subject: CN=OCSP Subsystem,O=My Lab
issued: 2023-09-20 14:21:33 UTC
expires: 2025-09-09 14:21:33 UTC
Request ID '20230920203125':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=FreeIPA Root Certificate Authority,O=My Lab
subject: CN=CA Subsystem,O=My Lab
issued: 2023-09-20 14:21:31 UTC
expires: 2025-09-09 14:21:31 UTC
Request ID '20230920203128':
status: MONITORING
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=FreeIPA Root Certificate Authority,O=My Lab
subject: CN=ipa1.my.lab,O=My Lab
issued: 2023-09-20 11:29:21 UTC
expires: 2023-12-20 11:29:21 UTC
Request ID '20230920203130':
status: MONITORING
stuck: no
key pair storage: type=FILE,location='/var/lib/ipa/ra-agent.key'
certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=FreeIPA Root Certificate Authority,O=My Lab
subject: CN=IPA RA,O=My Lab
issued: 2023-09-20 14:21:36 UTC
expires: 2025-09-09 14:21:36 UTC
Request ID '20230920203131':
status: CA_UNREACHABLE
ca-error: Error setting up ccache for "host" service on client
using default keytab: Cannot contact any KDC for requested realm.
stuck: no
key pair storage:
type=NSSDB,location='/etc/dirsrv/slapd-MY-LAB',nickname='Server-Cert',token='NSS
Certificate DB',pinfile='/etc/dirsrv/slapd-MY-LAB/pwdfile.txt'
certificate:
type=NSSDB,location='/etc/dirsrv/slapd-MY-LAB',nickname='Server-Cert',token='NSS
Certificate DB'
CA: IPA
issuer: CN=FreeIPA Root Certificate Authority,O=My Lab
subject: CN=ipa1.my.lab,O=My Lab
issued: 2023-09-20 14:21:39 UTC
expires: 2025-09-20 14:21:39 UTC
Request ID '20230920203134':
status: CA_UNREACHABLE
stuck: no
key pair storage:
type=FILE,location='/var/lib/ipa/private/httpd.key',pinfile='/var/lib/ipa/passwds/ipa1.my.lab-443-RSA'
certificate: type=FILE,location='/var/lib/ipa/certs/httpd.crt'
CA: IPA
issuer: CN=My Lab Root CA,O=My Lab
subject: CN=ipa1.my.lab,O=My Lab
issued: 2023-07-10 04:31:16 UTC
expires: 2023-08-09 04:31:16 UTC
Request ID '20230510042436':
status: MONITORING
ca-error: Updated certificate not available
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=My Lab Root CA,O=My Lab
subject: CN=FreeIPA Root Certificate Authority,O=My Lab
issued: 2022-08-10 04:29:31 UTC
expires: 2023-08-10 04:29:31 UTC
```
Note that even the `MONITORING` ones are unusable due to the
non-overlapping date. So when I do something like `ipa-certupdate` I get:
`The ipa-certupdate command failed, exception: DatabaseError: Connect
error: error:0A000086:SSL routines::certificate verify failed (unable to
get issuer certificate)` when trying to use `ipaldap.py`.
It is a mess, that's why I think it is better to have a way to simply
re-bootstrap all certificates.
There isn't really a good way to start over.
rob
On 2023/09/22 15:57, Florence Blanc-Renaud wrote:
> Hi,
>
> On Fri, Sep 22, 2023 at 12:36 PM Cristian Le <fedora(a)lecris.me> wrote:
>
> Hi Florence,
>
> Thanks for the feedback, let me clarify the situation on the
> certificates:
> - External CA is still valid and it is a self-signed certificate
> that we use for other services. So we can manually sign any
> service certificates to get them back up and running
> - IPA CA is expired, let's say Aug/10
> - I have managed to import a renewed IPA CA and ran `ipa-cert-fix`
> (and also seemed to have run `ipa-certupdate`) on a current date,
> let's say Sep/20. But not all services were recovered and now
> there is no overlap between earliest date in service certificates
> and the original IPA CA
>
> Which are the not-recovered services? Can you provide the output of
> "getcert list" at the current date?
> flo
>
> - I have run a backup, but also did some system upgrades to get
> the `ipa-cacert-manage prune` command, but when I've tried to
> recover it, I've found that the backup was not there.
>
> > you can still find the original certificates in the LDAP
> database (below ou=certificateRepository,ou=ca,o=ipaca) but it
> requires a bit of searching. You would need to restore the expired
> certificates, go back in time and force the renewal.
>
> I suspect we cannot do this all within LDAP right? If we get back
> the expired certificates, how do we restore them in each service?
> `httpd` is straightforward, and I guess `nssdb` should be doable,
> assuming the same key is used, but is there another database type
> where the certificates are located? Are all the certificates
> tracked by `getcert list`? Is it safe to assume that after running
> something like `ipa-cert-fix`, they are using the same private key?
>
> Some symptoms in the current setup:
> - When we are forward in time, `pki-tomcatd` is able to run, but
> then I can't do any `ipa-cert-fix` or `ipa-cacert-manage renew`.
> From what I've read, all of these commands (or at least
> `ipa-cacert-manage renew`) must be done backwards in time.
> - When we are backwards in time `pki-tomcatd` is unable to run,
> failing to access `:8080/ca/admin/ca/getStatus`. This then blocks
> various other services to be run. But about `ipactl restart`, only
> `pki-tomcatd` service is actually failing (and ipa service itself
> of course).
>
> I have navigated to `ou=certificateRepository,ou=ca,o=ipaca` and
> indeed there are still a bunch of certificates there in linear
> order. What are the services I should look for in there? I am
> using Apache Directory Studio and I can download the
> `userCertificate`. Should I just run `certutil -A` with those
> values with corresponding `subjectName`?
>
> BTW, I want to document this process on the website, should I make
> a PR on the github repo or is there somewhere else?
>
> Kind regards,
> Cristian
>
> On 2023/09/22 9:00, Florence Blanc-Renaud wrote:
>> Hi,
>>
>> On Thu, Sep 21, 2023 at 5:04 PM Cristian Le via FreeIPA-users
>> <freeipa-users(a)lists.fedorahosted.org> wrote:
>>
>> I have tried my luck around with all the helpers: `pki-server
>> cert-fix`, `ipa-cacert-manage`, `ipa-certupdate`, etc. but
>> each one is failing on me for multiple reasons.
>> - `ipa-cacert-manage` Cannot update the CA with
>> `--external-cert-file` because the root ca is not detected to
>> be in the trust list
>>
>> This command is useful if you need to trust a new external CA or
>> renew IPA CA. Is your IPA CA expired?
>>
>> - `ipa-cert-fix` Was run without overlapping validity time,
>> and the certificate were re-created, so now it is not
>> recoverable, neither back in time, nor in current time
>>
>> It is recommended to do a backup before running ipa-cert-fix. If
>> you didn't, and want to try the back-in-time method, you can
>> still find the original certificates in the LDAP database (below
>> ou=certificateRepository,ou=ca,o=ipaca) but it requires a bit of
>> searching. You would need to restore the expired certificates, go
>> back in time and force the renewal.
>>
>> - `pki-tomcat` is failing
>>
>> What is the current situation? Which certs are expired (getcert
>> list)? If you start the services with "ipactl start
>> --ignore-service-failures", is pki-tomcat the only service failing?
>> flo
>>
>> It is quite a mess and I would like to ask for some guidance
>> on how one could recover manually from such dependency issues:
>> - Is it possible to do a `ipa-server-install` and keep the
>> user data?
>>
>> - If I sign all of the service's certificates manually, what
>> are all of the manual steps needed to get the services back
>> up so that the helpers can be run.
>> - I've tried to install the CA certificate in the nssdb
>> database, ldap, and /etc/ipa/ca.crt. Are there other locations?
>> - I've recreated an httpd certificate signed by the root,
>> but I can't figure how to do the same with the ones located
>> in the nssdb database, i.e. to recreate a csr with the same
>> data as one of the certificates there
>> - What is the order of services that should be updated. My
>> understanding is CA -> `certutil`'s CA -> httpd + slapd +
>> pki-tomcat (not sure where the last one is or how to edit it)
>> -> `ipa-certupdate`
>> _______________________________________________
>> FreeIPA-users mailing list --
>> freeipa-users(a)lists.fedorahosted.org
>> To unsubscribe send an email to
>> freeipa-users-leave(a)lists.fedorahosted.org
>> Fedora Code of Conduct:
>> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>> List Guidelines:
>> https://fedoraproject.org/wiki/Mailing_list_guidelines
>> List Archives:
>>
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
>> Do not reply to spam, report it:
>> https://pagure.io/fedora-infrastructure/new_issue
>>
_______________________________________________
FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
4:43 a.m.
If tomcat fails to start then you need to stop and figure out why.
Any further poking at certificates won't yield anything useful without a
working CA.
So at all times I should make sure `pki-tomcatd` is working. Well it is
often failing with:
`ipa-pki-wait-running: Request failed unexpectedly, 404 Client Error:
for url: http://ipa1.my.lab:8080/ca/admin/ca/getStatus`
Presumably this is meant to be fixed by `ipa-cacert-manage renew`, but
as indicated it fails with `ERROR: CA
certificate chain in is incomplete`, specifically the traceback:
```
ipapython.admintool: DEBUG: File
"/usr/lib/python3.11/site-packages/ipapython/admintool.py", line 180, in
execute
return_value = self.run()
^^^^^^^^^^
File
"/usr/lib/python3.11/site-packages/ipaserver/install/ipa_cacert_manage.py",
line 138, in run
return self.renew()
^^^^^^^^^^^^
File
"/usr/lib/python3.11/site-packages/ipaserver/install/ipa_cacert_manage.py",
line 201, in renew
return self.renew_external_step_2(ca, cert)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/lib/python3.11/site-packages/ipaserver/install/ipa_cacert_manage.py",
line 277, in renew_external_step_2
cert_file, ca_file = installutils.load_external_cert(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/lib/python3.11/site-packages/ipaserver/install/installutils.py",
line 1194, in load_external_cert
raise ScriptError(
```
(disregard the slightly offset line numbers, I've added some debugging
to confirm that in the loop at [1] the trust chain has indeed only 1
item and it is not self-signed).
One thing suspicious is that the root CA is not set to `CT,C,C`:
```
ipapython.ipautil: DEBUG: args=['/usr/bin/certutil', '-d',
'sql:/tmp/tmprpz4cur1', '-M', '-n', 'CN=FreeIPA Root
Certificate
Authority,O=My Lab', '-t', 'C,,', '-f',
'/tmp/tmprpz4cur1/pwdfile.txt']
ipapython.ipautil: DEBUG: args=['/usr/bin/certutil', '-d',
'sql:/tmp/tmprpz4cur1', '-M', '-n', 'CN=My Lab Root CA,O=My
Lab', '-t',
'C,,', '-f', '/tmp/tmprpz4cur1/pwdfile.txt']
```
Do you think it is worth delving more into this issue and finding a way
to recover `ipa-cacert-manage renew`?
Manually add how?
Using `certutil -A -d` on the `nssdb` and individual files like in
`/etc/httpd`. I get the locations as reported in `getcert list`.
The KDC isn't running.
The KDC is running, but the `kdc.crt` certificates are busted.
[1]
https://pagure.io/freeipa/blob/master/f/ipaserver/install/installutils.py...
On 2023/09/25 21:06, Rob Crittenden wrote:
> Cristian Le via FreeIPA-users wrote:
>> Ok, let me walk through some of the specific errors, and I will also
>> censor out some of the output since this is going to the public
>> mail-list as well.
>>
>> Starting from the beginning.
>> - I have set the date to `1 month` before certificate expired with `sudo
>> date`
>> - I ran `ipactl restart --force` and `pki-tomcatd` failed
> If tomcat fails to start then you need to stop and figure out why. Any
> further poking at certificates won't yield anything useful without a
> working CA.
>
>> - I try to run `ipa-cacert-manage renew` with `--external-ca` and
>> `--external-cert-file`, with the CSR signed with `openssl x509 -req
>> -copy_extensions=copyall`, but this one failed because of `ERROR: CA
>> certificate chain in is incomplete: missing certificate with subject`,
>> even though I passed both the root and new CA
> What failed?
>
>> Then I have tried to manually add the new CA, but then, that did not
>> work. So moving on to the next layer, when I run to `sudo getcert list`,
>> the certificates I get:
> Manually add how?
>
>> ```
>> Request ID '20220810070559':
>> status: CA_UNREACHABLE
>> ca-error: Error setting up ccache for "host" service on
client
>> using default keytab: Cannot contact any KDC for requested realm.
> The KDC isn't running.
>
>> stuck: no
>> key pair storage:
type=FILE,location='/var/kerberos/krb5kdc/kdc.key'
>> certificate: type=FILE,location='/var/kerberos/krb5kdc/kdc.crt'
>> CA: IPA
>> issuer: CN=FreeIPA Root Certificate Authority,O=My Lab
>> subject: CN=ipa1.my.lab,O=My Lab
>> issued: 2023-09-20 14:21:40 UTC
>> expires: 2025-09-20 14:21:40 UTC
>> Request ID '20230920203122':
>> status: MONITORING
>> stuck: no
>> key pair storage:
>>
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
>> cert-pki-ca',token='NSS Certificate DB',pin set
>> certificate:
>>
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
>> cert-pki-ca',token='NSS Certificate DB'
>> CA: dogtag-ipa-ca-renew-agent
>> issuer: CN=FreeIPA Root Certificate Authority,O=My Lab
>> subject: CN=CA Audit,O=My Lab
>> issued: 2023-09-20 14:21:35 UTC
>> expires: 2025-09-09 14:21:35 UTC
>> Request ID '20230920203123':
>> status: MONITORING
>> stuck: no
>> key pair storage:
>>
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
cert-pki-ca',token='NSS
>> Certificate DB',pin set
>> certificate:
>>
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
cert-pki-ca',token='NSS
>> Certificate DB'
>> CA: dogtag-ipa-ca-renew-agent
>> issuer: CN=FreeIPA Root Certificate Authority,O=My Lab
>> subject: CN=OCSP Subsystem,O=My Lab
>> issued: 2023-09-20 14:21:33 UTC
>> expires: 2025-09-09 14:21:33 UTC
>> Request ID '20230920203125':
>> status: MONITORING
>> stuck: no
>> key pair storage:
>>
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
>> cert-pki-ca',token='NSS Certificate DB',pin set
>> certificate:
>>
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
>> cert-pki-ca',token='NSS Certificate DB'
>> CA: dogtag-ipa-ca-renew-agent
>> issuer: CN=FreeIPA Root Certificate Authority,O=My Lab
>> subject: CN=CA Subsystem,O=My Lab
>> issued: 2023-09-20 14:21:31 UTC
>> expires: 2025-09-09 14:21:31 UTC
>> Request ID '20230920203128':
>> status: MONITORING
>> stuck: no
>> key pair storage:
>>
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
>> cert-pki-ca',token='NSS Certificate DB',pin set
>> certificate:
>>
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
>> cert-pki-ca',token='NSS Certificate DB'
>> CA: dogtag-ipa-ca-renew-agent
>> issuer: CN=FreeIPA Root Certificate Authority,O=My Lab
>> subject: CN=ipa1.my.lab,O=My Lab
>> issued: 2023-09-20 11:29:21 UTC
>> expires: 2023-12-20 11:29:21 UTC
>> Request ID '20230920203130':
>> status: MONITORING
>> stuck: no
>> key pair storage:
type=FILE,location='/var/lib/ipa/ra-agent.key'
>> certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem'
>> CA: dogtag-ipa-ca-renew-agent
>> issuer: CN=FreeIPA Root Certificate Authority,O=My Lab
>> subject: CN=IPA RA,O=My Lab
>> issued: 2023-09-20 14:21:36 UTC
>> expires: 2025-09-09 14:21:36 UTC
>> Request ID '20230920203131':
>> status: CA_UNREACHABLE
>> ca-error: Error setting up ccache for "host" service on
client
>> using default keytab: Cannot contact any KDC for requested realm.
>> stuck: no
>> key pair storage:
>>
type=NSSDB,location='/etc/dirsrv/slapd-MY-LAB',nickname='Server-Cert',token='NSS
>> Certificate DB',pinfile='/etc/dirsrv/slapd-MY-LAB/pwdfile.txt'
>> certificate:
>>
type=NSSDB,location='/etc/dirsrv/slapd-MY-LAB',nickname='Server-Cert',token='NSS
>> Certificate DB'
>> CA: IPA
>> issuer: CN=FreeIPA Root Certificate Authority,O=My Lab
>> subject: CN=ipa1.my.lab,O=My Lab
>> issued: 2023-09-20 14:21:39 UTC
>> expires: 2025-09-20 14:21:39 UTC
>> Request ID '20230920203134':
>> status: CA_UNREACHABLE
>> stuck: no
>> key pair storage:
>>
type=FILE,location='/var/lib/ipa/private/httpd.key',pinfile='/var/lib/ipa/passwds/ipa1.my.lab-443-RSA'
>> certificate: type=FILE,location='/var/lib/ipa/certs/httpd.crt'
>> CA: IPA
>> issuer: CN=My Lab Root CA,O=My Lab
>> subject: CN=ipa1.my.lab,O=My Lab
>> issued: 2023-07-10 04:31:16 UTC
>> expires: 2023-08-09 04:31:16 UTC
>> Request ID '20230510042436':
>> status: MONITORING
>> ca-error: Updated certificate not available
>> stuck: no
>> key pair storage:
>>
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
>> cert-pki-ca',token='NSS Certificate DB',pin set
>> certificate:
>>
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
>> cert-pki-ca',token='NSS Certificate DB'
>> CA: dogtag-ipa-ca-renew-agent
>> issuer: CN=My Lab Root CA,O=My Lab
>> subject: CN=FreeIPA Root Certificate Authority,O=My Lab
>> issued: 2022-08-10 04:29:31 UTC
>> expires: 2023-08-10 04:29:31 UTC
>> ```
>> Note that even the `MONITORING` ones are unusable due to the
>> non-overlapping date. So when I do something like `ipa-certupdate` I get:
>> `The ipa-certupdate command failed, exception: DatabaseError: Connect
>> error: error:0A000086:SSL routines::certificate verify failed (unable to
>> get issuer certificate)` when trying to use `ipaldap.py`.
>>
>> It is a mess, that's why I think it is better to have a way to simply
>> re-bootstrap all certificates.
> There isn't really a good way to start over.
>
> rob
>
>> On 2023/09/22 15:57, Florence Blanc-Renaud wrote:
>>> Hi,
>>>
>>> On Fri, Sep 22, 2023 at 12:36 PM Cristian Le <fedora(a)lecris.me> wrote:
>>>
>>> Hi Florence,
>>>
>>> Thanks for the feedback, let me clarify the situation on the
>>> certificates:
>>> - External CA is still valid and it is a self-signed certificate
>>> that we use for other services. So we can manually sign any
>>> service certificates to get them back up and running
>>> - IPA CA is expired, let's say Aug/10
>>> - I have managed to import a renewed IPA CA and ran `ipa-cert-fix`
>>> (and also seemed to have run `ipa-certupdate`) on a current date,
>>> let's say Sep/20. But not all services were recovered and now
>>> there is no overlap between earliest date in service certificates
>>> and the original IPA CA
>>>
>>> Which are the not-recovered services? Can you provide the output of
>>> "getcert list" at the current date?
>>> flo
>>>
>>> - I have run a backup, but also did some system upgrades to get
>>> the `ipa-cacert-manage prune` command, but when I've tried to
>>> recover it, I've found that the backup was not there.
>>>
>>> > you can still find the original certificates in the LDAP
>>> database (below ou=certificateRepository,ou=ca,o=ipaca) but it
>>> requires a bit of searching. You would need to restore the expired
>>> certificates, go back in time and force the renewal.
>>>
>>> I suspect we cannot do this all within LDAP right? If we get back
>>> the expired certificates, how do we restore them in each service?
>>> `httpd` is straightforward, and I guess `nssdb` should be doable,
>>> assuming the same key is used, but is there another database type
>>> where the certificates are located? Are all the certificates
>>> tracked by `getcert list`? Is it safe to assume that after running
>>> something like `ipa-cert-fix`, they are using the same private key?
>>>
>>> Some symptoms in the current setup:
>>> - When we are forward in time, `pki-tomcatd` is able to run, but
>>> then I can't do any `ipa-cert-fix` or `ipa-cacert-manage renew`.
>>> From what I've read, all of these commands (or at least
>>> `ipa-cacert-manage renew`) must be done backwards in time.
>>> - When we are backwards in time `pki-tomcatd` is unable to run,
>>> failing to access `:8080/ca/admin/ca/getStatus`. This then blocks
>>> various other services to be run. But about `ipactl restart`, only
>>> `pki-tomcatd` service is actually failing (and ipa service itself
>>> of course).
>>>
>>> I have navigated to `ou=certificateRepository,ou=ca,o=ipaca` and
>>> indeed there are still a bunch of certificates there in linear
>>> order. What are the services I should look for in there? I am
>>> using Apache Directory Studio and I can download the
>>> `userCertificate`. Should I just run `certutil -A` with those
>>> values with corresponding `subjectName`?
>>>
>>> BTW, I want to document this process on the website, should I make
>>> a PR on the github repo or is there somewhere else?
>>>
>>> Kind regards,
>>> Cristian
>>>
>>> On 2023/09/22 9:00, Florence Blanc-Renaud wrote:
>>>> Hi,
>>>>
>>>> On Thu, Sep 21, 2023 at 5:04 PM Cristian Le via FreeIPA-users
>>>> <freeipa-users(a)lists.fedorahosted.org> wrote:
>>>>
>>>> I have tried my luck around with all the helpers: `pki-server
>>>> cert-fix`, `ipa-cacert-manage`, `ipa-certupdate`, etc. but
>>>> each one is failing on me for multiple reasons.
>>>> - `ipa-cacert-manage` Cannot update the CA with
>>>> `--external-cert-file` because the root ca is not detected to
>>>> be in the trust list
>>>>
>>>> This command is useful if you need to trust a new external CA or
>>>> renew IPA CA. Is your IPA CA expired?
>>>>
>>>> - `ipa-cert-fix` Was run without overlapping validity time,
>>>> and the certificate were re-created, so now it is not
>>>> recoverable, neither back in time, nor in current time
>>>>
>>>> It is recommended to do a backup before running ipa-cert-fix. If
>>>> you didn't, and want to try the back-in-time method, you can
>>>> still find the original certificates in the LDAP database (below
>>>> ou=certificateRepository,ou=ca,o=ipaca) but it requires a bit of
>>>> searching. You would need to restore the expired certificates, go
>>>> back in time and force the renewal.
>>>>
>>>> - `pki-tomcat` is failing
>>>>
>>>> What is the current situation? Which certs are expired (getcert
>>>> list)? If you start the services with "ipactl start
>>>> --ignore-service-failures", is pki-tomcat the only service
failing?
>>>> flo
>>>>
>>>> It is quite a mess and I would like to ask for some guidance
>>>> on how one could recover manually from such dependency issues:
>>>> - Is it possible to do a `ipa-server-install` and keep the
>>>> user data?
>>>>
>>>> - If I sign all of the service's certificates manually,
what
>>>> are all of the manual steps needed to get the services back
>>>> up so that the helpers can be run.
>>>> - I've tried to install the CA certificate in the nssdb
>>>> database, ldap, and /etc/ipa/ca.crt. Are there other locations?
>>>> - I've recreated an httpd certificate signed by the root,
>>>> but I can't figure how to do the same with the ones located
>>>> in the nssdb database, i.e. to recreate a csr with the same
>>>> data as one of the certificates there
>>>> - What is the order of services that should be updated. My
>>>> understanding is CA -> `certutil`'s CA -> httpd +
slapd +
>>>> pki-tomcat (not sure where the last one is or how to edit it)
>>>> -> `ipa-certupdate`
>>>> _______________________________________________
>>>> FreeIPA-users mailing list --
>>>> freeipa-users(a)lists.fedorahosted.org
>>>> To unsubscribe send an email to
>>>> freeipa-users-leave(a)lists.fedorahosted.org
>>>> Fedora Code of Conduct:
>>>> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>>>> List Guidelines:
>>>> https://fedoraproject.org/wiki/Mailing_list_guidelines
>>>> List Archives:
>>>>
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
>>>> Do not reply to spam, report it:
>>>> https://pagure.io/fedora-infrastructure/new_issue
>>>>
>> _______________________________________________
>> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
>> To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
>> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>> List Archives:
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
>> Do not reply to spam, report it:
https://pagure.io/fedora-infrastructure/new_issue
>>
265
days inactive
270
days old
freeipa-users@lists.fedorahosted.org
6 comments
3 participants
participants (3)
-
Cristian Le -
Florence Blanc-Renaud -
Rob Crittenden