I've got two ancient (3.1?) IPA servers that have been upgraded over time. Last January things got really goofy with certificates and I got it all sorted. However, now I've got an old issue creeping back in. The 'transportCert cert-pki-kra' is mismatched between the CS.cfg and the tracked certificate. This is a multi-master setup. The signing master seems to be the one that's off. It's tracking the updated original 'transportCert cert-pki-kra' certificate. However, the "secondary" master is tracking a newly generated 'transportCert cert-pki-kra', which is also what both CS.cfg's are referencing. Neither one of the certificates is expired. Everything else seems to be in working order. Here is ipa-healthcheck's only relevant error:
"source": "ipahealthcheck.dogtag.ca", "kw": { "msg": "Certificate 'transportCert cert-pki-kra' does not match the value of ca.connector.KRA.transportCert in /var/lib/pki/pki-tomcat/conf/ca/CS.cfg", "configfile": "/var/lib/pki/pki-tomcat/conf/ca/CS.cfg", "directive": "ca.connector.KRA.transportCert", "key": "transportCert cert-pki-kra" },
So, what should I copy where to get this sorted? It seems like the updated original 'transportCert cert-pki-kra' should be copied into the CS.cfg and then manually scp the NSS files from "primary" to "secondary"? What commands would you use to do this? I've got a lot of commands noted and am beginning to get confused as to which ones should be used to get this sorted. Thanks.
Hi,
Certmonger can provide information related to the certificates it's tracking (stored in a file or in an NSS database). In your case, the certificate nickname is "transportCert cert-pki-kra", and to know where it's stored you can run the following command:
# getcert list -n 'transportCert cert-pki-kra' Number of certificates and requests being tracked: 12. Request ID '20220201080534': status: MONITORING stuck: no key pair storage: type=NSSDB,*location='/etc/pki/pki-tomcat/alias'*,nickname='transportCert cert-pki-kra',token='NSS Certificate DB',pin set certificate: type=NSSDB,*location='/etc/pki/pki-tomcat/alias'*,nickname='transportCert cert-pki-kra',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=IPA.TEST subject: CN=KRA Transport Certificate,O=IPA.TEST issued: 2022-02-01 08:04:27 UTC expires: 2024-01-22 08:04:27 UTC key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-clientAuth profile: caTransportCert pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "transportCert cert-pki-kra" track: yes auto-renew: yes
The above command shows that the certificate and the key are stored in /etc/pki/pki-tomcat/alias, which is a NSS database. In order to see the certificate details, you can use certutil command (-L for displaying the cert, -d for the NSS DB path, -n for the cert nickname): # certutil -L -d /etc/pki/pki-tomcat/alias/ -n 'transportCert cert-pki-kra' Certificate: Data: Version: 3 (0x2) Serial Number: 11 (0xb) Signature Algorithm: PKCS #1 SHA-256 With RSA Encryption Issuer: "CN=Certificate Authority,O=IPA.TEST" Validity: Not Before: Tue Feb 01 08:04:27 2022 Not After : Mon Jan 22 08:04:27 2024 Subject: "CN=KRA Transport Certificate,O=IPA.TEST" [...]
If you want to see the output printed in ASCII format, simply add the -a option: # certutil -L -d /etc/pki/pki-tomcat/alias/ -n 'transportCert cert-pki-kra' -a -----BEGIN CERTIFICATE----- MIID6zCCAlOgAwIBAgIBCzANBgkqhkiG9w0BAQsFADAzMREwDwYDVQQKDAhJUEEu [...] KLVH0hPJY7vzphBJtKtPTuEjyxYLrU9eKHNe8e7XPBd8/nA2qDAYS08eLIHBlek= -----END CERTIFICATE-----
The ipa-healthcheck command compares the certificate with the value stored in /var/lib/pki/pki-tomcat/conf/ca/CS.cfg in the directive ca.connector.KRA.transportCert. So you need to figure out which cert you want to keep and then make everything consistent (both servers should use the same 'transportCert cert-pki-kra'). How to figure out which one? Well, if both are valid, not revoked, any choice would work, you can pick the one with the furthest expiration date. If you need to manipulate the certs in the NSSDB, use the certutil command (with -D to remove a cert, -A to add a cert). Always make a backup of the directory /etc/pki/pki-tomcat/alias first, this way any certificate can be recovered in case of issues. You will need to restart PKI after changing the cert, with # systemctl restart pki-tomcatd@pki-tomcat
HTH, flo
On Mon, Jan 31, 2022 at 10:15 PM GH via FreeIPA-users < freeipa-users@lists.fedorahosted.org> wrote:
I've got two ancient (3.1?) IPA servers that have been upgraded over time. Last January things got really goofy with certificates and I got it all sorted. However, now I've got an old issue creeping back in. The 'transportCert cert-pki-kra' is mismatched between the CS.cfg and the tracked certificate. This is a multi-master setup. The signing master seems to be the one that's off. It's tracking the updated original 'transportCert cert-pki-kra' certificate. However, the "secondary" master is tracking a newly generated 'transportCert cert-pki-kra', which is also what both CS.cfg's are referencing. Neither one of the certificates is expired. Everything else seems to be in working order. Here is ipa-healthcheck's only relevant error:
"source": "ipahealthcheck.dogtag.ca", "kw": { "msg": "Certificate 'transportCert cert-pki-kra' does not match thevalue of ca.connector.KRA.transportCert in /var/lib/pki/pki-tomcat/conf/ca/CS.cfg", "configfile": "/var/lib/pki/pki-tomcat/conf/ca/CS.cfg", "directive": "ca.connector.KRA.transportCert", "key": "transportCert cert-pki-kra" },
So, what should I copy where to get this sorted? It seems like the updated original 'transportCert cert-pki-kra' should be copied into the CS.cfg and then manually scp the NSS files from "primary" to "secondary"? What commands would you use to do this? I've got a lot of commands noted and am beginning to get confused as to which ones should be used to get this sorted. Thanks. _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste... Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Well, I've managed to goof something up. Copied the ASCII from the latest one, from "primary", to the CS.cfg file on both servers, copied the /etc/pki/pki-tomcat/alias directory from the "primary" to the "secondary" and restarted pki-tomcat on both servers. That all said it worked. However, restarting ipa on the "secondary" now dies at pki-tomcatd. Logs showed an error of "Enter password for Internal Key Storage Token" and then the dreaded repeating "WARNING: Exception processing realm com.netscape.cms.tomcat.ProxyRealm@14444612 background process" for five minutes until it fails. Ugh.
Okay, took a gamble and copied the old passwd.txt file back in on "secondary". No change. Then copied the old key3.db file back in and ... it started! Haven't copied secmod.db back into place. Should I? I guess I should have only copied the cert8.db file over?
GH via FreeIPA-users wrote:
Okay, took a gamble and copied the old passwd.txt file back in on "secondary". No change. Then copied the old key3.db file back in and ... it started! Haven't copied secmod.db back into place. Should I? I guess I should have only copied the cert8.db file over?
I wouldn't recommend manually copying anything between servers except as a last resort.
These certificates should be tracked and kept in sync by certmonger
One of your servers should be configured as the "renewal master". You can find out which one with:
# ipa config-show | grep renewal IPA CA renewal master: ipa.example.test
That is the CA that does the actual renewals of expiring certificates. The other servers get the updated certificates from this one.
So make sure that the most recent "renewed" certificate was done on the renewal master by certmonger otherwise it won't be distributed properly. How you have two valid certificates at the same time I don't know.
So I'd recommend trying to figure out what happened so we can prevent future problems and get things back in sync in the best and safest way.
rob
The best I could tell was an upgrade back in Dec. 2019/Jan. 2020. It seems like it was a move from NSS to SSL for a number of pieces? Anyways, I'd had Ipsilon configured on the same server, and that move didn't make things happy as there was a port overlap. (Unsupported configuration, I know.) Lots of reconfiguration and copying certs around to get it straightened out.
Right now, everything starts on both servers. However, on the "secondary" that is not the renewal master, there's a number of "certificate doesn't match the CS.cfg" errors. 'ocspSigningCert cert-pki-ca' 'subsystemCert cert-pki-ca' 'Server-Cert cert-pki-ca' 'auditSigningCert cert-pki-ca'
Along with a: "msg": "Incorrect NSS trust for Server-Cert cert-pki-ca. Got ,, expected u,u,u",
The "primary", which is the renewal master listed on both boxes, shows none of those errors. At one point, I had figured out how to "force sync" the certs, but I've since forgotten.
GH via FreeIPA-users wrote:
The best I could tell was an upgrade back in Dec. 2019/Jan. 2020. It seems like it was a move from NSS to SSL for a number of pieces? Anyways, I'd had Ipsilon configured on the same server, and that move didn't make things happy as there was a port overlap. (Unsupported configuration, I know.) Lots of reconfiguration and copying certs around to get it straightened out.
Right now, everything starts on both servers. However, on the "secondary" that is not the renewal master, there's a number of "certificate doesn't match the CS.cfg" errors. 'ocspSigningCert cert-pki-ca' 'subsystemCert cert-pki-ca' 'Server-Cert cert-pki-ca' 'auditSigningCert cert-pki-ca'
Along with a: "msg": "Incorrect NSS trust for Server-Cert cert-pki-ca. Got ,, expected u,u,u",
The "primary", which is the renewal master listed on both boxes, shows none of those errors. At one point, I had figured out how to "force sync" the certs, but I've since forgotten.
This means there is no associated private key with the certificate. The "Server-Cert cert-pki-ca" certificate is used by tomcat and is unique per installation. The others are common and need to be identical on all CAs.
What does getcert list show?
rob
status: MONITORING stuck: no key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent ... expires: 2023-08-08 12:55:26 UTC key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-emailProtection pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "Server-Cert cert-pki-ca" track: yes auto-renew: yes
There are the expected 15 for this box, all seem to be monitoring and not stuck.
Because 'Server-Cert cert-pki-ca' is part of '/etc/pki/pki-tomcat/alias' and should be unique, it needs to be regenerated on this machine. Whether that regeneration is by pulling it from the backup of cert8.db or a new creation, I'm not exactly sure. Obviously, I'm also not versed well enough to export it out of the backed up cert8.db and then re-install it into the new copy of cert8.db. I suspect that it's three to four commands. One that exports it, one or two that massage the export into or out of pk12, and then an import.
Okay, so I found this: To export cert+key from a NSSDB to a p12file: pk12util -d /etc/pki/pki-tomcat/alias -n 'auditSigningCert cert-pki-ca' -o file.p12 To import cert.key from a p12 file to a NSSDB: pk12util -d /etc/pki/pki-tomcat/alias -i file.p12
From Flo's post at: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
That import line needs a -n 'certNameHere givenName' added to it. Otherwise, after massaging names and dirs, it seems to have worked for that portion. Now to just get the final three CS.cfg entries straightened out. That may just be a systemctl restart certmonger issue.
Had to copy the ASCII into the CS.cfg on the "secondary" manually. Now everything shows that it's happy from my untrained eye. Is there a way to test that the CS.cfg will now copy over correctly or that certs will be replicated correctly? Appreciate all of the help so far to get me to this point.
GH via FreeIPA-users wrote:
Had to copy the ASCII into the CS.cfg on the "secondary" manually. Now everything shows that it's happy from my untrained eye. Is there a way to test that the CS.cfg will now copy over correctly or that certs will be replicated correctly? Appreciate all of the help so far to get me to this point.
A clean bill of health from ipa-healthcheck is a decent start. It isn't perfect but it covers a lot of the common issues.
I'll add that renewal across all servers isn't an immediate thing. certmonger knows when the current one(s) will expire and by default will start looking for new ones with 28 days left and go by halves after that. So even if you manage to renew a CA cert on one side the others aren't going to bother looking for it for quite some time.
If you want to check on replication you can always issue a test cert and ensure it appears on all the other servers.
rob
Thanks Rob. Will do. I believe we can mark this as Solved then. Appreciate the help and effort you and Flo put into keeping FreeIPA a solid resource.
ARRRGGGHHHH!!! ’Server-Cert cert-pki-ca’ is missing again. Trying to recover it from the /etc/pki/pki-tomcat/alias directory via pk12util is not giving me the key, so that I can re-import it and get it trusted. The certutil -L command is showing a trust of ‘,,’, rather than ‘u,u,u’ because of the missing key. At this point, I think that I need to regenerate that certificate, import it, and then reset it to tracking the new one again. The piece I can’t seem to piece together is how to generate that certificate. (Yeah, it’s probably simple and I’m so deep in that I can’t see it.)
Thanks,
GH
On Feb 1, 2022, at 3:03 PM, Rob Crittenden <rcritten@redhat.commailto:rcritten@redhat.com> wrote:
GH via FreeIPA-users wrote: The best I could tell was an upgrade back in Dec. 2019/Jan. 2020. It seems like it was a move from NSS to SSL for a number of pieces? Anyways, I'd had Ipsilon configured on the same server, and that move didn't make things happy as there was a port overlap. (Unsupported configuration, I know.) Lots of reconfiguration and copying certs around to get it straightened out.
Right now, everything starts on both servers. However, on the "secondary" that is not the renewal master, there's a number of "certificate doesn't match the CS.cfg" errors. 'ocspSigningCert cert-pki-ca' 'subsystemCert cert-pki-ca' 'Server-Cert cert-pki-ca' 'auditSigningCert cert-pki-ca'
Along with a: "msg": "Incorrect NSS trust for Server-Cert cert-pki-ca. Got ,, expected u,u,u",
The "primary", which is the renewal master listed on both boxes, shows none of those errors. At one point, I had figured out how to "force sync" the certs, but I've since forgotten.
This means there is no associated private key with the certificate. The "Server-Cert cert-pki-ca" certificate is used by tomcat and is unique per installation. The others are common and need to be identical on all CAs.
What does getcert list show?
rob
Greg Harris wrote:
ARRRGGGHHHH!!! ’Server-Cert cert-pki-ca’ is missing again. Trying to recover it from the /etc/pki/pki-tomcat/alias directory via pk12util is not giving me the key, so that I can re-import it and get it trusted. The certutil -L command is showing a trust of ‘,,’, rather than ‘u,u,u’ because of the missing key. At this point, I think that I need to regenerate that certificate, import it, and then reset it to tracking the new one again. The piece I can’t seem to piece together is how to generate that certificate. (Yeah, it’s probably simple and I’m so deep in that I can’t see it.)
What version of IPA is this?
It is unusual for a key to disappear.
rob
Thanks,
GH
On Feb 1, 2022, at 3:03 PM, Rob Crittenden <rcritten@redhat.com mailto:rcritten@redhat.com> wrote:
GH via FreeIPA-users wrote:
The best I could tell was an upgrade back in Dec. 2019/Jan. 2020. It seems like it was a move from NSS to SSL for a number of pieces? Anyways, I'd had Ipsilon configured on the same server, and that move didn't make things happy as there was a port overlap. (Unsupported configuration, I know.) Lots of reconfiguration and copying certs around to get it straightened out.
Right now, everything starts on both servers. However, on the "secondary" that is not the renewal master, there's a number of "certificate doesn't match the CS.cfg" errors. 'ocspSigningCert cert-pki-ca' 'subsystemCert cert-pki-ca' 'Server-Cert cert-pki-ca' 'auditSigningCert cert-pki-ca'
Along with a: "msg": "Incorrect NSS trust for Server-Cert cert-pki-ca. Got ,, expected u,u,u",
The "primary", which is the renewal master listed on both boxes, shows none of those errors. At one point, I had figured out how to "force sync" the certs, but I've since forgotten.
This means there is no associated private key with the certificate. The "Server-Cert cert-pki-ca" certificate is used by tomcat and is unique per installation. The others are common and need to be identical on all CAs.
What does getcert list show?
rob
It’s 4.6.8-5.el7.centos.12.
Yes, it’s strange that it would disappear. I believe that it renewed the certificate, but may not have updated correctly. The first thing I found was that the certificate wasn’t there. I was able to restore the .crt from the CS.cfg file, but that of course doesn’t have the key.
I also could have inadvertently compounded problems with another series of commands. Attempting to move to el9, I then found that I had to go through el8 the hard way, though that went sideways also for a different reason. I wouldn’t expect that adding and removing servers during this time should have fouled up the cert database, but I’ve done weirder things to machines before.
Thanks,
Greg Harris
On Nov 22, 2022, at 4:08 PM, Rob Crittenden <rcritten@redhat.commailto:rcritten@redhat.com> wrote:
Greg Harris wrote: ARRRGGGHHHH!!! ’Server-Cert cert-pki-ca’ is missing again. Trying to recover it from the /etc/pki/pki-tomcat/alias directory via pk12util is not giving me the key, so that I can re-import it and get it trusted. The certutil -L command is showing a trust of ‘,,’, rather than ‘u,u,u’ because of the missing key. At this point, I think that I need to regenerate that certificate, import it, and then reset it to tracking the new one again. The piece I can’t seem to piece together is how to generate that certificate. (Yeah, it’s probably simple and I’m so deep in that I can’t see it.)
What version of IPA is this?
It is unusual for a key to disappear.
rob
Thanks,
GH
On Feb 1, 2022, at 3:03 PM, Rob Crittenden <rcritten@redhat.commailto:rcritten@redhat.com mailto:rcritten@redhat.com> wrote:
GH via FreeIPA-users wrote: The best I could tell was an upgrade back in Dec. 2019/Jan. 2020. It seems like it was a move from NSS to SSL for a number of pieces? Anyways, I'd had Ipsilon configured on the same server, and that move didn't make things happy as there was a port overlap. (Unsupported configuration, I know.) Lots of reconfiguration and copying certs around to get it straightened out.
Right now, everything starts on both servers. However, on the "secondary" that is not the renewal master, there's a number of "certificate doesn't match the CS.cfg" errors. 'ocspSigningCert cert-pki-ca' 'subsystemCert cert-pki-ca' 'Server-Cert cert-pki-ca' 'auditSigningCert cert-pki-ca'
Along with a: "msg": "Incorrect NSS trust for Server-Cert cert-pki-ca. Got ,, expected u,u,u",
The "primary", which is the renewal master listed on both boxes, shows none of those errors. At one point, I had figured out how to "force sync" the certs, but I've since forgotten.
This means there is no associated private key with the certificate. The "Server-Cert cert-pki-ca" certificate is used by tomcat and is unique per installation. The others are common and need to be identical on all CAs.
What does getcert list show?
rob
I just discovered that ipa-certupdate is removing the 'Server-Cert cert-pki-ca’ from 'certutil -L -d /etc/pki/pki-tomcat/alias/‘ when the trust flags aren’t correct. However, the new cert is still in 'getcert list’ as monitoring.
I did a 'ipa-getcert request -d /etc/pki/pki-tomcat/alias -n 'Server-Cert cert-pki-ca' -r -P $thePin’ after stop-tracking the old one. It did create it as above, but the old one was still in certutil when I did that.
Thanks,
Greg Harris
On Nov 22, 2022, at 4:17 PM, Greg Harris <gharris@teamexpansion.orgmailto:gharris@teamexpansion.org> wrote:
It’s 4.6.8-5.el7.centos.12.
Yes, it’s strange that it would disappear. I believe that it renewed the certificate, but may not have updated correctly. The first thing I found was that the certificate wasn’t there. I was able to restore the .crt from the CS.cfg file, but that of course doesn’t have the key.
I also could have inadvertently compounded problems with another series of commands. Attempting to move to el9, I then found that I had to go through el8 the hard way, though that went sideways also for a different reason. I wouldn’t expect that adding and removing servers during this time should have fouled up the cert database, but I’ve done weirder things to machines before.
Thanks,
Greg Harris
On Nov 22, 2022, at 4:08 PM, Rob Crittenden <rcritten@redhat.commailto:rcritten@redhat.com> wrote:
Greg Harris wrote: ARRRGGGHHHH!!! ’Server-Cert cert-pki-ca’ is missing again. Trying to recover it from the /etc/pki/pki-tomcat/alias directory via pk12util is not giving me the key, so that I can re-import it and get it trusted. The certutil -L command is showing a trust of ‘,,’, rather than ‘u,u,u’ because of the missing key. At this point, I think that I need to regenerate that certificate, import it, and then reset it to tracking the new one again. The piece I can’t seem to piece together is how to generate that certificate. (Yeah, it’s probably simple and I’m so deep in that I can’t see it.)
What version of IPA is this?
It is unusual for a key to disappear.
rob
Thanks,
GH
On Feb 1, 2022, at 3:03 PM, Rob Crittenden <rcritten@redhat.commailto:rcritten@redhat.com mailto:rcritten@redhat.com> wrote:
GH via FreeIPA-users wrote: The best I could tell was an upgrade back in Dec. 2019/Jan. 2020. It seems like it was a move from NSS to SSL for a number of pieces? Anyways, I'd had Ipsilon configured on the same server, and that move didn't make things happy as there was a port overlap. (Unsupported configuration, I know.) Lots of reconfiguration and copying certs around to get it straightened out.
Right now, everything starts on both servers. However, on the "secondary" that is not the renewal master, there's a number of "certificate doesn't match the CS.cfg" errors. 'ocspSigningCert cert-pki-ca' 'subsystemCert cert-pki-ca' 'Server-Cert cert-pki-ca' 'auditSigningCert cert-pki-ca'
Along with a: "msg": "Incorrect NSS trust for Server-Cert cert-pki-ca. Got ,, expected u,u,u",
The "primary", which is the renewal master listed on both boxes, shows none of those errors. At one point, I had figured out how to "force sync" the certs, but I've since forgotten.
This means there is no associated private key with the certificate. The "Server-Cert cert-pki-ca" certificate is used by tomcat and is unique per installation. The others are common and need to be identical on all CAs.
What does getcert list show?
rob
Got it! A ‘ipa-getcert resubmit -I $Serial’ did it. It’s now showing in the certutil as trusted. Now to see if it will ipa-server-upgrade correctly. Thanks!
Thanks,
Greg Harris
On Nov 22, 2022, at 4:26 PM, Greg Harris <gharris@teamexpansion.orgmailto:gharris@teamexpansion.org> wrote:
I just discovered that ipa-certupdate is removing the 'Server-Cert cert-pki-ca’ from 'certutil -L -d /etc/pki/pki-tomcat/alias/‘ when the trust flags aren’t correct. However, the new cert is still in 'getcert list’ as monitoring.
I did a 'ipa-getcert request -d /etc/pki/pki-tomcat/alias -n 'Server-Cert cert-pki-ca' -r -P $thePin’ after stop-tracking the old one. It did create it as above, but the old one was still in certutil when I did that.
Thanks,
Greg Harris
On Nov 22, 2022, at 4:17 PM, Greg Harris <gharris@teamexpansion.orgmailto:gharris@teamexpansion.org> wrote:
It’s 4.6.8-5.el7.centos.12.
Yes, it’s strange that it would disappear. I believe that it renewed the certificate, but may not have updated correctly. The first thing I found was that the certificate wasn’t there. I was able to restore the .crt from the CS.cfg file, but that of course doesn’t have the key.
I also could have inadvertently compounded problems with another series of commands. Attempting to move to el9, I then found that I had to go through el8 the hard way, though that went sideways also for a different reason. I wouldn’t expect that adding and removing servers during this time should have fouled up the cert database, but I’ve done weirder things to machines before.
Thanks,
Greg Harris
On Nov 22, 2022, at 4:08 PM, Rob Crittenden <rcritten@redhat.commailto:rcritten@redhat.com> wrote:
Greg Harris wrote: ARRRGGGHHHH!!! ’Server-Cert cert-pki-ca’ is missing again. Trying to recover it from the /etc/pki/pki-tomcat/alias directory via pk12util is not giving me the key, so that I can re-import it and get it trusted. The certutil -L command is showing a trust of ‘,,’, rather than ‘u,u,u’ because of the missing key. At this point, I think that I need to regenerate that certificate, import it, and then reset it to tracking the new one again. The piece I can’t seem to piece together is how to generate that certificate. (Yeah, it’s probably simple and I’m so deep in that I can’t see it.)
What version of IPA is this?
It is unusual for a key to disappear.
rob
Thanks,
GH
On Feb 1, 2022, at 3:03 PM, Rob Crittenden <rcritten@redhat.commailto:rcritten@redhat.com mailto:rcritten@redhat.com> wrote:
GH via FreeIPA-users wrote: The best I could tell was an upgrade back in Dec. 2019/Jan. 2020. It seems like it was a move from NSS to SSL for a number of pieces? Anyways, I'd had Ipsilon configured on the same server, and that move didn't make things happy as there was a port overlap. (Unsupported configuration, I know.) Lots of reconfiguration and copying certs around to get it straightened out.
Right now, everything starts on both servers. However, on the "secondary" that is not the renewal master, there's a number of "certificate doesn't match the CS.cfg" errors. 'ocspSigningCert cert-pki-ca' 'subsystemCert cert-pki-ca' 'Server-Cert cert-pki-ca' 'auditSigningCert cert-pki-ca'
Along with a: "msg": "Incorrect NSS trust for Server-Cert cert-pki-ca. Got ,, expected u,u,u",
The "primary", which is the renewal master listed on both boxes, shows none of those errors. At one point, I had figured out how to "force sync" the certs, but I've since forgotten.
This means there is no associated private key with the certificate. The "Server-Cert cert-pki-ca" certificate is used by tomcat and is unique per installation. The others are common and need to be identical on all CAs.
What does getcert list show?
rob
I have an administrative user which hasn't logged into his account in some time - likely over a year. He can authenticate to any bound host, but cannot login to the FreeIPA servers. I verified this wasn’t an HABC issue.
I compared his account to my own and found he had an extra attribute - krblastadminunlock
grant@ef-idm01:~[20221123-4:41][#1003]$ ipa user-show --all waynev | grep krblastadminunlock krblastadminunlock: 20171006230951Z grant@ef-idm01:~[20221123-4:47][#1004]$ ipa user-show --all grant | grep krblastadminunlock grant@ef-idm01:~[20221123-4:47][#1005]$
I wasn’t able to find much on this, but did find this: https://github.com/freeipa/freeipa/commit/69b1a5fc04357d1771c527444e9ba06475...
How can I remove the krblastadminunlock attribute from this user without resetting the password?
thanx
- grant
Grant Janssen via FreeIPA-users wrote:
I have an administrative user which hasn't logged into his account in some time - likely over a year. He can authenticate to any bound host, but cannot login to the FreeIPA servers. I verified this wasn’t an HABC issue.
I compared his account to my own and found he had an extra attribute - krblastadminunlock
grant@ef-idm01:~[20221123-4:41][#1003]$ ipa user-show --all waynev | grep krblastadminunlock krblastadminunlock: 20171006230951Z grant@ef-idm01:~[20221123-4:47][#1004]$ ipa user-show --all grant | grep krblastadminunlock grant@ef-idm01:~[20221123-4:47][#1005]$I wasn’t able to find much on this, but did find this: https://github.com/freeipa/freeipa/commit/69b1a5fc04357d1771c527444e9ba06475...
How can I remove the krblastadminunlock attribute from this user without resetting the password?
This attribute allows authentications rather than restricting them. I don't think this is the root of the issue. Either way it wouldn't affect a per-machine authentication.
How did you rule out HBAC?
You might want to crank up sssd debugging and have this user try to log in. That may provide some guidance.
rob
On ke, 23 marras 2022, Grant Janssen via FreeIPA-users wrote:
I have an administrative user which hasn't logged into his account in some time - likely over a year. He can authenticate to any bound host, but cannot login to the FreeIPA servers. I verified this wasn’t an HABC issue.
I compared his account to my own and found he had an extra attribute - krblastadminunlock
grant@ef-idm01:~[20221123-4:41][#1003]$ ipa user-show --all waynev | grep krblastadminunlock krblastadminunlock: 20171006230951Z grant@ef-idm01:~[20221123-4:47][#1004]$ ipa user-show --all grant | grep krblastadminunlock grant@ef-idm01:~[20221123-4:47][#1005]$
I wasn’t able to find much on this, but did find this: https://github.com/freeipa/freeipa/commit/69b1a5fc04357d1771c527444e9ba06475...
How can I remove the krblastadminunlock attribute from this user without resetting the password?
Try this on the IPA server as root: # ipa -e in_server=true user-mod waynev --delattr=krblastadminunlock=20171006230951Z
Alexander
Thank You for your attention, but this did not work for me. I had tried earlier to remove this attribute in the conventional manner, but failed. (example again at the tail of my output)
[root@ef-idm01 ~]# ipa -e in_server=true user-mod waynev --delattr=krblastadminunlock=20171006230951Z ipa: ERROR: krblastadminunlock does not contain '20171006230951Z' [root@ef-idm01 ~]# exit logout grant@ef-idm01:~[20221123-6:59][#1012]$ klist Ticket cache: KEYRING:persistent:555:555 Default principal: grant@PRODUCTION.EFILM.COMmailto:grant@PRODUCTION.EFILM.COM
Valid starting Expires Service principal 11/23/2022 04:43:47 11/24/2022 04:43:34 HTTP/ef-idm01.production.efilm.com@PRODUCTION.EFILM.COMmailto:HTTP/ef-idm01.production.efilm.com@PRODUCTION.EFILM.COM 11/23/2022 04:43:37 11/24/2022 04:43:34 krbtgt/PRODUCTION.EFILM.COM@PRODUCTION.EFILM.COMmailto:krbtgt/PRODUCTION.EFILM.COM@PRODUCTION.EFILM.COM grant@ef-idm01:~[20221123-6:59][#1013]$ ipa user-mod --delattr=krblastadminunlock=20171006230951Z waynev ipa: ERROR: krblastadminunlock does not contain '20171006230951Z' grant@ef-idm01:~[20221123-6:59][#1014]$ ipa user-show --all waynev | grep krblastadminunlock krblastadminunlock: 20171006230951Z grant@ef-idm01:~[20221123-DING!][#1015]$
thanx
- grant
On ke, 23 marras 2022, Grant Janssen wrote:
Alexander
Thank You for your attention, but this did not work for me. I had tried earlier to remove this attribute in the conventional manner, but failed. (example again at the tail of my output)
[root@ef-idm01 ~]# ipa -e in_server=true user-mod waynev --delattr=krblastadminunlock=20171006230951Z ipa: ERROR: krblastadminunlock does not contain '20171006230951Z'
It says the attribute value is not there. Can you do ipa user-show --all --raw waynev |grep krblastadminunlock ?
This differs from your output below by not processing the attributes value's output. And then use that value to pass through to user-mod.
[root@ef-idm01 ~]# exit logout grant@ef-idm01:~[20221123-6:59][#1012]$ klist Ticket cache: KEYRING:persistent:555:555 Default principal: grant@PRODUCTION.EFILM.COMmailto:grant@PRODUCTION.EFILM.COM
Valid starting Expires Service principal 11/23/2022 04:43:47 11/24/2022 04:43:34 HTTP/ef-idm01.production.efilm.com@PRODUCTION.EFILM.COMmailto:HTTP/ef-idm01.production.efilm.com@PRODUCTION.EFILM.COM 11/23/2022 04:43:37 11/24/2022 04:43:34 krbtgt/PRODUCTION.EFILM.COM@PRODUCTION.EFILM.COMmailto:krbtgt/PRODUCTION.EFILM.COM@PRODUCTION.EFILM.COM grant@ef-idm01:~[20221123-6:59][#1013]$ ipa user-mod --delattr=krblastadminunlock=20171006230951Z waynev ipa: ERROR: krblastadminunlock does not contain '20171006230951Z' grant@ef-idm01:~[20221123-6:59][#1014]$ ipa user-show --all waynev | grep krblastadminunlock krblastadminunlock: 20171006230951Z grant@ef-idm01:~[20221123-DING!][#1015]$
thanx
- grant
I see a slight variation, but still cannot remove the attribute.
grant@ef-idm01:~[20221123-7:19][#1018]$ ipa user-show --all --raw waynev | grep krblastadminunlock grant@ef-idm01:~[20221123-7:20][#1019]$ ipa user-show --all --raw waynev | grep -i krblastadminunlock krbLastAdminUnlock: 20171006230951Z grant@ef-idm01:~[20221123-7:20][#1020]$ ipa user-mod --delattr=krbLastAdminUnlock=20171006230951Z waynev ipa: ERROR: krblastadminunlock does not contain '20171006230951Z' grant@ef-idm01:~[20221123-7:20][#1021]$
- grant
Grant Janssen via FreeIPA-users wrote:
I see a slight variation, but still cannot remove the attribute.
grant@ef-idm01:~[20221123-7:19][#1018]$ipa user-show --all --raw waynev | grep krblastadminunlock grant@ef-idm01:~[20221123-7:20][#1019]$ipa user-show --all --raw waynev | grep -i krblastadminunlock krbLastAdminUnlock: 20171006230951Z grant@ef-idm01:~[20221123-7:20][#1020]$ ipa user-mod --delattr=krbLastAdminUnlock=20171006230951Z waynev ipa: ERROR: krblastadminunlock does not contain '20171006230951Z' grant@ef-idm01:~[20221123-7:20][#1021]$
It's probably a difference between storage and representation. This is a case where ldapsearch is probably better to find the value.
Alternatively you can try deleting the entire attribute with:
--setattr krblastadminunlock=
But again, this would affect any authentication and not just IPA servers so it doesn't make sense that access is not universally allowed/denied.
rob
I was able to remove this by overwriting the attribute "ipa user-mod --setattr krblastadminunlock= waynev"
grant@ef-idm01:~[20221123-7:50][#1022]$ ipa user-show --all --raw waynev | grep -i krblastadminunlock grant@ef-idm01:~[20221123-7:51][#1023]$
I’ll have the user test and we’ll see if this resolves the 'no ssh login to IPA servers' issue for this user. If it’s a no, I will change his password.
thanx
- grant
krbLastAdminUnlock was only a part of my issue.
I was able to resolve this issue, but not in the manner I expected. A careless administrator overwrote the keytabs on two FreeIPA servers while he was generating keytabs for MacOS hosts. Somehow, FreeIPA still functioned, the only repercussion was that some users (but not all) were unable to ssh into the IPA servers. The syslog did log this as a keytab issue. I was able to recover the original keytabs with "ipa-getkeytab -r”
- grant
freeipa-users@lists.fedorahosted.org
-
Alexander Bokovoy -
Florence Blanc-Renaud -
GH -
Grant Janssen -
Greg Harris
-
Rob Crittenden