User X509 certificates

If user X509 credentials integrity is degraded, for example in course of an incident the users key and cert were accessible to unauthorized individuals, the following steps need to be taken:

(NOTE All communications have to done with RT-IR tickets (Investigations in the related Incident).)

Collection

The list of affected certificates, their DNs and serial numbers needs to be collected and triaged

  • identify the directories in which the certificates and associated private keys are located
  • certificates that are in a place without any associated private key files, are out of scope now. However, keep in mind that private keys are almost always in encrypted form, so if private key are found in the same directory and with similar modification time stamps as the certificates (within a 13 months time frame), these must be treated as 'associated private keys' to the certificate
  • If the filesystem has reliable atime  data, you can remove all certificates in directories where the atime  of the private key is outside of the incident compromise period (key not accessed during the incident). Otherwise, keep all certificates in the list of potentially compromised ones
  • Remove all certificated where the validUntil time is prior to the start of the incident. This will usually get rid of a large number of certificates that are 'left around'
  • For the remaining certificates, that are valid and where the private key was potentially accessed, collect: issuer DN, subject DN, and serial number.
    You may find https://software.nikhef.nl/experimental/tcstools/probecert useful, but beware not to mess with the private key atime  ...
  • If you find unencrypted private keys, flag these!

The resulting list of (subject, issuer, serial) tuples is the one used for the next steps. All other DNs can be discarded at this point.

Actions

  • collect all affected X509 Certificate DNs as per above.
  • add the DNs to the central suspension service at CERN (https://argus.cern.ch/).
  • For unencrypted private keys, send the certificate and the key to the CA for immediate revocation. You can also try to sign an email with the cert. You can find the CA by entering the subjectDN in https://www.eugridpma.org/showca
    You cannot do this for encrypted private keys, which anyway have an additional layer of protection in the form of the passphrase. The passphrase does no protect credentials on interactive sytems wherekeyloggers may have been active, so reliance just on encryption is insufficient.
  • unless there is suspicion that it is an insider attack where certificates have been used as the vector, the affected site should inform the end-users/certificate owners based on local account management data. The information should also include the certificate serial number. It is a certificate, not a subject DN, that will be revoked. The users should be asked to either (1) revoke their own certificate at their CA or (2) request from their organisational helpdesk to initiate revocation. For TCS ("/DC=org/DC=terena/DC=tcs/...") certificates, the users should tell their local institutiona help desk to forward the revocation request to their "RAO" (organisational RA) or "DRAO" (departmental RAO). If the incident is (also) managed by the home-organisational IT, they can themselves revoke these certficates in case of TCS or InCommon membership.
  • Inform the issuing CAs which DNs are affected. Note that the CA may not initiate revocation without proof of compromise of the private key. Which is why the user requesting it is at times quicker. Yu need to show proof of compromise in order to guard against 'denial of CA service'. Some CAs may revoke based on incident details if you share these details (which then should be under TLP:AMBER+STRICT in order for the CA to be able to act)
  • get information at which VOs these credentials were registered (https://operations-portal.egi.eu/vo/userTracking#).
  • inform VOs, ask the VO to check and report for activities (ex. job submission) where these DNs were used, in particular which RCs have received jobs using these identities.
  • If possible, filter out only those RCs that are affected
  • inform the RCs, RCs should investigate the activities related to the identities in question.
  • inform OSG about the possibly compromised credentials.

Recvery

  • Once revocation is done (you can verify in the CRL by certificate serial number - all IGTF CRLs are also at https://dl.igtf.net/certificates/ labelled by CANAME.r0.pem, use "openssl crl -noout -text -in  filename.r0.pem" to list), after ~ 24-36 hours the subjectDN can be removed from central suspension.


 

  • No labels