Skip to main content

 

 

Cisco Defense Orchestrator

Resolving Certificate Issues

CDO's Use of Certificates

CDO checks the validity of certificates when connecting to devices. Specifically, CDO requires that:

  1. The device uses a TLS version equal to or greater than 1.0.
  2. The certificate presented by the device is not expired, and its issuance date is in the past (i.e. it is already valid, not scheduled to become valid at a later date). 
  3. One of these conditions is true:
    • The device uses a self-signed certificate, and it is the same as the most recent one trusted by an authorized user.
    • The device uses a certificate signed by a trusted Certificate Authority (CA), and provides a certificate chain linking the presented leaf certificate to the relevant CA.

These are the ways CDO uses certificates differently than browsers::

  • In the case of self-signed certificates, CDO overrides the domain name check, instead checking that the certificate exactly matches the one trusted by an authorized user during device onboarding or reconnection.
  • CDO does not yet support internal CAs. There is currently no way to check a certificate signed by an internal CA.

It is possible to disable certificate checking for ASA devices on a per-device basis. When an ASA's certificate cannot be trusted by CDO, you will have the option of disabling certificate checking for that device. If you have attempted to disable certificate checking for the device and you are still unable to onboard it, it is likely that the IP address and port you specified for the device is incorrect or unreachable.  There is no way to disable certificate checking globally, or to disable certificate checking for a device with a supported certificate. There is no way to disable certificate checking for non-ASA devices.

When you disable certificate checking for a device, CDO will still use TLS to connect to the device, but it will not validate the certificate used to establish the connection. This means that a passive man-in-the-middle attacker will not be able to eavesdrop on the connection, but an active man-in-the-middle could intercept the connection by supplying CDO with an invalid certificate.

Identifying Certificate Issues

If you see the error "CDO cannot connect to the device using the certificate presented," on a version 6.2.2 ASA FirePOWER module or version 6.2.2 Firepower Threat Defense device, please see the instructions for how to replace the certificate on the device.

There are several reasons that CDO may not be able to onboard a device. When the UI shows a message that "CDO cannot connect to the device using the certificate presented," there is a problem with the certificate. When the UI does not show this message, the problem is more likely related to connectivity problems (the device is unreachable) or other network errors.

To determine why CDO rejects a given certificate, you can use the openssl command-line tool on the SDC host or another host that can reach the relevant device. Use the following command to create a file showing the certificates presented by the device:

openssl s_client -showcerts -connect <host>:<port> &> <filename>.txt

This command will start an interactive session, so you will need to use Ctrl-c to exit after a couple of seconds.

You should now have a file containing output like the following:

depth=2 C = US, O = GeoTrust Inc., CN = GeoTrust Global CA

verify return:1

depth=1 C = US, O = Google Inc, CN = Google Internet Authority G2

verify return:1

depth=0 C = US, ST = California, L = Mountain View, O = Google Inc, CN = *.google.com

verify return:1

CONNECTED(00000003)

---

Certificate chain

 0 s:/C=US/ST=California/L=Mountain View/O=Google Inc/CN=*.google.com

   i:/C=US/O=Google Inc/CN=Google Internet Authority G2

-----BEGIN CERTIFICATE-----

MIIH0DCCBrigAwIBAgIIUOMfH+8ftN8wDQYJKoZIhvcNAQELBQAwSTELMAkGA1UE

....lots of base64...

tzw9TylimhJpZcl4qihFVTgFM7rMU2VHulpJgA59gdbaO/Bf

-----END CERTIFICATE-----

 1 s:/C=US/O=Google Inc/CN=Google Internet Authority G2

   i:/C=US/O=GeoTrust Inc./CN=GeoTrust Global CA

-----BEGIN CERTIFICATE-----

MIID8DCCAtigAwIBAgIDAjqSMA0GCSqGSIb3DQEBCwUAMEIxCzAJBgNVBAYTAlVT

....lots of base64...

tzw9TylimhJpZcl4qihFVTgFM7rMU2VHulpJgA59gdbaO/Bf

-----END CERTIFICATE-----

 2 s:/C=US/O=GeoTrust Inc./CN=GeoTrust Global CA

   i:/C=US/O=Equifax/OU=Equifax Secure Certificate Authority

-----BEGIN CERTIFICATE-----

MIIDfTCCAuagAwIBAgIDErvmMA0GCSqGSIb3DQEBBQUAME4xCzAJBgNVBAYTAlVT

....lots of base64...

b8ravHNjkOR/ez4iyz0H7V84dJzjA1BOoa+Y7mHyhD8S

-----END CERTIFICATE-----

---

Server certificate

subject=/C=US/ST=California/L=Mountain View/O=Google Inc/CN=*.google.com

issuer=/C=US/O=Google Inc/CN=Google Internet Authority G2

---

No client certificate CA names sent

Peer signing digest: SHA512

Server Temp Key: ECDH, P-256, 256 bits

---

SSL handshake has read 4575 bytes and written 434 bytes

---

New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256

Server public key is 2048 bit

Secure Renegotiation IS supported

Compression: NONE

Expansion: NONE

No ALPN negotiated

SSL-Session:

    Protocol  : TLSv1.2

    Cipher    : ECDHE-RSA-AES128-GCM-SHA256

    Session-ID: 48F046F3360225D51BE3362B50CE4FE8DB6D6B80B871C2A6DD5461850C4CF5AB

    Session-ID-ctx:

    Master-Key: 9A9CCBAA4F5A25B95C37EF7C6870F8C5DD3755A9A7B4CCE4535190B793DEFF53F94203AB0A62F9F70B9099FBFEBAB1B6

    Key-Arg   : None

    PSK identity: None

    PSK identity hint: None

    SRP username: None

    TLS session ticket lifetime hint: 100800 (seconds)

    TLS session ticket:

    0000 - 7a eb 54 dd ac 48 7e 76-30 73 b2 97 95 40 5b de   z.T..H~v0s...@[.

    0010 - f3 53 bf c8 41 36 66 3e-5b 35 a3 03 85 6f 7d 0c   .S..A6f>[5...o}.

    0020 - 4b a6 90 6f 95 e2 ec 03-31 5b 08 ca 65 6f 8f a6   K..o....1[..eo..

    0030 - 71 3d c1 53 b1 29 41 fc-d3 cb 03 bc a4 a9 33 28   q=.S.)A.......3(

    0040 - f8 c8 6e 0a dc b3 e1 63-0e 8f f2 63 e6 64 0a 36   ..n....c...c.d.6

    0050 - 22 cb 00 3a 59 1d 8d b2-5c 21 be 02 52 28 45 9d   "..:Y...\!..R(E.

    0060 - 72 e3 84 23 b6 f0 e2 7c-8a a3 e8 00 2b fd 42 1d   r..#...|....+.B.

    0070 - 23 35 6d f7 7d 85 39 1c-ad cd 49 f1 fd dd 15 de   #5m.}.9...I.....

    0080 - f6 9c ff 5e 45 9c 7c eb-6b 85 78 b5 49 ea c4 45   ...^E.|.k.x.I..E

    0090 - 6e 02 24 1b 45 fc 41 a2-87 dd 17 4a 04 36 e6 63   n.$.E.A....J.6.c

    00a0 - 72 a4 ad                                          r..

    00a4 - <SPACES/NULS>

 

    Start Time: 1476476711

    Timeout   : 300 (sec)

    Verify return code: 0 (ok)

---

The first thing to note in this output is the last line, where you see the Verify return code. If there is a certificate issue, the return code will be non-zero and there will be a description of the error. See below for explanations of common errors and how to remediate them.

Verify return code: 21 (unable to verify the first certificate)
Description

This error indicates that there is a problem with the certificate chain, and openssl cannot verify that the certificate presented by the device should be trusted. Let's look at the certificate chain from the example above to see how certificate chains should work:

---

Certificate chain

 0 s:/C=US/ST=California/L=Mountain View/O=Google Inc/CN=*.google.com

   i:/C=US/O=Google Inc/CN=Google Internet Authority G2

-----BEGIN CERTIFICATE-----

MIIH0DCCBrigAwIBAgIIUOMfH+8ftN8wDQYJKoZIhvcNAQELBQAwSTELMAkGA1UE

....lots of base64...

tzw9TylimhJpZcl4qihFVTgFM7rMU2VHulpJgA59gdbaO/Bf

-----END CERTIFICATE-----

 1 s:/C=US/O=Google Inc/CN=Google Internet Authority G2

   i:/C=US/O=GeoTrust Inc./CN=GeoTrust Global CA

-----BEGIN CERTIFICATE-----

MIID8DCCAtigAwIBAgIDAjqSMA0GCSqGSIb3DQEBCwUAMEIxCzAJBgNVBAYTAlVT

....lots of base64...

tzw9TylimhJpZcl4qihFVTgFM7rMU2VHulpJgA59gdbaO/Bf

-----END CERTIFICATE-----

 2 s:/C=US/O=GeoTrust Inc./CN=GeoTrust Global CA

   i:/C=US/O=Equifax/OU=Equifax Secure Certificate Authority

-----BEGIN CERTIFICATE-----

MIIDfTCCAuagAwIBAgIDErvmMA0GCSqGSIb3DQEBBQUAME4xCzAJBgNVBAYTAlVT

....lots of base64...

b8ravHNjkOR/ez4iyz0H7V84dJzjA1BOoa+Y7mHyhD8S

-----END CERTIFICATE-----

---


The certificate chain is a list of certificates presented by the server, beginning with the server's own certificate and then including increasingly higher-level intermediate certificates linking the server's certificate with a Certificate Authority's top-level certificate. Each certificate lists its Subject (the line starting with 's:' and its Issuer (the line starting with 'i').

The Subject is the entity identified by the certificate. It includes the Organization name and sometimes the Common Name of the entity for which the certificate was issued.

The Issuer is the entity that issued the certificate. It also includes an Organization field and sometimes a Common Name.

If a server had a certificate issued directly by a trusted Certificate Authority, it would not need to include any other certificates in its certificate chain. It would present one certificate that looked like:

---

Certificate chain

 0 s:/C=US/ST=California/L=Anytown/O=ExampleCo/CN=*.example.com

   i:/C=US/O=Trusted Authority/CN=Trusted Authority

-----BEGIN CERTIFICATE-----

MIIH0DCCBrigAwIBAgIIUOMfH+8ftN8wDQYJKoZIhvcNAQELBQAwSTELMAkGA1UE

....lots of base64...

tzw9TylimhJpZcl4qihFVTgFM7rMU2VHulpJgA59gdbaO/Bf

-----END CERTIFICATE-----

---

Given this certificate, openssl would verify that the ExampleCo certificate for *.example.com was correctly signed by the Trusted Authority certificate, which would be present in openssl's built-in trust store. After that verification, openssl would successfully connect to the device.

However, most servers do not have certificates signed directly by a trusted CA. Instead, as in the first example, the server's certificate is signed by one or more intermediates, and the highest-level intermediate has a certificate signed by the trusted CA. OpenSSL does not trust these intermediate CAs by default, and can only verify them if it is given a complete certificate chain ending in a trusted CA.

It is critically important that servers whose certificates are signed by intermediate authorities supply ALL the certificates linking them to a trusted CA, including all of the intermediate certificates. If they don't supply this entire chain, the output from openssl will look something like this:

depth=0 OU = Example Unit, CN = example.com

verify error:num=20:unable to get local issuer certificate

verify return:1

depth=0 OU = Example Unit, CN = example.com

verify error:num=27:certificate not trusted

verify return:1

depth=0 OU = Example Unit, CN = example.com

verify error:num=21:unable to verify the first certificate

verify return:1

CONNECTED(00000003)

---

Certificate chain

 0 s:/OU=Example Unit/CN=example.com

   i:/C=US/ST=Massachusetts/L=Cambridge/O=Intermediate Authority/OU=http://certificates.intermediateauth...N=Intermediate Certification Authority/sn=675637734

-----BEGIN CERTIFICATE-----

...lots of b64...

-----END CERTIFICATE-----

---

Server certificate

subject=/OU=Example Unit/CN=example.com

issuer=/C=US/ST=Massachusetts/L=Cambridge/O=Intermediate Authority/OU=http://certificates.intermediateauth...N=Intermediate Certification Authority/sn=675637734

---

No client certificate CA names sent

---

SSL handshake has read 1509 bytes and written 573 bytes

---

New, TLSv1/SSLv3, Cipher is AES256-SHA

Server public key is 2048 bit

Secure Renegotiation IS NOT supported

Compression: NONE

Expansion: NONE

SSL-Session:

    Protocol  : TLSv1

    Cipher    : AES256-SHA

    Session-ID: 24B45B2D5492A6C5D2D5AC470E42896F9D2DDDD54EF6E3363B7FDA28AB32414B

    Session-ID-ctx:

    Master-Key: 21BAF9D2E1525A5B935BF107DA3CAF691C1E499286CBEA987F64AE5F603AAF8E65999BD21B06B116FE9968FB7C62EF7C

    Key-Arg   : None

    Krb5 Principal: None

    PSK identity: None

    PSK identity hint: None

    Start Time: 1476711760

    Timeout   : 300 (sec)

    Verify return code: 21 (unable to verify the first certificate)

---


This output shows that the server only provided one certificate, and the provided certificate was signed by an intermediate authority, not a trusted root. The output also shows the characteristic verification errors.

Remediation

This problem is caused by a misconfigured certificate presented by the device. The only way to fix this so that CDO or any other program can securely connect to the device is to load the correct certificate chain onto the device, so that it will present a complete certificate chain to connecting clients.

Verify return code: 9 (certificate is not yet valid)
Description

This error means that the issuance date of the certificate provided is in the future, so clients will not treat it as valid. This can be caused by a poorly-constructed certificate, or in the case of a self-signed certificate it can be cause by the device time being wrong when it generated the certificate.

You should see a line in the error including the notBefore date of the certificate:

depth=0 CN = ASA Temporary Self Signed Certificate

verify error:num=18:self signed certificate

verify return:1

depth=0 CN = ASA Temporary Self Signed Certificate

verify error:num=9:certificate is not yet valid

notBefore=Oct 21 19:43:15 2016 GMT

verify return:1

depth=0 CN = ASA Temporary Self Signed Certificate

notBefore=Oct 21 19:43:15 2016 GMT

From this error, you can determine when the certificate will become valid. 

Remediation

The notBefore date of the  certificate needs to be in the past. You can reissue the certificate with an earlier notBefore date. This issue can also arise when the time is not set correctly either on the client or issuing device.

Verify return code: 10 (certificate has expired)
Description

This error means that at least one of the certificates provided has expired. You should see a line in the error including the notBefore date of the certificate:

error 10 at 0 depth lookup:certificate has expired

The expiration date is located in the certificate body.

Remediation

If the certificate is truly expired, the only remediation is to get another certificate. If the certificate's expiration is still in the future, but openssl claims that it is expired, check the time and date on your computer. For instance, if a certificate is set to expire in the year 2020, but the date on your computer is in 2021, your computer will treat that certificate as expired.