Re-deriving AES and 3DES attribute encryption keys for 389 Directory Server installation

1. Introduction

Even if you do not use attribute encryption keys in your 389 Directory Server (DS) installation, you might be bothered by all those messages that repeat themselves in /var/log/dirsrv/slapd-instance_name/error ("instance name" is your actual instance name), after every start/restart of the 389 DS daemon:

[04/Nov/2019:02:03:16.808096029 +0200] - ERR - attrcrypt_cipher_init - No symmetric key found for cipher AES in backend userRoot, attempting to create one...
[04/Nov/2019:02:03:16.843731964 +0200] - ERR - attrcrypt_wrap_key - Failed to wrap key for cipher AES
[04/Nov/2019:02:03:16.808096029 +0200] - ERR - attrcrypt_cipher_init - No symmetric key found for cipher 3DES in backend userRoot, attempting to create one...
[04/Nov/2019:02:03:16.843731964 +0200] - ERR - attrcrypt_wrap_key - Failed to wrap key for cipher 3DES

What causes their appearance and how to get rid of them (and to solve the problem they point to)?

There are two reasons for those message to be seen in /var/log/dirsrv/slapd-instance_name/error:

• the RSA-based X.509 certificate, used by the 389 DS server, was replaced with a new one and it has new encryption key (that situation might be fatal if your LDAP directory hosts encrypted attributes and the old RSA keys were lost);
• the 389 DS server uses Elliptic Curves based X.509 certificate.

First, some important remarks. While it is true that failing to access the attribute encryption keys is not a fatal event to most of the 389 DS installations, where upon the attribute encryption is not necessary and therefore - not truly implemented, the appearance of those error messages might indicate a disruption of some other services (especially those that frequently query the LDAP server). That means, you really have to investigate the source of those errors, if your goal is a reliable 389 DS installation.

2. The RSA-based X.509 certificate, used by the 389 DS server, was replaced with a new one and it has new encryption key

Go down to "Deriving new attribute encryption keys" to see how to fix the problem.

3. The 389 DS server uses Elliptic Curves (ECC) based X.509 certificate

389 Directory Server stores the keys, certificates, and passwords in Mozilla NSS DB storage. It is usually located inside the folder /etc/dirsrv/slapd-instance_name, where "instance_name" is the actual instance name of your 389 Directory Server setup (depends on how did you name the instance). The NSS DB store is split into three files: cert8.db, keys3.db, and secmod.db.

If the 389 Directory Server is configured to use SSL (and STARTTLS), a X.509v3 certificate is installed in the NSS DB store. The problem is that by default (so far), the administrative server sets any X.509 certificate in use, as if it is RSA-based. Therefore, in /etc/dirsrv/slapd-instance_name/dse.ldif (that is the starting configuration of the instance), you will find the entry:

dn: cn=RSA,cn=encryption,cn=config
objectClass: top
objectClass: nsEncryptionModule
cn: RSA
nsSSLPersonalitySSL: server-cert
nsSSLToken: internal (software)
nsSSLActivation: on

even if the X.509 certificate with nickname "server-cert" installed there (the nickname the certificate is know under in the NSS DB store) is ECDSA-based. And this "masking" of the ECDSA-based certificate as RSA, causes those errors from above to appear, when the SSL certificated is ECDSA-based. Note that the current method for deriving 3DES and AES attribute encryption keys is based on RSA encryption keys only. To solve this problem (especially if you want to deliberately generate 3DES and AES keys), you need to implement some workaround. That workaround works in the following way. Define an entry for the ECDSA certificate in the starting configuration (replace "server-cert" with your actual ECDSA X.509 certificate nickname, listed in the NSS DB certificate store):

$ ldapadd -D "cn=Directory Manager" -x -W
dn: cn=ECDSA,cn=encryption,cn=config
objectClass: top
objectClass: nsEncryptionModule
cn: ECDSA
nsSSLPersonalitySSL: server-cert
nsSSLToken: internal (software)
nsSSLActivation: on

Then import into NSS DB certificate store new RSA certificate, that will be used only to support the attribute encryption key generation (we assume its nickname is "keygen-cert"), remove the old dn "cn=RSA,cn=encryption,cn=config" (if exists):

$ ldapdelete -D "cn=Directory Manager" -x -W "cn=RSA,cn=encryption,cn=config"

and add it again, pointing this time the attribute "nsSSLPersonalitySSL" to the RSA certificate nickname ("keygen-cert" in the example bellow):

$ ldapadd -D "cn=Directory Manager" -x -W
dn: cn=RSA,cn=encryption,cn=config
objectClass: top
objectClass: nsEncryptionModule
cn: RSA
nsSSLPersonalitySSL: keygen-cert
nsSSLToken: internal (software)
nsSSLActivation: off

PAY ATTENTION: To avoid the use of RSA certificate as a host certificate, and to force the use of ECDSA one for that purpose, you have to set nsSSLActivation to off. That is mandatory!

At this point, you might restart your 389 Directory Server instance:

$ sudo systemctl restart dirsrv@instance_name

Then proceed to "Deriving new attribute encryption keys".

4. Deriving new attribute encryption keys

If you don't know how many back-ends does your 389 DS currently supports, you might check that:

$ ldapsearch -D "cn=Directory Manager" -b "cn=config" -x -W "nsslapd-backend=*" nsslapd-backend | grep ^nsslapd-backend | awk '{print $2}'

(you will have to know the "cn=Directory Manager" password to complete the search). In most cases (default setup schema) the result will look like this:

userRoot

If the administrative server is also installed and configured, the NetscapeRoot back-end will be shown too:

NetscapeRoot
userRoot

The next step is to delete the dn-objects containing the AES and 3DES attribute encryption keys. For each back-end, you need to execute (replace BACKEND with the actual name of the back-end):

$ ldapdelete -D "cn=Directory Manager" -x -W "cn=AES,cn=encrypted attribute keys,cn=BACKEND,cn=ldbm database,cn=plugins,cn=config"
$ ldapdelete -D "cn=Directory Manager" -x -W "cn=3DES,cn=encrypted attribute keys,cn=BACKEND,cn=ldbm database,cn=plugins,cn=config"

If you receive "No such object" as a result, that might be a proof that only X.509 ECC-based certificates have been used with the current installation of 389 DS, ever since it was created.

You can restart now the 389 DS daemon (replace "instance_name" with your actual 389 Directory Server instance name):

$ sudo systemctl restart dirsrv@instance_name

and if the keys were successfully derived during the restart of 389 DS daemon, you will see the following messages at the end of /var/log/dirsrv/slapd-instance_name/error (two lines for each back-end):

[04/Nov/2019:04:57:08.393260367 +0200] - ERR - attrcrypt_cipher_init - No symmetric key found for cipher AES in backend userRoot, attempting to create one...
[04/Nov/2019:04:57:08.446317176 +0200] - INFO - attrcrypt_cipher_init - Key for cipher AES successfully generated and stored
[04/Nov/2019:04:57:08.470085086 +0200] - ERR - attrcrypt_cipher_init - No symmetric key found for cipher 3DES in backend userRoot, attempting to create one...
[04/Nov/2019:04:57:08.504787008 +0200] - INFO - attrcrypt_cipher_init - Key for cipher 3DES successfully generated and stored

No such messages will appear again during the next restarts, unless you remove the employed RSA-based X.509 certificates from the NSS DB.

Read More

Using SmartCard-HSM 4K USB-Token to work with ECDSA keys in RHEL, CentOS, and Enterprise Linux, versions 7 and 8

Content:

1. Introduction

2. Connecting the device

3. Downloading, compiling, and installing the last stable release of OpenSC and HSM code

4. Initializing the device (setting the PINs)

5. Generating 521-bit EC key pair

6. Using the USB token as OpenSSL PKCS#11 engine

7. Generating CSR, based on a key pair previously stored in the token, through OpenSSL PKCS#11 engine

8. Importing X.509 certificate to the token storage

 

1. Introduction

In the modern cryptographic systems, the use of Elliptic Curve cryptography (ECC) becomes de facto a standard. Meanwhile, even now (it is 2019), it is quite rare to find a cryptographic token that supports the generation and use of ECDSA keys, up to 512 bits, SHA512 hash function (embedded in the processor of the token), and comes with a driver for Linux OS support. Most of the available tokens support mostly RSA key pairs, and even if they support ECDSA, that support is partial and the maximum key size is limited bellow 512 bits.

Surprisingly, you can find a reliable crypto token, that supports ECDSA 512-bit keys and SHA512 at:

cardomatic.de

The SmartCard-HSM 4K USB-Token, they have in sale, is supported by the driver library libccid.so, which is part of the OpenSC package (version 0.16 or higher). If one can download, compile, and install locally both the latest OpenSC and HSM code release, that is sufficient to employ the SmartCard-HSM 4K USB-Token in OpenSSL (as an engine), Mozilla Firefox, and Mozilla Thunderbird (as a security device).

The notes given bellow explain how to make the SmartCard-HSM 4K USB-Token device available in a Linux-based system running CentOS 7, Scientific Linux 7, Red Hat Enterprise Linux 8, and (possibly) Red Hat Enterprise Linux 8, how initializing the device, and the modules to interact the opensc tools (like pkcs11-tools and pkcs15-tools).

 

2. Connecting the device

Install the following packages (there is a good chance they are already installed):

# yum install pcsc pcsc-lite-ccid opensc

Enable the daemon pcscd to be loaded when starting/restarting the system:

# systemctl enable pcscd

and run it manually afterwards:

# systemctl start pcscd

Now you can connect the SmartCard-HSM 4K USB-Token device to the system. Records like those given bellow will be added to the /var/log/messages by syslog:

May 10 15:00:06 argus kernel: usb 3-10: new full-speed USB device number 9 using xhci_hcd
May 10 15:00:06 argus kernel: usb 3-10: New USB device found, idVendor=04e6, idProduct=5816
May 10 15:00:06 argus kernel: usb 3-10: New USB device strings: Mfr=1, Product=2, SerialNumber=5
May 10 15:00:06 argus kernel: usb 3-10: Product: uTrust 3512 SAM slot Token
May 10 15:00:06 argus kernel: usb 3-10: Manufacturer: Identiv
May 10 15:00:06 argus kernel: usb 3-10: SerialNumber: 55511247701280

Their appearance there does not exactly mean the device is accessible by the applications. It only indicates that the USB device is properly recognized by libusb. In addition, the device will appear under the Vendor Name: SCM Microsystems, Inc., Vendor ID: 04e6, and Product ID: 5816, in the output generated by the execution of lsusb:

Bus 003 Device 009: ID 04e6:5816 SCM Microsystems, Inc.

If detailed debugging information, regarding the process of device detection, performed by pcscd daemon, is required, do modify the content of the file /usr/lib/systemd/system/pcscd.service (used by systemd to invoke the daemon pcscd), by adding the additional declaration --debug to the end of the line starting with "ExecStart":

ExecStart=/usr/sbin/pcscd --foreground --auto-exit --debug

Save the changes to the file, load the new list of declarations:

# systemctl daemon-reload

and finally restart the daemon pcscd:

# systemctl restart pcscd

At that moment, start monitoring the new content appended to the file /var/log/messages (# tailf /var/log/messages will do), then unplug and plug in the device, and watch what kind of messages the syslog daemon is appending to the file.

To prevent the fast growth of the /var/log/messages size on the disk, immediately after finishing with the analysis of the debug information, do restore the original declarations in /usr/lib/systemd/system/pcscd.service (remove the additional declaration --debug), reload it, and restart the daemon pcscd!

 

3. Downloading, compiling, and installing the last stable release of OpenSC and HSM code

First, install the required RPM packages (if they are not already installed):

# yum install git gcc gcc-c++ make

Then fetch the code of sc-hsm-embedded project, hosted on Github, configure, compile, and install it (change the destination folder after --prefix, if needed):

$ git clone https://github.com/CardContact/sc-hsm-embedded.git
$ cd sc-hsm-embedded
$ ./configure --prefix=/usr/local/sc-hsm-embedded
$ make
$ sudo make install

General note: Red Hat Enterprise Linux 8 (as well as CentOS 8 and Scientific Linux 8), ships OpenSC version 0.19. That means one does not need to compile OpenSC code, as explained bellow, in systems running EL8.

Once the installation is successfully completed, download, compile, and install the latest stable release of the OpenSC code (change the destination folder after --prefix and --with-completiondir, if needed):

$ wget https://github.com/OpenSC/OpenSC/releases/download/0.19.0/opensc-0.19.0.tar.gz
$ tar xvf opensc-0.19.0.tar.gz
$ cd opensc-0.19.0
$ ./configure --prefix=/usr/local/opensc-0.19.0 --with-completiondir=/usr/local/opensc-0.19.0/share/bash-completion/completions
$ make
$ sudo make install

The next step is to add the folders containing the installed binaries and libraries to the user's PATH and LD_LIBRARY_PATH environmental variables. Those additions can be made either system-wide (available to all users, inclusive) or user-specific (available only to specific users, exclusive):

• system-wide:

  Write down the following export declarations:

  export PATH=/usr/local/opensc-0.19.0/bin:$PATH
  export LD_LIBRARY_PATH=/usr/local/sc-hsm-embedded/lib:/usr/local/opensc-0.19.0/lib:$LD_LIBRARY_PATH

  to the file /etc/profile.d/opensc.sh (create the file, it does not exist by default there).

• user-specific:

  Open each ~/.bashrc file and append (to the end) the following lines:

  export PATH=/usr/local/opensc-0.19.0/bin:$PATH
  export LD_LIBRARY_PATH=/usr/local/sc-hsm-embedded/lib:/usr/local/opensc-0.19.0/lib:$LD_LIBRARY_PATH

 

4. Initializing the device (setting the PINs)

The SmartCard-HSM 4K USB-Token device comes uninitialized! That means the device cannot be used, unless complete the initialization process first.

Connect the token to the system and invoke the tool sc-hsm-tool by specifying the SO PIN (SO stands for "Security Officer", change SO PIN 3537363231383830, specified in the example bellow, with another unique 16-digit long decimal number and do not forget it) and the user's PIN (change 123456 to another number):

$ sc-hsm-tool --initialize --so-pin 3537363231383830 --pin 123456

Here, the number after --so-pin is the SO PIN and the one after --pin is the user's PIN. Note that the SO PIN is the most important PIN, for it allows write access to the all slots of the device storage. Also, it can change every PUK and PIN, previously stored there (that includes the SO PIN itself). So do not loose SO PIN number!

5. Generating 521-bit EC key pair

Invoke the tool pkcs11-tool and specify the key type (EC:specp521r1 for 521-bit EC encryption key) and key pair ID label (it helps later to access the keys):

$ pkcs11-tool --module libsc-hsm-pkcs11.so -l --keypairgen --key-type EC:secp521r1 --id 10 --label "EC pair ID 10"

The user's PIN will be requested next, to grant access to the device storage and processor:

Enter PKCS#11 token PIN for SmartCard-HSM:

If the provided user's PIN is correct, it will take up to 2-3 seconds to execute and complete the key generation process.

The most important part of the command line above is the ID number assigned to the key pair (given above is 10, but you may reserve some other number, including hexadecimal number syntax, which is not already taken), because from that moment onward, the keys of that pair can be selected and involved in operations only if their ID number is specified.

 

6. Using the USB token as OpenSSL PKCS#11 engine

Install the package openssl-pkcs11 to enable the OpenSSL PKCS#11 engines functionality:

# yum install openssl-pkcs11

After that, store the following minimal OpenSSL configuration into a local file (openssl_hsm.cnf, for the sake of our illustration process):

openssl_conf = openssl_def

[openssl_def]
engines = engine_section

[engine_section]
pkcs11 = pkcs11_section

[pkcs11_section]
engine_id = libsc-hsm-pkcs11
dynamic_path = /usr/lib64/engines-1.1/pkcs11.so
MODULE_PATH = /usr/local/opensc-0.19.0/lib/libsc-hsm-pkcs11.so
init = 0


[ req ]
distinguished_name      = req_distinguished_name
string_mask = utf8only

[ req_distinguished_name ]

The only declaration in openssl_hsm.cnf one might need to change is that of MODULE_PATH:

MODULE_PATH = /usr/local/opensc-0.19.0/lib/libsc-hsm-pkcs11.so

Replace /usr/local/opensc-0.19.0 there with the actual folder, where the file libsc-hsm-pkcs11.so is stored. Note that the file is brought there during the installation of the HSM code).

 

7. Generating CSR, based on a key pair previously stored in the token, through OpenSSL PKCS#11 engine

If the OpenSSL PKCS#11 engine configuration file is available, the generation of CSR PKCS#10 block requires to specify the key pair ID (that is the number assigned to the key pair during the process of key pair generation):

$ openssl req -new -subj "/CN=server.example.com" -out request.pem -sha512 -config openssl_hsm.cnf -engine pkcs11 -keyform engine -key 10

One can expand both command line arguments and the set of declarations stored in openssl_hsm.cnf, if the CSR should contain some specific extensions, requested by the certificate issuer.

 

8. Importing X.509 certificate to the token storage

That type of operation is possible only if the X.509 certificate is stored in DER format (binary). For example, if the file SU_ECC_Root_CA.crt contains PEM formatted X.509 certificate, that certificate might be converted into DER format, by using openssl:

$ openssl x509 -in SU_ECC_Root_CA.crt -outform DER -out SU_ECC_Root_CA.der

To import the DER formatted X.509 certificate to the USB storage token, use the tool pkcs11-tool:

$ pkcs11-tool --module ibsc-hsm-pkcs11.so --login --write-object SU_ECC_Root_CA.der --type cert --id `uuidgen | tr -d -` --label "SU_ECC_ROOT_CA"

The label string after --label, naming the X.509 certificate, should be specified in a way hinting the purpose of having and using that certificate.

Read More

Using ECC keys with NetworkManager for EAP-TLS authentication

Content:

1. Introduction

2. Description of the problem

3. Cause and solution

Appendix: How to create PKCS#12 file using OpenSSL

1. Introduction

NetworkManager is employed by almost all modern Linux distributions to manage a large variety of network connection types (Ethernet, WiFi, VPN, PPP) and has got both GUI (graphical user interface) and command line tools.

NetworkManager can read PEM-formatted RSA end-user certificates and keys, when a 802.1x-based WiFi connection requires the user authentication process to be based on Extensible Authentication Protocol (EAP) with Transport Layer Security (TLS) - EAP-TLS, defined in RFC5216. For example, the command line:

nmcli connection add type wifi con-name "eduroam" ifname wlan0 ssid "eduroam" -- wifi-sec.key-mgmt wpa-eap 802-1x.eap tls 802-1x.ca-cert /home/username/eduroam/ecc_ca.crt 802-1x.identity username@example.com 802-1x.phase2-ca-path /home/username/eduroam/ca.crt  802-1x.client-cert /home/username/end_user.crt 802-1x.private-key /home/username/end_user.key 802-1x.phase2-private-key-password "some_password" 802-1x.anonymous-identity "anonymous@example.com"

defines a new WiFi connection, manageable by NetworkManager, if the file /home/username/rsa_end_user.key contains PEM-formatted RSA (usually encrypted) private key that is complementary to the public key in /home/username/rsa_end_user.crt.

2. Description of the problem

When the key file, given after 802-1x.private-key, contains PEM-formatted private key generated using Elliptic-Curves, the NetworkManager fails to parse it and rises the following error message:

Error: failed to modify 802-1x.private-key: 802-1x.private-key: invalid private key.

3. Cause and solution

The construction of the PEM-formatted block, used for storing the private keys in text files, in case of RSA key:

-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: AES-256-CBC,CA02E2DB3CA8464D0A35527EAE024204

...
-----END RSA PRIVATE KEY-----

is completely different from the one specific to the EC keys:

-----BEGIN EC PARAMETERS-----
...
-----END EC PARAMETERS-----
-----BEGIN EC PRIVATE KEY-----
...
-----END EC PRIVATE KEY-----

Handling that kind of differences, requires the implementation of a complex text parser, which might become (in certain cases) a source of problems (parsing text files to match patterns is always a challenge). Instead of updating the text parser, NetworkManager team decided to unify and optimize the process of certificate/key reading. To introduce more flexible and robust procedure for reading and identifying cryptography objects, and to simplify the programming code, NetworkManager team adopted a PKCS#12 containers reader. The goal of PKCS#12 is to provide the software applications with a container format for storing (binary) cryptography objects, like keys and certificates (incl. chain of certificates). Having PKCS#12 functionality added to the programming code not only shortens that code, but also makes the process of reading, identifying, and verifying the cryptography objects a task easy to perform (at the expense of linking the executable to external libraries like OpenSSL and NSS libraries).

To solve the problem described above, the end user certificate and key need to be converted and stored into a single PKCS#12 file (with extension *.p12). Then, when invoking nmcli to create the new WiFi connection set of settings, that file (as full path) should be given after the switches 802-1x.client-cert and 802-1x.private-key:

nmcli connection add type wifi con-name "eduroam" ifname wlan0 ssid "eduroam" -- wifi-sec.key-mgmt wpa-eap 802-1x.eap tls 802-1x.ca-cert /home/username/eduroam/ecc_ca.crt 802-1x.identity username@example.com 802-1x.phase2-ca-path /home/username/eduroam/ca.crt  802-1x.client-cert /home/username/rsa_end_user.p12 802-1x.private-key /home/username/rsa_end_user.p12 802-1x.phase2-private-key-password "some_password" 802-1x.anonymous-identity "anonymous@example.com"

Short and useful recipe for creating PKCS#12 files is given in "Appendix" down bellow.

Appendix: How to create PKCS#12 file using OpenSSL

openssl pkcs12 -export -aes256 -maciter -in end_user.crt -inkey end_user.key -name "ECC_End_User" -chain -CAfile ca.crt -out end_user.p12

Read More