This document describes a secure file encryption system that uses a Hardware Security Module (HSM) for key derivation. The system ensures that encryption keys are derived securely within the HSM and never stored permanently on disk.
Table of Contents
- 1. Overview
- 2. Components
- 3. Security Features
- 4. HSM Object Inspection
- 5. Implementation Details
- 6. Important Note: Why --hash Instead of --derive
- 7. Hash Mechanism Flexibility
- 8. Cipher Mode Security
- 9. Usage
- 10. Security Considerations
- 11. Limitations
- 12. Cold Boot Attack Mitigation
- 13. TODO List
- 14. YubiHSM 2 and Cold Boot Attack Prevention
- 15. Dependencies
- 16. Usage Notes
Overview
Why HSM-Based Encryption?
Traditional file encryption methods often expose encryption keys or passwords to the operating system in various ways:
- Keys stored on disk (even if encrypted)
- Keys loaded into OS memory
- Keys accessible to system processes
- Keys vulnerable to memory dumps
- Keys potentially exposed through swap files
- Keys that could be intercepted by malware
This HSM-based approach provides superior security by:
- Hardware Isolation: All key derivation operations happen inside the HSM's secure boundary
- Minimal Key Exposure: Keys are derived on-demand and stored only briefly in RAM-based temporary files
- PIN Protection: Physical possession of the HSM alone is insufficient; PIN authentication is required
- Tamper Resistance: HSM hardware is designed to resist physical attacks
- Secure Cleanup: Temporary key files are securely wiped using
shredafter use - Process Isolation: Even if the host system is compromised, the HSM's internal operations remain secure
System Operation
The system consists of two main operations:
- File encryption with a unique key derived from a random salt
- File decryption using the same salt to derive the identical key
The system uses the HSM's SHA-256 mechanism to derive encryption keys, ensuring that key derivation happens within the secure hardware environment.
Components
System Requirements
This encryption system is designed and tested for Linux environments (tested on RHEL 9). While the core components (OpenSC, OpenSSL) are available on macOS, some adjustments might be needed for full compatibility:
- Linux (primary platform)
- Tested on RHEL 9
- Should work on most modern Linux distributions
- All components available through standard package managers
- macOS (potentially compatible)
- Requires OpenSC installation via Homebrew
- May need path adjustments for PKCS#11 module
- Limited testing available
- Windows (untested)
- OpenSC and OpenSSL are available for Windows
- Would require significant path and script adjustments
- Has not been tested by the author
- Might work with Cygwin or WSL (Windows Subsystem for Linux)
Hardware Requirements
The code snippets shown below match the features of the following HSM tokens:
- SmartCard-HSM (tested with Identiv uTrust 3512 SAM slot Token)
- Nitrokey HSM 2
They can be purchased from either:
- CardOMatic GmbH (Identiv uTrust 3xxx in various form factors):
- USB Tokens (recommended for desktop usage):
- SmartCard-HSM 4K USB-Token (Standard form factor)
- SmartCard-HSM 4K Mini USB-Token (Compact form factor)
- Card Formats:
- SmartCard-HSM-4K-Dual-IF (Dual Interface)
- SmartCard-HSM-4K-Mini-SIM
- SmartCard-HSM-4K-Micro-SIM
- Special Formats:
- SmartCard-HSM 4K MicroSD Card (For mobile devices)
- USB Tokens (recommended for desktop usage):
- Nitrokey Shop (Nitrokey HSM 2)
All devices feature 4K storage and support the required cryptographic operations.
⚠️ Important Compatibility Note:
This encryption system specifically requires HSM capabilities that are NOT available in PIV-based smartcards (like YubiKey PIV devices). PIV smartcards:
- Do not support direct hash operations within the device
- Cannot perform arbitrary SHA-256 operations on input data
- Lack the required key derivation mechanisms
- Have limited storage for custom objects
Only true HSM devices (like SmartCard-HSM or Nitrokey HSM 2) provide the necessary cryptographic operations for this system to work. YubiHSM 2 may work as well, but that statement requires a proper validation.
Software Components
- OpenSC PKCS#11 module (
/usr/lib64/pkcs11/opensc-pkcs11.so) aes_encrypt.sh- Encrypts files using HSM-derived keys (see the code snippets below)aes_decrypt.sh- Decrypts files using HSM-derived keys (see the code snippets below)
Security Features
- Hardware-Based Key Derivation
- All key derivation happens inside the HSM
- Uses HSM's SHA-256 mechanism
- Requires HSM PIN authentication for both encryption and decryption
- Unique Keys Per File
- Each encryption operation uses a new random 32-byte salt
- Salt is used to derive a unique key through the HSM
- Prevents key reuse across different files
- Secure Key Handling
- Derived keys exist only in RAM-based temporary files during operations
- Keys are securely wiped using
shredafter use - No keys are stored permanently on disk
- File Format
- First 32 bytes: Random salt used for key derivation
- Remaining bytes: AES-256 encrypted data (mode depends on implementation)
- Operational Security
- HSM Redundancy: Maintain at least two HSM tokens with identical ECDSA key pairs
- Critical for disaster recovery
- Protects against HSM loss, theft, or hardware failure
- Ensures access if one token gets locked out due to PIN attempts
- Both tokens should be stored securely in different locations
- Without a working HSM containing the correct key pair, decryption is impossible
- Regular testing of backup HSM tokens is recommended
- HSM Redundancy: Maintain at least two HSM tokens with identical ECDSA key pairs
- HSM Cryptographic Capabilities
You can inspect the HSM's supported mechanisms using:
pkcs11-tool --module /usr/lib64/pkcs11/opensc-pkcs11.so --list-mechanismsExample output:
Using slot 0 with a present token (0x0) Supported mechanisms: SHA-1, digest SHA224, digest SHA256, digest SHA384, digest SHA512, digest MD5, digest RIPEMD160, digest GOSTR3411, digest ECDSA, keySize={192,521}, hw, sign, verify, EC F_P, EC parameters, EC OID, EC uncompressed ECDSA-SHA384, keySize={192,521}, sign, verify ECDSA-SHA512, keySize={192,521}, sign, verify ECDSA-SHA1, keySize={192,521}, hw, sign, verify, EC F_P, EC parameters, EC OID, EC uncompressed ECDSA-SHA224, keySize={192,521}, hw, sign, verify, EC F_P, EC parameters, EC OID, EC uncompressed ECDSA-SHA256, keySize={192,521}, hw, sign, verify, EC F_P, EC parameters, EC OID, EC uncompressed ECDH1-COFACTOR-DERIVE, keySize={192,521}, hw, derive, EC F_P, EC parameters, EC OID, EC uncompressed ECDH1-DERIVE, keySize={192,521}, hw, derive, EC F_P, EC parameters, EC OID, EC uncompressed ECDSA-KEY-PAIR-GEN, keySize={192,521}, hw, generate_key_pair, EC F_P, EC parameters, EC OID, EC uncompressed RSA-X-509, keySize={1024,4096}, hw, decrypt, sign, verify RSA-PKCS, keySize={1024,4096}, hw, decrypt, sign, verify SHA1-RSA-PKCS, keySize={1024,4096}, sign, verify SHA224-RSA-PKCS, keySize={1024,4096}, sign, verify SHA256-RSA-PKCS, keySize={1024,4096}, sign, verify SHA384-RSA-PKCS, keySize={1024,4096}, sign, verify SHA512-RSA-PKCS, keySize={1024,4096}, sign, verify MD5-RSA-PKCS, keySize={1024,4096}, sign, verify RIPEMD160-RSA-PKCS, keySize={1024,4096}, sign, verify RSA-PKCS-PSS, keySize={1024,4096}, hw, sign, verify SHA1-RSA-PKCS-PSS, keySize={1024,4096}, sign, verify SHA224-RSA-PKCS-PSS, keySize={1024,4096}, sign, verify SHA256-RSA-PKCS-PSS, keySize={1024,4096}, sign, verify SHA384-RSA-PKCS-PSS, keySize={1024,4096}, sign, verify SHA512-RSA-PKCS-PSS, keySize={1024,4096}, sign, verify RSA-PKCS-OAEP, keySize={1024,4096}, hw, decrypt RSA-PKCS-KEY-PAIR-GEN, keySize={1024,4096}, generate_key_pairThe HSM supports a comprehensive set of cryptographic operations:
- Hash Functions:
- SHA family (SHA-1, SHA224, SHA256, SHA384, SHA512)
- MD5
- RIPEMD160
- GOSTR3411
- ECDSA Operations (key sizes 192-521 bits):
- Key pair generation
- Signing/verification with various hash combinations
- Hardware-accelerated operations
- Support for standard curves (EC F_P)
- RSA Operations (key sizes 1024-4096 bits):
- Key pair generation
- Encryption/decryption
- Signing/verification with various hash combinations
- Support for PKCS, PSS, and OAEP padding
- Key Derivation:
- ECDH1-DERIVE
- ECDH1-COFACTOR-DERIVE
This encryption system specifically uses the SHA256 mechanism for key derivation, ensuring the operation is performed securely within the HSM's hardware boundary.
- Hash Functions:
HSM Object Inspection
To inspect the objects (keys, certificates, data) stored on your HSM, use the pkcs11-tool --list-objects command:
pkcs11-tool --module /usr/lib64/pkcs11/opensc-pkcs11.so --list-objects
Expected Output Format
The command will show all objects stored on your HSM. Here's an example output:
Using slot 0 with a present token (0x0) Certificate Object; type = X.509 cert label: vesso_ecdsa#ssh subject: DN: CN=Test User serial: 65C25210C8EDD2269D3DFFB836C2485C0A17BC32 ID: 01 uri: pkcs11:model=PKCS%2315%20emulated;manufacturer=www.CardContact.de;serial=DECC0900169;token=SmartCard-HSM%20%28UserPIN%29;id=%01;object=test_ecdsa%23ssh;type=cert Public Key Object; EC EC_POINT 384 bits EC_POINT: 046104644baa533af20667849c4a6f38092e663f376d167cc1f64adca5cda8b72b1998d7757ac9057695cec32d1fc6239f8a282c9a803cb603dc1dd81907e411e737ab8e69129ea97035895bce14cc079043848c890ef969b6330c8a65418904a9d414 EC_PARAMS: 06052b81040022 label: test_ecdsa#ssh ID: 01 Usage: encrypt, verify Access: local uri: pkcs11:model=PKCS%2315%20emulated;manufacturer=www.CardContact.de;serial=DECC0900169;token=SmartCard-HSM%20%28UserPIN%29;id=%01;object=test_ecdsa%23ssh;type=public Public Key Object; EC EC_POINT 256 bits EC_POINT: 0441047bfd2ff666960d4186f1aab2370b138554fe2dc2163a7016e2967624b8ad1ee3e36d25c84cac28b9040cf80ff68fa63e06c73a44a21ef70a74701fa8ea30c3d4 EC_PARAMS: 06082a8648ce3d030107 label: ECDH-KEY ID: 02 Usage: verify Access: none uri: pkcs11:model=PKCS%2315%20emulated;manufacturer=www.CardContact.de;serial=DECC0900169;token=SmartCard-HSM%20%28UserPIN%29;id=%02;object=ECDH-KEY;type=public Public Key Object; EC EC_POINT 256 bits EC_POINT: 044104e1dc094f873e00eb86626f7ae0a845885df08cb89a797d29271d6b7f602e685bf9681cda2ecaeaa671f39837a11ee9259142c706d988558db907ad6560879ac3 EC_PARAMS: 06082a8648ce3d030107 label: ECDH-AES-Key ID: 03 Usage: verify Access: none uri: pkcs11:model=PKCS%2315%20emulated;manufacturer=www.CardContact.de;serial=DECC0900169;token=SmartCard-HSM%20%28UserPIN%29;id=%03;object=ECDH-AES-Key;type=public Profile object 1385173136 profile_id: CKP_PUBLIC_CERTIFICATES_TOKEN (4)
Understanding the Output
- Object Types:
Certificate Object: X.509 certificates with subject and serial informationPublic Key Object: Public keys with their parameters and pointsProfile Object: HSM configuration profiles
- Object Attributes:
label: Name of the object (e.g., "test_ecdsa#ssh", "ECDH-KEY")ID: Object identifier (01, 02, 03)Usage: Object capabilities (encrypt, verify)Access: Access restrictions (local, none)uri: PKCS#11 URI identifying the object
Key Objects in This Setup
ID 01: SSH ECDSA key (384-bit) with certificateID 02: ECDH key (256-bit) for key exchangeID 03: ECDH-AES key (256-bit) for encryption operations
Usage Notes
- If the HSM is locked, add
--loginand provide a PIN:pkcs11-tool --module /usr/lib64/pkcs11/opensc-pkcs11.so --login --list-objects - Use this command to:
- Verify key installation and parameters
- Check certificate details
- Audit HSM object configuration
- Confirm key availability before operations
Implementation Details
Encryption Process
- Salt Generation
openssl rand -out "$SALT_FILE" 32 - Key Derivation in HSM
pkcs11-tool --module /usr/lib64/pkcs11/opensc-pkcs11.so \ --slot 0 \ --login \ --pin "$HSM_PIN" \ --hash \ --mechanism SHA256 \ --input-file "$SALT_FILE" > "$KEY_FILE" - File Encryption
- Salt is written to the start of the output file
- Data is encrypted using AES-256-CTR with the derived key
cat "$SALT_FILE" > "$OUTPUT_FILE" openssl enc -aes-256-ctr -pbkdf2 -iter 1 -salt -in "$INPUT_FILE" -kfile "$KEY_FILE" >> "$OUTPUT_FILE" - Secure Cleanup
shred -u "$SALT_FILE" "$KEY_FILE"
Decryption Process
- Salt Extraction
dd if="$INPUT_FILE" of="$SALT_FILE" bs=32 count=1 dd if="$INPUT_FILE" of="$ENCRYPTED_DATA" bs=1 skip=32 - Key Derivation in HSM
- Uses the same HSM SHA-256 mechanism with the extracted salt
- Produces identical key as encryption
- File Decryption
openssl enc -aes-256-ctr -pbkdf2 -iter 1 -d -salt -in "$ENCRYPTED_DATA" -out "$OUTPUT_FILE" -kfile "$KEY_FILE" - Secure Cleanup
shred -u "$SALT_FILE" "$ENCRYPTED_DATA" "$KEY_FILE"
Important Note: Why --hash Instead of --derive
When implementing this encryption system, you might be tempted to use the --derive option with pkcs11-tool. However, this will result in an error:
pkcs11-tool --module /usr/lib64/pkcs11/opensc-pkcs11.so --derive --mechanism SHA256 --input-file salt.bin
error: Private key not found
Aborting.
Why --derive Doesn't Work
The --derive option in pkcs11-tool is designed for key derivation operations that require a private key, such as:
- ECDH (Elliptic Curve Diffie-Hellman) key exchange
- Key derivation using existing private keys
- Cryptographic operations that need a base key
The --derive option expects:
- A private key to be specified (using
--id,--label, or--object-index) - A mechanism that supports key derivation (like ECDH1-DERIVE)
- Input data that will be used with the private key for derivation
Why --hash is Correct
For our encryption system, we want to perform a simple hash operation on the salt to create a key. This is exactly what the --hash option provides:
pkcs11-tool --module /usr/lib64/pkcs11/opensc-pkcs11.so \
--slot 0 \
--login \
--pin "$HSM_PIN" \
--hash \
--mechanism SHA256 \
--input-file "$SALT_FILE" > final_key.bin
The --hash option:
- Takes input data (our salt)
- Performs the specified hash operation (SHA256) within the HSM
- Returns the hash result as output
- Doesn't require any private keys
HSM Private Keys vs Hash Operations
Your HSM contains several private keys (both RSA and EC), but these are for different purposes:
Private Keys with derive usage:
- Used for ECDH key exchange operations
- Require a corresponding public key for derivation
- Used in protocols like TLS key exchange
Hash Operations:
- Simple one-way functions
- Don't require any keys
- Perfect for our key derivation from salt
Verification
You can verify the available private keys and their capabilities:
# List private keys (requires PIN)
pkcs11-tool --module /usr/lib64/pkcs11/opensc-pkcs11.so --login --list-objects --type privkey
# Example output showing EC keys with derive capability:
# Private Key Object; EC
# label: test@example.com#59edd63
# ID: 02
# Usage: sign, derive
# Access: sensitive, always sensitive, never extractable, local
The derive usage on EC keys is for ECDH operations, not for simple hashing. Our encryption system correctly uses --hash for the SHA-256 operation on the salt.
Hash Mechanism Flexibility
The encryption system is not limited to SHA-256. You can use any hash mechanism supported by your HSM, as long as it provides sufficient output length for your AES key. The choice of hash mechanism depends on your security requirements and HSM capabilities.
Available Hash Mechanisms
Based on the HSM mechanisms list shown earlier, your HSM supports these hash functions:
- SHA-1 (160 bits) - Not recommended for new implementations
- SHA224 (224 bits) - Adequate for AES-256
- SHA256 (256 bits) - Recommended for AES-256
- SHA384 (384 bits) - Excellent for AES-256, provides extra security margin
- SHA512 (512 bits) - Maximum security, excellent for AES-256
- MD5 (128 bits) - Not recommended, insufficient for AES-256
- RIPEMD160 (160 bits) - Not recommended for new implementations
Recommended Hash Mechanisms
For AES-256 encryption, the following mechanisms are recommended in order of preference:
- SHA512 - Maximum security margin (512 bits → 256 bits)
- SHA384 - Excellent security margin (384 bits → 256 bits)
- SHA256 - Optimal fit (256 bits → 256 bits)
- SHA224 - Adequate but minimal margin (224 bits → 256 bits, will be padded)
Using Different Hash Mechanisms
To use a different hash mechanism, simply change the --mechanism parameter:
# Using SHA-512 (maximum security)
pkcs11-tool --module /usr/lib64/pkcs11/opensc-pkcs11.so \
--slot 0 \
--login \
--pin "$HSM_PIN" \
--hash \
--mechanism SHA512 \
--input-file "$SALT_FILE" > final_key.bin
# Using SHA-384 (excellent security margin)
pkcs11-tool --module /usr/lib64/pkcs11/opensc-pkcs11.so \
--slot 0 \
--login \
--pin "$HSM_PIN" \
--hash \
--mechanism SHA384 \
--input-file "$SALT_FILE" > final_key.bin
# Using SHA-224 (adequate but minimal margin)
pkcs11-tool --module /usr/lib64/pkcs11/opensc-pkcs11.so \
--slot 0 \
--login \
--pin "$HSM_PIN" \
--hash \
--mechanism SHA224 \
--input-file "$SALT_FILE" > final_key.bin
Security Considerations
- SHA-512/SHA-384: Provide security margin beyond AES-256 requirements
- SHA-256: Optimal fit for AES-256, widely trusted
- SHA-224: Adequate but provides minimal security margin
- SHA-1/MD5: Not recommended due to known vulnerabilities
- RIPEMD160: Not recommended for new implementations
Implementation Note
When changing hash mechanisms, ensure that:
- Both encryption and decryption scripts use the same mechanism
- The mechanism is supported by your HSM (verify with
--list-mechanisms) - The output length is sufficient for your AES key size
- You maintain consistency across all encrypted files
Checking HSM Support
Always verify that your HSM supports the chosen mechanism:
pkcs11-tool --module /usr/lib64/pkcs11/opensc-pkcs11.so --list-mechanisms | grep -E "(SHA|MD5|RIPEMD)"
Cipher Mode Security
⚠️ Important Security Warning:
The original implementation uses AES-256-CBC mode, which has known vulnerabilities including padding oracle attacks and the need for proper IV handling. Modern applications should use more secure cipher modes.
Available Secure Cipher Modes
OpenSSL supports these secure AES-256 modes:
- AES-256-CTR (Counter Mode) - Recommended for OpenSSL enc
- No padding required
- Resistant to padding oracle attacks
- Stream cipher mode
- Note: Does not provide authentication (use with caution)
- AES-256-GCM (Galois/Counter Mode) - Best Security (if available)
- Provides both encryption and authentication
- No padding required
- Built-in integrity protection
- Resistant to padding oracle attacks
- Note: Not supported by OpenSSL
enccommand in all versions
- AES-256-CCM (Counter with CBC-MAC) - Good Alternative (if available)
- Provides both encryption and authentication
- No padding required
- Built-in integrity protection
- Slightly more complex than GCM
- Note: Not supported by OpenSSL
enccommand in all versions
- AES-256-CBC (Cipher Block Chaining) - Not Recommended
- Vulnerable to padding oracle attacks
- Requires proper IV handling
- No built-in integrity protection
- Only use if compatibility is absolutely required
Why Avoid CBC Mode?
CBC mode has several security issues:
- Padding Oracle Attacks: Attackers can exploit padding validation to decrypt data
- IV Requirements: Requires cryptographically secure random IVs
- No Integrity: Provides no protection against tampering
- Predictable Patterns: Can leak information about plaintext structure
Recommended Implementation
Use AES-256-CTR for compatibility with OpenSSL enc command:
# Encryption with CTR (recommended for OpenSSL enc)
openssl enc -aes-256-ctr -pbkdf2 -iter 1 -salt -in "$INPUT_FILE" -kfile "$KEY_FILE" >> "$OUTPUT_FILE"
# Decryption with CTR (recommended for OpenSSL enc)
openssl enc -aes-256-ctr -pbkdf2 -iter 1 -d -salt -in "$ENCRYPTED_DATA" -out "$OUTPUT_FILE" -kfile "$KEY_FILE"
Note: If your OpenSSL version supports GCM/CCM in the enc command, use those modes instead for better security.
Alternative: CCM Mode
If your OpenSSL version supports CCM in the enc command:
# Encryption with CCM
openssl enc -aes-256-ccm -pbkdf2 -iter 1 -salt -in "$INPUT_FILE" -kfile "$KEY_FILE" >> "$OUTPUT_FILE"
# Decryption with CCM
openssl enc -aes-256-ccm -pbkdf2 -iter 1 -d -salt -in "$ENCRYPTED_DATA" -out "$OUTPUT_FILE" -kfile "$KEY_FILE"
Alternative: GCM Mode
If your OpenSSL version supports GCM in the enc command:
# Encryption with GCM
openssl enc -aes-256-gcm -pbkdf2 -iter 1 -salt -in "$INPUT_FILE" -kfile "$KEY_FILE" >> "$OUTPUT_FILE"
# Decryption with GCM
openssl enc -aes-256-gcm -pbkdf2 -iter 1 -d -salt -in "$ENCRYPTED_DATA" -out "$OUTPUT_FILE" -kfile "$KEY_FILE"
File Format with CTR/GCM/CCM
When using CTR, GCM, or CCM modes, the file format remains the same:
- First 32 bytes: Random salt for key derivation
- Remaining bytes: AES-256-CTR/GCM/CCM encrypted data
Compatibility Considerations
- CTR: Compatible with OpenSSL
enccommand, no padding oracle attacks - GCM/CCM: Modern, secure, but not supported by OpenSSL
enccommand in all versions - CBC: Legacy compatibility only, avoid for new systems due to padding oracle attacks
Verification
Check available cipher modes on your system:
# Check what ciphers are supported by the enc command
openssl enc -list | grep -E "(aes256)"
# Check if GCM/CCM are available (they may not be supported by enc command)
openssl list -cipher-algorithms | grep -E "(aes256-gcm|aes256-ccm)"
Usage
Setting Up the Scripts
Before using the encryption system, you need to create the encryption and decryption scripts. Copy the following code blocks and save them with the appropriate names:
1. Create the Encryption Script
Save the following content as aes_encrypt.sh:
#!/bin/bash
set -e
if [ "$#" -ne 1 ]; then
echo "Usage: $0 "
exit 1
fi
INPUT_FILE="$1"
OUTPUT_FILE="${INPUT_FILE}.enc"
SALT_FILE=$(mktemp)
KEY_FILE=$(mktemp -p /dev/shm)
# Configuration
SLOT=0
# Get HSM PIN
read -s -p "Enter HSM PIN: " HSM_PIN
echo
# Generate a random salt (32 bytes)
openssl rand -out "$SALT_FILE" 32
# Use HSM's SHA-256 to create key from salt
echo "Creating key in HSM..."
pkcs11-tool --module /usr/lib64/pkcs11/opensc-pkcs11.so \
--slot $SLOT \
--login \
--pin "$HSM_PIN" \
--hash \
--mechanism SHA256 \
--input-file "$SALT_FILE" > "$KEY_FILE"
# Encrypt the file using the derived key from HSM
echo "Encrypting file..."
# Write salt at the start of the output file (we'll need it for decryption)
cat "$SALT_FILE" > "$OUTPUT_FILE"
# Encrypt the actual data using AES-256-CTR (more secure than CBC)
openssl enc -aes-256-ctr -pbkdf2 -iter 1 -salt -in "$INPUT_FILE" -kfile "$KEY_FILE" >> "$OUTPUT_FILE"
# Clean up
shred -u "$SALT_FILE" "$KEY_FILE"
echo "File encrypted successfully: $OUTPUT_FILE"
2. Create the Decryption Script
Save the following content as aes_decrypt.sh:
#!/bin/bash
set -e
if [ "$#" -ne 1 ]; then
echo "Usage: $0 "
exit 1
fi
INPUT_FILE="$1"
OUTPUT_FILE="${INPUT_FILE%.enc}.decrypted"
SALT_FILE=$(mktemp)
ENCRYPTED_DATA=$(mktemp)
KEY_FILE=$(mktemp -p /dev/shm)
# Configuration
SLOT=0
# Get HSM PIN
read -s -p "Enter HSM PIN: " HSM_PIN
echo
# Extract salt from encrypted file
dd if="$INPUT_FILE" of="$SALT_FILE" bs=32 count=1 2>/dev/null
dd if="$INPUT_FILE" of="$ENCRYPTED_DATA" bs=1 skip=32 2>/dev/null
# Use HSM's SHA-256 to create key from salt
echo "Creating key in HSM..."
pkcs11-tool --module /usr/lib64/pkcs11/opensc-pkcs11.so \
--slot $SLOT \
--login \
--pin "$HSM_PIN" \
--hash \
--mechanism SHA256 \
--input-file "$SALT_FILE" > "$KEY_FILE"
# Decrypt the file using the derived key from HSM
echo "Decrypting file..."
openssl enc -aes-256-ctr -pbkdf2 -iter 1 -d -salt -in "$ENCRYPTED_DATA" -out "$OUTPUT_FILE" -kfile "$KEY_FILE"
# Clean up
shred -u "$SALT_FILE" "$ENCRYPTED_DATA" "$KEY_FILE"
echo "File decrypted successfully: $OUTPUT_FILE"
3. Make the Scripts Executable
After creating both files, make them executable:
chmod +x aes_encrypt.sh aes_decrypt.sh
Using the Scripts
Encrypting a File
./aes_encrypt.sh
- Prompts for HSM PIN
- Creates encrypted file with
.encextension - Original file remains unchanged
Decrypting a File
./aes_decrypt.sh
- Prompts for HSM PIN
- Creates decrypted file with
.decryptedextension - Encrypted file remains unchanged
Security Considerations
- HSM PIN Protection
- PIN is never stored in scripts
- PIN is required for both encryption and decryption
- Multiple incorrect PIN attempts may lock the HSM
- Temporary Files
- All temporary files are securely wiped using
shred - Includes salt files, key files, and intermediate data
- All temporary files are securely wiped using
- Key Handling Reality
- Important: Derived keys are temporarily stored in secure temporary files
- Keys are stored in regular temp files (not
/dev/shm) for binary data compatibility - Keys are securely wiped using
shredimmediately after use - No keys are stored permanently on disk
- The HSM's internal operations remain secure even if the host system is compromised
- Cold Boot Attack Mitigation
- Enhanced: Keys are stored in regular temp files (not
/dev/shm) - Keys are securely wiped using
shredimmediately after use - Temporary files are created in standard temp directories
- Reduced exposure compared to
/dev/shmstorage
- Enhanced: Keys are stored in regular temp files (not
- Salt Management
- Each file gets a unique random salt
- Salt is stored with the encrypted data
- 32 bytes of random data provides sufficient uniqueness
Limitations
- HSM must be available for both encryption and decryption operations
- HSM PIN must be known and HSM must be in an unlocked state
- System depends on the HSM's SHA-256 implementation
- No key rotation mechanism (each file is encrypted with its own unique key)
- Cold Boot Attack Mitigation: Keys are stored in RAM-based temp files (
/dev/shm), reducing exposure
Cold Boot Attack Mitigation
Understanding the Threat
Cold boot attacks exploit the fact that RAM contents can persist for minutes to hours after power loss, especially in cold environments. Attackers with physical access can:
- Extract RAM contents after system shutdown
- Analyze memory dumps to find encryption keys
- Recover keys that were stored in
/dev/shmfiles - Decrypt files using recovered keys
Enhanced Implementation Security
The updated scripts store derived keys in regular temporary files (not /dev/shm), which:
- ✅ Provides compatibility with binary data
- ✅ Reduces cold boot attack exposure compared to
/dev/shm - ✅ Keys are securely wiped using
shredimmediately after use - ✅ Uses standard temp directories instead of RAM-based filesystems
Mitigation Strategies
1. Current Implementation (Enhanced)
The scripts now use regular temporary files for binary data compatibility:
# Keys are stored in regular temp files (not /dev/shm)
pkcs11-tool --module /usr/lib64/pkcs11/opensc-pkcs11.so \
--slot $SLOT \
--login \
--pin "$HSM_PIN" \
--hash \
--mechanism SHA256 \
--input-file "$SALT_FILE" > "$KEY_FILE"
# Use the key file directly
openssl enc -aes-256-ctr -pbkdf2 -iter 1 -salt -in "$INPUT_FILE" -kfile "$KEY_FILE" >> "$OUTPUT_FILE"
2. Additional Security Enhancements
For even higher security, consider these additional measures:
#!/bin/bash
set -e
if [ "$#" -ne 1 ]; then
echo "Usage: $0 "
exit 1
fi
INPUT_FILE="$1"
OUTPUT_FILE="${INPUT_FILE}.enc"
SALT_FILE=$(mktemp)
# Configuration
SLOT=0
# Get HSM PIN
read -s -p "Enter HSM PIN: " HSM_PIN
echo
# Generate a random salt (32 bytes)
openssl rand -out "$SALT_FILE" 32
# Use HSM's SHA-256 to create key and use it directly
echo "Creating key in HSM..."
KEY_DATA=$(pkcs11-tool --module /usr/lib64/pkcs11/opensc-pkcs11.so \
--slot $SLOT \
--login \
--pin "$HSM_PIN" \
--hash \
--mechanism SHA256 \
--input-file "$SALT_FILE")
# Encrypt the file using the derived key directly
echo "Encrypting file..."
cat "$SALT_FILE" > "$OUTPUT_FILE"
echo "$KEY_DATA" | openssl enc -aes-256-ctr -pbkdf2 -iter 1 -salt -in "$INPUT_FILE" -kfile /dev/stdin >> "$OUTPUT_FILE"
# Clean up
shred -u "$SALT_FILE"
echo "File encrypted successfully: $OUTPUT_FILE"
3. System-Level Protections
Enable memory encryption:
# Check if memory encryption is available
dmesg | grep -i "memory encryption\|mktme\|tme"
# Enable secure boot and memory encryption in BIOS/UEFI
# Use Intel TME (Total Memory Encryption) or AMD SME if available
Use encrypted swap:
# Ensure swap is encrypted
swapon --show
# If not encrypted, consider disabling swap for sensitive operations
4. Operational Security
Physical security measures:
- Keep systems in secure locations
- Use full-disk encryption
- Implement proper shutdown procedures
- Consider using secure enclaves (Intel SGX, AMD SEV)
Operational procedures:
- Shut down systems immediately after sensitive operations
- Use dedicated machines for encryption operations
- Implement proper access controls
- Regular security audits
Current Implementation
The scripts now implement enhanced security with binary data compatibility:
- Use regular temp files - Keys stored in standard temp directories (not
/dev/shm) - Binary data compatibility - Properly handles HSM binary output
- Secure cleanup - Keys wiped using
shredimmediately after use - Reduced attack surface - No RAM-based filesystem storage
Security Trade-offs
| Approach | Cold Boot Risk | Performance | Complexity |
|---|---|---|---|
| Regular Temp Files (Current) | Medium | Fast | Low |
| Memory Encryption | Low | Medium | High |
| Dedicated Hardware | Very Low | Fast | Very High |
The current implementation provides a good balance of security and simplicity. For high-security environments, consider additional system-level protections like memory encryption.
Complete Implementation
The encryption system is implemented through two bash scripts that handle:
- Secure PIN input - Prompts for HSM PIN without displaying it
- Random salt generation - Creates unique 32-byte salts for each file
- HSM-based key derivation - Uses HSM's SHA-256 mechanism for key creation
- File encryption/decryption - Uses AES-256-CTR for secure encryption
- Secure cleanup - Wipes all temporary files using
shred
The complete scripts are provided in the Usage section above, where you can copy them and save as aes_encrypt.sh and aes_decrypt.sh.
TODO List
Hardware Testing
- YubiHSM 2 FIPS Integration
- Test compatibility with YubiHSM 2 FIPS
- Verify SHA-256 mechanism availability and behavior
- Test key derivation performance
- Document any required modifications for YubiHSM 2 support
- Test network sharing capabilities if relevant
- Validate PKCS#11 module path and configuration
- Multiple Key Testing
- Test behavior with multiple ECDSA keys
- Verify key selection logic
- Document needed modifications
- Test key ID specification requirements
YubiHSM 2 and Cold Boot Attack Prevention
Overview
While the current system uses SmartCard-HSM devices for key derivation with host-based encryption, YubiHSM 2 offers a fundamentally different approach that can provide complete protection against cold boot attacks by performing all cryptographic operations within the HSM hardware itself.
YubiHSM 2 Hardware-Based Encryption
YubiHSM 2 supports direct AES encryption and decryption operations within the HSM hardware, available with firmware version 2.3.1 or later:
Supported Operations
- AES-128/192/256 CBC Mode:
encrypt-cbcanddecrypt-cbccommands - AES-128/192/256 ECB Mode:
encrypt-ecbanddecrypt-ecbcommands - Hardware-accelerated: All operations performed within the secure hardware boundary
Cold Boot Attack Prevention
This hardware-based approach provides complete protection against cold boot attacks:
- ✅ Keys Never Leave the HSM: AES keys are stored and used entirely within the secure hardware
- ✅ Encryption in Hardware: All cryptographic operations occur inside the HSM's secure boundary
- ✅ No Host Memory Exposure: Plaintext and keys are never loaded into the host system's RAM
- ✅ Hardware Isolation: Even if the host system is compromised, the HSM's internal operations remain secure
- ✅ FIPS 140-2 Level 3: Certified tamper-resistant hardware with physical security protections
Implementation Differences
Current System (SmartCard-HSM)
# Key derivation in HSM, encryption on host
pkcs11-tool --hash --mechanism SHA256 --input-file salt.bin > key.bin
openssl enc -aes-256-ctr -kfile key.bin -in file.txt -out file.enc
YubiHSM 2 Approach
# Generate AES key in HSM
yubihsm-shell -a generate-symmetric-key -l "file-key" -d 1 -c encrypt-cbc,decrypt-cbc -A aes256
# Encrypt data directly in HSM (chunked)
yubihsm-shell -a encrypt-cbc -i -s -i data=
Data Size Limitations
Important Constraint: YubiHSM 2 has a maximum data size of approximately 2021 bytes per operation. This means:
- ✅ Small files: Can be encrypted entirely within the HSM
- ❌ Large files: Must be processed in ~2KB chunks
- ⚠️ Performance impact: Chunked processing adds complexity and latency
Security Comparison
| Aspect | Current System (SmartCard-HSM) | YubiHSM 2 |
|---|---|---|
| Key Derivation | ✅ HSM-based SHA-256 | ✅ HSM-based SHA-256 |
| Key Storage | ✅ Hardware protected | ✅ Hardware protected |
| File Encryption | ❌ Host-based (OpenSSL) | ✅ Hardware-based (limited size) |
| Cold Boot Protection | ⚠️ Partial (keys briefly in temp files) | ✅ Complete (all operations in HSM) |
| Large File Support | ✅ Full support | ❌ Limited to ~2KB chunks |
| Implementation Complexity | ✅ Simple | ❌ Complex (chunking required) |
Practical Considerations
Advantages of YubiHSM 2
- Complete cold boot attack prevention
- All cryptographic operations in hardware
- No key exposure to host memory
- FIPS 140-2 Level 3 certification
- Superior security model
Disadvantages of YubiHSM 2
- Limited to ~2KB chunks per operation
- More complex implementation for large files
- Higher latency due to chunked processing
- Requires custom chunking logic
- Not suitable for streaming encryption
Recommendation
For environments where cold boot attacks are the primary security concern, YubiHSM 2 provides superior protection by keeping all cryptographic operations within the secure hardware boundary. However, the current SmartCard-HSM approach offers a better balance of security and practicality for general file encryption use cases.
Choose YubiHSM 2 if:
- Cold boot attacks are your primary threat model
- You can accept the complexity of chunked processing
- You're working with smaller files or can implement chunking
- You require FIPS 140-2 Level 3 compliance
Choose SmartCard-HSM if:
- You need simple, practical file encryption
- You work with large files frequently
- You want straightforward implementation
- You accept the minimal cold boot attack exposure of temporary key storage
Dependencies
- OpenSC PKCS#11 module
- OpenSSL for AES encryption and random number generation
- Standard Unix tools (dd, cat, shred)
- Bash shell
For more detailed instructions about SmartCard-HSM initialization and key generation, refer to SmartCard-HSM USB Token Setup Guide.
Usage Notes
- If the HSM is locked, add
--loginand provide a PIN:pkcs11-tool --module /usr/lib64/pkcs11/opensc-pkcs11.so --login --list-objects - Use this command to:
- Verify key installation and parameters
- Check certificate details
- Audit HSM object configuration
- Confirm key availability before operations
