Configure OpenSSL for signing with PKCS11 integration

OpenSSL is a versatile open-source cryptography library that provides a set of tools and libraries for secure communications and digital signatures. Integrate DigiCert® Software Trust Manager PKCS11 library with OpenSSL to sign.

Caution

Scan your systems for uses of OpenSSL 3.0 and above, and if you find any instances, upgrade to 3.0.7. See OpenSSL releases patch for high level vulnerability in versions 3.0 and above.

Tip

OpenSSL does not support the following characters in sign commands: ; ! ‘ ( ) [ &

To avoid errors, remove unsupported characters from file paths before attempting to sign.

Prerequisites

Windows, Linux or macOS operating system
OpenSSL version 1.x.x
Note
OpenSSL 3.x.x is onlycompatible with Linux.
OpenSSL PKCS#11 engine
DigiCert® Software Trust Manager PKCS11 library

Install OpenSSL

To install OpenSSL and the OpenSSL PKCS#11 engine based on your operating system:

Windows

Download and install OpenSSL.
Manually compile the OpenSSL PKCS11 using one of the following methods:
Example 1. MSVC
To build libp11 git clone https://github.com/OpenSC/libp11.git:
Start a Visual Studio Command prompt.
Navigate to the libp11 directory.
Run:
nmake -f Makefile.mak
Once OpenSSL is installed, open a new directory and run:
nmake -f Makefile.mak OPENSSL_DIR=\your\openssl\directory
For x64 bit builds: open the Native x64 Visual Studio Command Prompt and run:
nmake /f Makefile.mak OPENSSL_DIR=c:\OpenSSL-Win64 BUILD_FOR=WIN64
Note
If any of your builds fail for any reason, clean the src directory of obj files before re-attempting the build.
Example 2. MSYS2
To build libp11:
Download and install msys2-i686-*.exe from https://msys2.org.
Start a MSYS2 MSYS console from the Start menu
Use:
pacman -S git pkg-config libtool autoconf automake make gcc openssl-devel git clone https://github.com/OpenSC/libp11.git cd libp11 autoreconf -fi ./configure --prefix=/usr/local make && make install
Example 3. Cygwin
To build libp11:
Download and install msys2-i686-*.exe from https://msys2.org.
Start a MSYS2 MSYS console from the Start menu
Use:
pacman -S git pkg-config libtool autoconf automake make gcc openssl-devel git clone https://github.com/OpenSC/libp11.git cd libp11 autoreconf -fi ./configure --prefix=/usr/local make && make install
Example 4. MinGW/MSYS
To build libp11:
Download and install mingw-get-setup.exe from https://osdn.net/projects/mingw/.
Note
Ensure that you have selected all necessary MinGW and MSYS packages during install.
Install:
pkg-config or pkg-config-lite
Update autoconf
Update OpenSSL
To configure OpenSSL to replace very old mingw’s version, run:
./configure --prefix=/mingw threads shared mingw make depend && make && make install
To download and unpack libp11 in its directory, run:
libtoolize --force aclocal -I m4 --install autoheader automake --force-missing --add-missing autoconf ./configure --prefix=/usr/local make && make install
Example 5. MinGW cross-compile on a Unix host
Example configuration for a 64-bit OpenSSL installed in /opt/openssl-mingw64:
```
PKG_CONFIG_PATH=/opt/openssl-mingw64/lib/pkgconfig ./configure --host=x86_64-w64-mingw32 --prefix=/opt/libp11-mingw64            
```
Example configuration for a 32-bit OpenSSL installed in /opt/openssl-mingw:
```
PKG_CONFIG_PATH=/opt/openssl-mingw/lib/pkgconfig ./configure --host=i686-w64-mingw32 --prefix=/opt/libp11-mingw
```
To build and install, run:
```
make && sudo make install            
```

Linux

To install OpenSSL, OpenSSL PKCS11 engine and P11tool, run:

macOS

To install OpenSSL, OpenSSL PKCS11 engine and P11tool, run:

To install the prerequisites, download this install_mac_prerequisites.sh script.
In Terminal, run the following command to make the script executable:
```
chmod +x install_mac_prerequisites.sh

#Run the script
./install_mac_prerequisites.sh
```
Note
You will be prompted to enter your password. This is required because the command uses sudo, which stands for "superuser do" and gives the command higher privileges.
Once the script completes, pkcs11.dylib will be present in the openssl@1.1 folder under lib/engines-1.1.
Note
Example location: /opt/homebrew/Cellar/openssl@1.1/1.1.1w/lib/engines-1.1/pkcs11.dylib
The openssl@1.1 location may vary according to the macOS architecture that you are using.

Download and configure PKCS11 library

A configuration file is required for OpenSSL PKCS#11 engine to use Software Trust Manager PKCS11 library. This file is required in related sign commands.

Download PKCS11 library

To download the Software Trust Manager PKCS11 library:

Sign in to DigiCert ONE.
In the manager menu (top right), select Software Trust.
Navigate to: Resources > Client tool repository.
Select your operating system.
Click the download icon next to PKCS11 Library.

Create configuration file

To create the configuration file for PKCS11:

Open an integrated development environment (IDE) or plain text editor.
Name the file as openssl.conf.
Copy and paste the following text for your operating system into the editor:

Example 8. Windows

openssl_conf = openssl_init
[openssl_init]
engines = engine_section
[engine_section]
pkcs11 = pkcs11_section
[pkcs11_section]

#Path to the Compiled OpenSSL PKCS11 from OpenSC - libp11
dynamic_path = <Path to compiled libp11 pkcs11.dll>

MODULE_PATH = <Path to smpkcs11.DLL>

Example 9. Linux

openssl_conf = openssl_init
[openssl_init]
engines = engine_section
[engine_section]
pkcs11 = pkcs11_section
[pkcs11_section]

#Path to the OpenSSL PKCS11 Engine
dynamic_path = "<Path to libpkcs11.so>
"

MODULE_PATH = <Path to smpkcs11.so>

Note

The smpkcs11.so path depends on where you stored the file.
The libpkcs11.so path depends on your Linux distribution and version:
- Ubuntu 18.04: /usr/lib/x86_64-linux-gnu/engines-1.1/libpkcs11.so
- Ubuntu 20.04: /usr/lib/x86_64-linux-gnu/engines-1.1/libpkcs11.so
- Ubuntu 22.04: /usr/lib/x86_64-linux-gnu/engines-3/libpkcs11.so
- RHEL 8: /usr/lib64/engines-1.1/libpkcs11.so
- RHEL 9: /usr/lib64/engines-3/libpkcs11.so

Example 10. macOS

dynamic_path = <Path to compiled pkcs11.dylib>

MODULE_PATH = <Path to smpkcs11.dylib>

Set environment variable for dc-openssl.conf

The OPENSSL_CONF environment variable must be set with the value of the path to openssl.conf.

To set the OPENSSL_CONF environment variable, add:

Example 11. Windows

Command:

set OPENSSL_CONF=<path to openssl.conf>

Command sample:

set OPENSSL_CONF=C:\Program Files\DigiCert\DigiCert One Signing Manager Tools\dc-openssl.conf

Example 12. Linux

Command:

export OPENSSL_CONF=<path to openssl.conf>

Command sample:

export OPENSSL_CONF=/home/<user>/smtools-linux-x64/dc-openssl.conf

Note

Environment variables will be reset upon logging out of the current session. To set them permanently. add the line to /home/<user>/.bashrc.

Example 13. macOS

Command:

export OPENSSL_CONF=<path to openssl.conf>

Command sample:

export OPENSSL_CONF=/home/<user>/smtools-mac-x64/dc-openssl.conf

Configure OpenSSL for signing with PKCS11 integration

Caution

Tip

Prerequisites

Note

Install OpenSSL

Windows

Note

Note

Linux

Note

macOS

Note

Note

Download and configure PKCS11 library

Download PKCS11 library

Create configuration file

Note

Set environment variable for dc-openssl.conf

Note

Related articles