Skip to main content

Verify signature with OpenSSL pkeyutl using PKCS11 library

OpenSSL pkeyutl performs low-level public key operations using any supported algorithm. Integrate the DigiCert​​®​​ Software Trust Manager PKCS11 library with OpenSSL to verify signatures using OpenSSL pkeyutl.

Prerequisites

Verify signature with OpenSSL pkeyutl

When you use OpenSSL dgst to sign, it digests the data internally. However, you cannot use OpenSSL pkeyutl to verify the original data with the digest produced by dgst. You must manually generate the hash as shown below and use has as the input value in the verify signature command for OpenSSL pkeyutl.

  1. To sign using OpenSSL dgst, run:

    $openssl dgst -engine <private key storage> -keyform engine -sign "pkcs11:object=<keypair alias>;type=private" -sha256 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:32 -sigopt rsa_mgf1_md:sha256 -out <file name after signed>
    engine "pkcs11" set.

    Command sample:

    $openssl dgst -engine pkcs11 -keyform engine -sign "pkcs11:object=keypair-1;type=private" -sha256 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:32 -sigopt rsa_mgf1_md:sha256 -out signature.bin sample.bin
    engine "pkcs11" set.
  2. To generate the hash using OpenSSL dgst, run:

    $openssl dgst -sha256 -binary sample.bin > hashed_output.bin

    Command sample:

    $openssl dgst -sha256 -binary sample.bin > hashed_output.bin
  3. To verify your signature using OpenSSL pkeyutl, run:

    $openssl pkeyutl -verify -in <hashed file> -sigfile <signed file> -pkeyopt rsa_padding_mode:pss -pkeyopt digest:sha256 -pubin -inkey keypair-1.pem
    

    Command sample

    $openssl pkeyutl -verify -in hashed_sample.bin -sigfile signature.bin -pkeyopt rsa_padding_mode:pss -pkeyopt digest:sha256 -pubin -inkey keypair-1.pem
    

Troubleshooting

Error:

$openssl pkeyutl -verify -in sample.bin -sigfile signature.bin -pkeyopt rsa_padding_mode:pss -pkeyopt digest:sha256 -pubin -inkey keypair-1.pem
Error: The input data looks too long to be a hash

Description: pkeyutl expects a digest as an input file to verify the signature.

Solution: Use the instructions above.