Delegate random number generation to crypto provider (fixes #418)

ChangeLog

@@ -1,5 +1,13 @@
 2020-04-06  Jay Berkenbilt  <ejb@ql.org>
 
+	* Move random number generation into the crypto providers. The old
+	os-based secure random number generation with fallback to insecure
+	random number generation (only if allowed at build time) has moved
+	into the native crypto provider. If using other providers
+	(currently gnutls or openssl), random number generation will use
+	those libraries. The old interfaces for supplying your own random
+	number generator are still in place. Fixes #418.
+
 	* Source-level incompatibility: remove QUtil::srandom. There was
 	no reason to ever call this, and it didn't do anything unless
 	insecure random number generation was compiled in, which it is not

README.md

@@ -111,7 +111,9 @@ If you are packaging qpdf for a distribution and preparing a build that is run b
 
 # Random Number Generation
 
-By default, when `qpdf` detects either the Windows cryptography API or the existence of `/dev/urandom`, `/dev/arandom`, or `/dev/random`, it uses them to generate cryptography secure random numbers.  If none of these conditions are true, the build will fail with an error.  This behavior can be modified in several ways:
+By default, qpdf uses the crypto provider for generating random numbers. The rest of this applies only if you are using the native crypto provider.
+
+If the native crypto provider is in use, then, when `qpdf` detects either the Windows cryptography API or the existence of `/dev/urandom`, `/dev/arandom`, or `/dev/random`, it uses them to generate cryptographically secure random numbers.  If none of these conditions are true, the build will fail with an error.  This behavior can be modified in several ways:
 * If you configure with `--disable-os-secure-random` or define `SKIP_OS_SECURE_RANDOM`, qpdf will not attempt to use Windows cryptography or the random device.  You must either supply your own random data provider or allow use of insecure random numbers.
 * If you configure qpdf with the `--enable-insecure-random` option or define `USE_INSECURE_RANDOM`, qpdf will try insecure random numbers if OS-provided secure random numbers are disabled.  This is not a fallback.  In order for insecure random numbers to be used, you must also disable OS secure random numbers since, otherwise, failure to find OS secure random numbers is a compile error.  The insecure random number source is stdlib's `random()` or `rand()` calls. These random numbers are not cryptography secure, but the qpdf library is fully functional using them.  Using non-secure random numbers means that it's easier in some cases to guess encryption keys.  If you're not generating encrypted files, there's no advantage to using secure random numbers.
 * In all cases, you may supply your own random data provider.  To do this, derive a class from `qpdf/RandomDataProvider` (since version 5.1.0) and call `QUtil::setRandomDataProvider` before you create any `QPDF` objects.  If you supply your own random data provider, it will always be used even if support for one of the other random data providers is compiled in.  If you wish to avoid any possibility of your build of qpdf from using anything but a user-supplied random data provider, you can define `SKIP_OS_SECURE_RANDOM` and not `USE_INSECURE_RANDOM`.  In this case, qpdf will throw a runtime error if any attempt is made to generate random numbers and no random data provider has been supplied.

include/qpdf/QPDFCryptoImpl.hh

@@ -41,6 +41,11 @@ class QPDF_DLL_CLASS QPDFCryptoImpl
     QPDF_DLL
     virtual ~QPDFCryptoImpl() = default;
 
+    // Random Number Generation
+
+    QPDF_DLL
+    virtual void provideRandomData(unsigned char* data, size_t len) = 0;
+
     // Hashing
 
     typedef unsigned char MD5_Digest[16];

include/qpdf/QUtil.hh

@@ -263,35 +263,25 @@ namespace QUtil
     QPDF_DLL
     std::vector<std::string> possible_repaired_encodings(std::string);
 
-    // If secure random number generation is supported on your
-    // platform and qpdf was not compiled with insecure random number
-    // generation, this returns a cryptographically secure random
-    // number.  Otherwise it falls back to random from stdlib and
-    // calls srandom automatically the first time it is called.
+    // Return a cryptographically secure random number.
     QPDF_DLL
     long random();
 
-    // Initialize a buffer with random bytes.  By default, qpdf tries
-    // to use a secure random number source.  It can be configured at
-    // compile time to use an insecure random number source (from
-    // stdlib).  You can also call setRandomDataProvider with a
-    // RandomDataProvider, in which case this method will get its
-    // random bytes from that.
-
+    // Initialize a buffer with cryptographically secure random bytes.
     QPDF_DLL
     void initializeWithRandomBytes(unsigned char* data, size_t len);
 
-    // Supply a random data provider.  If not supplied, depending on
-    // compile time options, qpdf will either use the operating
-    // system's secure random number source or an insecure random
-    // source from stdlib.  The caller is responsible for managing the
-    // memory for the RandomDataProvider.  This method modifies a
-    // static variable.  If you are providing your own random data
-    // provider, you should call this at the beginning of your program
-    // before creating any QPDF objects.  Passing a null to this
-    // method will reset the library back to whichever of the built-in
-    // random data handlers is appropriate based on how qpdf was
-    // compiled.
+    // Supply a random data provider. Starting in qpdf 10.0.0, qpdf
+    // uses the crypto provider as its source of random numbers. If
+    // you are using the native crypto provider, then qpdf will either
+    // use the operating system's secure random number source or, only
+    // if enabled at build time, an insecure random source from
+    // stdlib. The caller is responsible for managing the memory for
+    // the RandomDataProvider. This method modifies a static variable.
+    // If you are providing your own random data provider, you should
+    // call this at the beginning of your program before creating any
+    // QPDF objects. Passing a null to this method will reset the
+    // library back to its default random data provider.
     QPDF_DLL
     void setRandomDataProvider(RandomDataProvider*);
 

libqpdf/CryptoRandomDataProvider.cc

@@ -0,0 +1,24 @@
+#include <qpdf/CryptoRandomDataProvider.hh>
+#include <qpdf/QPDFCryptoProvider.hh>
+
+CryptoRandomDataProvider::CryptoRandomDataProvider()
+{
+}
+
+CryptoRandomDataProvider::~CryptoRandomDataProvider()
+{
+}
+
+void
+CryptoRandomDataProvider::provideRandomData(unsigned char* data, size_t len)
+{
+    auto crypto = QPDFCryptoProvider::getImpl();
+    crypto->provideRandomData(data, len);
+}
+
+RandomDataProvider*
+CryptoRandomDataProvider::getInstance()
+{
+    static CryptoRandomDataProvider instance;
+    return &instance;
+}

libqpdf/QPDFCrypto_gnutls.cc

@@ -29,6 +29,18 @@ QPDFCrypto_gnutls::~QPDFCrypto_gnutls()
     this->aes_key_len = 0;
 }
 
+void
+QPDFCrypto_gnutls::provideRandomData(unsigned char* data, size_t len)
+{
+    int code = gnutls_rnd (GNUTLS_RND_KEY, data, len);
+    if (code < 0)
+    {
+	throw std::runtime_error(
+	    std::string("gnutls: random number generation error: ") +
+            std::string(gnutls_strerror(code)));
+    }
+}
+
 void
 QPDFCrypto_gnutls::MD5_init()
 {

libqpdf/QPDFCrypto_native.cc

@@ -1,6 +1,41 @@
 #include <qpdf/QPDFCrypto_native.hh>
 #include <qpdf/QUtil.hh>
 
+#ifdef USE_INSECURE_RANDOM
+# include <qpdf/InsecureRandomDataProvider.hh>
+#endif
+#include <qpdf/SecureRandomDataProvider.hh>
+
+static RandomDataProvider* getRandomProvider()
+{
+#ifdef USE_INSECURE_RANDOM
+    static RandomDataProvider* insecure_random_data_provider =
+        InsecureRandomDataProvider::getInstance();
+#else
+    static RandomDataProvider* insecure_random_data_provider = 0;
+#endif
+    static RandomDataProvider* secure_random_data_provider =
+        SecureRandomDataProvider::getInstance();
+
+    static RandomDataProvider* provider = (
+        secure_random_data_provider ? secure_random_data_provider
+        : insecure_random_data_provider ? insecure_random_data_provider
+        : 0);
+
+    if (provider == 0)
+    {
+        throw std::logic_error("QPDFCrypto_native has no random data provider");
+    }
+
+    return provider;
+}
+
+void
+QPDFCrypto_native::provideRandomData(unsigned char* data, size_t len)
+{
+    getRandomProvider()->provideRandomData(data, len);
+}
+
 void
 QPDFCrypto_native::MD5_init()
 {

libqpdf/QPDFCrypto_openssl.cc

@@ -38,6 +38,12 @@ QPDFCrypto_openssl::~QPDFCrypto_openssl()
     EVP_MD_CTX_free(md_ctx);
 }
 
+void
+QPDFCrypto_openssl::provideRandomData(unsigned char* data, size_t len)
+{
+    check_openssl(RAND_bytes(data, QIntC::to_int(len)));
+}
+
 void
 QPDFCrypto_openssl::MD5_init()
 {

libqpdf/QUtil.cc

@@ -3,10 +3,7 @@
 
 #include <qpdf/QUtil.hh>
 #include <qpdf/PointerHolder.hh>
-#ifdef USE_INSECURE_RANDOM
-# include <qpdf/InsecureRandomDataProvider.hh>
-#endif
-#include <qpdf/SecureRandomDataProvider.hh>
+#include <qpdf/CryptoRandomDataProvider.hh>
 #include <qpdf/QPDFSystemError.hh>
 #include <qpdf/QTC.hh>
 #include <qpdf/QIntC.hh>
@@ -891,29 +888,9 @@ class RandomDataProviderProvider
 };
 
 RandomDataProviderProvider::RandomDataProviderProvider() :
-    default_provider(0),
+    default_provider(CryptoRandomDataProvider::getInstance()),
     current_provider(0)
 {
-#ifdef USE_INSECURE_RANDOM
-    static RandomDataProvider* insecure_random_data_provider =
-        InsecureRandomDataProvider::getInstance();
-#else
-    static RandomDataProvider* insecure_random_data_provider = 0;
-#endif
-    static RandomDataProvider* secure_random_data_provider =
-        SecureRandomDataProvider::getInstance();
-
-    this->default_provider = (
-        secure_random_data_provider ? secure_random_data_provider
-        : insecure_random_data_provider ? insecure_random_data_provider
-        : 0);
-
-    // QUtil.hh has comments indicating that getRandomDataProvider(),
-    // which calls this method, never returns null.
-    if (this->default_provider == 0)
-    {
-        throw std::logic_error("QPDF has no random data provider");
-    }
     this->current_provider = default_provider;
 }
 

libqpdf/build.mk

@@ -27,6 +27,7 @@ SRCS_libqpdf = \
 	libqpdf/BufferInputSource.cc \
 	libqpdf/ClosedFileInputSource.cc \
 	libqpdf/ContentNormalizer.cc \
+	libqpdf/CryptoRandomDataProvider.cc \
 	libqpdf/FileInputSource.cc \
 	libqpdf/InputSource.cc \
 	libqpdf/InsecureRandomDataProvider.cc \

libqpdf/qpdf/CryptoRandomDataProvider.hh

@@ -0,0 +1,22 @@
+#ifndef CRYPTORANDOMDATAPROVIDER_HH
+#define CRYPTORANDOMDATAPROVIDER_HH
+
+#include <qpdf/RandomDataProvider.hh>
+#include <qpdf/DLL.h>
+
+class CryptoRandomDataProvider: public RandomDataProvider
+{
+  public:
+    QPDF_DLL
+    CryptoRandomDataProvider();
+    QPDF_DLL
+    virtual ~CryptoRandomDataProvider();
+
+    QPDF_DLL
+    virtual void provideRandomData(unsigned char* data, size_t len);
+
+    QPDF_DLL
+    static RandomDataProvider* getInstance();
+};
+
+#endif // CRYPTORANDOMDATAPROVIDER_HH

libqpdf/qpdf/QPDFCrypto_gnutls.hh

@@ -15,6 +15,8 @@ class QPDFCrypto_gnutls: public QPDFCryptoImpl
     QPDF_DLL
     virtual ~QPDFCrypto_gnutls();
 
+    virtual void provideRandomData(unsigned char* data, size_t len);
+
     virtual void MD5_init();
     virtual void MD5_update(unsigned char const* data, size_t len);
     virtual void MD5_finalize();

libqpdf/qpdf/QPDFCrypto_native.hh

@@ -17,6 +17,8 @@ class QPDFCrypto_native: public QPDFCryptoImpl
     QPDF_DLL
     virtual ~QPDFCrypto_native() = default;
 
+    virtual void provideRandomData(unsigned char* data, size_t len);
+
     virtual void MD5_init();
     virtual void MD5_update(unsigned char const* data, size_t len);
     virtual void MD5_finalize();

libqpdf/qpdf/QPDFCrypto_openssl.hh

@@ -9,6 +9,7 @@
 #else
 #include <openssl/evp.h>
 #endif
+#include <openssl/rand.h>
 
 class QPDFCrypto_openssl: public QPDFCryptoImpl
 {
@@ -18,6 +19,8 @@ class QPDFCrypto_openssl: public QPDFCryptoImpl
     QPDF_DLL
     ~QPDFCrypto_openssl() override;
 
+    void provideRandomData(unsigned char* data, size_t len) override;
+
     void MD5_init() override;
     void MD5_update(unsigned char const* data, size_t len) override;
     void MD5_finalize() override;

libtests/qtest/random.test

@@ -7,10 +7,30 @@ require TestDriver;
 
 my $td = new TestDriver('random');
 
-$td->runtest("Random Data Providers",
-	     {$td->COMMAND => "random"},
-	     {$td->STRING => "random: end of tests\n",
-	      $td->EXIT_STATUS => 0},
-	     $td->NORMALIZE_NEWLINES);
+my @providers = ();
+if (exists $ENV{'QPDF_CRYPTO_PROVIDER'})
+{
+    push(@providers, $ENV{'QPDF_CRYPTO_PROVIDER'});
+}
+else
+{
+    open(Q, "qpdf --show-crypto|") or die;
+    while (<Q>)
+    {
+        s/\s+$//s;
+        push(@providers, $_);
+    }
+    close(Q);
+}
+foreach my $p (@providers)
+{
+    $ENV{'QPDF_CRYPTO_PROVIDER'} = $p;
 
-$td->report(1);
+    $td->runtest("Random Data Providers ($p)",
+                 {$td->COMMAND => "random"},
+                 {$td->STRING => "random: end of tests\n",
+                      $td->EXIT_STATUS => 0},
+                 $td->NORMALIZE_NEWLINES);
+}
+
+$td->report(scalar(@providers));

manual/qpdf-manual.xml

@@ -3914,25 +3914,16 @@ outfile.pdf</option>
    <title>Random Number Generation</title>
    <para>
     QPDF generates random numbers to support generation of encrypted
-    data.  Versions prior to 5.0.1 used <function>random</function> or
-    <function>rand</function> from <filename>stdlib</filename> to
-    generate random numbers.  Version 5.0.1, if available, used
-    operating system-provided secure random number generation instead,
-    enabling use of <filename>stdlib</filename> random number
-    generation only if enabled by a compile-time option.  Starting in
-    version 5.1.0, use of insecure random numbers was disabled unless
-    enabled at compile time.  Starting in version 5.1.0, it is also
-    possible for you to disable use of OS-provided secure random
-    numbers.  This is especially useful on Windows if you want to
-    avoid a dependency on Microsoft's cryptography API.  In this case,
-    you must provide your own random data provider.  Regardless of how
-    you compile qpdf, starting in version 5.1.0, it is possible for
-    you to provide your own random data provider at runtime.  This
-    would enable you to use some software-based secure pseudorandom
-    number generator and to avoid use of whatever the operating system
-    provides.  For details on how to do this, please refer to the
-    top-level README.md file in the source distribution and to comments
-    in <filename>QUtil.hh</filename>.
+    data. Starting in qpdf 10.0.0, qpdf uses the crypto provider as
+    its source of random numbers. Older versions used the OS-provided
+    source of secure random numbers or, if allowed at build time,
+    insecure random numbers from stdlib. Starting with version 5.1.0,
+    you can disable use of OS-provided secure random numbers at build
+    time. This is especially useful on Windows if you want to avoid a
+    dependency on Microsoft's cryptography API. You can also supply
+    your own random data provider. For details on how to do this,
+    please refer to the top-level README.md file in the source
+    distribution and to comments in <filename>QUtil.hh</filename>.
    </para>
   </sect1>
   <sect1 id="ref.adding-and-remove-pages">
@@ -4906,6 +4897,14 @@ print "\n";
         Library Enhancements
        </para>
        <itemizedlist>
+        <listitem>
+         <para>
+          Random number generation is now delegated to the crypto
+          provider. The old behavior is still used by the native
+          crypto provider. It is still possible to provide your own
+          random number generator.
+         </para>
+        </listitem>
         <listitem>
          <para>
           Add a new version of