2
1
mirror of https://github.com/qpdf/qpdf.git synced 2024-12-22 10:58:58 +00:00

Convert README files to markdown

This commit is contained in:
Jay Berkenbilt 2017-08-22 13:23:49 -04:00
parent 6219111ed7
commit 2a8cd4acdc
5 changed files with 200 additions and 576 deletions

View File

@ -1,49 +1,46 @@
Avoiding operator[]
===================
# Avoiding `operator[]`
During a security review by Red Hat security team (specifically
Florian Weimer), it was discovered that qpdf used std::string and
std::vector's operator[], which has no bounds checking by design.
Instead, using those objects' at() method is preferable since it does
bounds checking. Florian has a tool that can detect all uses of these
methods and report them. I have a short perl script that
automatically corrects any such uses. The perl script is not intended
to be general, but it could be reasonably general. The only known
shortcut is that it might not work very well with some cases of nested
[]'s like a[b[c]] or with cases where there are line breaks inside the
brackets. For qpdf's coding style, it worked on all cases reported.
During a security review by Red Hat security team (specifically Florian Weimer), it was discovered that qpdf used `std::string` and `std::vector`'s `operator[]`, which has no bounds checking by design. Instead, using those objects' `at()` method is preferable since it does bounds checking. Florian has a tool that can detect all uses of these methods and report them. I have a short perl script that automatically corrects any such uses. The perl script is not intended to be general, but it could be reasonably general. The only known shortcut is that it might not work very well with some cases of nested `[]`'s like `a[b[c]]` or with cases where there are line breaks inside the brackets. For qpdf's coding style, it worked on all cases reported.
To use this, obtain htcondor-analyzer, run it, and respond to the
report. Here's what I did.
To use this, obtain htcondor-analyzer, run it, and respond to the report. Here's what I did.
```
sudo aptitude install libclang-dev llvm llvm-dev clang
cd /tmp
git clone https://github.com/fweimer/htcondor-analyzer
# HEAD = 5fa06fc68a9b0677e9de162279185d58ba1e8477 at this writing
cd htcondor-analyzer
make
```
in qpdf
In qpdf:
```
./autogen.sh
/tmp/htcondor-analyzer/create-db
CC=/tmp/htcondor-analyzer/cc CXX=/tmp/htcondor-analyzer/cxx ./configure --disable-shared --disable-werror
# to remove conftest.c
\rm htcondor-analyzer.sqlite
/tmp/htcondor-analyzer/create-db
```
Repeat until no more errors:
```
/tmp/fix-at.pl is shown below.
```
```
make
/tmp/htcondor-analyzer/report | grep std:: | grep qpdf >| /tmp/r
perl /tmp/fix-at.pl /tmp/r
# move all *.new over the original file. patmv is my script. Can
# also use a for loop.
patmv -f s/.new// **/*.new
```
---------- /tmp/fix-at.pl ----------
/tmp/fix-at.pl:
```perl
#!/usr/bin/env perl
require 5.008;
use warnings;
@ -90,4 +87,4 @@ foreach my $file (sort keys %to_fix)
}
close(F) or die;
}
--------------------
```

View File

@ -1,178 +1,91 @@
Release Reminders
=================
* Test for binary compatility. The easiest way to do this is to check
out the last release, run the test suite, check out the new
release, run make build_libqpdf, check out the old release, and run
make check NO_REBUILD=1.
* When making a release, always remember to run large file tests and
image comparison tests (--enable-test-compare-images
--with-large-file-test-path=/path). For Windows, use a Windows
style path, not an MSYS path for large files. For a major release,
consider running a spelling checker over the source code to catch
errors in variable names, strings, and comments. Use ispell -p
ispell-words.
# Release Reminders
* Test for binary compatility. The easiest way to do this is to check out the last release, run the test suite, check out the new release, run `make build_libqpdf`, check out the old release, and run `make check NO_REBUILD=1`.
* When making a release, always remember to run large file tests and image comparison tests (`--enable-test-compare-images` `--with-large-file-test-path=/path`). For Windows, use a Windows style path, not an MSYS path for large files. For a major release, consider running a spelling checker over the source code to catch errors in variable names, strings, and comments. Use `ispell -p ispell-words`.
* Run tests with sanitize address enabled:
```
./configure CFLAGS="-fsanitize=address -g" \
CXXFLAGS="-fsanitize=address -g" \
LDFLAGS="-fsanitize=address" \
--enable-werror --disable-shared
As of gcc 6.3.0, this exposes some good things but appears to also
have some false positive leak reports. Valgrind is more reliable
but also may miss some things that this catches.
* Consider running tests with latest gcc and/or valgrind. To test
with valgrind:
```
As of gcc 6.3.0, this exposes some good things but appears to also have some false positive leak reports. Valgrind is more reliable but also may miss some things that this catches.
* Consider running tests with latest gcc and/or valgrind. To test with valgrind:
```
./configure --disable-shared
make -j8 -k VALGRIND=1
make -k check NO_REBUILD=1
This moves each binary into a subdirectory and replaces it with a
link to make/exec-z. See make/exec-z.
```
This moves each binary into a subdirectory and replaces it with a link to make/exec-z. See make/exec-z.
* Check all open issues in the sourceforge trackers and on github.
* If any interfaces were added or changed, check C API to see whether
changes are appropriate there as well. If necessary, review the
casting policy in the manual, and ensure that integer types are
properly handled.
* Remember to avoid using operator[] with std::string or
std::vector. See README-hardening.md for details.
* Increment shared library version information as needed (LT_* in
configure.ac)
* Update release notes in manual -- look at diffs and ChangeLog
* Add a release entry to ChangeLog
* Make sure version numbers are consistent in the following
locations:
configure.ac
libqpdf/QPDF.cc
manual/qpdf-manual.xml
make_dist verifies this consistency.
* Update release date in manual/qpdf-manual.xml. Remember to ensure
that the entities at the top of the document are consistent with
the release notes for both version and release date.
* Check TODO file to make sure all planned items for the release are
done or retargeted.
* Each year, update copyright notices. Just do a case-insensitive
search for copyright. Don't forget copyright in manual. Also update
debian copyright in debian package. Last updated: 2017.
* To construct a source distribution from a pristine checkout,
make_dist does the following:
* If any interfaces were added or changed, check C API to see whether changes are appropriate there as well. If necessary, review the casting policy in the manual, and ensure that integer types are properly handled.
* Remember to avoid using `operator[]` with `std::string` or `std::vector`. Instead, use `at()`. See README-hardening.md for details.
* Increment shared library version information as needed (`LT_*` in `configure.ac`)
* Update release notes in manual. Look at diffs and ChangeLog.
* Add a release entry to ChangeLog.
* Make sure version numbers are consistent in the following locations:
* configure.ac
* libqpdf/QPDF.cc
* manual/qpdf-manual.xml
`make_dist` verifies this consistency.
* Update release date in `manual/qpdf-manual.xml`. Remember to ensure that the entities at the top of the document are consistent with the release notes for both version and release date.
* Check `TODO` file to make sure all planned items for the release are done or retargeted.
* Each year, update copyright notices. Just do a case-insensitive search for copyright. Don't forget copyright in manual. Also update debian copyright in debian package. Last updated: 2017.
* To construct a source distribution from a pristine checkout, `make_dist` does the following:
```
./autogen.sh
./configure --enable-doc-maintenance --enable-werror
make build_manual
make distclean
* To create a source release, do an export from the version control
system to a directory called qpdf-version. For example, from this
directory:
```
* To create a source release, do an export from the version control system to a directory called qpdf-version. For example, from this directory:
```
rm -rf /tmp/qpdf-x.y.z
git archive --prefix=qpdf-x.y.z/ HEAD . | (cd /tmp; tar xf -)
From the parent of that directory, run make_dist with the directory
as an argument. Remember to have fop in your path. For internally
testing releases, you can run make_dist with the --no-tests option.
* To create a source release of external libs, do an export from the
version control system into a directory called qpdf-external-libs
and just make a zip file of the result called
qpdf-external-libs-src.zip. See the README.txt file there for
information on creating binary external libs releases. Run this
from the external-libs repository:
```
From the parent of that directory, run `make_dist` with the directory as an argument. Remember to have fop in your path. For internally testing releases, you can run make_dist with the `--no-tests` option.
* To create a source release of external libs, do an export from the version control system into a directory called `qpdf-external-libs` and just make a zip file of the result called `qpdf-external-libs-src.zip`. See the README.txt file there for information on creating binary external libs releases. Run this from the external-libs repository:
```
git archive --prefix=external-libs/ HEAD . | (cd /tmp; tar xf -)
cd /tmp
zip -r qpdf-external-libs-src.zip external-libs
* To create Windows binary releases, extract the qpdf source
distribution in Windows (MSYS + MINGW, MSVC). From the extracted
directory, extract the binary distribution of the external
libraries. Run ./make_windows_releases from there. You will need
to have zip in your path.
```
* To create Windows binary releases, extract the qpdf source distribution in Windows (MSYS2 + MSVC). From the extracted directory, extract the binary distribution of the external libraries. Run ./make_windows_releases from there.
* Before releasing, rebuild and test debian package.
* Remember to copy README-what-to-download.md separately onto the
download area.
* Remember to update the web page including putting new documentation
in the "files" subdirectory of the website on sourceforge.net.
* Create a tag in the version control system, and make backups of the
actual releases. With git, use git tag -s to create a signed tag:
* Remember to copy `README-what-to-download.md` separately onto the download area.
* Remember to update the web page including putting new documentation in the `files` subdirectory of the website on sourceforge.net.
* Create a tag in the version control system, and make backups of the actual releases. With git, use git `tag -s` to create a signed tag:
```
git tag -s release-qpdf-$version HEAD -m"qpdf $version"
```
* When releasing on sourceforge, `external-libs` distributions go in `external-libs/yyyymmdd`, and qpdf distributions go in `qpdf/vvv`. Make the source package the default for all but Windows, and make the 32-bit mingw build the default for Windows.
* When releasing on sourceforge, external-libs distributions go in
external-libs/yyyymmdd, and qpdf distributions go in qpdf/vvv.
Make the source package the default for all but Windows, and make
the 32-bit mingw build the default for Windows.
# General Build Stuff
QPDF uses autoconf and libtool but does not use automake. The only files distributed with the qpdf source distribution that are not controlled are `configure`, `libqpdf/qpdf/qpdf-config.h.in`, `aclocal.m4`, and some documentation. See above for the steps required to prepare a source distribution.
General Build Stuff
===================
QPDF uses autoconf and libtool but does not use automake. The only
files distributed with the qpdf source distribution that are not
controlled are "configure", "libqpdf/qpdf/qpdf-config.h.in",
"aclocal.m4", and some documentation. See above for the steps
required to prepare a source distribution.
A small handful of additional files have been taken from autotools
programs. These should probably be updated from time to time.
* config.guess, config.sub, ltmain.sh, and the m4 directory: these
were created by running libtoolize -c. To update, run libtoolize
-f -c or remove the files and rerun libtoolize.
A small handful of additional files have been taken from autotools programs. These should probably be updated from time to time.
* `config.guess`, `config.sub`, `ltmain.sh`, and the `m4` directory: these were created by running `libtoolize -c`. To update, run `libtoolize -f -c` or remove the files and rerun `libtoolize`.
* Other files copied as indicated:
```
cp /usr/share/automake-1.11/install-sh .
cp /usr/share/automake-1.11/mkinstalldirs .
```
The entire contents of the m4 directory came from libtool.m4. If we
had some additional local parts, we could also add those to the m4
directory. In order for this to work, it is necessary to run "aclocal
-I m4" before running autoheader and autoconf.
The entire contents of the `m4` directory came from `libtool.m4`. If we had some additional local parts, we could also add those to the `m4` directory. In order for this to work, it is necessary to run `aclocal -I m4` before running `autoheader` and `autoconf`. The `autogen.sh` script handles this.
If building or editing documentation, configure with
--enable-doc-maintenance. This will ensure that all tools or files
required to validate and build documentation are available.
If building or editing documentation, configure with `--enable-doc-maintenance`. This will ensure that all tools or files required to validate and build documentation are available.
If you want to run make maintainer-clean or make distclean and you
haven't run ./configure, you can pass CLEAN=1 to make on the command
line to prevent it from complaining about configure not having been
run.
If you want to run `make maintainer-clean`, `make distclean`, or `make autofiles.zip` and you haven't run `./configure`, you can pass `CLEAN=1` to make on the command line to prevent it from complaining about configure not having been run.
Local Windows Testing Procedure
===============================
If you want to run checks without rerunning the build, pass `NO_REBUILD=1` to make. This can be useful for special testing scenarios such as valgrind or binary compatibility.
# Local Windows Testing Procedure
This is what I do for routine testing on Windows.
From Linux, run ./autogen.sh and make autofiles.zip
From Linux, run `./autogen.sh` and `make autofiles.zip CLEAN=1`.
From Windows, git clone from my Linux clone, unzip external-libs, and
unzip autofiles.zip.
From Windows, git clone from my Linux clone, unzip `external-libs`, and unzip `autofiles.zip`.
Look at make_windows_releases. Set up path the same way and run
whichever ./config-* is appropriate for whichever compiler I need to
test with. Run from msys started from one of the visual studio command
line shells. The build doesn't work from git bash on my local system.
Look at `make_windows_releases`. Set up path the same way and run whichever `./config-*` is appropriate for whichever compiler I need to test with. Start one of the Visual Studio native compiler shells, and from there, run one of the msys shells. The Visual Studio step is not necessary if just building with mingw.

View File

@ -1,56 +1,30 @@
To build from source for Linux or other UNIX/UNIX-like systems, it is
generally sufficient to download just the source qpdf-<version>.tar.gz
file.
To build from source for Linux or other UNIX/UNIX-like systems, it is generally sufficient to download just the source `qpdf-<version>.tar.gz` file.
For Windows, there are several additional files that you might want to
download.
For Windows, there are several additional files that you might want to download.
* qpdf-<version>-bin-mingw32.zip
* `qpdf-<version>-bin-mingw32.zip`
If you just want to use the qpdf command line program or use the
qpdf DLL's C-language interface, you can download this file. You
can also download this version if you are using MINGW's gcc 4.6 (or
a binary compatible version) and want to program using the C++
interface.
If you just want to use the qpdf command line program or use the qpdf DLL's C-language interface, you can download this file. You can also download this version if you are using MINGW's gcc and want to program using the C++ interface.
* qpdf-<version>-bin-mingw64.zip
* `qpdf-<version>-bin-mingw64.zip`
A 64-bit version built with mingw. Use this for 64-bit Windows
systems. The 32-bit version will also work on Windows 64-bit.
Both the 32-bit and the 64-bit version support files over 2 GB in
size, but you may find it easier to integrate this with your own
software if you use the 64-bit version.
A 64-bit version built with mingw. Use this for 64-bit Windows systems. The 32-bit version will also work on Windows 64-bit. Both the 32-bit and the 64-bit version support files over 2 GB in size, but you may find it easier to integrate this with your own software if you use the 64-bit version.
* qpdf-<version>-bin-msvc32.zip
* `qpdf-<version>-bin-msvc32.zip`
If you want to program using qpdf's C++ interface and you are using
Microsoft Visual C++ 2010 in 32-bit mode, you can download this
file.
If you want to program using qpdf's C++ interface and you are using Microsoft Visual C++ 2015 in 32-bit mode, you can download this file.
* qpdf-<version>-bin-msvc64.zip
* `qpdf-<version>-bin-msvc64.zip`
If you want to program using qpdf's C++ interface and you are using
Microsoft Visual C++ 2010 in 64-bit mode, you can download this
file.
If you want to program using qpdf's C++ interface and you are using Microsoft Visual C++ 2015 in 64-bit mode, you can download this file.
* qpdf-external-libs-bin.zip
* `qpdf-external-libs-bin.zip`
If you want to build qpdf for Windows yourself with either MINGW or
MSVC 2010, you can download this file and extract it inside the
qpdf source distribution. Please refer to README-windows.md in
the qpdf source distribution for additional details. Note that you
need the 2012-06-20 version or later to be able to build qpdf 3.0
or newer. The 2009-10-24 version is required for qpdf 2.3.1 or
older.
If you want to build qpdf for Windows yourself with either MINGW or MSVC 2015, you can download this file and extract it inside the qpdf source distribution. Please refer to README-windows.md in the qpdf source distribution for additional details. Note that you need the 2017-08-21 version or later to be able to build qpdf 7.0 or newer. Generally grab the `external-libs` distribution that was the latest version at the time of the release of whichever version of qpdf you are building.
* qpdf-external-libs-src.zip
* `qpdf-external-libs-src.zip`
If you want to build the external libraries on your own (for
Windows or anything else), you can download this archive. In
addition to including an unmodified distribution zlib and the jpeg
libary, it includes a README file and some scripts to help you
build it for Windows. qpdf was built using a binary distribution of
libjpeg-turbo for Windows. You will also have to provide those. See
README-windows.md for details.
If you want to build the external libraries on your own (for Windows or anything else), you can download this archive. In addition to including an unmodified distribution `zlib` and the `jpeg` libary, it includes a `README` file and some scripts to help you build it for Windows. You will also have to provide those.
If you want to build on Windows, please see also README-windows.md in the qpdf source distribution.
If you want to build on Windows, please see also README-windows.md.

View File

@ -1,12 +1,9 @@
Common Setup
============
You may need to disable antivirus software to run qpdf's test suite.
You may need to disable antivirus software to run qpdf's test suite. Running Windows Defender on Windows 10 does not interfere with building or running qpdf or its test suite.
To be able to build qpdf and run its test suite, you must have MSYS2
installed. This replaces the old process of having a mixture of msys,
mingw-w64, and ActiveState perl. It is now possible to do everything
with just MSYS2.
To be able to build qpdf and run its test suite, you must have MSYS2 installed. This replaces the old process of having a mixture of msys, mingw-w64, and ActiveState perl. It is now possible to do everything with just MSYS2.
Here's what I did on my system:
@ -16,214 +13,95 @@ Here's what I did on my system:
* From the prompt:
* Run `pacman -Syuu` and follow the instructions, which may tell you
to close the window and rerun the command multiple times.
* pacman -S make base-devel git zip unzip
* pacman -S mingw-w64-x86_64-toolchain mingw-w64-i686-toolchain
* `pacman -S make base-devel git zip unzip`
* `pacman -S mingw-w64-x86_64-toolchain mingw-w64-i686-toolchain`
If you would like to build with Microsoft Visual C++, install a
suitable Microsoft Visual Studio edition. In early 2016, 2015
community edition with C++ support is fine. It may crash a few times
during installation, but repeating the installation will allow it to
finish, and the resulting software is stable.
If you would like to build with Microsoft Visual C++, install a suitable Microsoft Visual Studio edition. In early 2016, 2015 community edition with C++ support is fine. It may crash a few times during installation, but repeating the installation will allow it to finish, and the resulting software is stable.
To build qpdf with Visual Studio, start the msys2 mingw32 or mingw64
shell from a command window started from one of the Visual Studio
shell windows. You must use the mingw shell for the same word size (32
or 64 bit) as the Windows compiler since the MSVC build uses objdump
from the msys distribution. You must also have it inherit the path.
For example:
To build qpdf with Visual Studio, start the msys2 mingw32 or mingw64 shell from a command window started from one of the Visual Studio shell windows. You must use the mingw shell for the same word size (32 or 64 bit) as the Windows compiler since the MSVC build uses objdump from the msys distribution. You must also have it inherit the path. For example:
* Start x64 native tools command prompt from msvc
* set MSYS2_PATH_TYPE=inherit
* C:\msys64\mingw64
* `set MSYS2_PATH_TYPE=inherit`
* `C:\msys64\mingw64`
Image comparison tests are disabled by default, but it is possible to
run them on Windows. To do so, add --enable-test-compare-images from
the configure statements given below and install some additional
third-party dependencies. These may be provided in an environment such
as MSYS or Cygwin or can be downloaded separately for other
environments. You may extract or install the following software into
separate folders each and add the "bin" folder to your "PATH"
environment variable to make executables and DLLs available. If
installers are provided, they might do that already by default.
Image comparison tests are disabled by default, but it is possible to run them on Windows. To do so, add `--enable-test-compare-images` from the configure statements given below and install some additional third-party dependencies. These may be provided in an environment such as MSYS or Cygwin or can be downloaded separately for other environments. You may extract or install the following software into separate folders each and add the `bin` folder to your `PATH` environment variable to make executables and DLLs available. If installers are provided, they might do that already by default.
* LibJpeg (http://gnuwin32.sourceforge.net/packages/jpeg.htm)
* [LibJpeg](http://gnuwin32.sourceforge.net/packages/jpeg.htm): This archive provides some needed DLLs needed by LibTiff.
* [LibTiff](http://gnuwin32.sourceforge.net/packages/tiff.htm): This archive provides some needed binaries and DLLs if you want to use the image comparison tests. It depends on some DLLs from LibJpeg.
* [GhostScript](http://www.ghostscript.com/download/gsdnld.html): GhostScript is needed for image comparison tests. It's important that the binary is available as `gs`, while its default name is `gswin32[c].exe`. You can either copy one of the original files, use `mklink` to create a hard/softlink, or provide a custom `gs.cmd` wrapper that forwards all arguments to one of the original binaries. Using `mklink` with `gswin32c.exe` is probably the best choice.
This archive provides some needed DLLs needed by LibTiff.
# External Libraries
* LibTiff (http://gnuwin32.sourceforge.net/packages/tiff.htm)
In order to build qpdf, you must have a copy of `zlib` and the `jpeg` library. The easy way to get it is to download the external libs from the qpdf download area. There are packages called `external-libs-bin.zip` and `external-libs-src.zip`. If you are building with MSVC 2015 or MINGW with MSYS2, you can just extract the `qpdf-external-libs-bin.zip` zip file into the top-level qpdf source tree. Note that you need the 2017-08-21 version (at least) to build qpdf 7.0 or greater since this includes jpeg. Passing `--enable-external-libs` to `./configure` (which is done automatically if you follow the instructions below) is sufficient to find them.
This archive provides some needed binaries and DLLs if you want to
use the image comparison tests. It depends on some DLLs from
LibJpeg.
You can also obtain `zlib` and `jpeg` directly on your own and install them. If you are using mingw, you can just set `CPPFLAGS`, `LDFLAGS`, and `LIBS` when you run ./configure so that it can find the header files and libraries. If you are building with MSVC and you want to do this, it probably won't work because `./configure` doesn't know how to interpret `LDFLAGS` and `LIBS` properly for MSVC (though qpdf's own build system does). In this case, you can probably get away with cheating by passing `--enable-external-libs` to `./configure` and then just editing `CPPFLAGS`, `LDFLAGS`, `LIBS` in the generated autoconf.mk file. Note that you should use UNIX-like syntax (`-I`, `-L`, `-l`) even though this is not what cl takes on the command line. qpdf's build rules will fix it.
* GhostScript (http://www.ghostscript.com/download/gsdnld.html)
You can also download `qpdf-external-libs-src.zip` and follow the instructions in the README.txt there for how to build external libs.
GhostScript is needed for image comparison tests. It's important
that the binary is available as "gs", while its default name is
"gswin32[c].exe". You can either copy one of the original files,
use "mklink" to create a hard-/softlink, or provide a custom
"gs.cmd" wrapper that forwards all arguments to one of the original
binaries. Using "mklink" with "gswin32c.exe" is probably the best
choice.
# Building from version control
If you check out qpdf from version control, you will not have the files that are generated by autoconf. If you are not changing these files, you can grab them from a source distribution or create them from a system that has autoconf. To create them from scratch, run `./autogen.sh` on a system that has autoconf installed. Once you have them, you can run `make CLEAN=1 autofiles.zip`. This will create an autofiles.zip that you can extract on top of a fresh checkout.
External Libraries
==================
# Building with MinGW
In order to build qpdf, you must have a copy of zlib and the jpeg
library. The easy way to get it is to download the external libs from
the qpdf download area. There are packages called
external-libs-bin.zip and external-libs-src.zip. If you are building
with MSVC 2015 or MINGW with MSYS2, you can just extract the
qpdf-external-libs-bin.zip zip file into the top-level qpdf source
tree. Note that you need the 2017-08-21 version (at least) to build
qpdf 7.0 or greater since this includes jpeg. Passing
--enable-external-libs to ./configure (which is done automatically if
you follow the instructions below) is sufficient to find them.
You can also obtain zlib and jpeg directly on your own and install
them. If you are using mingw, you can just set CPPFLAGS, LDFLAGS, and
LIBS when you run ./configure so that it can find the header files and
libraries. If you are building with msvc and you want to do this, it
probably won't work because ./configure doesn't know how to interpret
LDFLAGS and LIBS properly for MSVC (though qpdf's own build system
does). In this case, you can probably get away with cheating by
passing --enable-external-libs to ./configure and then just editing
CPPFLAGS, LDFLAGS, LIBS in the generated autoconf.mk file. Note that
you should use UNIX-like syntax (-I, -L, -l) even though this is not
what cl takes on the command line. qpdf's build rules will fix it.
You can also download qpdf-external-libs-src.zip and follow the
instructions in the README.txt there for how to build external libs.
Building from version control
=============================
If you check out qpdf from version control, you will not have the
files that are generated by autoconf. If you are not changing these
files, you can grab them from a source distribution or create them
from a system that has autoconf. To create them from scratch, run
./autogen.sh on a system that has autoconf installed. Once you have
them, you can run make CLEAN=1 autofiles.zip. This will create an
autofiles.zip that you can extract on top of a fresh checkout.
Building with MinGW
===================
QPDF is known to build and pass its test suite with MSYS2 using the
32-bit and 64-bit compilers from that project and Microsoft Visual C++
2015, both 32-bit and 64-bit versions. MSYS2 is required to build as
well in order to get make and other related tools. See common setup at
the top of this file for installation and configuration of MSYS2.
Then, from the suitable 32-bit or 64-bit environment, run
QPDF is known to build and pass its test suite with MSYS2 using the 32-bit and 64-bit compilers. MSYS2 is required to build as well in order to get make and other related tools. See common setup at the top of this file for installation and configuration of MSYS2. Then, from the suitable 32-bit or 64-bit environment, run
```
./config-mingw
and then
make
```
Note that ./config-mingw just runs ./configure with specific
arguments, so you can look at it, make adjustments, and manually run
configure instead.
Note that `./config-mingw` just runs `./configure` with specific arguments, so you can look at it, make adjustments, and manually run configure instead.
Add the absolute path to the libqpdf/build directory to your PATH.
Make sure you can run the qpdf command by typing qpdf/build/qpdf and
making sure you get a help message rather than an error loading the
DLL or no output at all. Run the test suite by typing
Add the absolute path to the `libqpdf/build` directory to your `PATH`. Make sure you can run the qpdf command by typing qpdf/build/qpdf and making sure you get a help message rather than an error loading the DLL or no output at all. Run the test suite by typing
```
make check
```
If all goes well, you should get a passing test suite.
To create an installation directory, run make install. This will
create install-mingw/qpdf-VERSION and populate it. The binary
download of qpdf for Windows with mingw is created from this
directory.
To create an installation directory, run `make install`. This will create `install-mingw/qpdf-VERSION` and populate it. The binary download of qpdf for Windows with mingw is created from this directory.
You can also take a look at make_windows_releases for reference. This
is how the distributed Windows executables are created.
You can also take a look at `make_windows_releases` for reference. This is how the distributed Windows executables are created.
# Building with MSVC 2015
Building with MSVC 2015
=======================
These instructions would likely work with newer versions of MSVC and are known to have worked with versions as old as 2008 Express.
These instructions would likely work with newer versions of MSVC and
are known to have worked with versions as old as 2008 Express.
You should first set up your environment to be able to run MSVC from the command line. There is usually a batch file included with MSVC that does this. Make sure that you start a command line environment configured for whichever of 32-bit or 64-bit output that you intend to build for.
You should first set up your environment to be able to run MSVC from
the command line. There is usually a batch file included with MSVC
that does this. Make sure that you start a command line environment
configured for whichever of 32-bit or 64-bit output that you intend to
build for.
From that cmd prompt, you can start your MSYS2 shell with path inheritance as described above.
From that cmd prompt, you can start your MSYS2 shell with path
inheritance as described above.
Configure as follows:
Configure and build as follows:
```
./config-msvc
Once configured, run
make
```
Note that ./config-msvc just runs ./configure with specific arguments,
so you can look at it, make adjustments, and manually run configure
instead.
Note that `./config-msvc` just runs `./configure` with specific arguments, so you can look at it, make adjustments, and manually run configure instead.
NOTE: automated dependencies are not generated with the msvc build.
If you're planning on making modifications, you should probably work
with mingw. If there is a need, I can add dependency information to
the msvc build, but since I only use it for generating release
versions, I haven't bothered.
NOTE: automated dependencies are not generated with the msvc build. If you're planning on making modifications, you should probably work with mingw. If there is a need, I can add dependency information to the msvc build, but since I only use it for generating release versions, I haven't bothered.
Once built, add the full path to the libqpdf/build directory to your
path and run
Once built, add the full path to the `libqpdf/build` directory to your path and run
```
make check
```
to run the test suite.
If you are building with MSVC and want to debug a crash in MSVC's
debugger, first start an instance of Visual C++. Then run qpdf. When
the abort/retry/ignore dialog pops up, first attach the process from
within visual C++, and then click Retry in qpdf.
If you are building with MSVC and want to debug a crash in MSVC's debugger, first start an instance of Visual C++. Then run qpdf. When the abort/retry/ignore dialog pops up, first attach the process from within visual C++, and then click Retry in qpdf.
A release version of qpdf is built by default. If you want to link
against debugging libraries, you will have to change /MD to /MDd in
make/msvc.mk. Note that you must redistribute the Microsoft runtime
DLLs. Linking with static runtime (/MT) won't work; see "Static
Runtime" below for details.
A release version of qpdf is built by default. If you want to link against debugging libraries, you will have to change `/MD` to `/MDd` in `make/msvc.mk`. Note that you must redistribute the Microsoft runtime DLLs. Linking with static runtime (`/MT`) won't work; see "Static Runtime" below for details.
# Runtime DLLs
Runtime DLLs
============
Both build methods create executables and DLLs that are dependent on the compiler's runtime DLLs. When you run make install, the installation process will automatically detect the DLLs and copy them into the installation bin directory. Look at the `copy_dlls` script for details on how this is accomplished.
Both build methods create executables and DLLs that are dependent on
the compiler's runtime DLLs. When you run make install, the
installation process will automatically detect the DLLs and copy them
into the installation bin directory. Look at the copy_dlls script for
details on how this is accomplished.
Redistribution of the runtime DLL is unavoidable as of this writing; see "Static Runtime" below for details.
Redistribution of the runtime DLL is unavoidable as of this writing;
see "Static Runtime" below for details.
# Static Runtime
Static Runtime
==============
Building the DLL and executables with static runtime does not work
with either Visual C++ .NET 2008 (a.k.a. vc9) using /MT or with mingw
(at least as of 4.4.0) using -static-libgcc. The reason is that, in
both cases, there is static data involved with exception handling, and
when the runtime is linked in statically, exceptions cannot be thrown
across the DLL to EXE boundary. Since qpdf uses exception handling
extensively for error handling, we have no choice but to redistribute
the C++ runtime DLLs. Maybe this will be addressed in a future
version of the compilers. This has not been retested with the
toolchain versions used to create qpdf 3.0 distributions. (This has
not been revisited since MSVC 2008, but redistrbuting runtime DLLs is
extremely common and should not be a problem.)
Building the DLL and executables with static runtime does not work with either Visual C++ .NET 2008 (a.k.a. vc9) using `/MT` or with mingw (at least as of 4.4.0) using `-static-libgcc`. The reason is that, in both cases, there is static data involved with exception handling, and when the runtime is linked in statically, exceptions cannot be thrown across the DLL to EXE boundary. Since qpdf uses exception handling extensively for error handling, we have no choice but to redistribute the C++ runtime DLLs. Maybe this will be addressed in a future version of the compilers. This has not been retested with the toolchain versions used to create qpdf >= 3.0 distributions. This has not been revisited since MSVC 2008, but redistrbuting runtime DLLs is extremely common and should not be a problem.

216
README.md
View File

@ -1,59 +1,28 @@
This is the QPDF package. Information about it can be found at
http://qpdf.sourceforge.net. The source code repository is hosted
at github: https://github.com/qpdf/qpdf.
This is the QPDF package. Information about it can be found at https://qpdf.sourceforge.net. The source code repository is hosted at github: https://github.com/qpdf/qpdf.
QPDF is copyright (c) 2005-2017 Jay Berkenbilt
This software may be distributed under the terms of version 2 of the
Artistic License which may be found in the source distribution as
"Artistic-2.0". It is provided "as is" without express or implied
warranty.
This software may be distributed under the terms of version 2 of the Artistic License which may be found in the source distribution as "Artistic-2.0". It is provided "as is" without express or implied warranty.
# Prerequisites
Prerequisites
=============
QPDF depends on the external libraries [zlib](http://www.zlib.net/) and [jpeg](http://www.ijg.org/files/). The [libjpeg-turbo](https://libjpeg-turbo.org/) library is also known to work since it is compatible with the regular jpeg library, and QPDF doesn't use any interfaces that aren't present in the straight jpeg8 API. These are part of every Linux distribution and are readily available. Download information appears in the documentation. For Windows, you can download pre-built binary versions of these libraries for some compilers; see [README-windows.md](README-windows.md) for additional details.
QPDF depends on the external libraries "zlib" and "jpeg". These are
part of every Linux distribution and are readily available. Download
information appears in the documentation. For Windows, you can
download pre-built binary versions of these libraries for some
compilers; see README-windows.md for additional details.
QPDF requires a C++ compiler that works with STL. Your compiler must also support `long long`. Almost all modern compilers do. If you are trying to port qpdf to a compiler that doesn't support `long long`, you could change all occurrences of `long long` to `long` in the source code, noting that this would break binary compatibility with other builds of qpdf. Doing so would certainly prevent qpdf from working with files larger than 2 GB, but remaining functionality would most likely work fine. If you built qpdf this way and it passed its test suite with large file support disabled, you could be confident that you had an otherwise working qpdf.
QPDF requires a C++ compiler that works with STL. Your compiler must
also support "long long". Almost all modern compilers do. If you are
trying to port qpdf to a compiler that doesn't support long long, you
could change all occurrences of "long long" to "long" in the source
code, noting that this would break binary compatibility with other
builds of qpdf. Doing so would certainly prevent qpdf from working
with files larger than 2 GB, but remaining functionality would most
likely work fine. If you built qpdf this way and it passed its test
suite with large file support disabled, you could be confident that
you had an otherwise working qpdf.
# Licensing terms of embedded software
QPDF makes use of zlib and jpeg libraries for its functionality. These packages can be downloaded separately from their own download locations, or they can be downloaded in the external-libs area of the qpdf download site.
Licensing terms of embedded software
====================================
QPDF makes use of zlib and jpeg libraries for its functionality. These
packages can be downloaded separately from their own download
locations, or they can be downloaded in the external-libs area of the
qpdf download site.
The Rijndael encryption implementation used as the basis for AES
encryption and decryption support comes from Philip J. Erdelsky's
public domain implementation. The files libqpdf/rijndael.cc and
libqpdf/qpdf/rijndael.h remain in the public domain. They were
obtained from
http://www.efgh.com/software/rijndael.htm
http://www.efgh.com/software/rijndael.txt
The Rijndael encryption implementation used as the basis for AES encryption and decryption support comes from Philip J. Erdelsky's public domain implementation. The files `libqpdf/rijndael.cc` and `libqpdf/qpdf/rijndael.h` remain in the public domain. They were obtained from
* http://www.efgh.com/software/rijndael.htm
* http://www.efgh.com/software/rijndael.txt
The embedded sha2 code comes from sphlib 3.0
http://www.saphir2.com/sphlib/
* http://www.saphir2.com/sphlib/
That code has the following license:
```
Copyright (c) 2007-2011 Projet RNRT SAPHIR
Permission is hereby granted, free of charge, to any person obtaining
@ -74,166 +43,59 @@ That code has the following license:
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
```
# Building from a pristine checkout
Building from a pristine checkout
=================================
When building qpdf from a pristine checkout from version control,
documentation and automatically generated files are not present.
Building on Windows from a pristine checkout is not guaranteed to work
because of issues running autoconf; see README-windows.md for how to
handle this. For UNIX and UNIX-like systems, you must have some
addditional tools installed to build from the source repository. To
do this, you should run
When building qpdf from a pristine checkout from version control, documentation and automatically generated files are not present. Building on Windows from a pristine checkout is not guaranteed to work because of issues running autoconf; see [README-windows.md](README-windows.md) for how to handle this. For UNIX and UNIX-like systems, you must have some addditional tools installed to build from the source repository. To do this, you should run
```
./autogen.sh
./configure --enable-doc-maintenance
make
make install
```
If you don't have Apache fop and the docbook stylesheets installed,
you won't be able to build documentation. You can omit
--enable-doc-maintenance and produce working qpdf software that passes
its test suite, but make install will fail because the documentation
files won't exist. Depending on your purposes, you can either work
around this or grab the docs from a source distribution.
If you don't have Apache fop and the docbook stylesheets installed, you won't be able to build documentation. You can omit `--enable-doc-maintenance` and produce working qpdf software that passes its test suite, but make install will fail because the documentation files won't exist. Depending on your purposes, you can either work around this or grab the docs from a source distribution.
Building from source distribution on UNIX/Linux
===============================================
# Building from source distribution on UNIX/Linux
For UNIX and UNIX-like systems, you can usually get by with just
```
./configure
make
make install
```
Packagers may set DESTDIR, in which case make install will install
inside of DESTDIR, as is customary with many packages. For more
detailed general information, see the "INSTALL" file in this
directory. If you are already accustomed to building and installing
software that uses autoconf, there's nothing new for you in the
INSTALL file.
Packagers may set DESTDIR, in which case make install will install inside of DESTDIR, as is customary with many packages. For more detailed general information, see the "INSTALL" file in this directory. If you are already accustomed to building and installing software that uses autoconf, there's nothing new for you in the INSTALL file. Note that qpdf uses `autoconf` but not `automake`. We have our own system of Makefiles that allows cross-directory dependencies, doesn't use recursive make, and works better on non-UNIX platforms.
# Building on Windows
Building on Windows
===================
QPDF is known to build and pass its test suite with mingw (latest version tested: gcc 7.2.0), mingw64 (latest version tested: 7.2.0) and Microsoft Visual C++ 2015, both 32-bit and 64-bit versions. MSYS2 is required to build as well in order to get make and other related tools. See [README-windows.md](README-windows.md) for details on how to build under Windows.
QPDF is known to build and pass its test suite with mingw (latest
version tested: gcc 4.6.2), mingw64 (latest version tested: 4.7.0) and
Microsoft Visual C++ 2010, both 32-bit and 64-bit versions. MSYS plus
ActiveState Perl is required to build as well in order to get make
and other related tools. See README-windows.md for details on how to
build under Windows, see README-windows.md.
# Additional Notes on Build
QPDF's build system, inspired by [abuild](http://www.abuild.org), can optionally use its own built-in rules rather than using libtool and obeying the compiler specified with configure. This can be enabled by passing `--with-buildrules=buildrules` where buildrules corresponds to one of the `.mk` files (other than `rules.mk`) in the make directory. This should never be necessary on a UNIX system, but may be necessary on a Windows system. See [README-windows.md](README-windows.md) for details. There is a `gcc-linux.mk` file enable `gcc-linux` build rules, but it is intended to help test the build system; Linux users should build with the `libtools` rules, which are enabled by default.
Additional Notes on Build
=========================
The QPDF package provides some executables and a software library. A user manual can be found in the "doc" directory. The docbook sources to the user manual can be found in the `manual` directory.
QPDF's build system, inspired by abuild (http://www.abuild.org), can
optionally use its own built-in rules rather than using libtool and
obeying the compiler specified with configure. This can be enabled by
passing --with-buildrules=buildrules where buildrules corresponds to
one of the .mk files (other than rules.mk) in the make directory.
This should never be necessary on a UNIX system, but may be necessary
on a Windows system. See README-windows.md for details. There is a
gcc-linux.mk file enable "gcc-linux" build rules, but it is intended
to help test the build system; Linux users should build with the
"libtools" rules, which are enabled by default.
The software library is just `libqpdf`, and all the header files are in the `qpdf` subdirectories of `include` and `libqpdf`. If you link statically with `-lqpdf`, then you will also need to link with `-lz` and `-ljpeg`. The shared qpdf library is linked with `-lz` and `-ljpeg`, none of qpdf's public header files directly include files from `libz`, and only `Pl_DCT.hh` includes files from `libjpeg`, so for most cases, qpdf's development files are self contained. If you need to use `Pl_DCT` in your application code, you will need to have the header files for some libjpeg distribution in your include path.
The QPDF package provides some executables and a software library. A
user's manual can be found in the "doc" directory. The docbook
sources to the user's manual can be found in the "manual" directory.
To learn about using the library, please read comments in the header files in `include/qpdf`, especially `QPDF.hh`, `QPDFObjectHandle.hh`, and
`QPDFWriter.hh`. These are the best sources of documentation on the API. You can also study the code of `qpdf/qpdf.cc`, which exercises most of the public interface. There are additional example programs in the examples directory. Reading all the source files in the `qpdf` directory (including the qpdf command-line tool and some test drivers) along with the code in the examples directory will give you a complete picture of every aspect of the public interface.
The software library is just libqpdf, and all the header files are in
the qpdf subdirectory. If you link statically with -lqpdf, then you
will also need to link with -lz and -ljpeg. The shared qpdf library is
linked with -lz and -ljpeg, none of qpdf's public header files
directly include files from libz, and only Pl_DCT.hh includes files
from libjpeg, so for most cases, qpdf's development files are self
contained. If you need to use Pl_DCT in your application code, you
will need to have the header files for some libjpeg distribution in
your include path.
# Additional Notes on Test Suite
To learn about using the library, please read comments in the header
files in include/qpdf, especially QPDF.hh, QPDFObjectHandle.hh, and
QPDFWriter.hh. You can also study the code of qpdf/qpdf.cc, which
exercises most of the public interface. There are additional example
programs in the examples directory. Reading all the source files in
the qpdf directory (including the qpdf command-line tool and some test
drivers) along with the code in the examples directory will give you a
complete picture of every aspect of the public interface.
By default, slow tests and tests that require dependencies beyond those needed to build qpdf are disabled. Slow tests include image comparison tests and large file tests. Image comparison tests can be enabled by passing `--enable-test-compare-images` to ./configure. This was on by default in qpdf versions prior to 3.0, but is now off by default. Large file tests can be enabled by passing `--with-large-file-test-path=path` to `./configure` or by setting the `QPDF_LARGE_FILE_TEST_PATH` environment variable. On Windows, this should be a Windows path. Run `./configure --help` for additional options. The test suite provides nearly full coverage even without these tests. Unless you are making deep changes to the library that would impact the contents of the generated PDF files or testing this on a new platform for the first time, there is no real reason to run these tests. If you're just running the test suite to make sure that qpdf works for your build, the default tests are adequate. The configure rules for these tests do nothing other than setting variables in `autoconf.mk`, so you can feel free to turn these on and off directly in `autoconf.mk` rather than rerunning configure.
If you are packaging qpdf for a distribution and preparing a build that is run by an autobuilder, you may want to add the `--enable-show-failed-test-output` to configure options. This way, if the test suite fails, test failure detail will be included in the build output. Otherwise, you will have to have access to the `qtest.log` file from the build to view test failures. The debian packages for qpdf enable this option.
Additional Notes on Test Suite
==============================
# Random Number Generation
By default, slow tests are disabled. Slow tests include image
comparison tests and large file tests. Image comparison tests can be
enabled by passing --enable-test-compare-images to ./configure. This
was on by default in qpdf versions prior to 3.0, but is now off by
default. Large file tests can be enabled by passing
--with-large-file-test-path=path to ./configure or by setting the
QPDF_LARGE_FILE_TEST_PATH environment variable. On Windows, this
should be a Windows path. Run ./configure --help for additional
options. The test suite provides nearly full coverage even without
these tests. Unless you are making deep changes to the library that
would impact the contents of the generated PDF files or testing this
on a new platform for the first time, there is no real reason to run
these tests. If you're just running the test suite to make sure that
qpdf works for your build, the default tests are adequate. The
configure rules for these tests do nothing other than setting
variables in autoconf.mk, so you can feel free to turn these on and
off directly in autoconf.mk rather than rerunning configure.
By default, when `qpdf` detects either the Windows cryptography API or the existence of `/dev/urandom`, `/dev/arandom`, or `/dev/random`, it uses them to generate cryptography secure random numbers. If none of these conditions are true, the build will fail with an error. This behavior can be modified in several ways:
* If you configure with `--disable-os-secure-random` or define `SKIP_OS_SECURE_RANDOM`, qpdf will not attempt to use Windows cryptography or the random device. You must either supply your own random data provider or allow use of insecure random numbers.
* If you configure qpdf with the `--enable-insecure-random` option or define `USE_INSECURE_RANDOM`, qpdf will try insecure random numbers if OS-provided secure random numbers are disabled. This is not a fallback. In order for insecure random numbers to be used, you must also disable OS secure random numbers since, otherwise, failure to find OS secure random numbers is a compile error. The insecure random number source is stdlib's `random()` or `rand()` calls. These random numbers are not cryptography secure, but the qpdf library is fully functional using them. Using non-secure random numbers means that it's easier in some cases to guess encryption keys. If you're not generating encrypted files, there's no advantage to using secure random numbers.
* In all cases, you may supply your own random data provider. To do this, derive a class from `qpdf/RandomDataProvider` (since version 5.1.0) and call `QUtil::setRandomDataProvider` before you create any `QPDF` objects. If you supply your own random data provider, it will always be used even if support for one of the other random data providers is compiled in. If you wish to avoid any possibility of your build of qpdf from using anything but a user-supplied random data provider, you can define `SKIP_OS_SECURE_RANDOM` and not `USE_INSECURE_RANDOM`. In this case, qpdf will throw a runtime error if any attempt is made to generate random numbers and no random data provider has been supplied.
If you are packaging qpdf for a distribution and preparing a build
that is run by an autobuilder, you may want to add the
--enable-show-failed-test-output to configure options. This way, if
the test suite fails, test failure detail will be included in the
build output. Otherwise, you will have to have access to the
qtest.log file from the build to view test failures. The debian
packages for qpdf enable this option, for example.
Random Number Generation
========================
By default, when the qpdf detects either the Windows cryptography API
or the existence of /dev/urandom, /dev/arandom, or /dev/random, it
uses them to generate cryptography secure random numbers. If none of
these conditions are true, the build will fail with an error. This
behavior can be modified in several ways:
* If you configure with --disable-os-secure-random or define
SKIP_OS_SECURE_RANDOM, qpdf will not attempt to use Windows
cryptography or the random device. You must either supply your own
random data provider or allow use of insecure random numbers.
* If you configure qpdf with the --enable-insecure-random option or
define USE_INSECURE_RANDOM, qpdf will try insecure random numbers
if OS-provided secure random numbers are disabled. This is not a
fallback. In order for insecure random numbers to be used, you
must also disable OS secure random numbers since, otherwise,
failure to find OS secure random numbers is a compile error. The
insecure random number source is stdlib's random() or rand() calls.
These random numbers are not cryptography secure, but the qpdf
library is fully functional using them. Using non-secure random
numbers means that it's easier in some cases to guess encryption
keys. If you're not generating encrypted files, there's no
advantage to using secure random numbers.
* In all cases, you may supply your own random data provider. To do
this, derive a class from qpdf/RandomDataProvider (since 5.1.0) and
call QUtil::setRandomDataProvider before you create any QPDF
objects. If you supply your own random data provider, it will
always be used even if support for one of the other random data
providers is compiled in. If you wish to avoid any possibility of
your build of qpdf from using anything but a user-supplied random
data provider, you can define SKIP_OS_SECURE_RANDOM and not
USE_INSECURE_RANDOM. In this case, qpdf will throw a runtime error
if any attempt is made to generate random numbers and no random
data provider has been supplied.
If you are building qpdf on a platform that qpdf doesn't know how to
generate secure random numbers on, a patch would be welcome.
If you are building qpdf on a platform that qpdf doesn't know how to generate secure random numbers on, a patch would be welcome.