mirror of
https://github.com/qpdf/qpdf.git
synced 2024-12-23 03:18:59 +00:00
79f6b4823b
Have classes contain only a single private member of type PointerHolder<Members>. This makes it safe to change the structure of the Members class without breaking binary compatibility. Many of the classes already follow this pattern quite successfully. This brings in the rest of the class that are part of the public API.
307 lines
10 KiB
Plaintext
307 lines
10 KiB
Plaintext
ROUTINE DEVELOPMENT
|
|
|
|
**Remember to check pull requests as well as issues in github.**
|
|
|
|
Default:
|
|
|
|
./configure --enable-werror --disable-shared
|
|
|
|
Debugging:
|
|
|
|
./configure CFLAGS="-g" CXXFLAGS="-g" --enable-werror --disable-shared
|
|
|
|
Memory checks:
|
|
|
|
./configure CFLAGS="-fsanitize=address -fsanitize=undefined -g" \
|
|
CXXFLAGS="-fsanitize=address -fsanitize=undefined -g" \
|
|
LDFLAGS="-fsanitize=address -fsanitize=undefined" \
|
|
--enable-werror --disable-shared
|
|
|
|
GOOGLE OSS-FUZZ
|
|
|
|
* qpdf project: https://github.com/google/oss-fuzz/tree/master/projects/qpdf
|
|
|
|
* To test locally, see https://github.com/google/oss-fuzz/tree/master/docs/,
|
|
especially new_project_guide.md. Summary:
|
|
|
|
Clone the oss-fuzz project. From the root directory of the repository:
|
|
|
|
Add `-e GITHUB_FORK=fork -e GITHUB_BRANCH=branch` to build_fuzzers
|
|
from a qpdf fork/branch rather than qpdf/master.
|
|
|
|
python infra/helper.py build_image --pull qpdf
|
|
python infra/helper.py build_fuzzers qpdf
|
|
python infra/helper.py check_build qpdf
|
|
python infra/helper.py build_fuzzers --sanitizer coverage qpdf
|
|
python infra/helper.py coverage qpdf
|
|
|
|
The fuzzer is in build/out/qpdf. It can be run with a directory as
|
|
an argument to run against files in a directory. You can use
|
|
|
|
qpdf_fuzzer -merge=1 cur new >& /dev/null&
|
|
|
|
to add any files from new into cur if they increase coverage. You
|
|
need to do this with the coverage build (the one with
|
|
--sanitizer coverage)
|
|
|
|
* General documentation: http://libfuzzer.info
|
|
|
|
* Build status: https://oss-fuzz-build-logs.storage.googleapis.com/index.html
|
|
|
|
* Project status: https://oss-fuzz.com/ (private -- log in with Google account)
|
|
|
|
* Latest corpus:
|
|
gs://qpdf-backup.clusterfuzz-external.appspot.com/corpus/libFuzzer/qpdf_fuzzer/latest.zip
|
|
|
|
CODING RULES
|
|
|
|
* Avoid atoi. Use QUtil::string_to_int instead. It does
|
|
overflow/underflow checking.
|
|
|
|
* Remember to avoid using `operator[]` with `std::string` or
|
|
`std::vector`. Instead, use `at()`. See README-hardening.md for
|
|
details.
|
|
|
|
* Use QIntC for type conversions -- see casting policy in docs.
|
|
|
|
* Use QPDF_DLL on all methods that are to be exported in the shared
|
|
library/DLL. Use QPDF_DLL_CLASS for all classes whose type
|
|
information is needed. This is important for exception classes and
|
|
it seems also for classes that are intended to be subclassed across
|
|
the shared library boundary.
|
|
|
|
* Put private member variables in PointerHolder<Members> for all
|
|
public classes. Remember to use QPDF_DLL on ~Members().
|
|
|
|
RELEASE PREPARATION
|
|
|
|
* Each year, update copyright notices. Just do a case-insensitive
|
|
search for copyright. Don't forget copyright in manual. Also update
|
|
debian copyright in debian package. Last updated: 2019.
|
|
|
|
* Check all open issues and pull requests in github and the
|
|
sourceforge trackers.
|
|
|
|
* Check `TODO` file to make sure all planned items for the release are
|
|
done or retargeted.
|
|
|
|
* Run a spelling checker over the source code to catch errors in
|
|
variable names, strings, and comments.
|
|
|
|
ispell -p ispell-words **/*.hh **/*.cc manual/* ChangeLog README* TODO
|
|
|
|
* If needed, run large file and image comparison tests. Configure
|
|
options:
|
|
|
|
--enable-test-compare-images --with-large-file-test-path=/path
|
|
|
|
For Windows, use a Windows style path, not an MSYS path for large files.
|
|
|
|
* Test with clang. Pass `CC=clang CXX=clang++` to `./configure`. Test
|
|
with newer version of gcc if available.
|
|
|
|
* Test 32-bit. Pass `CC=i686-linux-gnu-gcc CXX=i686-linux-gnu-g++` to
|
|
`./configure`.
|
|
|
|
* Test build on a mac.
|
|
|
|
* Test with address sanitizer as described above.
|
|
|
|
* A small handful of additional files have been taken from autotools
|
|
programs. These should probably be updated from time to time.
|
|
|
|
* `config.guess`, `config.sub`, `ltmain.sh`, and the `m4` directory:
|
|
these were created by running `libtoolize -c`. To update, run
|
|
`libtoolize -f -c` or remove the files and rerun `libtoolize`.
|
|
|
|
* Other files copied as indicated:
|
|
```
|
|
cp /usr/share/automake-1.11/install-sh .
|
|
cp /usr/share/automake-1.11/mkinstalldirs .
|
|
```
|
|
|
|
The entire contents of the `m4` directory came from `libtool.m4`. If
|
|
we had some additional local parts, we could also add those to the
|
|
`m4` directory. In order for this to work, it is necessary to run
|
|
`aclocal -I m4` before running `autoheader` and `autoconf`. The
|
|
`autogen.sh` script handles this.
|
|
|
|
* If any interfaces were added or changed, check C API to see whether
|
|
changes are appropriate there as well. If necessary, review the
|
|
casting policy in the manual, and ensure that integer types are
|
|
properly handled with QIntC or the appropriate cast.
|
|
|
|
* Increment shared library version information as needed (`LT_*` in
|
|
`configure.ac`)
|
|
|
|
* Test for binary compatibility:
|
|
* Check out the last release
|
|
* ./configure --enable-werror && make -j$(nproc)
|
|
* Check out the current version
|
|
* ./configure --enable-werror && make -j$(nproc) build_libqpdf
|
|
* Checkout the last release
|
|
* make -k check NO_REBUILD=1
|
|
|
|
* Update release notes in manual. Look at diffs and ChangeLog. Update
|
|
release date in `manual/qpdf-manual.xml`. Remember to ensure that
|
|
the entities at the top of the document are consistent with the
|
|
release notes for both version and release date.
|
|
|
|
* Make sure version numbers are consistent in the following locations:
|
|
* configure.ac
|
|
* libqpdf/QPDF.cc
|
|
* manual/qpdf-manual.xml
|
|
* qpdf/qpdf.cc
|
|
`make_dist` verifies this consistency.
|
|
|
|
* Add a release entry to ChangeLog.
|
|
|
|
|
|
CREATING A RELEASE
|
|
|
|
* Push to master. The azure pipeline will create an artifact called
|
|
distribution which will contain all the distribution files. Download
|
|
these, verify the checksums from the job output, rename to remove
|
|
-ci from the names, and copy to the release archive area.
|
|
|
|
* Sign the source distribution:
|
|
|
|
version=x.y.z
|
|
gpg --detach-sign --armor qpdf-$version.tar.gz
|
|
|
|
* Build and test the debian package
|
|
|
|
* Sign the releases. The release archive area should contain the
|
|
Windows binaries, the AppImage, the source tarball, and the source
|
|
tarball signature.
|
|
|
|
\rm -f *.{md5,sha1,sha512}
|
|
files=(*)
|
|
for i in md5 sha1 sha512; do
|
|
${i}sum ${files[*]} >| qpdf-$version.$i
|
|
gpg --clearsign --armor qpdf-$version.$i
|
|
mv qpdf-$version.$i.asc qpdf-$version.$i
|
|
done
|
|
chmod 444 *
|
|
chmod 555 *.AppImage
|
|
|
|
* When creating releases on github and sourceforge, remember to copy
|
|
`README-what-to-download.md` separately onto the download area if
|
|
needed.
|
|
|
|
* Ensure that the master branch has been pushed to github. The
|
|
rev-parse command below should show the same commit hash for all its
|
|
arguments. Create and push a signed tag. This should be run with
|
|
HEAD pointing to the tip of master.
|
|
|
|
git rev-parse master upstream/master @
|
|
git tag -s release-qpdf-$version @ -m"qpdf $version"
|
|
git push upstream release-qpdf-$version
|
|
|
|
* In Azure Pipelines, retain the build that was used to generate the
|
|
release.
|
|
|
|
* Create a github release after pushing the tag. `gcurl` is an alias
|
|
that includes the auth token.
|
|
|
|
# Create release
|
|
TOKEN=$(cat ~/.github-token)
|
|
function gcurl() { curl -H "Authorization: token $TOKEN" ${1+"$@"}; }
|
|
url=$(gcurl -s -XPOST https://api.github.com/repos/qpdf/qpdf/releases -d'{"tag_name": "release-qpdf-'$version'", "name": "qpdf '$version'", "draft": true}' | jq -r '.url')
|
|
|
|
# Get upload url
|
|
upload_url=$(gcurl -s $url | jq -r '.upload_url' | sed -E -e 's/\{.*\}//')
|
|
echo $upload_url
|
|
|
|
# Upload all the files. You can add a label attribute too, which
|
|
# overrides the name.
|
|
for i in *; do
|
|
mime=$(file -b --mime-type $i)
|
|
gcurl -H "Content-Type: $mime" --data-binary @$i "$upload_url?name=$i"
|
|
done
|
|
|
|
If needed, go onto github and make any manual updates such as
|
|
indicating a pre-release, adding release notes, etc.
|
|
|
|
# Publish release
|
|
gcurl -XPOST $url -d'{"draft": false}'
|
|
|
|
* Upload files to sourceforge. Make the source package the default for
|
|
all but Windows, and make the 32-bit mingw build the default for
|
|
Windows. Publish a news item manually on sourceforge.
|
|
|
|
* Update the web page to indicate the new version and to put the new
|
|
documentation in the `files` subdirectory of the website on
|
|
sourceforge.net.
|
|
|
|
* Email the qpdf-announce list.
|
|
|
|
|
|
OTHER NOTES
|
|
|
|
To construct a source distribution from a pristine checkout,
|
|
`make_dist` does the following:
|
|
|
|
./configure --enable-doc-maintenance --enable-werror
|
|
make build_manual
|
|
make distclean
|
|
|
|
To create a source release of external libs, do an export from the
|
|
version control system into a directory called `qpdf-external-libs`
|
|
and just make a zip file of the result called
|
|
`qpdf-external-libs-src.zip`. See the README.txt file there for
|
|
information on creating binary external libs releases. Run this from
|
|
the external-libs repository:
|
|
|
|
git archive --prefix=external-libs/ HEAD . | (cd /tmp; tar xf -)
|
|
cd /tmp
|
|
zip -r qpdf-external-libs-src.zip external-libs
|
|
|
|
When releasing on sourceforge, `external-libs` distributions go in
|
|
`external-libs/yyyymmdd`, and qpdf distributions go in `qpdf/vvv`.
|
|
|
|
For local iteration on the AppImage generation, follow the release
|
|
procedures for building the AppImage, but instead of passing git clone
|
|
options to the docker command, copy qpdf to /tmp/build. You can also
|
|
pass -e SKIP_TESTS=1 to docker to skip the test suite, useful for
|
|
rapid iteration. Set up /tmp/build as in the release process.
|
|
|
|
cp -a $PWD /tmp/build
|
|
docker run --privileged -ti --rm -e SKIP_TESTS=1 -v /tmp/build:/tmp/build qpdfbuild
|
|
|
|
|
|
GENERAL BUILD STUFF
|
|
|
|
QPDF uses autoconf and libtool but does not use automake. The only
|
|
files distributed with the qpdf source distribution that are not
|
|
controlled are `configure`, `libqpdf/qpdf/qpdf-config.h.in`,
|
|
`aclocal.m4`, and some documentation. See above for the steps required
|
|
to prepare a source distribution.
|
|
|
|
If building or editing documentation, configure with
|
|
`--enable-doc-maintenance`. This will ensure that all tools or files
|
|
required to validate and build documentation are available.
|
|
|
|
If you want to run `make maintainer-clean` or `make distclean` and you
|
|
haven't run `./configure`, you can pass `CLEAN=1` to make on the
|
|
command line to prevent it from complaining about configure not having
|
|
been run.
|
|
|
|
If you want to run checks without rerunning the build, pass
|
|
`NO_REBUILD=1` to make. This can be useful for special testing
|
|
scenarios such as validation of memory fixes or binary compatibility.
|
|
|
|
|
|
LOCAL WINDOWS TESTING PROCEDURE
|
|
|
|
This is what I do for routine testing on Windows.
|
|
|
|
From Windows, git clone from my Linux clone, and unzip
|
|
`external-libs`.
|
|
|
|
Look at `make_windows_releases`. Set up path the same way and run
|
|
whichever `./config-*` is appropriate for whichever compiler I need to
|
|
test with. Start one of the Visual Studio native compiler shells, and
|
|
from there, run one of the msys shells. The Visual Studio step is not
|
|
necessary if just building with mingw.
|