Test json against schema only on demand

Testing json against schema requires an in-memory copy, so do it only
when requested by the test suite.
This commit is contained in:
Jay Berkenbilt 2022-05-06 17:49:28 -04:00
parent a3c9980395
commit 7f65a5c21f
5 changed files with 35 additions and 33 deletions

25
TODO
View File

@ -50,18 +50,6 @@ Output JSON v2
Before starting on v2 format:
* Some if not all of the json output functionality should move from
QPDFJob to QPDF. There can top-level QPDF methods that take a
pipeline and write the JSON serialization to it. For things that
generate smaller amounts of output (constant-size stuff, lists of
attachments), we can also have a version that returns a string. For
the benefit of users of other languages, we can have something that
takes a FILE* or writes to stdout as well. This would be a good time
to make sure all the information from --check and other
informational options (--show-linearization, --show-encryption,
--show-xref, --list-attachments, --show-npages) is available in the
json output.
* Writing objects should write in numerical order with the trailer at
the end.
@ -70,15 +58,18 @@ Before starting on v2 format:
the input), not by overwriting, in case this has any unwanted side
effects.
* Figure out how/whether to do schema checks with incremental write.
Consider changing the contract to allow fields to be absent even
when present in the schema. It's reasonable for people to check for
presence of a key. Most languages make this easy to do.
General things to remember:
* deprecate getJSON without a version
* Make sure all the information from --check and other informational
options (--show-linearization, --show-encryption, --show-xref,
--list-attachments, --show-npages) is available in the json output.
* Consider changing the contract to allow fields to be absent even
when present in the schema. It's reasonable for people to check for
presence of a key. Most languages make this easy to do.
* The choices for json_key (job.yml) will be different for v1 and v2.
That information is already duplicated in multiple places.

View File

@ -14,4 +14,4 @@ libqpdf/qpdf/auto_job_json_decl.hh 06caa46eaf71db8a50c046f91866baa8087745a947431
libqpdf/qpdf/auto_job_json_init.hh e7047a7c83737adfaae49abc295a579bb9b9e0a4644e911d1656a604cb202208
libqpdf/qpdf/auto_job_schema.hh cbbcae166cfecbdbdeb40c5a30870e03604a019a8b4f7a217d554a82431d2e5f
manual/_ext/qpdf.py 6add6321666031d55ed4aedf7c00e5662bba856dfcd66ccb526563bffefbb580
manual/cli.rst 8a5a12351df6a42f91d6d271b2d065a843d8daa5125d8723d474e4180d7abbf1
manual/cli.rst 8684ca1f601f2832cded52d1b2f74730f97b7b85b57e31a399231731fbe80d26

View File

@ -16,6 +16,7 @@
#include <qpdf/Pl_Flate.hh>
#include <qpdf/Pl_OStream.hh>
#include <qpdf/Pl_StdioFile.hh>
#include <qpdf/Pl_String.hh>
#include <qpdf/QTC.hh>
#include <qpdf/QUtil.hh>
@ -1592,6 +1593,13 @@ QPDFJob::json_out_schema_v1()
void
QPDFJob::doJSON(QPDF& pdf, Pipeline* p)
{
std::string captured_json;
std::shared_ptr<Pl_String> pl_str;
if (this->m->test_json_schema) {
pl_str = std::make_shared<Pl_String>("capture json", p, captured_json);
p = pl_str.get();
}
JSON j = JSON::makeDictionary();
// This version is updated every time a non-backward-compatible
// change is made to the JSON format. Clients of the JSON are to
@ -1651,23 +1659,22 @@ QPDFJob::doJSON(QPDF& pdf, Pipeline* p)
doJSONObjectinfo(pdf, j);
}
// Check against schema
*p << j.unparse() << "\n";
JSON schema = json_schema(&m->json_keys);
std::list<std::string> errors;
if (!j.checkSchema(schema, errors)) {
*(this->m->cerr)
<< "QPDFJob didn't create JSON that complies with its own rules.\n\
Please report this as a bug at\n\
https://github.com/qpdf/qpdf/issues/new\n\
ideally with the file that caused the error and the output below. Thanks!\n\
\n";
for (auto const& error: errors) {
*(this->m->cerr) << error << std::endl;
if (this->m->test_json_schema) {
// Check against schema
JSON schema = json_schema(&m->json_keys);
std::list<std::string> errors;
JSON captured = JSON::parse(captured_json);
if (!captured.checkSchema(schema, errors)) {
*(this->m->cerr) << "QPDFJob didn't create JSON that complies with"
" its own rules."
<< std::endl;
for (auto const& error: errors) {
*(this->m->cerr) << error << std::endl;
}
}
}
*p << j.unparse() << "\n";
}
void

View File

@ -3336,6 +3336,9 @@ Related Options
This is used by qpdf's test suite to check consistency between the
output of ``qpdf --json`` and the output of ``qpdf --json-help``.
This option causes an extra copy of the generated json to appear in
memory and is therefore unsuitable for use with large files. This
is why it's also not on by default.
.. _unicode-passwords:

View File

@ -1124,7 +1124,8 @@ foreach my $d (@json_files)
}
my $in = "$file.pdf";
$td->runtest("json $out",
{$td->COMMAND => ['qpdf', '--json', @$xargs, $in]},
{$td->COMMAND =>
['qpdf', '--json=1', '--test-json-schema', @$xargs, $in]},
{$td->FILE => "$out.out", $td->EXIT_STATUS => 0},
$td->NORMALIZE_NEWLINES);
}