Array optimization fixes performance issues (fixes #305, #311)

This commit is contained in:
Jay Berkenbilt 2019-08-17 22:37:05 -04:00
parent 522d2b2227
commit 89dacd8eba
3 changed files with 16 additions and 7 deletions

View File

@ -1,3 +1,11 @@
2019-08-17 Jay Berkenbilt <ejb@ql.org>
* Change internal implementation of QPDF arrays to use sparse
arrays, which results in using much less memory for arrays with
large numbers of nulls. Various files have been encountered in the
wild that contains thousands of arrays with millions of nulls.
Fixes #305, #311.
2019-07-03 Jay Berkenbilt <ejb@ql.org>
* Non-compatible API change: change

7
TODO
View File

@ -1,10 +1,3 @@
Performance
===========
There is some performance problem that happens with some large files.
Issues 305 and 311 both seem to be examples of this. Files saved in
../misc/bugs.
Fuzz Errors
===========

View File

@ -4480,6 +4480,14 @@ print "\n";
<function>getUIntValueAsUInt</function>.
</para>
</listitem>
<listitem>
<para>
The underlying implementation of QPDF arrays has been
enhanced to be much more memory efficient when dealing with
arrays with lots of nulls. This enables qpdf to use
drastically less memory for certain types of files.
</para>
</listitem>
<listitem>
<para>
A new helper method