torque 7f82c24584
parser: implement streaming parser
With my pathological 50MiB 10_000 line nested list test, this is
definitely slower than the one shot parser, but it has peak memory
usage of 5MiB compared to the 120MiB of the one-shot parsing. Not bad.
Obviously this result is largely dependent on the fact that this
particular benchmark is 99% whitespace, which does not get copied into
the resulting document. A (significantly) smaller improvement will be
observed in files that are mostly data with little indentation or
empty lines.

But a win is a win.
2023-09-25 01:18:09 -07:00
..
2023-09-24 15:14:58 -07:00
2023-09-25 01:18:09 -07:00