This was the final feature I wanted to add to the format. Also some
other things have been cleaned up a little bit (for example, the
inline parser does not need the dangling key to be attached to each
stack level just like the normal parser doesn't). There was also an
off-by-one error that bugged out detecting the pathological case of a
flow list consisting of only an empty string (`[ ]`, not to be
mistaken for the empty list `[]`).
Mixed multiline strings are a bit confusing but internally consistent.
> what character does this string end with?
|
ends with a newline character because that's the style of the
second-to-last line. However, seeing | last makes my brain think it
should end with a space. The reason it ends with a newline is because
our concatenation strategy consists of appending to the string early
(as soon as a line is added) rather than lazily. This is a tradeoff,
though. while lazy appending would make this result more intuitive
(the string would end with a space) and it would allow us to remove
the self-proclaimed cheesy hack, it would make the opposite boundary
condition confusing:
>
| what character does this string start with?
With lazy appending, this string would start with a space
(despite > making it look like it should have a leading newline).
While both of these are likely to be uncommon edge cases, it doesn't
seem we can have it both ways. Of the two options, I think the current
logic is a little bit more clear.
This was kind of a pain in the butt to implement because it basically
required a second full state machine parser (though this one is a bit
simpler since there are less possible value types). It seems likely to
me that I will probably shove this directly into the main parser
struct at some point in the near future.
There was no actual check that lines weren't being indented too far.
Inline strings weren't having their trailing newline get chopped.
Printing is still janky, but it's better than it was.
This hand-rolled wonder of switch statements is capable of parsing a 5
byte document in less than a gigasecond.
This was an interesting exercise in writing a non-recursive parser for
a nested structure format. There's a lot of very slightly different
repetition, which I'm not wild about, but it can handle deeply nested
documents. I tested with a 50 mb indented list tree document (10_000
lines of nesting) and a ReleaseFast build was able to parse it in
approximately 50 ms with a peak memory footprint of about 100 MB (of
which, half was the contents of the document itself, as the file is
read into a single allocated buffer that does not get freed until
program exit). I don't consider myself to be someone who writes high
performance software, but I think those results are quite acceptable,
and I doubt any recursive implementation would even be able to parse
that document at all (the python NestedText implementation smashes
directly into a RecursionError, unsurprisingly).
Anyway, let's call this a success. I will actually probably export this
to a separate project soon. The main problem is coming up with a name.
I also strongly suspect there are some lurking bugs still, and I think
I do want to add nested inline map/list support (and also parsing
directly into objects).
