Why am I inventing my own documentation format? It's hard to say, but
it's probably because I Am Stupid. The main problem here will be lack
of automatic hyperlinks generated in the documentation. Oh well, it's
experimental.
This is an interesting change. While I think generally passing in
constant userdata is not terribly useful, the previous implementation
precluded it entirely. Interface types, for example, are often passed
directly and stored as constants (they hold pointers to their mutable
state).
Since we type erase this so it can be bound to the generic interface
object, non-pointer objects must be passed by reference to avoid
binding the parser interface to a temporary stack copy of the object.
This means we have to handle these cases slightly differently. Also,
while technically being classified as pointers, slices don't really
behave like pointers, which is understandable but annoying. There's a
bit of asymmetry here, as CommandBuilder(*u32) and CommandBuilder
(u32) both require an *u32 when binding the parser interface. This is
of course because pointers do not need to be rewrapped to be type
erased. The same code path could be used for both cases, but then the
user would have to pass in a pointer to a pointer, which actually
looks a bit silly because then it potentially means having to
do &&my_var.
This recognizes block labels. Actually implementing this successfully
took more attempts than I'd like to admit. I originally had a
streaming version using a tail queue, but it had problems that seemed
to be intractable. So instead, everything is just jammed in an
arraylist and processed as a whole once the tokenizing is complete. It
increases the maximum memory usage to store all the intermediates
during tokenizing the whole file, but it does work, and frankly I'm ok
with it using a few MB of memory. It can tokenize itself.
Even though the goal is for this to be run with an arena allocator,
nothing is currently enforcing that, so we should try to keep tidy.
This still leaks memory like crazy without an arena allocator, though.
Turns out the line wrapping logic is kind of ugly. I think probably a
couple of helper functions would make a big difference. But it appears
to work and even handles the edge cases I've currently encountered.
I've decided (perhaps stupidly) to eschew existing static site
generators and documentation templating systems. It seems fitting to
reinvent the wheel, which is something this project is entirely about
doing. The actual justification is a drama in a few very pathetic
parts:
a) existing documentation tools and static site generators are written
primarily in other programming languages. It feels bad to have a
dependency on a completely separate programming language ecosystem to
build the documentation.
b) existing documentation tools on average do a lot of things I don't
like. Things like trying to haul in google analytics and fonts and in
general having a truckload of javascript jammed in the frontend. The
page should have minimal or (ideally) no javascript at all.
c) modern documentation generators have really standardized on a three
column layout that looks flashy but doesn't make very good use of
space, and I think there's room to experiment with some ideas I've
seen while looking around.
So here we are. The approach being taken is to hand-roll html+css
directly, and that will hopefully converge to something that isn't
garbage. Once that has run its course, I will start considering what
level of automation makes sense to add.
Important takeaways:
1. Modern CSS is absurdly complex. It has had so many features jammed into
it over the years, it's completely nuts. I don't think it's possible
for any single person to store the details of all of it in their head.
2. Using non-tiny fonts takes up a lot of room quite quickly. Even if
you want to have a column larger than 800px, it's very easy to run out
of space quickly.
3. there's more than one way to make a list in ascii.
There's going to be more where this came from. Since this is our main
API, directly exposing all of the built-in functionality at this layer
is desirable.
Having thought about this more, it seems likely that complex converters
could benefit from being able to parse their arguments on the fly
without having to structure them into a rigid type. This is sort of a
get out of jail free card for custom converters as they can dump into
the user context type or whatever they want directly.
This needs a bit more thought. To support outside-in error writing,
converters need access to an allocator so they can create new buffer
writers. This can be done by passing in a pointer to an ArrayList
object directly, rather than an already bound writer.
I believe we've produced a superset of the functionality that was
present before rewriting all of the code.
There are still a lot of fiddly little details that need to be thought
through in order to produce something that is righteously flexible,
but I think this is in reasonable shape to start inventing real-world
uses for it.
Adding some tests, cleaning up a little bit of the allocation handling
(make better use of the arena allocators—we are definitely sort of
leaking memory at the moment), and writing documentation are still on
the roadmap.
I am resisting the urge to try to codegolf this into fewer lines. It's
going to end up being a sprawl, but it is what it is. The main part of
this that will actually require intelligent thought is the column
wrapping and alignment. I think I will probably be implementing a
custom writer style thing to handle that.
There are a lot of annoying loose odds and ends here. Choice types
should list all the choices. But we can't necessarily assume that an
enum-typed parameter is a choice type (only if it uses the default
converter). Perhaps the conversion stuff should be turned into an
interface that can also be responsible for converting the default
value and providing additional information. For now I will probably
just hack it so that I can move on to other things.
This wasn't fully implemented, and I don't actually think there's a
real reason for it to exist. It was cargo culted over from my own
thinking about click functionality, but API differences mean it just
isn't really useful here.
The main difference is that regular conversion now only happens after
the entire command line (including all subcommands) have been parsed.
This means that a failing normal converter won't prevent subcommand
eager converters from running. However, parsing errors can still
preclude eager converters and subcommand parsing from happening.
the checklist of things to do is continuing to dwindle. hooray. last
big feature push is help text generation. Then improving error
reporting. Then writing some tests. Then writing documentation.
Ay carumba.
The converters are hooked up, including some weird magic under the hood
to automatically handle multi-values as well as the many-to-one case
of the structure. The many-to-many case of arrays/slices is not
currently handled, but it should be straightforward to do.
Hooking up subcommands should be pretty straightforward if I've
designed this correctly, so that will be next. The final major piece
of the puzzle is help text generation, which in theory can be largely
carried over from the old implementation.
The error handling is very poorly done right now as well, and I need to
figure out a strategy. My plan for converters is to pass in a writable
buffer that the parser owns that they can write messages to when they
fail, to provide useful context. I will need to figure out how this
works for recursive converters (e.g. the struct converter) since the
failure happens inside-out, but the error message will read much
better if it's composed outside-in. There are many ways to tackle this.
The code will also need to be restructured to not be a monolithic 1000
line file.
Had to refactor the multi-value parameter stuff to push some of the key
information in to the generics structure, as they necessarily change
the conversion function signature. Some code has gotten folded
together by being a bit sloppier with inputs and outputs.
This should put us well on our way to having functioning value
conversion, which I think is the main major feature remaining besides
help text generation. Hopefully I won't need to rewrite everything
like this again. While this design seems to be on track to incorporate
all of the main features I am interested in, it has been a lot of work
to wrangle it around, and there is still a lot of work left before I
can put a bow on it.
Also work on a generic runtime parser interface for attaching
subcommands. This will allow subcommands to live in a mapping or
something at runtime which will simplify their use.
This is basically a full rewrite but with a much more solid concept of
what the public API looks like, which has informed some of the
lower-level decisions. This is not at feature parity with the main
branch yet, but it does handle some things better. The main
functionality missing is the help text generation and subcommands.
There's still some design to think about on the subcommand side of
things.
There are still quirks here. It's worth deciding if help descriptions
should be automatically hard-wrapped. Multi-line descriptions require
appropriate indentation after the first line. Long descriptions will
automatically be wrapped by the terminal.
The refactoring itch continues to grow.
This mostly works. Subcommands are utterly broken because we blindly
consume an additional argument to get the program name, which we
should not do.
This code was always kind of spaghetti, but it's getting worse. I want
to refactor it into something that doesn't make me cringe, but at the
same time, this project was intended to be a means to an end rather
than the end itself, and it kind of feels a bit silly to spend a ton
of time on it. On the other hand, relying on it for other projects
seems silly if it's a fragile mess. The goal was to get it into a
usable state and then hack on it as necessary, but it still has a ways
to go to get there, and working on it is kind of painful, in an
existential fashion.
Perhaps I will attempt to rewrite it, get halfway, and stall forever.
Thanks for reading my cool commit message blog. Bye.
For loop syntax changed. Also `orelse` apparently no longer works with
non-optional types, which is reasonable but annoying. The path of
least resistance is to make the flag default type optional to mirror
options/arguments.
The bakery bakes the user context type into an object so that it
doesn't have to be specified over and over again. This ends up being a
nicer way of specifying the CLI parameters, except for the fact that
it requires a slightly odd comptime block construct due to `var` not
working at the top level for some reason (and `comptime var` also not
working).
The user can provide a context type and corresponding value that will
get passed into any executed callbacks. This allows for complex
behavior through side effects and provides a mechanism by which the
user can pass an allocator into argument handlers, etc.
There was also a lot of restructuring in this including a bit more
automagical behavior, like making parameters that wrap optional types
default to being optional. The start of automatic handler picking
(user overridable, of course) is in place as well.
Needing to specify the userdata context type makes things a bit more
verbose, and there's some other jank I'm interested in trying to
remove. I have some ideas, but I don't know how far I can go in my
abuse of the compiler.
However, this seems like it will be usable once I get around to writing
the help text generation.
