This works, but there are probably interesting edge cases around how it
works when directly adjacent to other paragraphs. I will have to think
about it a bit. This wrapping code in general would benefit from term
queries.
Perhaps violating the principle of least astonishment quite severely is
the fact that "> a" and ">" are detected as preformatted, but ">a" is
normal wrapped text. Supporting both ">a" and "> a" leads to
nonobvious whitespace behavior, but this code should not be able to
runtime error outside of the writer dying. This may be reevaluated in
the future, but I will leave it as-is for now.
This just creates an empty command with an auto-assigned noop callback.
This is useful sugar for creating a group of commands under a common
name because previously the user would have to define their own noop
callback and bind it. This just takes a description string
(and, optionally, a help flag override).
This is a feeble attempt to unify some logic, as I realized that
Command.createInterface had different logic for handling the user
context than Parser did, which broke certain use cases (using a slice
as the context for example).
I'm not convinced this really unifies the logic as much as wraps it in
another layer of indirection, but at least the core problem is solved.
I think I had initially intended 0-length descriptions to be "hidden"
options, but this doesn't really work well with arguments, and it also
doesn't make intention clear. Perhaps an additional field should be
added to the parameter specification to support hiding options
(this does not make sense for non-named options).
This was kind of an annoying change to make since 0.11.0 has issues
where it will point to the wrong srcloc on compile errors in generic
code (which this 100% is) fortunately fixed in master. The motivation
for this change is that the arg vector already contains 0-terminated
strings, so we can avoid a lot of copies. This makes forwarding
command-line arguments to C-functions that expect zero-terminated
strings much more straightforward, and they also automatically decay
to normal slices.
Unfortunately, environment variable values are NOT zero-terminated, so
they are currently copied with zero-termination. This seems to be the
fault of Windows/WASI, both of which already are performing
allocations (Windows to convert from UTF-16 to UTF-8, and WASI to get
a copy of the environment). By duplicating the std EnvMap
implementation, we could make a version that generates 0-terminated
env vars without extra copies, but I'll skip on doing that for now.
This means that ParserInterface can do all of the important things that
Parser can do, which makes Command.createInterface a lot more useful
(there wasn't previously a way to add subcommands to an interface
created that way without a mass of extremely suspect casting).
This commit also makes the language around this. They're subcommands,
not children, and they have names, not verbs, associated with them.
Glad we could clear that up.
I think I still prefer snake_case stylistically, but this style fits in
a lot better with other zig code and is the officially endorsed style,
so it makes sense to use it. Since I don't have good test coverage,
some conversions may have been missed, but the demo still builds,
which is a decent amount of coverage. This was pretty easy to do with
a find and replace, so it should be reasonably thorough.
The main value of this method is that it allows runtime access to the
help description of the subcommand. This could allow implementation of
a help flag that takes the name of a subcommand to print help for or
something. Anyway, it's probably useful.
This allocates the interface with its own arena allocator, allowing it
to live beyond its stack lifetime. This enables some useful patterns
for composing a CLI from multiple functions or files. This is actually
probably the preferred method over `create_parser` in most
circumstances.
There are a couple of other places where []u8 is treated implicitly
like a string, which isn't strictly correct. Ultimately, some kind of
metasignal will be required to make this type truly unambiguous in
interpretation.
Stay a while and listen to my story.
Due to the design of the parser execution flow, the only reasonable way
to avoid leaking memory in the parser is to use an arena allocator
because the parser itself doesn't have direct access to everything it
allocates, and everything it allocates needs to live for the duration
of whatever callbacks are called.
Now, you might say, if the items it allocates are stored for the
lifetime of whatever callbacks, then that means that the items it
allocates stay allocated for effectively the entire life of the
program. In which case there's really not much point in freeing them
at all, as it's just extra work on exit that the OS should normally
clean up. And you'd be right, except for two details: if the user uses
the current GeneralPurposeAllocator, it will complain about leaks when
deinitialized, which simply isn't groovy. The other detail is that
technically the user can run any code they want after the parser
execution finishes, so forcing the user to leak memory by having an
incomplete API is rude.
The other option would be, as before, forcing the user to supply their
own arena allocator if they don't want to leak, but that's kind of a
rude thing to do and goes against the "all allocators look the same"
design of the standard library, which is what makes it so easy to use
and create allocators with advanced functionality. That seems like an
ugly thing to do, so, instead, each parser gets to eat the memory cost
of storing a pointer to its arena allocator (and the heap cost of the
arena allocator itself).
In theory, subcommands could borrow the arena allocator of their parent
command to save a bit of heap space, but that would make a variety of
creation and cleanup-related tasks less isomorphic between the parents
and the subcommands. I like the current design where commands and
subcommands are the same thing, and I'm not in a rush to disturb that.
I don't think the overhead cost of the arena allocator itself, which
can be measured in double digit bytes, is a particularly steep price
to pay.
enumToInt changed to intFromEnum, and the casting builtins figured out
how to automagically infer the cast type. This results in some minor
simplification, which is nice.
At some point, (probably during the llvm 16 upgrade, though I haven't
done the legwork to actually narrow it down), zig developed a crash
around the way inline was used here. Since using `inline` was an air
quotes optimization, we can just chuck the designation for the time
being so that compilation will succeed. This may remove more inlines
than is strictly necessary, but I am bravely willing to make that
sacrifice.
See: ziglang/zig#15668
Why am I inventing my own documentation format? It's hard to say, but
it's probably because I Am Stupid. The main problem here will be lack
of automatic hyperlinks generated in the documentation. Oh well, it's
experimental.
This is an interesting change. While I think generally passing in
constant userdata is not terribly useful, the previous implementation
precluded it entirely. Interface types, for example, are often passed
directly and stored as constants (they hold pointers to their mutable
state).
Since we type erase this so it can be bound to the generic interface
object, non-pointer objects must be passed by reference to avoid
binding the parser interface to a temporary stack copy of the object.
This means we have to handle these cases slightly differently. Also,
while technically being classified as pointers, slices don't really
behave like pointers, which is understandable but annoying. There's a
bit of asymmetry here, as CommandBuilder(*u32) and CommandBuilder
(u32) both require an *u32 when binding the parser interface. This is
of course because pointers do not need to be rewrapped to be type
erased. The same code path could be used for both cases, but then the
user would have to pass in a pointer to a pointer, which actually
looks a bit silly because then it potentially means having to
do &&my_var.
This recognizes block labels. Actually implementing this successfully
took more attempts than I'd like to admit. I originally had a
streaming version using a tail queue, but it had problems that seemed
to be intractable. So instead, everything is just jammed in an
arraylist and processed as a whole once the tokenizing is complete. It
increases the maximum memory usage to store all the intermediates
during tokenizing the whole file, but it does work, and frankly I'm ok
with it using a few MB of memory. It can tokenize itself.
Even though the goal is for this to be run with an arena allocator,
nothing is currently enforcing that, so we should try to keep tidy.
This still leaks memory like crazy without an arena allocator, though.
Turns out the line wrapping logic is kind of ugly. I think probably a
couple of helper functions would make a big difference. But it appears
to work and even handles the edge cases I've currently encountered.
I've decided (perhaps stupidly) to eschew existing static site
generators and documentation templating systems. It seems fitting to
reinvent the wheel, which is something this project is entirely about
doing. The actual justification is a drama in a few very pathetic
parts:
a) existing documentation tools and static site generators are written
primarily in other programming languages. It feels bad to have a
dependency on a completely separate programming language ecosystem to
build the documentation.
b) existing documentation tools on average do a lot of things I don't
like. Things like trying to haul in google analytics and fonts and in
general having a truckload of javascript jammed in the frontend. The
page should have minimal or (ideally) no javascript at all.
c) modern documentation generators have really standardized on a three
column layout that looks flashy but doesn't make very good use of
space, and I think there's room to experiment with some ideas I've
seen while looking around.
So here we are. The approach being taken is to hand-roll html+css
directly, and that will hopefully converge to something that isn't
garbage. Once that has run its course, I will start considering what
level of automation makes sense to add.
Important takeaways:
1. Modern CSS is absurdly complex. It has had so many features jammed into
it over the years, it's completely nuts. I don't think it's possible
for any single person to store the details of all of it in their head.
2. Using non-tiny fonts takes up a lot of room quite quickly. Even if
you want to have a column larger than 800px, it's very easy to run out
of space quickly.
3. there's more than one way to make a list in ascii.
There's going to be more where this came from. Since this is our main
API, directly exposing all of the built-in functionality at this layer
is desirable.
Having thought about this more, it seems likely that complex converters
could benefit from being able to parse their arguments on the fly
without having to structure them into a rigid type. This is sort of a
get out of jail free card for custom converters as they can dump into
the user context type or whatever they want directly.
This needs a bit more thought. To support outside-in error writing,
converters need access to an allocator so they can create new buffer
writers. This can be done by passing in a pointer to an ArrayList
object directly, rather than an already bound writer.
I believe we've produced a superset of the functionality that was
present before rewriting all of the code.
There are still a lot of fiddly little details that need to be thought
through in order to produce something that is righteously flexible,
but I think this is in reasonable shape to start inventing real-world
uses for it.
Adding some tests, cleaning up a little bit of the allocation handling
(make better use of the arena allocators—we are definitely sort of
leaking memory at the moment), and writing documentation are still on
the roadmap.
I am resisting the urge to try to codegolf this into fewer lines. It's
going to end up being a sprawl, but it is what it is. The main part of
this that will actually require intelligent thought is the column
wrapping and alignment. I think I will probably be implementing a
custom writer style thing to handle that.
There are a lot of annoying loose odds and ends here. Choice types
should list all the choices. But we can't necessarily assume that an
enum-typed parameter is a choice type (only if it uses the default
converter). Perhaps the conversion stuff should be turned into an
interface that can also be responsible for converting the default
value and providing additional information. For now I will probably
just hack it so that I can move on to other things.
This wasn't fully implemented, and I don't actually think there's a
real reason for it to exist. It was cargo culted over from my own
thinking about click functionality, but API differences mean it just
isn't really useful here.
The main difference is that regular conversion now only happens after
the entire command line (including all subcommands) have been parsed.
This means that a failing normal converter won't prevent subcommand
eager converters from running. However, parsing errors can still
preclude eager converters and subcommand parsing from happening.
