this converts meta.doc into an md pointer, not an xml pointer. since we
no longer need xml for manual chapters we can also remove support for
manual chapters from md-to-db.sh
since pandoc converts smart quotes to docbook quote elements and our
nixos-render-docs does not we lose this distinction in the rendered
output. that's probably not that bad, our stylesheet didn't make use of
this anyway (and pre-23.05 versions of the chapters didn't use quote
elements either).
also updates the nixpkgs manual to clarify that option docs support all
extensions (although it doesn't support headings at all, so heading
anchors don't work by extension).
this is not yet able to produce manual-combined.xml, but the intention
is to add that support before too long. for now we'll concentrate on
getting the basics working: concatenating a list of chapters into a
manual-combined fragment, which will be rendered via docbook.
previously we did not detect whether lists were supposed to be compact
or not. this will make a difference for manual chapters, so let's stop
not doing that.
headings are not supported in options docs (since it's unclear what that
would be in the final manual, and the docbook stylesheets already have
trouble rendering all docbook constructs correctly). ordered lists
should be supported, but obviously nothing uses them yet.
options do not use comments, but a number of manual chapters do. since
we don't want to enable html just so we can then inspect the html and
figure out whether it's a comment we'll instead add a plugin that
detects comments natively.
we will soon add plugins to this tool to support nixos markdown features
that aren't readily supported with markdown-it plugins. since we will
have to test these plugin we'll need access to the parser, and since
we'll also want to add functions that require postprocessing of a parsed
token stream we also add the necessary hooks now.
with some fiddling and custom validation logic we can avoid needing
multiple instances of this plugin. originally this wasn't done because
it does need a type stack to emit correct docbook, but since we can
easily do that now we may as well.
using environment variables isn't great once multiple input or output
formats get involved (which will happen soon). now is a good time to set
a pattern for future converters.
while we won't have other converters (with other output formats) for a
while yet it still seems like a good idea to generalize *now* so we have
a pattern to follow.
these modules will be extended with more functionality. md will
ultimately contain MD-related code such as parsers and converter base
classes. docbook will contain renderers.
this new package shall eventually contain the rendering code necessary
to produce the entirety of the nixos (not nixpkgs) manual, in all of its
various output formats.
the rest of the nixos manual has them enabled, so we should enable them
here too for consistency.
this changes rendered output pervasively. changes also include quotes in
types (eg in `strings concatenated with "\n"`), but since those are not
code this is probably fine. if not we can probably add a myst role to
inhibit replacements.
the rules are fixed, and we want to support all of them (or throw a
useful error message). this will also become the base for a generic
renderer system, so let's just list all the rules statically.
as far as we can tell nixos has only ever had a total of one olink, and
currently has no olinks at all. we can't currently represent olinks in
markdown docs, and if we re-add support for cross-document links they
will take a different form (and not use docbook, which will have to be
phased out before we re-add anything).
the olinkdb is thus unused and takes 10 seconds on our machine to build,
holding up the rest of the manual for no benefit.
this restores mergeJSON to its former glory if…merging json, and
extracts the MD rendering into a new script that will run instead of the
py+nix+xslt pipeline we previously ran to convert options.json to docbook.
this change alone gives a noticable performance boost when building
docs (18s instead of 27s to build optionsDocBook).
no changes to rendered output, except for a single example in the
rsnapshot module that uses hard tabs for indentation instead of spaces.
this probably isn't important.
docbook warnings remain with mergeJSON since the other processing steps
output single files instead of directories. since we'll only keep the
check until 23.11 this is probably also not important to fix.
also contains a few improvements to error reporting in the MD renderers.
