The nixpkgs-unstable channel's programs.sqlite was used to identify
packages producing exactly one binary, and these automatically added
to their package definitions wherever possible.
* doc: fix and simplify stylesheets for the manuals, fix nrd bug
* Add anchorjs script to add links on section headers
* Fix another nrd bug, address style changes
* Use span instead of a for inline span syntax
Building a python environment with python3Minimal requires hydra
to bootstrap pip and build all packages used in the environment
which would otherwise not be built. This reduces cache re-use and duplicates things.
Also common dependencies normally included in python itself
are not properly checked and can cause hard to debug errors
because everyone just assumes those modules are there.
this was needed because the nixos/nixpkgs manuals, for some
reason, *did* drop the languages. we no longer have to render exactly as
the docbook toolchain did, so we can finally fix this bug.
This commit generalizes nixos-render-docs for manpage generation
to take the header preamble and footer postscript as file paths
as CLI flags --header and --footer.
parsing of escaped text has changed internally, but apart from Token
field contents that we only inspect in tests the change is *mostly*
invisible to us. some bits of documentation now parse differently and
need to be adjusted, but those were slightly questionable to begin with.
`children` only has to be not-None, not also not-[]. empty table cells
would can produce an empty inline token and cause these checks to fail,
even though the token actually is totally valid.
showed up in
https://github.com/NixOS/nixpkgs/pull/246956#discussion_r1283146120.
this is only used in the stdenv chapter, but footnotes could be useful
in other places as well. since markdown-it has a plugin to parse
footnote syntax we may as well just support them even if they're rare.
examples and figures behave identically regarding numbering and titling,
they just don't share a common number space. make the numbering/titling
function generic over block types now so figures can just use it.
currently only supported for html. docbook could also support images,
but it's on the way out for manual generation anyway so we won't add
image support there. options docs can't use images because they also
target manpages, which leaves no viable users.
the nixpkgs manual uses section tocs for eg the nixpkgs library function
reference. we will only not emit tocs for subsections at this point, but
maybe we should consider doing so in the future.
docbook always emits intra-file links if only a single file is emitted.
it doesn't seem to do this for intra-file links when multiple files are
emitted, so we don't do that yet either.
This automated de-linting has applied a few different refactors:
- Remove unused imports and variables
- Change f-strings with no variables to regular strings
- Remove trailing semicolon
docbook is now gone and we can flip the defaults. we won't keep the
command line args around (unlike the make-options-docs argument) because
nixos-render-docs should not be considered an exposed API.
the nixos manual contains enough examples to support them as a proper
toc entity with specialized rendering, and if in the future the nixpkgs
wants to use nixos-render-docs we will definitely have to support them.
this also allows us to restore some examples that were lost in previous
translation steps because there were too few to add renderer support
back then.
this converter is currently supposed to be able to reproduce the
docbook-generated html DOMs exactly, though not necessarily the
html *files*. it mirrors many docbook behaviours that seem rather odd,
such as top-level sections in chapters using the same heading depth as
understood by html as their parent chapters do. over time we can
hopefully remove all special casing needed to reproduce docbook
rendering, but for now at least it doesn't hurt *too* much.
this will be necessary for html since there we have to do chunking into
multiple files ourselves. writing one file from the caller of the
converter and all others from within the converter is unnecessarily
spread out, and returning a dict of file names and their contents is not
quite as meaningful for docbook (which has only one file to begin with).
it's not hooked up to anything yet, but that will come soon. there's a
bit of docbook compat here that must be interoperable with the actual
docbook exporter, but luckily it's not all that much.
the basic html renderer. it doesn't have all the docbook compatibility
codes embedded into it, but there is a good amount. this renderer is
unaware of manual structure and does not traverse structural include
tokens (if it finds any it'll just fail), that task falls to derived
classes. once we have more uses for structural includes than just the
manual we may revisit this decision.
the docbook toolchain uses docbook-xsl to generate its TOC, our html
renderer will have to do this on its own. this generator uses a very
straight-forward algorithm of only inspecting headings, but anything
else could be inspected as well. (examples come to mind, but those do
not have titles and would thus make for bad toc entries)
we also use path information (that will be taken from include block args
in the html renderer) to produce navigation information. the algorithm
we use mirrors what docbook does, linking to the next/previous files in
depth-first toc order.
toc entries are linked to the tokens they refer to for easy use later.
while docbook relies on external chunk-toc info to do chunking of the
rendered manual we have nothing of the sort for html. there it seems
easiest to add annotations to blocks to create new chunks. such
annotations could be extended to docbook to create the chunk-toc instead
of passing it in externally, but with docbook on the way out that seems
like a waste of effort.
text content in the toplevel file of a book will not render properly.
the first proper element will be a preface, part, or chapter anyway, and
those require includes to produce.
parts do not currently allow headings in the part file itself, but
that's mainly a renderer limitation. we can add support for headings in
part intros when we need them
in all other cases includes must be followed by either another include,
a heading, or end of file. text content could not be properly linked to
from a TOC without a preceding heading.
without this we cannot build a TOC to arbitrary depth without generating
ids for headings, but generated ids are fragile and liable to either
break or point to different things if the manual changes shape. we
already have the convention that all headings should have an id, this
formalizes it.
while not technically necessary for correct rendering of *contents* we
do need to disallow heading levels being skipped to build a correct
TOC. treating headings that have skipped a number of levels to actually
be headings that many levels up only gets confusing, and inserting
artifical intermediate headings suffers from problems, such as which ids
to use and what to call them.
check that all required headings are present during parsing, not during
rendering. building a correct TOC will need this since every TOC entry
needs a heading to set its title, and every included substructure needs
a title.
also improve the error message on repeated title headings slightly,
giving the end line turns out to not be very useful.
for most of our data classes we can use dataclasses.dataclass with
frozen=True or even plain named tuples. the TOC structure we'll need to
generate proper navigation links is most easily represented and used as
a cyclic structure though, and for that we can use neither. if we want
to make the TOC structures immutable (which seems like a good idea)
we'll need a hack of *some* kind, and this hack seems like the least intrusive.
we should really be rendering options at *rendering* time, not at parse
time. currently this is just an academic exercise, but the html renderer
will have to inspect the options.json data after the entire document has
been parsed, but before anything gets rendered.
these weren't used for anything. options never was (and does not contain
any information for the renderer that we *want* to honor), and env is
not used because typed renderer state is much more useful for all our cases.
our renderers carry significantly more state than markdown-it wants to
easily cater for, and the html renderer will need even more state still.
relying on the markdown-it-provided rendering functions has already
proven to be a nuisance, and since parsing and rendering are split well
enough we can just replace the rendering part with our own stuff outright.
this also frees us from the tyranny of having to set instance variables
before calling super().__init__ just to make sure that the renderer
creation callback has access to everything it needs.
the old method of pasting parts of options.json into a markdown document
and hoping for the best no longer works now that options.json contains
more than just docbook. given the infrastructure we have now we can
actually render options.md properly, so we may as well do that.
inline anchors are not allowed in option docs per the manual, and the
sole class we current have (.keycap) is never used anyway. disallow them
for now to avoid future surprises.
the same goes for examples, which aren't even documented in the manual yet.
move the restrictions we care about into a mixin class. a few more
restrictions will appear soon and a few new converters as well, the
renderers of which need not have these restrictions already baked in by
accident (like the manpage renderer does right now).
add the ability to set the info string for a newly created fenced code
block, and a flag to always emit a fenced block. the commonmark
converter will need this to faithfully recreate fenced and indented code
blocks.
