nixos-render-docs: init redirects system

Co-authored-by: Valentin Gagarin <valentin@gagarin.work>

ci/OWNERS

@@ -105,6 +105,11 @@ nixos/modules/installer/tools/nix-fallback-paths.nix  @NixOS/nix-team @raitobeza
 /nixos/modules/system/activation/bootspec.nix         @grahamc @cole-h @raitobezarius
 /nixos/modules/system/activation/bootspec.cue         @grahamc @cole-h @raitobezarius
 
+# NixOS Render Docs
+/pkgs/by-name/ni/nixos-render-docs @fricklerhandwerk @GetPsyched @hsjobeki
+/doc/redirects.json                @fricklerhandwerk @GetPsyched @hsjobeki
+/nixos/doc/manual/redirects.json   @fricklerhandwerk @GetPsyched @hsjobeki
+
 # NixOS integration test driver
 /nixos/lib/test-driver  @tfc
 

doc/README.md

@@ -21,7 +21,7 @@ Rendered documentation:
 - [Unstable (from master)](https://nixos.org/manual/nixpkgs/unstable/)
 - [Stable (from latest release)](https://nixos.org/manual/nixpkgs/stable/)
 
-The rendering tool is [nixos-render-docs](../pkgs/by-name/ni/nixos-render-docs/src/nixos_render_docs), sometimes abbreviated `nrd`.
+The rendering tool is [nixos-render-docs](../pkgs/by-name/ni/nixos-render-docs), sometimes abbreviated `nrd`.
 
 ## Contributing to this documentation
 
@@ -42,6 +42,12 @@ It is a daemon, that:
 2. HTTP serves the manual, injecting a script that triggers reload on changes
 3. opens the manual in the default browser
 
+### Testing redirects
+
+Once you have a successful build, you can open the relevant HTML (path mentioned above) in a browser along with the anchor, and observe the redirection.
+
+Note that if you already loaded the page and *then* input the anchor, you will need to perform a reload. This is because browsers do not re-run client JS code when only the anchor has changed.
+
 ## Syntax
 
 As per [RFC 0072](https://github.com/NixOS/rfcs/pull/72), all new documentation content should be written in [CommonMark](https://commonmark.org/) Markdown dialect.

doc/doc-support/package.nix

@@ -31,6 +31,7 @@ stdenvNoCC.mkDerivation (
         ../anchor-use.js
         ../anchor.min.js
         ../manpage-urls.json
+        ../redirects.json
       ];
     };
 
@@ -62,6 +63,7 @@ stdenvNoCC.mkDerivation (
 
       nixos-render-docs manual html \
         --manpage-urls ./manpage-urls.json \
+        --redirects ./redirects.json \
         --revision ${nixpkgs.rev or "master"} \
         --stylesheet style.css \
         --stylesheet highlightjs/mono-blue.css \

nixos/doc/manual/contributing-to-this-manual.chapter.md

@@ -17,6 +17,12 @@ There's also [a convenient development daemon](https://nixos.org/manual/nixpkgs/
 
 The above instructions don't deal with the appendix of available `configuration.nix` options, and the manual pages related to NixOS. These are built, and written in a different location and in a different format, as explained in the next sections.
 
+## Testing redirects {#sec-contributing-redirects}
+
+Once you have a successful build, you can open the relevant HTML (path mentioned above) in a browser along with the anchor, and observe the redirection.
+
+Note that if you already loaded the page and *then* input the anchor, you will need to perform a reload. This is because browsers do not re-run client JS code when only the anchor has changed.
+
 ## Contributing to the `configuration.nix` options documentation {#sec-contributing-options}
 
 The documentation for all the different `configuration.nix` options is automatically generated by reading the `description`s of all the NixOS options defined at `nixos/modules/`. If you want to improve such `description`, find it in the `nixos/modules/` directory, and edit it and open a pull request.

nixos/doc/manual/default.nix

@@ -122,6 +122,7 @@ in rec {
 
       nixos-render-docs -j $NIX_BUILD_CORES manual html \
         --manpage-urls ${manpageUrls} \
+        --redirects ${./redirects.json} \
         --revision ${escapeShellArg revision} \
         --generator "nixos-render-docs ${pkgs.lib.version}" \
         --stylesheet style.css \

pkgs/by-name/ni/nixos-render-docs/README.md

@@ -0,0 +1,77 @@
+# nixos-render-docs
+
+A [CommonMark](https://commonmark.org/) and [`man-pages`](https://www.man7.org/linux/man-pages/man7/man-pages.7.html) renderer for the NixOS and Nixpkgs manuals.
+
+## Summary
+
+`nixos-render-docs` implements [RFC 72](https://github.com/NixOS/rfcs/pull/72) and has enabled a lossless port of Nixpkgs and NixOS documentation, which was originally written in the [DocBook](https://docbook.org/whatis) format, to [CommonMark](https://commonmark.org/) with [custom extensions](../../../../doc/README.md#syntax).
+
+Maintaining our own documentation rendering framework may appear extreme but has practical advantages:
+- We never have to work around existing tools made under different assumptions
+- We don't have to deal with unexpected breakage
+- We can grow the framework with our evolving requirements without relying on external support or approval or the need to maintain a small diff to upstream
+- The amount of code involved is minimal because it's single-purpose
+
+Several alternatives to `nixos-render-docs` were discussed in the past.
+A detailed analysis can be found in a [table comparing documentation rendering framework](https://ethercalc.net/dc4vcnnl8zv0).
+
+## Redirects system
+
+Moving contents around can cause links to break.
+
+Since we have our own markdown parser, we can hook into the rendering process to extract all of the metadata around each content identifier.
+The [mechanism for checking correctness of redirects](./src/nixos_render_docs/redirects.py) takes the collection of identifiers and a mapping of the identified content to its historical locations in the output.
+It validates them against a set of rules, and creates a client-side redirect mapping for each output file, as well as a `_redirects` file for server-side redirects in [Netlify syntax](https://docs.netlify.com/routing/redirects/#syntax-for-the-redirects-file).
+
+This allows us to catch:
+- Identifiers that were removed or renamed
+- Content that was moved from one location to another
+- Various consistency errors in the redirects mapping
+
+### Design considerations
+
+The creation, movement, and removal of every identifier is captured in the Git history.
+However, analysing hundreds of thousands of commits during the build process is impractical.
+
+The chosen design is a trade-off between speed, repository size, and contributor friction:
+- Stricter checks always require more attention from contributors
+    - Checks should be reasonably fast and ideally happen locally, e.g. as part of the build, as anything else will substantially lengthen the feedback cycle.
+- Computing redirects against previous revisions of the repository would be more space-efficient, but impractically slow.
+    - It would also require keeping an impure or otherwise continuously updated reference to those other revisions.
+    - The static mapping acts like a semi-automatically updated cache that we drag along with version history.
+    - Other setups, such as a dedicated service to cache a history of moved content, are more complicated and would still be impure.
+- Checking in large amounts of data that is touched often, bears a risk of more merge conflicts or related build failures.
+
+The solution picked here is to have a static mapping of the historical locations checked into the Git tree, such that it can be read during the build process.
+This also ensures that an improper redirect mapping will cause `nixos-render-docs` to fail the build and thus enforce that redirects stay up-to-date with every commit.
+
+### Redirects Mapping Structure
+
+Here's an overview of this mapping:
+
+```json
+{
+  "<identifier>": [
+    "index.html#<identifier>",
+    "foo.html#foo",
+    "bar.html#foo"
+  ]
+}
+```
+
+- The keys of this mapping _must_ be an exhaustive list of all identifiers in the source files.
+- The first element of the value of this mapping _must_ be the current output location (path and anchor) of the content signified by the identifier in the mapping key.
+  - While the order of the remaining elements is unconstrained, please only prepend to this list when the content under the indentifier moves in order to keep the diffs readable.
+
+In case this identifier is renamed, the mapping would change into:
+
+```json
+{
+  "<identifier-new>": [
+    "index.html#<identifier-new>",
+    "foo.html#<identifier>",
+    "bar.html#foo",
+    "index.html#<identifier>"
+  ]
+}
+```

pkgs/by-name/ni/nixos-render-docs/src/nixos_render_docs/manual.py

@@ -16,6 +16,7 @@ from . import md, options
 from .html import HTMLRenderer, UnresolvedXrefError
 from .manual_structure import check_structure, FragmentType, is_include, make_xml_id, TocEntry, TocEntryType, XrefTarget
 from .md import Converter, Renderer
+from .redirects import Redirects
 
 class BaseConverter(Converter[md.TR], Generic[md.TR]):
     # per-converter configuration for ns:arg=value arguments to include blocks, following
@@ -216,14 +217,16 @@ class ManualHTMLRenderer(RendererMixin, HTMLRenderer):
     _base_path: Path
     _in_dir: Path
     _html_params: HTMLParameters
+    _redirects: Redirects
 
     def __init__(self, toplevel_tag: str, revision: str, html_params: HTMLParameters,
                  manpage_urls: Mapping[str, str], xref_targets: dict[str, XrefTarget],
-                 in_dir: Path, base_path: Path):
+                 redirects: Redirects, in_dir: Path, base_path: Path):
         super().__init__(toplevel_tag, revision, manpage_urls, xref_targets)
         self._in_dir = in_dir
         self._base_path = base_path.absolute()
         self._html_params = html_params
+        self._redirects = redirects
 
     def _pull_image(self, src: str) -> str:
         src_path = Path(src)
@@ -306,6 +309,11 @@ class ManualHTMLRenderer(RendererMixin, HTMLRenderer):
                 '   <hr />',
                 '  </div>',
             ])
+
+        redirects_path = f'{self._base_path}/{toc.target.path.split('.html')[0]}-redirects.js'
+        with open(redirects_path, 'w') as file:
+            file.write(self._redirects.get_redirect_script(toc.target.path))
+
         return "\n".join([
             '<?xml version="1.0" encoding="utf-8" standalone="no"?>',
             '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"',
@@ -317,7 +325,7 @@ class ManualHTMLRenderer(RendererMixin, HTMLRenderer):
             "".join((f'<link rel="stylesheet" type="text/css" href="{html.escape(style, True)}" />'
                      for style in self._html_params.stylesheets)),
             "".join((f'<script src="{html.escape(script, True)}" type="text/javascript"></script>'
-                     for script in self._html_params.scripts)),
+                     for script in [*self._html_params.scripts, redirects_path])),
             f' <meta name="generator" content="{html.escape(self._html_params.generator, True)}" />',
             f' <link rel="home" href="{home.target.href()}" title="{home.target.title}" />' if home.target.href() else "",
             f' {up_link}{prev_link}{next_link}',
@@ -501,6 +509,7 @@ class HTMLConverter(BaseConverter[ManualHTMLRenderer]):
     _revision: str
     _html_params: HTMLParameters
     _manpage_urls: Mapping[str, str]
+    _redirects: Redirects
     _xref_targets: dict[str, XrefTarget]
     _redirection_targets: set[str]
     _appendix_count: int = 0
@@ -509,9 +518,9 @@ class HTMLConverter(BaseConverter[ManualHTMLRenderer]):
         self._appendix_count += 1
         return _to_base26(self._appendix_count - 1)
 
-    def __init__(self, revision: str, html_params: HTMLParameters, manpage_urls: Mapping[str, str]):
+    def __init__(self, revision: str, html_params: HTMLParameters, manpage_urls: Mapping[str, str], redirects: Redirects):
         super().__init__()
-        self._revision, self._html_params, self._manpage_urls = revision, html_params, manpage_urls
+        self._revision, self._html_params, self._manpage_urls, self._redirects = revision, html_params, manpage_urls, redirects
         self._xref_targets = {}
         self._redirection_targets = set()
         # renderer not set on purpose since it has a dependency on the output path!
@@ -519,7 +528,7 @@ class HTMLConverter(BaseConverter[ManualHTMLRenderer]):
     def convert(self, infile: Path, outfile: Path) -> None:
         self._renderer = ManualHTMLRenderer(
             'book', self._revision, self._html_params, self._manpage_urls, self._xref_targets,
-            infile.parent, outfile.parent)
+            self._redirects, infile.parent, outfile.parent)
         super().convert(infile, outfile)
 
     def _parse(self, src: str, *, auto_id_prefix: None | str = None) -> list[Token]:
@@ -670,7 +679,13 @@ class HTMLConverter(BaseConverter[ManualHTMLRenderer]):
                 )
 
         TocEntry.collect_and_link(self._xref_targets, tokens)
-
+        self._redirects.validate(self._xref_targets)
+        server_redirects = self._redirects.get_server_redirects()
+        with open(outfile.parent / '_redirects', 'w') as server_redirects_file:
+            formatted_server_redirects = []
+            for from_path, to_path in server_redirects.items():
+                formatted_server_redirects.append(f"{from_path} {to_path} 301")
+            server_redirects_file.write("\n".join(formatted_server_redirects))
 
 
 def _build_cli_html(p: argparse.ArgumentParser) -> None:
@@ -683,16 +698,22 @@ def _build_cli_html(p: argparse.ArgumentParser) -> None:
     p.add_argument('--chunk-toc-depth', default=1, type=int)
     p.add_argument('--section-toc-depth', default=0, type=int)
     p.add_argument('--media-dir', default="media", type=Path)
+    p.add_argument('--redirects', type=Path)
     p.add_argument('infile', type=Path)
     p.add_argument('outfile', type=Path)
 
 def _run_cli_html(args: argparse.Namespace) -> None:
-    with open(args.manpage_urls, 'r') as manpage_urls:
+    with open(args.manpage_urls) as manpage_urls, open(Path(__file__).parent / "redirects.js") as redirects_script:
+        redirects = {}
+        if args.redirects:
+            with open(args.redirects) as raw_redirects:
+                redirects = json.load(raw_redirects)
+
         md = HTMLConverter(
             args.revision,
             HTMLParameters(args.generator, args.stylesheet, args.script, args.toc_depth,
                            args.chunk_toc_depth, args.section_toc_depth, args.media_dir),
-            json.load(manpage_urls))
+            json.load(manpage_urls), Redirects(redirects, redirects_script.read()))
         md.convert(args.infile, args.outfile)
 
 def build_cli(p: argparse.ArgumentParser) -> None:

pkgs/by-name/ni/nixos-render-docs/src/nixos_render_docs/redirects.js

@@ -0,0 +1,3 @@
+const anchor = document.location.hash.substring(1);
+const redirects = REDIRECTS_PLACEHOLDER;
+if (redirects[anchor]) document.location.href = redirects[anchor];

pkgs/by-name/ni/nixos-render-docs/src/nixos_render_docs/redirects.py

@@ -0,0 +1,174 @@
+import json
+from dataclasses import dataclass, field
+from pathlib import Path
+
+from .manual_structure import XrefTarget
+
+
+class RedirectsError(Exception):
+    def __init__(
+        self,
+        conflicting_anchors: set[str] = None,
+        divergent_redirects: set[str] = None,
+        identifiers_missing_current_outpath: set[str] = None,
+        identifiers_without_redirects: set[str] = None,
+        orphan_identifiers: set[str] = None
+    ):
+        self.conflicting_anchors = conflicting_anchors or set()
+        self.divergent_redirects = divergent_redirects or set()
+        self.identifiers_missing_current_outpath = identifiers_missing_current_outpath or set()
+        self.identifiers_without_redirects = identifiers_without_redirects or set()
+        self.orphan_identifiers = orphan_identifiers or set()
+
+    def __str__(self):
+        error_messages = []
+        if self.conflicting_anchors:
+            error_messages.append(f"""
+Identifiers must not be identical to any historical location's anchor of the same output path.
+    The following identifiers violate this rule:
+    - {"\n    - ".join(self.conflicting_anchors)}
+
+    This can break links or redirects. If you added new content, choose a different identifier.
+""")
+        if self.divergent_redirects:
+            error_messages.append(f"""
+All historical content locations must correspond to exactly one identifier.
+    The following locations violate this rule:
+    - {"\n    - ".join(self.divergent_redirects)}
+
+    It leads to inconsistent behavior depending on which redirect is applied.
+    Please update doc/redirects.json or nixos/doc/manual/redirects.json!
+""")
+        if self.identifiers_missing_current_outpath:
+            error_messages.append(f"""
+The first element of an identifier's redirects list must denote its current location.
+    The following identifiers violate this rule:
+    - {"\n    - ".join(self.identifiers_missing_current_outpath)}
+
+    If you moved content, add its new location as the first element of the redirects mapping.
+    Please update doc/redirects.json or nixos/doc/manual/redirects.json!
+""") # TODO: automatically detect if you just missed adding a new location, and make a tool to do that for you
+        if self.identifiers_without_redirects:
+            error_messages.append(f"""
+Identifiers present in the source must have a mapping in the redirects file.
+    - {"\n    - ".join(self.identifiers_without_redirects)}
+
+    This can happen when an identifier was added or renamed.
+    Please update doc/redirects.json or nixos/doc/manual/redirects.json!
+""") # TODO: add tooling in the development shell to do that automatically and point to that command
+        if self.orphan_identifiers:
+            error_messages.append(f"""
+Keys of the redirects mapping must correspond to some identifier in the source.
+    - {"\n    - ".join(self.orphan_identifiers)}
+
+    This can happen when an identifier was removed or renamed.
+    Please update doc/redirects.json or nixos/doc/manual/redirects.json!
+""") # TODO: add tooling in the development shell to do that automatically and point to that command
+
+        error_messages.append("NOTE: If your Manual build passes locally and you see this message in CI, you probably need a rebase.")
+        return "\n".join(error_messages)
+
+
+@dataclass
+class Redirects:
+    _raw_redirects: dict[str, list[str]]
+    _redirects_script: str
+
+    _xref_targets: dict[str, XrefTarget] = field(default_factory=dict)
+
+    def validate(self, initial_xref_targets: dict[str, XrefTarget]):
+        """
+        Validate redirection mappings against element locations in the output
+
+        - Ensure semantic correctness of the set of redirects with the following rules:
+          - Identifiers present in the source must have a mapping in the redirects file
+          - Keys of the redirects mapping must correspond to some identifier in the source
+          - All historical content locations must correspond to exactly one identifier
+          - Identifiers must not be identical to any historical location's anchor of the same output path
+          - The first element of an identifier's redirects list must denote its current location.
+        """
+        xref_targets = {}
+        ignored_identifier_patterns = ("opt-", "auto-generated-", "function-library-")
+        for id, target in initial_xref_targets.items():
+            # filter out automatically generated identifiers from module options and library documentation
+            if id.startswith(ignored_identifier_patterns):
+                continue
+            xref_targets[id] = target
+
+        identifiers_without_redirects = xref_targets.keys() - self._raw_redirects.keys()
+        orphan_identifiers = self._raw_redirects.keys() - xref_targets.keys()
+
+        client_side_redirects = {}
+        server_side_redirects = {}
+        conflicting_anchors = set()
+        divergent_redirects = set()
+        identifiers_missing_current_outpath = set()
+
+        for identifier, locations in self._raw_redirects.items():
+            if identifier not in xref_targets:
+                continue
+
+            if not locations or locations[0] != f"{xref_targets[identifier].path}#{identifier}":
+                identifiers_missing_current_outpath.add(identifier)
+
+            for location in locations[1:]:
+                if '#' in location:
+                    path, anchor = location.split('#')
+
+                    if anchor in identifiers_without_redirects:
+                        identifiers_without_redirects.remove(anchor)
+
+                    if location not in client_side_redirects:
+                        client_side_redirects[location] = f"{xref_targets[identifier].path}#{identifier}"
+                        for identifier, xref_target in xref_targets.items():
+                            if xref_target.path == path and anchor == identifier:
+                                conflicting_anchors.add(anchor)
+                    else:
+                        divergent_redirects.add(location)
+                else:
+                    if location not in server_side_redirects:
+                        server_side_redirects[location] = xref_targets[identifier].path
+                    else:
+                        divergent_redirects.add(location)
+
+        if any([
+            conflicting_anchors,
+            divergent_redirects,
+            identifiers_missing_current_outpath,
+            identifiers_without_redirects,
+            orphan_identifiers
+        ]):
+            raise RedirectsError(
+                conflicting_anchors=conflicting_anchors,
+                divergent_redirects=divergent_redirects,
+                identifiers_missing_current_outpath=identifiers_missing_current_outpath,
+                identifiers_without_redirects=identifiers_without_redirects,
+                orphan_identifiers=orphan_identifiers
+            )
+
+        self._xref_targets = xref_targets
+
+    def get_client_redirects(self, target: str):
+        paths_to_target = {src for src, dest in self.get_server_redirects().items() if dest == target}
+        client_redirects = {}
+        for locations in self._raw_redirects.values():
+            for location in locations[1:]:
+                if '#' not in location:
+                    continue
+                path, anchor = location.split('#')
+                if path not in [target, *paths_to_target]:
+                    continue
+                client_redirects[anchor] = locations[0]
+        return client_redirects
+
+    def get_server_redirects(self):
+        server_redirects = {}
+        for identifier, locations in self._raw_redirects.items():
+            for location in locations[1:]:
+                if '#' not in location and location not in server_redirects:
+                    server_redirects[location] = self._xref_targets[identifier].path
+        return server_redirects
+
+    def get_redirect_script(self, target: str) -> str:
+        client_redirects = self.get_client_redirects(target)
+        return self._redirects_script.replace('REDIRECTS_PLACEHOLDER', json.dumps(client_redirects))

pkgs/by-name/ni/nixos-render-docs/src/pyproject.toml

@@ -13,3 +13,6 @@ nixos-render-docs = "nixos_render_docs:main"
 
 [build-system]
 requires = ["setuptools"]
+
+[tool.setuptools.package-data]
+nixos_render_docs = ["redirects.js"]

pkgs/by-name/ni/nixos-render-docs/src/tests/test_auto_id_prefix.py

@@ -3,6 +3,7 @@ from pathlib import Path
 from markdown_it.token import Token
 from nixos_render_docs.manual import HTMLConverter, HTMLParameters
 from nixos_render_docs.md import Converter
+from nixos_render_docs.redirects import Redirects
 
 auto_id_prefix="TEST_PREFIX"
 def set_prefix(token: Token, ident: str) -> None:
@@ -10,7 +11,7 @@ def set_prefix(token: Token, ident: str) -> None:
 
 
 def test_auto_id_prefix_simple() -> None:
-    md = HTMLConverter("1.0.0", HTMLParameters("", [], [], 2, 2, 2, Path("")), {})
+    md = HTMLConverter("1.0.0", HTMLParameters("", [], [], 2, 2, 2, Path("")), {}, Redirects({}, ''))
 
     src = f"""
 # title
@@ -31,7 +32,7 @@ def test_auto_id_prefix_simple() -> None:
 
 
 def test_auto_id_prefix_repeated() -> None:
-    md = HTMLConverter("1.0.0", HTMLParameters("", [], [], 2, 2, 2, Path("")), {})
+    md = HTMLConverter("1.0.0", HTMLParameters("", [], [], 2, 2, 2, Path("")), {}, Redirects({}, ''))
 
     src = f"""
 # title
@@ -57,7 +58,7 @@ def test_auto_id_prefix_repeated() -> None:
     ]
 
 def test_auto_id_prefix_maximum_nested() -> None:
-    md = HTMLConverter("1.0.0", HTMLParameters("", [], [], 2, 2, 2, Path("")), {})
+    md = HTMLConverter("1.0.0", HTMLParameters("", [], [], 2, 2, 2, Path("")), {}, Redirects({}, ''))
 
     src = f"""
 # h1

pkgs/by-name/ni/nixos-render-docs/src/tests/test_redirects.py

@@ -0,0 +1,259 @@
+import json
+import unittest
+from pathlib import Path
+
+from nixos_render_docs.manual import HTMLConverter, HTMLParameters
+from nixos_render_docs.redirects import Redirects, RedirectsError
+
+
+class TestRedirects(unittest.TestCase):
+    def setup_test(self, sources, raw_redirects):
+        with open(Path(__file__).parent / 'index.md', 'w') as infile:
+            indexHTML = ["# Redirects test suite {#redirects-test-suite}\n## Setup steps"]
+            for path in sources.keys():
+                outpath = f"{path.split('.md')[0]}.html"
+                indexHTML.append(f"```{{=include=}} appendix html:into-file=//{outpath}\n{path}\n```")
+            infile.write("\n".join(indexHTML))
+
+        for filename, content in sources.items():
+            with open(Path(__file__).parent / filename, 'w') as infile:
+                infile.write(content)
+
+        redirects = Redirects({"redirects-test-suite": ["index.html#redirects-test-suite"]} | raw_redirects, '')
+        return HTMLConverter("1.0.0", HTMLParameters("", [], [], 2, 2, 2, Path("")), {}, redirects)
+
+    def run_test(self, md: HTMLConverter):
+        md.convert(Path(__file__).parent / 'index.md', Path(__file__).parent / 'index.html')
+
+    def assert_redirect_error(self, expected_errors: dict, md: HTMLConverter):
+        with self.assertRaises(RuntimeError) as context:
+            self.run_test(md)
+
+        exception = context.exception.__cause__
+        self.assertIsInstance(exception, RedirectsError)
+
+        for attr, expected_values in expected_errors.items():
+            self.assertTrue(hasattr(exception, attr))
+            actual_values = getattr(exception, attr)
+            self.assertEqual(set(actual_values), set(expected_values))
+
+    def test_identifier_added(self):
+        """Test adding a new identifier to the source."""
+        before = self.setup_test(
+            sources={"foo.md": "# Foo {#foo}"},
+            raw_redirects={"foo": ["foo.html#foo"]},
+        )
+        self.run_test(before)
+
+        intermediate = self.setup_test(
+            sources={"foo.md": "# Foo {#foo}\n## Bar {#bar}"},
+            raw_redirects={"foo": ["foo.html#foo"]},
+        )
+        self.assert_redirect_error({"identifiers_without_redirects": ["bar"]}, intermediate)
+
+        after = self.setup_test(
+            sources={"foo.md": "# Foo {#foo}\n## Bar {#bar}"},
+            raw_redirects={"foo": ["foo.html#foo"], "bar": ["foo.html#bar"]},
+        )
+        self.run_test(after)
+
+    def test_identifier_removed(self):
+        """Test removing an identifier from the source."""
+        before = self.setup_test(
+            sources={"foo.md": "# Foo {#foo}\n## Bar {#bar}"},
+            raw_redirects={"foo": ["foo.html#foo"], "bar": ["foo.html#bar"]},
+        )
+        self.run_test(before)
+
+        intermediate = self.setup_test(
+            sources={"foo.md": "# Foo {#foo}"},
+            raw_redirects={"foo": ["foo.html#foo"], "bar": ["foo.html#bar"]},
+        )
+        self.assert_redirect_error({"orphan_identifiers": ["bar"]}, intermediate)
+
+        after = self.setup_test(
+            sources={"foo.md": "# Foo {#foo}"},
+            raw_redirects={"foo": ["foo.html#foo"]},
+        )
+        self.run_test(after)
+
+    def test_identifier_renamed(self):
+        """Test renaming an identifier in the source."""
+        before = self.setup_test(
+            sources={"foo.md": "# Foo {#foo}\n## Bar {#bar}"},
+            raw_redirects={"foo": ["foo.html#foo"], "bar": ["foo.html#bar"]},
+        )
+        self.run_test(before)
+
+        intermediate = self.setup_test(
+            sources={"foo.md": "# Foo Prime {#foo-prime}\n## Bar {#bar}"},
+            raw_redirects={"foo": ["foo.html#foo"], "bar": ["foo.html#bar"]},
+        )
+        self.assert_redirect_error(
+            {
+                "identifiers_without_redirects": ["foo-prime"],
+                "orphan_identifiers": ["foo"]
+            },
+            intermediate
+        )
+
+        after = self.setup_test(
+            sources={"foo.md": "# Foo Prime {#foo-prime}\n## Bar {#bar}"},
+            raw_redirects={"foo-prime": ["foo.html#foo-prime", "foo.html#foo"], "bar": ["foo.html#bar"]},
+        )
+        self.run_test(after)
+
+    def test_leaf_identifier_moved_to_different_file(self):
+        """Test moving a leaf identifier to a different output path."""
+        before = self.setup_test(
+            sources={"foo.md": "# Foo {#foo}\n## Bar {#bar}"},
+            raw_redirects={"foo": ["foo.html#foo"], "bar": ["foo.html#bar"]},
+        )
+        self.run_test(before)
+
+        intermediate = self.setup_test(
+            sources={
+                "foo.md": "# Foo {#foo}",
+                "bar.md": "# Bar {#bar}"
+            },
+            raw_redirects={"foo": ["foo.html#foo"], "bar": ["foo.html#foo"]},
+        )
+        self.assert_redirect_error({"identifiers_missing_current_outpath": ["bar"]}, intermediate)
+
+        after = self.setup_test(
+            sources={
+                "foo.md": "# Foo {#foo}",
+                "bar.md": "# Bar {#bar}"
+            },
+            raw_redirects={"foo": ["foo.html#foo"], "bar": ["bar.html#bar", "foo.html#bar"]},
+        )
+        self.run_test(after)
+
+    def test_non_leaf_identifier_moved_to_different_file(self):
+        """Test moving a non-leaf identifier to a different output path."""
+        before = self.setup_test(
+            sources={"foo.md": "# Foo {#foo}\n## Bar {#bar}\n### Baz {#baz}"},
+            raw_redirects={"foo": ["foo.html#foo"], "bar": ["foo.html#bar"], "baz": ["foo.html#baz"]},
+        )
+        self.run_test(before)
+
+        intermediate = self.setup_test(
+            sources={
+                "foo.md": "# Foo {#foo}",
+                "bar.md": "# Bar {#bar}\n## Baz {#baz}"
+            },
+            raw_redirects={"foo": ["foo.html#foo"], "bar": ["foo.html#bar"], "baz": ["foo.html#baz"]},
+        )
+        self.assert_redirect_error({"identifiers_missing_current_outpath": ["bar", "baz"]}, intermediate)
+
+        after = self.setup_test(
+            sources={
+                "foo.md": "# Foo {#foo}",
+                "bar.md": "# Bar {#bar}\n## Baz {#baz}"
+            },
+            raw_redirects={
+                "foo": ["foo.html#foo"],
+                "bar": ["bar.html#bar", "foo.html#bar"],
+                "baz": ["bar.html#baz", "foo.html#baz"]
+            },
+        )
+        self.run_test(after)
+
+    def test_conflicting_anchors(self):
+        """Test for conflicting anchors."""
+        md = self.setup_test(
+            sources={"foo.md": "# Foo {#foo}\n## Bar {#bar}"},
+            raw_redirects={
+                "foo": ["foo.html#foo", "foo.html#bar"],
+                "bar": ["foo.html#bar"],
+            }
+        )
+        self.assert_redirect_error({"conflicting_anchors": ["bar"]}, md)
+
+    def test_divergent_redirect(self):
+        """Test for divergent redirects."""
+        md = self.setup_test(
+            sources={
+                "foo.md": "# Foo {#foo}",
+                "bar.md": "# Bar {#bar}"
+            },
+            raw_redirects={
+                "foo": ["foo.html#foo", "old-foo.html"],
+                "bar": ["bar.html#bar", "old-foo.html"]
+            }
+        )
+        self.assert_redirect_error({"divergent_redirects": ["old-foo.html"]}, md)
+
+    def test_no_client_redirects(self):
+        """Test fetching client side redirects and ignore server-side ones."""
+        md = self.setup_test(
+            sources={"foo.md": "# Foo {#foo}\n## Bar {#bar}"},
+            raw_redirects={"foo": ["foo.html#foo"], "bar": ["foo.html#bar", "bar.html"]}
+        )
+        self.run_test(md)
+        self.assertEqual(md._redirects.get_client_redirects("foo.html"), {})
+
+    def test_basic_redirect_matching(self):
+        """Test client-side redirects getter with a simple redirect mapping"""
+        md = self.setup_test(
+            sources={"foo.md": "# Foo {#foo}\n## Bar {#bar}"},
+            raw_redirects={
+                'foo': ['foo.html#foo', 'foo.html#some-section', 'foo.html#another-section'],
+                'bar': ['foo.html#bar'],
+            },
+        )
+        self.run_test(md)
+
+        client_redirects = md._redirects.get_client_redirects("foo.html")
+        expected_redirects = {'some-section': 'foo.html#foo', 'another-section': 'foo.html#foo'}
+        self.assertEqual(client_redirects, expected_redirects)
+
+    def test_advanced_redirect_matching(self):
+        """Test client-side redirects getter with a complex redirect mapping"""
+        md = self.setup_test(
+            sources={"foo.md": "# Foo {#foo}", "bar.md": "# Bar {#bar}"},
+            raw_redirects={
+                'foo': ['foo.html#foo', 'foo.html#some-section', 'bar.html#foo'],
+                'bar': ['bar.html#bar', 'bar.html#another-section'],
+            },
+        )
+        self.run_test(md)
+        self.assertEqual(md._redirects.get_client_redirects("index.html"), {})
+
+        client_redirects = md._redirects.get_client_redirects("foo.html")
+        expected_redirects = {'some-section': 'foo.html#foo'}
+        self.assertEqual(client_redirects, expected_redirects)
+
+        client_redirects = md._redirects.get_client_redirects("bar.html")
+        expected_redirects = {'foo': 'foo.html#foo', 'another-section': 'bar.html#bar'}
+        self.assertEqual(client_redirects, expected_redirects)
+
+    def test_server_redirects(self):
+        """Test server-side redirects getter"""
+        md = self.setup_test(
+            sources={"foo.md": "# Foo {#foo}", "bar.md": "# Bar {#bar}"},
+            raw_redirects={
+                'foo': ['foo.html#foo', 'foo-prime.html'],
+                'bar': ['bar.html#bar', 'bar-prime.html'],
+            },
+        )
+        self.run_test(md)
+
+        server_redirects = md._redirects.get_server_redirects()
+        expected_redirects = {'foo-prime.html': 'foo.html', 'bar-prime.html': 'bar.html'}
+        self.assertEqual(server_redirects, expected_redirects)
+
+    def test_client_redirects_to_ghost_paths(self):
+        """Test implicit inference of client-side redirects to ghost paths"""
+        md = self.setup_test(
+            sources={"foo.md": "# Foo {#foo}", "bar.md": "# Bar {#bar}"},
+            raw_redirects={
+                'foo': ['foo.html#foo', 'foo-prime.html'],
+                'bar': ['bar.html#bar', 'foo-prime.html#old'],
+            },
+        )
+        self.run_test(md)
+
+        client_redirects = md._redirects.get_client_redirects("foo.html")
+        expected_redirects = {'old': 'bar.html#bar'}
+        self.assertEqual(client_redirects, expected_redirects)