Merge branch 'master' into os-prober-test

.github/CODEOWNERS

@@ -58,11 +58,11 @@
 /doc/languages-frameworks/python.section.md     @FRidh
 
 # Haskell
-/pkgs/development/compilers/ghc                       @basvandijk
+/pkgs/development/compilers/ghc                       @basvandijk @cdepillabout
-/pkgs/development/haskell-modules                     @basvandijk
+/pkgs/development/haskell-modules                     @basvandijk @cdepillabout
-/pkgs/development/haskell-modules/default.nix         @basvandijk
+/pkgs/development/haskell-modules/default.nix         @basvandijk @cdepillabout
-/pkgs/development/haskell-modules/generic-builder.nix @basvandijk
+/pkgs/development/haskell-modules/generic-builder.nix @basvandijk @cdepillabout
-/pkgs/development/haskell-modules/hoogle.nix          @basvandijk
+/pkgs/development/haskell-modules/hoogle.nix          @basvandijk @cdepillabout
 
 # Perl
 /pkgs/development/interpreters/perl @volth
@@ -107,8 +107,8 @@
 # Eclipse
 /pkgs/applications/editors/eclipse @rycee
 
-# https://github.com/NixOS/nixpkgs/issues/31401
+# Licenses
-/lib/licenses.nix @ghost
+/lib/licenses.nix @alyssais
 
 # Qt / KDE
 /pkgs/applications/kde @ttuegel
@@ -130,6 +130,12 @@
 /nixos/tests/hardened.nix @joachifm
 /pkgs/os-specific/linux/kernel/hardened-config.nix @joachifm
 
+# Network Time Daemons
+/pkgs/tools/networking/chrony @thoughtpolice
+/pkgs/tools/networking/ntp @thoughtpolice
+/pkgs/tools/networking/openntpd @thoughtpolice
+/nixos/modules/services/networking/ntp @thoughtpolice
+
 # Dhall
 /pkgs/development/dhall-modules      @Gabriel439 @Profpatsch
 /pkgs/development/interpreters/dhall @Gabriel439 @Profpatsch
@@ -139,3 +145,19 @@
 
 # Bazel
 /pkgs/development/tools/build-managers/bazel @mboes @Profpatsch
+
+# NixOS modules for e-mail and dns services
+/nixos/modules/services/mail/mailman.nix    @peti
+/nixos/modules/services/mail/postfix.nix    @peti
+/nixos/modules/services/networking/bind.nix @peti
+/nixos/modules/services/mail/rspamd.nix     @peti
+
+# Emacs
+/pkgs/applications/editors/emacs-modes @adisbladis
+/pkgs/applications/editors/emacs       @adisbladis
+/pkgs/top-level/emacs-packages.nix     @adisbladis
+
+# Prometheus exporter modules and tests
+/nixos/modules/services/monitoring/prometheus/exporters.nix  @WilliButz
+/nixos/modules/services/monitoring/prometheus/exporters.xml  @WilliButz
+/nixos/tests/prometheus-exporters.nix                        @WilliButz

.github/ISSUE_TEMPLATE.md

@@ -8,5 +8,4 @@
 
 ## Technical details
 
-Please run `nix-shell -p nix-info --run "nix-info -m"` and paste the
+Please run `nix run nixpkgs.nix-info -c nix-info -m` and paste the result.
-results.

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,37 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: '0.kind: bug'
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. ...
+2. ...
+3. ...
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.
+
+**Additional context**
+Add any other context about the problem here.
+
+**Metadata**
+Please run `nix run nixpkgs.nix-info -c nix-info -m` and paste the result.
+
+Maintainer information:
+```yaml
+# a list of nixpkgs attributes affected by the problem
+attribute:
+# a list of nixos modules affected by the problem
+module:
+```

.github/ISSUE_TEMPLATE/packaging_request.md

@@ -0,0 +1,18 @@
+---
+name: Packaging requests
+about: For packages that are missing
+title: ''
+labels: '0.kind: packaging request'
+assignees: ''
+
+---
+
+**Project description**
+_describe the project a little_
+
+**Metadata**
+
+* homepage URL:
+* source URL:
+* license: mit, bsd, gpl2+ , ...
+* platforms: unix, linux, darwin, ...

.github/PULL_REQUEST_TEMPLATE.md

@@ -15,7 +15,9 @@
 - [ ] Tested compilation of all pkgs that depend on this change using `nix-shell -p nix-review --run "nix-review wip"`
 - [ ] Tested execution of all binary files (usually in `./result/bin/`)
 - [ ] Determined the impact on package closure size (by running `nix path-info -S` before and after)
-- [ ] Assured whether relevant documentation is up to date
+- [ ] Ensured that relevant documentation is up to date
 - [ ] Fits [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/.github/CONTRIBUTING.md).
 
----
+###### Notify maintainers
+
+cc @

.version

@@ -1 +1 @@
-19.09
+20.03

README.md

@@ -1,47 +1,113 @@
-[<img src="https://nixos.org/logo/nixos-hires.png" width="500px" alt="logo" />](https://nixos.org/nixos)
+<p align="center">
+  <a href="https://nixos.org/nixos"><img src="https://nixos.org/logo/nixos-hires.png" width="500px" alt="NixOS logo" /></a>
+</p>
 
-[![Code Triagers Badge](https://www.codetriage.com/nixos/nixpkgs/badges/users.svg)](https://www.codetriage.com/nixos/nixpkgs)
+<p align="center">
+  <a href="https://www.codetriage.com/nixos/nixpkgs"><img src="https://www.codetriage.com/nixos/nixpkgs/badges/users.svg" alt="Code Triagers badge" /></a>
+  <a href="https://opencollective.com/nixos"><img src="https://opencollective.com/nixos/tiers/supporter/badge.svg?label=Supporter&color=brightgreen" alt="Open Collective supporters" /></a>
+</p>
 
-Nixpkgs is a collection of packages for the [Nix](https://nixos.org/nix/) package
+[Nixpkgs](https://github.com/nixos/nixpkgs) is a collection of over
-manager. It is periodically built and tested by the [Hydra](https://hydra.nixos.org/)
+40,000 software packages that can be installed with the
-build daemon as so-called channels. To get channel information via git, add
+[Nix](https://nixos.org/nix/) package manager. It also implements
-[nixpkgs-channels](https://github.com/NixOS/nixpkgs-channels.git) as a remote:
+[NixOS](https://nixos.org/nixos/), a purely-functional Linux distribution.
 
-```
+# Manuals
-% git remote add channels https://github.com/NixOS/nixpkgs-channels.git
-```
 
-For stability and maximum binary package support, it is recommended to maintain
+* [NixOS Manual](https://nixos.org/nixos/manual) - how to install, configure, and maintain a purely-functional Linux distribution
-custom changes on top of one of the channels, e.g. `nixos-19.03` for the latest
+* [Nixpkgs Manual](https://nixos.org/nixpkgs/manual/) - contributing to Nixpkgs and using programming-language-specific Nix expressions
-release and `nixos-unstable` for the latest successful build of master:
+* [Nix Package Manager Manual](https://nixos.org/nix/manual) - how to write Nix expresssions (programs), and how to use Nix command line tools
 
-```
+# Community
-% git remote update channels
-% git rebase channels/nixos-19.03
-```
-
-For pull requests, please rebase onto nixpkgs `master`.
-
-[NixOS](https://nixos.org/nixos/) Linux distribution source code is located inside
-`nixos/` folder.
-
-* [NixOS installation instructions](https://nixos.org/nixos/manual/#ch-installation)
-* [Documentation (Nix Expression Language chapter)](https://nixos.org/nix/manual/#ch-expression-language)
-* [Manual (How to write packages for Nix)](https://nixos.org/nixpkgs/manual/)
-* [Manual (NixOS)](https://nixos.org/nixos/manual/)
-* [Community maintained wiki](https://nixos.wiki/)
-* [Continuous package builds for unstable/master](https://hydra.nixos.org/jobset/nixos/trunk-combined)
-* [Continuous package builds for 19.03 release](https://hydra.nixos.org/jobset/nixos/release-19.03)
-* [Tests for unstable/master](https://hydra.nixos.org/job/nixos/trunk-combined/tested#tabs-constituents)
-* [Tests for 19.03 release](https://hydra.nixos.org/job/nixos/release-19.03/tested#tabs-constituents)
-
-Communication:
 
 * [Discourse Forum](https://discourse.nixos.org/)
 * [IRC - #nixos on freenode.net](irc://irc.freenode.net/#nixos)
+* [NixOS Weekly](https://weekly.nixos.org/)
+* [Community-maintained wiki](https://nixos.wiki/)
 
-Note: MIT license does not apply to the packages built by Nixpkgs, merely to
+# Other Project Repositories
-the package descriptions (Nix expressions, build scripts, and so on). It also
+
-might not apply to patches included in Nixpkgs, which may be derivative works
+The sources of all offical Nix-related projects are in the [NixOS
-of the packages to which they apply. The aforementioned artifacts are all
+organization on GitHub](https://github.com/NixOS/). Here are some of
-covered by the licenses of the respective packages.
+the main ones:
+
+* [Nix](https://github.com/NixOS/nix) - the purely functional package manager
+* [NixOps](https://github.com/NixOS/nixops) - the tool to remotely deploy NixOS machines
+* [Nix RFCs](https://github.com/NixOS/rfcs) - the formal process for making substantial changes to the community
+* [NixOS homepage](https://github.com/NixOS/nixos-homepage) - the [NixOS.org](https://nixos.org) website
+* [hydra](https://github.com/NixOS/hydra) - our continuous integration system
+* [NixOS Artwork](https://github.com/NixOS/nixos-artwork) - NixOS artwork
+
+# Continuous Integration and Distribution
+
+Nixpkgs and NixOS are built and tested by our continuous integration
+system, [Hydra](https://hydra.nixos.org/).
+
+* [Continuous package builds for unstable/master](https://hydra.nixos.org/jobset/nixos/trunk-combined)
+* [Continuous package builds for the NixOS 19.03 release](https://hydra.nixos.org/jobset/nixos/release-19.03)
+* [Tests for unstable/master](https://hydra.nixos.org/job/nixos/trunk-combined/tested#tabs-constituents)
+* [Tests for the NixOS 19.03 release](https://hydra.nixos.org/job/nixos/release-19.03/tested#tabs-constituents)
+
+Artifacts successfully built with Hydra are published to cache at
+https://cache.nixos.org/. When successful build and test criteria are
+met, the Nixpkgs expressions are distributed via [Nix
+channels](https://nixos.org/nix/manual/#sec-channels). The channels
+are provided via a read-only mirror of the Nixpkgs repository called
+[nixpkgs-channels](https://github.com/NixOS/nixpkgs-channels).
+
+# Contributing
+
+Nixpkgs is among the most active projects on GitHub. While thousands
+of open issues and pull requests might seem a lot at first, it helps
+consider it in the context of the scope of the project. Nixpkgs
+describes how to build over 40,000 pieces of software and implements a
+Linux distribution. The [GitHub Insights](https://github.com/NixOS/nixpkgs/pulse)
+page gives a sense of the project activity.
+
+Community contributions are always welcome through GitHub Issues and
+Pull Requests. When pull requests are made, our tooling automation bot,
+[OfBorg](https://github.com/NixOS/ofborg) will perform various checks
+to help ensure expression quality.
+
+The *Nixpkgs maintainers* are people who have assigned themselves to
+maintain specific individual packages. We encourage people who care
+about a package to assign themselves as a maintainer. When a pull
+request is made against a package, OfBorg will notify the appropriate
+maintainer(s). The *Nixpkgs committers* are people who have been given
+permission to merge.
+
+Most contributions are based on and merged into these branches:
+
+* `master` is the main branch where all small contributions go
+* `staging` is branched from master, changes that have a big impact on
+  Hydra builds go to this branch
+* `staging-next` is branched from staging and only fixes to stabilize
+  and security fixes with a big impact on Hydra builds should be
+  contributed to this branch. This branch is merged into master when
+  deemed of sufficiently high quality
+
+For more information about contributing to the project, please visit
+the [contributing page](https://github.com/NixOS/nixpkgs/blob/master/.github/CONTRIBUTING.md).
+
+# Donations
+
+The infrastructure for NixOS and related projects is maintained by a
+nonprofit organization, the [NixOS
+Foundation](https://nixos.org/nixos/foundation.html). To ensure the
+continuity and expansion of the NixOS infrastructure, we are looking
+for donations to our organization.
+
+You can donate to the NixOS foundation by using Open Collective:
+
+<a href="https://opencollective.com/nixos#support"><img src="https://opencollective.com/nixos/tiers/supporter.svg?width=890" /></a>
+
+# License
+
+Nixpkgs is licensed under the [MIT License](COPYING).
+
+Note: MIT license does not apply to the packages built by Nixpkgs,
+merely to the files in this repository (the Nix expressions, build
+scripts, NixOS modules, etc.). It also might not apply to patches
+included in Nixpkgs, which may be derivative works of the packages to
+which they apply. The aforementioned artifacts are all covered by the
+licenses of the respective packages.

doc/Makefile

@@ -8,10 +8,10 @@ debug:
 	nix-shell --run "xmloscopy --docbook5 ./manual.xml ./manual-full.xml"
 
 .PHONY: format
-format:
+format: doc-support/result
 	find . -iname '*.xml' -type f | while read f; do \
 		echo $$f ;\
-		xmlformat --config-file "$$XMLFORMAT_CONFIG" -i $$f ;\
+		xmlformat --config-file "doc-support/result/xmlformat.conf" -i $$f ;\
 	done
 
 .PHONY: fix-misc-xml
@@ -21,19 +21,19 @@ fix-misc-xml:
 
 .PHONY: clean
 clean:
-	rm -f ${MD_TARGETS} .version manual-full.xml functions/library/locations.xml functions/library/generated
+	rm -f ${MD_TARGETS} doc-support/result .version manual-full.xml functions/library/locations.xml functions/library/generated
 	rm -rf ./out/ ./highlightjs
 
 .PHONY: validate
-validate: manual-full.xml
+validate: manual-full.xml doc-support/result
-	jing "$$RNG" manual-full.xml
+	jing doc-support/result/docbook.rng manual-full.xml
 
-out/html/index.html: manual-full.xml style.css highlightjs
+out/html/index.html: doc-support/result manual-full.xml style.css highlightjs
 	mkdir -p out/html
-	xsltproc ${xsltFlags} \
+	xsltproc \
 		--nonet --xinclude \
 		--output $@ \
-		"$$XSL/docbook/xhtml/docbook.xsl" \
+		doc-support/result/xhtml.xsl \
 		./manual-full.xml
 
 	mkdir -p out/html/highlightjs/
@@ -43,49 +43,48 @@ out/html/index.html: manual-full.xml style.css highlightjs
 	cp ./style.css out/html/style.css
 
 	mkdir -p out/html/images/callouts
-	cp "$$XSL/docbook/images/callouts/"*.svg out/html/images/callouts/
+	cp doc-support/result/xsl/docbook/images/callouts/*.svg out/html/images/callouts/
 	chmod u+w -R out/html/
 
 out/epub/manual.epub: manual-full.xml
 	mkdir -p out/epub/scratch
-	xsltproc ${xsltFlags} --nonet \
+	xsltproc --nonet \
 		--output out/epub/scratch/ \
-		"$$XSL/docbook/epub/docbook.xsl" \
+		doc-support/result/epub.xsl \
 		./manual-full.xml
 
 	cp ./overrides.css out/epub/scratch/OEBPS
 	cp ./style.css out/epub/scratch/OEBPS
 	mkdir -p out/epub/scratch/OEBPS/images/callouts/
-	cp "$$XSL/docbook/images/callouts/"*.svg out/epub/scratch/OEBPS/images/callouts/
+	cp doc-support/result/xsl/docbook/images/callouts/*.svg out/epub/scratch/OEBPS/images/callouts/
 	echo "application/epub+zip" > mimetype
 	zip -0Xq "out/epub/manual.epub" mimetype
 	rm mimetype
 	cd "out/epub/scratch/" && zip -Xr9D "../manual.epub" *
 	rm -rf "out/epub/scratch/"
 
-highlightjs:
+highlightjs: doc-support/result
 	mkdir -p highlightjs
-	cp -r "$$HIGHLIGHTJS/highlight.pack.js" highlightjs/
+	cp -r doc-support/result/highlightjs/highlight.pack.js highlightjs/
-	cp -r "$$HIGHLIGHTJS/LICENSE" highlightjs/
+	cp -r doc-support/result/highlightjs/LICENSE highlightjs/
-	cp -r "$$HIGHLIGHTJS/mono-blue.css" highlightjs/
+	cp -r doc-support/result/highlightjs/mono-blue.css highlightjs/
-	cp -r "$$HIGHLIGHTJS/loader.js" highlightjs/
+	cp -r doc-support/result/highlightjs/loader.js highlightjs/
 
 
 manual-full.xml: ${MD_TARGETS} .version functions/library/locations.xml functions/library/generated *.xml **/*.xml **/**/*.xml
 	xmllint --nonet --xinclude --noxincludenode manual.xml --output manual-full.xml
 
-.version:
+.version: doc-support/result
-	nix-instantiate --eval \
+	ln -rfs ./doc-support/result/version .version
-		-E '(import ../lib).version' > .version
 
-functions/library/locations.xml:
+doc-support/result: doc-support/default.nix
-	nix-build ./lib-function-locations.nix \
+	(cd doc-support; nix-build)
-		--out-link $@
 
-functions/library/generated: functions/library/locations.xml
+functions/library/locations.xml: doc-support/result
-	nix-build ./lib-function-docs.nix \
+	ln -rfs ./doc-support/result/function-locations.xml functions/library/locations.xml
-		--arg locationsXml $< \
+
-		--out-link $@
+functions/library/generated: doc-support/result
+	ln -rfs ./doc-support/result/function-docs functions/library/generated
 
 %.section.xml: %.section.md
 	pandoc $^ -w docbook+smart \

doc/coding-conventions.xml

@@ -921,7 +921,7 @@ src = fetchFromGitHub {
     <para>
      You can convert between formats with nix-hash, for example:
 <screen>
-$ nix-hash --type sha256 --to-base32 <replaceable>HASH</replaceable>
+<prompt>$ </prompt>nix-hash --type sha256 --to-base32 <replaceable>HASH</replaceable>
 </screen>
     </para>
    </listitem>
@@ -1038,7 +1038,7 @@ patches = [ ./0001-changes.patch ];
      <para>
       Move to the root directory of the source code you're patching.
 <screen>
-$ cd the/program/source</screen>
+<prompt>$ </prompt>cd the/program/source</screen>
      </para>
     </listitem>
     <listitem>
@@ -1046,8 +1046,8 @@ $ cd the/program/source</screen>
       If a git repository is not already present, create one and stage all of
       the source files.
 <screen>
-$ git init
+<prompt>$ </prompt>git init
-$ git add .</screen>
+<prompt>$ </prompt>git add .</screen>
      </para>
     </listitem>
     <listitem>
@@ -1060,7 +1060,7 @@ $ git add .</screen>
      <para>
       Use git to create a diff, and pipe the output to a patch file:
 <screen>
-$ git diff > nixpkgs/pkgs/the/package/0001-changes.patch</screen>
+<prompt>$ </prompt>git diff > nixpkgs/pkgs/the/package/0001-changes.patch</screen>
      </para>
     </listitem>
    </orderedlist>

doc/contributing.xml

@@ -12,9 +12,9 @@ xlink:href="https://github.com/NixOS/nixpkgs/tree/master/doc">doc</filename>
   You can quickly check your edits with <command>make</command>:
  </para>
 <screen>
-  $ cd /path/to/nixpkgs/doc
+<prompt>$ </prompt>cd /path/to/nixpkgs/doc
-  $ nix-shell
+<prompt>$ </prompt>nix-shell
-  [nix-shell]$ make
+<prompt>[nix-shell]$ </prompt>make
 </screen>
  <para>
   If you experience problems, run <command>make debug</command> to help
@@ -24,10 +24,10 @@ xlink:href="https://github.com/NixOS/nixpkgs/tree/master/doc">doc</filename>
   After making modifications to the manual, it's important to build it before
   committing. You can do that as follows:
 <screen>
-  $ cd /path/to/nixpkgs/doc
+<prompt>$ </prompt>cd /path/to/nixpkgs/doc
-  $ nix-shell
+<prompt>$ </prompt>nix-shell
-  [nix-shell]$ make clean
+<prompt>[nix-shell]$ </prompt>make clean
-  [nix-shell]$ nix-build .
+<prompt>[nix-shell]$ </prompt>nix-build .
 </screen>
   If the build succeeds, the manual will be in
   <filename>./result/share/doc/nixpkgs/manual.html</filename>.

doc/default.nix

@@ -1,8 +1,7 @@
 { pkgs ? (import ./.. { }), nixpkgs ? { }}:
 let
   lib = pkgs.lib;
-  locationsXml = import ./lib-function-locations.nix { inherit pkgs nixpkgs; };
+  doc-support = import ./doc-support { inherit pkgs nixpkgs; };
-  functionDocs = import ./lib-function-docs.nix { inherit locationsXml pkgs; };
 in pkgs.stdenv.mkDerivation {
   name = "nixpkgs-manual";
 
@@ -10,30 +9,8 @@ in pkgs.stdenv.mkDerivation {
 
   src = ./.;
 
-  # Hacking on these variables? Make sure to close and open
-  # nix-shell between each test, maybe even:
-  # $ nix-shell --run "make clean all"
-  # otherwise they won't reapply :)
-  HIGHLIGHTJS = pkgs.documentation-highlighter;
-  XSL = "${pkgs.docbook_xsl_ns}/xml/xsl";
-  RNG = "${pkgs.docbook5}/xml/rng/docbook/docbook.rng";
-  XMLFORMAT_CONFIG = ../nixos/doc/xmlformat.conf;
-  xsltFlags = lib.concatStringsSep " " [
-    "--param section.autolabel 1"
-    "--param section.label.includes.component.label 1"
-    "--stringparam html.stylesheet 'style.css overrides.css highlightjs/mono-blue.css'"
-    "--stringparam html.script './highlightjs/highlight.pack.js ./highlightjs/loader.js'"
-    "--param xref.with.number.and.title 1"
-    "--param toc.section.depth 3"
-    "--stringparam admon.style ''"
-    "--stringparam callout.graphics.extension .svg"
-  ];
-
   postPatch = ''
-    rm -rf ./functions/library/locations.xml
+    ln -s ${doc-support} ./doc-support/result
-    ln -s ${locationsXml} ./functions/library/locations.xml
-    ln -s ${functionDocs} ./functions/library/generated
-    echo ${lib.version} > .version
   '';
 
   installPhase = ''

doc/doc-support/default.nix

@@ -0,0 +1,45 @@
+{ pkgs ? (import ../.. {}), nixpkgs ? { }}:
+let
+  locationsXml = import ./lib-function-locations.nix { inherit pkgs nixpkgs; };
+  functionDocs = import ./lib-function-docs.nix { inherit locationsXml pkgs; };
+  version = pkgs.lib.version;
+
+  epub-xsl = pkgs.writeText "epub.xsl" ''
+    <?xml version='1.0'?>
+    <xsl:stylesheet
+      xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
+      version="1.0">
+      <xsl:import href="${pkgs.docbook_xsl_ns}/xml/xsl/docbook/epub/docbook.xsl" />
+      <xsl:import href="${./parameters.xml}"/>
+    </xsl:stylesheet>
+  '';
+
+  xhtml-xsl = pkgs.writeText "xhtml.xsl" ''
+    <?xml version='1.0'?>
+    <xsl:stylesheet
+      xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
+      version="1.0">
+      <xsl:import href="${pkgs.docbook_xsl_ns}/xml/xsl/docbook/xhtml/docbook.xsl" />
+      <xsl:import href="${./parameters.xml}"/>
+    </xsl:stylesheet>
+  '';
+in pkgs.runCommand "doc-support" {}
+''
+  mkdir result
+  (
+    cd result
+    ln -s ${locationsXml} ./function-locations.xml
+    ln -s ${functionDocs} ./function-docs
+
+    ln -s ${pkgs.docbook5}/xml/rng/docbook/docbook.rng ./docbook.rng
+    ln -s ${pkgs.docbook_xsl_ns}/xml/xsl ./xsl
+    ln -s ${epub-xsl} ./epub.xsl
+    ln -s ${xhtml-xsl} ./xhtml.xsl
+
+    ln -s ${../../nixos/doc/xmlformat.conf} ./xmlformat.conf
+    ln -s ${pkgs.documentation-highlighter} ./highlightjs
+
+    echo -n "${version}" > ./version
+  )
+  mv result $out
+''

doc/doc-support/lib-function-docs.nix

@@ -6,7 +6,7 @@
 
 with pkgs; stdenv.mkDerivation {
   name = "nixpkgs-lib-docs";
-  src = ./../lib;
+  src = ./../../lib;
 
   buildInputs = [ nixdoc ];
   installPhase = ''

doc/doc-support/lib-function-locations.nix

@@ -14,10 +14,10 @@ let
     builtins.map
       (subsetname: {
         subsetname = subsetname;
-        functions = libDefPos toplib."${subsetname}";
+        functions = libDefPos toplib.${subsetname};
       })
       (builtins.filter
-        (name: builtins.isAttrs toplib."${name}")
+        (name: builtins.isAttrs toplib.${name})
         (builtins.attrNames toplib));
 
   nixpkgsLib = pkgs.lib;

doc/doc-support/parameters.xml

@@ -0,0 +1,14 @@
+<?xml version='1.0'?>
+<xsl:stylesheet
+    xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
+    version="1.0">
+ <xsl:param name="section.autolabel" select="1" />
+ <xsl:param name="section.label.includes.component.label" select="1" />
+ <xsl:param name="html.stylesheet" select="'style.css overrides.css highlightjs/mono-blue.css'" />
+ <xsl:param name="html.script" select="'./highlightjs/highlight.pack.js ./highlightjs/loader.js'" />
+ <xsl:param name="xref.with.number.and.title" select="1" />
+ <xsl:param name="use.id.as.filename" select="1" />
+ <xsl:param name="toc.section.depth" select="3" />
+ <xsl:param name="admon.style" select="''" />
+ <xsl:param name="callout.graphics.extension" select="'.svg'" />
+</xsl:stylesheet>

doc/functions.xml

@@ -16,7 +16,9 @@
  <xi:include href="functions/fhs-environments.xml" />
  <xi:include href="functions/shell.xml" />
  <xi:include href="functions/dockertools.xml" />
+ <xi:include href="functions/snaptools.xml" />
  <xi:include href="functions/appimagetools.xml" />
  <xi:include href="functions/prefer-remote-fetch.xml" />
  <xi:include href="functions/nix-gitignore.xml" />
+ <xi:include href="functions/ocitools.xml" />
 </chapter>

doc/functions/appimagetools.xml

@@ -52,7 +52,7 @@ spot sensor temperature 0.000000, unit celsius, color scheme 0, calibration: off
 
 <prompt>$ </prompt>file -k type2.AppImage
 type2.AppImage: ELF 64-bit LSB executable, x86-64, version 1 (SYSV) (Lepton 3.x), scale 232-60668, spot sensor temperature -4.187500, color scheme 15, show scale bar, calibration: offset -0.000000, slope 0.000000 (Lepton 2.x), scale 4111-45000, spot sensor temperature 412442.250000, color scheme 3, minimum point enabled, calibration: offset -75402534979642766821519867692934234112.000000, slope 5815371847733706829839455140374904832.000000, dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.18, BuildID[sha1]=79dcc4e55a61c293c5e19edbd8d65b202842579f, stripped\012- data
-  </screen>
+</screen>
 
   <para>
    Note how the type 1 AppImage is described as an <literal>ISO 9660 CD-ROM

doc/functions/dockertools.xml

@@ -312,7 +312,23 @@ hello        latest   de2bf4786de6   About a minute ago   25.2MB
       Maximum number of layers to create.
      </para>
      <para>
-      <emphasis>Default:</emphasis> <literal>24</literal>
+      <emphasis>Default:</emphasis> <literal>100</literal>
+     </para>
+     <para>
+      <emphasis>Maximum:</emphasis> <literal>125</literal>
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry>
+    <term>
+     <varname>extraCommands</varname> <emphasis>optional</emphasis>
+    </term>
+    <listitem>
+     <para>
+       Shell commands to run while building the final layer, without access
+       to most of the layer contents. Changes to this layer are "on top"
+       of all the other layers, so can create additional directories
+       and files.
      </para>
     </listitem>
    </varlistentry>
@@ -480,9 +496,9 @@ pullImage {
     <literal>nix-prefetch-docker</literal> command can be used to get required
     image parameters:
 
-<programlisting>
+<screen>
-$ nix run nixpkgs.nix-prefetch-docker -c nix-prefetch-docker --image-name mysql --image-tag 5
+<prompt>$ </prompt>nix run nixpkgs.nix-prefetch-docker -c nix-prefetch-docker --image-name mysql --image-tag 5
-</programlisting>
+</screen>
 
     Since a given <varname>imageName</varname> may transparently refer to a
     manifest list of images which support multiple architectures and/or
@@ -491,17 +507,17 @@ $ nix run nixpkgs.nix-prefetch-docker -c nix-prefetch-docker --image-name mysql
     By default it will match the OS and architecture of the host the command is
     run on.
 
-<programlisting>
+<screen>
-$ nix-prefetch-docker --image-name mysql --image-tag 5 --arch x86_64 --os linux
+<prompt>$ </prompt>nix-prefetch-docker --image-name mysql --image-tag 5 --arch x86_64 --os linux
-</programlisting>
+</screen>
 
     Desired image name and tag can be set using
     <option>--final-image-name</option> and <option>--final-image-tag</option>
     arguments:
 
-<programlisting>
+<screen>
-$ nix-prefetch-docker --image-name mysql --image-tag 5 --final-image-name eu.gcr.io/my-project/mysql --final-image-tag prod
+<prompt>$ </prompt>nix-prefetch-docker --image-name mysql --image-tag 5 --final-image-name eu.gcr.io/my-project/mysql --final-image-tag prod
-</programlisting>
+</screen>
   </para>
  </section>
 
@@ -537,7 +553,7 @@ exportImage {
 
   name = someLayeredImage.name;
 }
-  </programlisting>
+</programlisting>
   </example>
 
   <para>

doc/functions/ocitools.xml

@@ -0,0 +1,76 @@
+<section xmlns="http://docbook.org/ns/docbook"
+         xmlns:xlink="http://www.w3.org/1999/xlink"
+         xmlns:xi="http://www.w3.org/2001/XInclude"
+         xml:id="sec-pkgs-ociTools">
+ <title>pkgs.ociTools</title>
+
+ <para>
+  <varname>pkgs.ociTools</varname> is a set of functions for creating
+  containers according to the
+  <link xlink:href="https://github.com/opencontainers/runtime-spec">OCI
+  container specification v1.0.0</link>. Beyond that it makes no assumptions
+  about the container runner you choose to use to run the created container.
+ </para>
+
+ <section xml:id="ssec-pkgs-ociTools-buildContainer">
+  <title>buildContainer</title>
+
+  <para>
+   This function creates a simple OCI container that runs a single command
+   inside of it. An OCI container consists of a <varname>config.json</varname>
+   and a rootfs directory.The nix store of the container will contain all
+   referenced dependencies of the given command.
+  </para>
+
+  <para>
+   The parameters of <varname>buildContainer</varname> with an example value
+   are described below:
+  </para>
+
+  <example xml:id='ex-ociTools-buildContainer'>
+   <title>Build Container</title>
+<programlisting>
+buildContainer {
+  args = [ (with pkgs; writeScript "run.sh" ''
+    #!${bash}/bin/bash
+    ${coreutils}/bin/exec ${bash}/bin/bash
+  '').outPath ]; <co xml:id='ex-ociTools-buildContainer-1' />
+
+  mounts = {
+    "/data" = {
+      type = "none";
+      source = "/var/lib/mydata";
+      options = [ "bind" ];
+    };
+  };<co xml:id='ex-ociTools-buildContainer-2' />
+
+  readonly = false; <co xml:id='ex-ociTools-buildContainer-3' />
+}
+
+    </programlisting>
+   <calloutlist>
+    <callout arearefs='ex-ociTools-buildContainer-1'>
+     <para>
+      <varname>args</varname> specifies a set of arguments to run inside the container.
+      This is the only required argument for <varname>buildContainer</varname>.
+      All referenced packages inside the derivation will be made available
+      inside the container
+     </para>
+    </callout>
+    <callout arearefs='ex-ociTools-buildContainer-2'>
+     <para>
+      <varname>mounts</varname> specifies additional mount points chosen by the
+      user. By default only a minimal set of necessary filesystems are mounted
+      into the container (e.g procfs, cgroupfs)
+     </para>
+    </callout>
+    <callout arearefs='ex-ociTools-buildContainer-3'>
+     <para>
+       <varname>readonly</varname> makes the container's rootfs read-only if it is set to true.
+       The default value is false <literal>false</literal>.
+     </para>
+    </callout>
+   </calloutlist>
+  </example>
+ </section>
+</section>

doc/functions/prefer-remote-fetch.xml

@@ -10,16 +10,16 @@
   upload while the builder can fetch faster directly from the source. To use
   it, put the following snippet as a new overlay:
 <programlisting>
-    self: super:
+self: super:
-      (super.prefer-remote-fetch self super)
+  (super.prefer-remote-fetch self super)
-  </programlisting>
+</programlisting>
   A full configuration example for that sets the overlay up for your own
   account, could look like this
-<programlisting>
+<screen>
-    $ mkdir ~/.config/nixpkgs/overlays/
+<prompt>$ </prompt>mkdir ~/.config/nixpkgs/overlays/
-    $ cat &gt; ~/.config/nixpkgs/overlays/prefer-remote-fetch.nix &lt;&lt;EOF
+<prompt>$ </prompt>cat &gt; ~/.config/nixpkgs/overlays/prefer-remote-fetch.nix &lt;&lt;EOF
-      self: super: super.prefer-remote-fetch self super
+  self: super: super.prefer-remote-fetch self super
-    EOF
+EOF
-  </programlisting>
+</screen>
  </para>
 </section>

doc/functions/snap/example-firefox.nix

@@ -0,0 +1,28 @@
+let
+  inherit (import <nixpkgs> { }) snapTools firefox;
+in snapTools.makeSnap {
+  meta = {
+    name = "nix-example-firefox";
+    summary = firefox.meta.description;
+    architectures = [ "amd64" ];
+    apps.nix-example-firefox = {
+      command = "${firefox}/bin/firefox";
+      plugs = [
+        "pulseaudio"
+        "camera"
+        "browser-support"
+        "avahi-observe"
+        "cups-control"
+        "desktop"
+        "desktop-legacy"
+        "gsettings"
+        "home"
+        "network"
+        "mount-observe"
+        "removable-media"
+        "x11"
+      ];
+    };
+    confinement = "strict";
+  };
+}

doc/functions/snap/example-hello.nix

@@ -0,0 +1,12 @@
+let
+  inherit (import <nixpkgs> { }) snapTools hello;
+in snapTools.makeSnap {
+  meta = {
+    name = "hello";
+    summary = hello.meta.description;
+    description = hello.meta.longDescription;
+    architectures = [ "amd64" ];
+    confinement = "strict";
+    apps.hello.command = "${hello}/bin/hello";
+  };
+}

doc/functions/snaptools.xml

@@ -0,0 +1,74 @@
+<section xmlns="http://docbook.org/ns/docbook"
+         xmlns:xlink="http://www.w3.org/1999/xlink"
+         xmlns:xi="http://www.w3.org/2001/XInclude"
+         xml:id="sec-pkgs-snapTools">
+ <title>pkgs.snapTools</title>
+
+ <para>
+  <varname>pkgs.snapTools</varname> is a set of functions for creating
+  Snapcraft images. Snap and Snapcraft is not used to perform these operations.
+ </para>
+
+ <section xml:id="ssec-pkgs-snapTools-makeSnap-signature">
+  <title>The makeSnap Function</title>
+
+  <para>
+   <function>makeSnap</function> takes a single named argument,
+   <parameter>meta</parameter>. This argument mirrors
+   <link xlink:href="https://docs.snapcraft.io/snap-format">the upstream
+   <filename>snap.yaml</filename> format</link> exactly.
+  </para>
+
+  <para>
+   The <parameter>base</parameter> should not be be specified, as
+   <function>makeSnap</function> will force set it.
+  </para>
+
+  <para>
+   Currently, <function>makeSnap</function> does not support creating GUI
+   stubs.
+  </para>
+ </section>
+
+ <section xml:id="ssec-pkgs-snapTools-build-a-snap-hello">
+  <title>Build a Hello World Snap</title>
+
+  <example xml:id="ex-snapTools-buildSnap-hello">
+   <title>Making a Hello World Snap</title>
+   <para>
+    The following expression packages GNU Hello as a Snapcraft snap.
+   </para>
+<programlisting><xi:include href="./snap/example-hello.nix" parse="text" /></programlisting>
+   <para>
+    <command>nix-build</command> this expression and install it with
+    <command>snap install ./result --dangerous</command>.
+    <command>hello</command> will now be the Snapcraft version of the package.
+   </para>
+  </example>
+ </section>
+
+ <section xml:id="ssec-pkgs-snapTools-build-a-snap-firefox">
+  <title>Build a Hello World Snap</title>
+
+  <example xml:id="ex-snapTools-buildSnap-firefox">
+   <title>Making a Graphical Snap</title>
+   <para>
+    Graphical programs require many more integrations with the host. This
+    example uses Firefox as an example, because it is one of the most
+    complicated programs we could package.
+   </para>
+<programlisting><xi:include href="./snap/example-firefox.nix" parse="text" /></programlisting>
+   <para>
+    <command>nix-build</command> this expression and install it with
+    <command>snap install ./result --dangerous</command>.
+    <command>nix-example-firefox</command> will now be the Snapcraft version of
+    the Firefox package.
+   </para>
+   <para>
+    The specific meaning behind plugs can be looked up in the
+    <link xlink:href="https://docs.snapcraft.io/supported-interfaces">Snapcraft
+    interface documentation</link>.
+   </para>
+  </example>
+ </section>
+</section>

doc/functions/trivial-builders.xml

@@ -35,25 +35,25 @@
      An example of using <literal>runCommand</literal> is provided below.
     </para>
 <programlisting>
-       (import &lt;nixpkgs&gt; {}).runCommand "my-example" {} ''
+(import &lt;nixpkgs&gt; {}).runCommand "my-example" {} ''
-         echo My example command is running
+  echo My example command is running
 
-         mkdir $out
+  mkdir $out
 
-         echo I can write data to the Nix store > $out/message
+  echo I can write data to the Nix store > $out/message
 
-         echo I can also run basic commands like:
+  echo I can also run basic commands like:
 
-         echo ls
+  echo ls
-         ls
+  ls
 
-         echo whoami
+  echo whoami
-         whoami
+  whoami
 
-         echo date
+  echo date
-         date
+  date
-       ''
+''
-     </programlisting>
+</programlisting>
    </listitem>
   </varlistentry>
   <varlistentry>

doc/languages-frameworks/android.section.md

@@ -185,10 +185,9 @@ with import <nixpkgs> {};
 
 androidenv.emulateApp {
   name = "emulate-MyAndroidApp";
-  platformVersion = "24";
+  platformVersion = "28";
-  abiVersion = "armeabi-v7a"; # mips, x86 or x86_64
+  abiVersion = "x86_64"; # armeabi-v7a, mips, x86
-  systemImageType = "default";
+  systemImageType = "google_apis_playstore";
-  useGoogleAPIs = false;
 }
 ```
 
@@ -201,7 +200,7 @@ with import <nixpkgs> {};
 androidenv.emulateApp {
   name = "emulate-MyAndroidApp";
   platformVersion = "24";
-  abiVersion = "armeabi-v7a"; # mips, x86 or x86_64
+  abiVersion = "armeabi-v7a"; # mips, x86, x86_64
   systemImageType = "default";
   useGoogleAPIs = false;
   app = ./MyApp.apk;

doc/languages-frameworks/beam.xml

@@ -131,8 +131,8 @@
    in <literal>beamPackages</literal>, use the following command:
   </para>
 
-<programlisting>
+<screen>
-$ nix-env -f &quot;&lt;nixpkgs&gt;&quot; -qaP -A beamPackages
+<prompt>$ </prompt>nix-env -f &quot;&lt;nixpkgs&gt;&quot; -qaP -A beamPackages
 beamPackages.esqlite    esqlite-0.2.1
 beamPackages.goldrush   goldrush-0.1.7
 beamPackages.ibrowse    ibrowse-4.2.2
@@ -140,16 +140,16 @@ beamPackages.jiffy      jiffy-0.14.5
 beamPackages.lager      lager-3.0.2
 beamPackages.meck       meck-0.8.3
 beamPackages.rebar3-pc  pc-1.1.0
-  </programlisting>
+</screen>
 
   <para>
    To install any of those packages into your profile, refer to them by their
    attribute path (first column):
   </para>
 
-<programlisting>
+<screen>
-$ nix-env -f &quot;&lt;nixpkgs&gt;&quot; -iA beamPackages.ibrowse
+<prompt>$ </prompt>nix-env -f &quot;&lt;nixpkgs&gt;&quot; -iA beamPackages.ibrowse
-  </programlisting>
+</screen>
 
   <para>
    The attribute path of any BEAM package corresponds to the name of that
@@ -178,22 +178,22 @@ $ nix-env -f &quot;&lt;nixpkgs&gt;&quot; -iA beamPackages.ibrowse
     </para>
 
 <programlisting>
-        { stdenv, fetchFromGitHub, buildRebar3, ibrowse, jsx, erlware_commons }:
+{ stdenv, fetchFromGitHub, buildRebar3, ibrowse, jsx, erlware_commons }:
 
-        buildRebar3 rec {
+buildRebar3 rec {
-          name = "hex2nix";
+  name = "hex2nix";
-          version = "0.0.1";
+  version = "0.0.1";
 
-          src = fetchFromGitHub {
+  src = fetchFromGitHub {
-            owner = "ericbmerritt";
+    owner = "ericbmerritt";
-            repo = "hex2nix";
+    repo = "hex2nix";
-            rev = "${version}";
+    rev = "${version}";
-            sha256 = "1w7xjidz1l5yjmhlplfx7kphmnpvqm67w99hd2m7kdixwdxq0zqg";
+    sha256 = "1w7xjidz1l5yjmhlplfx7kphmnpvqm67w99hd2m7kdixwdxq0zqg";
-          };
+  };
 
-          beamDeps = [ ibrowse jsx erlware_commons ];
+  beamDeps = [ ibrowse jsx erlware_commons ];
-        }
+}
-      </programlisting>
+</programlisting>
 
     <para>
      Such derivations are callable with
@@ -228,29 +228,29 @@ $ nix-env -f &quot;&lt;nixpkgs&gt;&quot; -iA beamPackages.ibrowse
     </para>
 
 <programlisting>
-        { buildErlangMk, fetchHex, cowlib, ranch }:
+{ buildErlangMk, fetchHex, cowlib, ranch }:
 
-        buildErlangMk {
+buildErlangMk {
-          name = "cowboy";
+  name = "cowboy";
-          version = "1.0.4";
+  version = "1.0.4";
 
-          src = fetchHex {
+  src = fetchHex {
-            pkg = "cowboy";
+    pkg = "cowboy";
-            version = "1.0.4";
+    version = "1.0.4";
-            sha256 = "6a0edee96885fae3a8dd0ac1f333538a42e807db638a9453064ccfdaa6b9fdac";
+    sha256 = "6a0edee96885fae3a8dd0ac1f333538a42e807db638a9453064ccfdaa6b9fdac";
-          };
+  };
 
-          beamDeps = [ cowlib ranch ];
+  beamDeps = [ cowlib ranch ];
 
-          meta = {
+  meta = {
-            description = ''
+    description = ''
-              Small, fast, modular HTTP server written in Erlang
+      Small, fast, modular HTTP server written in Erlang
-            '';
+    '';
-            license = stdenv.lib.licenses.isc;
+    license = stdenv.lib.licenses.isc;
-            homepage = https://github.com/ninenines/cowboy;
+    homepage = https://github.com/ninenines/cowboy;
-          };
+  };
-        }
+}
-      </programlisting>
+</programlisting>
    </section>
 
    <section xml:id="mix-packages">
@@ -262,56 +262,56 @@ $ nix-env -f &quot;&lt;nixpkgs&gt;&quot; -iA beamPackages.ibrowse
     </para>
 
 <programlisting>
-        { buildMix, fetchHex, plug, absinthe }:
+{ buildMix, fetchHex, plug, absinthe }:
 
-        buildMix {
+buildMix {
-          name = "absinthe_plug";
+  name = "absinthe_plug";
-          version = "1.0.0";
+  version = "1.0.0";
 
-          src = fetchHex {
+  src = fetchHex {
-            pkg = "absinthe_plug";
+    pkg = "absinthe_plug";
-            version = "1.0.0";
+    version = "1.0.0";
-            sha256 = "08459823fe1fd4f0325a8bf0c937a4520583a5a26d73b193040ab30a1dfc0b33";
+    sha256 = "08459823fe1fd4f0325a8bf0c937a4520583a5a26d73b193040ab30a1dfc0b33";
-          };
+  };
 
-          beamDeps = [ plug absinthe ];
+  beamDeps = [ plug absinthe ];
 
-          meta = {
+  meta = {
-            description = ''
+    description = ''
-              A plug for Absinthe, an experimental GraphQL toolkit
+      A plug for Absinthe, an experimental GraphQL toolkit
-            '';
+    '';
-            license = stdenv.lib.licenses.bsd3;
+    license = stdenv.lib.licenses.bsd3;
-            homepage = https://github.com/CargoSense/absinthe_plug;
+    homepage = https://github.com/CargoSense/absinthe_plug;
-          };
+  };
-        }
+}
-      </programlisting>
+</programlisting>
 
     <para>
      Alternatively, we can use <literal>buildHex</literal> as a shortcut:
     </para>
 
 <programlisting>
-        { buildHex, buildMix, plug, absinthe }:
+{ buildHex, buildMix, plug, absinthe }:
 
-        buildHex {
+buildHex {
-          name = "absinthe_plug";
+  name = "absinthe_plug";
-          version = "1.0.0";
+  version = "1.0.0";
 
-          sha256 = "08459823fe1fd4f0325a8bf0c937a4520583a5a26d73b193040ab30a1dfc0b33";
+  sha256 = "08459823fe1fd4f0325a8bf0c937a4520583a5a26d73b193040ab30a1dfc0b33";
 
-          builder = buildMix;
+  builder = buildMix;
 
-          beamDeps = [ plug absinthe ];
+  beamDeps = [ plug absinthe ];
 
-          meta = {
+  meta = {
-            description = ''
+    description = ''
-              A plug for Absinthe, an experimental GraphQL toolkit
+      A plug for Absinthe, an experimental GraphQL toolkit
-            '';
+    '';
-            license = stdenv.lib.licenses.bsd3;
+    license = stdenv.lib.licenses.bsd3;
-            homepage = https://github.com/CargoSense/absinthe_plug;
+    homepage = https://github.com/CargoSense/absinthe_plug;
-         };
+ };
-       }
+}
-      </programlisting>
+</programlisting>
    </section>
   </section>
  </section>
@@ -330,47 +330,47 @@ $ nix-env -f &quot;&lt;nixpkgs&gt;&quot; -iA beamPackages.ibrowse
     could do the following:
    </para>
 
-<programlisting>
+<screen>
-      $ nix-shell -A beamPackages.ibrowse.env --run "erl"
+<prompt>$ </prompt><userinput>nix-shell -A beamPackages.ibrowse.env --run "erl"</userinput>
-      Erlang/OTP 18 [erts-7.0] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]
+<computeroutput>Erlang/OTP 18 [erts-7.0] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]
 
-      Eshell V7.0  (abort with ^G)
+Eshell V7.0  (abort with ^G)</computeroutput>
-      1> m(ibrowse).
+<prompt>1> </prompt><userinput>m(ibrowse).</userinput>
-      Module: ibrowse
+<computeroutput>Module: ibrowse
-      MD5: 3b3e0137d0cbb28070146978a3392945
+MD5: 3b3e0137d0cbb28070146978a3392945
-      Compiled: January 10 2016, 23:34
+Compiled: January 10 2016, 23:34
-      Object file: /nix/store/g1rlf65rdgjs4abbyj4grp37ry7ywivj-ibrowse-4.2.2/lib/erlang/lib/ibrowse-4.2.2/ebin/ibrowse.beam
+Object file: /nix/store/g1rlf65rdgjs4abbyj4grp37ry7ywivj-ibrowse-4.2.2/lib/erlang/lib/ibrowse-4.2.2/ebin/ibrowse.beam
-      Compiler options:  [{outdir,"/tmp/nix-build-ibrowse-4.2.2.drv-0/hex-source-ibrowse-4.2.2/_build/default/lib/ibrowse/ebin"},
+Compiler options:  [{outdir,"/tmp/nix-build-ibrowse-4.2.2.drv-0/hex-source-ibrowse-4.2.2/_build/default/lib/ibrowse/ebin"},
-      debug_info,debug_info,nowarn_shadow_vars,
+debug_info,debug_info,nowarn_shadow_vars,
-      warn_unused_import,warn_unused_vars,warnings_as_errors,
+warn_unused_import,warn_unused_vars,warnings_as_errors,
-      {i,"/tmp/nix-build-ibrowse-4.2.2.drv-0/hex-source-ibrowse-4.2.2/_build/default/lib/ibrowse/include"}]
+{i,"/tmp/nix-build-ibrowse-4.2.2.drv-0/hex-source-ibrowse-4.2.2/_build/default/lib/ibrowse/include"}]
-      Exports:
+Exports:
-      add_config/1                  send_req_direct/7
+add_config/1                  send_req_direct/7
-      all_trace_off/0               set_dest/3
+all_trace_off/0               set_dest/3
-      code_change/3                 set_max_attempts/3
+code_change/3                 set_max_attempts/3
-      get_config_value/1            set_max_pipeline_size/3
+get_config_value/1            set_max_pipeline_size/3
-      get_config_value/2            set_max_sessions/3
+get_config_value/2            set_max_sessions/3
-      get_metrics/0                 show_dest_status/0
+get_metrics/0                 show_dest_status/0
-      get_metrics/2                 show_dest_status/1
+get_metrics/2                 show_dest_status/1
-      handle_call/3                 show_dest_status/2
+handle_call/3                 show_dest_status/2
-      handle_cast/2                 spawn_link_worker_process/1
+handle_cast/2                 spawn_link_worker_process/1
-      handle_info/2                 spawn_link_worker_process/2
+handle_info/2                 spawn_link_worker_process/2
-      init/1                        spawn_worker_process/1
+init/1                        spawn_worker_process/1
-      module_info/0                 spawn_worker_process/2
+module_info/0                 spawn_worker_process/2
-      module_info/1                 start/0
+module_info/1                 start/0
-      rescan_config/0               start_link/0
+rescan_config/0               start_link/0
-      rescan_config/1               stop/0
+rescan_config/1               stop/0
-      send_req/3                    stop_worker_process/1
+send_req/3                    stop_worker_process/1
-      send_req/4                    stream_close/1
+send_req/4                    stream_close/1
-      send_req/5                    stream_next/1
+send_req/5                    stream_next/1
-      send_req/6                    terminate/2
+send_req/6                    terminate/2
-      send_req_direct/4             trace_off/0
+send_req_direct/4             trace_off/0
-      send_req_direct/5             trace_off/2
+send_req_direct/5             trace_off/2
-      send_req_direct/6             trace_on/0
+send_req_direct/6             trace_on/0
-      trace_on/2
+trace_on/2
-      ok
+ok</computeroutput>
-      2>
+<prompt>2></prompt>
-    </programlisting>
+</screen>
 
    <para>
     Notice the <literal>-A beamPackages.ibrowse.env</literal>. That is the key
@@ -408,7 +408,7 @@ let
 in
 
   drv
-    </programlisting>
+</programlisting>
 
    <section xml:id="building-in-a-shell">
     <title>Building in a Shell (for Mix Projects)</title>
@@ -474,7 +474,7 @@ plt:
 analyze: build plt
         $(NIX_SHELL) --run "mix dialyzer --no-compile"
 
-    </programlisting>
+</programlisting>
 
     <para>
      Using a <literal>shell.nix</literal> as described (see
@@ -513,9 +513,9 @@ analyze: build plt
    <literal>nixpkgs</literal> repository:
   </para>
 
-<programlisting>
+<screen>
-$ nix-build -A beamPackages
+<prompt>$ </prompt>nix-build -A beamPackages
-    </programlisting>
+</screen>
 
   <para>
    That will attempt to build every package in <literal>beamPackages</literal>.

doc/languages-frameworks/crystal.section.md

@@ -0,0 +1,71 @@
+# Crystal
+
+## Building a Crystal package
+
+This section uses [Mint](https://github.com/mint-lang/mint) as an example for how to build a Crystal package.
+
+If the Crystal project has any dependencies, the first step is to get a `shards.nix` file encoding those. Get a copy of the project and go to its root directory such that its `shard.lock` file is in the current directory, then run `crystal2nix` in it
+```bash
+$ git clone https://github.com/mint-lang/mint
+$ cd mint
+$ git checkout 0.5.0
+$ nix-shell -p crystal2nix --run crystal2nix
+```
+
+This should have generated a `shards.nix` file.
+
+Next create a Nix file for your derivation and use `pkgs.crystal.buildCrystalPackage` as follows:
+```nix
+with import <nixpkgs> {};
+crystal.buildCrystalPackage rec {
+  pname = "mint";
+  version = "0.5.0";
+
+  src = fetchFromGitHub {
+    owner = "mint-lang";
+    repo = "mint";
+    rev = version;
+    sha256 = "0vxbx38c390rd2ysvbwgh89v2232sh5rbsp3nk9wzb70jybpslvl";
+  };
+
+  # Insert the path to your shards.nix file here
+  shardsFile = ./shards.nix;
+
+  ...
+}
+```
+
+This won't build anything yet, because we haven't told it what files build. We can specify a mapping from binary names to source files with the `crystalBinaries` attribute. The project's compilation instructions should show this. For Mint, the binary is called "mint", which is compiled from the source file `src/mint.cr`, so we'll specify this as follows:
+
+```nix
+  crystalBinaries.mint.src = "src/mint.cr";
+
+  # ...
+```
+
+Additionally you can override the default `crystal build` options (which are currently `--release --progress --no-debug --verbose`) with
+
+```nix
+  crystalBinaries.mint.options = [ "--release" "--verbose" ];
+```
+
+Depending on the project, you might need additional steps to get it to compile successfully. In Mint's case, we need to link against openssl, so in the end the Nix file looks as follows:
+
+```nix
+with import <nixpkgs> {};
+crystal.buildCrystalPackage rec {
+  version = "0.5.0";
+  pname = "mint";
+  src = fetchFromGitHub {
+    owner = "mint-lang";
+    repo = "mint";
+    rev = version;
+    sha256 = "0vxbx38c390rd2ysvbwgh89v2232sh5rbsp3nk9wzb70jybpslvl";
+  };
+
+  shardsFile = ./shards.nix;
+  crystalBinaries.mint.src = "src/mint.cr";
+
+  buildInputs = [ openssl_1_0_2 ];
+}
+```

doc/languages-frameworks/gnome.xml

@@ -0,0 +1,263 @@
+<section xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="sec-language-gnome">
+ <title>GNOME</title>
+
+ <section xml:id="ssec-gnome-packaging">
+  <title>Packaging GNOME applications</title>
+
+  <para>
+   Programs in the GNOME universe are written in various languages but they all use GObject-based libraries like GLib, GTK or GStreamer. These libraries are often modular, relying on looking into certain directories to find their modules. However, due to Nix’s specific file system organization, this will fail without our intervention. Fortunately, the libraries usually allow overriding the directories through environment variables, either natively or thanks to a patch in nixpkgs. <link xlink:href="#fun-wrapProgram">Wrapping</link> the executables to ensure correct paths are available to the application constitutes a significant part of packaging a modern desktop application. In this section, we will describe various modules needed by such applications, environment variables needed to make the modules load, and finally a script that will do the work for us.
+  </para>
+
+  <section xml:id="ssec-gnome-settings">
+   <title>Settings</title>
+
+   <para>
+    <link xlink:href="https://developer.gnome.org/gio/stable/GSettings.html">GSettings</link> API is often used for storing settings. GSettings schemas are required, to know the type and other metadata of the stored values. GLib looks for <filename>glib-2.0/schemas/gschemas.compiled</filename> files inside the directories of <envar>XDG_DATA_DIRS</envar>.
+   </para>
+
+   <para>
+    On Linux, GSettings API is implemented using <link xlink:href="https://wiki.gnome.org/Projects/dconf">dconf</link> backend. You will need to add <literal>dconf</literal> GIO module to <envar>GIO_EXTRA_MODULES</envar> variable, otherwise the <literal>memory</literal> backend will be used and the saved settings will not be persistent.
+   </para>
+
+   <para>
+    Last you will need the dconf database D-Bus service itself. You can enable it using <option>programs.dconf.enable</option>.
+   </para>
+
+   <para>
+    Some applications will also require <package>gsettings-desktop-schemas</package> for things like reading proxy configuration or user interface customization. This dependency is often not mentioned by upstream, you should grep for <literal>org.gnome.desktop</literal> and <literal>org.gnome.system</literal> to see if the schemas are needed.
+   </para>
+  </section>
+
+  <section xml:id="ssec-gnome-icons">
+   <title>Icons</title>
+
+   <para>
+    When an application uses icons, an icon theme should be available in <envar>XDG_DATA_DIRS</envar>. The package for the default, icon-less <link xlink:href="https://www.freedesktop.org/wiki/Software/icon-theme/">hicolor-icon-theme</link> contains <link linkend="ssec-gnome-hooks-hicolor-icon-theme">a setup hook</link> that will pick up icon themes from <literal>buildInputs</literal> and pass it to our wrapper. Unfortunately, relying on that would mean every user has to download the theme included in the package expression no matter their preference. For that reason, we leave the installation of icon theme on the user. If you use one of the desktop environments, you probably already have an icon theme installed.
+   </para>
+  </section>
+
+  <section xml:id="ssec-gnome-themes">
+   <title>GTK Themes</title>
+
+   <para>
+    Previously, a GTK theme needed to be in <envar>XDG_DATA_DIRS</envar>. This is no longer necessary for most programs since GTK incorporated Adwaita theme. Some programs (for example, those designed for <link xlink:href="https://elementary.io/docs/human-interface-guidelines#human-interface-guidelines">elementary HIG</link>) might require a special theme like <package>pantheon.elementary-gtk-theme</package>.
+   </para>
+  </section>
+
+  <section xml:id="ssec-gnome-typelibs">
+   <title>GObject introspection typelibs</title>
+
+   <para>
+    <link xlink:href="https://wiki.gnome.org/Projects/GObjectIntrospection">GObject introspection</link> allows applications to use C libraries in other languages easily. It does this through <literal>typelib</literal> files searched in <envar>GI_TYPELIB_PATH</envar>.
+   </para>
+  </section>
+
+  <section xml:id="ssec-gnome-plugins">
+   <title>Various plug-ins</title>
+
+   <para>
+    If your application uses <link xlink:href="https://gstreamer.freedesktop.org/">GStreamer</link> or <link xlink:href="https://wiki.gnome.org/Projects/Grilo">Grilo</link>, you should set <envar>GST_PLUGIN_SYSTEM_PATH_1_0</envar> and <envar>GRL_PLUGIN_PATH</envar>, respectively.
+   </para>
+  </section>
+ </section>
+
+ <section xml:id="ssec-gnome-hooks">
+  <title>Onto <package>wrapGAppsHook</package></title>
+
+  <para>
+   Given the requirements above, the package expression would become messy quickly:
+<programlisting>
+preFixup = ''
+  for f in $(find $out/bin/ $out/libexec/ -type f -executable); do
+    wrapProgram "$f" \
+      --prefix GIO_EXTRA_MODULES : "${getLib gnome3.dconf}/lib/gio/modules" \
+      --prefix XDG_DATA_DIRS : "$out/share" \
+      --prefix XDG_DATA_DIRS : "$out/share/gsettings-schemas/${name}" \
+      --prefix XDG_DATA_DIRS : "${gsettings-desktop-schemas}/share/gsettings-schemas/${gsettings-desktop-schemas.name}" \
+      --prefix XDG_DATA_DIRS : "${hicolor-icon-theme}/share" \
+      --prefix GI_TYPELIB_PATH : "${lib.makeSearchPath "lib/girepository-1.0" [ pango json-glib ]}"
+  done
+'';
+</programlisting>
+   Fortunately, there is <package>wrapGAppsHook</package>, that does the wrapping for us. In particular, it works in conjunction with other setup hooks that will populate the variable:
+   <itemizedlist>
+    <listitem xml:id="ssec-gnome-hooks-wrapgappshook">
+     <para>
+      <package>wrapGAppsHook</package> itself will add the package’s <filename>share</filename> directory to <envar>XDG_DATA_DIRS</envar>.
+     </para>
+    </listitem>
+    <listitem xml:id="ssec-gnome-hooks-glib">
+     <para>
+      <package>glib</package> setup hook will populate <envar>GSETTINGS_SCHEMAS_PATH</envar> and then <package>wrapGAppsHook</package> will prepend it to <envar>XDG_DATA_DIRS</envar>.
+     </para>
+    </listitem>
+    <listitem xml:id="ssec-gnome-hooks-dconf">
+     <para>
+      <package>gnome3.dconf.lib</package> is a dependency of <package>wrapGAppsHook</package>, which then also adds it to the <envar>GIO_EXTRA_MODULES</envar> variable.
+     </para>
+    </listitem>
+    <listitem xml:id="ssec-gnome-hooks-hicolor-icon-theme">
+     <para>
+      <package>hicolor-icon-theme</package>’s setup hook will add icon themes to <envar>XDG_ICON_DIRS</envar> which is prepended to <envar>XDG_DATA_DIRS</envar> by <package>wrapGAppsHook</package>.
+     </para>
+    </listitem>
+    <listitem xml:id="ssec-gnome-hooks-gobject-introspection">
+     <para>
+      <package>gobject-introspection</package> setup hook populates <envar>GI_TYPELIB_PATH</envar> variable with <filename>lib/girepository-1.0</filename> directories of dependencies, which is then added to wrapper by <package>wrapGAppsHook</package>. It also adds <filename>share</filename> directories of dependencies to <envar>XDG_DATA_DIRS</envar>, which is intended to promote GIR files but it also <link xlink:href="https://github.com/NixOS/nixpkgs/issues/32790">pollutes the closures</link> of packages using <package>wrapGAppsHook</package>.
+     </para>
+     <warning>
+      <para>
+       The setup hook <link xlink:href="https://github.com/NixOS/nixpkgs/issues/56943">currently</link> does not work in expressions with <literal>strictDeps</literal> enabled, like Python packages. In those cases, you will need to disable it with <code>strictDeps = false;</code>.
+      </para>
+     </warning>
+    </listitem>
+    <listitem xml:id="ssec-gnome-hooks-gst-grl-plugins">
+     <para>
+      Setup hooks of <package>gst_all_1.gstreamer</package> and <package>gnome3.grilo</package> will populate the <envar>GST_PLUGIN_SYSTEM_PATH_1_0</envar> and <envar>GRL_PLUGIN_PATH</envar> variables, respectively, which will then be added to the wrapper by <literal>wrapGAppsHook</literal>.
+     </para>
+    </listitem>
+   </itemizedlist>
+  </para>
+
+  <para>
+   You can also pass additional arguments to <literal>makeWrapper</literal> using <literal>gappsWrapperArgs</literal> in <literal>preFixup</literal> hook:
+<programlisting>
+preFixup = ''
+  gappsWrapperArgs+=(
+    # Thumbnailers
+    --prefix XDG_DATA_DIRS : "${gdk-pixbuf}/share"
+    --prefix XDG_DATA_DIRS : "${librsvg}/share"
+    --prefix XDG_DATA_DIRS : "${shared-mime-info}/share"
+  )
+'';
+</programlisting>
+  </para>
+ </section>
+
+ <section xml:id="ssec-gnome-updating">
+  <title>Updating GNOME packages</title>
+
+  <para>
+   Most GNOME package offer <link linkend="var-passthru-updateScript"><literal>updateScript</literal></link>, it is therefore possible to update to latest source tarball by running <command>nix-shell maintainers/scripts/update.nix --argstr package gnome3.nautilus</command> or even en masse with <command>nix-shell maintainers/scripts/update.nix --argstr path gnome3</command>. Read the package’s <filename>NEWS</filename> file to see what changed.
+  </para>
+ </section>
+
+ <section xml:id="ssec-gnome-common-issues">
+  <title>Frequently encountered issues</title>
+
+  <variablelist>
+   <varlistentry xml:id="ssec-gnome-common-issues-no-schemas">
+    <term>
+     <computeroutput>GLib-GIO-ERROR **: <replaceable>06:04:50.903</replaceable>: No GSettings schemas are installed on the system</computeroutput>
+    </term>
+    <listitem>
+     <para>
+      There are no schemas avalable in <envar>XDG_DATA_DIRS</envar>. Temporarily add a random package containing schemas like <package>gsettings-desktop-schemas</package> to <literal>buildInputs</literal>. <link linkend="ssec-gnome-hooks-glib"><package>glib</package></link> and <link linkend="ssec-gnome-hooks-wrapgappshook"><package>wrapGAppsHook</package></link> setup hooks will take care of making the schemas available to application and you will see the actual missing schemas with the <link linkend="ssec-gnome-common-issues-missing-schema">next error</link>. Or you can try looking through the source code for the actual schemas used.
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry xml:id="ssec-gnome-common-issues-missing-schema">
+    <term>
+     <computeroutput>GLib-GIO-ERROR **: <replaceable>06:04:50.903</replaceable>: Settings schema ‘<replaceable>org.gnome.foo</replaceable>’ is not installed</computeroutput>
+    </term>
+    <listitem>
+     <para>
+      Package is missing some GSettings schemas. You can find out the package containing the schema with <command>nix-locate <replaceable>org.gnome.foo</replaceable>.gschema.xml</command> and let the hooks handle the wrapping as <link linkend="ssec-gnome-common-issues-no-schemas">above</link>.
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry xml:id="ssec-gnome-common-issues-double-wrapped">
+    <term>
+     When using <package>wrapGAppsHook</package> with special derivers you can end up with double wrapped binaries.
+    </term>
+    <listitem>
+     <para>
+      This is because derivers like <function>python.pkgs.buildPythonApplication</function> or <function>qt5.mkDerivation</function> have setup-hooks automatically added that produce wrappers with <package>makeWrapper</package>. The simplest way to workaround that is to disable the <package>wrapGAppsHook</package> automatic wrapping with <code>dontWrapGApps = true;</code> and pass the arguments it intended to pass to <package>makeWrapper</package> to another.
+     </para>
+     <para>
+      In the case of a Python application it could look like:
+<programlisting>
+python3.pkgs.buildPythonApplication {
+  pname = "gnome-music";
+  version = "3.32.2";
+
+  nativeBuildInputs = [
+    wrapGAppsHook
+    gobject-introspection
+    ...
+  ];
+
+  dontWrapGApps = true;
+
+  # Arguments to be passed to `makeWrapper`, only used by buildPython*
+  makeWrapperArgs = [
+    "\${gappsWrapperArgs[@]}"
+  ];
+}
+</programlisting>
+      And for a QT app like:
+<programlisting>
+mkDerivation {
+  pname = "calibre";
+  version = "3.47.0";
+
+  nativeBuildInputs = [
+    wrapGAppsHook
+    qmake
+    ...
+  ];
+
+  dontWrapGApps = true;
+
+  # Arguments to be passed to `makeWrapper`, only used by qt5’s mkDerivation
+  qtWrapperArgs [
+    "\${gappsWrapperArgs[@]}"
+  ];
+}
+</programlisting>
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry xml:id="ssec-gnome-common-issues-unwrappable-package">
+    <term>
+     I am packaging a project that cannot be wrapped, like a library or GNOME Shell extension.
+    </term>
+    <listitem>
+     <para>
+      You can rely on applications depending on the library set the necessary environment variables but that it often easy to miss. Instead we recommend to patch the paths in the source code whenever possible. Here are some examples:
+      <itemizedlist>
+       <listitem xml:id="ssec-gnome-common-issues-unwrappable-package-gnome-shell-ext">
+        <para>
+         <link xlink:href="https://github.com/NixOS/nixpkgs/blob/7bb8f05f12ca3cff9da72b56caa2f7472d5732bc/pkgs/desktops/gnome-3/core/gnome-shell-extensions/default.nix#L21-L24">Replacing a <envar>GI_TYPELIB_PATH</envar> in GNOME Shell extension</link> – we are using <function>substituteAll</function> to include the path to a typelib into a patch.
+        </para>
+       </listitem>
+       <listitem xml:id="ssec-gnome-common-issues-unwrappable-package-gsettings">
+        <para>
+         The following examples are hardcoding GSettings schema paths. To get the schema paths we use the functions
+         <itemizedlist>
+          <listitem>
+           <para>
+            <function>glib.getSchemaPath</function> Takes a nix package attribute as an argument.
+           </para>
+          </listitem>
+          <listitem>
+           <para>
+            <function>glib.makeSchemaPath</function> Takes a package output like <literal>$out</literal> and a derivation name. You should use this if the schemas you need to hardcode are in the same derivation.
+           </para>
+          </listitem>
+         </itemizedlist>
+        </para>
+        <para xml:id="ssec-gnome-common-issues-unwrappable-package-gsettings-vala">
+         <link xlink:href="https://github.com/NixOS/nixpkgs/blob/7bb8f05f12ca3cff9da72b56caa2f7472d5732bc/pkgs/desktops/pantheon/apps/elementary-files/default.nix#L78-L86">Hard-coding GSettings schema path in Vala plug-in (dynamically loaded library)</link> – here, <function>substituteAll</function> cannot be used since the schema comes from the same package preventing us from pass its path to the function, probably due to a <link xlink:href="https://github.com/NixOS/nix/issues/1846">Nix bug</link>.
+        </para>
+        <para xml:id="ssec-gnome-common-issues-unwrappable-package-gsettings-c">
+         <link xlink:href="https://github.com/NixOS/nixpkgs/blob/29c120c065d03b000224872251bed93932d42412/pkgs/development/libraries/glib-networking/default.nix#L31-L34">Hard-coding GSettings schema path in C library</link> – nothing special other than using <link xlink:href="https://github.com/NixOS/nixpkgs/pull/67957#issuecomment-527717467">Coccinelle patch</link> to generate the patch itself.
+        </para>
+       </listitem>
+      </itemizedlist>
+     </para>
+    </listitem>
+   </varlistentry>
+  </variablelist>
+ </section>
+</section>

doc/languages-frameworks/go.xml

@@ -218,12 +218,12 @@ deis = buildGoPackage rec {
    <varname>bin</varname> includes program binaries. You can test build a Go
    binary as follows:
 <screen>
-    $ nix-build -A deis.bin
+<prompt>$ </prompt>nix-build -A deis.bin
-  </screen>
+</screen>
    or build all outputs with:
 <screen>
-    $ nix-build -A deis.all
+<prompt>$ </prompt>nix-build -A deis.all
-  </screen>
+</screen>
    <varname>bin</varname> output will be installed by default with
    <varname>nix-env -i</varname> or <varname>systemPackages</varname>.
   </para>

doc/languages-frameworks/idris.section.md

@@ -11,10 +11,21 @@ $ # On non-NixOS
 $ nix-env -i nixpkgs.idris
 ```
 
-This however only provides the `prelude` and `base` libraries. To install additional libraries:
+This however only provides the `prelude` and `base` libraries. To install idris with additional libraries, you can use the `idrisPackages.with-packages` function, e.g. in an overlay in `~/.config/nixpkgs/overlays/my-idris.nix`:
+
+```nix
+self: super: {
+  myIdris = with self.idrisPackages; with-packages [ contrib pruviloj ];
+}
+```
+
+And then:
 
 ```
-$ nix-env -iE 'pkgs: pkgs.idrisPackages.with-packages (with pkgs.idrisPackages; [ contrib pruviloj ])'
+$ # On NixOS
+$ nix-env -iA nixos.myIdris
+$ # On non-NixOS
+$ nix-env -iA nixpkgs.myIdris
 ```
 
 To see all available Idris packages:
@@ -113,3 +124,21 @@ in another file (say `default.nix`) to be able to build it with
 ```
 $ nix-build -A yaml
 ```
+
+## Passing options to `idris` commands
+
+The `build-idris-package` function provides also optional input values to set additional options for the used `idris` commands.
+
+Specifically, you can set `idrisBuildOptions`, `idrisTestOptions`, `idrisInstallOptions` and `idrisDocOptions` to provide additional options to the `idris` command respectively when building, testing, installing and generating docs for your package.
+
+For example you could set
+
+```
+build-idris-package {
+  idrisBuildOptions = [ "--log" "1" "--verbose" ]
+
+  ...
+}
+```
+
+to require verbose output during `idris` build phase.

doc/languages-frameworks/index.xml

@@ -14,6 +14,7 @@
  <xi:include href="beam.xml" />
  <xi:include href="bower.xml" />
  <xi:include href="coq.xml" />
+ <xi:include href="gnome.xml" />
  <xi:include href="go.xml" />
  <xi:include href="haskell.section.xml" />
  <xi:include href="idris.section.xml" />
@@ -32,4 +33,5 @@
  <xi:include href="titanium.section.xml" />
  <xi:include href="vim.section.xml" />
  <xi:include href="emscripten.section.xml" />
+ <xi:include href="crystal.section.xml" />
 </chapter>

doc/languages-frameworks/java.xml

@@ -10,7 +10,7 @@ stdenv.mkDerivation {
   name = "...";
   src = fetchurl { ... };
 
-  buildInputs = [ jdk ant ];
+  nativeBuildInputs = [ jdk ant ];
 
   buildPhase = "ant";
 }
@@ -30,7 +30,8 @@ stdenv.mkDerivation {
   <filename>foo.jar</filename> in its <filename>share/java</filename>
   directory, and another package declares the attribute
 <programlisting>
-buildInputs = [ jdk libfoo ];
+buildInputs = [ libfoo ];
+nativeBuildInputs = [ jdk ];
 </programlisting>
   then <envar>CLASSPATH</envar> will be set to
   <filename>/nix/store/...-libfoo/share/java/foo.jar</filename>.
@@ -46,7 +47,7 @@ buildInputs = [ jdk libfoo ];
   script to run it using the OpenJRE. You can use
   <literal>makeWrapper</literal> for this:
 <programlisting>
-buildInputs = [ makeWrapper ];
+nativeBuildInputs = [ makeWrapper ];
 
 installPhase =
   ''
@@ -68,7 +69,7 @@ installPhase =
   can be done in a generic fashion with the <literal>--set</literal> argument
   of <literal>makeWrapper</literal>:
 <programlisting>
-  --set JAVA_HOME ${jdk.home}
+--set JAVA_HOME ${jdk.home}
 </programlisting>
  </para>
 
@@ -76,7 +77,7 @@ installPhase =
   It is possible to use a different Java compiler than <command>javac</command>
   from the OpenJDK. For instance, to use the GNU Java Compiler:
 <programlisting>
-buildInputs = [ gcj ant ];
+nativeBuildInputs = [ gcj ant ];
 </programlisting>
   Here, Ant will automatically use <command>gij</command> (the GNU Java
   Runtime) instead of the OpenJRE.

doc/languages-frameworks/lua.xml

@@ -29,7 +29,7 @@ fileSystem = buildLuaPackage {
     maintainers = with maintainers; [ flosse ];
   };
 };
-  </programlisting>
+</programlisting>
  </para>
 
  <para>

doc/languages-frameworks/ocaml.xml

@@ -62,7 +62,7 @@ buildDunePackage rec {
     maintainers = with stdenv.lib.maintainers; [ sternenseemann ];
   };
 }
- </programlisting>
+</programlisting>
 
  <para>
   Here is a second example, this time using a source archive generated with
@@ -93,5 +93,5 @@ buildDunePackage rec {
     maintainers = [ maintainers.eqyiel ];
   };
 }
- </programlisting>
+</programlisting>
 </section>

doc/languages-frameworks/perl.xml

@@ -47,13 +47,13 @@ foo = import ../path/to/foo.nix {
   in <filename>all-packages.nix</filename>. You can test building a Perl
   package as follows:
 <screen>
-$ nix-build -A perlPackages.ClassC3
+<prompt>$ </prompt>nix-build -A perlPackages.ClassC3
 </screen>
   <varname>buildPerlPackage</varname> adds <literal>perl-</literal> to the
   start of the name attribute, so the package above is actually called
   <literal>perl-Class-C3-0.21</literal>. So to install it, you can say:
 <screen>
-$ nix-env -i perl-Class-C3
+<prompt>$ </prompt>nix-env -i perl-Class-C3
 </screen>
   (Of course you can also install using the attribute name: <literal>nix-env -i
   -A perlPackages.ClassC3</literal>.)
@@ -75,7 +75,8 @@ $ nix-env -i perl-Class-C3
      It adds the contents of the <envar>PERL5LIB</envar> environment variable
      to <literal>#! .../bin/perl</literal> line of Perl scripts as
      <literal>-I<replaceable>dir</replaceable></literal> flags. This ensures
-     that a script can find its dependencies.
+     that a script can find its dependencies. (This can cause this shebang line
+     to become too long for Darwin to handle; see the note below.)
     </para>
    </listitem>
    <listitem>
@@ -137,6 +138,36 @@ ClassC3Componentised = buildPerlPackage rec {
 </programlisting>
  </para>
 
+ <para>
+  On Darwin, if a script has too many
+  <literal>-I<replaceable>dir</replaceable></literal> flags in its first line
+  (its “shebang line”), it will not run. This can be worked around by calling
+  the <literal>shortenPerlShebang</literal> function from the
+  <literal>postInstall</literal> phase:
+<programlisting>
+{ stdenv, buildPerlPackage, fetchurl, shortenPerlShebang }:
+
+ImageExifTool = buildPerlPackage {
+  pname = "Image-ExifTool";
+  version = "11.50";
+
+  src = fetchurl {
+    url = "https://www.sno.phy.queensu.ca/~phil/exiftool/Image-ExifTool-11.50.tar.gz";
+    sha256 = "0d8v48y94z8maxkmw1rv7v9m0jg2dc8xbp581njb6yhr7abwqdv3";
+  };
+
+  buildInputs = stdenv.lib.optional stdenv.isDarwin shortenPerlShebang;
+  postInstall = stdenv.lib.optional stdenv.isDarwin ''
+    shortenPerlShebang $out/bin/exiftool
+  '';
+};
+</programlisting>
+ This will remove the <literal>-I</literal> flags from the shebang line,
+ rewrite them in the <literal>use lib</literal> form, and put them on the next
+ line instead. This function can be given any number of Perl scripts as
+ arguments; it will modify them in-place.
+ </para>
+
  <section xml:id="ssec-generation-from-CPAN">
   <title>Generation from CPAN</title>
 
@@ -148,7 +179,7 @@ ClassC3Componentised = buildPerlPackage rec {
   </para>
 
 <screen>
-$ nix-env -i nix-generate-from-cpan
+<prompt>$ </prompt>nix-env -i nix-generate-from-cpan
 </screen>
 
   <para>
@@ -156,7 +187,7 @@ $ nix-env -i nix-generate-from-cpan
    unpacks the corresponding package, and prints a Nix expression on standard
    output. For example:
 <screen>
-$ nix-generate-from-cpan XML::Simple
+<prompt>$ </prompt>nix-generate-from-cpan XML::Simple
   XMLSimple = buildPerlPackage rec {
     name = "XML-Simple-2.22";
     src = fetchurl {

doc/languages-frameworks/python.section.md

@@ -540,7 +540,8 @@ and the aliases
 #### `buildPythonPackage` function
 
 The `buildPythonPackage` function is implemented in
-`pkgs/development/interpreters/python/build-python-package.nix`
+`pkgs/development/interpreters/python/mk-python-derivation`
+using setup hooks.
 
 The following is an example:
 ```nix
@@ -594,6 +595,7 @@ All parameters from `stdenv.mkDerivation` function are still supported. The foll
 * `catchConflicts ? true`: If `true`, abort package build if a package name appears more than once in dependency tree. Default is `true`.
 * `disabled` ? false: If `true`, package is not build for the particular Python interpreter version.
 * `dontWrapPythonPrograms ? false`: Skip wrapping of python programs.
+* `permitUserSite ? false`: Skip setting the `PYTHONNOUSERSITE` environment variable in wrapped programs.
 * `installFlags ? []`: A list of strings. Arguments to be passed to `pip install`. To pass options to `python setup.py install`, use `--install-option`. E.g., `installFlags=["--install-option='--cpp_implementation'"]`.
 * `format ? "setuptools"`: Format of the source. Valid options are `"setuptools"`, `"pyproject"`, `"flit"`, `"wheel"`, and `"other"`. `"setuptools"` is for when the source has a `setup.py` and `setuptools` is used to build a wheel, `flit`, in case `flit` should be used to build a wheel, and `wheel` in case a wheel is provided. Use `other` when a custom `buildPhase` and/or `installPhase` is needed.
 * `makeWrapperArgs ? []`: A list of strings. Arguments to be passed to `makeWrapper`, which wraps generated binaries. By default, the arguments to `makeWrapper` set `PATH` and `PYTHONPATH` environment variables before calling the binary. Additional arguments here can allow a developer to set environment variables which will be available when the binary is run. For example, `makeWrapperArgs = ["--set FOO BAR" "--set BAZ QUX"]`.
@@ -602,6 +604,7 @@ All parameters from `stdenv.mkDerivation` function are still supported. The foll
 * `preShellHook`: Hook to execute commands before `shellHook`.
 * `postShellHook`: Hook to execute commands after `shellHook`.
 * `removeBinByteCode ? true`: Remove bytecode from `/bin`. Bytecode is only created when the filenames end with `.py`.
+* `setupPyGlobalFlags ? []`: List of flags passed to `setup.py` command.
 * `setupPyBuildFlags ? []`: List of flags passed to `setup.py build_ext` command.
 
 The `stdenv.mkDerivation` function accepts various parameters for describing build inputs (see "Specifying dependencies"). The following are of special
@@ -635,7 +638,7 @@ with import <nixpkgs> {};
         };
       });
     };
-  in pkgs.python3.override {inherit packageOverrides;};
+  in pkgs.python3.override {inherit packageOverrides; self = python;};
 
 in python.withPackages(ps: [ps.blaze])).env
 ```
@@ -756,6 +759,7 @@ specified packages in its path.
 * `extraLibs`: List of packages installed inside the environment.
 * `postBuild`: Shell command executed after the build of environment.
 * `ignoreCollisions`: Ignore file collisions inside the environment (default is `false`).
+* `permitUserSite`: Skip setting the `PYTHONNOUSERSITE` environment variable in wrapped binaries in the environment.
 
 #### `python.withPackages` function
 
@@ -794,6 +798,22 @@ such as `ignoreCollisions = true` or `postBuild`. If you need them, you have to
 Python 2 namespace packages may provide `__init__.py` that collide. In that case `python.buildEnv`
 should be used with `ignoreCollisions = true`.
 
+#### Setup hooks
+
+The following are setup hooks specifically for Python packages. Most of these are
+used in `buildPythonPackage`.
+
+- `flitBuildHook` to build a wheel using `flit`.
+- `pipBuildHook` to build a wheel using `pip` and PEP 517. Note a build system (e.g. `setuptools` or `flit`) should still be added as `nativeBuildInput`.
+- `pipInstallHook` to install wheels.
+- `pytestCheckHook` to run tests with `pytest`.
+- `pythonCatchConflictsHook` to check whether a Python package is not already existing.
+- `pythonImportsCheckHook` to check whether importing the listed modules works.
+- `pythonRemoveBinBytecode` to remove bytecode from the `/bin` folder.
+- `setuptoolsBuildHook` to build a wheel using `setuptools`.
+- `setuptoolsCheckHook` to run tests with `python setup.py test`.
+- `wheelUnpackHook` to move a wheel to the correct folder so it can be installed with the `pipInstallHook`.
+
 ### Development mode
 
 Development or editable mode is supported. To develop Python packages

doc/languages-frameworks/qt.xml

@@ -4,71 +4,182 @@
  <title>Qt</title>
 
  <para>
-  Qt is a comprehensive desktop and mobile application development toolkit for
+   This section describes the differences between Nix expressions for Qt
-  C++. Legacy support is available for Qt 3 and Qt 4, but all current
+   libraries and applications and Nix expressions for other C++ software. Some
-  development uses Qt 5. The Qt 5 packages in Nixpkgs are updated frequently to
+   knowledge of the latter is assumed. There are primarily two problems which
-  take advantage of new features, but older versions are typically retained
+   the Qt infrastructure is designed to address: ensuring consistent versioning
-  until their support window ends. The most important consideration in
+   of all dependencies and finding dependencies at runtime.
-  packaging Qt-based software is ensuring that each package and all its
-  dependencies use the same version of Qt 5; this consideration motivates most
-  of the tools described below.
  </para>
 
- <section xml:id="ssec-qt-libraries">
+ <example xml:id='qt-default-nix'>
-  <title>Packaging Libraries for Nixpkgs</title>
+   <title>Nix expression for a Qt package (<filename>default.nix</filename>)</title>
+   <programlisting>
+{ mkDerivation, lib, qtbase }: <co xml:id='qt-default-nix-co-1' />
 
+mkDerivation { <co xml:id='qt-default-nix-co-2' />
+  pname = "myapp";
+  version = "1.0";
+
+  buildInputs = [ qtbase ]; <co xml:id='qt-default-nix-co-3' />
+}
+   </programlisting>
+ </example>
+
+ <calloutlist>
+   <callout arearefs='qt-default-nix-co-1'>
+     <para>
+       Import <literal>mkDerivation</literal> and Qt (such as
+       <literal>qtbase</literal> modules directly. <emphasis>Do not</emphasis>
+       import Qt package sets; the Qt versions of dependencies may not be
+       coherent, causing build and runtime failures.
+     </para>
+   </callout>
+   <callout arearefs='qt-default-nix-co-2'>
+     <para>
+       Use <literal>mkDerivation</literal> instead of
+       <literal>stdenv.mkDerivation</literal>. <literal>mkDerivation</literal>
+       is a wrapper around <literal>stdenv.mkDerivation</literal> which
+       applies some Qt-specific settings.
+       This deriver accepts the same arguments as
+       <literal>stdenv.mkDerivation</literal>; refer to
+       <xref linkend='chap-stdenv' /> for details.
+     </para>
+     <para>
+       To use another deriver instead of
+       <literal>stdenv.mkDerivation</literal>, use
+       <literal>mkDerivationWith</literal>:
+<programlisting>
+mkDerivationWith myDeriver {
+  # ...
+}
+</programlisting>
+       If you cannot use <literal>mkDerivationWith</literal>, please refer to
+       <xref linkend='qt-runtime-dependencies' />.
+     </para>
+   </callout>
+   <callout arearefs='qt-default-nix-co-3'>
+     <para>
+       <literal>mkDerivation</literal> accepts the same arguments as
+       <literal>stdenv.mkDerivation</literal>, such as
+       <literal>buildInputs</literal>.
+     </para>
+   </callout>
+ </calloutlist>
+
+ <formalpara xml:id='qt-runtime-dependencies'>
+   <title>Locating runtime dependencies</title>
+   <para>
+     Qt applications need to be wrapped to find runtime dependencies.  If you
+     cannot use <literal>mkDerivation</literal> or
+     <literal>mkDerivationWith</literal> above, include
+     <literal>wrapQtAppsHook</literal> in <literal>nativeBuildInputs</literal>:
+<programlisting>
+stdenv.mkDerivation {
+  # ...
+
+  nativeBuildInputs = [ wrapQtAppsHook ];
+}
+</programlisting>
+   </para>
+ </formalpara>
+
+ <para>
+   Entries added to <literal>qtWrapperArgs</literal> are used to modify the
+   wrappers created by <literal>wrapQtAppsHook</literal>. The entries are
+   passed as arguments to <xref linkend='fun-wrapProgram' />.
+<programlisting>
+mkDerivation {
+  # ...
+
+  qtWrapperArgs = [ ''--prefix PATH : /path/to/bin'' ];
+}
+</programlisting>
+ </para>
+
+ <para>
+  Set <literal>dontWrapQtApps</literal> to stop applications from being
+  wrapped automatically. It is required to wrap applications manually with
+  <literal>wrapQtApp</literal>, using the syntax of
+  <xref linkend='fun-wrapProgram' />:
+<programlisting>
+mkDerivation {
+  # ...
+
+  dontWrapQtApps = true;
+  preFixup = ''
+      wrapQtApp "$out/bin/myapp" --prefix PATH : /path/to/bin
+  '';
+}
+</programlisting>
+ </para>
+
+ <note>
   <para>
-   Whenever possible, libraries that use Qt 5 should be built with each
+    <literal>wrapQtAppsHook</literal> ignores files that are non-ELF executables.
-   available version. Packages providing libraries should be added to the
+    This means that scripts won't be automatically wrapped so you'll need to manually
-   top-level function <varname>mkLibsForQt5</varname>, which is used to build a
+    wrap them as previously mentioned. An example of when you'd always need to do this
-   set of libraries for every Qt 5 version. A special
+    is with Python applications that use PyQT.
-   <varname>callPackage</varname> function is used in this scope to ensure that
-   the entire dependency tree uses the same Qt 5 version. Import dependencies
-   unqualified, i.e., <literal>qtbase</literal> not
-   <literal>qt5.qtbase</literal>. <emphasis>Do not</emphasis> import a package
-   set such as <literal>qt5</literal> or <literal>libsForQt5</literal>.
   </para>
+ </note>
 
-  <para>
+ <para>
-   If a library does not support a particular version of Qt 5, it is best to
+  Libraries are built with every available version of Qt. Use the <literal>meta.broken</literal>
-   mark it as broken by setting its <literal>meta.broken</literal> attribute. A
+  attribute to disable the package for unsupported Qt versions:
-   package may be marked broken for certain versions by testing the
+<programlisting>
-   <literal>qtbase.version</literal> attribute, which will always give the
+mkDerivation {
-   current Qt 5 version.
+  # ...
-  </para>
- </section>
 
- <section xml:id="ssec-qt-applications">
+  # Disable this library with Qt &lt; 5.9.0
-  <title>Packaging Applications for Nixpkgs</title>
+  meta.broken = builtins.compareVersions qtbase.version "5.9.0" &lt; 0;
+}
+</programlisting>
+ </para>
 
-  <para>
+ <formalpara>
-   Call your application expression using
+   <title>Adding a library to Nixpkgs</title>
-   <literal>libsForQt5.callPackage</literal> instead of
+   <para>
-   <literal>callPackage</literal>. Import dependencies unqualified, i.e.,
+     Add a Qt library to <filename>all-packages.nix</filename> by adding it to the
-   <literal>qtbase</literal> not <literal>qt5.qtbase</literal>. <emphasis>Do
+     collection inside <literal>mkLibsForQt5</literal>. This ensures that the
-   not</emphasis> import a package set such as <literal>qt5</literal> or
+     library is built with every available version of Qt as needed.
-   <literal>libsForQt5</literal>.
+     <example xml:id='qt-library-all-packages-nix'>
-  </para>
+       <title>Adding a Qt library to <filename>all-packages.nix</filename></title>
+<programlisting>
+{
+  # ...
 
-  <para>
+  mkLibsForQt5 = self: with self; {
-   Qt 5 maintains strict backward compatibility, so it is generally best to
+    # ...
-   build an application package against the latest version using the
+
-   <varname>libsForQt5</varname> library set. In case a package does not build
+    mylib = callPackage ../path/to/mylib {};
-   with the latest Qt version, it is possible to pick a set pinned to a
+  };
-   particular version, e.g. <varname>libsForQt55</varname> for Qt 5.5, if that
+
-   is the latest version the package supports. If a package must be pinned to
+  # ...
-   an older Qt version, be sure to file a bug upstream; because Qt is strictly
+}
-   backwards-compatible, any incompatibility is by definition a bug in the
+</programlisting>
-   application.
+     </example>
-  </para>
+   </para>
+ </formalpara>
+
+ <formalpara>
+   <title>Adding an application to Nixpkgs</title>
+   <para>
+     Add a Qt application to <filename>all-packages.nix</filename> using
+     <literal>libsForQt5.callPackage</literal> instead of the usual
+     <literal>callPackage</literal>. The former ensures that all dependencies
+     are built with the same version of Qt.
+     <example xml:id='qt-application-all-packages-nix'>
+       <title>Adding a Qt application to <filename>all-packages.nix</filename></title>
+<programlisting>
+{
+  # ...
+
+  myapp = libsForQt5.callPackage ../path/to/myapp/ {};
+
+  # ...
+}
+</programlisting>
+     </example>
+   </para>
+ </formalpara>
 
-  <para>
-   When testing applications in Nixpkgs, it is a common practice to build the
-   package with <literal>nix-build</literal> and run it using the created
-   symbolic link. This will not work with Qt applications, however, because
-   they have many hard runtime requirements that can only be guaranteed if the
-   package is actually installed. To test a Qt application, install it with
-   <literal>nix-env</literal> or run it inside <literal>nix-shell</literal>.
-  </para>
- </section>
 </section>

doc/languages-frameworks/ruby.section.md

@@ -0,0 +1,365 @@
+---
+title: Ruby
+author: Michael Fellinger
+date: 2019-05-23
+---
+
+# Ruby
+
+## User Guide
+
+### Using Ruby
+
+#### Overview
+
+Several versions of Ruby interpreters are available on Nix, as well as over 250 gems and many applications written in Ruby.
+The attribute `ruby` refers to the default Ruby interpreter, which is currently
+MRI 2.5. It's also possible to refer to specific versions, e.g. `ruby_2_6`, `jruby`, or `mruby`.
+
+In the nixpkgs tree, Ruby packages can be found throughout, depending on what
+they do, and are called from the main package set. Ruby gems, however are
+separate sets, and there's one default set for each interpreter (currently MRI
+only).
+
+There are two main approaches for using Ruby with gems.
+One is to use a specifically locked `Gemfile` for an application that has very strict dependencies.
+The other is to depend on the common gems, which we'll explain further down, and
+rely on them being updated regularly.
+
+The interpreters have common attributes, namely `gems`, and `withPackages`. So
+you can refer to `ruby.gems.nokogiri`, or `ruby_2_5.gems.nokogiri` to get the
+Nokogiri gem already compiled and ready to use.
+
+Since not all gems have executables like `nokogiri`, it's usually more
+convenient to use the `withPackages` function like this:
+`ruby.withPackages (p: with p; [ nokogiri ])`. This will also make sure that the
+Ruby in your environment will be able to find the gem and it can be used in your
+Ruby code (for example via `ruby` or `irb` executables) via `require "nokogiri"`
+as usual.
+
+#### Temporary Ruby environment with `nix-shell`
+
+Rather than having a single Ruby environment shared by all Ruby
+development projects on a system, Nix allows you to create separate
+environments per project.  `nix-shell` gives you the possibility to
+temporarily load another environment akin to a combined `chruby` or
+`rvm` and `bundle exec`.
+
+There are two methods for loading a shell with Ruby packages. The first and
+recommended method is to create an environment with `ruby.withPackages` and load
+that.
+
+```shell
+nix-shell -p "ruby.withPackages (ps: with ps; [ nokogiri pry ])"
+```
+
+The other method, which is not recommended, is to create an environment and list
+all the packages directly.
+
+```shell
+nix-shell -p ruby.gems.nokogiri ruby.gems.pry
+```
+
+Again, it's possible to launch the interpreter from the shell. The Ruby
+interpreter has the attribute `gems` which contains all Ruby gems for that
+specific interpreter.
+
+##### Load environment from `.nix` expression
+
+As explained in the Nix manual, `nix-shell` can also load an expression from a
+`.nix` file. Say we want to have Ruby 2.5, `nokogori`, and `pry`. Consider a
+`shell.nix` file with:
+
+```nix
+with import <nixpkgs> {};
+ruby.withPackages (ps: with ps; [ nokogiri pry ])
+```
+
+What's happening here?
+
+1. We begin with importing the Nix Packages collections. `import <nixpkgs>`
+   imports the `<nixpkgs>` function, `{}` calls it and the `with` statement
+   brings all attributes of `nixpkgs` in the local scope. These attributes form
+   the main package set.
+2. Then we create a Ruby environment with the `withPackages` function.
+3. The `withPackages` function expects us to provide a function as an argument
+   that takes the set of all ruby gems and returns a list of packages to include
+   in the environment. Here, we select the packages `nokogiri` and `pry` from
+   the package set.
+
+##### Execute command with `--run`
+
+A convenient flag for `nix-shell` is `--run`. It executes a command in the
+`nix-shell`. We can e.g. directly open a `pry` REPL:
+
+```shell
+nix-shell -p "ruby.withPackages (ps: with ps; [ nokogiri pry ])" --run "pry"
+```
+
+Or immediately require `nokogiri` in pry:
+
+```shell
+nix-shell -p "ruby.withPackages (ps: with ps; [ nokogiri pry ])" --run "pry -rnokogiri"
+```
+
+Or run a script using this environment:
+
+```shell
+nix-shell -p "ruby.withPackages (ps: with ps; [ nokogiri pry ])" --run "ruby example.rb"
+```
+
+##### Using `nix-shell` as shebang
+
+In fact, for the last case, there is a more convenient method. You can add a
+[shebang](https://en.wikipedia.org/wiki/Shebang_(Unix)) to your script
+specifying which dependencies `nix-shell` needs. With the following shebang, you
+can just execute `./example.rb`, and it will run with all dependencies.
+
+```ruby
+#! /usr/bin/env nix-shell
+#! nix-shell -i ruby -p "ruby.withPackages (ps: with ps; [ nokogiri rest-client ])"
+
+require 'nokogiri'
+require 'rest-client'
+
+body = RestClient.get('http://example.com').body
+puts Nokogiri::HTML(body).at('h1').text
+```
+
+### Developing with Ruby
+
+#### Using an existing Gemfile
+
+In most cases, you'll already have a `Gemfile.lock` listing all your dependencies.
+This can be used to generate a `gemset.nix` which is used to fetch the gems and
+combine them into a single environment.
+The reason why you need to have a separate file for this, is that Nix requires
+you to have a checksum for each input to your build.
+Since the `Gemfile.lock` that `bundler` generates doesn't provide us with
+checksums, we have to first download each gem, calculate its SHA256, and store
+it in this separate file.
+
+So the steps from having just a `Gemfile` to a `gemset.nix` are:
+
+```shell
+bundle lock
+bundix
+```
+
+If you already have a `Gemfile.lock`, you can simply run `bundix` and it will
+work the same.
+
+To update the gems in your `Gemfile.lock`, you may use the `bundix -l` flag,
+which will create a new `Gemfile.lock` in case the `Gemfile` has a more recent
+time of modification.
+
+Once the `gemset.nix` is generated, it can be used in a
+`bundlerEnv` derivation. Here is an example you could use for your `shell.nix`:
+
+```nix
+# ...
+let
+  gems = bundlerEnv {
+    name = "gems-for-some-project";
+    gemdir = ./.;
+  };
+in mkShell { buildInputs = [ gems gems.wrappedRuby ]; }
+```
+
+With this file in your directory, you can run `nix-shell` to build and use the gems.
+The important parts here are `bundlerEnv` and `wrappedRuby`.
+
+The `bundlerEnv` is a wrapper over all the gems in your gemset. This means that
+all the `/lib` and `/bin` directories will be available, and the executables of
+all gems (even of indirect dependencies) will end up in your `$PATH`.
+The `wrappedRuby` provides you with all executables that come with Ruby itself,
+but wrapped so they can easily find the gems in your gemset.
+
+One common issue that you might have is that you have Ruby 2.6, but also
+`bundler` in your gemset. That leads to a conflict for `/bin/bundle` and
+`/bin/bundler`. You can resolve this by wrapping either your Ruby or your gems
+in a `lowPrio` call. So in order to give the `bundler` from your gemset
+priority, it would be used like this:
+
+```nix
+# ...
+mkShell { buildInputs = [ gems (lowPrio gems.wrappedRuby) ]; }
+```
+
+
+#### Gem-specific configurations and workarounds
+
+In some cases, especially if the gem has native extensions, you might need to
+modify the way the gem is built.
+
+This is done via a common configuration file that includes all of the
+workarounds for each gem.
+
+This file lives at `/pkgs/development/ruby-modules/gem-config/default.nix`,
+since it already contains a lot of entries, it should be pretty easy to add the
+modifications you need for your needs.
+
+In the meanwhile, or if the modification is for a private gem, you can also add
+the configuration to only your own environment.
+
+Two places that allow this modification are the `ruby` derivation, or `bundlerEnv`.
+
+Here's the `ruby` one:
+
+```nix
+{ pg_version ? "10", pkgs ? import <nixpkgs> { } }:
+let
+  myRuby = pkgs.ruby.override {
+    defaultGemConfig = pkgs.defaultGemConfig // {
+      pg = attrs: {
+        buildFlags =
+        [ "--with-pg-config=${pkgs."postgresql_${pg_version}"}/bin/pg_config" ];
+      };
+    };
+  };
+in myRuby.withPackages (ps: with ps; [ pg ])
+```
+
+And an example with `bundlerEnv`:
+
+```nix
+{ pg_version ? "10", pkgs ? import <nixpkgs> { } }:
+let
+  gems = pkgs.bundlerEnv {
+    name = "gems-for-some-project";
+    gemdir = ./.;
+    gemConfig = pkgs.defaultGemConfig // {
+      pg = attrs: {
+        buildFlags =
+        [ "--with-pg-config=${pkgs."postgresql_${pg_version}"}/bin/pg_config" ];
+      };
+    };
+  };
+in mkShell { buildInputs = [ gems gems.wrappedRuby ]; }
+```
+
+And finally via overlays:
+
+```nix
+{ pg_version ? "10" }:
+let
+  pkgs = import <nixpkgs> {
+    overlays = [
+      (self: super: {
+        defaultGemConfig = super.defaultGemConfig // {
+          pg = attrs: {
+            buildFlags = [
+              "--with-pg-config=${
+                pkgs."postgresql_${pg_version}"
+              }/bin/pg_config"
+            ];
+          };
+        };
+      })
+    ];
+  };
+in pkgs.ruby.withPackages (ps: with ps; [ pg ])
+```
+
+Then we can get whichever postgresql version we desire and the `pg` gem will
+always reference it correctly:
+
+```shell
+$ nix-shell --argstr pg_version 9_4 --run 'ruby -rpg -e "puts PG.library_version"'
+90421
+
+$ nix-shell --run 'ruby -rpg -e "puts PG.library_version"'
+100007
+```
+
+Of course for this use-case one could also use overlays since the configuration
+for `pg` depends on the `postgresql` alias, but for demonstration purposes this
+has to suffice.
+
+#### Adding a gem to the default gemset
+
+Now that you know how to get a working Ruby environment with Nix, it's time to
+go forward and start actually developing with Ruby.
+We will first have a look at how Ruby gems are packaged on Nix. Then, we will
+look at how you can use development mode with your code.
+
+All gems in the standard set are automatically generated from a single
+`Gemfile`. The dependency resolution is done with `bundler` and makes it more
+likely that all gems are compatible to each other.
+
+In order to add a new gem to nixpkgs, you can put it into the
+`/pkgs/development/ruby-modules/with-packages/Gemfile` and run
+`./maintainers/scripts/update-ruby-packages`.
+
+To test that it works, you can then try using the gem with:
+
+```shell
+NIX_PATH=nixpkgs=$PWD nix-shell -p "ruby.withPackages (ps: with ps; [ name-of-your-gem ])"
+```
+
+#### Packaging applications
+
+A common task is to add a ruby executable to nixpkgs, popular examples would be
+`chef`, `jekyll`, or `sass`. A good way to do that is to use the `bundlerApp`
+function, that allows you to make a package that only exposes the listed
+executables, otherwise the package may cause conflicts through common paths like
+`bin/rake` or `bin/bundler` that aren't meant to be used.
+
+The absolute easiest way to do that is to write a
+`Gemfile` along these lines:
+
+```ruby
+source 'https://rubygems.org' do
+  gem 'mdl'
+end
+```
+
+If you want to package a specific version, you can use the standard Gemfile
+syntax for that, e.g. `gem 'mdl', '0.5.0'`, but if you want the latest stable
+version anyway, it's easier to update by simply running the `bundle lock` and
+`bundix` steps again.
+
+Now you can also also make a `default.nix` that looks like this:
+
+```nix
+{ lib, bundlerApp }:
+
+bundlerApp {
+  pname = "mdl";
+  gemdir = ./.;
+  exes = [ "mdl" ];
+}
+```
+
+All that's left to do is to generate the corresponding `Gemfile.lock` and
+`gemset.nix` as described above in the `Using an existing Gemfile` section.
+
+##### Packaging executables that require wrapping
+
+Sometimes your app will depend on other executables at runtime, and tries to
+find it through the `PATH` environment variable.
+
+In this case, you can provide a `postBuild` hook to `bundlerApp` that wraps the
+gem in another script that prefixes the `PATH`.
+
+Of course you could also make a custom `gemConfig` if you know exactly how to
+patch it, but it's usually much easier to maintain with a simple wrapper so the
+patch doesn't have to be adjusted for each version.
+
+Here's another example:
+
+```nix
+{ lib, bundlerApp, makeWrapper, git, gnutar, gzip }:
+
+bundlerApp {
+  pname = "r10k";
+  gemdir = ./.;
+  exes = [ "r10k" ];
+
+  buildInputs = [ makeWrapper ];
+
+  postBuild = ''
+    wrapProgram $out/bin/r10k --prefix PATH : ${lib.makeBinPath [ git gnutar gzip ]}
+  '';
+}
+```

doc/languages-frameworks/rust.section.md

@@ -336,9 +336,9 @@ with import <nixpkgs> {};
 let src = fetchFromGitHub {
       owner = "mozilla";
       repo = "nixpkgs-mozilla";
-      # commit from: 2018-03-27
+      # commit from: 2019-05-15
-      rev = "2945b0b6b2fd19e7d23bac695afd65e320efcebe";
+      rev = "9f35c4b09fd44a77227e79ff0c1b4b6a69dff533";
-      sha256 = "034m1dryrzh2lmjvk3c0krgip652dql46w5yfwpvh7gavd3iypyw";
+      sha256 = "18h0nvh55b5an4gmlgfbvwbyqj91bklf1zymis6lbdh75571qaz0";
    };
 in
 with import "${src.out}/rust-overlay.nix" pkgs pkgs;

doc/languages-frameworks/texlive.xml

@@ -26,7 +26,7 @@
 texlive.combine {
   inherit (texlive) scheme-small collection-langkorean algorithms cm-super;
 }
-      </programlisting>
+</programlisting>
      There are all the schemes, collections and a few thousand packages, as
      defined upstream (perhaps with tiny differences).
     </para>
@@ -44,7 +44,7 @@ texlive.combine {
   # elem tlType [ "run" "bin" "doc" "source" ]
   # there are also other attributes: version, name
 }
-      </programlisting>
+</programlisting>
     </para>
    </listitem>
    <listitem>

doc/languages-frameworks/vim.section.md

@@ -21,7 +21,7 @@ At the moment we support three different methods for managing plugins:
 
 Adding custom .vimrc lines can be done using the following code:
 
-```
+```nix
 vim_configurable.customize {
   # `name` specifies the name of the executable and package
   name = "vim-with-plugins";
@@ -32,11 +32,11 @@ vim_configurable.customize {
 }
 ```
 
-This configuration is used when vim is invoked with the command specified as name, in this case `vim-with-plugins`.
+This configuration is used when Vim is invoked with the command specified as name, in this case `vim-with-plugins`.
 
 For Neovim the `configure` argument can be overridden to achieve the same:
 
-```
+```nix
 neovim.override {
   configure = {
     customRC = ''
@@ -46,10 +46,10 @@ neovim.override {
 }
 ```
 
-If you want to use `neovim-qt` as a graphical editor, you can configure it by overriding neovim in an overlay
+If you want to use `neovim-qt` as a graphical editor, you can configure it by overriding Neovim in an overlay
-or passing it an overridden neovimn:
+or passing it an overridden Neovimn:
 
-```
+```nix
 neovim-qt.override {
   neovim = neovim.override {
     configure = {
@@ -63,16 +63,16 @@ neovim-qt.override {
 
 ## Managing plugins with Vim packages
 
-To store you plugins in Vim packages (the native vim plugin manager, see `:help packages`) the following example can be used:
+To store you plugins in Vim packages (the native Vim plugin manager, see `:help packages`) the following example can be used:
 
-```
+```nix
 vim_configurable.customize {
   vimrcConfig.packages.myVimPackage = with pkgs.vimPlugins; {
     # loaded on launch
     start = [ youcompleteme fugitive ];
     # manually loadable by calling `:packadd $plugin-name`
-    # however, if a vim plugin has a dependency that is not explicitly listed in
+    # however, if a Vim plugin has a dependency that is not explicitly listed in
-	# opt that dependency will always be added to start to avoid confusion.
+    # opt that dependency will always be added to start to avoid confusion.
     opt = [ phpCompletion elm-vim ];
     # To automatically load a plugin when opening a filetype, add vimrc lines like:
     # autocmd FileType php :packadd phpCompletion
@@ -83,7 +83,7 @@ vim_configurable.customize {
 `myVimPackage` is an arbitrary name for the generated package. You can choose any name you like.
 For Neovim the syntax is:
 
-```
+```nix
 neovim.override {
   configure = {
     customRC = ''
@@ -92,7 +92,7 @@ neovim.override {
     packages.myVimPackage = with pkgs.vimPlugins; {
       # see examples below how to use custom packages
       start = [ ];
-      # If a vim plugin has a dependency that is not explicitly listed in
+      # If a Vim plugin has a dependency that is not explicitly listed in
       # opt that dependency will always be added to start to avoid confusion.
       opt = [ ];
     };
@@ -102,7 +102,7 @@ neovim.override {
 
 The resulting package can be added to `packageOverrides` in `~/.nixpkgs/config.nix` to make it installable:
 
-```
+```nix
 {
   packageOverrides = pkgs: with pkgs; {
     myVim = vim_configurable.customize {
@@ -126,7 +126,7 @@ After that you can install your special grafted `myVim` or `myNeovim` packages.
 To use [vim-plug](https://github.com/junegunn/vim-plug) to manage your Vim
 plugins the following example can be used:
 
-```
+```nix
 vim_configurable.customize {
   vimrcConfig.packages.myVimPackage = with pkgs.vimPlugins; {
     # loaded on launch
@@ -137,7 +137,7 @@ vim_configurable.customize {
 
 For Neovim the syntax is:
 
-```
+```nix
 neovim.override {
   configure = {
     customRC = ''
@@ -161,89 +161,112 @@ assuming that "using latest version" is ok most of the time.
 
 First create a vim-scripts file having one plugin name per line. Example:
 
-    "tlib"
+```
-    {'name': 'vim-addon-sql'}
+"tlib"
-    {'filetype_regex': '\%(vim)$', 'names': ['reload', 'vim-dev-plugin']}
+{'name': 'vim-addon-sql'}
+{'filetype_regex': '\%(vim)$', 'names': ['reload', 'vim-dev-plugin']}
+```
 
 Such vim-scripts file can be read by VAM as well like this:
 
-    call vam#Scripts(expand('~/.vim-scripts'), {})
+```vim
+call vam#Scripts(expand('~/.vim-scripts'), {})
+```
 
 Create a default.nix file:
 
-    { nixpkgs ? import <nixpkgs> {}, compiler ? "ghc7102" }:
+```nix
-    nixpkgs.vim_configurable.customize { name = "vim"; vimrcConfig.vam.pluginDictionaries = [ "vim-addon-vim2nix" ]; }
+{ nixpkgs ? import <nixpkgs> {}, compiler ? "ghc7102" }:
+nixpkgs.vim_configurable.customize { name = "vim"; vimrcConfig.vam.pluginDictionaries = [ "vim-addon-vim2nix" ]; }
+```
 
 Create a generate.vim file:
 
-    ActivateAddons vim-addon-vim2nix
+```vim
-    let vim_scripts = "vim-scripts"
+ActivateAddons vim-addon-vim2nix
-    call nix#ExportPluginsForNix({
+let vim_scripts = "vim-scripts"
-    \  'path_to_nixpkgs': eval('{"'.substitute(substitute(substitute($NIX_PATH, ':', ',', 'g'), '=',':', 'g'), '\([:,]\)', '"\1"',"g").'"}')["nixpkgs"],
+call nix#ExportPluginsForNix({
-    \  'cache_file': '/tmp/vim2nix-cache',
+\  'path_to_nixpkgs': eval('{"'.substitute(substitute(substitute($NIX_PATH, ':', ',', 'g'), '=',':', 'g'), '\([:,]\)', '"\1"',"g").'"}')["nixpkgs"],
-    \  'try_catch': 0,
+\  'cache_file': '/tmp/vim2nix-cache',
-    \  'plugin_dictionaries': ["vim-addon-manager"]+map(readfile(vim_scripts), 'eval(v:val)')
+\  'try_catch': 0,
-    \ })
+\  'plugin_dictionaries': ["vim-addon-manager"]+map(readfile(vim_scripts), 'eval(v:val)')
+\ })
+```
 
 Then run
 
-    nix-shell -p vimUtils.vim_with_vim2nix --command "vim -c 'source generate.vim'"
+```bash
+nix-shell -p vimUtils.vim_with_vim2nix --command "vim -c 'source generate.vim'"
+```
 
 You should get a Vim buffer with the nix derivations (output1) and vam.pluginDictionaries (output2).
-You can add your vim to your system's configuration file like this and start it by "vim-my":
+You can add your Vim to your system's configuration file like this and start it by "vim-my":
 
-    my-vim =
+```
-     let plugins = let inherit (vimUtils) buildVimPluginFrom2Nix; in {
+my-vim =
-          copy paste output1 here
+  let plugins = let inherit (vimUtils) buildVimPluginFrom2Nix; in {
-     }; in vim_configurable.customize {
+    copy paste output1 here
-       name = "vim-my";
+  }; in vim_configurable.customize {
+    name = "vim-my";
 
-       vimrcConfig.vam.knownPlugins = plugins; # optional
+    vimrcConfig.vam.knownPlugins = plugins; # optional
-       vimrcConfig.vam.pluginDictionaries = [
+    vimrcConfig.vam.pluginDictionaries = [
-          copy paste output2 here
+       copy paste output2 here
-       ];
+    ];
-
-       # Pathogen would be
-       # vimrcConfig.pathogen.knownPlugins = plugins; # plugins
-       # vimrcConfig.pathogen.pluginNames = ["tlib"];
-     };
 
+    # Pathogen would be
+    # vimrcConfig.pathogen.knownPlugins = plugins; # plugins
+    # vimrcConfig.pathogen.pluginNames = ["tlib"];
+  };
+```
 
 Sample output1:
 
-    "reload" = buildVimPluginFrom2Nix { # created by nix#NixDerivation
+```
-      name = "reload";
+"reload" = buildVimPluginFrom2Nix { # created by nix#NixDerivation
-      src = fetchgit {
+  name = "reload";
-        url = "git://github.com/xolox/vim-reload";
+  src = fetchgit {
-        rev = "0a601a668727f5b675cb1ddc19f6861f3f7ab9e1";
+    url = "git://github.com/xolox/vim-reload";
-        sha256 = "0vb832l9yxj919f5hfg6qj6bn9ni57gnjd3bj7zpq7d4iv2s4wdh";
+    rev = "0a601a668727f5b675cb1ddc19f6861f3f7ab9e1";
-      };
+    sha256 = "0vb832l9yxj919f5hfg6qj6bn9ni57gnjd3bj7zpq7d4iv2s4wdh";
-      dependencies = ["nim-misc"];
+  };
+  dependencies = ["nim-misc"];
 
-    };
+};
-    [...]
+[...]
+```
 
 Sample output2:
 
-    [
+```nix
-      ''vim-addon-manager''
+[
-      ''tlib''
+  ''vim-addon-manager''
-      { "name" = ''vim-addon-sql''; }
+  ''tlib''
-      { "filetype_regex" = ''\%(vim)$$''; "names" = [ ''reload'' ''vim-dev-plugin'' ]; }
+  { "name" = ''vim-addon-sql''; }
-    ]
+  { "filetype_regex" = ''\%(vim)$$''; "names" = [ ''reload'' ''vim-dev-plugin'' ]; }
-
+]
+```
 
 ## Adding new plugins to nixpkgs
 
-In `pkgs/misc/vim-plugins/vim-plugin-names` we store the plugin names
+Nix expressions for Vim plugins are stored in [pkgs/misc/vim-plugins](/pkgs/misc/vim-plugins). For the vast majority of plugins, Nix expressions are automatically generated by running [`./update.py`](/pkgs/misc/vim-plugins/update.py). This creates a [generated.nix](/pkgs/misc/vim-plugins/generated.nix) file based on the plugins listed in [vim-plugin-names](/pkgs/misc/vim-plugins/vim-plugin-names). Plugins are listed in alphabetical order in `vim-plugin-names` using the format `[github username]/[repository]`. For example https://github.com/scrooloose/nerdtree becomes `scrooloose/nerdtree`.
-for all vim plugins we automatically generate plugins for.
+
-The format of this file `github username/github repository`:
+Some plugins require overrides in order to function properly. Overrides are placed in [overrides.nix](/pkgs/misc/vim-plugins/overrides.nix). Overrides are most often required when a plugin requires some dependencies, or extra steps are required during the build process. For example `deoplete-fish` requires both `deoplete-nvim` and `vim-fish`, and so the following override was added:
-For example https://github.com/scrooloose/nerdtree becomes `scrooloose/nerdtree`.
+
-After adding your plugin to this file run the `./update.py` in the same folder.
+```
-This will updated a file called `generated.nix` and make your plugin accessible in the
+deoplete-fish = super.deoplete-fish.overrideAttrs(old: {
-`vimPlugins` attribute set (`vimPlugins.nerdtree` in our example).
+  dependencies = with super; [ deoplete-nvim vim-fish ];
-If additional steps to the build process of the plugin are required, add an
+});
-override to the `pkgs/misc/vim-plugins/default.nix` in the same directory.
+```
+
+Sometimes plugins require an override that must be changed when the plugin is updated. This can cause issues when Vim plugins are auto-updated but the associated override isn't updated. For these plugins, the override should be written so that it specifies all information required to install the plugin, and running `./update.py` doesn't change the derivation for the plugin. Manually updating the override is required to update these types of plugins. An example of such a plugin is `LanguageClient-neovim`.
+
+To add a new plugin:
+
+  1. run `./update.py` and create a commit named "vimPlugins: Update",
+  2. add the new plugin to [vim-plugin-names](/pkgs/misc/vim-plugins/vim-plugin-names) and add overrides if required to [overrides.nix](/pkgs/misc/vim-plugins/overrides.nix),
+  3. run `./update.py` again and create a commit named "vimPlugins.[name]: init at [version]" (where `name` and `version` can be found in [generated.nix](/pkgs/misc/vim-plugins/generated.nix)), and
+  4. create a pull request.
 
 ## Important repositories
 
@@ -252,4 +275,3 @@ override to the `pkgs/misc/vim-plugins/default.nix` in the same directory.
 
 - [vim2nix](https://github.com/MarcWeber/vim-addon-vim2nix) which generates the
   .nix code
-

doc/meta.xml

@@ -30,7 +30,7 @@ meta = with stdenv.lib; {
   The meta-attributes of a package can be queried from the command-line using
   <command>nix-env</command>:
 <screen>
-$ nix-env -qa hello --json
+<prompt>$ </prompt>nix-env -qa hello --json
 {
     "hello": {
         "meta": {
@@ -70,7 +70,7 @@ $ nix-env -qa hello --json
   <command>nix-env</command> knows about the <varname>description</varname>
   field specifically:
 <screen>
-$ nix-env -qa hello --description
+<prompt>$ </prompt>nix-env -qa hello --description
 hello-2.3  A program that produces a familiar, friendly greeting
 </screen>
  </para>
@@ -272,11 +272,9 @@ meta.platforms = stdenv.lib.platforms.linux;
       <para>
        This attribute is special in that it is not actually under the
        <literal>meta</literal> attribute set but rather under the
-       <literal>passthru</literal> attribute set. This is due to a current
+       <literal>passthru</literal> attribute set. This is due to how
-       limitation of Nix, and will change as soon as Nixpkgs will be able to
+       <literal>meta</literal> attributes work, and the fact that they
-       depend on a new enough version of Nix. See
+       are supposed to contain only metadata, not derivations.
-       <link xlink:href="https://github.com/NixOS/nix/issues/2532">the relevant
-       issue</link> for more details.
       </para>
      </warning>
      <para>

doc/package-notes.xml

@@ -92,9 +92,9 @@ modulesTree = [kernel]
         <para>
          If needed you can also run <literal>make menuconfig</literal>:
 <screen>
-$ nix-env -i ncurses
+<prompt>$ </prompt>nix-env -i ncurses
-$ export NIX_CFLAGS_LINK=-lncurses
+<prompt>$ </prompt>export NIX_CFLAGS_LINK=-lncurses
-$ make menuconfig ARCH=<replaceable>arch</replaceable></screen>
+<prompt>$ </prompt>make menuconfig ARCH=<replaceable>arch</replaceable></screen>
         </para>
        </listitem>
        <listitem>
@@ -142,8 +142,8 @@ $ make menuconfig ARCH=<replaceable>arch</replaceable></screen>
   <para>
    The generator is invoked as follows:
 <screen>
-$ cd pkgs/servers/x11/xorg
+<prompt>$ </prompt>cd pkgs/servers/x11/xorg
-$ cat tarballs-7.5.list extra.list old.list \
+<prompt>$ </prompt>cat tarballs-7.5.list extra.list old.list \
   | perl ./generate-expr-from-tarballs.pl
 </screen>
    For each of the tarballs in the <filename>.list</filename> files, the script
@@ -160,8 +160,8 @@ $ cat tarballs-7.5.list extra.list old.list \
    A file like <filename>tarballs-7.5.list</filename> contains all tarballs in
    a X.org release. It can be generated like this:
 <screen>
-$ export i="mirror://xorg/X11R7.4/src/everything/"
+<prompt>$ </prompt>export i="mirror://xorg/X11R7.4/src/everything/"
-$ cat $(PRINT_PATH=1 nix-prefetch-url $i | tail -n 1) \
+<prompt>$ </prompt>cat $(PRINT_PATH=1 nix-prefetch-url $i | tail -n 1) \
   | perl -e 'while (&lt;>) { if (/(href|HREF)="([^"]*.bz2)"/) { print "$ENV{'i'}$2\n"; }; }' \
   | sort > tarballs-7.4.list
 </screen>
@@ -210,7 +210,7 @@ $ cat $(PRINT_PATH=1 nix-prefetch-url $i | tail -n 1) \
    often available. It is possible to list available Eclipse packages by
    issuing the command:
 <screen>
-$ nix-env -f '&lt;nixpkgs&gt;' -qaP -A eclipses --description
+<prompt>$ </prompt>nix-env -f '&lt;nixpkgs&gt;' -qaP -A eclipses --description
 </screen>
    Once an Eclipse variant is installed it can be run using the
    <command>eclipse</command> command, as expected. From within Eclipse it is
@@ -250,7 +250,7 @@ packageOverrides = pkgs: {
    available for installation using <varname>eclipseWithPlugins</varname> by
    running
 <screen>
-$ nix-env -f '&lt;nixpkgs&gt;' -qaP -A eclipses.plugins --description
+<prompt>$ </prompt>nix-env -f '&lt;nixpkgs&gt;' -qaP -A eclipses.plugins --description
 </screen>
   </para>
 
@@ -325,6 +325,18 @@ packageOverrides = pkgs: {
    elm2nix</link>.
   </para>
  </section>
+ <section xml:id="sec-kakoune">
+  <title>Kakoune</title>
+
+  <para>
+   Kakoune can be built to autoload plugins:
+<programlisting>(kakoune.override {
+  configure = {
+    plugins = with pkgs.kakounePlugins; [ parinfer-rust ];
+  };
+})</programlisting>
+  </para>
+ </section>
  <section xml:id="sec-shell-helpers">
   <title>Interactive shell helpers</title>
 

doc/package-specific-user-notes.xml

@@ -6,7 +6,7 @@
          answer some of the frequently asked questions
          related to Nixpkgs use.
 
-  Some useful information related to package use 
+  Some useful information related to package use
   can be found in <link linkend="chap-package-notes">package-specific development notes</link>.
 
  </para>
@@ -196,7 +196,7 @@ overrides = self: super: rec {
   haskell-mode = self.melpaPackages.haskell-mode;
   ...
 };
-((emacsPackagesNgGen emacs).overrideScope' overrides).emacsWithPackages (p: with p; [
+((emacsPackagesGen emacs).overrideScope' overrides).emacsWithPackages (p: with p; [
   # here both these package will use haskell-mode of our own choice
   ghc-mod
   dante
@@ -409,11 +409,19 @@ overrides = self: super: rec {
  </section>
 
  <section xml:id="sec-citrix">
-  <title>Citrix Receiver</title>
+  <title>Citrix Receiver &amp; Citrix Workspace App</title>
 
   <para>
-   The <link xlink:href="https://www.citrix.com/products/receiver/">Citrix
+   <note>
-   Receiver</link> is a remote desktop viewer which provides access to
+    <para>
+     Please note that the <literal>citrix_receiver</literal> package has been deprecated since its
+     development was <link xlink:href="https://docs.citrix.com/en-us/citrix-workspace-app.html">discontinued by upstream</link>
+     and has been replaced by <link xlink:href="https://www.citrix.com/products/workspace-app/">the citrix workspace app</link>.
+    </para>
+   </note>
+   <link xlink:href="https://www.citrix.com/products/receiver/">Citrix Receiver</link> and
+   <link xlink:href="https://www.citrix.com/products/workspace-app/">Citrix Workspace App</link>
+   are a remote desktop viewers which provide access to
    <link xlink:href="https://www.citrix.com/products/xenapp-xendesktop/">XenDesktop</link>
    installations.
   </para>
@@ -423,30 +431,35 @@ overrides = self: super: rec {
 
    <para>
     The tarball archive needs to be downloaded manually as the license
-    agreements of the vendor need to be accepted first. This is available at
+    agreements of the vendor for
-    the
+    <link xlink:href="https://www.citrix.com/downloads/citrix-receiver/">Citrix Receiver</link>
-    <link xlink:href="https://www.citrix.com/downloads/citrix-receiver/">download
+    or <link xlink:href="https://www.citrix.de/downloads/workspace-app/linux/workspace-app-for-linux-latest.html">Citrix Workspace</link>
-    page at citrix.com</link>. Then run <literal>nix-prefetch-url
+    need to be accepted first.
-    file://$PWD/linuxx64-$version.tar.gz</literal>. With the archive available
+    Then run <command>nix-prefetch-url file://$PWD/linuxx64-$version.tar.gz</command>.
+    With the archive available
     in the store the package can be built and installed with Nix.
    </para>
 
-   <para>
+   <warning>
-    <emphasis>Note: it's recommended to install <literal>Citrix
+    <title>Caution with <command>nix-shell</command> installs</title>
-    Receiver</literal> using <literal>nix-env -i</literal> or globally to
+    <para>
-    ensure that the <literal>.desktop</literal> files are installed properly
+     It's recommended to install <literal>Citrix Receiver</literal>
-    into <literal>$XDG_CONFIG_DIRS</literal>. Otherwise it won't be possible to
+     and/or <literal>Citrix Workspace</literal> using
-    open <literal>.ica</literal> files automatically from the browser to start
+     <literal>nix-env -i</literal> or globally to
-    a Citrix connection.</emphasis>
+     ensure that the <literal>.desktop</literal> files are installed properly
-   </para>
+     into <literal>$XDG_CONFIG_DIRS</literal>. Otherwise it won't be possible to
+     open <literal>.ica</literal> files automatically from the browser to start
+     a Citrix connection.
+    </para>
+   </warning>
   </section>
 
   <section xml:id="sec-citrix-custom-certs">
    <title>Custom certificates</title>
 
    <para>
-    The <literal>Citrix Receiver</literal> in <literal>nixpkgs</literal> trusts
+    The <literal>Citrix Workspace App</literal>
-    several certificates
+    in <literal>nixpkgs</literal> trust several certificates
     <link xlink:href="https://curl.haxx.se/docs/caextract.html">from the
     Mozilla database</link> by default. However several companies using Citrix
     might require their own corporate certificate. On distros with imperative
@@ -459,7 +472,7 @@ overrides = self: super: rec {
 <programlisting>
 <![CDATA[with import <nixpkgs> { config.allowUnfree = true; };
 let extraCerts = [ ./custom-cert-1.pem ./custom-cert-2.pem /* ... */ ]; in
-citrix_receiver.override {
+citrix_workspace.override {
   inherit extraCerts;
 }]]>
 </programlisting>

doc/platform-notes.xml

@@ -20,14 +20,14 @@
      scripts.
     </para>
 <programlisting>
-      stdenv.mkDerivation {
+stdenv.mkDerivation {
-        name = "libfoo-1.2.3";
+  name = "libfoo-1.2.3";
-        # ...
+  # ...
-        buildPhase = ''
+  buildPhase = ''
-          $CC -o hello hello.c
+    $CC -o hello hello.c
-        '';
+  '';
-      }
+}
-    </programlisting>
+</programlisting>
    </listitem>
    <listitem>
     <para>
@@ -39,12 +39,12 @@
      <function>fixupPhase</function>.
     </para>
 <programlisting>
-      stdenv.mkDerivation {
+stdenv.mkDerivation {
-        name = "libfoo-1.2.3";
+  name = "libfoo-1.2.3";
-        # ...
+  # ...
-        makeFlags = stdenv.lib.optional stdenv.isDarwin "LDFLAGS=-Wl,-install_name,$(out)/lib/libfoo.dylib";
+  makeFlags = stdenv.lib.optional stdenv.isDarwin "LDFLAGS=-Wl,-install_name,$(out)/lib/libfoo.dylib";
-      }
+}
-    </programlisting>
+</programlisting>
    </listitem>
    <listitem>
     <para>
@@ -62,19 +62,19 @@
      <manvolnum>1</manvolnum></citerefentry> manpage.
     </para>
 <programlisting>
-      dyld: Library not loaded: /nix/store/7hnmbscpayxzxrixrgxvvlifzlxdsdir-jq-1.5-lib/lib/libjq.1.dylib
+dyld: Library not loaded: /nix/store/7hnmbscpayxzxrixrgxvvlifzlxdsdir-jq-1.5-lib/lib/libjq.1.dylib
-      Referenced from: /private/tmp/nix-build-jq-1.5.drv-0/jq-1.5/tests/../jq
+Referenced from: /private/tmp/nix-build-jq-1.5.drv-0/jq-1.5/tests/../jq
-      Reason: image not found
+Reason: image not found
-      ./tests/jqtest: line 5: 75779 Abort trap: 6
+./tests/jqtest: line 5: 75779 Abort trap: 6
-    </programlisting>
+</programlisting>
 <programlisting>
-      stdenv.mkDerivation {
+stdenv.mkDerivation {
-        name = "libfoo-1.2.3";
+  name = "libfoo-1.2.3";
-        # ...
+  # ...
-        doInstallCheck = true;
+  doInstallCheck = true;
-        installCheckTarget = "check";
+  installCheckTarget = "check";
-      }
+}
-    </programlisting>
+</programlisting>
    </listitem>
    <listitem>
     <para>
@@ -85,15 +85,15 @@
      on xcode.
     </para>
 <programlisting>
-      stdenv.mkDerivation {
+stdenv.mkDerivation {
-        name = "libfoo-1.2.3";
+  name = "libfoo-1.2.3";
-        # ...
+  # ...
-        prePatch = ''
+  prePatch = ''
-          substituteInPlace Makefile \
+    substituteInPlace Makefile \
-              --replace '/usr/bin/xcrun clang' clang
+        --replace '/usr/bin/xcrun clang' clang
-        '';
+  '';
-      }
+}
-    </programlisting>
+</programlisting>
     <para>
      The package <literal>xcbuild</literal> can be used to build projects that
      really depend on Xcode. However, this replacement is not 100% compatible

doc/quick-start.xml

@@ -9,8 +9,8 @@
     <para>
      Checkout the Nixpkgs source tree:
 <screen>
-$ git clone https://github.com/NixOS/nixpkgs
+<prompt>$ </prompt>git clone https://github.com/NixOS/nixpkgs
-$ cd nixpkgs</screen>
+<prompt>$ </prompt>cd nixpkgs</screen>
     </para>
    </listitem>
    <listitem>
@@ -23,7 +23,7 @@ $ cd nixpkgs</screen>
      See <xref linkend="sec-organisation" /> for some hints on the tree
      organisation. Create a directory for your package, e.g.
 <screen>
-$ mkdir pkgs/development/libraries/libfoo</screen>
+<prompt>$ </prompt>mkdir pkgs/development/libraries/libfoo</screen>
     </para>
    </listitem>
    <listitem>
@@ -34,8 +34,8 @@ $ mkdir pkgs/development/libraries/libfoo</screen>
      as arguments, and returns a build of the package in the Nix store. The
      expression should usually be called <filename>default.nix</filename>.
 <screen>
-$ emacs pkgs/development/libraries/libfoo/default.nix
+<prompt>$ </prompt>emacs pkgs/development/libraries/libfoo/default.nix
-$ git add pkgs/development/libraries/libfoo/default.nix</screen>
+<prompt>$ </prompt>git add pkgs/development/libraries/libfoo/default.nix</screen>
     </para>
     <para>
      You can have a look at the existing Nix expressions under
@@ -180,7 +180,7 @@ $ git add pkgs/development/libraries/libfoo/default.nix</screen>
      with some descriptive name for the variable, e.g.
      <varname>libfoo</varname>.
 <screen>
-$ emacs pkgs/top-level/all-packages.nix</screen>
+<prompt>$ </prompt>emacs pkgs/top-level/all-packages.nix</screen>
     </para>
     <para>
      The attributes in that file are sorted by category (like “Development /
@@ -193,7 +193,7 @@ $ emacs pkgs/top-level/all-packages.nix</screen>
      To test whether the package builds, run the following command from the
      root of the nixpkgs source tree:
 <screen>
-$ nix-build -A libfoo</screen>
+<prompt>$ </prompt>nix-build -A libfoo</screen>
      where <varname>libfoo</varname> should be the variable name defined in the
      previous step. You may want to add the flag <option>-K</option> to keep
      the temporary build directory in case something fails. If the build
@@ -205,13 +205,17 @@ $ nix-build -A libfoo</screen>
     <para>
      If you want to install the package into your profile (optional), do
 <screen>
-$ nix-env -f . -iA libfoo</screen>
+<prompt>$ </prompt>nix-env -f . -iA libfoo</screen>
     </para>
    </listitem>
    <listitem>
     <para>
-     Optionally commit the new package and open a pull request, or send a patch
+     Optionally commit the new package and open a pull request <link
-     to <literal>https://groups.google.com/forum/#!forum/nix-devel</literal>.
+     xlink:href="https://github.com/NixOS/nixpkgs/pulls">to nixpkgs</link>, or
+     use <link
+     xlink:href="https://discourse.nixos.org/t/about-the-patches-category/477">
+     the Patches category</link> on Discourse for sending a patch without a
+     GitHub account.
     </para>
    </listitem>
   </orderedlist>

doc/reviewing-contributions.xml

@@ -153,11 +153,11 @@
        nixpkgs-unstable for easier review by running the following commands
        from a nixpkgs clone.
 <screen>
-$ git remote add channels https://github.com/NixOS/nixpkgs-channels.git <co
+<prompt>$ </prompt>git remote add channels https://github.com/NixOS/nixpkgs-channels.git <co
   xml:id='reviewing-rebase-1' />
-$ git fetch channels nixos-unstable <co xml:id='reviewing-rebase-2' />
+<prompt>$ </prompt>git fetch channels nixos-unstable <co xml:id='reviewing-rebase-2' />
-$ git fetch origin pull/PRNUMBER/head <co xml:id='reviewing-rebase-3' />
+<prompt>$ </prompt>git fetch origin pull/PRNUMBER/head <co xml:id='reviewing-rebase-3' />
-$ git rebase --onto nixos-unstable BASEBRANCH FETCH_HEAD <co
+<prompt>$ </prompt>git rebase --onto nixos-unstable BASEBRANCH FETCH_HEAD <co
   xml:id='reviewing-rebase-4' />
 </screen>
        <calloutlist>
@@ -197,7 +197,7 @@ $ git rebase --onto nixos-unstable BASEBRANCH FETCH_HEAD <co
        request url.
       </para>
 <screen>
-$ nix-shell -p nix-review --run "nix-review pr PRNUMBER"
+<prompt>$ </prompt>nix-shell -p nix-review --run "nix-review pr PRNUMBER"
 </screen>
      </listitem>
     </itemizedlist>

doc/stdenv.xml

@@ -31,9 +31,28 @@ stdenv.mkDerivation {
    separate Nix expression from <filename>pkgs/all-packages.nix</filename>, you
    need to pass it as a function argument.) Specifying a
    <varname>name</varname> and a <varname>src</varname> is the absolute minimum
-   you need to do. Many packages have dependencies that are not provided in the
+   Nix requires. For convenience, you can also use <varname>pname</varname> and
-   standard environment. It’s usually sufficient to specify those
+   <varname>version</varname> attributes and <literal>mkDerivation</literal>
-   dependencies in the <varname>buildInputs</varname> attribute:
+   will automatically set <varname>name</varname> to
+   <literal>"${pname}-${version}"</literal> by default. Since
+   <link xlink:href="https://github.com/NixOS/rfcs/pull/35">RFC 0035</link>,
+   this is preferred for packages in Nixpkgs, as it allows us to reuse the
+   version easily:
+<programlisting>
+stdenv.mkDerivation rec {
+  pname = "libfoo";
+  version = "1.2.3";
+  src = fetchurl {
+    url = "http://example.org/libfoo-source-${version}.tar.bz2";
+    sha256 = "0x2g1jqygyr5wiwg4ma1nd7w4ydpy82z9gkcv8vh2v8dn3y58v5m";
+  };
+}</programlisting>
+  </para>
+
+  <para>
+   Many packages have dependencies that are not provided in the standard
+   environment. It’s usually sufficient to specify those dependencies in the
+   <varname>buildInputs</varname> attribute:
 <programlisting>
 stdenv.mkDerivation {
   name = "libfoo-1.2.3";
@@ -324,7 +343,7 @@ let f(h, t, i) = i + (if i &lt;= 0 then h else t - 1)
 let f(h, h + 1, i) = i + (if i &lt;= 0 then h else (h + 1) - 1)
 let f(h, h + 1, i) = i + (if i &lt;= 0 then h else h)
 let f(h, h + 1, i) = i + h
-  </programlisting>
+</programlisting>
    This is where "sum-like" comes in from above: We can just sum all of the
    host offsets to get the host offset of the transitive dependency. The target
    offset is the transitive dependency is simply the host offset + 1, just as
@@ -680,7 +699,7 @@ passthru = {
      </para>
     </listitem>
    </varlistentry>
-   <varlistentry>
+   <varlistentry xml:id="var-passthru-updateScript">
     <term>
      <varname>passthru.updateScript</varname>
     </term>
@@ -709,19 +728,16 @@ passthru.updateScript = writeScript "update-zoom-us" ''
 <programlisting>
 passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ];
 </programlisting>
-      Note that the update scripts will be run in parallel by default; you
+     </para>
-      should avoid running <command>git commit</command> or any other commands
+     <para>
-      that cannot handle that.
+      The script will be usually run from the root of the Nixpkgs repository
+      but you should not rely on that. Also note that the update scripts will
+      be run in parallel by default; you should avoid running <command>git
+      commit</command> or any other commands that cannot handle that.
      </para>
      <para>
       For information about how to run the updates, execute
-      <cmdsynopsis>
+      <command>nix-shell maintainers/scripts/update.nix</command>.
-       <command>nix-shell</command> 
-       <arg>
-        maintainers/scripts/update.nix
-       </arg>
-      </cmdsynopsis>
-      .
      </para>
     </listitem>
    </varlistentry>
@@ -770,7 +786,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
         set, the default value is used, which is <literal>$prePhases
         unpackPhase patchPhase $preConfigurePhases configurePhase
         $preBuildPhases buildPhase checkPhase $preInstallPhases installPhase
-        fixupPhase $preDistPhases distPhase $postPhases</literal>.
+        fixupPhase installCheckPhase $preDistPhases distPhase $postPhases</literal>.
        </para>
        <para>
         Usually, if you just want to add a few phases, it’s more convenient
@@ -971,6 +987,16 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
       </para>
      </listitem>
     </varlistentry>
+    <varlistentry>
+     <term>
+      <varname>dontUnpack</varname>
+     </term>
+     <listitem>
+      <para>
+       Set to true to skip the unpack phase.
+      </para>
+     </listitem>
+    </varlistentry>
     <varlistentry>
      <term>
       <varname>dontMakeSourcesWritable</varname>
@@ -1093,6 +1119,16 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ]
       </para>
      </listitem>
     </varlistentry>
+    <varlistentry>
+     <term>
+      <varname>dontConfigure</varname>
+     </term>
+     <listitem>
+      <para>
+       Set to true to skip the configure phase.
+      </para>
+     </listitem>
+    </varlistentry>
     <varlistentry>
      <term>
       <varname>configureFlagsArray</varname>
@@ -1563,6 +1599,16 @@ installTargets = "install-bin install-doc";</programlisting>
 
    <variablelist>
     <title>Variables controlling the fixup phase</title>
+    <varlistentry>
+     <term>
+      <varname>dontFixup</varname>
+     </term>
+     <listitem>
+      <para>
+        Set to true to skip the fixup phase.
+      </para>
+     </listitem>
+    </varlistentry>
     <varlistentry>
      <term>
       <varname>dontStrip</varname>
@@ -2213,7 +2259,7 @@ someVar=$(stripHash $name)
    array by doing something like
 <programlisting language="bash">
 addEnvHooks "$hostOffset" myBashFunction
-  </programlisting>
+</programlisting>
   </para>
 
   <para>
@@ -2365,7 +2411,7 @@ addEnvHooks "$hostOffset" myBashFunction
       <para>
        The Bintools Wrapper was only just recently split off from CC Wrapper,
        so the division of labor is still being worked out. For example, it
-       shouldn't care about about the C standard library, but just take a
+       shouldn't care about the C standard library, but just take a
        derivation with the dynamic loader (which happens to be the glibc on
        linux). Dependency finding however is a task both wrappers will continue
        to need to share, and probably the most important to understand. It is
@@ -2583,13 +2629,11 @@ addEnvHooks "$hostOffset" myBashFunction
     </varlistentry>
     <varlistentry>
      <term>
-      GStreamer
+      GNOME platform
      </term>
      <listitem>
       <para>
-       Adds the GStreamer plugins subdirectory of each build input to the
+       Hooks related to GNOME platform and related libraries like GLib, GTK and GStreamer are described in <xref linkend="sec-language-gnome" />.
-       <envar>GST_PLUGIN_SYSTEM_PATH_1_0</envar> or
-       <envar>GST_PLUGIN_SYSTEM_PATH</envar> environment variable.
       </para>
      </listitem>
     </varlistentry>
@@ -2651,8 +2695,8 @@ addEnvHooks "$hostOffset" myBashFunction
        that is supposed to be inspected, add <literal>breakpointHook</literal>
        to <literal>nativeBuildInputs</literal>.
 <programlisting>
-         nativeBuildInputs = [ breakpointHook ];
+nativeBuildInputs = [ breakpointHook ];
-       </programlisting>
+</programlisting>
        When a build failure happens there will be an instruction printed that
        shows how to attach with <literal>cntr</literal> to the build sandbox.
       </para>
@@ -2668,6 +2712,49 @@ addEnvHooks "$hostOffset" myBashFunction
       </note>
      </listitem>
     </varlistentry>
+    <varlistentry>
+     <term>
+      installShellFiles
+     </term>
+     <listitem>
+      <para>
+       This hook helps with installing manpages and shell completion files. It
+       exposes 2 shell functions <literal>installManPage</literal> and
+       <literal>installShellCompletion</literal> that can be used from your
+       <literal>postInstall</literal> hook.
+      </para>
+      <para>
+       The <literal>installManPage</literal> function takes one or more paths
+       to manpages to install. The manpages must have a section suffix, and may
+       optionally be compressed (with <literal>.gz</literal> suffix). This
+       function will place them into the correct directory.
+      </para>
+      <para>
+       The <literal>installShellCompletion</literal> function takes one or more
+       paths to shell completion files. By default it will autodetect the shell
+       type from the completion file extension, but you may also specify it by
+       passing one of <literal>--bash</literal>, <literal>--fish</literal>, or
+       <literal>--zsh</literal>. These flags apply to all paths listed after
+       them (up until another shell flag is given). Each path may also have a
+       custom installation name provided by providing a flag <literal>--name
+       NAME</literal> before the path. If this flag is not provided, zsh
+       completions will be renamed automatically such that
+       <literal>foobar.zsh</literal> becomes <literal>_foobar</literal>.
+<programlisting>
+nativeBuildInputs = [ installShellFiles ];
+postInstall = ''
+  installManPage doc/foobar.1 doc/barfoo.3
+  # explicit behavior
+  installShellCompletion --bash --name foobar.bash share/completions.bash
+  installShellCompletion --fish --name foobar.fish share/completions.fish
+  installShellCompletion --zsh --name _foobar share/completions.zsh
+  # implicit behavior
+  installShellCompletion share/completions/foobar.{bash,fish,zsh}
+'';
+</programlisting>
+      </para>
+     </listitem>
+    </varlistentry>
     <varlistentry>
      <term>
       libiconv, libintl
@@ -2847,7 +2934,7 @@ addEnvHooks "$hostOffset" myBashFunction
          printf(help_message);
                             ^
 cc1plus: some warnings being treated as errors
-    </programlisting>
+</programlisting>
     </listitem>
    </varlistentry>
    <varlistentry>
@@ -2869,7 +2956,7 @@ cc1plus: some warnings being treated as errors
 <programlisting>
 bin/blib.a(bios_console.o): In function `bios_handle_cup':
 /tmp/nix-build-ipxe-20141124-5cbdc41.drv-0/ipxe-5cbdc41/src/arch/i386/firmware/pcbios/bios_console.c:86: undefined reference to `__stack_chk_fail'
-    </programlisting>
+</programlisting>
     </listitem>
    </varlistentry>
    <varlistentry>
@@ -2898,19 +2985,19 @@ bin/blib.a(bios_console.o): In function `bios_handle_cup':
 <programlisting>
 malloc.c:404:15: error: return type is an incomplete type
 malloc.c:410:19: error: storage size of 'ms' isn't known
-    </programlisting>
+</programlisting>
 <programlisting>
 strdup.h:22:1: error: expected identifier or '(' before '__extension__'
-    </programlisting>
+</programlisting>
 <programlisting>
 strsep.c:65:23: error: register name not specified for 'delim'
-    </programlisting>
+</programlisting>
 <programlisting>
 installwatch.c:3751:5: error: conflicting types for '__open_2'
-    </programlisting>
+</programlisting>
 <programlisting>
 fcntl2.h:50:4: error: call to '__open_missing_mode' declared with attribute error: open with O_CREAT or O_TMPFILE in second argument needs 3 arguments
-    </programlisting>
+</programlisting>
     </listitem>
    </varlistentry>
    <varlistentry>
@@ -2935,7 +3022,7 @@ fcntl2.h:50:4: error: call to '__open_missing_mode' declared with attribute erro
 <programlisting>
 ccbLfRgg.s: Assembler messages:
 ccbLfRgg.s:33: Error: missing or invalid displacement expression `private_key_len@GOTOFF'
-    </programlisting>
+</programlisting>
     </listitem>
    </varlistentry>
    <varlistentry>
@@ -2999,7 +3086,7 @@ ccbLfRgg.s:33: Error: missing or invalid displacement expression `private_key_le
      </para>
 <programlisting>
 intel_drv.so: undefined symbol: vgaHWFreeHWRec
-    </programlisting>
+</programlisting>
     </listitem>
    </varlistentry>
   </variablelist>

doc/submitting-changes.xml

@@ -36,8 +36,8 @@
          </listitem>
         </itemizedlist>
 <screen>
-$ git checkout 0998212
+<prompt>$ </prompt>git checkout 0998212
-$ git checkout -b 'fix/pkg-name-update'
+<prompt>$ </prompt>git checkout -b 'fix/pkg-name-update'
 </screen>
        </para>
       </listitem>
@@ -514,7 +514,7 @@ The original commit message describing the reason why the world was torn apart.
 (cherry picked from commit abcdef)
 Reason: I just had a gut feeling that this would also be wanted by people from
 the stone age.
-      </screen>
+</screen>
     </listitem>
    </itemizedlist>
   </section>

lib/attrsets.nix

@@ -354,7 +354,7 @@ rec {
        => { a = ["x" "y"]; b = ["z"] }
   */
   zipAttrsWith = f: sets: zipAttrsWithNames (concatMap attrNames sets) f sets;
-  /* Like `zipAttrsWith' with `(name: values: value)' as the function.
+  /* Like `zipAttrsWith' with `(name: values: values)' as the function.
 
     Example:
       zipAttrs [{a = "x";} {a = "y"; b = "z";}]

lib/customisation.nix

@@ -47,7 +47,7 @@ rec {
 
 
   /* `makeOverridable` takes a function from attribute set to attribute set and
-     injects `override` attibute which can be used to override arguments of
+     injects `override` attribute which can be used to override arguments of
      the function.
 
        nix-repl> x = {a, b}: { result = a + b; }

lib/default.nix

@@ -50,7 +50,7 @@ let
     filesystem = callLibs ./filesystem.nix;
 
     # back-compat aliases
-    platforms = systems.forMeta;
+    platforms = systems.doubles;
 
     inherit (builtins) add addErrorContext attrNames concatLists
       deepSeq elem elemAt filter genericClosure genList getAttr
@@ -71,7 +71,7 @@ let
       zipAttrsWithNames zipAttrsWith zipAttrs recursiveUpdateUntil
       recursiveUpdate matchAttrs overrideExisting getOutput getBin
       getLib getDev chooseDevOutputs zipWithNames zip;
-    inherit (lists) singleton foldr fold foldl foldl' imap0 imap1
+    inherit (lists) singleton forEach foldr fold foldl foldl' imap0 imap1
       concatMap flatten remove findSingle findFirst any all count
       optional optionals toList range partition zipListsWith zipLists
       reverseList listDfs toposort sort naturalSort compareLists take

lib/deprecated.nix

@@ -24,7 +24,7 @@ rec {
     let arg = (merger init (defaultMergeArg init x));
         # now add the function with composed args already applied to the final attrs
         base = (setAttrMerge "passthru" {} (f arg)
-                        ( z: z // rec {
+                        ( z: z // {
                             function = foldArgs merger f arg;
                             args = (lib.attrByPath ["passthru" "args"] {} z) // x;
                           } ));

lib/kernel.nix

@@ -1,7 +1,7 @@
 { lib, version }:
 
 with lib;
-rec {
+{
   # Common patterns/legacy
   whenAtLeast = ver: mkIf (versionAtLeast version ver);
   whenOlder   = ver: mkIf (versionOlder version ver);

lib/licenses.nix

@@ -7,7 +7,7 @@ let
 
 in
 
-lib.mapAttrs (n: v: v // { shortName = n; }) rec {
+lib.mapAttrs (n: v: v // { shortName = n; }) {
   /* License identifiers from spdx.org where possible.
    * If you cannot find your license here, then look for a similar license or
    * add it to this list. The URL mentioned above is a good source for inspiration.
@@ -434,12 +434,12 @@ lib.mapAttrs (n: v: v // { shortName = n; }) rec {
 
   lgpl21 = spdx {
     spdxId = "LGPL-2.1-only";
-    fullName = "GNU Library General Public License v2.1 only";
+    fullName = "GNU Lesser General Public License v2.1 only";
   };
 
   lgpl21Plus = spdx {
     spdxId = "LGPL-2.1-or-later";
-    fullName = "GNU Library General Public License v2.1 or later";
+    fullName = "GNU Lesser General Public License v2.1 or later";
   };
 
   lgpl3 = spdx {
@@ -457,9 +457,9 @@ lib.mapAttrs (n: v: v // { shortName = n; }) rec {
     fullName = "libpng License";
   };
 
-  libpng2 = {
+  libpng2 = spdx {
-    fullName = "libpng License v2"; # 1.6.36+
+    spdxId = "libpng-2.0"; # Used since libpng 1.6.36.
-    url = "http://www.libpng.org/pub/png/src/libpng-LICENSE.txt";
+    fullName = "PNG Reference Library version 2";
   };
 
   libtiff = spdx {

lib/lists.nix

@@ -21,6 +21,19 @@ rec {
   */
   singleton = x: [x];
 
+  /*  Apply the function to each element in the list. Same as `map`, but arguments
+      flipped.
+
+      Type: forEach :: [a] -> (a -> b) -> [b]
+
+      Example:
+        forEach [ 1 2 ] (x:
+          toString x
+        )
+        => [ "1" "2" ]
+  */
+  forEach = xs: f: map f xs;
+
   /* “right fold” a binary function `op` between successive elements of
      `list` with `nul' as the starting value, i.e.,
      `foldr op nul [x_1 x_2 ... x_n] == op x_1 (op x_2 ... (op x_n nul))`.
@@ -75,7 +88,7 @@ rec {
   /* Strict version of `foldl`.
 
      The difference is that evaluation is forced upon access. Usually used
-     with small whole results (in contract with lazily-generated list or large
+     with small whole results (in contrast with lazily-generated list or large
      lists where only a part is consumed.)
 
      Type: foldl' :: (b -> a -> b) -> b -> [a] -> b
@@ -446,11 +459,11 @@ rec {
       if length list < 2
       then # finish
            { result =  list; }
-      else if dfsthis ? "cycle"
+      else if dfsthis ? cycle
            then # there's a cycle, starting from the current vertex, return it
                 { cycle = reverseList ([ dfsthis.cycle ] ++ dfsthis.visited);
                   inherit (dfsthis) loops; }
-           else if toporest ? "cycle"
+           else if toporest ? cycle
                 then # there's a cycle somewhere else in the graph, return it
                      toporest
                 # Slow, but short. Can be made a bit faster with an explicit stack.

lib/modules.nix

@@ -323,16 +323,14 @@ rec {
         else
           mergeDefinitions loc opt.type defs';
 
-      # Check whether the option is defined, and apply the ‘apply’
+
-      # function to the merged value.  This allows options to yield a
+      # The value with a check that it is defined
-      # value computed from the definitions.
+      valueDefined = if res.isDefined then res.mergedValue else
-      value =
+        throw "The option `${showOption loc}' is used but not defined.";
-        if !res.isDefined then
+
-          throw "The option `${showOption loc}' is used but not defined."
+      # Apply the 'apply' function to the merged value. This allows options to
-        else if opt ? apply then
+      # yield a value computed from the definitions
-          opt.apply res.mergedValue
+      value = if opt ? apply then opt.apply valueDefined else valueDefined;
-        else
-          res.mergedValue;
 
     in opt //
       { value = builtins.addErrorContext "while evaluating the option `${showOption loc}':" value;

lib/options.nix

@@ -36,7 +36,7 @@ rec {
     example ? null,
     # String describing the option.
     description ? null,
-    # Related packages used in the manual (see `genRelatedPackages` in ../nixos/doc/manual/default.nix).
+    # Related packages used in the manual (see `genRelatedPackages` in ../nixos/lib/make-options-doc/default.nix).
     relatedPackages ? null,
     # Option type, providing type-checking and value merging.
     type ? null,
@@ -101,7 +101,7 @@ rec {
   mergeOneOption = loc: defs:
     if defs == [] then abort "This case should never happen."
     else if length defs != 1 then
-      throw "The unique option `${showOption loc}' is defined multiple times, in ${showFiles (getFiles defs)}."
+      throw "The unique option `${showOption loc}' is defined multiple times, in:\n - ${concatStringsSep "\n - " (getFiles defs)}."
     else (head defs).value;
 
   /* "Merge" option definitions by checking that they all have the same value. */

lib/sources.nix

@@ -36,29 +36,62 @@ rec {
   # allowing you to chain multiple calls together without any
   # intermediate copies being put in the nix store.
   #
-  #     lib.cleanSourceWith f (lib.cleanSourceWith g ./.)     # Succeeds!
+  #     lib.cleanSourceWith {
-  #     builtins.filterSource f (builtins.filterSource g ./.) # Fails!
+  #       filter = f;
-  cleanSourceWith = { filter, src }:
+  #       src = lib.cleanSourceWith {
+  #         filter = g;
+  #         src = ./.;
+  #       };
+  #     }
+  #     # Succeeds!
+  #
+  #     builtins.filterSource f (builtins.filterSource g ./.)
+  #     # Fails!
+  #
+  # Parameters:
+  #
+  #   src:      A path or cleanSourceWith result to filter and/or rename.
+  #
+  #   filter:   A function (path -> type -> bool)
+  #             Optional with default value: constant true (include everything)
+  #             The function will be combined with the && operator such
+  #             that src.filter is called lazily.
+  #             For implementing a filter, see
+  #             https://nixos.org/nix/manual/#builtin-filterSource
+  #
+  #   name:     Optional name to use as part of the store path.
+  #             This defaults `src.name` or otherwise `baseNameOf src`.
+  #             We recommend setting `name` whenever `src` is syntactically `./.`.
+  #             Otherwise, you depend on `./.`'s name in the parent directory,
+  #             which can cause inconsistent names, defeating caching.
+  #
+  cleanSourceWith = { filter ? _path: _type: true, src, name ? null }:
     let
       isFiltered = src ? _isLibCleanSourceWith;
       origSrc = if isFiltered then src.origSrc else src;
       filter' = if isFiltered then name: type: filter name type && src.filter name type else filter;
+      name' = if name != null then name else if isFiltered then src.name else baseNameOf src;
     in {
       inherit origSrc;
       filter = filter';
-      outPath = builtins.filterSource filter' origSrc;
+      outPath = builtins.path { filter = filter'; path = origSrc; name = name'; };
       _isLibCleanSourceWith = true;
+      name = name';
     };
 
   # Filter sources by a list of regular expressions.
   #
   # E.g. `src = sourceByRegex ./my-subproject [".*\.py$" "^database.sql$"]`
-  sourceByRegex = src: regexes: cleanSourceWith {
+  sourceByRegex = src: regexes:
-    filter = (path: type:
+    let
-      let relPath = lib.removePrefix (toString src + "/") (toString path);
+      isFiltered = src ? _isLibCleanSourceWith;
-      in lib.any (re: builtins.match re relPath != null) regexes);
+      origSrc = if isFiltered then src.origSrc else src;
-    inherit src;
+    in lib.cleanSourceWith {
-  };
+      filter = (path: type:
+        let relPath = lib.removePrefix (toString origSrc + "/") (toString path);
+        in lib.any (re: builtins.match re relPath != null) regexes);
+      inherit src;
+    };
 
   # Get all files ending with the specified suffices from the given
   # directory or its descendants.  E.g. `sourceFilesBySuffices ./dir

lib/systems/default.nix

@@ -3,7 +3,6 @@
 
 rec {
   doubles = import ./doubles.nix { inherit lib; };
-  forMeta = import ./for-meta.nix { inherit lib; };
   parse = import ./parse.nix { inherit lib; };
   inspect = import ./inspect.nix { inherit lib; };
   platforms = import ./platforms.nix { inherit lib; };
@@ -15,7 +14,9 @@ rec {
   # `parsed` is inferred from args, both because there are two options with one
   # clearly prefered, and to prevent cycles. A simpler fixed point where the RHS
   # always just used `final.*` would fail on both counts.
-  elaborate = args: let
+  elaborate = args': let
+    args = if lib.isString args' then { system = args'; }
+           else args';
     final = {
       # Prefer to parse `config` as it is strictly more informative.
       parsed = parse.mkSystemFromString (if args ? config then args.config else args.system);
@@ -57,13 +58,13 @@ rec {
       uname = {
         # uname -s
         system = {
-          "linux" = "Linux";
+          linux = "Linux";
-          "windows" = "Windows";
+          windows = "Windows";
-          "darwin" = "Darwin";
+          darwin = "Darwin";
-          "netbsd" = "NetBSD";
+          netbsd = "NetBSD";
-          "freebsd" = "FreeBSD";
+          freebsd = "FreeBSD";
-          "openbsd" = "OpenBSD";
+          openbsd = "OpenBSD";
-          "wasi" = "Wasi";
+          wasi = "Wasi";
         }.${final.parsed.kernel.name} or null;
 
          # uname -p
@@ -85,10 +86,10 @@ rec {
         else if final.isx86_64 then "x86_64"
         else if final.isx86 then "i386"
         else {
-          "powerpc" = "ppc";
+          powerpc = "ppc";
-          "powerpcle" = "ppc";
+          powerpcle = "ppc";
-          "powerpc64" = "ppc64";
+          powerpc64 = "ppc64";
-          "powerpc64le" = "ppc64le";
+          powerpc64le = "ppc64le";
         }.${final.parsed.cpu.name} or final.parsed.cpu.name;
 
       emulator = pkgs: let

lib/systems/doubles.nix

@@ -13,19 +13,27 @@ let
 
     "i686-cygwin" "i686-freebsd" "i686-linux" "i686-netbsd" "i686-openbsd"
 
-    "x86_64-cygwin" "x86_64-darwin" "x86_64-freebsd" "x86_64-linux"
+    "x86_64-cygwin" "x86_64-freebsd" "x86_64-linux"
     "x86_64-netbsd" "x86_64-openbsd" "x86_64-solaris"
 
+    "x86_64-darwin" "i686-darwin" "aarch64-darwin" "armv7a-darwin"
+
     "x86_64-windows" "i686-windows"
 
     "wasm64-wasi" "wasm32-wasi"
+
+    "powerpc64le-linux"
+
+    "riscv32-linux" "riscv64-linux"
+
+    "aarch64-none" "avr-none" "arm-none" "i686-none" "x86_64-none" "powerpc-none" "msp430-none" "riscv64-none" "riscv32-none"
   ];
 
   allParsed = map parse.mkSystemFromString all;
 
   filterDoubles = f: map parse.doubleFromSystem (lists.filter f allParsed);
 
-in rec {
+in {
   inherit all;
 
   none = [];
@@ -36,6 +44,7 @@ in rec {
   i686    = filterDoubles predicates.isi686;
   x86_64  = filterDoubles predicates.isx86_64;
   mips    = filterDoubles predicates.isMips;
+  riscv   = filterDoubles predicates.isRiscV;
 
   cygwin  = filterDoubles predicates.isCygwin;
   darwin  = filterDoubles predicates.isDarwin;
@@ -50,5 +59,7 @@ in rec {
   wasi    = filterDoubles predicates.isWasi;
   windows = filterDoubles predicates.isWindows;
 
+  embedded = filterDoubles predicates.isNone;
+
   mesaPlatforms = ["i686-linux" "x86_64-linux" "x86_64-darwin" "armv5tel-linux" "armv6l-linux" "armv7l-linux" "armv7a-linux" "aarch64-linux" "powerpc64le-linux"];
 }

lib/systems/examples.nix

@@ -24,27 +24,27 @@ rec {
     platform = platforms.powernv;
   };
 
-  sheevaplug = rec {
+  sheevaplug = {
     config = "armv5tel-unknown-linux-gnueabi";
     platform = platforms.sheevaplug;
   };
 
-  raspberryPi = rec {
+  raspberryPi = {
     config = "armv6l-unknown-linux-gnueabihf";
     platform = platforms.raspberrypi;
   };
 
-  armv7l-hf-multiplatform = rec {
+  armv7l-hf-multiplatform = {
     config = "armv7l-unknown-linux-gnueabihf";
     platform = platforms.armv7l-hf-multiplatform;
   };
 
-  aarch64-multiplatform = rec {
+  aarch64-multiplatform = {
     config = "aarch64-unknown-linux-gnu";
     platform = platforms.aarch64-multiplatform;
   };
 
-  armv7a-android-prebuilt = rec {
+  armv7a-android-prebuilt = {
     config = "armv7a-unknown-linux-androideabi";
     sdkVer = "24";
     ndkVer = "18b";
@@ -52,7 +52,7 @@ rec {
     useAndroidPrebuilt = true;
   };
 
-  aarch64-android-prebuilt = rec {
+  aarch64-android-prebuilt = {
     config = "aarch64-unknown-linux-android";
     sdkVer = "24";
     ndkVer = "18b";
@@ -65,17 +65,17 @@ rec {
     inherit (platform.gcc) fpu;
   };
 
-  pogoplug4 = rec {
+  pogoplug4 = {
     config = "armv5tel-unknown-linux-gnueabi";
     platform = platforms.pogoplug4;
   };
 
-  ben-nanonote = rec {
+  ben-nanonote = {
     config = "mipsel-unknown-linux-uclibc";
     platform = platforms.ben_nanonote;
   };
 
-  fuloongminipc = rec {
+  fuloongminipc = {
     config = "mipsel-unknown-linux-gnu";
     platform = platforms.fuloong2f_n32;
   };
@@ -97,6 +97,18 @@ rec {
   riscv64 = riscv "64";
   riscv32 = riscv "32";
 
+  riscv64-embedded = {
+    config = "riscv64-none-elf";
+    libc = "newlib";
+    platform = platforms.riscv-multiplatform "64";
+  };
+
+  riscv32-embedded = {
+    config = "riscv32-none-elf";
+    libc = "newlib";
+    platform = platforms.riscv-multiplatform "32";
+  };
+
   msp430 = {
     config = "msp430-elf";
     libc = "newlib";
@@ -224,4 +236,9 @@ rec {
     useLLVM = true;
   };
 
+  # Ghcjs
+  ghcjs = {
+    config = "js-unknown-ghcjs";
+    platform = {};
+  };
 }

lib/systems/for-meta.nix

@@ -1,38 +0,0 @@
-{ lib }:
-let
-  inherit (lib.systems) parse;
-  inherit (lib.systems.inspect) patterns;
-
-  abis = lib.mapAttrs (_: abi: builtins.removeAttrs abi [ "assertions" ]) parse.abis;
-
-in rec {
-  all     = [ {} ]; # `{}` matches anything
-  none    = [];
-
-  arm     = [ patterns.isAarch32 ];
-  aarch64 = [ patterns.isAarch64 ];
-  x86     = [ patterns.isx86 ];
-  i686    = [ patterns.isi686 ];
-  x86_64  = [ patterns.isx86_64 ];
-  mips    = [ patterns.isMips ];
-  riscv   = [ patterns.isRiscV ];
-
-  cygwin  = [ patterns.isCygwin ];
-  darwin  = [ patterns.isDarwin ];
-  freebsd = [ patterns.isFreeBSD ];
-  # Should be better, but MinGW is unclear.
-  gnu     = [
-    { kernel = parse.kernels.linux; abi = abis.gnu; }
-    { kernel = parse.kernels.linux; abi = abis.gnueabi; }
-    { kernel = parse.kernels.linux; abi = abis.gnueabihf; }
-  ];
-  illumos = [ patterns.isSunOS ];
-  linux   = [ patterns.isLinux ];
-  netbsd  = [ patterns.isNetBSD ];
-  openbsd = [ patterns.isOpenBSD ];
-  unix    = patterns.isUnix; # Actually a list
-  windows = [ patterns.isWindows ];
-  wasi    = [ patterns.isWasi ];
-
-  inherit (lib.systems.doubles) mesaPlatforms;
-}

lib/systems/inspect.nix

@@ -12,7 +12,7 @@ rec {
     isx86_32       = { cpu = { family = "x86"; bits = 32; }; };
     isx86_64       = { cpu = { family = "x86"; bits = 64; }; };
     isPowerPC      = { cpu = cpuTypes.powerpc; };
-    isPower = { cpu = { family = "power"; }; };
+    isPower        = { cpu = { family = "power"; }; };
     isx86          = { cpu = { family = "x86"; }; };
     isAarch32      = { cpu = { family = "arm"; bits = 32; }; };
     isAarch64      = { cpu = { family = "arm"; bits = 64; }; };
@@ -23,6 +23,7 @@ rec {
     isMsp430       = { cpu = { family = "msp430"; }; };
     isAvr          = { cpu = { family = "avr"; }; };
     isAlpha        = { cpu = { family = "alpha"; }; };
+    isJavaScript   = { cpu = cpuTypes.js; };
 
     is32bit        = { cpu = { bits = 32; }; };
     is64bit        = { cpu = { bits = 64; }; };
@@ -44,6 +45,8 @@ rec {
     isCygwin       = { kernel = kernels.windows; abi = abis.cygnus; };
     isMinGW        = { kernel = kernels.windows; abi = abis.gnu; };
     isWasi         = { kernel = kernels.wasi; };
+    isGhcjs        = { kernel = kernels.ghcjs; };
+    isNone         = { kernel = kernels.none; };
 
     isAndroid      = [ { abi = abis.android; } { abi = abis.androideabi; } ];
     isMusl         = with abis; map (a: { abi = a; }) [ musl musleabi musleabihf ];

lib/systems/parse.nix

@@ -106,11 +106,13 @@ rec {
 
     wasm32   = { bits = 32; significantByte = littleEndian; family = "wasm"; };
     wasm64   = { bits = 64; significantByte = littleEndian; family = "wasm"; };
-    
+
     alpha    = { bits = 64; significantByte = littleEndian; family = "alpha"; };
 
     msp430   = { bits = 16; significantByte = littleEndian; family = "msp430"; };
     avr      = { bits = 8; family = "avr"; };
+
+    js       = { bits = 32; significantByte = littleEndian; family = "js"; };
   };
 
   # Determine where two CPUs are compatible with each other. That is,
@@ -271,6 +273,7 @@ rec {
     solaris = { execFormat = elf;     families = { }; };
     wasi    = { execFormat = wasm;    families = { }; };
     windows = { execFormat = pe;      families = { }; };
+    ghcjs   = { execFormat = unknown; families = { }; };
   } // { # aliases
     # 'darwin' is the kernel for all of them. We choose macOS by default.
     darwin = kernels.macos;
@@ -384,6 +387,8 @@ rec {
         then { cpu = elemAt l 0; vendor = elemAt l 1;    kernel = elemAt l 2;                }
       else if (elem (elemAt l 2) ["eabi" "eabihf" "elf"])
         then { cpu = elemAt l 0; vendor = "unknown"; kernel = elemAt l 1; abi = elemAt l 2; }
+      else if (elemAt l 2 == "ghcjs")
+        then { cpu = elemAt l 0; vendor = "unknown"; kernel = elemAt l 2; }
       else throw "Target specification with 3 components is ambiguous";
     "4" =    { cpu = elemAt l 0; vendor = elemAt l 1; kernel = elemAt l 2; abi = elemAt l 3; };
   }.${toString (length l)}
@@ -403,7 +408,7 @@ rec {
     getKernel = name:  kernels.${name} or (throw "Unknown kernel: ${name}");
     getAbi    = name:     abis.${name} or (throw "Unknown ABI: ${name}");
 
-    parsed = rec {
+    parsed = {
       cpu = getCpu args.cpu;
       vendor =
         /**/ if args ? vendor    then getVendor args.vendor
@@ -428,7 +433,7 @@ rec {
 
   mkSystemFromString = s: mkSystemFromSkeleton (mkSkeletonFromList (lib.splitString "-" s));
 
-  doubleFromSystem = { cpu, vendor, kernel, abi, ... }:
+  doubleFromSystem = { cpu, kernel, abi, ... }:
     /**/ if abi == abis.cygnus       then "${cpu.name}-cygwin"
     else if kernel.families ? darwin then "${cpu.name}-darwin"
     else "${cpu.name}-${kernel.name}";

lib/systems/platforms.nix

@@ -456,16 +456,16 @@ rec {
   };
 
   selectBySystem = system: {
-      "i486-linux" = pc32;
+      i486-linux = pc32;
-      "i586-linux" = pc32;
+      i586-linux = pc32;
-      "i686-linux" = pc32;
+      i686-linux = pc32;
-      "x86_64-linux" = pc64;
+      x86_64-linux = pc64;
-      "armv5tel-linux" = sheevaplug;
+      armv5tel-linux = sheevaplug;
-      "armv6l-linux" = raspberrypi;
+      armv6l-linux = raspberrypi;
-      "armv7a-linux" = armv7l-hf-multiplatform;
+      armv7a-linux = armv7l-hf-multiplatform;
-      "armv7l-linux" = armv7l-hf-multiplatform;
+      armv7l-linux = armv7l-hf-multiplatform;
-      "aarch64-linux" = aarch64-multiplatform;
+      aarch64-linux = aarch64-multiplatform;
-      "mipsel-linux" = fuloong2f_n32;
+      mipsel-linux = fuloong2f_n32;
-      "powerpc64le-linux" = powernv;
+      powerpc64le-linux = powernv;
     }.${system} or pcBase;
 }

lib/tests/modules.sh

@@ -71,6 +71,15 @@ checkConfigError 'The option value .* in .* is not of type.*positive integer.*'
 checkConfigOutput "42" config.value ./declare-int-between-value.nix ./define-value-int-positive.nix
 checkConfigError 'The option value .* in .* is not of type.*between.*-21 and 43.*inclusive.*' config.value ./declare-int-between-value.nix ./define-value-int-negative.nix
 
+# Check either types
+# types.either
+checkConfigOutput "42" config.value ./declare-either.nix ./define-value-int-positive.nix
+checkConfigOutput "\"24\"" config.value ./declare-either.nix ./define-value-string.nix
+# types.oneOf
+checkConfigOutput "42" config.value ./declare-oneOf.nix ./define-value-int-positive.nix
+checkConfigOutput "[ ]" config.value ./declare-oneOf.nix ./define-value-list.nix
+checkConfigOutput "\"24\"" config.value ./declare-oneOf.nix ./define-value-string.nix
+
 # Check mkForce without submodules.
 set -- config.enable ./declare-enable.nix ./define-enable.nix
 checkConfigOutput "true" "$@"

lib/tests/modules/declare-either.nix

@@ -0,0 +1,5 @@
+{ lib, ... }: {
+  options.value = lib.mkOption {
+    type = lib.types.either lib.types.int lib.types.str;
+  };
+}

lib/tests/modules/declare-oneOf.nix

@@ -0,0 +1,9 @@
+{ lib, ... }: {
+  options.value = lib.mkOption {
+    type = lib.types.oneOf [
+      lib.types.int
+      (lib.types.listOf lib.types.int)
+      lib.types.str
+    ];
+  };
+}

lib/tests/release.nix

@@ -1,11 +1,9 @@
 { pkgs ? import ((import ../.).cleanSource ../..) {} }:
 
-pkgs.stdenv.mkDerivation {
+pkgs.runCommandNoCC "nixpkgs-lib-tests" {
-  name = "nixpkgs-lib-tests";
+  buildInputs = [ pkgs.nix (import ./check-eval.nix) ];
-  buildInputs = [ pkgs.nix ];
   NIX_PATH="nixpkgs=${pkgs.path}";
-
+} ''
-  buildCommand = ''
     datadir="${pkgs.nix}/share"
     export TEST_ROOT=$(pwd)/test-tmp
     export NIX_BUILD_HOOK=
@@ -22,10 +20,5 @@ pkgs.stdenv.mkDerivation {
     cd ${pkgs.path}/lib/tests
     bash ./modules.sh
 
-    [[ "$(nix-instantiate --eval --strict misc.nix)" == "[ ]" ]]
-
-    [[ "$(nix-instantiate --eval --strict systems.nix)" == "[ ]" ]]
-
     touch $out
-  '';
+''
-}

lib/tests/systems.nix

@@ -12,19 +12,19 @@ let
     expected = lib.sort lib.lessThan y;
   };
 in with lib.systems.doubles; lib.runTests {
-  testall = mseteq all (linux ++ darwin ++ freebsd ++ openbsd ++ netbsd ++ illumos ++ wasi ++ windows);
+  testall = mseteq all (linux ++ darwin ++ freebsd ++ openbsd ++ netbsd ++ illumos ++ wasi ++ windows ++ embedded);
 
-  testarm = mseteq arm [ "armv5tel-linux" "armv6l-linux" "armv7l-linux" ];
+  testarm = mseteq arm [ "armv5tel-linux" "armv6l-linux" "armv7l-linux" "arm-none" "armv7a-darwin" ];
-  testi686 = mseteq i686 [ "i686-linux" "i686-freebsd" "i686-netbsd" "i686-openbsd" "i686-cygwin" "i686-windows" ];
+  testi686 = mseteq i686 [ "i686-linux" "i686-freebsd" "i686-netbsd" "i686-openbsd" "i686-cygwin" "i686-windows" "i686-none" "i686-darwin" ];
   testmips = mseteq mips [ "mipsel-linux" ];
-  testx86_64 = mseteq x86_64 [ "x86_64-linux" "x86_64-darwin" "x86_64-freebsd" "x86_64-openbsd" "x86_64-netbsd" "x86_64-cygwin" "x86_64-solaris" "x86_64-windows" ];
+  testx86_64 = mseteq x86_64 [ "x86_64-linux" "x86_64-darwin" "x86_64-freebsd" "x86_64-openbsd" "x86_64-netbsd" "x86_64-cygwin" "x86_64-solaris" "x86_64-windows" "x86_64-none" ];
 
   testcygwin = mseteq cygwin [ "i686-cygwin" "x86_64-cygwin" ];
-  testdarwin = mseteq darwin [ "x86_64-darwin" ];
+  testdarwin = mseteq darwin [ "x86_64-darwin" "i686-darwin" "aarch64-darwin" "armv7a-darwin" ];
   testfreebsd = mseteq freebsd [ "i686-freebsd" "x86_64-freebsd" ];
   testgnu = mseteq gnu (linux /* ++ kfreebsd ++ ... */);
   testillumos = mseteq illumos [ "x86_64-solaris" ];
-  testlinux = mseteq linux [ "i686-linux" "x86_64-linux" "armv5tel-linux" "armv6l-linux" "armv7l-linux" "aarch64-linux" "mipsel-linux" ];
+  testlinux = mseteq linux [ "aarch64-linux" "armv5tel-linux" "armv6l-linux" "armv7l-linux" "i686-linux" "mipsel-linux" "riscv32-linux" "riscv64-linux" "x86_64-linux" "powerpc64le-linux" ];
   testnetbsd = mseteq netbsd [ "i686-netbsd" "x86_64-netbsd" ];
   testopenbsd = mseteq openbsd [ "i686-openbsd" "x86_64-openbsd" ];
   testwindows = mseteq windows [ "i686-cygwin" "x86_64-cygwin" "i686-windows" "x86_64-windows" ];

lib/trivial.nix

@@ -134,7 +134,7 @@ rec {
      On each release the first letter is bumped and a new animal is chosen
      starting with that new letter.
   */
-  codeName = "Loris";
+  codeName = "Markhor";
 
   /* Returns the current nixpkgs version suffix as string. */
   versionSuffix =

lib/types.nix

@@ -42,7 +42,7 @@ rec {
   # Default type functor
   defaultFunctor = name: {
     inherit name;
-    type    = types."${name}" or null;
+    type    = types.${name} or null;
     wrapped = null;
     payload = null;
     binOp   = a: b: null;
@@ -107,11 +107,11 @@ rec {
       merge = mergeEqualOption;
     };
 
-    int = mkOptionType rec {
+    int = mkOptionType {
         name = "int";
         description = "signed integer";
         check = isInt;
-        merge = mergeOneOption;
+        merge = mergeEqualOption;
       };
 
     # Specialized subdomains of int
@@ -136,7 +136,7 @@ rec {
         sign = bit: range: ign (0 - (range / 2)) (range / 2 - 1)
           "signedInt${toString bit}" "${toString bit} bit signed integer";
 
-      in rec {
+      in {
         /* An int with a fixed range.
         *
         * Example:
@@ -172,18 +172,18 @@ rec {
     # Alias of u16 for a port number
     port = ints.u16;
 
-    float = mkOptionType rec {
+    float = mkOptionType {
         name = "float";
         description = "floating point number";
         check = isFloat;
-        merge = mergeOneOption;
+        merge = mergeEqualOption;
     };
 
     str = mkOptionType {
       name = "str";
       description = "string";
       check = isString;
-      merge = mergeOneOption;
+      merge = mergeEqualOption;
     };
 
     strMatching = pattern: mkOptionType {
@@ -217,7 +217,8 @@ rec {
 
     # Deprecated; should not be used because it quietly concatenates
     # strings, which is usually not what you want.
-    string = separatedString "";
+    string = warn "types.string is deprecated because it quietly concatenates strings"
+      (separatedString "");
 
     attrs = mkOptionType {
       name = "attrs";
@@ -243,7 +244,7 @@ rec {
       name = "path";
       # Hacky: there is no ‘isPath’ primop.
       check = x: builtins.substring 0 1 (toString x) == "/";
-      merge = mergeOneOption;
+      merge = mergeEqualOption;
     };
 
     # drop this in the future:
@@ -415,7 +416,7 @@ rec {
         name = "enum";
         description = "one of ${concatMapStringsSep ", " show values}";
         check = flip elem values;
-        merge = mergeOneOption;
+        merge = mergeEqualOption;
         functor = (defaultFunctor name) // { payload = values; binOp = a: b: unique (a ++ b); };
       };
 
@@ -443,6 +444,13 @@ rec {
       functor = (defaultFunctor name) // { wrapped = [ t1 t2 ]; };
     };
 
+    # Any of the types in the given list
+    oneOf = ts:
+      let
+        head' = if ts == [] then throw "types.oneOf needs to get at least one type in its argument" else head ts;
+        tail' = tail ts;
+      in foldl' either head' tail';
+
     # Either value of type `finalType` or `coercedType`, the latter is
     # converted to `finalType` using `coerceFunc`.
     coercedTo = coercedType: coerceFunc: finalType:

lib/versions.nix

@@ -7,7 +7,7 @@ let
 
 in
 
-rec {
+{
 
   /* Get the major version string from a string.
 

maintainers/scripts/hydra-eval-failures.py

@@ -11,13 +11,15 @@ import click
 import requests
 from pyquery import PyQuery as pq
 
+def map_dict (f, d):
+    for k,v in d.items():
+        d[k] = f(v)
 
 maintainers_json = subprocess.check_output([
-    'nix-instantiate', '-E', 'import ./maintainers/maintainer-list.nix {}', '--eval', '--json'
+    'nix-instantiate', '-A', 'lib.maintainers', '--eval', '--strict', '--json'
 ])
 maintainers = json.loads(maintainers_json)
-MAINTAINERS = {v: k for k, v in maintainers.items()}
+MAINTAINERS = map_dict(lambda v: v.get('github', None), maintainers)
-
 
 def get_response_text(url):
     return pq(requests.get(url).text)  # IO
@@ -38,30 +40,39 @@ def get_maintainers(attr_name):
             '-A',
             '.'.join(nixname[1:]) + '.meta',
             EVAL_FILE[nixname[0]],
+            '--arg',
+            'nixpkgs',
+            './.',
             '--json'])
         meta = json.loads(meta_json)
-        if meta.get('maintainers'):
+        return meta.get('maintainers', [])
-            return [MAINTAINERS[name] for name in meta['maintainers'] if MAINTAINERS.get(name)]
     except:
        return []
 
+def filter_github_users(maintainers):
+    github_only = []
+    for i in maintainers:
+        if i.get('github'):
+            github_only.append(i)
+    return github_only
+
 def print_build(table_row):
     a = pq(table_row)('a')[1]
     print("- [ ] [{}]({})".format(a.text, a.get('href')), flush=True)
-    
+
-    maintainers = get_maintainers(a.text)
+    job_maintainers = filter_github_users(get_maintainers(a.text))
-    if maintainers:
+    if job_maintainers:
-        print("  - maintainers: {}".format(", ".join(map(lambda u: '@' + u, maintainers))))
+        print("  - maintainers: {}".format(" ".join(map(lambda u: '@' + u.get('github'), job_maintainers))))
     # TODO: print last three persons that touched this file
     # TODO: pinpoint the diff that broke this build, or maybe it's transient or maybe it never worked?
-    
+
     sys.stdout.flush()
 
 @click.command()
 @click.option(
     '--jobset',
-    default="nixos/release-17.09",
+    default="nixos/release-19.09",
-    help='Hydra project like nixos/release-17.09')
+    help='Hydra project like nixos/release-19.09')
 def cli(jobset):
     """
     Given a Hydra project, inspect latest evaluation
@@ -93,6 +104,7 @@ def cli(jobset):
         print_build(tr)
 
 
+
 if __name__ == "__main__":
     try:
         cli()

maintainers/scripts/luarocks-config.lua

@@ -0,0 +1,4 @@
+rocks_servers = {
+	"https://luarocks.org"
+}
+version_check_on_fail = false

maintainers/scripts/luarocks-packages.csv

@@ -1,41 +1,70 @@
-# nix name, luarocks name, server, version/additionnal args
+# nix name, luarocks name, server, version,luaversion,maintainers
-ansicolors,
+alt-getopt,,,,,arobyn
-argparse,
+ansicolors,,,,,
-basexx,
+argparse,,,,,
-binaryheap,
+basexx,,,,,
-dkjson
+binaryheap,,,,,vcunat
-fifo
+bit32,,,,lua5_1,lblasc
-http
+busted,,,,,
-inspect
+cjson,lua-cjson,,,,
-ldoc
+compat53,,,,,vcunat
-lgi
+coxpcall,,,1.17.0-1,,
-lpeg_patterns
+cqueues,,,,,vcunat
-lpty
+cyrussasl,,,,,vcunat
-lrexlib-gnu,
+digestif,,http://luarocks.org/dev,,lua5_3,
-lrexlib-posix,
+dkjson,,,,,
-ltermbox,
+fifo,,,,,
-lua-cmsgpack,
+http,,,,,vcunat
-lua_cliargs,
+inspect,,,,,
-lua-iconv,
+ldoc,,,,,
-lua-messagepack,
+lgi,,,,,
-lua-term,
+ljsyscall,,,,lua5_1,lblasc
-lua-toml,
+lpeg,,,,,vyp
-luabitop,
+lpeg_patterns,,,,,
-luaevent,
+lpeglabel,,,,,
-luacheck
+lpty,,,,,
-luaffi,,http://luarocks.org/dev,
+lrexlib-gnu,,,,,
-luuid,
+lrexlib-pcre,,,,,vyp
-markdown,
+lrexlib-posix,,,,,
-penlight,
+ltermbox,,,,,
-rapidjson,
+lua-cmsgpack,,,,,
-say,
+lua-iconv,,,,,
-std__debug,std._debug,
+lua-lsp,,http://luarocks.org/dev,,,
-std_normalize,std.normalize,
+lua-messagepack,,,,,
-luv,
+lua-term,,,,,
-luasystem,
+lua-toml,,,,,
-mediator_lua,,http://luarocks.org/manifests/teto
+lua-zlib,,,,,koral
-mpack,,http://luarocks.org/manifests/teto
+lua_cliargs,,,,,
-nvim-client,,http://luarocks.org/manifests/teto
+luabitop,,,,,
-busted,,http://luarocks.org/manifests/teto
+luacheck,,,,,
-luassert,,http://luarocks.org/manifests/teto
+luadbi,,,,,
-coxpcall,,https://luarocks.org/manifests/hisham,1.17.0-1
+luadbi-mysql,,,,,
+luadbi-postgresql,,,,,
+luadbi-sqlite3,,,,,
+luaevent,,,,,
+luaexpat,,,1.3.0-1,,arobyn flosse
+luaffi,,http://luarocks.org/dev,,,
+luafilesystem,,,1.7.0-2,,flosse vcunat
+luaossl,,,,lua5_1,vcunat
+luaposix,,,,,vyp lblasc
+luasec,,,,,flosse
+luasocket,,,,,
+luasql-sqlite3,,,,,vyp
+luassert,,,,,
+luasystem,,,,,
+luautf8,,,,,pstn
+luazip,,,,,
+lua-yajl,,,,,pstn
+luuid,,,,,
+luv,,,,,
+markdown,,,,,
+mediator_lua,,,,,
+mpack,,,,,
+moonscript,,,,,arobyn
+nvim-client,,,,,
+penlight,,,,,
+rapidjson,,,,,
+say,,,,,
+std__debug,std._debug,,,,
+std_normalize,std.normalize,,,,
+stdlib,,,,,vyp

maintainers/scripts/nixpkgs-lint.nix

@@ -5,7 +5,7 @@ stdenv.mkDerivation {
 
   buildInputs = [ makeWrapper perl perlPackages.XMLSimple ];
 
-  unpackPhase = "true";
+  dontUnpack = true;
   buildPhase = "true";
 
   installPhase =

maintainers/scripts/update-discord

@@ -0,0 +1,36 @@
+#!/usr/bin/env bash
+# script to generate `pkgs/networking/instant-messengers/discord/default.nix`
+
+set -e
+exec >${1:?usage: $0 <output-file>}
+
+cat <<EOF
+{ branch ? "stable", pkgs }:
+
+let
+  inherit (pkgs) callPackage fetchurl;
+in {
+EOF
+
+for branch in "" ptb canary; do
+    url=$(curl -sI "https://discordapp.com/api/download${branch:+/}${branch}?platform=linux&format=tar.gz" | grep -oP 'location: \K\S+')
+    version=${url##https://dl*.discordapp.net/apps/linux/}
+    version=${version%%/*.tar.gz}
+    echo "  ${branch:-stable} = callPackage ./base.nix {"
+    echo "    pname = \"discord${branch:+-}${branch}\";"
+    case $branch in
+        "") suffix="" ;;
+        ptb) suffix="PTB" ;;
+        canary) suffix="Canary" ;;
+    esac
+    echo "    binaryName = \"Discord${suffix}\";"
+    echo "    desktopName = \"Discord${suffix:+ }${suffix}\";"
+    echo "    version = \"${version}\";"
+    echo "    src = fetchurl {"
+    echo "      url = \"${url}\";"
+    echo "      sha256 = \"$(nix-prefetch-url "$url")\";"
+    echo "    };"
+    echo "  };"
+done
+
+echo "}.\${branch}"

maintainers/scripts/update-luarocks-packages

@@ -1,5 +1,5 @@
 #!/usr/bin/env nix-shell
-#!nix-shell -p nix-prefetch-scripts luarocks-nix -i bash
+#!nix-shell update-luarocks-shell.nix -i bash
 
 # You'll likely want to use
 # ``
@@ -8,48 +8,53 @@
 # to update all libraries in that folder.
 # to debug, redirect stderr to stdout with 2>&1
 
-
 # stop the script upon C-C
 set -eu -o pipefail
 
-if [ $# -lt 1 ]; then
-	print_help
-    exit 1
-fi
-
 CSV_FILE="maintainers/scripts/luarocks-packages.csv"
 TMP_FILE="$(mktemp)"
+# Set in the update-luarocks-shell.nix
+NIXPKGS_PATH="$LUAROCKS_NIXPKGS_PATH"
+export LUAROCKS_CONFIG="$NIXPKGS_PATH/maintainers/scripts/luarocks-config.lua"
 
-exit_trap()
+# 10 is a pretty arbitrary number of simultaneous jobs, but it is generally
-{
+# impolite to hit a webserver with *too* many simultaneous connections :)
-  local lc="$BASH_COMMAND" rc=$?
+PARALLEL_JOBS=10
-  test $rc -eq 0 || echo -e "*** error $rc: $lc.\nGenerated temporary file in $TMP_FILE" >&2
+
+exit_trap() {
+    local lc="$BASH_COMMAND" rc=$?
+    test $rc -eq 0 || echo -e "*** error $rc: $lc.\nGenerated temporary file in $TMP_FILE" >&2
 }
-trap exit_trap EXIT
 
 print_help() {
     echo "Usage: $0 <GENERATED_FILE>"
     echo "(most likely pkgs/development/lua-modules/generated-packages.nix)"
-	echo ""
+    echo ""
-	echo " -c <CSV_FILE> to set the list of luarocks package to generate"
+    echo " -c <CSV_FILE> to set the list of luarocks package to generate"
-	exit 1
+    exit 1
 }
 
+if [ $# -lt 1 ]; then
+    print_help
+    exit 1
+fi
+
+trap exit_trap EXIT
 
 while getopts ":hc:" opt; do
-  case $opt in
+    case $opt in
     h)
-      print_help
+        print_help
-      ;;
+        ;;
     c)
-      echo "Loading package list from $OPTARG !" >&2
+        echo "Loading package list from $OPTARG !" >&2
-      CSV_FILE="$OPTARG"
+        CSV_FILE="$OPTARG"
-      ;;
+        ;;
     \?)
-      echo "Invalid option: -$OPTARG" >&2
+        echo "Invalid option: -$OPTARG" >&2
-      ;;
+        ;;
-  esac
+    esac
-  shift $((OPTIND-1))
+    shift $((OPTIND - 1))
 done
 
 GENERATED_NIXFILE="$1"
@@ -72,43 +77,60 @@ FOOTER="
 /* GENERATED */
 "
 
-
+function convert_pkg() {
-function convert_pkg () {
     nix_pkg_name="$1"
     lua_pkg_name="$2"
-    server="${3:+--server=$3}"
+    server="$3"
-    version="${4:-}"
+    pkg_version="$4"
+    lua_version="$5"
+    maintainers="$6"
 
-    echo "looking at $lua_pkg_name (version $version) from server [$server]" >&2
+    if [ "${nix_pkg_name:0:1}" == "#" ]; then
-    cmd="luarocks nix $server $lua_pkg_name $version"
+        echo "Skipping comment ${*}" >&2
-    drv="$nix_pkg_name = $($cmd)"
+        return
-    if [ $? -ne 0 ]; then
+    fi
-        echo "Failed to convert $pkg" >&2
+    if [ -z "$lua_pkg_name" ]; then
-        echo "$drv" >&2
+        echo "Using nix_name as lua_pkg_name for '$nix_pkg_name'" >&2
+        lua_pkg_name="$nix_pkg_name"
+    fi
+
+    echo "Building expression for $lua_pkg_name (version $pkg_version) from server [$server]" >&2
+    luarocks_args=(nix)
+    if [[ -n $server ]]; then
+        luarocks_args+=("--only-server=$server")
+    fi
+    if [[ -n $maintainers ]]; then
+        luarocks_args+=("--maintainers=$maintainers")
+    fi
+    if [[ -n $lua_version ]]; then
+        lua_drv_path=$(nix-build --no-out-link "$NIXPKGS_PATH" -A "$lua_version")
+        luarocks_args+=("--lua-dir=$lua_drv_path/bin")
+    fi
+    luarocks_args+=("$lua_pkg_name")
+    if [[ -n $pkg_version ]]; then
+        luarocks_args+=("$pkg_version")
+    fi
+    echo "Running 'luarocks ${luarocks_args[*]}'" >&2
+    if drv="$nix_pkg_name = $(luarocks "${luarocks_args[@]}")"; then
+        echo "$drv"
     else
-        echo "$drv" | tee -a "$TMP_FILE"
+        echo "Failed to convert $nix_pkg_name" >&2
+        return 1
     fi
 }
 
 # params needed when called via callPackage
 echo "$HEADER" | tee "$TMP_FILE"
 
-# list of packages with format
+# Ensure parallel can run our bash function
-# name,server,version
+export -f convert_pkg
-while IFS=, read -r nix_pkg_name lua_pkg_name server version
+export SHELL=bash
-do
+# Read each line in the csv file and run convert_pkg for each, in parallel
-    if [ "${nix_pkg_name:0:1}" == "#" ]; then
+parallel --group --keep-order --halt now,fail=1 --jobs "$PARALLEL_JOBS" --colsep ',' convert_pkg {} <"$CSV_FILE" | tee -a "$TMP_FILE"
-        echo "Skipping comment ${nix_pkg_name}" >&2
-        continue
-    fi
-    if [ -z "$lua_pkg_name" ]; then
-        echo "Using nix_name as lua_pkg_name" >&2
-        lua_pkg_name="$nix_pkg_name"
-    fi
-    convert_pkg "$nix_pkg_name" "$lua_pkg_name" "$server" "$version"
-done < "$CSV_FILE"
 
 # close the set
 echo "$FOOTER" | tee -a "$TMP_FILE"
 
 cp "$TMP_FILE" "$GENERATED_NIXFILE"
+
+# vim: set ts=4 sw=4 ft=sh:

maintainers/scripts/update-luarocks-shell.nix

@@ -0,0 +1,9 @@
+{ nixpkgs ? import ../.. { }
+}:
+with nixpkgs;
+mkShell {
+  buildInputs = [
+    bash luarocks-nix nix-prefetch-scripts parallel
+  ];
+  LUAROCKS_NIXPKGS_PATH = toString nixpkgs.path;
+}

maintainers/scripts/update-ruby-packages

@@ -0,0 +1,13 @@
+#!/usr/bin/env nix-shell
+#!nix-shell -i bash -p bundler bundix
+
+set -euf -o pipefail
+
+(
+  cd pkgs/development/ruby-modules/with-packages
+  rm -f gemset.nix Gemfile.lock
+  bundle lock
+  bundix
+  mv gemset.nix ../../../top-level/ruby-packages.nix
+  rm -f Gemfile.lock
+)

maintainers/scripts/update.nix

@@ -20,7 +20,9 @@ let
       in
         [x] ++ nubOn f xs;
 
-  pkgs = import ./../../default.nix { };
+  pkgs = import ./../../default.nix {
+    overlays = [];
+  };
 
   packagesWith = cond: return: set:
     nubOn (pkg: pkg.updateScript)
@@ -67,9 +69,12 @@ let
     let
       attrSet = pkgs.lib.attrByPath (pkgs.lib.splitString "." path) null pkgs;
     in
-      packagesWith (name: pkg: builtins.hasAttr "updateScript" pkg)
+      if attrSet == null then
-                     (name: pkg: pkg)
+        builtins.throw "Attribute path `${path}` does not exists."
-                     attrSet;
+      else
+        packagesWith (name: pkg: builtins.hasAttr "updateScript" pkg)
+                       (name: pkg: pkg)
+                       attrSet;
 
   packageByName = name:
     let
@@ -122,9 +127,17 @@ let
   packageData = package: {
     name = package.name;
     pname = (builtins.parseDrvName package.name).name;
-    updateScript = pkgs.lib.toList package.updateScript;
+    updateScript = map builtins.toString (pkgs.lib.toList package.updateScript);
   };
 
+  packagesJson = pkgs.writeText "packages.json" (builtins.toJSON (map packageData packages));
+
+  optionalArgs =
+    pkgs.lib.optional (max-workers != null) "--max-workers=${max-workers}"
+    ++ pkgs.lib.optional (keep-going == "true") "--keep-going";
+
+  args = [ packagesJson ] ++ optionalArgs;
+
 in pkgs.stdenv.mkDerivation {
   name = "nixpkgs-update-script";
   buildCommand = ''
@@ -139,6 +152,6 @@ in pkgs.stdenv.mkDerivation {
   '';
   shellHook = ''
     unset shellHook # do not contaminate nested shells
-    exec ${pkgs.python3.interpreter} ${./update.py} ${pkgs.writeText "packages.json" (builtins.toJSON (map packageData packages))}${pkgs.lib.optionalString (max-workers != null) " --max-workers=${max-workers}"}${pkgs.lib.optionalString (keep-going == "true") " --keep-going"}
+    exec ${pkgs.python3.interpreter} ${./update.py} ${builtins.concatStringsSep " " args}
   '';
 }

nixos/doc/manual/Makefile

@@ -24,7 +24,7 @@ fix-misc-xml:
 clean:
 	rm -f manual-combined.xml generated
 
-generated: ./options-to-docbook.xsl
+generated:
 	nix-build ../../release.nix \
 		--attr manualGeneratedSources.x86_64-linux \
 		--out-link ./generated

nixos/doc/manual/administration/cleaning-store.xml

@@ -11,12 +11,12 @@
   Nix’s <emphasis>garbage collector</emphasis> to remove old, unreferenced
   packages. This is easy:
 <screen>
-$ nix-collect-garbage
+<prompt>$ </prompt>nix-collect-garbage
 </screen>
   Alternatively, you can use a systemd unit that does the same in the
   background:
 <screen>
-# systemctl start nix-gc.service
+<prompt># </prompt>systemctl start nix-gc.service
 </screen>
   You can tell NixOS in <filename>configuration.nix</filename> to run this unit
   automatically at certain points in time, for instance, every night at 03:15:
@@ -31,11 +31,11 @@ $ nix-collect-garbage
   configurations. The following command deletes old roots, removing the ability
   to roll back to them:
 <screen>
-$ nix-collect-garbage -d
+<prompt>$ </prompt>nix-collect-garbage -d
 </screen>
   You can also do this for specific profiles, e.g.
 <screen>
-$ nix-env -p /nix/var/nix/profiles/per-user/eelco/profile --delete-generations old
+<prompt>$ </prompt>nix-env -p /nix/var/nix/profiles/per-user/eelco/profile --delete-generations old
 </screen>
   Note that NixOS system configurations are stored in the profile
   <filename>/nix/var/nix/profiles/system</filename>.
@@ -45,7 +45,7 @@ $ nix-env -p /nix/var/nix/profiles/per-user/eelco/profile --delete-generations o
   Nix store) is to run Nix’s store optimiser, which seeks out identical files
   in the store and replaces them with hard links to a single copy.
 <screen>
-$ nix-store --optimise
+<prompt>$ </prompt>nix-store --optimise
 </screen>
   Since this command needs to read the entire Nix store, it can take quite a
   while to finish.

nixos/doc/manual/administration/container-networking.xml

@@ -11,10 +11,10 @@
   <literal>10.233.0.0/16</literal>. You can get the container’s IPv4 address
   as follows:
 <screen>
-# nixos-container show-ip foo
+<prompt># </prompt>nixos-container show-ip foo
 10.233.4.2
 
-$ ping -c1 10.233.4.2
+<prompt>$ </prompt>ping -c1 10.233.4.2
 64 bytes from 10.233.4.2: icmp_seq=1 ttl=64 time=0.106 ms
 </screen>
  </para>

nixos/doc/manual/administration/control-groups.xml

@@ -16,7 +16,7 @@
   <literal>systemd</literal> hierarchy, which is what systemd uses to keep
   track of the processes belonging to each service or user session:
 <screen>
-$ systemd-cgls
+<prompt>$ </prompt>systemd-cgls
 ├─user
 │ └─eelco
 │   └─c1

nixos/doc/manual/administration/imperative-containers.xml

@@ -29,6 +29,13 @@
   <xref linkend="opt-services.openssh.enable"/> = true;
   <link linkend="opt-users.users._name__.openssh.authorizedKeys.keys">users.users.root.openssh.authorizedKeys.keys</link> = ["ssh-dss AAAAB3N…"];
 '
+</screen>
+  By default the next free address in the <literal>10.233.0.0/16</literal> subnet will be chosen
+  as container IP. This behavior can be altered by setting <literal>--host-address</literal> and
+  <literal>--local-address</literal>:
+<screen>
+# nixos-container create test --config-file test-container.nix \
+    --local-address 10.235.1.2 --host-address 10.235.1.1
 </screen>
  </para>
 

nixos/doc/manual/administration/logging.xml

@@ -11,14 +11,14 @@
   The command <literal>journalctl</literal> allows you to see the contents of
   the journal. For example,
 <screen>
-$ journalctl -b
+<prompt>$ </prompt>journalctl -b
 </screen>
   shows all journal entries since the last reboot. (The output of
   <command>journalctl</command> is piped into <command>less</command> by
   default.) You can use various options and match operators to restrict output
   to messages of interest. For instance, to get all messages from PostgreSQL:
 <screen>
-$ journalctl -u postgresql.service
+<prompt>$ </prompt>journalctl -u postgresql.service
 -- Logs begin at Mon, 2013-01-07 13:28:01 CET, end at Tue, 2013-01-08 01:09:57 CET. --
 ...
 Jan 07 15:44:14 hagbard postgres[2681]: [2-1] LOG:  database system is shut down
@@ -29,7 +29,7 @@ Jan 07 15:45:13 hagbard postgres[2500]: [1-1] LOG:  database system is ready to
   Or to get all messages since the last reboot that have at least a
   “critical” severity level:
 <screen>
-$ journalctl -b -p crit
+<prompt>$ </prompt>journalctl -b -p crit
 Dec 17 21:08:06 mandark sudo[3673]: pam_unix(sudo:auth): auth could not identify password for [alice]
 Dec 29 01:30:22 mandark kernel[6131]: [1053513.909444] CPU6: Core temperature above threshold, cpu clock throttled (total events = 1)
 </screen>

nixos/doc/manual/administration/rollback.xml

@@ -33,7 +33,7 @@
   where <replaceable>N</replaceable> is the number of the NixOS system
   configuration. To get a list of the available configurations, do:
 <screen>
-$ ls -l /nix/var/nix/profiles/system-*-link
+<prompt>$ </prompt>ls -l /nix/var/nix/profiles/system-*-link
 <replaceable>...</replaceable>
 lrwxrwxrwx 1 root root 78 Aug 12 13:54 /nix/var/nix/profiles/system-268-link -> /nix/store/202b...-nixos-13.07pre4932_5a676e4-4be1055
 </screen>

nixos/doc/manual/administration/service-mgmt.xml

@@ -21,7 +21,7 @@
   <command>systemd</command>. Without any arguments, it shows the status of
   active units:
 <screen>
-$ systemctl
+<prompt>$ </prompt>systemctl
 -.mount          loaded active mounted   /
 swapfile.swap    loaded active active    /swapfile
 sshd.service     loaded active running   SSH Daemon
@@ -33,7 +33,7 @@ graphical.target loaded active active    Graphical Interface
   You can ask for detailed status information about a unit, for instance, the
   PostgreSQL database service:
 <screen>
-$ systemctl status postgresql.service
+<prompt>$ </prompt>systemctl status postgresql.service
 postgresql.service - PostgreSQL Server
           Loaded: loaded (/nix/store/pn3q73mvh75gsrl8w7fdlfk3fq5qm5mw-unit/postgresql.service)
           Active: active (running) since Mon, 2013-01-07 15:55:57 CET; 9h ago

nixos/doc/manual/administration/store-corruption.xml

@@ -18,7 +18,7 @@
   If the corruption is in a path in the closure of the NixOS system
   configuration, you can fix it by doing
 <screen>
-# nixos-rebuild switch --repair
+<prompt># </prompt>nixos-rebuild switch --repair
 </screen>
   This will cause Nix to check every path in the closure, and if its
   cryptographic hash differs from the hash recorded in Nix’s database, the
@@ -28,7 +28,7 @@
  <para>
   You can also scan the entire Nix store for corrupt paths:
 <screen>
-# nix-store --verify --check-contents --repair
+<prompt># </prompt>nix-store --verify --check-contents --repair
 </screen>
   Any corrupt paths will be redownloaded if they’re available in a binary
   cache; otherwise, they cannot be repaired.

nixos/doc/manual/administration/user-sessions.xml

@@ -10,7 +10,7 @@
   allows querying and manipulating user sessions. For instance, to list all
   user sessions:
 <screen>
-$ loginctl
+<prompt>$ </prompt>loginctl
    SESSION        UID USER             SEAT
         c1        500 eelco            seat0
         c3          0 root             seat0
@@ -21,7 +21,7 @@ $ loginctl
   devices attached to the system; usually, there is only one seat.) To get
   information about a session:
 <screen>
-$ loginctl session-status c3
+<prompt>$ </prompt>loginctl session-status c3
 c3 - root (0)
            Since: Tue, 2013-01-08 01:17:56 CET; 4min 42s ago
           Leader: 2536 (login)

nixos/doc/manual/configuration/ad-hoc-packages.xml

@@ -9,7 +9,7 @@
   With the command <command>nix-env</command>, you can install and uninstall
   packages from the command line. For instance, to install Mozilla Thunderbird:
 <screen>
-$ nix-env -iA nixos.thunderbird</screen>
+<prompt>$ </prompt>nix-env -iA nixos.thunderbird</screen>
   If you invoke this as root, the package is installed in the Nix profile
   <filename>/nix/var/nix/profiles/default</filename> and visible to all users
   of the system; otherwise, the package ends up in
@@ -25,7 +25,7 @@ $ nix-env -iA nixos.thunderbird</screen>
   Packages come from the NixOS channel. You typically upgrade a package by
   updating to the latest version of the NixOS channel:
 <screen>
-$ nix-channel --update nixos
+<prompt>$ </prompt>nix-channel --update nixos
 </screen>
   and then running <literal>nix-env -i</literal> again. Other packages in the
   profile are <emphasis>not</emphasis> affected; this is the crucial difference
@@ -34,21 +34,21 @@ $ nix-channel --update nixos
   their current versions in the NixOS channel. You can however upgrade all
   packages for which there is a newer version by doing:
 <screen>
-$ nix-env -u '*'
+<prompt>$ </prompt>nix-env -u '*'
 </screen>
  </para>
 
  <para>
   A package can be uninstalled using the <option>-e</option> flag:
 <screen>
-$ nix-env -e thunderbird
+<prompt>$ </prompt>nix-env -e thunderbird
 </screen>
  </para>
 
  <para>
   Finally, you can roll back an undesirable <command>nix-env</command> action:
 <screen>
-$ nix-env --rollback
+<prompt>$ </prompt>nix-env --rollback
 </screen>
  </para>
 

nixos/doc/manual/configuration/adding-custom-packages.xml

@@ -14,8 +14,8 @@
 xlink:href="http://nixos.org/nixpkgs/manual">Nixpkgs
   manual</link>. In short, you clone Nixpkgs:
 <screen>
-$ git clone https://github.com/NixOS/nixpkgs
+<prompt>$ </prompt>git clone https://github.com/NixOS/nixpkgs
-$ cd nixpkgs
+<prompt>$ </prompt>cd nixpkgs
 </screen>
   Then you write and test the package as described in the Nixpkgs manual.
   Finally, you add it to <literal>environment.systemPackages</literal>, e.g.
@@ -65,8 +65,8 @@ stdenv.mkDerivation rec {
 </programlisting>
   This allows testing the package easily:
 <screen>
-$ nix-build my-hello.nix
+<prompt>$ </prompt>nix-build my-hello.nix
-$ ./result/bin/hello
+<prompt>$ </prompt>./result/bin/hello
 Hello, world!
 </screen>
  </para>

nixos/doc/manual/configuration/customizing-packages.xml

@@ -24,8 +24,8 @@
  <para>
   Apart from high-level options, it’s possible to tweak a package in almost
   arbitrary ways, such as changing or disabling dependencies of a package. For
-  instance, the Emacs package in Nixpkgs by default has a dependency on GTK+ 2.
+  instance, the Emacs package in Nixpkgs by default has a dependency on GTK 2.
-  If you want to build it against GTK+ 3, you can specify that as follows:
+  If you want to build it against GTK 3, you can specify that as follows:
 <programlisting>
 <xref linkend="opt-environment.systemPackages"/> = [ (pkgs.emacs.override { gtk = pkgs.gtk3; }) ];
 </programlisting>
@@ -33,7 +33,7 @@
   function that produces Emacs, with the original arguments amended by the set
   of arguments specified by you. So here the function argument
   <varname>gtk</varname> gets the value <literal>pkgs.gtk3</literal>, causing
-  Emacs to depend on GTK+ 3. (The parentheses are necessary because in Nix,
+  Emacs to depend on GTK 3. (The parentheses are necessary because in Nix,
   function application binds more weakly than list construction, so without
   them, <xref linkend="opt-environment.systemPackages"/> would be a list with
   two elements.)

nixos/doc/manual/configuration/declarative-packages.xml

@@ -22,7 +22,7 @@
  <para>
   You can get a list of the available packages as follows:
 <screen>
-$ nix-env -qaP '*' --description
+<prompt>$ </prompt>nix-env -qaP '*' --description
 nixos.firefox   firefox-23.0   Mozilla Firefox - the browser, reloaded
 <replaceable>...</replaceable>
 </screen>

nixos/doc/manual/configuration/ipv6-config.xml

@@ -39,7 +39,7 @@
 <xref linkend="opt-networking.defaultGateway6"/> = {
   address = "fe00::1";
   interface = "enp0s3";
-}
+};
 </programlisting>
  </para>
 

nixos/doc/manual/configuration/matrix.xml

@@ -33,91 +33,91 @@
    <link xlink:href="https://github.com/matrix-org/synapse#synapse-installation">
    installation instructions of Synapse </link>.
 <programlisting>
+let
+  fqdn =
     let
-      fqdn =
+      join = hostName: domain: hostName + optionalString (domain != null) ".${domain}";
-        let
+    in join config.networking.hostName config.networking.domain;
-          join = hostName: domain: hostName + optionalString (domain != null) ".${domain}";
+in {
-        in join config.networking.hostName config.networking.domain;
+  networking = {
-    in {
+    hostName = "myhostname";
-      networking = {
+    domain = "example.org";
-        hostName = "myhostname";
+  };
-        domain = "example.org";
+  networking.firewall.allowedTCPPorts = [ 80 443 ];
-      };
-      networking.firewall.allowedTCPPorts = [ 80 443 ];
 
-      services.nginx = {
+  services.nginx = {
-        enable = true;
+    enable = true;
-        # only recommendedProxySettings and recommendedGzipSettings are strictly required,
+    # only recommendedProxySettings and recommendedGzipSettings are strictly required,
-        # but the rest make sense as well
+    # but the rest make sense as well
-        recommendedTlsSettings = true;
+    recommendedTlsSettings = true;
-        recommendedOptimisation = true;
+    recommendedOptimisation = true;
-        recommendedGzipSettings = true;
+    recommendedGzipSettings = true;
-        recommendedProxySettings = true;
+    recommendedProxySettings = true;
 
-        virtualHosts = {
+    virtualHosts = {
-          # This host section can be placed on a different host than the rest,
+      # This host section can be placed on a different host than the rest,
-          # i.e. to delegate from the host being accessible as ${config.networking.domain}
+      # i.e. to delegate from the host being accessible as ${config.networking.domain}
-          # to another host actually running the Matrix homeserver.
+      # to another host actually running the Matrix homeserver.
-          "${config.networking.domain}" = {
+      "${config.networking.domain}" = {
-            locations."= /.well-known/matrix/server".extraConfig =
+        locations."= /.well-known/matrix/server".extraConfig =
-              let
+          let
-                # use 443 instead of the default 8448 port to unite
+            # use 443 instead of the default 8448 port to unite
-                # the client-server and server-server port for simplicity
+            # the client-server and server-server port for simplicity
-                server = { "m.server" = "${fqdn}:443"; };
+            server = { "m.server" = "${fqdn}:443"; };
-              in ''
+          in ''
-                add_header Content-Type application/json;
+            add_header Content-Type application/json;
-                return 200 '${builtins.toJSON server}';
+            return 200 '${builtins.toJSON server}';
-              '';
+          '';
-            locations."= /.well-known/matrix/client".extraConfig =
+        locations."= /.well-known/matrix/client".extraConfig =
-              let
+          let
-                client = {
+            client = {
-                  "m.homeserver" =  { "base_url" = "https://${fqdn}"; };
+              "m.homeserver" =  { "base_url" = "https://${fqdn}"; };
-                  "m.identity_server" =  { "base_url" = "https://vector.im"; };
+              "m.identity_server" =  { "base_url" = "https://vector.im"; };
-                };
-              # ACAO required to allow riot-web on any URL to request this json file
-              in ''
-                add_header Content-Type application/json;
-                add_header Access-Control-Allow-Origin *;
-                return 200 '${builtins.toJSON client}';
-              '';
-          };
-
-          # Reverse proxy for Matrix client-server and server-server communication
-          ${fqdn} = {
-            enableACME = true;
-            forceSSL = true;
-
-            # Or do a redirect instead of the 404, or whatever is appropriate for you.
-            # But do not put a Matrix Web client here! See the Riot Web section below.
-            locations."/".extraConfig = ''
-              return 404;
-            '';
-
-            # forward all Matrix API calls to the synapse Matrix homeserver
-            locations."/_matrix" = {
-              proxyPass = "http://[::1]:8008";
             };
-          };
+          # ACAO required to allow riot-web on any URL to request this json file
+          in ''
+            add_header Content-Type application/json;
+            add_header Access-Control-Allow-Origin *;
+            return 200 '${builtins.toJSON client}';
+          '';
+      };
+
+      # Reverse proxy for Matrix client-server and server-server communication
+      ${fqdn} = {
+        enableACME = true;
+        forceSSL = true;
+
+        # Or do a redirect instead of the 404, or whatever is appropriate for you.
+        # But do not put a Matrix Web client here! See the Riot Web section below.
+        locations."/".extraConfig = ''
+          return 404;
+        '';
+
+        # forward all Matrix API calls to the synapse Matrix homeserver
+        locations."/_matrix" = {
+          proxyPass = "http://[::1]:8008"; # without a trailing /
         };
       };
-      services.matrix-synapse = {
-        enable = true;
-        server_name = config.networking.domain;
-        listeners = [
-          {
-            port = 8008;
-            bind_address = "::1";
-            type = "http";
-            tls = false;
-            x_forwarded = true;
-            resources = [
-              { names = [ "client" "federation" ]; compress = false; }
-            ];
-          }
-        ];
-      };
     };
-   </programlisting>
+  };
+  services.matrix-synapse = {
+    enable = true;
+    server_name = config.networking.domain;
+    listeners = [
+      {
+        port = 8008;
+        bind_address = "::1";
+        type = "http";
+        tls = false;
+        x_forwarded = true;
+        resources = [
+          { names = [ "client" "federation" ]; compress = false; }
+        ];
+      }
+    ];
+  };
+};
+</programlisting>
   </para>
 
   <para>
@@ -141,15 +141,15 @@
    <option>services.matrix-synapse.registration_shared_secret</option>. To
    create a new user or admin, run the following after you have set the secret
    and have rebuilt NixOS:
-<programlisting>
+<screen>
-    $ nix run nixpkgs.matrix-synapse
+<prompt>$ </prompt>nix run nixpkgs.matrix-synapse
-    $ register_new_matrix_user -k &lt;your-registration-shared-secret&gt; http://localhost:8008
+<prompt>$ </prompt>register_new_matrix_user -k <replaceable>your-registration-shared-secret</replaceable> http://localhost:8008
-    New user localpart: &lt;your-username&gt;
+<prompt>New user localpart: </prompt><replaceable>your-username</replaceable>
-    Password:
+<prompt>Password:</prompt>
-    Confirm password:
+<prompt>Confirm password:</prompt>
-    Make admin [no]:
+<prompt>Make admin [no]:</prompt>
-    Success!
+Success!
-   </programlisting>
+</screen>
    In the example, this would create a user with the Matrix Identifier
    <literal>@your-username:example.org</literal>. Note that the registration
    secret ends up in the nix store and therefore is world-readable by any user
@@ -177,16 +177,16 @@
    Matrix Now!</link> for a list of existing clients and their supported
    featureset.
 <programlisting>
-     services.nginx.virtualHosts."riot.${fqdn}" = {
+services.nginx.virtualHosts."riot.${fqdn}" = {
-       enableACME = true;
+  enableACME = true;
-       forceSSL = true;
+  forceSSL = true;
-       serverAliases = [
+  serverAliases = [
-        "riot.${config.networking.domain}"
+    "riot.${config.networking.domain}"
-       ];
+  ];
 
-       root = pkgs.riot-web;
+  root = pkgs.riot-web;
-     };
+};
-   </programlisting>
+</programlisting>
   </para>
 
   <para>