Commit Graph

423990 Commits

Author SHA1 Message Date
Yarny0
075d112131 foomatic-db-ppds (cups ppd files): init
`foomatic-db-ppds` uses the Foomatic database from the packages
`foomatic-db` and -- optionally -- `foomatic-db-nonfree`
and the perl modules from `foomatic-db-engine`
to generate about 8,800 ppd files.

The general structure of the build recipe is as follows:

* Merge `foomatic-db` and `foomatic-db-nonfree` into
  one package that represents the Foomatic database.
  The package `foomatic-db-nonfree` is optional
  as it taints the result license-wise;
  it will only be used if `withNonfreeDb`
  is to to `true` in the `callPackage` call.
  We create a tiny setup hook script that provides the combined
  database and sets an environment variable pointing to the
  database direcotry, which is expected by the foomatic engine.

* The final package's license and version are computed
  from the licenses and versions of the database packages.
  The license is set to `free` if each database-providing
  package has a free license, and to `unfree` otherwise.
  The version is simply the highest version
  of the database-providing packages.

* The final package uses `foomatic-compiledb`
  from the `foomatic-db-engine` package to extract
  all ppd files from the database packages.
  `patchPpdFilesHook` is used to patch most
  executable invocations in the ppd files
  so that they point to nix store paths.
  Finally, ppd files are gzipped to reduce storage
  (from about 550 MiB to 90 MiB installed).

The "nonfree" version of the package, i.e. the version that is
based on `foomatic-db-nonfree` in addition to `foomatic-db`,
contains about 120 additional ppd files
compared to the "free" version.
Since the "free" version already produces about 8,700
ppd files and hydra won't build the "nonfree" version,
the commit adds two package variables to `all-packages.nix`:

* `foomatic-db-ppds` is based on `foomatic-db` only
* `foomatic-db-ppds-withNonfreeDb`
  is also based on `foomaitc-db-nonfree`

The package introduced by this commit
is the result of combining other packages;
it is not the build product of a simple source tarball.
While it would also be possible to perform the ppd file
generation directly in the build process of the database
packages, this would yield further complexity as the
`foomatic-db-nonfree` package needs to be combined with the
`foomatic-db` package before ppd file extraction is possible.

There is no upstream product with a name that
could/should be used for the `name` attribute,
the variable name, or for the filename in nixpkgs.
Similar packages have different names across distributions:

* https://repology.org/projects/?search=openprinting
* https://repology.org/projects/?search=foomatic

The name `foomatic-db-ppds` seems to be most common
(albeit not really *that* common):

* https://repology.org/project/foomatic-db-ppds/versions

At least openSUSE splits their corresponding
package into multiple "binary" packages
(similar to our multi-output packages):

* https://build.opensuse.org/package/binaries/Printing/OpenPrintingPPDs/openSUSE_Tumbleweed

I considered something similar.
However, after doing some statistics,
I concluded that it's not worth the effort:
The biggest dependencies (`perl` and `cups-filters`) are
already present on most NixOS systems, and they cannot
be "split away" easily since it cannot be
done along a canonical line (e.g. printer driver).
Splitting directly by dependency risks that ppd files
unexpectedly "move from output to output" on package updates;
disappearing ppd files can be quite annoying for package users.
2022-11-06 19:28:15 +01:00
Yarny0
156cc612ff foomatic-db-nonfree (cups ppd files): init at unstable/2015-06-05
`foomatic-db-nonfree` contains -- similar to its sister
package `foomatic-db` -- knowledge about printers,
drivers, and driver options from OpenPrinting in xml files.
It needs to be combined with the `foomatic-db`
package to yield a working database.
It also provides about 100 (gzipped) ppd files.
In contrast to `foomatic-db`, this package contains
data that is provided under non-free licenses.

Quoting
https://github.com/OpenPrinting/foomatic-db-nonfree/blob/master/README

> This is a repository of PPD and Foomatic XML files that may
> have restrictions that keep them from being used on a variety
> of machines for licensing and other non-technical reasons.

ppd files can be found in
`$out/share/cups/model/foomatic-db-nonfree`.
The subdirectory `foomatic-db-nofree` is used to avoid conflicts
with other packages that might be combined with the package
`foomatic-db-nonfree` in `config.services.printing.drivers`.

ppd files in this package are *not* patched
to call executables from the nix store:
The only executable "rastertophaser6100" that is
called from ppd files isn't available in nixpkgs.

There is a daily snapshot of a source archive available at
https://www.openprinting.org/download/foomatic/ .
However, these files rotate daily and
cannot be used as a stable download source.
So we rely on OpenPrinting's Github repository
instead and pinpoint a fresh commit.
Note that the current version is from 2015,
so updates are unlikely.
2022-11-06 19:28:15 +01:00
Yarny0
90a8a78e7c foomatic-db (cups ppd files): init at unstable/2022-10-03
`foomatic-db` contains the collected knowledge about printers,
drivers, and driver options from OpenPrinting in xml files.
It also provides thousands of ppd files.
The build process patches those files to reference executables
in the nix store and gzips them to reduce storage
(from about 670 MiB to 90 MiB installed).

In contrast to the sister package `foomatic-db-nonfree`
(packaged in a follow-up commit),
this package only includes files published under a free license.
Most files are published under the GPL,
some under the MIT license.
For details see
https://github.com/OpenPrinting/foomatic-db/blob/master/COPYING .
To encompass the different licenses, the package's
license is set to "free" as the common denominator.

ppd files can be found in `$out/share/cups/model/foomatic-db`.
The subdirectory `foomatic-db` is used to avoid conflicts
with other packages that might be combined with the package
`foomatic-db` in `config.services.printing.drivers`.

There is a daily snapshot of a source archive available at
https://www.openprinting.org/download/foomatic/ .
However, these files rotate daily and
cannot be used as a stable download source.
So we rely on OpenPrinting's Github repository
instead and pinpoint a fresh commit.
2022-11-06 19:28:15 +01:00
Yarny0
cd4c8d63f5 foomatic-db-engine: init at unstable-2022-05-03
`foomatic-db-engine` contains several perl scripts to parse
and process XML files from the Foomatic database packages.
It can be used to extract ppd files,
which will be accomplished in a follow-up commit.

The package also contains scripts
to handle print queues and jobs.
It can -- optionally -- talk to the local cups server,
to network printers and to SMB print servers.
The build recipe contains switches to enable these features;
however, they are not needed when generating ppd files.

There is a daily snapshot of a source archive available at
https://www.openprinting.org/download/foomatic/ .
However, these files rotate daily and
cannot be used as a stable download source.
So we rely on OpenPrinting's Github repository
instead and pinpoint a fresh commit.
2022-11-06 19:28:15 +01:00
Yarny0
bafefd7ae2 samsung-unified-linux-driver (cups driver): patch all filters
ppd files in the `samsung-unified-linux-driver`
package invoke these filter commands:

* pstosecps
* pstospl
* rastertospl
* pstosplc

The paths to all of those commands excluding the last one got
patched with their absolute paths during the build process.
This commit adds the last one to
the list of commands to be patched.
2022-11-06 16:19:23 +01:00
Yarny0
bfe3271fc0 samsung-unified-linux-driver (cups driver): use patchPpdFilesHook
The `sed` script in the `installPhase` is removed.
Instead, the setup hook `patchPpdFilesHook` is
used to patch the path to the filter executables.

The result should essentially be the same.
Comparing the generated ppd files showed no difference,
short of the package's hash in the absolute paths and
added newline characters at the end of some ppd files.
Missing newline characters at the end of the last line are
apparently added by `awk`; this shouldn't affect functionality.
The new package also contains a `propagated-build-inputs`
file which propagates the package itself.
This ensures the package is available whenever
a ppd file is singled out by another package.
2022-11-06 16:19:23 +01:00
Yarny0
bf30b53817 cups-kyocera (cups driver): use patchPpdFilesHook
The `sed` script in the `installPhase` is removed.
Instead, the setup hook `patchPpdFilesHook`
is used to patch the path to `rastertokpsl`.

The result should essentially be the same.
Comparing the generated ppd files showed no difference
(short of the package's hash in the absolute paths).
The new package also contains a `propagated-build-inputs`
file which propagates the package itself.
This ensures the package is available whenever
a ppd file is singled out by another package.
2022-11-06 16:19:22 +01:00
Yarny0
ef8566f38b cups-drv-rastertosag-gdi (cups driver): patch ppd bin paths
The `cups-drv-rastertosag-gdi` CUPS printer driver
package provides a filter executable `rastertosag-gdi`
and two ppd files that reference the filter file.
The commit at hand uses `patchPpdFilesHook` to
replace calls to the filter executable with
absolute paths to increase package purity.
2022-11-06 16:19:22 +01:00
Yarny0
335a9083b0 patchPpdFilesHook: new setup hook for absolute executable paths
PostScript Printer Description (ppd) files
describe printer features and capabilities.
They are usually evaluated by CUPS to convert
print jobs into a format suitable for a printer.
The conversion is often accomplished by commands
or even short shell scripts inside the ppd files.

ppd files are included in many printer driver packages.
Their scripts sometimes refer to other executables;
some of them are more common (like `perl`),
others are more exotic (like `rastertohp`).
If an executable is called with its name alone,
the effects of the ppd file depend on whether
the executable is in the PATH of CUPS,
and on the executable's version.
If an executable is called with an absolut path
(like `/usr/bin/perl`), it won't work at all in NixOS.

The commit at hand adds a setup hook that uses
the `fixupPhase` to substitute certain executable's
invocations in pdd files with absolute paths.
To use it, add the hook to `nativeBuildInputs` and
provide a list of executable names in `ppdFileCommands`.
Each executable must be available in the
package that is being built, or in `buildInputs`.
The setup hook's script then looks for ppd files in
`share/cups/model` and `share/ppds` in each output,
and replaces executable names with their absolute paths.
If ppd files need to be patched in unorthodox locations or
the setup hook needs to be invoked manually for other reasons,
one may leave the list `ppdFileCommands` empty to
avoid automatic processing of ppd files, then call
the shell function `patchPpdFileCommands` directly.
Details are described in the file `patch-ppd-hook.sh`.

Notes on the motivation for this setup hook:

Most packages in nixpkgs that provide
ppd files do not patch those ppd files at all.
This is not fatal when the executables are just called
with their names since the user can add packages
with the executables to `services.printing.drivers`.
E.g. if the user adds `pkgs.perl`, then all ppd
files that invoke `perl` will work as expected.
Nevertheless, to make these ppd files independent of
their execution environment, command invocations should
be substituted with absolut paths into the nix store.
This is similar to patching shebang lines so scripts can be
called independently of having the interpreter in the PATH.

The hook script in this commit is meant to support new packages
`foomatic-db*` which will generate several thousands of
ppd files referencing a plethora of different executables.
During development of these packages, I realized that
it's quite hard to patch ppd files in a robust way.
While binary names like `rastertokpsl` seem to be sufficiently
unique to be patched with `sed`, names like `date` or `gs`
are hard to patch without producing "false positives",
i.e., coincidental occurences of the executable's name that do
*not* refer to the executable and should not be patched at all.
As this problem also affects other packages,
it seems reasonable to put a robust implementation
in its own setup hook so that other
packages can use it without much effort.

Notes on the implementation:

The ppd file format is far from trivial.
The basic structure are key-value pairs;
keys may occur multiple times.
Only a small subset of keys may contain
executable names or shell scripts in their values.
Some values may span multiple lines;
a linebreak might even occur in the middle of a token.
Some executable names also occur in other keys by accident
where they must not be substituted (e.g. `gs` or `date`).

It is necessary to provide the list of command
names that will be patched for two reasons:
ppd files often contain "tokens" that might look
like commands (e.g. "file" or "host") but aren't;
these would erroneously get patched.
Also, looking for everything that might be a command
would slow down the patching process considerably.

The implementation uses `awk` to detect
keys that might contain executable names;
only their values are treated for substitution.
This avoids most cases of "overzealous" substitutions.
Since values may span multiple lines,
`sed` alone (while faster than `awk`) cannot focus
its substitution capabilities on relevant keys.
An elaborate set of regular expressions further helps
to minimize the probability of "false positives".
Several tricks are employed to speed up `awk`.
Notably, relevant files are identified with
`grep` before `awk` is applied to those files only.
Note that the script probably cannot handle fancy command
names (like spaces or backslashes as part of the name).
Also, there are still edge cases that the script would
mistakenly skip, e.g. if a shell script contains a
line break in the middle of an executable's name;
although ppd files permit such constellations,
I have yet to see one.

ppd files may be gzipped.
The setup hook accepts gzipped ppd files:
It decompresses them, substitutes paths, then recompresses them.
However, Nix cannot detect substituted paths as
runtime dependencies in compressed ppd files.
To ensure substituted paths are propagated as
runtime dependencies, the script adds each substituted
path to the variable `propagatedBuildInputs`.
Since this might not be enough for multi-output packages,
those paths are also written directly to
`nix-support/propagated-build-inputs`.
See the comment in `patch-ppd-hook.sh` for details.

Finally, the setup hook comes with a small test that
probes some edge cases with an artificial ppd file.

References:

* https://www.cups.org/doc/spec-ppd.html
  * general ppd file specification
  * lists some keys that may contain
    executable names or shell scripts

* https://refspecs.linuxfoundation.org/LSB_4.0.0/LSB-Printing/LSB-Printing/ppdext.html
  * lists some keys that may contain
    executable names or shell scripts

* https://en.wikipedia.org/wiki/PostScript_Printer_Description#CUPS
  * lists the usual locations of ppd files
2022-11-06 16:19:22 +01:00
Yarny0
6f622e91c5 cups-drv-rastertosag-gdi (cups driver): gzip ppd files
ppd files are gzipped to save some space.
The `gzip` "-n" option prevents gzip from storing
a timestamp, thus facilitating reproducibility.
2022-11-06 16:19:22 +01:00
Yarny0
5222363936 cups-drv-rastertosag-gdi (cups driver): fix simple comment typo 2022-11-06 16:19:22 +01:00
Mario Rodas
3bacde6273
Merge pull request #199554 from r-ryantm/auto-update/awscli2
awscli2: 2.8.8 -> 2.8.9
2022-11-05 01:30:49 -05:00
Mario Rodas
ff2e4e2408
Merge pull request #199605 from fabaff/google-cloud-appengine-logging-bump
python310Packages.google-cloud-appengine-logging: 1.1.5 -> 1.1.6
2022-11-05 01:28:55 -05:00
github-actions[bot]
429feeb354 terraform-providers.tencentcloud: 1.78.7 → 1.78.8 2022-11-05 16:28:44 +10:00
github-actions[bot]
0bf54c5ee7 terraform-providers.sumologic: 2.19.1 → 2.19.2 2022-11-05 16:28:44 +10:00
github-actions[bot]
9389dec4a7 terraform-providers.snowflake: 0.49.0 → 0.50.0 2022-11-05 16:28:44 +10:00
github-actions[bot]
ea85340ba7 terraform-providers.skytap: 0.15.0 → 0.15.1 2022-11-05 16:28:44 +10:00
Mario Rodas
f815e016e3
Merge pull request #199604 from fabaff/whois-bump
python310Packages.whois: 0.9.16 -> 0.9.17
2022-11-05 01:28:32 -05:00
Mario Rodas
3d6507eed3
Merge pull request #199611 from figsoda/nextest
cargo-nextest: 0.9.42 -> 0.9.43, add figsoda as a maintainer
2022-11-05 01:27:01 -05:00
Mario Rodas
8268152e56
Merge pull request #199620 from figsoda/cargo-deny
cargo-deny: 0.13.3 -> 0.13.4
2022-11-05 01:24:45 -05:00
Mario Rodas
24ca01fb3a
Merge pull request #199643 from NickCao/sysdig
Revert "sysdig: pin to openssl_1_1"
2022-11-05 01:24:24 -05:00
Mario Rodas
a395a15960
Merge pull request #195604 from tjni/starship
starship: reenable default features on darwin
2022-11-05 01:14:07 -05:00
Mario Rodas
2a0350d496
Merge pull request #199555 from senorsmile/mutagen-0.16.0
mutagen: 0.14.0 -> 0.16.0
2022-11-05 00:39:33 -05:00
Mario Rodas
532ab107ba
Merge pull request #199626 from r-ryantm/auto-update/syft
syft: 0.60.2 -> 0.60.3
2022-11-05 00:24:13 -05:00
Mario Rodas
ed229c09e9
Merge pull request #199638 from marsam/update-nodejs
nodejs: 14.21.0 -> 14.21.1, 16.18.0 -> 16.18.1, 18.12.0 -> 18.12.1, 19.0.0 -> 19.0.1 

resolves #199578 
resolves #199607
resolves #199608
2022-11-05 00:22:56 -05:00
Mario Rodas
b473e04a37
Merge pull request #199540 from r-ryantm/auto-update/oh-my-posh
oh-my-posh: 12.10.0 -> 12.11.0
2022-11-05 00:14:31 -05:00
Nick Cao
a9fb9f3b06
Revert "sysdig: pin to openssl_1_1"
This reverts commit 240ace010d.
2022-11-05 13:10:51 +08:00
R. Ryantm
f9dc913011 v2ray-geoip: 202210270100 -> 202211030059 2022-11-05 13:07:29 +08:00
Mario Rodas
71d53b94c6 oh-my-posh: 12.11.0 -> 12.12.1 2022-11-05 04:20:00 +00:00
Mario Rodas
94d4893136
Merge pull request #199627 from r-ryantm/auto-update/terragrunt
terragrunt: 0.39.2 -> 0.40.0
2022-11-04 23:08:05 -05:00
R. Ryantm
44770699ea netbird: 0.10.3 -> 0.10.4 2022-11-05 12:01:19 +08:00
R. Ryantm
c8abf9c376 debootstrap: 1.0.127 -> 1.0.128 2022-11-05 11:58:03 +08:00
R. Ryantm
6c731843c3 pgloader: 3.6.8 -> 3.6.9 2022-11-05 11:53:56 +08:00
Mario Rodas
fa7b12e493
Merge pull request #199606 from fabaff/canonicaljson-bump
python310Packages.canonicaljson: 1.6.3 -> 1.6.4
2022-11-04 22:51:27 -05:00
R. Ryantm
2390991544 s3cmd: 2.2.0 -> 2.3.0 2022-11-05 11:50:46 +08:00
R. Ryantm
d163332ff9 jellyfin-ffmpeg: 5.1.2-2 -> 5.1.2-4 2022-11-05 11:47:12 +08:00
R. Ryantm
c4937f84a5 cloud-init: 22.3.3 -> 22.3.4 2022-11-05 11:44:04 +08:00
Anderson Torres
eb738c0fd3
Merge pull request #199492 from kilianar/calibre-6.8.0
calibre: 6.7.1 -> 6.8.0
2022-11-05 00:11:23 -03:00
Bobby Rong
bfd16c5676
Merge pull request #199469 from afh/afh-got-update
got: 0.77 -> 0.78
2022-11-05 11:03:05 +08:00
R. Ryantm
177eaa168a terragrunt: 0.39.2 -> 0.40.0 2022-11-05 02:55:17 +00:00
R. Ryantm
7cd0ea9d37 syft: 0.60.2 -> 0.60.3 2022-11-05 02:47:55 +00:00
Artturi
bed653c5fa
Merge pull request #164169 from martinetd/logrotate_post_cleanup
logrotate: cleanup legacy options
2022-11-05 04:27:33 +02:00
figsoda
5eaab876db cargo-deny: 0.13.3 -> 0.13.4 2022-11-04 22:18:58 -04:00
Sandro
c28018857a
Merge pull request #199426 from r-ryantm/auto-update/aws-c-sdkutils 2022-11-05 02:41:25 +01:00
Lin Jian
fb445b517e nixos/fish: use a local version of runCommand for babelfishTranslate
Building etc."fish/setEnvironment.fish" needs
config.system.build.setEnvironment, which can be very large. And what
babelfishTranslate does is to translate env vars exported by bash
syntax, which does not need much computing power.

This patch can reduce the network traffic when using remote builders
with almost no harm.
2022-11-04 18:35:18 -07:00
Dominique Martinet
665d896244 logrotate: add services.logrotate.settings example 2022-11-05 10:34:04 +09:00
Dominique Martinet
01ff1dd23f logrotate service: cleanup deprecated options 2022-11-05 10:34:03 +09:00
Theodore Ni
1dd8696f96 vector: 0.24.1 -> 0.25.0 2022-11-04 21:06:39 -04:00
figsoda
2b4a5cf5c1 cargo-nextest: 0.9.42 -> 0.9.43, add figsoda as a maintainer 2022-11-04 21:04:21 -04:00
Theodore Ni
7b9bb27e08
starship: reenable default features on darwin 2022-11-04 17:55:53 -07:00