paperless: remove package & module as it has been superseded by paperless-ng

The paperless project has moved on to paperless-ng and the original
paperless package in Nixpkgs has stopped working recently (due to
version incompatibility with the providede Django package).

Instead of investing more time into the old module we should migrate all
users to the new module instead.
This commit is contained in:
Andreas Rammhold 2021-06-03 15:59:17 +02:00 committed by Flakebi
parent 95f2dc650d
commit 6e1421013a
No known key found for this signature in database
GPG Key ID: 38E7ED984D7DCD02
14 changed files with 140 additions and 543 deletions

View File

@ -176,6 +176,90 @@
<section xml:id="sec-release-21.11-incompatibilities">
<title>Backward Incompatibilities</title>
<itemizedlist>
<listitem>
<para>
The <literal>paperless</literal> module and package have been
removed. All users should migrate to the successor
<literal>paperless-ng</literal> instead. The Paperless project
<link xlink:href="https://github.com/the-paperless-project/paperless/commit/9b0063c9731f7c5f65b1852cb8caff97f5e40ba4">has
been archived</link> and advises all users to use
<literal>paperless-ng</literal> instead.
</para>
<para>
Users can use the <literal>services.paperless-ng</literal>
module as a replacement while noting the following
incompatibilities:
</para>
<itemizedlist spacing="compact">
<listitem>
<para>
<literal>services.paperless.ocrLanguages</literal> has no
replacement. Users should migrate to
<link xlink:href="options.html#opt-services.paperless-ng.extraConfig"><literal>services.paperless-ng.extraConfig</literal></link>
instead:
</para>
</listitem>
</itemizedlist>
<programlisting language="bash">
{
services.paperless-ng.extraConfig = {
# Provide languages as ISO 639-2 codes
# separated by a plus (+) sign.
# https://en.wikipedia.org/wiki/List_of_ISO_639-2_codes
PAPERLESS_OCR_LANGUAGE = &quot;deu+eng+jpn&quot;; # German &amp; English &amp; Japanse
};
}
</programlisting>
<itemizedlist>
<listitem>
<para>
If you previously specified
<literal>PAPERLESS_CONSUME_MAIL_*</literal> settings in
<literal>services.paperless.extraConfig</literal> you
should remove those options now. You now
<emphasis>must</emphasis> define those settings in the
admin interface of paperless-ng.
</para>
</listitem>
<listitem>
<para>
Option <literal>services.paperless.manage</literal> no
longer exists. Use the script at
<literal>${services.paperless-ng.dataDir}/paperless-ng-manage</literal>
instead. Note that this script only exists after the
<literal>paperless-ng</literal> service has been started
at least once.
</para>
</listitem>
<listitem>
<para>
After switching to the new system configuration you should
run the Django management command to reindex your
documents and optionally create a user, if you dont have
one already.
</para>
<para>
To do so, enter the data directory (the value of
<literal>services.paperless-ng.dataDir</literal>,
<literal>/var/lib/paperless</literal> by default), switch
to the paperless user and execute the management command
like below:
</para>
<programlisting>
$ cd /var/lib/paperless
$ su paperless -s /bin/sh
$ ./paperless-ng-manage document_index reindex
# if not already done create a user account, paperless-ng requires a login
$ ./paperless-ng-manage createsuperuser
Username (leave blank to use 'paperless'): my-user-name
Email address: me@example.com
Password: **********
Password (again): **********
Superuser created successfully.
</programlisting>
</listitem>
</itemizedlist>
</listitem>
<listitem>
<para>
The <literal>staticjinja</literal> package has been upgraded

View File

@ -55,6 +55,53 @@ pt-services.clipcat.enable).
## Backward Incompatibilities {#sec-release-21.11-incompatibilities}
- The `paperless` module and package have been removed. All users should migrate to the
successor `paperless-ng` instead. The Paperless project [has been
archived](https://github.com/the-paperless-project/paperless/commit/9b0063c9731f7c5f65b1852cb8caff97f5e40ba4)
and advises all users to use `paperless-ng` instead.
Users can use the `services.paperless-ng` module as a replacement while noting the following incompatibilities:
- `services.paperless.ocrLanguages` has no replacement. Users should migrate to [`services.paperless-ng.extraConfig`](options.html#opt-services.paperless-ng.extraConfig) instead:
```nix
{
services.paperless-ng.extraConfig = {
# Provide languages as ISO 639-2 codes
# separated by a plus (+) sign.
# https://en.wikipedia.org/wiki/List_of_ISO_639-2_codes
PAPERLESS_OCR_LANGUAGE = "deu+eng+jpn"; # German & English & Japanse
};
}
```
- If you previously specified `PAPERLESS_CONSUME_MAIL_*` settings in
`services.paperless.extraConfig` you should remove those options now. You
now *must* define those settings in the admin interface of paperless-ng.
- Option `services.paperless.manage` no longer exists.
Use the script at `${services.paperless-ng.dataDir}/paperless-ng-manage` instead.
Note that this script only exists after the `paperless-ng` service has been
started at least once.
- After switching to the new system configuration you should run the Django
management command to reindex your documents and optionally create a user,
if you don't have one already.
To do so, enter the data directory (the value of
`services.paperless-ng.dataDir`, `/var/lib/paperless` by default), switch
to the paperless user and execute the management command like below:
```
$ cd /var/lib/paperless
$ su paperless -s /bin/sh
$ ./paperless-ng-manage document_index reindex
# if not already done create a user account, paperless-ng requires a login
$ ./paperless-ng-manage createsuperuser
Username (leave blank to use 'paperless'): my-user-name
Email address: me@example.com
Password: **********
Password (again): **********
Superuser created successfully.
```
- The `staticjinja` package has been upgraded from 1.0.4 to 3.0.1
- The `erigon` ethereum node has moved to a new database format in `2021-05-04`, and requires a full resync

View File

@ -550,7 +550,6 @@
./services/misc/ombi.nix
./services/misc/osrm.nix
./services/misc/packagekit.nix
./services/misc/paperless.nix
./services/misc/paperless-ng.nix
./services/misc/parsoid.nix
./services/misc/plex.nix

View File

@ -73,6 +73,14 @@ in
{
meta.maintainers = with maintainers; [ earvstedt Flakebi ];
imports = [
(mkRemovedOptionModule [ "services" "paperless"] ''
The paperless module has been removed as the upstream project died.
Users should migrate to the paperless-ng module (services.paperless-ng).
More information can be found in the NixOS 21.11 release notes.
'')
];
options.services.paperless-ng = {
enable = mkOption {
type = lib.types.bool;
@ -180,14 +188,6 @@ in
};
config = mkIf cfg.enable {
assertions = [
{
assertion = config.services.paperless.enable ->
(config.services.paperless.dataDir != cfg.dataDir && config.services.paperless.port != cfg.port);
message = "Paperless-ng replaces Paperless, either disable Paperless or assign a new dataDir and port to one of them";
}
];
# Enable redis if no special url is set
services.redis.enable = mkIf (!hasAttr "PAPERLESS_REDIS" env) true;

View File

@ -1,183 +0,0 @@
{ config, pkgs, lib, ... }:
with lib;
let
cfg = config.services.paperless;
defaultUser = "paperless";
manage = cfg.package.withConfig {
config = {
PAPERLESS_CONSUMPTION_DIR = cfg.consumptionDir;
PAPERLESS_INLINE_DOC = "true";
PAPERLESS_DISABLE_LOGIN = "true";
} // cfg.extraConfig;
inherit (cfg) dataDir ocrLanguages;
paperlessPkg = cfg.package;
};
in
{
options.services.paperless = {
enable = mkOption {
type = lib.types.bool;
default = false;
description = ''
Enable Paperless.
When started, the Paperless database is automatically created if it doesn't
exist and updated if the Paperless package has changed.
Both tasks are achieved by running a Django migration.
'';
};
dataDir = mkOption {
type = types.str;
default = "/var/lib/paperless";
description = "Directory to store the Paperless data.";
};
consumptionDir = mkOption {
type = types.str;
default = "${cfg.dataDir}/consume";
defaultText = "\${dataDir}/consume";
description = "Directory from which new documents are imported.";
};
consumptionDirIsPublic = mkOption {
type = types.bool;
default = false;
description = "Whether all users can write to the consumption dir.";
};
ocrLanguages = mkOption {
type = with types; nullOr (listOf str);
default = null;
description = ''
Languages available for OCR via Tesseract, specified as
<literal>ISO 639-2/T</literal> language codes.
If unset, defaults to all available languages.
'';
example = [ "eng" "spa" "jpn" ];
};
address = mkOption {
type = types.str;
default = "localhost";
description = "Server listening address.";
};
port = mkOption {
type = types.port;
default = 28981;
description = "Server port to listen on.";
};
extraConfig = mkOption {
type = types.attrs;
default = {};
description = ''
Extra paperless config options.
The config values are evaluated as double-quoted Bash string literals.
See <literal>paperless-src/paperless.conf.example</literal> for available options.
To enable user authentication, set <literal>PAPERLESS_DISABLE_LOGIN = "false"</literal>
and run the shell command <literal>$dataDir/paperless-manage createsuperuser</literal>.
To define secret options without storing them in /nix/store, use the following pattern:
<literal>PAPERLESS_PASSPHRASE = "$(&lt; /etc/my_passphrase_file)"</literal>
'';
example = literalExample ''
{
PAPERLESS_OCR_LANGUAGE = "deu";
}
'';
};
user = mkOption {
type = types.str;
default = defaultUser;
description = "User under which Paperless runs.";
};
package = mkOption {
type = types.package;
default = pkgs.paperless;
defaultText = "pkgs.paperless";
description = "The Paperless package to use.";
};
manage = mkOption {
type = types.package;
readOnly = true;
default = manage;
description = ''
A script to manage the Paperless instance.
It wraps Django's manage.py and is also available at
<literal>$dataDir/manage-paperless</literal>
'';
};
};
config = mkIf cfg.enable {
systemd.tmpfiles.rules = [
"d '${cfg.dataDir}' - ${cfg.user} ${config.users.users.${cfg.user}.group} - -"
] ++ (optional cfg.consumptionDirIsPublic
"d '${cfg.consumptionDir}' 777 - - - -"
# If the consumption dir is not created here, it's automatically created by
# 'manage' with the default permissions.
);
systemd.services.paperless-consumer = {
description = "Paperless document consumer";
serviceConfig = {
User = cfg.user;
ExecStart = "${manage} document_consumer";
Restart = "always";
};
after = [ "systemd-tmpfiles-setup.service" ];
wantedBy = [ "multi-user.target" ];
preStart = ''
if [[ $(readlink ${cfg.dataDir}/paperless-manage) != ${manage} ]]; then
ln -sf ${manage} ${cfg.dataDir}/paperless-manage
fi
${manage.setupEnv}
# Auto-migrate on first run or if the package has changed
versionFile="$PAPERLESS_DBDIR/src-version"
if [[ $(cat "$versionFile" 2>/dev/null) != ${cfg.package} ]]; then
python $paperlessSrc/manage.py migrate
echo ${cfg.package} > "$versionFile"
fi
'';
};
systemd.services.paperless-server = {
description = "Paperless document server";
serviceConfig = {
User = cfg.user;
ExecStart = "${manage} runserver --noreload ${cfg.address}:${toString cfg.port}";
Restart = "always";
};
# Bind to `paperless-consumer` so that the server never runs
# during migrations
bindsTo = [ "paperless-consumer.service" ];
after = [ "paperless-consumer.service" ];
wantedBy = [ "multi-user.target" ];
};
users = optionalAttrs (cfg.user == defaultUser) {
users.${defaultUser} = {
group = defaultUser;
uid = config.ids.uids.paperless;
home = cfg.dataDir;
};
groups.${defaultUser} = {
gid = config.ids.gids.paperless;
};
};
};
}

View File

@ -334,7 +334,6 @@ in
pam-oath-login = handleTest ./pam-oath-login.nix {};
pam-u2f = handleTest ./pam-u2f.nix {};
pantheon = handleTest ./pantheon.nix {};
paperless = handleTest ./paperless.nix {};
paperless-ng = handleTest ./paperless-ng.nix {};
pdns-recursor = handleTest ./pdns-recursor.nix {};
peerflix = handleTest ./peerflix.nix {};

View File

@ -1,36 +0,0 @@
import ./make-test-python.nix ({ lib, ... } : {
name = "paperless";
meta = with lib.maintainers; {
maintainers = [ earvstedt ];
};
machine = { pkgs, ... }: {
environment.systemPackages = with pkgs; [ imagemagick jq ];
services.paperless = {
enable = true;
ocrLanguages = [ "eng" ];
};
};
testScript = ''
machine.wait_for_unit("paperless-consumer.service")
# Create test doc
machine.succeed(
"convert -size 400x40 xc:white -font 'DejaVu-Sans' -pointsize 20 -fill black -annotate +5+20 'hello world 16-10-2005' /var/lib/paperless/consume/doc.png"
)
with subtest("Service gets ready"):
machine.wait_for_unit("paperless-server.service")
# Wait until server accepts connections
machine.wait_until_succeeds("curl -fs localhost:28981")
with subtest("Test document is consumed"):
machine.wait_until_succeeds(
"(($(curl -fs localhost:28981/api/documents/ | jq .count) == 1))"
)
assert "2005-10-16" in machine.succeed(
"curl -fs localhost:28981/api/documents/ | jq '.results | .[0] | .created'"
)
'';
})

View File

@ -1,168 +0,0 @@
{ stdenv
, lib
, fetchFromGitHub
, makeWrapper
, callPackage
, python3
, imagemagick
, ghostscript
, optipng
, tesseract
, unpaper
}:
## Usage
# ${paperless}/bin/paperless wraps manage.py
# ${paperless}/share/paperless/setup-env.sh can be sourced from a
# shell script to setup a Paperless environment
# paperless.withConfig is a convenience function to setup a
# configured Paperless instance. (See ./withConfig.nix)
# For WSGI with gunicorn, use a shell script like this:
# let
# pythonEnv = paperless.python.withPackages (ps: paperless.runtimePackages ++ [ ps.gunicorn ]);
# in
# writers.writeBash "run-gunicorn" ''
# source ${paperless}/share/paperless/setup-env.sh
# PYTHONPATH=$paperlessSrc ${pythonEnv}/bin/gunicorn paperless.wsgi
# ''
let
paperless = stdenv.mkDerivation rec {
pname = "paperless";
version = "2.7.0";
src = fetchFromGitHub {
owner = "the-paperless-project";
repo = "paperless";
rev = version;
sha256 = "0pkmyky1crjnsg7r0gfk0fadisfsgzlsq6afpz16wx4hp6yvkkf7";
};
nativeBuildInputs = [ makeWrapper ];
doCheck = true;
dontInstall = true;
pythonEnv = python.withPackages (_: runtimePackages);
pythonCheckEnv = python.withPackages (_: (runtimePackages ++ checkPackages));
unpackPhase = ''
srcDir=$out/share/paperless
mkdir -p $srcDir
cp -r --no-preserve=mode $src/src/* $src/LICENSE $srcDir
'';
postPatch = ''
# django-cors-headers 3.x requires a scheme for allowed hosts
substituteInPlace $out/share/paperless/paperless/settings.py \
--replace "localhost:8080" "http://localhost:8080"
'';
buildPhase = let
# Paperless has explicit runtime checks that expect these binaries to be in PATH
extraBin = lib.makeBinPath [ imagemagick ghostscript optipng tesseract unpaper ];
in ''
${python.interpreter} -m compileall $srcDir
makeWrapper $pythonEnv/bin/python $out/bin/paperless \
--set PATH ${extraBin} --add-flags $out/share/paperless/manage.py
# A shell snippet that can be sourced to setup a paperless env
cat > $out/share/paperless/setup-env.sh <<EOF
export PATH="$pythonEnv/bin:${extraBin}''${PATH:+:}$PATH"
export paperlessSrc=$out/share/paperless
EOF
'';
checkPhase = ''
source $out/share/paperless/setup-env.sh
tmpDir=$(realpath testsTmp)
mkdir $tmpDir
export HOME=$tmpDir
export PAPERLESS_MEDIADIR=$tmpDir
cd $paperlessSrc
# Prevent tests from writing to the derivation output
chmod -R -w $out
# Disable cache to silence a pytest warning ("could not create cache")
$pythonCheckEnv/bin/pytest -p no:cacheprovider
'';
passthru = {
withConfig = callPackage ./withConfig.nix {};
inherit python runtimePackages checkPackages tesseract;
};
meta = with lib; {
description = "Scan, index, and archive all of your paper documents";
homepage = "https://github.com/the-paperless-project/paperless";
license = licenses.gpl3;
maintainers = [ maintainers.earvstedt ];
};
};
python = python3.override {
packageOverrides = self: super: let
customPkgs = import ./python-modules super fetchFromGitHub; in
{
pyocr = pyocrWithUserTesseract super;
# Paperless is incompatible with factory_boy >= 3
factory_boy = customPkgs.factory_boy_2_12_0;
# These are pre-release versions, hence they are private to this pkg
django-filter = self.callPackage ./python-modules/django-filter.nix {};
django-crispy-forms = self.callPackage ./python-modules/django-crispy-forms.nix {};
};
};
runtimePackages = with python.pkgs; [
dateparser
python-dateutil
django
django-cors-headers
django-crispy-forms
django-filter
django_extensions
djangoql
djangorestframework
factory_boy
filemagic
fuzzywuzzy
langdetect
pdftotext
pillow
psycopg2
pyocr
python-dotenv
python-gnupg
pytz
termcolor
] ++ (lib.optional stdenv.isLinux inotify-simple);
checkPackages = with python.pkgs; [
pytest
pytest-django
pytest-env
pytest-xdist
];
pyocrWithUserTesseract = pyPkgs:
let
pyocr = pyPkgs.pyocr.override { inherit tesseract; };
in
if pyocr.outPath == pyPkgs.pyocr.outPath then
pyocr
else
# The user has provided a custom tesseract derivation that might be
# missing some languages that are required for PyOCR's tests. Disable them to
# avoid build errors.
pyocr.overridePythonAttrs (attrs: {
doCheck = false;
});
in
paperless

View File

@ -1,11 +0,0 @@
pyPkgs: fetchFromGitHub:
{
factory_boy_2_12_0 = pyPkgs.factory_boy.overridePythonAttrs (old: rec {
version = "2.12.0";
src = pyPkgs.fetchPypi {
inherit (old) pname;
inherit version;
sha256 = "0w53hjgag6ad5i2vmrys8ysk54agsqvgbjy9lg8g0d8pi9h8vx7s";
};
});
}

View File

@ -1,39 +0,0 @@
{ lib
, buildPythonPackage
, fetchFromGitHub
, pytestCheckHook
, pytest-django
, django
}:
buildPythonPackage rec {
pname = "django-crispy-forms";
version = "1.10.0";
src = fetchFromGitHub {
owner = "django-crispy-forms";
repo = "django-crispy-forms";
rev = version;
sha256 = "0y6kskfxgckb9npcgwx4zrs5n9px159zh9zhinhxi3i7wlriqpf5";
};
# For reasons unknown, the source dir must contain a dash
# for the tests to run successfully
postUnpack = ''
mv $sourceRoot source-
export sourceRoot=source-
'';
checkInputs = [ django pytest-django pytestCheckHook ];
preCheck = ''
export DJANGO_SETTINGS_MODULE=crispy_forms.tests.test_settings
'';
meta = with lib; {
description = "The best way to have DRY Django forms";
homepage = "https://github.com/maraujop/django-crispy-forms";
license = licenses.mit;
maintainers = with maintainers; [ earvstedt ];
};
}

View File

@ -1,26 +0,0 @@
{ lib, buildPythonPackage, python, pythonOlder, fetchFromGitHub
, django, django-crispy-forms, djangorestframework, mock, pytz }:
buildPythonPackage rec {
pname = "django-filter";
version = "2.1.0-pre";
disabled = pythonOlder "3.4";
src = fetchFromGitHub {
owner = "carltongibson";
repo = pname;
rev = "24adad8c48bc9e7c7539b6510ffde4ce4effdc29";
sha256 = "0hv4w95jnlzp9vdximl6bb27fyi75001jhvsbs0ikkd8amq8iaj7";
};
checkInputs = [ django django-crispy-forms djangorestframework mock pytz ];
checkPhase = "${python.interpreter} runtests.py";
meta = with lib; {
description = "A reusable Django application for allowing users to filter querysets dynamically.";
homepage = "https://github.com/carltongibson/django-filter";
license = licenses.bsd3;
maintainers = with maintainers; [ earvstedt ];
};
}

View File

@ -1,68 +0,0 @@
{ paperless, lib, writers }:
## Usage
#
# nix-build --out-link ./paperless -E '
# (import <nixpkgs> {}).paperless.withConfig {
# dataDir = /tmp/paperless-data;
# config = {
# PAPERLESS_DISABLE_LOGIN = "true";
# };
# }'
#
# Setup DB
# ./paperless migrate
#
# Consume documents in ${dataDir}/consume
# ./paperless document_consumer --oneshot
#
# Start web interface
# ./paperless runserver --noreload localhost:8000
{ config ? {}, dataDir ? null, ocrLanguages ? null
, paperlessPkg ? paperless, extraCmds ? "" }:
with lib;
let
paperless = if ocrLanguages == null then
paperlessPkg
else
(paperlessPkg.override {
tesseract = paperlessPkg.tesseract.override {
enableLanguages = ocrLanguages;
};
}).overrideDerivation (_: {
# `ocrLanguages` might be missing some languages required by the tests.
doCheck = false;
});
envVars = (optionalAttrs (dataDir != null) {
PAPERLESS_CONSUMPTION_DIR = "${dataDir}/consume";
PAPERLESS_MEDIADIR = "${dataDir}/media";
PAPERLESS_STATICDIR = "${dataDir}/static";
PAPERLESS_DBDIR = dataDir;
}) // config;
envVarDefs = mapAttrsToList (n: v: ''export ${n}="${toString v}"'') envVars;
setupEnvVars = builtins.concatStringsSep "\n" envVarDefs;
setupEnv = ''
source ${paperless}/share/paperless/setup-env.sh
${setupEnvVars}
${optionalString (dataDir != null) ''
mkdir -p "$PAPERLESS_CONSUMPTION_DIR" \
"$PAPERLESS_MEDIADIR" \
"$PAPERLESS_STATICDIR" \
"$PAPERLESS_DBDIR"
''}
'';
runPaperless = writers.writeBash "paperless" ''
set -e
${setupEnv}
${extraCmds}
exec python $paperlessSrc/manage.py "$@"
'';
in
runPaperless // {
inherit paperless setupEnv;
}

View File

@ -553,6 +553,7 @@ mapAliases ({
owncloudclient = owncloud-client; # added 2016-08
ocz-ssd-guru = throw "ocz-ssd-guru has been removed due to there being no source available"; # added 2021-07-12
p11_kit = p11-kit; # added 2018-02-25
paperless = paperless-ng; # added 2021-06-06
parity = openethereum; # added 2020-08-01
parquet-cpp = arrow-cpp; # added 2018-09-08
pass-otp = pass.withExtensions (ext: [ext.pass-otp]); # added 2018-05-04

View File

@ -7889,8 +7889,6 @@ with pkgs;
pamtester = callPackage ../tools/security/pamtester { };
paperless = callPackage ../applications/office/paperless { };
paperless-ng = callPackage ../applications/office/paperless-ng { };
paperwork = callPackage ../applications/office/paperwork/paperwork-gtk.nix { };