Commit Graph

121 Commits

Author SHA1 Message Date
Robert Hensing
5cacf0fcec dockerTools: use go.GOARCH as default arch 2020-12-15 02:15:35 -08:00
Terin Stock
8f66dc94a7 dockerTools: normalize arch to GOARCH
Docker (via containerd) and the the OCI Image Configuration imply and
suggest, respectfully, that the architecture set in images matches those
of GOARCH in the Go Language document.

This changeset updates the implimentation of getArch in dockerTools to
return GOARCH values, to satisfy Docker.

Fixes: #106695
2020-12-15 02:14:01 -08:00
Florian Klink
e054694925 dockerTools.binSh: init 2020-12-02 14:57:23 +01:00
Florian Klink
f7ee2706c2 dockerTools.fakeNss: init
This provides a /etc/passwd and /etc/group that contain root and nobody.

Useful when packaging binaries that insist on using nss to look up
username/groups (like nginx).

The current nginx example used the `runAsRoot` parameter to setup
/etc/group and /etc/passwd (which also doesn't exist in
buildLayeredImage), so we can now just use fakeNss there and use
buildLayeredImage.
2020-12-02 14:56:07 +01:00
Graham Christensen
bc49a0815a
utillinux: rename to util-linux 2020-11-24 12:42:06 -05:00
adisbladis
6847a5c12d
referencesByPopularity: Don't use buildPackages at call-site
It's cleaner to reference buildPackages directly in
build-support/docker directly.
2020-11-24 02:55:40 +01:00
adisbladis
8a3b33baed
dockerTools: Set correct architecture when cross compiling 2020-11-19 18:13:21 +01:00
adisbladis
4313ac6b29
dockerTools.buildLayeredImage: Fix cross compilation 2020-11-19 14:32:22 +01:00
Michael
24c5059468 docker: Fix broken pipe on docker layer creation
Add `-p` to the `tee` command to avoid exiting on breaking pipe due to
tarsum finishing before tar which creating docker layers.
2020-11-03 12:33:11 +01:00
Robert Hensing
11b0106cf9 dockerTools: add isExe to streamLayeredImage result 2020-10-05 10:48:17 +02:00
Utku Demir
ae82f81bfa
dockerTools.streamLayeredImage: Store the customisation layer as a tarball
This fixes as issue described here[1], where permissions set by 'extraCommands'
were ignored by Nix.

[1] https://github.com/NixOS/nixpkgs/pull/91084#issuecomment-669834938
2020-09-04 16:53:23 +12:00
Robert Hensing
8c0459f611 dockerTools: Always set imageTag attribute
The image tag can be specified or generated from the output hash.
Previously, a generated tag could be recovered from the evaluated
image with some string operations.

However, with the introduction of streamLayeredImage, it's not
feasible to compute the generated tag yourself.

With this change, the imageTag attribute is set unconditionally,
for the buildImage, buildLayeredImage, streamLayeredImage functions.
2020-07-11 16:58:25 +02:00
Utku Demir
4bff5b7f3d
dockerTools: Properly quote a shell variable 2020-06-21 12:35:39 +12:00
Utku Demir
307804d97d
dockerTools.streamLayeredImage: comments 2020-06-21 12:35:38 +12:00
Utku Demir
4e0109f873
Set mtime to get more deterministic builds 2020-06-21 12:35:37 +12:00
Utku Demir
560201da66
Implement dockerTools.streamLayeredImage 2020-06-21 12:35:32 +12:00
lewo
a498da343a
Merge pull request #87154 from utdemir/buildimage-optimizations
Some performance optimizations to dockerTools.build{,Layered}Image
2020-05-19 15:39:25 +02:00
zowoq
42232493a3 dockerTools: pass insecure-policy and tmpdir to skopeo 2020-05-16 08:46:13 +10:00
Utku Demir
f5a90a7aab
dockerTools.buildImage: Preserve environment variables from the parent image 2020-05-08 21:49:16 +12:00
Utku Demir
f12346d493
dockerTools: Calculate tarsum's on the fly
Calculating the tarsum after creating a layer is inefficient, since
we have to read the tarball we've just written from the disk.

This commit simultaneously calculates the tarsum while creating the
tarball.
2020-05-07 11:50:07 +12:00
adisbladis
fafb127947
dockertools: Add a buildLayeredImageWithNixDb function
This is analogous to buildImageWithNixDb but instead uses
buildLayeredImage under the hood.
2020-05-02 15:59:39 +01:00
Robert Hensing
6dab1b50a6 buildLayeredImage: Allow empty store, no paths to add
This is useful when buildLayeredImage is called in a generic way
that should allow simple (base) images to be built, which may not
reference any store paths.
2020-02-28 14:59:04 +01:00
Silvan Mosberger
0a351c3f65
dockerTools.*: Assertion against building for Darwin (#77952)
dockerTools.*: Assertion against building for Darwin
2020-01-31 21:17:40 +01:00
Antoine Eiche
01a68479cc dockerTools.buildLayeredImage: assert maxLayers > 1
Since a layer is reserved for "customization", the image can not
contains less than 2 layers.

The user gets the following message at evaluation:

    nix-instantiate nixos/tests/docker-tools.nix
    trace: the maxLayers argument of dockerTools.buildLayeredImage function must be greather than 1 (current value: 1)
2020-01-30 21:09:52 +01:00
Antoine Eiche
283bcc1003 dockerTools.buildLayeredImage: fix image with only 2 layers
A test is also added to ensure an image with 2 layers can be built.
2020-01-30 21:09:44 +01:00
Silvan Mosberger
b6a9211bf4
dockerTools.*: Assertion against building for darwin
Building a docker image with darwin binaries just yields a confusing
error when ran:

  standard_init_linux.go:211: exec user process caused "exec format error"

This change prevents people from building such images in the first place
2020-01-20 19:14:12 +01:00
Antoine Eiche
da261e3631 dockerTools.buildLayeredImage: fix typo in comments 2020-01-11 09:02:30 +01:00
Richard Wallace
3be767593b dockerTools.buildLayeredImage: fix building layered images in parallel
when tar'ing store paths into layered archives when building layered
images, don't use the absolute nix store path so that tar won't complain
if something new is added to the nix store

when building the final docker image, ignore any file changes tar
detects in the layers. they are all immutable and the only thing that
might change is the number of hard links due to store optimization
2019-12-30 14:47:11 -07:00
Graham Christensen
64453c8dbd
Merge pull request #75781 from grahamc/dockertools/remove-implementation-detail-layers
dockertools.buildLayeredImage: remove implementation detail layers
2019-12-17 16:03:11 -05:00
Graham Christensen
9c02760855
dockerTools.buildLayeredImage: update maxlayers from 24 to 100 to match documentation
mkManyPureLayers already was changed, and this function was not updated.
2019-12-16 13:14:21 -05:00
Graham Christensen
12e2416380
dockerTools.buildLayeredImage: Exclude top level implementation detail layers 2019-12-16 13:03:15 -05:00
Graham Christensen
700f4c5388
dockerTools.buildLayeredImage: prepare to exclude some paths
Without changing behavior, since this code is fiddly, make it possible
to add a filtering step before packaging individual paths.
2019-12-16 12:57:04 -05:00
Graham Christensen
aec80dddc0
dockerTools.buildLayeredImage: pass a list of closures to mkManyPureLayers so it can exclude the top-most level
Before, every docker image had three extra layers:

1. A `closure` layer which is an internal implementation detail of
   calculating the closure of the container
2. a `name-config.json` layer which is the images' run-time
   configuration, and has no business being *in* the image as a layer.
3. a "bulk-layers" layer which is again and implementation detail
   around collecting the image's closure.

None of these layers need to be in the final product.
2019-12-16 12:48:05 -05:00
Graham Christensen
f6d75f550e
dockerTools.buildLayeredImage: tweak formatting on contentsEnv 2019-12-16 12:36:45 -05:00
tomberek
81b0a20dfa buildImageWithNixDb: export USER (#74959)
dockerTools.buildImageWithNixDb: export USER

Changes to Nix user detection (./src/nix-channel/nix-channel.cc#L-166)
cause this function to error. Exporting USER fixes this.
2019-12-07 10:06:42 +01:00
Nick Spinale
8166bc934b build-support/docker: set default image arch to host arch
The architecture of an image should default to the architecture for
which that image is being composed or pulled. buildPackages.go.GOARCH is
an easy way to compute that architecture with the correct terminology.
2019-09-16 14:22:30 +00:00
Graham Christensen
8adaae2df2
dockertools.buildLayeredImage: default layer count up to 100 2019-08-12 19:59:40 -04:00
Ding Xiang Fei
ceab72e21b dockerTools: use skopeo on the right platform 2019-07-02 14:53:24 +08:00
volth
f3282c8d1e treewide: remove unused variables (#63177)
* treewide: remove unused variables

* making ofborg happy
2019-06-16 19:59:05 +00:00
Matthew Bauer
4e5c9b8cf4
Merge pull request #54921 from grabango/master
Use nativeBuildInputs for building Docker images
2019-06-10 21:38:47 -04:00
Antoine Eiche
a5a5820048 dockerTools: Fix the layer order
The layer order was not correct when a parent image was used: parent
image layers were above the new created layer.

This commits simplifies the code related to layer ordering. In
particular, layers in `layer-list` are ordered from bottom-most to
top-most. This is also the order of layers in the `rootfs.diff_ids`
attribute of the image configuration.
2019-05-07 16:52:25 +02:00
lewo
a8beff987d
Merge pull request #58350 from xtruder/pkgs/dockerTools/storePathToLayer/runtimeShell
dockerTools: storePathToLayer use runtimeShell in script
2019-04-06 19:01:19 +02:00
lewo
dc3ed336df
Merge pull request #58345 from xtruder/pkgs/dockerTools/pullImage/finalImageName
dockerTools: add finalImageName parameter for pullImage
2019-03-28 16:25:01 +01:00
Alex Biehl
1b1e23024b Strip leading ./ in customization layer 2019-03-27 12:08:50 +01:00
Jaka Hudoklin
468df177c4
dockerTools: add finalImageName parameter for pullImage 2019-03-26 19:35:14 +01:00
Jaka Hudoklin
5d429f6822
dockerTools: storePathToLayer use runtimeShell in script 2019-03-26 11:07:24 +01:00
Antoine Eiche
fe6860800b dockerTools.buildImage.runAsRoot: preserve layers ordering at image unpacking
This patch preserves the ordering of layers of a parent image when the
image is unpacked.

Fixes #55290
2019-03-12 12:04:22 +01:00
Wael M. Nasreddine
9b22a51712
build-support/docker: fix the build of tarsum with Go 1.12 2019-03-05 10:53:21 -08:00
Jörg Thalheim
dadc7eb329
treewide: use runtimeShell instead of stdenv.shell whenever possible
Whenever we create scripts that are installed to $out, we must use runtimeShell
in order to get the shell that can be executed on the machine we create the
package for. This is relevant for cross-compiling. The only use case for
stdenv.shell are scripts that are executed as part of the build system.
Usages in checkPhase are borderline however to decrease the likelyhood
of people copying the wrong examples, I decided to use runtimeShell as well.
2019-02-26 14:10:49 +00:00
Austin Seipp
c36c048c0e dockerTools: mark store-path-to-layer.sh as executable
bcf54ce5bb introduced a treewide change to
use ${stdenv.shell} where-ever possible. However, this broke a script
used by dockerTools, store-path-to-layer.sh, as it did not preserve the
+x mode bit. This meant the file got put into the store as mode 0444,
resulting in a build-time error later on that looked like:

    xargs: /nix/store/jixivxhh3c8sncp9xlkc4ls3y5f2mmxh-store-path-to-layer.sh: Permission denied

However, in a twist of fate, bcf54ce5bb
not only introduced this regression but, in this particular instance,
didn't even fix the original bug: the store-path-to-layer.sh script
*still* uses /bin/sh as its shebang line, rather than an absolute path
to stdenv. (Fixing this can be done in a separate commit.)

Signed-off-by: Austin Seipp <aseipp@pobox.com>
2019-02-21 08:30:47 -06:00