The patch that was previously fetched from GitHub is now part of the
4.5.1 release, and is no longer needed. Furthermore, 4.5.1 introduces
some new scripts to build manpages, and some new build inputs are
required to make that work.
This also rebases the Darwin patch. I don't have a Mac so I can't test
this personally. There was one conflict:
diff --cc squashfs-tools/read_xattrs.c
index 2067f80,b28c3a0..0000000
--- a/squashfs-tools/read_xattrs.c
+++ b/squashfs-tools/read_xattrs.c
@@@ -36,9 -38,7 +38,7 @@@
#include "xattr.h"
#include "error.h"
- #include <stdlib.h>
-
-extern int read_fs_bytes(int, long long, int, void *);
+extern int read_fs_bytes(int, long long, long long, void *);
extern int read_block(int, long long, long long *, int, void *);
static struct hash_entry {
Resolved by updating the signature from int to long long.
This is the same test which blocks nixos-unstable-small. It recently
caused a long blockage, due to a regression in squashfsTools itself
corrupting the iso image, see #132286.
I pretty much manually applied all the changes in the patch to the new codebase and then generated a new diff because the pull request mentioned at the top of the patch was not accepted and will probably never be updated again.
I had to update the 4k alignment patch for this. While it does apply,
and the result compiles, and even appears to work fine superficially,
I do not know if there have been any changes to squashfs internals that
now require other places to take the alignment flag into account. Will
do more testing soon.
the nix store may contain hardlinks: derivations may output them
directly, or users may be using store optimization which automatically
hardlinks identical files in the nix store.
The presence of these links are intended to be a 'transparent'
optimization. However, when creating a squashfs image, the image
will be different depending on whether hard links were present
on the filesystem, leading to reproducibility problems.
By passing '-no-hardlinks' to mksquashfs the files are stored
as duplicates in the squashfs image. Since squashfs has support
for duplicate files this does not lead to a larger image.
For more details see
https://github.com/NixOS/nixpkgs/issues/114331
This new patch is the patch between 4.4, and commit [1], a pull request
at [2] to upsteam Mac and BSD compatibility. That pull request is
itself a rebase of an older pull request for an earlier version of
squashfs-tools. I squashed all commits between 4.4 and [1], apart from
the BSD-specific ones, and exported the new patch from that. Attached
below is the git diff --ignore-space-change between squashfs-tools 4.4
with the previous patch applied, and with the new patch applied.
[1]: 7d31beec53e6245d3405d6ef2b96e9811ae07044
[2]: https://github.com/plougher/squashfs-tools/pull/69
---
diff --git a/squashfs-tools/info.c b/squashfs-tools/info.c
index 8efefe6..5c2f835 100644
--- a/squashfs-tools/info.c
+++ b/squashfs-tools/info.c
@@ -159,7 +159,7 @@ void *info_thrd(void *arg)
case EINTR:
continue;
default:
- BAD_ERROR("sigwaitfailed "
+ BAD_ERROR("sigwait failed "
"because %s\n", strerror(errno));
}
}
diff --git a/squashfs-tools/mksquashfs.c b/squashfs-tools/mksquashfs.c
index 67d86a5..3607448 100644
--- a/squashfs-tools/mksquashfs.c
+++ b/squashfs-tools/mksquashfs.c
@@ -35,7 +35,10 @@
#include <stddef.h>
#include <sys/types.h>
#include <sys/stat.h>
-#ifdef linux
+#ifndef linux
+#include <sys/sysctl.h>
+#else
+#include <sys/sysinfo.h>
#include <sys/sysmacros.h>
#endif
#include <fcntl.h>
@@ -52,7 +55,6 @@
#include <sys/wait.h>
#include <limits.h>
#include <ctype.h>
-#include <sys/sysinfo.h>
#ifndef FNM_EXTMATCH /* glibc extension */
#define FNM_EXTMATCH 0
@@ -5191,7 +5193,17 @@ int parse_mode(char *arg, mode_t *res)
int get_physical_memory()
{
+ /*
+ * Long longs are used here because with PAE, a 32-bit
+ * machine can have more than 4GB of physical memory
+ *
+ * sysconf(_SC_PHYS_PAGES) relies on /proc being mounted.
+ * If it fails use sysinfo, if that fails return 0
+ */
+ long long num_pages = sysconf(_SC_PHYS_PAGES);
+ long long page_size = sysconf(_SC_PAGESIZE);
int phys_mem;
+
#ifndef linux
#ifdef HW_MEMSIZE
#define SYSCTL_PHYSMEM HW_MEMSIZE
@@ -5221,16 +5233,6 @@ int get_physical_memory()
}
#undef SYSCTL_PHYSMEM
#else
- /*
- * Long longs are used here because with PAE, a 32-bit
- * machine can have more than 4GB of physical memory
- *
- * sysconf(_SC_PHYS_PAGES) relies on /proc being mounted.
- * If it fails use sysinfo, if that fails return 0
- */
- long long num_pages = sysconf(_SC_PHYS_PAGES);
- long long page_size = sysconf(_SC_PAGESIZE);
-
if(num_pages == -1 || page_size == -1) {
struct sysinfo sys;
int res = sysinfo(&sys);
diff --git a/squashfs-tools/mksquashfs.h b/squashfs-tools/mksquashfs.h
index 88d0b5c..1beefef 100644
--- a/squashfs-tools/mksquashfs.h
+++ b/squashfs-tools/mksquashfs.h
@@ -24,7 +24,6 @@
* mksquashfs.h
*
*/
-#include <pthread.h>
struct dir_info {
char *pathname;
diff --git a/squashfs-tools/pseudo.c b/squashfs-tools/pseudo.c
index f8fd529..48e6b27 100644
--- a/squashfs-tools/pseudo.c
+++ b/squashfs-tools/pseudo.c
@@ -30,7 +30,6 @@
#include <errno.h>
#include <string.h>
#include <stdlib.h>
-#include <sys/stat.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <sys/stat.h>
diff --git a/squashfs-tools/unsquashfs.c b/squashfs-tools/unsquashfs.c
index 00615ce..c1a6183 100644
--- a/squashfs-tools/unsquashfs.c
+++ b/squashfs-tools/unsquashfs.c
@@ -38,7 +38,6 @@
#include <sys/sysinfo.h>
#include <sys/sysmacros.h>
#endif
-
#include <sys/types.h>
#include <sys/time.h>
#include <sys/resource.h>
@@ -1085,7 +1084,7 @@ int create_inode(char *pathname, struct inode *i)
break;
case SQUASHFS_SYMLINK_TYPE:
case SQUASHFS_LSYMLINK_TYPE: {
- struct timespec times[2] = {
+ struct timeval times[2] = {
{ i->time, 0 },
{ i->time, 0 }
};
@@ -1104,8 +1103,7 @@ int create_inode(char *pathname, struct inode *i)
goto failed;
}
- res = utimensat(AT_FDCWD, pathname, times,
- AT_SYMLINK_NOFOLLOW);
+ res = lutimes(pathname, times);
if(res == -1) {
EXIT_UNSQUASH_STRICT("create_inode: failed to set time on "
"%s, because %s\n", pathname,
I took the patch, and applied it on top of the previous squashfs-tools
commit that we packaged. It applied cleanly. Then I rebased that on top
of master, and resolved the conflicts. I'm not sure I resolved them
correctly though, I don't have access to Darwin. Somebody needs to
review this.
I took the patch, and applied it on top of the previous squashfs-tools
commit that we packaged (which required editing one line in the patch,
as it assumed to be applied on top of the reproducibility patches). Then
I rebased that on top of master, resolved one conflict, and I formatted
a new patch for this.
A new release has been made upstream. Reproducibility issues were fixed
in that release, so we no longer need those patches. For a full overview
of the changes, see the 4.4-specific readme at [1].
The alignment patch no longer applies cleanly; I disabled it for now,
and I will try to restore it in a follow-up commit.
[1]: 52eb4c279c/README-4.4
The 4k-align squashfs patch was broken in the update to squashfs 4.4dev,
such that the patch was no longer actually applied in full (command line
option not even parsed).
Without these patches, the output of mksquashfs is not reproducible.
The patches are taken from https://github.com/squashfskit/squashfskit,
a fork of squashfs-tools, licensed under the GPL2 or later like
squashfs-tools itself.
This prevents failures like "libgcc_s.so.1 must be installed for
pthread_cancel to work" that occur because Glibc assumes libgcc_s.so.1
to be in Glibc's libdir.
This solution is pretty hacky, because the libgcc_s.so.1 from
bootstrap-tools might be too old. So if we update GCC, programs might
end up using an outdated libgcc_s.so.1. Ideally, we would build
libgcc_s.so.1 *before* Glibc, which might not be impossible...
Fixes#3548.
to be in the buildInputs). While at it, update to
version 4.2. "attr" is no longer needed as a dependency.
svn path=/nixpkgs/branches/stdenv-updates/; revision=31744
