2024-05-21 06:12:31 +00:00
/* *
2024-04-02 21:46:59 +00:00
< ! - - This anchor is here for backwards compatibility - ->
2023-11-18 22:46:50 +00:00
[ ] { #sec-fileset}
The [ ` lib . fileset ` ] ( #sec-functions-library-fileset) library allows you to work with _file sets_.
2023-11-18 23:57:21 +00:00
A file set is a ( mathematical ) set of local files that can be added to the Nix store for use in Nix derivations .
2023-11-18 22:46:50 +00:00
File sets are easy and safe to use , providing obvious and composable semantics with good error messages to prevent mistakes .
2024-05-21 06:12:31 +00:00
# Overview {#sec-fileset-overview}
2023-11-18 23:57:21 +00:00
Basics :
- [ Implicit coercion from paths to file sets ] ( #sec-fileset-path-coercion)
2023-11-07 00:04:13 +00:00
- [ ` lib . fileset . maybeMissing ` ] ( #function-library-lib.fileset.maybeMissing):
Create a file set from a path that may be missing .
2023-11-18 23:57:21 +00:00
- [ ` lib . fileset . trace ` ] ( #function-library-lib.fileset.trace)/[`lib.fileset.traceVal`](#function-library-lib.fileset.trace):
Pretty-print file sets for debugging .
- [ ` lib . fileset . toSource ` ] ( #function-library-lib.fileset.toSource):
Add files in file sets to the store to use as derivation sources .
2024-02-13 21:46:11 +00:00
- [ ` lib . fileset . toList ` ] ( #function-library-lib.fileset.toList):
The list of files contained in a file set .
2023-11-18 23:57:21 +00:00
Combinators :
- [ ` lib . fileset . union ` ] ( #function-library-lib.fileset.union)/[`lib.fileset.unions`](#function-library-lib.fileset.unions):
Create a larger file set from all the files in multiple file sets .
- [ ` lib . fileset . intersection ` ] ( #function-library-lib.fileset.intersection):
Create a smaller file set from only the files in both file sets .
- [ ` lib . fileset . difference ` ] ( #function-library-lib.fileset.difference):
Create a smaller file set containing all files that are in one file set , but not another one .
Filtering :
- [ ` lib . fileset . fileFilter ` ] ( #function-library-lib.fileset.fileFilter):
Create a file set from all files that satisisfy a predicate in a directory .
Utilities :
- [ ` lib . fileset . fromSource ` ] ( #function-library-lib.fileset.fromSource):
Create a file set from a ` lib . sources ` - based value .
- [ ` lib . fileset . gitTracked ` ] ( #function-library-lib.fileset.gitTracked)/[`lib.fileset.gitTrackedWith`](#function-library-lib.fileset.gitTrackedWith):
Create a file set from all tracked files in a local Git repository .
If you need more file set functions ,
see [ this issue ] ( https://github.com/NixOS/nixpkgs/issues/266356 ) to request it .
2023-11-18 22:46:50 +00:00
2024-05-21 06:12:31 +00:00
# Implicit coercion from paths to file sets {#sec-fileset-path-coercion}
2023-11-18 22:46:50 +00:00
All functions accepting file sets as arguments can also accept [ paths ] ( https://nixos.org/manual/nix/stable/language/values.html #type-path) as arguments.
Such path arguments are implicitly coerced to file sets containing all files under that path :
- A path to a file turns into a file set containing that single file .
- A path to a directory turns into a file set containing all files _recursively_ in that directory .
If the path points to a non-existent location , an error is thrown .
: : : { . note }
Just like in Git , file sets cannot represent empty directories .
Because of this , a path to a directory that contains no files ( recursively ) will turn into a file set containing no files .
: : :
: : : { . note }
File set coercion does _not_ add any of the files under the coerced paths to the store .
Only the [ ` toSource ` ] ( #function-library-lib.fileset.toSource) function adds files to the Nix store, and only those files contained in the `fileset` argument.
This is in contrast to using [ paths in string interpolation ] ( https://nixos.org/manual/nix/stable/language/values.html #type-path), which does add the entire referenced path to the store.
: : :
2024-05-21 06:12:31 +00:00
## Example {#sec-fileset-path-coercion-example}
2023-11-18 22:46:50 +00:00
Assume we are in a local directory with a file hierarchy like this :
` ` `
├ ─ a /
│ ├ ─ x ( file )
│ └ ─ b /
│ └ ─ y ( file )
└ ─ c /
└ ─ d /
` ` `
Here's a listing of which files get included when different path expressions get coerced to file sets :
- ` ./. ` as a file set contains both ` a/x ` and ` a/b/y ` ( ` c / ` does not contain any files and is therefore omitted ) .
- ` ./a ` as a file set contains both ` a/x ` and ` a/b/y ` .
- ` ./a/x ` as a file set contains only ` a/x ` .
- ` ./a/b ` as a file set contains only ` a/b/y ` .
- ` ./c ` as a file set is empty , since neither ` c ` nor ` c/d ` contain any files .
* /
2023-08-16 22:55:32 +00:00
{ lib }:
let
inherit ( import ./internal.nix { inherit lib ; } )
_coerce
2023-10-17 22:16:50 +00:00
_singleton
2023-09-13 21:29:28 +00:00
_coerceMany
2023-08-16 22:55:32 +00:00
_toSourceFilter
2023-10-17 22:16:50 +00:00
_fromSourceFilter
2024-02-13 21:46:11 +00:00
_toList
2023-09-13 21:29:28 +00:00
_unionMany
2023-10-10 17:10:02 +00:00
_fileFilter
2023-09-20 23:01:45 +00:00
_printFileset
2023-09-26 00:10:46 +00:00
_intersection
2023-10-04 21:23:30 +00:00
_difference
2023-12-11 21:56:11 +00:00
_fromFetchGit
2023-11-03 00:32:02 +00:00
_fetchGitSubmodulesMinver
2023-11-07 00:04:13 +00:00
_emptyWithoutBase
2023-08-16 22:55:32 +00:00
;
inherit ( builtins )
2023-11-03 00:32:02 +00:00
isBool
2023-09-13 21:29:57 +00:00
isList
2023-08-16 22:55:32 +00:00
isPath
pathExists
2023-09-20 23:01:45 +00:00
seq
2023-08-16 22:55:32 +00:00
typeOf
2023-11-03 00:32:02 +00:00
nixVersion
2023-08-16 22:55:32 +00:00
;
2023-09-13 21:29:57 +00:00
inherit ( lib . lists )
2023-09-26 00:10:46 +00:00
elemAt
2023-09-13 21:29:57 +00:00
imap0
;
2023-08-16 22:55:32 +00:00
inherit ( lib . path )
hasPrefix
splitRoot
;
inherit ( lib . strings )
isStringLike
2023-11-03 00:32:02 +00:00
versionOlder
2023-08-16 22:55:32 +00:00
;
inherit ( lib . filesystem )
pathType
;
inherit ( lib . sources )
cleanSourceWith
;
2023-09-14 22:08:04 +00:00
inherit ( lib . trivial )
2023-10-10 17:10:02 +00:00
isFunction
2023-09-14 22:08:04 +00:00
pipe
;
2023-08-16 22:55:32 +00:00
in {
2024-05-21 06:12:31 +00:00
/* *
2023-11-07 00:04:13 +00:00
Create a file set from a path that may or may not exist :
- If the path does exist , the path is [ coerced to a file set ] ( #sec-fileset-path-coercion).
- If the path does not exist , a file set containing no files is returned .
2024-05-21 06:12:31 +00:00
# Inputs
` path `
: 1 \ . Function argument
# Type
` ` `
maybeMissing : : Path -> FileSet
` ` `
# Examples
: : : { . example }
## `lib.fileset.maybeMissing` usage example
` ` ` nix
# All files in the current directory, but excluding main.o if it exists
difference ./. ( maybeMissing ./main.o )
` ` `
: : :
2023-11-07 00:04:13 +00:00
* /
maybeMissing =
path :
if ! isPath path then
if isStringLike path then
throw ''
lib . fileset . maybeMissing : Argument ( " ${ toString path } " ) is a string-like value , but it should be a path instead . ''
else
throw ''
lib . fileset . maybeMissing : Argument is of type $ { typeOf path } , but it should be a path instead . ''
else if ! pathExists path then
_emptyWithoutBase
else
_singleton path ;
2024-05-21 06:12:31 +00:00
/* *
2023-11-19 00:02:07 +00:00
Incrementally evaluate and trace a file set in a pretty way .
This function is only intended for debugging purposes .
The exact tracing format is unspecified and may change .
This function takes a final argument to return .
In comparison , [ ` traceVal ` ] ( #function-library-lib.fileset.traceVal) returns
the given file set argument .
This variant is useful for tracing file sets in the Nix repl .
2024-05-21 06:12:31 +00:00
# Inputs
` fileset `
: The file set to trace .
This argument can also be a path ,
which gets [ implicitly coerced to a file set ] ( #sec-fileset-path-coercion).
` val `
: The value to return .
# Type
` ` `
trace : : FileSet -> Any -> Any
` ` `
# Examples
: : : { . example }
## `lib.fileset.trace` usage example
` ` ` nix
trace ( unions [ ./Makefile ./src ./tests/run.sh ] ) null
= >
trace : /home/user/src/myProject
trace : - Makefile ( regular )
trace : - src ( all files in directory )
trace : - tests
trace : - run . sh ( regular )
null
` ` `
: : :
2023-11-19 00:02:07 +00:00
* /
2024-05-21 06:12:31 +00:00
trace = fileset :
2023-11-19 00:02:07 +00:00
let
# "fileset" would be a better name, but that would clash with the argument name,
# and we cannot change that because of https://github.com/nix-community/nixdoc/issues/76
actualFileset = _coerce " l i b . f i l e s e t . t r a c e : A r g u m e n t " fileset ;
in
seq
( _printFileset actualFileset )
( x : x ) ;
2024-05-21 06:12:31 +00:00
/* *
2023-11-19 00:02:07 +00:00
Incrementally evaluate and trace a file set in a pretty way .
This function is only intended for debugging purposes .
The exact tracing format is unspecified and may change .
This function returns the given file set .
In comparison , [ ` trace ` ] ( #function-library-lib.fileset.trace) takes another argument to return.
This variant is useful for tracing file sets passed as arguments to other functions .
2024-05-21 06:12:31 +00:00
# Inputs
` fileset `
: The file set to trace and return .
This argument can also be a path ,
which gets [ implicitly coerced to a file set ] ( #sec-fileset-path-coercion).
# Type
` ` `
traceVal : : FileSet -> FileSet
` ` `
# Examples
: : : { . example }
## `lib.fileset.traceVal` usage example
` ` ` nix
toSource {
root = ./. ;
fileset = traceVal ( unions [
./Makefile
./src
./tests/run.sh
] ) ;
}
= >
trace : /home/user/src/myProject
trace : - Makefile ( regular )
trace : - src ( all files in directory )
trace : - tests
trace : - run . sh ( regular )
" / n i x / s t o r e / . . . - s o u r c e "
` ` `
: : :
2023-11-19 00:02:07 +00:00
* /
2024-05-21 06:12:31 +00:00
traceVal = fileset :
2023-11-19 00:02:07 +00:00
let
# "fileset" would be a better name, but that would clash with the argument name,
# and we cannot change that because of https://github.com/nix-community/nixdoc/issues/76
actualFileset = _coerce " l i b . f i l e s e t . t r a c e V a l : A r g u m e n t " fileset ;
in
seq
( _printFileset actualFileset )
# We could also return the original fileset argument here,
# but that would then duplicate work for consumers of the fileset, because then they have to coerce it again
actualFileset ;
2024-05-21 06:12:31 +00:00
/* *
2023-08-16 22:55:32 +00:00
Add the local files contained in ` fileset ` to the store as a single [ store path ] ( https://nixos.org/manual/nix/stable/glossary #gloss-store-path) rooted at `root`.
The result is the store path as a string-like value , making it usable e . g . as the ` src ` of a derivation , or in string interpolation :
` ` ` nix
stdenv . mkDerivation {
src = lib . fileset . toSource { . . . } ;
# ...
}
` ` `
The name of the store path is always ` source ` .
2024-05-21 06:12:31 +00:00
# Inputs
Takes an attribute set with the following attributes
` root ` ( Path ; _required_ )
: The local directory [ path ] ( https://nixos.org/manual/nix/stable/language/values.html #type-path) that will correspond to the root of the resulting store path.
2023-08-16 22:55:32 +00:00
Paths in [ strings ] ( https://nixos.org/manual/nix/stable/language/values.html #type-string), including Nix store paths, cannot be passed as `root`.
` root ` has to be a directory .
2023-11-08 22:28:25 +00:00
: : : { . note }
Changing ` root ` only affects the directory structure of the resulting store path , it does not change which files are added to the store .
The only way to change which files get added to the store is by changing the ` fileset ` attribute .
: : :
2024-05-21 06:12:31 +00:00
` fileset ` ( FileSet ; _required_ )
: The file set whose files to import into the store .
2023-09-13 21:31:02 +00:00
File sets can be created using other functions in this library .
This argument can also be a path ,
which gets [ implicitly coerced to a file set ] ( #sec-fileset-path-coercion).
2023-11-08 22:28:25 +00:00
: : : { . note }
If a directory does not recursively contain any file , it is omitted from the store path contents .
: : :
2023-09-13 21:31:02 +00:00
2024-05-21 06:12:31 +00:00
# Type
` ` `
toSource : : {
root : : Path ,
fileset : : FileSet ,
} -> SourceLike
` ` `
# Examples
: : : { . example }
## `lib.fileset.toSource` usage example
` ` ` nix
# Import the current directory into the store
# but only include files under ./src
toSource {
root = ./. ;
fileset = ./src ;
}
= > " / n i x / s t o r e / . . . - s o u r c e "
# Import the current directory into the store
# but only include ./Makefile and all files under ./src
toSource {
root = ./. ;
fileset = union
./Makefile
./src ;
}
= > " / n i x / s t o r e / . . . - s o u r c e "
# Trying to include a file outside the root will fail
toSource {
root = ./. ;
fileset = unions [
./Makefile
./src
../LICENSE
] ;
}
= > <error>
# The root needs to point to a directory that contains all the files
toSource {
root = ../. ;
fileset = unions [
./Makefile
./src
../LICENSE
] ;
}
= > " / n i x / s t o r e / . . . - s o u r c e "
# The root has to be a local filesystem path
toSource {
root = " / n i x / s t o r e / . . . - s o u r c e " ;
fileset = ./. ;
}
= > <error>
` ` `
: : :
* /
toSource = {
root ,
2023-08-16 22:55:32 +00:00
fileset ,
} :
let
# We cannot rename matched attribute arguments, so let's work around it with an extra `let in` statement
2023-09-14 22:08:04 +00:00
filesetArg = fileset ;
2023-08-16 22:55:32 +00:00
in
let
2023-09-14 22:08:04 +00:00
fileset = _coerce " l i b . f i l e s e t . t o S o u r c e : ` f i l e s e t ` " filesetArg ;
2023-08-16 22:55:32 +00:00
rootFilesystemRoot = ( splitRoot root ) . root ;
filesetFilesystemRoot = ( splitRoot fileset . _internalBase ) . root ;
2023-09-14 22:08:04 +00:00
sourceFilter = _toSourceFilter fileset ;
2023-08-16 22:55:32 +00:00
in
if ! isPath root then
2023-11-08 20:44:44 +00:00
if root ? _isLibCleanSourceWith then
throw ''
lib . fileset . toSource : ` root ` is a ` lib . sources ` - based value , but it should be a path instead .
To use a ` lib . sources ` - based value , convert it to a file set using ` lib . fileset . fromSource ` and pass it as ` fileset ` .
Note that this only works for sources created from paths . ''
else if isStringLike root then
2023-08-16 22:55:32 +00:00
throw ''
2023-11-08 22:59:04 +00:00
lib . fileset . toSource : ` root ` ( $ { toString root } ) is a string-like value , but it should be a path instead .
2023-08-16 22:55:32 +00:00
Paths in strings are not supported by ` lib . fileset ` , use ` lib . sources ` or derivations instead . ''
else
throw ''
lib . fileset . toSource : ` root ` is of type $ { typeOf root } , but it should be a path instead . ''
# Currently all Nix paths have the same filesystem root, but this could change in the future.
# See also ../path/README.md
2023-09-25 22:26:00 +00:00
else if ! fileset . _internalIsEmptyWithoutBase && rootFilesystemRoot != filesetFilesystemRoot then
2023-08-16 22:55:32 +00:00
throw ''
2023-11-08 22:59:04 +00:00
lib . fileset . toSource : Filesystem roots are not the same for ` fileset ` and ` root ` ( $ { toString root } ) :
` root ` : Filesystem root is " ${ toString rootFilesystemRoot } "
` fileset ` : Filesystem root is " ${ toString filesetFilesystemRoot } "
Different filesystem roots are not supported . ''
2023-08-16 22:55:32 +00:00
else if ! pathExists root then
throw ''
2023-11-08 22:59:04 +00:00
lib . fileset . toSource : ` root ` ( $ { toString root } ) is a path that does not exist . ''
2023-08-16 22:55:32 +00:00
else if pathType root != " d i r e c t o r y " then
throw ''
2023-09-14 22:08:04 +00:00
lib . fileset . toSource : ` root ` ( $ { toString root } ) is a file , but it should be a directory instead . Potential solutions :
2023-08-16 22:55:32 +00:00
- If you want to import the file into the store _without_ a containing directory , use string interpolation or ` builtins . path ` instead of this function .
- If you want to import the file into the store _with_ a containing directory , set ` root ` to the containing directory , such as $ { toString ( dirOf root ) } , and set ` fileset ` to the file path . ''
2023-09-25 22:26:00 +00:00
else if ! fileset . _internalIsEmptyWithoutBase && ! hasPrefix root fileset . _internalBase then
2023-08-16 22:55:32 +00:00
throw ''
2023-09-14 22:08:04 +00:00
lib . fileset . toSource : ` fileset ` could contain files in $ { toString fileset . _internalBase } , which is not under the ` root ` ( $ { toString root } ) . Potential solutions :
2023-08-16 22:55:32 +00:00
- Set ` root ` to $ { toString fileset . _internalBase } or any directory higher up . This changes the layout of the resulting store path .
2023-09-14 22:08:04 +00:00
- Set ` fileset ` to a file set that cannot contain files outside the ` root ` ( $ { toString root } ) . This could change the files included in the result . ''
2023-08-16 22:55:32 +00:00
else
2023-11-13 22:57:52 +00:00
seq sourceFilter
2023-08-16 22:55:32 +00:00
cleanSourceWith {
name = " s o u r c e " ;
src = root ;
2023-09-14 22:08:04 +00:00
filter = sourceFilter ;
2023-08-16 22:55:32 +00:00
} ;
2023-09-13 21:29:28 +00:00
2024-02-13 21:46:11 +00:00
2024-05-21 06:12:31 +00:00
/* *
2024-02-13 21:46:11 +00:00
The list of file paths contained in the given file set .
: : : { . note }
This function is strict in the entire file set .
This is in contrast with combinators [ ` lib . fileset . union ` ] ( #function-library-lib.fileset.union),
[ ` lib . fileset . intersection ` ] ( #function-library-lib.fileset.intersection) and [`lib.fileset.difference`](#function-library-lib.fileset.difference).
Thus it is recommended to call ` toList ` on file sets created using the combinators ,
instead of doing list processing on the result of ` toList ` .
: : :
The resulting list of files can be turned back into a file set using [ ` lib . fileset . unions ` ] ( #function-library-lib.fileset.unions).
2024-05-21 06:12:31 +00:00
# Inputs
` fileset `
: The file set whose file paths to return . This argument can also be a path , which gets [ implicitly coerced to a file set ] ( #sec-fileset-path-coercion).
# Type
` ` `
toList : : FileSet -> [ Path ]
` ` `
2024-02-13 21:46:11 +00:00
2024-05-21 06:12:31 +00:00
# Examples
: : : { . example }
## `lib.fileset.toList` usage example
` ` ` nix
toList ./.
[ ./README.md ./Makefile ./src/main.c ./src/main.h ]
toList ( difference ./. ./src )
[ ./README.md ./Makefile ]
` ` `
: : :
2024-02-13 21:46:11 +00:00
* /
2024-05-21 06:12:31 +00:00
toList = fileset :
2024-02-13 21:46:11 +00:00
_toList ( _coerce " l i b . f i l e s e t . t o L i s t : A r g u m e n t " fileset ) ;
2024-05-21 06:12:31 +00:00
/* *
2023-09-13 21:29:28 +00:00
The file set containing all files that are in either of two given file sets .
2023-09-13 21:29:57 +00:00
This is the same as [ ` unions ` ] ( #function-library-lib.fileset.unions),
but takes just two file sets instead of a list .
2023-09-13 21:29:28 +00:00
See also [ Union ( set theory ) ] ( https://en.wikipedia.org/wiki/Union_ ( set_theory ) ) .
The given file sets are evaluated as lazily as possible ,
with the first argument being evaluated first if needed .
2024-05-21 06:12:31 +00:00
# Inputs
` fileset1 `
: The first file set . This argument can also be a path , which gets [ implicitly coerced to a file set ] ( #sec-fileset-path-coercion).
` fileset2 `
: The second file set . This argument can also be a path , which gets [ implicitly coerced to a file set ] ( #sec-fileset-path-coercion).
# Type
` ` `
union : : FileSet -> FileSet -> FileSet
` ` `
2023-09-13 21:29:28 +00:00
2024-05-21 06:12:31 +00:00
# Examples
: : : { . example }
## `lib.fileset.union` usage example
` ` ` nix
# Create a file set containing the file `Makefile`
# and all files recursively in the `src` directory
union ./Makefile ./src
# Create a file set containing the file `Makefile`
# and the LICENSE file from the parent directory
union ./Makefile ../LICENSE
` ` `
: : :
2023-09-13 21:29:28 +00:00
* /
union =
fileset1 :
fileset2 :
2023-09-14 22:08:04 +00:00
_unionMany
( _coerceMany " l i b . f i l e s e t . u n i o n " [
2023-09-13 21:29:28 +00:00
{
2023-11-08 22:59:04 +00:00
context = " F i r s t a r g u m e n t " ;
2023-09-13 21:29:28 +00:00
value = fileset1 ;
}
{
2023-11-08 22:59:04 +00:00
context = " S e c o n d a r g u m e n t " ;
2023-09-13 21:29:28 +00:00
value = fileset2 ;
}
2023-09-14 22:08:04 +00:00
] ) ;
2023-09-13 21:29:28 +00:00
2024-05-21 06:12:31 +00:00
/* *
2023-09-13 21:29:57 +00:00
The file set containing all files that are in any of the given file sets .
This is the same as [ ` union ` ] ( #function-library-lib.fileset.unions),
but takes a list of file sets instead of just two .
See also [ Union ( set theory ) ] ( https://en.wikipedia.org/wiki/Union_ ( set_theory ) ) .
The given file sets are evaluated as lazily as possible ,
with earlier elements being evaluated first if needed .
2024-05-21 06:12:31 +00:00
# Inputs
2023-09-13 21:29:57 +00:00
2024-05-21 06:12:31 +00:00
` filesets `
2023-09-13 21:29:57 +00:00
2024-05-21 06:12:31 +00:00
: A list of file sets . The elements can also be paths , which get [ implicitly coerced to file sets ] ( #sec-fileset-path-coercion).
2023-09-13 21:29:57 +00:00
2024-05-21 06:12:31 +00:00
# Type
` ` `
unions : : [ FileSet ] -> FileSet
` ` `
# Examples
: : : { . example }
## `lib.fileset.unions` usage example
` ` ` nix
# Create a file set containing selected files
unions [
# Include the single file `Makefile` in the current directory
# This errors if the file doesn't exist
./Makefile
# Recursively include all files in the `src/code` directory
# If this directory is empty this has no effect
./src/code
# Include the files `run.sh` and `unit.c` from the `tests` directory
./tests/run.sh
./tests/unit.c
# Include the `LICENSE` file from the parent directory
../LICENSE
]
` ` `
: : :
2023-09-13 21:29:57 +00:00
* /
unions =
filesets :
2023-09-14 22:08:04 +00:00
if ! isList filesets then
2023-11-08 22:59:04 +00:00
throw ''
lib . fileset . unions : Argument is of type $ { typeOf filesets } , but it should be a list instead . ''
2023-09-13 21:29:57 +00:00
else
2023-09-14 22:08:04 +00:00
pipe filesets [
# Annotate the elements with context, used by _coerceMany for better errors
( imap0 ( i : el : {
2023-11-08 22:59:04 +00:00
context = " E l e m e n t ${ toString i } " ;
2023-09-14 22:08:04 +00:00
value = el ;
} ) )
( _coerceMany " l i b . f i l e s e t . u n i o n s " )
_unionMany
] ;
2023-09-13 21:29:57 +00:00
2024-05-21 06:12:31 +00:00
/* *
2023-09-26 00:10:46 +00:00
The file set containing all files that are in both of two given file sets .
See also [ Intersection ( set theory ) ] ( https://en.wikipedia.org/wiki/Intersection_ ( set_theory ) ) .
The given file sets are evaluated as lazily as possible ,
with the first argument being evaluated first if needed .
2024-05-21 06:12:31 +00:00
# Inputs
` fileset1 `
: The first file set . This argument can also be a path , which gets [ implicitly coerced to a file set ] ( #sec-fileset-path-coercion).
` fileset2 `
: The second file set . This argument can also be a path , which gets [ implicitly coerced to a file set ] ( #sec-fileset-path-coercion).
# Type
` ` `
intersection : : FileSet -> FileSet -> FileSet
` ` `
# Examples
: : : { . example }
## `lib.fileset.intersection` usage example
` ` ` nix
# Limit the selected files to the ones in ./., so only ./src and ./Makefile
intersection ./. ( unions [ ../LICENSE ./src ./Makefile ] )
` ` `
: : :
2023-09-26 00:10:46 +00:00
* /
intersection =
fileset1 :
fileset2 :
let
filesets = _coerceMany " l i b . f i l e s e t . i n t e r s e c t i o n " [
{
2023-11-08 22:59:04 +00:00
context = " F i r s t a r g u m e n t " ;
2023-09-26 00:10:46 +00:00
value = fileset1 ;
}
{
2023-11-08 22:59:04 +00:00
context = " S e c o n d a r g u m e n t " ;
2023-09-26 00:10:46 +00:00
value = fileset2 ;
}
] ;
in
_intersection
( elemAt filesets 0 )
( elemAt filesets 1 ) ;
2024-05-21 06:12:31 +00:00
/* *
2023-10-04 21:23:30 +00:00
The file set containing all files from the first file set that are not in the second file set .
See also [ Difference ( set theory ) ] ( https://en.wikipedia.org/wiki/Complement_ ( set_theory ) #Relative_complement).
The given file sets are evaluated as lazily as possible ,
with the first argument being evaluated first if needed .
2024-05-21 06:12:31 +00:00
# Inputs
` positive `
: The positive file set . The result can only contain files that are also in this file set . This argument can also be a path , which gets [ implicitly coerced to a file set ] ( #sec-fileset-path-coercion).
` negative `
: The negative file set . The result will never contain files that are also in this file set . This argument can also be a path , which gets [ implicitly coerced to a file set ] ( #sec-fileset-path-coercion).
# Type
` ` `
union : : FileSet -> FileSet -> FileSet
` ` `
# Examples
: : : { . example }
## `lib.fileset.difference` usage example
` ` ` nix
# Create a file set containing all files from the current directory,
# except ones under ./tests
difference ./. ./tests
let
# A set of Nix-related files
nixFiles = unions [ ./default.nix ./nix ./tests/default.nix ] ;
in
# Create a file set containing all files under ./tests, except ones in `nixFiles`,
# meaning only without ./tests/default.nix
difference ./tests nixFiles
` ` `
: : :
2023-10-04 21:23:30 +00:00
* /
difference =
positive :
negative :
let
filesets = _coerceMany " l i b . f i l e s e t . d i f f e r e n c e " [
{
2023-11-08 22:59:04 +00:00
context = " F i r s t a r g u m e n t ( p o s i t i v e s e t ) " ;
2023-10-04 21:23:30 +00:00
value = positive ;
}
{
2023-11-08 22:59:04 +00:00
context = " S e c o n d a r g u m e n t ( n e g a t i v e s e t ) " ;
2023-10-04 21:23:30 +00:00
value = negative ;
}
] ;
in
_difference
( elemAt filesets 0 )
( elemAt filesets 1 ) ;
2024-05-21 06:12:31 +00:00
/* *
2023-11-19 00:02:07 +00:00
Filter a file set to only contain files matching some predicate .
2023-09-20 23:43:39 +00:00
2024-05-21 06:12:31 +00:00
# Inputs
` predicate `
: The predicate function to call on all files contained in given file set .
2023-11-19 00:02:07 +00:00
A file is included in the resulting file set if this function returns true for it .
2023-09-20 23:01:45 +00:00
2023-11-19 00:02:07 +00:00
This function is called with an attribute set containing these attributes :
- ` name ` ( String ) : The name of the file
- ` type ` ( String , one of ` " r e g u l a r " ` , ` " s y m l i n k " ` or ` " u n k n o w n " ` ) : The type of the file .
This matches result of calling [ ` builtins . readFileType ` ] ( https://nixos.org/manual/nix/stable/language/builtins.html #builtins-readFileType) on the file's path.
2023-11-09 00:29:38 +00:00
- ` hasExt ` ( String -> Bool ) : Whether the file has a certain file extension .
` hasExt ext ` is true only if ` hasSuffix " . ${ ext } " name ` .
This also means that e . g . for a file with name ` . gitignore ` ,
` hasExt " g i t i g n o r e " ` is true .
2023-11-19 00:02:07 +00:00
Other attributes may be added in the future .
2024-05-21 06:12:31 +00:00
` path `
: The path whose files to filter
# Type
` ` `
fileFilter : :
( {
name : : String ,
type : : String ,
hasExt : : String -> Bool ,
. . .
} -> Bool )
-> Path
-> FileSet
` ` `
# Examples
: : : { . example }
## `lib.fileset.fileFilter` usage example
` ` ` nix
# Include all regular `default.nix` files in the current directory
fileFilter ( file : file . name == " d e f a u l t . n i x " ) ./.
# Include all non-Nix files from the current directory
fileFilter ( file : ! file . hasExt " n i x " ) ./.
# Include all files that start with a "." in the current directory
fileFilter ( file : hasPrefix " . " file . name ) ./.
# Include all regular files (not symlinks or others) in the current directory
fileFilter ( file : file . type == " r e g u l a r " ) ./.
` ` `
: : :
* /
fileFilter =
2023-11-19 00:02:07 +00:00
predicate :
path :
if ! isFunction predicate then
throw ''
lib . fileset . fileFilter : First argument is of type $ { typeOf predicate } , but it should be a function instead . ''
else if ! isPath path then
if path . _type or " " == " f i l e s e t " then
throw ''
lib . fileset . fileFilter : Second argument is a file set , but it should be a path instead .
If you need to filter files in a file set , use ` intersection fileset ( fileFilter pred ./. ) ` instead . ''
else
throw ''
lib . fileset . fileFilter : Second argument is of type $ { typeOf path } , but it should be a path instead . ''
else if ! pathExists path then
throw ''
lib . fileset . fileFilter : Second argument ( $ { toString path } ) is a path that does not exist . ''
else
_fileFilter predicate path ;
2023-09-20 23:01:45 +00:00
2024-05-21 06:12:31 +00:00
/* *
Create a file set with the same files as a ` lib . sources ` - based value .
This does not import any of the files into the store .
2023-09-20 23:43:39 +00:00
2024-05-21 06:12:31 +00:00
This can be used to gradually migrate from ` lib . sources ` - based filtering to ` lib . fileset ` .
2023-09-20 23:43:39 +00:00
2024-05-21 06:12:31 +00:00
A file set can be turned back into a source using [ ` toSource ` ] ( #function-library-lib.fileset.toSource).
2023-09-20 23:43:39 +00:00
2024-05-21 06:12:31 +00:00
: : : { . note }
File sets cannot represent empty directories .
Turning the result of this function back into a source using ` toSource ` will therefore not preserve empty directories .
: : :
# Inputs
2023-09-20 23:43:39 +00:00
2024-05-21 06:12:31 +00:00
` source `
: 1 \ . Function argument
# Type
` ` `
2023-11-19 00:02:07 +00:00
fromSource : : SourceLike -> FileSet
2024-05-21 06:12:31 +00:00
` ` `
# Examples
: : : { . example }
## `lib.fileset.fromSource` usage example
2023-09-20 23:43:39 +00:00
2024-05-21 06:12:31 +00:00
` ` ` nix
2023-11-19 00:02:07 +00:00
# There's no cleanSource-like function for file sets yet,
# but we can just convert cleanSource to a file set and use it that way
toSource {
root = ./. ;
fileset = fromSource ( lib . sources . cleanSource ./. ) ;
}
# Keeping a previous sourceByRegex (which could be migrated to `lib.fileset.unions`),
# but removing a subdirectory using file set functions
difference
( fromSource ( lib . sources . sourceByRegex ./. [
" ^ R E A D M E \. m d $ "
# This regex includes everything in ./doc
" ^ d o c ( / . * ) ? $ "
] )
./doc/generated
# Use cleanSource, but limit it to only include ./Makefile and files under ./src
intersection
( fromSource ( lib . sources . cleanSource ./. ) )
( unions [
./Makefile
./src
] ) ;
2024-05-21 06:12:31 +00:00
` ` `
: : :
2023-11-19 00:02:07 +00:00
* /
fromSource = source :
2023-09-20 23:43:39 +00:00
let
2023-11-19 00:02:07 +00:00
# This function uses `._isLibCleanSourceWith`, `.origSrc` and `.filter`,
# which are technically internal to lib.sources,
# but we'll allow this since both libraries are in the same code base
# and this function is a bridge between them.
isFiltered = source ? _isLibCleanSourceWith ;
path = if isFiltered then source . origSrc else source ;
2023-09-20 23:43:39 +00:00
in
2023-11-19 00:02:07 +00:00
# We can only support sources created from paths
if ! isPath path then
if isStringLike path then
throw ''
lib . fileset . fromSource : The source origin of the argument is a string-like value ( " ${ toString path } " ) , but it should be a path instead .
Sources created from paths in strings cannot be turned into file sets , use ` lib . sources ` or derivations instead . ''
else
throw ''
lib . fileset . fromSource : The source origin of the argument is of type $ { typeOf path } , but it should be a path instead . ''
else if ! pathExists path then
throw ''
lib . fileset . fromSource : The source origin ( $ { toString path } ) of the argument is a path that does not exist . ''
else if isFiltered then
_fromSourceFilter path source . filter
else
# If there's no filter, no need to run the expensive conversion, all subpaths will be included
_singleton path ;
2023-11-03 00:24:55 +00:00
2024-05-21 06:12:31 +00:00
/* *
2023-11-03 00:24:55 +00:00
Create a file set containing all [ Git-tracked files ] ( https://git-scm.com/book/en/v2/Git-Basics-Recording-Changes-to-the-Repository ) in a repository .
This function behaves like [ ` gitTrackedWith { } ` ] ( #function-library-lib.fileset.gitTrackedWith) - using the defaults.
2024-05-21 06:12:31 +00:00
# Inputs
` path `
2023-11-03 00:24:55 +00:00
2024-05-21 06:12:31 +00:00
: The [ path ] ( https://nixos.org/manual/nix/stable/language/values #type-path) to the working directory of a local Git repository.
This directory must contain a ` . git ` file or subdirectory .
# Type
` ` `
gitTracked : : Path -> FileSet
` ` `
# Examples
: : : { . example }
## `lib.fileset.gitTracked` usage example
` ` ` nix
# Include all files tracked by the Git repository in the current directory
gitTracked ./.
# Include only files tracked by the Git repository in the parent directory
# that are also in the current directory
intersection ./. ( gitTracked ../. )
` ` `
: : :
2023-11-03 00:24:55 +00:00
* /
gitTracked =
path :
2023-12-11 21:56:11 +00:00
_fromFetchGit
" g i t T r a c k e d "
" a r g u m e n t "
path
{ } ;
2023-11-03 00:24:55 +00:00
2024-05-21 06:12:31 +00:00
/* *
2023-11-03 00:24:55 +00:00
Create a file set containing all [ Git-tracked files ] ( https://git-scm.com/book/en/v2/Git-Basics-Recording-Changes-to-the-Repository ) in a repository .
The first argument allows configuration with an attribute set ,
while the second argument is the path to the Git working tree .
2023-12-13 04:42:19 +00:00
` gitTrackedWith ` does not perform any filtering when the path is a [ Nix store path ] ( https://nixos.org/manual/nix/stable/store/store-path.html #store-path) and not a repository.
In this way , it accommodates the use case where the expression that makes the ` gitTracked ` call does not reside in an actual git repository anymore ,
and has presumably already been fetched in a way that excludes untracked files .
Fetchers with such equivalent behavior include ` builtins . fetchGit ` , ` builtins . fetchTree ` ( experimental ) , and ` pkgs . fetchgit ` when used without ` leaveDotGit ` .
2023-11-03 00:24:55 +00:00
If you don't need the configuration ,
you can use [ ` gitTracked ` ] ( #function-library-lib.fileset.gitTracked) instead.
This is equivalent to the result of [ ` unions ` ] ( #function-library-lib.fileset.unions) on all files returned by [`git ls-files`](https://git-scm.com/docs/git-ls-files)
( which uses [ ` - - cached ` ] ( https://git-scm.com/docs/git-ls-files #Documentation/git-ls-files.txt--c) by default).
: : : { . warning }
Currently this function is based on [ ` builtins . fetchGit ` ] ( https://nixos.org/manual/nix/stable/language/builtins.html #builtins-fetchGit)
As such , this function causes all Git-tracked files to be unnecessarily added to the Nix store ,
without being re-usable by [ ` toSource ` ] ( #function-library-lib.fileset.toSource).
This may change in the future .
: : :
2024-05-21 06:12:31 +00:00
# Inputs
` options ` ( attribute set )
: ` recurseSubmodules ` ( optional , default : ` false ` )
: Whether to recurse into [ Git submodules ] ( https://git-scm.com/book/en/v2/Git-Tools-Submodules ) to also include their tracked files .
If ` true ` , this is equivalent to passing the [ - - recurse-submodules ] ( https://git-scm.com/docs/git-ls-files #Documentation/git-ls-files.txt---recurse-submodules) flag to `git ls-files`.
` path `
: The [ path ] ( https://nixos.org/manual/nix/stable/language/values #type-path) to the working directory of a local Git repository.
This directory must contain a ` . git ` file or subdirectory .
# Type
` ` `
gitTrackedWith : : { recurseSubmodules : : Bool ? false } -> Path -> FileSet
` ` `
# Examples
: : : { . example }
## `lib.fileset.gitTrackedWith` usage example
` ` ` nix
# Include all files tracked by the Git repository in the current directory
# and any submodules under it
gitTracked { recurseSubmodules = true ; } ./.
` ` `
: : :
2023-11-03 00:24:55 +00:00
* /
gitTrackedWith =
{
2023-11-03 00:32:02 +00:00
recurseSubmodules ? false ,
2023-11-03 00:24:55 +00:00
} :
path :
2023-12-11 21:56:11 +00:00
if ! isBool recurseSubmodules then
2023-11-03 00:32:02 +00:00
throw " l i b . f i l e s e t . g i t T r a c k e d W i t h : E x p e c t e d t h e a t t r i b u t e ` r e c u r s e S u b m o d u l e s ` o f t h e f i r s t a r g u m e n t t o b e a b o o l e a n , b u t i t ' s a ${ typeOf recurseSubmodules } i n s t e a d . "
else if recurseSubmodules && versionOlder nixVersion _fetchGitSubmodulesMinver then
throw " l i b . f i l e s e t . g i t T r a c k e d W i t h : S e t t i n g t h e a t t r i b u t e ` r e c u r s e S u b m o d u l e s ` t o ` t r u e ` i s o n l y s u p p o r t e d f o r N i x v e r s i o n ${ _fetchGitSubmodulesMinver } a n d a f t e r , b u t N i x v e r s i o n ${ nixVersion } i s u s e d . "
2023-11-03 00:24:55 +00:00
else
2023-12-11 21:56:11 +00:00
_fromFetchGit
" g i t T r a c k e d W i t h "
" s e c o n d a r g u m e n t "
path
# This is the only `fetchGit` parameter that makes sense in this context.
# We can't just pass `submodules = recurseSubmodules` here because
# this would fail for Nix versions that don't support `submodules`.
( lib . optionalAttrs recurseSubmodules {
submodules = true ;
} ) ;
2023-08-16 22:55:32 +00:00
}