Upstream commit 0a5d9828c1f6b32e8462a00029b1ba2a720c867b changed how the
extension build script detects an Apple toolchain; adjust accordingly by
forcing it to be false. Keep both fixes so this is more compatible.
allows building bundix but most ruby gems still fail with
```
++ gem install --local --force --http-proxy http://nodtd.invalid --ignore-dependencies --install-dir /nix/store/...-ruby-aarch64-unknown-linux-gnu2.7.6-nio4r-2.5.8-aarch64-unknown-linux-gnu/lib/ruby/gems/2.7.0
--build-root / --backtrace --no-env-shebang -N /nix/store/...-nio4r-2.5.8.gem --
/nix/store/...-ruby-aarch64-unknown-linux-gnu-2.7.6/bin/gem: line 8: require: command not found
/nix/store/...-ruby-aarch64-unknown-linux-gnu-2.7.6/bin/gem: line 9: require: command not found
/nix/store/...-ruby-aarch64-unknown-linux-gnu-2.7.6/bin/gem: line 10: require: command not found
/nix/store/...-ruby-aarch64-unknown-linux-gnu-2.7.6/bin/gem: line 12: required_version: command not found
/nix/store/...-ruby-aarch64-unknown-linux-gnu-2.7.6/bin/gem: line 14: unless: command not found
/nix/store/...-ruby-aarch64-unknown-linux-gnu-2.7.6/bin/gem: line 15: abort: command not found
/nix/store/...-ruby-aarch64-unknown-linux-gnu-2.7.6/bin/gem: line 16: end: command not found
/nix/store/...-ruby-aarch64-unknown-linux-gnu-2.7.6/bin/gem: line 18: args: command not found
/nix/store/...-ruby-aarch64-unknown-linux-gnu-2.7.6/bin/gem: line 20: begin: command not found
/nix/store/...-ruby-aarch64-unknown-linux-gnu-2.7.6/bin/gem: line 21: Gem::GemRunner.new.run: command not found
/nix/store/...-ruby-aarch64-unknown-linux-gnu-2.7.6/bin/gem: line 22: rescue: command not found
/nix/store/...-ruby-aarch64-unknown-linux-gnu-2.7.6/bin/gem: line 23: exit: e.exit_code: numeric argument required
```
@ruby maintainers: please feel free to revert this and let me know,
should it cause any problems.
Stuff that's in the store shouldn't be writable! So let's disable
Bundler's requires_sudo? method, which checks if some files are
writable and suggests using sudo if not, entirely.
Previously, schleuder-cli would print to stderr when run:
Following files may not be writable, so sudo is needed:
/nix/store/qg40x0ysrf9x6sag6qgb1klg87lskdp5-schleuder-cli-0.1.0/lib/ruby/gems/2.7.0
/nix/store/qg40x0ysrf9x6sag6qgb1klg87lskdp5-schleuder-cli-0.1.0/lib/ruby/gems/2.7.0/bin
/nix/store/qg40x0ysrf9x6sag6qgb1klg87lskdp5-schleuder-cli-0.1.0/lib/ruby/gems/2.7.0/bin
/nix/store/qg40x0ysrf9x6sag6qgb1klg87lskdp5-schleuder-cli-0.1.0/lib/ruby/gems/2.7.0/build_info
/nix/store/qg40x0ysrf9x6sag6qgb1klg87lskdp5-schleuder-cli-0.1.0/lib/ruby/gems/2.7.0/bundler
/nix/store/qg40x0ysrf9x6sag6qgb1klg87lskdp5-schleuder-cli-0.1.0/lib/ruby/gems/2.7.0/cache
/nix/store/qg40x0ysrf9x6sag6qgb1klg87lskdp5-schleuder-cli-0.1.0/lib/ruby/gems/2.7.0/doc
/nix/store/qg40x0ysrf9x6sag6qgb1klg87lskdp5-schleuder-cli-0.1.0/lib/ruby/gems/2.7.0/extensions
/nix/store/qg40x0ysrf9x6sag6qgb1klg87lskdp5-schleuder-cli-0.1.0/lib/ruby/gems/2.7.0/gems
/nix/store/qg40x0ysrf9x6sag6qgb1klg87lskdp5-schleuder-cli-0.1.0/lib/ruby/gems/2.7.0/plugins
/nix/store/qg40x0ysrf9x6sag6qgb1klg87lskdp5-schleuder-cli-0.1.0/lib/ruby/gems/2.7.0/specifications
Otherwise you end up with a derivation that refers to the original
path (which is not in the Nix store and not accessible to
builders). This caused the derivation paths for the docbookrx package
(removed on master) to depend on the location of the nixpkgs source
tree.
This was ignored before. Now that it's passed through, it's possible
to do
(bundlerEnv {
name = "...";
gemdir = ./.;
document = [ "ri" ];
}).env
and get an environment where ri can be used to read gem documentation.
(It was requested by them.)
I left one case due to fetching from their personal repo:
pkgs/desktops/pantheon/desktop/extra-elementary-contracts/default.nix
pg_query 2.0.2 apparently has working ARM support.
This change also re-locks sqlint to use the newer version, and
generalises the platform spec to avoid being unnecessarily
restrictive.
As before, other packages which refer to pg_query are unaffected,
because they pin an older version of pg_query which is handled
separately in the gem-config expressions.
Follow-up to #116785
- Updates libpg_query to 2.0.0 in gem config, used by pg_query gem 2.0.1
- Continue to support older gem with older libpg_query, for gitlab
- Remove redundant gem config override in sqlint expression
- Add myself as a maintainer
It's been at least a year since I kept up to date with Ruby, and I
don't think I really have anything left to offer Nixpkgs in terms of
Ruby expertise.
Prior to this it doesn't seem to be possible to customize the version of
bundler used in a bundlerEnv app. This change allows something like the
following:
```nix
let bundler = pkgs.buildRubyGem rec {
gemName="bundler";
name="bundler-2.2.11";
version="2.2.11";
source.sha256="1izx6wsjdm6mnbxazgz1z5qbhwrrisbq0np2nmx4ij6lrqjy18jf";
};
in pkgs.bundlerEnv.override { inherit bundler; } {
name="test";
gemdir=./.;
}
```
to use bundler 2.2.11 rather than the 2.1.5 default.
this takes care of the following folders in pkgs/development:
* arduino
* chez-modules
* go-packages
* guile-modules
* idris-modules
* perl-modules
* r-modules
* ruby-modules
The way in which Nixpks builds Ruby gems means that certain operations
by bundler *will not work*, namely `bundle install --redownload`.
According to the source the _cache/_ directory should have been kept,
however it seems through revisions to the file it has been purged.
Here was the comment from the original commit that introduced
buildRubyGem:
```
# Note:
# We really do need to keep the $out/${ruby.gemPath}/cache.
# This is very important in order for many parts of RubyGems/Bundler to not blow up.
# See https://github.com/bundler/bundler/issues/3327
```
Why is the _cache_ directory needed?
Bundler and RubyGems uses the cache as a source of truth.
When bundler executes `bundler install --redownload`, any gems it
discovers in the _GEM_PATH_ it assums must have their _.gem_ file
present in the cache (unaware it was installed from Nix).
Rather than downloading the gem from RubyGems the bundler code forcibly
re-installs the gem from the cache directory instead and **fails** if it
does not exist.
I've opened https://github.com/rubygems/rubygems/issues/4088 to see if
this failure should be soft and not so explicit; or fallback to fetching
the gem from scratch.
Without this change the following is the error:
```bash
> [nix-shell:~/code/nix/playground/jruby-bundler-rake]$ bundle install --force
[DEPRECATED] The `--force` option has been renamed to `--redownload`
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.jruby.ext.openssl.SecurityHelper (file:/nix/store/fis6nzrpw9pmcivr84qh5byfgm07qn10-jruby-9.2.13.0/lib/ruby/stdlib/jopenssl.jar) to field java.security.MessageDigest.provider
WARNING: Please consider reporting this to the maintainers of org.jruby.ext.openssl.SecurityHelper
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Fetching gem metadata from https://rubygems.org/.
Using bundler 2.1.4
Installing hello-world 1.2.0
Bundler::GemNotFound: Could not find hello-world-1.2.0.gem for installation
An error occurred while installing hello-world (1.2.0), and Bundler
cannot continue.
Make sure that `gem install hello-world -v '1.2.0' --source
'https://rubygems.org/'` succeeds before bundling.
```
Wth the fix the following no woccurs:
```bash
[nix-shell:~/code/nix/playground/jruby-bundler-rake]$ bundle install --redownload
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.jruby.ext.openssl.SecurityHelper (file:/nix/store/69wjlj4yirp48rv1q03zxgd4xvf0150d-jruby-9.2.13.0/lib/ruby/stdlib/jopenssl.jar) to field java.security.MessageDigest.provider
WARNING: Please consider reporting this to the maintainers of org.jruby.ext.openssl.SecurityHelper
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Fetching gem metadata from https://rubygems.org/.
Using bundler 2.1.4
Installing hello-world 1.2.0
Bundle complete! 1 Gemfile dependency, 2 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
```
```
[nix-shell:~/code/nix/playground/jruby-bundler-rake]$ ls -l /nix/store/cwl9n5073hqgpfhnw4wic13nrrgg9dn8-gem-env/lib/jruby/gems/2.5.0/cache/
total 8
lrwxrwxrwx 1 fmzakari primarygroup 102 Dec 31 1969 bundler-2.1.4.gem -> /nix/store/ifc8a0gsfkrhkv953rd4rz8bcspahi8y-bundler-2.1.4/lib/jruby/gems/2.5.0/cache/bundler-2.1.4.gem
lrwxrwxrwx 1 fmzakari primarygroup 110 Dec 31 1969 hello-world-1.2.0.gem -> /nix/store/xi9ln6n1mz2is5ppykjxqhhkpjq9zm6i-hello-world-1.2.0/lib/jruby/gems/2.5.0/cache/hello-world-1.2.0.gem
```
I have a minimal project that demonstrates this issue at https://github.com/fzakaria/jruby-bundler-nix-failure
Changed ruby version to 2.7.x to match upstream.
Added a gem config for gitlab-pg_query as it tries to download a source
tarball during the build process.
Also removed a patch for gitaly that has become obsolete by upstream fix
[here](de04077c25).
The way ruby loads gems and keeps track of their paths seems to not
always work very well when the gems are accessed through
symlinks. Ruby will then complain that the same files are loaded
multiple times; it relies on the file's full path to determine whether
the file is loaded or not.
This adds an option to simply copy all gem files into the environment
instead, which gets rid of this issue, but may instead result in major
file duplication.
According to https://endoflife.software/programming-languages/server-side-scripting/ruby
ruby 2.4 will go end-of-life in march, where the new release of nixpkgs
will be cut. We won't be able to support it for security updates.
Remove all references to ruby_2_4 and add ruby_2_7 instead where
missing.
Mark packages that depend on ruby 2.4 as broken:
* chefdk
* sonic-pi
Setting a Bundler version with GEM_PATH doesn't seem to work in Ruby
2.7, so we need to use the LOAD_PATH instead. Without this,
bundlerEnv environments will always use the version of Bundler that
comes with Ruby, which won't necessarily work because it isn't the
version that was used to generate the bundle.
For example, building ronn with Ruby 2.7 without this change results
in a broken executable, but it works (when built with all packaged
Ruby versions) after this change.
In building a gem whose native extension is a Rakefile, the previous
version of this code will call essentially `rake ""`, when it means to
call `rake`.
This change converts `""` into `[]` rather than `[""]`.
This fixes
nix-shell -p 'ruby.withPackages (const [])' ruby.devdoc
which otherwise wouldn't find documentation, unlike
nix-shell -p ruby ruby.devdoc
which would, because ruby has setup hooks to accomodate for this, that
were being masked by the withPackages wrapper.
Go beyond the obvious setup hooks now, with a bit of sed, with a skipped case:
- cc-wrapper's `dontlink`, because it already is handled.
Also, in nix files escaping was manually added.
EMP
Because the gemdir was referenced on the derivation, it would cause the
whole gemdir to get added to the store, which would in turn force the
derivation to be rebuilt whenever unrelated folder files would change.
The `gemset` field can be an attribute set as well in `buildRubyGem`,
however attribute sets can't be coerced to strings for the environment
set.
The impact should be relatively small as the environment variables are
only used by the `runCommand` script for `bunlderApp` which doesn't
refer to `gemset` at all.
This has two benefits:
- Anyone else using the `mathematical` gem will benefit from the config.
- The gem config overrides the default one, so we were losing the fixes
for other gems.
I had to change how the update script works. Now it looks at the `gems`
passthru from `bundlerApp`.
The manual documents both bundlerEnv and bundlerApp as providing `env`
and `wrappedRuby` attributes on the resulting derivations. However only
bundlerEnv actually had these attributes. Update bundlerApp to provide
the same passthru that bundlerEnv does.
In case of the gem type 'git', nix-bundle-install.rb was called with
wrong path to the git checkout.
${src} does contain the sources, but not the newly generated .git dir,
which is created in the buildPhase of pkgs/development/ruby-modules/gem/default.nix
In some rare cases, this .git dir is needed at installPhase.
Note: on macOS, it works without this config. Testing on Ubuntu/Debian with the parent sha will produce an error about extconf.rb failing to find openssl/sha.h.
`digest-sha3` is a C-extension gem which fails to build on Nix because
it uses non-literals as format strings which is forbidden by the default
Nix hardening settings. There is a pull request to fix that ([1]), but
the gem seems to be abandoned.
This PR disables the "format" hardening for `digest-sha3`.
[1]: https://github.com/phusion/digest-sha3-ruby/pull/8
"default" isn't really a group, it's more the absence of one. With
Bundler, this means that a gem should be installed unconditionally,
regardless of which groups are specified. It doesn't really make sense
to allow these gems to be omitted from a bundlerEnv.
This wasn't really an issue until the latest minor release of Bundix
(2.4), because prior to then Bundix didn't emit group attributes, and so
this functionality of bundlerEnv wasn't really used. However, it is now
apparent that a better default for bundlerEnv would be to include all
gem groups by default, not just the default group. This matches the
behavior of Bundler, and makes more sense, because the default group
alone isn't necessarily useful for anything -- consider a Rails app with
production, development, and test groups. It has the additional benefit
of being backwards compatible with how this would have worked before the
Bundix update.
Suppose I have a Gemfile like this:
source "https://rubygems.org"
gem "actioncable"
gem "websocket-driver", group: :test
The gemset.nix generated by Bundix 2.4.1 will set ActionCable's groups
to [ "default" ], and websocket-driver's to [ "test" ]. This means that
the generated bundlerEnv wouldn't include websocket-driver unless the
test group was included, even though it's required by the default group.
This is arguably a bug in Bundix (websocket-driver's groups should
probably be [ "default" "test" ] or just [ "default" ]), but there's no
reason bundlerEnv should omit dependencies even given such an input --
it won't necessarily come from Bundix, and it would be good for
bundlerEnv to do the right thing.
To fix this, filterGemset is now a recursive function, that adds
dependencies of gems in the group to the filtered gemset until it
stabilises on the gems that match the required groups, and all of their
recursive dependencies.
> /nix/store/r2vsi140pys7jnzyk0qz1fj9aji6sq40-ruby2.5.3-rb-readline-0.5.5/lib/ruby/gems/2.5.0/gems/rb-readline-0.5.5/lib/rbreadline.rb:1097:in `<module:RbReadline>': HOME environment variable (or HOMEDRIVE and HOMEPATH) must be set and point to a directory (RuntimeError)
It would be reasonable to have a Ruby program that depends on some other
program being in the PATH. In this case, the obvious thing to do would
be something like this:
bundlerApp {
# ...
buildInputs = [ makeWrapper ];
postBuild = ''
wrapProgram "$out/bin/foo" \
--prefix PATH : ${lib.makeBinPath [ dep ]}
'';
}
However, this doesn't work, because even though it just forwards most of
its arguments to `runCommand`, `bundlerApp` won't take a `buildInputs`
parameter. It doesn't even specify its own `buildInputs`, which means
that the `scripts` parameter to `bundlerApp` (which depends on
`makeWrapper`) is completely broken, and, as far as I can tell, has been
since its inception. I've added a `makeWrapper` build input if the
scripts parameter is present to fix this.
I've added a `buildInputs` option to `bundlerApp`. It's also passed
through to bundled-common because `postBuild` scripts are run there as
well. This actually means that in this example we'd end up going through
two layers of wrappers (one from `bundlerApp` and one from
bundled-common), but that has always been the case and isn't likely to
break anything. That oddity does suggest that it might be prudent to
not forward `postBuild` to bundled-common (or to at least use a
different option) though...
FWIW, as far as I can tell no package in nixpkgs uses either the
`scripts` or `postBuild` options to `bundlerApp`.
