docker: Allow building for non-root user

[the-sun-will-rise-tomorrow]

Motivation

Add options uid, gid, uname, and gname to docker.nix.

Setting these to e.g. 1000, 1000, "user", "user" will build an image which runs and allows using Nix as that user.

Context

• Fixes Support running Nix as a non-root user nix-community/docker-nixpkgs#62

• Dependencies:

  • nixos/dockerTools: fix includeStorePaths when enableFakechroot nixpkgs#271976
  • dockerTools: Fix chown in fakeRootCommands nixpkgs#281520
  • Allow streaming layered containers with non-root Nix store nixpkgs#282886

Although (with NixOS/nixpkgs#281520) it's possible to simply do a chown ... /nix in fakeRootCommands, this is quite inefficient. A change only in ownership is not representable as a Docker layer, so the contents of the entire Nix store will end up being copied in the final layer. This is what NixOS/nixpkgs#282886 fixes and this PR builds on.

Priorities and Process

Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.

[Ericson2314]

I don't understand everything involved on the docker side, but the code change itself looks sensible good to me. Seems like a good companion to #789 too.

[thufschmitt]

(marked as draft since it depends on open Nixpkgs PRs, so we don't want to accidentally merge it right now).

I'm a bit reluctant in general when it comes to extending the Docker setup because I'm not sure that we want to maintain a non-trivial Docker setup. But that change is great since it's – mostly – a thin layer on top of the Nixpkgs-side heavy lifting. So 👍

[roberth]

Blocked on

• Backport of Allow streaming layered containers with non-root Nix store nixpkgs#282886
• Update to nixos-23.11 #9546

[dhess]

Is this ready to merge now?

[the-sun-will-rise-tomorrow]

• Backport of Allow streaming layered containers with non-root Nix store nixpkgs#282886

Just noticed this was still missing, PR opened: NixOS/nixpkgs#302113

[the-sun-will-rise-tomorrow]

Rebased; I think all dependencies are now satisfied.

[mmlb]

Looks like this has fallen through the cracks?

[dhess]

Yes, can we get this merged, please?

[roberth]

This PR still needs:

• tests: new test case in tests/nixos/?
• documentation, doc/manual/source/installation/installing-docker.md

I apologize for the other maintainers who have merged insufficiently documented and untested code. That should not have happened.

[the-sun-will-rise-tomorrow]

tests: new test case in tests/nixos/?

Added a very simple test. (I tried to put together a more complete test which does some store operations, but ran into this issue.)

documentation, doc/manual/source/installation/installing-docker.md

Thanks, added.

[the-sun-will-rise-tomorrow]

Added a very simple test. (I tried to put together a more complete test which does some store operations, but ran into this issue.)

I managed to get a more thorough test working with the help of a binary cache server machine, but I still can't figure out how to write a test that tries to actually build a derivation inside the container. We need to provide all dependencies offline (via the cache), but for some reason it's not happy with what's on the cache and tries to download some bootstrap tarballs, even if I try to warm up the cache by doing the same thing there first.

[roberth]

Nix's normal logging isn't the most helpful in this case because it gives you all the paths it needs, but if you pas -v or -vv it will tell you more about what it actually does. Probably the first path it queries is the one you need to provide (and repeat that solution strategy until it works).
What may also help is to build a simple derivation with builtins.derivation so that you control accurately what it depends on.
Otherwise you'll be copying Nixpkgs into the test derivation, slowing down the test.

I don't see the cache you're referring to, but I think you could bake the dependencies into the image.
For the expression file you put in the image, you could use a function storepath: builtins.appendContext storepath { ${storepath} = { path = true; } } to convert plain strings containing store paths into store paths that the Nix inside the image understands.

Feel free to push a work in progress commit so I can have a look.

[the-sun-will-rise-tomorrow]

What may also help is to build a simple derivation with builtins.derivation so that you control accurately what it depends on.

Great idea, it worked, thanks!

[roberth]

This is great!

[dhess]

Thanks for this, it'll be great to run Nix in containers as non-root.

If I understand correctly (from #5380 (comment)), pretty much the only thing that won't work is finding GC roots; is that right?

[the-sun-will-rise-tomorrow]

If I understand correctly (from #5380 (comment)), pretty much the only thing that won't work is finding GC roots; is that right?

That would apply to a daemon (multi-user) setup. This container has a single-user installation, which makes sense for OCI containers. I think GC should work fine if everything in the container runs as the same user.

[roberth]

The container has a proper store db for the store object parts that aren't the path or data, like the references for GC. Single user setups are supported. The only problems I would expect is to perhaps corrupting the store by accident because it is less protected. Also the sandbox may or may not be possible to isolate properly, depending on the container runtime and settings.