LXC

Setup Unprivileged Containers

Instructions.

Setup lxc user environment

Make sure to create lxc user and lxc group with user id matching entry in /etc/subuid and /etc/subgid.

Create lxc group with group id explicitly specifid:

groupadd -g 2000 lxc

Create lxc user with id explicitly specified:

useradd -u 2000 -g 2000 -d /srv/lxc -s /usr/bin/bash lxc

subuid and subguid

Permit user to map UID and GID.

/etc/subuid:

root:100000:65536
lxc:100000:65536

/etc/subguid is the same as /etc/subuid assuming lxc is a group id as well (?).

If root is present, then root can be used. The ranage numbers can be the same as lxc.

/etc/lxc/default.conf

The file format is documented in man lxc.container.conf.

# NAT bridge:
lxc.net.0.type = veth
lxc.net.0.link = lxcbr0
lxc.net.0.flags = up
lxc.net.0.hwaddr = 00:16:3e:xx:xx:xx

# The following 2 lines enable unprivileged containers:
lxc.idmap = u 0 100000 65536
lxc.idmap = g 0 100000 65536

lxc.idmap:

Thus, this file defines a mapping range, but /etc/subuid defines permission to map to that range.

Create Container

Create container (from lxc user):

lxc-create -n tuman -t download

Outputs: 3 failed to open TTY, then list.

It prompts for:

Ubuntu 20.04 LTS:

lxc-create -n tuman -t download -- --dist ubuntu --release focal --arch amd64

Message:

To enable SSH, run: apt install openssh-server
No default root or user password are set by LXC.

Start Container

lxc-start -n container-name --logfile /srv/lxc/container-name.log

foreground:

lxc-start -n tuman -F

with log file:

lxc-start -n tuman --logfile /srv/lxc/tuman.log

Get Root

from lxc user:

lxc-attach -n tuman

List conatiners with details

lxc-ls --fancy

Copy file from Host to Container

Example:

cat something_on_my_machine.sh | lxc-attach -n $1 -- /bin/sh -c "/bin/cat > /target/on/lxc/guest"

Credits go to Easy ways to add files to LXC containers.

Troubleshooting

Trace it with:

strace -e openat -f lxc-usernsexec

If ‘lxc-create’ command produces warnings

[lxc@stol ~]$ lxc-create -n tuman -t download
cmd/lxc_usernsexec.c: 64: opentty - Permission denied - Failed to open tty
cmd/lxc_usernsexec.c: 64: opentty - Permission denied - Failed to open tty
cmd/lxc_usernsexec.c: 64: opentty - Permission denied - Failed to open tty

It still should be successful.

from github issue/2764:

@Hethsron both /etc/subgid and /etc/subuid must have an entry for the user attempting to create the unprivileged container; otherwise the newuidmap call will fail. Try running usermod -v 100000-200000 -w 100000-200000 USERNAME.

lxc creates unprivileged container in ~/.local/share/lxc directory. Each subdirectory is based on the container name. When container is created its files have UID of user specified in the configuration files such as /etc/lxc/default.conf ands its local copy: ~/.config/lxc/default.conf.

groupadd -g 2000 lxc
useradd -u 2000 -g 2000 -d /srv/lxc -s /usr/bin/bash lxc

Create container (from lxc user):

lxc-create -n tuman -t download

Previous errors:

lxc@stol ~]$ lxc-create -n tuman -t download
lxc-create: tuman: conf.c: chown_mapped_root: 2978 lxc-usernsexec failed: cmd/lxc_usernsexec.c: 64: opentty - Permission denied - Failed to open tty
cmd/lxc_usernsexec.c: 64: opentty - No such file or directory - Failed to open tty
lxc-create: tuman: tools/lxc_create.c: main: 319 Failed to create container tuman
[lxc@stol ~]$ lxc-create -n tuman -t download
cmd/lxc_usernsexec.c: 64: opentty - Permission denied - Failed to open tty
cmd/lxc_usernsexec.c: 64: opentty - Permission denied - Failed to open tty
cmd/lxc_usernsexec.c: 64: opentty - Permission denied - Failed to open tty
lxc 20200430081438.273 ERROR    conf - conf.c:lxc_map_ids:2779 - newuidmap failed to write mapping "newuidmap: write to uid_map failed: Invalid argument": newuidmap 3333 0 100000 65536 2000 100000 65536 67536 2000 1
Failed to write id mapping for child process
lxc 20200430081438.273 ERROR    utils - utils.c:lxc_switch_uid_gid:1341 - Invalid argument - Failed to switch to gid 0
lxc-create: tuman: lxccontainer.c: create_run_template: 1626 Failed to create container from template
lxc-create: tuman: conf.c: lxc_map_ids: 2779 newuidmap failed to write mapping "newuidmap: write to uid_map failed: Invalid argument": newuidmap 3335 0 100000 65536 2000 100000 65536 67536 2000 1
lxc-create: tuman: conf.c: userns_exec_full: 4388 error setting up {g,u}id mappings for child process "3335"
lxc-create: tuman: lxccontainer.c: container_destroy: 3026 Error destroying rootfs for tuman
lxc-create: tuman: tools/lxc_create.c: main: 319 Failed to create container tuman

archlinux in unprivileged lxc cannot start #1678

Error:

Welcome to Debian GNU/Linux 10 (buster)!

Set hostname to <tuman>.
Initializing machine ID from random generator.
Failed to create /init.scope control group: Permission denied
Failed to allocate manager object: Permission denied
[!!!!!!] Failed to allocate manager object.
Exiting PID 1...

Solution:

Run that as root passing the username as first argument and the shell’s PID as the second argument. That will setup all your cgroups cleanly at which point your container will happily start.

#!/bin/sh
echo 1 > /sys/fs/cgroup/cpuset/cgroup.clone_children
for cgroup in /sys/fs/cgroup/*; do
    mkdir -p ${cgroup}/user.slice/user-$(id -u ${1}).slice
    chown -R $(id -u ${1}):$(id -g ${1}) ${cgroup}/user.slice/user-$(id -u ${1}).slice

    if [ "$(basename ${cgroup})" != "unified" ]; then
        echo ${2} > ${cgroup}/user.slice/user-$(id -u ${1}).slice/tasks
    fi
done

as lxc user:

echo $$
lxc_shell_pid

this_scipt lxc lxc_shell_pid

Resources

  1. https://ubuntu.com/server/docs/containers-lxc
  2. Exploring simple Linux containers with lxc posted January 30, 2020.
  3. https://openvz.org/
  4. How to Change Password Of An LXC Container User Account
  5. IBM resource mentions cgroup more than others.