LXD/LXC Basic Setup

Lin­ux Serv­er

Con­tain­ers

LXD Ini­tial Set­up

First add your Lin­ux user to the lxd group in or­der to op­er­ate with the con­tain­ers with­out su­do – note you nay need to cog-out and log-in.

sudo usermod -aG lxd <user>

Ini­tial­ize LXD.

lxd init

#Out­put

Would you like to use LXD clustering? (yes/no) [default=no]: no
Do you want to configure a new storage pool? (yes/no) [default=yes]: yes
Name of the new storage pool [default=default]: default
Name of the storage backend to use (dir, lvm, zfs, ceph, btrfs) [default=zfs]: dir
Would you like to connect to a MAAS server? (yes/no) [default=no]: no
Would you like to create a new local network bridge? (yes/no) [default=yes]: yes
What should the new bridge be called? [default=lxdbr0]: lxdbr0
What IPv4 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]: auto
What IPv6 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]: auto
Would you like the LXD server to be available over the network? (yes/no) [default=no]: no
Would you like stale cached images to be updated automatically? (yes/no) [default=yes]: yes
Would you like a YAML "lxd init" preseed to be printed? (yes/no) [default=no]: yes

config: {}
networks:
- config:
    ipv4.address: auto
    ipv6.address: auto
  description: ""
  name: lxdbr0
  type: ""
  project: default
storage_pools:
- config: {}
  description: ""
  name: default
  driver: dir
profiles:
- config: {}
  description: ""
  devices:
    eth0:
      name: eth0
      network: lxdbr0
      type: nic
    root:
      path: /
      pool: default
      type: disk
  name: default
projects: []
cluster: null

List the avail­able im­ages by the fol­low­ing com­mands and find the name of an im­age of a de­sired dis­tri­b­u­tion – in this case Ubun­tu Serv­er 22.04.

lxc image list
lxc image list ubuntu:22.04
lxc image list ubuntu:22.04 | grep x86_64

In­stall a cer­tain im­age. We will name the con­tain­er lxc-web­serv­er. We will add a ca­pa­bil­i­ty to the con­tain­er to run oth­er con­tain­ers in­side, for more de­tails read the ar­ti­cle Nest­ed con­tain­ers in LXD.

lxc launch ubuntu:22.04 lxc-webserver -c security.nesting=true

#Out­put

Creating lxc-webserver
Starting lxc-webserver

In or­der to add (or re­move) the nest­ing op­tion to an ex­ist­ing LXC, use:

lxc config set lxc-webserver security.nesting true

List the con­tain­ers avail­able with­in the LXD in use.

lxc list --columns ns4 # name, state and IPv4

#Out­put

+---------------+---------+-----------------------+
|   NAME        |  STATE  |         IPV4          |
+---------------+---------+-----------------------+
| lxc-webserver | RUNNING | 10.127.198.222 (eth0) |
+---------------+---------+-----------------------+

Con­fig­ure the con­tain­er to ob­tain a sta­t­ic IP as­sign­ment.

lxc config device override lxc-webserver eth0

Device eth0 overridden for lxc-webserver

lxc config device set lxc-webserver eth0 ipv4.address 10.127.198.222 # no output mean everything is fine
lxc restart lxc-webserver
lxc list --columns ns4 | grep eth0 # check the ip

| lxc-webserver | RUNNING | 10.127.198.222 (eth0) |

LXD Ba­sic op­er­a­tion

Restart the ser­vice.

sudo snap restart lxd

Mount a host di­rec­to­ry to a di­rec­to­ry in­side a con­tain­er. Map the per­mis­sions.

printf "lxd:$(id -u):1\nroot:$(id -u):1\n" | sudo tee -a /etc/subuid # Allow LXD’s use of our user uid
printf "lxd:$(id -g):1\nroot:$(id -g):1\n" | sudo tee -a /etc/subgid # Allow LXD’s use of our user gid
sudo snap restart lxd                                                # Restart LXD to have it load the new map
printf "uid $(id -u) 1000\ngid $(id -g) 1000" | lxc config set lxc-webserver raw.idmap - # Set a custom map for our container
lxc restart lxc-webserver                                            # Restart the container to have the new map apply
lxc config device add lxc-webserver Git disk source=/home/<user>/Git path=/home/<user>/Git # Mount the directory

Do the ac­tu­al share (mount) of some di­rec­to­ries.

lxc config device add lxc-webserver Git disk source=/home/<user>/Git path=/home/<user>/Git
lxc config device add lxc-webserver VSC disk source=/home/<user>/.vscode-server path=/home/<user>/.vscode-server

LXC Ba­sic op­er­a­tions

List avail­able con­tain­ers.

lxc list
lxc list -c ns4

Lo­gin to a con­tain­er (note lxc-web­serv­er is a con­tain­er name).

lxc shell lxc-webserver

Ex­e­cute a com­mand against the con­tain­er from the host.

lxc exec lxc-webserver -- apt install apache2

Start, stop or delete con­tain­er.

lxc (start|stop|delete) container-name

Cre­ate a snap­shot.

lxc snapshot lxc-webserver snapshot-name

Delete a snap­shot.

lxc delete lxc-webserver/snapshot-name

Re­store a snap­shot.

lxc restore lxc-webserver snapshot-name

Cre­ate a back­up.

lxc export lxc-webserver ./lxc-webserver-backup.tar.gz

Re­store a back­up.

lxc import ./lxc-webserver-backup.tar.gz

Get in­fo about the con­tain­er (and its snap­shots at the bot­tom).

lxc info lxc-webserver

Lim­it the container's mem­o­ry us­age.

lxc config set lxc-webserver limits.memory 1GB

Au­to-start a con­tain­er.

lxc config set lxc-webserver boot.autostart 1

Set an au­to-start de­lay for a con­tain­er.

lxc config set lxc-webserver boot.autostart.delay 30

Set an au­to-start or­der num­ber for a con­tain­er.

lxc config set lxc-database boot.autostart.order 2
lxc config set lxc-webserver boot.autostart.order 3

Dis­able IPv6 for the con­tain­ers – ref­er­ence.

lxc network set lxdbr0 ipv6.address none

Back­up LXC Con­tain­ers

Here are pro­vid­ed notes about back­ing up the con­tain­ers,

Lo­cal ex­port ap­proach

You can use a crontab en­try as the fol­low.

* */12 * * * /snap/bin/lxc export lxc-webserver $HOME/backups/lxc-webserver-backup.tar.gz >/tmp/crontab.${USER}.lxc-webserver-backup.tar.gz.log 2>&1

The com­mand could be a part of your back­up script. Al­so they say it is bet­ter to ex­port a snap shot… Then a re­mote in­stance can fetch it by rsync via SSH.

Ref­er­ences

• Lin­ux con­tain­ers LXD: Back­ing up a LXD serv­er
• CiberCi­ty: How to move/​​​migrate LXD VM to an­oth­er host on Lin­ux
• Nix­Craft: How to back­up and re­store LXD con­tain­ers
• Stack­Over­flow: Copy lxd con­tain­ers be­tween hosts

Re­mote ex­port ap­proach

In this sec­tion, the LXD that is run­ning the ac­tu­al LXC that we want to back­up will be called remote​-lxd​.host. The LXD that will fetch (ex­port) the back­ups will be called local​-lxd​.host. Here is used the most sim­ple set­tings and the con­nec­tion be­tween the servers will be car­ry out via SSH tun­nel.

1. First, at the remote​-lxd​.host run the fol­low­ing com­mand.

lxc config set core.https_address :8443
lxc config set core.trust_password 'p@s$********wD'

2. Then at the lo­cal serv­er local​-lxd​.host add the nec­es­sary en­try for a con­nec­tion with port for­ward­ing of port 8443 in the file ~/.ssh/config. For more de­tails read the SSH sec­tion be­low. Af­ter that test the con­nec­tion by the fol­low­ing com­mands.

ssh fwd.remote-lxd.host -fTN
sudo netstat -tnupa | grep 8443

3. If every­thing looks fine, do the Init set­up for this in­stance – local​-lxd​.host, an­swer with no at all ques­tions if wont run lo­cal LXCs, and then ex­e­cute the fol­low­ing com­mand to add the re­mote serv­er.

# Default auth type: TLS + password

lxc remote add remote-lxd.host 127.0.0.1:8443

ficate fingerprint: 1778ec79530...
ok (y/n/[fingerprint])? y
Admin password for remote-lxd.host: ***
Client certificate now trusted by server: remote-lxd.host

Then you can switch the de­fault re­mote serv­er and list the run­ning con­tain­ers as fol­low.

lxc remote list
lxc remote switch remote-lxd.host
lxc list

The above pro­ce­dure must be done for all users that will ma­nip­u­late the re­mote in­stance – in this count the root ac­count if need­ed!

4. In or­der to ex­port a back­up from the re­mote in­stance you can per­form the fol­low­ing steps.

lxc snapshot remote-lxd.host:lxc-webserver backup                               # Create a remote snapshot
lxc export remote-lxd.host:lxc-webserver/backup ./remote.lxc-webserver.tar.gz   # Create a local backup of the remote snapshot

Note you need to have enough space at the re­mote in­stance, be­cause the snap­shots are cre­at­ed there. Al­so when yo cre­ate re­mote back­ups they will be cre­at­ed as tem­po­rary file at the re­mote in­stance and ten will be trans­ferred to the lo­cal one.

lxc export remote-lxd.host:lxc-webserver ./remote.lxc-webserver.tar.gz   # Create a backup of the remote LXC withot a snapshot

Ref­er­ences

• Lin­ux con­tain­ers LXD: Ad­vanced guide In­tro­duc­tion, Sec­tion: Re­mote LXD Serv­er
• Lin­ux con­tain­ers LXD: Back­ing up a LXD serv­er
• Mi blog lah: How to use LXC re­mote with the LXD snap

Au­to­mate the Re­mote ex­port ap­proach

sudo nano /usr/local/bin/backup.remote-lxd.sh && chmod +x /usr/local/bin/backup.remote-lxd.sh

#!/bin/bash

# @author    Spas Z. Spasov <spas.z.spasov@metalevel.tech>
# @copyright 2022 Spas Z. Spasov
# @license   https://www.gnu.org/licenses/gpl-3.0.html GNU General Public License, version 3 (or later)
#
# @desc      Create a local backup of an remote LXC, create a local backup of the remote LXD's settings,
#            fetch other backup files.
#            Some of the parameters still hard coded - this should ne improoved:
#             -  remote-lxd.host
#             -  fwd.remote-lxd.host
#             -  lxc-webserver

# Define the input parameters. 
TODAY="$(date +%Y-%m-%d)"

BACKUP_DIR="/mnt/backups/remote-lxd.host"
LOG="$BACKUP_DIR/backup.${TODAY}.log"

SYSTEM="remote-lxd.host"
BACKUP_FILE_LXC="${BACKUP_DIR}/${SYSTEM}.lxc-webserver-backup-${TODAY}.tar.gz"
BACKUP_FILE_LXD="${BACKUP_DIR}/${SYSTEM}.lxd-init-backup-${TODAY}.yaml"

# Create the backup directory if doesn't exist
[[ ! -d $BACKUP_DIR ]] && mkdir "$BACKUP_DIR"

# Remove all backup files older than 14 days
find "$BACKUP_DIR" -mtime +14 -type f -delete

# Get the remote backup
main_ssh_export() {
    echo -e "***** $TODAY *****\n"
    
    # Establish a connection to the remote server
	while ! (netstat -tnpau 2>/dev/null | grep -q '8443')
	do
		autossh -o "ExitOnForwardFailure yes" fwd.remote-lxd.host -TNf
	done

	while ! (/snap/bin/lxc list 2>/dev/null | grep -q lxc-webserver)
	do
		sleep 1
	done

	# Export the container
	if (netstat -tnpau 2>/dev/null | grep -q '8443') && (/snap/bin/lxc list 2>/dev/null | grep -q lxc-webserver)
	then
		/snap/bin/lxc export remote-lxd.host:lxc-webserver "$BACKUP_FILE_LXC"
	else
		echo "Failed to establish a connection to the remote server"
		exit 1;
	fi

    # Wait a while
	sleep 3

    # Terminate the connection, try to kill the parrent first
	sudo kill -9 $(ps -ef | grep autossh | grep fwd.remote-lxd.host | awk '{print $2}' | head -n 1)  2>/dev/null
	for pid in $(sudo netstat -tnpau 2>/dev/null | grep '8443' | awk '{print $NF}' | awk -F'/' '{print $1}');
	do
		sudo kill -9 "$pid" 2>/dev/null;
	done

    # Dump the LXD configuration. It is intentionally done in this way. 
    # Otherwise, when we do a dump via the remote management,
    # some parameters as 'core.https_address: :8443' will be omitted.
    ssh remote-lxd.host 'lxd init remote-lxd.host --dump' > "$BACKUP_FILE_LXD"

    # Download the remote portable backup
    rsync --progress -avr -e "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" remote-lxd.host:"/backups/"* "${BACKUP_DIR}/"

    echo -e "\n******************\n"
}
main_ssh_export #> "$LOG" 2>&1

Then you can use a crontab en­try as the fol­low.

* 1 * * * /usr/local/bin/backup.remote-lxd.sh >/tmp/crontab.$USER.backup.remote-lxd.sh.log 2>&1

For­ward the HTTP Traf­fic from the LXD host to a LXC

First in­stall the ipt­a­bles-per­sis­tent pack­age and mi­grate to ipt­a­bles-nft.

• Red­Hat De­vel­op­ers: ipt­a­bles: The two vari­ants and their re­la­tion­ship with nfta­bles
• Ask Ubun­tu: Warn­ing: ipt­a­bles-lega­cy ta­bles present

sudo apt install iptables-persistent # iptables-save > /etc/iptables/rules.{v4,v6}
sudo apt install ipset-persistent    # ipset save > /etc/ipset/ipsets

sudo update-alternatives --set iptables /usr/sbin/iptables-nft
sudo update-alternatives --remove iptables /usr/sbin/iptables-legacy
sudo modprobe -r iptable_filter iptable_nat iptable_mangle iptable_raw iptable_security

The sec­ond step is to add and make per­sis­tent the Ipt­a­bles rules that will redi­rect the traf­fic at ports 80 and 433 on the remote​-lxd​.host to the lxc-web­serv­er con­tain­er with IP 10.127.198.222.

Let's as­sume remote​-lxd​.host the serv­er has the fol­low­ing IP ad­dress­es:

• eth0: 158.113.3.106: The pub­lic ad­dress of the serv­er, we are us­ing Float­ing IP 137.112.134.165 for the do­main remote​-lxd​.host, so we do not ex­pect any traf­fic at the web ports on this IP.
• eth0: 10.14.1.5: The Float­ing IP 137.112.134.165 redi­rects every­thing to this in­ter­face so we will for­ward the traf­fic at this port.
• eth1: 10.123.1.2: This is the IP in the pri­vate vir­tu­al net­work at Dig­i­talO­cean – so prob­a­bly in the fu­ture oth­er re­sources could ac­cess the droplet vi this in­ter­face so we will for­ward the traf­fic at this port.

Here is a tem­plate com­mand:

sudo iptables -t nat -I PREROUTING -i $IFACE -p TCP -d $PUBLIC_IP --dport $PORT -j DNAT --to-destination $CONTAINER_IP:$PORT -m comment --comment "forward to lxc-webserver"

#Rules Per IP/PORT

It should be ex­e­cut­ed for each set of the fol­low­ing val­ues.

PORT=80 PUBLIC_IP=10.14.1.5 CONTAINER_IP=10.127.198.222 IFACE=eth0

PORT=443 PUBLIC_IP=10.14.1.5 CONTAINER_IP=10.127.198.222 IFACE=eth0

PORT=80 PUBLIC_IP=10.123.1.2 CONTAINER_IP=10.127.198.222 IFACE=eth0

PORT=443 PUBLIC_IP=10.123.1.2 CONTAINER_IP=10.127.198.222 IFACE=eth0

sudo iptables -t nat -L PREROUTING
# sudo iptables -t nat -F PREROUTING

sudo iptables-save | sudo tee /etc/iptables/rules.v4
# sudo iptables-restore < /etc/iptables/rules.v4

SSH Prox­yJump to LXC via LXD

Fol­low the guide SSH Con­nec­tion Set­up and set­up SSH con­nec­tion from you lo­cal ma­chine to the serv­er where the LXD is host­ed. Then set­up SSH con­nec­tion from the LXD host to the LXC at its vir­tu­al net­work in­ter­face. You can use the same .pub in key – by pop­u­lat­ing the ~/.ssh/authorized_keys file of the LXD host to the LXC.

To ac­cess the con­tain­er di­rect­ly via SSH we can use the Proxy Jump op­tion in this way.

nano ~/.ssh/config

Host remote-lxd.host
    HostName 158.113.3.106
    IdentityFile ~/.ssh/id_ed25519
    User <user>
    Port 22

Host remote-lxd.host.lxc-webserver
    HostName 10.127.198.222
    IdentityFile ~/.ssh/id_ed25519
    User <user>
    Port 22
    ProxyJump remote-lxd.host
    
Host fwd.remote-lxd.host
    HostName 158.113.3.106
    IdentityFile ~/.ssh/id_ed25519
    User <user>
    Port 22
    LocalForward 8443 localhost:8443
    # Compression yes

Now we have the fol­low­ing short SSH com­mands with­in the com­mand line.

ssh remote-lxd.host                 # Connect to the remote LXD host
ssh remote-lxd.host.lxc-webserver   # Connect to the remote LXC via the LXD host
ssh fwd.remote-lxd.host -fTN        # Forward ports from/to the remote LXD host

For per­sis­tent port for­ward­ing read the guide SSH Per­sis­tent Tun­nel and SSHFS Mount via "sys­temd" units.

Ref­er­ences

• Lin­ux con­tain­ers LXD: Serv­er con­fig­u­ra­tion set­tings
• Lin­ux con­tain­ers LXD: Get­ting Start­ed with LXD
• Dig­i­talO­cean: How To In­stall and Con­fig­ure LXD on Ubun­tu 20.04
• Learn­Lin­uxTV at YouTube: Get­ting start­ed with LXD Con­tainer­iza­tion (Full Guide!)
• Canon­i­cal Ubun­tu Serv­er: LXD
• Canon­i­cal Ubun­tu Blog: Cus­tom user map­pings in LXD con­tain­ers
• Canon­i­cal Ubun­tu Blog: Mount­ing your home di­rec­to­ry in LXD
• Ask Ubun­tu: Adding a shared host di­rec­to­ry to an LXC/LXD Con­tain­er (for priv­i­leged con­tain­er)
• Ask Ubun­tu: apt-key dep­re­ca­tion warn­ing when up­dat­ing sys­tem
• Ask Ubun­tu: At what point is the ~/.bashrc file cre­at­ed?

Reg­is­ter: Lin­ux Serv­er

Идентифициране

Разработване

Авторизиране

Утвърждаване

Spas

Spas

8.09.2022

22.09.2022

31338

31688