Beyond network namespaces

I recently started using UTS and mount namespaces in addition to network namespaces to do even more with network emulation than I had done previously done with Linux network namespaces.

UTS namespaces allow me to change the hostname in the namespace. This has significant advantages over just network namespaces in that things like the the prompt and the hostname command will return the name of the namespace, not the name of the host that they are running on. Similarly, mount namespaces allow each namespace to have different mounted file systems. This enables me to have a separate instance of Quagga per namespace by having independent /etc/quagga and /run/quagga file systems. (I can even use vtysh, which usually has problems with multiple instances of Quagga.)

Below is a high level overview of how I do this on Linux without anything as heavy as Docker or LXC / LXD.

  1. Create the following directories to hold namespace mount points if they don't already exist:

    • /run/mountns
    • /run/netns
    • /run/utsns

  2. Touch the following files for each set of namespaces:

    • /run/mountns/testNS1
    • /run/netns/testNS1
    • /run/utsns/testNS1

  3. Create the namespaces, using the above mount points so they can easily be re-entered.

    unshare --mount=/run/mountns/testNS1 --net=/run/netns/testNS1 --uts=/run/utsns/testNS1 /bin/true

    This simply executes /bin/true when creating the namespaces. You can easily execute a different program if you want to. I find it convenient to always execute true and then launch additional different processes later.

  4. Run any additional programs you want to in the namespaces.

    nsenter --mount=/run/mountns/testNS1 --net=/run/netns/testNS1 --uts=/run/utsns/testNS1 /bin/hostname testNS1

    Repeat the above command as many times as you want to, substituting other programs as necessary.

  5. I ran the following commands to support running Quagga using it's default configuration / PID file / socket file locations.

    nsenter --mount=/run/mountns/testNS1 --net=/run/netns/testNS1 --uts=/run/utsns/testNS1 mount --bind /etc/quagga/testNS1 /etc/quagga
    nsenter --mount=/run/mountns/testNS1 --net=/run/netns/testNS1 --uts=/run/utsns/testNS1 mount -t tmpfs none /run/quagga

    Then I ran the following commands to start Quagga (zebra and bgpd).

    nsenter --mount=/run/mountns/testNS1 --net=/run/netns/testNS1 --uts=/run/utsns/testNS1 zebra -d
    nsenter --mount=/run/mountns/testNS1 --net=/run/netns/testNS1 --uts=/run/utsns/testNS1 bgpd -d

  6. Then any time I want to enter Quagga's configuration, I run the following command.

    nsenter --mount=/run/mountns/testNS1 --net=/run/netns/testNS1 --uts=/run/utsns/testNS1 /usr/bin/vtysh

    This will take me directly into Quagga's vty shell inside if testNS1's namespace.

I don't know about you, but I do get tired of typing nsenter --mount=/run/mountns/testNS1 --net=/run/netns/testNS1 --uts=/run/utsns/testNS1 <bla> all the time. So I created the following script named testNS1 that allows me to simply type testNS1 vtysh. I find that these wrapper scripts make it quite easy, especially when working with multiple sets of namespaces, i.e. multiple virtual routers. But more on that later.

nsenter --mount=/run/mountns/testNS1 --net=/run/netns/testNS1 --uts=/run/utsns/testNS1 ${@:-/bin/bash}

The variable expansion at the end will execute a command if one is specified, or launch bash inside the namespace. This allows me to enter the testNS1 namespace simply by typing testNS1.

Another advantage of reusing the /run/netns/testNS1 mount points is that it is compatible with ip netns exec testNS1. It is safe to use ip netns exec testNS1 to execute diagnostic commands like ping and traceroute. However it is better to actually enter the namespace if I'm going to do anything more complicated.

Finally, I should point out that using namespaces like I am does not require a copy of the OS files / libraries. That's because they mount namespace inherits the same mounted file systems that the host had. The only difference is the few things that I've changed after starting the namespace, notably /run/quagga and /etc/quagga. The rest is the exact same as the host.

Teaser: I'm working on an article talking about how to use mount, network, and UTS namespaces to create a virtual lab of 9 routers and 12 clients, each running Quagga, all using namespaces on a single machine. (I think it will use about 210 MB of memory, much less than full VMs.)