pages tagged namespaceyakkinghttp://yakking.branchable.com/tags/namespace/yakkingikiwiki2015-11-11T12:00:14ZNetworking - DHCPhttp://yakking.branchable.com/posts/networking-5-dhcp/Richard Maw2015-11-11T12:00:14Z2015-11-11T12:00:08Z
<p>I hope you've done your homework.
In a <a href="http://yakking.branchable.com/tags/namespace/networking-3-addresses-and-routes">previous article about networking</a>
we set an exercise to configure some machines with static IP addresses.</p>
<p>You probably agree that it would be annoying if you had to change the
address range as it is configured in all the different computers.</p>
<p>Wouldn't it be better if we could store this information in the network somehow?</p>
<p>Well, guess what? We can! Its called <a href="http://en.wikipedia.org/wiki/dhcp">DHCP</a>.</p>
<p>This works by nominating one computer on the network
to act as the master of networking configuration.</p>
<p>Computers which want to opt into the network
communicate with this server to be told what addresses to use.</p>
<p>Normally your router-modem that connects you to the internet
takes this responsibility,
but any machine, real or virtual, may do it.</p>
<h1>Example with network namespaces</h1>
<p>As before, we're going to use network namespaces
to simulate multiple machines sharing a network.</p>
<p>For DHCP to work you need a DHCP server on the managing machine,
and a DHCP client on every client machine that will join this network.
Since we are only virtualising the networking, we need both on the same machine.</p>
<p>You will likely have <a href="http://www.thekelleys.org.uk/dnsmasq/doc.html">dnsmasq</a> and <a href="http://linux.die.net/man/8/dhclient">dhclient</a> on your system,
since while systemd can provide a DHCP server and client through <a href="http://man7.org/linux/man-pages/man8/systemd-networkd.8.html">networkd</a>,
this currently is generally only used in servers, embedded and containers,
while <a href="https://wiki.gnome.org/Projects/NetworkManager">NetworkManager</a> uses <a href="http://www.thekelleys.org.uk/dnsmasq/doc.html">dnsmasq</a> and <a href="http://linux.die.net/man/8/dhclient">dhclient</a> on desktops.</p>
<p>While <a href="http://man7.org/linux/man-pages/man8/systemd-networkd.8.html">networkd</a> is capable of being both a DHCP client and server,
we can only demonstrate its use as a client,
since it requires a full container to isolate its configuration from the host.</p>
<h2>Configuring the virtual network</h2>
<p>We're going to use a network namespace called <code>dhcptest</code>,
which will be connected to the host network namespace
by a virtual ethernet device.</p>
<pre><code># ip netns add dhcptest
# ip link add host type veth peer name guest
# ip link set guest netns dhcptest
</code></pre>
<p>The subnet of the network linking us to <code>dhcptest</code> we use is <code>10.0.0.0/24</code>,
which ranges from <code>10.0.0.1</code> to <code>10.0.0.255</code>.</p>
<p>Since the machine in <code>dhcptest</code> will be the DHCP server,
we need to assign its own address statically.</p>
<pre><code># ip netns exec dhcptest ip addr add 10.0.0.1 dev guest
# ip netns exec dhcptest ip link set guest up
# ip netns exec dhcptest ip route add 10.0.0.0/24 dev guest
</code></pre>
<h2>Using dnsmasq as the DHCP server</h2>
<p>In a terminal, run:</p>
<pre><code># ip netns exec dhcptest dnsmasq --dhcp-range=10.0.0.2,10.0.0.254,255.255.255.0 --interface=guest --no-daemon
dnsmasq: started, version 2.72 cachesize 150
dnsmasq: compile time options: IPv6 GNU-getopt DBus i18n IDN DHCP DHCPv6 no-Lua TFTP conntrack ipset auth DNSSEC loop-detect
dnsmasq-dhcp: DHCP, IP range 10.0.0.2 -- 10.0.0.254, lease time 1h
dnsmasq: reading /etc/resolv.conf
dnsmasq: using nameserver 127.0.1.1#53
dnsmasq: read /etc/hosts - 1 addresses
</code></pre>
<p>This starts <a href="http://www.thekelleys.org.uk/dnsmasq/doc.html">dnsmasq</a> on <code>dhcptest</code>,
serving the address range <code>10.0.0.2</code> to <code>10.0.0.254</code>,
with the subnet <code>10.0.0.0/24</code> (this is the <code>255.255.255.0</code> netmask<code>),
on the</code>guest` network interface.</p>
<p><code>--no-daemon</code> forces <a href="http://www.thekelleys.org.uk/dnsmasq/doc.html">dnsmasq</a> to stay running in the foreground,
which makes it easier to clean up when finished,
since it can be cancelled with <code>^C</code>.</p>
<h2>Using dhclient as the DHCP client</h2>
<p>In a different terminal to the one that the DHCP server was started in, run:</p>
<pre><code># dhclient -d host
Internet Systems Consortium DHCP Client 4.3.1
Copyright 2004-2014 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/
Listening on LPF/host/16:c0:28:6b:ea:76
Sending on LPF/host/16:c0:28:6b:ea:76
Sending on Socket/fallback
DHCPDISCOVER on host to 255.255.255.255 port 67 interval 3 (xid=0xd6a3b68)
DHCPREQUEST of 10.0.0.105 on host to 255.255.255.255 port 67 (xid=0x683b6a0d)
DHCPOFFER of 10.0.0.105 from 10.0.0.1
DHCPACK of 10.0.0.105 from 10.0.0.1
bound to 10.0.0.105 -- renewal in 1643 seconds.
</code></pre>
<p>This shows that it has successfully requested <code>10.0.0.105</code> as its address.</p>
<p>The <a href="http://linux.die.net/man/8/dhclient">dhclient</a> process will continue to run,
as it hasn't been given that address forever,
it has been given that address for an hour,
at which point it will expire and it must stop using it,
but before then <a href="http://linux.die.net/man/8/dhclient">dhclient</a> may renew the address to allow us to keep using it.</p>
<h2>Using networkd as the DHCP client</h2>
<p>First create the config file:</p>
<pre><code># install -m 644 /dev/stdin /etc/systemd/network/dhcptest.network <<'EOF'
> [Match]
> Name=host
> [Network]
> DHCP=yes
> EOF
</code></pre>
<p>Since <a href="http://man7.org/linux/man-pages/man8/systemd-networkd.8.html">networkd</a> currently only reads its config at startup,
we must restart it for the new config to take effect.
(This also has the effect of starting it for the first time,
on distros that don't start networkd by default.)</p>
<pre><code># systemctl restart systemd-networkd.service
</code></pre>
<p>This won't give any immediate feedback whether it worked,
but you can inspect its status by running:</p>
<pre><code>$ networkctl status host
● 6: host
Link File: /lib/systemd/network/99-default.link
Network File: /etc/systemd/network/dhcptest.network
Type: ether
State: routable (configured)
Driver: veth
HW Address: 16:c0:28:6b:ea:76
MTU: 1500
Address: 10.0.0.105
fe80::14c0:28ff:fe6b:ea76
Gateway: 10.0.0.1
DNS: 10.0.0.1
</code></pre>
<p>Note that this has created some persistent configuration,
so the next time the <code>host</code> ethernet device is created networkd will DHCP.</p>
<p>If you don't want to do this in future, you can remove the configuration
by running <code>rm /etc/systemd/network/dhcptest.network</code>.</p>
<h2>DHCPing successfully</h2>
<p>In the terminal running <a href="http://www.thekelleys.org.uk/dnsmasq/doc.html">dnsmasq</a> you should see:</p>
<pre><code>dnsmasq-dhcp: DHCPDISCOVER(guest) 10.0.0.105 16:c0:28:6b:ea:76
dnsmasq-dhcp: DHCPOFFER(guest) 10.0.0.105 16:c0:28:6b:ea:76
dnsmasq-dhcp: DHCPREQUEST(guest) 10.0.0.105 16:c0:28:6b:ea:76
dnsmasq-dhcp: DHCPACK(guest) 10.0.0.105 16:c0:28:6b:ea:76 HOSTNAME
dnsmasq-dhcp: not giving name HOSTNAME to the DHCP lease of 10.0.0.105 because the name exists in /etc/hosts with address 127.0.1.1
</code></pre>
<p>In another terminal you can see that it configured the address by running:</p>
<pre><code>$ ip addr show dev host
6: host: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 16:c0:28:6b:ea:76 brd ff:ff:ff:ff:ff:ff
inet 10.0.0.105/24 brd 10.0.0.255 scope global host
valid_lft forever preferred_lft forever
inet6 fe80::14c0:28ff:fe6b:ea76/64 scope link
valid_lft forever preferred_lft forever
</code></pre>
<h2>Testing the link</h2>
<p>We can now prove it works with netcat.</p>
<p>Run <code>ip netns exec dhcptest nc -l 1234</code> in one terminal,
and <code>nc 10.0.0.1 1234</code> in another,
and you will be able to see that text entered is repeated in both terminals.</p>
Networking - Namespaceshttp://yakking.branchable.com/posts/networking-4-namespaces-and-multi-host-routing/Richard Maw2015-09-09T11:00:12Z2015-09-09T11:00:06Z
<h1>What are network namespaces</h1>
<p>Network Namespaces are a Linux feature,
that allows different processes to have different views of the network.</p>
<p>Aspects of networking that can be isolated between processes include:</p>
<ul>
<li><p>Interfaces</p>
<p>Different processes can connect to addresses on different interfaces.</p></li>
<li><p>Routes</p>
<p>Since processes can see different addresses from different namespaces,
they also need different routes to connect to networks on those interfaces.</p></li>
<li><p>Firewall rules</p>
<p>Firewalls are mostly out of scope for this article,
but since firewall rules are dependant on the source or target interfaces,
you need different firewall rules in different network namespaces.</p></li>
</ul>
<h1>How do you manage network namespaces</h1>
<ul>
<li><p>When a network namespace is created with the <a href="http://man7.org/linux/man-pages/man2/unshare.2.html">unshare(2)</a> or <a href="http://man7.org/linux/man-pages/man2/clone.2.html">clone(2)</a> system calls,
it is bound to the life of the current process,
so if your process exits,
the network namespace is removed.</p>
<p>This is ideal for sandboxing processes,
so that they have restricted access to the network,
as the network namespaces are automatically cleaned up
when the process they were isolating the network for
no longer exists.</p></li>
<li><p>A privileged process can make their network namespace persistent,
which is useful for if the network namespace needs to exist when there are no processes in it.</p>
<p>This is what the <a href="http://man7.org/linux/man-pages/man8/ip.8.html">ip(8)</a> command does when you run <code>ip netns add NAME</code>.</p></li>
<li><p>The <code>ip netns delete NAME</code> command undoes this,
and allows the namespace to be removed
when the last process using it leaves the namespace.</p></li>
<li><p><code>ip netns list</code> shows current namespaces.</p>
<pre><code>$ sudo ip netns add testns
$ ip netns list
testns
$ sudo ip netns delete testns
$ ip netns list
</code></pre></li>
<li><p>A network namespace is of no use if it has no interfaces,
so we can move an existing interface into it with the
<code>ip link set dev DEVICE netns NAME</code> command.</p>
<pre><code>$ ip link show dev usb0
5: usb0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN mode DEFAULT group default qlen 1000
link/ether 02:5e:37:61:60:66 brd ff:ff:ff:ff:ff:ff
$ sudo ip link set dev usb0 netns testns
$ ip link show dev usb0
Device "usb0" does not exist.
</code></pre></li>
<li><p>Network namespaces would be of limited use if we can't run a command in them.</p>
<p>We can run a command in a namespace with the <code>ip netns exec NAME COMMAND</code> command.</p>
<pre><code>$ sudo ip netns exec testns ip link show dev usb0
5: usb0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/ether 02:5e:37:61:60:66 brd ff:ff:ff:ff:ff:ff5: usb0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/ether 02:5e:37:61:60:66 brd ff:ff:ff:ff:ff:ff
</code></pre></li>
<li><p>There is no standard daemon to start persistent network namespaces on-boot,
however, the <a href="http://www.freedesktop.org/software/systemd/man/systemd.exec.html#PrivateNetwork=">PrivateNetwork=</a> option to systemd service files can be used
to start a process in a network namespace on boot.</p></li>
</ul>
<h1>Uses of network namespaces</h1>
<p>Network namespaces can be used to sandbox processes,
so that they can only connect through the interfaces provided.</p>
<p>You could have a machine with two network interfaces,
one which connects out to the internet,
the other connected to your internal network.
You could set up network namespaces
such that you can run the ssh server on your internal network,
and run a web server in a network namespace that has the internet-facing interface,
so if your web server is compromised,
it can't connect to your internal network.</p>
<p><a href="https://www.docker.com/">docker</a> and <a href="http://www.freedesktop.org/software/systemd/man/systemd-nspawn.html">systemd-nspawn</a> combine network namespaces
with other namespaces to provide containers.
<a href="https://www.docker.com/">docker</a> is primarily interested in application containers,
while <a href="http://www.freedesktop.org/software/systemd/man/systemd-nspawn.html">systemd-nspawn</a> is more interested in system containers,
though both technologies can be used for either.</p>
<p>A more novel use of network namespaces is virtual routers,
as implemented in <a href="https://wiki.openstack.org/wiki/Neutron">OpenStack Neutron</a>.</p>
<h2>Network namespaced routing</h2>
<p>We previously mentioned that routes were a property of network namespaces,
that virtual ethernet devices have a pair of ends,
and that we can move a network interface into another network namespace.</p>
<p>By combining these features,
we can construct a series of virtual computers and experiment with routing.</p>
<h2>Two node networking</h2>
<p>To start with we're going to add another virtual computer,
start a simple echo server on it,
and configure the network such that we can connect to it from our host computer.</p>
<p>As we've done in previous articles,
we're going to make the <code>left</code> and <code>right</code> virtual ethernet pair.</p>
<pre><code>$ sudo ip link add left type veth peer name right
</code></pre>
<p>But to demonstrate routing to another machine we need to create one,
so we're making another network namespace,
as we only care about its networking.</p>
<pre><code>$ sudo ip netns add testns
</code></pre>
<p>We're going to move one half of the virtual ethernet device pair
into another namespace,
to simulate a physical link between the two virtual computers.</p>
<pre><code>$ sudo ip link set dev right netns testns
</code></pre>
<p>We're going to say that we connect to interfaces in the <code>testns</code> namespace
by connecting to addresses in the <code>10.248.179.1/24</code> subnet.</p>
<pre><code>$ sudo ip address add 10.248.179.1/24 dev left
$ sudo ip netns exec testns ip address add 10.248.179.2/24 dev right
</code></pre>
<p>We're also going to say that there's another network in there
on the <code>10.248.180.1/24</code> subnet.</p>
<p>Rather than having any more complicated network interfaces,
we assigning it to the loopback interface.</p>
<pre><code>$ sudo ip netns exec testns ip address add 10.248.180.1/24 dev lo
</code></pre>
<p>Now that we've assigned addresses,
we need to bring the interfaces up.</p>
<pre><code>$ sudo ip link set dev left up
$ sudo ip netns exec testns ip link set dev right up
$ sudo ip netns exec testns ip link set dev lo up
</code></pre>
<p>This has created a chain of interfaces linking <code>left</code> → <code>right</code> → <code>lo</code>.
Where <code>left</code> is in your namespace,
and <code>right</code> is in <code>testns</code> with its own private <code>lo</code>.</p>
<p>Now we can demonstrate sending messages through the network namespaces,
by creating a server on the <code>lo</code> interface inside <code>testns</code>.</p>
<p>Start an echo server in the test namespace by running this command in a new
terminal.</p>
<p>It is <em>important</em> to run this in another terminal,
as if you background the process,
netcat may decide it doesn't need to actively output data as it recieves it.</p>
<pre><code>$ sudo ip netns exec testns nc -l 10.248.180.1 12345
</code></pre>
<p>If we were to try to connect to it,
we would not be able to send messages.</p>
<pre><code>$ nc -v 10.248.180.1 12345
nc: connect to 10.248.180.1 port 12345 (tcp) failed: Network is unreachable
</code></pre>
<p>This is because your host namespace does not understand how to reach that address,
since it is not an address in your network namespace,
nor is it an address on any of the subnets in the namespace,
and there are no defined rules for how to reach it.</p>
<p>So let's add a rule!</p>
<pre><code>$ sudo ip route add 10.248.180.0/24 dev left via 10.248.179.2
</code></pre>
<p>This says that you can find the <code>10.248.180.0/24</code> subnet
by sending packets to the <code>10.248.179.2</code> address
through the <code>left</code> interface.</p>
<p>You should now be able to type messages into the following netcat command,
and see the result in your other terminal.</p>
<pre><code>$ nc -v 10.248.180.1 12345
Connection to 10.248.180.1 12345 port [tcp/*] succeeded!
</code></pre>
<h2>3 namespaces</h2>
<p>This works fine,
but in the wider internet you need to connect through long chains of computers
before you reach your final destination,
so we're going to use 3 network namespaces to represent a longer chain.</p>
<p>We are going to have two extra network namespaces, called <code>near</code> and <code>far</code>.</p>
<pre><code>$ sudo ip netns add near
$ sudo ip netns add far
</code></pre>
<p>We are going to create a chain of network interfaces called <code>one</code>, <code>two</code>, <code>three</code> and <code>four</code>.</p>
<pre><code>$ sudo ip link add one type veth peer name two
$ sudo ip link add three type veth peer name four
</code></pre>
<p><code>one</code> will be in our network namespace, connecting to <code>two</code> in the <code>near</code>
namespace, and <code>three</code> will connect to <code>four</code> in the <code>far</code> namespace.</p>
<pre><code>$ sudo ip link set dev two netns near
$ sudo ip link set dev three netns near
$ sudo ip link set dev four netns far
</code></pre>
<p>This produces a chain <code>one</code> → <code>two</code> → <code>three</code> → <code>four</code>.</p>
<p>To speak to these interfaces, we need to assign address ranges, so for our
host to <code>near</code> link we will use <code>10.248.1.0/24</code>,
for our <code>near</code> to <code>far</code> link we will use <code>10.248.2.0/24</code>,
and for our destination address we will use <code>10.248.3.1/24</code>.</p>
<pre><code>$ sudo ip address add 10.248.1.1/24 dev one
$ sudo ip netns exec near ip address add 10.248.1.2/24 dev two
$ sudo ip netns exec near ip address add 10.248.2.1/24 dev three
$ sudo ip netns exec far ip address add 10.248.2.2/24 dev four
$ sudo ip netns exec far ip address add 10.248.3.1/24 dev lo
</code></pre>
<p>We need to bring these interfaces up.</p>
<pre><code>$ sudo ip link set dev one up
$ sudo ip netns exec near ip link set dev two up
$ sudo ip netns exec near ip link set dev three up
$ sudo ip netns exec far ip link set dev four up
$ sudo ip netns exec far ip link set dev lo up
</code></pre>
<p>Now let's this time start our server in the <code>far</code> namespace,
so we can be sure that we can't connect to it directly from our namespace.</p>
<p>As before, start this echo server in a new terminal.</p>
<pre><code>$ sudo ip netns exec far nc -l 10.248.3.1 12345
</code></pre>
<p>Now let's try to connect to it from our namespace.</p>
<pre><code>$ nc -v 10.248.3.1 12345
nc: connect to 10.248.3.1 port 12345 (tcp) failed: Network is unreachable
</code></pre>
<p>If you try to send a message, it won't arrive.
If you inspect the routing table with <code>ip route</code>,
you will see that there is no match.</p>
<p>The relevant routing rules are:</p>
<pre><code>$ ip route
default via 192.168.1.1 dev wlan0 proto static
10.248.1.0/24 dev one proto kernel scope link src 10.248.1.1
</code></pre>
<p>This says that if you're broadcasting from <code>10.248.1.1</code> and going to
anything in <code>10.248.1.0/24</code> then it goes to the <code>one</code> interface. However
this doesn't match because we want to go to the <code>four</code> interface which
has address <code>10.248.3.1</code>.</p>
<p>We can make this route to our <code>near</code> interface by adding a new rule.</p>
<pre><code>$ sudo ip route add 10.248.2.0/24 dev one via 10.248.1.2
</code></pre>
<p>We can prove this hop works by starting a server in the <code>near</code> namespace
(again, in a separate terminal).</p>
<pre><code>$ sudo ip netns exec near nc -l 10.248.2.1 12345
</code></pre>
<p>We can try to talk to it:</p>
<pre><code>$ nc -v 10.248.2.2 12345
</code></pre>
<p>This doesn't yet work, because it's a bidirectional protocol,
so the return route needs to work too.</p>
<pre><code>$ sudo ip netns exec near ip route change 10.248.1.0/24 dev two via 10.248.1.1
</code></pre>
<p>This still doesn't work,
since while there's now routing rules,
they aren't for any addresses which have desinations in the near network namespace,
so by default they get dropped.</p>
<p>To change this we can turn on ip forwarding with:</p>
<pre><code>sudo dd of=/proc/sys/net/ipv4/ip_forward <<<1
</code></pre>
<p>Now we can talk to the <code>near</code> namespace, but we can't yet talk to the <code>far</code> namespace.
We need to add another routing rule for that.</p>
<pre><code>$ sudo ip route add 10.248.3.0/24 dev one via 10.248.1.2
</code></pre>
<p>As before, this means you can route to the <code>near</code> namespace.
However, it doesn't know how to reach the <code>far</code> namespace from there,
so we need to add another routing rule.</p>
<pre><code>$ sudo ip netns exec near ip route add 10.248.3.0/24 dev three via 10.248.2.2
</code></pre>
<p>Now you can reach the <code>far</code> namespace, but it's still not working.
This is because tcp is a bidirectional communication protocol,
and it doesn't know how to send the response to a message from <code>10.248.1.1</code>,
so we need to add another rule.</p>
<pre><code>$ sudo ip netns exec far ip route add 10.248.1.0/24 dev four via 10.248.2.1
</code></pre>
<h2>Conclusion</h2>
<p>I think you will agree that it's far too much faff
to set up this routing just to talk to a machine on another network.</p>
<p>You are likely thinking that there must be a better way,
and you are correct, there's a couple of ways,
but we will cover those in later articles.</p>