Introduction
So having recently got my Carputer up and running I decided I wanted to connect it to my LAN at home and for my business. I’ve already got a pfsense installation for my border router so can easily add a new IPSec VPN node connecting in to it – with just a couple of quick changes to my existing setup.
Things I wanted to achieve:
- Change my car’s network over to the 10.2.0.0/24 network (10.0.0.0/16 is for my house network, 10.1.0.0/16 is for AWS services and 10.3.0.0/16 is where I allocate my /30s from). I’m not sure my car is going to need a /24, but I’m not exactly short on address space.
- Install a dynamic DNS client on the Pi to keep the IP updated whenever the vehicle is online
- Setup VPN connection from my car’s router to my main network border router – any 10.0.0.0/16 should flow here
Network Change
For some reason I setup my car as a 192.168.1.0/24 block. This made very little sense at the time and even less now – so time to change it. The first step is to change the address range that udhcpd offers. In my case all I need to change are the opt router and start/end addresses. This gives me a /etc/udhcpd.conf that looks like this:
start 10.2.0.100 end 10.2.0.254 interface wlan0 remaining yes opt dns 8.8.8.8 4.2.2.2 opt subnet 255.255.255.0 opt router 10.2.0.1 opt lease 864000 option domain guytp.org
Next it was to modify the IP of the Pi itself in /etc/network/interfaces to match the gateway above:
auto lo iface lo inet loopback iface eth0 inet dhcp iface wlan0 inet static address 10.2.0.1 netmask 255.255.255.0 up iptables-restore < /etc/iptables.ipv4.nat allow hotplug wwan0 iface wwan0 inet dhcp
Finally I just need to change the static IP on my OSMC connection to be 10.2.0.2. If your version of OSMC is newer you may be able to actually do this through the user interface. If that is the case then note that any changes you made manually will be wiped out. First check the My OSMC menu for a network option and if it isn’t there you need to manually edit files. OSMC uses connman to configure it’s IPs you need to select “Exit” from the power off menu in OSMC and keep pressing down escape. Eventually you’ll be at a command prompt and login as osmc for both the username and password. Then type in:
sudo bash cd /var/lib/connman ls
There will be a folder starting “wifi” and in there a file called settings. Edit this file and change the router and IP address and then you’re done. My file ended up looking like this:
[wifi_XXXXXXXXXXXX_XXXXXXXXXXXXXX_managed_psk] Name=XXXXXXX SSID=XXXXXXXXXXXXXX Frequency=2437 Favorite=true AutoConnect=true Modified=2015-04-26T00:18:35.264754Z Passphrase=XXXXXXXXXX IPv4.method=manual IPv6.method=auto IPv6.privacy=disabled IPv4.netmask_prefixlen=16 IPv4.local_address=10.2.0.2 IPv4.gateway=10.2.0.1 Nameservers=8.8.8.8;4.2.2.2; I thought I was done but soon found my web traffic blocked. If you installed squid don’t forget to edit /etc/squd3/squid.conf to change your local IP range as well. <h1>Dynamic DNS Client</h1> <p>I use no-ip for my dynamic DNS and then just have a CNAME from my guytp.org domain to the appropriate no-ip hosts with a 15 second TTL. So for me I wanted to download the no-ip source for linux. You need to download it, extract it and then compile/install by following the steps below. When do you "make install" you’ll be guided through your config settings.</p> guytp@GTP-RRSC-RTB-01:~$ wget http://www.no-ip.com/client/linux/noip-duc-linux.tar.gz … snipped … 100%[======================================>] 134,188 174K/s in 0.8s 2015-04-26 21:00:17 (174 KB/s) - `noip-duc-linux.tar.gz' saved [134188/134188] guytp@GTP-RRSC-RTB-01:~$ tar -xzf noip-duc-linux.tar.gz guytp@GTP-RRSC-RTB-01:~$ cd noip-2.1.9-1/ guytp@GTP-RRSC-RTB-01:~/noip-2.1.9-1$ make gcc -Wall -g -Dlinux -DPREFIX=\"/usr/local\" noip2.c -o noip2 … snipped … guytp@GTP-RRSC-RTB-01:~/noip-2.1.9-1$ sudo make install … snipped … Auto configuration for Linux client of no-ip.com. Multiple network devices have been detected. Please select the Internet interface from this list. By typing the number associated with it. 0 wlan0 1 wwan0 1 Please enter the login/email string for no-ip.com XXXXXXX@guytp.org Please enter the password for user 'guy@guytp.org' ******** 2 hosts are registered to this account. Do you wish to have them all updated?[N] (y/N) n Do you wish to have host [XXXXXXX.ddns.net] updated?[N] (y/N) n Do you wish to have host [XXXXXXX.ddns.net] updated?[N] (y/N) y Please enter an update interval:[30] 15 Do you wish to run something at successful update?[N] (y/N) n New configuration file '/tmp/no-ip2.conf' created. mv /tmp/no-ip2.conf /usr/local/etc/no-ip2.conf guytp@GTP-RRSC-RTB-01:~/noip-2.1.9-1$
To get this to run as a service on the Pi I created a file /etc/init.d/noip with the following contents:
#! /bin/sh
# /etc/init.d/noip
### BEGIN INIT INFO
# Provides: noip
# Required-Start: $remote_fs $syslog
# Required-Stop: $remote_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Simple script to start a program at boot
# Description: A simple script from www.stuffaboutcode.com which will start / stop a program a boot / shutdown.
### END INIT INFO
# If you want a command to always run, put it here
# Carry out specific functions when asked to by the system
case "$1" in
start)
echo "Starting noip"
# run application you want to start
/usr/local/bin/noip2
;;
stop)
echo "Stopping noip"
# kill application you want to stop
killall noip2
;;
*)
echo "Usage: /etc/init.d/noip {start|stop}"
exit 1
;;
esac
exit 0
I then made this executable, tested it worked and finally registered it to be included in startup/shutdown:
sudo chmod 755 /etc/init.d/noip sudo /etc/init.d/noip start sudo /etc/init.d/noip staop sudo update-rc.d noip defaults
And with that we’ve got dynamic DNS up and working. I can immediately see that my guytp.org alias is pointing at my 4G connection and I’ve got a way to determine my real public IP.
VPN Connection
This proved to be more than a little bit of a challenge. I ended up spending several evenings trying various attempts to get this working. At one end of my VPN (in my main network) I have a PFSense box running as border router and site-to-site VPN connections. To connect from the Raspberry Pi with full subnet routing rather than just host-to-host I decided to use IPSec. This required the use of the tool openswan.
There are several "gotchas" when you’re doing this in the scenario I describe (also acting as wireless LAN router and using 3G/4G as your connection).
- You’re more than likely behind NAT in your mobile connection as I discovered with dynamic DNS above
- You may even have something more weird than that depending on your provider
- There are a couple of steps that are missed out of several guides
I’ll go through my working configuration. To get my VPN connection up I first installed the requisite packages:
sudo apt-get install openswan uml-utilities chkconfig lsof
You now want to edit /etc/ipsec.secrets to setup your shared key. Since I’m only managing one tunnel I just use it to match any of my VPN connections. If you had more than one the first %any is the "left" side of the connection and the second is the "right" side’s identifier.
%any %any: PSK "Blah Blah Blah"
Next you need to setup the profile for the connection itself by editing /etc/ipsec.conf:
version 2.0 config setup protostack=netkey nat_traversal=yes keep_alive=30 conn Guytp type=tunnel authby=secret left=%defaultroute right=wan0.guytp.org leftsubnet=10.2.0.0/24 leftid=10.2.0.1 rightsubnet=10.0.0.0/16 rightid=@GTP-BSK-RTB-01 ike=aes128-sha1;modp1536! phase2alg=aes128-sha1;modp1536 keyexchange= ike pfs= no auto= start lifetime=1h
The important things in this config are:
- Guytp - the name of the connection
- leftsubnet - This is the subnet you are providing over WiFi from the Raspberry Pi in my scenario. If you’re not setup exactly like me this is fundamentally the subnet local to the Raspberry Pi.
- leftid - This is how the RaspberryPi identifies itself to the remote end (PFSense). You can leave this blank and a default is picked based on IP, or you can enter a hostname or even an arbitrary string prefixed with @. I could only get the use of an IP working in PFSense or I got negotiation failures
- rightsubnet - This is the subnet that you are connecting to at the remote end (PFSense). It’s important to know you can only route one subnet per tunnel and you cannot just add a route manually to a gateway at the other end of the connection. Since I haven’t got anything overlapping with 10.0.0.0/16 this wasn’t an issue but it may be depending on your mobile provider’s NAT
- rightid - This is the name that the remote end (PFSense) identifies with. In this instance I did manage to use a friendly name
The rest of these are values that finally worked after lots of playing around. Very important is the nat_traversal at the top.
Before we can bring the Openswan IPSec link up to the PFSense box from the Raspberry Pi we also need to configure a few other things. You need to disable redirects on all network cards for openswan to function. To do this add the following lines at the bottom of /etc sysctl.conf:
net.ipv4.conf.all.accept_redirects = 0 net.ipv4.conf.all.send_redirects = 0 net.ipv4.conf.default.send_redirects = 0 net.ipv4.conf.wlan0.send_redirects = 0 net.ipv4.conf.wlan0.accept_redirects = 0 net.ipv4.conf.default.accept_redirects = 0
To reload these values run:
sudo sysctl -p /etc/sysctl.conf
At this point you need to configure the remote end (PFSense). The only issues I had here were:
- To route to more than just my default /24 on 10.0.1.0 I needed to explicitly include 10.0.0.0/16 in the local network of Phase 2.
- Due to the dynamic IP and NAT nature of the mobile connection you need to enter the "Remote gateway" as your dynamic DNS hostname (i.e. non-NAT address). If your external IP happens to change before your dynamic DNS you won’t be able to bring the connection up
- Due to the identifier being a defined IP address the PFSense distribution doesn’t find the Pre-Shared key so I explicitly have to set the key in Pre-Shared Keys. This is really unacceptable but more about that later - this does kind of ruin it with NAT.
Here are some screenshots of my PFSense configuration and firewall.
Finally to make sure everything is looking good for Openswan run these two commands:
sudo unlink /bin/sh sudo ln -s /bin/bash /bin/sh
You can now try to bring the IPSec connection up.
sudo service ipsec restart
Check that your Raspberry Pi’s OS configuration is valid by running the below - everything should be OK, N/A or Disabled:
guytp@GTP-RRSC-RTB-01:~$ sudo ipsec verify Checking your system to see if IPsec got installed and started correctly: Version check and ipsec on-path [OK] Linux Openswan U2.6.37/K3.18.7-v7+ (netkey) Checking for IPsec support in kernel [OK] SAref kernel support [N/A] NETKEY: Testing XFRM related proc values [OK] Checking that pluto is running [OK] Pluto listening for IKE on udp 500 [OK] Pluto listening for NAT-T on udp 4500 [OK] Two or more interfaces found, checking IP forwarding [OK] Checking NAT and MASQUERADEing [OK] Checking for 'ip' command [OK] Checking /bin/sh is not /bin/dash [OK] Checking for 'iptables' command [OK] Opportunistic Encryption Support [DISABLED] guytp@GTP-RRSC-RTB-01:~$
Assuming your other router is set up you should see your connection come online. If you don’t then you can check out /var/log/auth.log and /var/log/syslog for more information on errors. You can also run:
sudo ipsec auto —status
Hopefully if everything is good you should see something like this at the bottom:
000 #1: "Guytp":4500 STATE_MAIN_I4 (ISAKMP SA established); EVENT_SA_REPLACE in 1468s; newest ISAKMP; lastdpd=10s(seq in:0 out:0); idle; import:admin initiate
Your remote end should also show that the tunnel is up. If you see anything other than STATE_MAIN_i4 with ISAKMP SA established something went wrong. Double, triple and quadruple check your configs then start poking around in the logs above and on the remote end. This can be a real pain.
Now to test some traffic. There are a few other important things here:
- The route to your remote network (on PFSense) will not show in the RPi routing table directly
- There are no additional interfaces such as tun0 or ipsec0 on the Raspberry Pi side
- Due to dynamic routing you cannot do route add -net <something> gw <IP on remote side> - this will fail
- If you do want to see the route you’ll need to try ip xfrm policy or ip xfrm state and take a read through there
- You cannot ping directly (by default) from either the Raspberry Pi or PFsense side without specifying a network interface
Try to ping from the remote end to your Raspberry Pi once the tunnel is up. If you’re pinging from the router itself you’ll need to specify the interface - i.e. from FreeBSD with the IP 10.0.1.1 as the internal part of the PFSense VPN it would be:
ping -I 10.0.1.1 10.2.0.1
From any other machine on the PFSense side of the network pings should work normally.
At the moment traffic will not be flowing from the Pi outwards though if the Pi initiates it. This took me days to solve but with one simple command to sort out the SNAT issues. This is listed as being a slight quirk of the way the NETKEY (the Linux IP stack in the Raspberry Pi kernel) works. This simple command should fix it though and store the updated ip-tables for later use.
sudo iptables -t nat -I POSTROUTING -s 10.2.0.0/24 -d 10.0.1.0/24 -j ACCEPT sudo iptables-save > /etc/iptables.ipv4.nat
You again cannot ping directly from the Pi without specifying an outgoing interface so in this case:
ping -I wlan0 10.0.1.1
And with that your Raspberry Pi / Wireless Router should now be connected to your VPN and capable of routing the traffic between subnets. At this point any device connected by WiFi in my car can directly connect to anywhere in my corporate or home networks. Pretty cool. I can get 10MBps over it as well which is about as much as I can get over 4G itself in my house.
My biggest issue here is that due to the PSK identification being based on a public-facing IP in PFSense. It seems even with hostname it stores the IP in Racoon (Openswan equivalent in PFSense) configuration. My solution rather than spending days more working this out has been to order a SIM card with no NAT and a static IP address which should be here shortly. I ordered mine from Andrews and Arnold (who can even do the termination with L2TP at mobile network level if required - but less fun) http://www.aa.net.uk/telecoms-mobile-data.html.
I didn’t bother routing traffic to AWS via the link. I could have done but this would have involved extra tunnels and since currently everything in AWS isn’t going to be touched by the car I’ll leave it. The easiest way to do this though is just duplicate your entries above to bring a second tunnel up between the Pi and router but changing the right subnet.
If that didn’t work for you prepare for many, many sleepless nights but I’ll happily help if I can. For now I’m just glad to have finally got this setup and working.
