Home LTE with Open5GS

Previous Post

This article is a follow-up of Your Own LTE Network at Home.

Following the blog post’s author that I was following in the last post, I also decided to switch to Open5GS. My reason was slightly different: I’ve been also biting the NixOS bug recently and my magma VM recently also ran out of disk space from the database (probably from all the logs), might as well revisit this setup with a fresh installation.

I followed most of his Nix config snippet closely except for a few things did not work for me and here’s once again a log of what it took me to get things to work.

Open5GS WebUI

The open5gs-webui snippet for some reason does not work for me, I keep ran into issues during npm build and there is a separately packaged open5gs-webui package now, so I use the following snippet to make use of it:

{pkgs, ...}: {
  systemd.services.open5gs-webui = {
    description = "Open5GS WebUI";
    wantedBy = ["multi-user.target"];
    after = [
      "network.target"
      "mongodb.service"
    ];
    requires = [
      "network.target"
      "mongodb.service"
    ];
    path = with pkgs; [
      bash
      nodejs
    ];
    environment = {
      HOSTNAME = "0.0.0.0";
      PORT = "9999";
    };
    preStart = ''
      export HOME=$(pwd)
    '';
    serviceConfig = {
      ExecStart = "${pkgs.nodejs}/bin/npm run start";
      CacheDirectory = "open5gs";
      WorkingDirectory = "${pkgs.open5gs-webui}/lib/node_modules/open5gs";
      User = "open5gs";
      Group = "open5gs";
      Restart = "always";
      RestartSec = "5";
    };
  };
}

GTP-U Address

This took me a loooooong time to figure out. I had everything setup, my phone would connect to the network but has no internet access. I ended up resorting to asking ChatGPT for some help in debugging this as I’m clueless about 4G/5G network architecture and it did help a lot! It pointed me to use tcpdump to see that during the device attachment request, my network is sending a non-sense response to tell my eNodeB to setup GTP-U tunnel to 127.0.0.6. GTP-U is GPRS¹ tunneling protocol - User Plane, which carries the user plane (data) traffic, with that being set to a loopback address, of course it is not going to do anything.

Mobile Network IP Stack

I actually learned a great deal while researching this: Did you know that your phone does not have a “default gateway” like it would have on a Wi-Fi network when it uses cellular network?

This is because the 4G/5G IP stack is point-to-point, it doesn’t really care about the next hop, it simply encapsulate the IP packet and send it over the radio. eNodeB also does not care about IP routing at all, it just forwards the packet to SGW using GTP-U. It is at SGW, the IP packet finally gets unwrapped and placed onto the tun interface we setup. Linux kernel will then pick it up and figure out what to do with it.

This came from the default Open5GS settings file. In sgwu.yaml, we need to update the gtpu server’s IP to the our server’s IP address that is reachable from eNodeB.

sgwu:
  gtpu:
    server:
      - address: 192.168.4.15

IP Forwarding & NAT

I also had to enable IP forwarding and NAT for the ogstun interface. I had the NixOS firewall disabled so I can’t just use the networking.firewall.extraCommands options, instead I created an oneshot systemd service to do it.

{
  boot.kernel.sysctl = {
    "net.ipv4.ip_forward" = 1;
    "net.ipv6.conf.all.forwarding" = 1;
  };
 
  systemd.services."ogstun-nat" = {
    description = "Add NAT rules for ogstun interface";
    wantedBy = [ "multi-user.target" ];
    after = [ "network-online.target" ];
    wants = [ "network-online.target" ];
 
    serviceConfig = {
      Type = "oneshot";
      RemainAfterExit = true;
    };
 
    script = ''
      ${pkgs.iptables}/bin/iptables -t nat -A POSTROUTING -s 10.45.0.0/16 ! -o ogstun -j MASQUERADE
      ${pkgs.iptables}/bin/ip6tables -t nat -A POSTROUTING -s 2001:db8:cafe::/48 ! -o ogstun -j MASQUERADE
    '';
  };
}

Happy Ending

Following the original article’s steps with these additions, I was able to get my new Open5GS-based core network setup and devices are connecting to it as if nothing happened.

Footnotes

1. oh wow I have not heard of GPRS in a while 😅. I still remember the good ol’ days of WAP websites… ↩