Network Segmentation for Techies

Among top control lists, including those issued by the New Zealand CERT, we see "network segmentation" as a critical control. However, testing experience suggests that a lot of our customers think this control is done and dusted, when their network is anything but secure. It's extremely common for a network to have one set of controls in place for "segmentation", while also having a fundamental design flaw that undermines those controls.

I want to help people understand why this occurs, what the hazards are around network segmentation, and share some principles for a reliable result. This article is primarily written for a technical audience, but hopefully covers some general principles to help everyone understand where issues arise, and to design things safely.

Chief among the challenges to this understanding is that networks are built out of layers, and our strategy for segmentation must consider strategies for each layer.

A massive thank you to Adam Boileau, who taught me a lot of the lower-level details about how networks are formed.

Purpose

When considering any of these controls, it's important to maintain sight of what we're trying to achieve. That aim is that a dubious and potentially compromised device shouldn't be able to talk to everything on our network.

Each device should only have lines of communication to the things it actually needs to talk to, to the maximum extent possible. This helps to mitigate the risk that we have vulnerable services and/or devices on the network - we can't exploit them if we can't talk to them!

Local Area Networks (LANs)

The lowest layer most of us have to think about is the Local Area Network (LAN), most commonly running over some variant of Ethernet. Each machine on this network layer has a MAC address, and then we have switches which allow communication either point-to-point across the network, or broadcast to everyone.

An important concept at the local network level is that we independently tell each machine what IP address it should have, and the range of addresses to expect on that LAN. There's no central authority controlling this. Even if we run DHCP, there's no enforcement mechanism, and any machine can simply adopt any address within that local network's address range. When two machines are assigned the same address, they have a little battle of ARP announcements trying to claim it.

A lot of attacks can emerge from this phenomenon. I can send messages claiming to have any address on the network, and I can do it faster than any normal computer. I can send confusing, different announcements to different parts of the network. There are also attacks based on internal mechanisms, like Spanning Tree Protocol, which allow an attacker to force communications to go through them.

Within a LAN, it's just best to assume I can impersonate any other machine's IP address, and can direct any requests you make to a particular machine to myself. Application-layer controls, such as TLS, or better division of networks so I never get on the same LAN as my targets are required to mitigate this.

Case study: A customer had deployed a pair of Tenable Nessus virtual appliances to their network to monitor for security problems. In order to allow scanning across the enterprise, the firewalls were configured to allow any and all connections from these hosts, to anywhere in the network. However, these devices were deployed to a large LAN shared with other virtual infrastructure. Any other host on that LAN could have "stolen" the IP address of the Nessus host, gaining the capability to connect to anywhere in the network.

Another key point is that at the local network level your traffic does not transit a central firewall. Communication occurs machine-to-machine, with only relatively passive switches in the middle. Your central firewall's rules do not apply to local network traffic.

I'll say that again: Your central firewall rules do not apply to local network traffic. The only firewalls that are effective between hosts on a local network are host-based firewalls on the endpoints in question, which have tended to be neglected due to the challenge of managing them.

Put together, these threats mean that putting machines of a similar trust level in a LAN is critical. Putting your dedicated management infrastructure in the same LAN as workstations would be a bad idea. Likewise IoT, phone systems, SCADA, etc all have very different purposes. Having one "DMZ" network which contains all the dubious stuff probably not the best idea either - consider dividing them up.

Determine a purpose for each network, consider that all devices in a network are particularly subject to attack from each other, and resist adding anything that doesn't clearly belong there.

Virtual LANs

While a traditional LAN is tied to a physical set of wiring, in the modern context it's very common to use a Virtual LAN (VLAN) arrangement, with multiple VLANs sharing a single set of physical wiring. The same virtual network can pop out in multiple places, linked together by a shared backbone.

This means that a "local" network may be somewhat less local than anticipated. Be careful how far you extend it, given the trust constraints above!

Case study: A customer had set up a backbone WAN between multiple sites across Auckland, and the same virtual networks popped out in multiple spots. Perhaps those vulnerabilities given a CVSS rating of AV:A (adjacent network position) aren't so "adjacent"!

Managed switches which support VLANs are also subject to some break-out methods, for example if obtaining access to a "trunk" (multiple network) port. Be aware of automatic configuration features like Cisco Discovery Protocol (CDP), Link-Layer Discovery Protocol (LLDP), etc, which can be used to say "hey, I'm a switch, please give me a trunk port". Ensure your configurations are sane, and disable such dangerous features.

Fine-grained slicing of networks is certainly not helped by service providers charging a substantial fee to "manage" each virtual network, which is really a way of rationing VLAN numbers. A lot of our customers lump a lot more things in to one place than they really should, because someone couldn't justify the cost of another network fee. This tends, however, to create a snowballing security risk.

Routed Networking

At the next layer, we have routed Internet Protocol (IP) traffic. Routed traffic goes through a router, which joins together multiple network segments. Routers sit between the local networks and allow hosts on those networks to talk to each other. Without this, we could only talk locally!

Routers usually have a set of firewall rules controlling which traffic may be forwarded from one network to another, and this is so fundamental to their behaviour that routers are often just marketed as "firewalls". Routers know which range of addresses to expect on each local network (or VLAN), so can't as easily be tricked in to sending traffic to the wrong place.

At the routing layer, it can be quite common to configure firewalls inadequately. We might split up our network in to different VLANs or network segments, in our mind achieving "network segmentation", but fail to actually filter the traffic between them. The solution here is to check your firewall rules to limit forwarding between network segments. Validate that this filtering occurs.

Case study: a customer had carefully segmented a number of networks throughout their business, including the IoT systems we were asked to review. Their office equipment was on a different VLAN to the IoT, and they believed everything was "segmented". However, firewall rules had not been set up, and were in a default state allowing nearly everything through. Almost every network could reach all the IoT devices, which lacked any form of authentication or access control.

The solution is, as usual, to allow-list only the traffic that is expected and necessary.

It's also possible to have multi-homed devices: sometimes a host, particularly servers, will be connected to multiple LANs. It's not uncommon, for example, for a traditional three-tier web application (web server, application, database) to have separated networks, but then also plug all those servers in to an "admin" LAN. If these hosts can talk to each other across the admin LAN, and don't have host-based firewalls, what's the point in the separate networks? Any host connected to more than one network is potentially something you could compromise to get from one side to the other.

Walking Through the Clouds

While all of the above set you up well for traditional, on-premises networks, it's important to understand that some cloud networks are different. They often make use of Software-Defined Networking (SDN), which allows network behaviours to be heavily customised, and controlled by external sources of data. If the cloud control plane knows where a given host has been deployed, why would it go through a discovery process to "find" that host?

For example, Amazon Web Services (AWS) networks, despite looking a lot like a LAN, behave very differently in practice. Instead of all the hosts sharing a switched network, every single host (actually network port) has only a point-to-point stub network with its VM host. This does just enough to say "send me all your traffic", then everything is forwarded using a proprietary routing layer. Firewalling is done through their custom process, with its own slightly esoteric edge cases.

Others will be much more similar to a regular network. Google Cloud Platform looks far more like a traditional network, which is useful for some purposes. I don't know enough about Azure to fill in on that front. OpenStack and Kubernetes have pluggable network layers, so may vary across providers.

When deploying to a cloud or virtualised environment, you'll need to refer to their own documentation to find where it varies from the traditional model.

Summary

Effective network segmentation is complicated, because it requires us to understand the layers making up a network. Errors are common, but you can achieve a high quality segmentation:

• Effective network segmentation requires that we separate hosts adequately in to local networks, and filter traffic between those networks adequately.
• Put only similarly trusted hosts together on a local area network. Preferably have an assigned "purpose" to each local network (LAN/VLAN).
• Route traffic between local area networks using a dedicated router+firewall appliance. Use an allow-list set of firewall rules, limiting the traffic which is permitted to be forwarded between networks.
• Be very cautious about connecting devices to multiple local networks.
• Consider using host-based firewalls, which allow filtering of local traffic. There are some central management tools available these days to help with this.
• Ensure that trunking, as well as switch discovery protocols such as LLDP, CDP, are only enabled where necessary. Be aware they may be enabled by default.