Proxmox - A Network Inside Network With VyOS - PART 2 - Configuring Vyos

January 30, 2025

In Part 1 - Setting up SDN we went over setting up Proxmox SDN to create a network inside a network with VXLAN. In this post we will go over the initial set up for VyOS including setting up:

Interfaces
SSH
Static routes
DNS
DHCP servers
DNS forwarding
SNAT
Firewall rules

At the moment, our VyOS firewall is unconfigured, can’t connect to the Internet, and our VMs on this vxlan network cannot either.

Step 1 - Setting up Interfaces

First we need to enter configure this will get us into configuration mode and allow us to configure VyOS.

Lets set up eth1 first, this is the interface that is connected to our HOME network, from now on I will be calling it OUTSIDE network you will see why later. There are 3 configuration commands that we can use, set edit and delete, these prefix the particular area that you want to configure.

In our case interfaces so to set up our eth1 interface lets add an address and a description to help us in the futre.

set interfaces ethernet eth1 address 192.168.1.251/24
set interfaces ethernet eth1 description OUTSIDE

Now we have done that we can set eth0, and this time we will put it as a sub-interface so we can use VLAN tags in Proxmox to further isolate VMs from eachother. Will also add a description and call it VXLAN_INSIDE as this is inside our isolated network.

set interfaces ethernet eth0 vif 10 address 10.0.10.254/24
set interfaces ethernet eth0 vif 10 description VXLAN_INSIDE

After typing these commands nothing has changed from what vyos is running, until we commit the changes. We can see what we are changing vs what is currently running with compare.

Step 2 - Verification

Now we have commited these changes lets see what our interfaces look like now. As we are still in configuration mode we have to prefix our OP commands with run.

run show interfaces

Now lets see if we can ping our VyOS VM from our OUTSIDE network.

Great, now lets see if we can ping google.com from our VyOS

Oh no! But we can ping our OUTSIDE gateway. Lets fix this but first lets get ssh set up so we dont have to keep using the console in Proxmox.

Step 3 - Setting up SSH

First we need to set up the ssh service in VyOS, we can do this with 1 line while in configure mode.

set service ssh port 22

Now lets ssh to the VyOS VM, as I set up the user vyos with password vyos.

ssh [email protected]

NICE.

Part 4 - Static routes

For vyos to be able to get to the OUTSIDE network and beyond, we need a route out. So we will statically set a default route to go via our HOME router. While in configure mode:

set protocols static route 0.0.0.0/0 next-hop 192.168.1.254

As you can see below after I partially typed the command I used TAB to show possible completions for the rest of the command. After doing this I commited the change.

Now lets ping google.com…

Same as before, This is because VyOS doesn’t have any name-servers set up for DNS (It’s Always DNS).

Lets fix that…

Part 5 - DNS

While in configure mode:

set system name-server 192.168.1.114
set system name-server 192.168.1.115

These are my Pi-hole servers but you can use whatever your DNS server/s are, generally you can use your HOME/OUTSIDE router IP and that will in turn use your ISP DNS. Lets commit that and test again.

WOOP!!

Part 6 - Lab VM networking

Now lets headover to our Lab VM dev-vm and go to Hardware and select the Network Device, and since we added the vif 10 to our VyOS vm we can tag this with VLAN Tag of 10 so it is on the same broadcast network as the eth1 sub-interface.

Lets headover to the dev-vm console and login and check our IP.

Nothing, can it ping the eth1 sub-interface address 10.0.10.254

Nope, As you may expect, It is because the dev-vm has no address, we could set one manually, but we have VyOS, lets turn on DHCP and let VyOS give it an address…

Part 7 - DHCP server + DNS forwarding

While we are setting up a DHCP server for clients on the network to get an IP address it would be handy if VMs could also reach the Internet, for that we need, an IP (from DHCP), DNS (from VyOS) and NAT, which I wont bore you with the technicals of, but if you want to know more can check Wikipedia.

First lets set up DHCP and DNS forwarding…

# DHCP
set service dhcp-server shared-network-name DEMO authoritative <- sets up VyOS to be the authoritative DNS
set service dhcp-server shared-network-name DEMO description DEMO
set service dhcp-server shared-network-name DEMO subnet 10.0.10.0/24 option default-router 10.0.10.254 <- tells the client its default route
set service dhcp-server shared-network-name DEMO subnet 10.0.10.0/24 option name-server 10.0.10.254 <- tells the client where to lookup DNS
set service dhcp-server shared-network-name DEMO subnet 10.0.10.0/24 range 0 start 10.0.10.1 <- DHCP pool start
set service dhcp-server shared-network-name DEMO subnet 10.0.10.0/24 range 0 stop 10.0.10.100 <- DHCP pool end
set service dhcp-server shared-network-name DEMO subnet 10.0.10.0/24 subnet-id 1

# DNS forwarding
set service dns forwarding allow-from 10.0.10.0/24 <- allow dns forwarding from this Subnet
set service dns forwarding listen-address 10.0.10.254 <- listen for dns on this address wich is eth1 
set service dns forwarding system <- use the system dns to forward in my case 192.168.1.114-115

Part 8 - Checking whats been done so far

Now we have typed a fair few commands, lets see how our VyOS config looks like now…

run show configuration commands <- If in config mode
or 
show configuration commands <- If in normal OP mode

Part 9 - Vif aint working

After some tinkering I could not get the vif working, and found the best way to VLAN tag was to add another Network Device in Proxmox and Tag it there. So I removed the vif and added the address on eth0 directly.

delete interfaces ethernet eth0 vif 10
set interfaces ethernet eth0 address 10.0.10.254/24

Now lets add the VLAN tag 10 to the Network Device in Proxmox that is attached to Vyos similar to what we did with the dev-vm.

Part 10 - Verification

Lets now check the VM and see if we can get an IP and ping google.com

Yes we have an IP address and if we check resolvectl we can see the DNS servers are there. Lets ping google.com, No luck, but If you look closely you can see that google.com resolved to 172.217.169.78 which means that our DNS forwarding also works.

Lets get NATTY.

Part 11 - NAT

NAT is the missing piece to get the VMs connected to the internet. To set this:

set nat source rule 1 outbound-interface name eth1
set nat source rule 1 source address 10.0.10.0/24
set nat source rule 1 translation address masquerade

Above you can see the three lines we typed in and how that is represented vs the compare command.

Lets verify this worked with our VM now.

IT WORKED!! But there is a problem…

Our VMs can reach inside our HOME network as well, If this is what you want you can draw the line here. But I want these VMs isolated from my HOME network. This is where firewalls come in.

Part 12 - Firewalls

First we will set some global options then we will set some firewll groups to make managing the firewall easier as we can use groups of network addresses to represent NET-INSIDE or we can just use it as a name instead of typing 10.0.10.0/24 all the time.

set firewall global-options state-policy established action 'accept'
set firewall global-options state-policy invalid action 'drop'
set firewall global-options state-policy related action 'accept'
set firewall group network-group NET-INSIDE network 10.0.10.0/24
set firewall group network-group NET-OUTSIDE network 192.168.1.0/24

This doesn’t get us the results we want so lets start adding some rules to the firewall. These First rules are input filter rules and apply to any packet where the destination IS the VyOS VM.

# This rule drops anything destined for VyOS by default and rule 10 allows ssh from
set firewall ipv4 input filter default-action drop
set firewall ipv4 input filter rule 10 action accept
set firewall ipv4 input filter rule 10 description 'Allow ssh from HOME/NET-OUTSIDE'
set firewall ipv4 input filter rule 10 destination port 22
set firewall ipv4 input filter rule 10 protocol tcp
set firewall ipv4 input filter rule 10 source group network-group 'NET-OUTSIDE' <- Missed this out in the screenshot
set firewall ipv4 input filter rule 10 state new

# rule 20
set firewall ipv4 input filter rule 20 action accept
set firewall ipv4 input filter rule 20 description 'Allow NET-INSIDE to firewall'
set firewall ipv4 input filter rule 20 source group network-group NET-INSIDE
set firewall ipv4 input filter rule 20 state new

Lets see if these work and try ssh from my laptop to VyOS. Yes, That works, and we can see if we do show firewall ipv4 input filter that there is a couple of packets hitting rule 10 which is our ssh rule.

Now we move on to forward filter rules, these are the rules that forward traffic from one subnet to another.

# Set default-action to drop and rule 10 do allow from OUTSIDE to INSIDE
set firewall ipv4 forward filter default-action drop
set firewall ipv4 forward filter rule 10 action accept
set firewall ipv4 forward filter rule 10 description 'Allow HOME/NET-OUTSIDE in NET-INSIDE'
set firewall ipv4 forward filter rule 10 destination group network-group NET-INSIDE <- TYPO in screenshot
set firewall ipv4 forward filter rule 10 source group network-group NET-OUTSIDE

# Set rule 20 drop INSIDE to OUTSIDE
set firewall ipv4 forward filter rule 20 action drop
set firewall ipv4 forward filter rule 20 description 'Drop NET-INSIDE to NET-OUTSIDE'
set firewall ipv4 forward filter rule 20 destination group network-group NET-OUTSIDE <- TYPO in screenshot
set firewall ipv4 forward filter rule 20 source group network-group NET-INSIDE

Verification time, lets test that I can no longer ping from VM to HOME network

OH NO…

We can no longer ping google.com lets find out why, lets monitor the forward filter and ping google.com at the same time and see whats going on.

monitor command 'show firewall ipv4 forward filter'

As we can see the dropped packet counter keeps going up, which means the packets are being dropped. Lets add another rule in to allow the VM access to the internet again.

set firewall ipv4 forward filter rule 15 action accepta
set firewall ipv4 forward filter rule 15 description 'Allow NET-INSIDE out to Internet'
set firewall ipv4 forward filter rule 15 destination address 0.0.0.0/0
set firewall ipv4 forward filter rule 15 source group network-group NET-INSIDE

Lets test again…

We can reach google.com again, lets just check we that we cant reach HOME address

OH NO… Because we set rule 15 to allow the destination to be 0.0.0.0/0 which is anywhere it allows our VMs into HOME agian. Lets reorder the firewall rules as they are processed lowest number to highest, so if we move rule 15 to rule 30 this should fix it. This is one of the main reasons for leaving gaps in firewall rule numbering so you can fit other rules in if needed.

In order to shuffle the rules around we need to delete the rule and re-add the rule with a new number.

# Deleting is easier
delete firewall ipv4 forward filter rule 15

# Now lets set the rule up to be 30
set firewall ipv4 forward filter rule 30 action accepta
set firewall ipv4 forward filter rule 30 description 'Allow NET-INSIDE out to Internet'
set firewall ipv4 forward filter rule 30 destination address 0.0.0.0/0
set firewall ipv4 forward filter rule 30 source group network-group NET-INSIDE

Final Verification

Can we ping out to the internet - google.com
Can we not ping HOME 192.168.1.0/24 addresses

SUCCESS , And if we monitor the forward filter again, we can see dropped packets for rule 20 and accept packets for VM out to the internet.

Now that was a long one , and If you made it this far, well done. There was a lot to cover in this one, for very little actual work. And this doesn’t even scratch the surface of what VyOS can do.

Next time in Part 3 We will look to put some of this into an Automated fashion using Ansible and Jinja2 templating.

And in future we will look at expanding the LAB to have two VyOS firewalls in a Highly available pair.