OPNsense and Wyse 5070: A Love Story ❤

When I first got my fiber internet connection, I was over the moon with its performance and the way I could instantly download some files. On the flip side, however… A terrible provider-supplied modem router (an Arcadyan VRV7519 rebranded as the Experiabox V10a) that did everything I needed but none of what I wanted. Okay, maybe it wasn’t that bad in hindsight, but I’m a tinkerer and there was nothing I could tinker with.

Table of Contents

• What came before
• Choosing a firewall solution
• Why the Wyse 5070 is great
• Adding a NIC to get seperate WAN and LAN interfaces
• Mounting the NIC daughter board
• Adding more (durable) storage
• BIOS settings and installing OPNsense
• Kit list

What came before

After careful consideration, I decided to go ahead with a Unifi Unified Security Gateway to replace the simpleton that was the V10a. Whilst this was acceptable for a while, the recently revamped Unifi USG control panel in combination with the USG’s age and ending manufacturer support had me longing for something more… something I could tinker with even more.

The solution would have to be able to completely replace my USG that I had connected to the NTU directly through ethernet and not require any additional hardware. My provider KPN uses different VLANs to deliver the different services (like internet, IPTV, telephony) to the modem, so VLAN support was also an important requirement. Lastly, IPTV would have to be routed in to the LAN from the provider’s IPTV VLAN and then work without quality or performance problems. That last bit means it needs IGMP proxy capabilities! The final requirement was to have some form of client VPN available.

Choosing a firewall solution

I had always wanted to try out OPNSense, which is a fork of pfSense. Both fulfilled all my requirements but in the end I picked OPNsense over pfSense for a couple of reasons:

• OPNsense uses HardenedBSD as a base OS where pfSense uses FreeBSD. Whilst both OSes are great and both share the same codebase, I feel HardenedBSD has a better policy on security and hygiene than vanilla FreeBSD. Do know that OPNsense is transitioning (back) to FreeBSD in the future. However, I have confidence that when they do, the OPNsense community will take with them the pillars HardenedBSD was built on.
• In my experience, OPNsense has a more intuitive and polished interface.
• OPNsense is maintained by the Dutch company Deciso. That means it has to adhere to strict Dutch/EU regulation on privacy and data.
• When comparing the mission statements between OPNsense and pfSense, the one OPNsense provides sounds easier on the (OSS) community to me.
  
  OPNsense mission statement:
  “Give users, developers and businesses a friendly, stable and transparent environment. Make OPNsense the most widely used open source security platform.“
  
  pfSense mission statement:
  “We provide leading-edge network security at a fair price – regardless of organizational size or network sophistication. We believe that an open-source security model offers disruptive pricing along with the agility required to quickly address emerging threats.“

Of course, this is a very personal preference and many features and inner workings are similar between the two products. Now for something to run it on…

Enter the Dell Wyse 5070 thin client that I came across for pennies. This super compact little machine has turned out to be a very capable device to run an OPNSense firewall on. Let me tell you why.

Why the Wyse 5070 is great

The Wyse 5070 comes in two form factors: slim (the one I got) and extended. The latter is about double the thickness of the former and features a slot for a half-size PCIe card. This is ideal for adding additional NIC(s) through a regular PCIe card. You can find these cards relatively cheap too. Other than that, the extended version has a couple more serial and parallel ports on the back, none of which were needed for my OPNSense firewall build.

The little machine, even in its slim form factor is great for my purpose for a plethora of reasons:

• The Intel Celeron™-based CPU supports AES–NI (Intel Advanced Encryption Standard New Instructions). This greatly improves the speed and performance of cryptographic operations. This is very nice to have when you want to use things like VPN. pfSense (for a brief moment) even had AES-NI as a requirement to run at all. They have since dropped this requirement but it illustrates the importance somewhat.
• It can run completely headless. (You can disable peripheral detection errors in the BIOS.)
• It has a low power consumption. I have not had it draw more than ~15 Watts whilst powering on. When Idle it runs at about a measily ~6 Watts.
• It generates almost no heat. My setup will eventually end up in a pretty cramped space and I don’t want it to overheat.
• It runs completely silent as there are no moving parts inside.
• It’s tiny and measures just 18,3 cm x 3,6 cm x 18,3 cm.
• It has an M.2 2260/2280 slot that takes SATA SSDs. (B)
• It has a second M.2 slot that Dell uses to offer a WiFi add-on card to the thin client. I was hoping to use this to add a second ethernet NIC to the setup. (C)
• It has expandable RAM and two SO-DIMM slots. Mine came with a single strip of 4GB DDR4 RAM but I was able to add another 4GB through the second (free) slot. I only need 8GB in total but the Wyse will happily accept 16GB. I had to update the BIOS to be able to use more than 4GB of RAM. (A)

Adding a NIC to get seperate WAN and LAN interfaces

There’s just one downside to owning the slim version and that is that there is no way of adding a second NIC as easily as it is with the extended version.

Whilst it’s theoretically possible to deploy a one-armed instance of OPNsense that utilizes a single physical NIC for both WAN and LAN traffic by using VLANs, this is a hassle to set up and maintain. This is why I opted to find a NIC that I could fit into the onboard M.2 socket. The socket is an M.2 2230 E-key socket so you need either an M.2 2230 E-key or A+E-key card to fit the Wyse.

Now when it comes to NICs, there are three major players on the market that produce the chipsets: Intel, Realtek and Broadcom. Whilst Intel chips are widely supported and are considered the most stable and feature-rich ones by many in the community, these are also very hard to come by in the form factor that we need. With Realtek and Broadcom chips, it’s a case of hit and miss in both OS/driver support and stability. Below are some of the options I explored and the option I went with eventually (in order of trying them).

• Getting an Intel in the correct M.2 2230 (A+)E-key form factor. Unfortunately, I was unable to get my hands on one. The one I went for in the first place was the Commell M2-210 as other users had reported success with using this card in combination with the Wyse 5070. However, Commell don’t actively produce them at this moment due to the ongoing chip shortages. They would spin up production for me only if I agreed to buy 50+ pieces. That was a bit much for me. But hey, if you want to get 50+, give them a call! ❌
• Getting an adapter that converts mini PCIe to M.2 A+E-key and then using a mini PCIe NIC:
  • Unbranded Chinese Intel I210AT-based NIC.
    Was not detected by BIOS or the OS. ❌
  • Digitus DN-10133 Realtek RTL8111F-based NIC.
    Was not detected by BIOS or the OS. ❌
• Getting an Aispark branded Winyao Realtek RTL8111F-based NIC in the correct M.2 2230 A+E-key form factor. These are available on plenty websites, usually advertised as an “M.2 NIC for a LattePanda Alpha or Delta” for around €20 including postage and packaging. SUCCESS! ✅

One pro to the NIC I did have success with is that it uses straight angle connectors for the ribbon cable out of the box. The others had straight upward-pointing connectors that prevented the lid from closing correctly. Had I gone with any of those, I would have had to replace the connectors by soldering on new ones.

Although I ended up having to use a Realtek-based NIC, I have had no performance problems since starting to use my OPNsense setup. It was detected and installed by OPNsense out-of-the-box and I could start configuring it through the GUI right away.

In order for the M.2 socket to work, you need to enable the WiFi add-on functionality in the BIOS. Otherwise, the Wyse will simply refuse to detect the card.

Mounting the NIC daughter board

So even though the NIC works fine now, I still have it dangling out of the back by its ribbon cable. Whilst functional, it’s not very sturdy and certainly an eyesore in my book. I’ll let the evidence speak for itself.

The solution to a problem like this seems fairly evident and is something that is (luckily) increasingly available for any tinkerer: 3D printing! I have taken a model from a Thingiverse post that is meant for a different board and edited that to fit the Aispark daughter board. Thanks, examiner!

Luckily, the original author used scad with dependencies that automatically calculate the correct dimensions depending on the daughterboard design. All I had to do was recreate the Aispark daughterboard in OpenSCAD and load it into the project’s existing master SCAD file. Tadaa…

All I have to do now is get it printed and perfect the fit. Once I make progress on that, I’ll report back here.

Adding more (durable) storage

The Wyse can come with either on-board storage or without, depending on the exact model you have. My version has 16GB of eMMC flash storage built in. You can easily install OPNsense on the on-board eMMC storage as I did at first, but eMMC is slow and not very durable. Remember Tesla having to replace a bunch of hardware because the eMMC chips had worn out due to writing logs to it all the time?

That’s why I opted to add a spare 256GB SK Hynix SATA SSD I had lying around that fit the M.2 2260/2280 slot. This should provide me with more than enough storage for the forseeable future.

Please note that the M.2 2260/2280 only takes SATA drives and does not support PCIe/NVMe-based drives.

BIOS settings and installing OPNsense

Installing OPNsense is fairly easy and the installation medium does not require any sort of slipstreaming drivers. I simply downloaded the latest amd64 vga image from the website and created a bootable USB drive using my favourite tool Rufus. Simply select the correct USB drive, proper iso file, and hit Start.

Power up your Wyse and once the Dell logo pops up, hit F12 repeatedly until the boot menu shows. This also has an option to enter the BIOS Setup. I have had to set the following settings:

• Under Secure Boot, uncheck Enable Secure Boot.
• Under System Configuration, USB Configuration, check Enable USB Boot Support.
• Under System Configuration, USB Configuration, make sure you have at least the front or the back ports enabled.
• Under System Configuration, 2nd NIC (RJ-45/SFP), select Enabled w/PXE.
• Under Wireless, Wireless Device Enable, check WLAN/WiGig and Bluetooth.
• Under POST behavior screen options, Keypad Error, uncheck Enable Keyboard Error Detection.

For a full BIOS option overview, please refer to the Dell documentation.

The default password for the BIOS is “Fireport” but there is also a jumper on the main board that allows you to reset the password.

After setting all necessary BIOS options, reboot with the USB drive in one of the USB ports that you have enabled in the BIOS in the previous section and hit that F12 button again like there’s no tomorrow.

This time, select the boot option that belongs to your USB drive and hit enter. The entry should usually have the brand and sometimes the capacity of your USB drive mentioned. When asked, enter the BIOS password to confirm USB boot.

All set! You should now be booting into the OPNsense live environment. I’m not including the exact settings I used to get my OPNsense setup working with my provider KPN in this post but I posted that in another post. In this post, I have only detailed what I had to do to be able to boot into the USB drive that has the OPNsense installation image on it.

Kit list

So, what went into this build?

• 1x Wyse 5070 slim form factor
• 1x Aispark branded Winyao Realtek RTL8111F-based NIC in M.2 2230 (A+)E-key form factor.
• 1x 3D-printed mount for NIC daughter board. Coming soon!
• 2x M3 screw to screw the 3D-printed mount to the casing.
• 1x SK Hynix 256GB M.2 2280 SATA SSD.
• 1x 4GB DDR4 SO-DIMM