Main container

Archive for the ‘Events’ Category

pfSense around the world, better IPsec, tryforward and netmap-fwd

Last week Renato Botelho do Couto, a pfSense developer and FreeBSD ports committer, presented a talk on pfSense at Lantinoware.  Renato reported that the room for his talk was full, and that many people wanted to talk after.  It’s great to see this type of response to pfSense in the world.

During the past month, I’ve attended vBSDcon 2015 in Virginia, USA, EuroBSDcon 2015 in Stockholm, Sweden, and BSDCon Brazil 2015, in Fortaleza, Brazil.   All along that way, (over 27,000 miles or 43,400 km), I’ve enjoyed having Groff, the BSD Goat as a traveling companion, and meeting many great BSD and pfSense people in each location.

At vBSDcon, EuroBSDcon and BSDCon Brazil, either George Neville-Neil or I spoke on, “Measure Twice, Code Once”.  This is our continuing series reporting on a continual, longitudinal study of networking performance in FreeBSD and pfSense.  The most recent developments here are the big improvement in IPsec performance with AES-NI support (1270 Mbps throughput, single stream, for AES-GCM with a 128-bit key on a pair of ~3GHz E5 Xeon CPUs), and the introduction of ‘tryforward’ to FreeBSD.  The IPSec changes are already in -CURRENT, and the MFC to -STABLE has been accomplished in our FreeBSD tree on github.  With some luck, these will also be present in FreeBSD 10.3-RELEASE, when it occurs.

FreeBSD has had a ‘turbo’ button of sorts since 2003.  Enabling this feature via “sysctl -w net.inet.ip.fastforwarding=1” on FreeBSD, or via System > Advanced > System Tunables on pfSense, improves forwarding, but at the expense of reception of packets on the box (a 4% hit compared to fastforwarding=0), and, more importantly for pfSense, disabling IPsec.  The tryforward code replaces the fastforward path with a tryforward() function.  Since this isn’t controlled by a sysctl, it is “always on”.  Importantly, tryforward() both improves the reception of packets on the box (around a 1% hit .vs the normal (non-fastforward) kernel path), and also results in functioning IPsec.

While this doesn’t improve the speed of IPsec, it  does allows us to be rid of the fake fastforwarding path and have good forwarding in the normal case while also having IPSEC in the kernel.   The tryforward() code should make it into pfSense version 2.3.

Also at BSDCon Brazil, Luiz Otavio Souza, a pfSense developer and FreeBSD src commiter, presented on his recent work, “netmap-forward: An IPv4 router over netmap for FreeBSD”.   This is basically the FreeBSD fastforward code ported to run in userspace over netmap.

When evaluating or measuring an Ethernet device’s (switches, routers, firewalls) performance capabilities, the main indicator that most will consider is the raw bandwidth that the device backplane can provide.  However it is also important to make sure that the device has the capacity or the ability to switch/route as many packets as required to achieve wire rate performance.  This metric is known as ‘Packets per Second’ or PPS.

Obviously, the smallest packet size will lead to the largest PPS rate, IF the system can handle it.

On Ethernet, the smallest frame size is 64 bytes, and if you look at router or switch literature very long, you’ll see reports of “64 byte packets”.  Importantly, this doesn’t count some additional framing overhead on Ethernet.  A true “minimum-sized” frame on Ethernet consists of a 12 byte inter-frame gap, 8 bytes of MAC preamble + SFD, 14 bytes of MAC header (6 bytes source address, 6 bytes destination address, 2 bytes of Ethernet ‘type’ (e.g. 0x0800 is IPv4, 0x0806 is ARP), 46 bytes of minimum payload (for IPv4 this includes any IP header, plus UDP or TCP headers, plus a very small amount of data.  A IPV4 + UDP frame would allow a mere 6 bytes of payload.  The headers for IPv6 are large enough to), and finally a 4 byte CRC.  Combined, this results in a minimum packet of 84 bytes (20 + 64).  Similarly, the maximum MTU Ethernet frame size is 1538 bytes for a 1500 byte frame.  (12 + 8) + 14 + 1500 + 4).  802.1q VLAN tagging allows four more bytes, if enabled.

The maximum frame rate of 1Gbps Ethernet is 10*10^8 bits/sec / (84 bytes * 8 bits/byte).  This equals 1,488,095 PPS.  10Gbps Ethernet is 10X the rate, or 14,880,952 PPS.   Increasing the frame size of 1500 byte (max MTU) packets substantially reduces the required PPS rate to ‘fill’ the interface.  Using the equation above, and substituting 1538 byte frames for 84 byte frames, we see that it only requires 81,274 PPS to fill a 1Gbps Ethernet with maximum-sized frames.  10Gigabit Ethernet is, again 10X this rate, requiring 812,743 PPS to fill a 10G interface with max-sized frames.

These are high, though achievable rates for software routers.   Using a Xeon E3-1275 (4 cores @ 3.5GHz) FreeBSD -CURRENT can forward at a rate of around 1.058 Mpps.  Turning on fastforwarding (or building a kernel with tryforward support) increases this rate to about 1.33Mpps.  While this is enough to ‘fill’ a 10Gbps link with full-sized frames, not all frames are full-sized, and the true test of a router is it’s ability to forward a mix of traffic, throttled only by the speed of its network interfaces.

Using a bit more pedestrian hardware, such as the C2758 that is for sale on the pfSense store, we find that we can forward at a rate of around 270 Kpps, and with fast forwarding or tryforward, we can obtain 426 Kpps.  A simple SG-2220 will support 123 Kpps until we enable fastforward or tryforward, when we can obtain 217 Kpps.

netmap-fwd, available as BSD licensed open source on github, substantially changes these results.

The SG-2220 that previously forwarded 123 Kpps or 217 Kpps with fastforward will obtain 945 Kpps with netmap-fwd.

The C2758 that could forwarded 270 Kpps or 426 Kpps with fastforward on, will obtain 1.683 Mpps with netmap-fwd over a Chelsio T-520 10G Ethernet interface.

And the Xeon E3-1275 that would previously strain to obtain 1.33 Mpps with fastfoward on will obtain 5.05 Mpps with netmap-fwd using an Intel X520 10G interface.  This is over 1/3 of the line-rate required to forward a full 10G of minimum-sized IP packets.

With netmap-fwd, the host stack is still available, so packets destined for the router are correctly routed to and from the host stack.  This means the applications you known and love still work.  Want to use ssh to manage your router?  It works.  Ansible?  It runs over ssh.  Saltstack?  It should work, we haven’t tried it yet.  VLANs are also supported.  Configuration is simple: you configure the interfaces via the normal mechanism on FreeBSD (ifconfig, rc.conf, etc), and start netmap-fwd, giving it a list of interfaces.

Importantly, the numbers cited are all without substantial tuning, and are using an early, and still in-development version of netmap-fwd  that is limited to running on a single core.  All of the devices above have multiple cores, and it is likely that we can substantially increase the performance obtained thus far using multi-threaded techniques.   We will also add ACLs, IPv6, BGPD / FIB integration, and better runtime statistics.  Additional protection will be gained by using Capsicum to sandbox the application.

If you want to read more, Luiz’s slides are available.

Back in February, I wrote a blog post that discussed our plans for pfSense software version 2.3, which is now in alpha, and our plans for pfSense 3.0.  While I promoted DPDK then, we’ve since found that netmap provides a simpler API, and substantially better safety, as the device drivers remain in the kernel, rather than running in userspace with DPDK.  Still, DPDK provides a set of libraries, such as longest-prefix match, which uses a variation of the DIR-24-8 algorithm for routing lookups, which we should find useful in our pursuit of the ultimate open source software router.

“Das ist sehr, sehr viel Arbeit die da versprochen wird.”, indeed.  But we are making good on that promise.

With the advent of netmap-fwd, the road ahead to pfSense 3.0 can be clearly seen.  As Tom Wolfe wrote, “Put your good where it will do the most!”

We’re doing that.  Join us.

March Hangout Announcement

Our March Hang Out for Gold Members will be Friday, March 27, at 13:00 CDT (-5 UTC). You’ll find the link and other information under “March 2015 Hang Out” after logging in to the members area.

The March 2015 hangout will cover Bandwidth Monitoring.. We will discuss how to monitor using built-in items like the RRD graphs as well as various add-on packages and techniques for tracking bandwidth by IP address.

In order to join us, you must be a pfSense Gold subscriber.

Subscribe to Gold Here!

edit: Apologies, the wrong time was sent out. It is Friday, March 27.

Here is a preview of the hangout:

AsiaBSDCon 2015

I’ll be leaving for AsiaBSDCon in a few days, so I thought I would show-off the custom-etched Beaglebone Black enclosures and the PC Engines APU case we’re doing as give-aways there. I’ll be passing through SLC for Saltconf and Niseko (skiing) on my way there.

If you attend either AsiaBSDCon or Saltconf, be sure to come talk to me about pfSense, high-performance networking, cryptography, or any other interesting topic.


February Hangout Announcement

Our February Hang Out for Gold Members will be Friday, February 27, at 13:00 US Central time. You’ll find the link and other information under “February 2015 Hang Out” after logging in to the members area.

The February 2015 hangout will cover User Management and Privileges. We will discuss how to grant access to the GUI using the pfSense privilege system to control what users may access, SSH access and command privileges using the sudo package, and securing access to pfSense firewall management.

Mark your calendar and we look forward to seeing you!

In order to join us, you must be a pfSense Gold subscriber.

Subscribe to Gold Here!

Here is a preview of the hangout:

November Hangout Announcement

Our monthly pfSense hangout has been scheduled! Please make note that because of the Thanksgiving holiday, we will have the event on Tuesday, November 25, 2014 at 1PM CST. Log-in to your portal account on the day of the event for details on how to join us. This month’s topic is: New and Improved Features in pfSense 2.2. Your host will be Jim Pingle

Here is a preview of the hangout:

pfSense October Hangout Announcement

We’re happy to announce the pfSense Gold Hangout for October 2014!

The older ALIX-based appliances (i.e. m1n1wall) are a tremendously popular and successful product. But as one might expect, this product is rapidly approaching its end of life (EOL). It’s successor, the VK-T40E is taking its place as the leader in small, form-factor, low power, entry level pfSense firewall appliances for small business, SOHO, and remote branch office environments.

Our October Hangout will cover the process for upgrading from an older ALIX unit to its successor, the VK-T40E. We will talk about upgrade precautions, how to make the move easier, common pitfalls, and deploying the new device.

When: Friday, October 24, 2014 @ 1PM Central US Time Where: Check your portal account on the day of the event for the link to join us Who: This event will be hosted by Jim Pingle

Mark your calendar and we look forward to seeing you!

In order to join us, you must be a pfSense Gold subscriber.

Subscribe to Gold Here!

Get your VK-T40E here

Here is a preview of the hangout:

August Hang Out – Network Address Translation

Our August hang out will be next Friday, August 15, at 13:00 US Central time. Join us for around an hour and a half of coverage of NAT, with time for questions to follow.

NAT is among one of the most widely used features in pfSense and one we haven’t yet gone over in detail in a hang out. Topics covered will include the following.

  • How NAT functions in general terms, and specifically with pfSense
  • Uses of NAT – more than just connecting your private network to the Internet.
  • NAT’s interaction with firewall rules
  • Live configuration examples of redirection using port forwards, 1:1, and outbound NAT
  • Troubleshooting guidance

Being tied up in the time-consuming materials preparation for our first pfSense University class last week, I unfortunately didn’t have time to adequately prepare for a hang out in July. We’ll make that up to you with an extra session in August or September, date to be determined.

This is an exclusive benefit for our Gold subscribers. The link to join the session can be found after logging into your account in the members area.

Thanks for your support, and look forward to having you there!

Here is a preview of the hangout:

June 2014 Hang Out – Firewalls and Virtualization

Our June 2014 hang out is Friday, June 27 at 13:00 US Central time. This month’s topic is firewalls and virtualization. This is an exclusive benefit for our Gold subscribers. Subscribers will find the meeting link after logging in to the members section. If you’re not yet a subscriber, sign up now and you’ll get immediate access. If you can’t make the live event, the video and audio recording and slides are available for members to download within a few hours of the session’s completion.

As companies and individuals have virtualized their server infrastructures, they have also looked to virtualize their firewalls. This brings many questions to mind. Is it a good idea? Is it secure? How does it work? What are my options for configuration? Can I get adequate performance?

pfSense Co-founder Chris Buechler will answer all these questions and more during June’s hang out.

Attendees will come away with the knowledge of where virtualized firewalls may be a good fit, where they’re probably a bad idea, the potential security implications, knowledge of the various network configuration options available in hypervisors, options for handling high availability, and more. Both desktop-class and server-class products will be covered, including bhyve, Hyper-V, KVM, Parallels, VirtualBox, VMware (Workstation, Player, Fusion and ESX/ESXi), and Xen.

Usage areas covered will include production systems, test and development environments, and fun but ugly hacks that can work temporarily if you’re in a bind.

Here is a preview of the hangout:

Announcing pfSense University

After months of preparation and high customer demand for official pfSense training, Electric Sheep Fencing, the company behind the pfSense project, is very excited to announce our new training initiative, pfSense University

Our first class, “pfSense Fundamentals and Practical Application” is a two-day event which will cover common usage scenarios, deployment considerations, step by step configuration guidance, and best practices. This course will also enhance your skills and abilities to install, configure and support pfSense in your environment. We have scheduled two sessions of this class – one in August and one in September.

To get more information about this class and to sign up, please see pfSense University!

Introductory price is $1999.00 per class, but use promotional code BBC425FF on the sign-up page for an instant $500 discount!

In addition to official pfSense training, attendees will receive a one-year subscription to pfSense Gold, a pfSense T-shirt, and an entry into a raffle to win a VK-T40E2.

Breakfast and a lunch buffet are provided both days.

Register quickly! Each class is limited to 20 participants!

Please send any questions to