A poster of our latest work, CrossRSS, a Stateless CPU-Aware Datacenter Load-Balancer

Today we will present a poster of our latest work, at CoNEXT’20 : CrossRSS! CrossRSS is a load-balancer that spreads the load uniformly even inside the servers. It uses knowledge of the dispatching done inside the servers, RSS, to purposely select less-loaded cores without any server modification, or inter-core communications on the server. Learn more by watching the short video!

The poster session will be held on the 4th of December, 2:30 CET on the Mozilla VR Hub

Extended Abstract ; Hub ; Video ; Poster-As-Slides

Our latest paper “Cheetah”, a load balancer that guarantees per-connection-consistency

Cheetah is a new load balancer that solves the challenge of remembering which connection was sent to which server without the traditional trade off between uniform load balancing and efficiency. Cheetah is up to 5 times faster than stateful load balancers and can support advanced balancing mechanisms that reduce the flow completion time by a factor of 2 to 3x without breaking connections, even while adding and removing servers.

More information at https://www.usenix.org/conference/nsdi20/presentation/barbette.

Our new paper RSS++: load and state-aware receive side scaling

I’m delighted to announce the publication of our latest paper titled “RSS++: load and state-aware receive side scaling” at CoNEXT’19.


While the current literature typically focuses on load-balancing among multiple servers, in this paper, we demonstrate the importance of load-balancing within a single machine (potentially with hundreds of CPU cores). In this context, we propose a new load-balancing technique (RSS++) that dynamically modifies the receive side scaling (RSS) indirection table to spread the load across the CPU cores in a more optimal way. RSS++ incurs up to 14x lower 95th percentile tail latency and orders of magnitude fewer packet drops compared to RSS under high CPU utilization. RSS++ allows higher CPU utilization and dynamic scaling of the number of allocated CPU cores to accommodate the input load while avoiding the typical 25% over-provisioning.

RSS++ has been implemented for both (i) DPDK and (ii) the Linux kernel. Additionally, we implement a new state migration technique which facilitates sharding and reduces contention between CPU cores accessing per-flow data. RSS++ keeps the flow-state by groups that can be migrated at once, leading to a 20% higher efficiency than a state of the art shared flow table.

Paper ; Video ; Slides

Our latest paper “Metron: NFV Service Chains at the True Speed of the Underlying Hardware” has been accepted at NSDI 2018 !


In this paper we present Metron, a Network Functions Virtualization (NFV) platform that achieves high resource utilization by jointly exploiting the underlying network and commodity servers’ resources. This synergy allows Metron to: (i) offload part of the packet processing logic to the network, (ii) use smart tagging to setup and exploit the affinity of traffic classes, and (iii) use tag-based hardware dispatching to carry out the remaining packet processing at the speed of the servers’ fastest cache(s), with zero intercore communication. Metron also introduces a novel resource allocation scheme that minimizes the resource allocation overhead for large-scale NFV deployments. With commodity hardware assistance, Metron deeply inspects traffic at 40 Gbps and realizes stateful network functions at the speed of a 100 GbE network card on a single server. Metron has 2.75-6.5x better efficiency than OpenBox, a state of the art NFV system, while ensuring key requirements such as elasticity, fine-grained load balancing, and flexible traffic steering.

Paper, slides & video

Do HUAWEI CloudEngine switches support OpenFlow?

No, no and no.

Despite what the ONF says (https://www.opennetworking.org/product-registry/) it is not. Huawei’s OpenFlow implementation is actually broken. The very first  HELLO OpenFlow message is broken. It reports support for OpenFlow 1.4 in the HELLO message, but the rest of the message is absolutely not structured as defined in the standard.

After contacting all parties, it is clear that nobody will move about that, especially HUAWEI which wants to sell the Agile controller for a high price. It would appear that an old firmware, announcing OpenFlow 1.3 was compliant at the certification time but only if using an old software compliant with OpenFlow 1.3.0 and not newer, as starting with 1.3.1 after that the message is broken too.

Funny, I recently bought a HUAWEI smartphone that had trouble with SmartWatches. The seller told me that most smartwatches worked with every phones except Huawei ones, because their bluetooth implementation is not compliant. Seems to be a habit…

Use tile-eclipse to launch a software on a remote Tilera TileEncoreGX36 through SSH

Here is the set-up to make it work :

First find the listening monitor port. On this picture it is 34531.

We’ll have to set up a reverse ssh forwarding for the tile-monitor to connect to our tile-eclipse instead of trying to connect to some local listener on the remote host.

Run :
ssh -R 34531:localhost:34531 sauron.run.montefiore.ulg.ac.be
Where in our case, the port 34531 is the one you found in tile-eclipse, and sauron.run.montefiore.ulg.ac.be is our host where the tile is connected.

Each time you re-run tile-eclipse you’ll have to redo that part as the port will change.

Then only once you have to set up your run configuration.

In the hardware part, correctly set up the hardware :

And in the monitor part, set remote as given in this picture and click on manage hosts :

And add your own :

I never found a better way to set it up (not using myself a ssh -R reverse forwarding), it should be possible to set it up automatically.

Making Tilera TileMDE work on Debian 7 with Kernel 3.14

This review how to install Tilera MDE, with the slight modifications to support recent kernel and the debian environment. Our device is a TILEncore-Gx36

Edit : with MDE 4.3.2, the patch isn’t necessary anymore. You may use ./tilera-compile –gpl-license to be able to go through compilation

Install & Unpack

  • cd /opt/
  • Extract the primary tareball with sudo tar -xvf /home/tom/TileraMDE-
  • Run “unpack”  sudo ./TileraMDE-
  • (optional instead of the last point) unpack the full tarball with sudo ./TileraMDE- /PATH_TO/TileraMDE-
  • We’ll keep the Tilera MDE’s root on /opt/tilera, so move it there with sudo mv ./TileraMDE- /opt/tilera && cd /opt/tilera

Setup environment

You’ve got to setup some environment variables. One way to do it is to add at the bottom of your ~/.bashrc file :


Compile and fail…

You’ve got to compile the Tilera module :

  • cd /opt/tilera/lib/modules/
  • Normally you would do “./tilepci-compile  –tilera-license”
  • Then “./tilepci intall”

But that would fail because :

  • It rely on redhat tools like chkconfig
  • It will compile a module not compatible with kernel 3.0+

You can do it anyway to extract a new folder for your current kernel, in my case “3.14-2-amd64-x86_64”

Apply the patch

So we’ll use my updated version of Sylvain Martin’s patch available here ( tom-fixes-3.14 ):

Warning : The patch is untested for many features (mostly related to /proc), but works with the stantard things you’ll want to do… Access the tilera, use tile-monitor, … It’s provided without any warranty.

  • rm -rf 3.14-2-amd64-x86_64
  • cp -rf tilera_src 3.14-2-amd64-x86_64
  • cd 3.14-2-amd64-x86_64/pcie
  • Apply the patch with patch -p1 < /PATH_TO/tom-fixes-3.14.txt
  • make INSTALL_PATH=/opt/tilera/lib/modules/3.14-2-amd64-x86_64


tilepci-install will want “chkconfig” which is a redhat tool. I provide a simple wrapper here : https://github.com/tbarbette/chkconfigwrapper/blob/master/chkconfig , you just have to copy it in /usr/sbin/ and launch ./tilepci-install


If you have any comment or can provide any help, do not hesitate to comment !