A poster of our latest work, CrossRSS, a Stateless CPU-Aware Datacenter Load-Balancer

Today we will present a poster of our latest work, at CoNEXT’20 : CrossRSS! CrossRSS is a load-balancer that spreads the load uniformly even inside the servers. It uses knowledge of the dispatching done inside the servers, RSS, to purposely select less-loaded cores without any server modification, or inter-core communications on the server. Learn more by watching the short video!

The poster session will be held on the 4th of December, 2:30 CET on the Mozilla VR Hub

Extended Abstract ; Hub ; Video ; Poster-As-Slides

Our latest paper “Cheetah”, a load balancer that guarantees per-connection-consistency

Cheetah is a new load balancer that solves the challenge of remembering which connection was sent to which server without the traditional trade off between uniform load balancing and efficiency. Cheetah is up to 5 times faster than stateful load balancers and can support advanced balancing mechanisms that reduce the flow completion time by a factor of 2 to 3x without breaking connections, even while adding and removing servers.

More information at https://www.usenix.org/conference/nsdi20/presentation/barbette.

Dynamic DNS with OVH

It may not be a clear thing, but OVH allows to have your own Dynamic DNS if you rent a domain name, surely a better thing than the weird paid website from dyndns.org. I will explain how to handle the update with Linux using ddclient.

On the manager

Connect to https://www.ovh.com/manager/web/#/configuration/domain/ , select your domain name, and create a new dynhost with the button on the right.

Enter a sub-domain name such as “mydyn” (.tombarbette.be), and add the actual IP for now, or just 8.8.8.8 for the time being.

Then it is not finished, you have to create a login that will be able to update that dns entry. Select the second button to handle accesses and create a new login.

Select a login, probably the name of the subdomain, the subdomain itself, and a password.

On the server

sudo apt install ddclient

Then edit /etc/ddclient.conf

protocol=dyndns2
use=web,web=checkip.dyndns.com
server=www.ovh.com
login=tombarbette.be-mydns
password='password'
mydns.tombarbette.be

Just do “sudo ddclient” to update once then “sudo service ddclient restart” to get it updated automatically.

May this be helpful to someone, personally I just forget it all the time so I wanted to leave a post-it somewhere.

Our new paper RSS++: load and state-aware receive side scaling

I’m delighted to announce the publication of our latest paper titled “RSS++: load and state-aware receive side scaling” at CoNEXT’19.

Abstract

While the current literature typically focuses on load-balancing among multiple servers, in this paper, we demonstrate the importance of load-balancing within a single machine (potentially with hundreds of CPU cores). In this context, we propose a new load-balancing technique (RSS++) that dynamically modifies the receive side scaling (RSS) indirection table to spread the load across the CPU cores in a more optimal way. RSS++ incurs up to 14x lower 95th percentile tail latency and orders of magnitude fewer packet drops compared to RSS under high CPU utilization. RSS++ allows higher CPU utilization and dynamic scaling of the number of allocated CPU cores to accommodate the input load while avoiding the typical 25% over-provisioning.

RSS++ has been implemented for both (i) DPDK and (ii) the Linux kernel. Additionally, we implement a new state migration technique which facilitates sharding and reduces contention between CPU cores accessing per-flow data. RSS++ keeps the flow-state by groups that can be migrated at once, leading to a 20% higher efficiency than a state of the art shared flow table.

Paper ; Video ; Slides

Our latest paper “Metron: NFV Service Chains at the True Speed of the Underlying Hardware” has been accepted at NSDI 2018 !

Abstract

In this paper we present Metron, a Network Functions Virtualization (NFV) platform that achieves high resource utilization by jointly exploiting the underlying network and commodity servers’ resources. This synergy allows Metron to: (i) offload part of the packet processing logic to the network, (ii) use smart tagging to setup and exploit the affinity of traffic classes, and (iii) use tag-based hardware dispatching to carry out the remaining packet processing at the speed of the servers’ fastest cache(s), with zero intercore communication. Metron also introduces a novel resource allocation scheme that minimizes the resource allocation overhead for large-scale NFV deployments. With commodity hardware assistance, Metron deeply inspects traffic at 40 Gbps and realizes stateful network functions at the speed of a 100 GbE network card on a single server. Metron has 2.75-6.5x better efficiency than OpenBox, a state of the art NFV system, while ensuring key requirements such as elasticity, fine-grained load balancing, and flexible traffic steering.

Paper, slides & video

Do HUAWEI CloudEngine switches support OpenFlow?

No, no and no.

Despite what the ONF says (https://www.opennetworking.org/product-registry/) it is not. Huawei’s OpenFlow implementation is actually broken. The very first  HELLO OpenFlow message is broken. It reports support for OpenFlow 1.4 in the HELLO message, but the rest of the message is absolutely not structured as defined in the standard.

After contacting all parties, it is clear that nobody will move about that, especially HUAWEI which wants to sell the Agile controller for a high price. It would appear that an old firmware, announcing OpenFlow 1.3 was compliant at the certification time but only if using an old software compliant with OpenFlow 1.3.0 and not newer, as starting with 1.3.1 after that the message is broken too.

Funny, I recently bought a HUAWEI smartphone that had trouble with SmartWatches. The seller told me that most smartwatches worked with every phones except Huawei ones, because their bluetooth implementation is not compliant. Seems to be a habit…

Project 4 deadline extended

Deadline extended to Thursday 13:59

Please take this time to write tests and merge them using the git at https://gitlab.montefiore.ulg.ac.be/INFO0940/project4scripts

Please already try to submit now with a barely working sys_pfstat allowing obvious error to be catched as early as possible.

Fork & Exec system calls to implement a shell

A shell, typically parse a command, then fork (duplicates itself).

The duplicated process replaces itself using Execvp by the program described in the command

The other process wait for the duplicated one to exit using waitpid. When that happens it prints the prompt again, ready for the next one to come. And the whole thing restarts again.

Not a very hard life.

Here are the important parts of the manpages of the 3 functions :

Fork

NAME
fork – create a child process

SYNOPSIS
#include <unistd.h>

pid_t fork(void);

DESCRIPTION
fork() creates a new process by duplicating the calling process. The
new process is referred to as the child process. The calling process
is referred to as the parent process.

The child process and the parent process run in separate memory spaces.
At the time of fork() both memory spaces have the same content.

RETURN VALUE
On success, the PID of the child process is returned in the parent, and
0 is returned in the child. On failure, -1 is returned in the parent,
no child process is created, and errno is set appropriately.

Execvp

EXEC(3) Linux Programmer’s Manual EXEC(3)

NAME
execvp – execute a file

SYNOPSIS
#include <unistd.h>
int execvp(const char *file, char *const argv[]);

DESCRIPTION

The execvp() function replaces the current process image with
a new process image.

The initial argument for this function is the name of a file that is
to be executed.

The const char *arg can be thought of as arg0, arg1, …, argn.
Together they describe a list of one or more pointers to null-termi‐
nated strings that represent the argument list available to the exe‐
cuted program. The first argument, by convention, should point to the
filename associated with the file being executed. The list of argu‐
ments must be terminated by a null pointer, and, since these are vari‐
adic functions, this pointer must be cast (char *) NULL.

RETURN VALUE
The exec() functions return only if an error has occurred. The return
value is -1, and errno is set to indicate the error.

waitpid

NAME
waitpid – wait for process to change state

SYNOPSIS

pid_t waitpid(pid_t pid, int *status, int options);

DESCRIPTION

The waitpid() system call suspends execution of the calling process
until a child specified by pid argument has changed state. By default,
waitpid() waits only for terminated children.

Home-Assistant : live camera feed and motion detection with a USB camera using motion

I want to display my webcam feed on home assistant. That’s easy and well explained on home assistant’s website. However they do not tell how to implement a motion detection system at the same time.

First step : set up the camera live feed as explained in the docs

In your configuration.yaml

camera:
- platform: mjpeg
mjpeg_url: http://localhost:8081
name: Salon

Install motion :

sudo apt-get install motion

Configure /etc/motion/motion.conf (change these values 🙂

daemon on
stream_port 8081
stream_quality 80
stream_maxrate 12
stream_localhost on

And then restart motion :

sudo service motion restart

And home assistant, then the webcam should appear ! Yeah !

Now the motion detection. The method I took is to use the mqtt protocol. A binary sensor will be the state of motion detection, motion will publish updates to the given topic to say if motion is on or off, and home assistant will subscribe to it.

Add this in your HA configuration.yaml

mqtt: #I pass the mqtt setup process
broker: 127.0.0.1
port: 1883
client_id: home-assistant
keepalive: 60
protocol: 3.1

binary_sensor:
- platform: mqtt
state_topic: "living_room/cam1"
name: cam1
sensor_class: motion

Install mosquitto-clients :

sudo apt-get install mosquitto-clients

The commande to start a motion event is :

mosquitto_pub -r -i motion-cam1 -t "living_room/cam1" -m "ON" 

-r sets the retain flag
-i is just a client id
-t is the topic, which should match the configuration in mqtt
-m Sets the message content, ON for motion being detected, OFF for a still image.

Then we have to update motion.conf accordingly:

on_event_start mosquitto_pub -r -i motion-cam1 -t "living_room/cam1" -m "ON"
on_event_end mosquitto_pub -r -i motion-cam1 -t "living_room/cam1" -m "OFF"

And restart motion ! And it’s finished !