Read trending IT updates for cloud businesses, managed service providers, IT pros & what innovation digital transformation is driving in tech industry.

Correction: Multi-Threaded Routing Daemons

0 5

After I wrote the Why Does Web Hold Breaking? weblog publish just a few weeks in the past, I claimed that FRR nonetheless makes use of single-threaded routing daemons (after a too-cursory learn of their documentation).

Donald Sharp and Quentin Younger politely instructed me I used to be an fool I ought to get my details straight, I eliminated the offending a part of the weblog publish, promised to write down one other one going into the main points, and Quentin improved the documentation within the meantime, so right here we’re…

Why Does It Matter?

In a phrase1: sanity, efficiency and responsiveness.

Networking engineers like to construct artisanal wheels, and routing protocol designers aren’t any higher. Each routing protocol has a bespoke implementation of the identical three main functionalities:

  • Cope with neighbors: uncover them, preserve them comfortable, and work out when one in all them retains quiet for too lengthy.
  • Cope with updates: obtain them, acknowledge them (when the protocol designer thought he might do higher than TCP), ship new info out, and retransmit it if wanted (but once more, just for individuals who assume TCP sucks)2
  • Cope with adjustments: Replace inside topology info based mostly on acquired updates, calculate new routing tables, push new stuff into routing desk to compete with different stuff.

Does it make sense to do all three issues in a single monolithic blob of code? Positive it does when you assume juggling is a superb pastime. Everybody else tries to remain sane by decoupling issues into smaller bits that may be executed independently.

For the sake of completeness: Cisco IOS programmers figured that out many years in the past. For instance, Cisco IOS OSPF implementation makes use of two processes per routing protocol: a hey course of and a router course of:

A number of OSPF-related processes are operating on a Cisco IOS field with a single OSPF routing course of

s1#present ip protocols abstract
Index Course of Title
0     linked
1     static
2     utility
3     ospf 1

s1#present processes | embrace OSPF
 224 Mwe  2D2CE17           23        285      80 8956/12000  0 OSPF-1 Router
 228 Mwe  2D30EB5           15        288      52 9044/12000  0 OSPF-1 Whats up

Processes or Threads?

The Cisco IOS printout talks about processes, FRR documentation talks about threads. What’s the distinction?

Right here’s a sweet-and-short reply I discovered on (the place else) StackOverflow:

The standard distinction is that threads (of the identical course of) run in a shared reminiscence house, whereas processes run in separate reminiscence areas.

From that perspective, Cisco IOS processes are actually threads as Cisco IOS doesn’t have inter-process isolation3.

The one right reply to “when ought to one use threads as an alternative of processes” is “it relies upon”, or we might preserve going for hours. To make a protracted story quick: each time unbiased bits of code share sockets (together with TCP classes) or reminiscence constructions, it’s simpler to say “who cares about reminiscence isolation” and use threads.


Now that I discussed Cisco IOS, I’ve so as to add one other little bit of trivia: Cisco IOS is a non-preemptive working system. So long as one course of retains operating, no one else can leap in to get one thing accomplished (like sending a badly wanted HELLO message)4.

Fashionable working programs like Linux can do higher. Processes or threads might be interrupted or ran in parallel on a number of CPU cores or sockets, which signifies that a conserving neighbors comfortable thread can preserve sending HELLO messages or BGP keepalives whereas the updating the BGP desk thread scratches its head attempting to determine what to do with one other half one million updates that simply got here in.

The advantages of having the ability to leap it at any time and get a small job accomplished are even larger as soon as we begin speaking about almost-real-time stuff like BFD.

Coming again to FRR: in accordance with course of structure documentation, they cut up BGP performance into three threads: an I/O thread, a keepalive processing thread, and a remainder of the stuff thread.


I discussed that you would additionally use threads to extend efficiency once you occur to have too many CPU cores. For instance, you would have a number of threads processing incoming BGP updates in parallel, and one other bunch of threads constructing outgoing updates. I’m optimistic there could be a noticeable profit in environments with dozens or a whole lot of BGP neighbors5.

Nonetheless, assuming your code shouldn’t be needlessly caught ready for I/O operations6, parallel threads improve efficiency solely when you’ll be able to assign extra CPU cores to the issue. Whereas BGP route reflectors or BGP route servers operating as VMs might expertise elevated efficiency, most {hardware} networking units nonetheless ship with the most cost effective CPU the seller can purchase, and people CPUs nonetheless don’t have quite a lot of cores or {hardware} threads, so why hassle.

You might also like