Pipes are ubiquitous in Unix --- but how fast can they go on Linux? In this post we'll iteratively improve a simple pipe-writing benchmark from 3.5GiB/s to 65GiB/s, guided by Linux `perf`.
  • Zachariah@lemmy.worldEnglish
    131·
    3 days ago

    We will proceed as follows:

    1. A first slow version of our pipe test bench;
    2. How pipes are implemented internally, and why writing and reading from them is slow;
    3. How the vmsplice and splice syscalls let us get around some (but not all!) of the slowness;
    4. A description of Linux paging, leading up to a faster version using huge pages;
    5. The final optimization, replacing polling with busy looping;
    6. Some closing thoughts.