Librecast Overlay Multicast - NGI Zero Core

Technology Tree

Roadmap

  • liblibrecast release 0.10

    Prepare a new release of the main Librecast C Library

    • Websocket API - completion, testing and merge into release
    • WebRTC API - completion, testing and merge into release
    • Peer Discovery - completion, testing and merge into release
    • Honeycomb / Fractal routing - completion, testing and merge into release
    • Socket/channel statistics - completion, testing and merge into release
    • Process feedback from security audit and produce patches
  • lcagent 0.1

    lcagent is a simple multicast agent which can be configured to run programs remotely and send status and output over multicast. We use this for our multicast CI system. Development started during NGI Assure. Prepare a release.

    • encryption & auth tokens
    • documentation: man page
    • documentation: website
  • Librecast Router API improvements

    Extending the liblibrecast Router API developed in NGI Assure. By adding callback hooks to the API we make it possible for new multicast routing protocols to be developed without changing the main Librecast library.

    • Router API callback hooks
    • Router API protocol examples
  • Documentation

    Write man pages for all liblibrecast library calls. At last count there were 205 API calls and only 72 man pages, leaving 133 undocumented functions.

    • man pages (liblibrecast)
    • update website documentation
  • Implement BATS Coding in LCRQ

    Implement network coding based on RFC 9426: BATched Sparse (BATS) Coding Scheme for Multi-hop Data Transport in the liblcrq library.

    Our existing RaptorQ Forward Error Correction in liblcrq helps us cope with packet loss in our UDP multicast streams. By extending this library to support network coding we can fully utilise multiple links with multicast, allowing us to maximize throughput across diverse links. This is particularly useful in combination with our peer to peer overlay multicast, as multiple links between peers will be the normal mode of operation.

    • Add BATS network encoding support to LCRQ library
  • LCRQ Optimisations

    Network encoding needs to be fast to be useful. We have previously optimised LCRQ for SSSE3 and AVX2 vector instructions on amd64. These milestones add support for newer CPU instruction sets (AVX512) and support for other common CPU architectures.

    Additionally, there are some optimisations discussed in RFC6330 which we'd like to revisit. When we tried these in 2022, the results were not faster, but we've since read results from others that suggest this is worth another look.

    For CPU architectures that we do not explicitly optimize for, we can add limited support by leveraging SIMDE (SIMD Everywhere), a library which provides polyfills for vector instructions on other CPUs.

    • Optimize for avx512
    • Optimize for arm64 NEON
    • Optimize for riscv64
    • Optimize for amdgpu
    • Re-visit RFC6330 optimisations, implement prebuilt plan
    • Multithreaded implementation
    • SIMD support for other non-amd64 arches
  • lcsync (multicast file syncing) improvements

    lcsync bugfixes, improvements and prepare new release

    • lcsync bugfixes, improvements and prepare new release