Building an Open-Source based audio streaming platform Maxime - PowerPoint PPT Presentation

Building an Open-Source based audio streaming platform Maxime Bugeia 2 February 2020

We stream audio! 2

French public broadcasting service 6 national channels 47 local channels 23 music webradios 2 Million listeners per day 200k simultaneous listeners 3

Before Black Box Icecast 3rd party 4

After Icecast HLS Our own cloud and open source based infrastructure 5

Audio streaming HLS ICECAST Segmented audio/video files Streaming server Sliding Playlist (m3u8) Master/Relay Architecture Adaptive bitrate Single bitrate Better mobile experience No cache Cacheable content Long persistent TCP Just “static” files connections 6

Audio to listeners Getting audio to the cloud 7

Audio to listeners Getting Icecast HLS audio to the cloud Producing stream formats 8

Audio to listeners Deliver content to listeners CDN Getting Icecast HLS audio to the cloud Producing stream formats 9

Audio to listeners Deliver content to listeners CDN Getting Icecast HLS audio to the cloud Producing stream formats Operate the platform 10

Getting audio to the cloud! 11

Transport: multicast problem NATIONAL CHANNELS LOCAL CHANNELS DIRECT CONNECT WAN MULTICAST WEBRADIOS 12

Transport: multicast problem NATIONAL CHANNELS ? LOCAL CHANNELS DIRECT CONNECT WAN MULTICAST WEBRADIOS 13

Transport: multicast problem NATIONAL CHANNELS ? LOCAL CHANNELS DIRECT CONNECT WAN MULTICAST WEBRADIOS No multicast in AWS! 14

Transport: SRT to the rescue 15

Transport: to the cloud! NATIONAL CHANNELS LOCAL CHANNELS DIRECT CONNECT WAN MULTICAST WEBRADIOS 16

Transport: resiliency b a c k u p s a t m a i n SRT CALLER 1 SRT LISTENER 1 backup m a i n SRT CALLER 2 SRT LISTENER 2 backup b a c k u p s a t 5 inputs per channel 17

Audio to cloud SAT BACKUP DIRECT CONNECT 18

Producing stream formats! 19

One program to rule them all Liquidsoap Open Source Audio & Video Streaming Language Collaboration with Radio France Receive Transcode Control Output AAC SRT listener Switch between sources Output MP3 Fallback logic Produce HLS Expose metrics Produce Icecast 20

Producing the streams SAT BACKUP Icecast Transcoder Liquidsoap HLS DIRECT CONNECT 21

Transcoder Architecture Icecast stream Icecast Icecast stream Liquidsoap Master Nginx HLS files HLS stream 22

Radio as Code Define inputs 23

Radio as Code Live: what we want to play 24

Radio as Code Fallback logic 25

Radio as Code Fallback logic Never fail! 26

Radio as Code Output HLS Source that never fails 27

Radio as Code Output HLS 28

Radio as Code Output HLS 29

Radio as Code Output Icecast Source that never fails 30

Transcoders Architecture Icecast stream Icecast Icecast stream Liquidsoap Master HLS files live.m3u8 Nginx HLS stream live_hifi.m3u8 segment1.ts segmentxxx.ts 31

Scaling to 200k listeners! 32

Scale: Icecast Icecast Master Icecast Relay Icecast Relay 1 HTTP :80 HTTPS :443 Transcoder 1 https://icecast.radiofrance.fr/fip-hifi.aac Icecast Relay 2 Transcoder 2 AWS NLB Icecast Relay X 33

Scale: Icecast Icecast Master Icecast Relay Icecast Relay 1 HTTP :80 HTTPS :443 Transcoder 1 https://icecast.radiofrance.fr/fip-hifi.aac Icecast Relay 2 No cache Transcoder 2 No autoscaling AWS NLB Icecast Relay automatic fallback Icecast Relay X 34

Scale: HLS HLS base stream Cache layer CDN Varnish 1 Playlists Transcoder 1 https://stream.radiofrance.fr/fip/fip.m3u8 https://stream.radiofrance.fr/fip/fip_hifi.m3u8 fip_aac_hifi_2_301960_1579533023.ts Varnish 2 Segments Transcoder 2 AWS NLB Varnish X 35

Scale: HLS HLS base stream Cache layer CDN Varnish 1 Playlists Transcoder 1 https://stream.radiofrance.fr/fip/fip.m3u8 https://stream.radiofrance.fr/fip/fip_hifi.m3u8 fip_aac_hifi_2_301960_1579533023.ts Varnish 2 Segments Transcoder 2 AWS NLB Autoscaling at cache layer CDN level fallback possible Varnish X 36

Scale: HLS HLS base stream Cache layer CDN Varnish 1 Playlists Transcoder 1 https://stream.radiofrance.fr/fip/fip.m3u8 https://stream.radiofrance.fr/fip/fip_hifi.m3u8 fip_aac_hifi_2_301960_1579533023.ts Varnish 2 Segments Transcoder 2 AWS NLB Just like web! Varnish X 37

Operate 38

Transcoder Architecture Icecast stream Icecast Icecast stream Liquidsoap Master Nginx HLS stream HLS files 39

Transcoder Architecture Icecast stream Icecast Icecast stream Liquidsoap Master Liquidsoap xxxx Liquidsoap xxxx Liquidsoap xxxx Liquidsoap xxxx Liquidsoap xxxx Liquidsoap xxxx Nginx HLS stream HLS files One Liquidsoap per channel 40

Automate Declarative everything Generating: ● SRT callers ● Liquidsoap scripts ● Icecast master & relay conf 41

Monitor Export metrics Visualize Collect Node Exporter Liquidsoap Icecast exporter 42

Monitor 43

Monitor: real time listeners HLS HLS access logs Custom logs Ingester HTTP POST Json Icecast Icecast prometheus Exporter 44

Monitor 45

Learn more https://github.com/mbugeia/srt2hls Receive SRT stream Produce HLS Prometheus Metrics HTTP API Command and fallback logic https://github.com/Haivision/srt https://www.liquidsoap.info/ 46

Learn more Special thanks to the Liquidsoap team! https://www.liquidsoap.info/ 47

Questions? For internal use only / prepared with love

For internal use only / prepared with love

HLS / DASH Compatibility 50

mp3 / aac Compatibility 51

ogg / flac Compatibility 52

Real HLS diffusion Cluster Kubernetes Segment .ts Storekeeper Bus RabbitMQ N Transcoders Segment L Playlists Forwarder B .m3u8 Segments HLS HTTP POST PGM PostgreSQL Player 53