1
Outline What is bufferbloat? Bufferbloat mitigation in Linux Fixing bloat in the WiFi stack Airtime fairness and policy Status of mainline drivers Future / ongoing work - Toke Høiland-Jørgensen <toke@toke.dk> - Toke Høiland-Jørgensen <toke@toke.dk> 2
Slow internet sucks - Toke Høiland-Jørgensen <toke@toke.dk> - Toke Høiland-Jørgensen <toke@toke.dk> 3
How does bufferbloat lead to slowness? - Toke Høiland-Jørgensen <toke@toke.dk> - Toke Høiland-Jørgensen <toke@toke.dk> 4
- Toke Høiland-Jørgensen <toke@toke.dk> - Toke Høiland-Jørgensen <toke@toke.dk> 5
- Toke Høiland-Jørgensen <toke@toke.dk> - Toke Høiland-Jørgensen <toke@toke.dk> 6
- Toke Høiland-Jørgensen <toke@toke.dk> - Toke Høiland-Jørgensen <toke@toke.dk> 7
- Toke Høiland-Jørgensen <toke@toke.dk> - Toke Høiland-Jørgensen <toke@toke.dk> 8
This is bloat 800 600 Latency (ms) 400 200 0 0 10 20 30 40 50 60 70 Time (s) - Toke Høiland-Jørgensen <toke@toke.dk> - Toke Høiland-Jørgensen <toke@toke.dk> 9
Fixing bufferbloat - AQM Inlet valve Float Tank Filler tube Flus h valve Supply line Bowl Figure 3: A well designed servo system - Toke Høiland-Jørgensen <toke@toke.dk> - Toke Høiland-Jørgensen <toke@toke.dk> 10
AQM - CoDel Timestamp k pkts now - timestamp > 5ms? CoDel - Toke Høiland-Jørgensen <toke@toke.dk> - Toke Høiland-Jørgensen <toke@toke.dk> 11
Fixing bufferbloat - FQ - Toke Høiland-Jørgensen <toke@toke.dk> - Toke Høiland-Jørgensen <toke@toke.dk> 12
FQ-CoDel sparse flow optimisation Credits exhausted Empty Empty Old Arrival New Empty or credits exhausted - Toke Høiland-Jørgensen <toke@toke.dk> - Toke Høiland-Jørgensen <toke@toke.dk> 13
Results at the qdisc layer (wired links) 9.6 9.5 Mean TCP goodput (Mbit/s) 9.4 fq_nocodel 9.3 sfq pfifo_fast fq_codel 9.2 pie codel 9.1 ared 9.0 8.9 8.8 100 80 60 40 20 0 Mean induced latency (ms) - Toke Høiland-Jørgensen <toke@toke.dk> - Toke Høiland-Jørgensen <toke@toke.dk> 14
Pieces of CAKE The sch_cake qdisc adds: cake cake_dst cake_src cake_triple fq_codel 2.5 Traffic shaping 2.0 Host-based fairness queueing Mbits/s 1.5 DiffServ handling 1.0 TCP ACK filtering 0.5 0.0 A->A A->B A->C A->C B->C B->D A->A A->B A->C A->C B->C B->D A->A A->B A->C A->C B->C B->D A->A A->B A->C A->C B->C B->D A->A A->B A->C A->C B->C B->D - Toke Høiland-Jørgensen <toke@toke.dk> - Toke Høiland-Jørgensen <toke@toke.dk> 15
Important feature: Byte Queue Limits (BQL) Queueing needs to happen where it can be managed There’s another queue in the hardware For Ethernet, we have BQL Enabled in most drivers now Check your driver, it’s a pretty small patch Other technologies (DSL, etc.) need this! - Toke Høiland-Jørgensen <toke@toke.dk> - Toke Høiland-Jørgensen <toke@toke.dk> 16
What about WiFi? - Toke Høiland-Jørgensen <toke@toke.dk> - Toke Høiland-Jørgensen <toke@toke.dk> 17
The qdisc approach doesn’t work for WiFi pie 60 pfifo_fast codel sfq 50 fq_codel Mean TCP goodput (Mbit/s) ared 40 fq_nocodel 30 20 10 fq_codel sfq ared fq_nocodel pie pfifo_fast codel 0 350 300 250 200 150 100 50 Mean induced latency (ms) - Toke Høiland-Jørgensen <toke@toke.dk> - Toke Høiland-Jørgensen <toke@toke.dk> 18
Fixing queueing in the WiFi stack Qdisc layer (bypassed) Per HW queue 1000 * Qdisc layer (x4) Assign TID FIFO* TID TID MAC layer Split flows Split flows *Can be replaced with an arbitrary configuration (Global limit) (Global limit) 8192 8192 MAC layer Assign TID FQ- FQ- CoDel CoDel Retries Retries TID TID 123 123 retry_q retry_q ath9k driver buf_q retry_q buf_q retry_q Prio Prio ath9k driver Prio Prio HW queue RR RR (x4) 2 aggr 2 aggr FIFO FIFO To hardware To hardware - Toke Høiland-Jørgensen <toke@toke.dk> - Toke Høiland-Jørgensen <toke@toke.dk> 19
Queueing latency impact 1.0 Cumulative probability 0.8 0.6 0.4 0.2 0.0 10 1 10 2 10 3 Latency (ms) With our solution Without our solution - Toke Høiland-Jørgensen <toke@toke.dk> - Toke Høiland-Jørgensen <toke@toke.dk> 20
Throughput impact Station 1 Station 2 Station 3 Average 40 30 Mbits/s 20 10 0 FIFO FQ-CoDel FQ-MAC Airtime FIFO FQ-CoDel FQ-MAC Airtime FIFO FQ-CoDel FQ-MAC Airtime FIFO FQ-CoDel FQ-MAC Airtime - Toke Høiland-Jørgensen <toke@toke.dk> - Toke Høiland-Jørgensen <toke@toke.dk> 21
Airtime: fairness and policy How can we control airtime usage of WiFi devices? - Toke Høiland-Jørgensen <toke@toke.dk> - Toke Høiland-Jørgensen <toke@toke.dk> 22
802.11 Performance Anomaly Effective transmission time and rate (for station ): T ( i ) R ( i ) i ∈ I 1 ⎧ with fairness ⎪ | I | ⎪ T _ data ( i ) T ( i ) = ⎨ otherwise ∑ _ j ∈ IT _ data ( j ) ⎪ ⎪ ⎩ R ( i ) = T ( i ) R _0( i ) Where is the effective rate of a station transmitting L _ i R _0( i ) = T _ data ( i )+ T _ oh without collisions. - Toke Høiland-Jørgensen <toke@toke.dk> - Toke Høiland-Jørgensen <toke@toke.dk> 23
Airtime fairness scheduler Idea: Schedule stations to equalise airtime usage We already have per-station queueing The driver schedules stations anyway So we can simply account airtime using DRR scheduling First prototype: Change the ath9k driver - Toke Høiland-Jørgensen <toke@toke.dk> - Toke Høiland-Jørgensen <toke@toke.dk> 24
Airtime fairness - achieved fairness (ath9k) FIFO FQ-CoDel FQ-MAC Airtime fair FQ 1.0 0.8 Airtime share 0.6 0.4 0.2 0.0 Fast 1 Fast 2 Slow Fast 1 Fast 2 Slow Fast 1 Fast 2 Slow Fast 1 Fast 2 Slow - Toke Høiland-Jørgensen <toke@toke.dk> - Toke Høiland-Jørgensen <toke@toke.dk> 25
Airtime fairness - throughput benefit (ath9k) Station 1 Station 2 Station 3 Average 40 30 Mbits/s 20 10 0 FIFO FQ-CoDel FQ-MAC Airtime FIFO FQ-CoDel FQ-MAC Airtime FIFO FQ-CoDel FQ-MAC Airtime FIFO FQ-CoDel FQ-MAC Airtime - Toke Høiland-Jørgensen <toke@toke.dk> - Toke Høiland-Jørgensen <toke@toke.dk> 26
Beyond fairness: “Polifi” Enable airtime policy enforcement, Userspace supporting: Policy User Station daemon configuration state tracker Assoc / disassoc notifications Prioritising single devices Set station weights Station queue state Use case, e.g.: “My TV needs more Kernel mac80211 subsystem airtime” Station Station Station Weight Weight Weight Balancing device groups Deficits Deficits Deficits Use case, e.g.: 5G network slicing Limiting groups to a maximum capacity Weighted DRR Airtime usage scheduler share Device driver Use case, e.g.: Guest network WiFi hardware - Toke Høiland-Jørgensen <toke@toke.dk> - Toke Høiland-Jørgensen <toke@toke.dk> 27
Policy operating modes Weights set by userspace daemon (hostapd), supporting three modes: Static mode: Specify MAC priority in config Dynamic mode: Specify weight per group (BSS) Limit mode: Like dynamic, but only limit some groups - Toke Høiland-Jørgensen <toke@toke.dk> - Toke Høiland-Jørgensen <toke@toke.dk> 28
Policy enforcement effect 1.00 0.75 Airtime share 0.50 0.25 0.00 0 10 20 30 40 Time (s) 1.00 0.75 Airtime share 0.50 0.25 0.00 0 10 20 30 40 Time (s) - Toke Høiland-Jørgensen <toke@toke.dk> - Toke Høiland-Jørgensen <toke@toke.dk> 29
Status in the mainline kernel Everything presented here is upstream as of 5.1! - Toke Høiland-Jørgensen <toke@toke.dk> - Toke Høiland-Jørgensen <toke@toke.dk> 30
Upstream feature support - kernel versions Feature Version BQL 3.3 sch_fq_codel 3.5 sch_cake 4.19 WiFi queueing 4.10 Airtime fairness (ath9k) 4.11 Airtime fairness + policy 5.1 - Toke Høiland-Jørgensen <toke@toke.dk> - Toke Høiland-Jørgensen <toke@toke.dk> 31
Recommend
More recommend
