1 Understanding and Genera-ng High Quality Patches for Concurrency Bugs Haopeng Liu , Yuxi Chen and Shan Lu
2 What are concurrency bugs • Synchroniza-on mistakes in mul--threaded programs
3 What are concurrency bugs • Synchroniza-on mistakes in mul--threaded programs //parent thread //child thread if(…){ unlock(f->mut); return; f->mut = NULL; }
4 What are concurrency bugs • Synchroniza-on mistakes in mul--threaded programs //parent thread //child thread if(…){ unlock(f->mut); return; f->mut = NULL; }
5 What are concurrency bugs • Synchroniza-on mistakes in mul--threaded programs //parent thread //child thread if(…){ unlock(f->mut); return; f->mut = NULL; }
6 Concurrency bugs need to be fixed • Prevalence – Widely exist in mul--threaded programs • Seriousness – Cause real world disasters • Difficulty – Time-consuming to fix [LuASPLOS’08] – Patches are the most error-prone [YinFSE’11] • 2X -mes more oXen to be buggy than memory/seman-c bugs
7 Concurrency bug automated fixing • Observa-on: – Program is correct in most -ming interleavings. • Solu-on: – Only need to remove some bad -ming. • add lock-related synchroniza-on [AFix PLDI’11] [CFix OSDI’12] [Grail FSE’14] .
8 An automated fixing example //child thread //parent thread if(…){ + lock(L) unlock(f->mut); + while(…){ + lock(L); + wait(con, L); + signal(con); + } + unlock(L); + unlock (L) return; } f->mut = NULL; + lock(L); Overall: • + signal(con); – 9 new synchroniza-ons + unlock(L); – 1 new while loop Reality is even worse!
9 The manual patch //child thread //parent thread if(…){ + thread_join(…); unlock(f->mut); f->mut = NULL; return; Overall: • } – New sycn.: 1 VS. 9 – New while loop: 0 VS. 1 Simpler patch à Easier to read and maintain Can we automa7cally generate such simple patches?
10 Contribu-ons • Manual patch study – 77 real-world concurrency bugs – The limita-on of exis-ng techniques – Guidance to design new tools • HFix Bug AFix[1] Reports Bug CFix[2] Reports Bug Grail[3] Reports Bug … Reports Bug HFix [1] Guoliang Jin etc. AFix. In PLDI’11 Reports [2] Guoliang Jin etc. CFix. In OSDI’12 [3] Peng Liu etc. Grail. In FSE’14
11 Outline • Mo-va-on • Manual patch study • HFix • Evalua-on • Conclusion
12 Research Ques-ons • Q1: Do manual patches mostly use lock- related synchroniza-ons? • Q2: Do manual patches mostly disable buggy -ming by adding synchroniza-on ops?
13 Manual patch study methodology [1] App. Source Descrip7on #. Bugs Apache Web Server 13 Mozilla Browser Suite 41 Lu ASPLOS’08 MySQL Database Server 12 OpenOffice Office Suite 5 Misc Recent papers[2-4] 6 Total 77 • Study developer patches & on-line discussion [1] Threads to validity are listed in paper. [2] Guoliang Jin etc. AFix. In PLDI’11 [3] Guoliang Jin etc. CFix. In OSDI’12 [4] Wei Zhang etc. ConSeq. In ASPLOS’11
14 Q1: How many patches use lock? Use Sync. primi7ves? Yes None
15 Q1: How many patches use lock? Use Sync. primi7ves? Synchroniza7on primi7ves Yes None Lock/Con. Var Create/Join Misc. AV patches OV patches
16 Q1: How many patches use lock? Use Sync. primi7ves? Synchroniza7on primi7ves Yes None Lock/Con. Var Create/Join Misc. AV patches OV patches
17 Q1: How many patches use lock? Use Sync. primi7ves? Synchroniza7on primi7ves Yes None Lock/Con. Var Create/Join Misc. AV patches OV patches
18 Q1: How many patches use lock? Use Sync. primi7ves? Synchroniza7on primi7ves Yes None Lock/Con. Var Create/Join Misc. AV patches OV patches Thread pc Thread r P R C
19 Q1: How many patches use lock? Use Sync. primi7ves? Synchroniza7on primi7ves Yes None Lock/Con. Var Create/Join Misc. AV patches OV patches Thread pc Thread r P R C
20 Q1: How many patches use lock? Use Sync. primi7ves? Synchroniza7on primi7ves Yes None Lock/Con. Var Create/Join Misc. AV patches OV patches Thread pc Thread r Thread A Thread B P A R B C
21 Q1: How many patches use lock? Use Sync. primi7ves? Synchroniza7on primi7ves Yes None Lock/Con. Var Create/Join Misc. AV patches OV patches Thread pc Thread r Thread A Thread B P A R B C
22 Q1 implica-on • Use non-lock synchroniza-on to fix OV. Synchroniza7on primi7ves Lock/Con. Var Create/Join Misc. AV patches OV patches
23 Q1 implica-on • Use non-lock synchroniza-on to fix OV. Synchroniza7on primi7ves Lock/Con. Var Create/Join Misc. � AV patches OV patches
24 Q2: What are developers’ fix strategy? Disable Tolerate
25 Q2: What are developers’ fix strategy? Disable (Adds)[1-3] Tolerate [1] Guoliang Jin etc. AFix. In PLDI’11 [2] Guoliang Jin etc. CFix. In OSDI’12 [3] Peng Liu etc. Grail . In FSE’14
26 Q2: What are developers’ fix strategy? Disable (Adds) Disable (Data priv.)[1] Tolerate [1] Jeff Huang etc . Execu-on Priva-za-on of Scheduler-Oblivious Concurrent Programs. In OOPSLA’11
27 Q2: What are developers’ fix strategy? Disable (Adds) Disable (Data priv.) Disable (Moves) Tolerate
28 Q2: What are developers’ fix strategy? Disable (Adds) Disable (Data priv.) Disable (Moves) Disable (Bypass) Tolerate
29 Q2 implica-on • Add-Sync widely applies, but oXen not best • Need to automate : � – Move synchroniza-on Disable (Adds) Disable (Data priv.) – Bypass Disable (Moves) Disable (Bypass) – Tolerate Tolerate • Different bugs are suitable different strategies
30 Outline • Mo-va-on • Manual patch studying • HFix – HFixJoin – HFixmove • Evalua-on • Conclusion
31 HFix overview HFix HFixJoin Sta-c Code analysis transform Thd pc Thd r Thd A Thd B HFixMove P R A B Sta-c Code C analysis transform • Goal: – Fix bugs with simple patches – Do not introduce new bugs
32 HFixjoin Intui-on • Which kind of bugs can be fixed by HFixjoin? Thread A Thread B create S cre A B
33 HFixjoin Intui-on • Which kind of bugs can be fixed by HFixjoin? Thread A Thread B create S cre A B
34 HFixjoin • Step1: Suitability checking – Goal: do not introduce new bugs. • Step2: Insert join synchroniza-on – Goal: fix the bug
35 Suitability checking (sta-c analysis) • Is Thread A a joinable child thread of Thread B ? • Only one Thread B ? • Is Thread A already joined? • Will there be deadlock/severe performance slow-down? Thread A Thread B create S cre A B
36 Suitability checking (sta-c analysis) • Is Thread A a joinable child thread of Thread B ? • Only one Thread B ? • Is Thread A already joined? • Will there be deadlock/severe performance slow-down? //parent thread //child thread + thread_join(…); fputs( fp ,…); fp = NULL; wait(condV); signal(condV);
37 Insert join synchroniza-on • Insert join before opera-on B • Patch Example //child thread //parent thread if(…){ pthread_create(&tid,…) unlock(f->mut); return; ... } f->mut = NULL;
38 Insert join synchroniza-on • Insert join before opera-on B • Patch Example //child thread //parent thread if(…){ pthread_create(&tid,…) unlock(f->mut); tmp = tid; return; ... pthread_join(tmp); } f->mut = NULL;
39 HFixmove Intui-on • What kind of bugs can be fixed by HFixmove? Thread A Thread B create S cre A B (a) Move create fixes OV bug move-up A or move-down S
40 HFixmove Intui-on • What kind of bugs can be fixed by HFixmove? Thread A Thread B Thread A Thread B A * create S cre A B join S join B (a) Move create fixes OV bug (b) Move join fixes OV bug move-up A or move-down S move-up S or move-down B
41 HFixmove Intui-on • What kind of bugs can be fixed by HFixmove? Thread A Thread B Thread A Thread B Thread pc Thread r Thread pc Thread r A * p create p S cre ck A S lock join S join r r B c ck S unlock B* c (a) Move create fixes OV bug (b) Move join fixes OV bug (c) Move unlock fixes AV bug (d) Move lock fixes AV bug move-up A or move-down S move-up S or move-down B move-up c or move-down S move-up S or move-down p
42 Step1: iden-fy two opera-ons to move Thread A Thread B Thread A Thread B Thread pc Thread r Thread pc Thread r ck A * S lock A create join p S cre S join P* C* c r r B B* ck S unlock (a) Move create fixes OV bug (b) Move join fixes OV bug (c) Move unlock fixes AV bug (d) Move lock fixes AV bug move-up A or move-down S move-up S or move-down B move-up c or move-down S move-up S or move-down p
43 Step1: iden-fy two opera-ons to move Thread A Thread B Thread A Thread B Thread pc Thread r Thread pc Thread r ck A * S lock A create p S cre join S join P* C* c r r B B* ck S unlock (a) Move create fixes OV bug (b) Move join fixes OV bug (c) Move unlock fixes AV bug (d) Move lock fixes AV bug move-up A or move-down S move-up S or move-down B move-up c or move-down S move-up S or move-down p X Y Y X
44 Step 2: decide the moving des-na-on X* • Goal # of execu-ons of X == # of execu-ons of X*
45 Strawman solu-on doesn’t work # of execu-ons of X ≠ # of execu-ons of X* 1≠0 Naïve sol X * Y Y X Originally, X executes once. AXer moving, X * does not execute.
Recommend
More recommend
