Parallelizing TTree::Draw functionality with PROOF Stefano Marinaci - PowerPoint PPT Presentation

Parallelizing TTree::Draw functionality with PROOF Stefano Marinaci (Supervisor: Gerardo Ganis) CERN PH-SFT

Introduction Today's desktop/laptop have many cores, lots of memory and performant drives (e.g. SSD): can we exploit all that to improve daily ROOT-based analysis tasks? Two approaches to exploit multi-cores: Threads: The future, but current ROOT non thread-safe Multi-process: PROOF, or PROOF-Lite for desktop/laptop Better integration of PROOF-Lite for local TTree processing is one of the items on the PROOF plan of work Parallelizing TTree::Draw functionality with PROOF 2

The Draw functionality 'TTree::Draw' used for many actions during daily analysis work: Variable plotting Histogram or ad-hoc output object creation Selection criteria design and optimization … Implemented using the TSelector framework with specialized selector TSelectorDraw calls process TTree::Draw TTreePlayer::DrawSelect TSelectorDraw canvas histogram Parallelizing TTree::Draw functionality with PROOF 3

TSelector framework Framework provided by ROOT to process trees Three main function blocks Begin(), called once before processing to initialize the job Process(), called for each tree entry, applying the analysis algorithm Terminate(), called once at the end to finalize the job PROOF works only with a TSelector framework, parallelizing the calls to Process() Parallelizing TTree::Draw functionality with PROOF 4

Draw in PROOF PROOF already provides a sub-sample of 'Draw' functionality via the TProof::DrawSelect and a set of specialized selectors, TProofDraw... TProofDraw selectors developed by a technical student (M. Biskup): Meant to replace TSelectorDraw but the work was never finished Missing macro processing, special keyword $ interpretation, … TSelectorDraw continued to evolve For functionality completeness and maintenance reasons keep only one selector TSelectorDraw has all the functionality implemented Make sense to use it in PROOF Parallelizing TTree::Draw functionality with PROOF 5

TSelectorDraw and PROOF However, TSelectorDraw is not usable in PROOF as it is because makes assumptions valid only locally In particular Availability of the TTree at constructor level (before start processing); Possibility to avoid using the output list for the output transfer. We cannot change the way PROOF works: But we can make the local flow similar to the PROOF one. Parallelizing TTree::Draw functionality with PROOF 6

Draw pseudo-code: current local vs PROOF CURRENT LOCAL PROOF f = TFile::Open(“filename”) dataset = new TChain(“treename”) USER tree = f->Get(“treename”) dataset->Add(“filename”) LEVEL tree->Draw(“var”, “cut”) TProof::Open(“lite://”) // start PROOF-Lite proof->DrawSelect(dataset, “var”, “cut”) treeplayer->DrawSelect(“var”, “cut”) proofplayer->DrawSelect(dataset, “var”, “cut”) input ->Add(“var”) input ->Add(“var”) BEHIND input ->Add(“cut”) input ->Add(“cut”) THE sel = new TSelectorDraw() SCENE sel->Begin (tree) Process(“TProofDraw...”) Process(sel) sel->Init (tree) //on first file opening out= sel->fObject <output from TSelector::fOutput > out->Draw() (canvas draw by the feedback mechanism) Formula compilation Parallelizing TTree::Draw functionality with PROOF 7

Changes to TSelectorDraw Main changes: Move variable formula initialization from Begin to Init Make the output object(s) available in the output list Details to pay attention to: Object ownership in list Conflicts between gDirectory and selector output list Make sure that proper inizialization is always run TSelectorDraw ctor not re-run in case of multiple Draw calls Parallelizing TTree::Draw functionality with PROOF 8

Status Branch in github repository with the changes in TSelectorDraw: Roottest runs through successfully PROOF interface: standalone macro implementing Draw in PROOF using TSelectorDraw exists Works fine for basic functionality: Plotting variables, applying standard math functions Parallelizing TTree::Draw functionality with PROOF 9

What next? Not working functionality requires dedicated transfer in/out PROOF (e.g. the user function or macro to be executed on the leaves): Not a technical showstopper, but needs to be implemented Integration of the PROOF interface in TTreePlayer::DrawSelect for transparent availability in TTree::Draw Requires a bit of brainstorming Parallelizing TTree::Draw functionality with PROOF 10

Personal impressions Homely place; Group work exciting; Reliable and willing people; Every day I learnt something new; Project initially incomprehensible but as time went by and with the help of my supervisor everything became clearer; Definitely an experience that I would do it again! Parallelizing TTree::Draw functionality with PROOF 11