Consorzio COMETA FESR Visualization Element: towards the - PowerPoint PPT Presentation

Consorzio COMETA FESR Visualization Element: towards the definition of a new Grid service Giuseppe ¡ANDRONICO ¡ (1) , ¡Roberto ¡BARBERA (1)(2) , ¡Andrea ¡FORNAIA ¡ (1) , ¡ Marcello ¡IACONO ¡MANNO ¡ (3) ¡and ¡Giuseppe ¡LA ¡ROCCA (1) ¡ ¡ ¡ (1) ¡INFN ¡– ¡Na*onal ¡Ins*tute ¡of ¡Nuclear ¡Physics, ¡Division ¡of ¡Catania, ¡Italy ¡ ¡ (2) ¡Department ¡of ¡Physics ¡and ¡Astronomy ¡of ¡the ¡University ¡of ¡Catania, ¡Italy ¡ ¡ (3) ¡Consorzio ¡COMETA, ¡Catania, ¡Italy ¡ ¡ ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan www.consorzio-cometa.it

Outline Outline • The Consorzio COMETA and the Sicilian Grid infrastructure; • Introduction to the Visualization Element service – Motivations; – Design and Implementation; – Architecture; – The Stereoscopy; – Examples; § Blender; § VISIVO; – Future plans. • Summary and Conclusions; • References. ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 2

The Italian e-Infrastructure (interoperable with common communication protocols) The Network FESR The Grid ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 3

The COMETA Consortium (http://www.consorzio-cometa.it) ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 4

Objectives of an e-Infrastructure in Sicily • Create a Virtual Laboratory in Sicily, both for scientific and industrial applications, built on top of a grid infrastructure • Connect the Sicilian e-Infrastructure to those already existing in Italy, in Europe, and in the rest of the world improving the scientific collaboration and increasing the “competitiveness” of e-Science and e-Industry “made in Sicily” • Disseminate the “grid paradigm” through the organization of dedicated events and training courses • Trigger/foster the creation of spin-offs in the ICT area in order to reduce the “brain drain” of brilliant young people to other parts of Italy and beyond ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 5

The COMETA Infrastructure Catania ~2500 ¡CPUs ¡ ~250 ¡TB ¡ ~15 ¡M ¡€ ¡in ¡3 ¡years ¡ >350 ¡people ¡involved! ¡ ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 6

Computing and storage resources available 1. ¡ ~2000 ¡cores ¡AMD ¡Opteron ¡2218 ¡ 2. ¡2 ¡GB ¡of ¡RAM ¡per ¡core ¡ ¡ 3. ¡LSF ¡as ¡LRMS ¡everywhere ¡ 4. ¡Infiniband-­‑4X ¡at ¡all ¡sites ¡ 1. ¡ 200+ ¡TB ¡of ¡raw ¡disk ¡storage ¡ ¡ 2. Distributed/parallel ¡GPFS ¡filesystem ¡ ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 7

Commercial software available (http://www.pi2s2.it/tecn/index.php?option=com_content&task=view&id=27&Itemid=58) ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 8

Motivations • Several Grid applications (e.g.: FLUENT) running in the COMETA Infrastructure produce media content (e.g.: images) that can be collected and encoded as a unique video to be visualized also when the job that produces the images is still running. – This feature maybe helpful for both demonstration purposes or to control job evolution. • Visualiza*on ¡Element is a post-processing tool allowing to show, starting from a list of images sequence, the video output produced by a producer application. 64 cores for 14 CPU days Environment protection Fig.1 ¡-­‑ ¡Some ¡frames ¡produced ¡by ¡FLUENT ¡and ¡streamed ¡by ¡the ¡VisualElement ¡service. ¡ ¡ ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 9

Deploying the VisualElement on Grid • VisualElement uses mencoder , VLC ¡VideoLAN ¡Client, ¡Xva and FFMPEG software packages to encode the media content and produce a video that will be send in streaming. • The needed packages for Visualization Element can be downloaded from repos. Top BDII • Published a new software tag, VO-cometa-VLC-0.9.9a, for the service. • A JDL file is created on-the-fly using a Grid portal to call the service with the corresponding inputs. • A first prototype of the Visualization Element on-line streaming tool has been deployed for the COMETA Grid infrastructure on board of the GENIUS Portal powered by EnginFrame from NICE srl ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 10

Integration of VisualElement in COMETA • Two different implementations of the service are available as shown in fig. 1 and fig. 2 Based ¡on ¡gLite ¡3.2 ¡CREAM ¡CE ¡ Based ¡on ¡LCG ¡Grid ¡CE ¡ Fig. ¡2 ¡– ¡Genera*on ¡of ¡the ¡video ¡output ¡using ¡a ¡DAG ¡ Fig. ¡1 ¡– ¡Genera*on ¡of ¡the ¡video ¡streaming ¡ ¡with ¡a ¡single ¡job ¡ ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 11

Commercial & industrial HPC applications on Grid: CFD simulations with Fluent 64 ¡cores ¡for ¡14 ¡CPU ¡days ¡ Environment ¡protec*on ¡ (“Le ¡Marmore” ¡falls) ¡ 48 ¡cores ¡for ¡7 ¡CPU ¡days ¡ Engine ¡design ¡ ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 12 12

What’s next ? The Stereoscopy • It is the ability of creating, or enhancing, the illusion of depth in a flat image: • by presenting two offset images, respectively for left and right eye, combined together in a single one, using some optical technique; • This is exactly what our brain does with the “images” taken from the eyes. • To implement the Stereoscopy, we can use three different methods: – Using two separated sources: – the user wears eyeglasses to combine the images taken from them; – Using only one source: – the user wears eyeglasses to split the image taken from it; – Using a self-stereoscopic source: – that splits the images directing the light into the viewer's eye, with no need of eyeglasses. ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 13

Stereoscopic streams in practice • We had to produce for each eye, two separated streams of images • observing the simulation from two different point of view; • Then, we have to combine them in a single stream • using appropriated video drivers, in according to the chosen stereoscopic technique; e.g., the anaglyph one . ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 14

First use case • The service has been tested for the first time encoding a BLENDER application. • – The – The application The reconstructed video will be send in streaming via UDP (default port 1234) to the client’s IP using VLC. , generates for each frame 2 different images; ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 15

Second use case • VIsIVO Server for COMETA Grid can create a 3D view from data tables; • It runs on every Linux based system: – The main characteristic is that VisIVO Server gives support for huge data; – No limits (in principle) are fixed to visualize your data. Some limits exist up to now, but the next version will solve these problems; – The FITS and HDF5 data format will be included among supported formats; – Output data from Gadget code will be soon supported; – The project is completed since October 2008. ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 16

Any questions, comments or remarks are very welcome. Please contact: giuseppe. andronico@ct.infn.it ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 17

References & Links • G. Andronico, R. Barbera, M. Iacono Manno and G. La Rocca. VisualGrid: On-Line Video Streaming for Application Control and Demonstration. Proceedings of the Final Workshop of Grid Projects “PON Ricerca 2000-2006, Avviso 1575”, 2009 ISGC 2011 & OGF 31 19-21 March 2011, Academia Sinica, Taipei, Taiwan 18