Abstract Deployment Model Problem: • Difficulties and lack of flexibility in deployment • Avoid scripting for: configuration, getting nodes, connecting, etc. A key principle: Virtual Node (VN) + XML deployment file • Abstract Away from source code: • Machines • Creation Protocols • Lookup and Registry Protocols Protocols and infrastructures: • Globus, ssh, rsh, LSF, PBS, … Web Services, WSRF, ... Denis Caromel 44
Descriptors: based on Virtual Nodes Virtual Node (VN): • Identified as a string name • Used in program source • Configured (mapped) in an XML descriptor file --> Nodes Operations specified in descriptors: • Mapping of VN to JVMs (leads to Node in a JVM on Host) • Register or Lookup VNs • Create or Acquire JVMs Program Source Descriptor (RunTime) |----------------------------------| |-------------------------------------------| Activities (AO) --> VN VN --> JVMs --> Hosts Runtime structured entities: 1 VN --> n Nodes in n JVMs Denis Caromel 45
Descriptors: Mapping Virtual Nodes Component Dependencies: Provides: … Uses: ... VirtualNodes: Dispatcher <RegisterIn RMIregistry, Globus, Grid Service, … > RendererSet Example of Mapping: an XML file Dispatcher --> DispatcherJVM descriptor: RendererSet --> JVMset JVMs: DispatcherJVM = Current // (the current JVM) JVMset=//ClusterSophia.inria.fr/ <Protocol GlobusGram … 10 > ... Denis Caromel 46
Mapping Virtual Nodes: example (1) <processDefinition id="linuxJVM"> <jvmProcess class="org.objectweb.proactive.core.process.JVMNodeProcess"/> </processDefinition> JVM on the current Host Infrastructure <processDefinition id=”sshProcess"> informations <sshProcess class="org.objectweb.proactive.core.process.ssh.SSHJVMProcess" hostname="sea.inria.fr"> <processReference refid="linuxJVM"/> </sshProcess> </processDefinition> JVM started using SSH Denis Caromel 47
Mapping Virtual Nodes: example (2) <processDefinition id=" clusterProcess "> <bsubProcess class="org.objectweb.proactive.core.process.lsf.LSFBSubProcess" hostname=”cluster.inria.fr"> <processReference refid=”singleJVM"/> <bsubOption> �������������� <processor>12</processor> ������������ </bsubOption> </bsubProcess> </processDefinition> Definition of LSF deployment, … Globus Denis Caromel 48
XML Deployment (Not in source) VNa VNb VNc = VN(a,b) C C A B A B Separate or Co-allocation Denis Caromel 49
IC2D: Interactive Control and Debugging of Distribution With any ProActive application Features: Graphical and Textual visualization Monitoring and Control Denis Caromel 50
Monitoring of RMI, Globus, Jini, LSF cluster Nice -- Baltimore ProActive IC2D: Width of links proportional to the number of com- munications Denis Caromel 51
Job Monitoring Denis Caromel 52
Denis Caromel 53
C3D: distributed-//-collaborative Collaborative 3D, Rendering in //, Application mobility Denis Caromel 54
Jem3D Denis Caromel 55
JEM 3D : Java 3D Electromagnetism together with Said El Kasmi, Stéphane Lanteri (caiman) Maxwell 3D equation solver, Finite Volume Method (FVM) Pre-existing Fortran MPI version: EM3D (CAIMAN team @ INRIA) Up to 294 machines at the same time (Intranet and cluster) Large data sets: 150x150x150 (100 million facets) temps d'exécution de la boucle principale (sur cluster) 900 800 700 taille du maillage temps (secondes) 600 21*21*21 500 31*31*31 400 43*43*43 55*55*55 300 81*81*81 200 97*97*97 113*113*113 100 121*121*121 0 0 10 20 30 40 50 60 70 nombre de processeurs Denis Caromel 56
Recent Benchmarks Seq. Java/Fortran: 2 Comparison: Jem3D over - ProActive/RMI Sun - ProActive/RMI Ibis Em3D in - Fortran/MPI On 16 machines: Fortran: 13.8 ProActive/Ibis: 12 ProActive/RMI: 8.8 Grid experiment on 5 clusters (DAS 2): Speed up of 100 on 150 machines Denis Caromel 57
Monte Carlo Simulations, Non-Linear Physics, INLN Denis Caromel 58
Electric Network Planning, E. Zimeo et al., Benevento (Naples), Italy On-line Power Systems Security Analysis (OPSSA) Denis Caromel 59
Electric Network Planning, E. Zimeo et al., Benevento (Naples), Italy On-line Power Systems Security Analysis (OPSSA) Denis Caromel 60
Electric Network Planning, E. Zimeo et al., Benevento (Naples), Italy On-line Power Systems Security Analysis (OPSSA) Denis Caromel 61
Electric Network Planning, E. Zimeo et al., Benevento (Naples), Italy On-line Power Systems Security Analysis (OPSSA) Denis Caromel 62
Electric Network Planning, E. Zimeo et al., Benevento (Naples), Italy On-line Power Systems Security Analysis (OPSSA) Denis Caromel 63
P2P N Queens Denis Caromel 64
Real Denis Caromel 65
Architectures: Server to Peer-To-Peer (P2P) Internet Apache Servlets EJB Databases SOA: Service Oriented Architectures Denis Caromel 66
Pure P2P: Definition Only PEERs, no above everything, top level, server(s) Every peer is, somehow, also a server No master … No slave ! System get organized dynamically, without static configuration Coherent, desired behavior, dynamically emerges Denis Caromel 67
P2P Examples (1) Denis Caromel 68
P2P can be difficult: need to be fault-tolerant, self healing Denis Caromel 69
Not a P2P system Denis Caromel 70
Neither a P2P system Denis Caromel 71
P2P Examples (2) Denis Caromel 72
P2P Examples (2bis) Denis Caromel 73
A P2P system at work Credit: from the movie Atlantis, Luc Besson Denis Caromel 74
Conclusion • GRIDs: - Scientific - Enterprise - Internet Strong convergence in process (infrastructure): WS, WSRF • Challenge: adequate and precise programming+composing model • Asynchronous distributed objects, Mobility, Components • High-level of abstraction, still Open and flexible • Modern languages: Java, or others not Fortran, MPI, C++, … not the answer Perspectives: Content • Real P2P • Interactive, Graphical, Deployment and Control: http://ProActive.ObjectWeb.org Denis Caromel 75
Interactive Composition in IC2D Instead of Composition View XML ADL Content <processDefinition id=" <jvmProcess class="org.objectweb </processDefinition> <processDefinition id=” <sshProcess class="org.objectweb hostname <processReferen </sshProcess> </processDefinition> Denis Caromel 76
Merging Component + Activity / JVM Views: Component Monitoring Dynamic View Content Denis Caromel 77
Merging Component + Activity / JVM Views: Component Monitoring Dynamic View Content Denis Caromel 78
Conclusion (2) Infrastructure and Deployment • Virtual Nodes and XML • Protocols: ssh, Globus, LSF, PBS, … • Transport: RMI, Ibis (Amsterdam) • Web Services: XML, WSDL, SOAP (ongoing) • OSGi (future work) Currently: a single application on 300 machines at once Goal: towards a few 1000s ! Available in LGPL with ProActive http://ProActive.ObjectWeb.org in The freedom your applications deserve! Denis Caromel 79
Next ProActive events EuroPar 2004, Pisa, Italy, Monday 30th August, Full Day Tutorial • ProActive Tutorial, and • Hands-on session User Group & Grid Interoperability, Nice, ETSI, Oct. 18-20 • Detailed presentation of the platform • User presentations and feedbacks • Contest & PLUGTESTS : N queen challenge on GRID’5000, … Denis Caromel 80
Denis Caromel 81
Denis Caromel 82
Denis Caromel 83
Exemple Enjeux: Passer à l’échelle 50 Machines, 1,5 Année de Calcul 5000 Machines, Efficacité de 50 % ==> 10 Jours Applications: • Rechercher un vaccin • Prévoir la progression d’un incendie • Prévoir en temps réel les dangers d’une inondation ... • etc. Denis Caromel 84
Ubiquitaire: Quelques chiffres PCs à l�’INRIA Sophia : ~ 1500 PACA : 1,3 Millions France : 25 Millions Europe : 108 Millions USA : 400 Millions Monde : 1 Milliard en 2002 (25 ans) Prévision: 2 Milliards en 2008 France : • 36 Millions de téléphones portables • 2.2 Millions dordinateurs portables • 630 Mille PDA (sources: ITU, Gartner Dataquest, IDC, 02-03, ) Denis Caromel 85
Some code ! // CREATE THE COMPONENTS ComponentIdentity speakers = ProActive.newActiveComponent(speakers_parameters); // OR USE THE COMPONENTS LOADER // ComponentIdentity speakers = ComponentsLoader.getComponent(«�speakers�»); // BIND THE COMPONENTS ((BindingController)cd_player .getFcInterface(BindingController.BINDING_CONTROLLER)) .bindFc(« output�», speakers.getFcInterface(«�input�»); // START THE LIFE CYCLE OF THE COMPONENTS (ENABLE THE COMPONENTS) ((LifeCycleController)speakers .getFcInterface(LifeCycleController.LIFECYCLE_CONTROLLER)) .startFc(); // INVOKE SOME ACTIONS ON FUNCTIONAL INTERFACES ((Input)speakers .getFcInterface(«input»)) .newMusic (music.mp3); ((PlayerFacade)cd_player .getFcInterface(«�control�»)) .play(); Denis Caromel 86
4. Composition, Deployment and Runtime Denis Caromel 87
VirtualNodes // names of the virtual nodes VirtualNode name= «�Node-facade�» VirtualNode name =«�Node-speaker�» - cyclic Deployment // what is behind the names of the nodes mapping // correspondance between the names of the VNs and the JVMs Node-facade --> JVM1 Node-speaker --> {JVM2, JVM3, JVM4} // 1 VN can be mapped onto a set of JVMs JVMs JVM1 created by process «�linuxJVM�» JVM2 created by process «�rsh-computer1�» JVM3 created by process «�rsh-computer2�» JVM4 created by process «�rsh-computer3�» Infrastructure // how and where the JVMs specified above are created process-definition «�linuxJVM�» // this process creates a JVM on the current host JVMProcess class= JVMNodeProcess process-definition «�rsh-computer1�» // this process establishes an rsh connection and starts a JVM on the remote host rshProcess class = RSHProcess host = «�computer1�» processReference = «�linuxJVM�» process-definition «�rsh-computer2�» rshProcess class = RSHProcess host = «�computer2�» processReference = «�linuxJVM process-definition «�rsh-computer2�» rshProcess class = RSHProcess host = «�computer3�» processReference = «�linuxJVM�» Denis Caromel 88
Component Orientedness • Level 1: Instantiate - Deploy - Configure • Simple Pattern • Meta-information (file, XML, etc.) JavaBeans, EJB • Level 2: Assembly (flat) • Server and client interfaces CCM • Level 3: Hierarchic • Composite Fractal, ProActive, ... • Level 4: Distributed + Reconfiguration • Binding, Inclusion, Location ProActive + On going work This talk Interactions / Communications: Functional Calls: service, event, stream Non-Functional: instantiate, deploy, start/stop, inner/outer, re-bind Denis Caromel 89
Distributed Components ComponentIdentity Cpt = newActiveComponent (params); A a = Cpt … .getFcInterface ("interfaceName"); V v = a.foo(param); A Example of component V instance Typed Group Java or Active Object JVM Denis Caromel 90
Migration Capability of composites Migrate sets of components, including composites Denis Caromel 91
Collective Operations : Example class A {… // Build a group of «�A�» V foo(P p){...} A ag = (A)ProActiveGroup.newGroup(«�A�», {a1,a2,b1}) } V v = ag.foo(param); // foo(param) invoked class B extends A // on each member { ...} // A group v of result of type V is created A a1= PA.newAct( A,); A a2= PA.newAct( A,); B b1= PA.newAct( B,); v.bar(); ProActiveGroup.waitForAll A a3= PA.newAct( A,); (v); //bloking -> all // starts bar() on // => modif. of each member of v.bar();//Group call group ag : the result group upon arrival Group ga = V vi = ProActiveGroup. ProActiveGroup.getOne(v); getGroup(ag); //bloking -> on ga.add(a3); vi.bar(); //a single call //ag is updated Denis Caromel 92
Two Representation Scheme Management of the group getGroup method of class Group Group of objects Typed group groupA gA getGroupByType static method of class ProActive Fonctional use of the group Denis Caromel 93
Two Representations (2) • Management operations add , remove , size , … • 2 possibility : static methods, second representation • 2 representations of a same group : Typed Group / Group of objects • ability to switch between those 2 representations Group gA = ProActiveGroup.getGroup(groupA); gA.add(new A()); gA.add(new B()); //B herits from A A groupA = (A) gA.getGroupeByType(); Denis Caromel 94
Denis Caromel 95
ProActive : Migration of active objects Migration is initiated by the active object itself through a primitive: migrateTo Can be initiated from outside through any public method The active object migrates with: • all pending requests • all its passive objects • all its future objects Automatic and transparent forwarding of: • requests (remote references remain valid) • replies (its previous queries will be fullfilled) Denis Caromel 96
Mapping Virtual Nodes: example (1) <virtualNodesDefinition> <virtualNode name="Dispatcher"/> Definition of Virtual Nodes </virtualNodesDefinition> <map virtualNode="Dispatcher"> <jvmSet> Definitions <vmName value="Jvm1"/> </jvmSet> and mapping </map> Mapping of Virtual Nodes <jvm name="Jvm1"> <acquisition method="rmi"/> <creation> <processReference refid="linuxJVM"/> </creation> </jvm> Denis Caromel 97
Mapping Virtual Nodes: example (2) <virtualNodesDefinition> <virtualNode name="Jem3DNode"/> Definition of Virtual Nodes </virtualNodesDefinition> <map virtualNode=" Jem3DNode"> <jvmSet> Definitions <vmName value=”clusterJvm"/> and mapping </jvmSet> </map> Mapping of Virtual Nodes <jvm name="clusterJvm"> <acquisition method="rmi"/> <creation> <processReference refid=”clusterProcess"/> </creation> </jvm> Denis Caromel 98
Mapping Virtual Nodes: example (3) <processDefinition id="linuxJVM"> <jvmProcess class="org.objectweb.proactive.core.process.JVMNodeProcess"/> </processDefinition> JVM on the current Host Infrastructure <processDefinition id="rshProcess"> informations <rshProcess class="org.objectweb.proactive.core.process.rsh.RSHJVMProcess" hostname="sea.inria.fr"> <processReference refid="linuxJVM"/> </rshProcess> </processDefinition> JVM started using RSH Denis Caromel 99
Mapping Virtual Nodes: example (4) <processDefinition id=”singleJVM"> <jvmProcess class="org.objectweb.proactive.core.process.JVMNodeProcess"/> </processDefinition> <processDefinition id=" clusterProcess "> �������������� <bsubProcess class="org.objectweb.proactive.core.process.lsf.LSFBSubProcess" ������������ hostname=”cluster.inria.fr"> <processReference refid=”singleJVM"/> <bsubOption> <processor>12</processor> </bsubOption> </bsubProcess> </processDefinition> Definition of bsub process Denis Caromel 100
Recommend
More recommend
