Scaling PyCSP Rune Mllegaard Friborg, John Markus Bjrndalen and - - PowerPoint PPT Presentation

▶
scaling pycsp
SMART_READER_LITE
LIVE PREVIEW

Scaling PyCSP Rune Mllegaard Friborg, John Markus Bjrndalen and - - PowerPoint PPT Presentation

Dec 12, 2023 7 likes •490 views

Scaling PyCSP Rune Mllegaard Friborg, John Markus Bjrndalen and Brian Vinter CPA 2013, Edinburgh August 25, 2013 1 Python for eScience Applications Flexible Can interface with most programming languages Many scientists already

slide-1
SLIDE 1

1

Scaling PyCSP

Rune Møllegaard Friborg, John Markus Bjørndalen and Brian Vinter CPA 2013, Edinburgh August 25, 2013

slide-2
SLIDE 2

2

Python for eScience Applications

  • Flexible
  • Can interface with most programming languages
  • Many scientists already know Python
  • Faster development cycle
  • Compute-intensive parts written in compilable languages are

easily integrated

  • Forces programmers to write readable code
slide-3
SLIDE 3

3

CSP for eScience!

  • Synchronized constructs for running a set of processes

– In parallel – In sequence

  • Synchronized communication through message passing

– One-way channels

  • Complete Process Isolation

– No shared data-structures – No side-effects from processes – Compositional structure – Reuse of processes

  • The data flow in scientific applications is often simple to model

in CSP.

slide-4
SLIDE 4

4

Introduction to PyCSP

  • 2007 - PyCSP is presented. The synchronization model for

channel communications is based on JCSP.

  • 2009 - A PyCSP with a new synchronization model is
  • presented. It is using the two-phase locking protocol to allow

any-to-any channels supporting both input and output guards.

  • 2011-2012 - A distributed version of the synchronization model

is presented and later implemented in PyCSP

slide-5
SLIDE 5

5

We want to run anywhere!

  • The user of PyCSP does not need to know

anything about the location of the hardware any process might run on

  • All channel ends are mobile
slide-6
SLIDE 6

6

Basic PyCSP Features

slide-7
SLIDE 7

7

Single Any-to-Any Channel

A = Channel(“A”)

slide-8
SLIDE 8

8

Buffered

A = Channel(“A”, buffer=10)

slide-9
SLIDE 9

9

Termination through Poisoning / Retiring

Cin = A.reader() Cin.poison() # propagate pill right now! Cin.retire() # propagate pill, when all readers on A have invoked retire

slide-10
SLIDE 10

10

External Choice

# Does not guarantee priority AltSelect(InputGuard(cin), OutputGuard(cout, msg))

slide-11
SLIDE 11

11

External Choice

# Guarantees priority, by adding a wait for an acknowledgement PriSelect(InputGuard(cin), OutputGuard(cout, msg))

slide-12
SLIDE 12

12

External Choice

# Uses PriSelect to perform a fair choice through reordering of guards, based on past selections FairSelect(InputGuard(cin), OutputGuard(cout, msg))

slide-13
SLIDE 13

13

Declaring Processes

# An OS thread @process def Increment(cin, cout): cout(cin() + 1)

slide-14
SLIDE 14

14

Declaring Processes

# An OS process @multiprocess def Increment(cin, cout): cout(cin() + 1)

slide-15
SLIDE 15

15

Starting Processes

# Blocking PAR - Natural number generator Parallel( Prefix(C.reader(), A.writer(), 1), Increment(A.reader(), B.writer(), Delta(B.reader(), C.writer(), D.writer()) ) Spawn(processes...) Sequence(processes...)

slide-16
SLIDE 16

16

Compositional

@process def Counter(cout): Parallel( Prefix(C.reader(), A.writer(), 1), Increment(A.reader(), B.writer(), Delta(B.reader(), C.writer(), cout) ) )

slide-17
SLIDE 17

17

Connecting Channels Host A

# Hosting channel A A = Channel(“A”) # Get address print(A.address) ('192.168.1.16', 63550)

slide-18
SLIDE 18

18

Connecting Channels Host B

# Connect to channel A A = Channel(“A”, connect=('192.168.1.16', 63550))

slide-19
SLIDE 19

19

@clusterprocess

slide-20
SLIDE 20

20

Declaring Remote Process

# A cluster process @clusterprocess def Increment(cin, cout): cout(cin() + 1)

slide-21
SLIDE 21

21

Declaring Remote Process

# A cluster process @clusterprocess( cluster_nodefile = <file containing list of nodes>, cluster_pin = <index for node in list>, cluster_hint = <'blocked' or 'strided'> ) def Increment(cin, cout): cout(cin() + 1)

slide-22
SLIDE 22

22

Executing Remote Process

# Spawn single increment process Spawn(Increment(A.reader(), B.writer())) # or # Spawn 5 increment processes and put them on 5 different hosts if available Spawn( 5 * Increment(A.reader(), B.writer()), cluster_hint = 'strided')

slide-23
SLIDE 23

23

Connecting Channels Implicitly

# Blocking PAR - Natural number generator # One clusterprocess per host Parallel( Prefix(C.reader(), A.writer(), 1), Increment(A.reader(), B.writer(), Delta(B.reader(), C.writer(), D.writer()), cluster_hint = 'strided' )

slide-24
SLIDE 24

24

Connecting Channels Implicitly

@clusterprocess def P1(cin): cin() # read value @clusterprocess def P2(cout): cout(42) # send value A = Channel() Parallel(P1(A.reader()),P2(A.writer()))

slide-25
SLIDE 25

25

Connecting Channels Implicitly

@clusterprocess def P1(cin): cin() # read value @clusterprocess def P2(cout): cout(42) # send value A = Channel(“A”) Parallel(P1(A.reader()),P2(A.writer())) Channels: “A”

slide-26
SLIDE 26

26

Connecting Channels Implicitly

@clusterprocess def P1(cin): cin() # read value @clusterprocess def P2(cout): cout(42) # send value A = Channel(“A”) Parallel(P1(A.reader()),P2(A.writer())) Channels: “A” cin() cout() Starting processes

  • n remote hosts using the SSH protocol.

PyCSP channels are used to transfer any function parameters P2 P1

slide-27
SLIDE 27

27

Connecting Channels Implicitly

@clusterprocess def P1(cin): cin() # read value @clusterprocess def P2(cout): cout(42) # send value A = Channel(“A”) Parallel(P1(A.reader()),P2(A.writer())) Channels: “A” cin() cout() The channel ends cin and cout reconnect to the channel home and registers as they are both new channel ends. P2 P1

slide-28
SLIDE 28

28

Connecting Channels Implicitly

@clusterprocess def P1(cin): cin() # read value @clusterprocess def P2(cout): cout(42) # send value A = Channel(“A”) Parallel(P1(A.reader()),P2(A.writer())) Channels: “A” cin() cout(42) The channel ends cin and cout now posts a request for communication at the channel home P2 P1 read write 42

slide-29
SLIDE 29

29

Connecting Channels Implicitly

@clusterprocess def P1(cin): cin() # read value @clusterprocess def P2(cout): cout(42) # send value A = Channel(“A”) Parallel(P1(A.reader()),P2(A.writer())) Channels: “A” cin() cout() The channel home then probes a read and write request for a potential match. Acquires the process locks and if successful, transfers the messages and notifies the processes P2 P1 L L

slide-30
SLIDE 30

30

512 processes (cores) in a ring

@clusterprocess def elementP(this_read, next_write): while True: token = this_read() next_write(token + 1)

slide-31
SLIDE 31

31

512 processes (cores) in a ring

Does not scale!

slide-32
SLIDE 32

32

Possible solutions

  • Avoid a channel home completely

– Requires a lot more messages for any-to-any

  • channels. The location of all processes connected

to a channel must always be known.

  • Request the user to redistribute another set of

channels, where each channel is hosted evenly

  • n the available hosts

– Difficult for the user

  • Add mobility to a channel home, such that it

may be moved during active use.

– Simple for the user. Our choice.

slide-33
SLIDE 33

33

Introducing Mobile Channel Homes in PyCSP

@clusterprocess def elementP(this_read, next_write): this_read.become_home() while True: token = this_read() next_write(token + 1)

slide-34
SLIDE 34

34

Introducing Mobile Channel Homes in PyCSP

  • Based on a transition model presented in 2011.

– When a channel is poisoned, all active requests

(processes) at a channel are notified with a POISON signal

– Similarly, when a channel home is moved, all active

requests (processes) at a channel are notified with a MOVE signal

– Processes then receive the new address of the

channel together with the MOVE signal. The processes then withdraws the active request from the “old” channel home and reposts the request at the new channel home.

slide-35
SLIDE 35

35

Introducing Mobile Channel Homes in PyCSP

@clusterprocess def elementP(this_read, next_write): this_read.become_home() while True: token = this_read() next_write(token + 1)

  • Based on a transition model presented in 2011
slide-36
SLIDE 36

36

Introducing Mobile Channel Homes in PyCSP

A main B C elementP this_read next_write elementP this_read next_write L L

slide-37
SLIDE 37

37

Introducing Mobile Channel Homes in PyCSP

A main B C elementP this_read next_write elementP this_read.become_home() next_write L L

slide-38
SLIDE 38

38

Introducing Mobile Channel Homes in PyCSP

A main B C elementP this_read next_write elementP this_read.become_home() next_write B_2 MOVE B to B_2 L L

slide-39
SLIDE 39

39

Introducing Mobile Channel Homes in PyCSP

A main B C elementP this_read next_write elementP this_read.become_home() next_write B_2 S e n d s a n y b u f f e r e d m e s s a g e s L L

slide-40
SLIDE 40

40

Introducing Mobile Channel Homes in PyCSP

A main B C elementP this_read next_write elementP this_read next_write B_2 MOVED to B_2 B_2 is now the official channel home of B. If any processes connects to B at main, they will receive the message “MOVED to B_2” L L

slide-41
SLIDE 41

41

Introducing Mobile Channel Homes in PyCSP

A main B C elementP this_read next_write elementP this_read next_write B_2 L L (moved)

slide-42
SLIDE 42

42

Introducing Mobile Channel Homes in PyCSP

  • The order of posted requests is not stable

during a move of a channel home. Thus, priorities can not be guaranteed during this step.

  • For most PyCSP applications, this is not

expected to be an issue.

slide-43
SLIDE 43

43

Results

slide-44
SLIDE 44

44

slide-45
SLIDE 45

45

slide-46
SLIDE 46

46

slide-47
SLIDE 47

47

Conclusions

  • PyCSP has distributed channels and remote

processes, but only with the introduction of the become_home() functionality is PyCSP now able to scale seamlessly to large clusters

  • With a few tweaks, we expect that PyCSP can

scale beyond clusters with more than 512 cores.

  • Thus, scientists does now have a tool which

can handle large computations using CSP in a working Python environment.

slide-48
SLIDE 48

48

Thanks