iv p e r k 1 s x t
play

IV P E R K 1 S X T Jw fshjxht Gjxfw n sn Erlang - PowerPoint PPT Presentation

Apr 02, 2024 •133 likes •751 views

FRANCESCO CESARINI presents IV P E R K 1 S X T Jw fshjxht Gjxfw n sn Erlang Solutions @FrancescoC francesco@erlang-solutions.com www.erlang-solutions.com W HAT I S S CALABILITY ? W HAT I S ( MASSIVE ) C ONCURRENCY ? W HAT I S H IGH

  1. FRANCESCO CESARINI presents IV P E R K 1 S X T Jw fshjxht Gjxfw n sn Erlang Solutions @FrancescoC francesco@erlang-solutions.com www.erlang-solutions.com

  3. W HAT I S S CALABILITY ?

  5. W HAT I S ( MASSIVE ) C ONCURRENCY ?

  7. W HAT I S H IGH A VAILABILITY ?

  9. W HAT I S F AULT T OLERANCE ?

  11. W HAT I S D ISTRIBUTION T RANSPARENCY ?

  13. Do you need a distributed system? Do you need a scalable system? Do you need a reliable system? Do you need a fault-tolerant system? Do you need a massively concurrent system? Do you need a distributed system? Do you need a scalable YES, PLEASE!!! system? Do you need a reliable system? Do you need a fault-tolerant system? Do distributed system? Do you need a scalable system? Do you need a reliable system? Do you need a fault-tolerant system? Do you need a massively

  14. TO THE RESCUE

  15. • O PEN SOURCE • C ONCURRENCY - ORIENTED WHAT IS ERLANG • L IGHTWEIGHT PROCESSES • A SYNCHRONOUS MESSAGE PASSING • S HARE - NOTHING MODEL • P ROCESS LINKING / MONITORING • S UPERVISION TREES AND RECOVERY STRATEGIES • T RANSPARENT DISTRIBUTION MODEL • S OFT - REAL TIME • L ET - IT - FAIL PHILOSOPHY • H OT - CODE UPGRADES

  16. WELL, IN FACT YOU NEED MORE.

  17. ERLANG IS JUST A PROGRAMMING LANGUAGE.

  18. YOU NEED ARCHITECTURE PATTERNS. YOU NEED MIDDLEWARE. YOU NEED LIBRARIES. YOU NEED TOOLS.

  19. Y OU NEED OTP.

  20. S OME T EXT

  21. W HAT IS M IDDLEWARE ?

  22. D ESIGN P ATTERNS Q M H H P I[ F AULT T OLERANCE E V I D ISTRIBUTION U PGRADES P ACKAGING

  23. W HAT ARE L IBRARIES ?

  24. S TORAGE P M F V E V O&M M IW I NTERFACES C OMMUNICATION

  25. W HAT T OOLS ?

  26. D EVELOPMENT S X T X S T EST F RAMEWORKS S P W R ELEASE & D EPLOYMENT D EBUGGING & M ONITORING

  27. OPEN SOURCE OTP IS PART OF THE ERLANG DISTRIBUTION

  28. OTP Less Code Servers Less Bugs Finite State Machines More Solid Code Event Handlers More Tested Code Supervisors More Free Time Applications

  29. Your Heading

  30. Let It Fail convert(Day) -> case Day of monday -> 1; tuesday -> 2; wednesday -> 3; thursday -> 4; friday -> 5; saturday -> 6; sunday -> 7; Other -> {error, unknown_day} end.

  31. Let It Fail convert(Day) -> case Day of monday -> 1; tuesday -> 2; wednesday -> 3; thursday -> 4; friday -> 5; saturday -> 6; sunday -> 7; end.

  32. I SOLATE T HE E RROR !

  33. Exit Signals P ROPAGATING E XIT S IGNALS {'EXIT', PidA, Reason} PidB PidA {'EXIT', PidB, Reason} PidC

  34. Trap Exit T RAPPING AN E XIT S IGNAL {'EXIT', PidA, Reason} PidB PidA PidC

  35. Supervisors Supervisor PidB PidA Workers PidC Application

  36. Releases Release Mongoose folsom lager IM snmp mnesia stdlib ERTS SASL kernel

  37. B EHAVIOURS

  38. W jw |jw GENERIC SPECIFIC BEHAVIOUR CALLBACK MODULE MODULE process

  39. OTP Less Code Servers Less Bugs Finite State Machines More Solid Code Event Handlers More Tested Code Supervisors More Free Time Applications

  40. {request, Pid, Message} Client Server {reply, Reply} call(Name, Message) -> Name ! {request, self(), Message}, receive {reply, Reply} -> Reply end. reply(Pid, Reply) -> Pid ! {reply, Reply}.

  41. {request, Pid, Message} {request, { Ref , self()}, Message} Client Server {reply, Ref , Reply} {reply, Reply} {reply, ??? , Reply} {reply, Reply} Server 2 call(Name, Msg) -> Ref = make_ref(), Name ! {request, { Ref , self()}, Msg}, receive {reply, Ref , Reply} -> Reply end. reply({ Ref , Pid}, Reply) -> Pid ! {reply, Ref , Reply}.

  42. {request, {Ref, PidA}, Msg} PidB PidA call(Name, Msg) -> Ref = erlang:monitor(process, Name), Name ! {request, {Ref, self()}, Msg}, receive � {reply, Ref, Reply} -> � erlang:demonitor(Ref), � Reply; � {'DOWN', Ref, process, _Name, _Reason} -> � {error, no_proc} end.

  43. {'DOWN', Ref, process, PidB, Reason} {request, {Ref, PidA}, Msg} {reply, Ref, Reply} PidB PidA call(Name, Msg) -> Ref = erlang:monitor(process, Name), Name ! {request, {Ref, self()}, Msg}, receive � {reply, Ref, Reply} -> � erlang:demonitor(Ref, [flush] ), � Reply; � {'DOWN', Ref, process, _Name, _Reason} -> � {error, no_proc} end.

  44. T IMEOUTS F IL E Z M D EADLOCKS S Y V W TRACING M ONITORING D ISTRIBUTION

  45. A UTOMATIC T AKEOVER AND F AILOVER

  46. {myApp, 2000, {n1@host, {n2@host, n3@host}]} N1 N2 N3 Application Master n1@host dies Application Masters on failover nodes Application

  47. {myApp, 2000, {n1@host, {n2@host, n3@host}]} N2 N3 n2@host dies Application is restarted on n2@host

  48. {myApp, 2000, {n1@host, {n2@host, n3@host}]} N1 N3 Application is restarted on n3@host n1@host comes back up

  49. {myApp, 2000, {n1@host, {n2@host, n3@host}]} N1 N3 N1 takes over N3

  50. RELEASE STATEMENT OF AIMS “To scale the radical concurrency-oriented programming paradigm to build reliable general-purpose software, such as server- based systems, on massively parallel machines (10^5 cores).”

  51. The Runtime Queues Erlang VM Scheduler #1 run queue migration migration Scheduler #2 Scheduler #2 run queue logic logic Scheduler #N run queue

  53. LIMITATIONS ARE PRESENT AT THREE LEVELS WP6 Case Studies WP4 Scalable Infrastructure WP5 Tools WP3 SD Erlang Language WP2 Virtual Machine

  54. VM LANGUAGE INFRASTRUCTURE • P USH THE RESPONSIBILITY FOR SCALABILITY FROM THE PROGRAMMER TO THE VM • A NALYZE PERFORMANCE AND SCALABILITY • I DENTIFY BOTTLENECKS AND PRIORITIZE CHANGES AND EXTENSIONS • T ACKLE WELL - KNOWN SCALABILITY ISSUES • E TS TABLES ( SHARED GLOBAL DATA STRUCTURE ) • M ESSAGE PASSING , COPYING AND FREQUENTLY COMMUNICATING PROCESSES

  55. VM LANGUAGE INFRASTRUCTURE • T WO MAJOR ISSUES • F ULLY CONNECTED CLUSTERS • E XPLICIT PROCESS PLACEMENT • S CALABLE D ISTRIBUTED (SD) E RLANG • N ODES GROUPING • N ON - TRANSITIVE CONNECTIONS • I MPLICIT PROCESS PLACEMENT • P ART OF THE STANDARD E RLANG /OTP PACKAGE • N EW CONCEPTS INTRODUCED • L OCALITY , A FFINITY AND D ISTANCE

  56. VM LANGUAGE INFRASTRUCTURE CCL /s ɪˈ s ɪ l ɪ / • M IDDLEWARE LAYER • S ET OF E RLANG A PPLICATIONS • C REATE AND MANAGE CLUSTERS OF ( HETEROGENEOUS ) ERLANG NODES • API TO MONITOR AND CONTROL ERLANG DISTRIBUTED SYSTEMS • E XISTING TRACING / LOGGING / DEBUGGING TOOLS PLUGGABLE • B ROKER LAYER BETWEEN USERS AND CLOUD PROVIDERS • A UTO - SCALING ... A ND M UCH M ORE

  57. C ONCLUSIONS

  58. Do you need a distributed system? Do you need a scalable system? Do you need a reliable system? Do you need a fault-tolerant system? Do you need a massively concurrent system? Do you need a distributed system? Do you need a scalable USE ERLANG system? Do you need a reliable system? Do you need a fault-tolerant system? Do distributed system? Do you need a scalable system? Do you need a reliable system? Do you need a fault-tolerant system? Do you need a massively

  59. Do you need a distributed system? Do you need a scalable system? Do you need a reliable system? Do you need a fault-tolerant system? Do you need a massively concurrent system? Do you need a distributed system? Do you need a scalable USE ERLANG/OTP system? Do you need a reliable system? Do you need a fault-tolerant system? Do distributed system? Do you need a scalable system? Do you need a reliable system? Do you need a fault-tolerant system? Do you need a massively

  60. Q UESTIONS ? @francescoC

Recommend

More recommend