Erlang: Developing Scalable Systems
Rise of collaborative systems � Sequential system cannot scale. � Need more concurrency. � Parallel code can scale Collaborative computing � Amdhal’s law � Don’t Scale up and Scale out
Hitting performance bottleneck…. � Clock speed not increasing � Not much gain in system performance � Demand for concurrency in application � Demand for fault tolerant technologies � Natural progression of existing � Natural progression of existing technologies � Reaching the limits of sequential programming Transistors Clock speed Power Performance / Clock
Scalability Scalability is the ability of a system, network, or process, to handle a growing amount of work in a capable manner or its ability to be enlarged to accommodate that growth. Scalability : 1. Distributed-ness of applications. 2. Fault tolerance and failure management. 2. Fault tolerance and failure management. 3. Concurrency. 4. Performance. So : 1. Go parallel . 2. Go asynchronous.
Background Erlang is a programming language designed for developing robust systems of programs that can be distributed among different computers in a network. Named for the Danish mathematician Agner Krarup Erlang, Named for the Danish mathematician Agner Krarup Erlang, the language was developed by the Ericsson Computer Sciences Lab to build software for its own telecommunication products. Erlang is a functional and actor based programming language
More on background Erlang has been successfully used in production systems for over 20 years (with reported uptimes of 9-nines — that's 31ms of downtime a year). Ericsson themselves have used Erlang extensively for many projects of varying sizes, both commercial and internal. The AXD301 ATM, one of Ericsson's flagship sizes, both commercial and internal. The AXD301 ATM, one of Ericsson's flagship products, may be the largest Erlang project in existence at over 1.1 million lines of Erlang
Guiding principles o Handling a very large number of concurrent activities. o Actions to be performed at a certain point of time or within a certain period of time. o Systems distributed over several computers. o Interaction with hardware. o Very large software systems. o Complex functionality such as feature interaction. o Complex functionality such as feature interaction. o Continuous operation over several years. o Software maintenance (reconfiguration, etc.) without stopping the system. o Stringent quality and reliability requirements. o Fault tolerance both to hardware failures and software errors.
So what Erlang is not, � Is not an object oriented programming language… � No shared states… � No concepts of threads… � No concept of String ( they are stored as list of integer )… � Does not have built in looping constructs like …… while , for… � There is no constructs conditions like ….. If .. then .. else…
Then what Erlang is , � Is Functional and concurrent programming language… � Light weight Processes… � Has message passing… ( between the processes ) � Does have recursion and tail recursion…for looping � Does have constructs for pattern matching , guards … for condition evaluation.
Erlang , brief overview � Constructs for sequential functional code. - var, numbers , tuples , list , atoms , functions , guards , records , bifs. � Constructs for concurrent code. - process/actors , OTP ( open telecom platform) � Inbuilt database / storage - Mnesia / ETS (Erlang Term Storage) , DETS disk . - Mnesia / ETS (Erlang Term Storage) , DETS disk . Erlang Virtual Machine - Memory management , portability , concurrency , garbage collection. - Schedulers , I/O Model (Event Based) Underlying Operating system
Erlang , variables and atoms … � Variables - In Erlang variable is immutable. - Once assigned cannot be reassigned. - Variable name starts with a capital letter. e.g. MyVariable = 54. MyVariable = 55. { this will throw an error.} � � Atoms Atoms - They start with lower case. - They should be enclosed in single quotes if not beginning in lower case. - They are literals with their name as value. - They are constants which cannot be changed. e.g. myatom
Erlang , Tuples and List … � Tuples - It is a way to organize data. - They can contain any kind of data. - Their structure cannot be altered once assigned. e.g. Types = { Orange , Apple }. Values = { 10 , 14}. Values = { 10 , 14}. Types = Values. value of Orange is 10 and the value of Apple is 14. � List and List Comprehensions - Used as a collection. - inbuilt pattern matching for Head and Tail . e.g. [Head | Tail ] = [ 1,2,3,4,5 ]. value of Head is 1 and the value of Tail is [2,3,4,5] e.g. [X || X <- [1,2,3,4,5,6,7,8,9,10] , X rem 2 =:= 0].
Erlang , functions and anonymous functions… � Functions - Higher order functions. - Can be passed around as Arguments . - e.g. arg1() -> 1. arg2() -> 2. sum(X,Y) -> X()+y(). %% mymod:sum( fun mymod:arg1/0 , fun mymod:arg2/0 ). - Called recursively .. � � Anonymous Functions .. Funs. Anonymous Functions .. Funs. - Similar in functionality as a normal function - Cannot be called recursively . � Guards
Erlang , storage (ETS) • Large key-value look up tables. • ETS is memory resident. • ETS stores tuples. • Data stored is transient (cannot survive system crash). • ETS data are not garbage collected. • First element of the tuple is the key of the table. Basic operations on ETS are: Basic operations on ETS are: 1. Create a new table or open an existing table. 2. Insert a tuple or tuples in the database. 3. Lookup for a tuple in the table. 4. Dispose/Delete a table. Process memory Insert Types of table: � Copied to .. • Sets • Ordered Sets, ETS Table • Bags, • Duplicate bags.
Erlang , storage (DETS) • Large key-value look up tables. • DETS is disk resident (DETS files can be of 2gb ). • DETS stores tuples. • Data stored is persistent (can survive system crash). • DETS data are not garbage collected. • First element of the tuple is the key of the table. • • DETS files are open before insertion and closed after insertion. DETS files are open before insertion and closed after insertion. • Process can share the DETS table. Basic operations on DETS are: 1. Create a new table or open an existing table. 2. Insert a tuple or tuples in the database. 3. Lookup for a tuple in the table. 4. Dispose/Delete a table.
Erlang , MNESIA • DB tables can be stored in RAM for speed / Disk for persistence. • Can be replicated in various machines for fault tolerance. • Conditionally selecting data from the table ( use of list comprehensions ). • Selecting Data from two tables and apply conditions. • Provision for Transaction (pessimistic locking). In case of a process accessing a “locked table” the “fun” in the process may be tried multiple number of times ( avoid side affects ). • Mnesia supports fragmented tables ( horizontal portioning ) which can be replicated across machines , and can have indexes. replicated across machines , and can have indexes. • Strategies for tables. - RAM resident single node. - RAM + disk copy on a single node. - Disk-only copy on a single node. - RAM resident table on two/more nodes. - Disk copies on two/mode nodes
Erlang , actors. � Light weight process. � Isolated from other process. � No shared state between the actors. � Inherently concurrent. Society of actors: � Communication is using asynchronous messaging. � Messages buffered in mail box. 1. Fails fast ( trap_exit) events � Massively scalable. 2. Supervisors. � Actors instead of Objects. � Actors can change the state of itself � � Actors send (immutable)messages to other Actor. Actors send (immutable)messages to other Actor. � Actors don’t compete for the shared data. Mail box or Channel Messages can be Actors manage serialized to survive their heap and crash stack
����������� �����������
Apache vs. Yaws http://www.sics.se/~joe/apachevsyaws.html Coordinated as thoughput (KBytes/second) vs. load. The red curve is yaws (running on an NFS file system). The blue curve is apache (running on an NFS file system). The green curve is apache (running on a local file system).
Erlang and scalability. Horizontal distribution (scaling ) of actors Automatically Erlang VM distributes the work load across CPU resources Single Core-2 Core-1 Core-3 Core
Comparison: Actor / Thread container based multi-threaded app servers Threads Incoming request Thread pool actor model --- Good for scaling out Good for scaling up Non Blocking calls Outgoing response Outgoing response Exporting these paradigms to act Blocking calls Pass by value inherently Pass by reference No State sharing State sharing Shared State State reference Action Action STATE Shared State State reference Action Action
Recommend
More recommend
