HAZELCAST DISTRIBUTED DATA STRUCTURES FOR JAVA
WHO AM I Fuad Malikov @fuadm Hazelcast Co-founder Java Developer since 2005 Leading Hazelcast Technical Operations
COMPANY: HAZELCAST Founded in 2008 Open Source Business Model Head Quartered in Palo Alto, CA Distributed Team with R&D mainly in Europe $14M+ Total Investment
PRODUCT: HAZELCAST In-Memory DataGrid Open Source - Apache License 2.0 In-Mempory Distributed Data Store Distributed Messaging Distributed Computation NoSQL
WHO USES HAZELCAST? and many more ...
USECASES
ALTERNATIVES? Oracle Coherence IBM eXtreme Scale VMware Gemfire Gridgain Redhat Infinispan Terracotta Gigaspaces
WHY HAZELCAST? Scale-out Computing Fast Application Performance Resilience Programming Model
FEATURES Java Collection API Map, Queue, Set, List, +MultiMap, +ReplicatedMap Topic (PubSub) Ring Buffer Java Concurrency API Lock, ExecutorService, AtomicLong, Semaphore, CountDownLatch, Entry Processor, Query, Map Reduce, Aggregations Transactions Custom Serialization Native Memory support (off-heap) Native client: C#, C++, Java, REST, memcached Hibernate 2nd Level Cache Web Session Replication
CACHE AS A SERVICE
EASY API // Creating a new Hazelcast node HazelcastInstance hz = Hazelcast.newHazelcastInstance(); // Getting a Map, Queue, Topic, ... Map map = hz.getMap("my-map"); Queue queue = hz.getQueue("my-queue"); ITopic topic = hz.getTopic("my-topic"); //Creating a Hazelcast Client HazelcastInstance client = Hazelcast.newHazelcastClient(); // Shutting down the node hz.shutdown();
HOW DOES IT WORK?
DATA PARTITIONING (1/2) Multiple partitions per node Consistent Hashing: partition_id = hash(serialize(key)) % partitionCount ; Controlled partitioning: "key@partitionkey", c.h.c.PartitionAware Possibility to find key owner for every key Support for Near-Caching and executions on key owner Automatic Fault-Tolerance Synchronous and Asynchronous backups Define sync / async backup counts
DATA PARTITIONING (2/2) With 4 cluster nodes every server holds 1/4 real data and 1/4 of backups
A HAZELCAST NETWORK
HAZELCAST IN NUMBERS Default partition count 271 Biggest cluster 600+ nodes 1 TB in-memory data Handles 1M+/sec topic messages Native Memory (Off-Heap) support for low GC overhead
COMMUNITY VS. ENTERPRISE Feature Community Enterprise Java Collection API X X Java Concurrency API X X JCache API X X Map Reduce / Aggregations X X HD Memory Store (off-heap) X Advanced Security X Native Clients (C++ / C#) X Management Center X
HEAP VS HD MEMORY STORE
CODE SAMPLES
EASY TO UNITTEST public class SomeTestCase { private HazelcastInstance[] instances; @Before public void before() throws Exception { // Multiple instances on the same JVM instances = new HazelcastInstance[2]; instances[0] = Hazelcast.newHazelcastInstance(); instances[1] = Hazelcast.newHazelcastInstance(); } @After public void after() throws Exception { Hazelcast.shutdownAll(); } }
SERIALIZATION // java.io.Serializable public class User implements Serializable {} // or java.io.Externalizable public class User implements Externalizable {} // or (com.hazelcast.nio.serialization).DataSerializable public class User implements DataSerializable {} // or (com.hazelcast.nio.serialization).DataSerializable public class User implements IdentifiedDataSerializable {} // or new in Hazelcast 3 (multi version support) Portable public class User implements Portable {} //Support for Pluggable Custom Serializer
MAP interface com.hazelcast.core.IMap<K, V> extends java.util.Map, java.util.ConcurrentMap HazelcastInstance hz = getHazelcastInstance(); //java.util.concurrent.ConcurrentMap implementation IMap<String, User> hzMap = hz.getMap("users"); hzMap.put("Peter", new User("Peter", "Veentjer")); hzMap.putIfAbsent("Peter", new User("Peter", "Veentjer")); //Distributed Lock hzMap.lock("Peter"); User peter = map.get("Peter");
LIST interface com.hazelcast.core.IList<E> extends java.util.List HazelcastInstance hz = getHazelcastInstance(); //java.util.List implementation L ist<User> l ist = hz.getList("users"); l ist.add(new User("Peter", "Veentjer")); User peter = list.get(0);
QUEUE interface com.hazelcast.core.IQueue<E> extends java.util.concurrent.BlockingQueue HazelcastInstance hz = getHazelcastInstance(); //java.util.concurrent.BlockingQueue implementation IQueue<Task> queue = hz.getQueue("tasks"); queue.offer(newTask()); queue.offer(newTask(), 500, TimeUnit.MILLISECONDS); Task task = queue.poll(); Task task = queue.poll(100, TimeUnit.MILLISECONDS); Task task = queue.take();
LOCK (1/3) interface com.hazelcast.core.ILock extends java.util.concurrent.locks.Lock
LOCK (2/3) HazelcastInstance hz = getHazelcastInstance(); // Distributed Reentrant L ock l ock = hz.getLock("myLock"); l ock.lock(); try { // Do something } finally { l ock.unlock(); }
LOCK (3/3) HazelcastInstance hz = getHazelcastInstance(); // Map (Row-)locks IMap<String, User> map = hz.getMap("users"); map.lock("Peter"); try { // Do something with Peter } finally { map.unlock("Peter"); }
TOPIC / PUBSUB public class Example implements MessageListener<String> { public void sendMessage { HazelcastInstance hz = getHazelcastInstance(); ITopic<String> topic = hz.getTopic("topic"); topic.addMessageListener(this); topic.publish("Hello World"); } @Override public void onMessage(Message<String> message) { System.out.println("Got message: " + message.getMessageObject()); } }
EXECUTORSERVICE public interface com.hazelcast.core.IExecutorService extends java.util.concurrent.ExecutorService HazelcastInstance hz = getHazelcastInstance(); //java.util.concurrent.ExecutorService implementation IExecutorService es = hz.getExecutorService("name"); es.executeOnAllMembers(buildRunnable()); es.executeOnKeyOwner(buildRunnable(), "Peter"); es.execute(buildRunnable()); Map<..> futures = es.submitToAllMembers(buildCallable()); Future<..> future = es.submitToKeyOwner(buildCallable(), "Peter"); es.submitToAllMembers(buildCallable(), buildCallback()); es.submitToKeyOwner(buildCallable(), "Peter", buildCallback());
MORE ADVANCED FEATURES
ADVANCED FEATURES Secondary Indexes and SQL-Like Query, Write-Behind / Write-Through persistence Read-Through (if key not loaded use MapLoader) Local & XA Transactions EntryListeners / EntryProcessors Map Reduce / Aggregators Hibernate 2nd Level Cache HTTP Session Clustering Automatic eviction Controlled partitioning and many more ...
CODE SAMPLES
INDEXING IMap<String, User> map = Hazelcast.getMap("users"); map.addIndex("age", true); // ordered map.addIndex("active", false); // not ordered <map name="employees"> ... <indexes> <index ordered="true">age</index> <index ordered="false">name</index> </indexes> </map>
DISTRIBUTED SQL-LIKE QUERIES HazelcastInstance hz = getHazelcastInstance(); IMap<String, User> map = hz.getMap("users"); Predicate predicate = new SqlPredicate("active AND age <= 30"); Set<User> users = map.values(predicate); Set<Entry<String, User>> entries = map.entrySet(predicate);
PERSISTENCY: MAPLOADER / MAPSTORE public class MapStorage implements MapStore<String, User>, MapLoader<String, User> { // Some methods missing ... @Override public User load(String key) { return loadValueDB(key); } @Override public Set<String> loadAllKeys() { return loadKeysDB(); } @Override public void delete(String key) { deleteDB(key); } @Override public void store(String key, User value) { storeToDatabase(key, value); } } <map name="users"> <map-store enabled="true"> <class-name>com.hazelcast.example.MapStorage</class-name> <write-delay-seconds>0</write-delay-seconds> </map-store> </map>
TRANSACTIONS HazelcastInstance hz = getHazelcastInstance(); TransactionContext context = hz.newTransactionContext(); context.beginTransaction(); TransactionalMap map = context.getMap("default"); TransactionalQueue queue = context.getQueue("default"); try { Tweet tweet = (Tweet) queue.poll(); processTweet(tweet); map.put(buildKey(tweet), tweet); context.commitTransaction(); } catch (Exception e) { context.rollbackTransaction(); }
Recommend
More recommend
