Counters and Async Infrastructures Proposal for Code Contribution - PowerPoint PPT Presentation

Counters and Async Infrastructures Proposal for Code Contribution Guy Sela, Senior Engineer, HPE

Who am I ? • Senior Engineer at HPE • Served as a developer and team leader at the Israeli NSA • Developing software since 2004 E-mail: guy.sela@hpe.com Linkedin: http://www.linkedin.com/pub/guy-sela/70/17/9aa

Agenda Counters Infrastructure • Why do we need Counters? • Proposed Implementation • Capabilities • Examples • Under the Hood • Third-Party Alternatives Async Infrastructure • Scope • Overview • Under the Hood

Why do we need Counters? • High-Frequency events will swamp the Log. • Good alternative for Log.debug() and Log.trace() • Useful information for troubleshooting. • Powerful tool for analyzing bottlenecks. • What events happened in my system until now? • Analytics

Proposed Implementation • Light-weight infrastructure that is very easy to integrate into your code • Supports the common use-cases for counters • Found to be extremely useful for monitoring and troubleshooting production systems

Capabilities • Incremental & State Counters • Configurable interval for “print counters delta to the log” • Runtime counters silencing based on the user’s log library • Almost no overhead on the system • Counter querying API based on regular expressions • REST API

Use Case – Incremental Counter public class UserService { // High-Scale Event void userEvent() { log.info(“Event Called”); // Will swamp the Log … // Business Logic } }

Use Case – Incremental Counter public class UserService { // High-Scale Event void userEvent() { UserServiceCounters.user_event_called.inc(); … // Business Logic } }

Log Output Example – 1 Minute Interval [2016-02-16 14:18:27,626] INFO : (com.hpe.test.TestClassA:252) – Did Something [2016-02-16 14:18:27,967] INFO : (com.hpe.test.TestClassB:254) – Did Something Else [2016-02-16 14:18:28,028] CNT : user_event_called : +37824 [2016-02-16 14:18:45,125] INFO : (com.hpe.test.TestClassC:63) – Yet Another Log Line [2016-02-16 14:19:28,028] CNT : user_event_called : +41215 [2016-02-16 14:19:50,026] INFO : (com.hpe.test.TestClassC:72) – I Love Log Lines [2016-02-16 14:20:30,028] CNT : user_event_called : +51621 [2016-02-16 14:20:50,026] INFO : (com.hpe.test.TestClassA:11) – Log Lines Rock! 10

Log Output Example – Multiple Counters [2016-02-16 15:18:27,626] INFO : (com.hpe.test.TestClassA:252) – Did Something [2016-02-16 15:18:27,967] INFO : (com.hpe.test.TestClassB:254) – Did Something Else [2016-02-16 15:18:28,028] CNT : user_event_called : +37824, other_counter: +72 [2016-02-16 15:18:45,125] INFO : (com.hpe.test.TestClassC:63) – Yet Another Log Line [2016-02-16 15:19:28,028] CNT : user_event_called : +41215, foo_counter: +17 11

import com.hpe.counter.OccurrenceCounter; public class UserService { // Boilerplate Code to define a new Counters Group enum UserServiceCounters { user_event_called() , other_user_counter(), third_counter(); private OccurenceCounter counter; public void inc() { counter.inc(); } UserServiceCounters() { counter = new OccurenceCounter(“UserServiceCounters”, name()); } } } 12

Counter Types • Incremental Counter – Value is manipulated using inc(). The log prints show the delta from the last log print. No delta  No log print. • State Counter – Value is manipulated using set(). The log prints show the current value of the counter in case it changed . Useful for detecting queue build-up. 13

Counters Silencing Log4J Config File // Standard mechanism to change Log Severity log4j.logger.UserService.user_event_called=WARN 14

Use Case - Using Counters to Identify Bottlenecks void myBusinessLogic { String value = DB.getKey(“key”); // … Socket.sendOnSocket(new Packet(value.toBytes())); // … invokeIntensiveMethod(value); } Scenario: 5000 Events per second which invoke myBusinessLogic(); Symptom: We only manage to handle 2000 Events per second 15

Use Case - Using Counters to Identify Bottlenecks void myBusinessLogic { BLCounters.before_db.inc(); String value = DB.getKey(“key”); BLCounters.after_db.inc(); // … BLCounters.before_socket.inc(); Socket.sendOnSocket(new Packet(value.toBytes())); BLCounters.after_socket.inc(); // … BLCounters.before_intensive.inc(); invokeIntensiveMethod(value); BLCounters.after_intensive.inc(); } 16

Log Output will tell us where is the Bottleneck [2016-02-16 15:18:28,028] CNT : before_db: +5000, after_db: +2000 , before_socket: +2000, after_socket: +2000, before_intensive: +2000, after_intensive: +2000 The DB is Slowing Us! 17

Log Output will tell us where is the Bottleneck [2016-02-16 15:18:28,028] CNT : before_db: +5000, after_db: +5000, before_socket: +5000, after_socket: +5000, before_intensive: +5000, after_intensive: +2000 The Intensive Method is Slowing Us! 18

What events happened in my System until Now? CountersDumperService API String getCounters(String regexp); • Call using REST API / Other Management Console 19

Analytics 20

Under the Hood • Counters based on AtomicLong & AtomicInteger causes insignificant overhead. • Using enum semantics instead of “register new counter” implementation • Background Thread that writes the delta of the counters to the Log in each configurable interval. 21

Third-Party Alternatives • Known third-party implementations with similar goals: Metrics, Parfait, JAMon, Java Simon, Perf4J. • We haven’t made a deep comparison between the proposed solution to these. 22

Questions? 23

And Now to a Completely Different Subject 24

Async Infrastructure 25

Introduction • In ODL’s Hydrogen version, every OSGI Bundle was activated using the BundleActivator pattern. • Every OSGI Service declared its implementations and dependencies in its Activator. • After the transformation to MD-SAL, Plugins/Applications define their dependencies, implementations and API using YANG Model. • Applications that reside in the Controller communicate with each other using MD-SAL: RPCs and Notifications. 26

Introduction • Application can be implemented by a number of OSGI Services and Bundles. • The presented Async Infrastructure facilitates communication between OSGI services, and is agnostic to YANG, RPCs, etc. • The infrastructure is currently only relevant for Bundles that still use the BundleActivator pattern. Specifically for Activators that extend ComponentActivatorAbstractBase. • If the presented capabilities interest the ODL community, we can discuss extension of the infrastructure to support the new OSGI Bundle Activation patterns, that came along with MD-SAL 27

Scope Application A Application B Async Infra (Unwanted?) OSGI Service OSGI Service OSGI Service OSGI Service Async Infra Async Infra RPC / Notifications RPC / Notifications MD-SAL 28

Overview • ServiceA implements PacketProcessingListener MD-SAL onPacketReceived() Notification doSomething() doSomethingElse() Service A Service B Service C Application A 29

public class ServiceA implements IServiceA, PacketProcessingListener { private IServiceB serviceB; public void onPacketReceived(PacketReceived notification) { serviceB.doSomething(); } } 30

public class ServiceB implements IServiceB { private IServiceC serviceC; public void doSomething() { serviceC.doSomethingElse(); } } 31

doSomething() doSomethingElse() Service A Service B Service C • The presented infrastructure enables you to convert these Java calls to Asynchronous Multi-Threaded calls just by changing a configuration file! 32

Example of the config-file /pools/OurThreadPool/poolSize=8 /handlers/com.hpe.ServiceB/PoolName=OurThreadPool /handlers/com.hpe.ServiceB/WorkerCount=5 /handlers/com.hpe.ServiceC/PoolName=OurThreadPool /handlers/com.hpe.ServiceC/WorkerCount=6 • We create a Thread Pool with 8 Threads, and define ServiceB and ServiceC to use it. • ServiceB will use up to 5 concurrent Threads while ServiceC will use up to 6 concurrent Threads. 33

Glimpse Under the Hood doSomething() ServiceB Proxy ServiceA ServiceB Proxy ServiceB Dedicated Queue doSomething() Worker Thread ServiceB Worker Thread Worker Thread 34

Benefits • The developers can focus in coding the business logic of the Service. • The API remains exactly the same with the help of Proxies and Reflection. • Dynamically decide which services will be Async and control Thread Pools. • No need to write bug-prone Boilerplate Code for Async method invocation. This includes BlockingQueues, Producer Threads, Consumer Threads, Executors, Thread Pools, synchronized blocks. Caveat: The code itself obviously must support multi-threaded execution, i.e. Concurrent data structures, etc. • The Queue can be capped, and messages that exceed the cap will be dropped. Important for prevention of memory leaks. 35

What if I want only part of the API to be ASYNC? public class MyService implements IMyService { public void methodWeWantAsync() { } public void anotherMethodWeWantAsync() { } @SyncMethod public void methodWeWantSync() { } } @SyncMethod - Will tell the infrastructure to run this method using the Caller’s Thread 36