Distributed Systems By Aaron Stannard, CEO, Petabridge What Were - PowerPoint PPT Presentation

Distributed Tracing: How the Pros Debug Concurrent and Distributed Systems By Aaron Stannard, CEO, Petabridge

What We’re Going to Cover • Microservices and common “people” problems they cause • How distributed tracing solves some of these problems • What distributed tracing is and how it works • The OpenTracing standard and its implementations • Distributed tracing best practices

Monoliths and Microservices

Microservices are for Scaling People

Microservice People Problems

Enter Distributed Tracing

Distributed Tracing Features

Collecting Distributed Traces

Tracing Output

Popular Tracing Implementations

OpenTracing • Vendor-neutral standard for facilitating distributed tracing • Enforces a common lexicon across all tracing products • Helps library and framework maintainers implement tracing • Defines common carrier formats for propagating traces between services

OpenTracing Terminology

Collecting Trace Data (Code) var url = "http://localhost:9411"; ITracer tracer = new ZipkinTracer(new ZipkinTracerOptions(url, "DemoApp", debug: true)); ISpanBuilder sb = null; ISpan current = tracer.ActiveSpan; sb = tracer.BuildSpan("actual-op").WithTag("empty", false); if (current != null) { current.Finish(); sb = sb.AsChildOf(current); } current = sb.Start(); current.Finish();

Trace Propagation

Carrier Formats B3 HTTP Headers: X-B3-TraceId: EFFC1D X-B3-ParentSpanId: ACC1DD X-B3-SpanId: 100FFD X-B3-Sampled: 0

Best Practices • Standardize on carrier formats inside your services – i.e. B3 HTTP headers, dictionary formats • Introduce tracing at the infrastructure level, if you can – i.e. inject into HTTP request processing pipeline • Use OpenTracing IScopeManager – Automatically resolves current active Span • Have logging infrastructure inject events into active span

Further Reading • http://opentracing.io • https://zipkin.io/ • https://github.com/openzipkin/docker-zipkin