FAULT TOLERANCE WITH PROXYSQL, MRM AND CONSUL ABOUT ME NARCS - PowerPoint PPT Presentation

Sep 23, 2023 •360 likes •543 views

FAULT TOLERANCE WITH PROXYSQL, MRM AND CONSUL ABOUT ME NARCS PILLAO Systems Engineer letgo.com | edreams.com | palominodb | emagister.com BEHIND THE SCENES 75M Downloads | 200M listings | 3B chats in 2,5 years ! AGENDA

FAULT TOLERANCE WITH PROXYSQL, MRM AND CONSUL
ABOUT ME NARCÍS PILLAO Systems Engineer letgo.com | edreams.com | palominodb | emagister.com
BEHIND THE SCENES 75M Downloads | 200M listings | 3B chats in 2,5 years !
AGENDA • Motivations • Describe the components • Architecture • Demo • Q/A
MOTIVATIONS
MOTIVATIONS • Automatic and repeatable process to reduce downtime when the master goes down • Query digest • Filter unexpected queries as ORM nightmares • Auto scalable design thanks to Consul and Puppet • Choose open source and emerging solutions
DESCRIBING THE COMPONENTS
REPLICATION MANAGER (MRM) • Open source solution (MySQL, Percona and MariaDB) • Manual or automatic failover options • Topology: Master and many slaves (ignore server option) • Multiple MRM nodes managed by arbitrator • ProxySQL integration ( just released ) • GH: https://github.com/signal18/replication-manager
PROXYSQL • Query digest • Query filtering • Query rewrite • Query routing • Maintenance operations • https://github.com/sysown/proxysql
CONSUL • Service discovery tool • Maintenance action feature • Introduce new ProxySQL nodes via its DNS • Monitoring checks
ARCHITECTURE
Replication Manager
ARCHITECTURE
ProxySQL and Consul
ARCHITECTURE
DEMO https://github.com/narcisbcn/fault_tolerant_db
QUESTIONS?
THANK YOU ! Github: narcisbcn Twitter: @narcisbcn

Recommend

Roadmap for Section 10.1 The Notion of Fault-Tolerance Fault-Tolerance Support in NTFS Volume

Unit OS10: Fault Tolerance Windows Operating System Internals - by David A. Solomon and Mark E. Russinovich with Andreas Polze Roadmap for Section 10.1 The Notion of Fault-Tolerance Fault-Tolerance Support in NTFS Volume Management - Striped

394 views • 25 slides

Fault Tolerance in Message Passing Fault Tolerance in Message Passing and in Action and in

Fault Tolerance in Message Passing Fault Tolerance in Message Passing and in Action and in Action Jack Dongarra, Innovative Computing Laboratory University of Tennessee and Computer Science and Mathematics Division Oak Ridge National

422 views • 14 slides

General Principles of Fault- Tolerance Daniel Gottesman Perimeter Institute Whats Left For

General Principles of Fault- Tolerance Daniel Gottesman Perimeter Institute Whats Left For Fault-Tolerance? Reduce the overhead (in space and in time) needed for fault-tolerance Better fault-tolerance in 1D? Better magic state

556 views • 22 slides

Rigorous fault-tolerance thresholds Ben Reichardt UC Berkeley N gate circuit 0/1 N gate

Rigorous fault-tolerance thresholds Ben Reichardt UC Berkeley N gate circuit 0/1 N gate circuit Need error 1/N 1/0 Quantum fault-tolerance problem Classical fault-tolerance: Von Neumann (1956) 0/1 Fault-tolerant, larger C High

846 views • 63 slides

Lecture 10: Fault Tolerance Fault Tolerant Concurrent Computing The main principles of fault

Lecture 10: Fault Tolerance Fault Tolerant Concurrent Computing The main principles of fault tolerant programming are: Fault Detection - Knowing that a fault exists Fault Recovery - having atomic instructions that can be rolled back in

361 views • 18 slides

CH NG 8: FAULT TOLERANCE TS. Tr n H i Anh Content 2 1. Introduction to fault

1 Tr n H i Anh Distributed System CH NG 8: FAULT TOLERANCE TS. Tr n H i Anh Content 2 1. Introduction to fault tolerance 2. Process resilience 3. Reliable client-Server Communication 4. Reliable Group Communication 5.

966 views • 50 slides

Distributed Systems 5. Fault Tolerant Systems Fault-Tolerance - 1 Lszl Bszrmnyi

Distributed Systems 5. Fault Tolerant Systems Fault-Tolerance - 1 Lszl Bszrmnyi Distributed Systems Fault tolerance A system or a component fails due to a fault Fault tolerance means that the system continues to provide its

428 views • 40 slides

CSci 5105 Introduction to Distributed Systems Fault Tolerance Last Time Replication and

CSci 5105 Introduction to Distributed Systems Fault Tolerance Last Time Replication and Consistency Today Fault tolerance Chapter 8 TVS Fault Tolerance Basics Availability short time horizon e.g down 1 msec every hour

292 views • 27 slides

Challenging Malicious Inputs with Fault Tolerance Techniques Bruno Luiz Agenda Threats

Challenging Malicious Inputs with Fault Tolerance Techniques Bruno Luiz Agenda Threats Fault Tolerance Fault Injection for Fault Tolerance Assessment Basic and classic techniques Decision Mechanisms Implementation

962 views • 61 slides

Distributed Systems (ICE 601) Fault Tolerance Dongman Lee ICU Class Overview Introduction

Distributed Systems (ICE 601) Fault Tolerance Dongman Lee ICU Class Overview Introduction Failure Model Fault Tolerance Models state machine primary-backup Distributed Systems - Fault Tolerance Introduction

146 views • 12 slides

Fibre bundle framework for unitary quantum fault tolerance Lucy Liuxuan Zhang University of

Motivations and main idea Basics of quantum fault tolerance Geometric picture of QECCs and unitary fault tolerance Conclusion Fibre bundle framework for unitary quantum fault tolerance Lucy Liuxuan Zhang University of Toronto December 18,

795 views • 16 slides

Hypervisor-Based Fault-Tolerance Thomas C. Bressoud, Isis

Hypervisor-Based Fault-Tolerance Thomas C. Bressoud, Isis Distributed Systems Fred B. Schneider, Cornell University SOSP, 1995 Problem Fault tolerance

2.97k views • 8 slides

Adaptability and Fault Tolerance Adaptability and Fault Tolerance Rog rio rio de Lemos de

Adaptability and Fault Tolerance Adaptability and Fault Tolerance Rog rio rio de Lemos de Lemos Rog University of Kent, UK University of Kent, UK Context: self-* and dependability; Focus: adaptability and fault tolerance;

640 views • 19 slides

Towards an Efficient Fault-Tolerance Scheme for GLB Claudia Fohry, Marco Bungart and Jonas Posner

GLB Fault Tolerance Scheme Experimental Results Towards an Efficient Fault-Tolerance Scheme for GLB Claudia Fohry, Marco Bungart and Jonas Posner Programming Languages / Methodologies June 14, 2015 1 / 18 GLB Fault Tolerance Scheme

744 views • 18 slides

iQIM 5 December 2011 Quantum fault tolerance Error correction and fault tolerance will be

Protected gates for superconducting qubits V( ) John Preskill with Peter Brooks and Alexei Kitaev

511 views • 33 slides

Fault Tolerance For Sparse Linear Algebra Computations Implemented In A Grid Environment

Fault Tolerance For Sparse Linear Algebra Computations Implemented In A Grid Environment Experiments with Fault Tolerant Linear Algebra Algorithms Jack Dongarra Jeffery Chen Zhiao Shi Asim YarKhan University of Tennessee Fault Tolerance:

412 views • 22 slides

Fault Tolerance and Robustness in Concurrent Systems Faults, errors, failures, and fault

Fault Tolerance and Robustness in Concurrent Systems Faults, errors, failures, and fault tolerance have many different definitions. What working definition should we use for fault? What does it mean to be fault-tolerant? 2 Faults, errors,

841 views • 16 slides

Improving Scalability and Fault Improving Scalability and Fault Tolerance in an Application

Improving Scalability and Fault Improving Scalability and Fault Tolerance in an Application Tolerance in an Application Management Infrastructure Management Infrastructure Nikolay Topilski , Jeannie Albrecht, and Amin Vahdat Williams College

560 views • 18 slides

Fault Tolerance at Speed Todd L. Montgomery @toddlmontgomery About me What type of Fault

StoneTor Fault Tolerance at Speed Todd L. Montgomery @toddlmontgomery About me What type of Fault Tolerance? What is Clustering? Why Aeron? Design for Speeding Up? What type of Fault Tolerance? What is Clustering? Why Aeron? Design

979 views • 77 slides

Formal Verification of Automatic Circuit Transformations for Fault-Tolerance Dmitry Burlyaev

Formal Verification of Automatic Circuit Transformations for Fault-Tolerance Dmitry Burlyaev Pascal Fradet 09/30/15, Austin, TX, USA @ FMCAD'15 Outline For a given (fault-tolerance) transformation T , we want to prove a property of the form

1.11k views • 79 slides

AFT: A Serverless Fault- Tolerance Shim Vikram Sreekanti , Chenggang Wu, Saurav Chhatrapati,

AFT: A Serverless Fault- Tolerance Shim Vikram Sreekanti , Chenggang Wu, Saurav Chhatrapati, Joseph E. Gonzalez, Joseph M. Hellerstein, Jose M. Faleiro RISE Lab, UC Berkeley 04/29/2020 Fault-Tolerance in Serverless Computing FaaS programs

93 views • 8 slides

Fault-Tolerance for PastryGrid Middleware erin 1 , Heithem Abbes 1 , 2 , Mohamed Jemni 2 , Yazid

Introduction PastryGrid Fault Tolerance in PastryGrid Conclusion Fault-Tolerance for PastryGrid Middleware erin 1 , Heithem Abbes 1 , 2 , Mohamed Jemni 2 , Yazid Christophe C Missaoui 2 1 LIPN, Universit e de Paris XIII, CNRS UMR 7030,

835 views • 71 slides

MC714 - Sistemas Distribuidos slides by Maarten van Steen (adapted from Distributed System - 3rd

MC714 - Sistemas Distribuidos slides by Maarten van Steen (adapted from Distributed System - 3rd Edition) Chapter 08: Fault Tolerance Version: May 16, 2019 Fault tolerance: Introduction to fault tolerance Basic concepts Dependability Basics

587 views • 47 slides

Fault tolerance with transactions: past, present and future Dr Mark Little Technical Development

Fault tolerance with transactions: past, present and future Dr Mark Little Technical Development Manager, Red Hat Overview Fault tolerance Transaction fundamentals What is a transaction? ACID properties Distributed

565 views • 31 slides