flyte mm a software based
play

Flyte-MM: A Software Based Sub-Floating Point GEMM Richard Veras - PowerPoint PPT Presentation

Dec 21, 2023 •111 likes •205 views

Flyte-MM: A Software Based Sub-Floating Point GEMM Richard Veras (Louisiana State University) David Gregg (Trinity College, Dublin) Want GEMM wit ith Hig igh Dynamic Range but Low Precision Targeting: Machine Have: Large Datasets and Need:

  1. Flyte-MM: A Software Based Sub-Floating Point GEMM Richard Veras (Louisiana State University) David Gregg (Trinity College, Dublin)

  2. Want GEMM wit ith Hig igh Dynamic Range but Low Precision Targeting: Machine Have: Large Datasets and Need: High Dynamic Range, Learning and Analytics. Constrained Systems. Not High Precision.

  3. Flyt lytes Preserve Dynamic Range at Reduced Space Float 32 (IEEE 754): Sign Exponent Mantissa 1 1 0 0 1 0 1 0 1 1 0 1 1 0 1 0 0 0 0 1 1 1 0 0 0 0 1 1 1 0 0 1 Flyte 24: 1 1 0 0 1 0 1 0 1 1 0 1 1 0 1 0 0 0 0 1 1 1 0 0 Flyte 16: 1 1 0 0 1 0 1 0 1 1 0 1 1 0 1 0 Flyte 16 to Float 32: 1 1 0 0 1 0 1 0 1 1 0 1 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 [Anderson, A., Muralidharan, S., Gregg, D. IEEE TOCS 2017]

  4. Efficiently Converting Flo loats to Flyt lyte in in GEMM BLIS Algorithm Convert in A/B in Pack: Fast, but cache block of float32. for( i = 0 .. mc ) Pack-b for( p = 0 .. kc ) buffer_A[buff_addr(i,p)] = flYTE_to_flOAT( A[a_addr(i,p)] ); Pack-a Convert in A in Kernel: Slow, but cache block of Flytes. SIMD Load and Convert: Kern-a SIMD Broadcast and Convert: Kern-b [van Zee, F., et. al. TOMS 2013]

  5. A Famil ily of f Im Implementations Software Based Sub- precision at little cost!

  6. A Famil ily of f Im Implementations Load Convert is cheaper than Broadcast Convert.

  7. Exploring the parameters for each strategy. PackA-Pack B uses same block Smaller data format for blocks allows for bigger block size as SGEMM, 6x16. sizes (96x256[SGEMM] vs. 128x256[FLYTE24]). 1 2 3 4 5 6 0 16 32 48 64 80 96 112 128 144 160 176 192 208 224 240 256 272 288 304 320 336 352 368 384 400 416 432 448 464 480 496 8 0.1 0.15 0.3 0.4 0.54 0.15 16 0.22 0.3 0.34 0.37 0.35 0.35 0.37 0.39 0.38 0.39 0.38 0.38 0.38 0.38 0.38 0.38 0.38 0.38 0.35 0.38 0.37 0.35 0.35 0.38 0.35 0.36 0.35 0.36 0.38 0.38 0.38 32 0.21 0.31 0.35 0.37 0.38 0.35 0.36 0.37 0.34 0.35 0.35 0.36 0.36 0.34 0.38 0.39 0.31 0.39 0.38 0.38 0.37 0.39 0.39 0.38 0.32 0.3 0.34 0.3 0.34 0.34 0.35 16 0.12 0.17 0.58 0.79 0.93 1 48 0.24 0.26 0.27 0.3 0.34 0.38 0.38 0.39 0.38 0.36 0.36 0.36 0.34 0.38 0.37 0.38 0.36 0.36 0.37 0.36 0.37 0.37 0.37 0.37 0.35 0.35 0.36 0.36 0.36 0.37 0.36 24 0.14 0.18 0.64 0.97 0.54 0.56 64 0.29 0.31 0.36 0.35 0.35 0.36 0.35 0.36 0.36 0.36 0.38 0.37 0.38 0.36 0.35 0.38 0.38 0.37 0.36 0.3 0.31 0.32 0.38 0.38 0.38 0.35 0.38 0.38 0.31 0.35 0.32 80 0.24 0.33 0.35 0.33 0.31 0.25 0.34 0.34 0.32 0.36 0.37 0.37 0.36 0.36 0.36 0.36 0.37 0.38 0.38 0.38 0.38 0.39 0.38 0.37 0.34 0.33 0.33 0.37 0.35 0.36 0.36 32 0.08 0.14 0.69 0.48 0.55 0.37 96 0.25 0.3 0.35 0.37 0.37 0.38 0.38 0.39 0.38 0.38 0.38 0.39 0.39 0.39 0.39 0.39 0.39 0.39 0.39 0.38 0.39 0.38 0.39 0.39 0.38 0.38 0.38 0.36 0.39 0.38 0.38 112 0.29 0.35 0.35 0.38 0.38 0.38 0.38 0.39 0.38 0.38 0.39 0.38 0.39 0.39 0.39 0.39 0.39 0.39 0.39 0.39 0.39 0.39 0.39 0.39 0.38 0.39 0.39 0.39 0.39 0.37 0.38 40 0.08 0.14 0.66 0.55 0.48 0.32 128 0.3 0.35 0.36 0.38 0.38 0.38 0.38 0.39 0.38 0.39 0.39 0.39 0.38 0.39 0.38 0.4 0.39 0.39 0.38 0.38 0.38 0.39 0.39 0.39 0.38 0.39 0.38 0.38 0.38 0.38 0.38 48 0.08 0.11 0.56 0.42 0.47 0.4 144 0.31 0.35 0.36 0.37 0.38 0.38 0.38 0.39 0.38 0.38 0.39 0.39 0.39 0.38 0.39 0.39 0.39 0.38 0.39 0.38 0.39 0.38 0.39 0.39 0.38 0.37 0.32 0.32 0.35 0.35 0.35 160 0.26 0.34 0.36 0.37 0.37 0.36 0.34 0.36 0.3 0.31 0.37 0.36 0.35 0.38 0.38 0.37 0.35 0.36 0.36 0.37 0.38 0.37 0.38 0.37 0.37 0.34 0.32 0.31 0.35 0.36 0.37 176 0.3 0.32 0.34 0.35 0.35 0.35 0.37 0.36 0.37 0.35 0.37 0.36 0.37 0.36 0.38 0.38 0.37 0.36 0.36 0.36 0.36 0.38 0.38 0.36 0.35 0.35 0.36 0.37 0.36 0.37 0.35 192 0.28 0.34 0.36 0.37 0.35 0.37 0.3 0.38 0.35 0.32 0.36 0.34 0.38 0.36 0.36 0.37 0.34 0.27 0.36 0.32 0.34 0.38 0.39 0.34 0.36 0.32 0.34 0.29 0.35 0.35 0.37 208 0.29 0.34 0.36 0.37 0.37 0.36 0.35 0.38 0.36 0.32 0.36 0.34 0.36 0.38 0.33 0.31 0.34 0.36 0.32 0.32 0.36 0.36 0.38 0.35 0.35 0.34 0.35 0.36 0.35 0.37 0.37 224 0.28 0.34 0.36 0.31 0.34 0.37 0.29 0.32 0.3 0.34 0.31 0.36 0.37 0.33 0.38 0.31 0.37 0.37 0.35 0.37 0.34 0.38 0.38 0.38 0.38 0.34 0.37 0.37 0.35 0.36 0.35 240 0.27 0.3 0.34 0.36 0.36 0.36 0.35 0.37 0.36 0.36 0.35 0.37 0.34 0.38 0.37 0.36 0.35 0.37 0.36 0.38 0.35 0.37 0.37 0.37 0.31 0.25 0.29 0.31 0.35 0.37 0.36 256 0.3 0.32 0.31 0.35 0.33 0.37 0.36 0.34 0.36 0.34 0.37 0.37 0.36 0.37 0.37 0.37 0.38 0.36 0.36 0.36 0.38 0.38 0.37 0.37 0.37 0.36 0.37 0.37 0.37 0.34 0.36 PackA-KernB uses MrxNr 3x32. 272 0.24 0.31 0.33 0.32 0.32 0.37 0.36 0.39 0.37 0.38 0.38 0.32 0.32 0.31 0.32 0.36 0.38 0.37 0.37 0.37 0.37 0.37 0.36 0.36 0.36 0.37 0.36 0.36 0.36 0.36 0.35 288 0.3 0.34 0.36 0.37 0.37 0.37 0.37 0.38 0.33 0.37 0.38 0.38 0.37 0.37 0.38 0.39 0.38 0.38 0.38 0.37 0.38 0.37 0.38 0.37 0.38 0.37 0.37 0.38 0.37 0.37 0.37 1 2 3 4 5 6 304 0.29 0.35 0.36 0.36 0.31 0.34 0.35 0.35 0.31 0.34 0.37 0.38 0.38 0.39 0.37 0.34 0.37 0.37 0.37 0.38 0.36 0.36 0.36 0.33 0.38 0.35 0.35 0.33 0.32 0.31 0.37 320 0.3 0.32 0.35 0.37 0.38 0.38 0.38 0.38 0.37 0.36 0.38 0.38 0.38 0.38 0.31 0.36 0.34 0.36 0.36 0.37 0.38 0.37 0.37 0.38 0.38 0.37 0.38 0.38 0.38 0.38 0.37 8 0.1 0.15 0.32 0.41 0.39 0.3 336 0.3 0.35 0.36 0.36 0.36 0.37 0.36 0.38 0.32 0.37 0.33 0.37 0.32 0.37 0.38 0.39 0.38 0.38 0.37 0.38 0.32 0.38 0.38 0.38 0.38 0.35 0.38 0.38 0.37 0.38 0.37 352 0.3 0.35 0.34 0.34 0.37 0.37 0.37 0.36 0.37 0.38 0.38 0.37 0.38 0.37 0.3 0.37 0.38 0.38 0.37 0.38 0.36 0.38 0.38 0.37 0.35 0.33 0.35 0.36 0.38 0.37 0.37 16 0.17 0.16 0.6 0.69 0.72 0.72 368 0.3 0.34 0.36 0.37 0.37 0.37 0.37 0.38 0.37 0.37 0.37 0.38 0.37 0.38 0.38 0.39 0.38 0.38 0.37 0.38 0.37 0.33 0.38 0.34 0.37 0.38 0.38 0.32 0.34 0.37 0.37 384 0.3 0.34 0.35 0.37 0.36 0.37 0.37 0.38 0.38 0.38 0.37 0.37 0.37 0.38 0.38 0.38 0.38 0.38 0.38 0.38 0.37 0.38 0.38 0.37 0.38 0.38 0.37 0.38 0.37 0.37 0.38 24 0.14 0.19 0.58 0.77 0.75 0.57 400 0.3 0.34 0.35 0.38 0.37 0.38 0.38 0.39 0.37 0.37 0.38 0.38 0.38 0.38 0.38 0.39 0.38 0.32 0.37 0.33 0.3 0.31 0.31 0.34 0.37 0.37 0.36 0.38 0.37 0.37 0.37 32 0.11 0.17 1 0.79 0.47 0.37 416 0.29 0.34 0.35 0.35 0.35 0.34 0.36 0.38 0.38 0.35 0.36 0.38 0.38 0.38 0.37 0.38 0.38 0.38 0.38 0.38 0.37 0.38 0.38 0.38 0.38 0.38 0.37 0.38 0.37 0.38 0.37 432 0.3 0.34 0.36 0.37 0.37 0.37 0.38 0.38 0.38 0.38 0.38 0.39 0.38 0.39 0.37 0.39 0.38 0.38 0.38 0.38 0.37 0.38 0.38 0.38 0.38 0.38 0.38 0.38 0.38 0.37 0.38 40 0.1 0.17 0.7 0.57 0.48 0.32 448 0.3 0.35 0.35 0.34 0.35 0.36 0.36 0.37 0.37 0.37 0.38 0.38 0.39 0.39 0.36 0.34 0.37 0.38 0.32 0.26 0.33 0.33 0.35 0.32 0.32 0.36 0.31 0.37 0.34 0.32 0.38 464 0.3 0.34 0.35 0.37 0.37 0.33 0.33 0.35 0.36 0.38 0.38 0.33 0.37 0.34 0.37 0.38 0.37 0.38 0.37 0.3 0.37 0.37 0.38 0.37 0.38 0.38 0.36 0.31 0.35 0.3 0.29 48 0.09 0.19 0.81 0.52 0.43 0.57 480 0.31 0.34 0.36 0.37 0.37 0.37 0.37 0.38 0.37 0.37 0.37 0.37 0.38 0.37 0.38 0.38 0.38 0.38 0.38 0.38 0.37 0.39 0.37 0.38 0.38 0.37 0.38 0.38 0.37 0.37 0.37 496 0.31 0.35 0.35 0.36 0.38 0.37 0.36 0.38 0.37 0.37 0.37 0.37 0.37 0.37 0.38 0.38 0.38 0.37 0.37 0.37 0.36 0.38 0.38 0.38 0.35 0.37 0.37 0.37 0.37 0.36 0.37

  8. Summary ry • Flytes are software based sub-precision Floating Point. • Keep the sign and exponent and truncate the mantissa . • Flyte based GEMM desirable for machine learning and analytics . • Family of implementations for computing on Flytes. • BLIS provides various opportunities to cast datatypes. • Code Generation makes kernel implementation trivial . • Opportunity for identifying optimal parameters . • Kernel throughput varies with register block dimensions . • Cache blocking parameters affected by datatype size .

Recommend

RCKMS - Technology The Thinking Behind RCKMS Software Build a software suite based on an open

RCKMS - Technology The Thinking Behind RCKMS Software Build a software suite based on an open

RCKMS - Technology The Thinking Behind RCKMS Software Build a software suite based on an open source software, best practices, and standards based principles, incorporating the following components: 1. Generalpurpose Public Health Decision

309 views • 13 slides
Configurable software- -based based Configurable software edge router architecture edge router

Configurable software- -based based Configurable software edge router architecture edge router

Configurable software- -based based Configurable software edge router architecture edge router architecture Wajdi LOUATI Badii JOUABER Djamal ZEGHLACHE Institut National des Tlcommunications 4 th Workshop on Applications and Services in

301 views • 17 slides
Software Methodologies for distributed systems Policy-based Management (PBM) Software

Software Methodologies for distributed systems Policy-based Management (PBM) Software

A Fuzzy, Utility-based Approach for Proactive Policy-based Management Christoph Frenzel, Henning Sanneck, and Bernhard Bauer RuleML 2013, July 11 13, Seattle, WA, USA Software Methodologies for distributed systems Policy-based Management

195 views • 16 slides
CSEP504: Advanced topics in software systems Software architecture: design-based Formal

CSEP504: Advanced topics in software systems Software architecture: design-based Formal

CSEP504: Advanced topics in software systems Software architecture: design-based Formal reasoning about properties Static vs. dynamic architectures Software architecture: property-based Autonomic systems Relationship to

1.46k views • 79 slides
State-based Behavior I SWEN-261 Introduction to Software Engineering Department of Software

State-based Behavior I SWEN-261 Introduction to Software Engineering Department of Software

State-based Behavior I SWEN-261 Introduction to Software Engineering Department of Software Engineering Rochester Institute of Technology A large part of software behavior is dependent on the state of the system, i.e. state-based. A

115 views • 9 slides
Reuse-based & Choreography-based Distributed Systems Reuse-based Software Engineering

Reuse-based & Choreography-based Distributed Systems Reuse-based Software Engineering

Reuse-based & Choreography-based Distributed Systems Reuse-based Software Engineering Distributed Systems Mar Marco Autili BPMN2 Choreographies basics University of LAquila Invited Seminar Lecture 1 Course: Advanced

724 views • 59 slides
Component- -based, Context based, Context- -aware aware Component Software Systems Software

Component- -based, Context based, Context- -aware aware Component Software Systems Software

Component- -based, Context based, Context- -aware aware Component Software Systems Software Systems Workshop on Spontaneous Networking Rutgers University Michael Przybilski 1 10.05.2006 michael.przybilski@cs.helsinki.fi Outline

593 views • 19 slides
Challenges and Opportunities of Big Software-based Innovation Jurgen J. Vinju Centrum Wiskunde

Challenges and Opportunities of Big Software-based Innovation Jurgen J. Vinju Centrum Wiskunde

S oftware Analysis And Transformation Challenges and Opportunities of Big Software-based Innovation Jurgen J. Vinju Centrum Wiskunde & Informatica TU Eindhoven INRIA Lille Big Software Matchmaking Day July 1st, 2015 Go Big Software!

224 views • 18 slides
State-based Behavior II SWEN-261 Introduction to Software Engineering Department of Software

State-based Behavior II SWEN-261 Introduction to Software Engineering Department of Software

State-based Behavior II SWEN-261 Introduction to Software Engineering Department of Software Engineering Rochester Institute of Technology There are a number of benefits for explicitly modeling state-based behavior. Explicitly defining

141 views • 13 slides
Evidence-Based Software Engineering Barbara Kitchenham Tore Dyb (SINTEF) Magne Jrgensen

Evidence-Based Software Engineering Barbara Kitchenham Tore Dyb (SINTEF) Magne Jrgensen

Evidence-Based Software Engineering Barbara Kitchenham Tore Dyb (SINTEF) Magne Jrgensen (Simula Laboratory) 1 Agenda The evidence-based paradigm Evidence-Based Software Engineering (EBSE) Goals Procedures Comparison

390 views • 25 slides
! MDA$based*Teaching*and* ! MDA based Reaserch ! Sample 1: Empirical Research

! MDA$based*Teaching*and* ! MDA based Reaserch ! Sample 1: Empirical Research

Plan ! Myself ! Model Driven Architecture ! MDA$based*Teaching*and* ! MDA based Reaserch ! Sample 1: Empirical Research Research*in*Software*Engineering* ! ! MDA based Teaching ! Sample 2: Software Development Course Ludwik!Kuniar z!

372 views • 7 slides
To Preserve or Not to Preserve Invalid Solutions in Search-Based Software Engineering: A Case

To Preserve or Not to Preserve Invalid Solutions in Search-Based Software Engineering: A Case

To Preserve or Not to Preserve Invalid Solutions in Search-Based Software Engineering: A Case Study in Cloud Cost Optimization Jianmei Guo Alibaba Group 2018.11.17 @CSBSE [Guo and Shi, ICSE18] 1 Search-based software engineering (SBSE) 2

231 views • 20 slides
Language-based methods for software security Gilles Barthe IMDEA Software, Madrid, Spain Part 1

Language-based methods for software security Gilles Barthe IMDEA Software, Madrid, Spain Part 1

Language-based methods for software security Gilles Barthe IMDEA Software, Madrid, Spain Part 1 Motivation Mobile code is ubiquitous: large distributed networks of JVM devices aimed at providing a global and uniform access to services provide

1.52k views • 136 slides
Playing with Binary Analysis Deobfuscation of VM based software protection Jonathan Salwan,

Playing with Binary Analysis Deobfuscation of VM based software protection Jonathan Salwan,

Playing with Binary Analysis Deobfuscation of VM based software protection Jonathan Salwan, Sbastien Bardin and Marie-Laure Potet SSTIC 2017 Topic Binary protection Virtualization-based software protection Automatic

933 views • 72 slides
Search-Based Genetic Optimization for Deployment and Reconfiguration of Software in the Cloud

Search-Based Genetic Optimization for Deployment and Reconfiguration of Software in the Cloud

Search-Based Genetic Optimization for Deployment and Reconfiguration of Software in the Cloud 35th International Conference on Software Engineering (ICSE 2013) San Francisco, CA, USA Sren Frey, Florian Fittkau, and Wilhelm Hasselbring

676 views • 41 slides
Web 2.0 Design Audio Video IM Convergence CSS Pay Per Click Software Engineering UMTS

Web 2.0 Design Audio Video IM Convergence CSS Pay Per Click Software Engineering UMTS

Based on: Pressman, Software Engineering, chapters 16+ Based on: Norrie and Grossniklaus, Web Engineering, ETH Zrich Wikis Aggregators Folksonomy http://www.globis.ethz.ch/education/ Joy of Use User Centered Blogs

413 views • 30 slides
Developing like there is no tomorrow Software as a Circus Frank Lyaruu Est. 2001 based in

Developing like there is no tomorrow Software as a Circus Frank Lyaruu Est. 2001 based in

Developing like there is no tomorrow Software as a Circus Frank Lyaruu Est. 2001 based in Amsterdam Enterprise software for large companies Heavily invested in Java & OSGi Focused on sports and entertainment Senior

469 views • 33 slides
SMT-based Software Model Checking: Experimental Comparison of Four Algorithms Matthias Dangl

SMT-based Software Model Checking: Experimental Comparison of Four Algorithms Matthias Dangl

SMT-based Software Model Checking: Experimental Comparison of Four Algorithms Matthias Dangl Joint work with Dirk Beyer University of Passau, Germany SMT-based Software Model Checking Bounded Model Checking ( Cbmc , CPAchecker , Esbmc ,

496 views • 26 slides
Software Design Software Engineering Software Engineering Andreas Zeller Saarland University

Software Design Software Engineering Software Engineering Andreas Zeller Saarland University

These slides are based on Grady Booch: Object-Oriented Analysis and Design (1998), updated from various sources Principles of Software Design Software Engineering Software Engineering Andreas Zeller Saarland University The Challenge

685 views • 23 slides
Software-based Fault Tolerance Mission (Im)possible? Peter Ulbrich The 29th CREST Open

Software-based Fault Tolerance Mission (Im)possible? Peter Ulbrich The 29th CREST Open

Software-based Fault Tolerance Mission (Im)possible? Peter Ulbrich The 29th CREST Open Workshop on Software Redundancy November 18, 2013 System Software Group

1.02k views • 47 slides
Distributed Applications Software Engineering 2017 Alessio Gambi - Saarland University Based on

Distributed Applications Software Engineering 2017 Alessio Gambi - Saarland University Based on

Distributed Applications Software Engineering 2017 Alessio Gambi - Saarland University Based on the work of Cesare Pautasso, Christoph Dorn, and other sources ReCap Software Architecture A software systems architecture is the set of

909 views • 78 slides
Web Applications Software Engineering 2017 Alessio Gambi - Saarland University Based on the work

Web Applications Software Engineering 2017 Alessio Gambi - Saarland University Based on the work

Web Applications Software Engineering 2017 Alessio Gambi - Saarland University Based on the work of Cesare Pautasso, Christoph Dorn, Andrea Arcuri, and others ReCap Software Architecture A software systems architecture is the set of

1.42k views • 108 slides
Performance Simulation of Runtime Reconfigurable Component-Based Software Architectures Robert von

Performance Simulation of Runtime Reconfigurable Component-Based Software Architectures Robert von

Performance Simulation of Runtime Reconfigurable Component-Based Software Architectures Robert von Massow , Andr van Hoorn , and Wilhelm Hasselbring Software Engineering Group http://se.informatik.uni-kiel.de/ Dept. of Computer Science

842 views • 58 slides
Software Reliability Estimation Based on Static Error Detection M. Moiseev, M. Glukhikh , A.

Software Reliability Estimation Based on Static Error Detection M. Moiseev, M. Glukhikh , A.

7 th Central and Eastern European Software Engineering Conference in Russia - CEE-SECR 2011 October 31 November 3, Moscow Software Reliability Estimation Based on Static Error Detection M. Moiseev, M. Glukhikh , A. Karpenko, H. Richter

884 views • 32 slides

More recommend