A Reinforcement Learning Based System for Minimizing Cloud Storage - PowerPoint PPT Presentation

Sep 24, 2022 •176 likes •360 views

A Reinforcement Learning Based System for Minimizing Cloud Storage Service Cost Haoyu Wang 1 , Haiying Shen 1 , Qi Liu 1 , Kevin Zheng 1 , and Jie Xu 2 1 University of Virginia and 2 George Mason University ICPP2020 Online presentation Web

A Reinforcement Learning Based System for Minimizing Cloud Storage Service Cost Haoyu Wang 1 , Haiying Shen 1 , Qi Liu 1 , Kevin Zheng 1 , and Jie Xu 2 1 University of Virginia and 2 George Mason University ICPP2020 Online presentation
Web application:
Type of storage User Hot Cloud Storage Cold Web Application Archive CSP: Cloud Service Provider
US West Hot $0.0055 US East Cold $0.01 Hot $0.005 Cold $0.01
Outline • How to minimize storage monetary cost • Related work • Wikipedia trace analysis • Markov decision process problem formulation • Main design • Performance evaluation • Conclusion
Minimize storage monetary cost Different price is determined by: • Storage type • Read/write operation frequencies • Storage period
Related work • Cloud storage payment minimization • Cloud resource pricing • Combining cloud providers Unlike the above methods, the goal of our method is to minimize the total payment a cloud storage service customer made to a CSP by leveraging the different types of storage provided by the CSP.
Trace analysis
Trace analysis
Trace analysis
Problem formulation Markov Decision Process M=(S,A,P,R) State space: Action space: Reward:
Main design 1. A3C algorithm used in MiniCost
Main design 2. Concurrent requested data files aggregation
Performance evaluation
Performance evaluation
Conclusion Analysis on the Wikipedia trace demonstrates that the substantial • request frequency variabilities may make it cost-inefficient for cloud storage service customer. An RL based data storage types assignment algorithm that generates • data storage types assignment plans periodically can save monetary cost significantly. Trace-driven experiment shows that our online RL based method can • achieve significant cost savings.
Thank you! Questions?

Recommend

Reinforcement Learning-Based End-to-End Parking for Automatic Parking System CS885

Reinforcement Learning-Based End-to-End Parking for Automatic Parking System CS885 Reinforcement Learning Paper by: P. Zhang, L. Xiong, Z. Yu, P. Fang, S. Yan, J. Yao, and Y. Zhou (Sensors 2019) Presented by: Neel Bhatt Context and

286 views • 14 slides

Advanced Model-Based Reinforcement Learning CS 294-112: Deep Reinforcement Learning Sergey

Advanced Model-Based Reinforcement Learning CS 294-112: Deep Reinforcement Learning Sergey Levine Class Notes 1. Homework 3 is extended by one week, to Wednesday after next Todays Lecture 1. Managing overfitting in model-based RL

686 views • 40 slides

Reinforcement Learning You can think of supervised learning as the teacher providing answers

Reinforcement Learning You can think of supervised learning as the teacher providing answers (the class labels) Reinforcement learning: In reinforcement learning, the agent learns Markov Decision Processes based on a punishment/reward

89 views • 6 slides

A Robotic Auto-Focus System based on Deep Reinforcement Learning Xiaofan Yu, Runze Yu, Jingsong

A Robotic Auto-Focus System based on Deep Reinforcement Learning Peking University A Robotic Auto-Focus System based on Deep Reinforcement Learning Xiaofan Yu, Runze Yu, Jingsong Yang, Xiaohui Duan* School of EECS, Peking University Beijing,

340 views • 21 slides

CS885 Reinforcement Learning Module 2: June 6, 2020 Maximum Entropy Reinforcement Learning

CS885 Reinforcement Learning Module 2: June 6, 2020 Maximum Entropy Reinforcement Learning Haarnoja, Tang et al. (2017) Reinforcement Learning with Deep Energy Based Policies, ICML . Haarnoja, Zhou et al. (2018) Soft Actor-Critic: Off-Policy

683 views • 24 slides

RL Overview of topics About Reinforcement Learning The Reinforcement Learning Problem

Introduction to Reinforcement Learning RL Overview of topics About Reinforcement Learning The Reinforcement Learning Problem Inside an RL agent Temporal difference learning Many faces of Reinforcement Learning What is

551 views • 35 slides

Model Based Reinforcement Learning Oriol Vinyals (DeepMind) @OriolVinyalsML May 2018 Stanford

Model Based Reinforcement Learning Oriol Vinyals (DeepMind) @OriolVinyalsML May 2018 Stanford University The Reinforcement Learning Paradigm OBSERVATIONS GOAL Agent Environment ACTIONS The Reinforcement Learning Paradigm Maximize Return -

1.26k views • 72 slides

Paths to Learning Reinforcement Learning Ever wonder how the orcas and dolphins at Sea World are

Paths to Learning Reinforcement Learning Ever wonder how the orcas and dolphins at Sea World are trained? They learn through a simple system of reinforcement learning, where the desired behaviors are met with rewards such as raw fish and back

248 views • 5 slides

Model Based Reinforcement Learning Katerina Fragkiadaki Model Anything the agent can use to

Carnegie Mellon School of Computer Science Deep Reinforcement Learning and Control Model Based Reinforcement Learning Katerina Fragkiadaki Model Anything the agent can use to predict how the environment will respond to its actions,

826 views • 65 slides

Reinforcement Learning Reinforcement Learning Reinforcement Learning in a nutshell g Imagine

Reinforcement Learning Reinforcement Learning Reinforcement Learning in a nutshell g Imagine playing a new game whose rules you dont know; after a hundred or so moves your don t know; after a hundred or so moves, your opponent announces, You

511 views • 30 slides

CSC2621 Topics in Robotics Reinforcement Learning in Robotics Week 4: Q-Value based RL Animesh

CSC2621 Topics in Robotics Reinforcement Learning in Robotics Week 4: Q-Value based RL Animesh Garg Deep Reinforcement Learning with Double Q-learning Hado van Hasselt, Arthur Guez, David Silver Dueling Network Architectures for Deep

535 views • 24 slides

Reinforcement Learning Based on Machine Learning, T. Mitchell, McGRAW Hill, 1997, ch. 13

0. Reinforcement Learning Based on Machine Learning, T. Mitchell, McGRAW Hill, 1997, ch. 13 Acknowledgement: The present slides are an adaptation of slides drawn by T. Mitchell 1. Reinforcement Learning Overview Task: Control

396 views • 24 slides

Reinforcement Learning and Simulation-Based Search David Silver Reinforcement Learning and

Reinforcement Learning and Simulation-Based Search Reinforcement Learning and Simulation-Based Search David Silver Reinforcement Learning and Simulation-Based Search Outline 1 Reinforcement Learning 2 Simulation-Based Search 3 Planning Under

425 views • 20 slides

Symbolic Regression for Reinforcement Learning and Dynamic System Modeling Robert Babuka 1

Symbolic Regression for Reinforcement Learning and Dynamic System Modeling Robert Babuka 1 Research interests Clustering for building locally linear models Reinforcement learning for continuous dynamic systems Neural

604 views • 31 slides

SOLAR: Deep Structured Representations for Model-Based Reinforcement Learning Marvin Zhang*,

SOLAR: Deep Structured Representations for Model-Based Reinforcement Learning Marvin Zhang*, Sharad Vikram*, Laura Smith, Pieter Abbeel, Matthew J Johnson, Sergey Levine UC Berkeley, UC San Diego, Google Efficient reinforcement learning from

212 views • 20 slides

Path following with reinforcement learning for autonomous cars - Mozzam Motiwala (IAS) Index

Path following with reinforcement learning for autonomous cars - Mozzam Motiwala (IAS) Index Basics of Reinforcement Learning Model Based vs Model Free Reinforcement Learning Autonomous Car collision avoidance What is Reinforcement

714 views • 24 slides

Deep he(a)p, big feat arXiv:1707.06887 A Distributional Perspective on Reinforcement Learning

Deep he(a)p, big feat arXiv:1707.06887 A Distributional Perspective on Reinforcement Learning arXiv:1702.08165 Reinforcement Learning with Deep Energy-Based Policies 1 / 25 Reinforcement Learning Environment Action Reward Interpreter State

531 views • 25 slides

Examples of Reinforcement Learning Robocup Soccer Teams Stone & Veloso, Reidmiller et al.

Reinforcement Learning Slides by Rich Sutton Mods by Dan Lizotte Refer to Reinforcement Learning: An Introduction by Sutton and Barto Alpaydin Chapter 16 Up until now we have been Supervised Learning Classifying, mostly

891 views • 31 slides

Bridging the Gap Between Value and Policy Based Reinforcement Learning Ofir Nachum, Mohammad

Bridging the Gap Between Value and Policy Based Reinforcement Learning Ofir Nachum, Mohammad Norouzi, Kelvin Xu, Dale Schuurmans Topic: Q-Value Based RL Presenter: Michael Pham-Hung Motivation Motivation i.e. Q-Learning Value Based RL +

520 views • 48 slides

Deep Reinforcement Learning 1 Outline 1. Overview of Reinforcement Learning 2. Policy Search 3.

Deep Reinforcement Learning 1 Outline 1. Overview of Reinforcement Learning 2. Policy Search 3. Policy Gradient and Gradient Estimators 4. Q-prop: Sample Efficient Policy Gradient and an Off-policy Critic 5. Model Based Planning in Discrete

769 views • 53 slides

GPU-BASED A3C FOR DEEP REINFORCEMENT LEARNING M. Babaeizadeh , , I.Frosio , S.Tyree

GPU-BASED A3C FOR DEEP REINFORCEMENT LEARNING M. Babaeizadeh , , I.Frosio , S.Tyree , J. Clemons , J.Kautz University of Illinois at Urbana-Champaign, USA NVIDIA, USA An ICLR 2017 paper A github project GPU-BASED

1.16k views • 48 slides

Reinforcement Learning Reinforcement Learning Now that you know a little about Optimal Control

Reinforcement Learning Reinforcement Learning Now that you know a little about Optimal Control Theory, you actually have some knowledge in RL. RL shares the overall goal with OCT: solving for a control policy such that the cumulative cost

610 views • 22 slides

Reinforcement Learning CS 4100: Artificial Intelligence Reinforcement Learning II Still

Reinforcement Learning CS 4100: Artificial Intelligence Reinforcement Learning II Still assume a Marko kov decision process (MDP): A se set o of st states s s S A se set o of a actions ( s (per st state) A A mo model

96 views • 5 slides

R i f R i f Reinforcement Learning III Reinforcement Learning III t L t L i i III III Dec

R i f R i f Reinforcement Learning III Reinforcement Learning III t L t L i i III III Dec 03 2008 1 Large State Spaces h When a problem has a large state space we can not longer represent the U or Q functions as explicit tables explicit

436 views • 18 slides