introduction to the course
play

Introduction to the course James Lamb Instructor DataCamp Time - PowerPoint PPT Presentation

Sep 06, 2023 •321 likes •590 views

DataCamp Time Series with data.table in R TIME SERIES WITH DATA . TABLE IN R Introduction to the course James Lamb Instructor DataCamp Time Series with data.table in R A data frame is a general-purpose data structure A data frame is not

  1. DataCamp Time Series with data.table in R TIME SERIES WITH DATA . TABLE IN R Introduction to the course James Lamb Instructor

  2. DataCamp Time Series with data.table in R A data frame is a general-purpose data structure A data frame is not something unique to R! It's a common data structure that meets these properties: List of lists All lists are of equal length Value type must be the same within each list (column) Value types can be different across columns someDF <- data.frame(x = rnorm(10), y = rep(TRUE, 100)) str(someDF) 'data.frame': 100 obs. of 2 variables: $ x: num -1.5456 -1.1905 0.6055 0.9489 0.0023 ... $ y: logi TRUE TRUE TRUE TRUE TRUE TRUE ...

  3. DataCamp Time Series with data.table in R data.table is an extension on data.frame data.frame = R's default data frame implementation data.table = extension of that base class data.table improvements: more expressive syntax more efficient memory use via pass-by-reference operators library(data.table) someDT <- data.table(x = rnorm(100), y = rep(TRUE, 100)) str(someDT) Classes ‘data.table’ and 'data.frame': 100 obs. of 2 variables: $ x: num -0.474 -0.944 0.382 -0.505 -1.128 ... $ y: logi TRUE TRUE TRUE TRUE TRUE TRUE ...

  4. DataCamp Time Series with data.table in R Selecting columns with .() You can select columns from a data.table with .() : baseballDT[, .(timestamp, winning_team)] timestamp winning_team 1: 2018-01-01 00:00:00 BOS 2: 2018-01-01 00:00:36 CWS 3: 2018-01-01 00:01:12 MIL

  5. DataCamp Time Series with data.table in R Column selection with .SD Use .SD ( S ubset of D ata) to reference a subset of columns. cols <- c("timestamp", "winning_team") baseballDT[, .SD, .SDcols = cols] This is identical: baseballDT[, .SD, .SDcols = c("timestamp", "winning_team")] "new data.table with specific columns" timestamp winning_team 1: 2018-01-01 00:00:00 BOS 2: 2018-01-01 00:00:36 CWS 3: 2018-01-01 00:01:12 MIL

  6. DataCamp Time Series with data.table in R Brief review of grep() grep() returns indexes of strings matching a pattern. grep(pattern = 'art', c('artistic', 'colorful')) [1] 1 Use value = TRUE to get values instead of indexes. grep(pattern = 'art', c('artistic', 'colorful'), value = TRUE) [1] "artistic" `

  7. DataCamp Time Series with data.table in R Using column suffixes and grep() Use column suffixes to group columns. innings_pitched_COUNT runs_allowed_COUNT era_AVERAGE 1: 10 8 7.2 2: 20 4 1.8 3: 30 22 6.6 Get just the count data count_cols <- grep('COUNT$', names(baseballDT), value = TRUE) countDT <- baseballDT[, .SD, .SDcols = count_cols] countDT innings_pitched_COUNT runs_allowed_COUNT 1: 10 8 2: 20 4 3: 30 22

  8. DataCamp Time Series with data.table in R Combining row and column selection Expressive subset statements with row selectors cols <- c("timestamp", "winning_team") baseballDT[ which.max(timestamp), .SD, .SDcols = cols ] "Get the most recent observation" timestamp winning_team 1: 2018-01-01 01:00:00 BOS

  9. DataCamp Time Series with data.table in R TIME SERIES WITH DATA . TABLE IN R Let's practice!

  10. DataCamp Time Series with data.table in R TIME SERIES WITH DATA . TABLE IN R Flexible data selection James Lamb Instructor

  11. DataCamp Time Series with data.table in R Explicit references Use direct name references in [] locDT <- data.table( cities = c("Chicago", "Boston", "Milwaukee"), ppl_mil = c(2.7, 0.673, 0.595) ) locDT[, cities] [1] "Chicago" "Boston" "Milwaukee"

  12. DataCamp Time Series with data.table in R Calling functions Functions in the i block to select rows locDT[which.max(ppl_mil)] cities ppl_mil 1: Chicago 2.7

  13. DataCamp Time Series with data.table in R Using get() get() : evaluate a string as a column reference locDT <- data.table( cities = c("Chicago", "Boston", "Milwaukee"), ppl_mil = c(2.7, 0.673, 0.595) ) city_col <- "cities" locDT[, get(city_col)] [1] "Chicago" "Boston" "Milwaukee"

  14. DataCamp Time Series with data.table in R get() is great when writing functions Write reusable functions without hard-coded column names: square_col <- function(DT, col_name){ return(DT[, get(col_name) ^ 2]) } square_col(locDT, "ppl_mil") [1] 7.290000 0.452929 0.354025

  15. DataCamp Time Series with data.table in R Using () Problem: get people in thousands from the ppl_mil column. locDT[, ppl_bil := ppl_mil * 1000] locDT[, ppl_bil] [1] 2700 673 595 But what if you want to parameterize the new column name? add_bil_ppl <- function(DT, new_name){ DT[, (new_name) := ppl_mil * 1000 } add_bil_ppl(locDT, "some_rand_name") print(locDT) cities ppl_mil some_rand_name 1: Chicago 2.700 2700 2: Boston 0.673 673 3: Milwaukee 0.595 595

  16. DataCamp Time Series with data.table in R Combining () and get() Function to create features by adding 10 to existing columns add10 <- function(DT, cols){ for (col in cols){ new_name <- paste0(col, "_plus10") DT[, (new_name) := get(col) + 10] } } add10(locDT, cols = "ppl_mil") locDT cities ppl_mil ppl_mil_plus10 1: Chicago 2.700 12.700 2: Boston 0.673 10.673 3: Milwaukee 0.595 10.595

  17. DataCamp Time Series with data.table in R Changing names with setnames() Change a single column's name: locDT <- data.table( cities = c("Chicago", "Boston", "Milwaukee"), ppl_mil = c(2.7, 0.673, 0.595) ) setnames(locDT, old = "cities", new = "city_names") names(locDT) [1] "city_names" "ppl_mil"

  18. DataCamp Time Series with data.table in R setnames() in functions Using setnames() in a function tag_important_columns <- function(DT, cols){ setnames(DT, old = cols, new = paste0(cols, "_important")) } Calling this function is efficient and doesn't copy the data! tag_important_columns(locDT, "ppl_mil") locDT cities ppl_mil_important 1: Chicago 2.700 2: Boston 0.673 3: Milwaukee 0.595

  19. DataCamp Time Series with data.table in R TIME SERIES WITH DATA . TABLE IN R Let's practice!

  20. DataCamp Time Series with data.table in R TIME SERIES WITH DATA . TABLE IN R Executing functions inside data.tables James Lamb Instructor

  21. DataCamp Time Series with data.table in R Use functions in the "i" block to select rows stockDT <- data.table( close_date = seq.POSIXt(as.POSIXct("2017-01-01"), as.POSIXct("2017-01-30"), MSFT = runif(100, 70, 80), AAPL = runif(100, 140, 180) ) Best day for Microsoft: stockDT[which.max(MSFT)] close_date MSFT AAPL 1: 2017-01-08 07:45:27 79.9235 159.9928 Final 8 hours of the dataset: stockDT[close_date > max(close_date) - 60 * 60 * 8] close_date MSFT AAPL 1: 2017-01-29 16:58:10 73.78340 157.9154 2: 2017-01-30 00:00:00 71.51727 141.8897

  22. DataCamp Time Series with data.table in R Using functions in the "j" block to summarize data cor() creates a correlation matrix between columns cor(stockDT[, .SD, .SDcols = c('AAPL', 'MSFT')]) AAPL MSFT AAPL 1.00000000 0.05680504 MSFT 0.05680504 1.00000000 You can call this directly inside a data.table ! corr_mat <- stockDT[, cor(.SD), .SDcols = c('AAPL', 'MSFT')] print(corr_mat) AAPL MSFT AAPL 1.00000000 0.05680504 MSFT 0.05680504 1.00000000

  23. DataCamp Time Series with data.table in R Use functions in the "j" block to generate new columns Add a new column: stockDT[, rand_noise := AAPL + rnorm(100)] close_date MSFT AAPL rand_noise 1: 2017-01-01 00:00:00 76.46907 163.6131 162.4594 2: 2017-01-01 07:01:49 78.68001 174.1177 174.9193

  24. DataCamp Time Series with data.table in R Using functions in the "by" block to dynamically group data Two-step process to generate "mean price by hour of the day": stockDT[, hour_of_day := as.integer(strftime(close_date, "%H"))] stockDT[, mean(AAPL), by = hour_of_day][order(hour_of_day)] hour_of_day V1 1: 0 155.4853 2: 1 163.5479 3: 2 152.5203 1-step process to generate "mean price by hour of day": stockDT[, mean(AAPL), by = .( hour_of_day = as.integer(strftime(close_date, "%H")) )][order(hour_of_day)] hour_of_day V1 1: 0 155.4853 2: 1 163.5479 3: 2 152.5203

  25. DataCamp Time Series with data.table in R Applying a function over every column with .SD Use lapply() if you want a data.table back Use sapply() if you want a vector or list back Count percent missing values by column: stockDT[, lapply(.SD, function(x){mean(is.na(x))})] close_date MSFT AAPL 1: 0 0.1 0.26 Count non-NA values: num_obs <- stockDT[, sapply(.SD, function(x){sum(!is.na(x), na.rm = TRUE)})] print(num_obs) close_date MSFT AAPL 100 90 74

  26. DataCamp Time Series with data.table in R TIME SERIES WITH DATA . TABLE IN R Let's practice!

Recommend

INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION

INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION

TRAINING COURSE ON SOCIAL PROTECTION &amp; FORMALIZATION TRINIDAD AND TOBAGO MARCH 15, 2017 INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION INTRODUCTION Design of the NIS Assistance from the

547 views • 23 slides
INTRODUCTION Introduction 2/42 INTRODUCTION Alternations I am giving bread to the

INTRODUCTION Introduction 2/42 INTRODUCTION Alternations I am giving bread to the

Patterns of variation in the expression of case and agreement Andrs Brny Leiden University Centre for Linguistics 9 November 2019, BaSIS Workshop a.barany@hum.leidenuniv.nl INTRODUCTION Introduction 2/42 INTRODUCTION

1.37k views • 42 slides
Introduction Introduction Introduction Introduction Outline Motivation Failures

Introduction Introduction Introduction Introduction Outline Motivation Failures

Introduction Introduction Introduction Introduction Outline Motivation Failures Definition and concepts Process and product properties Principles SE Approaches Motivation Software and the economy The economies of

1.03k views • 49 slides
Introduction Introduction Introduction Nationwide Cause for Concern 1

Introduction Introduction Introduction Nationwide Cause for Concern 1

Introduction Introduction Introduction Nationwide Cause for Concern 1 https://www.cdc.gov/std/stats17/infographic.htm Introduction Epidemiology in Western Colorado CPHE Surveillance Data Report, May 2018 Chlamydia Epidemiology in Western

977 views • 39 slides
DAQ introduction Purpose of this talk : (1) Introduction for those who have not been in every

DAQ introduction Purpose of this talk : (1) Introduction for those who have not been in every

DAQ introduction Purpose of this talk : (1) Introduction for those who have not been in every meeting (2) Some ideas of diagrams for section 7.1 (Introduction) of TP. Giles Barr 25 Jan 2018, Pembroke College On DUNE, the reality is there is a

474 views • 10 slides
INTRODUCTION cf. Schneider, Chapter 1 INTRODUCTION THEY ARE OUT TO GET YOU INTRODUCTION WHAT

INTRODUCTION cf. Schneider, Chapter 1 INTRODUCTION THEY ARE OUT TO GET YOU INTRODUCTION WHAT

ADVANCED COMPUTER SECURITY INTRODUCTION cf. Schneider, Chapter 1 INTRODUCTION THEY ARE OUT TO GET YOU INTRODUCTION WHAT IS SECURITY? A systems security policies describe What the system is supposed to do Store and provide access

479 views • 16 slides
Overview Introduction to SMIL Introduction to W3C and XML Introduction to SMIL

Overview Introduction to SMIL Introduction to W3C and XML Introduction to SMIL

Overview Introduction to SMIL Introduction to W3C and XML Introduction to SMIL Writing a SMIL file Adding Media Objects Martin Halvey Clipping media files January 2008 Timing and Synchronising Layout World Wide

387 views • 14 slides
14 Introduction Introduction Bad guys can put malware into Example: hosts

14 Introduction Introduction Bad guys can put malware into Example: hosts

Introduction Introduction source Encapsulation ISO/OSI reference model message M application segment H t H t M transport network datagram H n H n H t M presentation: allow applications to frame link H l H n H t M interpret

583 views • 4 slides
Previous Lecture Introduction to SMIL 2 Introduction to W3C and XML Introduction to SMIL

Previous Lecture Introduction to SMIL 2 Introduction to W3C and XML Introduction to SMIL

Previous Lecture Introduction to SMIL 2 Introduction to W3C and XML Introduction to SMIL Writing a SMIL file Adding Media Objects Martin Halvey Clipping media files January 2008 Timing and Synchronising Layout

322 views • 8 slides
Lecture 1 Andreas Habegger Introduction Zynq Introduction Zynq Introduction Zynq PS vs. PL

Lecture 1 Andreas Habegger Introduction Zynq Introduction Zynq Introduction Zynq PS vs. PL

Introduction Zynq Lecture 1 Andreas Habegger Introduction Zynq Introduction Zynq Introduction Zynq PS vs. PL Processing System Processor Peripherals Data Buses AXI Bus Conclusion BTE5380 - Embedded Systems Octobre 2014 Andreas Habegger

1.81k views • 30 slides
2018.06 01 SMILE5 Introduction S E 5 02 Alpha Cloud M I L 03 Company Introduction 04

2018.06 01 SMILE5 Introduction S E 5 02 Alpha Cloud M I L 03 Company Introduction 04

2018.06 01 SMILE5 Introduction S E 5 02 Alpha Cloud M I L 03 Company Introduction 04 Project References S E 5 PART.1 SMILE5 Introduction M I L Introduction STORION-SMILE5 Residential Energy Storage System Inverter 5kW Output Input

1.06k views • 25 slides
Introduction to CICS Course introduction Course introduction What is CICS? What is an

Introduction to CICS Course introduction Course introduction What is CICS? What is an

Introduction to CICS Course introduction Course introduction What is CICS? What is an application server? Why use an application server? Course introduction Services provided by CICS How CICS applications are defined Scaling CICS to meet

1.84k views • 146 slides
Network Visualization Introduction Presented by Shahed Introduction Introduction Basic

Network Visualization Introduction Presented by Shahed Introduction Introduction Basic

Network Visualization Introduction Presented by Shahed Introduction Introduction Basic building blocks Node Links (relationship between nodes) Spatial information Network data

584 views • 9 slides
Introduction Introduction (1) Main argument of the paper: universities are not only

Introduction Introduction (1) Main argument of the paper: universities are not only

Le jugement des pairs comme outil de management des universits Peer-review as a management tool Christine Musselin (CSO, Sciences Po et CNRS) Avril 2013, Lirrsistible ascension du capitalisme acadmique Introduction Introduction (1)

590 views • 28 slides
Introduction Introduction to storage and to storage and filesystems filesystems Introduction

Introduction Introduction to storage and to storage and filesystems filesystems Introduction

Moreno Baricevic Gilberto Daz Axel Kohlmeyer Stefano Cozzini ULA ICTP CNR-IOM DEMOCRITOS Merida, VENEZUELA Trieste, ITALY Trieste, ITALY Introduction Introduction to storage and to storage and filesystems filesystems Introduction

1.19k views • 51 slides
JABLOTRON JA-100 introduction November 2011 Introduction Todays goals First introduction of

JABLOTRON JA-100 introduction November 2011 Introduction Todays goals First introduction of

JABLOTRON JA-100 introduction November 2011 Introduction Todays goals First introduction of the brand new JA-100 alarm system Visions Basic architecture System components Setting Help us with final validations You are the FIRST ones

1.69k views • 111 slides
1 Introduction Introduction Communication System Baseband an adjective that describes

1 Introduction Introduction Communication System Baseband an adjective that describes

Introduction Introduction Physical Layer Mobile network Global ISP Provides the means to transmit bits Basic Concepts of from sender to receiver, Data Transmission Defines the mechanical, electrical, and Home network timing

463 views • 7 slides
Welcome! Todays Agenda: Topic Introduction Course Introduction Team Practical

Welcome! Todays Agenda: Topic Introduction Course Introduction Team Practical

INFOGR Computer Graphics Jacco Bikker - April-July 2015 - Lecture 1: Introduction Welcome! Todays Agenda: Topic Introduction Course Introduction Team Practical Details Assignments Field Study State of

869 views • 62 slides
Introduction (00) RNDr. Martin Madaras, PhD. madaras@skeletex.xyz Introduction Introduction

Introduction (00) RNDr. Martin Madaras, PhD. madaras@skeletex.xyz Introduction Introduction

Virtual and Augmented Reality Introduction (00) RNDr. Martin Madaras, PhD. madaras@skeletex.xyz Introduction Introduction Who am I? What do I do here? 2 About me Short research bio: - 2014 - finished PhD at FMFI UK - 2014 2016 -

1.3k views • 39 slides
PRESENTATION The Introduction What is the Introduction ? The introduction prepares the audience

PRESENTATION The Introduction What is the Introduction ? The introduction prepares the audience

PRESENTATION The Introduction What is the Introduction ? The introduction prepares the audience for your presentation. It tells them what your presentation is about, why it is important, and finally, what to listen for in your speech. Get the

505 views • 3 slides
How to make a presentation with L A T EX? Introduction to L A T EX Introduction to Beamer

How to make a presentation with L A T EX? Introduction to L A T EX Introduction to Beamer

Introduction to Beamer Hafida Benhidour How to make a presentation with L A T EX? Introduction to L A T EX Introduction to Beamer Introduction to Beamer Hafida Benhidour Department of computer science King Saud University November

732 views • 49 slides
Shenzhen Cuilu jewelry Co., Ltd was founded in 1996 and its a large private enterprise

Shenzhen Cuilu jewelry Co., Ltd was founded in 1996 and its a large private enterprise

Introduction Introduction - - Brief Brief Introduction Introduction of of Cuilu Cuilu Group Group Introduction Introduction - - Brief Brief Introduction Introduction of of Cuilu Cuilu Group Group Shenzhen Cuilu jewelry Co., Ltd

147 views • 14 slides
Introduction to ICS- -214 214 Introduction to ICS Official Unit / Incident Log - A V-C-N.org

Introduction to ICS- -214 214 Introduction to ICS Official Unit / Incident Log - A V-C-N.org

V olunteer C ommunications N etwork (V-C-N.org) Introduction to ICS- -214 214 Introduction to ICS Official Unit / Incident Log - A V-C-N.org Academy Course - 2012-02.24-2208 262: Introduction to ICS-214 : V-C-N.org : Slide: 1 V olunteer C

735 views • 34 slides
Introduction to ICS- -214 214 Introduction to ICS Official Unit / Incident Log - A V-C-N.org

Introduction to ICS- -214 214 Introduction to ICS Official Unit / Incident Log - A V-C-N.org

V olunteer C ommunications N etwork (V-C-N.org) Introduction to ICS- -214 214 Introduction to ICS Official Unit / Incident Log - A V-C-N.org Academy Course - 2012-02.24-2208 262: Introduction to ICS-214 : V-C-N.org : Slide: 1 V olunteer C

558 views • 18 slides

More recommend