inde x objects and labeled data
play

Inde x objects and labeled data MAN IP U L ATIN G DATAFR AME S W - PowerPoint PPT Presentation

Sep 20, 2023 •245 likes •525 views

Inde x objects and labeled data MAN IP U L ATIN G DATAFR AME S W ITH PAN DAS Anaconda Instr u ctor pandas data str u ct u res Ke y b u ilding blocks Index es : Seq u ence of labels Imm u table ( Like dictionar y ke y s ) Homogeneo u s in data t

  1. Inde x objects and labeled data MAN IP U L ATIN G DATAFR AME S W ITH PAN DAS Anaconda Instr u ctor

  2. pandas data str u ct u res Ke y b u ilding blocks Index es : Seq u ence of labels Imm u table ( Like dictionar y ke y s ) Homogeneo u s in data t y pe ( Like N u mP y arra y s ) Series : 1 D arra y w ith Inde x DataFrame s : 2 D arra y w ith Series as col u mns MANIPULATING DATAFRAMES WITH PANDAS

  3. Creating a Series import pandas as pd prices = [10.70, 10.86, 10.74, 10.71, 10.79] shares = pd.Series(prices) print(shares) 0 10.70 1 10.86 2 10.74 3 10.71 4 10.79 dtype: float64 MANIPULATING DATAFRAMES WITH PANDAS

  4. Creating an inde x days = ['Mon', 'Tue', 'Wed', 'Thur', 'Fri'] shares = pd.Series(prices, index=days) print(shares) Mon 10.70 Tue 10.86 Wed 10.74 Thur 10.71 Fri 10.79 dtype: float64 MANIPULATING DATAFRAMES WITH PANDAS

  5. E x amining an inde x print(shares.index) print(shares.index[-2:]) Index(['Mon', 'Tue', 'Wed', Index(['Thur', 'Fri'], 'Thur', 'Fri'], dtype='object') dtype='object') print(shares.index.name) print(shares.index[2]) None Wed print(shares.index[:2]) Index(['Mon', 'Tue'], dtype='object') MANIPULATING DATAFRAMES WITH PANDAS

  6. Modif y ing inde x name shares.index.name = 'weekday' print(shares) weekday Monday 10.70 Tuesday 10.86 Wednesday 10.74 Thursday 10.71 Friday 10.79 dtype: float64 MANIPULATING DATAFRAMES WITH PANDAS

  7. Modif y ing inde x entries shares.index[2] = 'Wednesday' TypeError: Index does not support mutable operations shares.index[:4] = ['Monday', 'Tuesday', 'Wednesday', 'Thursday'] TypeError: Index does not support mutable operations MANIPULATING DATAFRAMES WITH PANDAS

  8. Modif y ing all inde x entries shares.index = ['Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday'] print(shares) Monday 10.70 Tuesday 10.86 Wednesday 10.74 Thursday 10.71 Friday 10.79 dtype: float64 MANIPULATING DATAFRAMES WITH PANDAS

  9. Unemplo y ment data unemployment = pd.read_csv('Unemployment.csv') unemployment.head() Zip unemployment participants 0 1001 0.06 13801 1 1002 0.09 24551 2 1003 0.17 11477 3 1005 0.10 4086 4 1007 0.05 11362 MANIPULATING DATAFRAMES WITH PANDAS

  10. Unemplo y ment data unemployment.info() <class 'pandas.core.frame.DataFrame'> RangeIndex: 33120 entries, 0 to 33119 Data columns (total 3 columns): Zip 33120 non-null int64 unemployment 32556 non-null float64 particpants 33120 non-null int64 dtypes: float64(1), int64(2) memory usage: 776.3 KB MANIPULATING DATAFRAMES WITH PANDAS

  11. Assigning the inde x unemployment.index = unemployment['Zip'] unemployment.head() Zip unemployment participants Zip 1001 1001 0.06 13801 1002 1002 0.09 24551 1003 1003 0.17 11477 1005 1005 0.10 4086 1007 1007 0.05 11362 MANIPULATING DATAFRAMES WITH PANDAS

  12. Remo v ing e x tra col u mn unemployment.head(3) Zip unemployment participants Zip 1001 1001 0.06 13801 1002 1002 0.09 24551 1003 1003 0.17 11477 del unemployment['Zip'] unemployment.head(3) unemployment participants Zip 1001 0.06 13801 1002 0.09 24551 1003 0.17 11477 MANIPULATING DATAFRAMES WITH PANDAS

  13. E x amining inde x and col u mns print(unemployment.index) print(type(unemployment.index)) Int64Index([1001, 1002, 1003, ...], <class dtype='int64', 'pandas.indexes.numeric.Int64Index'> name='Zip', length=33120) print(unemployment.index.name) print(unemployment.columns) Zip Index(['unemployment', 'participants'], dtype='object') MANIPULATING DATAFRAMES WITH PANDAS

  14. read _ cs v() w ith inde x_ col unemployment = pd.read_csv('Unemployment.csv', index_col='Zip') unemployment.head() unemployment participants Zip 1001 0.06 13801 1002 0.09 24551 1003 0.17 11477 1005 0.10 4086 1007 0.05 11362 MANIPULATING DATAFRAMES WITH PANDAS

  15. Let ' s practice ! MAN IP U L ATIN G DATAFR AME S W ITH PAN DAS

  16. Hierarchical Inde x ing MAN IP U L ATIN G DATAFR AME S W ITH PAN DAS Anaconda Instr u ctor

  17. Stock data import pandas as pd stocks = pd.read_csv('datasets/stocks.csv') print(stocks) Date Close Volume Symbol 0 2016-10-03 31.50 14070500 CSCO 1 2016-10-03 112.52 21701800 AAPL 2 2016-10-03 57.42 19189500 MSFT 3 2016-10-04 113.00 29736800 AAPL 4 2016-10-04 57.24 20085900 MSFT 5 2016-10-04 31.35 18460400 CSCO 6 2016-10-05 57.64 16726400 MSFT 7 2016-10-05 31.59 11808600 CSCO 8 2016-10-05 113.05 21453100 AAPL MANIPULATING DATAFRAMES WITH PANDAS

  18. Setting inde x stocks = stocks.set_index(['Symbol', 'Date']) print(stocks) Close Volume Symbol Date CSCO 2016-10-03 31.50 14070500 AAPL 2016-10-03 112.52 21701800 MSFT 2016-10-03 57.42 19189500 AAPL 2016-10-04 113.00 29736800 MSFT 2016-10-04 57.24 20085900 CSCO 2016-10-04 31.35 18460400 MSFT 2016-10-05 57.64 16726400 CSCO 2016-10-05 31.59 11808600 AAPL 2016-10-05 113.05 21453100 MANIPULATING DATAFRAMES WITH PANDAS

  19. print(stocks.index) MultiIndex(levels=[['AAPL', 'CSCO', 'MSFT'], ['2016-10-03', '2016-10-04', ‘2016-10-05']], labels=[[1, 0, 2, 0, 2, 1, 2, 1, 0], [0, 0, 0, 1, 1, 1, 2, 2, 2]], names=['Symbol', 'Date']) print(stocks.index.name) None print(stocks.index.names) ['Symbol', 'Date'] MANIPULATING DATAFRAMES WITH PANDAS

  20. Sorting inde x stocks = stocks.sort_index() print(stocks) Close Volume Symbol Date AAPL 2016-10-03 112.52 21701800 2016-10-04 113.00 29736800 2016-10-05 113.05 21453100 CSCO 2016-10-03 31.50 14070500 2016-10-04 31.35 18460400 2016-10-05 31.59 11808600 MSFT 2016-10-03 57.42 19189500 2016-10-04 57.24 20085900 2016-10-05 57.64 16726400 MANIPULATING DATAFRAMES WITH PANDAS

  21. Inde x ing ( indi v id u al ro w) stocks.loc[('CSCO', '2016-10-04')] Close 31.35 Volume 18460400.00 Name: (CSCO, 2016-10-04), dtype: float64 stocks.loc[('CSCO', '2016-10-04'), 'Volume'] 18460400.0 MANIPULATING DATAFRAMES WITH PANDAS

  22. Slicing ( o u termost inde x) stocks.loc['AAPL'] Close Volume Date 2016-10-03 112.52 21701800 2016-10-04 113.00 29736800 2016-10-05 113.05 21453100 MANIPULATING DATAFRAMES WITH PANDAS

  23. Slicing ( o u termost inde x) stocks.loc['CSCO':'MSFT'] Close Volume Symbol Date CSCO 2016-10-03 31.50 14070500 2016-10-04 31.35 18460400 2016-10-05 31.59 11808600 MSFT 2016-10-03 57.42 19189500 2016-10-04 57.24 20085900 2016-10-05 57.64 16726400 MANIPULATING DATAFRAMES WITH PANDAS

  24. Fanc y inde x ing ( o u termost inde x) stocks.loc[(['AAPL', 'MSFT'], '2016-10-05'), :] Close Volume Symbol Date AAPL 2016-10-05 113.05 21453100 MSFT 2016-10-05 57.64 16726400 stocks.loc[(['AAPL', 'MSFT'], '2016-10-05'), 'Close'] Symbol Date AAPL 2016-10-05 113.05 MSFT 2016-10-05 57.64 Name: Close, dtype: float64 MANIPULATING DATAFRAMES WITH PANDAS

  25. Fanc y inde x ing ( innermost inde x) stocks.loc[('CSCO', ['2016-10-05', '2016-10-03']), :] Close Volume Symbol Date CSCO 2016-10-03 31.50 14070500 2016-10-05 31.59 11808600 MANIPULATING DATAFRAMES WITH PANDAS

  26. Slicing ( both inde x es ) stocks.loc[(slice(None), slice('2016-10-03', '2016-10-04')),:] Close Volume Symbol Date AAPL 2016-10-03 112.52 21701800 2016-10-04 113.00 29736800 CSCO 2016-10-03 31.50 14070500 2016-10-04 31.35 18460400 MSFT 2016-10-03 57.42 19189500 2016-10-04 57.24 20085900 MANIPULATING DATAFRAMES WITH PANDAS

  27. Let ' s practice ! MAN IP U L ATIN G DATAFR AME S W ITH PAN DAS

Recommend

Learning to Rank with Learning to Rank with Partially-Labeled Data Partially-Labeled Data Kevin

Learning to Rank with Learning to Rank with Partially-Labeled Data Partially-Labeled Data Kevin

Learning to Rank with Learning to Rank with Partially-Labeled Data Partially-Labeled Data Kevin Duh University of Washington 1 The Ranking Problem The Ranking Problem Definition: Given a set of objects, sort them by preference. objectA

716 views • 59 slides
Meas u ring Segregation : The Inde x of Dissimilarit y AN ALYZIN G U S C E N SU S DATA IN P YTH

Meas u ring Segregation : The Inde x of Dissimilarit y AN ALYZIN G U S C E N SU S DATA IN P YTH

Meas u ring Segregation : The Inde x of Dissimilarit y AN ALYZIN G U S C E N SU S DATA IN P YTH ON Lee Hachadoorian Asst . Professor of Instr u ction , Temple Uni v ersit y What is Segregation ? ANALYZING US CENSUS DATA IN PYTHON Inde x of

496 views • 46 slides
Learning to Rank Learning to Rank with Partially-Labeled Data with Partially-Labeled Data Kevin

Learning to Rank Learning to Rank with Partially-Labeled Data with Partially-Labeled Data Kevin

Learning to Rank Learning to Rank with Partially-Labeled Data with Partially-Labeled Data Kevin Duh University of Washington (Joint work with Katrin Kirchhoff) 1 Motivation Motivation Machine learning can be an effective solution for

472 views • 29 slides
Objects (Demo1) Objects Objects represent information They consist of data and behavior,

Objects (Demo1) Objects Objects represent information They consist of data and behavior,

Objects (Demo1) Objects Objects represent information They consist of data and behavior, bundled together to create abstractions Objects can represent things, but also properties, interactions, &amp; processes A type of object is

591 views • 30 slides
61A Lecture 12 Announcements Objects (Demo) Objects 4 Objects Objects represent

61A Lecture 12 Announcements Objects (Demo) Objects 4 Objects Objects represent

61A Lecture 12 Announcements Objects (Demo) Objects 4 Objects Objects represent information 4 Objects Objects represent information They consist of data and behavior, bundled together to create abstractions 4 Objects

1.66k views • 132 slides
Learning from Limited Labeled Data (but a lot of unlabeled data) NELL as a case study Tom M.

Learning from Limited Labeled Data (but a lot of unlabeled data) NELL as a case study Tom M.

Learning from Limited Labeled Data (but a lot of unlabeled data) NELL as a case study Tom M. Mitchell Carnegie Mellon University Thesis: We will never really understand learning until we build machines that learn many different things,

582 views • 43 slides
Mutable Values Announcements Objects (Demo) Objects 4 Objects Objects represent

Mutable Values Announcements Objects (Demo) Objects 4 Objects Objects represent

Mutable Values Announcements Objects (Demo) Objects 4 Objects Objects represent information 4 Objects Objects represent information They consist of data and behavior, bundled together to create abstractions 4 Objects

2.23k views • 197 slides
Ne w Me xic o s K inde rg a rte n Ob se rva tio n T o o l (K OT ) E ARL Y L E ARNI

Ne w Me xic o s K inde rg a rte n Ob se rva tio n T o o l (K OT ) E ARL Y L E ARNI

Ne w Me xic o s K inde rg a rte n Ob se rva tio n T o o l (K OT ) E ARL Y L E ARNI NG ADVI SORY COUNCI L MARCH 26, 2015 Ra c e to the T o p E a rly L e a rning Cha lle ng e RT T -E L C g ra nt a wa rde d to

500 views • 25 slides
Snorkel + Data Programming: Beyond Hand-Labeled Training Data Alex Ratner Stanford University,

Snorkel + Data Programming: Beyond Hand-Labeled Training Data Alex Ratner Stanford University,

AAAI DeLBP Workshop 2/3 / 2018 Snorkel + Data Programming: Beyond Hand-Labeled Training Data Alex Ratner Stanford University, InfoLab MOTIVATION: In practice, training data is often: The bottleneck The practical injection point for

671 views • 38 slides
Ma Margi ginal l Inde depe pende dence and d Co Condi diti tion onal l Inde depe

Ma Margi ginal l Inde depe pende dence and d Co Condi diti tion onal l Inde depe

Ma Margi ginal l Inde depe pende dence and d Co Condi diti tion onal l Inde depe pende dence Com omputer Sc Science c cpsc sc322, L , Lecture 2 26 (Te Text xtboo ook k Chpt 6.1-2) 2) June 1 13, 2017 Lecture Overview

251 views • 24 slides
An Intr oduc tion to the F utur e Re ady PA Inde x What is the F utur e Re ady PA Inde

An Intr oduc tion to the F utur e Re ady PA Inde x What is the F utur e Re ady PA Inde

An Intr oduc tion to the F utur e Re ady PA Inde x What is the F utur e Re ady PA Inde x? T he F uture Re a dy PA I nde x is the ne w sc ho o l me a sure me nt to o l fo r c o mmunitie s to e va lua te ho w sc ho o ls a re do

354 views • 9 slides
X:Y XandYmatch X::Y XandYareidentical sult goal, for example: X \:: Y X and Y are not identical

X:Y XandYmatch X::Y XandYareidentical sult goal, for example: X \:: Y X and Y are not identical

2.1 Data objects 33 data objects ,/\ simple objects structures /\ of Prolog constants variables /\ atoms numbers Figure 2.1 Data objects in Prolog. Atoms and numbers 2.1.1 44 In chapter 1 we have seen some examples of atoms and variables.

365 views • 11 slides
Co-Training Based on Combining Labeled and Unlabeled Data with Co-Training by A. Blum

Co-Training Based on Combining Labeled and Unlabeled Data with Co-Training by A. Blum

0. Co-Training Based on Combining Labeled and Unlabeled Data with Co-Training by A. Blum &amp; T. Mitchell, 1998 1. Problem: Learning to classify data (ex: web pages) when the description of each example can be partitioned in 2

560 views • 9 slides
/\ This method for data structuring is simple and powerful. It is one of the reasens 4'. why

/\ This method for data structuring is simple and powerful. It is one of the reasens 4'. why

2.1 Data objects 33 data objects ,/\ simple objects structures /\ of Prolog constants variables /\ atoms numbers Figure 2.1 Data objects in Prolog. Atoms and numbers 2.1.1 44 In chapter 1 we have seen some examples of atoms and variables.

711 views • 49 slides
Getting To Know Your Data Road Map 1. Data Objects and Attribute Types 2. Descriptive Data

Getting To Know Your Data Road Map 1. Data Objects and Attribute Types 2. Descriptive Data

Getting To Know Your Data Road Map 1. Data Objects and Attribute Types 2. Descriptive Data Summarization 3. Measuring Data Similarity and Dissimilarity Data Objects and Attribute Types Types of data sets Data objects

767 views • 42 slides
61A Lecture 12 Announcements Objects (Demo) Objects Objects represent information They

61A Lecture 12 Announcements Objects (Demo) Objects Objects represent information They

61A Lecture 12 Announcements Objects (Demo) Objects Objects represent information They consist of data and behavior, bundled together to create abstractions Objects can represent things, but also properties, interactions, &amp;

521 views • 16 slides
' ' ' ' DIE FACE-6 EIGHTH NOTE (Demo) 7 Some Objects Can Change Mutation Can Happen

' ' ' ' DIE FACE-6 EIGHTH NOTE (Demo) 7 Some Objects Can Change Mutation Can Happen

Announcements Mutable Values Objects Objects represent information They consist of data and behavior, bundled together to create abstractions Objects can represent things, but also properties, interactions, &amp; processes A type

207 views • 3 slides
Case st u d y: Ol y mpic medals MAN IP U L ATIN G DATAFR AME S W ITH PAN DAS Anaconda Instr u

Case st u d y: Ol y mpic medals MAN IP U L ATIN G DATAFR AME S W ITH PAN DAS Anaconda Instr u

Case st u d y: Ol y mpic medals MAN IP U L ATIN G DATAFR AME S W ITH PAN DAS Anaconda Instr u ctor Ol y mpic medals dataset MANIPULATING DATAFRAMES WITH PANDAS Reminder : inde x ing &amp; pi v oting Filtering and inde x ing One - le v el inde

411 views • 29 slides
Mutable Values Announcements Objects (Demo) Objects Objects represent information They

Mutable Values Announcements Objects (Demo) Objects Objects represent information They

Mutable Values Announcements Objects (Demo) Objects Objects represent information They consist of data and behavior, bundled together to create abstractions Objects can represent things, but also properties, interactions, &amp;

472 views • 23 slides
GANs for Limited Labeled Data MIX+GAN Ian Goodfellow, Sta ff Research Scientist, Google Brain

GANs for Limited Labeled Data MIX+GAN Ian Goodfellow, Sta ff Research Scientist, Google Brain

Progressive GAN CoGAN LR-GAN MedGAN CGAN IcGAN A ff GAN DiscoGAN LS-GAN b-GAN LAPGAN MPM-GAN AdaGAN AMGAN iGAN LSGAN InfoGAN IAN CatGAN GANs for Limited Labeled Data MIX+GAN Ian Goodfellow, Sta ff Research Scientist, Google Brain

807 views • 26 slides
F UE L RE VE NUE INDE XING T HE REG I O NAL T RANSPO RT AT I O N CO MMI SSI O N O

F UE L RE VE NUE INDE XING T HE REG I O NAL T RANSPO RT AT I O N CO MMI SSI O N O

4/21/2016 F UE L RE VE NUE INDE XING T HE REG I O NAL T RANSPO RT AT I O N CO MMI SSI O N O F SO UT HERN NEVADA T r ansit T r affic Manage me nt Syste ms T r anspor tation Planning &amp; F unding 1 4/21/2016 RT

390 views • 21 slides
Objects: Data Abstraction In Object-Oriented programming languages like Java, objects are used

Objects: Data Abstraction In Object-Oriented programming languages like Java, objects are used

Objects: Data Abstraction In Object-Oriented programming languages like Java, objects are used to represent data A class defines a type of object , including its data its permissible operations Once a type is defined,

969 views • 66 slides
INDE DEX OF F SL SLIDE DES Pre Pre-2017 Mo 2017 Money-Ba Based S System Pos Post-2016

INDE DEX OF F SL SLIDE DES Pre Pre-2017 Mo 2017 Money-Ba Based S System Pos Post-2016

INDE DEX OF F SL SLIDE DES Pre Pre-2017 Mo 2017 Money-Ba Based S System Pos Post-2016 2016 P Program ams an and Dat Data 2-4 Catch &amp; Release Crisis 11-12 ABQ Crime Rates 5-9 Constitutional Amendment 13 ABQ

357 views • 31 slides
Learning a Variable-Clustering Strategy for Octagon from Labeled Data Generated by a Static

Learning a Variable-Clustering Strategy for Octagon from Labeled Data Generated by a Static

1 Learning a Variable-Clustering Strategy for Octagon from Labeled Data Generated by a Static Analysis Kihong Heo 1 , Hakjoo Oh 2 , Hongseok Yang 3 Seoul National University 1 Korea University 2 University of Oxford 3 SAS 2016 @Edinburgh 2

823 views • 32 slides

More recommend