Multimedia Data Processing on CIEL Arman Idani 14 Feb 2012 R202 - PowerPoint PPT Presentation

Multimedia Data Processing on CIEL Arman Idani 14 Feb 2012 R202 – Data Centric Networking

Machine Learning on DC • Apache Mahout (library for Hadoop) • Tons of independent codes • Only on textual/graph content • No multimedia input • Speech, music, image, video

Goal • Developing an extension for CIEL • Prepare multimedia data for processing • Tasks should be heavily parallelized • Each task should be able to spawn new tasks (iteration)

Model-based Multimedia ML Labels Training Model 𝑦 1 𝑦 2 ⋮ 𝑦 n Feature Extraction 𝑧 1 𝑄(𝑏) y2 𝑄(𝑐) ⋮ y n 𝑄(𝑑) Recognition ⋮ 𝑄(𝑜)

Dataset • Multimedia input + time-stamped labels 352150000 416514000 Db 416514000 449336000 Ab 449336000 511888000 Ab 511888000 543612000 Gb 543612000 575153000 Db 575153000 639137000 Ab 639137000 670810000 Gb 670810000 701669000 Db 701669000 717149000 Db 717149000 732513000 Bb:min 732513000 764054000 F:min 764054000 796062000 Eb:min7

Challenge? • Terabytes of data • Very difficult training • Solution? • Group each label together

Solution 352150000 416514000 Db 352150000 416514000 Db 543612000 575153000 Db 416514000 449336000 Ab 670810000 701669000 Db 449336000 511888000 Ab 701669000 717149000 Db 511888000 543612000 Gb 543612000 575153000 Db 416514000 449336000 Ab 575153000 639137000 Ab 449336000 511888000 Ab 639137000 670810000 Gb 575153000 639137000 Ab 670810000 701669000 Db 701669000 717149000 Db 717149000 732513000 Bb:min 732513000 764054000 F:min 764054000 796062000 Eb:min7

Not that Simple! • Terabytes of data • or gigabytes of labels! • Image and Video sources • No longer can cut audio • Need to extract objects from visual sources • Huge in interactive ads

Extractor • def extractor( string <source_path>, string <transcription_path>, boolean <spawn> *, string <intermediate_path> )

System Overview Reducers Extractors Int 0 Int 1 Int 2 worker worker Input 0 Data Label 0 Int 3 Input 1 Data Label 1 Int 0 Input 2 Data Label 2 Int 4 worker worker Int 1 Input 3 Data Label 3 Int 3 … … Input n-1 Data Label m-1 Int 7 worker worker Input n Data Label m Int 2 Int 9 Int 3

System Overview (cont.) Data Analyser Feature 0 worker Data Label 0 Feature 1 Data Label 1 Feature 2 Data Label 2 Feature 3 worker Data Label 3 … … Feature t-1 Data Label m-1 Feature t worker Data Label m

Data Analyser • def analyser ( string <source_path>, boolean <spawn> *, string <destination_path> )

Overview • Developer only provides “extractor”, “analyser” and a config • CIEL (and the extension) takes care of the rest • Built- in support for audio and video “reducers” • Sample project for audio processing (MFCC) will be developed for evaluation