the experiments for
play

The experiments for You -Do, I- Learn Presenter: Wenguang Mao - PowerPoint PPT Presentation

Oct 18, 2023 •180 likes •597 views

The experiments for You -Do, I- Learn Presenter: Wenguang Mao Instructor: Kristen Grauman Author for the paper: Dima Damen Recap of the Paper Gaze attention Gaze point Clustering position Clustering TRO MOI Gaze area appearance

  1. The experiments for “You -Do, I- Learn” Presenter: Wenguang Mao Instructor: Kristen Grauman Author for the paper: Dima Damen

  2. Recap of the Paper Gaze attention Gaze point Clustering position Clustering TRO MOI Gaze area appearance Neighbor frames

  3. Experiment Setup • Dataset: Bristol Egocentric Object Interactions Dataset

  4. Experiment Setup • Dataset: Bristol Egocentric Object Interactions Dataset • Egocentric videos at 6 locations • Gaze point on each frame • Gaze positions in 3D space • Gaze fixation on each frame • Ground truth positions of TROs • 3D map for each location, 3D positions of the camera for each frame, …… • Code: VLFeat, Matlab toolboxes, and programs written by myself

  5. Why Need Gaze Info • Given an egocentric image, which part of the image do you think I am focusing on? • Center of image? • Blue point: center of image • Red point: gaze point

  6. Why Need Gaze Info • The distance between the center and the gaze point (a) Desk (b) Door

  7. Why Need Gaze Info • The distance between the center and the gaze point Center of image is not good approximation for the gaze point (a) Desk (b) Door

  8. Why Need Gaze Info • The distance between the center and the gaze point ( during gaze fixation ) (a) Desk (b) Door

  9. Why Need Gaze Info • The distance between the center and the gaze point ( during gaze fixation ) Center of image is not good approximation for the gaze point Even during attention period (a) Desk (b) Door

  10. How Gaze Fixation Helps • Do you think there is any TRO in the video clips • Red dot: gaze point

  11. How Gaze Fixation Helps • Do you think there is any TRO in the video clips • Red dot: gaze point Gaze fixation helps identify a TRO

  12. How Gaze Fixation Helps • Do you think there is any TRO in the video clips • Red dot: gaze point

  13. How Gaze Fixation Helps • Do you think there is any TRO in the video clips • Red dot: gaze point Gaze fixation alone is far from enough to find TROs

  14. How 3D Positions of Gaze Help • Blue circles: 3D positions of gazes in a video • Red cross: ground truth positions of TRO (a) Without gaze fixation filtering (a) With gaze fixation filtering

  15. How 3D Positions of Gaze Help • Blue circles: 3D positions of gazes in a video • Red cross: ground truth positions of TRO 3D gaze positions are very helpful to identify TROs (a) Without gaze fixation filtering (a) With gaze fixation filtering

  16. Clustering for Gaze 3D positions • Right number of clusters (kmeans) • Yellow square: cluster center (a) Without gaze fixation filtering (b) With gaze fixation filtering

  17. Clustering for Gaze 3D positions • Right number of clusters (kmeans) • Yellow square: cluster center With the knowledge of right number of TROs, they can be easily identified using 3D gaze positions (a) Without gaze fixation filtering (b) With gaze fixation filtering

  18. Clustering for Gaze 3D positions • Too less clusters • Yellow square: cluster center (a) Without gaze fixation filtering (b) With gaze fixation filtering

  19. Clustering for Gaze 3D positions • Too less clusters • Yellow square: cluster center If underestimating the number, low precision and low recall for identifying TROs (a) Without gaze fixation filtering (b) With gaze fixation filtering

  20. Clustering for Gaze 3D positions • Too much clusters • Yellow square: cluster center (a) Without gaze fixation filtering (b) With gaze fixation filtering

  21. Clustering for Gaze 3D positions • Too much clusters • Yellow square: cluster center If overestimating the number, high recall and low precision (a) Without gaze fixation filtering (b) With gaze fixation filtering

  22. Spectral Clustering • Right number of clusters (a) kmeans (b) spectral

  23. Spectral Clustering • Right number of clusters Same with K-means (a) kmeans (b) spectral

  24. Spectral Clustering • Too less clusters (a) kmeans (b) spectral

  25. Spectral Clustering • Too less clusters Same with k-means (a) kmeans (b) spectral

  26. Spectral Clustering • Too much clusters (a) kmeans (b) spectral

  27. Spectral Clustering • Too much clusters Outperform k-means, high precision and high recall. (a) kmeans (b) spectral

  28. What is the Limitation of Gaze Positions • Can we only use 3D gaze positions? • No, because of moving TRO • How to solve this problem? • Appearance

  29. Appearance • How HoG features represent an image

  30. Appearance • How HoG features represent an image HoG is good to describe the boundary

  31. Identify TROs based on Appearance • Extract HoG from the region near the gaze point for each frame • Generate BoW representation for each frame • Perform clustering on frames • Use the frame closest to the center to represent each cluster • Compare the appearance of center frames with the ground truth

  32. Appearance • Five TROs around the desk tape socket screwdriver charger box

  33. Results Success (box) Success (tape) Duplicated (box) Success (charger) Failure

  34. Results Success (box) Success (tape) Duplicated (box) Missing two TROs, the appearance is not as effective as the position Success (charger) Failure

  35. Using Neighbor frames Failure Success (charger) Success (box) Success (driver) Success (tape)

  36. Using Neighbor frames Failure Success (charger) Success (box) Missing one TRO, using neighbor frames is helpful to improve performance Success (driver) Success (tape)

  37. Over-Estimating No. of Clusters Failure Success (box) Success (driver) Success (charger) Success (tape) Duplicated (box) Duplicated (box) Duplicated (driver)

  38. Over-Estimating No. of Clusters Failure Success (box) Success (driver) Success (charger) Missing one TROs, over-estimating is helpful to identify more TROs Success (tape) Duplicated (box) Duplicated (box) Duplicated (driver)

  39. Also Using Neighbor frames Failure Success (tape) Success (socket) Duplicated (socket) Success (driver) Success (box) Success (charger) Duplicated (box)

  40. Also Using Neighbor frames Failure Success (tape) Success (socket) Duplicated (socket) Finding all TROs Success (driver) Success (box) Success (charger) Duplicated (box)

  41. Conclusion • Gaze information is important and necessary for egocentric videos, and the center of image is not a good approximation • Gaze fixation is helpful for identifying TROs, but itself is not enough • 3D positions of gaze give rich information for TROs, but clustering method and the estimation on the number of TROs is critical • Use spectral clustering and do not worry about overestimating • Appearance is another important feature for identifying TROs • Using neighbor frames is beneficial to improve performance • Over-estimating No. of TROs is helpful to reduce false negative

Recommend

Experiments Philosophy of Economics University of Virginia Matthias Brinkmann Contents 1.

Experiments Philosophy of Economics University of Virginia Matthias Brinkmann Contents 1.

Experiments Philosophy of Economics University of Virginia Matthias Brinkmann Contents 1. Public Goods Experiments 2. Types of Experiments 3. The Role of Experiments 29/10/2018 Experiments 2 A Standard Finding (Guala 2005, 22) Two

460 views • 14 slides
Hagner experiments status after 22 years Per-Erik Wikberg SLU Ume Hagner experiments

Hagner experiments status after 22 years Per-Erik Wikberg SLU Ume Hagner experiments

Hagner experiments status after 22 years Per-Erik Wikberg SLU Ume Hagner experiments Liberich liberation thinning, enrichment planting (in Swedish naturkultur) Principle: optimizing the net present value at each site of the

601 views • 34 slides
Collider Experiments and India Sunanda Banerjee January, 2019 Experiments in High Energy Physics

Collider Experiments and India Sunanda Banerjee January, 2019 Experiments in High Energy Physics

Collider Experiments and India Sunanda Banerjee January, 2019 Experiments in High Energy Physics Particle physics experiments in the world started with No accelerator Visual detection technique In absence of accelerators what was

472 views • 46 slides
Experiments designed to help the participants Maximilian Kasy 7/10/2019 Why experiments? Are

Experiments designed to help the participants Maximilian Kasy 7/10/2019 Why experiments? Are

Experiments designed to help the participants Maximilian Kasy 7/10/2019 Why experiments? Are your programs effective in helping refugees? How to find out? Possibility 1: Compare the outcomes of those who got the programs to others who

330 views • 13 slides
Planned OBT Experiments at CRL Planned OBT Experiments at CRL September 14, 2011 Sang Bog Kim

Planned OBT Experiments at CRL Planned OBT Experiments at CRL September 14, 2011 Sang Bog Kim

Planned OBT Experiments at CRL Planned OBT Experiments at CRL September 14, 2011 Sang Bog Kim and Vladimir Korolevych Research Scientist Environmental Technologies Branch Chalk River Laboratories Chalk River, Ontario Canada 1 UNRESTRICTED

594 views • 17 slides
Key Experiments Key Experiments and Simulation: and Simulation: Status Report Status Report

Key Experiments Key Experiments and Simulation: and Simulation: Status Report Status Report

Key Experiments Key Experiments and Simulation: and Simulation: Status Report Status Report E.Farnea INFN Sezione di Padova, Italy AGATA Week, Strasbourg, November 24 th , 2005 Outline Outline Performance of the Demonstrator Some

317 views • 19 slides
WISP searches by Tokyo tabletop experiments group UTokyo tabletop experiments group Toshio

WISP searches by Tokyo tabletop experiments group UTokyo tabletop experiments group Toshio

WISP searches by Tokyo tabletop experiments group UTokyo tabletop experiments group Toshio NAMBA 0 UT tabletop experiments group (only related to todays talk) UTokyo Physics & ICEPP : S. Asai, T. Inada, T. Yamaji, T. Yamazaki(KEK),

624 views • 49 slides
Lecture 5/Chapter 5 Experiments Variables Roles Outside Variables Single- or

Lecture 5/Chapter 5 Experiments Variables Roles Outside Variables Single- or

Lecture 5/Chapter 5 Experiments Variables Roles Outside Variables Single- or Double-Blind Experiments Flaws and Remedies in Experiments Roles of variables Most statistical studies of relationships attempt to establish evidence

763 views • 32 slides
Designs Chapter 11 Quasi-Experimentation Quasi-experiments resemble experiments, but lack

Designs Chapter 11 Quasi-Experimentation Quasi-experiments resemble experiments, but lack

Quasi-Experimental Designs Chapter 11 Quasi-Experimentation Quasi-experiments resemble experiments, but lack experimental control lack of random assignment is the key point of distinction between quasi- experiments and true

888 views • 31 slides
Experimental Design and the Search for Quasi-Experiments Department of Government London School

Experimental Design and the Search for Quasi-Experiments Department of Government London School

Review Randomized Experiments Quasi-Experiments Experimental Design and the Search for Quasi-Experiments Department of Government London School of Economics and Political Science Review Randomized Experiments Quasi-Experiments 1 A Review

823 views • 58 slides
Survey Experiments April 8 1 / 16 Outline Guest presentation Overview of survey experiments

Survey Experiments April 8 1 / 16 Outline Guest presentation Overview of survey experiments

Survey Experiments April 8 1 / 16 Outline Guest presentation Overview of survey experiments Discussion of readings Synopsis presentations 2 / 16 Guest Presentation Rune Slothuus, Professor in IFSK Discussing survey experiment by Druckman,

496 views • 16 slides
Remote Participation in in Remote Participation physical experiments physical experiments

Remote Participation in in Remote Participation physical experiments physical experiments

Remote Participation in in Remote Participation physical experiments physical experiments Piroska Giese KFKI RMKI e-mail : giese@rmki.kfki.hu Yahoo ID : kfki_Piroska_Giese April 18-19 2002, Cluj-Napoca RoEduNet Conference Slide 1 Overview

543 views • 40 slides
Social Experiments Handbook of Econometrics James J. Heckman and Edward J. Vytlacil The

Social Experiments Handbook of Econometrics James J. Heckman and Edward J. Vytlacil The

Social Experiments Handbook of Econometrics James J. Heckman and Edward J. Vytlacil The University of Chicago and Stanford University Handout Draft, July 28, 2004 1 1 Social Experiments 1.1 Introduction Consider ideal experiments with no

1.06k views • 79 slides
Non-LHC experiments in HEPDATA Matthew Wing (UCL) Initial comments. Data / experiments

Non-LHC experiments in HEPDATA Matthew Wing (UCL) Initial comments. Data / experiments

Non-LHC experiments in HEPDATA Matthew Wing (UCL) Initial comments. Data / experiments to be included ? More exotic data. British Museum HEPDATA Advisory Board Meeting 16 October 2015 Initial comments It would be

465 views • 5 slides
ACMS 20340 Statistics for Life Sciences Chapter 8: Designing Experiments Fishers Experiments

ACMS 20340 Statistics for Life Sciences Chapter 8: Designing Experiments Fishers Experiments

ACMS 20340 Statistics for Life Sciences Chapter 8: Designing Experiments Fishers Experiments Experiment Terminology individuals (if human) subjects explanatory variable factors In conducting an experiment, we try to keep as

340 views • 31 slides
THE ELECTRIC DISCHARGE EXPERIMENTS A Theorists Exploration of First Laboratory

THE ELECTRIC DISCHARGE EXPERIMENTS A Theorists Exploration of First Laboratory

THE ELECTRIC DISCHARGE EXPERIMENTS A Theorists Exploration of First Laboratory Plasmas I. INTRODUCTION Who? Experimentalists interested in electrical properties of matter What? Experiments with electromagnetic

160 views • 14 slides
More Than Two Factors Designing experiments with two factors extends easily to experiments with

More Than Two Factors Designing experiments with two factors extends easily to experiments with

ST 370 Probability and Statistics for Engineers More Than Two Factors Designing experiments with two factors extends easily to experiments with more factors: Choose a few (often 2 or 3) levels of each factor; All combinations of factor levels

412 views • 14 slides
Democratizing online controlled experiments at Booking.com How to run 1.000+ concurrent

Democratizing online controlled experiments at Booking.com How to run 1.000+ concurrent

Democratizing online controlled experiments at Booking.com How to run 1.000+ concurrent experiments Lukas Vermeer | Director of Experimentation Booking. Founded in 1996 28,000,000+ reported listings 1,550,000+ room nights per day 17.000+

608 views • 37 slides
Outline Experiments with Two Factors (14.1) Experiments with Three or More Factors (14.2)

Outline Experiments with Two Factors (14.1) Experiments with Three or More Factors (14.2)

2/15/2007 219323 Probability and Statistics for Software Statistics for Software and Knowledge Engineers Lecture 12: Multifactor Experimental Multifactor Experimental Design and Analysis Monchai Sopitkamon, Ph.D. Outline Experiments

462 views • 25 slides
Chapter 8. Experiments Chapter 8. Experiments Experimental Research Experimental Research

Chapter 8. Experiments Chapter 8. Experiments Experimental Research Experimental Research

Topics Appropriate for Topics Appropriate for Chapter 8. Experiments Chapter 8. Experiments Experimental Research Experimental Research Experiments usually involve two stages Experiments usually involve two stages Hypothesis testing

290 views • 4 slides
PIXEL DETECTOR for X-RAY experiments PIXEL DETECTOR for X-Rays Experiments Jean-Claude CLEMENS

PIXEL DETECTOR for X-RAY experiments PIXEL DETECTOR for X-Rays Experiments Jean-Claude CLEMENS

PIXEL DETECTOR for X-RAY experiments PIXEL DETECTOR for X-Rays Experiments Jean-Claude CLEMENS Centre de Physique des Particules de Marseille JC Clemens, CNRS/IN2P3/CPPM, FRANCE Pixel 2000, Genova, June 2000 PIXEL DETECTOR for

367 views • 15 slides
Last Time u Debugging It s a science use experiments to refine hypotheses about

Last Time u Debugging It s a science use experiments to refine hypotheses about

Last Time u Debugging It s a science use experiments to refine hypotheses about bugs It s an art creating effective hypotheses and experiments and trying them in the right order requires great intuition Today u

824 views • 34 slides
EDM experiments - Yannis K. Semertzidis Brookhaven National Lab Mercury EDM 199 Hg Mercury

EDM experiments - Yannis K. Semertzidis Brookhaven National Lab Mercury EDM 199 Hg Mercury

Indirect Searches for New Physics at the time of LHC Florence, 22-24 March 2010 + EDM experiments - Yannis K. Semertzidis Brookhaven National Lab Mercury EDM 199 Hg Mercury EDM 199 Hg Neutron EDM experiments Ra EDM, Electron EDM

799 views • 56 slides
A Hierarchal Approach to Validation Experiments in Magnetic Fusion Science Validation

A Hierarchal Approach to Validation Experiments in Magnetic Fusion Science Validation

A Hierarchal Approach to Validation Experiments in Magnetic Fusion Science Validation Experiments Working Group US Transport Task Force P.W. Terry, T. Carter, C. Hegna, C. Holland, M. GIlmore, M. Greenwald, B. LaBombard, R. Majeski, D.E.

593 views • 24 slides

More recommend