romip one step forward one step aside
play

ROMIP: one step forward, one step aside http://romip.ru/en/ Pavel - PowerPoint PPT Presentation

Mar 15, 2024 •219 likes •378 views

ROMIP: one step forward, one step aside http://romip.ru/en/ Pavel Braslavski, Ilia Chetviorkin, Maxim Gubin, Natalia Lukashevich, Igor Nekrestyanov, Marina Nekrestyanova, Natalia Vassileva CLEF 2012 ROMIP at a glance TREC-like Russian

  1. ROMIP: one step forward, one step aside http://romip.ru/en/ Pavel Braslavski, Ilia Chetviorkin, Maxim Gubin, Natalia Lukashevich, Igor Nekrestyanov, Marina Nekrestyanova, Natalia Vassileva CLEF 2012

  2. ROMIP at a glance • TREC-like Russian initiative • Started 2002 • Several freely available text and image collections • 10-15 participating teams each year • Remote participation + live meeting • Popular testbed for IR research in Russia • Related activities: RuSSIR 20.09.2012 ROMIP 2

  3. ROMIP 2004 20.09.2012 ROMIP 3

  4. Largest text collections Evaluated within Size Collection Documents Topics ad-hoc search (compressed) track ~300,000 Legal 2 Gb 14,794 220 By.Web 1,524,676 8 Gb ~ 60,000 1 500+ KM.RU 3,010,455 13 Gb ~ 60,000 ~250 20.09.2012 ROMIP 4

  5. (Retired) text document tracks � Ad-hoc text retrieval � Text categorization � Snippet generation � QA and fact extraction � News clustering � Search by sample document 20.09.2012 ROMIP 5

  6. Image collections � Photo collection: 20,000 images from Flickr � Dups collection: 15 hrs video � 37 800 frames � Panoramic series: 55,000 images (data recycled from Internet Math 2011) 20.09.2012 ROMIP 6

  7. Image tracks � Content based image retrieval � Near-duplicate detection � Image annotation � Finding panoramic series 20.09.2012 ROMIP 7

  8. ROMIP by 2011 • Low participation from academia • Fatigue of classical IR tasks available relevance tables – no need to participate • • overfitting on available datasets; • hard to model realistic settings and data; • well-studied tasks – new results are hard to expect. • Limited resources • ML challenges (e.g. www.kaggle.com) 20.09.2012 ROMIP 8

  9. ROMIP light 2011 • Sentiment analysis • Search by query image (low participation � ) • Schedule shifted to fall 20.09.2012 ROMIP 9

  10. ROMIP timeline 30 systems applied 25 systems participated # of tracks 20 15 10 5 0 2003 2004 2005 2006 2007 2008 2009 2010 2011 20.09.2012 ROMIP 10

  11. Sentiment analysis (SA) � Three domains: movies , books , and digital cameras � ‘Transfer learning’ (data from different sources) � Classification into 2, 3, and 5 classes � 23 teams registered � 12 submitted results � 6 reports published � 2-class: 105 runs, 3-class: 81 runs, 5-class: 30 runs 20.09.2012 ROMIP 11

  12. SA: data Training set 15,000+ movie reviews (10-point scale) 24,000+ book reviews (10-point scale) 10,000+ camera reviews (5-point scale) Test set blog posts collected via blog search w. subsequent filtering 275 posts on movies 329 posts on books 270 posts on digital cameras 20.09.2012 ROMIP 12

  13. Plans • New edition of SA track • Finer granularity (sentence) • Opinions for a given entity • Re-launch of image tracks (in cooperation with Graphicon conference) • Machine translation track 20.09.2012 ROMIP 13

  14. MT evaluation track (2012) • Strong industrial players • 1M parallel sentences (Ru-En) to release • Collaboration with TAUS Labs • Metrics BLEU • • human assessment 20.09.2012 ROMIP 14

  15. Thank you! Questions? Pavel Braslavski pb@kontur.ru 20.09.2012 ROMIP 15

Recommend

min { a i , a j } max { a i , a j } P j P j P i P i P i P j Step 1 Step 2 Step 3 Figure 9.1 A

min { a i , a j } max { a i , a j } P j P j P i P i P i P j Step 1 Step 2 Step 3 Figure 9.1 A

a i a j a i , a j a j , a i min { a i , a j } max { a i , a j } P j P j P i P i P i P j Step 1 Step 2 Step 3 Figure 9.1 A parallel compare-exchange operation. Processes P i and P j send their elements to each other. Process P i keeps min { a i ,

372 views • 22 slides
Step 1 Step 2 Step 3 Step 4 Step 5 Preparation of a sketch Submission of birth map of all

Step 1 Step 2 Step 3 Step 4 Step 5 Preparation of a sketch Submission of birth map of all

Step 1 Step 2 Step 3 Step 4 Step 5 Preparation of a sketch Submission of birth map of all customary information forms to Preparation and adoption of Convening and submission Submission of Application Ste land owned by ILG obtain

1.23k views • 4 slides
Russian Information Retrieval Evaluation Seminar (ROMIP) (ROMIP) http://romip.ru/en/ Igor

Russian Information Retrieval Evaluation Seminar (ROMIP) (ROMIP) http://romip.ru/en/ Igor

Russian Information Retrieval Evaluation Seminar (ROMIP) (ROMIP) http://romip.ru/en/ Igor Nekrestyanov, Pavel Braslavski CLEF 2010 ROMIP at a glance ROMIP at a glance TREC like Russian initiative Started 2002 Several text and

561 views • 12 slides
Using RtI for SLD eligibility: Using RtI for SLD eligibility: Step by step procedures Step by

Using RtI for SLD eligibility: Using RtI for SLD eligibility: Step by step procedures Step by

Using RtI for SLD eligibility: Using RtI for SLD eligibility: Step by step procedures Step by step procedures Webster Groves School District 2009 Response to Intervention (RtI) is a comprehensive early detection and prevention strategy

688 views • 23 slides
o o o o Step 1:

o o o o Step 1:

o o o o Step 1: Bachelor of Laws (Western Sydney School of Law) Step 2: Professional Legal Training (College of Law et ors) Step 3: Admission to the Supreme

366 views • 14 slides
Outbreak Investigation Outbreak Investigation Step by Step Step by Step Darin Areechokchai MD.,

Outbreak Investigation Outbreak Investigation Step by Step Step by Step Darin Areechokchai MD.,

Outbreak Investigation Outbreak Investigation Step by Step Step by Step Darin Areechokchai MD., DTM&H., MCTM. Surveillance and Investigation Section Bureau of Epidemiology, Department of Disease Control Ministry of Public Health, Thailand

1.27k views • 83 slides
Step by step guide Step 1: Purchasing an RSBlog! membership Step 2: Downloading RSBlog! Step 3:

Step by step guide Step 1: Purchasing an RSBlog! membership Step 2: Downloading RSBlog! Step 3:

Step by step guide Step 1: Purchasing an RSBlog! membership Step 2: Downloading RSBlog! Step 3: Installing RSBlog! 3.1 Installing the component 3.2 Installing a new language file Step 4: Import plugins (optional) 4.1 Import Joomla! articles

889 views • 67 slides
Quick guide Step 1: Purchasing an RSEvents! membership Step 2: Downloading RSEvents! Step 3:

Quick guide Step 1: Purchasing an RSEvents! membership Step 2: Downloading RSEvents! Step 3:

Quick guide Step 1: Purchasing an RSEvents! membership Step 2: Downloading RSEvents! Step 3: Installing RSEvents! Step 4: Configure RSEvents! Step 5: Add user permissions Step 6: Create event categories Step 7: Create event locations Step 8:

627 views • 35 slides
FROM XML TO RDF STEP BY STEP: APPROACHES FOR LEVERAGING

FROM XML TO RDF STEP BY STEP: APPROACHES FOR LEVERAGING

Co-funded by the Horizon 2020 Framework Programme of the European Union Grant Agreement Number 644771 FROM XML TO RDF STEP BY STEP: APPROACHES

613 views • 26 slides
FIRST DAY OF SCHOOL STEP BY STEP DIRECTION TO GETTING LOGGED ONTO YOUR FIRST DAY OF 7TH OR 8TH

FIRST DAY OF SCHOOL STEP BY STEP DIRECTION TO GETTING LOGGED ONTO YOUR FIRST DAY OF 7TH OR 8TH

FIRST DAY OF SCHOOL STEP BY STEP DIRECTION TO GETTING LOGGED ONTO YOUR FIRST DAY OF 7TH OR 8TH GRADE AUGUST 10,2020 IS A MINIMUM DAY STUDENTS WILL LOG 7:45AM - ONTO THEIR 12:27PM CLASSES AND ENGAGE WITH THEIR TEACHERS AND PEERS STEP 1:

276 views • 6 slides
Small step and Auto Zhuoran Liu May 30th Difference between small step and big step The big

Small step and Auto Zhuoran Liu May 30th Difference between small step and big step The big

Type Theory and Coq Small step and Auto Zhuoran Liu May 30th Difference between small step and big step The big step specify how a given expression can be evaluated to its final value. The style is simple and natural for many purposes and is

196 views • 19 slides
Step by step guide Step 1: Purchasing a RSTickets!Pro membership Step 2: Downloading

Step by step guide Step 1: Purchasing a RSTickets!Pro membership Step 2: Downloading

Step by step guide Step 1: Purchasing a RSTickets!Pro membership Step 2: Downloading RSTickets!Pro 2.1 Downloading the component 2.2 Downloading the plugins/modules 2.3 Downloading additional language packs Step 3: Installing RSTickets!Pro

548 views • 44 slides
Quick guide Step 1: Purchasing RSMail! Step 2: Download RSMail! Step 3: Installing RSMail! Step

Quick guide Step 1: Purchasing RSMail! Step 2: Download RSMail! Step 3: Installing RSMail! Step

Quick guide Step 1: Purchasing RSMail! Step 2: Download RSMail! Step 3: Installing RSMail! Step 4: RSMail! settings Step 5: Add Subscribers 5.1. Create subscriber lists 5.2. Add subscribers 5.2.1 Manual add 5.2.2 Import from CSV Step 6

625 views • 14 slides
Quick guide Step 1: Purchasing a RSComments! membership Step 2: Download RSComments! Step 3:

Quick guide Step 1: Purchasing a RSComments! membership Step 2: Download RSComments! Step 3:

Quick guide Step 1: Purchasing a RSComments! membership Step 2: Download RSComments! Step 3: Installing RSComments! Step 4: Configure RSComments 4.1 General Configuration 4.2 Configure comments Step 5: Add user permissions Step 6: Moderate

337 views • 8 slides
Step by step guide Step 1: Purchasing an RSEvents! membership Step 2: Downloading RSEvents! Step

Step by step guide Step 1: Purchasing an RSEvents! membership Step 2: Downloading RSEvents! Step

Step by step guide Step 1: Purchasing an RSEvents! membership Step 2: Downloading RSEvents! Step 3: Installing RSEvents! Step 4: Configure RSEvents! 4.1 General settings 4.2 Dashboard settings 4.3 Events 4.3.1 Event general settings 4.3.2

1.29k views • 80 slides
Background CLINICAL TRIAL AGREEMENT ACTIVITY TIME TRACKING PI / Department Reviews Other

Background CLINICAL TRIAL AGREEMENT ACTIVITY TIME TRACKING PI / Department Reviews Other

Background CLINICAL TRIAL AGREEMENT ACTIVITY TIME TRACKING PI / Department Reviews Other Intermountain Reviews Approvals & Signtures Manage and Close Step 1: Step 1a: Step 1b Step 1c Dept Step 2: Step 3: Step 4 : Step 5: Step 6 :

754 views • 31 slides
W W W . A V I A N . A E R O The Connected Trip Building a trip is a mess Step 1 Step 2 Step 1

W W W . A V I A N . A E R O The Connected Trip Building a trip is a mess Step 1 Step 2 Step 1

W W W . A V I A N . A E R O The Connected Trip Building a trip is a mess Step 1 Step 2 Step 1 According to Google, people are browsing more than 7 different websites before they buy a ticket They are spending an average of 10 hours on

723 views • 21 slides
Best Practices Step 1 Have clear internal and external case Step 2 Plant the seed early and

Best Practices Step 1 Have clear internal and external case Step 2 Plant the seed early and

Peer-to-Peer Campaign Best Practices Step 1 Have clear internal and external case Step 2 Plant the seed early and regularly Step 3 Skills, tools, and methods Step 4 Determine benchmarks Step 5 Challenge, Match, and Incentive Receive

440 views • 11 slides
Step Up Savannah 1 Step Up Savannah Mission: The Step Up collaborative will enhance economic

Step Up Savannah 1 Step Up Savannah Mission: The Step Up collaborative will enhance economic

Savannahs Poverty Reduction Initiative Step Up Savannah 1 Step Up Savannah Mission: The Step Up collaborative will enhance economic independence in Savannah by encouraging residents to take personal responsibility and organizations to

328 views • 29 slides
Credential Assessment Mapping Privilege Escalation at Scale Matt Weeks @scriptjunkie1 Adversary

Credential Assessment Mapping Privilege Escalation at Scale Matt Weeks @scriptjunkie1 Adversary

Credential Assessment Mapping Privilege Escalation at Scale Matt Weeks @scriptjunkie1 Adversary access (# boxes owned) 10000 1000 100 10 1 Step 1 Step 2 Step 3 Step 4 Step 5 Step 6 Step 7 Step 8 Step 9 Step 10 Adversary access (#

721 views • 45 slides
Measuring Gender Equality through a Composite Indicator 10 guiding principles Step 1.

Measuring Gender Equality through a Composite Indicator 10 guiding principles Step 1.

Measuring Gender Equality through a Composite Indicator 10 guiding principles Step 1. Developing a conceptual framework Step 2. Selecting indicators Step 3. Imputation of missing data Step 4. Multivariate analysis Step 5.

840 views • 37 slides
Revisiting the Visitor: the Just Do It Pattern C++ L ISP Step 1: plain L ISP Step 2: brute

Revisiting the Visitor: the Just Do It Pattern C++ L ISP Step 1: plain L ISP Step 2: brute

Visitor: Just Do It Didier Verna Introduction Revisiting the Visitor: the Just Do It Pattern C++ L ISP Step 1: plain L ISP Step 2: brute force Step 3: first class Didier Verna Step 4: mapping Step 5: generic map State

484 views • 22 slides
Activation, Transition & Close Out Step 7 May 2013 Project Management Concept Step 1:

Activation, Transition & Close Out Step 7 May 2013 Project Management Concept Step 1:

Activation, Transition & Close Out Step 7 May 2013 Project Management Concept Step 1: Needs Development Step 2: Scope Development Step 3: Procurement of Design Team Step 4: Design Step 5: Bid/Procurement of Contractor Step 6:

320 views • 12 slides
Quick guide Step 1: Purchasing a RSFirewall! membership Step 2: Download RSFirewall! Step 3:

Quick guide Step 1: Purchasing a RSFirewall! membership Step 2: Download RSFirewall! Step 3:

Quick guide Step 1: Purchasing a RSFirewall! membership Step 2: Download RSFirewall! Step 3: Installing RSFirewall! Step 4: Scan the Joomla! installation 4.1 Run the System Check 4.2 Fix the security vulnerabilities 4.2.1 Joomla! And

279 views • 17 slides

More recommend