text filtering in poesia
play

Text Filtering in POESIA A General Introduction Mark Hepple & - PowerPoint PPT Presentation

Mar 19, 2023 •360 likes •478 views

Text Filtering in POESIA A General Introduction Mark Hepple & Neil Ireson Sheffield University Internet Pornography (2003) Volume of material is immense: 4.2 million websites (12%) 372 million pages 68 million search engine

  1. Text Filtering in POESIA A General Introduction Mark Hepple & Neil Ireson Sheffield University

  2. Internet Pornography (2003) Volume of material is immense: • 4.2 million websites (12%) • 372 million pages • 68 million search engine requests/day (25%) • 1.5 billion downloads/month (35%)

  3. Pornographers Tricks • Cyber-squatting : buy legitimate sounding domain names for porn sites – whitehouse.com (c.f. whitehouse.org), civilwarbattles.com, tourdefrance.com • Mis-spelling : buy domain names that are mis-spellings of important sites for porn e.g. googlle.com

  4. Pornographers Tricks (cont) • Doorway scams : non-porn pages designed for indexing by search engines – user accessing page is redirected to porn site • Porn-napping : buy lapsed domains for porn sites (sell back to ex-owner for large fee) e.g. moneyopolis.org: money management site for kids by Ernst & Young

  5. Current Filtering Systems • Blacklists – addresses of ‘unacceptable’ sites • Keywords – block pages containing certain words/phrases BUT … • many words ambiguous, meaning depends on context – inclusion (resp. exclusion) of these terms as keywords results in over (resp. under) blocking – much offensive content in image form • need for effective filtering based on content

  6. Text filtering in POESIA • POESIA: multiple filters, including filters for image & textual content • Textual content filters use NLP methods to enhance recognition of categorisation relevant uses of terms • Textual content filters are language specific for English, Italian, Spanish (+ limited for French) • Approach requires a language identification component

  7. Language Identifier • Models probability distribution of 3- character n-grams – Parallel language text • 11 European languages (~560 Mbytes total) – Smoothing: Good-Turing Estimation – Term (Feature) selection: Information-Gain – Similarity metric: Entropy measure

  8. Language Identifier Evaluation • Non-porn pages = ~3% error • Porn pages = ~2% error • Pages with low amount of text – n-grams <30 (3.5% of pages): ~45% error – 30< n-grams <100 (6.5% of pages): ~12% error – 200< n-grams (80% of pages) ~0% error • Problems – Imperfect & impure language use – Specific terminology • Domain specific identifier – Proper names

  9. Language Specific Text Filters • NLP based filtering quite computationally expensive, useful to provide both two filtering modes: – Light: gives fast accept/reject for simple cases – Heavy: applies more complex methods for difficult cases, i.e. cases not resolved by light filtering • Light Filters – Statistical “bag-of-word” models – Stemming, term-selection, term-weighting… • Heavy Filters – NLP Techniques – Machine Learning Techniques – Localised context

  10. Data Collection for Text Filtering • Both porn/non-porn data collected automatically by spidering from the Google directory • Approach has advantages/disadvantages – Pros: • large dynamic sample of web • 72 specific language categories – Cons: • biased sample • misclassified pages

Recommend

Text Classification for Web Filtering Fabrizio Sebastiani Istituto di Scienza e Tecnologie

Text Classification for Web Filtering Fabrizio Sebastiani Istituto di Scienza e Tecnologie

c Fabrizio Sebastiani 1 Final POESIA Workshop Present and Future of Open-Source Content-Based Web filtering Pisa, IT 21-22 January, 2004 Text Classification for Web Filtering Fabrizio Sebastiani Istituto di Scienza e

196 views • 19 slides
POESIA: Public Open-source Environment for a Safer Internet Access Evaluation of POESIA Beta

POESIA: Public Open-source Environment for a Safer Internet Access Evaluation of POESIA Beta

POESIA: Public Open-source Environment for a Safer Internet Access Evaluation of POESIA Beta Release Sara Carro Martnez Telefnica I+D Presented by Steve Presland Liverpool Hope 2 Cdigo 00/00 Evaluation of POESIA Alpha and Beta

642 views • 33 slides
The POESIA Decision Mechanism Alberto Raggioli, Stefan Guerra M.E.T.A. S.r.l. POESIA Final

The POESIA Decision Mechanism Alberto Raggioli, Stefan Guerra M.E.T.A. S.r.l. POESIA Final

The POESIA Decision Mechanism Alberto Raggioli, Stefan Guerra M.E.T.A. S.r.l. POESIA Final Workshop Pisa 21-22/01/2004 What a Decision Mechanism (DM) is? A system that, given some input values, returns a response which summarizes input

360 views • 12 slides
TEXT FILTERING FOR SPANISH Enrique Puertas Sanz Universidad Europea de Madrid Contents

TEXT FILTERING FOR SPANISH Enrique Puertas Sanz Universidad Europea de Madrid Contents

TEXT FILTERING FOR SPANISH Enrique Puertas Sanz Universidad Europea de Madrid Contents Goals Scientific approach Design and implementation Current results Goals Effective filtering of Spanish text dealing with

266 views • 23 slides
1 An Filtering System that Monitors Document Search Engines Can Help, But Not Enough!

1 An Filtering System that Monitors Document Search Engines Can Help, But Not Enough!

Filtering May be Your Work Adaptive Information Filtering Using Bayesian Graphical Models Yi Zhang Baskin School of Engineering University of California Santa Cruz yiz@soe.ucsc.edu 1 2 Filtering May be Your Work Filtering May be Your Work

415 views • 8 slides
1 Text Nave Bayes Algorithm Text Nave Bayes Algorithm (Train) (Test) Let V be the

1 Text Nave Bayes Algorithm Text Nave Bayes Algorithm (Train) (Test) Let V be the

Text Categorization Applications Web pages Recommending Yahoo-like classification CS 391L: Machine Learning Newsgroup/Blog Messages Text Categorization Recommending spam filtering Sentiment analysis for marketing

418 views • 6 slides
Traffic Control Mechanisms Filtering Source address filtering Other forms of filtering

Traffic Control Mechanisms Filtering Source address filtering Other forms of filtering

Traffic Control Mechanisms Filtering Source address filtering Other forms of filtering Rate limits Protection against traffic analysis Padding Routing control Lecture 9 Page 1 CS 236 Online Source Address Filtering

366 views • 11 slides
CS490W: What is Collaborative Filtering? Collaborative Filtering (CF): Making recommendation

CS490W: What is Collaborative Filtering? Collaborative Filtering (CF): Making recommendation

CS490W: What is Collaborative Filtering? Collaborative Filtering (CF): Making recommendation decisions for a specific CS-490W user based on the judgments of users with similar tastes Collaborative Filtering Luo Si Train_User 1 1 5 3 3 4

628 views • 6 slides
FILTERING MACROECONOMIC DATA WienerKolmogorov Filtering of Stationary Sequences The classical

FILTERING MACROECONOMIC DATA WienerKolmogorov Filtering of Stationary Sequences The classical

FILTERING MACROECONOMIC DATA WienerKolmogorov Filtering of Stationary Sequences The classical theory of linear filtering was formulated independently by Norbert Wiener (1941) and Andrei Nikolaevich Kolmogorov (1941) during the Second World

437 views • 19 slides
Collaborative Filtering Alejandro Bellogn 1 , Jun Wang 2 , and Pablo Castells 1 1 Escuela

Collaborative Filtering Alejandro Bellogn 1 , Jun Wang 2 , and Pablo Castells 1 1 Escuela

Text Retrieval Methods for Item Ranking in Collaborative Filtering Alejandro Bellogn 1 , Jun Wang 2 , and Pablo Castells 1 1 Escuela Politcnica Superior, Universidad Autnoma de Madrid 2 Department of Computer Science, University College

246 views • 23 slides
Texture Filtering Longer bullets in the form of a paragraph are harder to read if there is

Texture Filtering Longer bullets in the form of a paragraph are harder to read if there is

Edit this text to create a Heading This subtitle is 20 points Cardinality-Constrained Bullets are blue They have 110% line spacing, 2 points before &amp; after Texture Filtering Longer bullets in the form of a paragraph are harder

783 views • 52 slides
Rao-Blackwellised Particle Filtering Based on Rao-Blackwellised Particle Filtering for Dynamic

Rao-Blackwellised Particle Filtering Based on Rao-Blackwellised Particle Filtering for Dynamic

Rao-Blackwellised Particle Filtering Based on Rao-Blackwellised Particle Filtering for Dynamic Bayesian Networks by Arnaud Doucet, Nando de Freitas, Kevin Murphy, and Stuart Russel Other sources Artificial Intelligence: A Modern

268 views • 26 slides
Text Mining in R ( tm 101) ViennaR Mario Annau, 22.2.2016

Text Mining in R ( tm 101) ViennaR Mario Annau, 22.2.2016

Text Mining in R ( tm 101) ViennaR Mario Annau, 22.2.2016 Textmining? Sta&lt;s&lt;cal analysis of textual data Use Cases include Spam Filtering

533 views • 11 slides
1 Recap: Particle Filtering Video of Demo Moderate Number of Particles Particles: track

1 Recap: Particle Filtering Video of Demo Moderate Number of Particles Particles: track

Particle Filtering CSE 473: Artificial Intelligence Particle Filters Filtering: approximate solution 0.0 0.1 0.0 Sometimes |X| is too big to use exact inference |X| may be too big to even store B(X) 0.0 0.0 0.2 E.g. X is

463 views • 3 slides
The Filtering Matrix Interrogating Internet Filtering and Surveillance Practices Worldwide Nart

The Filtering Matrix Interrogating Internet Filtering and Surveillance Practices Worldwide Nart

The Filtering Matrix Interrogating Internet Filtering and Surveillance Practices Worldwide Nart Villeneuve Director of Technical Research Citizen Lab, University of Toronto CACR 2004 The Filtering Matrix Technical &amp; non-technical

534 views • 15 slides
Title of an article [16 pt] Introduction [14 pt] Text. Text. Text. Text. Text. Text. Text. Text.

Title of an article [16 pt] Introduction [14 pt] Text. Text. Text. Text. Text. Text. Text. Text.

REQUIREMENTS FOR POSTER PRESENTATION 1. Volume of poster presentation (or Article) from 4 to 10 pages (A4 format). Please submit an electronic version by email. Along with the article must be submitted a review of a scientist . A review

71 views • 3 slides
Class 3: Advanced techniques Class Outline 3.1 Operators (define, uses) and filtering on

Class 3: Advanced techniques Class Outline 3.1 Operators (define, uses) and filtering on

Class 3: Advanced techniques Class Outline 3.1 Operators (define, uses) and filtering on properties (site:) 3.2 The filetype: operator 3.3 Removing invasive results with the minus (-) operator 3.4 OR operator and quoted text 3.5 The

328 views • 4 slides
An analysis of image filtering on WeChat Moments Jeffrey Knockel, Lotus Ruan, Masashi

An analysis of image filtering on WeChat Moments Jeffrey Knockel, Lotus Ruan, Masashi

An analysis of image filtering on WeChat Moments Jeffrey Knockel, Lotus Ruan, Masashi Crete-Nishihata Background Images increasingly used to communicate Image censorship understudied (Website blocking, text chat/posts, etc.)

1.06k views • 43 slides
A Gentle Introduction A Gentle Introduction to Bilateral Filtering to Bilateral Filtering and

A Gentle Introduction A Gentle Introduction to Bilateral Filtering to Bilateral Filtering and

A Gentle Introduction A Gentle Introduction to Bilateral Filtering to Bilateral Filtering and its Applications and its Applications Sylvain Paris MIT CSAIL Pierre Kornprobst INRIA Odysse Jack Tumblin Northwestern University

421 views • 19 slides
Collaborative Filtering Presentation by Alex Hugger Filtering Documents Mittwoch, 28. April 2010

Collaborative Filtering Presentation by Alex Hugger Filtering Documents Mittwoch, 28. April 2010

Collaborative Filtering Presentation by Alex Hugger Filtering Documents Mittwoch, 28. April 2010 Departement/Institut/Gruppe 2 Content-Based Methods Find other popular items by the same author or similar keywords Recommendation

858 views • 38 slides
Filtering Cubemaps Filtering Cubemaps Angular Extent Filtering and Edge Seam Fixup Methods

Filtering Cubemaps Filtering Cubemaps Angular Extent Filtering and Edge Seam Fixup Methods

Filtering Cubemaps Filtering Cubemaps Angular Extent Filtering and Edge Seam Fixup Methods Angular Extent Filtering and Edge Seam Fixup Methods John R. Isidoro 3D Application Research Group ATI Research Introduction Introduction Hardware

463 views • 25 slides
Deriving Filtering Algorithms Deriving Filtering Algorithms from Constraint Checkers from

Deriving Filtering Algorithms Deriving Filtering Algorithms from Constraint Checkers from

Deriving Filtering Algorithms Deriving Filtering Algorithms from Constraint Checkers from Constraint Checkers Nicolas Beldiceanu(1), Mats Carlsson(2) and Thierry Petit(1) (1) LINA FRE CNRS 2729, Ecole des Mines de Nantes, FR-44307 Nantes Cedex

642 views • 40 slides
Computer Graphics Texture Filtering Philipp Slusallek Filtering Magnification (Zoom-in)

Computer Graphics Texture Filtering Philipp Slusallek Filtering Magnification (Zoom-in)

Computer Graphics Texture Filtering Philipp Slusallek Filtering Magnification (Zoom-in) Map few texels onto many pixels Pixel Reconstruction filter: Nearest neighbor interpolation: Take the nearest texel Bilinear

392 views • 26 slides
Lesson 7 Rate Conversion Filtering and Downsampling interchange Filtering and Upsampling

Lesson 7 Rate Conversion Filtering and Downsampling interchange Filtering and Upsampling

Lesson 7 Rate Conversion Filtering and Downsampling interchange Filtering and Upsampling interchange Polyphase decomposition Polyphase Filtering Polyphase Filtering Polyphase filtering Example Example Example Example -n -n Example

474 views • 16 slides

More recommend