Natural Language Processing Historical Document Transcription Dan - PowerPoint PPT Presentation

Natural Language Processing Historical Document Transcription Dan Klein — UC Berkeley Joint work with Taylor Berg-Kirkpatrick and Greg Durrett [ACL 2013]

Historical Document

Historical Document Old Bailey Court Proceedings 1775

Transcription Document Image

Transcription Transcription Document Image (Google Tesseract) and Ch’: priftmer anhc bar. Jacob Lazarus and his IHP1 uh: prifoner. were both together when! rcccivcd lhczn. I fold eievén pair of than for xiirce guincas, and dclivcrcd the rcll'l:.in- d:r hack lo :11: prifuner. 1 fold ftvcn pairof filk to Mark Simpcr : nncpuir of mixcd. and. mo pair of Ifircad to lhz: foolnun, and on: pair of zhrzad to lh: barber. ' Q: What is the foolmarfs name? Fraum Mgfzr. I dun’: know. Hairy Hzrvir. l was flandingar the Camp Icr waizin far the thcrrilfs ufliceruo employ in: : Mo 3‘: daughter came for me to 0 am! take the prifoncr. 1 Wm! to |hc Old aailcy

Pipelined Approach

Pipelined Approach

Pipelined Approach

Pipelined Approach

Pipelined Approach

Pipelined Approach m

Pipelined Approach m o

Pipelined Approach m o d

Historical Document

Unknown Fonts

Unknown Fonts po

Unknown Fonts po

Unknown Fonts po

Unknown Fonts long s glyph

Wandering Baseline

Wandering Baseline

Wandering Baseline

Wandering Baseline

Uneven Inking

Uneven Inking

Uneven Inking

Uneven Inking

Various Historical Documents 1725 1875 1823 1883:

Our Approach

Our Approach po

Our Approach po

Our Approach po

Generative Model p r i s o n e r

Generative Model p r i s o n e r

Generative Model p r i s o n e r

Generative Model p r i s o n e r

Generative Model Language Model p r i s o n e r

Generative Model p r i s o n e r Typesetting Model

Generative Model p r i s o n e r Typesetting Model

Generative Model p r i s o n e r Typesetting Model

Generative Model p r i s o n e r Typesetting Model

Generative Model p r i s o n e r Typesetting Model

Generative Model p r i s o n e r Typesetting Model

Generative Model p r i s o n e r Typesetting Model

Generative Model p r i s o n e r Typesetting Model

Generative Model p r i s o n e r Typesetting Model

Generative Model p r i s o n e r Typesetting Model

Generative Model p r i s o n e r Typesetting Model

Generative Model p r i s o n e r Typesetting Model

Generative Model p r i s o n e r Typesetting Model

Generative Model p r i s o n e r Typesetting Model

Generative Model p r i s o n e r Rendering Model

Generative Model p r i s o n e r Rendering Model

Generative Model p r i s o n e r Rendering Model

Generative Model p r i s o n e r

Generative Model Language Model p r i s o n e r E P ( E )

Generative Model Language Model p r i s o n e r E P ( E ) Typesetting Model T · P ( T | E )

Generative Model Language Model p r i s o n e r E P ( E ) Typesetting Model T · P ( T | E ) Rendering Model X P ( X | E, T )

Generative Model Language Model E P ( E ) Typesetting Model T · P ( T | E ) Rendering Model X P ( X | E, T )

Language Model E

Language Model E

Language Model E r a t e i e i − 1 e i +1

Language Model E r a t e i e i − 1 e i +1 Kneser-Ney smoothed character 6-gram

Typesetting Model a e i

Typesetting Model a e i T

Typesetting Model a e i 5 1 T Left pad width l i

Typesetting Model a e i 5 1 T Left pad width l i g i 1 30 Glyph box width

Typesetting Model a e i 5 1 1 5 T Right pad Left pad width width l i g i r i 1 30 Glyph box width

Typesetting Model a e i 5 1 1 5 T Right pad Left pad width width l i g i r i 1 30 a Glyph box width

Typesetting Model a e i 5 1 1 5 T Right pad Left pad width width l i g i r i 1 30 a Glyph box width

Typesetting Model a e i 5 1 1 5 T Right pad Left pad width width l i g i r i 1 30 a Glyph box width a a a v i Vertical offset

Typesetting Model a e i 5 1 1 5 T Right pad Left pad width width l i g i r i 1 30 a d i Glyph box width a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a v i Vertical offset Inking level

Rendering Model

Rendering Model

Rendering Model Glyph box

Rendering Model Glyph box width g i Glyph box

Rendering Model Glyph box Vertical width offset g i v i Glyph box

Rendering Model Glyph box Vertical Inking width offset level d i g i v i Glyph box

Rendering Model Glyph box Vertical Inking width offset level d i g i v i Glyph box X

Rendering Model Glyph box Vertical Inking width offset level d i g i v i Glyph shape parameters Glyph box X

Rendering Model Glyph box Vertical Inking width offset level d i g i v i Glyph shape parameters Glyph box X

Rendering Model Glyph box Vertical Inking width offset level d i g i v i Glyph shape parameters Glyph box X Bernoulli pixel probs

Rendering Model Glyph box Vertical Inking width offset level d i g i v i Glyph shape parameters Glyph box X Sample pixels Bernoulli pixel probs

Rendering Model Glyph box Vertical Inking width offset level d i g i v i Glyph shape parameters Glyph box X Sample pixels Bernoulli pixel probs

Rendering Model Glyph box Vertical Inking width offset level d i g i v i Glyph shape parameters Glyph box X Sample pixels Bernoulli pixel probs

Rendering Model Glyph box Vertical Inking width offset level d i g i v i Glyph shape parameters Glyph box X Sample pixels Bernoulli pixel probs

Rendering Model Glyph box Vertical Inking width offset level d i g i v i Glyph shape parameters Glyph box X Sample pixels Bernoulli pixel probs

Rendering Model Glyph box Vertical Inking width offset level d i g i v i Glyph shape parameters Glyph box X Sample pixels Bernoulli pixel probs

Rendering Model Glyph box Vertical Inking width offset level d i g i v i Glyph shape parameters Glyph box X Sample pixels Bernoulli pixel probs

Rendering Model Glyph box Vertical Inking width offset level d i g i v i Glyph shape parameters Glyph box X Sample pixels Bernoulli pixel probs

Rendering Model Glyph box Vertical Inking width offset level d i g i v i Glyph shape parameters Glyph box X Sample pixels Bernoulli pixel probs

Rendering Model Glyph box Vertical Inking width offset level d i g i v i Glyph shape parameters Glyph box X Sample pixels Bernoulli pixel probs

Rendering Model Glyph box Vertical Inking width offset level d i g i v i Glyph shape parameters Glyph box X Sample pixels Bernoulli pixel probs

Rendering Model Glyph box Vertical Inking width offset level d i g i v i Glyph shape parameters Glyph box X Sample pixels Bernoulli pixel probs

Rendering Model Glyph box Vertical Inking width offset level d i g i v i Glyph shape parameters Glyph box X Sample pixels Bernoulli pixel probs

Rendering Model Glyph box Vertical Inking width offset level d i g i v i Glyph shape parameters Glyph box X Sample pixels Bernoulli pixel probs

Rendering Model Glyph box Vertical Inking width offset level d i g i v i Glyph shape parameters Glyph box X Sample pixels Bernoulli pixel probs

Log-linear Interpolation

Log-linear Interpolation Glyph shape parameters φ Bernoulli pixel probs θ

Log-linear Interpolation Glyph shape parameters φ Bernoulli pixel probs θ