Background Context Theories Tools Language as an Algebra Daoud Clarke Department of Computer Science University of Hertfordshire The Categorical Flow of Information in Quantum Physics and Linguistics, Oxford, 2010 Daoud Clarke Language as an Algebra
Background Context Theories Tools Overview Background 1 Distributional Semantics Context-theoretic Semantics Context Theories 2 Motivating Example Example Tools 3 Quotient Algebras Finite-dimensional Algebras Algebras from Monoids Daoud Clarke Language as an Algebra
Background Distributional Semantics Context Theories Context-theoretic Semantics Tools Beyond lexical distributional semantics Distributional semantics: Hypothesis of Harris (1968) LSA, distributional similarity etc. Many applications Good for words/short phrases How can we go beyond the lexical domain? Data sparseness Daoud Clarke Language as an Algebra
Background Distributional Semantics Context Theories Context-theoretic Semantics Tools Beyond lexical distributional semantics Distributional semantics: Hypothesis of Harris (1968) LSA, distributional similarity etc. Many applications Good for words/short phrases How can we go beyond the lexical domain? Data sparseness Daoud Clarke Language as an Algebra
Background Distributional Semantics Context Theories Context-theoretic Semantics Tools Physicists (xkcd.com) Daoud Clarke Language as an Algebra
Background Distributional Semantics Context Theories Context-theoretic Semantics Tools Physicists (xkcd.com) Daoud Clarke Language as an Algebra
Background Distributional Semantics Context Theories Context-theoretic Semantics Tools Context-theoretic Semantics Suggestion: explore the use of unital associative algebras over the real numbers An algebra is a vector space together with a bilinear multiplication: x · ( y + z ) = x · y + x · z ( x + y ) · z = x · z + y · z Associative algebras form a monoidal category K -Alg Daoud Clarke Language as an Algebra
Background Distributional Semantics Context Theories Context-theoretic Semantics Tools Context-theoretic Semantics Suggestion: explore the use of unital associative algebras over the real numbers An algebra is a vector space together with a bilinear multiplication: x · ( y + z ) = x · y + x · z ( x + y ) · z = x · z + y · z Associative algebras form a monoidal category K -Alg Daoud Clarke Language as an Algebra
Background Distributional Semantics Context Theories Context-theoretic Semantics Tools Context-theoretic Semantics Suggestion: explore the use of unital associative algebras over the real numbers An algebra is a vector space together with a bilinear multiplication: x · ( y + z ) = x · y + x · z ( x + y ) · z = x · z + y · z Associative algebras form a monoidal category K -Alg Daoud Clarke Language as an Algebra
Background Distributional Semantics Context Theories Context-theoretic Semantics Tools Al-kit¯ ab al-mukhtas .ar f¯ ı h .is¯ abi-l-jabr wa’l-muq¯ abala Daoud Clarke Language as an Algebra
Background Distributional Semantics Context Theories Context-theoretic Semantics Tools Algebra over a field An algebra over a field: abstraction of the space of operators on a vector space Hilbert space operators → C ∗ -algebras Vector lattice operators → lattice-ordered algebras Matrices of order n form an algebra under normal matrix multiplication Daoud Clarke Language as an Algebra
Background Distributional Semantics Context Theories Context-theoretic Semantics Tools Algebra over a field An algebra over a field: abstraction of the space of operators on a vector space Hilbert space operators → C ∗ -algebras Vector lattice operators → lattice-ordered algebras Matrices of order n form an algebra under normal matrix multiplication Daoud Clarke Language as an Algebra
Background Distributional Semantics Context Theories Context-theoretic Semantics Tools Algebra over a field An algebra over a field: abstraction of the space of operators on a vector space Hilbert space operators → C ∗ -algebras Vector lattice operators → lattice-ordered algebras Matrices of order n form an algebra under normal matrix multiplication Daoud Clarke Language as an Algebra
Background Distributional Semantics Context Theories Context-theoretic Semantics Tools Algebra over a field An algebra over a field: abstraction of the space of operators on a vector space Hilbert space operators → C ∗ -algebras Vector lattice operators → lattice-ordered algebras Matrices of order n form an algebra under normal matrix multiplication Daoud Clarke Language as an Algebra
Background Distributional Semantics Context Theories Context-theoretic Semantics Tools Context-theoretic Framework A context theory is a tuple � A , A , ˆ , φ � The meaning of a string x ∈ A ∗ is a vector ˆ x ∈ A A ∗ is the free monoid on an alphabet A A is a real associative unital algebra The mapping ˆ from A ∗ to A is a monoid homomorphism the cat sat = � � the · � cat · � sat Multiplication · is distributive with respect to vector space addition (bilinearity) φ is a linear functional on A indicating probability Daoud Clarke Language as an Algebra
Background Distributional Semantics Context Theories Context-theoretic Semantics Tools Context-theoretic Framework A context theory is a tuple � A , A , ˆ , φ � The meaning of a string x ∈ A ∗ is a vector ˆ x ∈ A A ∗ is the free monoid on an alphabet A A is a real associative unital algebra The mapping ˆ from A ∗ to A is a monoid homomorphism the cat sat = � � the · � cat · � sat Multiplication · is distributive with respect to vector space addition (bilinearity) φ is a linear functional on A indicating probability Daoud Clarke Language as an Algebra
Background Distributional Semantics Context Theories Context-theoretic Semantics Tools Context-theoretic Framework A context theory is a tuple � A , A , ˆ , φ � The meaning of a string x ∈ A ∗ is a vector ˆ x ∈ A A ∗ is the free monoid on an alphabet A A is a real associative unital algebra The mapping ˆ from A ∗ to A is a monoid homomorphism the cat sat = � � the · � cat · � sat Multiplication · is distributive with respect to vector space addition (bilinearity) φ is a linear functional on A indicating probability Daoud Clarke Language as an Algebra
Background Distributional Semantics Context Theories Context-theoretic Semantics Tools Context-theoretic Framework A context theory is a tuple � A , A , ˆ , φ � The meaning of a string x ∈ A ∗ is a vector ˆ x ∈ A A ∗ is the free monoid on an alphabet A A is a real associative unital algebra The mapping ˆ from A ∗ to A is a monoid homomorphism the cat sat = � � the · � cat · � sat Multiplication · is distributive with respect to vector space addition (bilinearity) φ is a linear functional on A indicating probability Daoud Clarke Language as an Algebra
Background Distributional Semantics Context Theories Context-theoretic Semantics Tools Context-theoretic Framework A context theory is a tuple � A , A , ˆ , φ � The meaning of a string x ∈ A ∗ is a vector ˆ x ∈ A A ∗ is the free monoid on an alphabet A A is a real associative unital algebra The mapping ˆ from A ∗ to A is a monoid homomorphism the cat sat = � � the · � cat · � sat Multiplication · is distributive with respect to vector space addition (bilinearity) φ is a linear functional on A indicating probability Daoud Clarke Language as an Algebra
Background Motivating Example Context Theories Example Tools Motivating Example: Context Algebras Meaning as context Formal language → syntactic monoid Fuzzy language → context algebra E.g. Latent Dirichlet Allocation → commutative algebra Daoud Clarke Language as an Algebra
Background Motivating Example Context Theories Example Tools Motivating Example: Context Algebras Meaning as context Formal language → syntactic monoid Fuzzy language → context algebra E.g. Latent Dirichlet Allocation → commutative algebra Daoud Clarke Language as an Algebra
Background Motivating Example Context Theories Example Tools Motivating Example: Context Algebras Meaning as context Formal language → syntactic monoid Fuzzy language → context algebra E.g. Latent Dirichlet Allocation → commutative algebra Daoud Clarke Language as an Algebra
Background Motivating Example Context Theories Example Tools Motivating Example: Context Algebras Meaning as context Formal language → syntactic monoid Fuzzy language → context algebra E.g. Latent Dirichlet Allocation → commutative algebra Daoud Clarke Language as an Algebra
Background Motivating Example Context Theories Example Tools Context Algebras: How it works Fuzzy language C : A ∗ → [ 0 , 1 ] x : A ∗ × A ∗ → [ 0 , 1 ] by For x ∈ A ∗ , define ˆ ˆ x ( y , z ) = C ( yxz ) Define A as vector space generated by { ˆ x : x ∈ A ∗ } Choose a basis and use multiplication on A ∗ to define multiplication on A Multiplication is the same, no matter what basis we choose! Daoud Clarke Language as an Algebra
Recommend
More recommend
