' $ Algebraic Semiotics and User In terface Design Joseph A. Goguen Departmen t of Computer Science & Engineering Univ ersit y of California at San Diego & %
' $ ABSTRA CT HCI lac ks scien ti�c theories for design; so new media, new metaphors (b ey ond the desktop), new hardw are, non-standard users (e.g., with disabilities) can b e c hallenging. Semiotics seems natural, but (1) lac ks mathematical basis, (2) considers single signs (no v els, �lms, etc.), not represen tations; (3) do esn't address dynamic signs, or (4) so cial issues, e.g., for co op erativ e w ork. Algebraic semiotics de�nes sign system & represen tation, giv es calculus of represen tation & represen tation qualit y . Case studies on bro wsable pro of displa ys, scien ti�c visualization, natural language metaphor, blending, h umor. So cial foundation uses ideas from ethnometho dology . & %
' $ Outline Motiv ation: Some Problems 1. Algebraic Semiotics 2. Calculus of Represen tation 3. Case Studies 4. Summary & F uture Researc h 5. & %
4 ' $ 1. Motiv ation: Some Problems Most HCI results are: sp ecialized & precise (e.g., Fitt's la w), or else � general but of uncertain reliabilit y & generalit y (e.g., proto col � analysis, questionnaires, case studies, usabilit y studies). What w e need are scien ti�c theories to guide design, e.g., for new media, � new metaphors (b ey ond the desktop), � new hardw are, � non-standard users (e.g., with disabilities). � & %
5 ' $ Semiotics, the general theory of signs, seems natural for a general HCI framew ork. But it 1. do es not ha v e mathematical st yle & so do es not supp ort engineering applications; 2. only considers single signs or sign systems (e.g., no v el, �lm), not represen ting signs in one system b y signs in another, as needed for in terfaces; 3. has not addressed dynamic signs, as needed for user in teraction; 4. has not considered so cial issues, as arise in co op erativ e w ork; 5. ignores the situated, em b o died asp ect of sign use. & %
6 ' $ 2. Algebraic Semiotics Algebraic Semiotics pro vides: precise algebraic de�nitions for sign system & represen tation; � calculus of represen tation, with la ws ab out op erations for � com bining represen tations; precise w a ys to compare qualit y of represen tations. � Ha v e case studies on bro wsable pro of displa ys, scien ti�c visualization, natural language metaphor, blending, & h umor. So cial foundations grounded in ideas from ethnometho dology: semiosis, the creation of meaning, is situated, em b o died, etc. & %
7 ' $ 2.1 Signs and Sign Systems Signs should not b e studied in isolation, but rather � as elemen ts of systems of related signs, e.g., � v o w el systems, tra�c signs, alphab ets, n umerals, n um b ers. Signs ma y ha v e parts, subparts, etc., of di�eren t sorts. � Sign parts ma y ha v e di�eren t saliency, determined b y ho w � constructed. Signs b ecome what they are b y ha ving di�eren t attributes than other signs { clear from mac hine learning of patterns. Same sign in di�eren t system has di�eren t meaning { e.g., alphab ets. Com bines ideas of P eirce (sign), Saussure (structure), Goguen (ADTs). & %
8 ' $ F ormalize sign system as algebraic theory with data, plus 2 sp eci�c semiotic items: - signature for sorts, subsorts & op erations (constructors & selectors); - axioms (e.g. equations) as constrain ts; - data sorts & functions; - lev els for sorts; - priorit y ordering on constructors. Sorts classify signs, op erations construct signs, data sorts pro vide v alues for attributes of signs, lev els & priorities indicate saliency . This is not the formal v ersion; also not necessarily �nal. Di�ers from approac hes of Gen tner, Carroll, etc. - axiomatic with lo ose seman tics, not set-based; giv es a language, not a mo del; this allo ws partial mo dels, op en structure, etc. & %
9 ' $ 2.2 Represen tation User in terface design means designing go o d represen tations. E.g., GUIs represen t functionalit y with icons, men us, etc. Basic insigh t: represen tations are maps M : S S of sign ! 1 2 systems, called semiotic morphisms, preserving as m uc h as reasonable: - sorts & subsorts, - ops, preserving source & target sorts, - axioms to consequences of axioms, - data & functions, - lev els of sorts, - priorit y of constructors. \Reasonable" quali�cation due to need for tradeo�s. & %
10 ' $ 2.3 Simple Examples 1. S { English sen tences. E 2. S { parse trees for English sen tences. T 3. S { prin ted page format. P 4. P : S S { parsing. ! E T 5. H : S S { phrase structure represen tation. ! T P Time �ies lik e an arro w. [[ time ] [[ f l ies ] [[ l ik e ] [[ an ] [ ar r ow ] ] ] ] ] . N V P Det N NP PP VP S Can't alw a ys preserv e ev erything - resulting displa y ma y b e to o complex for h umans. And sometimes just w an t to summarize some data set. & %
11 ' $ 2.4 Qualit y of Represen tation Con ten t means v alues of selector ops, e.g., size, color. Easy to de�ne sort preserving, constructor preserving, lev el � preserving, con ten t preserving, etc. But not v ery useful since often are preserv ed. � not Instead, de�ne more sort preserving, more lev el preserving, � more constructor preserving, more con ten t preserving, etc. These comparativ e notions de�ne orderings on morphisms. � Can com bine orderings to get righ t one for giv en application. � 0 Giv en S; S , one ma y preserv e more lev els, other more con ten t. � More imp ortan t to preserv e structure than con ten t. � More imp ortan t to preserv e lev els than priorit y . � Also it's easier to describ e structure. � & %
12 ' $ 3. Calculus of Represen tation Can comp ose morphisms & so study comp osed represen tations, as arise in iterativ e design. Ha v e iden tit y & asso ciativ e la ws: A ; 1 = A S 1 ; B = B S A ; ( B ; C ) = ( A ; B ) ; C Therefore ha v e a category. This giv es other simple la ws, plus notions: isomorphism of sign systems, sum & pro duct of sign systems & represen tations, plus m uc h more (see follo wing). & %
13 ' $ 3.1 Blending F auconnier & T urner studied blending metaphors, using conceptual spaces { sign systems with only constan ts & relations. Conceptual blend of maps with same source, the generic space, & targets called input spaces, com bining their features in blend space. B � @ I 6 � @ � @ I I 1 2 @ I � @ � @ � G W e generalize to arbitrary sign systems, morphisms, & diagrams. & %
14 ' $ Examples: house b oat; road kill; computer virus; arti�cial life; jazz piano; conceptual space; blend diagram; ... Blend diagram suggests categorical pushout { but do esn't w ork, since blends not unique. Example: \house b oat" has 4 di�eren t maximal blends: � 1. houseb oat; 2. b oathouse; 3. amphibious R V; 4. b oat for mo ving houses (!). But since ordered category, use \lax" pushout: has non-unique result; and � can actually calculate the 4 blends ab o v e! � Order b y f g i� g preserv es as m uc h con ten t as f , as man y � axioms as f , and is as inclusiv e as f . & %
15 ' $ 3.2 Some La ws � A A 1 = � � A A 1 = � � A B B A = � � A ( B C ) � ( A B ) C = � � � � a b � b a = � � a ( b c ) � ( b ; a ) c = � � � ( a b ) c � a ( b ; c ) = � � � A; B ; C can b e either sign systems or semiotic morphisms. Pro duct is sp ecial blend with common space empt y; sum of theories giv es mo del pro duct. So pro duct la ws are sp ecial blends la ws. & %
16 ' $ 4. Case Studies 1. Blending (already discussed). 2. Metaphor (similar to F auconnier & T urner). 3. Scien ti�c visualization. 4. Pro of presen tation. 5. Humor. So w e will do items 3, 4, 5. & %
Recommend
More recommend
