Outline read Chapter suggested exercises - PDF document

Outline �read Chapter �� �suggested exercises ���� ���� ���� ���� � Learning from examples � General�to�sp eci�c ordering o v er h yp otheses � V ersion spaces and candidate elimination algorithm � Pic king new examples � The need for inductiv e bias Note� simple approac h assuming no noise� illustrate s k ey concepts �� lecture slides for textb o ok Machine L e arning � T� Mitc hell� McGra w Hill� ����

T raining Examples for Enjo ySp ort Sky T emp Humid Wind W ater F orecst Enjo ySpt Sunn y W arm Normal Strong W arm Same Y es Sunn y W arm High Strong W arm Same Y es Rain y Cold High Strong W arm Change No Sunn y W arm High Strong Co ol Change Y es What is the general concept� �� lecture slides for textb o ok Machine L e arning � T� Mitc hell� McGra w Hill� ����

Represen ting Hyp otheses Man y p ossible represen tations Here� is conjunction of constrain ts on attributes h Eac h constrain t can b e � a sp ec�c v alue �e�g�� � m � W ater W ar � don�t care �e�g�� � W ���� ater � no v alue allo w ed �e�g���W ater� � �� F or example� Sky AirT emp Humid Wind W ater F orecst h S ame i � � � unny S tr ong S �� lecture slides for textb o ok Machine L e arning � T� Mitc hell� McGra w Hill� ����

Protot ypical Concept Learning T ask � Giv en� Instances � P ossible da ys� eac h describ ed b y � X the attributes Sky� A irT emp� Humidity� Wind� Water� F or e c ast T arget function c � � � f � � � g � E nj oy S por t X Hyp otheses � Conjunctions of literals� E�g� � H h � � � i � � � � � C ol d� H ig h� T raining examples � P ositiv e and negativ e � D examples of the target function h x � i � h x � i c � x c � x � � � � � m m � � � A h yp othesis in suc h that Determine� h H h � x � � c � x � for all in � x D �� lecture slides for textb o ok Machine L e arning � T� Mitc hell� McGra w Hill� ����

An y The inductiv e learning h yp othesis� h yp othesis found to appro ximate the target function w ell o v er a su�cien tly large set of training examples will also appro ximate the target function w ell o v er other unobserv ed examples� �� lecture slides for textb o ok Machine L e arning � T� Mitc hell� McGra w Hill� ����

Instance� Hyp otheses� and More� General�Than Instances X Hypotheses H Specific h h x 1 3 1 h x 2 2 General x = <Sunny, Warm, High, Strong, Cool, Same> h = <Sunny, ?, ?, Strong, ?, ?> 1 1 x = <Sunny, Warm, High, Light, Warm, Same> h = <Sunny, ?, ?, ?, ?, ?> 2 2 h = <Sunny, ?, ?, ?, Cool, ?> 3 �� lecture slides for textb o ok Machine L e arning � T� Mitc hell� McGra w Hill� ����

Algorithm Find�S �� Initiali ze to the most sp eci�c h yp othesis in h H �� F or eac h p ositiv e training instance x � F or eac h attribute constrain t in a h i If the constrain t in is satis�ed b y a h x i Then do nothing Else replace in b y the next more a h i general constrain t that is satis�ed b y x �� Output h yp othesis h �� lecture slides for textb o ok Machine L e arning � T� Mitc hell� McGra w Hill� ����

Hyp othesis Space Searc h b y Find�S Instances X Hypotheses H h 0 Specific - x 3 h 1 h 2,3 x + + x 1 2 General + h 4 x 4 < ∅ , ∅ , ∅ , ∅ , ∅ , ∅ > h = 0 x = < S u n n y W a r m N o r m a l S h = <Sunny Warm Normal Strong Warm Same t r o n g W a r m S a m e > , + 1 1 x = <Sunny Warm High Strong Warm Same>, + h = <Sunny Warm ? Strong Warm Same> 2 2 x = <Rainy Cold High Strong Warm Change>, - h = <Sunny Warm ? Strong Warm Same> 3 3 x = <Sunny Warm High Strong Cool Change>, + h = <Sunny Warm ? Strong ? ? > 4 4 �� lecture slides for textb o ok Machine L e arning � T� Mitc hell� McGra w Hill� ����

Complain ts ab out Find�S � Can�t tell whether it has learned concept � Can�t tell when training data inconsisten t � Pic ks a maximally sp eci�c �wh y�� h � Dep ending on � there migh t b e sev eral� H �� lecture slides for textb o ok Machine L e arning � T� Mitc hell� McGra w Hill� ����

V ersion Spaces A h yp othesis is with a set of h consisten t training examples of target concept if and D c only if h � x � � c � x � for eac h training example h x� c � x � i in � D � � �h x� c � x � i � onsistent � h� � � h � x � � c � x � C D D The space � � with resp ect to v ersion V S H �D h yp othesis space and training examples � H D is the subset of h yp otheses from consisten t H with all training examples in � D � f h � j C onsistent � h� � g V S H D H �D �� lecture slides for textb o ok Machine L e arning � T� Mitc hell� McGra w Hill� ����

The Algorithm� List�Then�Elimin ate �� � a list con taining ev ery V er sionS pace h yp othesis in H �� F or eac h training example� h x� c � x � i remo v e from an y h yp othesis for V er sionS pace h whic h h � x � � � c � x � �� Output the list of h yp otheses in V er sionS pace �� lecture slides for textb o ok Machine L e arning � T� Mitc hell� McGra w Hill� ����

Example V ersion Space { <Sunny, Warm, ?, Strong, ?, ?> } S: <Sunny, ?, ?, Strong, ?, ?> <Sunny, Warm, ?, ?, ?, ?> <?, Warm, ?, Strong, ?, ?> { <Sunny, ?, ?, ?, ?, ?>, <?, Warm, ?, ?, ?, ?> } G: �� lecture slides for textb o ok Machine L e arning � T� Mitc hell� McGra w Hill� ����

Represen ting V ersion Spaces The oundary � G� of v ersion space General b is the set of its maximally general V S H �D mem b ers The oundary � S� of v ersion space Sp eci�c b is the set of its maximally sp eci�c V S H �D mem b ers Ev ery mem b er of the v ersion space lies b et w een these b oundaries � f h � j � � s � �� � g � G �� g � � s � g V S H S h H �D where � means is more general or equal to x y x y �� lecture slides for textb o ok Machine L e arning � T� Mitc hell� McGra w Hill� ����

Candidate Eliminati o n Algorithm � maximally general h yp otheses in G H � maximally sp eci�c h yp otheses in S H F or eac h training example d � do � If is a p ositiv e example d Remo v e from an y h yp othesis inconsisten t � G with d F or eac h h yp othesis in that is not � s S consisten t with d � Remo v e from s S � Add to all minimal generalizations of S h s suc h that �� is consisten t with d � and h �� some mem b er of is more general than G h � Remo v e from an y h yp othesis that is more S general than another h yp othesis in S � If is a negativ e example d �� lecture slides for textb o ok Machine L e arning � T� Mitc hell� McGra w Hill� ����

Remo v e from an y h yp othesis inconsisten t � S with d F or eac h h yp othesis in that is not � g G consisten t with d � Remo v e from g G � Add to all minimal sp ecializat i ons of G h g suc h that �� is consisten t with d � and h �� some mem b er of is more sp eci�c than S h � Remo v e from an y h yp othesis that is less G general than another h yp othesis in G �� lecture slides for textb o ok Machine L e arning � T� Mitc hell� McGra w Hill� ����

Example T race S0: {<Ø, Ø, Ø, Ø, Ø, Ø>} G 0: {<?, ?, ?, ?, ?, ?>} �� lecture slides for textb o ok Machine L e arning � T� Mitc hell� McGra w Hill� ����

What Next T raining Example� { <Sunny, Warm, ?, Strong, ?, ?> } S: <Sunny, ?, ?, Strong, ?, ?> <Sunny, Warm, ?, ?, ?, ?> <?, Warm, ?, Strong, ?, ?> { <Sunny, ?, ?, ?, ?, ?>, <?, Warm, ?, ?, ?, ?> } G: �� lecture slides for textb o ok Machine L e arning � T� Mitc hell� McGra w Hill� ����