Extending the P o w er and Capacit y of Constrain t Satisfaction Net w orks Xinc h uan Zeng and T on y R� Martinez Computer Science Departmen t Brigham Y oung Univ ersit y July ���� �
Abstract This w ork fo cuses on impro ving the Hop�eld network for solv� ing optimization problems� Although m uc h w ork has b een done in this area� the p erformance of the Hop�eld net w ork is still not satisfactory in terms of v alid con v ergence and qualit y of solutions� W e address this issue in this w ork b y com bing a new activ ation function � E B A � and a new relax� ation pro cedure � C R � in order to impro v e the p erformance of the Hop�eld net w ork� Eac h of and has b een in� E B A C R dividually demonstrated capable of substan tially impro ving the p erformance� The com bined approac h has b een ev al� uated through ������ sim ulations based on ��� randomly generated cit y distributions of the �� �cit y tr aveling salesman pr oblem � The result sho ws that com bining the t w o metho ds �
is able to further impro v e the p erformance� Compared to without com bining with A � the com bined approac h C R E B increases the p ercen tage of v alid tou� rs b y ����� and de� creases the error rate b y ������ As compared to the original Hop�eld metho d �using neither E B A nor C R �� the com bined approac h increases the p ercen tage of v alid tours b y ������ and decreases the error rate b y ������ �
Com binatorial optimization problems Most of them are problems N P � Man y maxim um or minim um p oin ts � Examples� � T ra v eling Salesman Problem � T � S P � Routing in comm unication net w orks � Graph partitioning in circuit design � Wh y study b y Hop�eld net w ork� T S P is one early application b y Hop�eld net w ork �HN� T S P � is represen tativ e of com binational T S P � problems is a b enc hmark for comparing algorithms� T S P � �
A go o d algorithm for could b e mapp ed to solv e T S P � other com binatorial problems Hop�eld net w orks vs other heuristics Examples of other heuristics� �Divide�and�Conquer�� � �F orm outline con tour� The pro cedure of HN is more gener al while other metho ds � are more pr oblem�sp e ci�c � HN can use p ar al lel pro cessing while other metho ds usu� � ally use se quential pro cessing� HN can ha v e con v enien t hardw are implemen tation� � HN can ac hiev e an optimal or sub�optimal solution in � short time� �
Example� Arc hitecture for a ��cit y TSP� POSITION 1 2 3 4 5 6 7 A B V B2 C CITY D V E E6 F G F ully connected based on an energy function � Prop er random initial activ ations for neurons� � Net w ork is relaxed un til reac hing an equilibrium � �
Energy function for a �cit y TSP � N A B N N N N N N E � V V � V V X X X X X X X i X j X i Y i � � X �� i �� �� �j � � i i �� X �� �� �Y � � X j Y C D N N N N N � � � � � � V � � V N d V V X X X X X � X i � X Y X i Y �i �� Y �i � � � � i �� i �� X �� X �� Y �� �Y � � X ��� �where � � � ��� � � ��� � � �� � A B D C N � Corresp onding constrain ts � One and only one �on� neuron eac h ro w� � One and only one �on� neuron eac h column� � T otal n um b er of �on� neuron� � Encourage short tour based on the distance matrix � �
Pro cedure of the Hop�eld Net wrok Eac h neuron has an input v alue and an activ a� � X � i � U � X i tion �output� v alue V X i Connecting w eigh t b et w een � X i � and � Y � � � � j � � �� � �� � W � A� � B � � � � � X i�Y j X Y ij ij X Y � � � � ��� � C D d � � j�i �� j�i � � X Y �Where � � if � � and � � otherwise� � i j � ij ij Neuron � X i � is also connected to an external input cur� � � ren t �bias�� � ��� I C N X i � Initial v alue of is set to b e a constan t v alue �deter� U � X i N N mined b y V � N � and is then p erturb ed with P P X i X �� i �� small random noise� �
� n ��� During relaxation� at step �n��� is up dated b y� U � X i � n ��� � n � U � U � � U ��� X i X i X i � U is giv en b y the equation� � X i U N N X i � n � � U � � � � � � � t ��� W V I X X X i X i�Y j Y j X i � �� j �� Y �where � � � � is time constan t of R C circuit� � � n ��� � n ��� Activ ation is determined b y through an V U � X i X i activation function �whic h is sigmoid in HN�� � n ��� � U � n ��� X i � �� � tanh � �� ��� V X i � u � Hop�eld and T ank ������ sho w ed that the net w ork is � guaran teed to con v erge to a lo cal minim um for sym� metric connecting w eigh ts� �
Problems with the Hop�eld Net w ork F requency of p o or qualit y and in v alid solutions � Sensitivit y to v ariations of cit y distributions in T S P � Sensitivit y to the c hoice of parameters � Result of Wilson and P a wley ������� � ��� v alid tours � ��� froze in to in v alid tours� � ��� did not con v erge in ���� iterations� � �Based on �� sets of randomly ���cit y TSP� Previous w ork has fo cused on impro ving the Hop�eld � net w ork b y mo difying the energy function� ��
Purp ose This w ork fo cuses on impro ving the Hop�eld network for solv� ing optimization problems� Although m uc h w ork has b een done in this area� the p erformance of the Hop�eld net w ork is still not satisfactory in terms of v alid con v ergence and qualit y of solutions� In particular� w ork follo wing the initial use of the Hop�eld net w ork for the � tr aveling salesman T S P pr oblem � demonstrated that the net w ork con v erged to v alid tours only a small p ercen tage of the time� W e address this issue in this w ork b y com bing an Evidence Based Activ ation F unction � E A � and a Con trolled Relaxation � C � pro ce� B R dure in order to impro v e the p erformance of the Hop�eld net w ork� ��
Metho ds This w ork impro v es the p erformance of the Hop�eld net w ork b y com bing an Evidence Based Activ ation F unction � E A � B and a Con trolled Relaxation � C �� R Evidence Based Activ ation F unction The original � sigmoid � activ ation function is giv en b y Eq� ���� Evidence Based Activ ation F unction � E A �� B � x U � � ��� � tanh � �� � X i u V � � � U � �� X i X i x � � tanh � � � u � x U � x tanh � � � � � � ��� � tanh � � �� X i u u � � � � U �� ��� V � X i X i x � � tanh � � � u � ��
Comparison b et w een t w o functions� 1 A B 0.9 C 0.8 0.7 0.6 0.5 V 0.4 0.3 A: sigmoid 0.2 B: EBA (x0=0.03) 0.1 C: EBA (x0=0.05) 0 -0.1 -0.05 0 0.05 0.1 U has a larger threshold �con trolled b y �� and is more E B A x � � robust against noise� E B A increases the p ercen tage of v alid tours b y ����� and � decreases the error rate of tour length b y ������ ��
Recommend
More recommend
