Learning Based Auction for Cognitive Radio Networks AK Oloyede and - PowerPoint PPT Presentation

Learning Based Auction for Cognitive Radio Networks AK Oloyede and David Grace Communications Research Group Department of Electronics University of York 10 th October 2013

Outline n Introduction n Cognitive Radio / Dynamic Spectrum Access n Auction Based Dynamic Spectrum Access n Bid Learning Model n Result n Future Work n Conclusions 2

Introduction Traffic on mobile systems is growing rapidly Global mobile data grew by 70% in 2012 n Mobile video traffic exceeded 50 % for the first time in 2012 n Average smartphone usage grew 81% in 2012 n There were 161 million laptops on the mobile network in 2012, and each laptop generated 7 n times more traffic than the average smartphone Mobile network connection speeds more than doubled in 2012 n Trend continues Global mobile data traffic will increase 13-fold between 2012 and 2017. Mobile data traffic will n grow at a compound annual growth rate (CAGR) of 66 % from 2012 to 2017, reaching 11.2 EB per month by 2017 The number of mobile-connected devices will exceed world’s population in 2013 and by 2017 n there will be nearly 1.4 mobile devices per capita. The Middle East and Africa will have the strongest mobile data traffic growth of any region at 77 % n CAGR. This region will be followed by Asia Pacific at 76% and Latin America at 67% Source: Cisco report on major global mobile data traffic projections and growth trends - 3

Implications of Mobile Traffic Growth n Network operators would be forced to find alternatives n Dynamic Spectrum Access(DSA) and Spectrum sharing n Deployment of Cognitive Radio System n It might lead to more delay on the mobile network leading to WWW(World Wide Wait) 4

Cognitive Radio and DSA n Cognitive Radio systems are defined as wireless systems that are designed to interact and observe the transmission environment n Cognitive Radio would provide dynamic access to the radio spectrum n Access to the radio spectrum using DSA requires a fair means of allocation. Thus, the use of an auction provides a fair allocation process by granting access to the highest bidder(s) 5

Revenue Growth and Data Traffic Growth Source: Analysis of operator options to reduce the impact of the revenue gap caused by flat rate mobile broadband subscription By B. Molleryd, J. Markendahl and O.Makitalo 6

Auction n An auction is a process of buying and selling by offering them up for bid, taking bids, and then selling the item to the highest bidder (Wikipedia) n There are different types of Auction process n First price auction n Second Price Auction n sealed bid auction n Discriminatory auction n Uniform Auction single unit, multiple unit etc n Auctions that maximize sellers revenue are referred to as optimal and those that maximize social welfare are referred to as efficient . 7

DSA Architecture 1.5 1.8 1.7 1 1.2 1 2 1.3 RP=1.75 1.4 1.8 1.7 1.5 12 1.4 1.3 1.2 1 8

The Users Bid ​𝑐↓𝑛𝑗𝑜 = 𝑁𝑗𝑜𝑗𝑛𝑣𝑛 ¡ 𝑞𝑝𝑡𝑡𝑗𝑐𝑚𝑓 ¡ 𝑐𝑗𝑒 ​𝑐↓𝑛𝑏𝑦 = 𝑁𝑏𝑦𝑗𝑛𝑣𝑛 ¡ 𝑞𝑝𝑡𝑡𝑗𝑐𝑚𝑓 ¡ 𝑐𝑗𝑒 ¡ 9

The Reserve Price 10

The Bidding and Transmitting Period n The bidding period is the time window allowed for the users who intend to transmit on the spectrum to submit a bid -Long bidding period would introduce more delay into the system n The bidding period is dependent on the traffic load. n The transmission period is longer than the bidding period Bidding ¡Period ¡ Bidding ¡Period ¡ Bidding ¡Period ¡ Bidding ¡Period ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ Transmi(ng ¡period ¡ Transmi(ng ¡period ¡ Transmi(ng ¡period ¡ Transmi(ng ¡period ¡ 11

Learning Based Models n Q Reinforcement Learning (QL) ​ ¡ ¡ ¡Q ↑ π ( s,a ) =R+γ ​ max ┬ a ⁠ Q( ​ s ↑ ′ , ​ a ↑ ′ ) n Bayesian framework for Rainforcement Learning (BRL) ¡P ( A\B ) = ​ P(B\A)×P(A) / P(B) ¡ Where P(A) is the prior probability distribution of hypothesis A , P ( B ) is the probability of the training data B (likelihood) and P(B\A) is the probability of A given B (Posterior probability) 12

The Bid Learning Model n The Utility Function Assumptions We assume that users are price sensitive and therefore want to win the bid with the least n possible amount The utility function measures how much a winning bidder deviates from the lowest winning n bid The lower the value of the winning bid the higher the value of utility n If the user is not among the winning bidders then the utility is zero n 𝒄↓𝒏 } 𝑵 ={ ​𝒄↓ 𝒄↓ 𝟐 , ​𝒄↓ 𝒄↓ 𝟑 , ​𝒄↓ 𝒄↓ 𝟒 … ​𝒄↓𝒏 where 𝜺 = ​𝒏𝒋𝒐 𝒏𝒋𝒐 ⁠ 𝑵 𝒗↓𝒋 ( 𝒖 ) = ¡ ¡ ¡ ¡ ¡ {█□​ 𝟑 ↑​𝜺/ 𝒕𝒇 ​𝒗↓ 𝜺/​𝒄↓𝒋 𝒄↓𝒋 −𝟐 @ 𝟏 █□ ¡ ¡ ¡ ¡ 𝑮𝒑𝒔 𝑮𝒑𝒔 ¡ 𝒙𝒋𝒐𝒐𝒋 𝒐𝒐𝒋𝒐𝒉 𝒐𝒉 ¡ 𝒄𝒋𝒆𝒆𝒇𝒔𝒕@ 𝒕@𝒑𝒖𝒊𝒇𝒔𝒙𝒋𝒕𝒇 13

Learning Model 14

Learning Model The summary of the Q learning process is as shown below: 1: Users pick a bid value 2: The utility function of the user based on the bid is calculated 3: Other records is also calculated in equation 4: The weight table is computed at time 𝑢 5: After 𝜐 trials, the best bidding value is exploited ​𝜌↑ ∗ =max( ​𝑋↓𝑢 ) The summary of the Bayesian Learning process is as shown below: 1: 𝑣 and the priori probability are calculated 2: The likelihood is generated from QL and converted into probability 3: Bayes rule is applied 4: Users pick the action with the highest utility and the winning percentage ( ​ 15 𝜌↑ ∗ ) .

Flow Chart 16

Q Learning and Bayesian Learning 80 60 Weight 40 20 0 45 46 47 48 49 50 51 52 53 54 55 (a) Price (Price Unit) 100 Events 80 500 Events 60 Weight 40 20 0 45 46 47 48 49 50 51 52 53 54 55 (b) Price (Price Unit) 17

Future Work n Implication of learning on system performance such as delay and energy consumption n Compare different bidding strategies to explore the capability of learning n Problems with learning n Learning different parameters 18

Conclusions n Bayesian learning and Q learning can be applied to an auction based cognitive network n Bayesian learning allows prior information to be incorporated into Q learning to reduce the exploration time n Bayesian learning converges faster than QL 19

Question and Suggestions AK Oloyede n aao500@york.ac.uk Communications Research Group University of York York David Grace n dg@ohm.york.ac.uk Communications Research Group University of York York 20