Identifying MMORPG Bots: Identifying MMORPG Bots: A Traffic - PowerPoint PPT Presentation

Identifying MMORPG Bots: Identifying MMORPG Bots: A Traffic Analysis Approach A Traffic Analysis Approach (MMORPG: Massively Multiplayer Online Role Playing Game) (MMORPG: Massively Multiplayer Online Role Playing Game) Kuan-Ta Chen National Taiwan University Collaborators: Jhih-Wei Jiang Polly Huang Hao-Hua Chu Chin-Laung Lei Wen-Chin Chen

Talk Outline Talk Outline Motivation Trace collection Traffic analysis and bot identification schemes Performance evaluation S cheme Robustness Conclusion Identifying MMORPG Bots: A Traffic Analysis Approach 2

Game Bots Game Bots AI programs that can perform many tasks in place of gamers Can reap rewards efficiently in 24 hours a day � break the balance of power and economies in the game world Therefore bots are forbidden in most games Identifying MMORPG Bots: A Traffic Analysis Approach 3

Bot Detection Bot Detection Detecting whether a character is controlled by a bot is difficult since a bot obeys the game rules perfectly No general detection methods are available today The state of practice is identifying via human intelligence (as bots cannot talk like humans) Labor-intensive and may annoy innocent players This work is dedicated to automatic detection of game bots (without intrusion in players’ gaming experience) Identifying MMORPG Bots: A Traffic Analysis Approach 4

Key Contributions Key Contributions We proposed to detect bots with a traffic analysis approach We proposed four strategies to distinguish bots from human players based on their traffic characteristics Identifying MMORPG Bots: A Traffic Analysis Approach 5

Bot Detection: A Decision Problem Bot Detection: A Decision Problem Q: Whether a bot is controlling a game client given the traffic stream it generates? A: Yes or No Game client Game server Traffic stream Identifying MMORPG Bots: A Traffic Analysis Approach 6

Ragnarok Online -- -- a screen shot a screen shot Ragnarok Online Ragnarok Online One of the most popular MMORPGs (they claimed 17 million subscribers worldwide recently) Notorious for the prevalence of the use of game bots Figure courtesy of www.Ragnarok.co.kr Identifying MMORPG Bots: A Traffic Analysis Approach 7

Game Bots in Ragnarok Online Game Bots in Ragnarok Online Two mainstream bot series: Kore -- KoreC , X-Kore , modKore , S olos, Kore , wasu , Erok , iKore , and VisualKore DreamRO (popular in China and Taiwan) Both bots are standalone (game clients not needed), fully-automated, script-based, and interactive Identifying MMORPG Bots: A Traffic Analysis Approach 8

DreamRO -- -- A Screen Shot A Screen Shot DreamRO View S cope World Map Character is here Character S tatus Identifying MMORPG Bots: A Traffic Analysis Approach 9

Trace Collection Trace Collection Category Trace # Participants Average Length Network Human 8 traces 2 rookies 2.6 hours ADSL, players 2 experts Cable Modem, Campus Network Bots 11 traces 2 bots 17 hours Heterogeneity was preserved Player skills Character levels / equipments Network connections Network conditions (RTT, loss rate, etc) 206 hours and 3.8 million packets were traced in total Identifying MMORPG Bots: A Traffic Analysis Approach 10

Traffic Analysis of Collected Game Traces Traffic Analysis of Collected Game Traces Traffic is analyzed in terms of Command timing Traffic burstiness Reaction to network conditions Four bot identification strategies are proposed Identifying MMORPG Bots: A Traffic Analysis Approach 11

Command Timing Command Timing Observation Bots often issue their commands based on arrivals of server packets, which carry the latest status of the character and environment e a t p d u e a t S t t1 Response time T = t2 – t1 game server C l i e n t t2 c o m game client m a n d time Client response time (response time) Time difference between the release of a client packet and the arrival of the most recent server packet Identifying MMORPG Bots: A Traffic Analysis Approach 12

CDF of Response Times CDF of Response Times DreamRO > 50% response times are extremely small Kore Zigzag pattern (multiples of a certain value) Identifying MMORPG Bots: A Traffic Analysis Approach 13

Histograms of Response Times (DreamRO traces) Histograms of Response Times (DreamRO traces) Many client packets are sent in response to server packets 1 ms 1 ms multiple peaks multiple peaks Identifying MMORPG Bots: A Traffic Analysis Approach 14

Histograms of Response Times Histograms of Response Times S cheme #1: Command Timing Regularity in the distribution of bots’ A traffic stream is considered from a bot if it has … response times Quick response times (< 10 ms) clustered Regularity in the distribution of response times, i.e., if any frequency component exists Identifying MMORPG Bots: A Traffic Analysis Approach 15

Traffic Burstiness Traffic Burstiness Traffic burstiness An indicator of how traffic fluctuates over time The variability of packet/ byte counts observed in successive periods Index of Dispersion for Counts (IDC) The IDC at time scale t is de fi ned as I t = Var( N t ) E ( N t ) , where N t indicates the number of arrivals in intervals of time t . Identifying MMORPG Bots: A Traffic Analysis Approach 16

Example: Wine Sales and IDC Example: Wine Sales and IDC The period is approximately 12 months The IDC at 12 months is the lowest Identifying MMORPG Bots: A Traffic Analysis Approach 17

The Trend of Traffic Burstiness The Trend of Traffic Burstiness Conj ecture for Bot Traffic 1. Each iteration of the bot program’ s main loop takes roughly the same amount of time 2. Each iteration of the main loop sends out roughly the same number of packets 3. Bot traffic burstiness will be the lowest in the time scale around the time needed to complete each iteration Traffic generated by human players, of course, has no reason to exhibit such property Identifying MMORPG Bots: A Traffic Analysis Approach 18

Examining the Trend of Traffic Burstiness Examining the Trend of Traffic Burstiness S cheme #2: Trend of Traffic Burstiness Regularity in the distribution of bots’ A traffic stream is considered from a bot if … response times the IDC curve has a falling trend at first and after that a rising trend, and both trends are detected at time scales < 10 sec Identifying MMORPG Bots: A Traffic Analysis Approach 19

The Magnitude of Traffic Burstiness The Magnitude of Traffic Burstiness Conj ecture Bot traffic is relatively smooth than human player traffic Difficulty no “ typical” burstiness of human player traffic S olution compare the burstiness of client traffic with that of the corresponding server traffic (as servers treat all game clients equally) S cheme #3: Burstiness Magnitude A traffic stream is considered to be generated by a bot if the client traffic burstiness is much lower than the corresponding server traffic burstiness Identifying MMORPG Bots: A Traffic Analysis Approach 20

Human Reaction to Network Conditions Human Reaction to Network Conditions Conj ecture for Human Player Traces 1. The network delay of packets will influence the pace of game playing (the rate of screen updates, character movement) 2. Human players will unconsciously adapt to the game pace (the faster the game pace is, the faster the player acts) Traffic j am!! server Is there any relationship between network delay and the pace of user actions? Identifying MMORPG Bots: A Traffic Analysis Approach 21

Packet Rate vs. Network Delay Packet Rate vs. Network Delay Human player traces: downward trend S cheme #4: Pacing A traffic stream is considered from a bot if … correlation between pkt rate vs. network delay is non- negative Identifying MMORPG Bots: A Traffic Analysis Approach 22

Performance Evaluation Performance Evaluation Metrics Correct rate the ratio the client type of a trace is correctly determined False positive rate the ratio a player is misj udged as a bot False negative rate the ratio a bot is misj udged as a human player Evaluate the sensitivity of input size by dividing traces into segments, and computing the above metrics on a segment basis Identifying MMORPG Bots: A Traffic Analysis Approach 23

Performance Evaluation Results Performance Evaluation Results [Burst iness magnit ude] always achieves low false positive rates (< 5% ) and yields a moderate correct rate ( ≈ 75% ) [Command t iming and Burst iness t rend] Correct rates higher than 95%and false negative rates lower than 5%given an input size > 2,000 packets Identifying MMORPG Bots: A Traffic Analysis Approach 24

An Integrated Approach An Integrated Approach In practice, we can carry out multiple schemes simultaneously and combine their results according to preference Conservative approach: command t iming AND burst iness t rend Aggressive approach: command t iming OR burst iness t rend Identifying MMORPG Bots: A Traffic Analysis Approach 25

An Integrated Approach -- -- Results Results An Integrated Approach Aggressive Aggressive approach (2,000 packets): Conservative approach (10,000 packets): ≈ 0%false positive rate and > 90%correct rate false negative rate < 1%and 95%correct rate Identifying MMORPG Bots: A Traffic Analysis Approach 26