One of the main hurdles associated with creating a poker bot is avoiding detection.

I imagine most people when initially considering the problem look at it from the vantage point of the poker site. You ask yourself: “If I was charge of security, what would I look for to detect poker bots?”

So you make a short list of everything you can think of and start brainstorming ways not to violate those principles.

Your list might look something like this:

Mouse movement – Figure out an algorithm that closely models human behavior

Mouse clicks – Don’t click in the same location every time

Number of hands – Don’t play an excessive number of hands in given period of time

etc etc. You get the idea.

It’s critical that you take these precautions because its likely the first thing the poker sites look at when determining who’s been naughty and who’s been nice (or maybe not).

However, in my experience, the biggest challenge is NOT how to fool the poker site, its how to fool your opponents. These are the people that the bot will spend the most time with you and if it plays in a robotic or in an abnormal manner, someone might get suspicious. All it takes is one person emailing the poker site saying “Hey, what’s up with this guy?” and you’re toast.

Since most bots rely on playing a large number of hands, it’s vital that you devote a lot of time and energy to how you’re going to deal with people.

One of the problems you’ll have to figure out is what you want to do when people talk to your bot.

Do you want to create some IF/THEN rules that talk back when someone says a specific word or phrase? Maybe have the bot text your phone and then send your response back to that chat? You can get pretty creative, but I recommend keeping it simple. The more complicated your system, the more likely it is to fail.

Here’s the thing: most players never chat so if your bot never does, most people won’t raise an eyebrow. 99% of the time you can safely ignore what people say and you’ll be just fine.

However, in certain circumstances, especially when you’re heads up and someone says something provocative, you definitely want to respond. Take, for instance, if someone says: “Agh I hate bots”

You could ignore it… but its probably not the best way to increase your chances of long term success.

Here’s what I did: I created a list of words like “bot”, “computer”,  and “cheater” and if anyone said one of those words, my computer would start playing a loud, annoying bell. If I was in the around I could pause the bot, answer the the belligerent individual, and hopefully quell his suspicion.

Simple and effective.

What do you do if you’re not around?

Tip #2: be around. It’s not as sexy (?) as picturing yourself out to the movies while your bot’s at home paying for your evening out, but it will drastically increase how long your bot survives. There will be bugs. Things will break. Stuff will happen that you’ve never seen before. Be around. You don’t have to be there at your desk watching every move, but be in the vicinity. Also, the easiest way to ensure the bot isn’t playing an excessive number of hours is to make sure that you’re around when the bot is playing. In theory, you could be playing the same number of hours.

In general, you don’t want to stick out at all. Leaving your bot on all night is a sure-fire way to fail at that. You want the bot’s behavior to be as average as possible.

…except for its results :)

When I was working on the poker bot, my friends and family were fond of asking: How’s it doing?  Sure, it was probably just small talk, but it did raise an interesting question: how do you measure its success?

Net profit seems like an obvious choice, but that answer is close to meaningless unless you are very specific with the details and even if you are, it still probably won’t mean much… unless its just for small talk. For example, what period of time are you measuring? How many games did it play during that time? Once you’ve agreed on a time period and a way to count the results, how can you be sure that the results weren’t a product of luck and variance instead of skill?

Win rate seems like a decent choice, but you face the same questions as well as some new ones: do you measure $/game, big blinds/100 hands, $/hour, or something else? And once you calculate those, is that number good or not? What do you compare it to? Other bots? Other people? What’s your benchmark?

What about development time? If you spend 500 hours developing a bot that eventually makes $500, that means you made a dollar an hour… not exactly something to brag to your in-laws about. But if it made that $500 in two days, then you might be on to something.

Also, do you only count the games you know its competent at, or do you also include the shots you’ve had it take at higher limits?

You get the idea: it’s complicated.

With that in mind, I offer the following graph, which shows the poker bot’s profits at PokerStar’s $11+$0.50 No Limit Heads Up Sit-n-Go’s for all of Sept 2008, the last full month it played before PokerStars nixed my account. Midway though the month I also started mixing in a few $22+$1s to see how it would do, but I didn’t include those results here because there weren’t enough games for the results to mean anything.

For the curious non-poker geeks: “$11+$0.50” means that the buyin for the game was $11 and that the rake, which is what you pay PokerStars to play, was $0.50. So when you beat the other guy, you win his $11 minus the $0.50 rake for a total profit of +$10.50. When you lose, you’re out the rake and the buyin, or -$11.50.

Analysis like this is good in that it tells you what your results are, but it doesn’t tell you what led to those results. For example, a graph like this lets you say “Hey honey, I made $250 dollars last week” whereas with a thorough analysis you’d be able to say, if you were so inclined, something like “Hey honey, I widened my 20BB to 30BB three bet range to include low pocket pairs and Ace-x-suited hands and after 200 occurrences the bot is winning 2bb/100 more at that level than it was before.”

Now that dollar results are kind of out of the way, future posts should lean more towards that end of the spectrum.

As soon as my account was suspended, Jeff, head of PokerStar’s security team, emailed me to inform me that I had violated their Terms of Service by running a bot and that they were shutting down my account and seizing the funds I had in it.

What follows is the email conversation I had with Jeff about being suspended, creating bots, and improving PokerStars’s bot detection techniques.

I hope that posting this will create an awareness of the issues that will ultimately lead to better security for the online poker community.

October 2, 2008

Hello Matthew,

Upon a review of your account, we find that you are running an automated player program, commonly known as a “bot”.  The running of a bot is against the terms of service to which you agreed when opening your PokerStars account.  Those terms of service are located here:

http://www.pokerstars.com/tos.html

Among them are these terms:

5.5 AUTOMATIC PLAYERS (BOTS). The use of artificial intelligence including, without limitation, “robots” is strictly forbidden in connection with the Software and the Games. All actions taken in relation to the Games by a User must be executed personally by players through the user interface accessible by use of the Software.

5.8 FRAUDULENT BEHAVIOUR In the event that PokerStars deems that a User has engaged or attempted to engage in fraudulent, unlawful, dishonest or improper activity while using the Service, including without limitation, engaging in any of the activities set forth above […] PokerStars shall be entitled to take such action as it sees fit, including immediately blocking access to the Service, [and] terminating such User’s account with PokerStars […]

As such, your PokerStars account has been closed, and the funds within it have been seized and will not be returned.  they will be used to compensate the victims of illicit activity on PokerStars. Please do not attempt to return to the games on another account.  You are no longer welcome to play on PokerStars.

Thank You,

Jeff
PokerStars Game Security

October 9

Hello Matthew,

Thank you for being forthcoming here.  It is refreshing, as many
developers in your situation will try to “deny, deny, deny”, in an attempt
to return to the site.

Unfortunately, we cannot be lenient in such matters.  We cannot re-open
your account.  We have a zero tolerance policy toward bots, and as such
your account must remain closed.

Best Regards,

Jeff
PokerStars Game Security

October 10

Hello Matthew,

Your email has been forwarded to Jeff.

Thank you for your patience.

Regards,

EddieC
PokerStars Support Team

October 10

Hello Matthew,

>If you’re interested I’m willing to share everything I
>have–the tools, methodology, code, analysis and ideas to improve
>PokerStars’s security.

As long as you understand that what you’re offering is to travel along a one-way street, we would welcome the opportunity to see what you have to share.  We will not, however, be able to share any of the details of what we’re already doing to detect bots such as yours.

I will say that should the data you provide give us any insight that we’ve overlooked, we would *possibly* reconsider the permanence of your barring.

>It’s important to understand that it was never about the money. My poker
>income was good when I took the leap and it was unlikely that I’d ever
>earn more with a bot. I liked the challenge and that kept me motivated
>for a long time–probably too long.

This is a concept and a mentality that we’re all-too-familiar with.  The same is true of the vast majority of people who develop and run bots. I myself am a programmer and poker theorist as well, and if I wasn’t challenging myself by catching bots, I could definitely see the allure of the challenge of writing one.

Let’s see what you have to share, and perhaps it will ultimately (though not immediately) result in some sort of reprieve for you in the long run.

October 11

Jeff,

You may recall that several weeks ago we had a brief discussion after my account was suspended for operating a bot. I said that I’d like to share some information with you, you said don’t expect anything in return, and I said okay. It’s taken a little bit longer than I would have liked, but here’s my follow-on email.

Below are my best guesses as far as what you currently do to identify bots as well as some thoughts on how they might be improved.

Hand Quantity

The easiest and most obvious way to narrow down the field is to look for players with abnormally high numbers of hands in a given time period. This stems from the fact that most bots can’t compete in terms of skill, but can compensate for this disadvantage by playing massive quantities of hands. This one’s pretty obvious and I’m sure PokerStars considers it.

When I personally played HUSNGs, I probably played 10-15 on a given day, rarely playing more than 25. At 40-50 games/day, my bot was probably in the top 1% in terms of volume of HUSNGs played and yet it still went many months before the account was finally suspended. Even now, I’m not positive that’s what finally did it in, though I’d guess it probably was a big factor.

The “easiest” way to circumvent this is to set up multiple accounts, so that no individual account raises a flag. Fortunately, the logistical problems associated with creating multiple accounts with multiple addresses (assuming PokerStars look into the playing habits of players from the same address) or having friends run the software and exchange money is probably more than most people want to or can do successfully.

Abnormal Play

Along the way I made several big mistakes that probably should have raised a flag or two.

I had two programs: one that played the games and one that registered for new ones. One day the program that played the games crashed while I was out. I hadn’t built in a check for this, so the other program kept opening new ones for several hours, even though I was sitting out for all of them. By the time I got home, the bot had sat out for more than 20 consecutive games. I waited for the suspension, but nothing ever happened.

Another oddity was that the bot stuck to the same buyin for extended periods of time, which you wouldn’t expect from a human. There were periods of something like 800 games where it played nothing by $6+$0.25 Turbo HUSNGs. I’m sure there are people that do this, but most people tend to move up every now and then, even if only a little bit.

CAPTCHAs

I saw a CAPTCHA once while I was personally playing at a NL100 Heads Up table and I think once while the bot was playing a HUSNG. There may have been more, but my general impression was that you didn’t do them very often and didn’t target my account despite the other warning signs.

I realize there are lots of considerations to make when deciding whether to use CAPTCHAs.

As with any test, you’re still faced with false-positives (people who either don’t see the CAPTCHA or do but answer incorrectly) and false-negatives (correct answers, though still a bot). I imagine most people who are prompted with a CAPTCHA notice it, but a lot probably get it wrong due to their complexity.

You also have to worry about bots that can detect and solve CAPTCHAs. I never tried to write or purchase CAPTCHA solving software because it didn’t appear to be a major threat. (Though who knows, missing a few of them may have eventually led to the suspension). One of the problems of detecting and solving a PokerStars CAPTCHA is that they occur so infrequently so its hard to gather a large enough sample size to be confident in whatever system you develop to solve them. In that sense, the scarcity was good for you.

As I’m sure you know you can’t just throw a CAPTCHA at anyone who demonstrates suspicious activity. You have to consider the impact on legitimate players. If you prompt players with too many CAPTCHAs you may risk them getting annoyed and switching to one of your competitors. Additionally, the fact that you have to have it in the first place implies there are bots that they should be concerned about.  You’ll likely also have a high false-positive rate, resulting in angry explosions on poker forums of players crying foul.

You may have realized the tradeoff and purposely chosen a more passive, non-intrusive approach rather than aggressively testing suspicious accounts. In exchange you get some false-negatives, which might be the underlying reason my account lasted as long as it did.

That being said, I think you should still test the high volume players more often. If a player is in the top 5% by volume he is probably the type of player who will probably appreciate your attempts to protect against bots.

Additionally, knowing that CAPTCHAs are a threat probably discourages most people from trying to develop a PokerStars bot. The semi-frequent threads on poker forums about your CAPTCHAs have probably done more to minimize the number of bots on PokerStars than the CAPTCHA itself.

When I first saw a PokerStars CAPTCHA I attempted to take a screen shot of it, but was unable to. MSPaint said it couldn’t load the data, and no other software could either. It seemed that the PokerStars software disabled print screen during the CAPTCHA test. It could have been the result of an error on my part and not an intentional move by the software. If you don’t do this, maybe you should, as it makes it a lot harder to program the bot to detect a CAPTCHA when it doesn’t know what it’s looking for.

My hack was to have the bot check for abnormalities in the chat area of the screen. If the colors or window hierarchy were abnormal, the computer played loud obnoxious noises to draw my attention. Another hypothetical solution was to extract the CAPTCHA and send it to me on my cell phone, which I could quickly access and respond to.

It’d be hard to find a method that doesn’t result in any false positives or false negatives, but taken together with other flags you can definitely increase the likelihood of identifying bots.

Window Resizing

I think you also attempt to hinder bots by making slight changes to the dimensions of the game window every now and then. I’m not sure if this was intentional or a software bug or whether it was targeted or random, but it caused me a bit of trouble. At the time my bot (and most bots) operated on a fixed mapping which assumed that certain pixel are static. By resizing the windows you made it a lot harder to extract important information.

I had the bot resize the window by having the mouse click and drag it to the correct position, but sometimes things were still slightly off. I was able to overcome this by having the bot calculate where to look versus having it based on fixed positions, but it was a messy fix. Luckily, the images for things like the cards didn’t change; their positions just shifted by a few pixels.

This method can never fully prevent bot detection, as any changes you make developers can and will adapt to, but it will disable the majority of the commercial botting platforms.

Mouse Analysis

I’m not sure what PokerStars does with this, if anything, but I have a good idea of what it doesn’t do. You don’t check to see whether the locations that I click are consistent with a human. At least not very well.

Here’s what I did, but looking back, I’m really not sure it was worth the effort. Using the PokerStars log file, I extracted the locations I clicked during the course of a day and overlaid it on a screenshot of the table. With this, I could see where I clicked and set up the bot so that it clicked in similar locations. For example, when the bot would click call, fold, and raise the distribution resembled two bell curves in the shape of a cross, so that most of the clicks usually occurred near the center with very few at the corners. Again, at this point, I think this was mostly unnecessary.

One thing that you should check for is where a normal player clicks on the table itself; how often does he check out the Instant Hand History, how often does he click on the table graphics, how does he double click a player’s avatar to edit notes, etc. My bot rarely strayed from the area around the action buttons, which should have been a large flag if the software had been looking for abnormal behavior.

Another thing I did when starting out was to have the mouse to drag around versus having it jump from location to location. Theoretically this was supposed to make it look more human, though it just looked ridiculous in practice. A friend even worked out acceleration and deceleration so that it would look more natural. After doing this for a few days I switched it back to mouse hopping, figuring it was a huge waste of time if you weren’t actually checking it and if you were, the silly methods I used weren’t likely to fool you.

I saw on some botting forum that people claimed to have mouse movement down to a science. They proudly announced they had discovered what normal mouse movement looks like so they don’t have to fear being caught by site security. These developers likely wasted a lot of time working on inconsequential preventative measures. Probably not unlike me with some of these things…

Nonetheless, there is a big opportunity in mouse movement.  Have the PokerStars software look for abnormal behavior not only in where the user clicks, but the path and speed between those nodes.  If the behavior is abnormal, have the software notify your security team, who can then do further investigation. If the software doesn’t detect strange behavior, at least store a history of the user’s mouse movement in some encrypted file for a few days. If you do suspect an account of using automation software, have the software send you that log and do your own analysis on it, looking for indications of bot use.

Again, abnormal behavior is not a sure sign of a bot. PokerStars allows lots of scripts that aid multitablers and these tools do not violate the terms of service. However, when analyzed with respect to other available data it is one more piece of evidence you can look at before making your final decision.

Chat Box

Since PokerStars uses a custom control for the chat window, it’s not an easy task to extract the relevant information. In the beginning this was a major challenge, as most of the information the bot needs to make quality decisions is contained there (with the exception of stack sizes and holecards, which have to be read using less elegant methods).

I started off using some unreliable character recognition techniques, but eventually found a much easier way. While talking to another bot developer about the problem, he said he’s never had the problem. After some detective work we discovered that he ran PokerOffice, which adds its own control to the window which can be read using normal methods. So rather than doing the extensive work required to have the bot obtain the text, I just let PokerOffice run in the background and let it do the majority of the work.

Make the chat box harder to read and you’ll make it harder for bots to get the information they need to make intelligent decisions.

…

It might seem that I think PokerStars’s current bot detection algorithms are less than stellar. On the contrary, based on what I’ve seen and read in various forums about the security on other sites, PokerStars is second to none. Your site is so notoriously difficult to operate a bot on that most do not even try.

I don’t think there’s a single method you can use to tell whether someone is or is not a bot. The best you can do is use the data you have to make educated decisions about who might be, and then use your own judgment to make the final call.

You probably have terabytes worth of information available to you and as I’m sure you know it’s not a trivial task to analyze it all. Fortunately, the methods above should be relatively easy for PokerStars to implement on the client side. Until the final judgment has to be made, you shouldn’t have to rely on hand histories to narrow down the field.

I wish you guys the best of luck. It’s a tricky and important problem without an easy solution.

November 11

Hello Matthew,

Thank you for your email.

I have now forwarded your email to the department responsible for handling
this matter. As this department is currently receiving a bulk number of
requests, please allow some time before being contacted. Our apologies for
any inconvenience that this may have caused.

Thank you for your patience, cooperation and for choosing PokerStars.

Regards,

Andrew
PokerStars Support Team