When I was working on the poker bot, my friends and family were fond of asking: How’s it doing?  Sure, it was probably just small talk, but it did raise an interesting question: how do you measure its success?

Net profit seems like an obvious choice, but that answer is close to meaningless unless you are very specific with the details and even if you are, it still probably won’t mean much… unless its just for small talk. For example, what period of time are you measuring? How many games did it play during that time? Once you’ve agreed on a time period and a way to count the results, how can you be sure that the results weren’t a product of luck and variance instead of skill?

Win rate seems like a decent choice, but you face the same questions as well as some new ones: do you measure $/game, big blinds/100 hands, $/hour, or something else? And once you calculate those, is that number good or not? What do you compare it to? Other bots? Other people? What’s your benchmark?

What about development time? If you spend 500 hours developing a bot that eventually makes $500, that means you made a dollar an hour… not exactly something to brag to your in-laws about. But if it made that $500 in two days, then you might be on to something.

Also, do you only count the games you know its competent at, or do you also include the shots you’ve had it take at higher limits?

You get the idea: it’s complicated.

With that in mind, I offer the following graph, which shows the poker bot’s profits at PokerStar’s $11+$0.50 No Limit Heads Up Sit-n-Go’s for all of Sept 2008, the last full month it played before PokerStars nixed my account. Midway though the month I also started mixing in a few $22+$1s to see how it would do, but I didn’t include those results here because there weren’t enough games for the results to mean anything.

For the curious non-poker geeks: “$11+$0.50” means that the buyin for the game was $11 and that the rake, which is what you pay PokerStars to play, was $0.50. So when you beat the other guy, you win his $11 minus the $0.50 rake for a total profit of +$10.50. When you lose, you’re out the rake and the buyin, or -$11.50.

Analysis like this is good in that it tells you what your results are, but it doesn’t tell you what led to those results. For example, a graph like this lets you say “Hey honey, I made $250 dollars last week” whereas with a thorough analysis you’d be able to say, if you were so inclined, something like “Hey honey, I widened my 20BB to 30BB three bet range to include low pocket pairs and Ace-x-suited hands and after 200 occurrences the bot is winning 2bb/100 more at that level than it was before.”

Now that dollar results are kind of out of the way, future posts should lean more towards that end of the spectrum.

As soon as my account was suspended, Jeff, head of PokerStar’s security team, emailed me to inform me that I had violated their Terms of Service by running a bot and that they were shutting down my account and seizing the funds I had in it.

What follows is the email conversation I had with Jeff about being suspended, creating bots, and improving PokerStars’s bot detection techniques.

I hope that posting this will create an awareness of the issues that will ultimately lead to better security for the online poker community.

October 2, 2008

Hello Matthew,

Upon a review of your account, we find that you are running an automated player program, commonly known as a “bot”.  The running of a bot is against the terms of service to which you agreed when opening your PokerStars account.  Those terms of service are located here:

http://www.pokerstars.com/tos.html

Among them are these terms:

5.5 AUTOMATIC PLAYERS (BOTS). The use of artificial intelligence including, without limitation, “robots” is strictly forbidden in connection with the Software and the Games. All actions taken in relation to the Games by a User must be executed personally by players through the user interface accessible by use of the Software.

5.8 FRAUDULENT BEHAVIOUR In the event that PokerStars deems that a User has engaged or attempted to engage in fraudulent, unlawful, dishonest or improper activity while using the Service, including without limitation, engaging in any of the activities set forth above […] PokerStars shall be entitled to take such action as it sees fit, including immediately blocking access to the Service, [and] terminating such User’s account with PokerStars […]

As such, your PokerStars account has been closed, and the funds within it have been seized and will not be returned.  they will be used to compensate the victims of illicit activity on PokerStars. Please do not attempt to return to the games on another account.  You are no longer welcome to play on PokerStars.

Thank You,

Jeff
PokerStars Game Security

October 9

Hello Matthew,

Thank you for being forthcoming here.  It is refreshing, as many
developers in your situation will try to “deny, deny, deny”, in an attempt
to return to the site.

Unfortunately, we cannot be lenient in such matters.  We cannot re-open
your account.  We have a zero tolerance policy toward bots, and as such
your account must remain closed.

Best Regards,

Jeff
PokerStars Game Security

October 10

Hello Matthew,

Your email has been forwarded to Jeff.

Thank you for your patience.

Regards,

EddieC
PokerStars Support Team

October 10

Hello Matthew,

>If you’re interested I’m willing to share everything I
>have–the tools, methodology, code, analysis and ideas to improve
>PokerStars’s security.

As long as you understand that what you’re offering is to travel along a one-way street, we would welcome the opportunity to see what you have to share.  We will not, however, be able to share any of the details of what we’re already doing to detect bots such as yours.

I will say that should the data you provide give us any insight that we’ve overlooked, we would *possibly* reconsider the permanence of your barring.

>It’s important to understand that it was never about the money. My poker
>income was good when I took the leap and it was unlikely that I’d ever
>earn more with a bot. I liked the challenge and that kept me motivated
>for a long time–probably too long.

This is a concept and a mentality that we’re all-too-familiar with.  The same is true of the vast majority of people who develop and run bots. I myself am a programmer and poker theorist as well, and if I wasn’t challenging myself by catching bots, I could definitely see the allure of the challenge of writing one.

Let’s see what you have to share, and perhaps it will ultimately (though not immediately) result in some sort of reprieve for you in the long run.

October 11

Jeff,

You may recall that several weeks ago we had a brief discussion after my account was suspended for operating a bot. I said that I’d like to share some information with you, you said don’t expect anything in return, and I said okay. It’s taken a little bit longer than I would have liked, but here’s my follow-on email.

Below are my best guesses as far as what you currently do to identify bots as well as some thoughts on how they might be improved.

Hand Quantity

The easiest and most obvious way to narrow down the field is to look for players with abnormally high numbers of hands in a given time period. This stems from the fact that most bots can’t compete in terms of skill, but can compensate for this disadvantage by playing massive quantities of hands. This one’s pretty obvious and I’m sure PokerStars considers it.

When I personally played HUSNGs, I probably played 10-15 on a given day, rarely playing more than 25. At 40-50 games/day, my bot was probably in the top 1% in terms of volume of HUSNGs played and yet it still went many months before the account was finally suspended. Even now, I’m not positive that’s what finally did it in, though I’d guess it probably was a big factor.

The “easiest” way to circumvent this is to set up multiple accounts, so that no individual account raises a flag. Fortunately, the logistical problems associated with creating multiple accounts with multiple addresses (assuming PokerStars look into the playing habits of players from the same address) or having friends run the software and exchange money is probably more than most people want to or can do successfully.

Abnormal Play

Along the way I made several big mistakes that probably should have raised a flag or two.

I had two programs: one that played the games and one that registered for new ones. One day the program that played the games crashed while I was out. I hadn’t built in a check for this, so the other program kept opening new ones for several hours, even though I was sitting out for all of them. By the time I got home, the bot had sat out for more than 20 consecutive games. I waited for the suspension, but nothing ever happened.

Another oddity was that the bot stuck to the same buyin for extended periods of time, which you wouldn’t expect from a human. There were periods of something like 800 games where it played nothing by $6+$0.25 Turbo HUSNGs. I’m sure there are people that do this, but most people tend to move up every now and then, even if only a little bit.

CAPTCHAs

I saw a CAPTCHA once while I was personally playing at a NL100 Heads Up table and I think once while the bot was playing a HUSNG. There may have been more, but my general impression was that you didn’t do them very often and didn’t target my account despite the other warning signs.

I realize there are lots of considerations to make when deciding whether to use CAPTCHAs.

As with any test, you’re still faced with false-positives (people who either don’t see the CAPTCHA or do but answer incorrectly) and false-negatives (correct answers, though still a bot). I imagine most people who are prompted with a CAPTCHA notice it, but a lot probably get it wrong due to their complexity.

You also have to worry about bots that can detect and solve CAPTCHAs. I never tried to write or purchase CAPTCHA solving software because it didn’t appear to be a major threat. (Though who knows, missing a few of them may have eventually led to the suspension). One of the problems of detecting and solving a PokerStars CAPTCHA is that they occur so infrequently so its hard to gather a large enough sample size to be confident in whatever system you develop to solve them. In that sense, the scarcity was good for you.

As I’m sure you know you can’t just throw a CAPTCHA at anyone who demonstrates suspicious activity. You have to consider the impact on legitimate players. If you prompt players with too many CAPTCHAs you may risk them getting annoyed and switching to one of your competitors. Additionally, the fact that you have to have it in the first place implies there are bots that they should be concerned about.  You’ll likely also have a high false-positive rate, resulting in angry explosions on poker forums of players crying foul.

You may have realized the tradeoff and purposely chosen a more passive, non-intrusive approach rather than aggressively testing suspicious accounts. In exchange you get some false-negatives, which might be the underlying reason my account lasted as long as it did.

That being said, I think you should still test the high volume players more often. If a player is in the top 5% by volume he is probably the type of player who will probably appreciate your attempts to protect against bots.

Additionally, knowing that CAPTCHAs are a threat probably discourages most people from trying to develop a PokerStars bot. The semi-frequent threads on poker forums about your CAPTCHAs have probably done more to minimize the number of bots on PokerStars than the CAPTCHA itself.

When I first saw a PokerStars CAPTCHA I attempted to take a screen shot of it, but was unable to. MSPaint said it couldn’t load the data, and no other software could either. It seemed that the PokerStars software disabled print screen during the CAPTCHA test. It could have been the result of an error on my part and not an intentional move by the software. If you don’t do this, maybe you should, as it makes it a lot harder to program the bot to detect a CAPTCHA when it doesn’t know what it’s looking for.

My hack was to have the bot check for abnormalities in the chat area of the screen. If the colors or window hierarchy were abnormal, the computer played loud obnoxious noises to draw my attention. Another hypothetical solution was to extract the CAPTCHA and send it to me on my cell phone, which I could quickly access and respond to.

It’d be hard to find a method that doesn’t result in any false positives or false negatives, but taken together with other flags you can definitely increase the likelihood of identifying bots.

Window Resizing

I think you also attempt to hinder bots by making slight changes to the dimensions of the game window every now and then. I’m not sure if this was intentional or a software bug or whether it was targeted or random, but it caused me a bit of trouble. At the time my bot (and most bots) operated on a fixed mapping which assumed that certain pixel are static. By resizing the windows you made it a lot harder to extract important information.

I had the bot resize the window by having the mouse click and drag it to the correct position, but sometimes things were still slightly off. I was able to overcome this by having the bot calculate where to look versus having it based on fixed positions, but it was a messy fix. Luckily, the images for things like the cards didn’t change; their positions just shifted by a few pixels.

This method can never fully prevent bot detection, as any changes you make developers can and will adapt to, but it will disable the majority of the commercial botting platforms.

Mouse Analysis

I’m not sure what PokerStars does with this, if anything, but I have a good idea of what it doesn’t do. You don’t check to see whether the locations that I click are consistent with a human. At least not very well.

Here’s what I did, but looking back, I’m really not sure it was worth the effort. Using the PokerStars log file, I extracted the locations I clicked during the course of a day and overlaid it on a screenshot of the table. With this, I could see where I clicked and set up the bot so that it clicked in similar locations. For example, when the bot would click call, fold, and raise the distribution resembled two bell curves in the shape of a cross, so that most of the clicks usually occurred near the center with very few at the corners. Again, at this point, I think this was mostly unnecessary.

One thing that you should check for is where a normal player clicks on the table itself; how often does he check out the Instant Hand History, how often does he click on the table graphics, how does he double click a player’s avatar to edit notes, etc. My bot rarely strayed from the area around the action buttons, which should have been a large flag if the software had been looking for abnormal behavior.

Another thing I did when starting out was to have the mouse to drag around versus having it jump from location to location. Theoretically this was supposed to make it look more human, though it just looked ridiculous in practice. A friend even worked out acceleration and deceleration so that it would look more natural. After doing this for a few days I switched it back to mouse hopping, figuring it was a huge waste of time if you weren’t actually checking it and if you were, the silly methods I used weren’t likely to fool you.

I saw on some botting forum that people claimed to have mouse movement down to a science. They proudly announced they had discovered what normal mouse movement looks like so they don’t have to fear being caught by site security. These developers likely wasted a lot of time working on inconsequential preventative measures. Probably not unlike me with some of these things…

Nonetheless, there is a big opportunity in mouse movement.  Have the PokerStars software look for abnormal behavior not only in where the user clicks, but the path and speed between those nodes.  If the behavior is abnormal, have the software notify your security team, who can then do further investigation. If the software doesn’t detect strange behavior, at least store a history of the user’s mouse movement in some encrypted file for a few days. If you do suspect an account of using automation software, have the software send you that log and do your own analysis on it, looking for indications of bot use.

Again, abnormal behavior is not a sure sign of a bot. PokerStars allows lots of scripts that aid multitablers and these tools do not violate the terms of service. However, when analyzed with respect to other available data it is one more piece of evidence you can look at before making your final decision.

Chat Box

Since PokerStars uses a custom control for the chat window, it’s not an easy task to extract the relevant information. In the beginning this was a major challenge, as most of the information the bot needs to make quality decisions is contained there (with the exception of stack sizes and holecards, which have to be read using less elegant methods).

I started off using some unreliable character recognition techniques, but eventually found a much easier way. While talking to another bot developer about the problem, he said he’s never had the problem. After some detective work we discovered that he ran PokerOffice, which adds its own control to the window which can be read using normal methods. So rather than doing the extensive work required to have the bot obtain the text, I just let PokerOffice run in the background and let it do the majority of the work.

Make the chat box harder to read and you’ll make it harder for bots to get the information they need to make intelligent decisions.

…

It might seem that I think PokerStars’s current bot detection algorithms are less than stellar. On the contrary, based on what I’ve seen and read in various forums about the security on other sites, PokerStars is second to none. Your site is so notoriously difficult to operate a bot on that most do not even try.

I don’t think there’s a single method you can use to tell whether someone is or is not a bot. The best you can do is use the data you have to make educated decisions about who might be, and then use your own judgment to make the final call.

You probably have terabytes worth of information available to you and as I’m sure you know it’s not a trivial task to analyze it all. Fortunately, the methods above should be relatively easy for PokerStars to implement on the client side. Until the final judgment has to be made, you shouldn’t have to rely on hand histories to narrow down the field.

I wish you guys the best of luck. It’s a tricky and important problem without an easy solution.

November 11

Hello Matthew,

Thank you for your email.

I have now forwarded your email to the department responsible for handling
this matter. As this department is currently receiving a bulk number of
requests, please allow some time before being contacted. Our apologies for
any inconvenience that this may have caused.

Thank you for your patience, cooperation and for choosing PokerStars.

Regards,

Andrew
PokerStars Support Team

October 2, 2008 was the last day I played online poker… well sort of.

I actually hadn’t played for many months prior to that, though that’s probably not what you’d have concluded if you looked at the activity on my account. The hand histories would have painted a different story–one of an extremely active, fairly profitable player at PokerStars’s Heads Up No Limit Hold’em Sit-n-Go’s, which are basically short one on one poker tournaments.

The thing is I didn’t play any of those games — a poker bot I built called EVe did.

October 2 was the day PokerStars figured it out.

October 2 was the day they suspended my account:

I’ve debated for a long time whether to write about it.

On one hand, the online poker community usually doesn’t react well to this type of thing (go figure). Bot discussions tend to get a lot of “People like you destroy the games”, “What a scumbag you are…I hope you get banned” and “I hope you die in a grease fire”-type responses and I definitely don’t want to die in a grease fire.

On the other hand, I learned in the process and I’d like to share some of my experiences. I have nearly two years of stories, spreadsheets, hand histories, and results that I think a lot of people will find fascinating.  Maybe, just maybe, talking about it will result in better bot detection techniques, which the poker sites very much need. I’d also like to try to convince any ambitious programmers out there that they’re better off eating glass than trying to create a poker bot (because at least with the glass you’ll quickly feel the pain and realize what a terrible idea it was, unlike building a poker bot).

After weighing the options I’ve decided to go ahead and write about it a little bit at a time. If the feedback is positive, I’ll keep writing and if not, I’ll probably stop. We’ll see how it goes.

More to follow.