In late 2006 and early 2007, I tried to build a shortstacking poker bot.
Shortstacking, in a nutshell, is a strategy that advocates buying in for a relatively small amount of money compared to your opponents and then playing very aggressive preflop poker. Most opponents do not adjust correctly to your stack size, and, in theory, you can make a lot of money from their mistakes.
Here’s a quote from shortstacking strategy guide on what used to be DonkeyDevestation.com:
By buying in with 20bb, you control the size of the stacks that the hand is being played for, not your opponents. Most of your opponents will be making plays preflop that are suitable for a full stack game but would be losers against 20bb stacks, and therefore losers against you. They take the worst of it against you to play better against the others, and you profit from this. Many players will complain about short stack players, claiming they are “lame” or “unskilled”. And they are right. It is lame, and it does take less skill to play a short stack. But it is very profitable, especially in today’s games. Furthermore, unlike most strategies, short stack play actually becomes more effective as you move up in limits, because other players tend to be more aggressive preflop, meaning they are putting more money in with hands they would not raise if they were only playing against you.
Shortstacking was appealing to me as a first time bot developer because it had been written about fairly extensively and the strategy, at least from a preflop perspective, was pretty well defined.
Here, for example, is a chart of a player named cuzco, who successfully employed a shortstacking strategy:
How hard could it be to study his hand histories and have a bot emulate his decisions?
I already had a bit of experience writing add-ons for the PokerStars software (see PokerShark), so I figured it would only take a few weeks to get a prototype together.
In November 2006, I started programming.
The first major hurdle was figuring out how to read the chat box on the PokerStars client. PokerStars uses some custom controls which can’t easily be read using standard Windows API techniques, so I tried using Optical Character Recognition (OCR) to convert the pixels to words:
And later:
Eventually I figured out that PokerOffice, which makes a popular add-on for PokerStars, adds their own chat control on top of the default PokerStars one and it can be read using standard techniques. This was a big step because I no longer had to rely on OCR to get the text, which didn’t work very well (a lowercase L and a capital I look the same, for example).
Reading the text on the PokerStars Lobby was also important in order to automatically open new tables:
Using some shortstacking strategy guides as a starting point, I put together extensive preflop and postflop logic charts which were eventually implemented in the bot:
By the end of January, I had a working product:
It was capable of multitabling like the machine it was:
Most days it lost money, which was to be expected during development, but some days it did very well:
And that often gave me hope that the changes I had made the previous day were all that was needed to finally make it profitable.
I did a lot of A/B testing to try to measure what effect, if any, the changes I was making were effective:
Another thing I had to worry about were CAPTCHAs, which PokerStars is fond of using to identify bots:
Eventually I realized that conditional statements were not enough. There are simply too many situations in No Limit Texas Hold’em to account for–even when you’re shortstacking. For example, how do you take into account bet sizes and your opponent’s stack size? “IF … THEN” could not adequately handle that, so I started playing around with Expected Value (EV) calculations:
Unfortunately at the time I didn’t have the tools to implement these techniques.
Nonetheless, I persisted with the shortstacking bot, believing that it only would take a few more changes before it was profitable.
March rolled by…
And April came around…
But, despite my best efforts to turn it profitable, it kept losing.
By the time I decided to quit at the end April, the bot had played roughly 49K hands and blown through more than $1,300:
Eesh.
Shortstacking, it seemed, was harder than it looked.
What I eventually realized is that your profits from shortstacking come from a few key decisions and if you didn’t make the correct moves at those moments, you couldn’t win enough to be profitable. In other words, I may have had a 90% solution–though I doubt it–but because it couldn’t make the correct decision that remaining 10% of the time, it could never be profitable in the long run.
Frustrated but determined not to let the work go to waste, I decided to start from scratch and to build a No Limit Heads Up Sit-n-Go (HUSNG) bot. That was my game of choice and I figured as long as I’m going to work on something, it might as well be with a game that I enjoy and had a lot of experience with.
Tatta, one of the most profitable Sit-n-Go players on PokerStars, was my inspiration:
“If I could only achieve a fraction of his success,” I reasoned to myself, “this will have been worth it.”
For the rest of this story, I’ll point you to the previous 11 posts about the HUSNG bot.
To sum it it: after several more months of work it was profitable, but shortly thereafter, PokerStars caught it and closed my account.
Doh. :)
Matt Mazur 
How did you get cuzco’s hand histories. Did he give them to you or did you get them at handhq.com or somewhere?
I purchased a collection of every NL cash hand played on Stars from early Dec 06 to March 07. About 7M hands total. From that I parsed out all the hands that had a shortstacker and then parsed those hands to see who had played the most hands. Cuzco was one of the heaviest winners out of that group.
Do you think you have enough knowledge about short stacking to be profitable at NL100 even though you weren’t able to make a profitable bot because of programming issues.
If you were able to make a bot, do you think many other knowledgable people out there have done the same and are profitable?
I believe the problems you had were in your approach to strategy. Successful poker players need to be able to put their opponent on a range of hands. Simply evaluating your own hand is meaningless if you don’t know what your opponent is likely holding. For this reason you need to be able to keep track of an opponent’s history and be able to classify them according to how weak/tight they are, and how loose/aggressive they are at a minimum. That information can then be used in an algorithm to determine the likelihood of the opponent holding any given hand, and therefore the likelihood of you holding a superior hand.
It takes a lot of overlapping skills to build a profitable poker bot. I had a lot of experience building add ons for third party software (see AOL-Files, TetriNET bot, PokerShark), so it didn’t take too long to get the interaction part down. That’s the first major hurdle. The second is programming it with a profitable strategy, which is the second and largest hurdle.
If your goal is to make money, you’re much MUCH better off playing.
I’m sure it would be easy doing a limit holdem bot and there must be hundreds or thousands of people that are both good at poker/math and programming. I bet there are at least a handful of bots that are profitable.
In your ABTesting screenshot, what stakes did the bot play to achieve these results?
For any consolation, if it was 50NL, then losing $9.40 after 6507 hands is not so bad (-0.29bb/100 — granted a small sample size). With rakeback on a site like FullTilt, it would be profitable.
Intuitively, it seems that a shortstacking bot’s bread-and-butter positions will be the positions last to act preflop (CO, BTN, SB, BB), making +EV shoves/re-raises over limpers/raisers. I’m interested in knowing if in the blinds, your bot would re-raise raises/limpers with a fairly wide range given that your opponent’s opening/limping ranges are wide themselves, such that there is a good deal of fold equity? Did the bot use simple statistics such as positional VPIP to estimate opponent hand ranges? If not, I can understand why the bot would be slightly unprofitable.
Hi Matt,
very, very interesting article series — please keep it up if you have more material.
I’ve also written a bot. It started as a little helper tool for my own, manual SNG-playing, showing a HUD on the table with the other players’ OPR stats etc etc. Having that in place, I couldn’t resist the temptation to see how hard it would be to do some table scraping, and when it turned out that was really straightforward, I started coding up a little rule-set that I believed would be able to win at low-stakes SNGs, where play is less than stellar…
After about 4-5 weeks, of off-hour, hobby programming during some evenings and weekends (I have a family with two small kids, so time is limited), I had it absolutely *crushing* the 18-man turbo $1.75 and $3.40 SNGs on Pokerstars — with a 50% ROI vs the field (ie not counting the rake). Actual ROI (counting the rake) was ~35%.
Table scraping etc was effective enough that the bot could easily 24-table (and probably way more — I never tested with over 24), but I kept it at 12-18 simultaneous games, to not go completely overboard on the volume.
After about 1.5 months, and 6-7k games, Stars’ security team busted me. I’m not sure what triggered it — it might have been a captcha I missed (I usually just left the bot playing overnight while I was sleeping…), it might have been decision patterns that set off an alarm, it might have been things they scraped off my computer (mouse movements or something like that — although as you also mention, I believe that’s actually quite unlikely, as they will get an insane amount of false positives, for TableNinja users etc), or — most likely — it might have been the volume that led to a manual investigation of the account.
Just before I was busted, there were seemingly random changes to the layout of the hole cards and board cards, and they were sometimes antialiased a little different. That was probably a trap in the Stars client that they can trigger from the server side, to see if the other side starts timing out or doing something else stupid. And my bot obviously fell right into it.
A couple of days after that happened, I then got the same email you got, about terminating my account, confiscating the money I had (I had made a withdrawal the day before, so that was a tiny sum), and expelling me forever from playing at Stars.
Oh, well. It was a short but interesting ride — but as you also say, it wasn’t really worth it.
gotta protect yourself from being detected next time.
Try fulltilt, ultimate bet, etc and try again, but this time use stealth!