PokerShark: Gaining an Edge at Online Poker

This is post #9 in an ongoing series of articles about my work as a poker bot developer.

Imagine you’re an online poker player and you suddenly have the ability to only play bad opponents. Say goodbye to the tough, aggressive, profitable players and say hello to the legions of loose, passive fish that make up the online poker community.

With a little bit of work, that’s exactly the situation I found myself in.

Some background: When I started playing online poker in early 2005 I was immediately drawn to one vs one No Limit Hold’em tournaments, known as Heads Up Sit-n-Go’s (HUSNGs). They’re fast, exciting, and they require an increased attention to psychology that you don’t get as much of at the full ring (9 players) tables.

Here’s how you join a HUSNG (this will become important later):

1. Find an upcoming tournament.

The PokerStars lobby helps you find exactly the type of game you want to play.

2) Open up the Tournament Lobby.

Once you’ve found a game that you like, double click it to open up the tournament lobby:

The lobby displays important information about the tournament such as how much it costs to play (the buyin plus the rake), the payout, the blind structure, etc.

It also shows you how many other people are registered in the tournament. If you’re the first one to register,  the list on the right will be blank, otherwise it will show the name of the person who already signed up.

3) Register. When you’re ready to play, click Register and join the tournament.

As soon as the table fills up with two players the game begins.

You can join a tournament any time you’d like. As soon as one fills up, another one is automatically created. During peak hours it can actually be a challenge to sign up for one because so many people are trying to register at once.

With thousands of people playing at the low and medium stakes, you can literally play for weeks without facing the same opponent twice.

This is fantastic news if you’re up against a tough opponent because you know that you’ll probably never face him or her again. BUT, if you’re playing some donk (a donkey–a bad player), it’s frustrating for the exact same reason: you’ll likely never play him again. And you want to. You really want to.

One important point: For a few minutes after a HUSNG ends, you can still locate it and check the results:

Why PokerStars let’s you access this is beyond me.

Anyway, after a few months of playing the HUSNG opponent lottery, I decided to see what I could do to improve the situation. I set out to create a program that would record the results of every HUSNG played on PokerStars and then use those results to determine which opponents to play and which to skip.

Several months and many iterations later, it was built. I called it PokerShark. (The original version was called PokerSanta but I decided that was kind of girly and changed it to PokerShark.)

24/7 I had a program running that recorded the winners and losers for every HUSNG and when I was ready to play, I ran a second program which opened up tournament lobbies and waiting for a player to register. As soon as someone registered, the program checked the player’s previous results and determined if he was mediocre enough to play. If he was, the program would automatically register me for that tournament.

The results were simply incredible:

Here’s what the software looked like: (click to expand the screenshots)

July 13, 2005:

At first, I focused on the software that recorded the results.

The window on the left was my attempt at concisely visualizing the recorded results.

The small, busy window below the PokerStars lobby displayed collection statistics and the large “Tournament Intercept Window” intercepted the Completed tournament windows as they were opened so it didn’t steal the focus away from anything else I was doing.

December 16, 2005

The ugly maroon window on the bottom left was my first shot at automating the registration process.

February 16, 2006

Eventually I added support for multiple buyins and made the interception window much smaller.

There were a lot of interesting graphs…

July 7, 2006

Over time I improved the design (note the little icons next to the buyins) and added extra analysis criteria such as jump ratio, which measured the stakes a player was currently playing compared to his average stake. Higher jump ratio = more tilt = more I want to play.

October 5, 2006

Notable on this one is the statistic overlays on each of the tables, which was another program I wrote that helped me make better decisions.

Using this software was a trade off. On one hand I didn’t have to face any tough opponents, so the money came easily and I didn’t have to face a lot of the stress typically associated with heads up games. However, because I was playing against a terrible opponents, I picked up a lot of bad habits which actually made me a worse poker player. It’s funny how things work out.

By October 2006 I had had acquired a ton of experience programming add-ons for the PokerStars software, and decided to take it one step further and to try to build a bot.

I mean, how hard could it be? =)

No Limit Hold’em Poker Bot Profits by Effective Stack Size

This is post #8 in an ongoing series of articles about my work as a poker bot developer.

Two weeks ago I posted a chart of the poker bot’s net income for its last full month of play, September 2008. As a poker player, your net income is a crucial figure because it determines what you can actually buy with your hard earned profits. That’s the number you would tell your non-poker playing friends if they asked how you were doing. However, net income only paints part of the picture. To truly measure your results and your progress, you have to break down net income and analyze exactly where your profits and losses are coming from.

As a bot developer, identifying and eliminating weaknesses in the bot’s play were crucial to making it profitable. One way I did this was to break down its results into groups based on its effective stack size.

Effective stack size is best explained by example: Say you’re playing No Limit Hold’em with one opponent and you have $25 and he has $10. The important thing to realize is that your opponent cannot risk more than $10 in a single hand because that’s all he has in front of him. If he can’t risk more than $10, you can’t win or lose more than $10 either. Your effective stacks, therefore, are $10. It’s what you’re effectively playing with.

Put another way, your extra $15 does you no good playing against an opponent that can only risk $10. Because that extra $15 is not in play, you should devise your strategy based on the effective stack size; not your actual stack size (stack size should play a central role in your poker strategy).

Furthermore, you shouldn’t measure the size of your stack in chips or in dollars; you should measure it in terms of big blinds.

Consider this: you’re playing $1/$2 Heads Up No Limit Hold’em with a $200 stack. You’re the small blind and dealt pocket tens. All other things considered equal, you should make the same move as if you were playing $100/$200 with a $20,000 stack because in both cases, you’re playing with a 100 big blind stack ($200/$2 = 100, $20,000/$200 = 100). You could also measure your stack in terms of how many small blinds you have, but the standard is usually big blinds.

Q. You’re playing a Heads Up No Limit Hold’em Sit-n-Go (HUSNG) and the blinds are 25/50. You have 2,000 chips and your opponent has 1,000. What’s your effective stack size in big blinds?

A. You can’t risk more than 1,000 chips in a hand, so your effective stack is 1,000 chips, or 1,000/50 = 20 big blinds (bb).

When you start a HUSNG you’re given 1,500 chips and the blinds are 10/20, so you have a 75bb effective stack, meaning that in a HUSNG, you can never have more than a 75bb effective stack. If the blinds jump to 15/30 and you still have 1500 chips, you have a 1500/30 = 50bb stack.

The bot’s strategy changed depending on the effective stack size, which is why I broke down the net income into the different groups. The groups might seem strange (ie: why “22-35 bb’s?”) but there’s a method to the madness. Some of it is based on postflop stack to pot ratios and some of it simply based on preflop stack size.

Finally, “5 bb/100″ means that the bot won, on average, 5 big blinds every 100 hands. Measuring bb/100 is the standard way to gauge a player’s ability and what is a good number will vary based on the stakes and type of game.

This is all a long way of saying here are the results of my poker bot for its last three months of play broken down by effective stack size.

50 – 75 Big Blinds

21,479 hands @ 31 bb/100

~

35 – 50 Big Blinds

12,279 hands @ 14 bb/100

~

22 – 35 Big Blinds

11,615 hands @ 4 bb/100

~

15 – 22 Big Blinds

5,688 hands @ 5 bb/100

~

10 – 15 Big Blinds

6,609 hands @ 5 bb/100

~

0 – 10 Big Blinds

5,076 hands @ 3 bb/100

ALL IN Expert on PokerStrategyForums.com

Pardon me if it seems like I’m bragging–it’s just that I had pretty given up on ALL IN Expert so it’s really a pleasure to finally see it being discussed.

From PokerStrategyForums:

Another cool program I just recently found is called All-in-Expert. Its kind of similar to pokerstove in that it deals with ranges but its just for preflop play. You put in the amount in the current pot, the amount you have to call all-in and define your opponents range, and the software pumps out a grid of all possible hands and tells you whether you should fold or go all in with each hand and it gives you your ROI against your viillains range.

This tool has helped me a lot in dealing with those super tight shortstack players that only play premium hands. I was raising them all-in with hands like AK and JJ which was actually a mistake. The only hands I could profitably raise them with were QQ+. Even when I raised first in to 3xbb and they pushed all in for 20bb I found that calling with AK was a mistake. The better your pot odds get, the more hands you can profitably add into your calling range.

“Determine Never to Be Idle”

This is one of the most creative things I’ve ever seen.

It’s hard to describe why. Something about the author’s juxtaposition of the paintings and the photography or the simple, clever writing in that playful white font. My initial impression was that it was a sloppy mess, but the more I looked at it the more I realized that every word of every sentence was deliberate and purposeful. It’s inspiring.

The author wrote a piece on Lincoln too.

Introduction: Heads Up No Limit Hold’em Sit-n-Go’s

This is post #7 in an ongoing series of articles about my work as a poker bot developer.

There are two ways to play poker: cash games and tournaments.

In cash games, your chips are worth dollar amounts. When you bet $20, you actually risk $20. You can usually start and stop playing at any point and when you’re done, you trade your remaining chips in for their cash value.

In tournament poker, chips don’t represent dollar amounts. This is usually the type of poker you see airing on ESPN and the Travel Channel. You pay money to join the tournament, get a certain number of chips, and then play until you lose all your chips or win everyone else’s. You get paid based on how many people are left when you lose all your chips and that amount varies based on the structure of the tournament.

Sit-n-Go’s are basically fast tournaments. They’re specially designed not to last very long.

There are many types of Sit-n-Go’s depending on what type of poker you enjoy playing: 2, 4, 8, 9, 27, 180 people, Limit, Pot Limit, No Limit, Omaha, Stud, Razz, HORSE, etc.

The poker bot I designed was for Pokerstar’s Low Stakes Turbo No Limit Hold’em Heads Up Sit-n-Go’s, or, simply, HUSNGs.

That means:

  • PokerStars – One of the leading online poker sites
  • Low Stakes – The bot is profitable at the $2 + $0.20, $6 + $0.25, and $11 + $0.50 levels
  • Turbo – The blinds increase every 5 minutes instead of the normal 10
  • No Limit – You can bet your entire stack at any point
  • Hold’em – Two holecards, five community cards (flop, turn, and river).
  • Heads Up – You and one other opponent
  • Sit-n-Go’s – They usually last about 15 minutes.

HUSNGs were my specialty (I’ve played about 130K hands) and what I knew the most about, which is why I chose them for the bot.

To join a HUSNG, you pay a buyin and a rake. $6 + $0.25 means that the buyin is $6 and that you also have to pay $0.25 to PokerStars in order to play. That’s how they make their money. If you win the tournament, you win both buyins ($12) for a net profit of $12 – $6 – $0.25 = $5.75. If you lose, your opponent gets your buyin, so you’re out $6.25.

When the HUSNG starts you get 1500 chips and the blinds are 10/20 (the players alternate posting 10 chips and 20 chips before the cards are dealt). Five minutes later they jump to 15/30, then five minutes later 25/50, 50/100, 75/150, 100/200, and so on. Most games end before the blinds reach 75/150.

Here’s what it looks like:

With that out of the way, we can start to discuss the theory behind the poker bot’s decision making algorithms.

More to follow…