A Poker Bot Comment Worth Reading

This is post #13 in an ongoing series of articles about my work as a poker bot developer.

An anonymous poster left the following comment on my last poker bot post:

Hi Matt,

very, very interesting article series — please keep it up if you have more material.

I’ve also written a bot. It started as a little helper tool for my own, manual SNG-playing, showing a HUD on the table with the other players’ OPR stats etc etc. Having that in place, I couldn’t resist the temptation to see how hard it would be to do some table scraping, and when it turned out that was really straightforward, I started coding up a little rule-set that I believed would be able to win at low-stakes SNGs, where play is less than stellar…

After about 4-5 weeks, of off-hour, hobby programming during some evenings and weekends (I have a family with two small kids, so time is limited), I had it absolutely *crushing* the 18-man turbo $1.75 and $3.40 SNGs on Pokerstars — with a 50% ROI vs the field (ie not counting the rake). Actual ROI (counting the rake) was ~35%.

Table scraping etc was effective enough that the bot could easily 24-table (and probably way more — I never tested with over 24), but I kept it at 12-18 simultaneous games, to not go completely overboard on the volume.

After about 1.5 months, and 6-7k games, Stars’ security team busted me. I’m not sure what triggered it — it might have been a captcha I missed (I usually just left the bot playing overnight while I was sleeping…), it might have been decision patterns that set off an alarm, it might have been things they scraped off my computer (mouse movements or something like that — although as you also mention, I believe that’s actually quite unlikely, as they will get an insane amount of false positives, for TableNinja users etc), or — most likely — it might have been the volume that led to a manual investigation of the account.

Just before I was busted, there were seemingly random changes to the layout of the hole cards and board cards, and they were sometimes antialiased a little different. That was probably a trap in the Stars client that they can trigger from the server side, to see if the other side starts timing out or doing something else stupid. And my bot obviously fell right into it.

A couple of days after that happened, I then got the same email you got, about terminating my account, confiscating the money I had (I had made a withdrawal the day before, so that was a tiny sum), and expelling me forever from playing at Stars.

Oh, well. It was a short but interesting ride — but as you also say, it wasn’t really worth it.

Awesome.

And wow: 4-5 weeks to program a winning low stakes SNG bot with a 35% ROI. That’s pretty remarkable.

I’ve always had a suspicion that the abnormally high number of HUSNGs that my bot played/day was what ultimately led to its demise. I suspect that may be the case here too: 6-7K SNGs in 6 weeks–I bet that’s quite an outlier ;)

Another thing he mentions is the seemingly random changes to the layout, which I can also attest to. The tables would randomly resize by a few pixels, which would normally be imperceptible to a human player but for a bot that depends on certain pixels being in exact positions, it causes a lot of trouble.

Anonymous poster whoever you are, thanks for the fascinating story.

Building a Shortstacking Poker Bot – A Visual History

This is post #12 in an ongoing series of articles about my work as a poker bot developer.

In late 2006 and early 2007, I tried to build a shortstacking poker bot.

Shortstacking, in a nutshell, is a strategy that advocates buying in for a relatively small amount of money compared to your opponents and then playing very aggressive preflop poker. Most opponents do not adjust correctly to your stack size, and, in theory, you can make a lot of money from their mistakes.

Here’s a quote from shortstacking strategy guide on what used to be DonkeyDevestation.com:

By buying in with 20bb, you control the size of the stacks that the hand is being played for, not your opponents. Most of your opponents will be making plays preflop that are suitable for a full stack game but would be losers against 20bb stacks, and therefore losers against you. They take the worst of it against you to play better against the others, and you profit from this. Many players will complain about short stack players, claiming they are “lame” or “unskilled”. And they are right. It is lame, and it does take less skill to play a short stack. But it is very profitable, especially in today’s games. Furthermore, unlike most strategies, short stack play actually becomes more effective as you move up in limits, because other players tend to be more aggressive preflop, meaning they are putting more money in with hands they would not raise if they were only playing against you.

Shortstacking was appealing to me as a first time bot developer because it had been written about fairly extensively and the strategy, at least from a preflop perspective, was pretty well defined.

Here, for example, is a chart of a player named cuzco, who successfully employed a shortstacking strategy:

cuzco

How hard could it be to study his hand histories and have a bot emulate his decisions?

I already had a bit of experience writing add-ons for the PokerStars software (see PokerShark), so I figured it would only take a few weeks to get a prototype together.

In November 2006, I started programming.

The first major hurdle was figuring out how to read the chat box on the PokerStars client. PokerStars uses some custom controls which can’t easily be read using standard Windows API techniques, so I tried using Optical Character Recognition (OCR) to convert the pixels to words:

PokerStarsOCRInitial

And later:

PokerStarsOCRProgress

Eventually I figured out that PokerOffice, which makes a popular add-on for PokerStars, adds their own chat control on top of the default PokerStars one and it can be read using standard techniques. This was a big step because I no longer had to rely on OCR to get the text, which didn’t work very well (a lowercase L and a capital I look the same, for example).

Reading the text on the PokerStars Lobby was also important in order to automatically open new tables:

LobbyOCR

Using some shortstacking strategy guides as a starting point, I put together extensive preflop and postflop logic charts which were eventually implemented in the bot:

PreflopLogic

By the end of January, I had a working product:

InterfaceJan

It was capable of multitabling like the machine it was:

12Tables

Most days it lost money, which was to be expected during development, but some days it did very well:

PokerTrackerSwings

And that often gave me hope that the changes I had made the previous day were all that was needed to finally make it profitable.

I did a lot of A/B testing to try to measure what effect, if any, the changes I was making were effective:

ABTesting

Another thing I had to worry about were CAPTCHAs, which PokerStars is fond of using to identify bots:

CAPTCHA

Eventually I realized that conditional statements were not enough. There are simply too many situations in No Limit Texas Hold’em to account for–even when you’re shortstacking. For example, how do you take into account bet sizes and your opponent’s stack size? “IF … THEN” could not adequately handle that, so I started playing around with Expected Value (EV) calculations:

DecisionTreePostflop

Unfortunately at the time I didn’t have the tools to implement these techniques.

Nonetheless, I persisted with the shortstacking bot, believing that it only would take a few more changes before it was profitable.

March rolled by…

InterfaceMarch

And April came around…

InterfaceApril

But, despite my best efforts to turn it profitable, it kept losing.

By the time I decided to quit at the end April, the bot had played roughly 49K hands and blown through more than $1,300:

PokerTrackerGraph

Eesh.

Shortstacking, it seemed, was harder than it looked.

What I eventually realized is that your profits from shortstacking come from a few key decisions and if you didn’t make the correct moves at those moments, you couldn’t win enough to be profitable. In other words, I may have had a 90% solution–though I doubt it–but because it couldn’t make the correct decision that remaining 10% of the time, it could never be profitable in the long run.

Frustrated but determined not to let the work go to waste, I decided to start from scratch and to build a No Limit Heads Up Sit-n-Go (HUSNG) bot. That was my game of choice and I figured as long as I’m going to work on something, it might as well be with a game that I enjoy and had a lot of experience with.

Tatta, one of the most profitable Sit-n-Go players on PokerStars, was my inspiration:

tatta

“If I could only achieve a fraction of his success,” I reasoned to myself, “this will have been worth it.”

For the rest of this story, I’ll point you to the previous 11 posts about the HUSNG bot.

To sum it it: after several more months of work it was profitable, but shortly thereafter, PokerStars caught it and closed my account.

Doh. :)

Quote: “Every system has two sets of rules…”

Every system has two sets of rules: The rules as they are intended or commonly perceived, and the actual rules (“reality”). In most complex systems, the gap between these two sets of rules is huge.

Sometimes we catch a glimpse of the truth, and discover the actual rules of a system. Once the actual rules are known, it may be possible to perform “miracles” — things which violate the perceived rules.

Read more on Paul Buchheit’s recent blog post, Applied Philosophy, aka ‘Hacking’.

Paul is also speaking at Startup School–I can’t wait.

Notes from Startup Bootcamp

I was fortunate enough to be able to attend Startup Bootcamp at MIT yesterday, a one day event that featured talks from a diverse group of entrepreneurs and VCs on how to start a company.

These are the notes I took from the presentations. A lot of it is verbatim, but enough is paraphrased that you should check video recording on Justin.tv before quoting anybody directly. If there are any glaring mistakes please let me know.

P.S. : I’m flying out to California for Startup School on Oct 24. Shoot me an email if you want to grab a drink sometime.

Robin Chase, founder and former CEO of ZipCar

There are these huge giants out there that are toppling or are getting ready to topple.

It’s a great time to start a company.

A lot to do, a lot to be done.

Every single thing you’ve done since you were two comes into play. Two founders: twice as much. Three: three times as much.

Everyone you come in contact with is your free consultant. They’re not a jerk for not getting it. You’re a jerk for not explaining it to them so they get it. They’re an amazing resource. You should hone your idea and the way you express it and move forward.

Be honest with yourself about what sucks.

Execution is everything.

Keep your eye on your own stuff. You want to be best at your particular thing.

Start light. Listen, iterate, improve. Become a learning organization.

The one thing we have going for us is we are nimble, we can turn on the dime.

Making mistakes is OK, not learning from your mistakes is a problem.

We are the stories we tell. We could have created this list of ten thousand dos and donts, but instead we created a culture. We created a very strong sense of the company and what it was.

Being the CEO of a startup is very lonely. Can’t be intellectually honest with others. Make sure you’re telling stories that you want repeated. Ripple effect when you produce a high value product.

Goal #1: Sustainable (Profitable)
– A needed product or service
– Intellectual honest to get right
– Learn fast. Be a learning organization
– Listen to your customers
– Values and consistency
– Your team should reflect the world you want to live in
– Watch the cash! Focus. Manage risk, You are start up!
– Luck is when preparation meets opportunity.
– Problems are opportunities.
– The status quo is open they often this amazing opportunity.

Zipcar Stats:

325,000 people.

6,500 Zipcars.

100,000 cars not bought.

300,000 parking spaces not needed.

500,000 tons CO2 not emitted.

Sharmesh Shah, created OnStartups and founder of HubSpot

Maximize your odds of a modest outcome. HubSpot, $17 million funding, 90 employees.

Don’t like the process of taking funding. Thinking about founding a startup? Congrats, you have a genetic flaw. Admit it to yourself. Take the leap.

Why are you waiting? Your idea stinks. Get over it. The idea of finding an idea that’s going to win dramatically improves after you startup.

Stealth mode is for fighter jets, not startups. Most of you should not have a stealth startup. You have this idea. You’re going to spend years convincing people to pursue your idea. The odds that someone else being convinced: slim to none. The best barrier to entry: its really hard. The next best thing is marketing. You can build a barrier to entry with marketing. It’s very hard to pull that off. Once you run through the process and get something working its very hard to replicate.

Outbound marketing is about pushing messages out. You spray the world as best you can with that message and hope that a fraction of people come back to your site. Inbound is about pulling people in. Instead of spending all this money trying to go out and get people to come back, make it easier for people that do want it to find you. Your job should be to connect to that market. Make marketing about creativity, not cash. Inbound is good for startups because it turns out it gives us leverage.

It’s impossible to hate a speaker that has a photo of a kitten on his slides.

Places to get found: Google, blogs, and social media. “As ye SEO, so shall ye reap”. Single largest source of free traffic.

WebsiteGrader.com – analyzes your website for SEO. Keywords: Don’t pick a fight with a ninja, unless you are one. 1) Relevant to your business, 2) How many people search for that keyword. Steep decline for click-throughs for results. (No one checks the long tail). If you have a new website, don’t go after the keywords with heavy competition.

Google Ranking Algorithm: f(n) = Context + Authority. Authority = # of inbound links.

The page title is the strongest indicator to Google what your page is about. Don’t just put your name, describe your site too. Don’t waste it. Put the description first. The earlier the keyword, the stronger the signal to Google. “iPhone Blackjack Software | Itsosoft” is better than the opposite. To rank well, get powerful links. Don’t get too obsessed with SEO though, don’t forget about the humans.

Write a blog, not a business plan. Writing a business plan? Please stop. No one is going to read your business plan, especially VCs. If you need to articulate your thoughts, write a blog. Build a following before you build a product. Get a dummy website out there. Start saying something interesting. Start marketing the day you decide you might someday start a company.

For a blog, don’t be afraid to polarize. Sitting in the middle doesn’t help. Take a stand on something you believe in. Use Facebook ads to do market research. Use Twitter.

Look deep into your data. Marketing has turned into a geeky, data driven business.

Not having fun? You’re doing it wrong. Build a team. The hardest thing to do is building a team. If you’re in the area start connecting to people.

Be a superhero. Find your superpower. Find what you’re exceptionally good at and go do it.

Blog: should have a personal tone to it. Part of the value of blogging is self discovery. People think social marketing is another way to blast your message out. Trickle launch. The sooner you get it out there the sooner you get good feedback. On average we tend to overestimate the impact of that first impression. Be apologetic. Say this is what it is. Get it out there. Charge early. It’s OK to suck early on.

Dan Bricklin, founder of Visicalc

Check out the Justin.tv link at the top of the page to watch Dan’s excellent presentation on VisiCalc.

Angus Davis, founder of Tellme Networks

Inspiration books: Memos from the Chairman. Selling the Invisible.

Which vignettes matter the most? WWSBD? What would Steve Blank do? Four Steps to Epiphany. Customer Development. Before you go out spending $200M on a product, talk to the customers and find out what they really want.

Think first like a shareholder. What’s going to optimize my value as a shareholder? What if you’re the CEO… what if the VC’s want to kick me out? One of the last things they want to do. It’s probably one of the best ways to screw up a company. The CEO has attachment to the employees and the idea. It’s a disruptive thing for them to do. If you have the opportunity to hire a Jim Barksdale… do it.

If you want to do a Yahoo vs a Wufoo you need to prepare to make some major sacrifices.

Drew Houston, founder of DropBox

Everyone starts out clueless. It’s easier to go from engineer to business than vice versa—how can we take advantage of this?

You’re clueless about a large number of things that are essential to starting a company. Drink from the fire hose: sales, marketing financing. Product design. Psychology influence negotiation. Startup stories. Management and leadership. Business strategy. HN, VC and entrepreneur blogs, MIT EClub, WebInno, BarCamp, etc.

Take on responsibility; Build people skills; get out of your comfort zone. Join a startup; Learn on someone else’s dime. Take on positions for which you are completely unqualified. It’s a lot easier to start on your own when you have already seen the road ahead. Surround yourself with kindred spirits.

Forgotten USB drive → Dropbox. Some of the best startups are ones that solve your own problems.

Technically 1) challenging, 2) Large market, 3) Explain to normal people so they can understand what you’re talking about.

Passion for the idea is whats going to carry you through.

Not shown: 2.5 years of crazy hours, equal parts terror, panic, and elation, and a savage obsession with making a product people love.

A startup CEO:

Idea: assess the market, prototype the product

Early: Hire a few engineers, build and launch the product

Growth: Hire exec team, prove business model

Scale: Inspire and lead, build an enduring culture

Beginning: Tech & Product Skills most important. Later: Management and leadership skills most important.

As founder/CEO, your job description is rewritten every 12 months. Less coding, more leading and managing. Two challenges: scaling your company while scaling yourself. Immense personal growth. Opportunity to create something from nothing, put in millions of people’s hands.

Failing is irrelevant; you only have to be right once.

Alexis Ohanian, founder of Reddit

One overarching principle: no one wants to use your website. Only your mom wants to use your website. It’s a constant struggle to build your app… you need evangelists. Google… you didn’t hear about it from an ad campaign. Google didn’t need advertisements because it was good. That’s called organic traffic. Investors absolutely love it. You don’t need PR firms. If its good enough, its only a click away.

At the day of our acquisition our biggest expense was rent: $1,500.

Keep it real. Businesses in general have set the bar so incredibly low. Especially online… you don’t expect that kind of humanity there.

Don’t be exceptionally good… just be exceptional. T-Shirts: do it.

Facilitate serendipity – frankly a lot of it is luck. Work… intelligence… yes, but luck is important. We like to believe we can control it.

People love underdogs. People want someone to root for. Be root-for-able. Karma matters.

You will fuck up. Being good is insurance for when you’re dumb. Often times it makes things happen.

Make something people want.

Make something people love.

Do something you love.

Adam Smith, founder of Xobni

If we can improve email just a little bit we can create a lot of value.

What was most important is that it was something people wanted. We created something people wanted and there’s a market for it.

Before you reach product market fit its important you stay small; that you don’t grow past 5 or 6 people. Once you get big its no longer about creating a product people want… its more about real raw execution. You have something people want but don’t have a clear proven business model. There’s a lot of execution to do.

You have to hire a team. It took Google from 1998 to 2004 to go from product market fit to IPO. For the first year and a half of our company, 7 days a week all waking hours. It was too much looking back. Xobni Analytics was like Google Analytics for your email. They might look at it once a quarter. It doesn’t create enough value for the user. So, enterprise software analytics or come up with something completely different. It’s a brutal fact of startups you have to be completely nimble. Paypal went through four product ideas before settling on online payments. These other successes went though several evolutions…

The most important thing is to be nimble.

How to execute well: Hire good people. You want people that get things done. Fire and forget.

External deadlines are very useful. Work tends to expand to fill the time allotted. Sign yourself up for something like TC40. Put the onus on action. Barring a reason not to do something always do it. Otherwise you’re not going to be experimenting enough.

Risk. Expect ¼ of your projects to fail. If everything you do is working you’re not taking enough risk; you’re not being ambitious enough.

Focus on the user. Have lots of experience but make sure they’re strategically focused. Yahoo: spread too thin. Xobni does not solve the email triage problem. There are lots of dead bodies in the battlefield. Not something a lot of people think about, but its key.

Hire an outsider.

Build your DNA explicitly into the company. You don’t get to build your own DNA, but you can for your company.

Get people with diverse experience.

If you can find things that only you can do well, thats awesome.

Raise barriers to entry.

Wow your users at the same time. 90% of execution is keeping going.

Half of experienced people say Boston or SV doesn’t matter, but I do think the west coast is the center of all life in the startup universe. Investors… lawyers… startups… the culture… all out west. Center of the startup world. Generally speaking its great to be in California. Investors are all looking for great details.

Founders at Work. High Stakes No Prisoners. PG’s essays… going to be my kid’s first readings.

Aaron Swartz, founder of Infogami and Jottit

What type of startup are you?

Regarding big launches, DHH style: is Hollywood really the type of industry you want to emulate? If not the Hollywood launch, then what? The Gmail launch. Get people from day 0. Even if it is just you and your cofounder. What’s the smallest possible piece of this I can make? Every single day he had something he was using; something he was improving.

While your software is annoying, fix it. Keep sanding away all the edges that make things annoying. While people are happy, get more users, then repeat. Grow slowly but strongly. Do not underestimate how much users will help you define your product.

Every time we slacked off, traffic went up. Less changes = more traffic? When you make changes, people get mad.

Hemant Taneja, VC at General Catalyst Partners.

Harvard Square, 75 companies, $1.7 billion under management. Eight partners. All entrepreneurs. Early stage investments in consumer, IT, and energy.

Backing great people with big ideas.

– Social Media is a key enabler for consumer businesses

– Mobile computing is finally here thanks to the iPhone

– Tech innovations will solve global climate and energy crisis

– Major changes in healthcare and financial services sector

– Synthetic biology is in its infancy

– Such a fertile area in terms of great ideas and big problems we could be solving.

Startups we love…

– Have brilliant founder(s)

– Solve very hard problems

– Address very large markets

– Are ahead of the curve

– Are capital efficient

Should you raise VC money? Only if you have to. You’re inviting another partner into your ecosystem. If you’re doing consumer internet site, launch, eat a lot of peanut better and jelly, and see what happens. Might as well create something, show some data, get some data about how consumer are interacting with your product, and then go raise money. You have to weigh the options.

Choosing a VC. Smart… good listener. You want somebody who is open minded who can bring their experiences and understand why you’re doing what you’re doing. Easy going. Make sure its the kind of person you can reasonably resolve and work though issues. Bandwidth to help. Have to do due diligence. Will they have time to devote to being partners with you? To really engage. The more they engage the more they can open their resources to take advantage. Transparent. You want the type of VC who will be open with you. Somebody who can really sit down and engage you. Somebody who doesn’t already have their mind made up. You want someone you can have those easy transparent conversations with. Relevant network. Stable VC firm. That they’re going to be around to work with you for the next 5 – 7 years. Ask them for their CEO references. Talk with the people they have worked with in the past.

Spirit of a good termsheet…

– Let’s make sure there’s enough capital for significant milestones.

– Legally simple.

– Balance of ownership for VC, founder, and management. There’s no magic answer to this. Figure out what your goals are. Work the numbers from there. There’s a balance between founder and VC incentives.

VC not focused on downside.

Board of directors with domain expertise. The more they can bring smart people around you the more you can scale and the more interesting business you can grow.

Daniel Theobald, cofounder of Vecna

You really have to think for yourself. You really have to question everything.

Never hire anyone. As soon as you hire someone you become a manager. And if that doesn’t work: hire the smartest people you can find and take good care of them. A great engineer is 10x as productive as a good engineer who is 10x as productive as an average engineer. If you can’t find the right person don’t just hire anybody.

Other people’s money: it does make you stupid. … it can really cause you a lot of trouble. Avoid it as long as absolutely possible. Make sure you’re doing the math. Make sure that that is actually going to get you what you want. You want to do a really good job taking care of your employees and your customers. When you have to worry about start to take care of your investors… how do you avoid other people’s money?

Read this: On the Folly or Rewarding A, While Hoping for B. If you reward a behavior that is the behavior that will be exhibited. It really is worthwhile to get this right. You have to reward the behavoirs you want from your employees and your customers.

Do something you’re passionate about. Have fun. If you’re not thinking about something while in the shower, while laying in bed at night, thinking about it in all different ways you’re just not going to make it happen. Find something you’re really passionate about.

Don’t put a lot of stock in what other people tell you. I’ve always been surprised by what people can make a success of because they’re passionate about it. Make a difference in the world. It’s hard to know who is providing value in your company. We don’t give bonuses if there are no profits. Taking outside investments: like pushing the self destruct button at the end of the Alien movie. You’ve got one hour to get out or blow up. Spend less than you make. It requires sacrifice. You have to be creative.

So much of the success you have is because of luck. Something other than the effort you put into it.

Kyle Vogt, founder Justin.tv

Trough of sorrow: are you doing the right thing? Should I go back to school? You start questioning yourself. Once you hit that magic inflection point things start to take off.

Startup Productivity Hacks:

1. Buy catered lunch. If you keep lunch in the office, it increases the number of hours you get in from people. With 10 people, its equivalent to an extra employee.

2. Use Google Apps. Better than emailing around attachments, don’t need to worry about all the IT tasks that you usually need to get started.

3. Use Data Driven Development. It’s tempting to try to predict what users what and what features the customers need. The best way to do it is to throw it out there, A/B test it with something like Google Web Optimizer.

4. Use hiring screeners. One thing that works really well is making people do the thing you want them to do before you hire them. Helps you not hire people who are not qualified.

5. Keep job interviews short. The goal when you’re hiring someone is to find out if they can get the job don’t in a reasonable amount of time. Prove their competence by getting them to do what you want to do. It takes 10 minutes to develop your instinct. Anything more than that you wind up second guessing yourself. Use the screeners. Make sure they fit the company culture.

6. Don’t hire a PR firm. It seems good on paper. Educate customers to buy your product. Increase sales. Brand name recognition. The PR form will pitch you on how they’re going to develop your company. Once you sign, you have to spend a lot of time teaching them about your company. Hire someone who is a marketing associate. He spent all his time absorbing their product and working his contacts.

7. Put one guy on fund raising.

8. Work from home. Especially if you’re bootstrapping. Save the money until you run out of room in your apartment or it gets too distracting. Need a separate room to make phone calls. An office lowered their productivity.

9. Use hosted servers. Don’t try to build or buy your own hardware. If you just use hosted stuff. If its too expensive, you can build out. The thing you have to factor is the resource it takes to build and run your servers.

10. Listen to your users right away. They can reliably tell you what goes wrong. The best way to figure out what you need to include is to watch them use your product and see what they use. Watch for their pain points. Not fonts and colors.

11. Stick to .com’s. JustinTV.com for sale for $2M. People get confused.

12. Be transparent with employees. People need to know how much money is in the bank. Are we profitable? Are we hiring people? If employees know about it they’ll be happier and contribute things back up the chain.

13. Don’t outsource core products. If you’re building a website and you want to outsource the video site if your video website, you’re going to run into problems. Draw a line. Write out the things you want to build and what you don’t and draw a line down the middle. Figure out what you can outsource.

14. Hire specialists when needed if no one on your team has the domain knowledge.

15. Hire people smarter than yourself. One bad person can pull your entire company down.

16. Have a plan for actually making money. This was our biggest lesson learned. It seems obvious that you need customers and revenue. It’s a painful process to turn a company with no business model into one so you can survive.

Ken Zolot, founder MIT’s Innovation Team.

What makes tech the basis of a viable company. What happens before you know its time to startup a company? Ask “who cares?” figuring out who cares and then how you’re going to get to them is something founders often overlook.  One of the first rule of business school: IRR: internal rate of return. Most investors are only interested in a land grab. Does it work yet? Is it special? Who cares? What do I have/know? Who can help?

Team technology and market. “Progress is about taking and managing Risk” – Cooper.

You have to be somewhere and bump into people in order to continue this process of iterations.

General Mills dumps strawberry seeds into mix, but no nutritional value because parents buy 10x as much. What is that magic strawberry seed that conveys credibility?

Engineers like to know how processes work. Machine learning: cognitive model or sense of environment? Roomba figures out where it is and responds. Stimulus. That’s the model for a good entrepreneur. Watch ants move a crumb across a room. They don’t plan with a conference. Sometimes responding to stimuli from the environment and knowing how to act is the magic.

Knowing how to move forward. Wanting to move forward. Really wanting something. The will to take that step.

Propulsion and will: 1. Do something. 2. Sense; reflect. 3. Do something else. Not complicated. If you look at really interesting engineering problems this is often how they are solved as well. “Man who waits for roast duct to fly into mouth must wait very very long time” (thanks to Woodie Flowers).

Presentation and advocacy: public speaking, advocacy, “enrollment”, relationship building. Getting folks excited about what you’re doing. The power of enrollment and evangelism and becoming really enthusiastic about what you’re doing. Knowing your passion and being able to convey that to something else is one of the most important skills of an entrepreneur. Take 90 seconds try to get someone you don’t know interested in something you’re doing. Find something that activates you. Something that you’re passionate about. Never underestimate serendipity, magic, luck.

Remember: Do something.

Experimenting with a Neural Network-based Poker Bot

This is post #11 in an ongoing series of articles about my work as a poker bot developer.

At one point or another most poker bot developers have an epiphany. Their eyes open wide and they excitedly shout to the next stranger they see: “I’ll do it with a neural network!” Seems like such a good idea, right?

I tested it out and my definitive conclusion is “Maybe“.

A neural network (NN) is an AI technique that intelligently maps input values to output values. “Huh?” you say? Here’s the idea for poker: You give a NN data from previous hands you played (position, card values, hand rank, etc) and the decision you made in those situations (call, raise, fold, bet, check) and the NN will learn how to mimic those decisions. For a poker bot, this is a pretty appealing idea: you find the hand history of a winning, high stakes player, train the NN, and then set you poker bot loose to win a boat load of money.

Designing a Simple Test

A little background: My original goal for the poker bot was a full ring shortstacking bot. Shortstacking is a nasty little poker strategy that advocates aggressive play with with a relatively small amount chips. You see, when you don’t have a lot of chips to play with, you wind up making a lot of all in decisions preflop and relatively few postflop and most opponents do not adjust correctly to your strategy; most opponents play like they’re playing against someone with a normal stack, which is the absolute worst thing you can do against a talented shortstacker. This made shortstacking the perfect strategy for my fledgling poker bot.

At that time, the shortstacking bot made its decisions based on some elaborate conditional statements (ex: if you have QQ, KK, or AA and in early position, then raise). I had been testing the it for several weeks when I decided to try out the neural network idea, so I had plenty of data to work with.

To test it out, I picked a very specific situation that the shortstacking bot had faced many times in the past:  Everyone folds to you preflop at a full ring (8-9 players) table, do you raise or fold? The bot never called in those situations, so I didn’t have to factor that in.

There were a total of 10,461 hands that met that criteria. For the NN, I used 7 input values:

1. The numeric value of the first hole card scaled from 0 to 1. So, for example, 2 = 2/14 and Ace = 14/14.
2. The numeric value of the second hole card scaled from 0 to 1.
3. Whether or not they were suited. 0 = unsuited, 1 = suited
4. My position at the table where 0 = first to act, 7/9 = 0.778 = Dealer
5. The average value of the two hole cards
6. The difference between the first card’s value and the average
7. The difference between the second card’s value and the average

The output was simply a 1 if it had raised and a 0 if it had folded.

Here’s what the data looks like in an Excel sheet:

Poker Bot Neural Network

You can also download the spreadsheet by clicking here.

The Results

The predictions of the neural network were stunningly accurate:

Correctly predicted Raise: 776/865 = 88.6%
Correctly predicted Fold: 9475/9596 = 98.7%
Overall Accuracy: 10241/10461 = 97.9%

The predictions indicate that it is possible to make quality decisions based on the output of a neural network.

However, and this is a big however: the conditional statements that controlled the decisions in the training data were not very complicated so it wasn’t very hard for the NN to learn the pattern. Training it based on a human’s behavior may have led to very different results because a human’s thought process is much more complicated than “if this then that”. Normal decisions are not simply based on your hole cards and position at the table. You also have to take into account your stack size, your image, your opponents, the dynamics at the table, and a host of other factors. But, interestingly, this is exactly what a NN is good at: learning how a wide range of variables affect a decision.

Despite the success of this test, I ultimately decided not to pursue a neural network based poker bot. The problem is that you don’t have much control of the decision making process. You can’t, for example, look back at a hand and analyze why it made a specific decision. The neural network will spit out a number and the bot acts accordingly. There is no why; it’s merely a feeling it had. It’s also difficult to be precise. Say you want to always raise with AA, raise with KK half the time, and always call with QQ. It’s not a trivial task to adjust a neural network to make those type of decision if the training data indicates you did something else.

One final note: When I first started developing the poker bot in late 2006 I spoke with someone online who claimed to have built a profitable Heads Up No Limit Sit-n-go bot based solely on the predictions of a neural network that he had trained on his own hand histories. Legit? Who knows. Makes you wonder though…

Bitmap File Format

I put together this chart showing the organization of a small Bitmap file:

(click to expand)

I used Wikipedia’s Bitmap File Format article in conjunction with UltraEdit and PowerPoint to analyze this 10 x 10 pixel Bitmap image:

Block

There are only two bytes unaccounted for: the 6B 22 at the end of the file. Anyone know what their purpose is?