As a web developer who is also interested in A/B testing and evolution, it occurred to me that it would be fascinating to try to build a website that optimizes itself. I’ve been kicking around this idea for a while and wanted to share a few thoughts on it here to get feedback from anyone else that finds the idea promising.

Here’s the idea:

In traditional A/B testing, you specify multiple variations of some aspect of your site, measure which variation performs the best, then make the winner the default, and repeat. The types of things you test can range from simple headline and button color tests to complex tests that affect the user’s entire experience with your site.

In all of these cases though you have to figure out what variations you want to test. If you’re testing the headline, you need to specify which headlines you want to test. If you’re testing button color, you need to specify which colors to test, etc.

In the natural world, we see stunningly complex and optimized life forms that evolved little by little over billions of years. Evolution is similar to A/B testing in a sense, except that in evolution the variations happen by accident through genetic copying errors. Most of those mutations decrease the organism’s odds of reproducing, but occasionally they confer a benefit that causes that organism to have a better chance at surviving and reproducing. When that happens, the mutation gets passed on and may eventually spread to the rest of species over time. That “simple” process is what has led to all of the variety of life on earth.

Optimizing a web page through evolution poses many issues though.

How do you algorithmically mutate something on the page? Could you write a function that generates variations of a headline? Maybe. Would those variations be any good? Would you trust it enough to deploy into production?

I bet by analyzing tens of thousands of webpages, you could algorithmically identify common headline wording structures. Then maybe you could write code that intelligently customizes those headlines to your service.

You might be able to do the same with design. If you tried to categorize hundreds of homepage layouts, I expect you’d probably wind up with 20-30 common layouts that cover 90%+ of websites. Could you then write an algorithm that automatically tweaks your site’s layout to test these different layouts on your visitors? Could you do the same with color schemes? Maybe.

There’s also the problem of statistical significance. Running a simple two variation A/B test can take a long time if you don’t get a lot of traffic. Trying to get significant results for lots of algorithmically generated headlines might take forever. But maybe there are ways around that like the multi-armed bandit algorithm.

To sum it up, I think you’d need the following: a way to intelligently generate mutations on headlines, buttons, layout, etc + a ton of traffic or some novel method for picking the best variations + an organization that is willing to try such an experiment. It would not be easy, but I think it’s possible.

Imagine if it did work. Maybe your homepage converts users into sign ups at 10% now. You flip a switch, and in 6 months it increases your conversion rate 30% without any intervention on your part.

It would be revolutionary… if it worked.