One of the many people who saw the incorrect Google Quick Answer about Kenya was an editor at The Atlantic who asked Caroline Mimbs Nyce, one of their reporters, to look into it. Caroline interviewed me for the article they just published which focused on the challenges Google is facing in the age of AI-generated content.
From the article:
Given how nonsensical this response is, you might not be surprised to hear that the snippet was originally written by ChatGPT. But you may be surprised by how it became a featured answer on the internet’s preeminent knowledge base. The search engine is pulling this blurb from a user post on Hacker News, an online message board about technology, which is itself quoting from a website called Emergent Mind, which exists to teach people about AI—including its flaws. At some point, Google’s crawlers scraped the text, and now its algorithm automatically presents the chatbot’s nonsense answer as fact, with a link to the Hacker News discussion. The Kenya error, however unlikely a user is to stumble upon it, isn’t a one-off: I first came across the response in a viral tweet from the journalist Christopher Ingraham last month, and it was reported by Futurism as far back as August. (When Ingraham and Futurism saw it, Google was citing that initial Emergent Mind post, rather than Hacker News.)
One thing I learned from the article is the reason why Google hasn’t removed the Kenya quick answer, despite it being obviously incorrect and existing since at least August, is that it doesn’t violate their Terms of Service, and they are more focused on addressing the larger accuracy issue, not dealing with one-off instances of incorrect answers:
The Kenya result still pops up on Google, despite viral posts about it. This is a strategic choice, not an error. If a snippet violates Google policy (for example, if it includes hate speech) the company manually intervenes and suppresses it, Nayak said. However, if the snippet is untrue but doesn’t violate any policy or cause harm, the company will not intervene. Instead, Nayak said the team focuses on the bigger underlying problem, and whether its algorithm can be trained to address it.
The Atlantic article was published before I was alerted earlier this week by Full Fact, a UK fact-checking organization, about a more egregious example where Google misinterpreted a creative writing example on Emergent Mind about the health benefits of eating glass and was showing it as a Quick Answer:
On Thursday evening Chris Ingraham, a journalist with 100k followers on Twitter, shared a screenshot of the now-famous “african country that starts with k” Google Quick Answer, which quickly went viral, garnering over 82k likes and 3 million views as of the time of this writing on Monday morning:
Google was rotting from the inside out before AI came around but it's going to get 10 times worse pic.twitter.com/5NYlw4WTEy
Preceden’s designer, Milan, saw it on his feed and shared it with me on Friday morning, which I first saw when I loaded Twitter on my phone at the gym that morning.
I was like alright, here we go again. I’ll reply and explain what’s going on so anyone seeing the screenshot has some context:
And then my notifications started exploding with… shall we say… mixed reactions.
Some screenshots for posterity… 🤣
And the tweet that inspired the header image on this post:
In retrospect, in my original response to Chris I should have given more context, and not just assume people would click through and read the blog post. Or maybe I should not have responded at all, though I’m glad I was able to provide context, even if it resulted in some mean tweets against me, hah.
I also stand by my decision to keep that Emergent Mind page online. It’s a harmless and obviously incorrect answer that has now become the canonical example of how Google quick answers can get things wrong. Identifying and eliminating incorrect Quick Answers is no doubt hard to do at scale, but I hope this snippet and others like it contribute to Google addressing accuracy issues in time.
Sometime tells me this won’t be the last time this goes viral…
Emergent Mind, my AI news site that I’ve wavered on for months
A few months ago I announced I was going to try to sell Emergent Mind, my AI news aggregator, so I could focus on Preceden, my SaaS timeline maker.
I wound up having a lot of discussions with potential buyers, but in the end the offers I received were either too low to be worth the effort ($1k-$5k), or the offers were solid ($10k, $11k) but fell through during due diligence because of Emergent Mind’s lackluster growth.
That left me with a tough decision: keep it and continue trying to sell it, keep it and stop trying to sell it, or shut it down.
Me, explaining to potential buyers why they should take over Emergent Mind
There’s some chance that I could have kept up the sale page and eventually sold it for $10k, which is the minimum I was looking to sell it for. But, I’ve become increasingly skeptical that I could have found a buyer for $10k.
A big hurdle is that in order for a someone to take it over, they need to be a Ruby on Rails developer, or be willing to pay one to continue the development work. Ruby on Rails developers are in short supply, and most individual developers are not in a position where they can justify forking over $10k for a product with mediocre metrics (more on that below) that would likely require a lot of marketing work to grow (and most Ruby on Rails developers, myself included, would rather be coding than marketing). I did chat with one agency that had Ruby on Rails developers (who would have used it in part to promote their business), but that presented other challenges: multiple people in the organization had to be convinced it was worth $10k and worth reprioritizing their limited resources to work on it. Easier said than done.
And why not do the deal for like $5k? It probably would have taken 30 hours of work to complete the deal once I factored in the paperwork, account transfers, training, Zoom calls, emails, inevitable issues, etc, and I felt my time was much better spent focusing on Preceden vs going through all that for $5k, even less once you factored in the ~25% taxes I’d have to pay on that amount. And that’s assuming I skipped hiring a lawyer (which can easily cost a few thousand dollars) and just used some boilerplate legal templates I found online, which comes with its own risks.
So, I probably wouldn’t be able to sell it for $10k, and didn’t want to bother for less than that.
Reasons to just shut it down
There are plenty:
Visitors and subscribers have mostly trended down and to the right
Here’s weekly unique visitors to the site since launching last December:
But wait, you say, look at those spikes recently! Problem is that they’re entirely due to a single ChatGPT hallucination example going viral. The real trend is that gradual decline between April and early August where each week fewer and fewer people visit the site.
And it makes sense too. I haven’t done any marketing for the site in months, and AI hype has dissipated somewhat since its peak earlier this year in the months after ChatGPT launched.
New newsletters subscribers is a better representation:
You can understand why when buyers started diving into things, they were understandably hesitant to fork over $10k+ for the site.
Questionable product/market fit
Don’t get me wrong, some people really liked it.
There are about 700 people who opt to receive the email daily, and another 200 or so that opt to read it weekly, for a total of about 900 active subscribers. The emails have about a 75% open rate and a 30% click-through rate, which I’m told are stellar.
Here’s what the email looks like:
And when I announced I was selling it and potentially shutting it down, I got a number of emails like this:
Thank you very much for your work on Emergent Mind. As an AI hobbyist with a IT background, I am looking forward to reading your email every morning.
But, my impression is that it is a nice to have for most people. There was a bug a few months ago that took the newsletter down for a few days and no one wrote in about the email not going out. Almost no one sent in suggestions for how to improve it. Very few people shared it with others. Very few people have mentioned it on social media during its time running.
Competition is fierce
And there are no shortages of AI news resources, many of which are run by highly competent individuals or teams. Matt Wolfe and Ben Tossell come to mind, but there are countless others. Their work is incredibly well done, and Emergent Mind with its short news summaries pales in comparison (though of course there are many ways to consume news, and Emergent Mind may be a better fit for some types of people).
Poor founder/product fit
I like building tools. Preceden, my other product, is a tool, and pretty much everything else I’ve built and stuck with is a tool of some kind.
Emergent Mind is a… news site. I really enjoyed automating the news aggregation and summaries, as well as building the daily newsletter. Pretty sure it was the first automated newsletter entirely written by GPT.
But I don’t try to stay on top of the news much. Not national news, not AI news. I check the Emergent Mind newsletter periodically, but am just not that passionate about reading or being a source of AI news.
But lets say I wanted to stick with it. Now what? Is this website and newsletter what Emergent Mind is and now I have to market it and grow it into a real business? That sounds like a lot of (not enjoyable) work to me. I could partner with someone who enjoys that side of things, but even that would require a lot of work. Which brings me to the biggest reason to shut it down…
I should focus my time and attention on my other product
My other product, Preceden, is doing well. Not sail-into-the-sunset well, but well enough that I’m able to work on it full time and it can support my family financially with some buffer. And I’ve got a long, long list of projects that I’m really excited about to improve the product, market it better, and grow the business. Every minute I spent thinking about or working on Emergent Mind is one less minute I’m spending on my actual business.
Why split my time at all with another product, especially one I’m not that excited about taking to the next level?
I’ve told everyone I’m selling it/shutting it down
I’ve publicly gone back and forth on Emergent Mind’s future several times before, and the thought of writing this blog post where I justify doing it again was not very appealing to me.
But, avoiding a few eye-rolls should not be a factor in my decision-making process.
Why keep it
For most of the last few months, the “sell it or shut it down” route made the most sense to me. But, after a lot of reflection, I realized I was looking at it the wrong way.
The curse of inflated expectations
I made big mistake early on in the course of building Emergent Mind: I expected too much of it, and myself really. Rather than be some fun little side project (which I’ve had a ton of over the years), I saw it as an ambitious startup that had the potential to see meteoric growth and become the top AI news source. I told everyone I knew how big it would be, formed a Delaware LLC (to make converting it into a Delaware C Corp easier if I choose to raise money), spent $760 chatting with an IP lawyer for an hour to avoid getting myself in trouble with the direction I was going, and even thought about putting Preceden on the back-burner to focus 100% on Emergent Mind.
But then reality set in, and I realized how much work would be required to achieve those goals. There’s only so much time in the day, and faced with the prospect of trying to build/market/run two products by myself or focus on just one, I convinced myself to end Emergent Mind because I wouldn’t be able to achieve those initial ambitions.
It’s a false dichotomy though. It took me a long time to realize, but I don’t have to choose between shutting it down and devoting 50% of my time to growing it into an AI news juggernaut. I could work on it one day a month. Hell, I don’t actually have to work on it at all. I could make some tweaks to get the costs down and put it on the back-burner for 2 months or 2 years. It can be a small site with a small audience and that is completely okay.
And while it’s true that I’m super focused on Preceden right now and probably will be for the foreseeable future, my past indicates there’s a strong chance I may want to take a break and work on something else in the future. I will hopefully be in this full-time indie-hacker mode for years to come, and it’s unlikely Preceden will be the only thing I work on for the rest of my career.
I also have a ton of interesting ideas and potential directions for Emergent Mind that I may want to devote some time to in the future, not because I think they will change its growth (though maybe), but because they seem fun to work on. For example, how could I change the newsletter to make it appealing and informative to someone like myself who is only casually following the latest AI developments? How can I surface big picture trends, and not just list summaries of yesterday’s news items? I have some ideas.
And that’s what Emergent Mind should have been all along: a small side project that I can work on when I want to work on it, with no pressure to grow it into something larger.
And lets not forget: there are those 900 active subscribers with a 75% daily open rate. There are a lot of people getting value from the site, and while it may never be a top-3 AI news source with hundreds of thousands of readers, I should be (am am!) thrilled that it has as many readers as it does, and its a privilege to be serving them.
Also, I can likely get the costs down to something like $150/month, which is really not bad. If it was costing me $500/month, there would be a much stronger argument for pulling the plug, but for $150/month, it makes sense to keep it going and keep my options open.
What’s next
I’m going to put Emergent Mind on the back-burner for the foreseeable future. When I’m feeling inspired to work on it, I will, but am not going to put any pressure on myself to grow it by taking on large product and marketing initiatives that I’m not excited about.
Maybe it sits on that back-burner for months or years and continues its gradual decline. Maybe I get a meaningful offer one day and do sell it. Maybe in a year I announce I’m shutting it down and get a few more eye-rolls. Or maybe with some small product, marketing, and monetization projects I’m able to turn things around, and gradually want to spend more and more of my time on it.
Last week, the Twitter account Everything Out of Context posted a screenshot showing Google’s incorrect response to the search query “country in africa that starts with k”:
The tweet went viral, garnering over 133k likes, 6k retweets, and 1k replies.
Someone eventually tagged me in the post because it turns out Google’s Quick Answer is based on a ChatGPT hallucination example that was posted on Emergent Mind (then called LearnGPT) earlier this year to help people understand that ChatGPT responses should not be blindly trusted:
Even though I pivoted the site away from ChatGPT examples, those examples are still around, and apparently being shown in Quick Answers on Google when Google deems it relevant.
Someone also joined Emergent Mind’s Discord suggesting I take the page down:
[Update] On Oct 26, Christopher Ingraham, a journalist with a large following, shared it as well, resulting in a lot of attention. Here’s a newer blog post about the fallout from his tweet.
I’m not going to take down the page down because it seems like Google’s responsibility to figure out how to identify and exclude false information from Quick Answers. Also, as of this writing, I’m only seeing it as a Quick Answer some percentage of the time. I hope this obviously incorrect example raises awareness about inaccuracies in Google Search result snippets, and contributes to Google addressing them in time.
It is funny though: earlier this year when I was working heavily on Emergent Mind, I tried pretty hard to write a viral tweet about it. I never succeeded, but here we are now, months later, finally with it going viral, just not for a reason I imagined 🤣.
Matt Mazur 