I’m still experimenting with applications of GPT-3, specifically around reducing the amount of time it takes for a user to respond to a support email. While GPT-3 is fascinating, its complete lack of knowledge about Help Scout means it’s probably not going to be that useful for this purpose. We’ll see though.
Building a proof of concept for this has had me diving into Tampermonkey, a Chrome extension that lets you write custom JavaScript files that can be automatically run on certain webpages. For example, automatically adding a download button to all YouTube pages. These custom scripts (“user scripts”) can be shared too. This will enable me to quickly prototype ML tools for Help Scout’s support team to try within the product without going through engineering which would take a lot more time.
What I’m studying
Finally wrapped up DataCamp’s machine learning track!
Back in April I set a goal for myself to complete @DataCamp's Machine Learning for Everyone career track. After 6 months of working through the courses, I just finished 🎉 pic.twitter.com/JfPKw1iT8Z
I wound up exceeding my goal of 1 course/week towards the end because I really wanted to finish and start applying what I’ve learned.
The next day I headed to Kaggle and started on their old House Prices competition. The goal is to take a bunch (80!) data points about a house that sold and try to create a machine learning model that accurately predicts its sale price. There have been about 4.8k teams that have competed and my goal is to keep working on it until I hit top 5% or about top ~240. After about 6 hours of working on it, I broke through top 2000, so making progress but still have a ways to go. I’m learning a lot though and arguably should have been doing more of this sooner vs spending most of my time doing courses.
What I’m listening to
Love Too Much by Keane:
Every now and then I hear a song and wind up listening to it over and over again for weeks. This is one of those. The whole album, Cause and Effect, is excellent.
What I’m watching
Just wrapped up watching Picard on CBS All Access:
It’s probably my favorite Star Trek series after The Next Generation.
Between this and Raised by Wolves though, I’ve watched enough android television series to last me a while.
Last week my wife and I headed out to Luray, Virginia to relax for a few days without our kids. We stayed in a cabin at the Shadown Mountain Escape which is located right outside of Shenandoah National Park. We hiked the Appalachian Trail each day, read a lot, watched a lot of movies, and generally just tried to relax and unwind. It was lovely. If you ever wind up going, tell Karen and Ralph that I said hi.
What I’m working on at Help Scout
I finally got access to GPT-3 a few days ago and have been playing around with it and brainstorming applications at Help Scout and beyond. Specifically I wanted to see what kind of responses it would generate to support emails customers had sent us.
Here’s a real conversation I received today in the Preceden inbox:
Hello,
I upgraded my Preceden account this month, but it turns out its not a tool I will be using. My project management system can create a grantt chart and that is sufficient for my company’s needs. I would like to request a refund.
Thank you, Edna
Plugging that into the script I set up for this (which I seeded with some examples of real responses in Help Scout):
Hello Edna,
I’m sorry to hear that you won’t be using Preceden. I’ve refunded the account for the remaining days of the month.
If you have any other questions, please don’t hesitate to contact us.
Not bad right?
Thing is, it doesn’t have any idea what it’s talking about. For example, Preceden charges yearly and I give full refunds (not prorated) so the response about the partial refund for the month isn’t accurate.
Still, pretty interesting.
What I’m working on at Preceden
A worthwhile investment I made a few years back was creating this list of timeline maker tools. It is IMHO the most comprehensive list out there and over time it has come to rank well for search terms like “timeline software”. Because Preceden is at the top of the list of course so it winds up driving a decent amount of leads and sales.
This week I added two new tools, Kronoli and ChronoFlo, bringing the total to 35 tools.
What I’m working on at Timeglider
I recently decided to shut down Timeglider, a competitor timeline maker tool that I acquired last year.
After I cancelled the active subscriptions, I emailed those people as well as anyone who had used it in the last year to inform them about the closure and how to migrate their timelines to Preceden if they wanted.
But what about everyone else who had used it prior to that? All time, Timeglider had about 500k sign ups. Emailing everyone though seems unnecessary because many of those never did anything in the app. Limiting it to those that had created at least 11 events got the number down to 100k.
I imported their detailed (email, sign up date, timeline names) into a database in Preceden and then set up a rake task to email a bunch each day to inform them about the closure (I chose a rake task because signing up and trying to use like Mailchimp to email 100k people with a new account seemed like a bad idea). Here’s an example email:
Hi Elijah,
You’re receiving this email because you signed up for Timeglider in 2018 and still have 2 timelines on the site.
My name is Matt and I purchased Timeglider from its prior owner last summer. There aren’t a lot of people using it these days so I’ve made the tough decision to shut Timeglider down at the end of November. This will allow me to focus my efforts on Preceden, my primary timeline maker tool.
Here’s a list of your timelines:
* French Revolution timeline
You have a few options:
* If you don’t care about your timelines anymore, there’s nothing you need to do
* You can download a CSV of your data so it’s not lost when the site shuts down
* You can download a CSV and import it into Preceden to continue working on your timeline
* You can use the Timeglider jQuery widget to host your timeline on another site
For instructions on all of these options and answers to common questions, check out our Timeglider Closure FAQs.
Sorry for any hassle this causes.
Please don’t hesitate to respond if you have any questions.
Matt Mazur
info@timeglider.com
I started small and ramped up the daily emails as I gained confidence that the message was solid and that Timeglider didn’t have any issues that might cause problems (it had a few).
The last of the emails went out today:
Preceden uses SendGrid to its emails, including these going out to the Timeglider users. Over the course of sending out these emails my reputation score dropped from 99% to 79% due to a fair amount of bounced emails (since many of these users signed up 10+ years ago) and a few spam reports.
Scores between 70% and 80% are categorized as “This is considered a poor reputation and you should consider taking action to identify and fix problems with your sending practices.” Fortunately since its through the 100k emails this should start to rise back up to shortly.
The responses were overwhelmingly positive with lots of people thanking me for telling them and wishing me luck on Preceden.
I haven’t set up reporting for this yet, but it did seem to drive a lot of Preceden sales too as these people moved their timelines over to Preceden.
Chai Time Data Science, a podcast where the host, Sanyam Bhutani, interviews folks in the data science community with a focus on those who compete in Kaggle competitions. It’s got me pumped about diving into Kaggle in the coming year to level up my ML skills.
What I’m watching
Speaking of AI, I’ve been watching Raised by Wolves on HBO. If you like dystopian sci-fi movies, you’ll enjoy this show.
You can watch the entire first episode for free on YouTube:
That’s it for now my friends, thanks for reading 👋
At Preceden, I’ve been thinking a lot about data retention. For example, it has users that signed up a decade ago, used it briefly, and then never again. Unless they deleted their account, their user and timeline data still exist in Preceden’s database. In the past, I never gave any thought to whether it made sense to keep this data around forever. Maybe the user eventually returns one day, so why not keep it around? Analyzing all of that data has also been tremendously valuable. And for publicly shared timelines, there are SEO and user acquisition advantages to keeping those timelines around. But imagine a person that signed up in 2012 to create a private timeline about a messy divorce and hasn’t used it since then. Should Preceden retain that data forever? Probably not.
Addressing this is way easier said than done though. After I tweeted about this dilemma, Emilie Schario pointed me to an insightful Basecamp podcast episode about how they built something called the Data Incinerator to deal with this at their company. It’s worth a listen for anyone thinking about data retention. I’m probably going to build something like this to improve data retention in Preceden in the coming months.
—
Last summer I acquiredTimeglider, a competitor timeline maker tool. I recouped the acquisition cost in two ways: I kept Timeglider running for its existing customers (which have continued to bring in revenue) and also redirected Timeglider’s homepage to Preceden (which brought in a lot of new customers for Preceden).
Because Timeglider hasn’t had any new sign ups in the last 14 months, its recurring revenue has gradually dwindled as its customers have churned. So the question I’ve been debating for a while now is how long to keep Timeglider running. On one hand, maintenance and costs are minimal, so it’s basically a small stream of completely passive income that would probably continue for years. On the other hand, the remaining recurring revenue wasn’t significant, most of the customers were inactive, its codebase and infrastructure are very brittle, and operating it takes up headspace that I’ve love to free up. After weighing it all, I decided earlier this week to announce it was shutting down at the end of November.
Executing on this plan took more work than I anticipated though.
Many Timeglider customers paid annually. If someone paid $50 for access to Timeglider through August 2021, but I shut down the service in December 2020, it would be unethical to keep their full payment. I wound up issuing a lot of prorated refunds for recent annual payments.
Stripe doesn’t provide a way in the UI to cancel multiple subscriptions at once. The quantity was more than I wanted to do manually, so had to write a little Ruby script to iterate over all of the subscriptions and cancel them.
I had to notify all of the customers whose subscriptions I cancelled to make sure they knew the site was shutting down. Timeglider also has a free plan which had some active users, so I had to email them as well. Probably not the most efficient way to do this, but I BCC’d all of the impacted users 25 at a time in Help Scout (its BCC limit), using the Saved Reply functionality to compose each message.
Most of the responses from users were understanding and they appreciated that I was giving them notice and also provided a way to move their timelines over to Preceden.
There were some bugs I had to deal with though. By cancelling everyone’s subscriptions, it reverted everyone to Timeglider’s free plan which had feature limits including the inability to access more than 3 of their timelines. For users with more, this meant they couldn’t export their data. Had to hardcode in some logic to give everyone access to Timeglider’s top premium plan until the site shuts down.
Running that Ruby script to cancel all of the subscriptions was a little bit painful, but in the end I think shutting it down will be a good decision.
At Help Scout, I started on a project to clean up our Looker instance. We’ve been using Looker for business intelligence reporting for more than 3 years and over that time we’ve accumulated hundreds of dashboards and more than a thousand individual Looks. We mostly try to organize these by function: a folder for marketing reporting, a folder for sales reporting, a folder for finance reporting, etc. But over time it’s gotten very messy. Lots of unused dashboards and Looks. Lots of duplicate reporting. Even some completely empty folders. For any employee venturing into Looker to try to understand our metrics, figuring out where to go was a challenge to say the least. I’ve been cleaning it up and with just a few hours of work so far it’s way, way better than it was. I regret not spending more time on it earlier.
What I’m Reading
The Psychology of Money by Morgan Housel. Really enjoyed this book. Lots of sage advice on how to think about personal finances, uncertainty, risk, etc. Morgan is worth following on Twitter too. And for you podcast fans, here’s an interview with him from earlier this month (thanks Jason for the recommendation).
What I’m Watching
This deep dive explanation of a 3 minute Super Mario Bros 3 speed run is mind blowing:
I thought it was just going to be someone playing through the game perfectly, but it’s so much more than that. The record is made possible by painstaking work to reverse engineering the game to figure out and exploit a very subtle glitch. Here’s the HackerNews discussion about it.
What Else
I wrapped up the 28-Day Keto Challenge. Writeup here. After being back on my normal carb-heavy diet now for 5 days and feeling like a glutton, I’m giving serious thought to going back on keto. We’ll see.
At Help Scout, a customer using Beacon wrote in asking about a warning that Chrome was displaying on their site:
The root cause seemed to be some attribution tracking cookies that were set by a script I worked on, so the support request made its way to me. Digging into it, we can see a lot of Help Scout cookies (including attribution tracking ones like _A_FirstTouchURL) are being set when Beacon is loaded on the customer’s site:
Despite having worked a lot with cookies and tracking scripts in the past, I was fuzzy on how these cookies were being set. Why, for example, were there Mixpanel and Google Analytics cookies when Beacon wasn’t loading them? Turns out that because the customer is loading Beacon from a helpscout.net domain, all of the helpscout.net cookies are made available on their site unless they’re explicitly set the cookie not to be passed along.
This can be achieved by setting the SameSite cookie attribute to Strict. In this case, there’s no need for most of these helpscout.net cookies to be passed along to third party sites loading Beacon, so we’re going through the steps to mark the cookies as strict when possible. Some cookies on helpscout.net are set by third party scripts and can’t be set to strict, so some amount of cookies getting passed along is inevitable, but at least we can minimize it.
This week I knocked out two DataCamp courses in my machine learning adventures: Hyperparameter Tuning in Python and Introduction to Natural Language Processing in Python. Typically I only am able to finish one course per week, but a lot of the material in these was covered by other courses so it made these courses fairly quick to work through.
One new thing was the introduction of TPOT, a tool that uses genetic programming to find an optimal classifier and hyperparameters for a given data set.
Fascinated with TPOT, a machine learning tool that helps you identify a classifier and hyperparameters that fit your data well using genetic programming 😲: https://t.co/Vc1P34txZtpic.twitter.com/0laU8Ktq2x
Long time readers may remember my Evolution of Color project in 2014. It uses genetic algorithms to evolve a population of colors towards a goal color. Unlike most of the other Emergent Mind projects that were I re-implemented other people’s projects in JavaScript, the Evolution of Color was completely original, making it one of the projects there I’m most proud of.
I hope as I continue down this machine learning path I get more opportunities to work with genetic algorithms.
And on that note, I’ve been mulling over where to focus in the coming months to keep leveling up my machine learning skills. Currently my plan is:
Finish the 11 remaining machine learning DataCamp courses by end of the year
Spend a few months in early 2021 focusing on Kaggle competitions to gain experience applying what I’ve learned
Figure out how to integrate machine learning features into a web application, whether it be for Preceden, a new app, or maybe even Help Scout if there’s an opportunity. Apparently deploying ML applications is quite difficult though. We will see.
What I’m watching
My wife and I just finished watching Umbrella Academy on Netflix:
It’s about a dysfunctional family of superheroes that need to work together to save the world. I enjoyed it and would recommend.
Product Recommendations
I’m in a mastermind group with Tom Davies and Jason Rudolph. Tom has two popular Shopify apps, Best Sellers and Flair, and Jason has BuildPulse, a tool that helps automatically identify flaky tests (automated tests that randomly pass sometimes and fail other times).
If you run a Shopify store or your team is banging their head against the wall dealing with flaky tests, their tools are definitely worth checking out.
What else
I’m on day 26 of a 28-day keto challenge. I’m participating through the Wearable Challenge initiative which basically means I’ve been wearing a Continuous Glucose Monitor (CGM) on my arm and for every day that I stay below a certain blood glucose limit I get $25 on my own money back. I’ve lost about 5 pounds and feel pretty healthy, but probably won’t continue keto after the challenge ends in a few days. I’ll write more about this whole thing in a separate post.
Matt Mazur 