Friday Updates: More GPT-3 Experiments, ML Courses Completed, Kaggle, Keane, and Picard

What I’m working on at Help Scout

I’m still experimenting with applications of GPT-3, specifically around reducing the amount of time it takes for a user to respond to a support email. While GPT-3 is fascinating, its complete lack of knowledge about Help Scout means it’s probably not going to be that useful for this purpose. We’ll see though.

Building a proof of concept for this has had me diving into Tampermonkey, a Chrome extension that lets you write custom JavaScript files that can be automatically run on certain webpages. For example, automatically adding a download button to all YouTube pages. These custom scripts (“user scripts”) can be shared too. This will enable me to quickly prototype ML tools for Help Scout’s support team to try within the product without going through engineering which would take a lot more time.

What I’m studying

Finally wrapped up DataCamp’s machine learning track!

I wound up exceeding my goal of 1 course/week towards the end because I really wanted to finish and start applying what I’ve learned.

The next day I headed to Kaggle and started on their old House Prices competition. The goal is to take a bunch (80!) data points about a house that sold and try to create a machine learning model that accurately predicts its sale price. There have been about 4.8k teams that have competed and my goal is to keep working on it until I hit top 5% or about top ~240. After about 6 hours of working on it, I broke through top 2000, so making progress but still have a ways to go. I’m learning a lot though and arguably should have been doing more of this sooner vs spending most of my time doing courses.

What I’m listening to

Love Too Much by Keane:

Every now and then I hear a song and wind up listening to it over and over again for weeks. This is one of those. The whole album, Cause and Effect, is excellent.

What I’m watching

Just wrapped up watching Picard on CBS All Access:

It’s probably my favorite Star Trek series after The Next Generation.

Between this and Raised by Wolves though, I’ve watched enough android television series to last me a while.

Hope everyone is doing well πŸ‘‹.

Friday Updates: Shenandoah Vacation, Experimenting with GPT-3, Timeglider Closure Updates, ML Progress & Raised by Wolves

Last week my wife and I headed out to Luray, Virginia to relax for a few days without our kids. We stayed in a cabin at the Shadown Mountain Escape which is located right outside of Shenandoah National Park. We hiked the Appalachian Trail each day, read a lot, watched a lot of movies, and generally just tried to relax and unwind. It was lovely. If you ever wind up going, tell Karen and Ralph that I said hi.

What I’m working on at Help Scout

I finally got access to GPT-3 a few days ago and have been playing around with it and brainstorming applications at Help Scout and beyond. Specifically I wanted to see what kind of responses it would generate to support emails customers had sent us.

Here’s a real conversation I received today in the Preceden inbox:

Hello,

I upgraded my Preceden account this month, but it turns out its not a tool I will be using. My project management system can create a grantt chart and that is sufficient for my company’s needs. I would like to request a refund.

Thank you, Edna

Plugging that into the script I set up for this (which I seeded with some examples of real responses in Help Scout):

Hello Edna,

I’m sorry to hear that you won’t be using Preceden. I’ve refunded the account for the remaining days of the month.

If you have any other questions, please don’t hesitate to contact us.

Not bad right?

Thing is, it doesn’t have any idea what it’s talking about. For example, Preceden charges yearly and I give full refunds (not prorated) so the response about the partial refund for the month isn’t accurate.

Still, pretty interesting.

What I’m working on at Preceden

A worthwhile investment I made a few years back was creating this list of timeline maker tools. It is IMHO the most comprehensive list out there and over time it has come to rank well for search terms like “timeline software”. Because Preceden is at the top of the list of course so it winds up driving a decent amount of leads and sales.

This week I added two new tools, Kronoli and ChronoFlo, bringing the total to 35 tools.

What I’m working on at Timeglider

I recently decided to shut down Timeglider, a competitor timeline maker tool that I acquired last year.

After I cancelled the active subscriptions, I emailed those people as well as anyone who had used it in the last year to inform them about the closure and how to migrate their timelines to Preceden if they wanted.

But what about everyone else who had used it prior to that? All time, Timeglider had about 500k sign ups. Emailing everyone though seems unnecessary because many of those never did anything in the app. Limiting it to those that had created at least 11 events got the number down to 100k.

I imported their detailed (email, sign up date, timeline names) into a database in Preceden and then set up a rake task to email a bunch each day to inform them about the closure (I chose a rake task because signing up and trying to use like Mailchimp to email 100k people with a new account seemed like a bad idea). Here’s an example email:

Hi Elijah,

You’re receiving this email because you signed up for Timeglider in 2018 and still have 2 timelines on the site.

My name is Matt and I purchased Timeglider from its prior owner last summer. There aren’t a lot of people using it these days so I’ve made the tough decision to shut Timeglider down at the end of November. This will allow me to focus my efforts on Preceden, my primary timeline maker tool.

Here’s a list of your timelines:

* French Revolution timeline

You have a few options:

* If you don’t care about your timelines anymore, there’s nothing you need to do

* You can download a CSV of your data so it’s not lost when the site shuts down

* You can download a CSV and import it into Preceden to continue working on your timeline

* You can use the Timeglider jQuery widget to host your timeline on another site

For instructions on all of these options and answers to common questions, check out our Timeglider Closure FAQs.

Sorry for any hassle this causes.

Please don’t hesitate to respond if you have any questions.

Matt Mazur

info@timeglider.com

I started small and ramped up the daily emails as I gained confidence that the message was solid and that Timeglider didn’t have any issues that might cause problems (it had a few).

The last of the emails went out today:

Preceden uses SendGrid to its emails, including these going out to the Timeglider users. Over the course of sending out these emails my reputation score dropped from 99% to 79% due to a fair amount of bounced emails (since many of these users signed up 10+ years ago) and a few spam reports.

Scores between 70% and 80% are categorized as “This is considered a poor reputation and you should consider taking action to identify and fix problems with your sending practices.” Fortunately since its through the 100k emails this should start to rise back up to shortly.

The responses were overwhelmingly positive with lots of people thanking me for telling them and wishing me luck on Preceden.

I haven’t set up reporting for this yet, but it did seem to drive a lot of Preceden sales too as these people moved their timelines over to Preceden.

What I’m studying

I finished DataCamp’s Introduction to Deep Learning in Python, Introduction to Deep Learning with Keras, and Advanced Deep Learning with Keras courses. The last one there was probably the most difficult course I’ve done yet. Four courses to go.

Also started reading Hands-On Machine Learning with Scikit-Learn and TensorFlow which is fantastic. For anyone interested in machine learning who doesn’t want to go through an online course, this book is a great place to start.

What I’m listening to

Chai Time Data Science, a podcast where the host, Sanyam Bhutani, interviews folks in the data science community with a focus on those who compete in Kaggle competitions. It’s got me pumped about diving into Kaggle in the coming year to level up my ML skills.

What I’m watching

Speaking of AI, I’ve been watching Raised by Wolves on HBO. If you like dystopian sci-fi movies, you’ll enjoy this show.

You can watch the entire first episode for free on YouTube:

That’s it for now my friends, thanks for reading πŸ‘‹

Friday Updates: Data Retention at Preceden, Closing Down Timeglider, Organizing Looker, the Psychology of Money, and the 3m Mario Speed Run

What I’m Working On

At Preceden, I’ve been thinking a lot about data retention. For example, it has users that signed up a decade ago, used it briefly, and then never again. Unless they deleted their account, their user and timeline data still exist in Preceden’s database. In the past, I never gave any thought to whether it made sense to keep this data around forever. Maybe the user eventually returns one day, so why not keep it around? Analyzing all of that data has also been tremendously valuable. And for publicly shared timelines, there are SEO and user acquisition advantages to keeping those timelines around. But imagine a person that signed up in 2012 to create a private timeline about a messy divorce and hasn’t used it since then. Should Preceden retain that data forever? Probably not.

Addressing this is way easier said than done though. After I tweeted about this dilemma, Emilie Schario pointed me to an insightful Basecamp podcast episode about how they built something called the Data Incinerator to deal with this at their company. It’s worth a listen for anyone thinking about data retention. I’m probably going to build something like this to improve data retention in Preceden in the coming months.

Last summer I acquired Timeglider, a competitor timeline maker tool. I recouped the acquisition cost in two ways: I kept Timeglider running for its existing customers (which have continued to bring in revenue) and also redirected Timeglider’s homepage to Preceden (which brought in a lot of new customers for Preceden).

Because Timeglider hasn’t had any new sign ups in the last 14 months, its recurring revenue has gradually dwindled as its customers have churned. So the question I’ve been debating for a while now is how long to keep Timeglider running. On one hand, maintenance and costs are minimal, so it’s basically a small stream of completely passive income that would probably continue for years. On the other hand, the remaining recurring revenue wasn’t significant, most of the customers were inactive, its codebase and infrastructure are very brittle, and operating it takes up headspace that I’ve love to free up. After weighing it all, I decided earlier this week to announce it was shutting down at the end of November.

Executing on this plan took more work than I anticipated though.

  • Many Timeglider customers paid annually. If someone paid $50 for access to Timeglider through August 2021, but I shut down the service in December 2020, it would be unethical to keep their full payment. I wound up issuing a lot of prorated refunds for recent annual payments.
  • Stripe doesn’t provide a way in the UI to cancel multiple subscriptions at once. The quantity was more than I wanted to do manually, so had to write a little Ruby script to iterate over all of the subscriptions and cancel them.
  • I added a prominent notice at the top of Timeglider with a link to FAQs about the closure including instructions on how to move their Timeglider timelines to Preceden.
  • I had to notify all of the customers whose subscriptions I cancelled to make sure they knew the site was shutting down. Timeglider also has a free plan which had some active users, so I had to email them as well. Probably not the most efficient way to do this, but I BCC’d all of the impacted users 25 at a time in Help Scout (its BCC limit), using the Saved Reply functionality to compose each message.
  • Most of the responses from users were understanding and they appreciated that I was giving them notice and also provided a way to move their timelines over to Preceden.
  • There were some bugs I had to deal with though. By cancelling everyone’s subscriptions, it reverted everyone to Timeglider’s free plan which had feature limits including the inability to access more than 3 of their timelines. For users with more, this meant they couldn’t export their data. Had to hardcode in some logic to give everyone access to Timeglider’s top premium plan until the site shuts down.

Running that Ruby script to cancel all of the subscriptions was a little bit painful, but in the end I think shutting it down will be a good decision.

In my ongoing machine learning journey, I finished DataCamp’s Feature Engineering for NLP in Python course and started on its Introduction to TensorFlow in Python course. 8 courses including this one to completely finish the ML track.

At Help Scout, I started on a project to clean up our Looker instance. We’ve been using Looker for business intelligence reporting for more than 3 years and over that time we’ve accumulated hundreds of dashboards and more than a thousand individual Looks. We mostly try to organize these by function: a folder for marketing reporting, a folder for sales reporting, a folder for finance reporting, etc. But over time it’s gotten very messy. Lots of unused dashboards and Looks. Lots of duplicate reporting. Even some completely empty folders. For any employee venturing into Looker to try to understand our metrics, figuring out where to go was a challenge to say the least. I’ve been cleaning it up and with just a few hours of work so far it’s way, way better than it was. I regret not spending more time on it earlier.

What I’m Reading

The Psychology of Money by Morgan Housel. Really enjoyed this book. Lots of sage advice on how to think about personal finances, uncertainty, risk, etc. Morgan is worth following on Twitter too. And for you podcast fans, here’s an interview with him from earlier this month (thanks Jason for the recommendation).

What I’m Watching

This deep dive explanation of a 3 minute Super Mario Bros 3 speed run is mind blowing:

I thought it was just going to be someone playing through the game perfectly, but it’s so much more than that. The record is made possible by painstaking work to reverse engineering the game to figure out and exploit a very subtle glitch. Here’s the HackerNews discussion about it.

What Else

I wrapped up the 28-Day Keto Challenge. Writeup here. After being back on my normal carb-heavy diet now for 5 days and feeling like a glutton, I’m giving serious thought to going back on keto. We’ll see.

Hope everyone is doing well. Thanks for reading.

28 Days of Keto with the Wearable Challenge

In my ongoing effort to make the most of quarantine, I just wrapped up a 28-day experiment trying a keto diet. This diet had a twist though: if I stayed below a certain blood glucose level every day, I stood to make $700. Kind of.

Wearable Challenge

Freestyle Libre is a Continuous Glucose Monitor (CGM) which means its a sensor you wear to continually monitor your blood glucose levels.

The Freestyle Libre CGM sensor. It’s painless to apply and you barely notice it.

It’s intended for use by type I or type II diabetics, but this startup called Levels is repurposing it for non-diabetics to help them improve their diet and health.

And last but not least there’s the Wearable Challenge, a group that has partnered with Levels to help people adopt Levels to improve their health.

Here’s how the Wearable Challenge works:

  • You pay the Wearable Challenge team $850 after you’re accepted into the program.
  • They work with Levels to mail you two 14-day CGM sensors.
  • Every day for 28 days, you have to wear a sensor and scan it periodically with an app on your phone.
  • For every day that you stay below your max target blood sugar level (120 mg/dL) and meet certain other requirements (taking a picture of everything you eat, not having any gaps in your data) you get $25 back. If you do this every day for the entire challenge, you get a full $25/day * 28 days = $700 back. The remaining $150 pays for the sensor and you don’t get it back.

I’m not diabetic nor do I have a ton of weight to lose, but I’ve wanted to experiment with keto for some time and the Wearable Challenge seemed like an innovative way to go for it. I applied to the program, was accepted, and officially started the challenge on August 17th.

Keto

On a keto diet, you drastically reduce your carbs in your diet so your body switches over to using fat as its primary energy source.

I don’t enjoy cooking so headed to YouTube while preparing for keto to plan for what I’d eat throughout the challenge. I found this meal plan by Water Jug Fitness which I wound up following pretty closely throughout the challenge:

My meals look like this:

  • Breakfast: Omelette with two-eggs, mushrooms, spinach, and cheese with a side of cooked spinach, avocado, and two pieces of bacon.
  • Lunch: Taco salad with ground beef, onions, romaine lettuce, baby tomatoes, sour cream, avocado, and cheese.
  • Dinner: Some combination of a protein (fish, steak, chicken) with a side of vegetables.
  • Snacks:
    • Almond milk protein shake with MCT oil
    • Almonds
    • Pork rinds πŸ˜‹

I usually meal prepped for lunch. I’d prepare 4 days worth of taco salad ingredients (by cooking the ground beef, cutting up the lettuce, etc) and placing them in a meal prep storage container which I’d put in the fridge and grab for lunch each day. This was really helpful because it made it super easy to stick to keto when normally I’d be drawn towards some less healthy meal for lunch.

Taco salad for lunch every day

While it may not work for everyone, I was mostly fine with having the same meals for breakfast and lunch every day with a little variety for dinner.

Scanning the sensor

The Freestyle Libre sensor can only hold 8 hours of blood sugar readings, which means that you have to scan it with your phone at least that often otherwise you’ll have gaps in the data (the Wearable Challenge team did make exception for gaps overnight though since depending on how long ou slept you might have some gaps in the data right after you headed to bed).

Every few hours I would load the Freestyle Libre app on my phone and hold it up to the sensor on my arm. It would scan the sensor and show me my blood sugar level along with a chart showing the trends that day:

For most of the challenge I stayed well below 100 mg/dL, but toward the end I started incorporating more carbs which had my numbers jumping to 110 mg/dL here and there.

The Freestyle Libre app is set up to pass data along to Levels, which also has an app:

It’s the Levels app that you use to take photos of what you’re eating, one of the requirements to get the $25 for that day.

The Wearable Challenge has access to the data in Levels so they can monitor how you’re doing to determine whether to pay you $25 for that day or not. Even though the Wearble Challenge is fairly new (I was part of the third cohort) I’m assuming most of the day to day management of it are automated at this point.

Keto Impact

The first week of switching to keto you go through something known as the keto flu which is a result of your body adapting to the new diet. I had many of the symptoms: dizziness, keto breath, sugar cravings, and headaches. After the first week though most of it diminished significantly.

One thing that really blew my mind was how little I thought about food after the first week. I suspect that’s part of keto in general, but also because I was eating the same meals almost every day. Food just got very… boring. Cravings and hunger almost completely disappeared.

I didn’t exercise at all during the first week, but after my body adapted to keto I picked it up again, doing 20-30m YouTube HIIT workouts a few times per week. I expected going into this that working out would be tough due to to low energy, but wound up not having any problems with it.

I actually had a lot of energy despite only consuming 40-50g of carbs per day. Your body really does adapt.

Most of the last ten years I’ve been around 185 pounds, varying a few pounds in either direction. At the start of the challenge I was on the low end at 183.2 pounds. After about 5 days, I was down to 179 pounds, and then slowly got down to 178 by the end, the lowest I’ve been since college.

What’s Next

In the end, I managed meet the requirements each day and will be getting the full $700 back:

But I’m not going to stay on keto. I enjoy too many non-keto foods and am healthy enough where it’s not a huge deal to have some excess sugar in my diet right now. That said, I hope to find some balance between my previous carb-heavy diet and keto.

And who knows, if quarantine goes on long enough, maybe I wind up going back on full keto and trying to get down another few pounds. We will see.

Kudos

This little adventure wouldn’t have been possible without the Freestyle Libre, Levels, or the team organizing the Wearable Challenge. Huge thanks to everyone who helped make it happen. Also, shout out to my wife for putting up with my keto breath for most of the last month πŸ₯°.

If the Wearable Challenge sounds intriguring to you, I’d encourage you to check out the website and consider applying.