Friday Updates: Preceden Design Hire, GPT-3 Response Generation, Housing Prices Progress, Star Trek Discovery, and the Almanack of Naval

Photo courtesy of Josh Berquist

What I’m working on at Preceden

I’ve got this thing where I’m never quite satisfied with Preceden‘s design. It’s not that bad, but when I compare it to a site designed by professional designers like Help Scout or Stripe, it’s clear that there’s a large gap, and it’s not one that I’m talented enough to close on my own.

And so this week I posted a job on Upwork to try to find a designer to help improve Preceden’s design:

I was optimistic that this would receive a lot of applications, but it didn’t work out that way. It received maybe 10 applications, out of which I’d say 9 hadn’t tailored their application even one bit to the job description which is an immediate red flag for me (even just a short “Hey, working on Preceden sounds interesting, here’s a bit about me…” goes a long way). The 1 that did wasn’t a good fit. In the end I did make an offer to 2 people (so I could work with them for a short project then pick one to work with longer term), but one declined saying he was too busy and another wanted a min $3k initial commitment which I wasn’t comfortable with.

At Automattic and now Help Scout, there are some incredibly talented designers who can create mockups and then take that design and implement it in HTML/CSS within the codebase. But this hiring experience reminded me that those people are rare and in high demand and probably not looking for short projects on Upwork.

It’s more common for different people to tackle these tasks: one to create the design, another to implement it. For round 2 of hiring for this, I’m going to try to split it: find and hire one person (probably on Dribbble) to create the mockups, and hire a separate front-end developer to implement that design in Preceden.

What I’m working on at Help Scout

I shipped a fun little experiment that lets people at Help Scout use GPT-3 to automatically draft a response to customer conversations within the Help Scout editor:

This uses Tampermonkey to add the GPT-3 button using custom JavaScript that’s run on conversation pages, which when clicked takes the customer’s name and last message, posts it to an API endpoint which runs it through GPT-3 and returns the completed text, which is then simulated as being typed out in the editor to look cool.

To be 100% clear, we have no intention of actually building this into the real product. It’s meant to demonstrate GPT-3’s strengths in weaknesses. If I had to boil them down, I’d say that GPT-3 is great at generating text that looks like it was written by a human, but at least in this support context, the responses are terrible because GPT-3 has no knowledge of the product its trying to answer a question about. Maybe down the road there will be some future version of this technology where it can be trained on your support history or docs, but until that happens, GPT-3 probably won’t be augmenting customer service reps in this way anytime soon.

What I’m studying

Still chugging away at Kaggle’s old housing prices prediction competition, managing to break into the top 500 out of 4.6k:

My approach to this is to make as much progress as I can on my own, then peak at others’ publicly shared notebooks to get ideas for how to improve, then iterate on my own notebook accordingly. I’ve been learning a lot this way and probably stick with this competition for a few more weeks until I feel like I’ve maxed out how much I can learn from it.

What I’m watching

Star Trek Discovery:

Unlike most of the older Star Trek series which basically consisted of the ships exploring and getting into different adventures each episode, Discovery (and Picard) have an single story arc that’s told across each episode. It’s really good.

What I’m reading

The Almanack of Naval Ravikant – a summary of his podcast interviews and writings. I’m a huge fan of Naval. If you’re new to him, check out his Farnam Street interview which I’ve listened to several times.

That’s it for now πŸ‘‹

Friday Updates: More GPT-3 Experiments, ML Courses Completed, Kaggle, Keane, and Picard

What I’m working on at Help Scout

I’m still experimenting with applications of GPT-3, specifically around reducing the amount of time it takes for a user to respond to a support email. While GPT-3 is fascinating, its complete lack of knowledge about Help Scout means it’s probably not going to be that useful for this purpose. We’ll see though.

Building a proof of concept for this has had me diving into Tampermonkey, a Chrome extension that lets you write custom JavaScript files that can be automatically run on certain webpages. For example, automatically adding a download button to all YouTube pages. These custom scripts (“user scripts”) can be shared too. This will enable me to quickly prototype ML tools for Help Scout’s support team to try within the product without going through engineering which would take a lot more time.

What I’m studying

Finally wrapped up DataCamp’s machine learning track!

I wound up exceeding my goal of 1 course/week towards the end because I really wanted to finish and start applying what I’ve learned.

The next day I headed to Kaggle and started on their old House Prices competition. The goal is to take a bunch (80!) data points about a house that sold and try to create a machine learning model that accurately predicts its sale price. There have been about 4.8k teams that have competed and my goal is to keep working on it until I hit top 5% or about top ~240. After about 6 hours of working on it, I broke through top 2000, so making progress but still have a ways to go. I’m learning a lot though and arguably should have been doing more of this sooner vs spending most of my time doing courses.

What I’m listening to

Love Too Much by Keane:

Every now and then I hear a song and wind up listening to it over and over again for weeks. This is one of those. The whole album, Cause and Effect, is excellent.

What I’m watching

Just wrapped up watching Picard on CBS All Access:

It’s probably my favorite Star Trek series after The Next Generation.

Between this and Raised by Wolves though, I’ve watched enough android television series to last me a while.

Hope everyone is doing well πŸ‘‹.

Friday Updates: Shenandoah Vacation, Experimenting with GPT-3, Timeglider Closure Updates, ML Progress & Raised by Wolves

Last week my wife and I headed out to Luray, Virginia to relax for a few days without our kids. We stayed in a cabin at the Shadown Mountain Escape which is located right outside of Shenandoah National Park. We hiked the Appalachian Trail each day, read a lot, watched a lot of movies, and generally just tried to relax and unwind. It was lovely. If you ever wind up going, tell Karen and Ralph that I said hi.

What I’m working on at Help Scout

I finally got access to GPT-3 a few days ago and have been playing around with it and brainstorming applications at Help Scout and beyond. Specifically I wanted to see what kind of responses it would generate to support emails customers had sent us.

Here’s a real conversation I received today in the Preceden inbox:

Hello,

I upgraded my Preceden account this month, but it turns out its not a tool I will be using. My project management system can create a grantt chart and that is sufficient for my company’s needs. I would like to request a refund.

Thank you, Edna

Plugging that into the script I set up for this (which I seeded with some examples of real responses in Help Scout):

Hello Edna,

I’m sorry to hear that you won’t be using Preceden. I’ve refunded the account for the remaining days of the month.

If you have any other questions, please don’t hesitate to contact us.

Not bad right?

Thing is, it doesn’t have any idea what it’s talking about. For example, Preceden charges yearly and I give full refunds (not prorated) so the response about the partial refund for the month isn’t accurate.

Still, pretty interesting.

What I’m working on at Preceden

A worthwhile investment I made a few years back was creating this list of timeline maker tools. It is IMHO the most comprehensive list out there and over time it has come to rank well for search terms like “timeline software”. Because Preceden is at the top of the list of course so it winds up driving a decent amount of leads and sales.

This week I added two new tools, Kronoli and ChronoFlo, bringing the total to 35 tools.

What I’m working on at Timeglider

I recently decided to shut down Timeglider, a competitor timeline maker tool that I acquired last year.

After I cancelled the active subscriptions, I emailed those people as well as anyone who had used it in the last year to inform them about the closure and how to migrate their timelines to Preceden if they wanted.

But what about everyone else who had used it prior to that? All time, Timeglider had about 500k sign ups. Emailing everyone though seems unnecessary because many of those never did anything in the app. Limiting it to those that had created at least 11 events got the number down to 100k.

I imported their detailed (email, sign up date, timeline names) into a database in Preceden and then set up a rake task to email a bunch each day to inform them about the closure (I chose a rake task because signing up and trying to use like Mailchimp to email 100k people with a new account seemed like a bad idea). Here’s an example email:

Hi Elijah,

You’re receiving this email because you signed up for Timeglider in 2018 and still have 2 timelines on the site.

My name is Matt and I purchased Timeglider from its prior owner last summer. There aren’t a lot of people using it these days so I’ve made the tough decision to shut Timeglider down at the end of November. This will allow me to focus my efforts on Preceden, my primary timeline maker tool.

Here’s a list of your timelines:

* French Revolution timeline

You have a few options:

* If you don’t care about your timelines anymore, there’s nothing you need to do

* You can download a CSV of your data so it’s not lost when the site shuts down

* You can download a CSV and import it into Preceden to continue working on your timeline

* You can use the Timeglider jQuery widget to host your timeline on another site

For instructions on all of these options and answers to common questions, check out our Timeglider Closure FAQs.

Sorry for any hassle this causes.

Please don’t hesitate to respond if you have any questions.

Matt Mazur

info@timeglider.com

I started small and ramped up the daily emails as I gained confidence that the message was solid and that Timeglider didn’t have any issues that might cause problems (it had a few).

The last of the emails went out today:

Preceden uses SendGrid to its emails, including these going out to the Timeglider users. Over the course of sending out these emails my reputation score dropped from 99% to 79% due to a fair amount of bounced emails (since many of these users signed up 10+ years ago) and a few spam reports.

Scores between 70% and 80% are categorized as “This is considered a poor reputation and you should consider taking action to identify and fix problems with your sending practices.” Fortunately since its through the 100k emails this should start to rise back up to shortly.

The responses were overwhelmingly positive with lots of people thanking me for telling them and wishing me luck on Preceden.

I haven’t set up reporting for this yet, but it did seem to drive a lot of Preceden sales too as these people moved their timelines over to Preceden.

What I’m studying

I finished DataCamp’s Introduction to Deep Learning in Python, Introduction to Deep Learning with Keras, and Advanced Deep Learning with Keras courses. The last one there was probably the most difficult course I’ve done yet. Four courses to go.

Also started reading Hands-On Machine Learning with Scikit-Learn and TensorFlow which is fantastic. For anyone interested in machine learning who doesn’t want to go through an online course, this book is a great place to start.

What I’m listening to

Chai Time Data Science, a podcast where the host, Sanyam Bhutani, interviews folks in the data science community with a focus on those who compete in Kaggle competitions. It’s got me pumped about diving into Kaggle in the coming year to level up my ML skills.

What I’m watching

Speaking of AI, I’ve been watching Raised by Wolves on HBO. If you like dystopian sci-fi movies, you’ll enjoy this show.

You can watch the entire first episode for free on YouTube:

That’s it for now my friends, thanks for reading πŸ‘‹

Friday Updates: Data Retention at Preceden, Closing Down Timeglider, Organizing Looker, the Psychology of Money, and the 3m Mario Speed Run

What I’m Working On

At Preceden, I’ve been thinking a lot about data retention. For example, it has users that signed up a decade ago, used it briefly, and then never again. Unless they deleted their account, their user and timeline data still exist in Preceden’s database. In the past, I never gave any thought to whether it made sense to keep this data around forever. Maybe the user eventually returns one day, so why not keep it around? Analyzing all of that data has also been tremendously valuable. And for publicly shared timelines, there are SEO and user acquisition advantages to keeping those timelines around. But imagine a person that signed up in 2012 to create a private timeline about a messy divorce and hasn’t used it since then. Should Preceden retain that data forever? Probably not.

Addressing this is way easier said than done though. After I tweeted about this dilemma, Emilie Schario pointed me to an insightful Basecamp podcast episode about how they built something called the Data Incinerator to deal with this at their company. It’s worth a listen for anyone thinking about data retention. I’m probably going to build something like this to improve data retention in Preceden in the coming months.

Last summer I acquired Timeglider, a competitor timeline maker tool. I recouped the acquisition cost in two ways: I kept Timeglider running for its existing customers (which have continued to bring in revenue) and also redirected Timeglider’s homepage to Preceden (which brought in a lot of new customers for Preceden).

Because Timeglider hasn’t had any new sign ups in the last 14 months, its recurring revenue has gradually dwindled as its customers have churned. So the question I’ve been debating for a while now is how long to keep Timeglider running. On one hand, maintenance and costs are minimal, so it’s basically a small stream of completely passive income that would probably continue for years. On the other hand, the remaining recurring revenue wasn’t significant, most of the customers were inactive, its codebase and infrastructure are very brittle, and operating it takes up headspace that I’ve love to free up. After weighing it all, I decided earlier this week to announce it was shutting down at the end of November.

Executing on this plan took more work than I anticipated though.

  • Many Timeglider customers paid annually. If someone paid $50 for access to Timeglider through August 2021, but I shut down the service in December 2020, it would be unethical to keep their full payment. I wound up issuing a lot of prorated refunds for recent annual payments.
  • Stripe doesn’t provide a way in the UI to cancel multiple subscriptions at once. The quantity was more than I wanted to do manually, so had to write a little Ruby script to iterate over all of the subscriptions and cancel them.
  • I added a prominent notice at the top of Timeglider with a link to FAQs about the closure including instructions on how to move their Timeglider timelines to Preceden.
  • I had to notify all of the customers whose subscriptions I cancelled to make sure they knew the site was shutting down. Timeglider also has a free plan which had some active users, so I had to email them as well. Probably not the most efficient way to do this, but I BCC’d all of the impacted users 25 at a time in Help Scout (its BCC limit), using the Saved Reply functionality to compose each message.
  • Most of the responses from users were understanding and they appreciated that I was giving them notice and also provided a way to move their timelines over to Preceden.
  • There were some bugs I had to deal with though. By cancelling everyone’s subscriptions, it reverted everyone to Timeglider’s free plan which had feature limits including the inability to access more than 3 of their timelines. For users with more, this meant they couldn’t export their data. Had to hardcode in some logic to give everyone access to Timeglider’s top premium plan until the site shuts down.

Running that Ruby script to cancel all of the subscriptions was a little bit painful, but in the end I think shutting it down will be a good decision.

In my ongoing machine learning journey, I finished DataCamp’s Feature Engineering for NLP in Python course and started on its Introduction to TensorFlow in Python course. 8 courses including this one to completely finish the ML track.

At Help Scout, I started on a project to clean up our Looker instance. We’ve been using Looker for business intelligence reporting for more than 3 years and over that time we’ve accumulated hundreds of dashboards and more than a thousand individual Looks. We mostly try to organize these by function: a folder for marketing reporting, a folder for sales reporting, a folder for finance reporting, etc. But over time it’s gotten very messy. Lots of unused dashboards and Looks. Lots of duplicate reporting. Even some completely empty folders. For any employee venturing into Looker to try to understand our metrics, figuring out where to go was a challenge to say the least. I’ve been cleaning it up and with just a few hours of work so far it’s way, way better than it was. I regret not spending more time on it earlier.

What I’m Reading

The Psychology of Money by Morgan Housel. Really enjoyed this book. Lots of sage advice on how to think about personal finances, uncertainty, risk, etc. Morgan is worth following on Twitter too. And for you podcast fans, here’s an interview with him from earlier this month (thanks Jason for the recommendation).

What I’m Watching

This deep dive explanation of a 3 minute Super Mario Bros 3 speed run is mind blowing:

I thought it was just going to be someone playing through the game perfectly, but it’s so much more than that. The record is made possible by painstaking work to reverse engineering the game to figure out and exploit a very subtle glitch. Here’s the HackerNews discussion about it.

What Else

I wrapped up the 28-Day Keto Challenge. Writeup here. After being back on my normal carb-heavy diet now for 5 days and feeling like a glutton, I’m giving serious thought to going back on keto. We’ll see.

Hope everyone is doing well. Thanks for reading.