Friday Updates: Boone, Putting ML on the Back Burner, Embulk, Mandalorian, and A Princess of Mars

Last week my family and I vacationed in a cabin near Boone, North Carolina for a few days. We hiked, visited Grandfather Mountain, made s’mores, enjoyed the hot tub, did gem mining, and generally just tried to unwind for a few days. It was nice.

What I’m working on at Preceden

A few months ago when I was deep into the DataCamp machine learning courses, I wasn’t sure whether after I finished I would keep focusing on machine learning or start spending more time on Preceden again.

It turns out that the break wound up invigorating me to work on Preceden again. For the past few weeks my mornings have been spent almost entirely on Preceden and not on machine learning at all. I don’t regret spending the time getting smart on machine learning – it’s a skillset I’m glad I now somewhat have – but right now I’m just more excited to work on Preceden, so that’s what I’m doing.

I have three big projects in various stages:

  1. Redesign: As I’ve written about in previous updates, I’m working with a designer and front-end developer to modernize the site. This involves migrating over to Tailwind and a lot of other changes to update the appearance of the site.
  2. Data Cleanup & Reactivation Project: there’s a lot of old data in Preceden that doesn’t need to stick around forever. Imagine someone creating a timeline about a divorce back in 2011. Unless the user deleted their timeline or account, it’s still in there. I’m going to build out a system for emailing long-inactive users to remind them it’s in there and say if they want to keep it they just need to log back in. If not, I’ll have Preceden purge their data after some period of time. This should let me both nix a lot of that old unneeded data and also bring back some users who haven’t thought about Preceden in a long time.
  3. Pricing: Preceden’s revenue is mostly from one-time payments. I’d love to figure out a way to get more recurring revenue. Much easier said than done though due to the nature of the tool (most people don’t need to use it very long). I’m playing around with spreadsheet models to figure out whether there’s a recurring revenue model that could work. We’ll see.

What I’m working on at Help Scout

We use a service called Fivetran to get data from various sources (like MySQL, Salesforce, HubSpot, and Mixpanel) over to our data warehouse (BigQuery) for analysis in Looker. Problem is, Fivetran’s pricing model changed this year and combined with heavier data volume it was going to cause our renewal cost to be much higher than in the past.

Most of our volume was from syncing MySQL data over to BigQuery. Our ops team was able to figure out a way to use an open source tool called Embulk to get our MySQL data over to BigQuery, eliminating our need to use Fivetran for it, greatly reducing our bill for 2021.

What I’m watching

Mandalorian season 2 on Disney+:

What I’m reading

A Princess of Mars by Edgar Rice Burroughs (same guy who wrote Tarzan). I read this when I was younger, but saw it sitting on my bookshelf the other day and started reading it again. It’s a fantastic series.

Stay sane my friends ๐Ÿ‘‹

Friday Updates: Election, Hiring a Tailwind Dev, NPS, and the Queen’s Gambit

Photo courtesy of Element5 Digital

As I write this, the election still hasn’t been called was just called for Biden. Hallelujah ๐ŸŽ‰.

What I’m working on at Preceden

Asif is still busy putting together mockups for Preceden’s ongoing redesign work. Here’s the latest Case Studies page mockup:

Tom Davies (in my mastermind group) suggested checking out the #jobs channel in Tailwind’s Discord group to find a front-end developer. That proved to be a great resource, and I quickly found and hired a talented developer in Bosnia named Milan to work on implementing Asif’s designs and help move Preceden over to Tailwind. Work is ongoing there.

What I’m working on at Help Scout

Net Promoter Score (NPS) is a way to gauge how satisfied your users are. You ask them how likely they are to recommend your service to a friend, and they give you a 1 to 10 score. You then take the % of 9s and 10s (your promoters) and subtract the % of 0s through 6s (your detractors) to get your NPS. At Help Scout, surveyed our users to determine our NPS, and I’ve been helping analyze the results.

The trick is knowing how to interpret the score. Is 30 good? 40? 50? 60? In a perfect world you would know what NPS score your competitors have, but that’s often not public and even if it is, it’s not always clear how reliable it is. TurboTax has a NPS score of 58. But which users do they ask and when? Are they asking people right after they get notified they’ll receive a refund? If so, they wouldn’t be surveying people who get frustrated and give up earlier in the funnel, which may give them a higher score than they would receive otherwise. If you’re a TurboTax competitor and survey users earlier in the funnel and get a lower score, it would be a mistake to compare it to TurboTax’s number, assuming you knew when users were surveyed.

The ideal process is likely automatically surveying a fraction of your users every month and tracking the trends over time. This will give you a sense of whether the changes you’re making to the product are making users happier or not, and you can segment the results in a variety of ways to understand how different groups of customers score your product.

What I’m studying

Didn’t spend much time on machine learning this week due to the focus on Preceden and election distractions. I did start porting my local Jupyter Notebook for the Housing Pricing Competition over to a Kaggle Notebook, but it’s not ready to be shared quite yet. Probably next week.

What I’m watching

The Queen’s Gambit on Netflix:

Be well my friends ๐Ÿ‘‹

Friday Updates: Preceden Redesign Progress, Learning Tailwind, Data Survey, Housing Prices Ranking, and Yoga with Adrienne

Photo courtesy of Marius Ciocirlan

What I’m working on at Preceden

Work continues on the site redesign project.

I found a talented designer on Dribble named Asif Howlader who I wound up hiring to put together some mockups to brainstorm ways to improve Preceden’s design.

Here for example is Preceden’s current Case Studies page:

And after a few iterations here’s where the redesigned page stands:

Not bad right?

He’s currently working on redesigning the homepage and later will tackle other areas of the site as well.

The next task is actually implementing this on the site. My plan is to hire a front-end developer freelancer who can knock it out. I’d also like to have him or her transition Preceden to Tailwind in the process which should go a long way towards making the site easier to develop in the future and also standardize a lot of the design elements (right now it’s a hodgepodge of colors, font sizes, margins, etc). Before I hire someone though, I want to get some experience with Tailwind, so this week has had me diving into Tailwind courses and trying to get it set up in Preceden.

This in turn had me upgrading Preceden from Rails 5.2.2.4 to 6.0 and Ruby from 2.6.5 to 2.6.6, which has me thinking about this Malcom in the Middle clip:

What I’m working on at Help Scout

In an effort to get a better feel for where to focus my efforts in the coming months, I created a survey and posted it in our metrics Slack channel for anyone to take. Here were the questions:

Score from 1 (strongly disagree) to 5 (strongly agree):

  1. I rely on data to make decisions in my job
  2. I am satisfied with my ability to use Looker
  3. Looker has reports that are useful to me
  4. Looker has reporting on everything I want
  5. Itโ€™s easy to find reports on specific things in Looker
  6. I am confident that I understand the meaning of the metrics and data fields I use
  7. I trust the accuracy of the reporting in Looker
  8. I am satisfied with the speed that my data questions are answered

Open ended questions:

  1. If you could wave a magic wand and change one thing about our data work at Help Scout, what would it be? (optional)
  2. Anything else you would like to add? (optional)
  3. What department are you in? (optional)
  4. Whatโ€™s your name? (optional)

Two big takeaways from the responses are that people want to learn more about how to use Looker and also that there’s a demand for more product analytics (we have some, but comparatively little compared to our marketing and finance reporting).

What I’m studying

After a few more hours of iterating on my housing prices prediction notebook, I finally made it into the top 5% of scores:

Around 2% of the high scores are cheaters though (because the full data set is public so people can find it and just submit the actual house prices to get near perfect scores) so this score is closer to top 3%. And many of the higher scores cheat in a more subtle way, by creating a model that incorporates the training data (aka data leakage) which gives you an advantage over someone who is just creating a model based on the training data (which is the right way to do it).

I’m at the point of diminishing returns on this though so will probably wrap up soon. I’ve read through a lot of the public notebooks and don’t feel like there’s a ton more to learn, at least relative to jumping into another competition. I’ll likely put together a final notebook and share it on Kaggle next week for others to learn from.

What I’m watching

Too much time at the computer has inevitably led to me to have frequent back pain. A lot of things help though:

  • Standing instead of sitting
  • Weight lifting
  • Chiropractor
  • Yoga

For this last one, I’ve been doing a lot of Yoga With Adrienne on YouTube. Here’s an example:

If you have back issue or just want to try to work more yoga into your life, check out her videos.

Adios for now ๐Ÿ‘‹

Friday Updates: Preceden Design Hire, GPT-3 Response Generation, Housing Prices Progress, Star Trek Discovery, and the Almanack of Naval

Photo courtesy of Josh Berquist

What I’m working on at Preceden

I’ve got this thing where I’m never quite satisfied with Preceden‘s design. It’s not that bad, but when I compare it to a site designed by professional designers like Help Scout or Stripe, it’s clear that there’s a large gap, and it’s not one that I’m talented enough to close on my own.

And so this week I posted a job on Upwork to try to find a designer to help improve Preceden’s design:

I was optimistic that this would receive a lot of applications, but it didn’t work out that way. It received maybe 10 applications, out of which I’d say 9 hadn’t tailored their application even one bit to the job description which is an immediate red flag for me (even just a short “Hey, working on Preceden sounds interesting, here’s a bit about me…” goes a long way). The 1 that did wasn’t a good fit. In the end I did make an offer to 2 people (so I could work with them for a short project then pick one to work with longer term), but one declined saying he was too busy and another wanted a min $3k initial commitment which I wasn’t comfortable with.

At Automattic and now Help Scout, there are some incredibly talented designers who can create mockups and then take that design and implement it in HTML/CSS within the codebase. But this hiring experience reminded me that those people are rare and in high demand and probably not looking for short projects on Upwork.

It’s more common for different people to tackle these tasks: one to create the design, another to implement it. For round 2 of hiring for this, I’m going to try to split it: find and hire one person (probably on Dribbble) to create the mockups, and hire a separate front-end developer to implement that design in Preceden.

What I’m working on at Help Scout

I shipped a fun little experiment that lets people at Help Scout use GPT-3 to automatically draft a response to customer conversations within the Help Scout editor:

This uses Tampermonkey to add the GPT-3 button using custom JavaScript that’s run on conversation pages, which when clicked takes the customer’s name and last message, posts it to an API endpoint which runs it through GPT-3 and returns the completed text, which is then simulated as being typed out in the editor to look cool.

To be 100% clear, we have no intention of actually building this into the real product. It’s meant to demonstrate GPT-3’s strengths in weaknesses. If I had to boil them down, I’d say that GPT-3 is great at generating text that looks like it was written by a human, but at least in this support context, the responses are terrible because GPT-3 has no knowledge of the product its trying to answer a question about. Maybe down the road there will be some future version of this technology where it can be trained on your support history or docs, but until that happens, GPT-3 probably won’t be augmenting customer service reps in this way anytime soon.

What I’m studying

Still chugging away at Kaggle’s old housing prices prediction competition, managing to break into the top 500 out of 4.6k:

My approach to this is to make as much progress as I can on my own, then peak at others’ publicly shared notebooks to get ideas for how to improve, then iterate on my own notebook accordingly. I’ve been learning a lot this way and probably stick with this competition for a few more weeks until I feel like I’ve maxed out how much I can learn from it.

What I’m watching

Star Trek Discovery:

Unlike most of the older Star Trek series which basically consisted of the ships exploring and getting into different adventures each episode, Discovery (and Picard) have an single story arc that’s told across each episode. It’s really good.

What I’m reading

The Almanack of Naval Ravikant – a summary of his podcast interviews and writings. I’m a huge fan of Naval. If you’re new to him, check out his Farnam Street interview which I’ve listened to several times.

That’s it for now ๐Ÿ‘‹