Friday Updates: Preceden’s Data Retention Dashboard, Hiring a Data Lead at Help Scout, Hanna

Photo by Jason Leung

What I’m working on at Preceden

This week I created a data retention dashboard (using Mode) to help me monitor how everything is progressing.

There are 225k users scheduled for deletion in the next few months and 46k have already been deleted (these are folks who hadn’t been active in years and had little to no content in Preceden). It has sent out 8k “Your account will be cancelled in 60 days” notices which has led to 63 users logging back into Preceden, and a whopping $58 in reactivation revenue. These emails have gone almost entirely to people who never paid in the past, so it’s not surprising not many have paid.

This week I got former customers added to the schedule and they should reactivate at a much higher rate. Not expecting a ton of reactivation revenue from this project, but if it winds up bringing in a few thousand extra dollars in revenue this year I’ll be happy with it.

Here’s the number of users deleted each day:

We can see some initial experiments in the beginning there, followed by a few days of bug fixes before it ramped up again.

It’s sending about 1,500 notifications per day now informing folks their accounts will be deleted in 60 days (and eventually 7-day reminders too):

And here’s the 225k scheduled for deletion:

These are spread out just to reduce the risk if something goes awry.

The only segment of inactive users remaining to handle are those with public timelines. Some of these timelines have hundreds of thousands of page views so I don’t want to delete them even if the user who created it hasn’t been active in years (since people are still getting value from them, plus some people go on to sign up and pay for Preceden). This week I spent a lot of time building out a system for quickly measuring how many total views a timeline has had. Preceden does have a views table that tracks every view a timeline has gotten, but with tens of millions of records it was impossible to query efficiently to say like “give me a list of timelines with more than 100 views”. Took creating a daily timeline views table to make it possible to query quickly.

It’s all coming along. Should wrap up this project in the next 2 weeks or so and move onto something new.

What I’m working on at Help Scout

This week we published a Data Lead job opening that I’ve been working on recently. I’ve been the Data Lead for the past 4 years, but with me being a part-time contractor now and preferring an IC role anyway, it makes sense to bring on a full time person who can take Help Scout’s data initiatives to the next level.

If this role sounds interesting to you, I’d encourage you to apply (and bonus points for being a reader of this blog!). Some highlights:

As our Data Lead, you will
  • We’d look to you to lead, develop, mentor, and manage our data team, currently consisting of an analytics engineer, and oversee the hiring efforts to grow the team to 5 people in the next 12-18 months.
  • Collaborate with leadership from across the company including marketing, sales, finance, support, and product to gain a deep understanding how data is used at Help Scout today and how we can make it even more useful in years to come.
  • Take ownership of our data stack which currently includes Looker, dbt, BigQuery and Fivetran. You’ll maintain the long-term roadmap for our data infrastructure and make sure it can grow alongside the company. 
  • Evaluate and implement new tools and processes to expand Help Scout’s data capabilities.
  • Prioritize and assist with data initiatives and data requests and communicate analyses to stakeholders.
  • Initially spend roughly 50% of your time on individual contributor/technical/analysis work and 50% on management/strategy work, but gradually spend a larger % of your time on the latter as you grow the team.
In your first 90 days at Help Scout, you will
  • Learn the ins and outs of the business, our data stack, our processes, our key metrics, etc.
  • Gain experience by applying what you learn to solving real-world data requests.
  • Meet with other team leads to understand how they use data currently and brainstorm opportunities for improvement that the data team can tackle in the future.
  • After you get spun up and better understand the state of data and the needs of the business, you’ll decide on what role we need to hire for next and lead the effort to hire that person in Q3.

What I’m watching

Hanna on Prime:

Cheers!

Friday Updates: Account Deletion Notifications, Removable Branding, Prophet, Exhalation, Synchronic, and Burpees

Photo by Aniket Bhattacharya

What I’m working on at Preceden

Account deletion notifications

As part of this ongoing data retention project, I’ve reached the stage where I’m handling inactive users who do have a meaningful amount of content in Preceden and therefore need to be notified and given some time so they can log back in and prevent their account from being deleted.

Here’s what the first notification looks like, this one sent 60 days prior to deletion:

This email was inspired by a similar email that Basecamp sends to inactive users:

Basecamp considers inactive to be 1 year and give users 30 days notice to log in before the account is “cancelled” and then an additional 30 before it’s deleted.

For Preceden, I’m considering inactive to be 2 years and giving them 60 days notice before their account is deleted (with an additional reminder 7 days out). I suspect the reason why Basecamp does the cancelation and then the deletion later is a way give users once last chance to return, since they notify them the account has been canceled (but not yet deleted). Preceden doesn’t have a way to soft delete users and their content like this, so for this v1 I went with the simpler route of just saying their account will be deleted at the end of the window. Might iterate on this though, possibly even going down to 1 year like Basecamp.

Removable Branding

Tom in my mastermind group threw out an interesting idea: what if I added Preceden branding to exported timelines on the Free and Student plans, but hid it on the Personal and Professional plans as a way of encouraging more people to upgrade to the higher priced plans. I thought it was a great idea and so knocked that out yesterday.

Here’s what the revised plans look like:

On the Free and Student plans, when users go to export your timeline they will see a notice about the branding:

And their content will include a “Created with Preceden” branding section:

This branding is automatically removed if they upgrade to the Personal or Professional plans.

There are plenty of people who use Preceden for work who only need one historical timeline, so the Student plan has been fine for them functionality wise in the past. Hopefully this branding will nudge some of them over to the Personal plan in the future, driving additional revenue.

What I’m working on at Help Scout

Shipped a project to use Facebook’s Prophet to improve the accuracy of our monthly metrics projections. I’ll likely write up a separate post about this detailing how it works in case anyone else wants to try it in their organization.

Also got some attention on Twitter thanks to a post on some GPT-3 experimentation I did at Help Scout a few months back (hat-tip Mat Patto for the fantastic writeup):

What I’m reading

Exhalation, a collection of short stores by Ted Chiang, who wrote the book that Arrival is based on. My favorite story so far is the first one titled The Merchant and the Alchemist’s Gate, a trippy story about time travel.

What I’m watching

And speaking of time travel…

What I’m exercising to

For a few fee weeks I’ve been rowing 5km four-five nights/week (which almost has me through Better Call Saul). I realized though I needed to incorporate more into my exercise routine than rowing, so have been alternating between 15 minutes of yoga (with Adrienne!) and a short HIIT workout before rowing.

For the HIIT workout, I’ve been just trying to make it through this burpee workout:

It’s 100 jumping jacks followed by 100 burpees, with the burpees being 30 seconds on then 30 seconds rest spread over 10 minutes. I can barely make it through this, with the last set taking almost the full minute and my movements barely being resembling burpees at that point. If you’re looking for a short intense workout, give it a shot.

Adios 👋

Friday Updates: Scheduling 200k Preceden Users for Deletion, Yuval Noah Harari Interview, AlphaGo

Photo by Gary Chan on Unsplash

What I’m working on at Preceden

As you might recall from previous updates, I’m working on a big data retention project for Preceden. For example, imagine someone who signed up for Preceden in 2015 and never did anything. This account brings up some interesting questions IMO:

  • Should I keep that user’s account around forever?
  • If the answer is no, do I need to notify the person that their account will be deleted?
  • How long does a user need to be inactive before being subject to deletion?
  • If I do notify them, how much time do I give them to log back in? How many reminder emails do I send?
  • If the user does have content, does it affect the decision about notifying them or not?

This is what I’ve been figuring out and implementing in Preceden recently.

Here’s where I’ve landed so far:

  • If a user hasn’t been active in 1 year, has never paid, and has no content in Preceden, I’ll have Preceden delete their account without any email notifications.
  • If a user hasn’t been active in 2 years, has never paid, and has little content in Preceden (1-4 events), I’ll also have Preceden delete their account without any email notifications. There’s an argument for notifying them, but it’s not clear cut: if someone barely used the product and hasn’t used it in years, do I really need to email them? The vast majority won’t care and many would find the email annoying and someone would report it as spam. And for the handful that do care, it’s easy enough for them to create a new account and recreate the little amount of content they previously had.
  • If a user gets deleted and then the user attempts to log in, I’ll display a message letting them know what happened.
  • For users with 5+ events who haven’t been active in 2 years and aren’t paying, Preceden will send an email notification and give them some time (maybe 60 days) to log back in which would stop the deletion. Probably will sent 2 emails about this, one at 60 days and another at like 10 days. Will make some exceptions for users who have popular publicly shared timelines since there’s value keeping them around. This process should also help reactive a bunch of inactive users, hopefully leading to additional revenue.

Last two weeks I worked on the first three parts of this dealing with inactive users that don’t require notification. All in all, there are now 200k users scheduled for deletion over the next 60 days. The reason for 60 days is so that if there are any problems, I can stop the process and investigate before Preceden deletes too many users. I’ve got internal notifications set up so that if someone who is scheduled for deletion logs in, I’ll get an email so I can investigate. In theory there shouldn’t be too many of these, so if there are a lot it indicates something is amiss.

The code for all of this is probably the most thoroughly tested code I’ve ever written, hah. Want to be super careful and not accidentally delete an account that shouldn’t be deleted. It’s coming along.

What I’m working on at Help Scout

The usual: working on a job description for a data lead, project managing the setup of Heap in the app, data requests, Looker training videos, and using ML to improve our monthly KPI forecasts.

What I’m watching

This documentary on AlphaGo, the software DeepMind created to play the game of Go:

What I’m listening to

Tim Ferriss’s recent interview with Yuval Noah Harari. The discussion around 1 hr 25 minutes into it about the major global problems, especially about technology, was very thought provoking.

Be well my friends 👋

Friday Updates: Data Retention Progress, Better Call Saul, Doomscrolling

Photo by the Creative Exchange on Unsplash

What I’m working on at Preceden

I’ve been thinking a lot about data retention at Preceden. After people are done using the service, unless they manually delete their timelines and account, that data stays around forever. And since Preceden has been around 11 years now, that means a lot of accounts and a lot of timeline data that’s sitting there that’s serving no purpose. By figuring out a way to clean it up, the database will perform better, there’s less impact in the event of a data breach, it serves as a way to retain paying customers (“we’re not just going to hold onto your data forever if you stop paying”) and in general it feels like the responsible thing to do.

Easier said than done though.

For a v1, I’m attempting to automate a solution to delete accounts that have no timeline data and haven’t been active in at least 2 years. There’s actually a lot of accounts that fit this criteria: about 25% of users that sign up never create a single event. Spread out over 11 years, and the number is pretty high.

Preceden isn’t set up to handle this type of thing well. Users have timelines, timelines have layers, layers have events. And some timelines have collaborators, some users have teams, and some users are set up as teachers with student sub-accounts. Iterating over each user and performing the necessary queries puts a ton of strain on the database, causing performance issues for people actively using it. And I haven’t tracked last active date in the past, adding further complication.

And for v2, I want to handle situations where users have timeline data, and that’s going to require notifying them, giving them some time to log in and indicate they want to keep using it, or to export their data if interested, etc. Lots of and lots of complexity here and it all has to be done super carefully: don’t want to ever delete any accounts accidentally.

This is all solvable, just taking some time to work through it one issue at a time.

What I’m watching

Better Call Saul on Netflix:

I loved Breaking Bad back in the day, but never got into this spinoff. Finally started watching it and am really enjoying it.

What I’m (not) doomscrolling

Twitter.

I got into a bad habit the last few years of keeping Tweetbot open during the day and scrolling through tweets every 10-15 minutes throughout the day. It was largely driven by following politics on Twitter, ugh.

I recently got a new Macbook and decided not to install Tweetbot. It’s a hard habit to break, but I’ve definitely noticed my anxiety level decrease recently and I think not having Twitter open all day has contributed to that. It definitely helps that Biden won the presidency and Twitter politics is a lot more boring now.

Thank goodness.