Alternative title: what happens when a backend engineer tries to make a product and predictably fails

A few years ago, I decided to take some time off and explore what's next in my career. While looking for work, I decided to build a side project to keep my tech skills sharp and revisit how the world of full stack development had changed since the last time I'd touched it, way back in college.

So, I built trackmy.games, a product that solved a personal pain point: I kept forgetting which games I own on which platform (console, switch, steam, etc) and then sometimes re-purchased games I already had.

Looking back at it almost 2 years later, depending on how you slice it it was either a terrible failure or a smashing success. I'm writing this as a mini postmortem for myself before I forget the details - and sharing in case it helps anyone else.

About me

I'm a backend / systems guy by heart. As I've told friends who have tried to get my advice on full stack/UI projects, they're probably better off without me - my participation on such projects is likely to result in negative expected value. At the same time, a lot of "useful" side projects don't really need a backend - they need good UI and product, so I've always wanted to learn and get better at doing this end to end.

I also am (or more honestly, was) an avid gamer. I'm also a hoarder/collector and often buy a lot of games on sale and through humble bundle. So I would often end up buying games I already had on another platform, or buy something I already had as part of a bundle. If you know me, you may have seen the periodic "here's a few dozen steam keys" giveaways. Not great for the budget. So I thought I'd build something that saved me money by easily keeping track of what I owned where and helped me avoid unnecessary purchases. And then maybe find other users in case anyone found this useful.

Goals

Beyond actually solving a real problem with this project, I think I had a few additional goals. Bucketed into a few groups:

  • General "side project" goals:

    • I wanted to solve a real pain point and save myself some money
    • Get experience building and shipping a side project
    • See if this (working full-time for myself) is something I want to do more of, or less of

  • Personal goals:

    • I like writing code, I wanted to have fun building something
    • ... and have fun writing some more Rust
    • ... and explore some cool tech ideas
    • I wanted to try out and learn more SQLite and see how far I could scale with it
    • ... and put some system design/scaling challenges to the test and see how far modern machines can go.

  • Professional goals:

    • Get my coding skills less rusty, so I could better prepare for interviews
    • Have some non-trivial things to approach in a "systems design" interview manner, for the same reason
    • Have something to put on my resume to show off what I'd built.
    • Actually learn about the non-technical side of shipping too: marketing, legal, compliance, etc etc - things beyond just the code

Well, that's certainly a mouthful. But I was sinking a lot of time into this, and I had a lot of goals!

So, what did you build/do?

Well you can check this out for yourself. But to recap, here's what I built:

  • A website where you can browse the latest and greatest games, and see which games you own across all platforms.
  • A browser extension used to gather data on which games you own, and show in-line warnings on each store before you purchase something you already have.

This was built by a backend guy so I kept things simple: it was primarily just static HTML/CSS with very minimal one off JavaScript. And this was built pre-AI so it was all hand-crafted, artisinal, human-written code (or copied from Stack Overflow).

In addition to the product, there were a few things I did behind the scenes that were new for me (to trial it full-time):

  • Paid for a real proper logo / core graphic
  • Paid for a real legal privacy policy/terms of service after researching options
  • Actually used some cloud services/APIs this time and and did the legal/compliance legwork for them (e.g. sendgrid, google cloud oauth, etc)
  • Set up a proper business (legally registered a corporation in the state of CA, got an FBN, set up a separate domain/email hosting, etc)

A bunch of engineering work went into the backend that's not easily apparent. Beyond project specific stuff (lots of business logic for scraping data, massaging it, etc), I also started to build reusable components I could use for the next few projects. This included stuff (some of it open sourced, some private but intended to be reusable) like:

I also took the time to blog about what I was doing where possible, partly because I like writing and partly so I wouldn't forget what I learned and did. Here's some of my blogposts that were an outcome of working on this:

How I built it

I decided to trial how it would look like if I did this full time, so I approached this like a full time job. I didn't spend full-time hours on this, but perhaps close. I spent the majority of my non-interview prep and non-leisure time (I was burnt out and relaxing) for about 3 months on this. More concretely, I'd estimate I spent about 250 hours on this.

This included a few things that I'd not normally do otherwise on side projects.

From a project management perspective this meant:

  • I had a full ticketing system to track my work (119 closed issues, 60 still open) - I had these meticulously tagged, with various categorizations and effort estimates (some were weeks long, some just a few minutes).
  • Built reusable components where I thought it'd be useful, for future business needs
  • Kept accounts/emails/etc separate and not super tied to my normal personal email

Engineering wise:

  • Professional quality code and commit messages, I wrote what I'd expect to see at $dayjob
  • Stuck to a tech stack I was familiar with, so this meant no javascript, used Rust for the backend and mostly pure HTML/CSS
  • Designed the system to scale, I over engineered the crap out of it:
    • As the monitoring blog showed, p95 response time was ~3-5ms (from reverse proxy to app backend), without caching.
    • Did various load tests/etc to show it'd scale to many tens of thousands of daily users at minimum. I had ~10 users total.
    • Scaled to ~millions of games, ~100ks of jobs processed.
  • Set up proper monitoring for the first time

In terms of the actual work, I:

  • Prototyped the job fetching/scraping part and made that scale up
  • Designed a multi-store datamodel and ensured I could ingest and deduplicate things across various platforms
  • Then built out the auth/product
  • Figured out how to build a browser extension from scratch
  • ... and then fleshed out various features till I felt comfortable shipping

I would say somewhere during that second bullet above I realized I kept hitting scope creep, so I made myself write an MVP list of what I'd be happy with, cut that a few times, then worked backwards and just did ticket by ticket till I got there.

Did it fail or succeed?

Well, by most measures of side project success, it was an abject failure: I had 1 regular user (me), a dozen signups (but almost no one used it), made 0 in income from it, and it cost me about $25/mo in recurring costs plus about $3-400 in fixed costs. But hey, even just this month, it stopped me from paying $70 for a game I had access to for free, so on net it's probably worth it for that alone financially.

I still consider it a success though, because my main motivation wasn't to do this full time. Here's the useful stuff I got out of this:

  • I kept my coding skills sharp for my interviews, those went smoothly
  • Having "real" systems design interview style prep helped
  • At least one interview asked me to implement a job queue in Rust and well...
  • I got to explore and learn a number of ideas I'd always wanted to learn about
  • Ended up building a reusable stack for future projects, as called out above. This has already helped me in one future project (more on that soon).
  • Had an excuse to try out Akita (where I interviewed) and blog about both that adn other lessons I had. Blogging has been fun and also helpful
  • It also just kept me happy and occupied, so I consider that a success

Lessons learned

This could fill a whole novel. Keeping it brief though, here's what I learned:

  • Discipline mattered:
    • My progress wnet up a lot when I actually put tickets on a board and did serious project management.
    • Working backwards from the end goal helped see where I was going and made it easier to prioritize
    • Ensure there is enough "fun" work to balance out the grunt work, to keep myself motivated.
  • Engineering is make or break:
    • Don't over-do it with over-engineering, but at the same time make sure you have fun
    • sqlite is underrated, you can quickly develop your features and avoid the N+1 query problem
    • Don't discount devex: having a single static binary and a single file DB makes deployment/debugging SO much easier
    • Server-side html is "good enough" for a lot of sites, esp. if you can get response times in 10s of milliseconds
    • If you know some sysadmin stuff and a bit of backend, you can get by on a super-cheap VPS (this can run on a $5-10 vps for many thousands of concurrent users)
    • Prioritize familiar tech stacks, I kept getting into design paralysis otherwise trying to learn the latest and greatest
    • Keep off-site backups at a different provider (not just the same one)
  • Marketing/product is critical:
    • Need to talk to users upfront, and prioritize an MVP. I had n=2 interested users but it turned out to be a super biased set and no one else needed it
    • I ran reddit ads w/free credits but by that time it was too late, there was no PMF
  • Legal/compliance:
    • It is surprisingly hard as a small shop to hit all the GDPR/etc compliance. So much confusing terminology out there
    • Beyond the safe basics (I tried to keep no unnecessary data) it would be great to have clear guidelines on what applies where
    • Even basic terms of services/privacy policies are expensive
    • So to cover my bases I just did what I want from an ideal product: no tracking, minimal data gathering, etc
    • Company registration takes a while!

Incidents

I did have one "incident". About a year and a half in I forgot to pay my Hetzner bill as the credit card expired and it went to my other email (which I didn't check often enough). This went on long enough that my server got wiped and I didn't have off-site backups. Not much lost (since I was still the only user) but I did just restart the whole thing from scratch on my main VPS (took about an hour to get back and running, switching over DNS, redeploying, running the scrapes again, etc - yay for keeping all of that version controlled). The separation of concerns isn't warranted at this stage, IMO the simplicity of a unified stack (I'm not forgetting the bills on this one...) is too hard to beat.

How would I approach this differently going forward?

I'm already trying to do some of this with future projects. But in general I think it'll still depend on my goals: if I'm having fun I'll focus on the engineering more, if I want to make $$$ I'll probably focus more on product/marketing.

But:

  • Get an MVP up ASAP
  • Screw the backend, fake it with mock data, iterate on the product experience and make sure it works
  • Use a lot more AI for my development now that I know how to (more on that soon), and get a reusable stack for myself
  • Market early, market often, get the right folks' eyeballs for feedback
  • Keep it simple, don't do any of the corporation/registration stuff unless super serious and until I have PMF