Hello HN,

I recently noticed I'd crossed 100 commits on my personal site and realized it's been over three years since the initial commit. I wrote down some thoughts on the journey.

Curious to hear if others here still maintain a personal site/blog and what your experience has been.

Last weekend, I dove into CMU’s [15-445/645](https://15445.courses.cs.cmu.edu/) database course and got hit with a deceptively simple problem: count the number of unique users visiting a website per day. Easy, right? Just throw user IDs into an unordered_set and return its size—classic LeetCode.

But what happens when you’re at Facebook scale? Tracking a billion unique users means burning through GBs of memory just to count. And in the real world, users are streaming in constantly, not sitting in a neat, static list. Storing every ID? Not happening.

I explored practical workarounds (like “last seen” timestamps and full table scans), but they’re either inefficient or put massive strain on your DB. Then the assignment introduces HyperLogLog: a probabilistic algorithm that estimates cardinality with just 1.5KB of memory—accurate to within 2% for billions of users.

The magic? Pure mathematics. It’s distributable, and powers real-world systems like Redis and Google Analytics. I break down how it works (with illustrations!), check out my deep dive.

Curious to hear from HN: Who’s using HyperLogLog in production? And have you run into accuracy issues, and how did you handle them?

Sounds like if-else with extra steps.

MIT engineers invent something -> it becomes talk of the town -> every one forgets about it the next day/week.

Not quite everyone. Last year they got $1M from the Musk Foundation and $80M from investors including Breakthrough Energy Ventures, and they're working with their first commercial client.

https://news.mit.edu/2022/cracking-carbon-removal-challenge-....

I have a presumption that this is going to be the talk of the tech-town for the next few days.

"sorting with O(n^2) is no longer a bottleneck as we have fast processors" /s

That makes no sense. Just the brutal math of a polynomial is always going to be poor enough to notice than subpolynomial times.

Often but not always. Cool trick: any bounded limit is always O(1)!

Pick a small enough bound and an O(n^2) algorithm behaves better than an O(n log n). This is why insertion sort is used for sorting lengths less than ~64, for example.

Pick a small enough bound and certain O(n^2) algorithms will behave better than certain O(n log n) algorithms.

Big O notation doesn't take into account constant factors of overhead or plain old once-per-run overhead.

Sorry it was indeed a typo

GitHub Copilot moment