This may be my experience as a Googler talking, but I also somewhat disagree with the notion that the data can't fit in memory. I operate a X0,000 task service, and our monitoring data could fit into memory in a large server, if need be.
Of course, we don't keep per-task data permanently, that would be prohibitive at a 5s monitoring interval like the one we use, even if it were put on disk. Instead, we accomplish what I described by aggregating away some dimensions, mostly task number, and then holding the aggregated series in memory, for fast queries. There are some nuances, particularly around having foresight over the cases where you do want to see individual tasks.
But suffice it to say that this person's experience does not match mine in terms of what I need from a TSDB. Perhaps his ops background comes from a different set of needs than mine, but if you're building a TSDB for many customers, I wouldn't take this list as gospel.
If I need a few terabytes of data to be stored reliably enough that I can lose a node (or two), support mostly writes and some reads, for a duration of a few years (at least three), would you still suggest storing this in RAM?
This may be my experience as a Googler talking, but I also somewhat disagree with the notion that the data can't fit in memory. I operate a X0,000 task service, and our monitoring data could fit into memory in a large server, if need be.
Of course, we don't keep per-task data permanently, that would be prohibitive at a 5s monitoring interval like the one we use, even if it were put on disk. Instead, we accomplish what I described by aggregating away some dimensions, mostly task number, and then holding the aggregated series in memory, for fast queries. There are some nuances, particularly around having foresight over the cases where you do want to see individual tasks.
But suffice it to say that this person's experience does not match mine in terms of what I need from a TSDB. Perhaps his ops background comes from a different set of needs than mine, but if you're building a TSDB for many customers, I wouldn't take this list as gospel.