> In terms of relational databases, Galera Cluster’s group replication is also imperfect, but closer to the ideal.
As a longtime Galera user, I have to admit this "closer to the ideal" has nothing in common with reality. It fails, it loses data, quorum kills healthy nodes, transactions add significant latency. The more nodes you have, the lower performance and fault tolerance. One mysql node could literally endure triple of load, which could be deadly for Galera cluster of 3 nodes. Also, it rollbacks transactions silently.
> document the meetings behind decisions, document the rationale for cutting corners
... document the API, document the project goals, document the company values, document checklists and routines, document processes and personal instructions. DOCUMENT THE GOD DAMN HISTORY OF FAILURES AND INCIDENTS!
And then rely in job on all those documents, not on talks and experience or memory of some key engineers. It hurts when you start, people tend to blame all those unnecessary staff, but it pays off in so high rates later. Often it's possible to say if a company will be successful on long run only by seeing its documentation.
As a longtime Galera user, I have to admit this "closer to the ideal" has nothing in common with reality. It fails, it loses data, quorum kills healthy nodes, transactions add significant latency. The more nodes you have, the lower performance and fault tolerance. One mysql node could literally endure triple of load, which could be deadly for Galera cluster of 3 nodes. Also, it rollbacks transactions silently.