Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Even with the Record layer, it doesn't have support for JOINs in the same way as an RDBMS would.

Yes, the Record Layer helps you define and index into _hierarchies_ of entities, but I suspect it doesn't have an answer for other access patterns (e.g. producing "report" views that relate or aggregate non-hierarchical data).

You could retroactively construct a custom view _after the fact_, but only if you can do so within 5 seconds. And (if you want continued access to that view) you'll need to ensure that that view is maintained thereafter (you would have to define your own layer -- it cannot be entrusted to application logic unless you can atomically switch to a new version of your stack). Maintaining such a view is made more difficult if your data allows updates/deletes.

The same caveat affects schema migrations. You would need to be able to fit the migration into a 5 second transaction (or tell your applications to stop modifying the data for a while, and handle it as a series of smaller transactions).

My assessment is that if you have (non-hierarchical) relational use-cases at scale, FDB really requires you to plan your access patterns from day 1. Whereas a typical RDBMS fares far better at satisfying emergent needs. That said: FDB's model is brilliant for document store and key-value use-cases.



You're right the Record Layer doesn't yet have join support, but note PR #306 [1]. We support aggregate indexes but can't run aggregate “reports” that aren't backed by indexes. The rationale for this is covered in the paper (see [2]), but note that doesn't preclude such support being added into or on top of Record Layer (also discussed in the paper).

The Record Layer does a whole bunch of work to deal with index maintenance (see our docs [3])). Our “index maintainer” abstraction (discussed in the paper) makes maintaining our indexes (including those that are basically materialized views) completely seamless from the user's perspective, even for updates and deletes. We also have a lot of tooling for making efficient schema migrations. For example, schema migrations are performed lazily (when the data is accessed), so they aren't limited by the 5 second transaction limit. If you add/remove/change indexes, they'll be put into a “write-only” mode where they'll keep accepting writes while an “online indexer” builds the index over multiple transactions. We even have fancy logic to automatically adjust the size of the transactions if they start failing due to contention or timeouts!

Basically, the Record Layer solves a lot (but not all) of the pain points that shows up when you don't know your access patterns from the beginning. The paper talks a bit about how CloudKit uses some of those features.

[1] https://github.com/FoundationDB/fdb-record-layer/pull/306

[2] https://foundationdb.github.io/fdb-record-layer/FAQ.html — search for “aggregation”

[3] https://foundationdb.github.io/fdb-record-layer/SchemaEvolut...




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: