Draft: How does XTDB compare to other databases?

[jacobobryant]

Edit: draft moved to the main website. Will publish officially soon.

[jacobobryant]

Just wrote up the XTDB vs. Postgres section.

(Also, this essay writing stuff takes forever…)

[jacobobryant]

Started on the XTDB vs. Datomic section and slightly updated the remaining outline.

[jacobobryant]

Heh, already ranking.

[jacobobryant]

Just wrote up the XTDB vs. Datomic section and made some light edits to the rest of the essay. The draft is now complete.

@refset would you mind giving the whole essay a read to make sure I’m not spreading any misinformation ha ha/if anything else comes to mind?

[jacobobryant]

@jacobobryant

As mentioned above, XTDB leaves schema enforcement to you—my understanding was that with bitemporality, it was just too tricky to do schema in a general way.

A fact-check on this would also be nice.

[refset]

Thanks for the ping - no misinformation that I can see! It’s perhaps worth adding/clarifying (if only here):

1. Datomic’s schema offers an opinionated, RDF-inspired information model (with defined scalar types, composite attributes, reified transactions, upsert etc.) whereas XTDB is ‘dynamic’ and defers all control to userspace. As for “with bitemporality, it was just too tricky to do schema in a general way” – it’s more that the solutions to the bitemporal schema problem all required creating the dynamic core first, rather than there being some ~artificial implementation-related compromise
2. Datomic exposes indexes for direct data access and queries rely on user-specified join ordering, whereas XTDB only encourages data access via high-level queries
3. Datomic Cloud promotes running compute using ‘Ions’ and leans heavily into AWS for scaling (with EFS etc.), whereas XTDB is purely offering node-local indexes running on SSDs. There are a huge number of differences in implementation details and operational considerations consequently
4. Performance between the two could vary substantially for different use cases. It probably goes without saying, but when evaluating any database it’s definitely important to validate the performance and scaling for your application workload - there is no “one size fits all”

[jacobobryant]

thank you this is great!

[axelbdt]

This is a great read already, thanks for making the draft public.

Can I ask for more detail (or a reference link) about storing the schema in the database? After 15 years with RDBMS, using schema-less DBs feels like driving without a seat belt and I am looking for resources to make this paradigm shift and use XTDB more confidently. The link about migrations was eye opening in that regard.

Cheers

[jacobobryant]

Glad it’s helpful! If you’re not already familiar with how Biff’s schema-in-code approach, you can take a look at these:

• How Biff’s schema definitions look
• What it looks like to submit a transaction with Biff that’s checked against the schema
• Biff reference docs for schema and transactions

To clarify, this differs from what the datomic articles refers to as “schema-implicit”–the schema is explicit, it’s just stored in a Clojure var and only enforced by clients to the database, not by the database itself. That also means the current version of the schema that’s active is tied to the currently deployed code. For my projects that’s been fine, but if you had a database that was accessed by multiple services, you might prefer the database itself to be the source of truth for what version of the schema is currently active. So that would be the main benefit of going beyond what Biff already does for schema.

As for the implementation, there might be an XT document for each document type, like {:xt/id :biff.schema/user, :biff.schema/malli [:map ...]}… or there could even just be a single document for all the schema, like {:xt/id :biff/schema, :biff.schema/malli {:user ..., :msg ...}}.

Though I should mention that even in this case, it would still be the client’s responsibility to load the schema and enforce it when submitting transactions; there wouldn’t be any schema-checking enforced by XT itself, similar to what you get in RDBMS/Datomic. So there could be some footgun potential there for submitting transactions without going through the schema checks

[axelbdt]

Thanks a ton for the detailed answer.

I appreciate the clarification about what “storing the schema” entails, and especially that it is still user responsibility to retrieve said schema and check against it.

[jacobobryant]

@refset

As for “with bitemporality, it was just too tricky to do schema in a general way” – it’s more that the solutions to the bitemporal schema problem all required creating the dynamic core first, rather than there being some ~artificial implementation-related compromise

By the way—does this mean that XT might at some point come with (optional?) schema enforcement baked in?

[jacobobryant]

Made a few edits. Now I’ll probably publish this next week.

[refset]

@jacobobryant

does this mean that XT might at some point come with (optional?) schema enforcement baked in?

That’s the plan, “gradual schema”