Databases: Part 5 of How to Commission a Software Project
This database article is part of the “How to Commission a Software Project” series.
Databases are Changing
Network computing is at the edge of a strange epoch. Database architecture is a particular part of the stack currently subject to the changing landscape. Big data, serverless and virtualized computing, and high-availability data centers are pushing especially hard on the database world. The change is great news if you’re prepared for it. But I’m afraid for organizations who dig their heels into legacy relational database implementations. Like any season of tech hotness, the nerds will love to go all-in on the new thing. I’d recommend a more even-keeled, hybrid, and intentional approach.
You might remember how much I love documentation. That includes documenting your data model by measuring potential reads, writes, scale, longevity, and accessibility against the tools available and put each of your data objects in the right buckets. That might mean your customers live in a relational database (SQL or something SQL-like) and your transactional data live in a big data store. The trick is balancing desired outcomes with the resource allocation available to connect a data model in one data store to its related models in another data store. Depending on the number of sources and the capabilities of your tech stack, there are potential development considerations.
I intend to make these articles as stack-agnostic as possible. However, since it’s hard to speak meaningfully about these technologies without dropping names, I’ll mention a few platforms within Fivable’s preferred stack as I go. Since we are an Amazon Web Services (AWS) shop, our databases are mostly spread across Aurora instances and DynamoDB tables. Apart a bit of a gray area in the middle, our litmus test for answering “what goes where?” is pretty simple. Small, inflexible, and highly relational sets go into Aurora and large, free flowing and less correlated sets go into DynamoDB. That method isn’t axiomatic, but it’s a fantastic starting point. Here’s a comparison figure:
Measure your data models, understand the capabilities of your resources, and anticipate how accessible each component needs to be. Draw it out on a whiteboard with your tech team and determine how you can leverage a hybrid approach to your databases. Of course these details and the other ten thousand I’ve simply skipped for the purposes of this article may not require much of your attention as a founder or operations professional. But the goal is to afford you enough fluency with tech-stack items to talk to your own nerds.
Interested in getting help with designing or implementing your databases?