At work, we recently had to move a few tables from one PostgreSQL instance to another. In my previous post, I discussed how to use Google’s Database Migration Service (DMS) to migrate data from one instance to another. Unfortunately, that option was not available here, since DMS only allows the...
Lately, I've been interested in how third party vendors can best authenticate into their customers' cloud accounts. The status quo in AWS is usually role assumption from the vendor's account to the customers', but what about GCP and Azure? Can OIDC be used to authenticate into all three clouds in...
At work, we recently upgraded our multi-terabyte PostgreSQL database from version 13 to 16 in Google Cloud Platform (GCP). This post shares lessons learned during the process, focusing on our experience using Google’s Database Migration Service (DMS). DMS supports various migration types, but...
In the world of data engineering, ensuring data quality is paramount. From business analysts relying on dashboards to C-level executives making strategic decisions, and data scientists training machine learning models — everyone depends on the quality of the data. If the data is flawed, the...
About a year ago I quit my job to attempt my startup idea (Deadpendency), writing it in Haskell. One key decision to make was what cloud provider would I go with. Like many developers, I have a bunch of professional experience developing against Amazon Web Services (AWS). The Haskell community...