In the early days of our startup, speed was everything, so we adopted an off-the-shelf, generic SaaS offering to integrate customer data quickly. It worked well at first, but as we onboarded more clients, expanded to more sources, and encountered more edge cases, the limitations of generic ETL started to show.
In this talk, I’ll share how we transitioned from a one-size-fits-all tool to a custom-built, AWS-native ingestion system using Step Functions, Fargate, S3, DynamoDB, and RDS. You’ll hear: