Lightweight ETL with AWS Lambda, DuckDB, and delta-rs
Introduction I’m Aki, an AWS Community Builder (@jitepengin). In my previous articles, I’ve focused mainly on Apache Iceberg: https://dev.to/aws-builders/lightweight-etl-with-aws-lambda-duckdb-and-pyiceberg-1l5p https://dev.to/aws-builders/build-a-lightweight-serverless-etl-pipeline-to-iceberg-tables-with-aws-lambda-athena-661 https://dev.to/aws-builders/lightweight-etl-with-aws-glue-python-shell-duckdb-and-pyiceberg-153l While Iceberg is quickly becoming the de facto standard for Open Table Formats (OTF), Delta Lake is still the first choice when using Databricks. In this article, I’ll walk through an approach to reduce…