MWAA UndocumentedAmazon MWAA is the managed version of Airflow offered by AWS. In this small document I am going to add findings from experience and…May 9, 20241May 9, 20241
Monitoring Task Execution in Managed Workflows for Apache Airflow (MWAA) WorkersApache Airflow has become a go-to solution for orchestrating complex data workflows, and AWS’s Managed Workflows for Apache Airflow (MWAA)…Oct 11, 2023Oct 11, 2023
Devoxx 2023 (Athens)Initially, I was skeptical. In the age of Covid, do people even attend conferences anymore? With free educational content on YouTube and…May 7, 2023May 7, 2023
Bring back all deleted S3 keys in a version—enabled S3 BucketIt is not hard but it is nice to have in our toolbox a small python script that can revert easily our deleted s3 objects from a…Mar 20, 2023Mar 20, 2023
AWS Rest API for GlueThis is a small script to help access AWS Glue using Rest API. AWS Glue is a serverless data integration service that makes it easier to…Feb 18, 2023Feb 18, 2023
Create easily python packages for Airflow MWAALately, we have decided to move our custom airflow workflow servers to the more managed approach of AWS MWAA. The basic thought behind this…Jan 31, 2023Jan 31, 2023
Create a company in 4 weeksThese are my notes on listening to Dan Sullivan of Crowdly. His talk on youtube in Harvard Innovation Labs is called “Fake it Till you Make…Nov 22, 2022Nov 22, 2022
Python Hidden GemsI am a fan of Raymond Hettinger. For those who do not know him, he is one of the Python core developers with significant contributions like…Oct 18, 2022Oct 18, 2022
Data Engineering SQL TipsThis can be an ongoing list of small SQL scripts I am reusing in various projects.Sep 12, 2022Sep 12, 2022
Artist Guide to AI Image GenerationWhat keywords should you use to better control ai image generation (for Dall-e-2 and MidJourney)Sep 1, 2022Sep 1, 2022
