PinnedAntonio SiHow Intuit Debug Consumer Lags in Apache BeamAt Intuit, we have been using Apache Beam with FlinkRunner as a real time data processing pipelines platform for our data pipelines…8 min read·Apr 8, 2021--1--1
Antonio SiA Few Tips On Remote Debugging Golang Applications Running in an M1 Docker ContainerI recently picked up Go programming. One technique that I always find helpful in development is the ability to remote debug a program…9 min read·May 1, 2023----
Antonio SiSupplementary Notes to Deploy Argo-Events in Managed Namespace Scope— coauthored with Prema Kuppuswamy4 min read·Feb 8, 2023----
Antonio SiinApache Beam State ProcessingMicroscopic Look at the States Inside Apache Beam Stateful Pipeline (Part two of three)— coauthored with Prema Kuppuswamy and harish nagu sana10 min read·Jan 27, 2023----
Antonio SiHow To Obtain Kafka Consumer Lags in Pyspark Structured Streaming (Part 2)In this article, I will describe another alternative to obtaining consumer lag based on spark checkpoint file. As I did in my previous…7 min read·Jul 8, 2022----
Antonio SiHow To Obtain Kafka Consumer Lags in Pyspark Structured Streaming (Part 1)Pyspark is a common BigData computational engine used by data scientists. Intuit Stream Processing Platform (SPP) provides a common…9 min read·Apr 2, 2022--3--3