ENGINEERING & SYSTEMS

Rathish Kumar B

Building software, exploring systems, and understanding tradeoffs

I build and think about software systems - how they behave under real constraints, how they scale, and the tradeoffs that make them hold up over time. Over the past 14+ years, my work has spanned databases, data platforms, cloud architectures, and AI engineering workflows. I write practical notes for engineers and technical leaders who care about understanding systems beyond the surface.

I am open to discussing real-world systems challenges with engineering teams, particularly around data platforms, AI system architecture, and infrastructure cost optimization. If your team is navigating complex technical tradeoffs or looking for an experienced sounding board to review architecture, feel free to reach out on LinkedIn or via the contact form.

About Me → Get in touch → Latest Writing ↓
Focus Areas
Database Internals Distributed Systems AI Systems Architecture Data Modeling Machine Learning

Writing

Recent technical notes and systems deep dives.

Showing posts with label Distributed Systems. Show all posts
Showing posts with label Distributed Systems. Show all posts

Amazon Aurora Deep Dive Series: From Monolith to Modular - Inside Amazon Aurora’s Cloud-Native Database Architecture

/ by 

An Aurora Deep Dive Series by Rathish Kumar B - Part 2
An Aurora Deep Dive Series by Rathish Kumar B - Part 2 
Amazon Aurora reimagines the database as a set of decoupled, distributed services—each built to scale, fail, and recover independently.

In our previous article we discussed why monolithic databases hit scalability and availability limits as workloads grow. Traditional RDBMS engines bundle query processing, transaction management, caching and storage into one tightly-coupled system. In such a monolithic design, every SQL write passes through a single process that parses the query, locks data, updates in-memory buffers, logs changes, and flushes to disk. By definition, “monolithic” means all functionally distinguishable components (parsing, processing, logging, etc.) are interwoven rather than separate. This coupling creates bottlenecks: for example, all sessions share one buffer pool and one write-ahead log (WAL) stream on the same machine. The rest of this article examines the traditional SQL transaction path and its tradeoffs, and then shows how Aurora breaks these layers apart into cloud-native services for greater throughput and resilience.

GCP Cloud Pub/Sub Replay: Seeking to timestamp & Seeking to snapshots

/ by 

Google Cloud Pub/Sub Replay (Pixabay)
Google Cloud Pub/Sub Replay (Pixabay) 


Let's assume, you have data pipeline deployed on Google Cloud Platform, events are published to Cloud Pub/Sub topic from publisher client, and subscribed by a data processing application, which reads data from the Cloud Pub/Sub subscription, process it and write it to BigQuery table.

Streaming Analytics in Google Cloud Platform (GCP) - Building Data Pipeline with Apache Beam

/ by 

Building Apache Beam Data Pipeline
Building Apache Beam Data Pipeline (Source: Pixabay) 


In introduction article of this series Streaming Analytics in Google Cloud Platform (GCP) - Introduction, we have seen the basics of streaming analytics, its importance and example uses cases, and short introduction about the Google Cloud Services, we will be using to build Streaming Analytics system in Google Cloud Platform.

Streaming Analytics in Google Cloud Platform (GCP) - Setting Up The Environment

/ by 

Streaming Analytics in GCP
Streaming Analytics in GCP (Source: Pixabay) 


Hello everyone, in the previous article Streaming Analytics in Google Cloud Platform - Introduction, we have covered what is streaming analytics, what services we are going to use and a quick introduction to each service. In this part of the series, we will begin the installation of SDKs, and libraries and set up our environment.


Streaming Analytics in Google Cloud Platform (GCP) - Introduction

/ by 

Streaming Analytics in Google Cloud Platform
Streaming Analytics in Google Cloud Platform (image source - pixabay) 

From data-to-decision in real-time 

Welcome to our new series on building a streaming analytics system in the Google Cloud Platform!. Let's begin with a quick introduction. Streaming analytics is the process of analysing data in real-time as it is received. Streaming analytics enables an organisation to gain insights and make decisions based on the most up-to-date data, in real time. This is crucial for business as it allows organisations to respond to changes and opportunities in a timely manner.

Real-time Monitoring and Log Streaming in Google Cloud Platform (GCP)

/ by 

Monitoring Dashboard with charts - credit pixabay 



Getting insights into performance, availability and health status of infrastructure and application is very critical for building and managing reliable systems. When we are dealing with clusters of instances, it is becoming very challenging to collect, aggregate and derive actionable insights from data in real time. There are monitoring tools available to address this challenge, both open source and commercial products, we are going to discuss about how to achieve real-time log streaming analytics and monitoring in Google Cloud Platform.

Subscribe to: Posts (Atom)