Amar Prakash Pandey

Amar Prakash Pandey aka @amarlearning

I’m a senior data engineer at Citi in Bengaluru, with 8+ years of building and fixing data systems. Most of my days go into Spark. I’m usually the one who gets pulled in when a job is slow, expensive, or failing every night, and I’ve cut resource usage on some of those by up to 90%. Then I build the tooling so nobody has to dig through those logs by hand again.

Before Citi, I spent four years at Sahaj Software owning terabyte-scale ad-tech pipelines on AWS EMR and Airflow, and before that, three years at TCS building real-time pricing APIs for Cummins.

Outside work, I’ve contributed to Apache Airflow, was a maintainer at CRI-O and DuckDuckGo, and did Google Summer of Code with TEAMMATES at NUS. When a problem annoys me enough, I turn it into a tool, like spot-optimizer (11k+ downloads) or uv-bundler, and I write about Spark internals, databases, and functional programming.

Always happy to talk data, open source, or side projects. Say hello on GitHub, Medium, or LinkedIn.


Right now, I'm building Litmus, an interactive playground for practicing system design. Instead of handing you a finished architecture, Litmus gives you the requirements and an empty canvas to build your own system.

You'll tackle the constraints that shape real-world systems: 10K requests/sec, 100M DAU, p99 latency targets, petabyte-scale storage, fault tolerance, and more. Change the workload, experiment with different architectures, and discover what it takes to build systems that scale.

Litmus / Learn when to use a component, not just what it is try it →

Lore / Decision memory for Claude Code uv-bundler / Linux-ready Python builds from any machine Spot Optimizer / Picks the cheapest stable AWS Spot instances GeohashViz / Visualize geohashes in bulk

Your AI Agent Forgets Everything. Lore Fixes That. Functors and Monads: Wrapping Your Data (Not Your Head) Around Them Currying and Partial Application: Shaping Intent Choosing Functional Python (Without Forcing It) Spot Optimizer ≠ Spot Fleet: decide smart, then launch View all →