When Anthropic released prompt caching for Claude API, I saw an opportunity to revisit AI agent systems due to the significant cost savings. Little did I know, I was about to coordinate code changes across two separate open-source teams to get it working.
OpenHands
OpenHands, an open-source AI code agent (32k ⭐), allows users to create entire code projects via a chat-based agent. This is a perfect candidate for prompt caching due to the prior messages remaining constant, so I gave it a go hoping to see some low cost automation. Instead, I uncovered a web of issues preventing the feature from working as intended.
When Your Dependencies Have Bugs Too
What started as a quick fix evolved into changes in two separate open-source repos. I found myself:
-
Submitting PR #1 to address initial implementation issues in OpenHands
-
Uncovering a bug in a key project dependency (13k ⭐)
-
Coordinating fixes across two separate open-source projects
-
Navigating PR approvals and merges
The result? A 90% cost reduction for EVERY user of this AI agent. We're talking game-changing savings for startups and developers everywhere.
Moral of the story: Sometimes, the biggest impact comes from being the bridge between two disconnected teams.