I checked my analytics on a Tuesday morning. Flatlined. Two days of zero digests sent. No errors, no crash reports, nothing in the logs.
Here’s the thing about pipelines that run on a schedule: when they break, they break quietly. There’s no user screaming in your DMs because your email automation failed — the whole point of the automation is that nobody’s watching it. So it failed, and the silence itself was the cover.
The root cause was so small I almost laughed when I found it.
The Setup
I had built an email pipeline on Google Cloud. Cloud Functions, Pub/Sub, Vertex AI for categorization, and a scheduled job that sends a beautiful morning digest to my inbox. It had been running smoothly for weeks. Categorizing newsletters from personal mail from server alerts. Summarizing. Delivering. All without me lifting a finger.
Until it stopped.
I didn’t notice at first. Day one, no digest — maybe I missed it. Day two — hmm. Day three I actually opened my inbox and scrolled past the spot where the digest should be. Empty. That’s when the knot formed in my stomach.
The pipeline hadn’t crashed. It had just… stopped sending.
The Silent Failure
Google OAuth access tokens expire after one hour. That’s by design. Your app gets a token, uses it for sixty minutes, then exchanges a refresh token for a new one. The flow is standard. Documented. Boring. Every tutorial handles it the same way — get a token, refresh it when it expires, move on. Nobody tells you what happens when the refresh itself fails.
I had implemented it correctly — or so I thought. The refresh logic was there. The token store was persisting. Everything looked fine in the code.
But here’s the thing about refresh tokens: Google gives them a lifespan. If your app is in testing — and let’s be honest, most personal projects stay in testing forever — those refresh tokens expire after seven days.
Seven days.
So here’s what was happening. Week one: perfect. Tokens fresh, emails flying, digests landing. Week five: still running, still working — because you’d never know by looking at the dashboard.
Week two was the critical moment. The refresh token expired. My code tried to exchange it for a new access token. Silent failure. No exception I was catching. The Gmail API just returned a 401 and my pipeline kept marching forward like nothing was wrong — because the way I’d written it, a failed send was just a skipped email. Better to miss one than crash the whole pipeline, I thought. Better to be resilient.
Resilience without visibility isn’t resilience. It’s denial.
The Debug
I traced the bug backward from the symptom. No digests. Check the Cloud Functions logs. Functions ran fine. Check the Pub/Sub queue. Messages published successfully. Check the Gmail API calls. 401. Unauthorized.
Wait. Unauthorized? The token was valid last week.
That’s when I fell down the OAuth rabbit hole. Google’s documentation is thorough, but it’s also sprawling. You have to know what you’re looking for. I didn’t know test-mode refresh tokens had a shelf life. Why would I? The whole point of a refresh token is that it lasts.
The Fix
The fix was trivial. Move the app to production. One toggle in the Google Cloud Console. The refresh token I already had would now live indefinitely (unless revoked). Done in fifteen minutes.
But the fix wasn’t the point. The point was that I had built a system designed to be invisible, and its failure mode was to remain invisible.
Think about that for a second. Everything we build as developers eventually breaks. Code rots. APIs change. Certificates expire. The question isn’t whether your pipeline will fail. The question is: when it fails, will you know?
I had monitoring. I had logging. I had error handling. I had all the things the blog posts tell you to set up. What I didn’t have was a check for the thing I assumed would never happen — the auth layer quietly dying. Because auth is infrastructure. It’s the thing that’s supposed to just work. You don’t wake up worrying about whether your front door will open. You assume it will.
The Real Lesson
I ended up adding a health check that sends a heartbeat email every day — an empty message to myself with a timestamp. If the heartbeat doesn’t arrive, I get an alert. It’s stupid. It’s elegant. It catches the exact class of bug that would otherwise live in the dark for weeks.
This is the same pattern I keep seeing in every system I build. The most dangerous bugs aren’t the ones that crash your app. Those are loud. They page you at 2 AM. They get fixed. The dangerous ones are the ones that turn the volume down gradually until you stop noticing the silence.
And it makes me wonder: what else in my life is running on an expired token right now? What assumption am I making that hasn’t been tested lately?
Maybe that’s the real takeaway here. Not about Google OAuth. Not about token lifetimes. Not even about monitoring. But about the invisible contracts we make with our tools — and the quiet ways they can break while we’re not looking. The things you trust the most are the things you check the least.
I fixed the bug in fifteen minutes. But I spent a lot longer thinking about what else I’m not checking.