NewsCatcher’s Own Blog Was the Manual for AI Agents

Last year, I finally automated my big cats news pipeline. I have been searching, sometimes translating, and publishing big cats news since 2007. I quickly found I didn’t have the resources to maintain and extend it myself. Then, at my day job, I started working with AI agents and saw I could manage them the way I manage junior developers. That’s what made my personal project sustainable as a one-person operation, with AI help.

I rebuilt one specific layer: the search itself. I’d hand-coded the rest of the pipeline through 2025: translation, analysis, edit, re-analysis. It was running fine. The search came from NewsCatcherAPI’s free tier for non-commercial projects. The free tier renews every few months, but only if you ask for it and show that the project is still alive and still non-commercial. On 1 September 2025, my access ended. I hadn’t asked them to renew it. By that point I was already overwhelmed by other things and hadn’t returned to the personal project for months.

At the start of 2026, my NewsCatcherAPI subscription started working again for a few days. I don’t know why. Some articles came into my review queue and reminded me the project still existed. By then, I’d been using AI agents at work every day, and I’d started using them for my own coding too. I also remembered a couple of articles on the NewsCatcher blog about how to build a service like theirs from scratch. I decided to direct an AI agent to read those articles and rebuild the search.

In 2025, I built an AI agent system without knowing that’s what it was. There was no tutorial. The pipeline runs several AI models in cycles, switching providers when one fails. An editor model and an analyzer model send the article back and forth until they stop finding problems, or they hit the cycle limit. At the end of 2025, I started working with AI agents at my day job. I lead a Tier-3 support team, and most of my time goes to bug investigations and managing a product in production. Once I’d learned the vocabulary, I recognized what I’d built.

I pointed the AI at the NewsCatcher blog and asked it to do wider research too. It came back with what I needed for the search, including a NewsCatcher post that figured out the Google News RSS query parameters Google itself doesn’t document. From there, I told it to implement the search. The hardest part was decoding the URLs Google News returns. Google doesn’t return the article URL directly; it returns a redirect with an encoded payload, and reading that payload isn’t trivial. The AI worked out a decoder that handled the encoding correctly.

Other small things came up over the next few weeks. I had to add a cleaning step on top of the search, because the free sources return raw HTML with boilerplate, ads, and navigation that NewsCatcher’s paid product had been stripping out for me. There was a Google News query bug too, where wildcards in the positive search terms silently returned zero results for a few hours. I recognized the pattern from production at work, and the fix was straightforward. GDELT also had a rate-limit problem where the retry logic was making things worse; the logs showed less than 5% of retries succeeded. The fix was to remove them. The new search fit into the 2025 article-level pipeline. But the pipeline itself did not stand still while I worked on the search.

Logs are how I investigate everything at the day job. I brought the same discipline to the personal project. I told the AI to add good logging from the start, and to analyze the logs and report back to me. That was how I found that many articles were failing — sites that returned nothing useful from a normal HTTP request. I told the AI to research a self-hosted scraper and add it to the fallback chain. The AI pushed back against using a headless browser. It said browsers were heavy, slow, and one more thing that could break. I let it try the lighter version first. Then I watched what came back, and the lighter version wasn’t enough. I told it to add the headless-browser fallback anyway.

All of the 2026 pipeline changes were AI-coded under my direction, the same way the search rebuild was. Flip-flop detection got the most iteration. I had AI implement it in 2026 because the analyzer and the applier were running too many cycles on articles where the two models could not settle. I pay for the tokens the pipeline uses, so I notice when iteration counts go up on my bill before I notice them in the logs.

The first version hashed the whole article and looked for full reversions. AI moved it to per-sentence hashing to catch partial reversions, and then to correction-pair detection for the smaller flip-flops that happen inside sentences. When a flip-flop confirms after two checks in a row, a separate AI call picks the better of the two competing versions and sends the article back to the analyzer for a final pass. That keeps escalation as a last resort, not an automatic next step. AI reported the iteration counts; I decided what to fix next.

The applier needed its own fixes. Stale Apply jobs were colliding with running Analyze jobs, trailing characters were accumulating across edit cycles, sentence-count guards were missing, and the diff verifier — the step that checks what changed between two versions — did not have the original corrections it needed to handle merge and split edits.

Running the new search every hour also exposed problems in the queue worker layer. AI built a worker pool that scaled itself, a kill-recovery mechanism for long jobs that the hosting environment kept stopping, and a fix for worker startups that had been failing silently because the wrong version of PHP was being run. None of this is novel work. Every production system that runs jobs at scale solves these eventually. Solving them on a one-person personal project meant using the same patterns I use at work for a production product.

NewsCatcherAPI’s product is the strongest of its kind. I’ve compared it to other services, and the others are weaker. The free access for non-commercial and academic projects is generous. For commercial use, I’d pay for it myself. NewsCatcherAPI is a Ukrainian-founded company, and since I’ve been searching news for 19 years, I love that Ukrainians built a service like this. My own project publishes its news in Ukrainian, which is the whole point of running it at all. This rebuild isn’t a complaint about NewsCatcherAPI. They return more news than one person can review by hand, and the rebuild fits the size of a project I run alone to keep my skills sharp.

The project is sustainable again. The new search runs every hour and the AI agents flag what needs human review. I review what they flag the way I’d review what an assistant brings me. If you’re managing AI agents yourself, I’d be interested to hear how you handle it.