TL;DR
- A volunteer cat rescue posted kittens on Instagram constantly and got likes without adoptions, so the account’s data was pulled with an Apify scraper and handed to Claude.
- The cause was structural: the bio link that every caption pointed to led to a donations page, leaving the adoption form unreachable from Instagram.
- A comparison rescue of similar size had three times the followers on thirty percent fewer posts, following one cat’s story across posts, ending each with a single clear action, and celebrating every adoption.
- The scrape cost about $0.04 and an afternoon, and the method only suits measurable questions: a vague “should we rebrand” has no answer in the data.
In plain English
A scraper is a program that visits a public web page and collects what is on it in a structured form, so profiles, posts, likes and comments arrive as data you can count rather than as a feed you scroll. Apify is a marketplace of these programs, just over 44,000 of them at time of writing, including one that reads a public Instagram account. It was pointed at the rescue’s last fifty posts, then run again against three comparable rescues, and the raw numbers given to Claude to look for patterns.
Apify also exposes its whole catalogue through an MCP server. MCP is a standard way of handing an AI agent access to outside tools, so the agent can run the scrape and read the results back inside the same conversation rather than you running it yourself. The caution is about when to bother: scraping to confirm something you already believe is a slower guess.
Stop guessing and pull the actual data
Don’t have time to read this week’s issue? Why not copy/paste it into your AI agent and ask it for insights?
I help out a small, volunteer-run cat rescue. They do good work, and they post on Instagram constantly: kittens looking for homes, a couple a day. The posts get likes, but the cats don’t get adopted. Which also means that the kittens I’m fostering are…never leaving. So why isn’t any of their social media activity turning into adoptions?
Instead of guessing about their social media strategy, I pulled the actual data, with Claude’s help, using a tool called Apify.
Apify is like a giant app store of (at time of writing, just north of 44,000) robots that can visit websites on your behalf, collect information, and plug that data into your tools and AI agents so you don’t have to do it manually.
One of them reads a public Instagram account: the profile, the recent posts, the likes and comments on each. I pointed it at the rescue’s account and pulled the last fifty posts, then ran it again on three other cat rescues of a similar size to compare against. The whole thing cost under a dollar and took an afternoon. Then I handed the raw numbers to Claude and asked it to look for patterns. And the content wasn’t the main problem.
The link in the account’s bio — the one every caption points to, the one that says “fill in the form, link in bio” — went to a donations page instead of an adoption form. So: someone reads a post, falls for a kitten, taps the link to apply, and lands on a “give us money” page. The entire adoption form was unreachable from Instagram. Nobody would have caught that by scrolling the feed, because the feed looks fine. It only showed up when you traced the actual path a real person takes.
I called on Claude again to do some research of competitors in the region, to see how they were performing and what they’re doing differently. One of the other rescues with the same kind of shoestring setup, no building, fostering cats in volunteers’ homes had three times the followers on fewer posts. They weren’t spending more. The data showed what they did differently: they follow one cat’s story across several posts, they end each post with a single clear action instead of a menu of donation options, and they celebrate every adoption. You can argue with an opinion about good content; you can’t argue with three times the followers on thirty percent fewer posts.
And the total cost to run the Apify bots was about $0.04.
Diagnosis and reporting
A diagnosis stuck in my head is no use to the rescue. So I had Claude turn the data into a short report they could act on: the problems in plain language, the fixes ranked by impact, and a simple content plan to follow. Because the team works in another language, I had it produce the report bilingually in one pass — a polished version in their own language rather than a rough translation they’d have to redo. The afternoon’s scrape became a plan they could pick up and run with.
Try it on a question you’d otherwise guess at
Most of the questions a small business guesses at are sitting in public data: how are my competitors actually pricing, which of their posts land, what do their reviews complain about, where does my own funnel leak. You used to either guess or pay someone to find out. Now a cheap scraper pulls the real data and an AI reads it back to you, for about a dollar and an afternoon.
Apify’s store has a ready-made scraper for most public sites. And if you’re handing the job to an AI agent rather than running it yourself, Apify exposes the whole catalogue through an MCP server, so the agent pulls the data and reads it back without you leaving the chat.
Don’t scrape to confirm a hunch
The skill is knowing when to bother — because scraping a feed to confirm something you already believe is just a slower guess.
The test I’d use: is the question measurable?
- “We’re not converting,”
- “their posts do better than ours,”
- “is this the going rate?”
those have answers sitting in the data, so go and get them. A vague “should we rebrand” doesn’t, and no amount of scraping will settle it.
And watch out for the trap the cat rescue fell into. When results disappoint, the cause usually turns out to be something structural nobody thought to check: a link pointing at the wrong page and a form no one reaches.
- Jim
Just launched
Talking Utilities went live this month — a daily news-and-intelligence publication I designed and built end to end for a niche UK energy-sector publisher.
Read the full case study at Solopreneur Superpowers →
A tool I actually use: most of my task system runs on Notion. If you’re a startup or a solo founder, you can get 3 months of Notion Business free, with unlimited AI and no credit card, through my link. Try Notion free for 3 months →
Notion affiliate link — I get a small credit if you start a trial, and I only run ads for tools I actually use.
Signal Over Noise helps you sort the AI worth your time from the hype. If it was useful, pass it to someone who’d want it too.