Skip to content
Menu
All projects

AI agent skill / Creative testing

E-commerce Paid Social Creative Testing

"Which ad won?" is a harder question than it looks, because most ad sets were never a fair test. I built a skill that helps e-commerce teams plan creative tests their budget can support, and read the results without flattering them.

My contribution
Research, testing method, evidence rules and build
Timeframe / scope
Open-source skill for Claude Code and Codex. Published October 2026.
Current stage
Working release with tests. Platform rules reviewed on 1 October 2026; every example is synthetic.

At a glance

Situations covered, from a launch plan to a fading winner
6
Hook families in the creative taxonomy
16
Separate verdicts: evidence strength, business viability, next action
3

Where the idea started

Several ads in one optimising ad set isn't an A/B test. The platform spends unevenly, one ad takes most of the budget, and the report still crowns a winner. It's an easy conclusion to draw and a shaky one to scale.

I wanted a way to plan and read creative tests that's honest about what the data can and can't say.

The approach

Start from what a team already has: a product brief, the creatives, and an ad-level export. No ad account access and no API keys.

Then keep three verdicts apart. How strong is the evidence? Does it work for the business? What should happen next? An ad the platform barely spent on hasn't been tested, and a day with no purchase data is unknown, not zero.

How I built it

The skill covers six situations, from a launch test plan to "our winner is getting expensive". It tags creatives by angle, concept, hook, proof, format and offer, maps the gaps, and writes production briefs that name the proof each line needs.

A standard-library Python script does the arithmetic so the model doesn't have to. It recomputes ratios from totals instead of averaging rows, keeps different currencies and attribution settings apart, and flags underdelivery. A feasibility command shows how many outcomes each test cell can expect from the budget.

It's Meta-first, with TikTok rules kept separate. It never touches an ad account.

Where it stands

It's published with tests for the script and its files, and a fictional brand to try it on.

One limit worth knowing: current Meta help pages couldn't be reliably retrieved during the research. So the skill tells the agent to verify experiment options and attribution settings against live documentation before giving exact setup instructions.

What I learned

"Inconclusive" is a useful result. Saying it out loud costs less than scaling an ad that only looked like a winner.

Tools used

  • Python
  • Claude Code
  • Codex
  • Meta Ads
  • TikTok Ads
  • CSV exports