All insights

Build a useful creative testing matrix

Plan which ads to compare and why: separate distinct concepts from cosmetic variants, follow a worked example and give the launch team a clear handoff.

Kevin Dosanjh
Co-founder / CMO, AI Vidia · Updated
Overhead view of a testing grid of blank white cards in seven columns on pale oak, several cards in burnt orange and ink blue, a hand placing one card out of line.
On this page7 sections
  1. 01Start from one decision
  2. 02Use a simple structure when it is enough
  3. 03Agree measurement before delivery
  4. 04Order the next batch from what you learned
  5. 05Fill in the matrix before production starts
  6. 06Turn the review into a decision
  7. 07What the team launching the ads receives

A testing matrix is an overview of the ads you want to compare and the differences between them. It makes production easier to plan and the result easier to read. The number of cells matters less than the question each cell is meant to help answer. If your best ad is wearing out, a matrix helps you turn what it proved into genuinely new concepts instead of more copies of the same one.

Choose the size by your test question and capacity. Four variants can hold a simple comparison, while 35 can cover several products or markets. Neither number is a platform limit or a guarantee of learning, and the larger plan is only useful when the extra cells have a clear job.

Start from one decision

Choose what the team wants to know. Should the product be shown in use or on its own? Is a practical argument more relevant than an argument about looks? Write the question down before you spread ideas across cells.

Separate a new idea from a small change. A new text colour is rarely the same as a new selling argument. Both can be worth testing, but they answer different questions and should be named accordingly.

A new concept changes the reason to buy or the story that shows it, so it asks a new question. A new crop, opening line or caption on the same story is a variant: useful for keeping a working ad fresh, but it still tests the same idea. When your best ad is wearing out, write down what it seems to prove before briefing anything new. Then either keep that reason and tell it through a different situation, or test the next reason to buy. The guides to spotting creative fatigue and to a weekly creative cadence cover the timing.

To see which reasons to buy your live ads already give, paste them into the free Creative Blind Spot Map. It shows which buying questions they answer, which they repeat and which of your priorities no ad answers yet.

Use a simple structure when it is enough

A hypothetical example is two product arguments with two openings for each. That gives four variants. It is an illustrative structure, not a recommendation for every account. If both parts change at once, be careful about explaining which part caused the difference.

A larger plan can hold more products, languages or situations. Expand only when the extra difference is relevant and there is capacity to produce, approve and test it. Otherwise more cells can spread a limited budget without giving a clearer answer.

Agree measurement before delivery

Keep the reference ad, variant names, period and any significant changes along the way. Do not let today's best number rewrite the decision rule afterwards.

Before launch, agree four things with whoever launches the ads: which observations count as comparable, who approves each variant before it runs, what else changes in the campaign at the same time, and who reads the result and decides the next brief. Exact settings and thresholds belong in the internal test plan, not in the matrix.

Record raw counts and differences in delivery. If some variants are barely shown, the matrix has not necessarily answered anything about them. "Unresolved" is a useful conclusion when it is accurate.

Need new ads?
Describe your product and what you want to test.
Send your brief

Order the next batch from what you learned

Review what the test showed and what is still missing. It may be right to simplify the message or to test a completely different argument. It is not always necessary to make more versions of the ad that got the most clicks.

Keep an eye on production cost too. See the cost guide, and check the planned batch against what is included in AI video ads.

Fill in the matrix before production starts

In a hypothetical example with a bag, the two arguments can be organisation and the properties of the material. For each argument, choose two openings. One shows the product straight away, the other starts with a relevant situation. Each of the four cells needs its own name and a short script, so the differences can be seen in the finished material.

Check that both arguments have a real basis. If a material is not documented as water-resistant, one direction must not show a demonstration that implies it. Build every cell on a product property you can document.

Also write down what stays the same: product variant, offer, destination and the part of the video that is not part of the test. When one of these changes anyway, whoever is responsible should note it. The test can still be useful, but the explanation of the result has to fit what actually happened.

A second hypothetical: a pet-food subscription whose first-box ad is wearing out. Separate concepts can answer three different questions: what arrives in the box, how feeding fits into a morning routine, and what the approved ingredient statement means for the owner. Three versions of the same pack shot would test one presentation of one idea, not three reasons to buy. These directions are illustrations, not a promised number of concepts or tested winners.

Turn the review into a decision

A review should start with the question and end with the next action. If the organisation argument looks promising, the next test can explore a more precise demonstration. If both openings are unclear, a new shoot may be more relevant than more small text changes.

Keep what was not settled as well. Perhaps a variant got almost no delivery, or the offer changed halfway through the period. Then the next plan should remove that limitation before the team concludes the idea does not work. A clearly unresolved observation can save a whole batch of unnecessary videos.

Do not treat the completed matrix as a promise that every cell has to be tested at the same time. It can also work as a prioritised queue, where an early learning changes the later tasks. That keeps the plan easy to adjust without losing the link between question, file and result.

What the team launching the ads receives

Whoever launches the ads needs a handoff that keeps each file tied to its question. A useful sheet has, for every file:

  • Concept and question: the concept name and the customer question it tests.
  • Claim and source: the reason to buy, in approved wording, with the evidence behind it.
  • Format: length, ratio and placement it was made for.
  • Variant label: the same name in the sheet and in the file name, so results map back without guessing.
  • Constants: what stays the same across the variants of that concept.
  • Rights: where and how long the people, music and footage may be used.

How many concepts and variants you receive depends on the plan; a concept is usually 3 to 5 videos. The process page shows who does what, and the Control Test is a bounded way to put new work against your best ad.

Frequently asked questions

01How many variants should a matrix have?
Choose the number by the question, the budget and your capacity to produce, approve and test the material.
02What is the difference between a concept and a variant?
A concept is the core idea of the ad. A variant changes one described part of that idea, for example the opening or the text.
03When is a test unresolved?
When delivery, events or comparability are not enough for the decision you want to make.

Next step

Get a proposal for your next ads.

Tell us about your product, audience and ad production needs. We review your brief and reply with a proposed next step.

Send your brief

Read next