How to review creative software so the test is designed before access is granted

A defensible review says exactly what evidence was examined and reserves scores for criteria that were actually observed. Product-page claims, contract terms, a hands-on task and campaign outcomes belong in separate columns. One cannot stand in for another.

What to take away

Define the buying decision

Write the user's job, workload and unacceptable failure. A local retailer making occasional static assets has a different decision from a regulated organisation feeding approved data into thousands of variants.

Record required formats, collaborators, approval lanes, destinations, retention and exit needs. Our buyer personas article shows how to write them without guessing.

Set gates before preferences. A product that cannot meet a mandatory rights, security or data condition should not recover points through attractive templates. For public-sector buyers, the Technology Code of Practice starts with defining user needs and says technology should work with existing infrastructure and adapt to future demands. Private buyers can adapt both questions.

Build an evidence ledger

Give every claim an exact source, capture date and status:

For example, a vendor claim that approved templates with locked elements and brand kits keep assets, fonts and colours on brand in Adobe Express is evidence that the vendor publishes the claim, not proof that a particular plan contains the controls for an English buyer or that they work in the buyer's environment. The best providers article sets out how to shortlist by use case.

The same discipline applies to security answers. The NCSC guidance on choosing a cloud provider describes a confidence scale: at the low end you rely on a straightforward promise from the supplier with no verification, and at the high end on independently assured components. Use that distinction rather than awarding a tick for a polished white paper.

Design the test before access

Prepare a fixed specimen: approved source pack, awkward long copy, restricted image, missing data field, collaborator roles and required exports.

Specimen task example: take a 12-word headline from the source pack and add a 90-character awkward line. Place the restricted image in a square crop, leave the campaign code field blank, assign editor and approver roles, then export one 1080 by 1080 PNG and one CSV of alt text.

State the permitted help, device, browser, timing rule and pass conditions. Use the same ordinary task for each eligible product. The product comparison article shows a specimen across several products.

Record the starting state, actions, observed output, failure and intervention. Do not score a feature that the reviewer could not access. A vendor demonstration may answer a question, but it is not hands-on buyer evidence. See software selection for how a task specimen replaces a sales demo.

Repeat a corrected path once, because recovery behaviour may expose a control that the ordinary run never reached.

Score with evidence caps

Use a short scale with anchors. Zero might mean the requirement was not met; one, unresolved or documented only; two, observed under the test; three, observed twice including an exception.

Worked example for the criterion restricted image export: a supplier page alone gives score 1 and the evidence cap is 1. A documented test export gives score 2. A repeat with an exception or rights change gives score 3. A failed export gives score 0.

Publish weights before results. Report raw findings as well as totals because equal scores can conceal different failures. If a legal, privacy or security gate remains open, the verdict should remain conditional regardless of arithmetic. Our data protection article shows where creative data travels.

Review the review

Audit the review against the ledger: confirm the declared product version, include rejected evidence and record any undisclosed relationship.

Where public ratings are cited, remember that the Digital Markets, Competition and Consumers Act 2024 treats a fake consumer review as one that purports to be based on a genuine experience when it is not. A qualified lawyer must assess its application.

End with one of four states: suitable within the tested boundary, conditionally suitable pending evidence, unsuitable for a stated requirement, or not enough evidence. Set an expiry triggered by material product, contract, law or buyer-workflow change. The reviews and comparison methods article covers scope declarations.

Before you act

Common questions

What should a review record for each claim?

An exact source, capture date and status. Status options are a supplier statement on a current first-party page, a contractual commitment, an independently assured fact, an observed result from a documented test, or an unresolved question.

How should a reviewer score a feature they could not access?

Do not score it. Zero means the requirement was not met, one unresolved or documented only, two observed under the test, three observed twice including an exception.

What states can a review end with?

Suitable within the tested boundary, conditionally suitable pending evidence, unsuitable for a stated requirement, or not enough evidence. Set an expiry triggered by a material product, contract, law or workflow change.