We run programs to make a difference. To open doors, change trajectories, celebrate excellence. But here’s the uncomfortable question we rarely ask ourselves: How do we know if we’re actually doing any of that?
Not in a vague, feel-good way. But really knowing. Being able to point to specific people and say “this program changed things for them.” Understanding which parts of what we do create real impact, and which parts are just… nice.
Most of us can’t. We run our programs, announce our winners, share the press release, and move on to next year. Meanwhile, someone who won three years ago might have landed their first commercial client because of the credibility we gave them. Or maybe they’re still struggling and our award made zero difference.
We don’t know, and that should bother us (it certainly bothers me since we’ve failed to do so with our past programs!)
The measurement trap nobody talks about
You decide to start measuring impact. You add questions to your application form. Create follow-up surveys. Build tracking systems.
Then something weird happens.
The moment you start measuring “career advancement,” every application suddenly claims they’re on the verge of a breakthrough. Start tracking “commercial success” and watch artistic risk-taking disappear overnight.
A competition which starts heavily promoting alumni success stories – gallery representation, international residencies, the works, might unexpectedly lose 30% of their entries next year scaring away emerging artists who thought “if that’s the standard, I’m not ready.”
The very act of measuring success can make programs less accessible (but increase the level of quality entries) – it’s a very delicate balance.
But not measuring is worse
“We think we’re making a difference” isn’t a strategy. We tend to default to thinking this way when we’re too busy, or we’re intimidated by the process of doing so.
Looking under the bonnet may be scary but it shows us what works and what doesn’t. It will give you insights on how to trim things that don’t work for your participants and increase resources on the work that makes a real difference in their lives.
Plus – stories expire. You can’t depend on that one winner from 2019 anymore. Funders want proven patterns, not anecdotes. They want to know this year’s impact and last year’s and the year before.
Impact is not built of hope, it comes to life through carefully planned action.
Numbers are nothing without stories (but both unlock funding)
Traditional impact reports are far too generic in their outlook:
- 73% career advancement
- 32% income increase
- 84% valuable connections
What does that even mean? Career advancement could be anything from “got an intern position” to “now running the Tate Modern.”
Compare that to: “A photographer was working part-time in retail while building their practice. Six months after winning our award, they landed their first commercial commission. The client referred directly to the award when they approached them. They’re now fully self-employed and just hired an assistant.”
Then add: “This photographer is one of 73% who report similar career shifts.”
Story bring the statistic to life. Statistic proves the story isn’t cherry-picked. You need both to complement each other or you are missing a critical piece of your jigsaw puzzle.
With more competition for the same funding it’s now more vital than ever to stand out for grants and sponsorships. Everyone’s claiming impact. But when you can say “73% of our participants increase their creative income within 12 months, here’s how” and back it up with real stories? That’s when funders lean in. That’s when sponsors renew for another year.
Numbers get you attention, stories get you the check.
Light-touch tracking that won’t give you (and your participants) migraines
The best impact measurement is invisible. Your participants shouldn’t feel like they’re being studied.
Start simple. In your application, ask just three things:
- Where they’re at (student/emerging/juggling day job/etc)
- Biggest challenge (money/credibility/connections/skills)
- One specific goal for next year
That’s it. You’re not running a census.
Six months later, send something like this:
“Hi [Name],
Been thinking about everyone from [Program Name] and wondering how you’re doing.
Quick question – remember you mentioned wanting to [their specific goal]? How’s that going?
No pressure to respond, but would love to hear. Even two lines would make my day.
[Your name]P.S. If something amazing happened, I want to know. If things are rough, that’s real too.”
Response rates to emails like this beat generic surveys, and with AI are easier to analyze than ever before (speaking of surveys – we’ve written about human-centered survey approaches that actually work).
Your data will be messy and incomplete
Some winners vanish. Response rates hover around 40% on a (very) good day. Success happens without you knowing. People’s lives are complicated – you’re one factor among hundreds.
This bothers people who like clean data (and who doesn’t!)
It’s best to take a step back and remember you’re not running a clinical trial. You’re trying to understand if your work matters and how to make it matter more. Even messy data brings about patterns:
- Which bits participants actually value
- What support creates lasting change
- Where you’re accidentally creating barriers
- How reality differs from your intentions
Perfect data that never gets collected helps nobody. Messy data you actually have can change everything. Besides if no one answers – that might be data in-itself (are they more likely to respond if they got value from what you do?)
What happens when you start measuring properly
Past participants engage more when you check in. Turns out people appreciate you caring beyond the award ceremony. Some become your best advocates – those ambassadors we talked about who recommend you because they’ve seen the long-term value.
Sponsors start seeing you differently. Not as event organizers but as change-makers with receipts. They still care about reach, but transformation stories? That’s what they remember at board meetings.
Your team remembers why they do this. When you’re drowning in logistics, regular impact stories are like oxygen. Reminds everyone the admin serves a purpose.
It’s best to just start somewhere
Pick three things to track. Set up one check-in email. See what happens.
If you’ve run programs for years without measuring, don’t try reconstructing history. Start now. Future you will be grateful.
The best measurement system is the one you’ll actually use. Not the perfect one you’ll design but never implement.
Imperfect measurement beats perfect ignorance” is still quotable but “Doing nothing leaves us in the dark” immediately softens any harshness. Then listing who it doesn’t serve brings it back to purpose and people.
That’s the uncomfortable truth we need to live with (and nobody wants to say out loud!)
Let us know you want us to write more content like this with a love!
Share
Guy Armitage is the founder of Zealous and author of “Everyone is Creative“. He is on a mission to amplify the world’s creative potential.
