Having a clear understanding of how you will make your selection before submissions open helps you define the information you need from candidates, reduce enquiries and plan your schedule better. It is also vital should you be inviting external parties to help you make your selection.
Knowing how you will select successful candidates will also help you better communicate what you are looking for in your guidelines and get the best out of every application made to you.
This article covers:
- Calculating how much effort your selection will take
- Selecting in rounds
- Inviting judges to help select candidates
- Anonymous judging
- Running a public vote
Common Mistakes
When crafting your competition, you are likely to focus more on the next actionable step (e.g. submissions opening) and less on the processes that come afterwards. Doing so will likely lead to problems later on. Here are a couple of examples:
- Missing data to allow you to make your selection.
- Getting flooded by emails asking you when the results will be published.
- Applications not fitting your exact needs.
- Processing too many applications in too little time.
- Judges finding it difficult to score so many unexpected entries
Focusing on your selection first will save you valuable time and give your stakeholders a better experience.
Working backwards
When planning your new programme, work backwards through your process. Start with what kind of candidate will be successful and work back towards the process of submitting. To reap the benefits of foresight, you need to do so before you launch your programme.
Questions worth asking yourself, in order, are:
- What criteria do you need for someone to be successful?
- How are you going to select them? (e.g. number of Rounds)
- When are you going to pick successful candidates?
- Who is going to pick successful candidates? (and in which Round?)
- What information do you need from candidates to make your selection?
Fostering diversity & inclusion
Answering all the questions above will allow you to review your process and route out any systematic discrimination early (e.g. are your selection criteria fair? Are you asking for too much? Are those making the selection diverse and impartial?)
Calculating effort
The more popular your opportunity is, the more effort it will be to make your selection. If this is your first, it may be hard to model how many submissions you will get. Planning for the most demanding scenario will allow you to cope regardless of the success of your competition.
You can calculate how much effort is required by doing the following:
1. Establish how many submissions you will get. Picking a number isn’t easy (especially if this is your first), so instead, think of it as a range.
e.g. between 1,200 to 1,800 entries
2. To be safe, add 20% to your upper range.
e.g. 2,160 (1,800 x 1.2)
3. Think about how long it will take for someone to process one application. If you are unsure, do a dress rehearsal. Create an example submission (make it as legitimate as possible) and give it to someone with your criteria for selection. Then time them to see how long it takes to make a decision.
e.g. 1 – an image & description could be processed in 10 seconds
e.g. 2 – a short film and five questions could take you 10 minutes
4. To calculate the total effort for processing entries, multiply your maximum applications by the average amount of time it will take you to process them. So in our examples above, taking an image submission that would be 2,160 x 10 seconds = 12 hours of processing (assuming an average workday of 8 hours – it would take a day and a half of pure processing).
e.g. 1 – 12 hours = 2,160 x 10 seconds (1.5 days with an 8 hours workday)
e.g. 2 – 120 hours = 2,160 x 10 minutes (15 days with an 8 hours workday)
Knowing how long processing the maximum amount of applications will take you allows you to plan for sufficient time to do so. This may push you to rethink your process to reduce the time taken to go through each application.
Filtering in Rounds
You may need to consider selecting candidates across rounds if you:
- Expect more than a couple of hundred submissions.
- Want to guarantee the quality of the submissions going onto your judges.
- Want to manually check entries fulfil specific criteria before moving them onto the next stage.
- Want to mix all the above (e.g. you could have a Round for each)
As seen in the previous section, it can take a long time to process submissions. Breaking your decision making into Rounds allows you to reduce the time taken processing applications in earlier Rounds and spending more time per submission in later rounds. An ideal structure for large programs.
Rounds allow you to select the entries proceeding onto the next step giving you control over how many works judges need to score (no one like discovering they have to do ten times more work than expected!).
Once you have established if you want any rounds, draft a [quick timeline of dates] in your process. Remember to give your judges and yourselves plenty of time.
Fostering diversity & inclusion
Rounds allow you to view a more manageable snapshot of the candidates going onto the next stage and question if unconscious bias came into play when making that selection.
What needs to be submitted
Asking for large amounts of information when people submit will seriously slow down your selection process. Asking for too little could make it hard for you to differentiate successful candidates.
Rethink what information you need from candidates and keep it to an absolute minimum whilst allowing you to make a meaningful decision. You may also wish to hide answers to specific questions from judges to lighten their loads.
If your first Round depends only on a couple of checkboxes, this will be quick. However, if you have twenty text boxes to read through, this will slow you down radically. If you expect many submissions, a good rule is to keep the process lighter for earlier rounds and gradually go into more detail for later rounds.
Fostering diversity & inclusion
The data points you focus on may lead to systematic discrimination (e.g. Education). Review each one and think about the repercussions of using them as part of your selection process.
Inviting Judges
Inviting judges allow you to:
- Share the workloads internally at earlier rounds
- Reduce bias by having multiple people help with the selection
- Inspire candidates to submit with big names linked to your competition
- Represent your community in your selection process
Including members of your organisation and partners in earlier rounds allows you to open dialogue around the work submitted to your programme. People will often feel proud to be asked and enjoy experiencing the impact of the programme you are running.
Getting external members of your communities and more recognisable names to take part is particularly useful for later rounds. You should not be using their time to qualify hundreds of submissions (e.g. for compliance, meeting specific criteria…); instead, you should aim to capture their individual opinions on the best entries submitted.
Remember, inviting external judges requires a very clear ask. Do not swamp them with work. Be sure to have a Round before theirs so you can control the number of submissions they will have to go through.
Fostering diversity & inclusion
The more diverse your judging panel is, the less biased your selection is likely to be. Judges may have little time for you at present. Making the process as quick as possible will be the difference between a high-level judge accepting your request and declining it.
Judging criteria
Having judges is one thing. Making sure the judges know what is required of them is another. What should the judges be looking at when making their decision? What makes for a successful candidate?
Drafting a shortlist of criteria and how important they are in your decision-making process will allow you to formalise how you will select successful candidates. Criteria could be:
- A simple Yes, No, Maybe
- A total score out of ten
- A sum of scores relating to specific criteria (e.g. eight points for Originality, twelve for Execution, thirty for Purpose…)
Remember that each decision you ask a judge to make will slow the process down, so try to keep criteria to a minimum in earlier rounds (e.g. Yes/No/Maybe) and allow for richer criteria in the later rounds (e.g. A score out of 10 for Originality, Execution, Purpose). A good rule of thumb is 5 is an ideal maximum amount of criteria (8 at a push).
Another benefit of knowing your criteria before your submissions open is that you can share them in your guidelines allowing candidates to make submissions that better match your requirements.
Anonymous selection
Making entries anonymous is a great way to reduce judges bias. It is worth noting that this is only really relevant where the artist does not feature in their work (e.g. performance, self-portraiture, etc.) and requires to think carefully about what the candidate will be submitting.
Ensure all information that can identify the candidate is not made available to judges. You may also want to warn candidates as they submit to make sure things like artist statements do not include information that could directly identify them.
Fostering diversity & inclusion
Not knowing who submitted can go a long way in reducing bias when judges are processing submissions (e.g. listening to music without knowing who is playing it).
Public Vote
Public voting can give your communities a voice, but it can also end up being a popularity contest. Depending solely on candidates to share a link to vote for their work will likely measure how sociable a candidate is, not the merits of their application.
If you wish to capture the voices of specific communities, you will need to actively drive public votes through these groups yourself. This requires carefully planned communications with all your stakeholders during the voting period (judges, partners, community groups, etc.).
You might also consider a sign/screen to remind people to vote as they visit your building (if you have a physical space).
It is also worth noting that rewarding the candidate chosen by the public with large cash prizes will incentivise them to cheat. The rise of increasingly sophisticated click farms makes catching cheaters increasingly hard. Balance the incentives for the winners of the public vote. They should be valuable enough for the candidates to take part but not enough to pay for a click farm with their winnings. To put this in perspective, a thousand likes on Instagram cost as little as $50.
Fostering diversity & inclusion
Public votes can give you a false sense of being more inclusive. If voting turns into a popularity contest, it may well be the opposite. (e.g. favouring extroverted candidates over introverted ones or those with more resources).
Conclusion
Thinking about how you will make your selection too late in your process is likely to lead to issues that will waste a huge amount of your time later on. In the worst of cases, it could lead to souring relationships with your candidates, judges and stakeholders.
Each minute you spend planning your selection will save ten later on and guarantees everyone who takes part has a good experience.
Share
