This page contains an edited selection of feedback received about references over the course of this project, along with our responses.
Ranking
- Personal assessment rankings are too subjective to be of use in many cases
- Ranking by academic referees is very useful and relatively reliable
- Even personal assessments from professional references can be ranked intuitively or even on the basis of objective comparative data
- Suggest that they're specifically asked to explain any difference between their personal assessment ranking and official class rank
The ranking in the first release has been designed with ease of use by those completing the references in mind first and foremost. We are aware that the ranking is used by the funding bodies and we are working with them to ensure that adequate guidance is produced by the next funding round. However, there are many different opinions within the two groups (those writing and those reading the references), and so finding an optimal solution will take time. We will continue to gather and discuss feedback as it comes in.
Evaluation Grids
On whether more granular rankings (rate the applicant on some set of characteristics on a scale of 1 to 10) are helpful:
- [Some UK University]'s form is awful - all those artificially precise numerical responses
- As part of ref comments, I would prefer also to see a "evaluation grid" sim. to [The Same UK University]
As mentioned earlier, there are a wide range of opinions on this. For the time being we will stick with the simple options for ranking, but will revisit this if certain departments feel strongly about having an 'evaluation grid'.
Guidance
The only problem with a generic statement is that specific requirements of the proposed course can not be highlighted e.g. for a PhD course independent research ability is particularly important whereas this might not be as important for a taught MPhil. So it relies on the Reviewer figuring this out.
We will be looking into allowing course-specific guidance for referees when we move into the application form development.