REF Factory

Writing the REF submission for Computing at Kent is drawing to a close, agonisingly slowly, but there we are … Now is the time to capture any insights about how it might have been done, and how it could be done next time (I'm not volunteering!). I'll write this now, and then finish revising my first publication for REF 2020: final copy is due to be with the ACM in a couple of days' time.

Kent made the decision to hold a full-scale pilot in 2012, so that we had been through the process of writing all the narrative sections, the impact case studies, and – in the case of computing and a few others – the 100 words accompanying each output, about 120 for us. This had some strong points, and it was particularly good in giving us a deadline by which we had to have written a first draft of everything, so that the last year has been a process of refining those drafts.

On reflection, though, it has – at least for me – meant that we (or I) have conflated two quite different processes:

• the process of marshalling the arguments, information and data, and 
• the process of writing that within the constraints of the 100 word limit, or the various templates.

I did have long drafts – essentially compilations of data and info – prior to the pilot, but since then, we've been working on drafts in the form of the final submission. What I'd do if we were doing this again is to keep the separation between the two until much later in the process, and work on the raw information and data until a much later point. Doing that means that we're able to get an overall view of all that might be pertinent, rather than selecting, and then re-selecting, which is what we've done. For example, I had included data on staff study leave in a draft of REF 5 in May, but that was removed in June, only to be re-introduced in November. 

The same applies to the 100 words, I think; a subject I discussed with another REF coordinator recently. Aiming to get all the pertinent information collected for each potential paper (with a few prompts about different kinds of info), and to keep that up to date and reviewed, would allow the final 100 words to be generated close to the deadline, rather than evolving over two years in many cases.

I realise that this may week be a case of the other person's grass always being greener, and that we would most likely find problems with this approach, but this is what I would try were I to do it again. In the case of the pilot submission, if we had to write it in that format, we should see it as a throw-away draft, and keep on marshalling until we write a new final draft from scratch.

Finally, the more I think about this, the more I think that this is very like a problem we come across in writing successful research grant applications. The most useful feedback on an application is at the stage where the ideas begin to be articulated clearly. One way of expressing this is as a “one pager” that summarises the essence of the ideas, the work packages, the methodology and the impact, all on one side of A4. As a reviewer it's possible to engage with this, and say “yes, but it would be so much better if you were to push it in this direction” – it's almost impossible to do this once it has been set in concrete in a complete application. In a roundabout way, that explains why I called this post “REF Factory” – Kent's well-regarded grant application training is called “Grants Factory”, a name coined by Andrew Derrington.