Unanticipated impact …

The latest incarnation of research evaluation in the UK, REF 2014, has taken the radical step of assessing the impact of research, roughly interpreted as "the effects that the work has had beyond academia" (that's my paraphrase, you can get chapter and verse at the REF website). I call it radical because it's untried - apart from a pilot -  and counts for 20% of the overall assessment, which you could identify with 20% of the money that's begin allocated on the basis of the results of the REF.

Whatever misgivings I might have about the mechanism, it's interesting to try and find out what effect your own work has had, and so we've been looking at what impact Wrangler has had since the project to build this refactoring tool for Erlang began in 2005.

The development of Wrangler was first funded by EPSRC, and then the EU's Seventh Framework Programme (FP7) in the ProTest and Release projects. This kind of European project encourages industrial / academic crossover and so one level of impact comes from Wrangler being used in industrial collaborators in these projects, including Ericsson, Quviq and LambdaStream. Moreover, some of them continue to use and develop the system after the end of the project.

What's fascinating is to see the effect of making the project open source and putting it on github, a community collaboration site. This means that anyone can contribute to the project, rather than it being restricted to the project team. Github provides a great set of tools for visualising what's happening on your project, and for Wrangler the best seem to be the Network Graph, showing all the branches, commits and merges:

and the impact analysis, showing the impact of what individuals have contributed:






which together show us that we've had a collection of contributors, some of whom we know - for example adapting the tool to work with other tools - and others who have pitched in to help to add what they needed - e.g. updating the system to support a parallel make. So, opening up the system in this case seems pretty clearly to have added to its overall impact, and take up, and so we've got some evidence to provide as a part of the REF submission …

[One thing that it seems harder to find out in github is the number of downloads, but it looks like the github API provides an answer … more on that anon.]