Image by Sarah Goodier.
ROER4D Evaluation Advisor Sarah Goodier discusses her attendance at the 12th European Evaluation Society Biennial conference in Maastrict, Netherlands.
Sarah Goodier, ROER4D’s Evaluation Advisor, attended the 12th European Evaluation Society (EES) Biennial Conference held in Maastricht, Netherlands, from 26 to 30 September 2016. This conference brought together members of the EES and other evaluators from around the globe with an interest in the conference theme of evaluation futures, focusing on connectivity, innovation and use.
The conference covered the major areas of interest for those undertaking evaluation, including ethics, methods, use and communication. There were also several overarching themes, such as gender and sustainable development, running through the conference presentations’ strands. The mix of presentation topics provided a good overview of how evaluators are currently responding to various different views on the importance of evidence, with several embracing a more realist approach focused on what works for whom in what contexts. One of the five major presentation strands was about evaluation methods and research, with two sessions specifically focused on social network analysis (SNA).
Sarah presented on Day 2 of the conference on evaluating network growth: a case study using SNA. This presentation covered a component of the evaluation work looking at networking in the ROER4D project. As ROER4D has clusters of research activity across the global south, consisting of over 100 researchers in 26 countries, building an audience network of those interested in the project’s progress and findings is a key consideration. Part of the project’s network building strategy involved the use of social media, including Twitter, to increase the project’s reach and engagement. This led to the formulation of a key evaluation question considering to what extent this building of a network has occurred.
From discussion with the ROER4D team, the uses of evaluation results generated with regard to the Twitter activity were established as being: 1) to improve ROER4D’s communications strategy and 2) to have proof of concept about ROER4D’s networking model on the social media channel Twitter. Three appropriate tools were selected for data collection: Twitter Analytics, Twitter Archiving Google Spreadsheet (TAGS), and the Network Overview Discovery and Exploration Add-in for Excel (NodeXL). Through making use of these tools it was possible to assess both the amount of engagement at various time-points and the breadth of the audience and the network. Overall, these three sources provide data on audiences as well as engagement (account activity and interactions), both over the history of the Twitter account (interactions – over time) as well as at key time-points (interactions – 7-day window), and enable an archive of tweets to be kept for any further analyses. As part of the ROER4D evaluation, data is generally collected monthly or when the team requires an update, for example, during or after a conference. Updates on findings and recommendations are presented to the ROER4D team periodically in order to best facilitate use of the evaluation results. The same methods and the tools workflow as presented here could feasibly be applied to other research projects or programmes that are using Twitter.
Sarah’s full presentation is available here.