Innovation in information and communications technology (ICT) has ushered in a new era of monitoring and evaluation (M&E), which is marked by new opportunities for remote, real-time, participatory and advanced forms of data collection. Accompanying this shift towards the increased use of ICT techniques are new considerations for evaluators, including the organizational capacity to implement new technology and the comparative value of tools. Furthermore, ICT users must address contemporary challenges posed by ICT methods.
ICT Tools for M&E
ICT encompasses a broad set of data collection, evaluation and presentation tools. The most common ICT approaches are visualized below from the 2013 UNDP discussion paper on Innovations in Monitoring & Evaluating Results. Further explanations of each of these techniques can be found following the link.
Opportunities
The ICT tools outlined above open the door to continuous evaluation, and thereby improvements in implementation, throughout the project cycle. This marks a significant shift from the from traditional baseline, mid-term and final M&E framework. Harnessing the power of ICT tools allows for real-time evaluation, which facilitates increased communication among stakeholders. ICT approaches also allow for prompt decision-making in time-sensitive situations, as well as efficient resource management. Furthermore, ICT techniques promote participatory methods and offer more opportunities for triangulation, thereby fostering robust data collection. Finally, these innovative tools can minimize the use of resources by gathering, aggregating, examining and visualizing data in a quicker and cheaper manner.
Qualifications
Before jumping on the latest technology bandwagon, organizations must discern if it is both feasible and valuable to implement ICT mechanisms into their M&E processes. Utilizing ICT tools may not always be practical given the context of M&E activities. To guide this decision-making, the UNDP developed criteria for conducting a cost-benefit analysis of the advantages of using, or not using, ICT instruments. The reasoning primarily revolves around the capacity of the utilizing organization, including possessing:
- 1) The required expertise to efficiently and effectively implement the ICT methods;
- 2) The ability and drive to consequently adapt M&E processes to fit ICT techniques;
- 3) And, sufficient funds to make an initial investment in the ICT tools.
Other criteria assess the value of utilizing ICT tools themselves. For example, ICT techniques must:
- 1) Meet a need that cannot be fulfilled by traditional or less expensive tools;
- 2) And, the tool must be able to incentivize the participation of stakeholders.
In using these guiding questions, organizations can evaluate the advantages of adopting new ICT techniques. This ensures that the implementing actor is utilizing the most appropriate and relevant data collection, analysis and presentation tools.
Challenges
The benefits brought by ICT tools are accompanied by contemporary risks that implementing organizations must acknowledge and mitigate.
- First, organizations should be wary of utilizing ICTs in a manner that generates volume rather than depth. Collecting large amounts of data may multiply information, but it will not necessarily foster knowledge (information applied to answer questions) or wisdom (knowledge applied to guide actions). This is particularly true if the implementing organization does not have sufficient human capacity to synthesize and analyze all of the information. Data volume is fruitless unless an organization can extract depth from it.
- Furthermore, the data collected may not be representative. As previously explained, ICT tools facilitate large-scale data collection and may, in some contexts, foster greater participation from a diverse range of stakeholders. Yet the data may not be representative if the temptation to quickly gather “big data” leads to imprecise sampling. For example, “access to technology often follows existing inequalities, such as wealth, gender or geographic inequalities.” Already marginalized groups may thus be further marginalized as they lack access to the channels in which they can voice their feedback. For instance, mobile data collection in the form of text messages can easily exclude illiterate individuals, those who do not own a personal phone or mobile plan, and communities living in remote areas without reception. Finally, the nature of ICT tools and concerns about data privacy may reduce the incentives of participation. Methods such as mobile phone data collection and crowdsourcing may leave potential respondents feeling as though they are submitting information into a void, wondering: “Where does my information go? Do I see the results? Why should I participate? Is anyone responding to my input?”
- Finally, ICT users must be wary of the accompanying ethical concerns surrounding data security and privacy. This is particularly relevant with “passive data collection.” This approach refers to information that is gathered without the direct engagement, and sometimes awareness, of the data provider. For example, data exhaust captures information from the digital trail left by users through their online activity, such as cookies and social media engagement. Remote sensing collects data on physical conditions, for instance the use of satellites to assess damage after a natural disaster. In these cases, obtaining informed consent may be unfeasible. Yet this raises concerns, as informed consent is often a standard of ethical research. Upholding this principle protects data privacy and ensures that individuals agree to the collection of their data. Finally, since ICT approaches may prioritize top-down M&E rather than bolstering local capacities and partners, organizations should complement innovative technology with redoubled efforts to ensure local engagement.
In conclusion, ICT techniques offer an innovative set of tools that can enhance data collection, analysis and visualization. However, organizations utilizing these methods must first evaluate the necessity and value of these tools as opposed to traditional M&E approaches, and subsequently address the corresponding risks concerning robust data, sample representation and ethics.
References
- https://www.intrac.org/wpcms/wp-content/uploads/2017/01/ICT-in-monitoring-and-evaluation.pdf
- https://lindaraftree.com/2013/04/17/benefits-barriers-and-tips-for-ict-enabled-me/
- https://www.betterevaluation.org/en/theme/MandE_technology_insecure_settings
About the Author
Fiona Eichinger is a Junior Officer with five years of experience in humanitarian action, particularly with migrants and refugees, across Europe and the USA. She is earning a Master’s in International Relations at Syracuse University and Sciences Po, specializing in humanitarian assistance, development and post conflict reconstruction.
Read more about Fiona on LinkedIn.
