NGO Aid Map: Balancing More Data with Data Quality

This last blog post of the series delves into the challenge of balancing the demand for more data with the need to ensure a sufficient level of data quality. For NGO Aid Map to be successful, people must view it as a source of reliable and useful information, which suggests the need to prioritize data quality. On the other hand, if the initiative seems to lack forward momentum and starts to feel stale, this also invites failure. So what direction will NGO Aid Map be taking? For reasons we’ll explain below, we’ve chosen to go wide over going deep.

First though, it’s worth explaining what we mean by data quality. For us, this has three dimensions: completeness, accuracy and comprehensiveness. Completeness relates to individual project records, and refers to how many of the required and optional fields information has been submitted for. As you can see in the table below, this aspect of data quality has improved significantly over time. Of the three data quality dimensions, this is the easiest to monitor.

Accuracy refers to whether the information provided correctly reflects the work taking place on the ground, while comprehensiveness refers to how much of its work an organization is submitting information on. Where secondary sources of information are available – such as an organization’s website or donor websites and fact sheets – we try to use these to verify accuracy and comprehensiveness. So we encourage members to publish what they can. We also sometimes catch errors through our reviews of individual projects. Seeing how their peers are reporting to NGO Aid Map is also prompting organizations to make improvements in these areas. In an interview, one NGO told us, “[Haiti Aid Map] also made us realize that others had more capacity to give you better data. We talked internally about that. We asked ourselves, ‘What do we need to do to get to that level?’ It has helped us look for areas of improvement.” In general, we rely on our members to provide accurate, comprehensive information on their work.

None of this is meant to suggest that we have cracked the data quality nut. Indeed, anytime we take a look at the data, we find things that we’d like to improve upon. Still, we are confident that things are moving in the right direction. Which is why we’ve chosen to focus more attention on another area: expanding NGO Aid Map.

This spring, we will launch three new subsites: China, India and Mexico. Next year, we hope to launch a global NGO Aid Map covering all sectors and countries our members work in. We know this is a hugely ambitious step, and conversations with members this year will tell us how feasible it is. In doing this, we are going against the suggestion that we improve on the sites we’ve already created, notably by making sure that all InterAction members who could be contributing data (what we call eligible members) actually are. Here’s why.

One of the most important lessons we’ve learned in doing this is not to go after perfection. As we emphasized in the second post in this series, NGO Aid Map is a completely voluntary initiative. In the short- to medium-term, we are unlikely to get 100 percent of eligible members to participate. This is true for various reasons, including genuine capacity constraints.

Yet this should not stop forward progress. If you believe, as we do, that making aid data public can lead to better development, then increasing the amount of information available must be a priority. We will not launch a global map with data from just a handful of organizations. But if a critical mass is ready to share information on their work around the world, we’d like to provide the platform for making that information public.

There are also very practical reasons for taking this step. A single, comprehensive NGO Aid Map will be much easier to market and will likely attract much more traffic than six separate sites. More traffic means participating organizations will get more visibility, one of the primary incentives for contributing data. Going global also allows more InterAction members to participate. Expanding to China, India and Mexico brings the number of eligible organizations to 126, from the current 96 that can contribute data to the Haiti, Food Security and Horn of Africa Aid Maps. Expanding to all sectors and countries will push that number up to around 150 (not all InterAction members implement projects, which means we will never have participation from 100 percent of  members). The more organizations that participate, the more others will feel compelled to do so.

One of the mantras of aid transparency is, “Start with what you have and improve over time.” NGO Aid Map has very much operated in this spirit. In two years, we’ve come a long way – from trying to convince members one by one to contribute information to the initiative, to being asked by some organizations when they can submit more data. Despite its limitations, NGO Aid Map remains one of the few – if not the only – places where you can find project level information on the work of U.S. NGOs. Our focus going forward is on making it an even better, more comprehensive resource.

Laia Grino is Manager of Transparency, Accountability and Results at InterAction. Read the other posts in this blog series: Two Years Older and Wiser, We Built It. Have They Come?, We Built It. Have They Come (Part 2: Making It Easy to Participate), and The Tricky Question of Use. Follow us on Twitter at @NGOAidMap.