To contribute to large-scale change, international NGOs have to influence policymakers. So we engage in advocacy. But how do we know if our efforts are paying off? How do we convince our organizations and donors to continue investing in our advocacy over time?

Monitoring and evaluation are key. Monitoring helps advocates identify and respond to changes in a policy context and understand if they need to adjust their tactics. Evaluations help advocates identify successes, understand shortcomings and shape future efforts.

Yet those engaged in advocacy may feel too busy to pause for monitoring activities. They may think effectiveness is too hard to measure. The experiences of six international NGOs (Bread for the World, CARE, Mercy Corps, Oxfam America, RESULTS and Save the Children, all of which provided valuable input for this article) help answer these challenges.

M&E of advocacy investments

The challenges of evaluating advocacy are similar to those faced evaluating humanitarian response, peacebuilding and human rights efforts. Rapidly changing contexts require constant monitoring and flexible interventions to ensure that approaches remain cost-effective and efficient.

There are many factors beyond an intervention influencing a desired policy change, both positively and negatively. Changes in a political environment affect strategies required for success. Expected and unexpected events can change the playing field. And advocacy efforts by others (allies and/or opponents) working on the same issue can also change the political dynamic for a campaign.

This complexity makes it unrealistic to think that an NGO can isolate the impact of its intervention. Instead, evaluators seek to build credible and defensible cases demonstrating that an advocacy effort was indeed one of the factors that contributed to change.

What can we monitor and evaluate?

NGOs can measure their level of advocacy effort (outputs) and what happens as a result of their advocacy (outcomes). Activity and output indicators let an organization know if it is on track to raise awareness on an issue and increase its organizational visibility. Output indicators may include reports disseminated, events held and meetings organized.

Outcome indicators let an NGO know if it has selected effective strategies and tactics. Outcome indicators may include instances in which policymakers reach out to an NGO for help on a topic, evidence that the NGO’s efforts have helped shape a debate, and signs of strengthening coalitions that support its goals.

Policy context. If an NGO wants to make sure its advocacy strategy is effective, it has to monitor changes in the policy context in which it is operating. Periodic reviews are a common tool. For example, Save the Children, with the International Food and Policy Research Institute and other partners, is using social network analysis to map decision-makers and decision-making processes and to identify advocacy opportunities.

Advocacy capacity. Because influencing policy change depends on the ability to intervene at the right moment, evaluating an organization’s advocacy capacity is critical. Advocacy capacity measures policymaker relationships, grassroots organization, media relationships and web campaign capacity. Bread for the World’s internal evaluation tool assesses its advocacy capacity and organizational strength. The tool tracks items such as changes in its activist base and network of churches; responsiveness to Bread’s advocacy alerts to show how many activists consider Bread an authority on a given issue; communications with members of Congress; and media hits. The evaluation tool also tracks progress towards legislative and policy goals.

Country offices’ advocacy capacities and efforts. At CARE and Save the Children, country offices report on their advocacy activities, which are seen as an integral item in the full range of tools staff can use to help improve people’s lives. This reporting helps an organization guide its internal advocacy capacity building efforts and identify and link up similar advocacy initiatives being conducted in different countries. This reporting also highlights opportunities for case studies that can serve as learning tools.

Outreach to policymakers. Many NGOs track the number and content of their meetings with policymakers and staff. Capturing this information in a database and sharing it in staff meetings can improve coordination. These measures can also show whether advocacy is on track and inform decision-making. However, other NGOs voice caution. They argue that while reaching many offices and having lots of meetings can raise awareness, to actually shape policy, in depth conversations and ongoing relationships with a few policymakers is a better tactic and measure.

Grassroots advocacy, including Web advocacy. Many organizations use Web advocacy to build grassroots support for their issues. Web analytics, although imperfect, show who is joining a site and taking action, offering snapshots of online reach and influence. They reveal whether an NGO is building an online constituency, and provide guidance for targeting advocacy actions.

Some NGOs use offline action to foster grassroots advocates. Measures of success include grassroots advocates organizing events and holding meetings with elected officials in their in-district offices. Monitoring these activities and their outcomes can be difficult since volunteers have no reporting obligations. Bread for the World addresses this by encouraging its members to share their successes and highlighting those successes in its internal newsletter, where the success stories can promote more activism.

Media coverage, media relationships. Many NGOs track media coverage of issues. Some undertake qualitative assessments of a piece’s placement and framing, or analyze how their messages resonate in the media to measure their influence. To assess media capacity, CARE developed a tool that captures its media relationships. With this tool, CARE knows which media outlets are most likely to help spread its message and can identify with which outlets it needs to foster stronger relations.

Policymaker champions. Since connecting an NGO’s advocacy efforts to policy change or changes in people’s lives is such a complex task, many advocates measure their success according to changes in policymakers who are influential on an issue. They then attempt to link those changes to their actions.

A number of tools have been developed to track policymaker champions, such as the Harvard Family Research Project’s Policymaker Ratings tool. Oxfam America uses such a measure as part of its quarterly campaign team check-ins. CARE and the Aspen Institute’s Continuous Progress Strategic Services created a similar tool with well-defined traits and scoring. RESULTS created the Champion Scale for activists to use as a grassroots advocacy tool to build relationships with their members of Congress and to evaluate their own advocacy strategies.

Yet, it is nearly impossible for raters to know everything a policymaker does on an issue. CARE took that challenge as an opportunity to meet its dual advocacy purposes of advancing issues and raising CARE’s profile. The measure captures the policymaker’s support for an issue in part as a result of CARE’s effort.

When an NGO can access policymakers, direct follow-up can link an organization to a policymaker’s action. After significant advocacy activity, Mercy Corps occasionally contacts policymakers and their staff to hear their assessments of its efforts. Similarly, CARE asked an external evaluator to reach out to policymakers who had traveled on a CARE Learning Tour. If access and trust exist, policymakers are likely to report honestly on an organization’s performance. Verifying this with other data provides greater credibility.

Facilitating advocacy M&E

The following steps can create an environment in an organization that increases its ability to effectively monitor its advocacy work.

Establish a clear advocacy model. Assessing advocacy success requires a well-defined advocacy model and measurement plan. A clear theory of change explaining how an NGO believes that policy change might come about (e.g., through citizen action? through behind-the-scenes conversations?) helps advocates monitor a policy context, assess if they are on track, and identify needed adjustments.

Ensure support from an organization’s leadership. For an organization to benefit from advocacy evaluation, its leadership must champion it. While staff may value evaluation, their busy schedules will only let them prioritize it if management does so. Leadership must understand that measurement approaches that work for other types of interventions often do not work with advocacy. When leadership clearly understands this, advocates are free to choose ambitious objectives and strategic indicators.

Ensure staff participate and use evaluations. To ensure staff participation, evaluators need to identify reasonable time periods to gather, analyze and use information, taking into account staff workloads. Monitoring must be limited to what is needed to support strategic decision-making. Reporting formats must make information easy to understand and use. Demonstrating the utility of monitoring data and evaluation findings to decision-making helps increase buy-in. Showing how data can be used to influence policymakers helps people better understand the full value of monitoring and evaluation.

Work with donors. Save the Children notes that because advocacy takes place in fluid environments with many actors, advocacy approaches and measures sometimes need to change. Making these changes requires regular conversations with donors that build trust and allow for this flexibility.

Embed monitoring and evaluation within advocacy efforts. Because policy contexts change rapidly, advocates must constantly ask themselves whether their strategies remain relevant and their tactics effective. An embedded evaluator (internal or external to the organization) has access to information, a better understanding of the context and advocacy initiative, and greater trust with the advocacy team, leading to more accurate, timely and useful evaluation findings. Additionally, an embedded evaluator can highlight advances and challenges that advocates might not perceive.

A truly external evaluator can provide new learning and new perspectives. External evaluations tend to receive more attention from leadership and outside an organization. But external evaluators are not always needed. Advocates, working with internal evaluation staff or on their own, can build monitoring, evaluation and learning processes into their advocacy efforts. Mercy Corps, Oxfam America and Save the Children use after-action reviews following any major advocacy effort. Participants reflect on their actions and the results they achieved; tracking data reinforces participants’ perceptions. These reviews inform planning and help improve practice. The reports help explain to the rest of the organization the value of advocacy.

The six organizations highlighted in this article are using advocacy evaluation to improve their effectiveness. As evidenced by their experiences, advocacy evaluation, when integrated into an advocacy effort, can help ultimately improve advocacy outcomes and strengthen support for this important part of an organization’s work.