Testing the Assumptions of the Data Revolution

 A World that Counts: Mobilizing the Data Revolution for Sustainable Development  provided a roadmap for how the data revolution might accelerate the development agenda. Prepared at the request of the United Nations Secretary-General, Ban Ki-moon, its launch predated the General Assembly’s adoption of the  2030 Agenda —a universal, goal-oriented plan to advance global development equitably and sustainably—by ten months. This gave data practitioners a jump on acting on its recommendation to align innovations in data to the development framework underpinning the 2030 Agenda. 

Ten years have passed since the release of A World that Counts and the formal adoption of the  Sustainable Development Goals (SDGs) . This seems an appropriate time for national governments and the global data community to reflect on where progress has been made so far. 

This report supports this objective in three ways: it evaluates the assumptions that underpin A World that Counts’ core hypothesis that the data revolution would lead to better outcomes across the 17 SDGs, it summarizes where and how we have made progress, and it identifies knowledge gaps related to each assumption. These knowledge gaps will serve as the foundation for the next phase of the SDSN TReNDS research program, guiding our exploration of emerging data-driven paradigms and their implications for the SDGs. By analyzing these assumptions, we can consider how SDSN TReNDs and other development actors might adapt their activities to a new set of circumstances in the final six years of the SDG commitments.

Given that the 2030 Agenda established a 15-year timeframe for SDG attainment, it is to be expected that some of A World that Counts’ key assumptions would fall short or require recalibration along the way. Unforeseen events such as the COVID-19 pandemic would inevitably shift global attention and priorities away from the targets set out in the SDG framework, at least temporarily. 

We revisit these assumptions by synthesizing published research and interpreting select country and technology cases. Our interpretation builds on the experience of TReNDS members (many of whom were key authors of A World that Counts) and data practitioners within the  Global Partnership for Sustainable Development Data (GPSDD)  network, which was itself established in response to one of the report’s key recommendations. Our audience includes these key communities as well as others working to advance a data-driven agenda that will reinforce the SDGs.

Our report is structured as follows. The present introduction provides a quick summary of A World that Counts and distills six key assumptions that underpin its recommendations. Following this, six sections reflect on the continued relevance and validity of these assumptions based on subsequent experience over the eight years since the report’s release. The final section brings together the major themes that have emerged and suggests some concrete actions to help leverage the data revolution for the final six years of the 2030 Agenda.

To assess and test the assumptions, each section in this report draws on a synthesis of available evidence along with expert input from the authors of A World that Counts and the community of practice established in response to their report. We find clear evidence that some assumptions still hold true and will likely stay so for the remaining years of the SDG framework. However, our synthesis of evidence is insufficient to validate others. In these cases, we suggest further areas of research or ways to fill evidence gaps, although we do not seek to fill these gaps in the context of this report. 

The analysis in this report is a starting point, not an endpoint. These sections are designed to serve as prompts for further discussion and thought, rather than presenting explicit, immediate recommendations. Our hope is to support colleagues and peers across the global data and statistics community, providing encouragement to continue the discussion and collective ideation for concrete recommendations.

For consistency, the report sections follow a common format as they review each of the six assumptions. Each section begins with a brief summary and recap of the assumption it addresses. This is followed by a review of subsequent experience since the publication of A World that Counts at the end of 2014, through which we synthesize the evidence available in order to assess the assumption’s validity. A discussion of the evidence and concluding reflections round out each section. 

Readers should anticipate some overlap across the assessment sections. It is inevitable that some contextual changes in the external environment will impact the continued validity of more than one assumption. As a result, the report provides a concluding section that consolidates the main findings.

At the midway point of the 2030 Agenda and eight years since the release of A World that Counts, our progress toward achieving the 17 Sustainable Development Goals (SDGs) falls short of initial expectations. Recognizing this pivotal juncture, TReNDS' Expert Members deemed it timely to examine the foundational assumptions of the report, "A World that Counts" – the hypothesis that the data revolution would serve as a catalyst for SDG progress.

In response, the TReNDS secretariat, in collaboration with select Expert Members, came together to test six core assumptions. This effort emphasizes the imperative to assess the report's priorities in light of the rapidly evolving data landscape. The reflections highlighted throughout this project can help to propel the SDGs forward in their concluding seven years.

What began as an online debate has transformed into an interactive platform, extending an invitation for input and discussions from readers and the wider data community. This project prompts a more extensive conversation, urging stakeholders within the data and statistics communities to actively participate, share insights, and collaboratively shape the future of data-driven development. In doing so, we seek to bridge knowledge gaps and collectively pave the way for a more impactful and inclusive data revolution.

Join us in testing the assumptions and engaging in the dialogue it fosters. Together, let us contribute to a world that counts for all.

Maryam Rabiee

Acting Head of TReNDS

We express our sincere gratitude to the authors, Castelline Tilus (SDSN TReNDS), Grant Cameron (Consultant and past TReNDS Director), and Alex Fischer (Monash Sustainable Development Institute), whose dedication and expertise have been instrumental in bringing this project to fruition. We are also deeply appreciative of the valuable feedback and insights provided by Shaida Badiee (Open Data Watch), Bob Chen (CIESIN, Columbia University), Chris Maloney (Hewlett Foundation), Eric Swanson (Open Data Watch), Tom Orrell (Data Ready), Jessica Espey (University of Bristol), Lorenz Noe (Open Data Watch), Jenna Slotin (Global Partnership for Sustainable Development Data), and Sam Custer (AidData). 

Special appreciation goes to our esteemed funder, the Hewlett Foundation, whose support made this project possible. Last but not least, we extend our thanks to Steph Pietras (SDSN TReNDS), who crafted an engaging platform for the dissemination of this project.

To everyone who contributed to this endeavor, we are truly grateful for your collaboration and support.

In August 2014, the UN Secretary-General’s office established the Independent Expert Advisory Group (IEAG) on a Data Revolution for Sustainable Development, which in November that year produced its seminal report, A World that Counts: Mobilizing the Data Revolution for Sustainable Development. The report positions data firmly at the core of the 2030 Agenda, with recommended actions organized under four pillars:  (1) principles and standards; (2) technology, innovation, and analysis; (3) capacity and resources; and (4) leadership and governance. While primarily geared toward the global statistics and development communities, many of its recommendations also engaged data producers from the private sector, the non-government sector, and from citizen-generated data initiatives. This reflected the landscape of opportunities for collaboration. 

A World that Counts principally focused on two pathways through which data could be an agent of change for development. First, it assumed that improving the public and civil society’s use of data would enhance government accountability (the accountability pathway). Second, it assumed that introducing new sources of data and new systems would help governments design more effective policies and programs resulting in greater impact (the policymaking pathway).

For both pathways, the report highlighted the importance of strengthening connections between data producers and users to ensure that improvements in data production met policymaking and accountability needs. Stronger connections would contribute to policymaking by helping governments develop products and services that supported the five data-intensive steps in the policy process—agenda setting, prioritizing actions, formulating policies, monitoring, and evaluation (University of Oxford, 2023)—and would also support accountability by ensuring that data was used ethically, with privacy-protection structures firmly in place. As such, it recognized that the accountability function of non-state actors involved ensuring that they improved their services consistently with cultural norms and sensibilities, as well as holding governments to account for their policy performance. 

A World that Counts recognized that a comprehensive approach would be essential to successfully harness data to further the development agenda. Of their recommendations, many were adopted early on. For example, the UN began holding regular World Data Forums to explore sustainable development data, corresponding to recommendations listed under “governance and leadership”, These led to new networks and innovative forms of collaboration, including partnerships between state and non-state actors, in an attempt to channel and combine new and old forms of data to support the SDGs.

The launch of the  Sustainable Development Solutions Network ’s  Thematic Research Network on Data and Statistics (SDSN TReNDS)  was motivated in part by the recommendation made in A World that Counts that research gaps at the intersection of data and the 2030 Agenda should be identified and filled. Among our contributions are two flagship reports—Counting on the World (2017) and Counting on the World to Act (2019)—together laying out an action plan for a modern, integrated statistical system to support the SDGs consistent with the key principles identified in A World that Counts. The 2017 report has influenced other thought leaders in the development space, most notably in the UN Statistics Commission’s work program on data stewardship and the World Bank’s launch of its flagship report, World Development Report: Data for Better Lives (2021).

The present report benefits from a series of consultations with SDSN TReNDS members as well as comments from other experts in the SDG data space. It builds on their experiences, expertise, and commitment to the SDG data agenda. SDSN TReNDS members are leaders within the global scientific, development, and public- and private-sector data communities. From October 2021 to April 2023, they formed a working group to review the assumptions underlying A World that Counts, with the aim of identifying lessons learned that could advance the “data for development” agenda and pinpointing areas where shifts are needed in our thinking. In doing so, they brought to light a number of key assumptions that were largely implicit in A World that Counts. 

Assumptions are too often unrecognized, unstated, unquestioned. Yet they are often predicated on future potentials. Our analysis seeks to reveal and examine the assumptions that were embedded within A World that Counts, and to validate the extent to which these beliefs and the change processes they anticipated have held true. 

Guided by the four enabling pillars of actions recommended in A World that Counts, we developed the following criteria to identify the key assumptions underpinning its implicit theory of change:

• The key assumptions should reflect how the use of data would differentiate the SDGs and the 2030 Agenda from past global policy initiatives (how government and non-government sectors would harness big data to further an inclusive, globally-adopted development agenda).
• The key assumptions must align to the four pillars/enablers under which the actions recommended in A World that Counts are organized.
• The number of assumptions must be limited to represent only those most critical to success and which must be valid for the expected results to happen.

Using these criteria, we identified six key assumptions within A World that Counts’ four pillars of recommendations:

A set of core assumptions shaped the SDG-driven data revolution and drove advocacy for it. While they held true for a time, serving as a tool to assess whether recommended actions would be appropriate to influence change, the present report provides a reflexive check on these assumptions and an opportunity to update them in a changing world. Since the release of A World that Counts in 2014, many factors have shifted the conditions under which it linked actions to harness the data revolution towards improved SDG outcomes. 

In this concluding section, we summarize the main findings of the previous six sections of this report, which examined the key assumptions underlying A World that Counts. This summary also provides the basis to review its implicit theory of change. 

In identifying and evaluating the underlying assumptions of A World that Counts, we uncovered a number of key opportunities to further advance the core objective over the next six years, of achieving better SDG outcomes. We view these actionable insights as “eureka moments.” Our Eureka moments are summarized as follows:

Part 1 (Technical progress would enable greater data availability): A World that Counts correctly assumed that new technologies would continue the exponential growth in the volume of data available for decision-making. However, data from the greatest applications of these technologies remain siloed in large private-sector companies and industries traditionally outside the sustainable development sector. Creating incentives for open data partnerships between public- and private-sector institutions could allow for greater use of private data for the public good while also addressing existing SDG data gaps. 

Part 2 (The SDGs would be the driving force for data innovations for the public good): Our findings confirm that the extent to which the SDGs have driven data innovations for the public good in relation to SDG policy formulation remains obscure. The COVID-19 pandemic provided concrete examples of what is possible when a sense of shared urgency drives political will to match technical capacity. However, the interest and ability of governments to invest in data for policy formulation outside of their national statistical bodies remains unknown. While investment continues to be driven by the SDG indicator measurements, we now have an opportunity to assess the marginal return on data investments so far. 

Part 3 (Information gaps would be the major reason for policy failure): More data does not immediately result in better policy formulation or improved outcomes. Many institutional decision-making processes are not ready for more data, lacking either the analytic capacity or the appropriate processes to integrate them into their policymaking cycles. Moreover, combining the requisite political acumen with analytic capabilities is essential if policymaking is to have a development impact—a dynamic that received proportionately less attention in the assumptions underpinning the data revolution.

Part 4 (The SDGs would enable resource mobilization for the data revolution, accelerating progress toward outcomes): Funding gaps have persisted since 2015, suggesting that the issue extends not only to planning and accountability barriers, but to wider geopolitical and economic factors and national politics of donor countries. Despite COVID-19 pandemic-related investments, funding directed at statistical systems has plateaued without reaching target levels. The pandemic resulted in numerous innovations in data systems, with many countries moving beyond traditional surveys as funding shifted from traditional grants to in-kind technical support and assistance. There is an opportunity to rigorously target financing in national plans to specific elements of the pathways outlined in A World that Counts, where data has the greatest impact on shaping development outcomes across government and civil society, or to target financing at improving the efficacy of SDG implementation initiatives. 

Part 5 (The public sector would guide and drive data innovations to target sustainable development): The private sector and civil society are playing a much greater role in generating data innovations than the authors of A World that Counts had envisioned. As the private sector has the potential to continue to generate data consistent with responsible and sustainable economic opportunities, targeted synergies between the private and public sectors should be created to establish processes for mutual data improvements. A clear focus area could be environmental and biodiversity accounting.

Part 6 (Data would be a standardizing force and a mechanism for greater participation and accountability): A World that Counts motivated actions to safeguard data to ensure its ethical use, and to open data to improve accountability. More research is needed, however, to resolve existing disputes regarding the links between transparency and accountability. 

A World that Counts undoubtedly had an impact on mobilizing attention and efforts to harness the data revolution for the SDG agenda. Ten years after its release, the pathways of change it identified and its four enabling pillars remain relevant, and unlike other thematic global initiatives, data remains central to the political discourse. As attention to data has not receded, funders, implementers, standard-setters, and researchers can all still benefit from its core premise—a data revolution for sustainable development—to support their missions. The knowledge gaps identified through this work will serve as the foundational framework upon which the next phase of the SDSN TReNDS research program will be constructed, guiding our exploration of emerging data-driven paradigms and their implications for the SDGs. By analyzing these assumptions, we hope to reflect on how SDSN TReNDs and other development actors might adapt their activities to the new set of circumstances in the final six years of the SDG commitments.

SDSN TReNDS. (2017). Counting on the World: Building Modern Data Systems for Sustainable Development. Sustainable Development Solutions Network, Thematic Research Network on Data and Statistics. Available at  https://files.unsdsn.org/sdsn-trends-counting-on-the-world-2017.pdf  

SDSN TReNDS. (2019). Counting on the World to Act: A Roadmap for Governments to Achieve Modern Data Systems for Sustainable Development. Sustainable Development Solutions Network, Thematic Research Network on Data and Statistics. Available at  https://files.unsdsn.org/19COTW.pdf . 

University of Oxford. (2023). “Guidance note 2: Understanding the policy process.” In Guidance on Policy Engagement Internationally. Available at  https://www.ox.ac.uk/research/using-research-engage/policy-engagement/guidance-policy-engagement-internationally/guidance-note-2-understanding-policy-process .