Leveraging Non-Traditional Data For The Covid-19 Socioeconomic Recovery Strategy

This article is co-authored with Selva Ramachandran, Resident Representative, UNDP Philippines.

Data is now recognized as the “new oil” for the digital economy. While development actors have relied on traditional sources of data, such as that sourced from public surveys and government administration, there is great potential to harness the value of unconventional or non-traditional sources such as data from the private sector, which can help fuel a more nimble, agile, and inclusive brand of governance.

Indeed, private companies routinely collect, analyze and use large volumes of data—both sourced from their own operations and from other firms—to derive actionable insights and inform business strategies. The ability and pace at which this data is harnessed with the aid of data science, analytics and artificial intelligence tools has allowed data savvy businesses to successfully navigate through several forms of crisis, including the Covid-19 pandemic. In this dynamic and uncertain environment, the importance of high-frequency, timely and granular data to inform decision-making has become invaluable.

To this end, it is opportune to ask the following questions: Can we harness the power of data routinely collected by companies—including transportation providers, mobile network operators, social media networks and others—for the public good? Can we bridge the data gap to give governments access to data, insights and tools that can inform national and local response and recovery strategies?

The Potential of Non-Traditional Data

There is increasing recognition that traditional and non-traditional data should be seen as complementary resources. Non-traditional data can bring significant benefits in bridging existing data gaps but must still be calibrated against benchmarks based on established traditional data sources. These traditional datasets are widely seen as reliable as they are subject to established stringent international and national standards. However, they are often limited in frequency and granularity, especially in low- and middle-income countries, given the cost and time required to collect such data. For example, official economic indicators such as GDP, household consumption and consumer confidence may be available only up to national or regional level with quarterly updates.

Meanwhile, non-traditional data such as market research routinely collected monthly from nationwide household surveys may only be specific to certain products and brands, but can provide more frequent and granular information, with disaggregation by geographical area, socio-economic group of households, gender and other attributes. Further, data collected from mobile devices, internet platforms and satellite images are often available in real-time and offer high granularity in location. These do not always comply with traditional statistical standards of data sampling and collection and often require novel “big data” methodologies to process and analyze. Innovative approaches that combine indicators from these different kinds of data can demonstrate their consistency and complementarity, exploit the advantages of each and produce novel insights.

Examples From The Philippines

In the Philippines, UNDP, with support from The Rockefeller Foundation and the government of Japan, recently setup the Pintig Lab: a multidisciplinary network of data scientists, economists, epidemiologists, mathematicians and political scientists, tasked with supporting data-driven crisis response and development strategies. In early 2021, the Lab conducted a study which explored how household spending on consumer-packaged goods, or fast-moving consumer goods (FMCGs), can been used to assess the socioeconomic impact of Covid-19 and identify heterogeneities in the pace of recovery across households in the Philippines. The Philippine National Economic Development Agency is now in the process of incorporating this data for their GDP forecasting, as additional input to their predictive models for consumption. Further, this data can be combined with other non-traditional datasets such as credit card or mobile wallet transactions, and machine learning techniques for higher-frequency GDP nowcasting, to allow for more nimble and responsive economic policies that can both absorb and anticipate the shocks of crisis.

Non-traditional data also has the potential to provide insights on the status of vulnerable groups, including the informal sector, which are not always captured by official statistics. In recognition of this, the Department for Information Communication and Technology and UNDP have begun to explore the use of satellite imagery to identify “last mile” communities living in geographically isolated and disadvantaged areas and understand their level of connectivity in terms of WiFi, electricity, roads, education, healthcare and markets. Furthermore, UNDP has utilized chatbots on social media platforms to rapidly collate information from disadvantaged sectors and small enterprises, to understand the ways in which the pandemic has impacted them, and the extent to which the social amelioration programs have worked.

These are powerful examples as to how non-traditional data can and has shed light on disadvantaged groups previously invisible, allowing for more inclusive plans and programs so that no one gets left behind.

Non-Traditional Data Can Facilitate Inclusivity

Currently, the ability of governments and development organizations to appreciate, access and responsibly use non-traditional data sources from the private sector is limited— this applies globally, but even more so in the developing world. On the supply side, companies may not yet fully appreciate how their data can be leveraged to support public and development needs. Further, there is a need to harmonize and operationalize international and national standards for data licensing, privacy and security to address legal and financial concerns and lower the barriers for data sharing. In this work it must be recognized that risks need to be identified and a mitigation strategy in place—including representation accuracy, digital security risks, risks of confidentiality and privacy breaches, and potential violation of intellectual property rights and other commercial interests. On the demand side, government agencies and development organizations have varying levels of technical capacity and resources for data-related work. Moreover, even within units where technical data-related work is performed, there may still be a need to innovate on approaches that incorporate these new kinds of data to augment official datasets and methodologies. Existing challenges including methodological, legal, privacy and security issues need to be addressed to promote practical use of non-traditional data.

Broadening The Data For Development Community

Unlocking private sector data for public good at scale requires setting up of the necessary market, legal and technical infrastructure, building on pillars of legal foundation, data governance, secure IT architecture, partnership management and multidisciplinary teams. A trailblazing initiative that has pioneered this is the Development Data Partnership, a private-public consortium founded by the World Bank, IMF and IADB with support from The Rockefeller Foundation. So far, it has 26 major companies as data partners—including Google, Facebook, Twitter, Waze and LinkedIn—and 6 development partners—namely, UNDP, IADB, IMF, World Bank, OECD and The Rockefeller Foundation. Multidisciplinary teams around the world are leveraging the rich non-traditional data sources offered through the partnership to innovate solutions to address the Covid-19 pandemic as well as major developmental challenges encompassing climate change, poverty, food security, transportation services and gender inequality.

Just to cite a few examples, the partnership’s non-traditional data are being used to track the impact of Covid-19 restrictions on mobility in Vietnam to assess the effectiveness of localized lockdowns, map urban mobility in Haiti to inform transportation policy and investments and fill data gaps on the impact of economic activity on climate change to enable policymakers to do robust economic and financial analysis. The use of nontraditional data to support the monitoring of the sustainable development goals has also been officially recognized, with the UN Committee of Experts on Big Data and Data Science for Official Statistics^{tasked to promote their practical use for SDG monitoring, including as basis for new indicators or proxies of indicators, with improved timeliness and granular social and geo-spatial breakdown.}

We have only begun to open the door to a parallel world of non-traditional data that has existed alongside us for decades now. As we engage in public discourse on the responsibilities of companies that collect and monetize our data and their positive and negative effects on society, there is space to consider the potential benefits if such powerful data and tools are harnessed for public good.

Data is inherently political and maximizing its positive impacts for society, particularly in unveiling the faces of vulnerable groups which had previously been invisible, will require a concerted effort from a community of practitioners and advocates within government, businesses, civil society and international organizations to shape the ways in which data is accessed, analyzed and used beyond the confines of their “for-profit” origins. Doing so could very well unlock the potential for more rapid and inclusive evidence-based interventions for those who need it the most.