The volume of data in the world is increasing exponentially. By some estimates, 90% of the data in the world has been created in the last two years, and it is projected to increase by 40% annually. A large share of this output is “data exhaust,” or passively collected data deriving from everyday interactions with digital products or services, including mobile phones, credit cards, and social media. This deluge of digital data is known as big data. Data is growing because it is increasingly being gathered by inexpensive and numerous information‐sensing, mobile devices and because the world’s capacity for storing information has roughly doubled every 40 months since the 1980s.
The Data Revolution
The data revolution -- which encompasses the open data movement, the rise of crowdsourcing, new ICTs for data collection, and the explosion in the availability of big data, together with the emergence of artificial intelligence and the Internet of Things -- is already transforming society. Advances in computing and data science now make it possible to process and analyse big data in real time. New insights gleaned from such data mining can complement official statistics and survey data, adding depth and nuance to information on human behaviours and experiences. The integration of this new data with traditional data should produce high-quality information that is more detailed, timely and relevant.
Data is the lifeblood of decision-making and the raw material for accountability. Today, in the private sector, analysis of big data is commonplace, with consumer profiling, personalised services, and predictive analysis being used for marketing, advertising and management. Similar techniques could be adopted to gain real-time insights into people’s wellbeing and to target aid interventions to vulnerable groups. New sources of data, new technologies, and new analytical approaches, if applied responsibly, can enable more agile, efficient and evidence-based decision-making and can better measure progress on the Sustainable Development Goals (SDGs) in a way that is both inclusive and fair.
Fundamental elements of human rights have to be safeguarded to realize the opportunities presented by big data: privacy, ethics and respect for data sovereignty require us to assess the rights of individuals along with the benefits of the collective. Much new data is collected passively – from the ‘digital footprints’ people leave behind and from sensor-enabled objects – or is inferred via algorithms. Because big data is the product of unique patterns of behaviour of individuals, removal of explicit personal information may not fully protect privacy. Combining multiple datasets may lead to the re-identification of individuals or groups of individuals, subjecting them to potential harms. Proper data protection measures must be put in place to prevent data misuse or mishandling.
There is also a risk of growing inequality and bias. Major gaps are already opening up between the data haves and have-nots. Without action, a whole new inequality frontier will split the world between those who know, and those who do not. Many people are excluded from the new world of data and information by language, poverty, lack of education, lack of technology infrastructure, remoteness or prejudice and discrimination. There is a broad range of actions needed, including building the capacities of all countries and particularly the Least Developed Countries (LDCs), Land-locked Developing Countries (LLDCs), and Small Island Developing States (SIDS).
Big Data for Development and Humanitarian Action
In 2015, the world embarked on a new development agenda underpinned by the Sustainable Development Goals (SDGs). Achieving these goals requires integrated action on social, environmental and economic challenges, with a focus on inclusive, participatory development that leaves no one behind.
Critical data for global, regional and national development policymaking is still lacking. Many governments still do not have access to adequate data on their entire populations. This is particularly true for the poorest and most marginalized, the very people that leaders will need to focus on if they are to achieve zero extreme poverty and zero emissions by 2030, and to ‘leave no one behind’ in the process.
Big data can shed light on disparities in society that were previously hidden. For example, women and girls, who often work in the informal sector or at home, suffer social constraints on their mobility, and are marginalized in both private and public decision-making.
Much of the big data with the most potential to be used for public good is collected by the private sector. As such, public-private partnerships are likely to become more widespread. The challenge will be ensuring they are sustainable over time, and that clear frameworks are in place to clarify roles and expectations on all sides.
The role of the UN
One of the key roles of the UN and other international or regional organisations is setting principles and standards to guide collective action around the safe use of big data for development and humanitarian action within a global community and according to common norms. These standards seek to increase the usefulness of data through a much greater degree of openness and transparency, avoid invasion of privacy and abuse of human rights from misuse of data on individuals and groups, and minimise inequality in production, access to and use of data. Achievement of the SDGs in our digital world will require recognition of the need not only to prevent misuse of data, but also to ensure that when data can be used responsibly for the public good, it is.
The Secretary-General’s Independent Expert Advisory Group on a Data Revolution for Sustainable Development (IEAG) has made specific recommendations on how to address these challenges, calling for a UN-led effort to mobilise the data revolution for sustainable development.
- Fostering and promoting innovation to fill data gaps.
- Mobilising resources to overcome inequalities between developed and developing countries and between data-poor and data-rich people.
- Leadership and coordination to enable the data revolution to play its full role in the realisation of sustainable development.
Uptake of big data analytics is accelerating across the UN system with a growing number of UN agencies, funds and programmes implementing and scaling operational applications for development and humanitarian use.
The UN Development Group has issued general guidance on data privacy, data protection and data ethics concerning the use of big data, collected in real time by private sector entities as part of their business offerings, and shared with UNDG members for the purposes of strengthening operational implementation of their programmes to support the achievement of the 2030 Agenda.
The first UN World Data Forum held in January 2017 brought together over 1,400 data users and producers from the public and private sectors, policy makers, academia and civil society to explore ways to harness the power of data for sustainable development. It produced important outcomes, including the launch of the Cape Town Global Action Plan for Sustainable Development Data.
UN Global Pulse
Global Pulse is an innovation initiative of the UN Secretary-General on data science. Global Pulse promotes awareness of the opportunities big data presents for sustainable development and humanitarian action, develops high-impact analytics solutions for UN and government partners through its network of data science innovation centres, or Pulse Labs, in Indonesia (Jakarta), Uganda (Kampala) and the UN Headquarters (New York), and works to lower barriers to adoption and scaling.
To safely and responsibly unlock the value of data, Global Pulse established a data privacy programme, part of which involves ongoing research into privacy-protective uses of big data for humanitarian and development purposes. Global Pulse set up a Data Privacy Advisory Group, comprised of privacy experts from the regulatory community, private sector and academia, that engages in dialogue on the critical issues around big data and advises on the development of privacy tools and guidelines across the UN. To better understand the risks linked to big data, Global Pulse developed a two-phase “Risk, Harms and Benefits Assessment” tool, which includes guidelines to help practitioners assess the proportionality of the risks, harms, and utility in a data-driven project.
Global Pulse was also involved in the organization of the UN Data Innovation Lab workshop series, an initiative led by UNICEF and WFP. Consisting of five thematic workshops, the series aimed to understand existing data innovation capabilities and needs within the UN system.
Public - Private Partnerships
To ensure that access to insights from big data across many industries is widely available, Global Pulse has been working with the private sector to operationalize the concept of ‘data philanthropy,’ whereby companies’ data can be safely and responsibly used for sustainable development and humanitarian action. For example, in 2016, Global Pulse formed a partnership with the social media network Twitter.
Every day, people around the world send hundreds of millions of tweets in dozens of languages. Such social conversations contain real-time information on many issues, including food costs, the availability of jobs, access to health care, quality of education, and reports of natural disasters. The partnership will allow UN development and humanitarian agencies to turn the public data into actionable information to aid communities around the globe.
Other examples of partnerships include the GSMA’s “Big Data for Social Good” initiative, which leverages mobile operators’ big data capabilities to address humanitarian crises, including epidemics and natural disasters; Data for Climate Action, a competition which connected researchers around the world with data and tools from leading companies to enable data-driven climate solutions; and Data Collaboratives, a new form of collaboration beyond the public-private partnership model, in which participants from different sectors (and companies in particular) exchange their data to create public value.
- Predictive Analytics and the Future of Humanitarian Response
- An animated introduction to the UN's Global Pulse initiative
- A visualization of postal network big data - understanding nations' wellbeing
- The Data Gender Gap You Don't Know About
- Videos from the World Data Forum
- UNDG Guidance Note on Big Data for Achievement of the 2030 Agenda: Data Privacy, Ethics and Protection
Initiatives and Collaborations
- Data Revolution Group
- Data Privacy Advisory Group
- GSMA Data for Social Good Initiative
- Data for Climate Action challenge
- Data collaboratives from NYU GovLab & UNICEF
- Global Partnership for Sustainable Development Data