Data Utilization is at the Heart of Governance: Conducting a Census of Somalia

It has been almost 50 years since a reliable census was conducted in Somalia. Decades of civil unrest, armed conflict, and climatic shocks have made doing so exceptionally difficult. However, this year the Somali government intends to change that. The Somali Population and Housing Census 2024 was announced on May 2nd, marking the beginning of a national effort aimed at attaining a true count of the number of people residing in the country.

Many of the conditions that have prevented a census from being taken in the last decades persist. However, in this instance the Somali government will be assisted by both the United Nations Population Fund (UNFPA) and other development partners. This includes the UK’s Office for National Statistics (ONS) Data Science Campus, which, as part of its remit, supports the delivery of wider UK government international aid by helping improve the use of statistics in low and middle-income countries.

Data scientists at the ONS have been working to develop a new tool, based on high-resolution satellite imagery from Planet, that will allow for a more effective and efficient allocation of the limited human resources available to conduct a census. Their novel machine learning model applied to Planet imagery has the potential, not only to assist Somalia in this instance, but to help coordinate future humanitarian relief efforts globally.  

Conducting a census is never easy. Obtaining an accurate count of every man, woman, and child in a country involves huge amounts of preparation, organization, diligence, and in most cases, footwork. Ideally, the end result of these efforts is a reliable dataset that enables the effective and equitable implementation of public policy. It is a difficult endeavor even under ideal conditions, but the data it provides is always invaluable.

“Generation of data and utilization of data is at the heart of governance,” Salah Jama, Deputy Prime Minister of Somalia, said during the announcement of the project. “It is paramount for good governance. It is paramount for development gains.”

The people of Somalia have faced many challenges in the intervening decades since the last census was conducted. Political unrest and climate change have upended communities and driven large portions of the population from their homes. The UN High Commissioner for Refugees (UNHCR) and the International Organization for Migration estimate that 3.8 million people are displaced in the country.

The vast majority of those uprooted from their communities have not left the country, but rather have settled in informal encampments on the outskirts of larger urban centers. These settlements are commonly termed Internally Displaced People (IDP) camps. They most often do not have access to basic infrastructure such as electricity and running water, and the institutional structures to facilitate and manage the coordination of humanitarian assistance within the camps are often weak.  

The characteristics of these camps create a particularly difficult context for collecting accurate population statistics. This is because they are dynamic, changing quickly in terms of size and location. This creates an information gap that is difficult to fill.

Gathering accurate data on IDP camps poses a significant challenge. Overcoming it would require time and human resources that are simply unavailable to both the Somali authorities and the numerous humanitarian organizations working in the country. However, the development of automated, satellite imagery-based change detection systems for these settlements has the potential to greatly reduce the human resource cost of maintaining an accurate and timely understanding of where IDP populations have settled.

The UNFPA generates IDP population estimates through integrating remote sensing and field survey data. The current process involves using very high-resolution satellite imagery to manually count structures (buildings, tents) within IDP camps. This process is time consuming and prevents rapid response to on-going or escalating situations. This is why data scientists at the UK Office for National Statistics (ONS) Data Science Campus set out to develop a novel machine learning model, based on Planet high-resolution SkySat imagery, with the goal of creating an analytical solution that can automatically detect the footprints of tents and other structures in IDP camps. The goal is that such a system would enable the timely and efficient allocation of the human resources required to collect and confirm accurate population statistics.

The ONS team faced a variety of challenges in applying such a model to the particular context of Somalia. Refugees who cross international borders are often provided a certain amount of resources by the UNHCR, including standardized, white-colored tents. However, internally displaced people are left to build shelters with whatever is available to them.

According to Nicci Potts, the ONS data scientist leading the effort, this poses an exceptional challenge from a satellite imagery and machine learning perspective. “The material used to construct shelters tends to be really ad hoc, which makes this really challenging from a technical point of view, because we are telling the machine, this is what you should expect, and where you should expect it. In a refugee camp, you tend to have a bit of structure in the layout of tents that isn’t present in IDP camps,” she said.

The landscape itself also poses a challenge. Most of Somalia is arid desert. Its exposed earth ranges in hue between rich brown tones, to the deep reds and oranges of soil rich in iron oxide. Dust matching the soil in color is inevitably deposited on the structures within the IDP camps. The result is that, in satellite imagery, you are left with many heterogeneous structures that provide little contrast with the surrounding landscape.

Despite these challenges, the ONS data scientists were able to successfully train a machine learning model with Planet data that was able to accurately identify structures in IDP camps in Somalia, helping reduce labor costs.

The resources saved by the model are also expected to increase over time. The initial implementation was relatively labor intensive, as the model requires the production of a large amount of training data. However, this is a one-off task, so once implemented, the ONS team predicts that labor cost savings will apply with each subsequent use of the model. 

This compounding of labor cost savings is an important characteristic of the tool developed by the ONS. While it will play an important role in helping Somalia conduct a successful census in the coming year, it has the potential to be utilized in other contexts. For Potts and her colleagues, the goal is to create a system that can be operated by the UNFPA in support of its humanitarian work globally. In their view, Somalia was the right place to test the feasibility of the project given the interest and engagement from stakeholders and the potential for the work to have a direct impact beyond the proof of concept research. “If we overcome the challenges that Somalia poses for building this kind of model, then we can reasonably hope that it can be embedded as a business process elsewhere,” Potts said. “The aspiration is that we can upskill staff within both UNFPA and the Government of Somalia and give them a product that they can run themselves.”

The effects of climate change, and the continuation of armed conflicts globally can mean that the displacement of vulnerable populations will continue around the world. Tools like this can help the UNFPA and other humanitarian relief organizations effectively provide for those most in need, and satellite imagery will continue to play a critical role in those efforts.