Using machine learning to find
policy interventions to end hunger

Given the amount of information available, relying only on keyword searching doesn’t work. If we want to understand the fullness of human knowledge, we need to incorporate new methods of discovery that account for the way we describe similar things in different ways.

Over the past decade, there have been enormous advances in artificial intelligence that enable computers to analyze the way we use language. This involves training a computer program to recognize relationships between words, so that it can capture the different ways people describe similar things.

We used machine learning and natural language processing (NLP) to create and analyze a preliminary dataset of ~50,000 articles and reports (2008-2018) about smallholder farmers from science journals and research and development organizations. We used a variety of search terms, such as small-scale food producers, rural farmers, and subsistence and contract farmers.

In order to increase coverage of materials published in low and middle income countries, we included the full table of contents from the African Journal of Biotechnology, African Journal of Agricultural Research, African Journal of Food, Agriculture, Nutrition and Development, African Crop Science Journal, Indian Journal of Agronomy, and the Indian Journal of Agricultural Economics.

We used machine learning and natural language processing (NLP) to create and analyze a preliminary dataset of ~50,000 articles and reports (2008-2018) about smallholder farmers from science journals and research and development organizations. We used a variety of search terms, such as small-scale food producers, rural farmers, and subsistence and contract farmers.

In order to increase coverage of materials published in low and middle income countries, we included the full table of contents from the African Journal of Biotechnology, African Journal of Agricultural Research, African Journal of Food, Agriculture, Nutrition and Development, African Crop Science Journal, Indian Journal of Agronomy, and the Indian Journal of Agricultural Economics.

Want to learn more about machine learning-driven evidence synthesis—and how it can be applied to your research interests? Use the form below!

SYNONYMS FOR INTERVENTION

In agricultural research, the word intervention isn’t used consistently as a way of describing “what works” to address a particular problem, even though there are, in reality, many interventions designed to tackle agricultural and food security problems that have been researched.

Semantic associations, or synonyms, are a key building block for us. They enrich our ability to discover what we are looking for and what we hope to find. For example, when we conduct a keyword search using only “interventions” and “greenhouse gas emissions” in our dataset, we find only about 10 percent of the relevant materials. But when we create associations between interventions and possible synonyms, this increases to 61 percent of the dataset.

Now that we knew how interventions were described in agricultural research, we could set about analyzing our sample of articles to find and classify specific interventions—see “How we created an evidence map.”

Word

Intervention

Policy

Strategy

Measure

Program

Project

Programme

Outcome

Recommendation

Initiative

Targeting

Capacity building

Participatory approach

Programming

Social protection

Entry point

Policy option

Nutrition education

Multi-sectoral approach

Articles using the word (out of 49,000)

2561

7234

5752

2822

2785

2609

1961

1773

1180

1085

674

428

393

323

263

166

138

62

14