Itinai.com mockup of branding agency website on laptop. moder 03f172b9 e6d0 45d8 b393 c8a3107c17e2 2
Itinai.com mockup of branding agency website on laptop. moder 03f172b9 e6d0 45d8 b393 c8a3107c17e2 2

Contextual SDG Research Identification: An AI Evaluation Agent Methodology

Contextual SDG Research Identification: An AI Evaluation Agent Methodology

Universities and Global Competition

Universities are facing tough competition worldwide. Their rankings are increasingly linked to the United Nations’ Sustainable Development Goals (SDGs), which assess their social impact. These rankings affect funding, reputation, and student recruitment.

Challenges with Current Research Tracking

Currently, tracking SDG-related research relies on traditional keyword searches in academic databases. This method often misclassifies papers as relevant even when they do not contribute meaningfully to SDG goals.

Limitations of Traditional Methods

Existing research has attempted to improve the identification of SDG-related research but still struggles to separate meaningful contributions from superficial mentions.

Innovative AI Solutions

Researchers from Virginia Tech Libraries have proposed a new AI approach to identify SDG research. This method uses a specialized Large Language Model (LLM) to evaluate research abstracts and identify genuine contributions to SDGs.

Structured Evaluation Process

The AI evaluates research based on structured guidelines, focusing on measurable actions that align with SDG objectives. It uses data science and text analytics for a deeper understanding of context and language.

Research Methodology

The researchers analyzed 20,000 journal articles and conference abstracts for each of the 17 SDGs, using Scopus as a data source. They utilized search queries from Elsevier’s SDG Research Mapping Initiative, recognizing the interconnectedness of the goals.

Implementation of LLMs

The evaluation agent employs three compact LLMs: Microsoft’s Phi-3.5-mini-instruct, Mistral-7B-Instruct-v0.3, and Meta’s Llama-3.2-3B-Instruct. These models are efficient, easy to host locally, and capable of precise classification of abstracts.

Results and Insights

The research shows different LLMs interpret relevance variably. For instance, Phi-3.5-mini labels 52% of abstracts as ‘Relevant’, while Llama-3.2 is much stricter, marking only 15% as ‘Relevant’.

Conclusion and Future Directions

This research highlights the potential of small LLMs in accurately classifying research related to SDGs. Despite promising results, the researchers note limitations, including the sensitivity of prompt design and the focus on SDG 1.

Get Involved

Check out the full paper for more insights. Follow us on Twitter, join our Telegram Channel, and connect on LinkedIn. Subscribe to our newsletter for updates and insights.

Transform Your Organization with AI

Embrace AI to enhance your operations:

  • Identify Automation Opportunities: Find key areas for AI integration.
  • Define KPIs: Ensure measurable impacts from AI initiatives.
  • Select an AI Solution: Choose tools that fit your needs.
  • Implement Gradually: Start small, gather data, and expand.

Contact Us

For AI management advice, reach out at hello@itinai.com. Stay updated on AI insights through our Telegram and Twitter channels.

List of Useful Links:

Itinai.com office ai background high tech quantum computing 0002ba7c e3d6 4fd7 abd6 cfe4e5f08aeb 0

Vladimir Dyachkov, Ph.D
Editor-in-Chief itinai.com

I believe that AI is only as powerful as the human insight guiding it.

Unleash Your Creative Potential with AI Agents

Competitors are already using AI Agents

Business Problems We Solve

  • Automation of internal processes.
  • Optimizing AI costs without huge budgets.
  • Training staff, developing custom courses for business needs
  • Integrating AI into client work, automating first lines of contact

Large and Medium Businesses

Startups

Offline Business

100% of clients report increased productivity and reduced operati

AI news and solutions