Measuring the impact of artificial intelligence in the public sector: towards a practical and trustworthy evaluation framework

Governments are increasingly adopting artificial intelligence (AI) to improve public service delivery and policymaking. However, there is no widely accepted framework for evaluating the actual impact of AI in the public sector. Existing approaches often prioritise either technical performance or abstract ethical principles, with limited attention to measurable outcomes that reflect both effectiveness and democratic values. This PhD project aims to develop and validate a practical, multidimensional evaluation framework for public sector AI. By integrating both quantitative performance indicators and qualitative public value dimensions—such as trust, accountability, and transparency—the research addresses a critical gap in AI governance. The framework will be tested through case studies in Estonia and other digitally advanced countries, offering both theoretical innovation and actionable guidance for policymakers.

Governments worldwide are increasingly experimenting with artificial intelligence (AI) to enhance public service delivery, policymaking, and administrative efficiency. However, there is a growing need to understand not only how AI is being implemented, but also what impact it has on public service delivery, organizational performance, and democratic governance. While policy documents and AI strategies often highlight broad benefits—such as increased efficiency, innovation, and better decision-making—there remains a gap in systematically evaluating whether these benefits are realised in practice. At the same time, there is a pressing need to consider the broader societal and ethical implications of AI adoption in public administration. This includes questions of transparency, accountability, equity, and trust in government.

There is a lack of the frameworks for evaluating the actual impact of AI in the public sector. Most existing evaluations focus either on technical performance or broad ethical principles, with limited attention given to measurable outcomes or the interplay between innovation and governance. This gap limits the ability of governments to make evidence-based, value-driven decisions about AI deployment. A flexible framework is needed – one that provides a common foundation but can be customized for different domains or national contexts.

The goal of the PhD project is to develop a multi-dimensional, modular framework that evaluates AI implementations along three main axes: 1) Efficiency and Performance: Including traditional public administration metrics such as cost-effectiveness, service quality, and operational improvement. 2) Trust and Accountability: Assessing dimensions such as transparency, fairness, explainability, and citizen trust—currently underrepresented in most evaluation models. 3) Readiness and Sustainability: Focusing on organizational capacities, data quality, legal compliance, skills development, and long-term governance.

The proposed framework builds on and extends earlier academic models but differs in its emphasis on balance between quantitative KPIs and qualitative democratic values, practical applicability in multiple domains, and alignment with real-world policymaking needs. Using Estonia as a living laboratory—given its advanced digital infrastructure and proactive AI strategy—the research will analyse real-life AI use cases. Through interviews, workshops, and participatory research with Estonian public sector institutions, the framework will be iteratively refined and validated.

The project’s novelty lies in its focus on bridging theory and practice: offering not just conceptual tools, but also practical guidance, standardised indicators, and policy recommendations tailored to public sector contexts. In doing so, it will help public administrations govern AI systems in a way that is both effective and aligned with democratic principles.

The research should focus on the following questions:

1. What are the key indicators—both quantitative and qualitative—for evaluating the impact of AI in the public sector?
2. To what extent can existing evaluation frameworks support cross-sectoral, evidence-based policymaking in the context of AI deployment?
3. How can public institutions integrate ethical and accountability concerns into measurable evaluation processes?
4. What conditions and capacities enable or hinder meaningful AI impact evaluation at the organizational level?
5. How can a modular framework be designed to balance flexibility (for local contexts) with comparability (across cases)?

Responsibilities and tasks:

1. Map and critically review existing academic and policy evaluation frameworks for for assessing AI in public administration at national and international levels.
2. Examine the applicability and limitations of traditional evaluation methods (e.g., cost-benefit analysis, socio-economic impact assessment) in the context of AI.
3. Design a modular AI impact evaluation framework tailored to the needs of public sector organizations and aligned with principles of trustworthy AI.
4. Empirically test the framework through case studies of AI use in Estonia and other digitally advanced countries.
5. Develop standardised metrics and indicators for assessing both performance and public value dimensions.
6. Disseminate results through academic publications, practical toolkits, and stakeholder workshops.

Applicants should fulfil the following requirements:

• a master’s degree (preferably in ICT, public administration, or a related field)
• public sector work experience experience in AI-related policymaking, implementation, or research
• strong analytical skills and knowledge of digital governance
• excellent command of English
• strong and demonstrable writing and analytical skills
• capacity to work both as an independent researcher and as part of an international team

The candidate should submit a research plan for the topic, including the overall research and data collection strategy. The candidate can expand on the listed questions and tasks and propose theoretical lenses to be used.