In AI FORA, three research approaches bridging the gap between the technical and the social sciences (empirical social research, scenario simulation from computational social science, and AI experimentation as participatory modelling) will be combined and tri-angulated addressing these issues of technology assessment (TA).
Case studies for empirical social research
For empirical social research, a mixed-methods approach is used that will incorporate the methodological development and conduction of participatory multi-stakeholder workshops as conceptualised within the AI FORA Planning Grant, interviews with stakeholders, and media discourse analysis in each of the five carefully-selected case study countries. Case study partners having selected an AI-based social assessment domain that created considerable public debate within their country while being accessible for socio-technical investigation (Germany: a; Estonia: b; USA: c; China: d; India: e) will lead and organise the context-specific elements of the research process.
Applying a methodologically-rephrased version of Hofstede's cultural dimensions theory and factor analysis model set up to describe the effects of a society's culture on the values and value-related behaviour of its members, AI FORA will empirically investigate and compare the effects and future role of increased AI use in value judgements made by social assessment practices across societies on the globe.
For all case studies, empirical research will centre around the organisation and coordination of multi-stakeholder engagement on case study level. Relying on participatory and interactive formats for qualitative research, AI FORA will firstly investigate how the chosen social assessment domain had been organised without machines, which norms and values had been embedded in organisational practices, and what institutional infrastructures, policies and public debate had supported non-machine social assessment routines. Secondly, AI FORA’s empirical research will analyse how and up to what degree such routines have been substituted or changed by AI, where machine and non-machine social assessment practices differ in value implementations, and how institutional infrastructures, policies and public debate react on AI use in the chosen social assessment domain. Focus groups in the different cultural settings will explore possible scenarios of potential social value change induced by the current developments in the different societies.
Scenario simulations of Computational Social Science
Using scenario simulation, AI FORA will apply agent-based modelling (ABM) informed and calibrated by empirical data stemming from the research above to investigate on the level of case studies, whether and with what consequences there is a trend towards AI-induced change of social value frameworks responding to changed social assessment practices. Building on the participatory workshop formats and local focus groups of the different cultural settings above, which have explored possible scenarios of potential social value change induced by the current developments in the different societies, AI FORA will build an ABM simulation for an ex-ante investigation of potential future developments. Agent-based simulation enables studying “what-if” questions of the mechanisms of potential scenarios that provide a test-bed for policy interventions. Specifically, the simulation will mimic real-life discussions on AI-based social assessment systems and the recursive relation between social change and technological development. For this purpose, the simulation will examine and consequently induce changes in an artificial AI-based social assessment system that is subject of the virtual assessment in the simulation.
AI FORA will employ a participatory Companion Modeling approach (ComMod) to ABM simulation, where multiple stakeholders are involved by default in the modeling processes of object formation, problem definition, discussion, and elaboration to capture the complex contextual dynamics with heterogeneous interests in conflict-prone discussion and negotiation arenas with high uncertainty requiring many loops, de-briefings and societal reflections. Simulations experiments will be set up to analyse whether and how AI use can be expected to change societies – their social assessment routines and practices, but also their general value reference framework – and what consequences and effects this change might have on future societies. Furthermore, AI FORA will experiment with policies and interventions to investigate their requirements and appropriateness to support or prevent certain scenarios for impact assessment and ex-ante evaluation of potential developments and trends.
AI experimentation for social assessment technologies in the co-creation lab
Last but not least, using the insights from social research and scenario modelling, AI FORA will join up forces with society for helping to build better, i.e. context-sensitive, socially-informed AI for future societies. Building on the demonstrator and feasibility study from the Planning Grant phase, the project will further conceptualise and develop a co-creation methodology and an experimental lab infrastructure for building and critically discussing AI social assessment technologies in a user-friendly fab lab / living lab environment located at one of the technical partners’ venue.
In this co-creation lab, stakeholders will experiment with and develop AI-assisted/AI-based social assessment applications with a variety of tools that are needed for such purpose. The lab environment will provide stakeholders with a range of data sets, applications, algorithms, as well as training and education material for co-creative development. Building on the background of high-level multi-stakeholder initiatives to integrate societal and ethical issues in AI research (e.g. AI4EU, Humane AI etc.), this experience-based lab context will implement, test and evaluate participatory technology co-design on the operational ground level of local work practices, work routines and workplace settings to bring societal values and need to the very heart of AI technology production.