Keynote talk by Associate Professor Jahna Otterbacher at the 19th International Conference On Innovations In Intelligent Systems And Applications (INISTA 2025)

Dr Jahna Otterbacher, Associate Professor at the Faculty of Pure and Applied Sciences of the Open University of Cyprus and co-director of Cyprus Center for Trustworthy AI (CyCAT), is a keynote speaker at the 19th International Conference on Innovations in Intelligent Systems and Applications (INISTA 2025) held in Ras Al Khaimah, UAE, on October 29-31, 2025. The title of her talk is: “Trust in and Trustworthiness of AI: Building Socio-Technical Infrastructure for Assessing AI Alignment on the Large Scale.

Abstract: Generative AI and its foundation models dominate the conversations on the potential of AI to transform nearly every aspect of human digital activity. Current models are powerful and multimodal, generating “human-like” responses to prompts across vast subject domains, and exhibiting increasing understanding of linguistic and visual expression. Foundation models, which are easily adaptable for use in downstream applications, are transforming AI into a general-purpose technology; it is impossible to predict the range of innovative applications that will result. System-focused evaluations (i.e., capability testing) are key to understanding a model’s range of behaviors but tell us little about how the user experiences an application built upon it, and the extent to which ethical principles (e.g., privacy, fairness, accountability) are perceived as being respected. Thus, there is a crucial need to involve the public in evaluation, as to assess the alignment of (system) trustworthiness and (user) trust. In this talk, I will share insights from our efforts to build socio-technical infrastructure for performing such evaluations on the large scale. Specifically, we developed a methodology for user-sourcing evaluations of deployed AI applications in the wild. Tasks based on this methodology have been integrated into our open and distance-learning course “AI in Everyday Life,” offered to members of the public. The key challenges, as well as our plan for scaling up the approach, will be discussed.