Research
Publications
Algorithmic Bias Algorithmic Fairness Algorithmic Transparency Artificial Intelligence Diversity Education Human-Computer Interaction Information Retrieval Information Studies Information Systems Language Diversity
Monica Lestari Paramita Kalia Orphanou, Evgenia Christoforou Jahna Otterbacher Frank Hopfgartner Do you see what I see? Images of the COVID-19 pandemic through the lens of Google Journal Article Information Processing & Management, 2021. Abstract | Links | BibTeX | Tags: Algorithmic Bias, Artificial Intelligence @article{Paramita2021, title = {Do you see what I see? Images of the COVID-19 pandemic through the lens of Google}, author = {Monica Lestari Paramita, Kalia Orphanou, Evgenia Christoforou, Jahna Otterbacher, Frank Hopfgartner}, url = {https://www.sciencedirect.com/science/article/pii/S0306457321001424}, year = {2021}, date = {2021-09-05}, journal = {Information Processing & Management}, abstract = {During times of crisis, information access is crucial. Given the opaque processes behind modern search engines, it is important to understand the extent to which the “picture” of the Covid-19 pandemic accessed by users differs. We explore variations in what users “see” concerning the pandemic through Google image search, using a two-step approach. First, we crowdsource a search task to users in four regions of Europe, asking them to help us create a photo documentary of Covid-19 by providing image search queries. Analysing the queries, we find five common themes describing information needs. Next, we study three sources of variation – users’ information needs, their geo-locations and query languages – and analyse their influences on the similarity of results. We find that users see the pandemic differently depending on where they live, as evidenced by the 46% similarity across results. When users expressed a given query in different languages, there was no overlap for most of the results. Our analysis suggests that localisation plays a major role in the (dis)similarity of results, and provides evidence of the diverse “picture” of the pandemic seen through Google.}, keywords = {Algorithmic Bias, Artificial Intelligence}, pubstate = {published}, tppubtype = {article} } During times of crisis, information access is crucial. Given the opaque processes behind modern search engines, it is important to understand the extent to which the “picture” of the Covid-19 pandemic accessed by users differs. We explore variations in what users “see” concerning the pandemic through Google image search, using a two-step approach. First, we crowdsource a search task to users in four regions of Europe, asking them to help us create a photo documentary of Covid-19 by providing image search queries. Analysing the queries, we find five common themes describing information needs. Next, we study three sources of variation – users’ information needs, their geo-locations and query languages – and analyse their influences on the similarity of results. We find that users see the pandemic differently depending on where they live, as evidenced by the 46% similarity across results. When users expressed a given query in different languages, there was no overlap for most of the results. Our analysis suggests that localisation plays a major role in the (dis)similarity of results, and provides evidence of the diverse “picture” of the pandemic seen through Google. |
Maria Kasinidou Styliani Kleanthous, Jahna Otterbacher ‘Expected Most of the Results, but Some Others... Surprised Me’: Personality Inference in Image Tagging Services Inproceedings Fogli, Daniela ; Tetteroo, Daniel ; Barricelli, Barbara Rita ; Borsci, Simone ; Markopoulos, Panos ; Papadopoulos, George A (Ed.): International Symposium on End User Development, pp. 187–195, Springer, 2021, ISBN: 978-3-030-79840-6. Abstract | Links | BibTeX | Tags: Algorithmic Bias @inproceedings{Kasinidou2021c, title = {‘Expected Most of the Results, but Some Others... Surprised Me’: Personality Inference in Image Tagging Services}, author = {Maria Kasinidou, Styliani Kleanthous, Jahna Otterbacher}, editor = {Fogli, Daniela and Tetteroo, Daniel and Barricelli, Barbara Rita and Borsci, Simone and Markopoulos, Panos and Papadopoulos, George A.}, url = {https://link.springer.com/chapter/10.1007/978-3-030-79840-6_12}, isbn = {978-3-030-79840-6}, year = {2021}, date = {2021-07-06}, booktitle = {International Symposium on End User Development}, pages = {187--195}, publisher = {Springer}, series = { IS-EUD '21}, abstract = {Image tagging APIs, offered as Cognitive Services in the movement to democratize AI, have become popular in applications that need to provide a personalized user experience. Developers can easily incorporate these services into their applications; however, little is known concerning their behavior under specific circumstances. We consider how two such services behave when predicting elements of the Big-Five personality traits from users’ profile images. We found that personality traits are not equally represented in the APIs’ output tags, with tags focusing mostly on Extraversion. The inaccurate personality prediction and the lack of vocabulary for the equal representation of all personality traits, could result in unreliable implicit user modeling, resulting in sub-optimal – or even undesirable – user experience in the application.}, keywords = {Algorithmic Bias}, pubstate = {published}, tppubtype = {inproceedings} } Image tagging APIs, offered as Cognitive Services in the movement to democratize AI, have become popular in applications that need to provide a personalized user experience. Developers can easily incorporate these services into their applications; however, little is known concerning their behavior under specific circumstances. We consider how two such services behave when predicting elements of the Big-Five personality traits from users’ profile images. We found that personality traits are not equally represented in the APIs’ output tags, with tags focusing mostly on Extraversion. The inaccurate personality prediction and the lack of vocabulary for the equal representation of all personality traits, could result in unreliable implicit user modeling, resulting in sub-optimal – or even undesirable – user experience in the application. |
Maria Kasinidou Styliani Kleanthous, Kalia Orphanou Jahna Otterbacher Association for Computing Machinery, 2021, ISBN: 9781450382144. Abstract | Links | BibTeX | Tags: Algorithmic Fairness, Algorithmic Transparency @proceedings{Kasinidou2021b, title = {Educating Computer Science Students about Algorithmic Fairness, Accountability, Transparency and Ethics}, author = {Maria Kasinidou, Styliani Kleanthous, Kalia Orphanou, Jahna Otterbacher}, url = {https://dl.acm.org/doi/abs/10.1145/3430665.3456311}, doi = {10.1145/3430665.3456311}, isbn = {9781450382144}, year = {2021}, date = {2021-06-26}, publisher = {Association for Computing Machinery}, series = {ITiCSE '21}, abstract = {Professionals are increasingly relying on algorithmic systems for decision making however, algorithmic decisions occasionally perceived as biased or not just. Prior work has provided evidences that education can make a difference on the perception of young developers on algorithmic fairness. In this paper, we investigate computer science students' perception of FATE in algorithmic decision-making and whether their views on FATE can be changed by attending a seminar on FATE topics. Participants attended a seminar on FATE in algorithmic decision-making and they were asked to respond to two online questionnaires to measure their pre- and post-seminar perception on FATE. Results show that a short seminar can make a difference in understanding and perception as well as the attitude of the students towards FATE in algorithmic decision support. CS curricula need to be updated and include FATE topics if we want algorithmic decision support systems to be just for all.}, keywords = {Algorithmic Fairness, Algorithmic Transparency}, pubstate = {published}, tppubtype = {proceedings} } Professionals are increasingly relying on algorithmic systems for decision making however, algorithmic decisions occasionally perceived as biased or not just. Prior work has provided evidences that education can make a difference on the perception of young developers on algorithmic fairness. In this paper, we investigate computer science students' perception of FATE in algorithmic decision-making and whether their views on FATE can be changed by attending a seminar on FATE topics. Participants attended a seminar on FATE in algorithmic decision-making and they were asked to respond to two online questionnaires to measure their pre- and post-seminar perception on FATE. Results show that a short seminar can make a difference in understanding and perception as well as the attitude of the students towards FATE in algorithmic decision support. CS curricula need to be updated and include FATE topics if we want algorithmic decision support systems to be just for all. |
Fausto Giunchiglia Styliani Kleanthous, Jahna Otterbacher Tim Draws Transparency Paths - Documenting the Diversity of User Perceptions Inproceedings Adjunct Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization, Association for Computing Machinery, 2021, ISBN: 9781450383677. Abstract | Links | BibTeX | Tags: Algorithmic Transparency, Diversity @inproceedings{Giunchiglia2021, title = {Transparency Paths - Documenting the Diversity of User Perceptions}, author = {Fausto Giunchiglia, Styliani Kleanthous, Jahna Otterbacher, Tim Draws}, url = {https://dl.acm.org/doi/abs/10.1145/3450614.3463292}, doi = {10.1145/3450614.3463292}, isbn = {9781450383677}, year = {2021}, date = {2021-06-21}, booktitle = {Adjunct Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization}, publisher = {Association for Computing Machinery}, series = {UMAP '21}, abstract = {We are living in an era of global digital platforms, eco-systems of algorithmic processes that serve users worldwide. However, the increasing exposure to diversity online – of information and users – has led to important considerations of bias. A given platform, such as the Google search engine, may demonstrate behaviors that deviate from what users expect, or what they consider fair, relative to their own context and experiences. In this exploratory work, we put forward the notion of transparency paths, a process by which we document our position, choices, and perceptions when developing and/or using algorithmic platforms. We conducted a self-reflection exercise with seven researchers, who collected and analyzed two sets of images; one depicting an everyday activity, “washing hands,” and a second depicting the concept of “home.” Participants had to document their process and choices, and in the end, compare their work to others. Finally, participants were asked to reflect on the definitions of bias and diversity. The exercise revealed the range of perspectives and approaches taken, underscoring the need for future work that will refine the transparency paths methodology.}, keywords = {Algorithmic Transparency, Diversity}, pubstate = {published}, tppubtype = {inproceedings} } We are living in an era of global digital platforms, eco-systems of algorithmic processes that serve users worldwide. However, the increasing exposure to diversity online – of information and users – has led to important considerations of bias. A given platform, such as the Google search engine, may demonstrate behaviors that deviate from what users expect, or what they consider fair, relative to their own context and experiences. In this exploratory work, we put forward the notion of transparency paths, a process by which we document our position, choices, and perceptions when developing and/or using algorithmic platforms. We conducted a self-reflection exercise with seven researchers, who collected and analyzed two sets of images; one depicting an everyday activity, “washing hands,” and a second depicting the concept of “home.” Participants had to document their process and choices, and in the end, compare their work to others. Finally, participants were asked to reflect on the definitions of bias and diversity. The exercise revealed the range of perspectives and approaches taken, underscoring the need for future work that will refine the transparency paths methodology. |
Pınar Barlas Kyriakos Kyriakou, Styliani Kleanthous Jahna Otterbacher Person, Human, Neither: The Dehumanization Potential of Automated Image Tagging Proceeding 2021, ISBN: 9781450384735. Abstract | Links | BibTeX | Tags: Artificial Intelligence @proceedings{Barlas2021, title = {Person, Human, Neither: The Dehumanization Potential of Automated Image Tagging}, author = {Pınar Barlas, Kyriakos Kyriakou, Styliani Kleanthous, Jahna Otterbacher}, url = {https://dl.acm.org/doi/abs/10.1145/3461702.3462567}, doi = {10.1145/3461702.3462567}, isbn = {9781450384735}, year = {2021}, date = {2021-05-19}, series = {AIES '21}, abstract = {Following the literature on dehumanization via technology, we audit six proprietary image tagging algorithms (ITAs) for their potential to perpetuate dehumanization. We examine the ITAs' outputs on a controlled dataset of images depicting a diverse group of people for tags that indicate the presence of a human in the image. Through an analysis of the (mis)use of these tags, we find that there are some individuals whose 'humanness' is not recognized by an ITA, and that these individuals are often from marginalized social groups. Finally, we compare these findings with the use of the 'face' tag, which can be used for surveillance, revealing that people's faces are often recognized by an ITA even when their 'humanness' is not. Overall, we highlight the subtle ways in which ITAs may inflict widespread, disparate harm, and emphasize the importance of considering the social context of the resulting application.}, keywords = {Artificial Intelligence}, pubstate = {published}, tppubtype = {proceedings} } Following the literature on dehumanization via technology, we audit six proprietary image tagging algorithms (ITAs) for their potential to perpetuate dehumanization. We examine the ITAs' outputs on a controlled dataset of images depicting a diverse group of people for tags that indicate the presence of a human in the image. Through an analysis of the (mis)use of these tags, we find that there are some individuals whose 'humanness' is not recognized by an ITA, and that these individuals are often from marginalized social groups. Finally, we compare these findings with the use of the 'face' tag, which can be used for surveillance, revealing that people's faces are often recognized by an ITA even when their 'humanness' is not. Overall, we highlight the subtle ways in which ITAs may inflict widespread, disparate harm, and emphasize the importance of considering the social context of the resulting application. |
Evgenia Christoforou Pınar Barlas, Jahna Otterbacher It’s About Time: A View of Crowdsourced Data Before and During the Pandemic Proceeding Association for Computing Machinery, 2021, ISBN: 9781450380966. Abstract | Links | BibTeX | Tags: Algorithmic Bias @proceedings{Christoforou2021, title = {It’s About Time: A View of Crowdsourced Data Before and During the Pandemic}, author = {Evgenia Christoforou, Pınar Barlas, Jahna Otterbacher}, url = {https://dl.acm.org/doi/abs/10.1145/3411764.3445317}, doi = {10.1145/3411764.3445317}, isbn = {9781450380966}, year = {2021}, date = {2021-05-08}, publisher = {Association for Computing Machinery}, series = {CHI '21}, abstract = {Data attained through crowdsourcing have an essential role in the development of computer vision algorithms. Crowdsourced data might include reporting biases, since crowdworkers usually describe what is “worth saying” in addition to images’ content. We explore how the unprecedented events of 2020, including the unrest surrounding racial discrimination, and the COVID-19 pandemic, might be reflected in responses to an open-ended annotation task on people images, originally executed in 2018 and replicated in 2020. Analyzing themes of Identity and Health conveyed in workers’ tags, we find evidence that supports the potential for temporal sensitivity in crowdsourced data. The 2020 data exhibit more race-marking of images depicting non-Whites, as well as an increase in tags describing Weight. We relate our findings to the emerging research on crowdworkers’ moods. Furthermore, we discuss the implications of (and suggestions for) designing tasks on proprietary platforms, having demonstrated the possibility for additional, unexpected variation in crowdsourced data due to significant events.}, keywords = {Algorithmic Bias}, pubstate = {published}, tppubtype = {proceedings} } Data attained through crowdsourcing have an essential role in the development of computer vision algorithms. Crowdsourced data might include reporting biases, since crowdworkers usually describe what is “worth saying” in addition to images’ content. We explore how the unprecedented events of 2020, including the unrest surrounding racial discrimination, and the COVID-19 pandemic, might be reflected in responses to an open-ended annotation task on people images, originally executed in 2018 and replicated in 2020. Analyzing themes of Identity and Health conveyed in workers’ tags, we find evidence that supports the potential for temporal sensitivity in crowdsourced data. The 2020 data exhibit more race-marking of images depicting non-Whites, as well as an increase in tags describing Weight. We relate our findings to the emerging research on crowdworkers’ moods. Furthermore, we discuss the implications of (and suggestions for) designing tasks on proprietary platforms, having demonstrated the possibility for additional, unexpected variation in crowdsourced data due to significant events. |
Maria Kasinidou Styliani Kleanthous, Pınar Barlas Jahna Otterbacher Association for Computing Machinery, 2021, ISBN: 9781450383097. Abstract | Links | BibTeX | Tags: Algorithmic Bias, Algorithmic Fairness, Algorithmic Transparency @proceedings{Kasinidou2021, title = {I agree with the decision, but they didn't deserve this: Future Developers' Perception of Fairness in Algorithmic Decisions}, author = {Maria Kasinidou, Styliani Kleanthous, Pınar Barlas, Jahna Otterbacher}, url = {https://dl.acm.org/doi/abs/10.1145/3442188.3445931}, doi = {10.1145/3442188.3445931}, isbn = {9781450383097}, year = {2021}, date = {2021-03-08}, publisher = {Association for Computing Machinery}, series = {FAccT '21}, abstract = {While professionals are increasingly relying on algorithmic systems for making a decision, on some occasions, algorithmic decisions may be perceived as biased or not just. Prior work has looked into the perception of algorithmic decision-making from the user's point of view. In this work, we investigate how students in fields adjacent to algorithm development perceive algorithmic decisionmaking. Participants (N=99) were asked to rate their agreement with statements regarding six constructs that are related to facets of fairness and justice in algorithmic decision-making in three separate scenarios. Two of the three scenarios were independent of each other, while the third scenario presented three different outcomes of the same algorithmic system, demonstrating perception changes triggered by different outputs. Quantitative analysis indicates that a) 'agreeing' with a decision does not mean the person 'deserves the outcome', b) perceiving the factors used in the decision-making as 'appropriate' does not make the decision of the system 'fair' and c) perceiving a system's decision as 'not fair' is affecting the participants' 'trust' in the system. In addition, participants found proportional distribution of benefits more fair than other approaches. Qualitative analysis provides further insights into that information the participants find essential to judge and understand an algorithmic decision-making system's fairness. Finally, the level of academic education has a role to play in the perception of fairness and justice in algorithmic decision-making.}, keywords = {Algorithmic Bias, Algorithmic Fairness, Algorithmic Transparency}, pubstate = {published}, tppubtype = {proceedings} } While professionals are increasingly relying on algorithmic systems for making a decision, on some occasions, algorithmic decisions may be perceived as biased or not just. Prior work has looked into the perception of algorithmic decision-making from the user's point of view. In this work, we investigate how students in fields adjacent to algorithm development perceive algorithmic decisionmaking. Participants (N=99) were asked to rate their agreement with statements regarding six constructs that are related to facets of fairness and justice in algorithmic decision-making in three separate scenarios. Two of the three scenarios were independent of each other, while the third scenario presented three different outcomes of the same algorithmic system, demonstrating perception changes triggered by different outputs. Quantitative analysis indicates that a) 'agreeing' with a decision does not mean the person 'deserves the outcome', b) perceiving the factors used in the decision-making as 'appropriate' does not make the decision of the system 'fair' and c) perceiving a system's decision as 'not fair' is affecting the participants' 'trust' in the system. In addition, participants found proportional distribution of benefits more fair than other approaches. Qualitative analysis provides further insights into that information the participants find essential to judge and understand an algorithmic decision-making system's fairness. Finally, the level of academic education has a role to play in the perception of fairness and justice in algorithmic decision-making. |
Kyriakos Kyriakou Pınar Barlas, Styliani Kleanthous Jahna Otterbacher OpenTag: Understanding Human Perceptions of Image Tagging Algorithms Conference HCOMP-20 2020. Abstract | Links | BibTeX | Tags: Artificial Intelligence @conference{Kyriakou2020b, title = {OpenTag: Understanding Human Perceptions of Image Tagging Algorithms}, author = {Kyriakos Kyriakou, Pınar Barlas, Styliani Kleanthous, Jahna Otterbacher}, url = {http://www.cycat.io/hcomp_2020_paper_76-2/}, year = {2020}, date = {2020-10-25}, series = {HCOMP-20}, abstract = {Image Tagging Algorithms (ITAs) are extensively used in our information ecosystem, from facilitating the retrieval of images in social platforms to learning about users and their preferences. However, audits performed on ITAs have demonstrated that their behaviors often exhibit social biases, especially when analyzing images depicting people. We present OpenTag, a platform that fuses the auditing process with a crowdsourcing approach. Users can upload an image, which is then analyzed by various ITAs, resulting in multiple sets of descriptive tags. With OpenTag, the user can observe and compare the output of multiple ITAs simultaneously, while researchers can study the manner in which users perceive this output. Finally, using the collected data, further audits can be performed on ITAs.}, keywords = {Artificial Intelligence}, pubstate = {published}, tppubtype = {conference} } Image Tagging Algorithms (ITAs) are extensively used in our information ecosystem, from facilitating the retrieval of images in social platforms to learning about users and their preferences. However, audits performed on ITAs have demonstrated that their behaviors often exhibit social biases, especially when analyzing images depicting people. We present OpenTag, a platform that fuses the auditing process with a crowdsourcing approach. Users can upload an image, which is then analyzed by various ITAs, resulting in multiple sets of descriptive tags. With OpenTag, the user can observe and compare the output of multiple ITAs simultaneously, while researchers can study the manner in which users perceive this output. Finally, using the collected data, further audits can be performed on ITAs. |
Evgenia Christoforou Pınar Barlas, Jahna Otterbacher Crowdwork as a Snapshot in Time: Image Annotation Tasks during a Pandemic Conference HCOMP-20 2020. Abstract | Links | BibTeX | Tags: Artificial Intelligence @conference{Christoforou2020, title = {Crowdwork as a Snapshot in Time: Image Annotation Tasks during a Pandemic}, author = {Evgenia Christoforou, Pınar Barlas, Jahna Otterbacher}, url = {http://www.cycat.io/hcomp_2020_paper_79/}, year = {2020}, date = {2020-10-25}, series = {HCOMP-20}, abstract = {While crowdsourcing provides a convenient solution for tapping into human intelligence, a concern is the bias inherent in the data collected. Events related to the COVID-19 pandemic had an impact on people globally, and crowdworkers were no exception. Given the evidence concerning mood and stress on work, we explore how temporal events might affect crowdsourced data. We replicated an image annotation task conducted in 2018, in which workers describe people images. We expected 2020 annotations to contain more references to health, as compared to 2018 data. Overall, we find no evidence that health-related tags were used more often in 2020, but instead we find a significant increase in the use of tags related to weight (e.g., fat, chubby, overweight). This result, coupled with the “stay at home” act in effect in 2020, illustrate how crowdwork is impacted by temporal events. }, keywords = {Artificial Intelligence}, pubstate = {published}, tppubtype = {conference} } While crowdsourcing provides a convenient solution for tapping into human intelligence, a concern is the bias inherent in the data collected. Events related to the COVID-19 pandemic had an impact on people globally, and crowdworkers were no exception. Given the evidence concerning mood and stress on work, we explore how temporal events might affect crowdsourced data. We replicated an image annotation task conducted in 2018, in which workers describe people images. We expected 2020 annotations to contain more references to health, as compared to 2018 data. Overall, we find no evidence that health-related tags were used more often in 2020, but instead we find a significant increase in the use of tags related to weight (e.g., fat, chubby, overweight). This result, coupled with the “stay at home” act in effect in 2020, illustrate how crowdwork is impacted by temporal events. |
Barlas, Pınar ; Kyriakou, Kyriakos ; Guest, Olivia ; Kleanthous, Styliani ; Otterbacher, Jahna 2020. Abstract | Links | BibTeX | Tags: Algorithmic Bias, Artificial Intelligence @proceedings{Barlas2020b, title = {To "See" is to Stereotype: Image Tagging Algorithms, Gender Recognition, and the Accuracy-Fairness Trade-off}, author = {Barlas, Pınar and Kyriakou, Kyriakos and Guest, Olivia and Kleanthous, Styliani and Otterbacher, Jahna}, url = {https://dl.acm.org/doi/abs/10.1145/3432931}, doi = {10.1145/3432931}, year = {2020}, date = {2020-10-17}, series = {CSCW3 20}, abstract = {Machine-learned computer vision algorithms for tagging images are increasingly used by developers and researchers, having become popularized as easy-to-use "cognitive services." Yet these tools struggle with gender recognition, particularly when processing images of women, people of color and non-binary individuals. Socio-technical researchers have cited data bias as a key problem; training datasets often over-represent images of people and contexts that convey social stereotypes. The social psychology literature explains that people learn social stereotypes, in part, by observing others in particular roles and contexts, and can inadvertently learn to associate gender with scenes, occupations and activities. Thus, we study the extent to which image tagging algorithms mimic this phenomenon. We design a controlled experiment, to examine the interdependence between algorithmic recognition of context and the depicted person's gender. In the spirit of auditing to understand machine behaviors, we create a highly controlled dataset of people images, imposed on gender-stereotyped backgrounds. Our methodology is reproducible and our code publicly available. Evaluating five proprietary algorithms, we find that in three, gender inference is hindered when a background is introduced. Of the two that "see" both backgrounds and gender, it is the one whose output is most consistent with human stereotyping processes that is superior in recognizing gender. We discuss the accuracy--fairness trade-off, as well as the importance of auditing black boxes in better understanding this double-edged sword.}, keywords = {Algorithmic Bias, Artificial Intelligence}, pubstate = {published}, tppubtype = {proceedings} } Machine-learned computer vision algorithms for tagging images are increasingly used by developers and researchers, having become popularized as easy-to-use "cognitive services." Yet these tools struggle with gender recognition, particularly when processing images of women, people of color and non-binary individuals. Socio-technical researchers have cited data bias as a key problem; training datasets often over-represent images of people and contexts that convey social stereotypes. The social psychology literature explains that people learn social stereotypes, in part, by observing others in particular roles and contexts, and can inadvertently learn to associate gender with scenes, occupations and activities. Thus, we study the extent to which image tagging algorithms mimic this phenomenon. We design a controlled experiment, to examine the interdependence between algorithmic recognition of context and the depicted person's gender. In the spirit of auditing to understand machine behaviors, we create a highly controlled dataset of people images, imposed on gender-stereotyped backgrounds. Our methodology is reproducible and our code publicly available. Evaluating five proprietary algorithms, we find that in three, gender inference is hindered when a background is introduced. Of the two that "see" both backgrounds and gender, it is the one whose output is most consistent with human stereotyping processes that is superior in recognizing gender. We discuss the accuracy--fairness trade-off, as well as the importance of auditing black boxes in better understanding this double-edged sword. |
Barlas, Pınar ; Kyriakou, Kyriakos ; Chrysanthou, Antrea ; Kleanthous, Styliani ; Otterbacher, Jahna OPIAS: Over-Personalization in Information Access Systems Inproceedings Adjunct Publication of the 28th ACM Conference on User Modeling, Adaptation and Personalization, pp. 103–104, 2020, ISBN: 9781450379502. Links | BibTeX | Tags: Artificial Intelligence @inproceedings{Barlas2020, title = {OPIAS: Over-Personalization in Information Access Systems}, author = {Barlas, Pınar and Kyriakou, Kyriakos and Chrysanthou, Antrea and Kleanthous, Styliani and Otterbacher, Jahna}, url = {https://dl.acm.org/doi/abs/10.1145/3386392.3397607}, doi = {10.1145/3386392.3397607}, isbn = {9781450379502}, year = {2020}, date = {2020-07-12}, booktitle = {Adjunct Publication of the 28th ACM Conference on User Modeling, Adaptation and Personalization}, pages = {103–104}, series = {UMAP '20 Adjunct}, keywords = {Artificial Intelligence}, pubstate = {published}, tppubtype = {inproceedings} } |
Kyriakou, Kyriakos ; Kleanthous, Styliani ; Otterbacher, Jahna ; Papadopoulos, George A Emotion-based Stereotypes in Image Analysis Services Inproceedings Adjunct Publication of the 28th ACM Conference on User Modeling, Adaptation and Personalization, pp. 252–259, 2020, ISBN: 9781450379502. Abstract | Links | BibTeX | Tags: Artificial Intelligence @inproceedings{Kyriakou2020, title = {Emotion-based Stereotypes in Image Analysis Services}, author = {Kyriakou, Kyriakos and Kleanthous, Styliani and Otterbacher, Jahna and Papadopoulos, George A.}, url = {https://dl.acm.org/doi/abs/10.1145/3386392.3399567}, doi = {10.1145/3386392.3399567}, isbn = {9781450379502}, year = {2020}, date = {2020-07-12}, booktitle = {Adjunct Publication of the 28th ACM Conference on User Modeling, Adaptation and Personalization}, pages = {252–259}, series = {UMAP '20 Adjunct}, abstract = {Vision-based cognitive services (CogS) have become crucial in a wide range of applications, from real-time security and social networks to smartphone applications. Many services focus on analyzing people images. When it comes to facial analysis, these services can be misleading or even inaccurate, raising ethical concerns such as the amplification of social stereotypes. We analyzed popular Image Tagging CogS that infer emotion from a person's face, considering whether they perpetuate racial and gender stereotypes concerning emotion. By comparing both CogS and Human-generated descriptions on a set of controlled images, we highlight the need for transparency and fairness in CogS. In particular, we document evidence that CogS may actually be more likely than crowdworkers to perpetuate the stereotype of the "angry black man" and often attribute black race individuals with "emotions of hostility".}, keywords = {Artificial Intelligence}, pubstate = {published}, tppubtype = {inproceedings} } Vision-based cognitive services (CogS) have become crucial in a wide range of applications, from real-time security and social networks to smartphone applications. Many services focus on analyzing people images. When it comes to facial analysis, these services can be misleading or even inaccurate, raising ethical concerns such as the amplification of social stereotypes. We analyzed popular Image Tagging CogS that infer emotion from a person's face, considering whether they perpetuate racial and gender stereotypes concerning emotion. By comparing both CogS and Human-generated descriptions on a set of controlled images, we highlight the need for transparency and fairness in CogS. In particular, we document evidence that CogS may actually be more likely than crowdworkers to perpetuate the stereotype of the "angry black man" and often attribute black race individuals with "emotions of hostility". |
Chrysanthou, Antrea ; Barlas, Pınar ; Kyriakou, Kyriakos ; Kleanthous, Styliani ; Otterbacher, Jahna Bursting the Bubble: Tool for Awareness and Research about Overpersonalization in Information Access Systems Inproceedings Proceedings of the 25th International Conference on Intelligent User Interfaces Companion, pp. 112–113, 2020, ISBN: 9781450375139. Abstract | Links | BibTeX | Tags: Artificial Intelligence @inproceedings{chrysanthou2020bursting, title = {Bursting the Bubble: Tool for Awareness and Research about Overpersonalization in Information Access Systems}, author = {Chrysanthou, Antrea and Barlas, Pınar and Kyriakou, Kyriakos and Kleanthous, Styliani and Otterbacher, Jahna}, url = {https://dl.acm.org/doi/abs/10.1145/3379336.3381863}, doi = {10.1145/3379336.3381863}, isbn = {9781450375139}, year = {2020}, date = {2020-03-17}, booktitle = {Proceedings of the 25th International Conference on Intelligent User Interfaces Companion}, pages = {112–113}, series = {IUI '20}, abstract = {Modern information access systems extensively use personalization, automatically filtering and/or ranking content based on the user profile, to guide users to the most relevant material. However, this can also lead to unwanted effects such as the "filter bubble." We present an interactive demonstration system, designed as an educational and research tool, which imitates a search engine, personalizing the search results returned for a query based on the user's characteristics. The system can be tailored to suit any type of audience and context, as well as enabling the collection of responses and interaction data.}, keywords = {Artificial Intelligence}, pubstate = {published}, tppubtype = {inproceedings} } Modern information access systems extensively use personalization, automatically filtering and/or ranking content based on the user profile, to guide users to the most relevant material. However, this can also lead to unwanted effects such as the "filter bubble." We present an interactive demonstration system, designed as an educational and research tool, which imitates a search engine, personalizing the search results returned for a query based on the user's characteristics. The system can be tailored to suit any type of audience and context, as well as enabling the collection of responses and interaction data. |
Otterbacher, Jahna ; Barlas, Pınar ; Kleanthous, Styliani ; Kyriakou, Kyriakos How Do We Talk about Other People? Group (Un)Fairness in Natural Language Image Descriptions Inproceedings Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, pp. 106-114, 2019. Abstract | Links | BibTeX | Tags: Artificial Intelligence @inproceedings{otterbacher2019we, title = {How Do We Talk about Other People? Group (Un)Fairness in Natural Language Image Descriptions}, author = {Otterbacher, Jahna and Barlas, Pınar and Kleanthous, Styliani and Kyriakou, Kyriakos}, url = {https://ojs.aaai.org/index.php/HCOMP/article/view/5267}, year = {2019}, date = {2019-10-28}, booktitle = {Proceedings of the AAAI Conference on Human Computation and Crowdsourcing}, volume = {7}, number = {1}, pages = {106-114}, series = {HCOMP-19}, abstract = {Crowdsourcing plays a key role in developing algorithms for image recognition or captioning. Major datasets, such as MS COCO or Flickr30K, have been built by eliciting natural language descriptions of images from workers. Yet such elicitation tasks are susceptible to human biases, including stereotyping people depicted in images. Given the growing concerns surrounding discrimination in algorithms, as well as in the data used to train them, it is necessary to take a critical look at this practice. We conduct experiments at Figure Eight using a controlled set of people images. Men and women of various races are positioned in the same manner, wearing a grey t-shirt. We prompt workers for 10 descriptive labels, and consider them using the human-centric approach, which assumes reporting bias. We find that “what’s worth saying” about these uniform images often differs as a function of the gender and race of the depicted person, violating the notion of group fairness. Although this diversity in natural language people descriptions is expected and often beneficial, it could result in automated disparate impact if not managed properly.}, keywords = {Artificial Intelligence}, pubstate = {published}, tppubtype = {inproceedings} } Crowdsourcing plays a key role in developing algorithms for image recognition or captioning. Major datasets, such as MS COCO or Flickr30K, have been built by eliciting natural language descriptions of images from workers. Yet such elicitation tasks are susceptible to human biases, including stereotyping people depicted in images. Given the growing concerns surrounding discrimination in algorithms, as well as in the data used to train them, it is necessary to take a critical look at this practice. We conduct experiments at Figure Eight using a controlled set of people images. Men and women of various races are positioned in the same manner, wearing a grey t-shirt. We prompt workers for 10 descriptive labels, and consider them using the human-centric approach, which assumes reporting bias. We find that “what’s worth saying” about these uniform images often differs as a function of the gender and race of the depicted person, violating the notion of group fairness. Although this diversity in natural language people descriptions is expected and often beneficial, it could result in automated disparate impact if not managed properly. |
Kyriakou, Kyriakos; Barlas, Pınar; Kleanthous, Styliani; Otterbacher, Jahna Fairness in Proprietary Image Tagging Algorithms: A Cross-Platform Audit on People Images Conference ICWSM 2019 AAAI, 2019, ISSN: 2334-0770. Abstract | Links | BibTeX | Tags: Artificial Intelligence @conference{KyriakouICWSM2019, title = {Fairness in Proprietary Image Tagging Algorithms: A Cross-Platform Audit on People Images}, author = {Kyriakos Kyriakou and Pınar Barlas and Styliani Kleanthous and Jahna Otterbacher}, url = {http://www.cycat.io/wp-content/uploads/2019/05/ICWSM_tagging_b_eye_as_v4-2.pdf}, issn = {2334-0770}, year = {2019}, date = {2019-06-15}, publisher = {AAAI}, series = {ICWSM 2019}, abstract = {There are increasing expectations that algorithms should be- have in a manner that is socially just. We consider the case of image tagging APIs and their interpretations of people im- ages. Image taggers have become indispensable in our in- formation ecosystem, facilitating new modes of visual com- munication and sharing. Recently, they have become widely available as Cognitive Services. But while tagging APIs of- fer developers an inexpensive and convenient means to add functionality to their creations, most are opaque and propri- etary. Through a cross-platform comparison of six taggers, we show that behaviors differ significantly. While some of- fer more interpretation on images, they may exhibit less fair- ness toward the depicted persons, by misuse of gender-related tags and/or making judgments on a person’s physical appear- ance. We also discuss the difficulties of studying fairness in situations where algorithmic systems cannot be benchmarked against a ground truth.}, keywords = {Artificial Intelligence}, pubstate = {published}, tppubtype = {conference} } There are increasing expectations that algorithms should be- have in a manner that is socially just. We consider the case of image tagging APIs and their interpretations of people im- ages. Image taggers have become indispensable in our in- formation ecosystem, facilitating new modes of visual com- munication and sharing. Recently, they have become widely available as Cognitive Services. But while tagging APIs of- fer developers an inexpensive and convenient means to add functionality to their creations, most are opaque and propri- etary. Through a cross-platform comparison of six taggers, we show that behaviors differ significantly. While some of- fer more interpretation on images, they may exhibit less fair- ness toward the depicted persons, by misuse of gender-related tags and/or making judgments on a person’s physical appear- ance. We also discuss the difficulties of studying fairness in situations where algorithmic systems cannot be benchmarked against a ground truth. |
Barlas, Pınar; Kyriakou, Kyriakos; Kleanthous, Styliani; Otterbacher, Jahna Social B(eye)as: Human and Machine Descriptions of People Images Conference ICWSM 2019 AAAI, 2019, ISSN: 2334-0770. Abstract | Links | BibTeX | Tags: Artificial Intelligence @conference{BarlasICWSM2019, title = {Social B(eye)as: Human and Machine Descriptions of People Images}, author = {Pınar Barlas and Kyriakos Kyriakou and Styliani Kleanthous and Jahna Otterbacher}, url = {http://www.cycat.io/wp-content/uploads/2019/05/ICWSM_dataset_CAMERAREADY-2.pdf}, issn = {2334-0770}, year = {2019}, date = {2019-06-15}, publisher = {AAAI}, series = {ICWSM 2019}, abstract = {Image analysis algorithms have become an indispensable tool in our information ecosystem, facilitating new forms of visual communication and information sharing. At the same time, they enable large-scale socio-technical research which would otherwise be difficult to carry out. However, their outputs may exhibit social bias, especially when analyzing people images. Since most algorithms are proprietary and opaque, we propose a method of auditing their outputs for social biases. To be able to compare how algorithms interpret a controlled set of people images, we collected descriptions across six image tagging algorithms. In order to compare these results to human behavior, we also collected descriptions on the same images from crowdworkers in two anglophone regions. The dataset we present consists of tags from these eight taggers, along with a typology of concepts, and a python script to calculate vector scores for each image and tagger. Using our methodology, researchers can see the behaviors of the image tagging algorithms and compare them to those of crowdworkers. Beyond computer vision auditing, the dataset of human- and machine-produced tags, the typology, and the vectorization method can be used to explore a range of research questions related to both algorithmic and human behaviors.}, keywords = {Artificial Intelligence}, pubstate = {published}, tppubtype = {conference} } Image analysis algorithms have become an indispensable tool in our information ecosystem, facilitating new forms of visual communication and information sharing. At the same time, they enable large-scale socio-technical research which would otherwise be difficult to carry out. However, their outputs may exhibit social bias, especially when analyzing people images. Since most algorithms are proprietary and opaque, we propose a method of auditing their outputs for social biases. To be able to compare how algorithms interpret a controlled set of people images, we collected descriptions across six image tagging algorithms. In order to compare these results to human behavior, we also collected descriptions on the same images from crowdworkers in two anglophone regions. The dataset we present consists of tags from these eight taggers, along with a typology of concepts, and a python script to calculate vector scores for each image and tagger. Using our methodology, researchers can see the behaviors of the image tagging algorithms and compare them to those of crowdworkers. Beyond computer vision auditing, the dataset of human- and machine-produced tags, the typology, and the vectorization method can be used to explore a range of research questions related to both algorithmic and human behaviors. |
Barlas, Pınar; Kyriakou, Kyriakos; Kleanthous, Styliani; Otterbacher, Jahna What Makes an Image Tagger Fair? - Proprietary Auto-tagging and Interpretations on People Images Conference UMAP 2019 ACM, 2019. Abstract | Links | BibTeX | Tags: Artificial Intelligence @conference{BarlasUMAP2019, title = {What Makes an Image Tagger Fair? - Proprietary Auto-tagging and Interpretations on People Images}, author = {Pınar Barlas and Kyriakos Kyriakou and Styliani Kleanthous and Jahna Otterbacher}, url = {http://www.cycat.io/wp-content/uploads/2019/05/Barlas-et-al.-2019-What-Makes-an-Image-Tagger-Fair-Proprietary-Auto-tagging-and-Interpretations-on-People-Images-1.pdf}, doi = {10.1145/3320435.3320442}, year = {2019}, date = {2019-06-13}, publisher = {ACM}, series = {UMAP 2019}, abstract = {Image analysis algorithms have been a boon to personalization in digital systems and are now widely available via easy-to-use APIs. However, it is important to ensure that they behave fairly in applications that involve processing images of people, such as dating apps. We conduct an experiment to shed light on the factors influencing the perception of “fairness." Participants are shown a photo along with two descriptions (human- and algorithm-generated). They are then asked to indicate which is “more fair" in the context of a dating site, and explain their reasoning. We vary a number of factors, including the gender, race and attractiveness of the person in the photo. While participants generally found human-generated tags to be more fair, API tags were judged as being more fair in one setting - where the image depicted an “attractive," white individual. In their explanations, participants often mention accuracy, as well as the objectivity/subjectivity of the tags in the description. We relate our work to the ongoing conversation about fairness in opaque tools like image tagging APIs, and their potential to result in harm.}, keywords = {Artificial Intelligence}, pubstate = {published}, tppubtype = {conference} } Image analysis algorithms have been a boon to personalization in digital systems and are now widely available via easy-to-use APIs. However, it is important to ensure that they behave fairly in applications that involve processing images of people, such as dating apps. We conduct an experiment to shed light on the factors influencing the perception of “fairness." Participants are shown a photo along with two descriptions (human- and algorithm-generated). They are then asked to indicate which is “more fair" in the context of a dating site, and explain their reasoning. We vary a number of factors, including the gender, race and attractiveness of the person in the photo. While participants generally found human-generated tags to be more fair, API tags were judged as being more fair in one setting - where the image depicted an “attractive," white individual. In their explanations, participants often mention accuracy, as well as the objectivity/subjectivity of the tags in the description. We relate our work to the ongoing conversation about fairness in opaque tools like image tagging APIs, and their potential to result in harm. |
Kleanthous, Styliani; Otterbacher, Jahna HAPPIE 2019 ACM, 2019. Abstract | Links | BibTeX | Tags: Information Retrieval, Information Studies, Information Systems @workshop{KleanthousHAPPIE2019, title = {Shaping the Reaction: Community Characteristics and Emotional Tone of Citizen Responses to Robotics Videos at TED versus YouTube}, author = {Styliani Kleanthous and Jahna Otterbacher}, url = {http://www.cycat.io/wp-content/uploads/2019/05/happ03-kleanthous.pdf}, year = {2019}, date = {2019-06-09}, publisher = {ACM}, series = {HAPPIE 2019}, abstract = {When modelling for the social we need to consider more than one medium. Little is known as to how platform community characteristics shape the discussion and how communicators could best engage each community, taking into consideration these characteristics. We consider comments on TED videos featuring roboticists, shared at TED.com and YouTube. We find evidence of different social norms and importantly, approaches to comment writing. The emotional tone is more positive at TED; however, there is little emotional escalation in either platform. The study highlights the importance of considering the community characteristics of a medium, when communicating with the public in a case study of emerging technologies.}, keywords = {Information Retrieval, Information Studies, Information Systems}, pubstate = {published}, tppubtype = {workshop} } When modelling for the social we need to consider more than one medium. Little is known as to how platform community characteristics shape the discussion and how communicators could best engage each community, taking into consideration these characteristics. We consider comments on TED videos featuring roboticists, shared at TED.com and YouTube. We find evidence of different social norms and importantly, approaches to comment writing. The emotional tone is more positive at TED; however, there is little emotional escalation in either platform. The study highlights the importance of considering the community characteristics of a medium, when communicating with the public in a case study of emerging technologies. |
Tal, Avital Shulner; Batsuren, Khuyagbaatar; Bogina, Veronika; Giunchiglia, Fausto; Hartman, Alan; Kleanthous-Loizou, Styliani; Kuflik, Tsvi; Otterbacher, Jahna 14th International Workshop On Semantic And Social Media Adaptation And Personalization, SMAP 2019 ACM, 2019. Abstract | Links | BibTeX | Tags: Algorithmic Bias, Algorithmic Fairness, Algorithmic Transparency @workshop{endtoend2019, title = {"End to End" - Towards a Framework for Reducing Biases and Promoting Transparency of Algorithmic Systems}, author = {Avital Shulner Tal and Khuyagbaatar Batsuren and Veronika Bogina and Fausto Giunchiglia and Alan Hartman and Styliani Kleanthous-Loizou and Tsvi Kuflik and Jahna Otterbacher}, url = {http://www.cycat.io/wp-content/uploads/2019/07/1570543680.pdf}, year = {2019}, date = {2019-06-09}, booktitle = {14th International Workshop On Semantic And Social Media Adaptation And Personalization}, publisher = {ACM}, series = {SMAP 2019}, abstract = {Algorithms play an increasing role in our everyday lives. Recently, the harmful potential of biased algorithms has been recognized by researchers and practitioners. We have also witnessed a growing interest in ensuring the fairness and transparency of algorithmic systems. However, so far there is no agreed upon solution and not even an agreed terminology. The proposed research defines the problem space, solution space and a prototype of comprehensive framework for the detection and reducing biases in algorithmic systems.}, keywords = {Algorithmic Bias, Algorithmic Fairness, Algorithmic Transparency}, pubstate = {published}, tppubtype = {workshop} } Algorithms play an increasing role in our everyday lives. Recently, the harmful potential of biased algorithms has been recognized by researchers and practitioners. We have also witnessed a growing interest in ensuring the fairness and transparency of algorithmic systems. However, so far there is no agreed upon solution and not even an agreed terminology. The proposed research defines the problem space, solution space and a prototype of comprehensive framework for the detection and reducing biases in algorithmic systems. |
Batsuren, Khuyagbaatar; Bella, Gabor; Giunchiglia, Fausto CogNet: A Large-Scale Cognate Database Inproceedings Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 3136–3145, Florence, Italy, 2019. Abstract | Links | BibTeX | Tags: @inproceedings{batsuren-etal-2019-cognet, title = {CogNet: A Large-Scale Cognate Database}, author = {Khuyagbaatar Batsuren and Gabor Bella and Fausto Giunchiglia}, url = {https://www.aclweb.org/anthology/P19-1302}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics}, pages = {3136--3145}, address = {Florence, Italy}, abstract = {This paper introduces CogNet, a new, large-scale lexical database that provides cognates -words of common origin and meaning- across languages. The database currently contains 3.1 million cognate pairs across 338 languages using 35 writing systems. The paper also describes the automated method by which cognates were computed from publicly available wordnets, with an accuracy evaluated to 94%. Finally, it presents statistics about the cognate data and some initial insights into it, hinting at a possible future exploitation of the resource by various fields of lingustics.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } This paper introduces CogNet, a new, large-scale lexical database that provides cognates -words of common origin and meaning- across languages. The database currently contains 3.1 million cognate pairs across 338 languages using 35 writing systems. The paper also describes the automated method by which cognates were computed from publicly available wordnets, with an accuracy evaluated to 94%. Finally, it presents statistics about the cognate data and some initial insights into it, hinting at a possible future exploitation of the resource by various fields of lingustics. |
Zembylas, Michalinos Professional standards for teachers and school leaders Journal Article Journal of Professional Capital and Community, 3 (3), pp. 142–156, 2018. Abstract | Links | BibTeX | Tags: Education @article{Zembylas2018, title = {Professional standards for teachers and school leaders}, author = {Michalinos Zembylas}, url = {https://doi.org/10.1108/jpcc-12-2017-0029}, doi = {10.1108/jpcc-12-2017-0029}, year = {2018}, date = {2018-07-01}, journal = {Journal of Professional Capital and Community}, volume = {3}, number = {3}, pages = {142--156}, publisher = {Emerald}, abstract = { Purpose The purpose of this paper is to contribute to recent work that interrogates the affective conditions in standardizing processes taking place in schools by asking: what are the relations between affect and biopower, when standardizing processes take place in schools, and how can we better understand the constitution of affective spaces and atmospheres that enable some transformative potentials while preventing others? Design/methodology/approach The main argument is that professional standards for teachers and school leaders create ambivalent (i.e. both positive and negative) affective spaces and atmospheres in schools that require one to look for the ways in which biopower works affectively through specific technologies. This ambivalence produces not only governable and self-managed teachers and school leaders who simply implement professional standards, but also affective spaces and atmospheres that might subvert the normalizing effects (and affects) of standards. Findings While attention has been directed to the involvement of affectivity in standardizing processes, what has been theorized less in the field of professional capital is the entanglement of affect and biopower in the spread of professional standards. Engaging with recent work surrounding the affective turn in the social sciences and humanities, the encounter between affect and biopower opens methodological, ethical and political possibilities to examine the affective impact of standards on the professional capital of teachers and school leaders. The analysis displaces emotions from their dominant positionality in discourses about professional standards, reinvigorating theoretical explorations of the affective spaces and atmospheres that co-constitute subjectivities, organizations, governance and social practices in standardizing processes. Originality/value The spatiotemporal and organizational arrangements of schools while undergoing standardizing processes constitute crucial constellations for ethical and political reproduction of affective relations. Thus, the destabilizing and inventive potentials of affects, spaces and atmospheres – to name a few conceptual resources – are extremely important in exposing the normalizing as well as resisting aspects of standardizing processes.}, keywords = {Education}, pubstate = {published}, tppubtype = {article} } Purpose The purpose of this paper is to contribute to recent work that interrogates the affective conditions in standardizing processes taking place in schools by asking: what are the relations between affect and biopower, when standardizing processes take place in schools, and how can we better understand the constitution of affective spaces and atmospheres that enable some transformative potentials while preventing others? Design/methodology/approach The main argument is that professional standards for teachers and school leaders create ambivalent (i.e. both positive and negative) affective spaces and atmospheres in schools that require one to look for the ways in which biopower works affectively through specific technologies. This ambivalence produces not only governable and self-managed teachers and school leaders who simply implement professional standards, but also affective spaces and atmospheres that might subvert the normalizing effects (and affects) of standards. Findings While attention has been directed to the involvement of affectivity in standardizing processes, what has been theorized less in the field of professional capital is the entanglement of affect and biopower in the spread of professional standards. Engaging with recent work surrounding the affective turn in the social sciences and humanities, the encounter between affect and biopower opens methodological, ethical and political possibilities to examine the affective impact of standards on the professional capital of teachers and school leaders. The analysis displaces emotions from their dominant positionality in discourses about professional standards, reinvigorating theoretical explorations of the affective spaces and atmospheres that co-constitute subjectivities, organizations, governance and social practices in standardizing processes. Originality/value The spatiotemporal and organizational arrangements of schools while undergoing standardizing processes constitute crucial constellations for ethical and political reproduction of affective relations. Thus, the destabilizing and inventive potentials of affects, spaces and atmospheres – to name a few conceptual resources – are extremely important in exposing the normalizing as well as resisting aspects of standardizing processes. |
Otterbacher, Jahna; Checco, Alessandro; Demartini, Gianluca; Clough, Paul Investigating User Perception of Gender Bias in Image Search Inproceedings The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval - SIGIR textquotesingle18, ACM Press, 2018. Abstract | Links | BibTeX | Tags: Artificial Intelligence, Human-Computer Interaction, Information Retrieval @inproceedings{Otterbacher2018, title = {Investigating User Perception of Gender Bias in Image Search}, author = {Jahna Otterbacher and Alessandro Checco and Gianluca Demartini and Paul Clough}, url = {https://doi.org/10.1145/3209978.3210094}, doi = {10.1145/3209978.3210094}, year = {2018}, date = {2018-01-01}, booktitle = {The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval - SIGIR textquotesingle18}, publisher = {ACM Press}, abstract = {There is growing evidence that search engines produce results that are socially biased, reinforcing a view of the world that aligns with prevalent social stereotypes. One means to promote greater transparency of search algorithms - which are typically complex and proprietary - is to raise user awareness of biased result sets. However, to date, little is known concerning how users perceive bias in search results, and the degree to which their perceptions differ and/or might be predicted based on user attributes. One particular area of search that has recently gained attention, and forms the focus of this study, is image retrieval and gender bias. We conduct a controlled experiment via crowdsourcing using participants recruited from three countries to measure the extent to which workers perceive a given image results set to be subjective or objective. Demographic information about the workers, along with measures of sexism, are gathered and analysed to investigate whether (gender) biases in the image search results can be detected. Amongst other findings, the results confirm that sexist people are less likely to detect and report gender biases in image search results.}, keywords = {Artificial Intelligence, Human-Computer Interaction, Information Retrieval}, pubstate = {published}, tppubtype = {inproceedings} } There is growing evidence that search engines produce results that are socially biased, reinforcing a view of the world that aligns with prevalent social stereotypes. One means to promote greater transparency of search algorithms - which are typically complex and proprietary - is to raise user awareness of biased result sets. However, to date, little is known concerning how users perceive bias in search results, and the degree to which their perceptions differ and/or might be predicted based on user attributes. One particular area of search that has recently gained attention, and forms the focus of this study, is image retrieval and gender bias. We conduct a controlled experiment via crowdsourcing using participants recruited from three countries to measure the extent to which workers perceive a given image results set to be subjective or objective. Demographic information about the workers, along with measures of sexism, are gathered and analysed to investigate whether (gender) biases in the image search results can be detected. Amongst other findings, the results confirm that sexist people are less likely to detect and report gender biases in image search results. |
Giunchiglia, Fausto; Batsuren, Khuyagbaatar; Bella, Gabor Understanding and Exploiting Language Diversity Inproceedings Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, International Joint Conferences on Artificial Intelligence Organization, 2017. Abstract | Links | BibTeX | Tags: Language Diversity @inproceedings{Giunchiglia2017, title = {Understanding and Exploiting Language Diversity}, author = {Fausto Giunchiglia and Khuyagbaatar Batsuren and Gabor Bella}, url = {https://doi.org/10.24963/ijcai.2017/560}, doi = {10.24963/ijcai.2017/560}, year = {2017}, date = {2017-08-01}, booktitle = {Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence}, publisher = {International Joint Conferences on Artificial Intelligence Organization}, abstract = {The main goal of this paper is to describe a general approach to the problem of understanding linguistic phenomena, as they appear in lexical semantics, through the analysis of large scale resources, while exploiting these results to improve the quality of the resources themselves. The main contributions are: the approach itself, a formal quantitative measure of language diversity; a set of formal quantitative measures of resource incompleteness and a large scale resource, called the Universal Knowledge Core (UKC) built following the methodology proposed. As a concrete example of an application, we provide an algorithm for distinguishing polysemes from homonyms, as stored in the UKC.}, keywords = {Language Diversity}, pubstate = {published}, tppubtype = {inproceedings} } The main goal of this paper is to describe a general approach to the problem of understanding linguistic phenomena, as they appear in lexical semantics, through the analysis of large scale resources, while exploiting these results to improve the quality of the resources themselves. The main contributions are: the approach itself, a formal quantitative measure of language diversity; a set of formal quantitative measures of resource incompleteness and a large scale resource, called the Universal Knowledge Core (UKC) built following the methodology proposed. As a concrete example of an application, we provide an algorithm for distinguishing polysemes from homonyms, as stored in the UKC. |
Otterbacher, Jahna; Talias, Michael S/hetextquotesingles too Warm/Agentic! Inproceedings Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction - HRI textquotesingle17, ACM Press, 2017. Abstract | Links | BibTeX | Tags: Artificial Intelligence, Human-Computer Interaction, Information Retrieval @inproceedings{Otterbacher2017, title = {S/hetextquotesingles too Warm/Agentic!}, author = {Jahna Otterbacher and Michael Talias}, url = {https://doi.org/10.1145/2909824.3020220}, doi = {10.1145/2909824.3020220}, year = {2017}, date = {2017-01-01}, booktitle = {Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction - HRI textquotesingle17}, publisher = {ACM Press}, abstract = {Gender stereotypes are strong influences on human behavior. Given our tendency to anthropomorphize, incorporating gender cues into a robot's design can influence acceptance by humans. However, little is known about the interaction between human and robot gender. We focus on the role of gender in eliciting negative, ``uncanny" reactions from observers. We create a corpus of YouTube videos featuring robots with female, male and no gender cues. Our experiment is grounded in Gray and Wegner's (2012) model, which holds that uncanny reactions are driven by the perception of robot agency (i.e., ability to plan and control) and experience (i.e., ability to feel), which in turn, is driven by robot appearance and behavior. Participants watched videos and completed questionnaires to gauge perceptions of robots as well as affective reactions. We used Structural Equation Modeling to test whether the model explains reactions of both men and women. For gender-neutral robots, it does. However, we find a salient human-robot gender interaction. Men's uncanny reactions to robots with female cues are best predicted by the perception of experience, while women's negativity toward masculine robots is driven by perceived agency. The result is interpreted in light of the ``Big Two" dimensions of person perception, which underlie expectations for women to be warm and men to be agentic. When a robot meets these expectations, it increases the chances of an uncanny reaction in the other-gender observer.}, keywords = {Artificial Intelligence, Human-Computer Interaction, Information Retrieval}, pubstate = {published}, tppubtype = {inproceedings} } Gender stereotypes are strong influences on human behavior. Given our tendency to anthropomorphize, incorporating gender cues into a robot's design can influence acceptance by humans. However, little is known about the interaction between human and robot gender. We focus on the role of gender in eliciting negative, ``uncanny" reactions from observers. We create a corpus of YouTube videos featuring robots with female, male and no gender cues. Our experiment is grounded in Gray and Wegner's (2012) model, which holds that uncanny reactions are driven by the perception of robot agency (i.e., ability to plan and control) and experience (i.e., ability to feel), which in turn, is driven by robot appearance and behavior. Participants watched videos and completed questionnaires to gauge perceptions of robots as well as affective reactions. We used Structural Equation Modeling to test whether the model explains reactions of both men and women. For gender-neutral robots, it does. However, we find a salient human-robot gender interaction. Men's uncanny reactions to robots with female cues are best predicted by the perception of experience, while women's negativity toward masculine robots is driven by perceived agency. The result is interpreted in light of the ``Big Two" dimensions of person perception, which underlie expectations for women to be warm and men to be agentic. When a robot meets these expectations, it increases the chances of an uncanny reaction in the other-gender observer. |
Otterbacher, Jahna; Bates, Jo; Clough, Paul Competent Men and Warm Women Inproceedings Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems - CHI textquotesingle17, ACM Press, 2017. Abstract | Links | BibTeX | Tags: Artificial Intelligence, Human-Computer Interaction, Information Retrieval @inproceedings{Otterbacher2017b, title = {Competent Men and Warm Women}, author = {Jahna Otterbacher and Jo Bates and Paul Clough}, url = {https://doi.org/10.1145/3025453.3025727}, doi = {10.1145/3025453.3025727}, year = {2017}, date = {2017-01-01}, booktitle = {Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems - CHI textquotesingle17}, publisher = {ACM Press}, abstract = {There is much concern about algorithms that underlie information services and the view of the world they present. We develop a novel method for examining the content and strength of gender stereotypes in image search, inspired by the trait adjective checklist method. We compare the gender distribution in photos retrieved by Bing for the query "person" and for queries based on 68 character traits (e.g., "intelligent person") in four regional markets. Photos of men are more often retrieved for "person," as compared to women. As predicted, photos of women are more often retrieved for warm traits (e.g., "emotional") whereas agentic traits (e.g., "rational") are represented by photos of men. A backlash effect, where stereotype-incongruent individuals are penalized, is observed. However, backlash is more prevalent for "competent women" than "warm men." Results underline the need to understand how and why biases enter search algorithms and at which stages of the engineering process.}, keywords = {Artificial Intelligence, Human-Computer Interaction, Information Retrieval}, pubstate = {published}, tppubtype = {inproceedings} } There is much concern about algorithms that underlie information services and the view of the world they present. We develop a novel method for examining the content and strength of gender stereotypes in image search, inspired by the trait adjective checklist method. We compare the gender distribution in photos retrieved by Bing for the query "person" and for queries based on 68 character traits (e.g., "intelligent person") in four regional markets. Photos of men are more often retrieved for "person," as compared to women. As predicted, photos of women are more often retrieved for warm traits (e.g., "emotional") whereas agentic traits (e.g., "rational") are represented by photos of men. A backlash effect, where stereotype-incongruent individuals are penalized, is observed. However, backlash is more prevalent for "competent women" than "warm men." Results underline the need to understand how and why biases enter search algorithms and at which stages of the engineering process. |
Bates, Jo; Lin, Yu-Wei; Goodale, Paula Data journeys: Capturing the socio-material constitution of data objects and flows Journal Article Big Data & Society, 3 (2), pp. 205395171665450, 2016. Abstract | Links | BibTeX | Tags: Information Studies @article{Bates2016, title = {Data journeys: Capturing the socio-material constitution of data objects and flows}, author = {Jo Bates and Yu-Wei Lin and Paula Goodale}, url = {https://doi.org/10.1177/2053951716654502}, doi = {10.1177/2053951716654502}, year = {2016}, date = {2016-07-01}, journal = {Big Data & Society}, volume = {3}, number = {2}, pages = {205395171665450}, publisher = {SAGE Publications}, abstract = {In this paper, we discuss the development and piloting of a new methodology for illuminating the socio-material constitution of data objects and flows as data move between different sites of practice. The data journeys approach contributes to the development of critical, qualitative methodologies that can address the geographic and temporal scale of emerging knowledge infrastructures, and capture the ‘life of data’ from their initial generation through to re-use in different contexts. We discuss the theoretical development of the data journeys methodology and the application of the approach on a project examining meteorological data on their journey from initial production through to being re-used in climate science and financial markets. We then discuss three key conceptual findings from this project about: (1) the socio-material constitution of digital data objects, (2) ‘friction’ in the movement of data through space and time and (3) the mutability of digital data as a material property that contributes to driving the movement of data between different sites of practice.}, keywords = {Information Studies}, pubstate = {published}, tppubtype = {article} } In this paper, we discuss the development and piloting of a new methodology for illuminating the socio-material constitution of data objects and flows as data move between different sites of practice. The data journeys approach contributes to the development of critical, qualitative methodologies that can address the geographic and temporal scale of emerging knowledge infrastructures, and capture the ‘life of data’ from their initial generation through to re-use in different contexts. We discuss the theoretical development of the data journeys methodology and the application of the approach on a project examining meteorological data on their journey from initial production through to being re-used in climate science and financial markets. We then discuss three key conceptual findings from this project about: (1) the socio-material constitution of digital data objects, (2) ‘friction’ in the movement of data through space and time and (3) the mutability of digital data as a material property that contributes to driving the movement of data between different sites of practice. |
Michael, Loizos Cognitive Reasoning and Learning Mechanisms Inproceedings AIC, 2016. Abstract | BibTeX | Tags: Artificial Intelligence @inproceedings{Michael2016CognitiveRA, title = {Cognitive Reasoning and Learning Mechanisms}, author = {Loizos Michael}, year = {2016}, date = {2016-01-01}, booktitle = {AIC}, abstract = {With an eye towards the development of systems for common every-day tasks — that operate in a manner that is cognitively-compatible with the argumentative nature of human reasoning — we present mechanisms for reasoning with and learning cognitive programs: common sense, symbolically represented in the form of prioritized association rules. The FLASH mechanism supports a fast argumentation-flavored type of reasoning, while sidestepping the rigidness of traditional proof procedures in formal logic. The NERD mechanism supports the never-ending acquisition of cognitive programs, without human supervision.}, keywords = {Artificial Intelligence}, pubstate = {published}, tppubtype = {inproceedings} } With an eye towards the development of systems for common every-day tasks — that operate in a manner that is cognitively-compatible with the argumentative nature of human reasoning — we present mechanisms for reasoning with and learning cognitive programs: common sense, symbolically represented in the form of prioritized association rules. The FLASH mechanism supports a fast argumentation-flavored type of reasoning, while sidestepping the rigidness of traditional proof procedures in formal logic. The NERD mechanism supports the never-ending acquisition of cognitive programs, without human supervision. |
Otterbacher, Jahna Crowdsourcing Stereotypes: Linguistic Bias in Metadata Generated via GWAP Inproceedings Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 1955–1964, ACM, Seoul, Republic of Korea, 2015, ISBN: 978-1-4503-3145-6. Abstract | Links | BibTeX | Tags: Artificial Intelligence, Human-Computer Interaction, Information Retrieval @inproceedings{Otterbacher:2015:CSL:2702123.2702151, title = {Crowdsourcing Stereotypes: Linguistic Bias in Metadata Generated via GWAP}, author = {Jahna Otterbacher}, url = {http://doi.acm.org/10.1145/2702123.2702151}, doi = {10.1145/2702123.2702151}, isbn = {978-1-4503-3145-6}, year = {2015}, date = {2015-01-01}, booktitle = {Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems}, pages = {1955--1964}, publisher = {ACM}, address = {Seoul, Republic of Korea}, series = {CHI '15}, abstract = {Games with a Purpose (GWAP) is a popular approach for metadata creation, enabling institutions to collect descriptions of digital artifacts on a mass scale. Creating metadata is challenging not only because one must recognize the artifact; the description must then be encoded into natural language. Language behaviors are influenced by many social factors, particularly when we are asked to describe other people. We consider labels for images of people generated via the ESP Game. While ESP has been shown to produce relevant labels, critics claim they are obvious and stereotypical. Based on theories of linguistic biases, we examine whether there are systematic differences in the ways players describe images of men versus women. Our first analysis considers images of people generally, and reveals a tendency for women to be described with subjective adjectives. A second analysis compares images depicting men and women within each of six occupational roles. Images of women receive more labels related to appearance, whereas those depicting men receive more occupation-related labels. Our work exposes the presence of gender-based stereotypes through linguistic biases, illustrates the forms in which they manifest, and raises important implications for those who design systems or train algorithms using data produced via GWAP.}, keywords = {Artificial Intelligence, Human-Computer Interaction, Information Retrieval}, pubstate = {published}, tppubtype = {inproceedings} } Games with a Purpose (GWAP) is a popular approach for metadata creation, enabling institutions to collect descriptions of digital artifacts on a mass scale. Creating metadata is challenging not only because one must recognize the artifact; the description must then be encoded into natural language. Language behaviors are influenced by many social factors, particularly when we are asked to describe other people. We consider labels for images of people generated via the ESP Game. While ESP has been shown to produce relevant labels, critics claim they are obvious and stereotypical. Based on theories of linguistic biases, we examine whether there are systematic differences in the ways players describe images of men versus women. Our first analysis considers images of people generally, and reveals a tendency for women to be described with subjective adjectives. A second analysis compares images depicting men and women within each of six occupational roles. Images of women receive more labels related to appearance, whereas those depicting men receive more occupation-related labels. Our work exposes the presence of gender-based stereotypes through linguistic biases, illustrates the forms in which they manifest, and raises important implications for those who design systems or train algorithms using data produced via GWAP. |
Beloglazov, Anton; Banerjee, Dipyaman; Hartman, Alan; Buyya, Rajkumar Improving Productivity in Design and Development of Information Technology (IT) Service Delivery Simulation Models Journal Article Journal of Service Research, 18 (1), pp. 75–89, 2014. Abstract | Links | BibTeX | Tags: Information Systems @article{Beloglazov2014, title = {Improving Productivity in Design and Development of Information Technology (IT) Service Delivery Simulation Models}, author = {Anton Beloglazov and Dipyaman Banerjee and Alan Hartman and Rajkumar Buyya}, url = {https://doi.org/10.1177/1094670514541002}, doi = {10.1177/1094670514541002}, year = {2014}, date = {2014-07-01}, journal = {Journal of Service Research}, volume = {18}, number = {1}, pages = {75--89}, publisher = {SAGE Publications}, abstract = {The unprecedented scale of Information Technology (IT) service delivery requires careful analysis and optimization of service systems. The simulation is an efficient way to handle the complexity of modeling and optimization of real-world service delivery systems. However, typically developed custom simulation models lack standard architectures and limit the reuse of design and implementation artifacts across multiple models. In this work, following the design science research methodology, based on a formal model of service delivery systems and applying an adapted software product line (SPL) approach, we create a design artifact for building product lines of IT service delivery simulation models, which vastly simplify and reduce the cost of simulation model design and development. We evaluate the design artifact by constructing a product line of simulation models for a set of IBM’s IT service delivery systems. We validate the proposed approach by comparing the simulation results obtained using our models with the results from the corresponding custom simulation models. The case study demonstrates that the proposed approach leads to 5–8 times reductions in the time required to design and develop related simulation models. The potential implications of the application of the proposed approach within an organization are quicker responses to changes in the business environment, more information to assist in managerial decisions, and reduced workload on the process reengineering specialists.}, keywords = {Information Systems}, pubstate = {published}, tppubtype = {article} } The unprecedented scale of Information Technology (IT) service delivery requires careful analysis and optimization of service systems. The simulation is an efficient way to handle the complexity of modeling and optimization of real-world service delivery systems. However, typically developed custom simulation models lack standard architectures and limit the reuse of design and implementation artifacts across multiple models. In this work, following the design science research methodology, based on a formal model of service delivery systems and applying an adapted software product line (SPL) approach, we create a design artifact for building product lines of IT service delivery simulation models, which vastly simplify and reduce the cost of simulation model design and development. We evaluate the design artifact by constructing a product line of simulation models for a set of IBM’s IT service delivery systems. We validate the proposed approach by comparing the simulation results obtained using our models with the results from the corresponding custom simulation models. The case study demonstrates that the proposed approach leads to 5–8 times reductions in the time required to design and develop related simulation models. The potential implications of the application of the proposed approach within an organization are quicker responses to changes in the business environment, more information to assist in managerial decisions, and reduced workload on the process reengineering specialists. |
Loizou, Styliani Kleanthous; Dimitrova, Vania Adaptive notifications to support knowledge sharing in close-knit virtual communities Journal Article User Modeling and User-Adapted Interaction, 23 (2-3), pp. 287–343, 2012. Abstract | Links | BibTeX | Tags: Human-Computer Interaction @article{KleanthousLoizou2012, title = {Adaptive notifications to support knowledge sharing in close-knit virtual communities}, author = {Styliani Kleanthous Loizou and Vania Dimitrova}, url = {https://doi.org/10.1007/s11257-012-9127-y}, doi = {10.1007/s11257-012-9127-y}, year = {2012}, date = {2012-09-01}, journal = {User Modeling and User-Adapted Interaction}, volume = {23}, number = {2-3}, pages = {287--343}, publisher = {Springer Nature}, abstract = {Social web-groups where people with common interests and goals communicate, share resources, and construct knowledge, are becoming a major part of today’s organisational practice. Research has shown that appropriate support for effective knowledge sharing tailored to the needs of the community is paramount. This brings a new challenge to user modelling and adaptation, which requires new techniques for gaining sufficient understanding of a virtual community (VC) and identifying areas where the community may need support. The research presented here addresses this challenge presenting a novel computational approach for community-tailored support underpinned by organisational psychology and aimed at facilitating the functioning of the community as a whole (i.e. as an entity). A framework describing how key community processes—transactive memory (TM), shared mental models (SMMs), and cognitive centrality (CCen)—can be utilised to derive knowledge sharing patterns from community log data is described. The framework includes two parts: (i) extraction of a community model that represents the community based on the key processes identified and (ii) identification of knowledge sharing behaviour patterns that are used to generate adaptive notifications. Although the notifications target individual members, they aim to influence individuals’ behaviour in a way that can benefit the functioning of the community as a whole. A validation study has been performed to examine the effect of community-adapted notifications on individual members and on the community as a whole using a close-knit community of researchers sharing references. The study shows that notification messages can improve members’ awareness and perception of how they relate to other members in the community. Interesting observations have been made about the linking between the physical and the VC, and how this may influence members’ awareness and knowledge sharing behaviour. Broader implications for using log data to derive community models based on key community processes and generating community-adapted notifications are discussed.}, keywords = {Human-Computer Interaction}, pubstate = {published}, tppubtype = {article} } Social web-groups where people with common interests and goals communicate, share resources, and construct knowledge, are becoming a major part of today’s organisational practice. Research has shown that appropriate support for effective knowledge sharing tailored to the needs of the community is paramount. This brings a new challenge to user modelling and adaptation, which requires new techniques for gaining sufficient understanding of a virtual community (VC) and identifying areas where the community may need support. The research presented here addresses this challenge presenting a novel computational approach for community-tailored support underpinned by organisational psychology and aimed at facilitating the functioning of the community as a whole (i.e. as an entity). A framework describing how key community processes—transactive memory (TM), shared mental models (SMMs), and cognitive centrality (CCen)—can be utilised to derive knowledge sharing patterns from community log data is described. The framework includes two parts: (i) extraction of a community model that represents the community based on the key processes identified and (ii) identification of knowledge sharing behaviour patterns that are used to generate adaptive notifications. Although the notifications target individual members, they aim to influence individuals’ behaviour in a way that can benefit the functioning of the community as a whole. A validation study has been performed to examine the effect of community-adapted notifications on individual members and on the community as a whole using a close-knit community of researchers sharing references. The study shows that notification messages can improve members’ awareness and perception of how they relate to other members in the community. Interesting observations have been made about the linking between the physical and the VC, and how this may influence members’ awareness and knowledge sharing behaviour. Broader implications for using log data to derive community models based on key community processes and generating community-adapted notifications are discussed. |