About the project
In this thesis, Knaus examines the emergent infrastructures of voice, speech, and language technologies, primarily what is known in critical AI studies as the field of Machine Listening. This term refers to the process of converting sound into data, which is then analyzed by machine learning algorithms. When machines listen to human speech and body sounds, these sounds can reveal a wealth of 'fine-grained' information captured alongside language that may indicate a person's emotional, physical, mental, or general health state. Well-known examples include digital assistants (Siri, Alexa, Cortana), speech transcription, speaker identification, voiceprints, voice synthesis, and the more esoteric cases of classifying a speaker's personality, race, gender, sexual preference, truthfulness, and so on, known as aural phrenology for voice forensics. Natural Language Processing also deals with speech recognition, for instance, in digital assistants, voice identification, and voice analysis. Within these practices of machine listening, particularly the analysis of health states, industry experts describe it as a 'new field of analysis' that is opening up for machine listening companies. It is, therefore, hailed as a new extractive site for venture capital, which has led to a proliferation of actors.
The sheer proliferation of actors and their differing methodologies makes it increasingly difficult to follow and delineate a field. A key factor contributing to this difficulty is that these actors analyze various body or voice sounds in different combinations, depending on the product. The thesis does not focus on a specific application or user's experiences but on several key aspects of an AI assembly line for voice, speech, and language technologies: algorithmic techniques – that become more networked and modular; the problematic production of data - that trains these algorithmic techniques; how new algorithmic techniques may or may not be aligned with human values; and the adoption of machine listening techniques by genome research infrastructures.
For the analysis of such technical specifics, Knaus primarily conducted a practice-based document analysis of highly technical documents, including computer science research articles that evaluate code, review articles in the field, news articles, company websites, advertisement materials, and academic project websites. Along with such data, she analyzed open-source datasets produced to evaluate algorithms and treated them as text and ethnographic materials. She performed a 'close reading' of such texts and datasets—a reading explicitly conducted from a social science perspective, not aimed at uncovering hidden meanings or understanding code, but at tracing how the field constructs its research object and forms its assumptions, narratives, and metaphors. The key findings suggest that AI restructures and reconfigures how the body and mind are studied.
Thesis supervisor
- Professor Susanne Bauer (TIK Centre for Technology, Innovation and Culture, University of Oslo)
Commentator
- Professor Eric Monteiro (Department of Computer Science, NTNU – Norwegian University of Science and Technology)
About the candidate
Tanja Knaus is a PhD candidate at the TIK Centre for Technology, Innovation and Culture at the University of Oslo. Her PhD project is part of the work package "AI in Society" at the Norwegian Centre for Artificial Intelligence Innovation (NorwAI), and concerns itself with the socio-technical implications of automation and speech emotion recognition (SER).
How to participate
The seminar is open for everyone to join. Welcome!
The manuscript is available upon request. Please send the request to Tanja Knaus tanja.knaus@tik.uio.no.