For the latest issue of Information Professional magazine, my Scripturient column features an interview with Nick Seaver, Assistant Professor of Anthropology at Tufts University.
Nick’s new book Computing Taste uses ethnographic fieldwork to explore how the makers of music recommendation algorithms understand and go about their work – from product managers thinking about their relationship with users to scientists theorising the act of listening itself as a kind of data processing and engineers for whom the world of music is a geography to be cared for and controlled.
You can read the latest Scripturient column featuring Nick as a PDF download here, and there’s a full transcript of our conversation, digging even deeper into these issues and other elements of Nick’s research, below.
Matt: Music recommender systems are often presented as an antidote to a surfeit of choice in the age of streaming, but in your book, you point out that “information overload” is a concept which predates digital consumer capitalism, going right back to the early modern period or even antiquity.
You’ve talked elsewhere about how, when students come to your class to study a particular problem, they end up “problematising the problem” – getting a fresh perspective rather than necessarily resolving it neatly.
In the world of streaming music, there’s a notion that “information overload is a problem, and algorithmic recommendation is the remedy”. How did that become problematised for you in the course of your research?
Nick: My attitude towards information overload shifted over the course of this research because it becomes a kind of obvious reason to have a recommender system in the first place. It seems to make sense to everybody: “Of course it’s an issue, we need a system like this to help us.”
What’s funny is that, for a long time when I was talking to people working on these systems, I took the idea of overload for granted. Then I realised it was playing such an important role in the justification of these systems, and that my job as an anthropologist is in part to point out the things that people think are obvious and say “I don’t know, is that obvious? What if it was not?”
The idea that opening a music streaming service today and there’s forty million songs waiting for you is a problem is actually kind of weird! Does it have to be a problem necessarily, in the way that information overload is?
My approach is classically anthropological, in the sense that I take something which people in the field, people that I interviewed, and even readers of the book, thing is obvious – and try to undermine that. If we imagine that we don’t have to take that for granted, we can find other ways of thinking about the world.
There are a bunch of avenues available to someone who wants to take an academic approach to this. One is to say: “We do feel information overload, there are too many things, and I’d like a system to help me figure out which of those things I might like.” And a response to this, which I don’t think is necessarily the most helpful one, is just to say, “Well, this is what capitalism does. It tries to induce some new kind of desire in you, to make you want something you didn’t realise you needed, and then when you have that, to make you want something else.”
This makes sense at a macro scale, but for me, that story doesn’t capture the personal and interpersonal quality of the feeling of information overload, and the motivations of the people who build these systems. None of the people I talked to who were working on these things were going to say, “My goal is to induce desire in people which they didn’t have before!” They wanted to help people.
They don’t have to be right, there don’t have to be good consequences just because they had good intentions, but figuring out the terms by which that works for the people who build these systems was, for me, the core ethnographic work that I was trying to do in the book.
So many interesting and memorable terms come up from your fieldwork: dogfooding, data plumbers, the celestial jukebox. To what extent was this fieldwork like learning a foreign language?
The conventional work of anthropology in the field often involves some kind of literal language learning; even doing fieldwork in English, which is my first language, I did have this encounter with terminology and a way of thinking. We have a long history in anthropology of using linguistic metaphors for thinking about culture anyway, and what I want to know about in this research is the culture of the people who are building these systems: how do they think about the world? What’s important to them? What kinds of symbols do they use, what comparisons do they tend to draw?
Over the course of doing ethnographic fieldwork, you learn how to speak the language and learn what makes sense to people in that domain. Like a lot of anthropologists who study industries, I came to the point where, if I hadn’t gotten an academic job, the second most likely job I would actually get was doing the thing that I had been studying!
There’s a delight, for those of us who study the anthropology of science, technology, and engineers, in the many interesting, confusing, and weird little terminologies. It’s fun to gather them together and say: “Look at this! Isn’t this weird?”
If you’re living and working and socialising alongside your informants, and it’s even possible that you might end up with a career in their industry, how do you manage the boundaries? And what about the boundary between anthropology and “STS” – science and technology studies?
The boundary between STS and anthropology has, for some people at some times, been particularly important. For me, that’s not really the case; right now, I’m director of STS at my university, as well as being in the anthropology department. We have a pretty heterodox STS programme, insofar as such a thing exists – people from disciplines all over the place. At the end of the twentieth century, there was a lot of anxiety in the anthropology of science and technology about positioning vis-a-vis STS – but by the time my cohort of anthropologists were being trained, those distinctions were less foregrounded. And, of course, despite being an “ology” discipline, anthropology is pretty disciplinary already.
In terms of going into the field, this question of positionality is a big and important one, especially as critiques of the sector have shifted over the time I’ve been doing this work. I conceptualised this project back in 2010: over that time, much has changed in how people talk about these things. One of the huge changes in relation to algorithms of search and recommendation is the question of bias, and particularly racial biases.
There are different positions you can take when you want to talk about these systems in an academic mode. As an anthropologist, one of my primary goals is to understand people in the terms in which they understand themselves – making a bridge across those two kinds of positionality, to say “How do people working in this field talk about what they’re doing? How might I bring that into conversation with the ways that people in anthropology and adjacent disciplines talk about similar problems?”
That facilitates a kind of critique of these systems, and also, inversely, a critique of anthropology. There’s a lot more critique of this sector than there was when I started, and as a white guy who could just as easily have been working in this company as studying it, I’m not necessarily the best person to make those critiques, as I’m not the kind of person who would be affected by them. So I take my job to be someone who can foreground those critiques, draw them into conversation with what I see, and provide empirical material that people who want to build those critiques can draw on.
My position affords me a kind of access and a kind of conversation that could be harder for people who didn’t fit in demographically the way that I did. I look like someone who might understand how to programme a computer – which I don’t really! – and that affects how people talk to me and what they’re willing to tell me.
One issue for some critique in this sector is that it’s been hard to get empirical grounding. We may have to talk about algorithms where we don’t really know how they work; we may take for granted what companies say about how they work; we sometimes have to rely on unreliable interface methods, almost experimenting with what we can get out of the app when we open it as a user, without always really knowing what’s happening under the hood. What I put together empirically may be useful for people who want to make critical arguments, but may not have been able to get the kind of access I did, or may be from a discipline which doesn’t incorporate fieldwork.
Clifford Geertz spoke of fieldwork as “deep hanging out”. At times you’re speaking to informants in climbing gyms, and I had a vision of you hanging off a rope while conducting your ethnographic study.
The method involves living with people and learning to do what they do; if you do an ethnography that’s organised around an occupation or a business sector, it can be a little weird, because it’s not always clear where the interest starts and stops.
These are often people for whom their jobs are a big part of their lives, but if you’re going to study music recommendation, it doesn’t really make sense that you’re going to go stay at someone’s house or something like that. But working on this project for a long time, and making friends with people who worked in the field, I would stay with them if, say, I were visiting a town where they lived. It’s more in the background of the book, but you do learn about people’s personal lives and participate in activities outside of the office. When I was in San Francisco, I did an interview with someone at a rock climbing gym, where we went climbing together. That felt very “of the moment” in the history of the tech scene in San Francisco.
They say that ethnography is “history but not yet”, but so much has changed so swiftly in that sector, in these systems and their popularity, since I started the work. I used to have to explain to people what a recommender system was, when I started giving talks. Now “the algorithm” is a thing. It’s a different problem, a different set of things that I need to do, when I talk to audiences about these issues.
The fieldwork gradually recedes into the past. You’ve moved from the west coast to the east. What happened in your thought process during the writing of the book, at this stage of the journey?
The biggest thing, which I try to address in the epilogue to the book, is that recommender systems used to be technologies that were part of the system: something off to one side, part of the interface, making suggestions for you in the corner alongside editorial recommendations, your own collection that you’ve gathered yourself, and so on.
What’s happened since, in music but also in other domains, is that recommendation has encompassed everything. Most of the media that people encounter today will have been algorithmically filtered in one way or another. The stories you see on the front page of a newspaper online, the music you find, what you watch, social media; it’s all going to be filtered somehow.
That really changes what the power of a recommender system is, and what it means to work on one. For example, Pandora Radio, once upon a time, made kiosks for record stores, so you would go into a Tower Records, and by entering an artist you liked into a terminal, you’d be presented with some suggestions for other artists to listen to. That’s a very different spot for a recommendation to live, compared to anything you hear being filtered through one of these systems. What’s more, they work differently in technical terms, and the consequences can be substantial, such as the creation of feedback loops where the act of recommending causes a thing to become more popular, and hence get recommended even more. Previously, there was always an outside; there was something external which could inject novelty into the system. That’s not as obvious now. Recommendation has expanded from an isolated technical component to the container for everything.
The other thing is that people working in this sector have been critiqued a lot, and, being aware of that, they’ve been trying to come to terms with it, figure out what they can do about it. How do people working in music recommendation think about ethics, and what are they going to do, given the situation? That’s partly why I’m so interested in the detail of how these people think. Merely saying “capitalism demands the production of desire” tells us little about, for example, the ambivalence felt by one of my interviewees at the end of the book, about the work that he’s done. These ambivalent people are still powerful, they still get to make design decisions, and to decide what the next thing is going to be, which will be branded as solving the problems of the previous thing. If we want to know what’s going to happen, how these systems work and how they might work in the near future, it is useful to look at that dynamic, and how those people are thinking about what they are doing.
When we talk about recommendation being everywhere, there’s two meanings.First, in the technical sense I’ve just set out, a lot of media flows are literally being shaped by algorithmic systems, which might raise questions for us about how they work and what they do.
The other is the argument which I’ve heard a lot from people in the field, that we can think of algorithmic recommendation as one instance of a much broader phenomenon of gathering advice from other people. Joe Konstan has this common opening trope in his classes and talks, where he talks about the very first recommender system probably being this prehistoric caveman situation where there’s a plant outside the cave, and people are trying to figure out whether they want to eat it; one person eats it, and everyone watches to see what happens.
What’s fascinating about that story is that it suggests that recommendation as an idea, for at least some people in this space, is very generic. It’s not about collaborative filters or technical implementations we’ve had since the mid-1990s. It boils down to what anthropologists might simply call society: the fact that we live with other people, the division of labour, the fact that people learn from one another, can be taken as a recommender system.
Now this can function as a kind of naturalising: “Don’t worry about whatever they’re doing at this social media company, it’s just a computerised version of a thing we’ve been doing since time immemorial.” That’s a bit ridiculous, of course, to argue, because it’s not the same. There are many ways of learning from other people, and the problem is not the learning, it’s how it is implemented and who gets to decide.
However, it’s also useful in terms of thinking through how people in this sector are going to respond to critiques and the cosmology they have and the world they understand themselves to live in. If we raise the questions of filter bubbles and echo chambers, for example, a response might be: “Is it worse than the echo chamber you were already in, before you had this system?” Because at least if you know you’re in an echo chamber, then you know enough to get out. But even if it’s claimed that, say, the small town you lived in was an echo chamber or a filter bubble, that town was not an algorithmic recommender system that was designed by a relatively elite set of engineers at a relatively small set of companies. It becomes useful and interesting to try such ideas on, and see what the consequences might be, and then to ask: “What other ways might we have of thinking about what’s going on here?”
You said that in some ways, this was a very orthodox ethnographic project – but at the same time, it sounds like you’ve learned a lot about how to conduct fieldwork under the conditions of corporate privacy, secrecy, and legal regimes such as intellectual property.
This is a tremendous problem, and one which occupied most of my thinking and writing on this project. The first academic publications from this project were largely methodological, and focussed on this question of access rights.
Not all of the ways we think about access to the communities we want to do fieldwork with in the discipline are helpful. Sometimes it’s imagined that once you get in, once you’re there, you simply see stuff and it all makes sense to you. That’s certainly not how it happens in office fieldwork, and I don’t think it really happens anywhere. The challenge of gaining access is part of the fieldwork experience.
It’s easy to think that what I went through, signing an NDA on an iPad at the front desk of an office I was entering for fieldwork, is a brand new world, but there are a lot of similarities to past ethnographies too. One of the things I did was read ethnographic accounts of other secretive groups; it’s not like companies are the first people to have secrets! Recent examples I’ve loved included Graham Jones’ book about magicians, and Lilith Mahmud on freemasons in Italy. These groups have secrets, but there’s also a permeability that’s distinctive to them. It’s possible to become a freemason, to become a magician, to enter a company. There are terms by which these groups of people produce and maintain secrecy. Thinking you’ll just get inside those groups and “bring those facts out” is not going to do justice to the texture of access.
It also doesn’t do justice to the fact that once you gain access, you won’t see anything. There’s no there, there! There’s only more insides. Once I got something that looked like access, doing an internship for a short period inside a company, I found that once you’re inside, you’re just in an office. There’s more doors, there’s more people; you can’t see inside their heads just because you’re standing next to them. They’re going to keep secrets from each other as well as from you. You have to keep going, once you get “inside”.
For me, that was frustrating, because we have this mythology in anthropology of just arriving at the beach and getting your fieldwork going, but it also was an opportunity. Since this work has come out, I’ve had grad students approach to thank me for being open about how gradual and painful and partial the process of fieldwork in this space can be.
It’s an interesting problem, and I think it’s going to be more widely experienced across a range of domains, because for better or for worse – probably mostly for the better – more groups of people are having more control over their public representation. It’s not just US corporate groups who may expect people conducting ethnographic fieldwork to sign an NDA, who may want to have some control over their representation. It’s good that you’re obligated to be in a relationship with the people you’re writing about, such that you can’t just go in, extract some stuff, leave, and spit it out.
Colonial history was an obvious moment when anthropologists could feel entitled and actually have a certain degree of access as a result of their affiliations. But it’s clear that groups should have more say over how they are represented. That leads into a much bigger question about how ethics in ethnographic fieldwork works. When we think of what anthropologists call “studying up”, looking at powerful entities like corporations and thinking about respecting their interests, a lot of these questions get turned on their head. Should powerful engineers be able to have a say in how they’re represented? How do you decide who is powerful in a given setting?
How did you reflect on your own power in your ethnographic encounters?
I didn’t feel like I had a whole lot of power in that setting! I did have the privilege of mobility, that many people working in that space didn’t have; I could talk with people working at many different companies, for example. That’s not always easy for people working in rival firms within the same sector, though of course people did have friends at competitor companies and there were informal conversations and sharing arrangements.
The other thing is that it’s very possible for an anthropologist, even working in an ostensibly not-sensitive domain like I do, to put things out there which might cause harm to people they are talking about. So people absolutely told me things that would have got them in trouble, had I published them. My solution for some of those things was to use artful pseudonyms, to composite people into a single voice, or to change certain details so that they can’t be identified. There are debates about how good it is to do these things, which usually hinge on what we assume the purpose of ethnography is. Some people think there should be no such obfuscations. I was writing this book as my doctoral dissertation, so I was learning how to be an anthropologist, and my experiences on this project have certainly informed my subsequent work.
Writing this book was particularly strange because I had a collaborator who is effectively me, ten years ago – someone a couple of years into a PhD programme for a subject he’s never studied before, and kind of doesn’t know what he’s doing! It makes for an interesting long-duration project to have, and I think we have to be sensitive to the fact that this is just how these projects come together.
You talk about the privilege of mobility, and that’s true in another sense, too: you write in the book about “discursive mobility”, bringing in anthropological work on animal traps to reflect on how attention is captured in digital consumer products.
These are my favourite things to do – one of my goals is to take a sometimes passive local discourse and draw it out, imagining alternatives to the current situation, and draw on my corpus of references to do so.
In this case, it involved taking people more literally than they take themselves: “Hey, you talk about trapping people! What else can I do with that?”
A friend from grad school, a wonderful anthropologist named Chima Anyadike-Danes, would send me articles that he thought would interest me, usually about the anthropology of technology. He knew that I have a long-standing interest in gizmos and diagrams, and the anthropology of trapping is a heavily gizmo and diagram laden subfield! I’m pretty sure it came via him, but the precise moment was lost to time. He would send me weird mid-century anthropology about trapping and time management, and then I saw the link to the work I was doing in the present.
Intel funded some of my doctoral work, and the first time I spoke on this link to trapping, geeking out on the anthropological literature, it was at Intel. It was raw, and it was weird, and people in the audience were like, “What was that?”
It was just one of those things that you roll around in your head and it sounds interesting. In some sense, the ideas catch you.
And I guess Chima is then a kind of recommender.
An excellent one!
You speak of your interest in how people who deal with technology handle cultural materials. When did that topic first hook you?
Firstly, the conventional shape of an STS argument is to take something scientific or technological which people don’t think is especially cultural, and highlight how it is cultural. You might not think that particle physics is cultural, for example, but then you can look into how the ways in which people approach it are cultural. And I find this approach super useful, I use it in my teaching, I draw on it all the time. It’s so provocative.
What I do is study a group of people who are working on a thing they know is cultural, such as what music people like. That’s clearly cultural in most senses of the word. The question then becomes not, “Did you know that what you are doing is cultural?” It becomes, what do they do when they know that what they are doing is cultural. How do they try to account for the fact that they are doing cultural stuff, in whatever terms they understand culture to mean?
That’s been the organising principle for my work, in general, and as I move into new projects, such as my latest work on attention, I am particularly interested in how these technical experts relate to cultural concepts and the idea of culture itself.
I’ve always been interested in culture and technology. As an undergrad, I was a literature major, but ended up writing a thesis about noise, music, and audio recording. The technical fact of being able to record and reproduce something exactly, what does that mean for the idea of noise?
From that, I went on to do a master’s degree in media studies at MIT, where I wrote a history of the player piano, which could record performances and play them back. There were similar issues there around automation and music being understood as being a domain which is so much about feelings and human expression, and yet one which is so technical. Instruments are machines, and they’re surrounded by recording devices and playback devices and amplification systems, and all of these things which we think of as being so human are caught up in technology. So a robotic piano player raised fun questions about what the boundary is.
When I was thinking about projects to pursue in a PhD programme, I was looking for something that combined music and automation in the present day. That led me to music recommender systems.
Kurt Vonnegut’s first novel, Player Piano, is also a meditation on what it is to be an individual in the corporate machine, right?
Right. The player piano becomes this classic figure of the badness of machinery in human life. If you ask people to imagine what a player piano sounds like in their heads, people always imagine that it’s somewhat out of tune! Also, the modern player piano and the phonograph were invented within a couple of years of each other. So we have this idea in our minds that the player piano must have been the old technology, which was replaced by the phonograph, but there was a time when the player piano was seen as legit, and the phonograph seemed like a gimmick or a toy. So the thought of recuperating the plausibility of technologies like that was interesting to me: recordings versus robotic versions of instruments.
We’re coming off a hundred plus years of life with audio recording that sounds a certain way, a feeling of how it makes sense to encounter certain media, which we’ve come to take for granted. Yet when you think about, for example, how does a music recommender do what it does? What does it recommend? Well, it doesn’t recommend music in the abstract – there is no such thing: it recommends recordings. It only makes sense in a world where recordings dominate. You have to really stretch your mind to imagine what recommendation looks like in a world where we all use robotic instruments, player pianos and their equivalents.
Now that we have AI-produced art, however, and music might similarly be synthesised, perhaps we would move back towards the player piano and away from the recording…
This was an argument that I was making back at the time. The player piano involves remaking music from an existing order, not merely capturing the audio. When we look at AI-produced music, we find it used where people want background music. Instead of paying for sync rights to put a recording behind a commercial on TV or another recording, it’s possible now to automate away certain kinds of musical occupations. The thought of generating music through AI or using AI to edit music that had already been recorded is certainly something that came up in discussions during my fieldwork. However, while there was some progress on this about ten years ago, effectively it’s been stomped down by the fact that most people want to listen to recorded music by artists that they know. And record labels are not keen to grant rights for people to hack music up, remix it, and so on as well as the simple right to replay; you could imagine a version of Spotify that’s full of options to remix and transform music. The reason we don’t have that is not so much technical as corporate-legal.
How has this work affected your own relationship to music?
Having kids really changed my relationship to music more than anything. It’s a pretty common phenomenon, that people become a little less exploratory when it comes to hunting for new music. I had my first kid right before I filed my dissertation, and so a lot of music I listen to, even now, is music that I learned about from people who were building algorithmic recommender systems. These people shared music with me, and it’s another way that the book lives on in my life. I listen to the music that I listened to while I was working on it.
Additionally, when you listen to an algorithmically generated playlist, you do this mental operation: “Where did it come from? Why is it doing that?” From my research, I have a pretty good sense of the range of things that might be going into that. So I’ll listen to something and take a guess at how it came about, listening for the relationships between recommendations, where once we listened for the intentionality of why the DJ chose the next track.
While I was wrapping up my fieldwork, I had dinner with my family at a restaurant. We were in some steakhouse with terrible music, a weird mix of soft rock, crooners, and such. My sister, who’d been talking with me about my research, said: “I think this is a Michael Bublé Pandora station.”
The reasoning was that if you were this cool steakhouse, you wanted something that was Frank Sinatra like, but not Sinatra. And if you left it alone, it would keep playing things in that vein, but along various dimensions. One would be Vegas-style crooning, another would be easy listening, another AM radio. My sister asked the bartender, who was playing it off their phone — and she was right. So you can develop this weird intuition for the logic by which these systems associate tracks. I think everyone is starting to learn this now, wondering if that cool new track you heard in the coffee shop was on their Discover Weekly as well as yours, and so on.
Nick Seaver’s Computing Taste is available from University of Chicago Press in December.
matt finch / mechanical dolphin 
