Joy Kim, a research scientist, studies how technology can connect people—and how people can accomplish creative things once they’re linked together in online communities. She’s based in Adobe Research’s San Francisco office.
What is the focus of your research, and how did you decide to pursue this work at Adobe Research?
I am interested in how computers and humans work together to do cool things—a field known as human-computer interaction (HCI). In particular, I’m looking at how technology lets people connect with each other and how to design online communities. I study social computing and crowdsourcing, tapping into a lot of people working together online.
I got into HCI as an undergrad. I liked research and went onto grad school, where I met a professor who specialized in crowdsourcing. I enjoy art and storytelling, and I thought I’d try to combine that with HCI and working with the crowd. That’s how I started the Mechanical Novel project.
By the end of my grad studies, I knew two things: I wanted to make things that go out into the real world and affect people, and I wanted to continue to work on creativity. I had an internship with Adobe Research with Mira Dontcheva. I saw that Adobe Research was a really good fit for everything I hoped to do.
Thankfully, I got an offer—and I’m really enjoying it! I am surrounded by people really interested in creativity, and I can gush about cool things I find. And people who work here often have creative hobbies outside work—they’re musicians, photographers, artists. They care about supporting creative work.
Could you tell us about the Mechanical Novel project?
The Mechanical Novel was a storytelling project I pursued for my doctoral research. It’s named after “Mechanical Turk,” a term used for crowdsourcing, where numerous people do small tasks online as part of a bigger project. [Note: The Mechanical Turk moniker originated with an 18th-century robot-like machine that played chess. Despite appearing to be mechanical, the Turk actually had a human chess-master hidden inside it. Similarly, crowdsourcing today provides human labor—often that of hundreds of people—for small tasks that computers can’t do alone. Its most well-known platform is Amazon Mechanical Turk.]
I wanted to find out if crowdworkers – hundreds of strangers on the internet — could write a story together. You might have heard of storytelling games where people add a sentence or paragraph to a story, one at a time. But this “divide and conquer” method of crowdsourcing doesn’t usually provide a coherent plot or theme. Nobody coordinates. I asked if there was a better way to do this, to coordinate people’s contributions to a story through goals.
We began with a rough draft of a story and asked a few hundred crowdworkers to help set goals for editing. For example: Should we make the grandpa character more friendly? If so, crowdworkers next needed to pick which parts of the story need to change. Then, the next batch of crowdworkers would know exactly which parts of the story to edit, rather than just choosing what to change on their own. Strangers could jump in and out of it. The Mechanical Novel had coordination, because we had high level goals that workers had brainstormed and chosen for themselves.
The research study compared a typical crowdsourcing workflow of “divide and conquer” and a mechanical novel workflow of coordination to revise the stories. We found that they were good at different things. The typical crowdsourcing approach could catch details like grammar and spelling errors. The Mechanical Novel workflow was better at creating a coherent plot and writing character description.
This Mechanical Novel story excerpt shows how crowdworkers edit and revise a text.
What does this kind of work tell us about the potential of crowdworkers?
Crowdsourcing is another approach to AI. We could be using crowdsourcing more for creative tasks. With crowdworkers, there are humans in the loop. That’s important, because in creative work, there’s not always one right answer. High-level things can be hard to express to a machine. People can understand more “fuzzy” things, like the feeling of a certain design or image.
Machines aren’t super good at brainstorming ideas. Humans are pretty good at that. Creative people may want human input on their projects—for example, would the design of this flyer make you want to go to an event? There’s a value to a set of humans saying, “Yes.”
If you are stuck, need ideas, it would be nice if you could turn to a crowdsourced assistant or an “expert crowd” that could jump in and do a task with you quickly. This is a high-level dream!
What have you been working recently at Adobe Research?
I have been involved in great intern projects over the summer, looking at how to make collaboration and feedback exchange easier for graphic designers. We’ve been doing a lot of interviews with designers to learn about problems and pain points in this space and are working on prototypes of tools to help.
Joy Kim at Adobe Research’s San Francisco office
Based on an interview with Meredith Alexander Kunz
Photos by Claire (Qin) Li
