Good Design is 💩

October 23, 2018

“ People use emoji as who they are. They’re like, “I am 🌸⚡🐬That’s who I am.” Jennifer Daniel

In this episode we tell the unexpected origin story of our new universal screen language…..Emoji! We talk to the designers behind some of the most iconic emoji, and the steps and missteps emoji design took before finding its way to our screens.

 

TRANSCRIPT

 

KHOI: Welcome to Wireframe, from Adobe. A podcast about good design. And specifically about user experience design — how we shape technology to fit into our lives.

I’m Khoi Vinh, principal designer at Adobe.

Each week on this show, I’ll be joined by one of our producers to explore one aspect of good design.

This week, I’ve got Isabella Kulkarni here in the studio.

Hi Isabella…

ISABELLA: Hey Khoi! So I brought something to show you It is what you might call a WHALE of a book.

KHOI: Oh really?

ISABELLA: It’s 700 pages; it’s heavier than a motel Bible… So, if I could just get you to open up a random page and tell me what you see…

KHOI: Okay, chapter 89. Heart, woman, cat, cactus, angry face, American flag, film camera, film camera, unidentifiable, [laughs] and factory?

ISABELLA: And we’re looking at emoji right?

KHOI: Yes.

ISABELLA: So, this is translation of Herman Melville’s… Moby Dick. But if you flip the book over you’ll see this version is called.

KHOI: “Emoji Dick.” (HAH!) I get it. With a whale emoji on the cover.

ISABELLA: Yeah, Emoji Dick! It’s an all emoji translation of Moby Dick. Compiled and edited by a man named Fred Benenson.

And so on each page of Emoji Dick you can see strings and strings of emoji – there are thousands in total and together, it tells the story of Ahab and the whale. Emoji Dick is even shelved in the Library of Congress, right next to Melville’s original.

KHOI: Whoa- that’s kind of remarkable.

ISABELLA: Yeah, pretty wild that emoji have found their way into a book! I mean, Khoi, do you remember the first time you saw an emoji?

KHOI: Vaguely, I mean I remember the poo emoji sticking out as me as being weirdly disgusting but kinda cute. I didn’t think that this was gonna be around or have a meaningful impact on the way we communicate. Certainly could not have predicted that emoji would have an impact on the world that it did.

ISABELLA: Yeah and so in this episode, we’re going to talk all about Emoji.

How these, scrappy, experimental icons — became so popular. and ultimately grew so unwieldy that designers had to step in to control them.

So Khoi– I learned that the earliest emoji date back to the late 90s in Japan. They were introduced by a telecom company to help liven up people’s texts and pagers.

And you may remember the early set of emoji and they had this pixelated look – with a lot of geometric shapes. They’re very early internet.

KHOI: Yeah. I remember them from the early 2000s when they showed up in gmail. I mean, gmail was pretty primitive back then. They were really sort of pixelated and basic, they were very different than today.

ISABELLA: So, I actually spoke to one of the designers back in the day, who created this emoji set for gmail.

SUSIE SAHIM: It was one of the funnest projects I ever did at Google. These animated gifs with these little faces and they were just so fun to design

This is Susie Sahim. She worked on the design team at Google.

SUSIE: … and I was one of the original designers of the first emoji set for Google called Goomoji.

Susie says in 2008, her team at Gmail was approached by Google to bring these emoji — which remember were already popular in Japan, to Gmail – so Japanese users would use their service.

SUSIE: It was important at that time to provide emojis for gmail users in Asia// because it was part of that culture there using emoji was just so pervasive as the way people communicated there that it made sense from a user perspective to offer something within gmail.

So these emoji were designed in a set. There were smileys, a flower…

SUSIE: The crab, a monkey, and a robot. There’s also one of two characters hugging. There’s a character twirling its mustache.

ISABELLA: And all these characters were in the Google colors, red, yellow, blue and green. And then of course there was the POO that you remember so well, Khoi.

KHOI: And I found out that Poo is an auspicious symbol in Japan. The word in Japanese sounds a bit like the term for… good luck.

In the the version Susie’s team designed for Gmail – because you better believe it, poo was in there — they decided to liven it up a bit.

SUSIE: We had animated it so that it (LAUGHS) was even more disgusting than the original concept because of the way the poop flopped onto the screen. We thought it was both gross and hilarious at the same time.

KHOI: Yeah, that is so great. I mean, so much of design is so serious and it’s about results for the business, but design is also about being human and bringing character and fun.  And if something’s a little odd or weird and it can make it through? I think that’s terrific. It’s probably one of the main reasons emoji worked so well is they connected with people.

ISABELLA: Absolutely. Yeah. And Susie actually says that approaching design this way – with humor and play –  that was the spirit of this team. In fact, bringing these emoji to the global stage in Gmail was sort of an afterthought.

SUSIE: We were really happy with what we created and decided it wasn’t going to hurt us to make it available everywhere

ISABELLA: The global launch was kind of a lark. Not even emoji’s first designers expected much from them.

But over the course of a couple years… that changed. The first emoji keyboard was officially announced on a smartphone in 2011. On the iPhone. And, Khoi, you may remember suddenly a lot of people were using these little pictures in their texts.

KHOI: Yeah that’s a huge moment bc it suddenly becomes much much easier to use emoji. Because they’re everywhere. And you know everyone has access to it. That is a make or break moment. They’ve gone from this frivolous thing that maybe you think people have a lot of time on their hands are using, to something that’s built into just about every single phone out there. It totally levels it up.

ISABELLA: Exactly, and people were really using emoji to communicate in all kinds of ways. As people were spending more time on their screens. Emoji started to permeate every day communication. In, email, text… even dating apps. …it became a clever way of adding something lively to text conversations. Like, say, you were sending a flirtatious text.

MARCEL DANESI: Undoubtedly almost all of the time the texts would end with a little heart.

That’s Marcel Danesi, he’s a linguist at University of Toronto. And as a professor of semiotics, Marcel is always on the hunt for interesting developments in language.

MARCEL: If it was something that was ironic, there’s a cat emoji which I started to love. Love that cat emoji [laughter]. It would be there at the end with its sardonic smile.

To Marcel, emoji looked a lot like a language. There are patterns and structure, and a kind of grammar…

MARCEL: So at the end of a thought it has a punctuation function. But you tell me the difference between a period and a heart. Wow! What a difference.

So, Emoji didn’t replace words; it punctuated them in the same way gestures work alongside some languages like Marcel’s native Italian.

MARCEL: If you go to Italy and places like Naples, you can see people actually communicating to each other through gesture with very few words. It is a kind of visual gesture language that, however, goes a little bit beyond the semantics of ordinary language.

In just a few years — emoji became a global phenomenon.

In large part because you could find them NEARLY everywhere you went online.

From Facebook to Twitter to Slack.

Emoji moved into the realm of politics and pop culture too – President Obama even gave them a shoutout in a speech to the Prime Minister of Japan.

And on July 17,  20-14, emoji enthusiasts celebrated their first World Emoji Day.

Then in 2015, the Oxford English Dictionary named an emoji the word of the year. It was the tears of joy emoji. That face that looks like it’s laughing and crying at the same time.

For Marcel, this was a big moment for emoji…

MARCEL: It acknowledged that indeed emoji was now part of the way, an intrinsic part of the way we communicate and that went way beyond anybody’s expectations.

People like Marcel and his students were using emoji for another reason, too. Not just that they offered something new as a language — but also because of how it was designed.

MARCEL: They make it so easy. So easy to use, look I go to my new outlook and i see the whole set of things all i have to do is go click! Simplicity is one of the first laws of communicative change.

In other words, because emoji are so simple they’re actually changing the way we communicate.

So “communicative change” is a linguistic concept — that’s also a pretty basic part of interaction design, is that right Khoi?

KHOI: Absolutely. If you wanna change people’s behaviors, making something simpler is one of the best ways to do that.

But at the same simplicity can create other kinds of design challenges especially when things get so simple they start getting so simple, they start getting used in ways that was never intended in the first place.

After the break, we’re going to find out what happens when something that was never supposed to be a language gets bigger than anyone.. ever… intended.

We’ll be right back.

Welcome back to Wireframe, I’m Khoi Vinh, and I’m here with producer Isabella Kulkarni.

ISABELLA: When we left off, emoji had exploded across our devices, making their way to Facebook, Instagram and Slack. But there’s a problem for emoji—and it has to do with the technology that supports it.

Khoi, have you had the experience of sending an emoji and getting a jumbled mess of question marks? Or having a black box appear instead of the thing you meant to send?

KHOI: Yes I have. That’s a problem with the way the devices are communicating the language to one another. The technical term is encoding. And in those early days, the encoding wasn’t happening properly for emoji.

ISABELLA: Right, so this problem starts to surface in a big way in 2011 … Some devices weren’t showing emoji properly. So people were getting this frustrating black box instead of the emoji that they meant to send.

And as emoji grow in popularity, more and more people are seeing this …

Which isn’t what we’re used to with our communication — you know, we expect things to mostly just work.

KHOI: Yeah, we do. And the reason that mostly just works for language is because an organization called UNICODE regulates the way these languages are encoded.

UNICODE is made up of software engineers and designers from all kinds of  big tech companies like Microsoft, Google apple. Adobe’s a part of UNICODE too. And their job is in part to make sure that anything you send on your phone comes out the same way on the other side.

ISABELLA: But they weren’t doing this for emoji … until emoji started to get super popular. So again this is around 2011 … and it’s at this point that designers from the major tech companies got together and they made a decision: they’re gonna ask UNICODE to standardize emoji. And UNICODE says yes.

And this was a really big deal for emoji. UNICODE is saying “emoji is an important part of how we communicate.”

But the part of that that I thought was a little weird, Khoi, was that designers would be asking for MORE oversight over their designs.

KHOI: Designers respond to constraints. They really try to make things work based on what the rules are. And UNICODE’s involvement is imposing a kind of constraint on the way emoji are supposed to work. And that actually helps designers come up with better solutions that are more consistent and work in the way users want them to work.

ISABELLA: So UNICODE stepped in and took on regulating emoji, and this mostly solved the problem of this little black box. Mostly. But once emoji was under the governance of UNICODE, the organization was now the default referee for other problems in the emoji arena — like — who and what — emoji looked like.

Here’s Rob Giampietro the Director of Design at MoMa, and a former designer at Google.

ROB GIAMPIETRO: Initially emojis come in one skin tone color. You know, like just yellow. And then you know people start to say, hey this isn’t really a very representative skin tone for most people.

Emoji reflecting a spectrum of skin tones was leading to a lot of public criticism.

And Rob says once UNICODE got wind of this inclusivity problem, they set about trying to solve it.  

ROB: So they use use this thing called the Fitzpatricks skin tone scale which is kind of used by dermatologists to give the different skin tones in the emoji set.

But inclusivity isn’t just about skin tone. Emoji fell short in other ways…

For example…

  • Some professional emoji, like a construction worker or spy, only came in a male character.
  • There was a limited menu of food emoji…
  • And a handful of national flags were missing…

So, UNICODE started to make emoji design more democratic. In 2015, they invited the outside world to propose new emojis.

A few years later, after UNICODE introduced a more racially inclusive skin-tone slate.

There was also a wider representation of culture… for example, a girl in a headscarf was added to the emoji set, as were same-sex couples.

And we got chopsticks and a burrito emoji.

So emoji was continuing to evolve. But, another problem quickly emerged.

Emoji often looked different on different platforms — an iPhone vs an Android might have a different version of the same icon.

JENNIFER DANIEL: Look at the picture frame emoji. Everyone had a different picture in the picture frame. So Microsoft had a cat, Twitter has a red lighthouse. We have a mountain.

That’s Jennifer Daniel. She’s responsible for Google’s emoji program. She says the new problem for emoji was making them universal.

JENNIFER: All those pictures communicate something very, very different. And if you are using the picture frame emoji to punctuate a sentence, having a picture of a cat versus a picture of a mountain mean completely different things. And that’s a problem for everybody.

And these variations were endless. Apple might have a different idea of what something should look like then, say, Google.

Like, the emoji of a dancing woman on an iPhone — showed up as a dancing blob with a rose in its mouth on Android.

But this all came to a head. With the gun emoji. Here’s Rob again.

ROB: The gun emoji was actually you know represented by Apple as a squirt gun. So you know I might be sending you on an Android platform a picture of a handgun and you as an Apple user might get a picture of a squirt gun because you’re on that platform.

Obviously those two representations of the gun suggested very different things.

ROB: ….therefore you could create a miscommunication simply because of the difference in platforms across the two users.

ISABELLA: So, something had to change. But standardizing how the designs should look across platforms, that was beyond the purview of UNICODE. Here’s Jennifer again.

JENNIFER: The only reason why UNICODE oversees emoji is because they’re in charge of universal translation. And they are in a position to really be able to support this for all devices. But because they’re thinking about it from a database point of view, they’re not really thinking about the user.

In other words, as long as emoji were showing up on people’s devices, UNICODE’s job was more or less done. But that didn’t solve the problem for designers.

In order to limit the misinterpretation of emoji, designers like Jennifer say they started to shift their thinking in how they designed emoji.

They started to make emoji look more universal to prevent these differences across platforms.

This meant thinking through every element of a single emoji—down to a feather on a peacock.

Here’s Jennifer again.

JENNIFER: What do people imagine when they close their eyes and think of a Peacock? Likely, they think of the peacock with the beautiful plumage and the color. But there’s actually lots of different kinds of peacocks. And so you have to think about what is the symbol for a peacock, not literally what is a peacock.

Eventually designers started to design emoji to be more similar – to prevent these miscommunications through designs.

Now, when you send a dancing lady no more blob with a rose, almost all platforms show identical dancing ladies. Everything started to look the same, platform to platform.

But, for some designers, this has a downside. Jennifer says that in this push to homogenize their design, emoji have lost a piece of what made them so fun.

JENNIFER: If you looked at emoji designs maybe just two years ago, not that long ago, you really saw some personality. The Samsung death emoji, where it’s like this face that looks like it’s from the painting The Scream — had this little ghost coming out of it, it was really scary, it was great.  
So designers have gotten on the same page — but you can’t stop USERS from using emojis however they want.  

JENNIFER: The loudly crying emoji is a really good example of that. There’s crying because I’m so proud of you. And there’s crying because I am actually upset. There’s like a whole spectrum of why loudly crying can mean good and bad. And that’s the beautiful thing about language is that it’s fluid. It changes over time. It doesn’t matter what Unicode calls that emoji. They can call it peach. But everyone knows it’s a butt.

ISABELLA: So, Khoi, isn’t that part of the strength of emoji that it can mean different things to different people? That we could use peach as a stand-in for butt?

KHOI: Yes absolutely. They’re kind of a language. And just like any language, the constituent parts can mean different things in different contexts.

So that actually makes it more robust. So like the poo emoji. It now has new meaning to me now that I know it means good luck.

These types of misinterpretations of emoji have added to their power. They’ve actually created new meanings.

ISABELLA: Yeah, And in this way, we’re kind of back to the start of this wild west of designing a new language for screens. where the meaning of emojis are fluid, constantly changing and evolving.

Here’s Jennifer again.

JENNIFER: People use emoji to connote their gender. People use emoji as who they are. They’re like, I am pink flower, lightning bolt, dolphin. That’s who I am.

ISABELLA: So Khoi, if we’re at a place with emoji where they represent how people identify, that seems like a really big design task….

KHOI: Yeah it is. I think what you see with emoji is the designers are watching what users are doing and trying to reflect those changes, those patterns of behavior in new versions of emoji and that, in turn, influences users again. So it’s a virtuous cycle any technology needs to keep pace with the way users are changing if it wants to sustain itself and be relevant.

ISABELLA: Yeah, Jennifer agrees with you.

JENNIFER: If emoji can’t keep up with the complexities of gender, of identity, of what people talk about, then it will become outdated. The world moves very very fast, right, And so to keep up, to be relevant, but also be timelines there’s a lot of tension in there.

KHOI: Yeah. I mean it may sound like the story of emoji has been very rocky but this is actually what it means to design things that people use. This is Interaction Design. There’s a balancing act between technology, design and the user, and to make it work there has to be a feedback loop that unites all three of them.

So the lesson here, I think, is that you can’t ever be sure what the effect of your design will be. You have to be open to reinterpreting your work. You might think you’re making a toy, but you might wind up creating this powerful communication tool. An entire language. And then one day—-your toy… it winds up in the Library of Congress.

 

CREDITS

This episode was produced by Isabella Kulkarni, Rikki Novetsky, Abbie Ruzicka and Amy Standen.

Devon Taylor is our editor.

Catherine Anderson is our engineer. And Keegan Sanford created our show art. Special thanks to Fred Benenson for lending us his copy of Emoji Dick.

Learn more about the show at adobe dot ly slash wireframe.

You can subscribe to Wireframe on Apple Podcasts, Google Play, Spotify, or wherever you get your podcasts.

And leave us a review! We’d love to hear what you think.

I’m Khoi Vinh. ….Thanks for listening.