Ex-Yahoo CEO Marissa Mayer on her new photo app, souped up by AI

Q: You started off with a contacts app and now you have a photo app. Can you explain the strategy?

A: Photos and contacts go together. It’s obvious when you think about it, but the contacts you’re closest to, you tend to spend time with, and if you spend time with them in this day and age you have photos of and with them. So you can understand people’s relationships if you can look at their camera roll. And now with the beauty of AI, we can actually analyze someone’s camera roll really succinctly and have an understanding of who you spend time with, when, where and why, at least in broad brushstrokes.

That’s really useful when it comes to keeping people in touch with each other, helping take those mundane, everyday touches and make them magical and really helping people feel connected to each other. Contacts are a big part of that. It was the foundation and that’s why we started there. But photos are perhaps one of the most meaningful ways that we all interact with each other. The way we all build those relationships.

Q: It resonates with me because I’ve never loved social media but I do want to share with people I know. Do you see consumer tastes changing these days in the post-web 2.0 era?

A: Yes, maybe. I also think that AI and our expectations from it are going to influence it. One way to think of it is if Instagram is one perfectly filtered, edited photo that you push out to everyone you know and don’t know who is in your follower base, and person-to-person photo sharing is me snapping a picture and texting it or air dropping it to you, Shine tries to sit in that space in between. The ’okay, I was just somewhere with 10, 20, 50 people.

And it was an interesting night, but I didn’t kind of want to take myself out of the moment and take a photo. But I know I took some, they took some, let’s just make a big photo pool of what that trip, party, conference, event felt like.’ That’s really the space we’re trying to occupy.

There’s not a lot happening in that space. I refer to it as smart, small-scale sharing. The early social networks and early social media tried to fill that space. But then people would join, they wouldn’t see a lot of the people there that they wanted to share with. So they looked for who is famous, which is why you suddenly started to get these counts into the millions for celebrity followers.

But at the same time, celebrities are interesting, but so are your friends, families and your close contacts and they’ve got stuff going on in their lives that they want to share with you, too. That’s really the space we want to play in. I think Snapchat showed this in a different way — you can share in a way that’s a little bit less perfect, and a little bit temporary and in the moment. There’s something really powerful about that.

Plus the fact that, as engineers and product people, we look for everyday things that just feel broken. And the fact that you’re like, ‘I know this picture was taken, and I don’t have it, or my friend forgot to send it to me.’

Or that feeling that you have when you’re at an event and you see someone snap a scene of the first dance, the child blowing out the birthday candles and you feel the need to pull your phone out and take the same picture. Because you’re like ‘okay, the only way I’m going to be sure I get it is I actually take it.’ To me, these types of tools can let us be much more in the moment. At the same time we don’t want to just move a lot of photos around because there are lots of duplicates that aren’t organized.

Q: And that’s how you’re using AI? You’re picking the best?

A: I talked about how AI is going to change people’s expectations. That’s a big part of it. Because now we can take photo sharing and make it so what you get is actually an organized album of things that people would want to share, that you would want to see. And yes, if they took eight pictures and they weren’t sure which one has all the eyes open, and they weren’t sure which one wasn’t blurry, AI can do all of that.

Q: How else are you using AI?

A: So we’re doing AI-powered curation for the albums to understand what’s shareworthy. When you’re at an event there are things that aren’t logical to show up in the album and there are things that are. We’re using AI there and also to do the grouping near duplicates. We also use AI in our suggested albums that live in our ‘For You’ tab.

The suggested albums are one of the most powerful features in the app because it doesn’t mean that you have to necessarily have a Shine album right now. You can actually build Shine albums in the past. We’ve built AI algorithms that go back over the metadata of your camera roll. Where were you, when did you take pictures? Basically analysis of it, and try to understand where and when you take boring, mundane photos that you probably don’t want to share, and where and when, and what do you take pictures of when they’re pictures that you do want to share? So we got AI operating there as well.

And then one of the things that we noticed is that this is the kind of smart, small-scale sharing that you want to do at events. The natural place to get the lead-in from that is invitations. So we have our Shine website that does events. The nice thing is that in that case, you know the group of people, you know they’re all going to be there. And if you can actually just make the photo sharing contiguous.

If you have all the RSVPs, it’s all part of Shine. You’re just naturally joined to the album, and you can enjoy the event, naturally participate in the photo sharing, receive the photo sharing. Our invitations use a different form of AI, which is generative AI. We’ve got really breathtaking, clever and custom invitations that are based on AI prompts.

Q: Those are great.

A: Some of them have taken our breath away. Some of them make us laugh out loud. They’re really fun and they really add a lot of personality to the events that people are throwing.

Q: You’re a technical founder and you’ve run teams of coders. Do you see AI tools changing the way software is developed?

A: One of the most immediate impacts is in testing. The AI software is clearly there. Lots of tools do a really good job with test harnesses. You can always do more and you can always find more with testing, but I think that there are some elements of coding and the process of bringing a product to market that are going to be AI-led sooner rather than later. Some of the end stage processes, like testing and QA, are clearly going to be better run and more efficiently run because of the AI tools.

It doesn’t mean those jobs go away completely, but rather than writing the test harness yourself, you can actually have the tools really do a lot of the development for you, which helps you cover a lot more cases and a lot more ground quickly.

Q: As somebody who started out in symbolic systems, do you geek out on the architecture of these models these days?

A: The thing that I’ve found the most surprising and delicious in all of this is symbolic systems. Symbolic systems at Stanford is a unique major. It’s basically cognitive psychology with a really heavy CS bent. But the actual makeup of the major is cognitive psychology: How do people learn? Philosophy: How do they reason? Linguistics: How do they express themselves? And computer science: Can you create a computer that can learn reason and express itself?

I was fascinated by this in the mid 90s. Both my degrees at Stanford are in the concentration or a specialization in AI for my bachelor’s and my master’s, respectively. I thought that was very interesting and there’s all these symbolic systems. We see that thread, learn, reason and express yourself.

I loved computer science from the very first time I coded. I didn’t know much about linguistics. I understood it was important for the intelligence to express itself, but my focus and I will say overall, the industry’s focus, has over the past decade or two, been much more on learning. Building the models, reinforcement learning, all of the different techniques that go into building a modern day model right now, and then deploying those models, giving them new things to reason about and see how they perform. That’s where all the energy is gone.

And the big surprising thing that happened with OpenAI and ChatGPT, at least for me, was that expression piece, that linguistics piece, that was perhaps under-appreciated in that cycle of symbolic systems is what captured everyone’s imagination, the fact that it can talk and express itself and write in a way that you can understand and feels really intelligent. The fact that Dall-E can draw these wonderfully complicated pictures that would take an artist days or weeks to express. That really captured the imagination.

So it’s funny because for a long time, I didn’t understand and kind of discounted that expression piece. And in this moment, it’s that expression piece that has been the big unlock for AI, and really helping everyone see its potential.

Q: When you were at Google, all the deep learning that was happening then was really a result of compute power. The ability to test a lot of these theories that had been developed in academia. Now you have the data and you have the compute resources to do it. But if you look at what’s happening now, it just dwarfs what was going on then. Does that surprise you? Just how massive the resources are that are going into developing these things?

A: Sure, I think that one they did not discount is the data, right? One of the reasons Google can do this is just the huge amount of data, as well as the compute. It’s easier to connect the dots backwards, than it sometimes is to connect them forwards. I do think that when you look at everything Google did from spell check to Google Translate, this notion of building things that weren’t modern day LLMs but basically used huge amounts of data, huge amounts of compute to make predictions about what comes next and how things should be constructed, whether in another language, whether in terms of spelling.

And so I think that it does boggle the mind what they’re doing today. But when you look back at those underpinnings and where Google was working, we were using it on spelling and then later, we used it across languages. If you use those same techniques, applied slightly differently within a language, you can also get remarkable results.

Q: If you put your CEO of a large company hat on, do you spend tens of billions of dollars to scale up transformer models or do you use what’s happened with ChatGPT as the inspiration to find the next breakthrough and architecture?

A: The foundations of this, those underlying models are incredibly important. At the same time, they are also even more incredibly expensive to build. So either you have to know you have natural applications that are going to sit on top of those LLMs and those models that you build, which are going to be very useful for people and therefore very profitable. Or you’re going to have to make APIs where other people are building applications on those models that have that same type of lucrative nature.

That’s why I remain really bullish on Google, because they have the data and they have lots of natural applications in search, in email, in maps and docs, in lots of different areas where it absolutely makes sense for them to invest in that foundational model. They can build their applications and features on top of it, and they can also make it available in various formats and APIs.

I think for a medium or a small company, it makes sense to say no, I’m not going to build a foundational model. I’m going to count on technology giants to do it and then I’m going to pick the model of theirs that works right for my domain.

Q: So you won’t be developing your own foundation models?

A: As a small start up, that doesn’t necessarily make sense. I also feel empowered that we can pick among many different models. Even the cases that we’ve talked about here, from the AI that runs behind suggested albums, to curated albums to the generative AI that powers our invitations, we’ve got a bunch of different models and we can go in and shop for them and get them a la carte.

Q: So you’re taking those and then fine tuning them?

A: Exactly.

Q: Is it good timing to be at a startup right now, as this technology becomes so powerful and potentially disruptive?

A: It’s a great time to be in technology because the big companies are doing interesting things and so are the smaller companies. There are whole new greenfield areas that are being opened up in terms of how this can be applied, how it can be thought about and used.

Q: But you still are within these walled gardens of Apple and maybe to a lesser extent, Google. Those restrictions put limits on your abilities. With the recent antitrust litigation, what parts of that walled garden would you like to break down?

A: Location, maps, all things geo is something that is close to my heart because I worked on maps for a long time when I was at Google. I think that location is so powerful: Where we go, how we spend our time. It does affect your photos, it affects who’s in your contacts, or it should. Location can be a big part of it. But I do think that the two platforms, Android and Apple, are different in terms of granularity, what they really allow you to access, and with how much ease.

They have different philosophies in terms of users and what they want to do, and the types of authorization they want to give. But it’d be great if those two became more standardized, and really allowed developers to have access to the same type of information that the platforms have.

Q: Apple’s line here is ‘we’re protecting our user privacy.’ What do you say to that?

A: I think user privacy and the trust that it builds is incredibly important. It’s important for Apple, it’s important for startups. We know that especially in the type of work we’re doing. Your contacts, your photos, are among the most private things on your phone. So we need to really be very respectful of privacy.

At the same time, we’ve seen big historical examples and even modern everyday examples, of people willing to, with a trusted source, offer views into their private information, if it’s super useful to them. The classic example is your driver’s license. Why would you go and tell the state or the government, where you live, what you weigh, how tall you are, your hair color, eye color, right? It turns out driving is really, really helpful. So people make those kinds of tradeoffs.

My browser keeps track of my history and everywhere I’ve gone and that’s a lot of information about me, what I’m interested in, what I’m learning about. That’s very personal. But it’s also helpful to later be able to ask ‘wait, where was I? Where did I see that? Can I go back to my history?’

If sharing data helps me get photos more easily and share photos more routinely with less forgetting, then, I think there’s a lot of people who would say ‘sure.’ It’s personal, but it’s not even that personal in terms of where you were. Obviously your cell phone carrier has the same information, knowing where your phone was at different times. So there’s other people who have access to that same information that benefits you and also doesn’t benefit you.

This is a case that can clearly just benefit you. So overall I think there are users who want to make that choice. They cannot be misled. That really violates trust. But if they want to make that tradeoff, I think they should be able to do so more easily than they can today.

Q: That tradeoff at Google and other companies means that your data will be monetized. You’re offering a freemium model. So does that mean you’ll have the choice of not having it monetized? I don’t think you’re doing any advertising at this point.

A: Our contacts model is free, but you can pay us to export it back and become your iOS contacts. So if you want your contacts upgraded on iOS, we’ll take the enhanced contacts we’ve created, deduplicate, and enhance them. For Shine, our photos app, and our events app, right now they are just free. In our privacy pledge that we laid out when we started the company, we don’t sell your information.

We want you to understand what information we have, exactly how it’s used, and it should always benefit you as your data. You can take it with you at any time. When we look at business models, I learned never to say never, but our proclivity and our actions to date are much more to go direct to consumers and to have services that people pay for either on a one time or an ongoing subscription basis to ultimately monetize these services. So there’s not a third party in the mix because, especially for someone’s very personal information, you don’t want a third party involved.

Q: It seems like the internet business model is shifting a bit, too, and people are willing to pay for this technology in a way that they were not in 1.0 and 2.0. Do you think that’s true?

A: I‘ll give you an analogy and a story. When I was at Google, we got very interested in personalized search. When you search for an engine, we know that you’re interested in cars, where someone else might be interested in planes or different train engines, or what have you. So we were like how can we make this more personalized for people?

So we did an acquisition and it was a company called Kaltix. They came in, we looked at all kinds of different things we could use to personalize your search, where you live and what you’re an expert in to the extent we had the ability to look at that. We looked at all these different factors and then we looked at our overall search happiness and search satisfaction with results. And what do you think performed best? If you only get one thing to personalize someone’s search, what’s the one thing that’s most useful?

Q: I have no idea. Tell me.

A: The search you just did. If I type carburetor, and then I type engine I’m probably looking for an automobile engine, right? Or I’m looking for something that has to do with the carburetor. Those types of things. So it was kind of fascinating because more than where you live, more than what you’re an expert on, more than what you know, if you know the search they just did, you get the best result.

When you start to look at some of these subscriptions and sign up services, ChatGPT is almost the same principle. If they know the AI tasks you were just working on, they’re going to be able to meet you where you are and help you move it forward that much faster. It’s funny because even in our invitation, we see that people have this tendency when they do prompts, as we’re learning how to help them do better prompts for their invitations. They’ve even come to expect, like make that one but with more red, that kind of iterative notion of building off of where they already were.

That’s something people are coming to expect from these AI technologies and chatbots. They want to be able to pick up the conversation where they left it or the task where they left it. That’s among other reasons why it makes all the sense in the world to have an account because you can do a good job with limited history or with no history at all, just like Google did for years before it had personalized search. But the moment you have an element of history and that kind of longevity of getting to know someone, they’re just going to be that much better.

Another unlock moment will happen in AI when we see these current AI models meet personal information. Today, if you ask ChatGPT, ‘What should I buy my nephew for his birthday next month?’ They’re like, ‘Who is he? Where does he live? How old is he?’

But I think once you actually start to have a sense of people’s personal information, personal relationships, personal preferences, and that comes to bear with a language model, or an image model that is strong and powerful, that’s when there’ll be another big step up in terms of the unlock of what people feel day today.

Q: Do you think we’ll get to this point where there is a model that’s able to really reason and consequently act as an agent for us and be reliable?

A: The current models, which have some amount of reasoning in them but are pretty limited in the philosophical sense, are doing well enough that I’m not sure there’s going to be that much pressure to do that type of reasoning. Over time, it is a very intellectually interesting space and I do think that we will see research advances there. But right now, with the current techniques and the current style of reasoning, the technology is getting us far enough that I’m not sure that type of causal reasoning model, which is pretty different than the way these models are built today, is really called for and is really being driven towards that hard.

Q: Do you do much angel investing and, if so, is there anything that’s blown your mind recently on the startup front?

A: I do some angel investing, but it’s pretty limited. Obviously, a lot of my investment in time and resources goes to Sunshine these days. I invested in a company called Halo AI a few years ago that is basically doing AI analysis, radiology images, and ultimately doing very early detection screens on things like cancer and various conditions. I wanted to be a doctor for a long time when I was growing up. Those types of medical advances and medical applications always get me really excited.

Q: It does seem like that area is so promising and drug discovery too. Do you enjoy the startup life, or is the corporate CEO life more fun?

A: I love all phases of companies. I had a funny conversation with my mom, many years ago, where she was talking about one of my friends who had just said, ‘oh my gosh, 4 is my favorite age.’ And afterwards, she said, ‘I feel so bad for her because someday, she’s going to have a 5-year-old.’ And my mom was like, ‘I love all phases of my children.’ And it was before I had children, before I was married, and I said, ‘you know, Mom, I think I love all phases of companies.’ I was like, ’I like it when they’re small, when it’s single digits, double digits, triple, quadruple. I love it when I’m an individual contributor, I like managing small teams, big teams, teams of thousands. I love the hypergrowth, I even love the challenge of turning around a company that’s in decline or a product line that is in decline. It’s one of the richest, most interesting intellectual and design problems you can work on. I just love building things. That’s really what a company is about. It’s about building products, businesses, teams and cultures. That work and the way that all comes together is just truly fascinating.

Q: You said you were going to be a doctor. And in a way, the human body is this amazing system. Is a company similar to that?

A: Maybe it’s the fundamental engineering mindset. I just love systems. I love symbolic systems and the way they support what is now modern day AI. That’s a system, the system of cognition, the system of a body, I love the system of a company. I like thinking about really complicated systems and how to make them work better. And that’s what products are, too, whether it’s a search engine or a photo sharing app like Shine. They’re all complicated, interesting systems that overall can be made to work better and more efficiently and have a bigger impact.