How Adobe Design creates experiences for nascent technologies

The process that keeps us focused on the possibilities of new platforms amid the excitement and hype

Wildflower image created in Adobe Firefly

.

Which is exactly what happened when we began envisioning the possibilities for immersive experiences on Apple’s Vision Pro.

The opportunity was exciting but daunting: From a design standpoint, how do you begin to tackle a new platform when you’re among the first to design for it? From keeping your team as small as possible, to making sure your pitch (to leadership) is precise, there’s a process that can help keep teams grounded as everyone around them gets carried away by excitement and possibility.

Step 1: Dive in to see what’s possible and try to break things

When being among the first to design for new technologies, it’s impossible to ignore the upfront excitement that accompanies them—the possibilities of what can be done and what could be most fun to work on. That anticipation is what sparks creative exploration. Even though ideas will spring from this initial intensity, the most important first step is to take a close look at the technology to determine both what’s possible now and what might be possible in the future.

This in-depth interaction with the new technology will make possibilities feel more tangible and start to ground your ideas. It's during these high-level first experiences that you’ll learn what makes the technology and the interaction different, and when you’ll start to understand the possibilities of designing for it.

For the Vision Pro, that possibility was its spatiality: The augmented reality of Passthrough, creates new modes of interaction by layering digital information (from, say, an application like Adobe Firefly) on top of your physical environment; and the immersive experience of Environments, that transports people to virtual reality locations like Yosemite or the surface of the moon, points to new methods of interaction, storytelling, and consuming media.

An example of Passthrough where the digital information from an application is superimposed on actual surroundings (top) and the same image in the immersive experience of Environments (bottom).

No longer were these things only in front of you on a single screen, they were transporting you to a location or immersing you in the spaces where you use them. It was the first time I'd seen physical and virtual worlds merge in a way that could change the way people interact with digital experiences.

It’s critical at this stage to dive into the new technology by simply trying it for yourself. Explore, be curious, play—and allow your instincts to guide you. Put yourself in a variety of situations, then try to break things. Afterward, ask people around you to try it and interview them about what stood out. Use what you’ve learned to dive back in and explore more.

Step 2: Shoot for the moon

Start to imagine the most amazing things you can do with your company’s product. What would the experience be like if you could do whatever you wanted? Try not to think about whether you’ll get there, just think big. There are no bad ideas during this phase—play the game with a focus on quantity. A small team can be beneficial at this stage because it’s more comfortable to brainstorm with fewer people—being creative in a crowded room is difficult. With three or four people the level of comfort gets quickly to a place where you can be as ridiculous as you want.

The simplest trick for capturing moonshot ideas is to play a simple game of “What if?” Take something that surprised or delighted you while exploring the new technology and combine it with a key goal or capability of your product. In our case, we asked ourselves, “What if we took the generative power of Firefly and combined it with the immersive experience of Vision Pro’s Environments?”

That first moonshot idea helped us envision how the technologies could coexist simply by generating images from a series of over-the-top prompts. Virtual locations (like a city made of pasta, a cheeseburger island, or a magical paper forest) that had been difficult to imagine just days earlier were possible thanks to creative generative AI prompting.

Step 3: Rein it in

This is the time for more realistic conversations about what’s possible within the constraints of time and technology. At this point you’ll realize that not everything you had in mind is possible, but you’ll be able to start thinking more clearly about what is, and how you can create a product that’s valuable to your users. Resist the temptation to do too much and instead lean in to one key aspect you know you can build in the time that’s available. Focus on one simple thing to make great, make sure that piece works as cleanly as possible, and worry about the rest later. In other words, think about a minimum viable product while keeping it in the context of long-range goals and ideas.

For Firefly, the first-time experience on Vision Pro would be a rotating slideshow of images and prompts—and the ability for people to input new prompts.

For Firefly, we envisioned the first-time experience on Vision Pro as a rotating slideshow of images and prompts—with the ability for people to input a new prompt and generate images. As we continued to explore that idea, there were three development considerations we kept at the forefront to maintain the scope of our project:

1. That it make sense to people who’d used Firefly on other platforms. We didn’t want people who’d used Firefly on the web or in one of our applications to have to relearn how to use it.
2. That it be a logical first step in a longer journey for Firefly integrations on Vision Pro. If our moonshot idea was to be able immerse people in an environment generated by a prompt, the most logical first step was to enable prompting and visible outputs.
3. That the scope of the project was practical and realistic for the development time we had. We knew that our development time consisted of weeks, not months so we planned accordingly.

Again, team size at this step is key: Fewer people means easier decisions and more runway for pushing possibilities. Time will ultimately decide what’s possible, so be honest about what you have time to do, and design accordingly.

Step 4: Pitch ideas to leadership

It’s important when pitching to leadership that you make clear both what you’d like to do (a forecast of what can be built on the shoulders of what you’re building now) and what you can do (what's reasonable and manageable given time constraints). When presenting ideas for nascent tech, start with the long-term vision (Why is this even something we should be thinking about?) so that your presentation not only shows what you want to do, but what it will take to get there and what you need to work on first. Remember, when you’re forecasting, you’re also planning.

Since everyone will be excited in the moment, you’ll be able to talk about what's reasonable from the standpoint of what’s already been done and what could be done next (when there’s more time). And if you’re lucky you'll end up with some intermittent steps. And remember, pitch meetings are for discussion: Be prepared to explain your decisions and the reasons they were made.

From a pitch deck: A description and depiction of a first-time, first-launch experience for Firefly on Vision Pro that replicates what exists currently for Firefly on the web.

Our pitch for the Vision Pro was an extreme example. The possibilities we wanted to share—our moonshots—were inherently futuristic and exciting, but the constraints we were weighing—to ship on the same day the hardware went on sale to the public—were significant. Ultimately, we had a successful pitch, showing how a simple, curated set of rotating widescreen images, along with their associated prompts, could be the first step toward people typing in new prompts, signing in to Firefly, and generating four new images.

One key tip for pitching work for nascent technology is to avoid the weeds. Things that might come up during the pitch that will undoubtedly need to be refined later anyway:

• Specific copy choices
• Minor design elements
• Interop with other apps
• Testing plans
• Technical impacts like server load

It’s okay that they’re mentioned—since they could be critical, eventually—but prolonged discussion can derail the pitch and make the task ahead seem overly complicated. Capture the thoughts and move on. There are many small, and even medium-sized, mysteries that will pop up once you start down the path of development. Let those be part of the joy of the next steps and maintain clarity on the key aspects that make the project worth pursuing. Focus on those, and the pitch will go well.

Step 5: Build what you've envisioned

At this point, a prototyper might stand in for the product team to create high-fidelity prototypes to get ideas across and address specific needs of the technology. This is also a good time to uncover any legal or brand requirements that need to be checked off.

Getting through this stage quickly can be a challenge. It’s still important to keep the core team small, but it’s also extremely useful to reach out to other teams that have traveled similar paths. Relying on the experiences of others who’ve been in similar situations is one advantage of working within a big organization. People may not have time to actively pitch in, but almost everyone has time to provide a time-saving tip.

Despite an extremely short timeframe, our work for Vision Pro went well. New technology is exciting to experience designers because we're always going to be interested in exploring new spaces. One of the advantages of working within the MINT team is that it enables more focused development time—our designers have a place to explore UX for immersive experiences. There will always be new places to create experiences but for now there’s what’s next for Firefly in Vision Pro: The Adobe Research team has some explorations underway... one of them even shoots for the moon.

Special thanks to Arie Huang, Bria Alexander, Greg Muscolino, Danielle Morimoto, and Brooke Hopper.