The Challenge
It was 2019, and I was a designer on Salesforce’s AI product, dubbed Einstein.
Although conversational UIs had been available for well over a decade, the previous year had been an important inflection point for voice technology. Dedicated hardware such as Apple’s HomePod were accelerating adoption, and consumer adoption numbers were undeniable.
Around this time, Salesforce also dabbled in this trend with an enterprise desktop device called Einstein Voice Assistant, targeted towards Sales Cloud customers.
Yet despite this clear trend in HCI, I realized our team had absolutely no knowledge or perspective on how conversational UIs could help us better serve our users on Marketing Cloud — so I set out to fill this gap.
The Plan
I set the goal of developing a strong POV for how conversational technology could serve marketers.
Yet we didn’t have a research budget, and there was no executive sponsorship for this effort on Marketing Cloud. But with the world changing rapidly around us, I saw this as a necessary moment, and one that would energize our team.
I floated a few paths forward, ultimately landing on the idea of running a simulation workshop.
Part 1: Research
We were starting from square one. No one on my immediate team had dabbled in conversational UIs at the time. So we needed a shared foundation for structuring our ideas around conversational design, which would serve to level up all of our understandings.
Externally, I attended a local voice technology summit in San Francisco, connected with a couple of leaders in the industry, read books, newsletters, articles, and soaked up everything I possibly could about this cutting-edge domain.
Internally, I connected with several experts who’d already put a lot of thought into this on other teams at Salesforce, building relationships, and learning the tech stack that was being worked on to enable these in the future.
All of this equipped me with enough knowledge to define a set of frameworks that would underpin our POV. Not to mention new relationships in the space, and the street cred to be its advocate for our team.
Part 2: Setting a Foundation
Our foundation was composed of the following elements:
- Skills
Pegged as the basic units of Conversational UI - Levels of Analysis
Which level was most helpful to focus on between Stories and Microinteractions? - Workflow Depths
Intuitively, we knew that some use-cases happen on the go, some early in a workflow, and some later on in a workflow. This framework helped structure that thinking. - Objects vs Actions
Skills apply actions to objects. - Multi-Modality
When and where do responses necessitate visual feedback, in addition to conversational feedback? - Workshop Framework
Finally, we needed a framework to guide our ideation in identifying use-cases. We called this the Object-Phase Framework. (More on that later.)
Part 3: The Simulation Workshops
Workshop 1: San Francisco
We gathered a team of PMs and designers within the Einstein team to run the first workshop.
Some people in NYC and other places wanted to join in too, so they participated in a parallel whiteboard grid I spun up in LucidChart.
The workshop went as follows:
- Each participant would represent a real customer
- For a set of realistic marketing campaigns, participants would step through the entire campaign lifecycle – planning, building, execution, and learning.
- At each moment in this lifecycle, participants would log stickies describing a conversational way of achieving their task, if possible.
Workshop 2: Indianapolis (Remote)
With all of the participants coming from the Einstein team, we knew we might have biases in the data that would be need to be controlled for. So we ran a second workshop with our UX and product partners in Indy, following the same structure.
We didn’t have budget for me to fly out to Indy, so I had to guide some volunteers to do the setup prior to the workshop. We ran it the same way as Workshop #1, with me standing by and facilitating remotely.
Afterward all was done, since I would be the one synthesizing the stickies, (and photos were difficult to capture the data), a collaborator carefully stacked the stickies according to the grid sections, and mailed them to us.
Synthesis & Output
With 400+ data points now in hand, our final readout sought to answer several questions:
- Which objects were mentioned most frequently?
- Which actions were mentioned most frequently
- Which objects were most often paired with secondary objects?
- What modality of output was most expected?
I pulled in a data analyst to help with the synthesis here, and together we came up with some very clear takeaways that would become our POV.
Impact
Initial Impact
This isn’t a story about some immediate win in a blaze of glory. Shortly after this effort, Salesforce abandoned all voice initiatives, and the immediate potential fell by the wayside.
However, our team now felt energized and prepared for what we intuitively knew was to come. We weren’t watching the cutting edge from the sidelines — we were ready for it to hit.
Along comes ChatGPT and Agentforce
Seven years later, Salesforce is now all-in on conversational UIs with Agentforce.
And because of this effort, we were ready. This intrapreneurial effort has been informing our approach to Agentforce, one of the biggest bets in Salesforce’s history.
Reflection
One of the biggest critiques was: “This was done using internal people, not actual marketers.”
And that’s totally valid. Lack of funding drove us to use simulation rather than external research. What’s important is that we started with no expertise in conversation use-cases, and using what resources we had, developed strong hunches that could later be validated once it became a corporate priority.
And sure enough, that’s exactly how things are unfolding.