For our testing of the first response, we wanted to learn if an information-based or inquiry-based first response was most effective at engaging users with art and the app. From the standpoint of the user, what type of response would help the user understand the nature of the app experience, and look more closely at art or engage more deeply with the artwork? From the standpoint of the Team, what type of response was best at providing an immediate response, and what type would be most compelling to the user?

In our first round of tests we experimented with having the first response from the Team be strictly information.  The user received the opening prompt, “What work of art are you looking at?” and the Team responded with information only. Information-based responses varied, some provided information about a specific detail that the user would be able to observe on an object; other answers included broader information that provided geographic, historical, or other types of contextual information about the object. For example, here is an exchange the draws the user to look at specific details, and provides contextual information:

In testing information-based responses, we learned users appreciated information—it provided context and a broader view of the object they are looking at, and had the potential to support closer looking. We also learned that the right type of information helps to establish the authority of the Team. Users trusted the app experience more when they believed that the individuals responding to them were knowledgeable, and could offer information that they would not have access to just by looking at the work of art or reading the label.

Our second round of tests experimented using inquiry-based response, the question was simple, and we used the same wording consistently, “What drew you to that object?” A majority of users told us that they liked being posed a question, “I liked being asked the question, it made me look at it [the object] again,” and “I liked that I had to think.” Additionally, using this simple and proscribed first response had the advantage of providing an immediate response to the user, and also helped the Team member have additional time to gather information about the object.  While the user was crafting a response, the Team was able to collect information about the object using the Dashboard, and our collection of wikis.

In addition to providing time, using inquiry had the advantage requiring action on the part of the user, and functioned as tool to immediately engage the user with the work.  It addressed one of our big picture goals for the ASK app experience — provide visitors with an experience that has them engaged with works of art.  By asking, “What drew you to this object?” the Team was able to quickly gather some information about the user’s interest, and helped us engage the user on a more personal level, and generally led to a deeper discussion about the work of art.  By answering our question, users found themselves looking back at works of art more closely, and thinking more critically about the objects in the Museum.

With this in mind we will use inquiry as our first response moving forward, and also integrate it into the next version of the opening prompt.  In my next blog post I will discuss our process of crafting a new prompt, and what we learned from our second round of testing using inquiry.

The ASK app functions like a text message conversation between users and the Audience Engagement Team.  Users can send a text message or a photo. In our first few testing sessions we learned, very quickly, some basic rules which have remained constant in our two months of testing—in retrospect these basics are obvious—users wanted the experience to be similar to how they use text messages in their daily life, and they wanted the experience to feel personal:

• Users wanted to receive an immediate response after they sent their first message.
• Users preferred short messages in response, rather than a large blurb of text.  We could send the same information, we just needed to do it in bite size bits.
• Users enjoyed when the conversation had an informal tone to it as it helped establish that there was indeed a real person responding.
• Users appreciated receiving new information that they didn’t know, and they also appreciated when we revealed that we didn’t have an immediate answer to their questions—it actually helped to create more trust from the users—as per above, it helped to establish a sense of familiarity and a personal conversation.

Our original prompt.

Using this basic information as a starting point we set out to deconstruct our text message conversations, focusing specifically on the first message within the text message exchange.  We wanted to learn how users would respond to our “opening prompt” (the first message that the user receives when opening the ASK app).

The opening prompt that the app presents to the user has a huge responsibility. We learned from early testing that users did not want to read lengthy directions or go through a multistep “onboarding” process.  With this in mind we knew that the prompt needed to be short, and needed to get the user actually using the app immediately.  We created a prompt that was short, directed, and began with art: “What work of art are you looking at right now?”

Through our testing sessions, we wanted to know if the opening prompt was effective in quickly generating a conversation between the visitor and ASK team. From the user’s standpoint, what will get people interested and using the app quickly? From the Team’s standpoint, what will provide the best starting point for conversation?

Prompt was changed to elicit more deliberate action on the part of the user, a prompt that would require to the user to not just immediately engage with the app, but also immediately engage with the art in the Museum in a thoughtful manner.

Data from post-testing feedback sessions (group conversations with testers), and information gathered from surveys brought us to the conclusion that the opening prompt was successful in getting users to use the app because it was easy to respond to and testers began using the app immediately. However, while the prompt was easy to respond to, testers were confused as to what would happen next. Additionally, we’d see users arbitrarily choose an artwork to send and that was frequently the first work of art they saw, and not necessarily an object they were interested in.

Based on this information we knew that we needed a prompt that, like this one, motivated testers to begin using the app immediately.  The prompt needed to be equally directed, but somehow provide the user with an idea of what the app experience would be, and have the user motivated to want to continue the conversation.  We decided that the prompt needed to elicit more deliberate action on the part of the user, a prompt that would require to the user to not just immediately engage with the app, but also immediately engage with the art in the Museum in a thoughtful manner.  This led us to our new prompt, “Find a work of art that intrigues you.  Send us a photo.”

It immediately proved to be a positive change.  As with the previous prompt, users engaged with the app immediately, and in addition, they remarked on how the prompt initiated them to start looking at the art more closely, to really consider what work of piqued their curiosity and interested them.  Some users continued to note some confusion as to what the full app experience was “supposed” to be.  However, we received at least half of the number of these types of comments as compared to when users tested with the first prompt.

We will continue to use this new prompt, and experiment with ways in which the Team is following up to users first message.  I will discuss the process of finding the best type of first response in my next blog post.