We began with a very short list of options in a drop down menu that included the special exhibitions, a few specific collection areas, and the collection more generally. We quickly found the need to add a few more options. For example, the admissions team asked for a “just in the neighborhood” option as it’s a common response to the question (though the data shows it’s not as common as they likely felt it was).

The survey appears in a pop-up window and has a drop-down menu of options. Unfortunately, the option to skip or cancel is baked-in; we can’t make this a required question to complete the transaction.

Results for the first two months are interesting. In July, the permanent collection ranked highest in response rate, while for August it was our Pierre Cardin special exhibition. The initial lack of options is one of the reasons for the high “other” response rate in July, which dipped the second month as more options were added. Currently, we have 16 options plus skip/decline. This feels like a lot, but maybe it’s ok. In particular, I wonder about including Korean art and African art in the list at the moment since both are temporarily off-view, but it would help us track an uptick once those collections are on view once more. We also have to remember to update the list regularly as special exhibitions open and close. For example, both Liz Johnson Artur and One: Egúngún exhibitions closed mid-August, which explains the dip in responses.

A quick comparison of the total number of survey responses (which should be every transaction) to total number of visitors who were required to visit the admissions desk shows the transaction count is about 60-65% of the visitor count. Multiple tickets can be purchased through a single transaction—and we know most of our visitors come in pairs or groups—and that feels close to the right percentage. I think we are still getting more cancellations than we should and we’ll keep working on it. The admissions team is meant to pose the question in a casual and conversational manner so it doesn’t feel like a survey (or an interrogation!) and select a response in order to proceed with the sale, although it is possible to cancel and move on. To avoid cancellation, we included a skip/decline option. Unfortunately, not everyone is consistently asking the survey question, which we know because we can run reports on who is logging which responses. For example, we found one person mostly just cancelled the survey in the first week and were able to speak with them. While we don’t want survey completion rate to become punitive, we do want to encourage completion because the information is important for us as an institution. Finding that balance can be tricky.

After two months, we are still working out the kinks, mostly in terms of making this process habit for the admissions team. A next step is to work with our Tech team to create a report that would knit together the survey answer, ticket info, and zip code from each transaction so we can compare the data set as a whole. That would be a pretty powerful triumvirate. 

1. Are ASK users first-time or returning museum visitors?
2. Are ASK users avid museum goers?
3. What kind of experience are users having?
4. Are the users participating in the Frida Kahlo ASK activities?
5. Are the ASK users using the app in Spanish?

The students determined a survey was best, which the ASK team sent via a link as part of the chat. This was more challenging than we thought because figuring out the timing for when to send the survey link proved difficult. You don’t want to share it first-thing, before anyone’s engaged, but finding the end of the conversation can be tough; some users just stop texting. The team had to play with timing a bit and I’m not sure we ever found the perfect moment. 

As with the Kahlo visitor study, we offered an incentive for completion: every 25th respondent got free tickets to Frida Kahlo. We ended up switching that incentive to free Kahlo Dance Party tickets since most users during this time had already seen the exhibition. In the end, we netted 115 responses over 6 weeks. As with the Kahlo evaluation, the results I’m sharing here include some of their interpretation of the data (reviewed by me) as well as some of my own analysis. 

Let’s break down the results. 

Are ASK users first-time or returning museum visitors?

Respondents were almost 50/50 split, with just slightly more repeat visitors. A caveat here though, is that we make this question time-bound, i.e. “within the last year,” so we don’t know if visitors meant life-time visits or something else.

Are ASK users avid art museum goers?

Yes! A majority (72%) visit art museums three plus times in a typical year (this question was timebound). Only 7% of respondents noted it was their first visit to any art museum, while almost 40% visit 6+ times a year.

What kind of experience are users having?

As indicated in the past through positive app store reviews, I’m happy to say we continue to provide a good experience for users. Over 98% of respondents would recommend the app to a friend. The most popular reasons given for this answer included descriptions that could be coded as “fun,” “informative,” and “helpful.” 

Are the users participating in the Frida Kahlo ASK activities?

Yes. Roughly a third (34%) of respondents checked the scavenger hunt as a way they used ASK, while 19% used the Kahlo themed cards.  Interestingly, although this question was a “check all,” a slight majority (57%) only checked on option. It’s possible this could be attributed to users have one main reason for using the app, though it could also be people simply filling out the survey quickly without stopping to think critically about all the ways they used it. Users tend to rely heavily on photos in chats, so the fact that only 22% check that use suggests it’s wise to take these responses with a grain of salt. 

Are the ASK users using the app in Spanish?

Unfortunately, the dashboard doesn’t track this for us automatically, so the way the students chose to get at this was to determine users’ preferred language. It doesn’t quite answer the question, but helps unpack user preferences. Over 80% of respondents (93) listed English and only 2 people listed Spanish. We do know, however, from the Kahlo visitor study that a little over a third of visitors interviewed used the Spanish language materials. Perhaps the lack of synergy between studies means Spanish-speakers aren’t using the app, despite invitations in Spanish to do so. More data would have been required to be really sure.

So, what does it all mean?

We learned a few things from the study. First, our assumption that more first-time visitors would use the app for the Kahlo activities—mostly because we felt like it was an easy way into exploring a new place—was correct. Those least likely to use the scavenger hunt in particular were the users who had visited us the most. 

Second, it’s nice to know most people have a good experience with us, no matter how they use ASK. A few comments include: 

The ask app totally enhanced my museum going experience. Typically I can find usage of phones in the galleries to be pretty distracting, but this app advances the experience. It’s great to have my questions answered in a friendly way- and by a person!

Due to ASK at the Brooklyn Museum, I rate this museum the top museum out of all I visited before!

Keep it up please! This is the best form of digital technology yet!

And third, as can be the case with visitor studies, this one leaves me wanting a bit more. Not through any fault of the students who worked on it, but due to the fact that it’s whetted my appetite for more info about our users. This study paints a small picture of what some of our users experienced, but what would happen if we ran this for longer? Switched out some questions? Got more responses? Something we may have to explore…

A visitor study of “Frida Kahlo” was a perfect project for some of my Pratt grad students.

In my last post, I posited that although we don’t have a CRM, we are gathering data in the ways we can to help inform our decisions. One recent example are the two evaluation projects completed by Pratt graduate students in the Museums and Digital Culture program last semester. I had eight students in the Audience Research & Evaluation course, so I split them into two groups of four for final projects. One team was responsible for completing a visitor study about ASK users and the other about visitors to Frida Kahlo. For this post, I’ll focus on the latter. The student project team for the Kahlo evaluation was Sydney Stewart, Megan Paqua, Grace Poole, and Caroline Baer. The results I’m sharing here are their interpretation of the data (reviewed by me) and their work (including the visuals).

The objective of the study, as defined by our institutional needs and finalized by project team, was to determine: who is coming to Frida Kahlo and will they return to the Museum? In addition, we wanted to know if the Spanish language materials were useful. The first part was really to help us check our assumption that Frida Kahlo might bring in a lot of first-time visitors. The second was to give us some insight on the ROI of the translation work we did for this exhibition, which was the most extensive we’ve ever done—ticketing pages, signs in the building, floor plan, and labels were all provided in both English and Spanish. While we’ve provided label copy in other languages in the past, we had yet to translate logistical information beyond the floor plan. We have a small team and none of our editors are native Spanish speakers. Best practice requires someone for translation as well as proofreading and layout review. Thankfully we have several fluent Spanish speakers (some of whom are native speakers) who helped in the review process and even translated our last minute signs. (Thank you Stephanie, Rachel, Jaime, and Allison!) Working with Eriksen Translations and our in-house powerhouse, we were able to get the translation completed, but it took a lot of extra time and resources.

We determined that the best approach to getting answers to our research objectives were interviews with Kahlo visitors. While we had visitor time and attention, we also asked about how they heard about the exhibition and how their experiences was. All eight students, myself, and Rachel Lewis (who conducted interviews in Spanish for us, thank you!) conducted interviews in the exhibition shop, which was exit point of the exhibition. We offered 10% shop coupons as an incentive to take the survey.

Results

We interviewed visitors during class time on four Thursday afternoons from about 3:30pm-5:30pm in March-April 2019. In total we completed 259 interviews. Below are the results, as written in the students’ final report (in quotes):

Who came to Frida Kahlo?

“Visitors came from a wide geographic area. Visitors to Frida Kahlo were almost evenly split between tourists and New York City residents. The majority of New York City residents were from Brooklyn, but the show also attracted visitors across the city. Overall, domestic visitors came from 26 states plus Puerto Rico, and international visitors came from 8 different countries.” Side note: the data is skewed since it only accounts for responses during a certain time window: Thursday afternoons. Responses would likely be different on different days. Weekends, for example, likely bring in more locals on their day off.

Visitorship was almost evenly split between locals and tourists, which may be accounted for by the fact that interviews took place Thursday afternoons only.

Did they use the Spanish language materials?

“Visitors responded positively to the inclusion of Spanish language materials, particularly the exhibition labels, whether or not they used them personally, and indicated that continued inclusion of bilingual materials would make them more likely to make a return visit to the museum.

Visitors frequently remarked on the relevance of the Spanish language materials for an exhibition about Frida Kahlo. Many visitors expressed an interest in expanding foreign language offerings in future exhibitions when appropriate, especially for exhibitions featuring artist from non-English speaking communities.”

The majority of visitors interviewed did not use the Spanish language materials.

How did they hear about the exhibition?

“35% of visitors reported learning about Frida Kahlo through word of mouth, although social media posts as well as reviews in major news publications were the next most effective at reaching visitors. On-site signage was also useful for attracting visitors who had not heard about the exhibition prior to their arrival at the museum.”

Word-of-mouth ranked highest among ways people learned about the exhibition.

How was their experience? Will they come back to the Museum?

“No matter where visitors are coming from, Frida Kahlo was the primary motivation for visitors to come to the Brooklyn Museum, whether it was out of personal interest, to visit with a friend or family member, or based on a recommendation or advertisement. The majority of visitors expressed having positive opinions about the exhibition and their visit overall, even when they had complaints about other aspects of their experience.

Visitors had a clear interest in returning to the museum for special exhibitions, especially if the focus is on diverse communities or artists of color. Many visitors stated that they associated the Brooklyn Museum with this type of programming and were more positively disposed towards the institution as a result.”

For 71% of repeat visitors, “Frida Kahlo” specifically and special exhibitions in general are the reason to return to the Museum.

What would visitors like to see more of at the Museum?

“When asked what they would like to see more of at the museum, 10.71% of visitors responded with requests for more activities designed for children. These visitors were overwhelmingly returning visitors, indicating that this is a current gap in the Museum’s programming within the exhibition space.”

Learnings

This evaluation confirmed some of what we already knew:

• Special exhibitions are a big draw for first time and repeat visitors alike.
• Word-of-mouth is important, though it’s sometimes tough to parse out the nuances of this answer. While some people genuinely couldn’t remember exactly where they heard about it, for others there might be more to the story. For example, their friend told them about it either through social media or because that friend saw it on social media. So does that count as word-of-mouth or social media? Or both? More to play with here to get better data to help our marketing team.

And gave us some new insights:

• The majority of visitors felt that the Spanish materials contributed positively to their experience whether they used them or not.
• There’s a desire for more activities for children specifically in the exhibition galleries. As reported: “Of the individuals who responded to what they would like to see more of at the museum, 10.71% indicated a desire for more kid-friendly activities. The kid-friendly advocating visitors are also primarily returning visitors (88.9% who desire kid friendly activities).”

Those interviewed felt that the Spanish materials were positive and useful whether they speak Spanish or not.

The results of this evaluation have been really useful as we think about planning our upcoming exhibitions. In particular the usefulness of languages has come up a few times as we plan for our Pierre Cardin: Future Fashion exhibition opening next month. Do we translate to French, the designer’s native tongue? Ultimately we decided not to go that route, but mostly because we don’t think we’ll have high attendance of Francophiles (since Cardin has his own museum in Paris).

In addition to providing some good information about visitors to Frida Kahlo, the evaluation also gave us good practice as crafting questions and things to think about in terms of how to tease out nuances of how people hear about the exhibitions. As always, it’s a work-in-progress.

Without a unified CRM, the data landscape can look a little bleak.
Stephen McMillan (American, born 1949). Zabriskie Point, 1976, 1976. Aquatint on paper, sheet: 22 1/8 x 29 3/4 in. (56.2 x 75.6 cm). Brooklyn Museum, Gift of ADI Gallery, 77.152.2. © artist or artist’s estate (Photo: Image courtesy of Stephen McMillan, CUR.77.152.2_StephenMcmillan_photograph.jpg)

Despite our lack of CRM, we’ve been working hard to gather data in ways we can. For example, we started a single question survey as part of our admissions process—What brought you to the Museum today?—that our front-of-house team is meant to ask as part of a casual and welcoming conversation. That, along with the zip codes we gather as part of every onsite transaction (a requirement of city funding is to report on zip codes), helps us begin to get some basic information about who is coming and why.

On top of regular focused data points like zip codes and single question surveys, we will be running an annual visitor study. We recently partnered with SightX for our research needs and have launched a baseline visitor study to get a better idea of visitor museum-going behavior and demographics. We haven’t run such a survey consistently for 3 years or more. I’m really looking forward to having that data ongoing. We’ve tried to be thoughtful in not only what questions we pose, but also how we phrase them.

In addition to our own visitor study efforts, one amazing resource we’ve been able to tap to build our data stores is our relationship with Pratt Institute School of Information and their Museums and Digital Culture graduate program in particular. In addition to our ongoing fellowship and Sydney’s recent work on ASK data, I’m able to work with the students I teach (and other professors’ classes) on projects that both give the students real-world experience and help the Museum—a true win-win. This past semester, I taught Audience Research & Evaluation. Through class work, students ran one evaluation related to the Frida Kahlo exhibition and another on our ASK app. I’ll report on the findings of these studies in a future post.

So data gathering is happening, albeit piecemeal sometimes, and we’re trying to smart about knitting it into processes and moments that make sense such as with the admissions process, or a short, three-question survey with our thank-you emails to online ticket holders. What does our data landscape really look like and what can we do with the data we do have?  I’m happy to say, that’s where a dedicated data analyst comes in. Thanks to Bloomberg Philanthropies’ ongoing support, we are able to contract a data analyst (job description forthcoming) to help us begin to make sense of all these data points, with ASK data as a pivotal element in the data landscape. I’m really looking forward to carving out time to get the big picture of all the ways we gather data and streamlining those moments. If you know of any good people, please send them our way. In the meantime, check back as we share the results of our recent visitor studies.

Don’t get me wrong, we still care about use rate and want as many visitors as possible to use ASK, but we are no longer consumed by that metric. We have kept the tactics that work best. For example, our ASK Ambassador program is going strong and still makes the greatest difference: there is a direct correlation between Ambassador staffing and use rate. We’ve seen this repeatedly since launching the Ambassador program in 2017. We’ve also continued to play with engagement via ASK, particularly in relation to major special exhibitions, which—let’s face it—is why a majority of visitors come to the Museum. For the David Bowie is exhibition, we created a special trivia activity that was so popular we could barely keep up. Jessica will share more about that in a future post. She’ll also share about what we are currently doing in relation to Frida Kahlo: Appearances Can Be Deceiving, on view until May 12.

The ASK Ambassadors wear branded t-shirts and hats (hats optional).

In addition to these initiatives, I’m delighted to say we’ve added a Pratt fellow to our team this year who has been focusing on the data. Sydney Stewart is a second year graduate student in the Museums and Digital Culture program in the School of Information at Pratt Institute. (Full-disclosure, I teach for that program and Sydney is one of my stellar students). It’s amazing what we have been able to learn by having one person focus on the data, and I’ve invited Sydney to share her research and results here in the coming weeks.  

When it comes to truly getting into the ASK data, I find we’re constantly bumping up against what happens when you build minimal viable product (MVP) as part of an agile process: short-sightedness. We purposefully weren’t thinking long-term when developing the dashboard (the interface the team uses to answer questions and process chats), only what we needed in the moment. Because we were building MVP,  we didn’t plan for or build ways to access the larger data set. We only created tools for our initial needs, which were basic metrics and ways to share conversations with curators for fact-checking purposes. Now that we are trying to get a better handle on the scope and possibilities of the data, we are having to look into building tools to access it. For now, the Tech team runs reports for us when we know what to ask for. I suppose you could say we’re having to get agile once again by using Sydney’s research path as a way to help us understand what we actually want and need to know from the data set. Unfortunately, that makes it a little difficult for her as there is a delay between her determination of needed data and our ability to give her that data. Fortunately, she’s been more than up to the task, and she’ll share some of her creative workarounds and what she’s been able to do with existing metrics.

Our period of radio silence is over, so stay tuned!

Five star app reviews fueled our increased use rate desire. With all these great reviews, surely more visitors would use the app.

Turns out, not that many more people are using it. After a great deal of testing and improved marketing efforts with insights from an outside evaluator, we have managed to double our use rate from 1% to just over 2% pretty consistently. That’s with a lot of effort on our part: incentives and contests, staff hired specifically to promote the app, and marketing materials including palm cards and object labels. When the stars align and our team is really on fire, we’ve seen over 3%, but that is likely the best we are going to get with what we’ve got. And you know what? Maybe that’s OK.

I’ve been doing a lot of soul searching about this project, about what success should look like, and about the original goals for ASK. And to me, the raw numbers are becoming less important. We created ASK in order to facilitate engagement with art. Do I wish more people used it? Sure. But those that do get something really special out of it. That might just be enough. Interestingly, when you compare ASK to other engagement strategies we’ve offered in the past, it fits within a norm. For example, our audio guides (offered until 2012) and our In Conversation kiosks also netted about 2% use rate.

So where do we go from here?

It’s time to shift our focus away from “more” and towards one of the most unique aspects of ASK: the data. We have over 11,000 chats just waiting to be examined. We have metrics not only on app use, but also most-asked-about artworks, information related to where and how many places people ask us questions, and more. We can determine the kinds of questions people have about works of art, how they think about or look at artworks, how they experience the museum, and more. All of this data would be of interest to researchers of all kinds: educators, art historians, and technologists. It’s certainly of interest to us.

The ASK dashboard tracks certain metrics already, but we have yet to delve into the 11,000+ chats we’ve had with users.

That is where we are headed. ASK is no longer about “more,” but instead about identifying the unique insights we can gain about how visitors view and understand works of art. As Shelley stated in an early post about measuring success: “Three components help us determine the health of the ASK: engagement goals, use rates, and (eventually) institutional knowledge gained from the incoming data.” We’ve nailed the engagement goals. Use rate is the best we can get with the tools at our disposal. It’s time to shift to the final measurement: institutional knowledge.

The next step is to reach out to our colleagues across various departments—curatorial, education, visitor services—and determine what questions they have. We’ll then add those to our list of questions and begin to narrow down a research focus. As we delve into the data, we will be sharing insights along with way. I can’t wait to see what we learn.

One of the questions we’ve had since the beginning of the project was if ASK is appropriate for a mounted kiosk of some kind. We originally thought of this in terms of accessibility—providing kiosks for those who didn’t have smartphones. Most people have smartphones nowadays, and since we now offer texting in addition the app, those that do not have smartphones can text us. We have had a few people with flip phones use the texting option. However, we still wondered if providing a kiosk would encourage ASK use; that if people had a good experience on the kiosk, they might use ASK on their own device. So we decided to test this in our Luce Visible StorageStudy Center.

Luce gives open access to some 2,000 of the many thousands of American objects held in storage. Visitors can use iPads mounted throughout the space to search works by accession number.

Luce has five iPad kiosks for visitors to search the collection online since none of the works have any information beyond an accession number. We loaded ASK on 4 of the 5 iPads (the fifth was being used for a survey) and tried it for a week. During that time we had 14 chats come into the dashboard and learned some important things in the process:

• We need to identify the kiosk function. On the iPad, the app is already open so users don’t have the benefit of seeing the prompts. We need a sign letting people know the iPad is how they can find out information about the works in Luce.
• We need to respond very quickly. Because users are stationary, it feels like a really long time before an answer comes our prompt to put down the phone and look at art doesn’t apply here. We have to make sure to send a response fast, even if it’s a partial answer just to start. This was an initial concern about kiosk-ifying ASK and we were right to be concerned.
• We need to turn off the camera feature. Because the iPads are mounted, the user can’t easily access it anyway. One iPad got stuck on the camera feature somehow and since the iPad case covers the camera button, we had to restart the kiosk.
• We need a way to refresh the conversation. The app stays open, which means that people can read the running conversation from the entire day. We watched several people do just this and it’s a behavior we’ve seen before in other iterations of Q&A kiosks. However, we need to be able to start each day with a fresh conversation, otherwise it goes on too long.
• We need to remove access to emojis. Quite a few silly incoming messaging were in the form on nonsensical emoji streams. Since we tested this right before school started, we think it was probably bored kids. The team’s dashboard can’t read or send emojis anyway, so this is a moot feature.
• We need to be able to hide selected messages from appearing on the iPads. If someone does send nonsense, we want to be able to remove this from view so others aren’t tempted to do the same and so that those who want to read the conversation thread can do so without interruption. Plus it’s simply annoying for the team.

While some visitors using the Luce iPads had genuine questions, like a regular app exchange, others were compelled to send us silly messages. The team dealt with them quite politely, but eventually started ignoring them.

We are working with HFC, our contract developers, to create a special version of the app that has the features we need to be able to install it on a stationary kiosk. We will give it another go. Fingers crossed!

Our texting palm card included the phone number as well as “helpful hints” on what kinds of things to text. Unlike the app (which is geo-fenced), functionally you can text us anytime, but we won’t answer outside of Museum hours. The system will autofire the same “out of office” notification” app users also receive.

Our developers did some backend magic so that the Twilio messages would push to the dashboard, which allows us to keep that single source for incoming messages. Unlike the app, the SMS messages do not utilize location aware, which meant the team was flying somewhat blind in the dashboard. Normally, when a visitor sends us a message, the nearest beacon responds and the dashboard populates with the artworks on view in that gallery. Each artwork has the associated metadata from our collection online as well as “snippets” (question and answer pairs) from previous conversations that have been tagged to the work. SMS messages provided few of these tools to the team, who had to manually search the collection online (or good, old Google) if they didn’t know the work already. However, this isn’t the first time we’ve dealt with this challenge, so going into this pilot we felt pretty confident we could handle it. As expected, response time suffered a bit, but overall the team did really well and lack of location data didn’t hinder them much at all. One happy discovery was that the image recognition the developers put into place last year occasionally worked with MMS message, so if a user sent us an image (many do), there was a chance the dashboard would find it and pull the metadata as well.

I have to say, out of all the pilots, I thought this would be the most successful, and I was right. Visitors really responded well to the idea of texting, and we ended up having to reorder palm cards twice. Despite this enthusiasm, use rate was not what I would like to have seen; we averaged just above 2%. Interestingly, not all of that traffic was from texting. We still had some iOS and Android users, only 19 of whom were repeat users, so some folks were finding and using the app despite our promotion of texting (and likely not all international visitors). Out of curiosity, I wanted to compare this most successful pilot to our most successful two non-pilot weeks. Turns out the average use rate of those two weeks is higher than any of the pilots. So alas, while this pilot had the best use rate of the three, it did not hint at a “solution” to our use rate plateau and was not better than our best two (non-pilot) weeks combined.

By way of quick recap: pilot 1 (provided devices) proved charging for devices doesn’t work, while free devices didn’t give us more app traffic; pilot 2 (ASK on Demand) showed that while people liked the idea of an in-person chat, few took us up on it; and pilot 3 (texting) did not show better numbers than really successful non-pilot weeks.

What’s next? I’m not 100% sure, but we’re kicking around a few ideas. We just had a big team meeting with the ASK team and Ambassadors to discuss these metrics and share observations, which I’m still chewing over. Some things we are exploring include providing devices for scheduled groups and keeping the texting service as an alternative for those that just don’t want to download.

It’s going to take a little while to figure out, but I promise more to come after vacation (yay!) and time to think. 

Over the course of ASK, we have seen distinct patterns in the ways people engage with the app and the ASK team: they ask us questions, seek more information, and share their opinions. Visitors have also responded very well to opportunities to meet the ASK team in person during  pop-up tours, Art History Happy Hour events, or when team members are stationed in the galleries, which they occasionally are. Users also enjoyed the chance to meet the ASK team when their office space was in a public space, though not enough to keep them there for good. Knowing all this, we began to wonder, what if we updated the user experience of ASK to reflect these aspects that visitors enjoy and provide visitors clearer choice within their experience?

While still providing the opportunity to chat via text, the core of the ASK experience, could we also offer FAQs, most-asked about objects, or a “surprise me” feature that provides bite-sized content? This would help address the reported pressure some visitors feel about having to ask a question. In addition to texting and FAQs, could we offer the ultimate personal experience by offering visitors the opportunity to have conversation on-demand, in-person with one of the ASK team members? Functioning something like an airplane “call button,” this option would allow visitors who prefer an in-person conversation or are just really enjoying their text conversation to request the ASK team member to join them in the gallery. Should this concept work, we could play with incentives like only surfacing the “call button” after a certain number of exchanges or galleries visited or we could offer the option faster for repeat users. There are lots of possibilities to explore here.

This kind of approach would require changes to the app functionality and design, which we’re prepared to tackle, but only if we could prove some of the basic concepts valid. In particular, I wanted to confirm people would take us up on the in-person request component. We created a dedicated palm card for this pilot highlighting the various ways people might engage with us and the ASK Ambassadors emphasized the in-person option in the pitches.

Dedicated palm cards highlight the in-person concept. The “helpful hints” simultaneously provide instructions as well as concrete suggestions on to how to use the app based on needs and interests.

I’m somewhat surprised to report that over the course of the two weeks, our app traffic was right within the normal range and only six people took us up on the offer of an in-person appearance. One of the things I was curious about was how many people would bypass texting altogether and just request in-person time. Timing of the request really varied. Two of the five requested a team member immediately, one (a family group) ended their very engaged time via the app with an in-person request, one refused to download the app and the ASK Ambassador requested a team member on their behalf, and two took the ASK team member up on her offer to join them one or two messages into the conversation.

Our ASK Ambassadors managed to snap two quick photos of visitor interactions during this pilot. On the right, Isabella speaks with a visitor in the O’Keeffe exhibition. On the left, Rachel speaks with visitors in “We Wanted a Revolution.”

I will say that I think one reason for the limited uptake on the in-person interaction was our over-zealous guarding of the ASK team’s time. We were so worried that the team member would be unable to extract herself from a very interested visitor (this has happened often on tours) or be expected to present some manner of tour, that we asked the Ambassadors to really stress that it was an opportunity to “say hello.” I understand from the Ambassadors, who did a great job executing exactly what I asked, would often preface the opportunity with some kind of explanation that the team might be busy answering questions via the app and would only be free for a minute. In retrospect, I’m sure this made the in-person request feel like a total imposition. I know if I were a visitor, I at least would have hesitated before requesting someone if it were presented to me in such a way.

Again, I come away from this pilot without any true conclusions except that we might want to revisit it, but with less protective language around the team’s time and a more simple invitation to have someone join them in the gallery for a bit. August might be our month to take the learnings from running these pilots try them again. So far, the only thing we’ve been able to definitively say based on the pilots is that charging visitors for iPods loaded with the app won’t work. I suppose that’s something!