It’s Not About “More” Anymore

For the majority of this project, we have been fixated on use rate. After all, it’s easy to track and is a very clear measurement of success: just how many people use this thing? With soft launch, we saw about 1% use rate and we automatically focused on the need to increase that. We felt that we were offering such a great experience, that all we had to do was figure out the right way to explain it to people and they would naturally want to use it. We’ve spent almost two years working on increasing that use rate. During that time, engagement via ASK has only improved. Our average number of exchanges between user and ASK team increased from 11.9 to 14. Our app store reviews continued to be stellar with users remarking on how helpful the team was (often citing the team member by name), how much their conversation opened their eyes or changed their experience in a personal way. This reaction only fueled our desire to get ASK in more people’s hands. Surely more people want to use it!

Five star app reviews fueled our increased use rate desire. With all these great reviews, surely more visitors would use the app.

Turns out, not that many more people are using it. After a great deal of testing and improved marketing efforts with insights from an outside evaluator, we have managed to double our use rate from 1% to just over 2% pretty consistently. That’s with a lot of effort on our part: incentives and contests, staff hired specifically to promote the app, and marketing materials including palm cards and object labels. When the stars align and our team is really on fire, we’ve seen over 3%, but that is likely the best we are going to get with what we’ve got. And you know what? Maybe that’s OK.

I’ve been doing a lot of soul searching about this project, about what success should look like, and about the original goals for ASK. And to me, the raw numbers are becoming less important. We created ASK in order to facilitate engagement with art. Do I wish more people used it? Sure. But those that do get something really special out of it. That might just be enough. Interestingly, when you compare ASK to other engagement strategies we’ve offered in the past, it fits within a norm. For example, our audio guides (offered until 2012) and our In Conversation kiosks also netted about 2% use rate.

So where do we go from here?

It’s time to shift our focus away from “more” and towards one of the most unique aspects of ASK: the data. We have over 11,000 chats just waiting to be examined. We have metrics not only on app use, but also most-asked-about artworks, information related to where and how many places people ask us questions, and more. We can determine the kinds of questions people have about works of art, how they think about or look at artworks, how they experience the museum, and more. All of this data would be of interest to researchers of all kinds: educators, art historians, and technologists. It’s certainly of interest to us.

The ASK dashboard tracks certain metrics already, but we have yet to delve into the 11,000+ chats we’ve had with users.

That is where we are headed. ASK is no longer about “more,” but instead about identifying the unique insights we can gain about how visitors view and understand works of art. As Shelley stated in an early post about measuring success: “Three components help us determine the health of the ASK: engagement goals, use rates, and (eventually) institutional knowledge gained from the incoming data.” We’ve nailed the engagement goals. Use rate is the best we can get with the tools at our disposal. It’s time to shift to the final measurement: institutional knowledge.

The next step is to reach out to our colleagues across various departments—curatorial, education, visitor services—and determine what questions they have. We’ll then add those to our list of questions and begin to narrow down a research focus. As we delve into the data, we will be sharing insights along with way. I can’t wait to see what we learn.