ASK 2.0: Providing Devices? Maybe.

As I prefaced in my post last week, while ASK has been successful from an engagement standpoint, we are stalled at between 1-2% use rate. We’ve learned through evaluation, both our own and that conducted by ERm in December 2016, that the biggest barrier to more visitors using ASK Brooklyn Museum is reluctance to download an app. What is more, ERm’s evaluation revealed that although most study participants (57%) felt that the app would enhance their museum visit, many (44%) still would choose not download it! Seriously? Argh. Clearly the format of the current ASK Brooklyn Museum experience is the barrier to adoption. Simply put, people are reluctant to download an app and our recent additional marketing efforts, while they helped us break the previous 1% average, have not provided the final solution to this issue.

We are running a series of three pilot projects that will help us determine the direction of the next two years of the program, which we’re calling ASK 2.0. This is the first of (at least) three posts on the results.

We began our pilots with the low-hanging fruit: provided devices. If downloading is a barrier, don’t make people download anything. Providing devices would also eliminate the related “excuses” we hear from visitors: not having enough storage, data, or battery life. Of course, it’s not quite as simple as just handing someone a device. We had to make sure we’d get it back, which means there had to be some kind of check out process. This is its own kind of barrier, albeit a familiar one for anyone who has ever rented an audio guide. Would a check-out process prevent people from using it? I also wondered if people would be willing to potentially juggle two devices—our iPod Touch and their phone—because despite some focus group and survey participants proclaiming that they don’t like to use their phones during a museum visit, 76% of those same survey respondents admitted that they use their phones at some point during their visit (to take photos, text, use social media, Google something, etc.). Finally, I wondered about the perception of value. What would happen if we put a dollar amount on the experience and charged for the devices? Would it turn people away? Increase download rates? Encourage people to chat more to get their money’s worth? I had to know.

Seems fitting to return to iPod touches for this pilot since that’s what we used for our 2014 pilot that ultimately led us to build ASK. Here we set them up with lanyards and protective cases. We enabled “guided access mode” to lock them down on the app only.

We ran the test for two weeks and included the A/B test around value. For the first week, the device (iPod touches) were available to check-out for free. For the second week, the iPods cost $5. I have to say, the results surprised me a bit.

For the first (free) week, we checked out out 42 devices, with 30 chats coming from those devices. So not everyone who checked out a device used it (not the surprise). The ASK Ambassadors asked visitors returning devices what they thought about their experience and for most people that didn’t use it, they admitted they didn’t “need” it because the museum was well-curated and explained. I suppose that’s hard to argue with! One surprising result we saw were that the iPods averaged more than double the messages of all users that week. The average number of exchanges for the week was 11.9, but for the iPods specifically it was 24.3! Now this could partially be explained by the fact that several kids used the iPods, and kids tends to send a flurry of photos and comments, so there is a lot of back-and-forth in a short time, but it’s something to think about. Another surprising (and disappointing) thing was that providing devices didn’t net us any more chats than our usual weekly average. So while 30 people used it, that wasn’t 30 more people than usually use it. Hmm.

Alex and Kahlah help a visitor check out an iPod during the free week. We used the Info Desk in the lobby as home base for storing visitor IDs collected as collateral and for storing and charging the iPods.

The second week of the test ($5 rental fee) didn’t work at all. We only had one visitor rent an iPod, and it was for her kid. The engagement numbers don’t mean much with one sample (though it was really high at 36 exchanges). This was also a slow week for us, under the average number of chats for the week. This may have nothing to do with the iPods and perhaps more to do with a few of our ASK Ambassadors being out sick, but hard to know. We did see an uptick in download rate, however, and the ASK Ambassadors told me that it was an easy “sell” to get people to download for free as opposed to renting an iPod. Unfortunately, those downloads did not translate to app use.

So what does this all mean? Honestly, I think it means we need to revisit providing devices for free again. We’ll work this into the plan for August, unless one of our remaining two pilots is wildly successful. So far the second pilot, which I’ve named “ASK on Demand,” is showing some promise. More to come next week!