Getting Visibility on the iBeacon Problem

It’s been just over a year since I wrote about the realities of installing ibeacon to scale. Our ASK app, funded by Bloomberg Philanthropies, has been active for the past year and the beacon-powered solution that sends a visitor’s location to the ASK team continues to be difficult; this post will help quantify some of the issues we have been seeing.

We started to look deeper into our beacon difficulties roughly six months ago because the project had been on the floor long enough to give us patterns in the data. The timing was critical, too, because we were about to begin Android development; working to solve the problems now would help make that deploy more straightforward. To get going quickly, we implemented a manual tracking system using Survey Monkey. The ASK team would log each time they ran into an issue with the location information in an ASK chat. This was not an easy thing to juggle because mid-chat they’d need to fill out a long and detailed form about what was happening and where, but it showed us two very clear trends. Problem messages were either coming in with “no location found” or nearby signal bleed was causing an incorrect location to be sent to the team.

If you thought the manual tracking system was labor intensive what comes next may boggle your mind. As we started to troubleshoot beacon issues, we wanted a clean slate. This meant updating the firmware on all the beacons, checking the battery life, and turning off the advanced power settings that Estimote provides. This was a painstakingly manual process where I’d have to go and update each unit one-by-one. In some cases, I’d use Estimote’s cloud tool to pre-select certain actions, but I’d still have to walk to each unit to execute the changes and use of the tool hardly made things faster. More frustrating was the process to update a unit’s firmware because the phone has to be in close physical proximity to the beacon you are trying to update; this meant carting a ladder around all day in the galleries because all of our beacons are installed high on the wall.

Our solve for this—and I can’t claim credit for coming up with this one because it was our Head of IT, Christina, who drummed this up—was to use a selfie stick to circumvent the need for a ladder. In the end, it took 11 staff hours to make the needed updates/changes to the roughly 150 beacons installed in the Museum. There is some good news. After a year on the floor, the batteries are still holding strong with most units reporting 36+ months of battery life remaining.

With two clear issues on the table, we started down a technical path toward solving them. The web development team pulled data to see if we could quantify how big these issues were. Turns out, “no location found” was overwhelming larger than the “incorrect location” problem; roughly 15% of messages coming into ASK didn’t have a location. This was puzzling because if you walked around with the Estimote app ranging for beacons, it would show that the chance for a dead spot was unlikely; if anything we have too many beacons creating too much noise (as evidenced by the incorrect location problem). Diving in a bit more, the data was showing us that within the 15% of messages with no location data, 26% had no location on a user’s first message in a chat. This pointed to a problem with the beacon detection code in our ASK app.

While this is about the last thing I wanted to hear, it was a darn good time to figure it out. With Android development on the horizon, we decided to evaluate the iOS code to try and figure out why this was happening, but fix the issue in the Android code. Then we could go back after learning those lessons to refactor how the iOS code is working. I’m going to expand on this at length—code release included—in my next post.

For now, let’s move on to the issue not affected by code—the incorrect location problem. Beacon placement in a situation like this one is challenging—we’ve got an older and complicated building with walls of all different material types, special exhibitions that change the physical spaces constantly, object cases that affect signal, and even things like attendance can cause fluctuations in signal that are not expected. To get a handle on signal bleed, we needed two things: a way quantify where we had problems and a way the ASK team could work around them because, no matter what, things may get better with bleed, but that problem is always going to be there with this technology.

The solve for the incorrect location problem gives the ASK team a tool to work around the issue while leveraging crowdsourcing to diagnose where we have the problem. In the ASK dashboard, the ASK team has a way to toggle the location. If it’s clear that an incorrect location has been sent, they can use the next/previous arrows to see the objects in adjacent locations; I have the ability to set the adjacencies in an admin tool. As an example, I can define that beacon group 59 (European) is adjacent to beacon group 34 (Assyrian) and 29 (Egyptian); if a message gets sent from European, but the visitor is clearly asking about an Egyptian object, the team can use the toggle to get to the adjacent Egyptian objects.

The toggle helps the ASK team work around the bleed in the most immediate sense, but the toggle is also helping us discover our pain points. The web development team is now tracking clicks on the toggle and a report in our admin tool displays toggle clicks and where they land. This then helps me go into the galleries and adjust the transmit rates and/or placement of beacons to lessen the instances of the problem.

Speaking of, when I go into the galleries and adjust the transmit rates here are the general rules of thumb that I’m using; keep in mind, for ASK all we care about is knowing the room a visitor is standing in and we don’t need more granularity. Larger, cavernous spaces with fixed boundaries and little chance of bleed—our lobby and west wing special exhibition galleries—get fewer beacons with higher transmit rates for better coverage. Smaller galleries closer together—most of our permanent collections—tend to get more beacons with lower transmit rates; this helps us get the coverage we need while keeping the bleed at a minimum.