Gaming Click!??!?
Yikes! This week I wanted to take a moment and look at some rather amusing things (or scary things, depending on your perspective) that happened during the evaluation process. We stayed mum on this during the eval period to avoid bringing more attention to it, but it’s worth looking at now while revisiting chapter three of The Wisdom of Crowds.
In chapter three, Surowiecki discusses information cascades, the effect of one person making a decision and others following suit. He writes, “The fundamental problem with an information cascade is that after a certain point it becomes rational for people to stop paying attention to their own knowledge—their private information—and to start looking instead at the actions of others and imitate them.” A bit later in the chapter he concludes with, “Collective decisions are most likely to be good ones when they’re made by people with diverse opinions reaching independent conclusions, relying primarily on their private information.” Avoiding a cascade was the primary reason for suppressing comments and direct links in the evaluation tool, but, of course, we couldn’t think of every situation that might present itself. It’s important to keep in mind that there were very few instances of this, but the fact I didn’t see this coming makes me wonder about my optimistic state of mind…
The evaluation was being conducted on the web and we were specifically asking for people to help us spread the word. Some of the posts, like this one, were awesome – here’s an artist who’s spreading the word using our widget which minimizes influence by randomizing images. But, then we noticed posts like this and this popping up (even more here) and, well, I felt a little daft for not instituting a specific rule about it. But, hey, a big part of this project is on the ‘net and who wants to post to their blog without a little self-promo? So, I get it. The reality is these posts are fascinating to read and provide a great opportunity to see the process from the point of view of some of the artists – yay! And it’s really awesome to come across a post like this, where the artist says, “i can’t/won’t tell which image i selected, as i believe it’s against the rules.”—well, it still makes me smile.
All this self-promotion begs the question…did it do any good? We don’t think so. Evaluation was not a quick and easy process and you can see how this type of implementation starts to help when issues like this crop up. We carefully looked at the data to see how much of a difference this kind of self-promotion may have made. Luckily for us, we had enough participation (data) to counteract any skewing and the results were fairly balanced (thank you, again). Of course, this could have easily changed had any one instances above (or similar) generated a lot of traffic with very determined users.
Have fun exploring some of these posts, it will be even more interesting to reference them when you see the results on June 27. Next week we’ve invited a special guest blogger to post about the subject matter in the submitted works…more soon.
