The finalists of the Scientific Blogging Competition were announced last Sunday. I was delighted to find my essay among them:
Yay! Honor and delight! Now, how are they going to select grand, second, and third prize winners from these thirty finalists? Public online voting:
Now, here are five things that make me feel weird about this contest:
1. The contestant pool. When I described the competition to a labmate and mentioned that it was only open to grad students from the ton ten (eleven) universities in the country, he immediately responded, "Well that's kind of elitist." Scientific Blogging has responded to similar accusations in their comment threads:
More than anything, this got me thinking about the original US News & World Report study. Who do they think they are, ranking the best science schools in the country? "Questionnaires were sent to the department heads and directors of graduate studies at each program in each discipline." I would sure love to know what kind of rankings they'd get if they sent questionnaires to the grad students instead . . .
2. I don't like asking people to "go vote for me" instead of "go read all the essays and vote for the one you think is best, and if that's really mine, then thank you for your praise." But Rule 3 of the contest is: "Encourage friends, family, and faculty to vote for you." So I'm following instructions. Selfishly.
3. Online voting makes me nervous. It's just too easy to game the system. How do you identify voters? As originally stated, the plan was that only registered users could vote. However, when they announced the winners, they announced new rules: voting is now open to anyone, registered or not. But they've got to track voters somehow--I'm guessing by IP address, a standard but imperfect practice. It allows people to vote as many times as they have network connections (at least one contestant has stated explictly in their article comments: "one vote per computer per day") and I rather wish they'd stayed with registered users only. True, there's nothing to prevent you from registering multiple accounts if you have multiple e-mail addresses, but it takes a lot more work.
4. And why does the voting go on for 22 days? I can certainly see the value of a long voting period in terms of giving voters time to hear about the contest, read all the essays, and decide on their favorite. But why give each voter (or rather, each network connection) 22 votes instead of one? And if you really want to give out multiple votes, still, is there any reason not to let people allocate them all in one day?
As I've talked about this system with other people, it's become obvious that this is the online voting paradigm. Upon reflection, it makes sense. These competitions are a great way for websites to drum up readership (which translates directly into ad revenue)*. From this perspective, you want to make people vote every day for as long as possible, so all the contestants' friends and relations keep coming back to your site, day after day. Not only does this create a big temporary boost in readership, but with sufficient exposure, some newbies might decide they like the site enough to stick around after the competition. Or maybe they keep coming back out of habit. In any case, win!
5. So, is this a networking competition or a writing competition? Not surprisingly, several comments on the website have addressed the idea that open voting to determine the winners is "unfair". After all, who's to say the "plebes" have the discriminative ability to pick the best essay? Scientific Blogging's Kim responded:
In answer to this question, SciBlogging's Hank argued:
Sure, the site gets a million readers a month. Do they visit the site every day, though, or just once a month? And however often they visit the site, do they know about the contest? The article announcing the finalists was one of the five featured articles on the front page for a few days, but it's no longer there. It wasn't mentioned in the e-newsletter at all. And even if you happened to visit the article, you had to read half-way down before you learned how to find the finalist essays (by clicking on the red CONTEST button). Maybe that should be obvious, but that red button had been there for the last two months, before finalists were chosen and voting opened, so unless you'd been keeping pretty close tabs on things, there's no reason you'd assume you could click on it and vote now. And even if every one of those million readers know about the contest, do they care enough to vote? Do they care enough to vote every day?
All these concerns are trivial, perhaps, but in my mind, they add up to the hypothesis that the vast majority of votes are coming not from the permanent sciblogging readership, but from friend mobs, voters visiting the site to read one article--their friend's--and vote for it. So, the more friends you have, and the more frequently you remind, cajole, and nag them, the more votes you'll get.
I realize this could all sound pretty negative and critical. The thing is, I like Scientific Blogging. I like Hank and Kim, and all the other bloggers there that I've been reading. Over the last month or two that I've been blogging with them, I got more and more excited about the idea of winning the internship and working with them.
But over the last week of voting, I've realized that this phase of the competition makes me uncomfortable. It's not that I think networking is an evil skill. If I were trying to drum up votes or donations for a larger cause (like Sasha does for her non-profit SOIL) then I would probably be more comfortable sending out lots of e-mails. In this case, though, the cause at stake is science communication. I'm passionate about it, but honestly, any one of the finalists is going to do a fine job. My desire to win is therefore purely selfish, and I don't want to harass people for a selfish cause. Besides, I've never had a very competitive spirit.
This experience might just be teaching me that I don't belong in internet competitions. And that's a useful thing to know about myself.
* Just to be clear: I'm not saying that drumming up readership is the only, or even the primary, motivation for holding a contest like this. Also, I'm not saying that it is a bad motivation. I am 100% in favor of good websites (like sciblogging) building up both their readership and their revenue so they can stick around and keep serving up good content.
Autumn has arrived, bringing firework foliage, delicious squash, and, at least in the Pacific Northwest, an invasion of squid.
Humboldt or jumbo squid, sometimes mistakenly called giant squid, are grabbing fishing lures and washing up on beaches from Oregon to British Columbia. As a marine biologist fielding questions from reporters and citizens, my heart always sinks when I hear the inevitable query--delivered with a mixture of horror and fascination--"They eat people, right?" . . .
Yay! Honor and delight! Now, how are they going to select grand, second, and third prize winners from these thirty finalists? Public online voting:
Now it's the readers turn to tell us who their favorites are. The red CONTEST tab now lists all of our finalists. If you wish to vote for an article, click on the article of your choice, and then click the "VOTE" widget that comes up with that article. You can vote for an article only once per day, but you may vote for more than one article each day if you have several favorites. And be sure to come back the next day to vote again if you want to help your favorite finalists win.So, go vote for my essay! Okay?
Individuals do NOT need to be registered members of Scientific Blogging in order to vote. However, voting will close at Midnight Pacific Time on Sunday, November 22nd. So get your votes in while you can, and then stay tuned for the announcement of our winners on December 1st.
Now, here are five things that make me feel weird about this contest:
1. The contestant pool. When I described the competition to a labmate and mentioned that it was only open to grad students from the ton ten (eleven) universities in the country, he immediately responded, "Well that's kind of elitist." Scientific Blogging has responded to similar accusations in their comment threads:
We have to keep the pool of writers small. If it goes off without too much of a hassle, we will open up the next one to all schools but we have a limited amount of people who can read the papers and help pare them down to a manageable number. . . . Obviously we're the only science site that is open to everyone so we aren't elitist, we are just making this first one manageable.A site that's open to everyone isn't necessarily immune to accusations of elitism, but I certainly sympathize with the need to keep the applicant pool small, having read and graded my fair share of essays.
More than anything, this got me thinking about the original US News & World Report study. Who do they think they are, ranking the best science schools in the country? "Questionnaires were sent to the department heads and directors of graduate studies at each program in each discipline." I would sure love to know what kind of rankings they'd get if they sent questionnaires to the grad students instead . . .
2. I don't like asking people to "go vote for me" instead of "go read all the essays and vote for the one you think is best, and if that's really mine, then thank you for your praise." But Rule 3 of the contest is: "Encourage friends, family, and faculty to vote for you." So I'm following instructions. Selfishly.
3. Online voting makes me nervous. It's just too easy to game the system. How do you identify voters? As originally stated, the plan was that only registered users could vote. However, when they announced the winners, they announced new rules: voting is now open to anyone, registered or not. But they've got to track voters somehow--I'm guessing by IP address, a standard but imperfect practice. It allows people to vote as many times as they have network connections (at least one contestant has stated explictly in their article comments: "one vote per computer per day") and I rather wish they'd stayed with registered users only. True, there's nothing to prevent you from registering multiple accounts if you have multiple e-mail addresses, but it takes a lot more work.
4. And why does the voting go on for 22 days? I can certainly see the value of a long voting period in terms of giving voters time to hear about the contest, read all the essays, and decide on their favorite. But why give each voter (or rather, each network connection) 22 votes instead of one? And if you really want to give out multiple votes, still, is there any reason not to let people allocate them all in one day?
As I've talked about this system with other people, it's become obvious that this is the online voting paradigm. Upon reflection, it makes sense. These competitions are a great way for websites to drum up readership (which translates directly into ad revenue)*. From this perspective, you want to make people vote every day for as long as possible, so all the contestants' friends and relations keep coming back to your site, day after day. Not only does this create a big temporary boost in readership, but with sufficient exposure, some newbies might decide they like the site enough to stick around after the competition. Or maybe they keep coming back out of habit. In any case, win!
5. So, is this a networking competition or a writing competition? Not surprisingly, several comments on the website have addressed the idea that open voting to determine the winners is "unfair". After all, who's to say the "plebes" have the discriminative ability to pick the best essay? Scientific Blogging's Kim responded:
The voting is to determine the writers that not just "scientists" or science professionals find interesting, but that the general public find interesting as well. We are looking for writers that can cast a broad net, and make even those that "are not even interested in science" want to stop, and read, and potentially learn something new from one of our posts.Hear hear! I fully agree with this definition of good science writing. But the big question is: are the people casting votes actually "stopping, reading, and learning something" or are they just taking two seconds to click a button? In other words, is the electorate actually made up of the "general public"? Or is it made up of the contestants' "friend mobs", who haven't actually read all the essays and may not have even read their friend's, but are nonetheless voting loyally and daily?
In answer to this question, SciBlogging's Hank argued:
There's nothing unconventional about letting the smartest audience on the planet pick the winner of a writing competition. The notion that some vague nepotism is somehow going to overrule a million people a month who read this site is not realistic.Is it all that unrealistc? I'm not sure. Nepotism isn't the right word (commentor's word choice btw, not Hank's) but personal popularity (as opposed to writing popularity) or networking ability just might be.
Sure, the site gets a million readers a month. Do they visit the site every day, though, or just once a month? And however often they visit the site, do they know about the contest? The article announcing the finalists was one of the five featured articles on the front page for a few days, but it's no longer there. It wasn't mentioned in the e-newsletter at all. And even if you happened to visit the article, you had to read half-way down before you learned how to find the finalist essays (by clicking on the red CONTEST button). Maybe that should be obvious, but that red button had been there for the last two months, before finalists were chosen and voting opened, so unless you'd been keeping pretty close tabs on things, there's no reason you'd assume you could click on it and vote now. And even if every one of those million readers know about the contest, do they care enough to vote? Do they care enough to vote every day?
All these concerns are trivial, perhaps, but in my mind, they add up to the hypothesis that the vast majority of votes are coming not from the permanent sciblogging readership, but from friend mobs, voters visiting the site to read one article--their friend's--and vote for it. So, the more friends you have, and the more frequently you remind, cajole, and nag them, the more votes you'll get.
I realize this could all sound pretty negative and critical. The thing is, I like Scientific Blogging. I like Hank and Kim, and all the other bloggers there that I've been reading. Over the last month or two that I've been blogging with them, I got more and more excited about the idea of winning the internship and working with them.
But over the last week of voting, I've realized that this phase of the competition makes me uncomfortable. It's not that I think networking is an evil skill. If I were trying to drum up votes or donations for a larger cause (like Sasha does for her non-profit SOIL) then I would probably be more comfortable sending out lots of e-mails. In this case, though, the cause at stake is science communication. I'm passionate about it, but honestly, any one of the finalists is going to do a fine job. My desire to win is therefore purely selfish, and I don't want to harass people for a selfish cause. Besides, I've never had a very competitive spirit.
This experience might just be teaching me that I don't belong in internet competitions. And that's a useful thing to know about myself.
* Just to be clear: I'm not saying that drumming up readership is the only, or even the primary, motivation for holding a contest like this. Also, I'm not saying that it is a bad motivation. I am 100% in favor of good websites (like sciblogging) building up both their readership and their revenue so they can stick around and keep serving up good content.
I'm amused by your concern on point number 3, knowing you, your husband, and your circle of friends. If anyone wanted to game the anonymous voting system, you know the people to make it happen. Good luck!
Megan and I had some fun speculating about how one might conceivably be able to implement an online voting system that would actually only allow each person to vote once. It doesn't seem a priori impossible, but so far it seems like much more energy has gone into fancy ways to verify that people are who they say they are online, but hardly any into ways to verify that no one says they are more than one person.
Of course, as you point out, "one man, 22 votes" may be perfectly in line with sciblogging's goals, but it's an interesting thing to think about.
Sure it is. It happens all the time.
As I was saying on IM the other day, my boss's fiance (then girlfriend) entered a nationwide, radio-station-and-beer-brewer-sponsored competition where the winning band would get to tour with the country's biggest summer rock festival. Even with all that commercial exposure, the race came down to the two bands that were best at exhorting their friends, message-board readers and boyfriend's employees to vote every day.
@Luke--Yeah, it's just disconcerting to have those people ask me "How badly do you want to win?" with a slightly maniacal glint in their eyes.
@Mike--It is a fun thought exercise, isn't it? We can continue brainstorming at TG with the whole family...
@Charles--Thanks much! I had the feeling there'd be some really obvious citation to make that point.
Hi Danna, this is Patrick, one of the developers at ScientificBlogging. I noticed your link in one of the comments on the contest-announcement articles, so I thought I'd drop by and say hello.
I'm currently working on the contest myself, and I suppose I want to reassure you that I personally have been sifting through just about every single vote that was submitted during the competition and making sure that it's valid and not the result of a trolling attempt like Charles pointed out. Hank's taking this verification very seriously, and he wants to make sure that only the best writers win, not those that get their hacker friends to autovote for them.
(Note: this is why the contest winners haven't been announced yet, even though it's Dec 1st. Verification is taking a bit longer than we thought to do. Shifty hacker-types and their.. hacker ways. :P)
@Mike: if you had more time than I had to implement all this, and more talented developers than I (not to mention if HTTP was a slightly more-sane protocol), then maybe you could develop a more secure system.
@Danna, keep up the good work, really. Squid-a-Day is an interesting blog, and I like reading this blog because I so rarely get a look into what our users think about the site in a well-thought-out and elucidated manner. Keep the feedback coming, and we'll be doing our best to make the site better for people like you. :)
Anyways, I've probably ranted on for too long, but I thought your analysis of our competition was so good that I felt like ranting here. Sorry if it's unwelcome; delete it if you wish.
Tl;dr: Contest winner announcements are coming soon; I think I've got the winners pinned down, I just need verification from Hank and Kim before we actually put anything up.
So long, and thanks for all the feedback!
Patrick
Developer, ScientificBlogging.com
Hi Patrick. I really appreciate you stopping by--I was hoping to stimulate discussion! I'm glad to hear that you and Hank took vote verification seriously, although I never thought otherwise. I question not the intention, but the implementation.
I'm curious what "personally sifting through every vote" entails? Were you looking for unique IP addresses? If you disregard multiple votes from the same IP address, then you only allow one vote per firewall, although many people may be voting from behind the same firewall. I mentioned this problem in a chat with Hank and received no response, so I would be very interested to hear your take on it. Perhaps you've found a way around it. Or perhaps you decided to accept that consequence and say: tough luck. One vote per firewall it is. I would certainly understand the need to make a decision like that; I won't get out the pitchforks and tar! I would just like to know.
It may be that you and Hank are not comfortable discussing the details of "sifting votes" because you think that might compromise the security of future contests and votes. Obscurity is a weak, but not totally ineffective, form of security, so I personally think you'd be better off being transparent about the voting process so you can get criticism and improvement from "the internet's smartest readers".
But the most important point I tried to make is #5. Even if you secured the system perfectly against autovoting, that wouldn't change my suspicion that "the vast majority of votes are coming not from the permanent sciblogging readership, but from friend mobs, voters visiting the site to read one article--their friend's--and vote for it." Networking isn't hacking--but it isn't writing, either. Without supporting evidence (which I would love to see, if you have it), it seems naive to pretend that the winners were the best writers, not merely the best networkers.
Please rant back at me. I find these topics interesting, and would very much appreciate a continued dialogue.
And, I'm very glad to hear you've been enjoying my writing!
Hey Danna, it's Patrick again.
"Personally sifting through every vote" entails looking at each of the tens of thousands of votes we got over the voting period and looking for a few criteria:
Since we can only collect so much data on someone's vote (namely, their IP address, the time of vote submission, whether they're logged-in, and the referring URL), we decided that those criteria were the only ways that we could filter out bogus votes (and there were a lot of them).
In short, yeah, we decided to say "tough luck". As far as I know, those are the only real validation methods we can use for anonymous voting, and it's inevitable that someone's real vote is going to get lost in the noise.
As for #5, which I forgot to mention in my last post, you are absolutely right. I wish I had evidence to support the idea that the best writers were chosen for their writing skill, but I really don't. 99% of the votes we got were just people voting for their friends (I can't say whether they actually read the articles in question, since that's not a statistic that I can collect in any meaningful way).
The unfortunate thing is that a contest like ours requires time and dedication to reading and analyzing different articles and coming to an objective conclusion about which one is written best, and therefore deserves higher praise. In the spirit of what we're trying to do, you can't just go to the page, click the vote button, and leave.
The problem is, in my opinion, that asking people of any nature (be they the world's greatest scientists or not) to take more time out of their days to read a bunch of articles that they're probably not very interested in in the first place is nearly impossible, and asking them to honestly and conclusively evaluate which one is better is a feat I think even God couldn't accomplish, nowadays.
I think we got lucky in this competition; I actually did read all of the articles that were submitted and I think the ones that came out in the end were damn fine ones. The best? Maybe not. But at least, they were consistently good without people having to cheat to upvote them, and that must count for something. :)
Something tells me I should resurrect my old column on ScientificBlogging to talk about subjects like this. I'll have to talk to Hank about that when I see him next.
Well, I think that's enough for me. I hope I haven't ranted too much, but I do think that transparency is a good idea. At the very least, it might give me ideas on how to make the system better for next time. :)
Patrick
Developer, ScientificBlogging.com
P.S.: By the way, congratulations on winning the camera prize! I know it's not exactly the best of prizes, but hopefully it will help you give us better pictures and videos of squid for your articles, because that would be pretty cool.
Hey Patrick! Thanks again for the time you've taken to respond thoughtfully and thoroughly.
Just out of curiosity, why did you decide to change the rules from the original "only registered users can vote" to allow anonymous voting? Sticking with registered users would have saved a lot of vote-sifting trouble.
In the end, it's the nature of the online voting beast to turn into a popularity contest. I'm glad to hear you recognize that "99% of the votes we got were just people voting for their friends" although it conflicts with the fact that "in the spirit of what we're trying to do, you can't just go to the page, click the vote button, and leave."
If you're going to have more of these competitions, I think there will have to be a very conscious decision whether you want the prize to go to the best writing or the best networking. Because you're absolutely right--it's not possible to get a bunch of folks on the internet to actually sit down and read all of the articles and evaluate which is the best before casting their vote. So if you want the prize to go to the best writing, you need a panel of expert judges who are committed to giving the contest the time and dedication it deserves. Alternatively, if you're going to run future competitions like this one and give the prize to the best networking, that's fine, but you'll have to acknowledge it up-front.
And I'm actually quite excited about the flip camera! I feel a little odd being congratulated for winning it, since it was a random drawing--yay me! I got my name pulled out of a hat!--but the sentiment is kind and I appreciate it.
Hi again, Danna. Sorry it's taken a week for me to get back to you; I keep forgetting to write! :)
From what I understand, we wanted more people to join the ScientificBlogging community and participate in the voting process, so we opened voting up to anonymous users.
See, people on the Internets don't like having to register for yet another site in order to comment on something they see or to vote on something. It's a big barrier to people participating on any site. The general hope is that you can let them use the service, and they'll like what they see or be intrigued by other services available to registered members that then they'll sign up and come more often.
I honestly don't know how one should conduct competitions on the Internet anymore; if it's anything more complex than rating pictures of LOLcats (or LOLPlankton, as the case may be... :P), then people are just going to give up unless they're a part of the community (which takes time and networking) and really want to see the best person win. Which is kind of ironic, since competitions like this are held so we can make people think that we think their opinions matter, thereby making them want to sign up and become parts of the community (but without the time investment).
Also, how does one hire judges for a multi-disciplinary science site like ours? A physicist like Tommaso might not be able to appreciate the interesting points of a hardcore biology article, for instance. (Which actually raises an interesting point: wouldn't the best article be the one that scientists from all the disciplines get, representing our reader base?)
Damn, I'm talking myself in circles again.
Anyways, that's all I can think of for this installment of Patrick Should Be Doing Troubleshooting But Decided To Talk To Users Instead, the new hit TV show!
Patrick Adair,
Developer, ScientificBlogging.com
Yikes! I am sooooo sorry. Unbeknownst to me, this comment has been languishing in the bowels of Movable Type for the last month. Don't know what happened! Anyway, I finally unearthed & posted it.
I certainly understand that logins are a barrier to participation. People of the interwebs don't like registering, but they also don't like being manipulated. They know you're trying to "make people think that we think their opinions matter" but they also know that online contests are won by gaming the system. I stand by my assertion that "if you're going to run future competitions like this one and give the prize to the best networking, that's fine, but you'll have to acknowledge it up-front" because if you don't, people are going to feel cheated and manipulated.
As for hiring judges, wasn't the whole point of the competition to find good writing that appeals to the general public? If a biology article is too hardcore to be appreciated by a physicist, then it's probably too hardcore to be appreciated by the general public. You want generalist judges, people who know good writing, not necessarily people with PhDs. Professional writers and editors would probably be a good pool to draw from. But really, so would your next family reunion. I think it matters less exactly who the judges are, and more that they are people with no conflicts of interest who agree to sit down, read, and score every single article.
I'm honored to be entertained for this long as a guest on the new hit TV show. ;)