We Fought the Comment Spam (and the Comment Spam Didn't Win)

Posted by FeliciaCrawford

All across the Internet, comments sections are disappearing.

From your high-profile news sites to those that share the online marketing space, more and more sites are banishing that unassuming little text box at the bottom of a post. And frankly, it’s not hard to understand why.

First, you have your good ol’-fashioned spam comments. These are the commenters that hold dear the idea that those nofollowed comment links are valuable:

The usual.

Spicing it up a bit with some solid industry advice.

Really going for the gold!

Then you have your thin comments. Often left with (we assume) good intentions, they don’t add much value to the discussion:

These poor souls usually end up with a lot of downvotes, and if they receive upvotes, it’s often a clear sign that there’s a nefarious MozPoint scheme afoot.

Sometimes even the best of us are lured by the glamour of spamming:

Finally, lest we forget, you have your inflammatory comments. Those comments that, although perhaps on-topic, are derailing or just downright unkind. We don’t get too much of that here on the Moz blog, thank goodness (or thank TAGFEE), but I’m sure we’ve all read enough of those to last us several lifetimes.

And comment moderation is a thankless, wearying task. Though we fight the good fight, comment spammers are constantly finding ways around our barriers, poking their links into the chinks in our armor. It takes valuable time out of a Mozzer’s busy workday to moderate those comments.

So why are we battling to keep them?

In the beginning, there was the blog.

Before the Moz Pro toolset was even a twinkle in Roger’s optical sensors, Moz was a blog. A community of brave folks banding together to tackle the mysteries and challenges of SEO. If you look back across the years and rifle through the many, many comments, you’ll begin to notice a few things:

• People learned from one another.
• People leaned on one another.
• People networked and cultivated relationships that otherwise may not have blossomed.

Google says they're good for SEO, and I'm not gonna fight with Google.

Now, I don’t want to cheapen the sentiment here, but it has to be said: the smart folks over at Google have made it clear that a healthy, vibrant online community is one signal of a site’s quality. Comments can be considered part and parcel of what constitutes good (nay, even great) content, and have even been spotted in a featured snippet or two.

I don’t know about you, but I’m not one to argue with the Big G.

But there's always been comment spam. Why do you care now?

Comment spam isn’t a new or novel phenomenon. It’s been plaguing blogs almost since the very first public bloggers put fingers to keyboard. Most blog posts on Moz show traces of its corrupt spamalicious influence in the comments section. So what was the catalyst that steeled our resolve?

It just got annoying.

Authors pour heart and soul into crafting their posts. They take valuable time out of their regular work day to engage in the comments section, answering questions and driving thoughtful discussion. They deserve better than a slew of spammers aiming to place a link.

Readers devote hours of their ever-so-precious lives to reading the blog. Some folks even read for the comment conversations alone. They deserve to benefit from those invested hours, to be inspired to join the conversation.

We knew we had to do something. What that was seemed unclear, though.

We began to notice something. When we promoted a YouMoz post to the main blog, it tended to garner more of what we’d call quality comments. Comments with depth, that ask pertinent questions, that respectfully challenge the article in question. These posts came prepackaged with their own discussions already in full swing from their time on YouMoz; often, the first few comments were engaging ones, and they were just as often upvoted to remain on top (the blog auto-sorts comments by popularity).

Conversely, when the first several comments on a brand-new post were thin, spammy, or otherwise low-quality, it seemed to grind any potential discussion to a screeching halt. Internally, our Mozzer authors like Dr. Pete and Rand began to take notice. I received some concerned questions from other frequent contributors. At first, I wasn’t sure how to tackle the problem. After all, we already seemed to be doing so much.

Comment moderation? Check. Certain triggers catch comments in a queue, which we clear out daily.

Subject every comment to approval by an editor? No, that would stymy the natural discussions that make our blog comments section special in the first place. No one should have to wait for my morning meetings to finish before they can engage in intellectual banter with their peers.

Close the comments section? No way. This was never on the table. It simply didn’t make sense; we’re fortunate in that a good majority of comments on the blog are high quality.

It boiled down to the fact that there was the potential for our comments section to nurture not only good content, not only great content, but fantastic content. 10X, if you prefer that term. The most royal darn content outside of Buckingham Palace.

Okay, that might be going a little far. But something incredibly special happens here on the blog. You can ask questions about a Whiteboard Friday and Rand will do his best to answer, thoughtfully refute, or discuss your point. You can get to know your peers in an industry largely cooped up behind a screen half a world away. You can joke with them, disagree with them, metaphorically high-five them. And it’s not limited to a relatively low character count, nor is there pressure to approve the friend request of anyone you’ve just hotly debated.

We had to preserve that.

And that’s when we devised our grand experiment.

We began to seed discussion questions as the first comment.

Inspired by sites like the New York Times with their “NYT Pick” featured comment option, we decided there was a better way.

Marvel at that nifty gold badge!

For one week in August (8/1 through 8/5), I asked authors to contribute a discussion question, something to spark a decent conversation in the comments early on, before you could even say “thanks for the nice post.”

This question would appear at the top of our comments section, the first thing a reader would see after consuming the post and potentially feeling inspired enough to share their thoughts.

Rand kicked it off a little early, in fact, with this zinger on July 29th:

Those upvotes looked mighty promising to a despairing blog editor.

Keep in mind that, for the most part, posting these discussion questions is a very manual process. We don’t currently have the framework built to display a “featured question.” We tend to publish around 12am Seattle time; to get these little puppies in place early enough to make a difference, I would...

• Stay up until midnight
• Assume the identity of the author (with permission, of course) using magical Moz admin abilities
• Publish the comment
• Sneak back to my main account and — yes, here’s the shady bit — thumbs it up to ensure it stayed “on top” for a few hours

I do struggle with the guilt of these small betrayals (that is, gaming the thumb system), but ‘twas for the greater good, I swear it! As you can see from the screenshot above, that high visibility — combined with a ready-to-go thought-provoking question — earned more upvotes as the day wore on. Almost without fail, each seeded discussion question remained the top-voted comment on every post that week. And it seemed to be working — more and more comments seemed to be good quality. Great quality. Sometimes even fantastic quality. (I just shivered.)

What's spam to me might be a sandwich to you.

Now, quality is a very subjective thing. I can’t vouch for the absolute science of this experiment, because it was very squarely rooted in a subjective analysis of the comments. But when we compared the results from our experiment week (8/1 through 8/5) to two separate weeks in which we didn’t make any special effort in the comments (7/18 through 7/22 and 6/27 through 7/1), the results were quite telling.

Cut to the chase — what happened?!

Manually going through the comments section of each post, I tallied how many comments I considered high-quality or useful that were not given by the author, and how many comments I considered so thin or spammy as to be detrimental to the section as a whole.

For the control week of June 27th through July 1st, 26% of total comments were high-quality and 26% were spammy.

For the control week of July 18th through July 22nd, 23% were high-quality and 29% were spammy.

For the week of our discussion questions, August 1st through August 5th, 35% of total comments were high-quality and 11% were spammy.

My subjective, unscientific experiment had great results. Since then, I’ve asked our authors to contribute discussion questions to kick off a good conversation in the comments. Every time, I can anecdotally say that the commentary was more vibrant, more overtly helpful, and more alive than when we don’t meddle.

You like it, you really like it!

Seeded discussion questions far and away have more upvotes than your regularly scheduled top comments. Often they top the double digits, and this very apt discussion question by Gianluca (a long-time supporter and champion of the Moz community) earned a whopping 27 thumbs pointing toward the heavens:

In addition, people are answering those questions. They're answering each other answering those questions. The questions are helping to get the gears turning, adding another layer of thoughtfulness to a piece that you otherwise might be content to skim and then bounce off to another magical corner of the Internet.

The greatest and most humbling triumph, of course, would be to help transform the spammers into supporters, to inspire everyone to think critically and communicate boldly. If even one person hesitates before dropping in a promotional link and instead asks the community's advice, my spirit shall rest easy forevermore.

There's a light at the end of the tunnel.

Sure, there are still comment spammers. There have always been comment spammers. And, though it pains me to say it, there will always be comment spammers. It’s just a part of life we must accept, like the mud that comes along with a beautifully rainy Seattle afternoon or when your last sip of delicious coffee is muddled with grounds.

But I want to give you hope, O ye commenters and readers and editors of the world. You need not sacrifice the intrinsic goodness of a community-led comments section to the ravages of spam. There is another way. And though the night is dark and full of spammers, we’re strong enough and smart enough to never yield, to hold firm to our values, and to nourish what goodness and helpfulness we can in our humble territory of the Internet.

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!