For those of us working in the world of SEO, the Rap Genius link spam controversy will go down among the big SEO events of 2013. I’ve written about it before, but just to summarize—the lyrics site got caught red-handed asking bloggers to include links to Rap Genius in unrelated blog posts. That’s pretty egregious spam, and once it went public, Google’s head of Webspam, Matt Cutts, crushed the site’s search rankings with a manual penalty.
First it’s important to understand what Rap Genius did wrong. Google penalized the site for “unnatural links,” meaning that many of the sites that linked to Rap Genius didn’t do so in a way that seemed “organic.” Think of it this way: if a blogger writing about Jay-Z linked to a Rap Genius page, it would make sense. But if a post about the best toaster ovens included a Rap Genius link to Justin Bieber lyrics…
It looks spammy.
To resolve its manual penalty, Rap Genius had to get rid of these links (or as many of them as possible). Remember, these were links on other sites, pointing to Rap Genius. There are a two ways to get rid of links to your site that you no longer want:
Use Google’s Disavow links tool (more on this later)
The challenge for Rap Genius is that the site had thousands of links pointing to it – 177,781, according to a recent blog post. So, how to ferret out the bad ones without getting rid of the good stuff? Let’s get technical.
How to Find Bad Links
Rap Genius had the advantage of knowing exactly why they’d been penalized—because they’d asked bloggers to add off-topic links to Rap Genius sites on their blog posts. This was part of a quid pro quo scheme where Rap Genius would tweet links to the blog post—boosting that blogger’s traffic—in exchange for those unnatural links.
So the first step for Rap Genius was to examine which sites it had made this exchange with. By singling out only links from those domains, Rap Genius was able to winnow the list down to 3,333—still a big number, but a lot smaller than 177,781.
Next, the site divided those links into categories:
“Natural links” that were woven into the text
Lists of links (usually track lists from albums) that were relevant to the post, i.e. a Miley Cyrus track list on a post about Miley Cyrus
Lists of links that were irrelevant to the post, i.e. a Billy Ray Cyrus track list on a post about Eminem
Rap Genius decided to rip out all of the link lists, even those that were relevant to the subject of the blog post. These may not have been technically against Google’s guidelines, but it’s just the kind of self-flagellation that Google loves to see in penitent webmasters. The site emailed the webmasters for all of those sites and asked them to take down their links.
Getting Help from a Robot
Next, Rap Genius built a program to scour every link on its list of 178,781. This robot is not unlike Google’s indexing “spider,” in that it crawled across the websites, read the behind-the-scenes HTML code, and collected certain data. Rap Genius decided that certain characteristics indicated that a site was more likely to contain unnatural links. The site used these characteristics to give links a numeral rating on their “suspiciousness.” Indications of suspiciousness included:
A high number of links to Rap Genius
Linked text that included the word “Lyrics”
Links grouped closely together
After the robot did its work, the folks at Rap Genius went through and looked one-by-one at the most suspicious sites. This generated another 590 pages with unnatural links, and Rap Genius again emailed the owners of these sites to ask them to remove the links.
How to Remove Links from Other Sites
Rap Genius had one more major problem – “scraped” sites. These sites use robots to automatically steal content from other, more popular pages and put it on their own. The idea is to replicate the success of those sites and get people to click, enabling the owners to charge for ads. These sites had “scraped” other sites that contained links to Rap Genius—multiplying Rap Genius’s problem.
So Rap Genius wrote another program. This one drew on publicly available information about the owners of the scraped sites—what’s called the WHOIS listing. Anyone can use a site like Whois.net to look up contact information about the owners of any domain. This can be names, email addresses, phone numbers, and even home addresses. Rap Genius’s program robot collected this information automatically so that they could email the owners en masse and ask them to take down their Rap Genius links.
And finally, after Rap Genius made what Google calls a “good faith effort” to get webmasters to remove bad links, the site used Google’s Disavow links tool to cancel out any remaining bad links. Google discourages wanton use of Disavow links, but in this case apparently decided Rap Genius had tried hard enough to contact site owners.
And apparently it worked. In just a week, Rap Genius recovered its rankings—and got a nice boost in traffic from the controversy, as well. The whole episode was enough to get SEOBook’s Aaron Wall pretty worked up, and not without justification:
Most “spammers” would never attempt such a campaign because they would view it as being far too spammy. They would have a zero percent chance of recovery as Google effectively deletes their site from the web… It comes without surprise that on a Saturday barely a week after being penalized Google restored RapGenius’s rankings.
In short, it took Rap Genius a week to do what takes other sites—with far fewer links, and far fewer resources—weeks or months to accomplish. And many sites never recover at all. Undoubtedly, a big part of the Rap Genius recovery was the publicity that surrounded it.
Although this may seem like a Herculean effort, it’s almost exactly what we do here at Blue Corona. We often work with clients who come to us after receiving a manual penalty, and so the bulk of our work can be sorting through their links and submitting reconsideration requests to Google. Unlike with Rap Genius, there’s rarely a shortcut for penalty recovery. It requires careful examination of a site’s entire link profile—a painstaking process that many business owners don’t have the time or expertise to do themselves.