mYfAVE'StOYOU: Disavowed: Secrets of Google's Most Mysterious Tool

Update: Check out the new findings at the end of the post.

To many webmasters, Google’s Disavow Tool seems a lifesaver. If you’ve suffered a Google penalty or been plagued by shady link building, simply upload a file of backlinks you want to disavow, and BOOM - you’re back in good graces. Traffic city!

Or nothing happens at all.

Few Google products have produced more fear, rumors and speculation. No one outside Google knows exactly how it works, and fewer understand how to succeed with it. To better understand, I used the tool myself to disavow 1000s of links, and talked with dozens of SEOs who used it in attempts to recover from Google penalties.

How Dangerous Is Disavow?

When you first log into the Disavow Tool, Google does everything in its power to dissuade you from actually using it with scary messaging.

Do Not Disavow

What’s the worst that could happen?

To find out how much damage I could do, I performed an experiment: Disavowing every link pointing to my website. Over 35,000 of them.

In this case, no reconsideration request was filed. Would the disavow tool work on its own?

Experiment

Disavow 35,000 Links to a Single Website

URL: http://cyrusshepard.com

Process:

Download all links from Google Webmaster Tools
Upload 35,000 properly formatted links to Google's Disavow Tool
Wait 2 Months

Results:

After 2 months, nothing happened. No drop in traffic.

The evidence suggests one of three possibilities:

You must file a reconsideration request after disavowing your links, or...
The disavow has built-in safeguards in order to protect you from disavowing good links, or...
It takes longer than 2 months for Google to process all the links.

We’ve heard conflicting accounts from Googlers whether the tool works automatically, or if must file a reconsideration request for it to work. The data implies the later, although some SEOs say they’ve seen results from using the Disavow without filing a reconsideration request.

Google also states they reserve the right to ignore your disavowed links if they think you made a mistake, much like rel=”canonical”.

Best Advice: Safeguards or not, you might still shoot yourself in the foot. Be careful disavowing links!

Can You Use Disavow for Penguin?

Can you use the Disavow Tool if you haven't received a manually penalized? For example, will it work for Penguin?

The answer: maybe.

Here's a reminder: Google updates like Panda and Penguin are part of Google's overall algorithm. They automatically affect your rankings without direct human intervention. On the other hand, a manual penalty is often applied when you violate Google's guidelines. These can be both applied and lifted manually by Google's Webspam team.

Google representatives, including Matt Cutts, have gone on record to say the Disavow Tool could be used to help if you’ve been hit by Penguin (an algorithmic action), but also suggests that this applies to links that also violate Google’s Quality Guidelines.

Penguin and Google’s Unnatural Link Warnings often go hand in hand. So if you were hit by one, you are often hit by the other. Conversely, certain SEOs have claimed benefits from using the disavow on sites that were not penalized.

Best Advice: If you’ve been hit with a manual penalty, you need to file a reconsideration request if using the Disavow Tool. If you haven't been manually penalized, the benefits of using the tool are inconclusive.

1. Remove First, Disavow Last

Google wants you to remove links first. Disavow is a last resort.

100% accuracy isn’t required, but effort counts.

Google’s Webspam team keeps a historical index of your backlink profile, so that when you file a reconsideration request they can see the links you’ve worked to remove.

2. Gather Your Links

You can use any source you want, but Google recommends downloading your Latest Links report directly from Webmaster Tools.

3. Find the Bad Links

You can do this two ways, with either automatic tools or manual analysis. Realistically, you should use both. Best Manual Analysis Resource:

Easy Step by Step Guide to Finding Low Quality Links

Best Link Removal Research Tools:

Link Detox
Remove’em
SEOgadgets Link API Extension for Excel. They also have contact information on file for millions of URLs.

Link Removal Resources

4. Outreach, Outreach, Outreach

Next, you’re going to send emails to get those links removed. Lots of emails.

Resources for Link Removal Outreach:

4. Trust in Google Docs

When you document your efforts, don’t submit random links to the Webspam team; they may not click on them. By sharing all your evidence via Google Docs, you provide a level of protection that helps ensure the Webspam team sees your evidence.

5. When in Doubt, Disavow Entire Domains

Google’s Disavow Tool gives you 2 options when disavowing links: individual URLs or entire domains.

Many webmasters fail at their reconsideration requests the first time because they miss too many links. The fear is that you’ll disavow something valuable, but if you’ve been rejected time and time again, this one change often leads to success.

Here’s a screenshot from Dr. Pete’s post showing both formats.

Disavow Format

Best Advice: If you are rejected after disavowing individual URLs, try disavowing entire domains.

6. Formatting Counts

Google rejects many disavow files because of bad formatting, but webmasters usually never know. Guidelines state the file type should be .txt only and “must be encoded UTF-8 or 7-bit ASCII.”

7. Bonus: Extra "Removed" Links with Screaming Frog

Google’s link index of your site is rarely up to date. They undoubtedly include links that no longer exist. To find dead links quickly, download a complete file of your latest links from Google Webmaster Tools into Screaming Frog (use List Mode) or another crawler of your choice.

When finished, take any links that return a 404 and download these into a spreadsheet. Be sure to include these dead links as "Removed" when you submit your reconsideration request to Google, otherwise they may not know about them.

Conclusion

The Disavow Tool is useful, but damn tricky.

Someday, perhaps Google can get away from tools like the Disavow. Today, good SEOs can't keep up with what's considered a good link or a bad, and Google continually cracks down on what it considers a “bad link.”

For successful marketers, it’s much more fulfilling to build new links, than disavow old ones.

I suppose that's Google's point, no?

Additional Resources:

Update 5/31: Closer to an Answer

As it turns out, my ranking started dropping the same day Penguin 2.0 was released, the Thursday before this post. (I actually researched the post before this, and the traffic drop happened around a holiday weekend, so it went largely unnoticed)

As of today my rankings and traffic are definitely down:

Rankings Change

Tim Grice of Branded3 was sharp enough to notice something similar. Tim has experience with the Disavow Tool himself and saw something similar last Thursday. His theory is that Google processes your Disavow file either:

When you file for reconsideration, or
When there’s a big data refresh, such a Penguin update.

Tim Grice Twitter

Does this mean my site was hit by Penguin? No. It simply indicates Google processed my disavowed links with the Penguin refresh. Those disavowed sites no longer count towards my domain, so my rankings dropped - at least in theory.

Also today, Marie Haynes published a collection of mini-case studies showing dramatic movement in SERP visibility during the Penguin refresh on sites that disavowed links.

We’ve also heard anecdotal accounts of other sites seeing much faster results from the Disavow without filing a reconsideration request, but even these webmasters saw a much bigger change happen during Penguin 2.0.

If this all proves to be true, the following is possible:

Google’s Disavow Tool may work automatically, but doesn’t process until you file for reconsideration or there is a major data refresh.
You could, in theory, use the Disavow Tool to recover from Penguin, even if no manual penalty was applied and you don’t file a reconsideration request.

What we don’t know at all is how often Google makes data refreshes that would incorporate disavowed links. My own site waited 2 months for a change to take place, and last year saw only 3-4 Penguin refreshes.

On the other hand, we’ve also heard anecdotal accounts of other sites seeing much faster results from the Disavow without filing a reconsideration request, but even these webmasters saw a much bigger change happen during Penguin 2.0.

A better question might be: Should you use the Disavow Tool to try to recover from Penguin if you haven’t received a manual penalty?

My answer: Recovery stories have been few and far between, but they do happen. In my personal experience, it seems more likely to recover from Penguin if you earn new and more diverse links from high quality sites. This, along with obvious link cleanup, seems to be the best course of action every time.

By the way, I’ve removed the file from the Disavow Tool. Let’s see what happens.

Fingers crossed.

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!