Over the last few days the search community has noticed that dmoz was no longer showing up in the search engine results page. Many thought that it could have been banned do to Google backlash on directories or of duplicate content issues.

Looks like that is not the case. Our all mighty Matt Cutts explains what happened over at sphinn:

Hey all, I dug into this a little bit with the help of a couple crawl folks. It looks like when Googlebot tried to fetch http://www.dmoz.org/, we got a 301 redirect back to http://www.dmoz.org/ . It looks like that self-loop has been going on for several days. We were last able to fetch the root page successfully on Sept. 10th, but from that point on DMOZ was returning these 301-to-itself pages, and after a few days Googlebot gave up on trying to fetch the url.

It looks like the rest of the site is fine, so I suspect that if DMOZ gets 301/redirects for their root page sorted out on their webserver, we’ll recrawl and index the page pretty quickly.

Just to be 100% clear, “Googlebot gave up” is not the root reason. I was just introducing a bit of levity. The real reason was of course the infinite redirect loop that lasted for days. If I 301 page A to point back to page A and do that infinite loop for a week (or more), it’s probably a bad user experience to return that infinite loop to users. But if the loop stops, then our system is set up to get the page again fairly quickly.

So, remember folks 301 redirects are very powerful so make sure you are doing them correctly and never place a 301 redirect from http://www.domain.com to http://www.domain.com.

Its almost like a dog running around chasing his tail!

Thanks Dmoz.org for the lesson!