Past Time For MSN To Pony Up To The Real Truth About Referrer Spam
Way back on August 17th, the first reports about MSN sending weird referrer spam, started trickling in. This referrer spam includes weird referral strings such as adult keywords, pharmaceutical keywords, etc. A few weeks later, msndude, made a statement about this saying they were official Microsoft quality check tests. Specifically, he said, "The traffic you are seeing is part of a quality check we run on selected pages. While we work on addressing your conerns, we would request that you do not actively block the IP addreses used by this quality check; blocking these IP addresses could prevent your site from being included in the Live Search index."
For various reasons, msndude's answer is completely unsatisfactory, and the fact that this is still happening months later, without any further clarification from msndude is even worse. I've had some people bugging me to look into this for a while now, but I assumed it would just disappear and not be worth the time. Now, I have to wonder if that was a mistake on my part.
Something fishy appears to be going on, and the one comment from msndude about it doesn't get rid of the fishy smell. I think a little more noise will need to be made if we hope to get some real answers about this.
Here are a few points from previous forum threads and blog posts that you might want to think about:
I am getting thousands of hits where the items in my log show the referrer as follows;
http://search.live.com/result.aspx?q=KEYWORD&mrt=en-us&FORM=LVSP
When I load the referred page then I am told that there are no results. Also there is no relationsfip between the keyword and the page requested. The Kkeywords are single words and seem to be mainly concerned with the normal spam areas.
- Landing pages appear to be pseudo-random (seemed to request 'batches' of related pages)
- IPs changeable in the range 65.55.165.*
- Spoofed referrers are search.live.com/result.aspx?q=[KEYWORD]&mrt=en-us&FORM=LVSP. However, the bot also makes requests without a referrer, some times in the course of the same visit
- Keywords are usually single words, varying from obvious commercial (but inoffensive) to drug names and pr0n-related words
- System details are identical for each 'visitor': MSIE 7.0, Windows 2003, 32 bit colour res, 800×600 screen res, both cookies and javascript accepted
To add to it, they started hammering my poetry site on Thursday, and they were also downloading my AdSense blocks, completely inflating my stats. Myblogblog however, didn't recognize them as visitors, so the discrepancy was immediately obvious.
For several weeks I have been annoyed by this "quality check" as it imposes it self as real behaviour with full fledge browser capabilities and a standard user agent. All of the sudden my customers are all excited over getting all this search engine traffic from Live Search which they are in fact NOT.
some people on other forums think that MSN is sending them traffic.
That's one way to make it look like someone is using their SE.
am getting severely hacked off with this. My logs are the most important tool I've got and they are being wrecked.
My opinion is simple, as I've stated here in WebmasterWorld for two months also, that Microsoft is intentionally screwing up our log files for their own gain.
It takes the .css and .js stuff despite it being off limits in robots.txt.
All of the above quotes come from the WMW thread mentioned in the first sentence of this post. The thread is fairly long, so although I don't usually "rip off" this much content, it seemed the only way to really show the meat of the problem without making everyone read the entire thread (which of course, 99% wouldn't bother doing).
Surely, there's something really bizarre about all of this, and I think everyone deserves a better answer than msndude initially provided. In addition, it needs to STOP happening. Microsoft, just stop it. It's wrong. Period.
E-Newsletter
Top SEP Blog Posts
[...] 1: DazzlinDonna from SEO Scoop has written an excellent background to this fiasco, and Michael VanDeMar is reporting that Microsoft is interfering with AdSense. [...]
SearchCap: The Day In Search, November 13, 2007…
Below is what happened in search today, as reported on Search Engine Land and from other places across the web…….
Actually I may disagree . MSN has to choose between a quality check and get rid of a LOT of spam from their index and a few pissed webmasters
Now Donna you want MSN to have better rankings ? If Yes why you don't want accept some errors in your logs .
I'll explain , as I understand they created a fully capable bot MSIE and check for cloacking , usually BH look for referrer if is MSN redirect to ppc or aff program this is done 3 ways , ip cloacking / JS redirection (that's why your adsense is messed up) / server side redirection (same as JS but done on server using PHP etc ).
I would say this is a good idea tho it's not bulletproof especially IP cloacking but with freehosts like blogger , it's perfect .
So you guys have to choose between a higher ranking (less spam ) or some messed up logs .
On the other hand I don't understand why they didn't create a white list of urls from where not to load/execute JS like Google adsense/ analytics etc .
"It takes the .css and .js stuff despite it being off limits in robots.txt." – js that does the redirection could be placed there too so this is a must .
And I am thinking this is a real browser out there (MSIE) incorporated in a program that does the quality check so technically speaking you have lots of visitors from the same IP.
No, them looking for "cloacking" does not make any of this reasonable or explain the whys behind their behavior. That is one of the more ridiculous theories I have heard about this, and obviously you really do not understand the situation at all.
I suggest you familiarize yourself a little more thoroughly with the whole story, and the way things work, before posting something critical towards someone who quite obviously knows more about the workings of the industry than you do.
By the way, posting the label next to the text box as your name, when asked to provide your name when commenting, isn't clever, it's rude.
Microsoft Live.com Plays Referrer Stats Games and Ruins Your AdSense Income…
While it is common, nowadays, to whine how Google shouldn't tell people how to make money with their websites (like selling links), this time Microsoft has done the same.
Lately, it's been noticed that Live.com spams the referrer logs of the websites…
[...] http://www.seo-scoop.com/2007/11/13/past-time-for-msn-to-pony-up-to-the-real-truth-about-referrer-sp… [...]
Making Live Search error free is not my pain… Don't threaten me with a ban if I block these spam referrals… I never depend on you for my food…
Do you think MS allow me to spam them with an excuse of 'quality check'? Give respect and take respect… Monopoly won't rule…
[...] Spam referrals. Pierre: Yell if Microsoft’s Live.com Spammed You Too – Updated Dazzlin Donna:Past Time For MSN To Pony Up To The Real Truth About Referrer Spam Michael Vandemar: Microsoft Needs To Quit Fucking With My AdSense Scripts – this is something [...]
[...] reports that MSN has acknowledged this odd behavior and requests that webmasters not block their bots. It's not happening on any of my sites, but if it were I'd be inclined to tell MS to [...]
I'm glad you brought this up because I'm having the same issue and the "quality checks" are increasing to the point where this is playing havoc with the Referral Traffic Numbers for the site I manage at my office.
I would prefer that they not do this. Why can't Microsoft perform the quality check against the data in the crawl bucket right after it gets fetched?
Hmmm… I never noticed this traffic before, have to go and check it out now.
[...] Past Time For MSN To Pony Up To The Real Truth About Referrer Spam [...]
[...] Webmasters across the globe weren't that excited to find promotinal messages from Live Search in their log files, so a somewhat confused MSN dude [...]
[...] 3 weeks ago, I said it was Past Time For MSN To Pony Up To The Real Truth About Referrer Spam. Now, they [...]
I suspect there's a simple explanation for all this … just follow the money.
I think M$ is trying to convince people that their live.search.com is sending you all kinds of visitors … and, of course, that means you should be advertising with them.
It's possible that search.live.com is not only showing people the search hits BUT also visits (at least) some (large!) number of pages in that list.
E.G., when I look at the referrers for the last 3 days for one of our sites, I see entries like:
- 10 hits : http://search.live.com/results.aspx?q=license
- 1 hit : http://search.live.com/results.aspx?q=licenses
- 1 hit : http://search.live.com/results.aspx?q=upgrades
- 7 hits : http://search.live.com/results.aspx?q=contacts
… and on and on.
So, if they're hitting our pages for these kind of search terms, you can imagine that they're hitting hundreds of thousands of pages — possibly for each person's search.
That way it looks like there are "sooooo many people" visiting your site because they've used search.live.com … and, of course, that'd mean you really should be advertising with them.
When all this started, we suddenly saw referrals go from about 3% of the number of Google search referrals to more than the number of Google-search referrals … all within a few days. Analyzing the resulting traffic from these referrals makes it fairly obvious that they're not real people visiting.
In the end, I just attribute this to the normal kind of business behavior that's consistent for Microsoft. What sleazebags!
Jim – I am in the same boat – I never noticed the increased traffic and was always wondering why I ranked much higher with MSN. I have to look at my analytics and see if i can follow the traffic.