Yellow Mouse Robots Spray: Repel Search Engine Robots From Indexing Your Pages
- Date: 2007-08-10 - Word Count: 456
Share This!
Don't you hate it if your download page or thank you page can be found in Google, Yahoo or MSN? What about your eBook, is it listed in Google, too?
Your result of your hard work can be accessed by everyone because your page is simply indexed by search engines such as Google. You invested your money, time and energy to create your best product but your product can be found by everyone easily without even paying you. You could loose thousands of dollars. You don't want this happen to you, do you?
So here comes the yellow mouse robots spray. Repel search engine robots from indexing your pages!
Robots.txt
Robots.txt is a simple text file that tells the search engine robots not to crawl certain directories and pages of your site.
When a robot crawls your site it looks for the robots.txt file. If it doesn't find one it assumes automatically that it may crawl and index the entire site. Not having a robots.txt file can also create unnecessary 404 errors (page not found error) in your server logs, making it more difficult to track the real 404 errors.
Creating robots.txt file is easy.
1. Open your notepad or other text editor (not HTML editor).
2. Type this following to prevent all robots to crawl or index your certain directory:
User-agent:*
Disallow:/directory/
where directory is the directory that you don't want to be crawled or indexed by search engine robots.
3. Save the file as robots.txt.
4. Upload it to your domain root directory so that it looks like this: http://yourdomain.com/robots.txt
5. Confirm if everything is correct using robots.txt validator. (You can easily find robots.txt validator using Google Search).
For more information on robots.txt, you can visit http://www.robotstxt.org.
Are these two tricks enough to protect your product download page, thank you page and private page? Perhaps, because nothing is perfect in this world.
So I'll give you another trick to double protect your precious page.
Meta tags
Another trick, which may be more convenient for page usage, adds a META tag to an HTML page to tell search engine robots not to index the page.
To prevent all robots from indexing a page on your site, place this following META tag into the HEAD section of your page:
META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW"
(Add "" after the code).
That should helps protect your precious pages.
One more trick, exclusive to blog only!
Disallow Pings
All of those tricks above are useless if you still allow pings for your blog page or blog post.
Disallow pings for your precious page.
If you use Wordpress:
1. Expand your discussion tab.
2. Uncheck "Allow Pings".
These tricks should help you protect your product download page, thank you page and private page. However, these tricks don't protect your precious page if your buyer adds it to his Internet Explorer's Favorite, Firefox's Bookmarks or social bookmarks and share the link with his friends.
Your result of your hard work can be accessed by everyone because your page is simply indexed by search engines such as Google. You invested your money, time and energy to create your best product but your product can be found by everyone easily without even paying you. You could loose thousands of dollars. You don't want this happen to you, do you?
So here comes the yellow mouse robots spray. Repel search engine robots from indexing your pages!
Robots.txt
Robots.txt is a simple text file that tells the search engine robots not to crawl certain directories and pages of your site.
When a robot crawls your site it looks for the robots.txt file. If it doesn't find one it assumes automatically that it may crawl and index the entire site. Not having a robots.txt file can also create unnecessary 404 errors (page not found error) in your server logs, making it more difficult to track the real 404 errors.
Creating robots.txt file is easy.
1. Open your notepad or other text editor (not HTML editor).
2. Type this following to prevent all robots to crawl or index your certain directory:
User-agent:*
Disallow:/directory/
where directory is the directory that you don't want to be crawled or indexed by search engine robots.
3. Save the file as robots.txt.
4. Upload it to your domain root directory so that it looks like this: http://yourdomain.com/robots.txt
5. Confirm if everything is correct using robots.txt validator. (You can easily find robots.txt validator using Google Search).
For more information on robots.txt, you can visit http://www.robotstxt.org.
Are these two tricks enough to protect your product download page, thank you page and private page? Perhaps, because nothing is perfect in this world.
So I'll give you another trick to double protect your precious page.
Meta tags
Another trick, which may be more convenient for page usage, adds a META tag to an HTML page to tell search engine robots not to index the page.
To prevent all robots from indexing a page on your site, place this following META tag into the HEAD section of your page:
META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW"
(Add "" after the code).
That should helps protect your precious pages.
One more trick, exclusive to blog only!
Disallow Pings
All of those tricks above are useless if you still allow pings for your blog page or blog post.
Disallow pings for your precious page.
If you use Wordpress:
1. Expand your discussion tab.
2. Uncheck "Allow Pings".
These tricks should help you protect your product download page, thank you page and private page. However, these tricks don't protect your precious page if your buyer adds it to his Internet Explorer's Favorite, Firefox's Bookmarks or social bookmarks and share the link with his friends.
Related Tags: make money online, robots, meta tag, robots txt, search engine robots, index page
Visit make money online to get the ultimate protection for your pages. Why WASTE Your Valuable Time and Money? Learn how to make money online with topics like marketing blog, blogging for money, affiliate marketing, advertising network, Agloco, RSS feeds and a lot more. Your Article Search Directory : Find in Articles
Recent articles in this category:
- Many Different Content Management Systems Are Available ( Cms)
At this time the rule is that going to the content management system open source and a commercial. H - Establish Your Credibility Online
Shoppers are now careful these days. They are more conscious that there are fraud sellers lurking on - Make Money Online Fast - Is It Possible?!
When you see the phrase "make money online fast" you may wonder to yourself, is this for real?! And - What Is Auto Traffic Avalanche?
Many people is talking about this great tool, and that became an excellent software, that can genera - Beginners Guide To Blogging
A blog is basically an online journal wherein you can digitally pen down your thoughts, ideas, opini - Say No To Just Having Enough To Get By...look For Wealth After A Fixed Paycheck
For the last year, I am proud to say that my regular weekly salary was earned through direct deposit - How Outsourcing Works - Hiring An Outsourced Virtual Assistant
Outsourcing is a proven means of helping businesses cut resource costs while getting the same level - How, Where Should You Start And Get Paid In Affiliate Marketing?
Before internet, freelance jobs like insurance, network marketing and real estate used to be the onl - Saratoga Builders Help You Make Informed Decisions
Numerous American central heating designs implement a furnace which blows forced hot air by means of - Basic Features Of A Good Web Hosting Plan
There are a few features that you should look for when you are picking a good web hosting plan, ther
Most viewed articles in this category:
- Tools for Adsense
If you're just boarding on the AdSense train, and looking to find a quick way to make the profits yo - Information Products - Your Action Plan
If you've been thinking of writing your own information product, here are a few tips to help you get - Are You Missing Out On This Method Of Generating Instant Cash?
If you have not yet heard of Bum Marketing, you're missing out. It's a great online marketing metho - How to Find a Home Based Business Internet Opportunity Thats Right for you
Copyright © Jagraj Gill http://www.payitforward4profits.com/jaggiegill Finding the right hom - Can It Be True? Can You Really Use The Internet to Create Money?
Is it hype? Or is it real? Is dramatized and exaggerated? Or can I really use the internet to cre - How to Create a Virtual Grand Opening
Copyright © 2007 Donna Gunter Grand openings and ribbon cuttings don't have to be limited to bu - What Is The Hottest Product To Sell Online
After going through a survey from all the internet Guru Alex Mandossian, Armand Morin, Derek Gehl an - Web Based Employee Time Clock
The power of the internet is in its connectivity. Many people think that the internet is great becau - Driving Hordes of Traffic to your Site
Getting enough traffic to your site is the overwhelming problem for anyone new to Internet or Affili - So You Want to Start an Online Business Part Two - Analyzing Keywords
Analyzing Keywords Continuing on from Part One of 'So You Want to Start an Online Business'; we wil