How to prevent google access/indexing/listing to script pages? - php

Question

So, i have some PHP script pages that are accessed trough AJAX, others trough POST or GET and are used to send emails and access the database, and although i know that a search engine probably wont have interest in this pages i do not want it to even know that those exist.
I want a solid way to separate the pages that should be seen by a search engine and the ones that shouldn't.
I've seen Matt Cutts video (https://www.youtube.com/watch?v=nM2VDkXPt0I) in which he explains that the best way to prevent a page to viewed by Google is by using .htacess with password protection... The problem is that my script pages must be accessed by users.
Id like to know if there is a solution that only involves .htacess once in this video Matt Cutts explains that noindex, robots.txt are not very effective.
So the solution must follow the rules:
Use only .htacess (or something that works, but with no exceptions)
No HTML tags because of the specific response I'm getting in .responseText (these pages don't even have html, just php)
Allow single page restriction (not full directories for example)
Allow user access
I've searched a lot, and seen many solutions out there, but nothing that works for me, so, any ideas ?

Answer 1

Create a directory for your ajax pages and then set the htaccess to block Google from accessing it.
For directory redirects:
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^googlebot
RewriteRule ^/ajax/ - [F,L]
For single page redirects:
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^googlebot
RewriteRule ^([^/\.]+)/?$ yourpage.php [L]
Just in case you want to redirect multiple files (as i assume you do)
RewriteCond %{HTTP_USER_AGENT} ^googlebot
RewriteRule ^(file1|file2|file3|file4)\.html$ http://www.yoursite.com [R=301,NC,L]
Hope this helps.
Note that this must be uploaded to the parent directory and not the ajax folder.
Editing for a different solution, as you seem keen on single file redirects, you could return a PHP 301 redirect if a search engine bot enters your site
function bot_detected() {
if (isset($_SERVER['HTTP_USER_AGENT']) && preg_match('/bot|crawl|slurp|spider/i', $_SERVER['HTTP_USER_AGENT'])) {
return TRUE;
}
else {
return FALSE;
}
}
if(bot_detected() {
header (“http/1.1 301 Moved Permanently”);
header (“Location: http://www.yourwebsite.com”);
}