When getting the Google site in cURL PHP I get the old Google page (The one with the blackbar and weird buttons). So is there a way to get the Google as it appears on google.com? My code:
function file_get_contents_curl($url, $proxy) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_AUTOREFERER, TRUE);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_PROXY, $proxy);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
echo file_get_contents_curl('https://www.google.com/', 'proxy:8080');
Thanks, if there is no way I am ok with the weird looking Google.
Related
I am trying to login to a web page and then retrieve data from a page that is only visible to logged in members. I seem to now be able to log in using CURL, however I'm unable to retrieve the page that I want.
function login($url,$postData){
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postData);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_REFERER, "http://domain.com/".$jobUrl."");
curl_setopt($ch, CURLOPT_USERAGENT, "MozillaXYZ/1.0");
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
$output = curl_exec($ch);
curl_close($ch);
return $output;
}
$html=login('https://www.domain.com/users/ajaxonlogin.php',
'username=myemailaddress&passwd=mypassword');
echo $html;
I am able to get the following response:
{"status":"success","goto_url":"https:\/\/www.domain.com\/dashboard\/"}
I have tried adding to the bottom of the login function:
$url = "http://www.domain.com/page-that-i-want-but-requires-log-in.php";
$results_page = curl($url);
echo $results_page;
But I only receive the page as if I was NOT logged in...
Any pointers please?
Thanks.
Bruce
I have a problem to get the data using curl operation. Here i hide the token, If i use the url only in my browser then it returns the data but here its null.
<?php
$token = "TOKEN"; //the actual token hidden
$url = "https://crm.zoho.com/crm/private/xml/Leads/getRecords?authtoken=".$token."&scope=crmapi";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0);
$result = curl_exec($ch);
curl_close($ch);
echo $result; //does not return anything
?>
Where i do mistake please help me.
This is how you can try with CURLOPT_RETURNTRANSFER which is used to return the output and curl_errno() to track the errors :
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://example.com/my_url.php" );
curl_setopt($ch, CURLOPT_POST, 1 );
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_data);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$postResult = curl_exec($ch);
if (curl_errno($ch)) {
print curl_error($ch);
}
curl_close($ch);
Helpful Links: curl_errno(), curl_error()
Try adding these lines:
<?php
$token = "TOKEN"; //the actual token hidden
$url = "https://crm.zoho.com/crm/private/xml/Leads/getRecords";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, array("authtoken"=>$token,"scope"=>"crmapi"));
$result = curl_exec($ch);
curl_close($ch);
echo $result; //does not return anything
?>
please see below article , not just turn off verify , you should update your php.ini with pem file
https://snippets.webaware.com.au/howto/stop-turning-off-curlopt_ssl_verifypeer-and-fix-your-php-config/
Anyone running a recent Linux distribution is probably already OK, because they get the curl libraries bundled in through their package managers and recent curl libraries come with the latest CA root certificate bundle from Mozilla.org.
For a PHP installation that doesn’t come with this file, like the Windows PHP distribution, you need to download the CA root certificate bundle and tell PHP where to find it.
I'm learning regex, php & cUrl and would like to get Google Image html
(for example : https://www.google.fr/search?q=terrorist&biw=1920&bih=1008&source=lnms&tbm=isch&gws_rd=ssl#tbm=isch&q=love)
I tried a lot of different answers but I don't understand why, surprisingly, when I do
<?php
function curl_get_contents($url)
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
curl_setopt($ch, CURLOPT_COOKIEJAR, "cookie.txt");
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($conn2, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($conn2, CURLOPT_SSL_VERIFYHOST, false);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
$get_page = curl_get_contents("https://www.google.fr/search?q=terrorist&biw=1920&bih=1008&source=lnms&tbm=isch&gws_rd=ssl#tbm=isch&q=love");
echo $get_page;
?>
I've got a very different result than in browser. All images links are dead for example.
Does anyone know why ? And what could I do to fix it?
Thanx a lot !!
There are 2 search queries in this request as shown below
https://www.google.fr/search?q=terrorist&biw=1920&bih=1008&source=lnms&tbm=isch&gws_rd=ssl#tbm=isch&q=love
try
https://www.google.fr/search?q=love&biw=1920&bih=1008&source=lnms&tbm=isch&gws_rd=ssl#tbm=isch
It looks like its returning the first query in the browser and then running the second, but it wont do the second request via curl.
This works for me:
<?php
function curl_get_contents($url)
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
curl_setopt($ch, CURLOPT_COOKIEJAR, "cookie.txt");
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
$data = curl_exec($ch);
var_dump(curl_error($ch));
curl_close($ch);
return $data;
}
$get_page = curl_get_contents("https://www.google.fr/search?q=love&biw=1920&bih=1008&source=lnms&tbm=isch&gws_rd=ssl#tbm=isch");
echo $get_page;
?>
EDIT: After further research this is an unsupported way of doing this, You should be using Google Custom Search API. The way that you are doing it will result in Google detecting misuse and presenting you with Captcha requests or maybe even blocking you.
I'm trying to get content from bank site using curl.
http://www.zaba.hr/home/wps/wcm/connect/zaba_hr/zabapublic/tecajna
Site is specific becouse it using ajax to fill currency exchange table. There is a link for download data in to file but you have to have same session id to able to do that.
Im trying this code:
$url="http://www.zaba.hr/home/wps/wcm/connect/zaba_hr/zabapublic/tecajna";
$useragent = $_SERVER['HTTP_USER_AGENT'];
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, $useragent);
curl_setopt($ch, CURLOPT_URL,$url);
$cl = curl_exec($ch);
$dom = new DOMDocument();
#$dom->loadHTML($cl);
#$link = $dom->getElementById('tecajPrn');
echo $suburl = "http://www.zaba.hr".$link->getAttribute('href');
After this I got link to file but I can't open it.
Another strange situation is that link I got with curl is http://www.zaba.hr/home/ZabaUtilsWeb/utils/tecaj/danasPrn but real link when I click on icon is http://www.zaba.hr/ZabaUtilsWeb/utils/tecaj/prn/62/2014
You are messing with cookie and ajax(may be!). Here is the lookaround. Try this:
First send a request to the page to obtain the cookie.
$url="http://www.zaba.hr/home/wps/wcm/connect/zaba_hr/zabapublic/tecajna";
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, "mozilla 5.0");
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_COOKIEFILE,"cookie.txt");
curl_setopt($ch, CURLOPT_COOKIEJAR,"cookie.txt");
$cl = curl_exec($ch);
curl_close($ch);
After that make another curl request. This time to obtain the json data:
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, "mozilla 5.0");
curl_setopt($ch, CURLOPT_HTTPHEADER, array("X-Requested-With: XMLHttpRequest", "Referer: http://www.zaba.hr/home/wps/wcm/connect/zaba_hr/zabapublic/tecajna"));
curl_setopt($ch, CURLOPT_URL,"http://www.zaba.hr/ZabaUtilsWeb/utils/tecaj/danas");
curl_setopt($ch, CURLOPT_COOKIEFILE,"cookie.txt");
curl_setopt($ch, CURLOPT_COOKIEJAR,"cookie.txt");
$cl = curl_exec($ch);
curl_close($ch);
Your json is available at this variable. Parse it using json_decode()
// now parse json from $cl
print $cl;
Anything required, help yourself straightway!
Note: Make sure you have write permission for the cookie.txt file. Also, its better to use absolute path like c:/test/cookie.txt or /var/tmp/cookie.txt.
I am working on core php.
I am requesting a URL by curl. This is working on my development server but same thing are not working on live server.
Below is my Code:
$url = "http://www.streamatemodels.com/login.php";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 15);
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
curl_setopt($ch, CURLOPT_TIMEOUT, 15);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, 'submitted=1&g=&sausr='.urlencode('username').'&sapwd='.urlencode('password').'');
$result = curl_exec($ch);
$error = curl_errno($ch);
$code = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);
print_r($result);
Please help me if anyone have solution.
Update
I checked this with new username and password but same things happen. Its working on my development server but not working on live and local host.
The following line:
curl_setopt($ch, CURLOPT_POSTFIELDS, 'submitted=1&g=&sausr='.urlencode('username').'&sapwd='.urlencode('password').'');
should be changed to:
curl_setopt($ch, CURLOPT_POSTFIELDS, 'submitted=1&g=&sausr='.urlencode($username).'&sapwd='.urlencode($password).'');
If it still doesn't work try to create the query-string outside of curl_setopt:
$params = 'submitted=1&g=&sausr='.urlencode($username).'&sapwd='.urlencode($password).'';
curl_setopt($ch, CURLOPT_POSTFIELDS, $params);
Another advise: it's better practice to use http_build_query($params)
UPDATE
I had the time to check the code, and it works fine on my local host (meaning: check your username/password!):