Why is this curl code not returning any thing as it was working while I tried for my local testing site.
public function test_curl()
{
//step1
$cSession = curl_init();
//step2
curl_setopt($cSession,CURLOPT_URL,"https://www.google.com/search?q=hehe");
curl_setopt($cSession,CURLOPT_RETURNTRANSFER,true);
//step3
$result=curl_exec($cSession);
//step4
curl_close($cSession);
//step5
echo $result;
}
This is what I use:
$URL = "https://www.google.com/search?q=hehe";
$agent= 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)';
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
curl_setopt($ch, CURLOPT_URL, $URL);
$result =curl_exec($ch);
echo $result;
It works on sites that have blocked file_get_contents. Should do the trick!
Related
I am trying to get string from external website but i am getting 403 error. I have tried using custom user agent and same error. Can you please check and help me to get it fixed.
Code :
$agent= 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)';
function file_get_contents_curl($url) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_AUTOREFERER, TRUE);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
curl_setopt($ch, CURLOPT_VERBOSE, true);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
$homepage = file_get_contents_curl("https://www.transunion.com/");
echo $homepage;
Help me fix this.
i want to get web page of this url "http://namnak.com" in my server but this url blocked curl request from my server this is my code :
<?php
$proxy = "138.68.173.29 :8080";
$proxy = explode(':', $proxy);
$url = "http://namnak.com";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)');
curl_setopt($ch, CURLOPT_PROXY, $proxy[0]);
curl_setopt($ch, CURLOPT_PROXYPORT, $proxy[1]);
curl_setopt($ch, CURLOPT_HEADER, 1);
$exec = curl_exec($ch);
echo curl_error($ch);
print_r(curl_getinfo($ch));
echo $exec;
You must define the type of proxy you want to use, for example, SOCKS5:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT
5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)');
curl_setopt($ch, CURLOPT_PROXY, $proxy[0]);
curl_setopt($ch, CURLOPT_PROXYPORT, $proxy[1]);
curl_setopt($ch, CURLOPT_PROXYTYPE, CURL_SOCKS5);
curl_setopt($ch, CURLOPT_HEADER, 1);
Start by trying that and we'll figure it out from there!
EDIT: In some cases, SOCKS5 isn't defined in curl, you can use its value which is equal to 7
I´m trying to get google trends data by parsing this csv file:
https://trends.google.com/trends/api/widgetdata/comparedgeo/csv?req={'geo':{'country':'US'},'comparisonItem':[{'time':'2017-10-06 2018-10-06','complexKeywordsRestriction':{'keyword':[{'type':'BROAD','value':'travel'}]}}],'resolution':'REGION','locale':'de','requestOptions':{'property':'','backend':'IZG','category':0}}&token=APP6_UEAAAAAW7o-Y1_D87AoOXJJqulrVGiPmc3Cz6_Z&tz=-120
I tried it with php and curl, doesn´t work. I get an error page of google. Telling me "Bad Request". Alternatively I could download it with a cronjob but also this doesn´t work for me. Any chance to get/download this file?
<?php
ini_set('display_errors', 1);
ini_set('display_startup_errors', 1);
error_reporting(E_ALL);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://trends.google.com/trends/api/widgetdata/comparedgeo/csv?req={'geo':{'country':'US'},'comparisonItem':[{'time':'2017-10-06 2018-10-06','complexKeywordsRestriction':{'keyword':[{'type':'BROAD','value':'travel'}]}}],'resolution':'REGION','locale':'de','requestOptions':{'property':'','backend':'IZG','category':0}}&token=APP6_UEAAAAAW7o-Y1_D87AoOXJJqulrVGiPmc3Cz6_Z&tz=-120");
$agent= 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)';
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 15); //time out of 15 seconds
$output = curl_exec($ch);
curl_close($ch);
?>
It's because there's a literal space in your URL. Here's a working example:
<?php
ini_set('display_errors', 1);
ini_set('display_startup_errors', 1);
error_reporting(E_ALL);
$ch = curl_init();
$url = "https://trends.google.com/trends/api/widgetdata/comparedgeo/csv?req={'geo':{'country':'US'},'comparisonItem':[{'time':'2017-10-06 2018-10-06','complexKeywordsRestriction':{'keyword':[{'type':'BROAD','value':'travel'}]}}],'resolution':'REGION','locale':'de','requestOptions':{'property':'','backend':'IZG','category':0}}&token=APP6_UEAAAAAW7o-Y1_D87AoOXJJqulrVGiPmc3Cz6_Z&tz=-120";
$url = str_replace(" ", '%20', $url);
curl_setopt($ch, CURLOPT_URL, $url);
$agent= 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)';
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 15); //time out of 15 seconds
$output = curl_exec($ch);
curl_close($ch);
print_r($output);
?>
But you really don't need to use CURL here:
<?php
$url = "https://trends.google.com/trends/api/widgetdata/comparedgeo/csv?req={'geo':{'country':'US'},'comparisonItem':[{'time':'2017-10-06 2018-10-06','complexKeywordsRestriction':{'keyword':[{'type':'BROAD','value':'travel'}]}}],'resolution':'REGION','locale':'de','requestOptions':{'property':'','backend':'IZG','category':0}}&token=APP6_UEAAAAAW7o-Y1_D87AoOXJJqulrVGiPmc3Cz6_Z&tz=-120";
$url = str_replace(" ", '%20', $url);
$result = file_get_contents($url);
print_r($result);
?>
i am trying to get web page content with curl from some websites but they return 400 bad request ( file_get_contents return empty ) here's the function i am using :
function file_get_contents_curl($url) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HTTPGET, TRUE);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)');
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
Put error_reporting(E_ALL); line at the top file where you are calling this function.
It will generate the cause of an error.
Hello i have problem with loading page via curl. Page html looks like this
<!--showModel--><!DOCTYPE html>
<html>
<head>
...
and then curl load page i get only
<!--showModel-->
how can i load full page?
php code looks like this
$agent = 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)';
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
curl_setopt($ch, CURLOPT_URL, $link);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
$response = curl_exec($ch);