I am trying to make a cURL request. The problem I am facing is that the page have different text depending on which country it is. So I would like the cURL request to have the language en_US (English). So it will get the English text on the website.
Currently I have this code, but its not getting the US text.
$url = 'http://testurl.com'; // Not the real URL
$options = array(
CURLOPT_RETURNTRANSFER => true, // return web page
CURLOPT_HEADER => false, // don't return headers
CURLOPT_FOLLOWLOCATION => true, // follow redirects
CURLOPT_HTTPHEADER => array("Accept-Language: en-US;q=0.6,en;q=0.4"),
CURLOPT_USERAGENT => Array("User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.15) Gecko/20080623 Firefox/2.0.0.15"),
);
$ch = curl_init($url);
curl_setopt_array($ch, $options);
$content = curl_exec($ch);
$err = curl_errno($ch);
$errmsg = curl_error($ch);
$header = curl_getinfo($ch);
curl_close($ch);
echo htmlspecialchars($content);
So to make this simple, I would like the cURL request to send the request with the US language, if possible.
Right now it has the language 'dutch' I think this is because my hosting server is located in Netherlands. So therefore it is deutch. But I would like to change it to English.
Related
I am trying to fetch a response from here (example url), and first, I thought I should use file_get_contents()
When I tried this, I got the following error:
Warning: file_get_contents(https://steamcommunity.com/market/pricehistory/?country=US¤cy=1&appid=730&market_hash_name=SG%20553%20|%20Damascus%20Steel%20(Factory%20New)): failed to open stream: HTTP request failed! HTTP/1.0 400 Bad Request
I know this is because it is converting & to &. I have tried numerous ways to counter this, however they have all failed and after a quick google I came to the conclusion that file_get_contents() converts & to & automatically.
My next step was to try curl. I tried the below code first:
// Get cURL resource
$curl = curl_init();
// Set some options - we are passing in a useragent too here
curl_setopt_array($curl, array(
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_URL => 'http://steamcommunity.com/market/pricehistory/?country=US¤cy=1&appid=730&market_hash_name='.$hash,
CURLOPT_USERAGENT => 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.2 (KHTML, like Gecko) ChromePlus/4.0.222.3 Chrome/4.0.222.3 Safari/532.2'
));
// Send the request & save response to $resp
$resp = curl_exec($curl);
// Close request to clear up some resources
curl_close($curl);
But this returned ‹ŠŽÿÿ)»L as the response. I wondered if this was to do with json encoding, so I tried putting it through json_decode() but it didn't work.
Next, I tried:
// Get cURL resource
$curl = curl_init();
// Set some options - we are passing in a useragent too here
curl_setopt_array($curl, array(
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_URL => 'http://steamcommunity.com/market/pricehistory/',
CURLOPT_USERAGENT => 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.2 (KHTML, like Gecko) ChromePlus/4.0.222.3 Chrome/4.0.222.3 Safari/532.2',
CURLOPT_POST => 1,
CURLOPT_POSTFIELDS => array(
country => "US",
currency => 1,
appid => 730,
market_hash_name => "SG%20553%20|%20Damascus%20Steel%20(Factory%20New)"
)
));
// Send the request & save response to $resp
$resp = curl_exec($curl);
// Close request to clear up some resources
curl_close($curl);
But again got the response ‹ŠŽÿÿ)»L.
What does this response mean, and can I parse it? If not, how should I correctly fetch this data? Furthermore, why didn't file_get_contents() work?
I'm pretty sure this is happening because you need some type of access token to access the steam web API.
See this answer on SO.
Essentially, Steam is returning an error with the "400 Bad Request" status. This error can be ignored, however, by doing this:
<?php
$url = "https://steamcommunity.com/market/pricehistory/?country=US¤cy=1&appid=730&market_hash_name=SG%20553%20%7C%20Damascus%20Steel%20(Factory%20New)";
$context = stream_context_create(array(
'http' => array(
'ignore_errors'=>true,
'method'=>'GET'
// for more options check http://www.php.net/manual/en/context.http.php
)
));
$response = file_get_contents($url, false, $context);
echo $response; // returns "[]"
?>
Make sure you take a look at this answer on SO.
May be your response is gzip, try to use CURLOPT_ENCODING.
curl_setopt($curl ,CURLOPT_ENCODING, '')
If you use https don't forget to disable CURLOPT_SSL_VERIFYPEER.
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false)
One thing, if I follow your link with my browser and open my debug console.
I see you request have a 400 Status code (Bad Request).
I cant say about your enpoint, but you can get around your Bad Request error by using urlencode():
$url = urlencode('https://steamcommunity.com/market/pricehistory/?country=US¤cy=1&appid=730&market_hash_name=SG%20553%20%7C%20Damascus%20Steel%20(Factory%20New))'
file_get_contencts($url);
I do a php curl to this website
http://www.hoovers.com/company-information/company-search.html
But it returned 404. Looks like something encrypted or what.
Can you give some clue about this problem.
Thanks
// Get cURL resource
$curl = curl_init();
// Set some options - we are passing in a useragent too here
curl_setopt_array($curl, array(
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_URL => 'http://www.hoovers.com/company-information/company-search.html',
));
// Send the request & save response to $resp
$resp = curl_exec($curl);
// Close request to clear up some resources
curl_close($curl);
Looks like their web server is rejecting requests based on HTTP headers. Or it might be on the application level as well. Try this
<?php
// Get cURL resource
$curl = curl_init();
// Set some options - we are passing in a useragent too here
curl_setopt_array($curl, array(
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_HEADER=>1,
CURLOPT_URL => 'http://www.hoovers.com/company-information/company-search.html',
CURLOPT_HTTPHEADER=> array(
'User-Agent: Mozilla/5.0 (Windows NT 6.3; rv:36.0) Gecko/20100101 Firefox/36.0', 'Accept-Language: en-US,en;q=0.5'
)
));
// Send the request & save response to $resp
$resp = curl_exec($curl);
// Close request to clear up some resources
curl_close($curl);
//debug
print_r($resp);
?>
I have read many question regarding the title. Basically I'm using combination of getheader and curl to check wether a url is exist.
$url = "http://www.asdkkk.com";
$headers = get_headers($url);
if(strpos($headers[0],'404') === false){
$ch = curl_init($url);
curl_setopt_array($ch,array(
CURLOPT_HEADER => true,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_FOLLOWLOCATION => true,
CURLOPT_SSL_VERIFYPEER => false,
CURLOPT_HTTPHEADER => array("Accept-Language: en-US;q=0.6,en;q=0.4"),
CURLOPT_USERAGENT => 'Mozilla/5.0 (Windows NT 5.1) AppleWebKit/535.6 (KHTML, like Gecko) Chrome/16.0.897.0 Safari/535.6'
));
$data = curl_exec($ch);
$httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
if($httpCode != 404){
curl_close($ch);
return $data;
}
}else{
echo "URL Not Exists";
}
Both function will return status code 200 for the url("http://www.asdkkk.com"). In the url is a page not found website. But it seem like it is hosted and the header of the page doesn't set to 404. I have try out not only this url but others too. So how can I determine a URL is actually existence in a very accurate way?
I think the issue with your example code is you are confusing a 404 HTTP response code for 'Not Found' from a server with the case of a URL that doesn't point to any server at all. If there's no server response at all, cURL will return '0' as the HTTP response, rather than 404. Try running the below code and see if it works for your purposes:
$urls = array(
"http://www.asdkkk.com",
"http://www.google.com/cantfindthisurl",
"http://www.google.com",
);
$ch = curl_init();
foreach($urls as $url){
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_exec($ch);
$http_status = curl_getinfo($ch, CURLINFO_HTTP_CODE);
echo "$http_status for $url <br>";
}
I have created an internal billing system where i need to generate invoices for a customer based on their billing schedule however i have run into a problem when running PHP scripts from CURL and was wondering if there is any way round it
I currently have a CRON task that runs a php script called crontask.php
crontask.php then calculates if the customer needs an invoice generated and sent to them via email. If it calculates that it does then it will try and call an url that will create the Invoice and send the email using CURL i.e (www.internal.co.uk/invoicing/geninvoice.php?CUST=10)
function get_web_page($url)
{
$ua = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.A.B.C Safari/525.13';
echo "curl:url<pre>".$url."</pre><BR>";
$options = array(
CURLOPT_RETURNTRANSFER => true, // return web page
CURLOPT_HEADER => true, // don't return headers
CURLOPT_FOLLOWLOCATION => true, // follow redirects
CURLOPT_ENCODING => "", // handle all encodings
CURLOPT_USERAGENT => $ua, // who am i
CURLOPT_AUTOREFERER => true, // set referer on redirect
CURLOPT_CONNECTTIMEOUT => 15, // timeout on connect
CURLOPT_TIMEOUT => 15, // timeout on response
CURLOPT_MAXREDIRS => 10, // stop after 10 redirects
);
$ch = curl_init($url);
curl_setopt_array( $ch, $options );
$content = curl_exec( $ch );
$err = curl_errno( $ch );
$errmsg = curl_error( $ch );
$header = curl_getinfo( $ch,CURLINFO_EFFECTIVE_URL );
curl_close( $ch );
if(isset($header['errno'])) {
$header['errno'] = $err;
$header['errmsg'] = $errmsg;
}
//change errmsg here to errno
if (isset($errno)) {
echo "CURL:".$errmsg."<BR>";
}
return $content;
}
When running this i am getting access denied when trying to run from curl in PHP,
The server is running on virtualmin/webmin and i have root access, is there something i need to change or add authentication to the script?
I'm constructing this URL with PHP and getting the result from Flickr with CURL.
http://api.flickr.com/services/rest?api_key=APIKEY&format=php_serial&method=flickr.photosets.getPhotos&photoset_id=72157594403088940&per_page=200&extras=description,url_l,url_c,url_z,url_m,url_n,url_s,url_t
There is a real API key there of course. Anyway it sometimes returns bool(false), sometimes the proper list of images. Usually it's like first time check on a given day returns false, then after a refresh it gets the list properly. My CURL function I use to get the result:
function file_get_contents_curl($url, $curlopt = array()){
if(in_array('curl', get_loaded_extensions())){
$ch = curl_init();
$default_curlopt = array(
CURLOPT_TIMEOUT => 2,
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_FOLLOWLOCATION => false,
CURLOPT_USERAGENT => "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 AlexaToolbar/alxf-1.54 Firefox/3.6.13 GTB7.1"
);
$curlopt = array(CURLOPT_URL => $url) + $curlopt + $default_curlopt;
curl_setopt_array($ch, $curlopt);
$response = curl_exec($ch);
if($response === false)
trigger_error(curl_error($ch));
curl_close($ch);
return $response;
}else{
return file_get_contents($url);
}
}
What is this and why does it happen? Maybe it has something to do with my CURL function (my best bet)?
You have set the CURLOPT_TIMEOUT to 2 seconds. Can you verify if the code times out before the execution completes. If so, try increasing it a bit and see if it works.
Just a guess.