I use curl in php to get data from a website and add to my mysql database.
But the string I get is formatted strange. i've tried some method but didn't help me. who used to deal with it please give me your solution.
My curl method here:
function grab_page($site){
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
curl_setopt($ch, CURLOPT_TIMEOUT, 40);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
curl_setopt($ch, CURLOPT_URL, $site);
ob_start();
return curl_exec($ch);
ob_end_clean();
curl_close ($ch);
}
After get it, i echo the html, my method here:
function getDetailPage(){
$detailData = grab_page("https://www.deliverynow.vn/ho-chi-minh/hanuri-quan-an-han-quoc-xo-viet-nghe-tinh");
echo htmlspecialchars($detailData);
}
The html string i got is change to weird character like: Món khác . But it should be like this: Món khác
Use html_entity_decode to restore characters.
Related
I am calling three methods which executes xml using curl
$clientID=$newInvoice->reateClient($name,$organization,$email,$address,$country);
$invoiceID=$newInvoice->CreateInvoice($clientID,$invoiceNumber,$itemName,$cost,$quantity);
echo $newInvoice->SendInvoice($invoiceID);
when I use these all three method then first executes properly but second and third shows error that : String could not be parsed as XML
Its is working fine if I call each method one at a time by commenting other two method. Can anyone please help
php code sample($xml_request contains xml and it seems ok because I can execute without any error if I execute one method at a time by commenting any other two ):
$ch = curl_init($api_url);
curl_setopt($ch, CURLOPT_USERPWD, $token.':123');
curl_setopt($ch, CURLOPT_POSTFIELDS, 'xml_request='.$xml_request);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_TIMEOUT, 240);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_USERAGENT, "Invoicera API Test 1.0");
$curl_result = curl_exec($ch);
curl_close ($ch);
return $curl_result;
I'm learning regex, php & cUrl and would like to get Google Image html
(for example : https://www.google.fr/search?q=terrorist&biw=1920&bih=1008&source=lnms&tbm=isch&gws_rd=ssl#tbm=isch&q=love)
I tried a lot of different answers but I don't understand why, surprisingly, when I do
<?php
function curl_get_contents($url)
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
curl_setopt($ch, CURLOPT_COOKIEJAR, "cookie.txt");
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($conn2, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($conn2, CURLOPT_SSL_VERIFYHOST, false);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
$get_page = curl_get_contents("https://www.google.fr/search?q=terrorist&biw=1920&bih=1008&source=lnms&tbm=isch&gws_rd=ssl#tbm=isch&q=love");
echo $get_page;
?>
I've got a very different result than in browser. All images links are dead for example.
Does anyone know why ? And what could I do to fix it?
Thanx a lot !!
There are 2 search queries in this request as shown below
https://www.google.fr/search?q=terrorist&biw=1920&bih=1008&source=lnms&tbm=isch&gws_rd=ssl#tbm=isch&q=love
try
https://www.google.fr/search?q=love&biw=1920&bih=1008&source=lnms&tbm=isch&gws_rd=ssl#tbm=isch
It looks like its returning the first query in the browser and then running the second, but it wont do the second request via curl.
This works for me:
<?php
function curl_get_contents($url)
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
curl_setopt($ch, CURLOPT_COOKIEJAR, "cookie.txt");
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
$data = curl_exec($ch);
var_dump(curl_error($ch));
curl_close($ch);
return $data;
}
$get_page = curl_get_contents("https://www.google.fr/search?q=love&biw=1920&bih=1008&source=lnms&tbm=isch&gws_rd=ssl#tbm=isch");
echo $get_page;
?>
EDIT: After further research this is an unsupported way of doing this, You should be using Google Custom Search API. The way that you are doing it will result in Google detecting misuse and presenting you with Captcha requests or maybe even blocking you.
I am trying to decode the JSON at https://api.twitter.com/1/statuses/user_timeline.json?include_entities=true&incslude_rts=0&screen_name=microsoft&count=200&exclude_replies=1&contributor_details=0 with json_decode() in PHP (decoding as an associative array, so the second parameter is set to TRUE.
The problem is that it seems to not do anything (no error, warning either). The data contains 200 tweets + some extra data about them. If I fetch only let's say 50 tweets, the json_decode function runs successfully.
So my question is: is json_decode not able to decode large strings?
Edit: my code:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url); // $url is the above mentioned URL
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST,0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER,0);
$content = curl_exec($ch);
print_r(json_decode($content,true));
There is something wrong with your cURL I suppose. I just tried the following code with the URL you have provided and works just fine:
$curl = curl_init();
$url = 'https://api.twitter.com/1/statuses/user_timeline.json?include_entities=true&incslude_rts=0&screen_name=microsoft&count=200&exclude_replies=1&contributor_details=0';
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_URL, $url);
$content = curl_exec($curl);
var_dump(json_decode($content, true));
Why am I not getting the HTML code when I use PHP curl? This is my code:
// $content = file_get_contents('http://www.datadiary.com/Company/311734/dimsinstituteofhotelmanagement');
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://www.datadiary.com/Company/311734/dimsinstituteofhotelmanagement');
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_COOKIESESSION, false);
$content = curl_exec($ch);
curl_close($ch);
echo $content;
By default curl_exec sends the response to the output (usually the browser). Set the CURLOPT_RETURNTRANSFER option if you want curl_exec to return the result instead:
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
Relevant manual entries:
http://php.net/curl-exec
http://php.net/curl-setopt
You need to add the option to return the transfer as a string:
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
How do I hide the output from curl in PHP?
My code as it stands is the following:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_USERPWD, $pass);
$result = curl_exec($ch);
curl_close ($ch);
The problem is that is spews out the entire page to the user. Instead I want to capture the output and simply show a "success" or "failed" message?
Use this option to curl_setopt():
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
This will make curl_exec return the data instead of outputting it.
To see if it was successful you can then check $result and also curl_error().
Also make sure to turn off this option:
curl_setopt($ch, CURLOPT_VERBOSE, 0);
Or else it will still print everything to screen.