I am not getting readable format data using CURL?

I am not getting readable format data using CURL? - php

Below is my code
$url = 'http://api.walmartlabs.com/v1/feeds/bestsellers?apikey=xxxxx';
$ch = curl_init();
curl_setopt($ch, CURLOPT_AUTOREFERER, TRUE);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
'Content-Type: application/xml',
'Connection: Keep-Alive'
));
//curl_setopt($ch, CURLOPT_RETURNTRANSFER, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)");
$data = curl_exec($ch);
curl_close($ch);
print_r($data);
exit;
my output is ....
��ks�Ȓ&�}~EE�������;N���ִdkM�=gg6� ��(��;����*��fQ���'�E#�*+++�����!k����`��_�o���f�g�f8碒Es|��k��B�$����_�����|v6-���*�����\=h� ���<���5ƁC�Ƣ�e�<͔�� ��e����Ue�����W��zZV́��*�7YYP#?�ͯ�/�_�+ /`yY\�\�MV\����U&k�L0z��Y�(V���f�]e͔5Ӭf�#v)��JJz�8f�SYK6��Hd�.r��敬k�E�ʂ�b���K�mE?UЦhX�]Y����Le�=Y"%�,�ضh~�Y$c���P?L>%R��Ǣl~e���?���h{C���Bu��i 4g)����岻�1Q0�X΀��5+.�F4�Q$��ٟ�%t��G�ihg�ț[��|�+��iO� �W�^�$�fS��B�Y�d�j[VK��1�M��4�����d~�IQ���+������X�pϘT� A���eU�r�<��)Q ���o�;�B�-�r��G�F �}��K|����/̌���,A�0)�%�Kl��^��z�N�G����c��"ޡU�V:.⡅�-���O(������Z���<�O�ט~mMУ������7�y�΃kt��-���lo��nPd)�����foJ��׸�Z�;j��D����v�z)�KY�.5��,*����t}�4������_�|&���T��r�#�j�LL�tc�{Q�s�"xn�m+ ���,+�?Xc�NK�q'�/�����3,ב�H��s./�^&�+ Tk����}�4������R�O�2������jW���梺�_�����S��ZO�J�8�D���*�zγ�c���X�=�>�5��b'�a��{�����L^��K\��ݲl÷Ǿ�,��o��#�:p%��^D���q ��fs�h���ۃg�oOͿ����ڻ��uD�\]]�v�~���Gibd'%���SW�Z�m�4���������#��=���u�e1�y`��f9��u��4�ρLH�T�T�)�����B)w��n?/' ��)�ɦ��2��kX��R*�ۗ����6a�&*S��&�فaFT�=��Dj-oR�踉�= [ ���v=�n�å���4��W�6���`�K�"_v�m���0f��X��6���`�cz�ѢH�k��s)#�]';h�o�DQoe2�2�H��mwoղh�*����*�����jy*�i������)D��_�dA�gz���Rm'�y d|�|u�u�V� ��/�����"w���jz�t����mw�ؾ�и&3H��sWɇ�A���� "�-";�E�� �k�9��:_v��(G��T�/�5�Qq�^C�ՋD}]62*ˏ#�\�]�21���%�Px8E����?�D��X#��P�w<#����.vz��,��}v���|�+�2�:fg�X�j#���V��:�o9V"�6��x^k�;�`m��E���ӟO��T��/����T��ʸD��;��{�Q}�-��{Ѩ�� ��eY�i���D����Y����W�=�Œ�Yx�~��&}�.�:�ޮ�����Y�1��rЉ�!Y��\���h� ,���d�kDV���[V�#k}�>��t�������E$G�l�<�4b�~?�}�vg�ϲ`�n� {�%�b*P�WLR�/C�ݞ�nG}+:��(��x�sД��<�`#X)E�S��#ۅ�q�UU�m���,�XR�02� �&�d���دV��^x]^���.%ި4��F4�-�1;n��{��Au歏_�#I ?��S�.|���� �Rv S�I��Z���L�)ș�O� 4R�`G8�Y���$����|¬��ݵ7�ޚȸ�,���.�.�k���� pB���S�k���H��0v��-C�ش�莇{�U�n��ߩe7b�E��o��ul�1��E��8y��o��M-����6�2kd<-#W�X���o�UZ��2U4�-�����h��K)������0D��r6i�F�U����C�=6�\>��⃏� ��مZyD�fb�Ǖ�ޏF�*�x���A�c��5st���xl.|�s/�"��#< \�[��ؾ�!��A3K�$��F6�dbٞ�$"6��������c�-��������4�2����#w�z�s#��?���{��?>��;8�X�alJ�A���������C���l}pn��v���=���N��_2w3�2?2�ȣ/
please help me to solve this issue...

Try this:
XML is now deprecated and JSON is the supported format. The api returns data gzipped. So, you will have to decode the response appropriately.
$url = 'http://api.walmartlabs.com/v1/feeds/bestsellers?apikey=<apiKey>&format=json';
$ch = curl_init();
curl_setopt($ch, CURLOPT_AUTOREFERER, TRUE);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
'Content-Type: application/json; charset=utf-8',
'Content-Encoding:gzip'
));
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
$data = curl_exec($ch);
curl_close($ch);
print_r(gzdecode($data)); // un-gzip before using it
Hope this helps.

Related

Http 411 php curl due to post request

With this code:
<?php
function request($url, $post, $cook)
{
$ch = curl_init();
$cookie_file_path = "vskcookies.txt";
$urll = 'http://auto.vsk.ru/login.aspx';
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_NOBODY, false);
curl_setopt($ch, CURLOPT_URL, $urll);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file_path);
curl_setopt($ch, CURLOPT_USERAGENT,'"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 10.0; Trident/7.0; Touch; .NET4.0C; .NET4.0E; Tablet PC 2.0)"');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, $urll);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
curl_setopt( $ch, CURLOPT_COOKIESESSION, true );
$result = curl_exec($ch);
echo "FIELDS:\n\n".$post;
echo "\n\nHEADERS:\n\n";
curl_close($ch);
return $result;
}
$result = request($_POST['url'], $_POST['data'], $_POST['cook']);
if ($result === FALSE)
echo('error');
else
echo($result);
?>
I am getting two cookies as I need but with http 411 error in body:
The same request, the same code, but in the end, right before curl_exec I add
curl_setopt($ch, CURLOPT_HTTPHEADER, array("Content-Type: application/x-www-form- urlencoded;charset=utf-8"));
As the result, I am getting correct body, but now only one cookie (I need both):
Another variants:
This code
curl_setopt($ch, CURLOPT_FRESH_CONNECT, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Cache-Control: no-cache;Content-Type: application/x-www-form- urlencoded;charset=utf-8;Content-Length: '.strlen($post)));
Gives 411 error and correct cookies for reason.
This
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Length:'.strlen($post)));
Causes nothing: just nothing happens.
Also after both (first)variants, vskcookies.txt contains only one (pool) cookie.
Why that?
It looks like, when I add header it erases request’s body(post fields).
UPDATE
For this code
curl_setopt($ch, CURLOPT_FRESH_CONNECT, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Cache-Control: no-cache;Content-Type: application/x-www-form- urlencoded;'));
I just sitting and clicking on button that makes ajax request of the script above on my page, getting every time 411 error response (also cookies/time in header updates every time), but after about 10 clicks I got http 200 and the page I need. Then again many times 411 and then again a single 200. By the way cookies file still has only one cookie.
Wtf?

As I mentioned in question’s update, sometimes I get 200 response with two cookies and body I need.
So I wrote this is bad unstable solution, often it can remain about 20 seconds:
<?php
function request($url, $post, $cook)
{
$sta=0;
while($sta != '200')
{
$ch = curl_init();
$cookie_file_path = "vskcookies.txt";
$urll = 'http://auto.vsk.ru/login.aspx';
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_NOBODY, false);
curl_setopt($ch, CURLOPT_URL, $urll);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file_path);
curl_setopt($ch, CURLOPT_USERAGENT,'"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 10.0; Trident/7.0; Touch; .NET4.0C; .NET4.0E; Tablet PC 2.0)"');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, $urll);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
curl_setopt($ch, CURLOPT_MAXREDIRS, 10);
curl_setopt( $ch, CURLOPT_COOKIESESSION, true );
//curl_setopt($ch, CURLOPT_FRESH_CONNECT, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Cache-Control: no-cache;Content-Type: application/x-www-form- urlencoded;'));
//curl_setopt($ch, CURLOPT_POSTREDIR, 2);
$result = curl_exec($ch);
$sta = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);
}
echo "FIELDS:\n\n".$post;
echo "\n\nHEADERS:\n\n";
return $result;
}
$result = request($_POST['url'], $_POST['data'], $_POST['cook']);
if ($result === FALSE)
echo('error');
else
echo($result);
?>
Bet there is a much better correct answer.

How to pass custom headers to cURL?

Sorry for my English. Tell me how to pass non-standard headers in cURL
$headers = array("Accept: application/json, text/plain, /*",
"Captcha: NDRMR9",
"Referer: https://avtokod.mos.ru/AutoHistory/");
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://avtokod.mos.ru/AutoHistory/GetByLicensePlate");
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query(array('licensePlate' => $licensePlate, 'sts' => $sts)));
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLINFO_HEADER_OUT, true);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:61.0) Gecko/20100101 Firefox/61.0');
curl_setopt($ch, CURLOPT_COOKIEFILE, $_SERVER["DOCUMENT_ROOT"]."/cookie.txt");
curl_setopt($ch, CURLOPT_COOKIEJAR, $_SERVER["DOCUMENT_ROOT"]."/cookie.txt");
$response = curl_exec($ch);
$info = curl_getinfo($ch);
curl_close($ch);
echo "<pre>";
echo $response;
print_r($info['request_header']);
But I get an error
{"code":1,"message":"Необходимо ввести капчу"}
I. e. I didn't send a captcha.
I can see that in the debugger the Captcha parameter is passed in the header
open image
What am I doing wrong?

Token invalid Travian when call from curl

I try to Exchange silver to gold but i got error message:
{"response": {"error":true,"errorMsg":"Token invalid","data":[]}}
The token is right but i don't know way this error can you help me to fixed
$url ='http://ts1.travian.com/dorf1.php';
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_NOBODY, false);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIEFILE,$dir . '/cookie.txt');
curl_setopt($ch, CURLOPT_COOKIE, "cookiename=0");
curl_setopt($ch, CURLOPT_USERAGENT,"Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, $_SERVER['REQUEST_URI']);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
$response = curl_exec($ch);
preg_match("/ajaxToken = '(.*?)'/s", $response, $s3);
$url ='http://ts1.travian.com/ajax.php?cmd=silverExchange';
$data = array('cmd'=>'silverExchange','exTyp'=>'SilverToGold','s' =>'200','g'=>'1','ajaxToken'=>$s3[1]);
$data_json = json_encode($data);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: application/json'));
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS,$data_json);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_exec($ch);

Curl command works, but written in PHP it does not work. Why?

I am running Linux on a VPS.
When I run the this command:
curl https://www.bloomingdales.com/account/signin -H "Cookie: ewqeqweq" -X GET
... I get the source code of the site.
I tried to write the same command in PHP:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://www.bloomingdales.com/account/signin');
curl_setopt($ch, CURLOPT_ENCODING ,"");
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_COOKIESESSION, 1);
curl_setopt($ch, CURLOPT_HEADER, array("Cookie: ewqeqweq"));
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookie.txt');
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookie.txt');
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, FALSE);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
curl_setopt($ch, CURLOPT_REFERER, 'https://www.bloomingdales.com/account/signin');
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:47.0) Gecko/20100101 Firefox/47.0');
$get_ = curl_exec($ch);
echo $get_;
curl_close($ch);
However, the result of this code is: NULL.
I am wondering what is wrong in my code?

You can use my source code
function _curl($url,$post="",$usecookie = false,$_sock = false,$timeout = false) {
$ch = curl_init();
if($post) {
curl_setopt($ch, CURLOPT_POST ,1);
curl_setopt ($ch, CURLOPT_POSTFIELDS, $post);
}
if($timeout){
curl_setopt($ch, CURLOPT_TIMEOUT,$timeout);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
}
if($_sock){
curl_setopt($ch, CURLOPT_PROXY, $_sock);
curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5);
}
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.10) Gecko/20100914 Firefox/3.6.10");
if ($usecookie) {
curl_setopt($ch, CURLOPT_COOKIEJAR, $usecookie);
curl_setopt($ch, CURLOPT_COOKIEFILE, $usecookie);
}
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'Accept-Language: en-US,en;q=0.5',
'Accept-Encoding: zip, deflate, sdch'
));
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
$result=curl_exec ($ch);
curl_close ($ch);
return $result;
}
$socks5 = '176.126.196.52:24369';
$cookie = tempnam('cookies','coo'.rand(1000000,9999999));
$url = "https://www.bloomingdales.com/account/signin";
$post = "";
$s = _curl($url,$post,$cookie,$socks5,'');
echo $s;
unlink($cookie);
My result

POST Request using CURL and PHP not working properly

I am trying to programmatically process my statement using the form at http://nlp.stanford.edu:8080/corenlp/process. I have the following code snippet in PHP/CURL. However, instead of processing the statement, it is returning the HTML for the form - as if the post parameters are not being sent. I checked that I am sending the required paramaters. Can someone guide me as to what I am doing wrong?
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://nlp.stanford.edu:8080/corenlp/process");
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 30);
curl_setopt($ch, CURLOPT_USERAGENT,"Mozilla/14.0 (compatible; MSIE 6.0; Windows NT 5.1)");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_POST, true);
$data = array(
'outputFormat' => 'xml',
'input' => 'Here is a statement to process',
'Process' => 'Submit Query'
);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
$result = curl_exec($ch);
echo $result;

You have to convert the $data array into a string... Also, Be sure to urlencode() the individual values.
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://nlp.stanford.edu:8080/corenlp/process");
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 30);
curl_setopt($ch, CURLOPT_USERAGENT,"Mozilla/14.0 (compatible; MSIE 6.0; Windows NT 5.1)");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_POST, true);
$data = array(
"outputFormat" => "xml",
"input" => "Here is a statement to process",
"Process" => "Submit Query"
);
$data_string = "";
foreach($data as $key=>$value){ /// YOU HAVE TO DO THIS
$data_string .= $key.'='.urlencode($value).'&'; /// AND THIS
}
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POSTFIELDS, $data_string);
$result = curl_exec($ch);
echo $result;

We Keep Coding

PHP, A popular general-purpose scripting language that is especially suited to web development.

I am not getting readable format data using CURL? - php

Related

Http 411 php curl due to post request

How to pass custom headers to cURL?

Token invalid Travian when call from curl

Curl command works, but written in PHP it does not work. Why?

POST Request using CURL and PHP not working properly

Categories

Resources