PHPUnit and curl wrong charset, cannot compare equal strings - php

i have a PHPUnit test with an assertion:
$this->assertEquals(json_encode($expected), json_encode($result));
The assertion fails because the strings differ in their encoding: (the diff is actually larger, from a large json object)
Failed asserting that two strings are equal.
--- Expected
+++ Actual
## ##
-'Wir holen Ihre P\u00e4ckchen im Wunschzeitraum ab.'
+'Wir holen Ihre P\u00c3\u00a4ckchen im Wunschzeitraum ab.'
I get the actual string from a curl request, the expected string is defined inside the php file as a string.
I use the following curl code to retrieve the JSON from a server:
$headers = array(
"GET /HTTP/1.1",
"User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1",
"Content-type: application/json;charset=\"utf-8\"",
"Accept: application/json",
"Accept-Language: en-us,en;q=0.5",
"Accept-Encoding: gzip,deflate",
"Accept-Charset: utf-8;q=0.7,*;q=0.7",
"Keep-Alive: 300",
"Connection: keep-alive");
$ch = curl_init('http://localhost/frontend_test.php/api');
curl_setopt_array($ch, array(
CURLOPT_POST => TRUE,
CURLOPT_RETURNTRANSFER => TRUE,
CURLOPT_HTTPHEADER => $headers,
CURLOPT_POSTFIELDS => json_encode($some_data),
CURLOPT_HEADER => 1
));
// Send the request
$response = curl_exec($ch);
list($header, $body) = explode("\r\n\r\n", $response, 2);
$result = json_decode($body);
What may be the cause of this?

okay so the solution is simple in my case: The PDO connection used the wrong charset. (The API retrieved the String in question from the database, which was not encoded properly) Dumb mistake

Related

Trying to get html respons

I was trying to get html response using PHP but always get something like this
screenshot [picture hidden from post due to binary characters]
<?php
$header_request = array (
"accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
"accept-encoding: gzip",
"accept-language: id-ID,id;q=0.9,en-US;q=0.8,en;q=0.7",
"cookie: csrf_cookie_name=9316014c9d7860019da66a78edfaf926; _data_pop=115-1_274-1; ci_session=607f0be4e56b8b08ee2398b892f115c9e660192e; _data_cpc=1-2_15-2_190-4",
"referer: https://ptc4btc.com/",
"user-agent: Mozilla/5.0 (Linux; Android 7.0; Moto C Plus) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.56 Mobile Safari/537.36",
"upgrade-insecure-requests: 1"
);
$ch = curl_init();
curl_setopt_array($ch, array(
CURLOPT_URL => "https://ptc4btc.com/dashboard",
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_FOLLOWLOCATION => 1,
CURLOPT_HTTPHEADER => $header_request,
CURLOPT_SSL_VERIFYPEER => 0,
CURLOPT_SSL_VERIFYHOST => 2,
));
$exec = curl_exec($ch);
echo($exec);
curl_close($exec);
You said you were willing to accept gzip data with accept-encoding: gzip in the headers. So decode it:
echo gzdecode($exec);

POSTMAN export don't work with PHP cURL

I created a POST request on POSTMAN, it work great. But when I export header and form body in PHP cURL code and execute it with PHP CLI, the server does not show me data asked in my form.
Thereafter the PHP cURL exported with POSTMAN :
<?php
$curl = curl_init();
curl_setopt_array($curl, array(
CURLOPT_URL => "http://renovation-info-service.gouv.fr/trouvez-un-professionnel",
CURLOPT_RETURNTRANSFER => true,
CURLOPT_ENCODING => "",
CURLOPT_MAXREDIRS => 10,
CURLOPT_TIMEOUT => 30,
CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
CURLOPT_CUSTOMREQUEST => "POST",
CURLOPT_POSTFIELDS => "all%5Bcols-1%5D%5Bcode_postal%5D=03&all%5Bcols-1%5D%5Bcommune%5D=03&all%5Brayon%5D=5&all%5Bdomaine_travaux%5D=30&op=Localiser%2Bdes%2Bentreprises&form_build_id=form-48ViCRy2vxhZBo452cPiRi22VR7K2mZ5ZUwzR-tlO3U&form_id=front_office__locate_all_entreprise_form",
CURLOPT_HTTPHEADER => array(
"accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
"accept-encoding: gzip, deflate",
"accept-language: fr,fr-FR;q=0.8,en-US;q=0.5,en;q=0.3",
"cache-control: no-cache",
"connection: keep-alive",
"content-length: 253",
"content-type: application/x-www-form-urlencoded",
"cookie: UUID=3ab96f55-1e99-d7c4-39a9-59345773b4ca; has_js=1; _ga=GA1.3.1779497884.1533297982; _gid=GA1.3.1474901001.1533297982; _gat=1",
"host: renovation-info-service.gouv.fr",
"postman-token: 570c2dd8-8289-dacd-db57-dcb0d107ea29",
"referer: http://renovation-info-service.gouv.fr/trouvez-un-professionnel",
"upgrade-insecure-requests: 1",
"user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0"
),
));
$response = curl_exec($curl);
$err = curl_error($curl);
curl_close($curl);
if ($err) {
echo "cURL Error #:" . $err;
} else {
echo $response;
}
Thank you for your help ! And sorry with my bad english ^^'
POSTMAN answer is a html code with data asked in POST request. PHP cURL answer is a html code without data asked in POST request.

How do i convert this cURL command into PHP code? And what is "Request Payload"?

So, I was inspecting the network while I'm uploading an JPG image.
A copy as a curl command :
curl "https://admin-official.line.me/11702069/home/api/objects" -H "Origin: https://admin-official.line.me" -H "Accept-Encoding: gzip, deflate, br" -H "Accept-Language: en-US,en;q=0.9,id;q=0.8,de-DE;q=0.7,de;q=0.6" -H "User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36" -H "Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryBv1B9k1i89zlgjnA" -H "Accept: */*" -H "Referer: https://admin-official.line.me/11702069/home/send/" -H "Cookie: tzoffset=-420; _trmccid=6b030cba93d9db09; _ga=GA1.2.1058855825.1526457491; _gid=GA1.2.814129237.1526457491; ldsuid=CiDum1r75zWBJ3f7oLqRAg==; _trmcdisabled2=-1; _trmcuser=^{^\^"id^\^":^\^"^\^"^}; _ga=GA1.3.1058855825.1526457491; _gid=GA1.3.814129237.1526457491; plack_session_localaccount=XXX; minesota=XXX; minesota2=XXX-jdT5f_x10ov7Re12NliQw_zFyIe9osY4oOOc2Rxdy1gMyBhXbKdPmnpA2LHB9VhvlqlL_4hkeSNQK5SgU; plack_session=6cd2e4790ae9990303ea54f7f35b305fba801dc0; _trmcsession=^{^\^"id^\^":^\^"b8ba0724^\^",^\^"path^\^":^\^"/11702069/home/send/^\^",^\^"query^\^":^\^"^\^",^\^"time^\^":1526735963213,^\^"_dice^\^":0.09452730791211028^}; __try__=1526737592637; _trmcpage=/11702069/home/send/" -H "Connection: keep-alive" --data-binary ^"------WebKitFormBoundaryBv1B9k1i89zlgjnA^
Content-Disposition: form-data; name=^\^"file^\^"; filename=^\^"alpen.jpg^\^"^
Content-Type: image/jpeg^
^
^
------WebKitFormBoundaryBv1B9k1i89zlgjnA^
Content-Disposition: form-data; name=^\^"csrf_token^\^"^
^
vJiSf8AZPpAvqCp0JQtrNOMgrZRtrDpa1TEHbYLz^
------WebKitFormBoundaryBv1B9k1i89zlgjnA--^
^" --compressed
I've never seen this before, i'm not an expert at curl. But I'm using curl to PHP coverter, but it's not working in this case.
EDIT : here's what I've tried.
<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://admin-official.line.me/11702069/home/api/objects");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, array("file" => "#".realpath('test.jpg'), "csrf_token" => $split2[0]));
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_ENCODING, 'gzip, deflate');
$headers = array();
$headers[] = "Origin: https://admin-official.line.me";
$headers[] = "Accept-Encoding: gzip, deflate, br";
$headers[] = "Accept-Language: en-US,en;q=0.9,id;q=0.8,de-DE;q=0.7,de;q=0.6";
$headers[] = "User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36";
$headers[] = "Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryBv1B9k1i89zlgjnA";
$headers[] = "Accept: */*";
$headers[] = "Referer: https://admin-official.line.me/11702069/home/send/";
$headers[] = 'Cookie: tzoffset=-420; _trmccid=6b030cba93d9db09; _ga=GA1.2.1058855825.1526457491; _gid=GA1.2.814129237.1526457491; ldsuid=CiDum1r75zWBJ3f7oLqRAg==; _trmcdisabled2=-1; _trmcuser={"id":""}; _ga=GA1.3.1058855825.1526457491; _gid=GA1.3.814129237.1526457491; plack_session_localaccount=XXX; minesota=XXX; minesota2=XXX; plack_session=XXX; _trmcsession={"id":"b8ba0724","path":"/11702069/home/send/","query":"","time":1526735963213,"_dice":0.09452730791211028}; __try__=1526737592637; _trmcpage=/11702069/home/send/';
$headers[] = "Connection: keep-alive";
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookie.txt');
curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookie.txt');
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
$result = curl_exec($ch);
if (curl_errno($ch)) {
echo 'Error:' . curl_error($ch);
}
curl_close ($ch);
echo $result;
?>
the "request payload" is the request body. a HTTP request is split into 2 parts, the headers, and the body (and Chrome calls the body for the request payload here, not sure if that's an official name, or just the whim of a chromium dev)
anyway, you're using the # method, which was discouraged as of PHP 5.5, disabled-by-default in php 5.6, and completely removed in PHP7.
use CURLFile instead, eg:
curl_setopt($ch, CURLOPT_POSTFIELDS, array(
"file" => new CURLFile('test.jpg'),
"csrf_token" => $split2[0]
));
also, don't set the Accept-Encoding header manually. you specify gzip, deflate, br, and curl doesn't support br, and depending on how libcurl was compiled, might not even support gzip/deflate (although it usually does), remove that.
also, set CURLOPT_ENCODING to emptystring, and curl will list all encodings it was built to support (which is usually gzip and deflate, but is not guaranteed to be)
also, when using the # or CURLFile method, don't set the Content-Type header manually, because if curl's boundary doesn't match what you set in the Content-Type header, the server won't be able to read the request properly. curl will set this header for you, if you don't override it, so don't.
Figure it out using Postman.
$curl = curl_init();
curl_setopt_array($curl, array(
CURLOPT_URL => "https://admin-official.line.me/11702069/home/api/objects",
CURLOPT_RETURNTRANSFER => true,
CURLOPT_ENCODING => "",
CURLOPT_MAXREDIRS => 10,
CURLOPT_TIMEOUT => 30,
CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
CURLOPT_CUSTOMREQUEST => "POST",
CURLOPT_POSTFIELDS => array(
"csrf_token" => $split2[0],
"file" => new CURLFile("test.jpg")
),
//CURLOPT_POSTFIELDS => "------WebKitFormBoundary7MA4YWxkTrZu0gW\r\nContent-Disposition: form-data; name=\"csrf_token\"\r\n\r\nETukCyg_-OM72cZ0MuAyunvvy4cb8Cx2QqoP41T7\r\n------WebKitFormBoundary7MA4YWxkTrZu0gW\r\nContent-Disposition: form-data; name=\"file\"\r\n\r\n\r\n------WebKitFormBoundary7MA4YWxkTrZu0gW--",
CURLOPT_HTTPHEADER => array(
"Accept: */*",
"Accept-Encoding: gzip, deflate, br",
"Accept-Language: en-US,en;q=0.9,id;q=0.8,de-DE;q=0.7,de;q=0.6",
"Cache-Control: no-cache",
"Connection: keep-alive",
"Content-Type: multipart/form-data",
"Cookie: tzoffset=-420; _trmccid=6b030cba93d9db09; _ga=GA1.2.1058855825.1526457491; _gid=GA1.2.814129237.1526457491; ldsuid=CiDum1r75zWBJ3f7oLqRAg==; _trmcdisabled2=-1; _trmcuser=^{^\^id^^:^\^^^^}; _ga=GA1.3.1058855825.1526457491; _gid=GA1.3.814129237.1526457491; plack_session_localaccount=COOKIE; minesota=COOKIE; minesota2=COOKIE; _trmcsession=^{^\^id^^:^\^36b6efe6^^,^\^path^^:^\^/11702069/home/send/^^,^\^query^^:^\^^^,^\^time^^:1526880852499,^\^_dice^^:0.4099096438552394^}; plack_session=519cce226206bd15cba86f2153c9dc91a56df4fb; _trmcpage=/11702069/home/send/; __try__=1526881572577",
"Origin: https://admin-official.line.me",
"Postman-Token: 8b9dc9c8-166a-4c50-93c4-b16ca72d72dc",
"Referer: https://admin-official.line.me/11702069/home/send/",
"User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36",
//"content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW"
),
));
curl_setopt($curl, CURLOPT_COOKIEJAR, 'cookie.txt');
curl_setopt($curl, CURLOPT_COOKIEFILE, 'cookie.txt');
$response = curl_exec($curl);
$err = curl_error($curl);
curl_close($curl);
if ($err) {
echo "cURL Error #:" . $err;
} else {
echo $response;
}

is it required to decode curl result in my case?

I have created php curl to send xml request as under, where it seems that login works. because if there is anything wrong in login detail then it shows error. but if everything is ok then it give response in some garbage value:
Source main code :
$contentlength=strlen($postdata);
$curl = curl_init();
curl_setopt_array($curl, array(
CURLOPT_URL =>$this->url ,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_TIMEOUT => 30,
CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_0,
CURLOPT_CUSTOMREQUEST => "POST",
CURLOPT_POSTFIELDS => "$postdata",
CURLOPT_HTTPHEADER => array(
"Content-Type: text/xml;charset=utf-8",
"Content-Length: 839",
"Accept: text/xml",
"Accept-Encoding: gzip",
"Authorization : $Authorization",
"User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.11) Gecko/20071204 Ubuntu/7.10 (gutsy) Firefox/2.0.0.11",
"Host: $host",
"Connection: close"
),
));
Ouput looks like this :
‹d1oÂ0…÷üŠS–,Û)J›ÈuèÔ"$–²šäH£Æv°MUþ}MÒ€J·»{ŸŸï_}«¾ÐºÖèeÂæ4Ô•©[Ý,““?¤OÉJDÜ¢ëv¸ïLõyåã‡yþ‹ Ç:oñ€6 x
what is wrong ? should i decode anything ? if yes then in which forgat i need to debug. I use securetrading.com site to send xml request.
Simply, you have to remove "Accept-Encoding: gzip" from CURLOPT_HTTPHEADER array.
If still not working, use to decompress output using gzdecode() function
$original_output= gzdecode($curl_output);
For resolve issue of content length you have to count string length of post data, which you have already calculated and stored in $contentlength variable.
"Content-Length:".$contentlength
curl_setopt($curl, CURLOPT_ENCODING , "gzip");
will show properly the result

How to post data with cURL?

I want to return the response as url instead boolean. I looks header from browser and match it but it only return boolean (1). Is there some options missing?
<?php
$ch = curl_init();
$url = 'http://www.tusfiles.net/anfossy50rrd';
// Post field and value
$post = array(
'op' => 'download2',
'id' => 'anfossy50rrd',
'rand' => 'q56tfpiklusrizipkforyjqsykxzqlsi7ur3hyi',
'referer' => '',
'method_free' => '',
'method_premium' => '',
'down_script' => '1'
);
its the header
$headers = [
'Host: www.tusfiles.net',
'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:28.0) Gecko/20100101 Firefox/28.0',
'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'Accept-Language: en-US,en;q=0.5',
'Accept-Encoding: gzip, deflate',
'Cache-Control: no-cache',
'Content-Type: application/x-www-form-urlencoded; charset=utf-8',
'Referer: http://www.tusfiles.net/anfossy50rrd',
'Cookie: lang=english; login=lynxpravoka; xfss=; aff=2513158; __atuvc=17%7C36;
__atuvs=57cf088127fd55f9004; cookiescriptaccept=visit',
'Connection: keep-alive',
'Upgrade-Insecure-Requests: 1'
];
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_POSTFIELDS,$post);
$server_output = curl_exec ($ch);
print_r($server_output);
?>
You need to use something like this:
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
'X-Header-1: value1',
'X-Header-2: value2'
));
Or you can also build an array where the key is the header name and the value is of course the header value. It would look something like this:
$headers = array(
'Host' => 'www.tusfiles.net',
'User-Agent' => 'Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:28.0) Gecko/20100101 Firefox/28.0'
);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
http://www.php.net/manual/en/function.curl-setopt.php
the problem is that the server returns a HTTP/1.1 302 Found HTTP redirect, expecting you to send the request to the "found" URL. but because you do not have CURLOPT_FOLLOWLOCATION - nor do you manually handle redirects (nor read the redirect headers at all), you do not follow the redirect. enable CURLOPT_FOLLOWLOCATION , and you should get the response.
also, you should practically never give the 'Accept-Encoding: gzip, deflate', header manually with curl. if curl was not actually compiled with gzip or deflate and the server decides to use it, the transfer will fail, furthermore, if, in the future, both the server and curl supports an even better transfer encoding, it won't be used because you explicitly said only gzip/deflate was supported. instead, make curl give all supported encodings automatically, by giving an empty string '' to CURLOPT_ENCODING.
and a protip: in the future, to debug why curl transfers isn't working as expected, enable CURLOPT_VERBOSE

Categories