I'm trying to POST some form data to a site using cURL. It's not a secure site, and doesn't require logging in. It's just a form used to get back some information.
The form is here, and the form action is to the same page (it's in Turkish). This is how I have been trying to send the POST request in PHP:
$headers = array(
"content-length: 138",
"accept-language: en-US,en;q=0.8",
"accept-encoding: gzip, deflate",
"referer: http://objektifsonuc.com/",
"content-type: application/x-www-form-urlencoded",
"user-agent: Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 Safari/537.36",
"origin: http://objektifsonuc.com",
"accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
"cache-control: max-age=0",
"connection: keep-alive"
"host: objektifsonuc.com"
);
$query="il=1&ilce=1%3B18&okul=1%3B18%3B743729%3BTEOGS%2CSBS&sinav=100%3BTEOGS&sinif=8&ogrno=1941&ograd=BERKANT+%DDPEK&ogr=%D6%F0renci+Veli+Giri%FE";
$url = "http://objektifsonuc.com/index.php";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POSTFIELDS, $query);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
echo curl_exec($ch);
Current result: The form page renders
Desired result: A different page containing some tables renders
Here are valid form inputs:
Şehir: ADANA
İlçe: SARIÇAM
Okul: Hacı Özcan Sinağ Ortaokulu
Sınav Türü: TEOGS | ADANA İL MİLLİ EĞİTİM OKDS
Sınıf: 8
Öğrenci No: 1941
Öğrenci Ad: BERKANT İPEK
Here is what the form looks like filled out:
Here are the form headers taken from Chrome on a successful POST request:
POST /index.php HTTP/1.1
Host: objektifsonuc.com
Connection: keep-alive
Content-Length: 138
Cache-Control: max-age=0
Origin: http://objektifsonuc.com
User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 Safari/537.36
Content-Type: application/x-www-form-urlencoded
DNT: 1
Referer: http://objektifsonuc.com/index.php
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.8
And here is the POST body:
il=1&ilce=1%3B18&okul=1%3B18%3B743729%3BTEOGS%2CSBS&sinav=100%3BTEOGS&sinif=8&ogrno=1941&ograd=BERKANT+%DDPEK&ogr=%D6%F0renci+Veli+Giri%FE
I'm stumped. What is causing the POST to fail? Can anyone make this POST succeed?
This should work for you:
//open connection
$url = 'http://objektifsonuc.com/index.php';
$string = 'il=1&ilce=1%3B18&okul=1%3B18%3B743729%3BTEOGS%2CSBS&sinav=100%3BTEOGS&sinif=8&ogrno=1941&ograd=BERKANT+%DDPEK&ogr=%D6%F0renci+Veli+Giri%FE';
$ch = curl_init();
//set the url, number of POST vars, POST data
curl_setopt($ch,CURLOPT_URL, $url);
curl_setopt($ch,CURLOPT_POST, count($string));
curl_setopt($ch,CURLOPT_POSTFIELDS, $string);
//execute post
$result = curl_exec($ch);
//close connection
curl_close($ch);
Related
I am trying to send a GET request to a python server with an Authorization Token in the header.
I do receive the Request on the server side but without the authorization token
Here is the php code that is generating the request :
<?php
$url = "http://localhost:7432/f.php";
$curl = curl_init($url);
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
$headers = array(
"Authorization: sdfascvthsgdgdssgvsgscf",
);
curl_setopt($curl, CURLOPT_HTTPHEADER, $headers);
//for debug only!
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
$resp = curl_exec($curl);
curl_close($curl);
var_dump($resp);
?>
When I generate the request from Reqbin, I do Receive the request with the Authorization token in the header , but when I try to generate the request from the php file then it does not send the Authorization token to the server
Here is the request received on server side when I use the Reqbin website to make the request:
127.0.0.1 - - [03/Sep/2021 13:27:10] "GET /f.php HTTP/1.1" 200 -
Host: localhost:7432
Connection: keep-alive
sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="92", "Opera";v="78"
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159
Safari/537.36 OPR/78.0.4093.184
sec-ch-ua-mobile: ?0
Authorization: sdfascvthsgdgdssgvsgscf
Accept: */*
Sec-Fetch-Site: none
Sec-Fetch-Mode: cors
Sec-Fetch-Dest: empty
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Cookie: _ga=GA1.1.1359401486.1622671713
And This is the received Request when I use the php code mentioned above :
127.0.0.1 - - [03/Sep/2021 13:27:20] "GET /f.php HTTP/1.1" 200 -
Host: localhost:7432Connection: keep-alive Cache-Control: max-age=0
sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="92", "Opera";v="78"
sec-ch-ua-mobile: ?0 Upgrade-Insecure-Requests: 1 User-Agent:
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML,
like Gecko) Chrome/92.0.4515.159 Safari/537.36 OPR/78.0.4093.184
Accept:
text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Sec-Fetch-Site: none Sec-Fetch-Mode: navigate Sec-Fetch-User: ?1
Sec-Fetch-Dest: document Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9 Cookie:
_ga=GA1.1.1359401486.1622671713
What am I doing wrong at the php side ?
Okay, i have some website which i should parse...
Firstly, i open debugger in Firefox hitting F12, and look at Network tab, then enter needed website, and reading first root GET request, like
Doman => website.com
File => /
I get there all the request headers and write them into php array manually, then in code i call
curl_setopt($curl, CURLOPT_HTTPHEADER, $headerArray);
and also other options, then call
curl_exec();
while inspecting the Network tab in Firefox, i see that request headers are maybe such as default, and no specific headers written manually into array were sent. Similar problem with CURLOPT_COOKIEFILE and CURLOPT_COOKIEJAR, cookies are just written to cookie file on server, but in fact, there are another cookies in next request instead of previously saved in cookies file.
Actual request headers in browser's inspector:
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip, deflate
Accept-Language: ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3
Cache-Control: max-age=0
Connection: keep-alive
Cookie: _ga=GA1.1.1951751996.1563984714; _gid=GA1.1.1564173251.1563984714; _userGUID=0:jyhg490v:AIQdD2Qpm9rmbla1U93mK2a45CFRe49c; jv_enter_ts_2VumZAPpbr=1563984717382; jv_visits_count_2VumZAPpbr=1; .....
Host: localhost
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0
PHP Code:
<?php
$headers = ['Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'Accept-Language: ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3',
'Cache-Control: max-age=0',
'Connection: keep-alive',
'Cookie: visid_incap_1987259....,
'Host: website.com',
'TE: Trailers',
'Upgrade-Insecure-Requests: 1',
'User-Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)'];
$curl = curl_init("https://www.website.com/");
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($curl, CURLOPT_HTTPHEADER, $headers);
curl_setopt($curl, CURLOPT_COOKIEFILE, dirname(__FILE__)."/cookies.txt");
curl_setopt($curl, CURLOPT_COOKIEJAR, dirname(__FILE__)."/cookies.txt");
echo curl_exec($curl);
?>
You will not be able to see the headers send CURL in the Browser Dev Tools. All requests are executed on the server side. Your headers are sent successfully. You can check it out like this:
curl_setopt($curl, CURLINFO_HEADER_OUT, true);
$sentHeaders = curl_getinfo($curl, CURLINFO_HEADER_OUT);
print_r($sentHeaders);
I have 1 REST api :
http://www.animemobile.com/service/v2/mobile2.php?episode_id=47272
I use curl to request it, in my PC with xampp, it works well and returns the correct results. This is results from my PC with xampp:
[
{
"Title":"English Subbed",
"link":"\/[HorribleSubs] Pascal-sensei - 01 [720p]_af.mp4?
st=14GwNjlMxuI8524DS56IUA&e=1495183034"
}
]
I use
/[HorribleSubs] Pascal-sensei - 01 [720p]_af.mp4?
st=14GwNjlMxuI8524DS56IUA&e=1495183034
to create a link as:
http://st2.anime1.com/[HorribleSubs]%20Pascal-sensei%20-
%2001%20[720p]_af.mp4?st=14GwNjlMxuI8524DS56IUA&e=1495183034.
This link is a video that can be played when request from browser (now).
But when I use the curl in my SERVER, it still works well but does not return the correct results. This is results from my Server:
[
{
"Title":"English Subbed",
"link":"\/[HorribleSubs] Pascal-sensei - 01 [720p]_af.mp4?
st=ghDP4290fsBNdmfsSKCD=1495195645"
}
]
When I use
/[HorribleSubs] Pascal-sensei - 01 [720p]_af.mp4?
st=ghDP4290fsBNdmfsSKCD=1495195645
to create a link as:
http://st2.anime1.com/[HorribleSubs]%20Pascal-sensei%20-
%2001%20[720p]_af.mp4?%20st=ghDP4290fsBNdmfsSKCD=1495195645.
It doesn't play on my browser.
This is my curl:
$c = curl_init();
curl_setopt($c, CURLOPT_URL, $url);
curl_setopt($c, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($c, CURLOPT_ENCODING, 'gzip,deflate');
curl_setopt($c, CURLINFO_HEADER_OUT, true);
$headers = [
'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8',
'Accept-Encoding: gzip, deflate, sdch',
'Accept-Language: vi,en-US;q=0.8,en;q=0.6',
'Cache-Control: max-age=0',
'Connection: keep-alive',
'Cookie: __cfduid=d7bf11c717fbcd54ec9b259e301a966d71480412679',
'Host: www.animemobile.com',
'Upgrade-Insecure-Requests: 1',
'User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36'
];
curl_setopt($c, CURLOPT_HTTPHEADER, $headers);
$data = curl_exec($c);
What is the problem here? Please help me!
Edit1: If you want to test the results, you need to request the REST-api again because it had limited time for link to be created. Important that request the REST-api on PC returns correct results but request from server returns wrong results although they look very similar!
I'm trying to set up a proxy server to make a post request. Problem is when I make the request I am not seeing the payload.
One thing I notice is that curl seems to be adding an extra "boundary" to the content-type in the request.
Am I missing something?
The Code:
$contentType = $_SERVER["HTTP_CONTENT_TYPE"];
$post = http_build_query($_POST);
$ch = curl_init();
$header = array("Content-Type:" . $contentType,
"Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
"Accept-Encoding:gzip, deflate, br",
"Accept-Language:en-US,en;q=0.8",
"Connection:keep-alive",
"User-Agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36",
"Cache-Control:max-age=0",
"Upgrade-Insecure-Requests:1",
"Origin:<url here>");
echo "<b>POST</b><br>" . var_dump($_POST) . "<br><br>";
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, count($_POST));
curl_setopt($ch, CURLOPT_POSTFIELDS, $_POST);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLINFO_HEADER_OUT, true);
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookiejar.txt");
curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
curl_setopt($ch, CURLOPT_HEADER, 1);
$result = curl_exec($ch);
$headerSent = curl_getinfo($ch, CURLINFO_HEADER_OUT );
echo "<b>Request Header</b><br>$headerSent<br><br>";
$header_size = curl_getinfo($ch, CURLINFO_HEADER_SIZE);
$header = substr($result, 0, $header_size);
$body = substr($result, $header_size);
echo "<b>Response Header</b><br>$header<br><br>";
echo "<b>Response Body</b><br>$body";
Response
$_POST = array(5) { ["formFields_Complaint_Type"]=> string(9) "1-GM2-226"
["formFields_Descriptor_1"]=> string(10) "1-GM3-3085"
["formFields_Descriptor_2"]=> string(9) "1-GM4-903"
["formFields_Date/Time_of_Occurrence"]=> string(0) "" ["_target1"]=> string(1) " " }
Request Header:
POST <relative address> HTTP/1.1 Host: <url>
Cookie:
JSESSIONID=mDMJZQdLV4bhvJQ6vPyQvxqHVTynGS3byBnYsTpjDvY1xBnB93R8!-759339305!-1867032216 Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Encoding:gzip, deflate, br
Accept-Language:en-US,en;q=0.8
Connection:keep-alive
User-Agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36
Cache-Control:max-age=0 Upgrade-Insecure-Requests:1
Origin: <url>
Content-Length: 633
Expect: 100-continue
Content-Type:multipart/form-data; boundary=----
WebKitFormBoundarybdBepqnmjSF86t50; boundary=------------------------
f8e2ad22b9bb626c
best guess: your (biggest, code-breaking, but not only) problem is that the target server supports only application/x-www-form-urlencoded-encoded POST requests, but your curl code converts both application/x-www-form-urlencoded-encoded requests, and multipart/form-data requests to multipart/form-data, regardless of what the client used. (this is because PHP transparently translates both of them to an equal native PHP array called $_POST)
this will use multipart/form-data encoding:
curl_setopt($ch, CURLOPT_POSTFIELDS, $_POST);
this will use application/x-www-form-urlencoded encoding:
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($_POST));
you must decide which encoding to use, based on $_SERVER["HTTP_CONTENT_TYPE"];
and if its neither of those (for example, if its application/json), you must add special code to handle each, and you should probably error out whenever $_SERVER["HTTP_CONTENT_TYPE"]; is not 1 of the types you have made a special case for (like raw $_POST for multipart, and http_build_query($_POST) x-www-form-urlencoded)
also you're not forwarding arbitrary http headers, you should probably add some code for that
and if you really need to support Upgrade-Insecure-Requests:1 header, you need to implement specific code to handle that at the proxy side (go read the http specs on the subject - https://www.w3.org/TR/upgrade-insecure-requests/ )
and you say to the target that you accept Accept-Encoding:gzip, deflate, br , but provide no code to decode any of them, so it will look like garbage binary data to the client if the target server decide to use any of them (curl can decode them for you though, using CURLOPT_ENCODING, if libcurl was compiled with gzip and deflate and br support. i've never seen a libcurl with br support, and i bet your curl doesn't have it. probably have gzip/deflate support compiled-in though)
I am trying to cURL a base64_encoded xml string to a c# WebAPI that i dont have control over. I can cURL the string successfully but it is not being accepted by the API.
Logging the output shows that cURL is stripping + characters from the base64 string which i believe to be the problem.
The code i have is:
$username = "username";
$password = "$23hrlkbl";
$xml = "<Envelope><Header><User>".$username."</User><Password>".$password."</Password></Header></Envelope>";
$passThru = "https://api.domain.com/SignIn.aspx?passthruUrl=/Management/Api/DataEnrichment/GetAddresses/?buildingNumber=1%26streetName=Nightingale%20Road%26postcode=L12%200QN
$post_packet_data = 'XMLdataPacket='.urlencode(base64_encode($xml));
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $passThru);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_packet_data);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
$response = curl_exec ($ch);
curl_close($ch);
print_r($response);
Posting the following string (non urlencoded $post_packet_data) via an html form to the $passThru address works successfully
XMLdataPacket=PEVudmVsb3BlPjxIZWFkZXI+PFVzZXI+dXNlcm5hbWU8L1VzZXI+PFBhc3N3b3JkPiQyM2hybGtibDwvUGFzc3dvcmQ+PC9IZWFkZXI+PC9FbnZlbG9wZT4=
However when posting the same string via cURL the following is sent and not accepted
XMLdataPacket=PEVudmVsb3BlPjxIZWFkZXI PFVzZXI dXNlcm5hbWU8L1VzZXI PFBhc3N3b3JkPiQyM2hybGtibDwvUGFzc3dvcmQ PC9IZWFkZXI PC9FbnZlbG9wZT4=
UPDATE
After speaking to the developers of the API they have confirmed the requests are now coming in in the correct format but believe the calls are not being processed due to incorrect headers being sent. They have sent me the headers of a working call which they got using fiddler:
POST https://api.domain.com/SignIn.aspx?passthruUrl=/Management/Api/DataEnrichment/GetAddresses/?buildingNumber=1%26streetName=Nightingale%20Road%26postcode=L12%200QN HTTP/1.1
Host: test.domain.com
Connection: keep-alive
Content-Length: 164
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Origin: null
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.146 Safari/537.36
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-GB,en-US;q=0.8,en;q=0.6
Cookie: ASP.NET_SessionId=uzwfjq3zojao3l5fw141l453
XMLdataPacket=PEVudmVsb3BlPjxIZWFkZXI%2BPFVzZXI+dXNlcm5hbWU8L1VzZXI%2BPFBhc3N3b3JkPiQyM2hybGtibDwvUGFzc3dvcmQ%2BPC9IZWFkZXI%2BPC9FbnZlbG9wZT4%3D
How would i amend the headers of the cURL call to replicate the above. Currently again using fiddler the cURL call headers appear as the following:
GET http://test.domain.com/test_call.php HTTP/1.1
Host: test.domain.com
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-gb,en;q=0.5
Accept-Encoding: gzip, deflate
Cookie: CAKEPHP=g22d7kf7qs8b1v02uui2mnop30
Authorization: Basic bmV0aG91c2VwcmljZXMxOnRlbXAxODkx
Connection: keep-alive
Cache-Control: max-age=0
The data is receieved at the api end as POST but the curl headers are coming through as GET according to the above
Try using rawurlencode() instead of urlencode() - it encodes the space character using '%20' instead of using the '+' character.