I made a POST request, printed out header out information and noticed that is gets treated as GET. What is the reason for such behaviour?
HEADER OUT DATA:
GET /inx/aeGDrYQ HTTP/1.1
Host: example.com
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3
Accept: */*
Cookie: PHPSESSID=t762fd0nbi12p3hrgb9sgx9k20; ____ri=4485; safemode=1; session=eyJpdiI6Im1HQzlNR1JhMTNDc0JRelYyRVwveUp6N0JxZG56Z2p5K094eSs3YU5HQ3dzPSIsInZhbHVlIjoiVXBPYzN4TVNReURhVnMxQlZ1TndLZ0dYUjltbUVEcW11bkJJMDdMRVZoZ0hHMjRXZ2p6azlcL1FWXC93NnZWN3oreDcxQms3aGlcL3l0MG1vTjd1V21FcmVCVzFnQjVuMUY5dHBWeUlTbU9NSjJcL1d5TlwvTW11ZWp1eHpNd3d4eFZTamV6aThsNldkdlN3aFo0XC9sTnVnU0tXVDRKbWVBU25VU0hJaDREQ1J5M2xDXC9zRUc5OXhWMWJWWG9jYndhczYyZW4xMkUxb3BoU3FmQmMrNVdzM3RqQmgzeHY1NVJ5RXRTNGZOdmQ4dTRCbmRtWVZBN210QVVEVk1BNTFPc1NQcFU3bnd4NEpKbnRaTFliRWNzbkZaXC9YWUF1Nld1ekZSbjVGRXBuZzNoRlBNND0iLCJtYWMiOiI4OWEwNmMyZGVkYjFiYTlmNDY0MDE5MTQwNzE1YzNhYWJjYTA5YjJ3MWMyZjgwMTViN2MyYmI0OWUyNmMwNjM0In0%3D; toastMsg=2; ts1=11e2bb0a86bfb9669c36Xcc407e1e3b3decefcce
REST OF THE CODE:
$ch = curl_init('https://example.com/login');
$postData = [
'name' => $name,
'pass' => $pass
];
$postDataStr = http_build_query($postData);
# Append some fields to the CURL options array to make a POST request. I left out headers, since
# they don't change and added return_transfer for echoing end results
$options[CURLOPT_POST] = 1;
$options[CURLOPT_POSTFIELDS] = $postDataStr;
$options[CURLOPT_HEADER]=1;
$options[CURLOPT_COOKIEJAR]=$cookie;
$options[CURLOPT_USERAGENT]= $useragent;
$options[CURLOPT_FOLLOWLOCATION] = true;
$options[CURLOPT_RETURNTRANSFER] = true;
$options[CURLINFO_HEADER_OUT] = true;
curl_setopt_array($ch, $options);
# Execute
$response = curl_exec($ch);
// echo $response;
$request = curl_getinfo($ch, CURLINFO_HEADER_OUT);
echo "Request sent: $request<br>";
You're only showing one request, and I suspect it is the second request where the first was a POST and the GET you see here is the one done after a redirect has been followed.
curl may switch to a GET when following a redirect based on which 30x code is in the response and the behavior is guided by the HTTP 1.1 spec (RFC 7230 and friends).
Related
My problem is pretty straightforward, but I cannot for the life of me figure out what is wrong. I've done something similar with another API, but this just hates me.
Basically, I'm trying to get information from https://owapi.net/api/v3/u/Xvs-1176/blob and use the JSON result to get basic information on the user. But whenever I try to use file_get_contents, it just returns
Warning: file_get_contents(https://owapi.net/api/v3/u/Xvs-1176/blob): failed to open stream: HTTP request failed! HTTP/1.1 400 BAD REQUEST in Z:\DevProjects\Client Work\Overwatch Boost\dashboard.php on line
So I don't know what's wrong, exactly. My code can be seen here:
$apiBaseURL = "https://owapi.net/api/v3/u";
$apiUserInfo = $gUsername;
$apiFullURL = $apiBaseURL.'/'.$apiUserInfo.'/blob';
$apiGetFile = file_get_contents($apiFullURL);
Any help would be largely appreciated. Thank you!
You need to set user agent for file_get_contents like this, and you can check it with this code. Refer to this for set user agent for file_get_contents.
<?php
$options = array('http' => array('user_agent' => 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:53.0) Gecko/20100101 Firefox/53.0'));
$context = stream_context_create($options);
$response = file_get_contents('https://owapi.net/api/v3/u/Xvs-1176/blob', false, $context);
print_r($response);
That's what page is sending: "Hi! To prevent abuse of this service, it is required that you customize your user agent".
You can customize it using curl like that:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://owapi.net/api/v3/u/Xvs-1176/blob");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch,CURLOPT_USERAGENT,'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13');
$output = curl_exec($ch);
$output = json_decode($output);
if(curl_getinfo($ch, CURLINFO_HTTP_CODE) !== 200) {
var_dump($output);
}
curl_close($ch);
If you do curl -v https://owapi.net/api/v3/u/Xvs-1176/blob you will get a response and you will see what headers cURL includes by default. Namely:
> Host: owapi.net
> User-Agent: curl/7.47.0
> Accept: */*
So then the question is, which one does owapi care about? Well, you can stop cURL from sending the default headers like so:
curl -H "Accept:" -H "User-Agent:" -H "Host:" https://owapi.net/api/v3/u/Xvs-1176/blob
... and you will indeed get a 400 response. Experimentally, here's what you get back if you leave off the "Host" or "User-Agent" headers:
{"_request": {"api_ver": 3, "route": "/api/v3/u/Xvs-1176/blob"}, "error": 400, "msg": "Hi! To prevent abuse of this service, it is required that you customize your user agent."}
You actually don't need the "Accept" header, as it turns out. See the PHP docs on how to send headers along with file_get_contents.
I want to fetch response of a GET request by CURL, but get the Error "HTTP/1.1 405 Method Not Allowed" and no content is returned (except the header).
The following is my code.
$url = "http://api.example.com/Q5PLCmwYzho=/7avEU7ptYyummfheg9!0KA==?aqPI=93566&aqIIO=false&aqIP=false&aqSK=0&aqTK=14&aqSO=date&aqCI=0"
$options = array(
CURLOPT_URL => $url,
CURLOPT_HTTPGET => true, //set request type post or get
CURLOPT_RETURNTRANSFER => true, // return web page
CURLOPT_HEADER => true,
CURLOPT_USERAGENT => 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.84 Safari/537.36',
CURLOPT_HTTPHEADER => array(
'Accept: application/json, text/javascript, */*; q=0.01',
'Accept-Encoding: gzip, deflate, sdch',
//'Content-Type: application/json; charset=utf-8',
//'Access-Control-Allow-Origin: http://localhost:14189',
//'Origin: http://www.example.com',
//'Host: api.example.com',
//'Access-Control-Allow-Credentials: true',
),
CURLOPT_COOKIE => 'Cookie:scarab.profile=%2293566%7C1465411024%22; scarab.mayAdd=%5B%7B%22i%22%3A%2293566%22%7D%2C%7B%22i%22%3A%22117313%22%7D%5D; _ceg.s=o8gwtw; _ceg.u=o8gwtw; _gat=1; _dc_gtm_UA-13212406-1=1; DK-Client=CWUU,09b6ab76-f662-48b3-a1e6-f08779519236; __auc=764bbe2e155314db7ac1a950571; scarab.visitor=%22566B1DCD74FCD3A5%22; _ga=GA1.2.412464545.1465411024'
);
$ch = curl_init();
curl_setopt_array( $ch, $options );
$content = curl_exec( $ch );
$err = curl_errno( $ch );
$errmsg = curl_error( $ch );
$header = curl_getinfo( $ch );
curl_close( $ch );
$header['errno'] = $err; //no error!
$header['errmsg'] = $errmsg;
$header['content'] = json_decode($content); // no content!
echo $response['http_code']; //output: 405
I checked the corresponding ajax request in the webpage and every curl options seems to be set correctly. The details (captured from chrome Dev tools) are shown in the image.
![1]: http://i.stack.imgur.com/XW8k1.png
What am I missing?? I searched a lot but couldn't solve the problem! I know one of the causes of this error is using an unauthorized request method for curl but the corresponding AJAX call uses GET method and get the result.
What is wrong? Any help would be appreciated.
Update: I wrote an AJAX request to fetch the data in local server (localhost) and got this error : "Cross-Origin Request Blocked". It means that I can't make request to a different domain (api.example.com) from localhost. Are there any workarounds or hack to get the response content from that server?
I was getting this error in Postman. In my case it was because I was using GET when I should have used POST. Also I was using GET params instead of POST params. Checking the proper method (specified in my java code) and using the "code" button in postman to display the curl CLI helped me.
Try to send User-Agent in your request.
If page that you requested needs any authorization then you need to send cookies with auth data.
I am getting an odd server php curl error in both my local and production servers (Ubuntu 14.04.2 LTS, PHP 5.5.9-1ubuntu4.11, Apache 2.4.7).
Basically, a curl request to a remote API returns a status code 500 response, ONLY in wp_remote_get(), where it returns status 200 in both curl_exec() and a browser request.
My debug code:
<?php
$url = 'https://yoast.com?edd_action=activate_license&license=my-license-key-here&item_name=WooCommerce+Yoast+SEO&url=https://google.com';
// this return status 200:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$result = curl_exec($ch);
curl_close($ch);
echo '<pre>' . print_r($result, true) . '</pre>';
// this return status 500:
$testResp = wp_remote_get($url);
echo '<pre>' . print_r($testResp, true) . '</pre>';
I cannot figure out why it responds 500 for wp_remote_get(). I've tried adjusting args passed to wp_remote_get(), but still a 500 with it.
I've also disabled all plugins in debugging.
Any Ideas?
OK, after a bit of debugging, I believe the issue is the default User-Agent string Wordpress sets in wp-includes/class-http.php, set when creating an http request for wp_remote_get().
The option has a filter, but the default is created like so:
'user-agent' => apply_filters( 'http_headers_useragent', 'WordPress/' . $wp_version . '; ' . get_bloginfo( 'url' ) ),
So in my case, the 'user-agent' header value was: "Wordpress/4.3.1; http://myurl.com"
When I hook into the filter http_headers_useragent and return an empty string, or even a different user-agent string such as: 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8) AppleWebKit/535.6.2 (KHTML, like Gecko) Version/5.2 Safari/535.6.2', the request will return a successful 200 response.
Not sure if the semicolon is the true culprit, but if I remove it and set the user-agent string to just "Wordpress/4.3.1", the request is successful as well.
I had the same problems - wp_remote_get was not working while the classic Curl calls were making the calls. Indeed the problem is on 'user agent' . This is my solution based on "chuuke" findings
$args = array(
'user-agent' => 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8) AppleWebKit/535.6.2 (KHTML, like Gecko) Version/5.2 Safari/535.6.2',
);
$data = wp_remote_get($new_url_signed,$args);
Thanks
A server I am working on appears to be denying outbound HTTP requests. The reason I think this is because I've tried both Guzzle and curl requests to the API.
The API lives on the same domain as the web server (this is temporary at clients request). I can make requests to the API server via Postman (Chrome plugin), but when I run that same request on the server, it doesn't return anything.
Here are the headers from the 'Postman' request:
POST /api2/user/session HTTP/1.1
Host: example.com
Connection: keep-alive
Content-Length: 49
Cache-Control: no-cache
Origin: chrome-extension://fdmmgilgnpjigdojojpjoooidkmcomcm
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.76 Safari/537.36
Content-Type: application/x-www-form-urlencoded
Accept: */*
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.8
Cookie: PHPSESSID=d9ad79c4c0822fc5c86f4d8799307f1b; _ga=GA1.2.1674422587.1425409444
Post data:
token=a559d5bba5a9e9517d5c3ed7aeb62db6&user=30972
This works. It returns the data. But when I call the same endpoint from within my web app, I get nothing.
$data = urlencode("token=a559d5bba5a9e9517d5c3ed7aeb62db6&user=30972");
$ch = curl_init('http://example.com/api2/user/session');
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
'Content-Type: application/x-www-form-urlencoded',
'Content-Length: ' . strlen($data))
);
$result = curl_exec($ch);
What I don't understand is I can run the following, and it returns the content:
print file_get_contents("http://www.google.com");
When I var_dump the $_POST fields on the endpoint user/session it returns the array of postdata using Postman but $_POST fields are blank when sending via the web app. Even before it makes any request to the database, the post fields should be set right?
Via SSH this also works:
curl -F token=a559d5bba5a9e9517d5c3ed7aeb62db6 -F user=30972 http://example.com/api2/user/session
As suggested in comments I've tried:
var_dump(function_exists('curl_version'));
// bool(true)
I can't figure out what's going on.
Edit: This works ... but I don't want to use sockets. Must be a curl issue.
$fp = fsockopen('example.com', 80);
$vars = array(
'token' => 'a559d5bba5a9e9517d5c3ed7aeb62db6',
'user' => '30972'
);
$content = http_build_query($vars);
fwrite($fp, "POST /api2/user/session HTTP/1.1\r\n");
fwrite($fp, "Host: example.com\r\n");
fwrite($fp, "Content-Type: application/x-www-form-urlencoded\r\n");
fwrite($fp, "Content-Length: ".strlen($content)."\r\n");
fwrite($fp, "Connection: close\r\n");
fwrite($fp, "\r\n");
fwrite($fp, $content);
header('Content-type: text/plain');
while (!feof($fp)) {
echo fgets($fp, 1024);
}
Edit:
curl_error() also returns no error.
To better understand the differences between the PHP code and cURL, I created a RequestBin instance and tried both on it. They yielded drastically different results:
It seemed like the POST data from the PHP script yielded an incorrect result for what was sent. This can be fixed by using a built-in PHP function http_build_query.
It will yield a more apt result:
This can be caused by a session lock... If you use curl to access the same server, the same session is used. While the script is running, the session is locked by default, this means that the current request has to finish before another is handled for the same session. This would explain a timeout of the request in curl, as your first request is not completed and another is made...
Using session_write_close() before the curl_exec will unlock the session and correct the problem.
It turns out I needed to use http_build_query.
$vars = array(
'token' => 'a559d5bba5a9e9517d5c3ed7aeb62db6',
'user' => '30972'
);
$content = http_build_query($vars);
$ch = curl_init('http://example.com/api2/user/session');
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
'Content-Type: application/x-www-form-urlencoded',
'Content-Length: ' . strlen($content))
);
$result = curl_exec($ch);
I have a while loop, that takes ip's and passwords from a text file and logins to some servers that I rent using HTTP Auth.
<?php
$username = 'admin';
function login($server, $login){
global $username, $password, $server;
$options = array(
CURLOPT_URL => $server,
CURLOPT_HEADER => 1,
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_USERAGENT => "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 GTB5",
CURLOPT_HTTPHEADER => array("
Host: {$server}
User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Authorization: Basic {$login}
"));
$ch = curl_init();
curl_setopt_array($ch, $options);
$result = curl_exec($ch);
$http_status = curl_getinfo($ch, CURLINFO_HTTP_CODE);
if ( $http_status == 200 ) {
//do something
echo "Completed";
}
else { echo "Something went wrong";}
};
$file = fopen('myServers.txt', 'r');
while (! feof($file)) {
$m = explode(fgets($file), ':');
$password = $m[0];
$server = $m[1];
$login = base64_encode("{$username}:{$password}");
login($server, $login);
};
?>
The script works fine. However, when I load the page on my localhost, it takes forever to load and then prints out everything at once when its done with the entire file.
I want to print out Something went wrong or completed each time it does the file, I don't want it to wait for the entire file to go through the loop.
You're probably going to want to take a look at PHP flushing, which pushes content to the browser before continuing on with creating more page content. Note that from what I remember of PHP, you need to ob_flush() and flush() at the same time in order to properly flush content to the browser.
http://us3.php.net/flush
[Edit]
Example: You might try changing your echo statements to something resembling the below:
echo "Completed";
ob_flush();
flush();
Whether you can do what you want to do depends on the web server being used, and how it's configured, with regards to output buffering.
A good place to start reading would be the documentation for PHP's flush function.
A call to flush is intended to push output to the end user - but sometimes the web server implements it's own output buffering, which defeats the effect.
From the flush documentation:
Several servers, especially on Win32, will still buffer the output from your script until it terminates before transmitting the results to the browser.