We got a couple of apps, utilizing the car2go Rest-API with OAuth 1.0.
All our web apps stopped working 2 days ago. All curl POST requests are failing now with the following error:
400 Bad Request
Your browser sent a request that this server could not understand.
Error code: 53
Parser Error: [Content-Length: -]
I spend a lot of time trying to figure out if the problem is my oauth workflow. But in the end all parameters and signatures and stuff is correct. I successfully fire the POST via Postman (REST-Client)
So my conclusion is that somehow the php code for the curl is suddenly not working anymore.
This is the (very ugly) curl function. A difference to most tutorials about curl POST is, that I'm passing a full URL with all parameters already attached, so I don't need CURLOPT_POSTFIELDS.
function curlAPI($params) {
//open connection
$ch = curl_init();
$url = $params['url'];
curl_setopt($ch,CURLOPT_HEADER,false);
//set the url, number of POST vars, POST data
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION,true);
curl_setopt($ch, CURLOPT_MAXREDIRS,50);
curl_setopt($ch, CURLOPT_TIMEOUT_MS, 5000);
if($params['type'] == 'POST') {
// POST
curl_setopt($ch,CURLOPT_POST, true);
} else if($params['type'] == 'DELETE') {
// DELETE
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "DELETE");
} else if($params['type'] == 'PUT') {
$update_json = array();
// PUT
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "PUT");
curl_setopt($ch, CURLOPT_POSTFIELDS,'');
} else {
// GET
curl_setopt($ch,CURLOPT_POST,0);
}
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
//execute post
$result['result'] = curl_exec($ch);
// debug
if (FALSE === $result['result']) {
$result['errorInfo'] = curl_error($ch).' - '.curl_errno($ch);
}
$reponseInfo = array();
$reponseInfo['info'] = curl_getinfo($ch);
$reponseInfo['error'] = curl_error($ch);
//close connection
curl_close($ch);
$result['reponseInfo'] = $reponseInfo;
return json_encode($result);
}
Ok, this is what fixed this nightmare:
curl_setopt($ch,CURLOPT_HTTPHEADER ,array('Content-Length: 0'));
Aparrently it's not ok to send a curl POST without a header.
Related
I'm working on a project and am attempting to link a Java application to my website/database via an API.
I'm running on an aws ec2 instance. The problem I am encountering is using Curl to POST data to another webpage. (The end-product will require proxies, and I've decided to use curl to manage this.)
The PHP code that makes the request:
<?php
function get_url($url)
{
$ch = curl_init();
if($ch === false)
{
die('Failed to create curl object');
}
// Temporary, just trying to get POST value to work.
$data = "password=temp";
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$data2 = curl_exec($ch);
curl_close($ch);
return $data2;
}
echo get_url("www.mywebsite.org/API/myscript.php");
?>
The server-side code that handles the request:
<?php
echo "Password: " . $_POST["password"];
?>
The issue I am having is with passing the post variables through curl and retrieving them.
The output on the page is:
Password:
and the POST variables are not being set. I'm thinking I am missing a CURLOPT somewhere but can't figure out which one!
I'm doing a project and one of the requests is to automatically submit a form to a random site, which is specified from time to time
This is my cURL code :
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url_post);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $dati_post);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$result = curl_exec($ch);
curl_close($ch);
Code to get POST parameters :
foreach ($html->find('form[method=post],form[method=POST]') as $forms_post) {
$n_formPOST++;
$formPOST_action = $forms_post->action;
foreach ($forms_post->find('input') as $input) {
if ($input->type == 'text') {
$dati_testo[$input->name] = "text";
} else if ($input->type == 'email' || $input->name == 'email') {
$dati_random[$input->name] = "emailrandom#gmail.com";
} else if ($input->type == 'hidden') {
$dati_random[$input->name] = $input->value;
} else {
$dati_random[$input->name] = "random";
}
}
foreach ($forms_post->find('textarea') as $textarea) {
if ($textarea->disabled != true) {
$dati_testo[$textarea->name] = "text";
}
}
foreach ($forms_post->find('button') as $bottone) {
if ($bottone->type == 'submit') {
$dati_random[$bottone->name] = "random";
}
}
The problem is that in some sites POST is done correctly and I receive the right answer, which corresponds to the answer I would receive by doing it manually.
On other sites, it seems that the form is not submitted.
I have repeatedly checked the URL that I insert in the cURL and also the data that I pass and if I use them manually it works.
I even tried using online tools that perform POST / GET passing the same URL and the same data that I get in my project and it works.
The url_post is made from url host+form action.
I don't understand if there is something wrong in my curl code, considering I'm pretty sure the data I'm passing to the curl are corrects to complete the POST.
Data :
Site URL : http://www.comune.ricigliano.sa.it/
Form Action : index.php?action=index&p=228
Url_post : http://www.comune.ricigliano.sa.it/index.php?action=index&p=228
POST data :
'qs' => 'something to research'
'Submit2' => 'Cerca'
You need to use the curl_exec() function in order to execute the cURL. It takes the $ch param, like such:
// Execute cURL
curl_exec($ch);
// Close cURL
curl_close($ch);
More info here
If I get title of the page, I can tell the download link is active or dead.
For example: "Free online storage" is title of dead link and "[file name]" is the title of active link (mediafire). But my page takes too long to respond, so is there any other way to check if a download link is active or dead?
That is what i have done:
<?php
function getTitle($Url){
$str = file_get_contents($Url);
if(strlen($str)>0){
preg_match("/\<title\>(.*)\<\/title\>/",$str,$title);
return $title[1];
}
}
?>
Do not perform a GET request, which downloads the whole page/file, but HEAD request, which gets only the HTTP headers, and check if the status is 200, and the content-type is not text/html
Something like this...
function url_validate($link)
{
#[url]http://www.example.com/determining-if-a-url-exists-with-curl/[/url]
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $link);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_NOBODY, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_MAXREDIRS, 10); //follow up to 10 redirections - avoids loops
$data = curl_exec($ch);
curl_close($ch);
preg_match_all("/HTTP\/1\.[1|0]\s(\d{3})/",$data,$matches);
$code = end($matches[1]);
if(!$data)
{
return(false);
}
else
{
if($code==200)
{
return(true);
}
elseif($code==404)
{
return(false);
}
}
}
You can safely use any cURL library function. It is legitimate and thus would not regarded as a hacking attempt. The only requirement is that your web hosting company has cURL extension installed, which is very likely.
cURL should do the job. You can check the headers returned and the text content as well if you want.
I have the following code for cURL using PHP;
$product_id_edit="Playful Minds (1062)";
$item_description_edit="TEST";
$rank_edit="0";
$price_type_edit="2";
$price_value_edit="473";
$price_previous_value_edit="473";
$active_edit="1";
$platform_edit="ios";
//set POST variables
$url = 'https://www.domain.com/adm_test/phpgen/offline_items.php?operation=insert';
$useragent = 'Mozilla/5.0 (Windows NT 6.1; rv:8.0.1) Gecko/20100101 Firefox/8.0.1';
$fields = array(
'product_id_edit'=>urlencode($product_id_edit),
'item_description_edit'=>urlencode($item_description_edit),
'rank_edit'=>urlencode($rank_edit),
'price_type_edit'=>urlencode($price_type_edit),
'price_value_edit'=>urlencode($price_value_edit),
'price_previous_value_edit'=>urlencode($price_previous_value_edit),
'active_edit'=>urlencode($active_edit),
'platform_edit'=>urlencode($platform_edit)
);
$fields_string="";
//url-ify the data for the POST
foreach($fields as $key=>$value) { $fields_string .= $key.'='.$value.'&'; }
rtrim($fields_string,'&');
//open connection
$ch = curl_init();
//set the url, number of POST vars, POST data
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
//add useragent
curl_setopt($ch, CURLOPT_USERAGENT, $useragent);
curl_setopt ($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch,CURLOPT_POSTFIELDS,$fields_string);
curl_setopt($ch,CURLOPT_POST,count($fields));
//execute post
$result = curl_exec($ch);
if(curl_errno($ch)){
print "" . curl_error($ch);
}else{
//print_r($result);
}
$httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
//echo "HTTP Response Code: " . curl_error($ch);
echo $httpCode;
//close connection
curl_close($ch);
I have $httpCode printed; I get the code 200; I presume this is OK as I have read in the Manual Pages, however, when I check against the site, the POSTed values does not exist,
does this have something to do with cross-domains as I am not posting it on the same domain?, I'm doing it on 127.0.0.1/site/scrpt.php but how do I get the response code 200 if its not successful?
I also tried to get a 404 which I did by removing a part on the request URL it did return a 404 (this means that cURL is working properly in my assumption)
Does having the url https://www.domain.com/adm_test/phpgen/offline_items.php?operation=insert with the "?operation=insert" has something to do with it?
Let's presume(tho not implied), I'm from another site and I want post values into the form of another website sort'a a robot. tho my objective does not imply any evil intentions, is it that I have to encode thousand lines of info if this is not doable.
Likewise, I don't need a response from the server (but if one is available, then its just fine)
The operation should be passed with CURLOPT_POSTFIELDS. Along with other paramters.
Cross-domain issue happens in case of browser. And your code seems to be a php server side code so this should not be an issue.
Not sure if this is the solution or the problem is different, but this line:
rtrim($fields_string,'&');
Should be this:
$fields_string = rtrim($fields_string,'&');
curl_setopt($ch,CURLOPT_POST,TRUE);
CURLOPT_POST - boolean, it's not a count of values, it's use post flag.
Code 200 indicates that the connection is set up correctly and received a response from the server, but it does not mean that the requested action has been implemented.
Print $result after request to see the response from a web server.
I have a frontend code
$ch = curl_init();
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
//curl_setopt($ch, CURLOPT_HTTPHEADER, array('Accept: application/json'));
curl_setopt($ch, CURLOPT_URL, $url);
//make the request
$responseJSON = curl_exec($ch);
$response_status = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);
if ($response_status == 200) { // success
// remove any "problematic" characters from the json string and then decode
if (debug) {
echo "----finish API of getAPI inside basic_function with status==200---";
echo "<br>";
echo "-------the json response is-------" ; //.$responseJSON;
var_dump($responseJSON);
//print_r($responseJSON);
echo "<br>";
}
return json_decode( preg_replace( '/[\x00-\x1F\x80-\xFF]/', '', $responseJSON ) );
}
and I have a backend code which executed when cURL fired its operation with its URL. The backend code would therefore activated. So, I know cURL is operating.
$output=array (
'status'=>'OK',
'data'=>'12345'
)
$output=json_encode($output)
echo $output;
and $output shown on browser as {"status":"OK","data":"12345"}
However, I gone back to the frontend code and did echo $responseJSON, I got nothing. I thought the output of {"status":"OK","data":"12345"} would gone to the $responseJSON. any idea?
Here's output on Browser, something is very odd! the response_status got 200 which is success even before the parsing of API by the backend code. I expect status =200 and json response after the {"status":"OK","data":"12345"}
=========================================================================================
inside the get API of the basic functions
-------url of cURL is -----http://localhost/test/api/session/login/?device_duid=website&UserName=joe&Password=1234&Submit=Submit
----finish API of getAPI inside basic_function with status==200---
-------the json response is-------string(1153)
"************inside Backend API.php******************
---command of api is--/session/login/
---first element of api is--UserName=joe
--second element of api is---Password=1234
---third element of api is----Submit=Submit
----fourth element of api is---
-------inside session login of api-------------
{"status":"OK","data":"12345"}
Have you tried with curl_setopt($ch, CURLOPT_TIMEOUT, 10); commented?
See what happends if you comment that line.
Also try with the a basic code, if that works, smthing you added later is wrong:
// create a new cURL resource
$ch = curl_init();
// set URL and other appropriate options
curl_setopt($ch, CURLOPT_URL, "http://www.example.com/");
curl_setopt($ch, CURLOPT_HEADER, false);
// grab URL and pass it to the browser
curl_exec($ch);
// close cURL resource, and free up system resources
curl_close($ch);
Try var_dump($responseJSON)
If it returns false try
curl_error ( $ch )
Returns a clear text error message for the last cURL operation.
Are you sure your $url is correct?