I have this situation:
I want to make a PUT request using curl to a form but it keeps retrieving this error: "send failure: connection aborted". Here is the function I use:
function PutCurl($url,$parametros_post){
$ch = curl_init();
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch,CURLOPT_USERAGENT,'Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)');
curl_setopt($ch,CURLOPT_HEADER,false);
curl_setopt($ch,CURLOPT_RETURNTRANSFER,true);
curl_setopt($ch,CURLOPT_COOKIEJAR,'cookies.txt');
curl_setopt($ch,CURLOPT_COOKIEFILE,'cookies.txt');
curl_setopt ($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch,CURLOPT_PUT,true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Length: '.strlen($parametros_post)));
curl_setopt($ch,CURLOPT_POSTFIELDS,$parametros_post);
$result = curl_exec($ch);
$rerror = curl_error($ch);
curl_close($ch);
if ($rerror != ""){
echo '<h3>'.$rerror.'</h3>';
return false;
}
else
return $result;
}
I get this error:
While trying to process the request:
PUT /textos-del-anuncio/ HTTP/1.0
User-Agent: Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)
Host: www.somesite.com
Pragma: no-cache
Accept: /
Cookie: PHPSESSID=gkmb7ksi1o82d91rino35ihma3
Expect: 100-continue
Connection: Close
X-Forwarded-For: Ip
Via: 1.0 Proxy+ (v4.00 http://www.proxyplus.cz)
Proxy-Authorization: Basic a2F2OjNHd3BiK3J3QiE=
The following error was encountered:
* Invalid Request
Some aspect of the HTTP Request is invalid. Possible problems:
* Missing or unknown request method
* Missing URL
* Missing HTTP Identifier (HTTP/1.0)
* Request is too large
* Content-Length missing for POST or PUT requests
* Illegal character in hostname; underscores are not allowed
Then I tried to set Content-Length in the header adding this line to my function:
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Length: '.strlen($parametros_post)));
but then I get a "Send failure: Connection was aborted" error.
Please, someone who can help me. Thanks.
Something being wrong with the request, have you tried doing it like this:
curl_setopt($curl, CURLOPT_CUSTOMREQUEST, "PUT");
curl_setopt($curl, CURLOPT_POSTFIELDS, http_build_query($data));
So you do not use curl_setopt($ch,CURLOPT_PUT,true); but CURLOPT_CUSTOMREQUEST instead.
I assume your $parametros_post is an associative array like $parametros_post = array('post_field_name' => 'post_field_value');
Related
I'm trying to make a script that makes things easier for Apple users, I don't really want to expose it right now but those are the steps I need to follow.
Login to Apple and saving the cookies.
Getting the data from apple with the saved cookies.
Creating automation tools.
And i'm stuck in the first step, Curl isn't submitting the custom headers as I need to submit the json header.
Here is my code:
<?php
function login($url,$data){
$data=json_encode($data);
$login = curl_init();
curl_setopt($login, CURLOPT_COOKIEJAR, "cookie.txt");
curl_setopt($login, CURLOPT_COOKIEFILE, "cookie.txt");
curl_setopt($login, CURLOPT_NOBODY, false);
curl_setopt($login, CURLOPT_HEADER, false);
curl_setopt($login, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($login, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($login, CURLOPT_TIMEOUT, 40000);
curl_setopt($login, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($login, CURLOPT_URL, $url);
curl_setopt($login, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
curl_setopt($login, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($login, CURLOPT_POST, TRUE);
curl_setopt($login, CURLOPT_POST, 1);
curl_setopt($login, CURLOPT_HTTPHEADER, array(
'Content-Type: application/json',
'Content-Length: ' . strlen($data))
);
curl_setopt($login, CURLOPT_POSTFIELDS, $data);
//ob_start();
$rez=curl_exec ($login);
}
$data = array("accountName" => "myemail#me.com", "password" => "MyPassWord","rememberMe" => "true");
$url="https://idmsa.apple.com/appleauth/auth/signin";
login($url,$data);
?>
Whenever I try tampering the data when sending request, it never shows the custom headers
Apple uses X-Apple-Widget-Key like CSRF. I think you need to catch it. You can find it on source code. You can get status by using curl_getinfo()
// your code .....
curl_setopt($login, CURLOPT_POSTFIELDS, $data);
//ob_start();
$rez = curl_exec($login);
$status = curl_getinfo($login);
print_r($status);
This would be unmanageable as a comment, so here is how i see what goes on the wire:
$verbiage = null;
if ($this->verbose) {
curl_setopt($ch , CURLINFO_HEADER_OUT , false);
curl_setopt($ch , CURLOPT_VERBOSE , true);
$verbiage = fopen('php://temp' , 'w+');
curl_setopt($ch , CURLOPT_STDERR , $verbiage);
}
$resp = curl_exec($ch);
if ($this->verbose) {
rewind($verbiage);
$verboseLog = stream_get_contents($verbiage);
$log->info("Verbose information:\n$verboseLog");
fclose($verbiage);
}
curl_close($ch);
where $log is a log4php object, you pick your display method.
EDIT adding a sample log with this 'trace' option enabled
INFO - testCreateLabRequest starts.
TRACE - Query for lab request [dispensary[1]inboundLabRequest[2]ForPatient[1]] returned no records for feed [toubibCreamedFeedGuid]
DEBUG - Sending json
{ *redacted* , "trackingId":"5ac4473b-407f-4d7d-83eb-1a3f9abe5b8b"}
INFO - Verbose information:
* Trying fe80::3636:3bff:fecf:804c...
* Connected to darkmax.local (fe80::3636:3bff:fecf:804c) port 80 (#0)
> POST /ehr/feed/lab/request HTTP/1.1
Host: darkmax.local
Protocol_version : 1.1
Connection: Close
Content-Type: application/json ; charset=utf-8
Proxy-Connection: Close
User-Agent: Portable EHR(106)
Accept: application/json, text/html;
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Content-Length: 851
* upload completely sent off: 851 out of 851 bytes
< HTTP/1.1 200 OK
< Date: Wed, 31 Aug 2016 02:04:03 GMT
< Server: Apachee/2.4.12
< Access-Control-Allow-Origin: *
< Origin: http://darkmax.local
< Connection: close
< Transfer-Encoding: chunked
< Content-Type: application/json; charset=utf-8
<
* Closing connection 0
The lines with a < correspond to what my cURL session is receiving from the remote peer. Having some visibility into the actual HTTP protocol flying from me-to-peer-back-to-me has helped me many times figure out the nature of some underlying issue.
I have a php app which posts variable info to a flask app (which does some calculations and returns a result ). I'm running both locally on win7
When I test the url "127.0.0.1:5000/index" using a post with postman, I get a 200 status code (screenshot). However when the php app posts to the flask app I get:
The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.
I'm using CURL , and the verbose output is:
* About to connect() to 127.0.0.1 port 5000 (#0)
* Trying 127.0.0.1...
* connected
* Connected to 127.0.0.1 (127.0.0.1) port 5000 (#0)
> POST /index/ HTTP/1.1
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)
Host: 127.0.0.1:5000
Accept: */*
Content-Length: 338
Expect: 100-continue
Content-Type: multipart/form-data; boundary=----------------------------d5cb02e2edea
< HTTP/1.1 100 Continue
* HTTP 1.0, assume close after body
< HTTP/1.0 404 NOT FOUND
< Content-Type: text/html
< Content-Length: 233
< Server: Werkzeug/0.10.4 Python/2.7.5
My php code looks like:
$data= array('a'=>$a, 'token'=>$token);
$url="http://127.0.0.1:5000/index/";
$output = $this->my_model->get_data($url, $data);
public function get_data($url,$postFieldArray=FALSE) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookieFile);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)");
if ($postFieldArray!= FALSE) {
curl_setopt($ch, CURLOPT_POST, TRUE);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postFieldArray); //for django
}
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_STDERR, $verbose);
curl_setopt($ch, CURLOPT_URL, $url);
$html = curl_exec($ch);
curl_close($ch);
.......
return $result;
}
Simplified Flask app:
app = Flask(__name__)
app.debug = True
#app.route('/')
def hello_world():
return 'Hello World!'
#app.route('/index',methods=['POST'])
def index():
token = request.form['token']
a = request.form['a']
......
return
if __name__ == '__main__':
app.run()
What am I doing wrong?
You have a trailing slash in the $url variable in the PHP code. That won't work, because you don't have a trailing slash in your Flask code. Look here for more info, under the section "Unique URLs / Redirection Behavior"
I am opening a HTTPS page using cURL. The page I request issues a redirect request. I have set cURL to follow the redirect, but I cannot seem to be able to get it to request the correct page. I have tracked the same request in a browser and I see my browser making a different request to what cURL makes. What can I do to correct this? The correct URL is shown in the output of a verbose cURL dump. It follows the "* Issue another request to this URL"
Here is a snippet of the output from cURL's verbose output:
< HTTP/1.1 302 Moved Temporarily
< Location: /XXX
< Content-Type: text/html; charset=UTF-8
< Date: Tue, 31 Dec 2013 15:51:46 GMT
< Expires: Tue, 31 Dec 2013 15:51:46 GMT
< Cache-Control: private, max-age=0
< X-Content-Type-Options: nosniff
< X-Frame-Options: SAMEORIGIN
< X-XSS-Protection: 1; mode=block
< Server: GSE
< Alternate-Protocol: 443:quic
< Transfer-Encoding: chunked
<
* Ignoring the response-body
* Connection #0 to host 127.0.0.1 left intact
* Issue another request to this URL: 'XYYYZ'
* Re-using existing connection! (#0) with host 127.0.0.1
* Connected to 127.0.0.1 (127.0.0.1) port 8888 (#0)
> GET /??? HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:25.0) Gecko/20100101 Firefox/25.0
The PHP code I use follows:
$ch = curl_init();
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; rv:25.0) Gecko/20100101 Firefox/25.0');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_COOKIEJAR, COOKIE_FILE);
curl_setopt($ch, CURLOPT_COOKIEFILE, COOKIE_FILE);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_PROXY, '127.0.0.1:8888');
$target = ADDR;
curl_setopt($ch, CURLOPT_URL, $target);
$page = curl_exec($ch);
cURL follows the Location: Header, but be sure to send the exact headers (content-language, referer) browser does using CURLOPT_HTTPHEADER option because some servers refuse connectios to prevent automated requests. In Firefox you have live http headers to see what browser does.
Also make sure the Location: header contains the absolute url and not a relative path according to http 1.1.
If that dosen't work you can use the option CURLOPT_HEADER with curl_info to catch the 302 and redirect it manually.
Here i post an example to do it manually so you check if would produce an infinite loop.
$ch = curl_init();
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; rv:25.0) Gecko/20100101 Firefox/25.0');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_COOKIEJAR, COOKIE_FILE);
curl_setopt($ch, CURLOPT_COOKIEFILE, COOKIE_FILE);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_PROXY, '127.0.0.1:8888');
$target = ADDR;
curl_setopt($ch, CURLOPT_URL, $target);
$page = curl_exec($ch);
$curl_info = curl_getinfo($ch);
if ($curl_info['http_code'] == 302 || $curl_info['http_code'] == 301)
{
$response_headers = substr($page, 0, $curl_info['header_size']);
if (preg_match('#Location: (.*)#', $response_headers, $location_header))
{
// Call again curl to follow location; Better to wrap the curl process in a function called follow_location
// echo $location_header return an Array
// echo $location_header[0] return "Location: http//blablabla"
// echo $location_header[1] return URL only "http://blablbalba.com" and you can process with cURL :D
echo $location_header[1];
}
}
I am using PHP 5.3.6 and it seems I am unable to make a PUT request using CURL for PUTting just a string.
function put_data($url, $data)
{
$useragent="SimpleAgent-1.0";
$fh = fopen('php://memory', 'rw');
fwrite($fh, $data);
rewind($fh);$ch = curl_init();
curl_setopt($ch, CURLOPT_USERAGENT, $useragent);
curl_setopt($ch, CURLOPT_INFILE, $fh);
curl_setopt($ch, CURLOPT_INFILESIZE, strlen($data));
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
curl_setopt($ch, CURLOPT_PUT, 1);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$result = curl_exec($ch);
curl_close($ch);
fclose($fh);
return $result;
}
Here, $data is the string that I want to PUT.
Doesn't work and returns the following error:
500 Internal Server Error The server has either erred or is incapable
of performing the requested operation.
expected string or buffer
I used your code so defined a url and filled the data with a string and all worked as expected. Being I was rejected by the website as there was no receiving end that could deal with a put. To get info easily just add the line
curl_setopt($ch, CURLOPT_VERBOSE, true);
and you will get something like:
* About to connect() to yyyy.xxxx.com port 80 (#0)
* Trying 62.221.196.28...
* connected
* Connected to yyyy.xxxx.com (zz.221.196.28) port 80 (#0)
> PUT / HTTP/1.1
User-Agent: SimpleAgent-1.0
Host: yyyy.xxxx.com
Accept: */*
Content-Length: 14
Expect: 100-continue
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 405 Method Not Allowed
< Date: Thu, 09 Feb 2012 19:46:28 GMT
< Server: Apache
< Allow: GET,HEAD,POST,OPTIONS
< Vary: Accept-Encoding
< Content-Length: 231
< Content-Type: text/html; charset=iso-8859-1
<
* Connection #0 to host yyy.xxxx.com left intact
* Closing connection #0
As you can see from the logging the request went out, however when you want to put the data, the apache setting doesn't allow you to put data to that url. So depending upon the server you will have to take care of a receiving url that accepts the PUT.
I am only able to pass the array as a data with the versions I am using. So this is what I am doing now:
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "PUT");
curl_setopt($ch, CURLINFO_HEADER_OUT, TRUE);
curl_setopt($ch, CURLOPT_COOKIE, $curl_cookie);
$arr = array();
if (isset($data)) {
$arr['my_data'] = $data;
}
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($arr));
curl_exec($ch);
Using cURL to scrape a secure (i.e. login) page, and I'm at my wits' end. I managed to successfully scrape two sites with little or no problems, and now I just can't log into this one. cURL gets all the pages I ask it to, but they're all not logged in, which doesn't help. So maybe someone could spot a mistake I've missed?
The code is:
$url_to = 'http://fastorder.newrock.es/store2009/index.php/customer/account/loginPost/';
$url_from = 'http://fastorder.newrock.es/store2009/index.php/customer/account/login/';
$url_get = 'http://fastorder.newrock.es/store2009/index.php/';
$name_pass = 'login%5Busername%5D=*****&login%5Bpassword%5D=*****&send=';
function login($link,$user,$from) {
$fp = fopen("cookie.txt", "w");
fclose($fp);
$log = curl_init();
curl_setopt($log, CURLOPT_REFERER, $from);
curl_setopt($log, CURLOPT_URL, $link);
curl_setopt($log, CURLOPT_COOKIEJAR, "cookie.txt");
curl_setopt($log, CURLOPT_COOKIEFILE, "cookie.txt");
curl_setopt($log, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6");
curl_setopt($log, CURLOPT_TIMEOUT, 40);
curl_setopt($log, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($log, CURLOPT_HEADER, TRUE);
curl_setopt($log, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($log, CURLOPT_POST, TRUE);
curl_setopt($log, CURLOPT_POSTFIELDS, $user);
$data = curl_exec($log);
curl_close($log);
}
login($url_to,$name_pass,$url_from);
function get($url) {
$get = curl_init();
curl_setopt($get, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($get, CURLOPT_COOKIEFILE, "cookie.txt");
curl_setopt($get, CURLOPT_URL, $url);
return curl_exec ($get);
curl_close ($get);
}
$html = get($url_get);
echo $html;
This is the (more or less) same script that worked on the other two sites, and it manages to log in fine. What threw me off in the start are the codes in the $name_pass. Turns out the site has named name and password input fields as login[username] and login[password]. Why the hell for, I've no idea, but I've tried sending it both with codes and with brackets, and nothing helped.
Live HTTP Headers is giving me the following for the page:
http://fastorder.newrock.es/store2009/index.php/customer/account/loginPost/
POST /store2009/index.php/customer/account/loginPost/ HTTP/1.1
Host: fastorder.newrock.es
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Referer: http://fastorder.newrock.es/store2009/index.php/customer/account/login/
Cookie: frontend=6tjul97q4mvn0046ier0k79li8
Content-Type: application/x-www-form-urlencoded
Content-Length: 81
login%5Busername%5D=*****&login%5Bpassword%5D=*****&send=
HTTP/1.1 302 Found
Date: Fri, 26 Feb 2010 12:29:19 GMT
Server: Apache/2.0.63 (CentOS)
X-Powered-By: PHP/5.2.10
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Location: http://fastorder.newrock.es/store2009/index.php/customer/account/
Content-Length: 0
Connection: close
Content-Type: text/html; charset=UTF-8
I've tried to copy everything I could to the cURL script, thinking there's some obscure way of blocking the scrip from logging in. But right now I'm totally stuck and I've got no idea what to do next. And I've dug through a lot of tutorials, and they all give advices that worked like a charm for the first two sites.
Halp?
It may be this:
login%5Busername%5D=*****&login%5Bpassword%5D=*****&send=
I'm no curl guru, but your script seems to be OK, so maybe you should not escape the characters.
I would do local tests with curl and this kind of login forms. Maybe you can debug what's wrong from there. If I'm right, there will be empty fields.
Suggestion: Use Fiddler (www.fiddler2.com) to diff the request traffic, CURL vs your browser.
There is something broken with that store's registration/login. The activation email said to just login to activate the account. I've tried logging in multiple times but I get the error "This account is not activated." everytime I try to login.
Below is a quick change that prints the returned login page.
$url_to = 'http://fastorder.newrock.es/store2009/index.php/customer/account/loginPost/';
$url_from = 'http://fastorder.newrock.es/store2009/index.php/customer/account/login/';
$url_get = 'http://fastorder.newrock.es/store2009/index.php/';
$name_pass = 'login%5Busername%5D=*****&login%5Bpassword%5D=*****&send=';
function login($link,$user,$from) {
$fp = fopen("cookie.txt", "w");
fclose($fp);
$log = curl_init();
curl_setopt($log, CURLOPT_REFERER, $from);
curl_setopt($log, CURLOPT_URL, $link);
curl_setopt($log, CURLOPT_COOKIEJAR, "cookie.txt");
curl_setopt($log, CURLOPT_COOKIEFILE, "cookie.txt");
curl_setopt($log, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6");
curl_setopt($log, CURLOPT_TIMEOUT, 40);
curl_setopt($log, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($log, CURLOPT_HEADER, TRUE);
curl_setopt($log, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($log, CURLOPT_POST, TRUE);
curl_setopt($log, CURLOPT_POSTFIELDS, $user);
$data = curl_exec($log);
curl_close($log);
return $data;
}
echo login($url_to,$name_pass,$url_from);
function get($url) {
$get = curl_init();
curl_setopt($get, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($get, CURLOPT_COOKIEFILE, "cookie.txt");
curl_setopt($get, CURLOPT_URL, $url);
return curl_exec ($get);
curl_close ($get);
}
$html = get($url_get);
echo $html;
Edit:
Is the cookies data is being written to the cookies file (cookie.txt)? If not...
Check the file permissions, make sure its writable.
A bug in earlier versions of php5 caused the cookies file option to be ignored.
Details on the bug are here: http://bugs.php.net/bug.php?id=33475
Solution: Add unset($log) after curl_close($log);
Its hard to debug this script w/o being able to test it.