Here is the dump from WireShark:
POST /drm/drm_production_v2.php HTTP/1.1
content-length: 278
content-type: text/xml
User-Agent: UserAgent 1.0
Connection: Keep-Alive
Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=
host: www.example.com
<methodCall>
<methodName>aMethod</methodName>
<params>
<param>
<value>
<base64>dXNlcm5hbWU6cGFzc3dvcmQ=</base64>
</value>
</param>
<param>
<value>
<struct/>
</value>
</param>
</params>
</methodCall>
I have the xml saved into a seperate file. Here's what I am doing:
<?php
function httpsPost($Url, $xml_data, $headers)
{
// Initialisation
$ch=curl_init();
// Set parameters
curl_setopt($ch, CURLOPT_FORBID_REUSE, 1);
curl_setopt($ch, CURLOPT_FRESH_CONNECT, 1);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_URL, $Url);
// Return a variable instead of posting it directly
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_USERPWD,"username:password");
// Activate the POST method
curl_setopt($ch, CURLOPT_POST, 1) ;
curl_setopt($ch,CURLOPT_USERAGENT,"UserAgent 1.0");
// Request
curl_setopt($ch, CURLOPT_POSTFIELDS, $xml_data);
curl_setopt($ch, CURLOPT_TIMEOUT, 999);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
// execute the connexion
$result = curl_exec($ch);
// Close it
curl_close($ch);
return $result;
}
$str='username:password';
$auth=base64_encode($str);
$request_file = "./request.xml";
$fh = fopen($request_file, 'r');
$filesize=filesize($request_file);
echo $filesize;
$xml_data = fread($fh,$filesize);
fclose($fh);
$url = 'http://www.example.com';
$header = array();
$header[] = "POST /drm/drm_production_v2.php HTTP/1.1";
$header[] = "Content-type: text/xml";
$header[] = "content-length: ".$filesize. "\r\n";
$header[] = "User-Agent: UserAgent 1.0";
$header[] = "Connection: Keep-Alive";
$header[] = "Authorization: Basic ".$auth;
$header[] = "host: www.example.com";
$Response = httpsPost($url, $xml_data, $header);
echo $Response;
?>
It returns a 'Bad Request' from the server. Any suggestions?
My first guess is that the extra "\r\n" after the content-length header makes the server think that the post content starts there. I'd also change "content-length", "Content-type", and "host" to "Content-Length", "Contnet-Type", and "Host", just in case.
Edit: That, and Ronald Bouman's answer.
I think your argument to
curl_setopt($ch, CURLOPT_POSTFIELDS, $xml_data);
is not correct. The postfields option should be URL encoded name/value pairs. From the docs:
"
This can either be passed as a urlencoded string like 'para1=val1¶2=val2&...' or as an array with the field name as key and field data as value. If value is an array, the Content-Type header will be set to multipart/form-data
"
see http://php.net/manual/en/function.curl-setopt.php
There's the pcap2curl tool that allows one to convert a pcap file of HTTP requests into cURL.
But if you want to replay some web requests from your browser then you can do it without Wireshark and instead enter Web developer mode in the browser. You then go to the network requests view and right click on the request of interest then, in most modern browsers, there is an option to Copy as cURL after which you can then paste the resulting command into a terminal and rerun the captured command as you see fit using the curl tool. Some browsers (e.g. Mozilla) also offer the option to edit and resend from within the browser.
Related
I need to execute 2 requests in parallel using cURL to get a reply from the web service.
The problem is that I need to get the encrypted password from the first XML's output and pass it to the second XML to get 100 success response from the API.
Currently, I have created 2 cURL to achieve this but the API responds "101 Password Expired" because the encrypted password is valid only for the first request.
Here is my code for reference:
1st cURL:
$soapUrl = "http://localhost:54934/frmMutualFund.asmx?op=getPassword"; // asmx URL of WSDL
// xml post structure
$xml_post_string = '<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
<getPassword xmlns="http://tempuri.org/">
<pUserId>12345</pUserId>
<pPassword>98765</pPassword>
<pPasskey>ksjhdfksj</pPasskey>
</getPassword>
</soap:Body>
</soap:Envelope>'; // data from the form, e.g. some ID number
$headers = array(
"Content-type: text/xml;charset=\"utf-8\"",
"Accept: text/xml",
"Cache-Control: no-cache",
"Pragma: no-cache",
"SOAPAction: http://tempuri.org/getPassword",
"Content-length: ".strlen($xml_post_string),
); //SOAPAction: your op URL
$url = $soapUrl;
// PHP cURL for https connection with auth
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 1);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
//curl_setopt($ch, CURLOPT_USERPWD, $soapUser.":".$soapPassword); // username and password - declared at the top of the doc
curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $xml_post_string); // the SOAP request
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
// converting
$response = curl_exec($ch);
2nd cURL:
$soapUrl = "http://localhost:54934/frmMutualFund.asmx"; // asmx URL of WSDL
// xml post structure
$xml_post_string = '<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
<MFUAPI xmlns="http://tempuri.org/">
<pFlag>06</pFlag>
<pUserId>12345</pUserId>
<pEncPassword>'.$response.'</pEncPassword>
<pParam>some_parameters</pParam>
</MFUAPI>
</soap:Body>
</soap:Envelope>'; // data from the form, e.g. some ID number
$headers = array(
"Content-type: text/xml;charset=\"utf-8\"",
"Accept: text/xml",
"Cache-Control: no-cache",
"Pragma: no-cache",
"SOAPAction: http://tempuri.org/MFUAPI",
"Content-length: ".strlen($xml_post_string),
); //SOAPAction: your op URL
$url = $soapUrl;
// PHP cURL for https connection with auth
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 1);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
//curl_setopt($ch, CURLOPT_USERPWD, $soapUser.":".$soapPassword); // username and password - declared at the top of the doc
curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $xml_post_string); // the SOAP request
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
// converting
$response = curl_exec($ch);
curl_close($ch);
// converting
$response1 = str_replace("<soap:Body>","",$response);
$response2 = str_replace("</soap:Body>","",$response1);
// convertingc to XML
$parser = simplexml_load_string($response2);
// user $parser to get your data out of XML response and to display it.
print_r($parser);
first
I need to execute 2 requests in parallel
then
The problem is that I need to get the encrypted password from the
first XML's output and pass it to the second XML to get 100 success
response from the API.
if the request body of the 2nd request depends on the response of the first request, then you cannot do these 2 requests in parallel.
(generally speaking. you might be able to do some micro-optimization where you send the first request, and partially send the 2nd request, then wait for the password of the first request to be retrieved, then finish the 2nd request, but it'd be difficult to pull off, the gains would probably be very small and if you time it wrong you'll probably get a timeout for the 2nd request and have to start the 2nd request all over again, which would be even slower than just doing the 2 requests sequentially in the first place. you'll also need reliable recovery code for 2nd request timeout'ed while waiting for the password of the 1st request, it would be easy to code it wrong.)
I need to form a request under this format:
POST /CJW/cjws.asmx HTTP/1.1
Host: www.somesite.abc.edu.sg
Content-Type: text/xml; charset=utf-8
Content-Length: length
SOAPAction: "http://tempuri.org/GetAll"
<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
<GetAll xmlns="http://tempuri.org/">
<ErrorMsg>string</ErrorMsg>
</GetAll>
</soap:Body>
</soap:Envelope>
So here is what I tried:
<?php
$username = '';
$password = '';
$credentials = $username.":".$password;
//$url = "http://tempuri.org/GetAll";
$url = "http://www.somesite.abc.edu.sg/GetAll";
$soap_request = '<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
<GetAll xmlns="http://tempuri.org/">
<ErrorMsg>string</ErrorMsg>
</GetAll>
</soap:Body>
</soap:Envelope>';
$header = array(
"POST /CJW/cjws.asmx HTTP/1.1 \n",
"Host: www.somesite.abc.edu.sg \n",
"Content-type: text/xml; charset=\"utf-8\" \n",
"Content-length: ".strlen($soap_request)." \n",
"SOAPAction: ".$url." \n");
$curl = curl_init();
curl_setopt($curl, CURLOPT_PROXY, '127.0.0.1:8888');
curl_setopt($curl, CURLOPT_HTTPAUTH, CURLAUTH_BASIC ) ;
curl_setopt($curl, CURLOPT_USERPWD, $credentials);
curl_setopt($curl, CURLOPT_SSLVERSION,3);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, 2);
curl_setopt($curl, CURLOPT_HEADER, true);
curl_setopt($curl, CURLOPT_POST, 1);
curl_setopt($curl, CURLOPT_POSTFIELDS, $soap_request );
curl_setopt($curl, CURLOPT_HTTPHEADER, $header);
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)");
curl_setopt($curl, CURLOPT_URL, $url);
// perform the request
$xml_result = curl_exec($curl);
// check for errors
if ($xml_result === false) {
$error_occurred = true;
}
else {
echo $xml_result;
}
?>
But I cannot get the response correctly, actually, I do not see the response except this error:
Length Required
HTTP Error 411. The request must be chunked or have a content length.
I tried many way including remove the length part, change it with some fix number, etc. but it changes nothing.
So what I am doing wrong here ? How is the right way ?
Thanks a lot !
(You can remove Host: and Content-Length: from the headers you synthesize because curl will add them both itself if you don't set them.)
You should inspect your outgoing request in detail and verify that it looks exactly the way you want it to. Using CURLOPT_VERBOSE is usually enough but you need to be able to get that output somewhere for inspection.
I believe your custom headers all have a trailing space and a newline that you should remove. They will severely screw up the outgoing request. Oh, and the POST part must be removed from your custom headers (it is not a header, it is the "request-line").
I need a little help here:
I have 2 files
index.php
form0.html
form0.html automatically fill a form and send it.
When I go straight to the fill it works fine, but when I try to access it through my php script it won't work unless I print the results.
PHP CODE:
<?php
set_time_limit(30);
$delay_time = 2; // time to wait before looping again.
$loop_times = 1; // number of times to loop.
$url = array("http://localhost/htmlfile0.html");
for($x=0;$x<$loop_times;$x++)
{
echo count($url);
for($i=0;$i<count($url);$i++)
{
$url1=$url[$i];
$curl = curl_init(); // Initialize the cURL handler
$header = array();
$header[0] = "Accept: text/xml,application/xml,application/xhtml+xml,";
$header[0] .= "text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
$header[] = "Cache-Control: max-age=0";
$header[] = "Connection: keep-alive";
$header[] = "Keep-Alive: 30000";
$header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
$header[] = "Accept-Language: en-us,en;q=0.5";
$header[] = "Pragma: "; // browsers keep this blank.
$var_host = parse_url($url1,PHP_URL_HOST);
$cookieJar = 'cookies/'.$var_host.'.txt'; // change it according to your requirement. make dynamic for multi URL
curl_setopt($curl, CURLOPT_HTTPHEADER, $header); // Browser like header
curl_setopt($curl, CURLOPT_COOKIEJAR, $cookieJar); // file for cookies if site requires
curl_setopt($curl, CURLOPT_COOKIEFILE, $cookieJar); // file for cookies if site requires
curl_setopt($curl, CURLOPT_HEADER, 0);
curl_setopt($curl, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9) Gecko/2008052906 Firefox/3.0');
curl_setopt($curl, CURLOPT_ENCODING, 'gzip,deflate');
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, 1); // Follow any redirects, just in case
curl_setopt($curl, CURLOPT_AUTOREFERER, true);
curl_setopt($curl,CURLOPT_RETURNTRANSFER, true); // set curl to return the page
curl_setopt($curl, CURLOPT_POSTFIELDS, '');
curl_setopt($curl, CURLOPT_POST, true); //post the form
curl_setopt($curl, CURLOPT_URL, $url1); // Set the URL
$ch=curl_exec($curl); // Display page
curl_close($ch); // Close cURL handler
echo date('h:i:s') . "\n";
if($ch) echo "Success: ".$url1;
else echo "Fail: ".$url1;
echo '<hr>';
sleep(5);
echo date('h:i:s') . "\n";
}
if($x < $loop_times) sleep($delay_time);
}
?>
How can I get pass this?
Thanks.
Correct me if I'm wrong but you're trying to execute JS using CURL request which you cant' at least not directly.
When you go straight to file, it works fine - your browser's JS interpreter executes JS proprerly and form is submitted, but by using cURL you are doing something different - you are sending http request headers to certain URL(http://localhost/htmlfile0.html) and you may and may not fetch the response content.
If you do fetch the content (and parse JS in your browser's js interpreter) javascript may be set to refuse correct action based on wheather it is or it is not correct url.
Example:
do some action if it is right URL - if you reached code on http://example.com/script.html
do not do anything if it's not above URL - and that's the case when you perform action using your cURL php script to reach above URL with and to output document's code in http://localhost.
I've been trying to post XML and get response from the server but with no luck.
Here are the conditions on server side:
Requests to the server should be sent as XML over HTTP 1.1.
The following requirements apply to the HTTP request:
The request type should be POST;
A Content-Length header should be present, and the total length of the request should be below 16KB;
A Content-Type header should be present, containing the media type value text/xml;
Here is my script:
$url = "http://somedomain.com";
$xml = '<?xml version="1.0" encoding="UTF-8"?>
<Request PartnerID="asasdsadsa" Type="TrackSearch"> <TrackSearch> <Title>love</Title> <Tags> <MainGenre>Blues</MainGenre> </Tags> <Page Number="1" Size="20"/> </TrackSearch> </Request>';
$header = "POST HTTP/1.1 \r\n";
$header .= "Content-type: text/xml \r\n";
$header .= "Content-length: ".strlen($xml)." \r\n";
$header .= "Connection: close \r\n\r\n";
$header .= $xml;
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, $header);
$data = curl_exec($ch);
echo $data;
if(curl_errno($ch))
print curl_error($ch);
else
curl_close($ch);
This gives me:
HTTP Error 400. The request URL is invalid.
Bad Request - Invalid URL
Does this help?
<?php
$url = "http://stackoverflow.com";
$xml = '<?xml version="1.0" encoding="UTF-8"?><Request PartnerID="asasdsadsa" Type="TrackSearch"> <TrackSearch> <Title>love</Title> <Tags> <MainGenre>Blues</MainGenre> </Tags> <Page Number="1" Size="20"/> </TrackSearch> </Request>';
$headers = array(
"Content-type: text/xml",
"Content-length: " . strlen($xml),
"Connection: close",
);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $xml);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
$data = curl_exec($ch);
echo $data;
if(curl_errno($ch))
print curl_error($ch);
else
curl_close($ch);
?>
From the documentation.
CURLOPT_CUSTOMREQUEST
A custom request method to use instead of "GET"
or "HEAD" when doing a HTTP request. This is useful for doing "DELETE"
or other, more obscure HTTP requests. Valid values are things like
"GET", "POST", "CONNECT" and so on; i.e. Do not enter a whole HTTP
request line here. For instance, entering "GET /index.html
HTTP/1.0\r\n\r\n" would be incorrect.
The parameter for CURLOPT_CUSTOMREQUEST should be simply POST and you can use CURLOPT_HTTPHEADER to set headers.
I am trying to post a xml string to a remote perl script via cURL. I want the xml string to be posted as a post parameter 'myxml'. See the code I am using below:
$url = 'http://myurl.com/cgi-bin/admin/xml/xml_append_list_init.pl';
$xml = '<?xml version="1.0" standalone="yes"?>
<SUB_appendlist>
<SUB_user>username</SUB_user>
<SUB_pass>password</SUB_pass>
<list_id>129</list_id>
<append>
<subscriber>
<address>test#test.comk</address>
<first_name>Test</first_name>
<last_name>Test</last_name>
</subscriber>
</append>
</SUB_appendlist>';
$ch = curl_init(); //initiate the curl session
curl_setopt($ch, CURLOPT_URL, $url); //set to url to post to
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); // tell curl to return data in a variable
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array("Content-Type: text/xml", "Content-length: ".strlen($xml)));
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, 'myxml='.urlencode($xml)); // post the xml
curl_setopt($ch, CURLOPT_TIMEOUT, (int)30); // set timeout in seconds
$xmlResponse = curl_exec($ch);
curl_close ($ch);
However the remote server is not seeing the data in the 'myxml' parameter. And I get the following response back in $xmlResponse
HTTP/1.1 200 OK
Date: Fri, 15 Apr 2011 12:00:44 GMT
Server: Apache/2.2.9 (Debian)
Vary: Accept-Encoding
Content-Length: 0
Content-Type: text/html; charset=ISO-8859-1
I'm not a cURL expert by any measure so I may be doing something in mu cURL request which is obviously wrong. Would appreciate it if anyone can shed any light or spot any problems in this. Hope that is enough information.
Cheers,
Adrian.
The body of your message is not text/xml data. It is application/x-www-form-urlencoded data. You have form data containing XML, not plain XML.
Your problem is akin to trying to open MyDoc.zip in MS Word. You have to deal with it as a zip file before dealing with it as Word.
Based on my reading of the PHP manual, you want to remove:
curl_setopt($ch, CURLOPT_HTTPHEADER, array("Content-Type: text/xml", "Content-length: ".strlen($xml)));
and change the POSTFIELDS line to:
curl_setopt($ch, CURLOPT_POSTFIELDS, array(
'myxml' => $xml
));
That is not the correct content type for all browsers.
see this article
sometimes the content type for xml is: application/rss+xml