Using php curl to fill form on asp.net page - php

There is a asp.net driven site where I can login and fill a form to place a order after a succesful login. This needs to be done by my server with curl.
Whenever I do this with my browser there is nothing special.
I go to example.com/login.aspx. Fill in fields and post it back. I get a index page back for logged in users. After this i need to click on a link to the form for order placement. Doing this gives me the form, which I fill and send back.This yields the wanted result of an placed order. Analysing the source revealed the page gets submitted through javascript,but the server cannot know how the request is sent if the request is identical. During this process i kept track of all the post requests send from my browser.
Having all the post requests, i wrote some code to do the same with curl.
The algorithm is:
Send a get request to obtain the example.com/login.aspx page
Parse the responds for any hidden fields(as they are used to mimic a statefull connection)
Post the state and the logindata to example.com/login.aspx
Parse the state send back and update it
Get form for order placement
Parse & update the state again
Post the state and the same post data that my browser sent
Print info to file instead of STDERR
During the process cookies are used with curl.
Everything goes smooth until the last POST. Instead of a "Success" or "Failure" result, I get my original form returned to me, with no validation errors mentioned. If I delibarely fill in gibberish or leave some things empty with my browser I get validation errors shown.
I have no experience with asp.net and am completely stuck here, so forgive me if this question is too specific.
Question:
If the headers and post data are exactly the same as from my browser, how can my curl operation have a different result?
For completeness:
POST from browser:
POST /DataEntry.aspx HTTP/1.1
Host: www.iftin.co
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:16.0) Gecko/20100101 Firefox/16.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Language: nl,en-us;q=0.7,en;q=0.3
Accept-Encoding: gzip, deflate
Connection: keep-alive
Referer: https://www.iftin.co/DataEntry.aspx
Cookie: ASP.NET_SessionId=SomethingX; .hawAuth=SomethingY
Content-Type: application/x-www-form-urlencoded
Content-Length: 10111
Post send by Curl:
POST /DataEntry.aspx HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:16.0) Gecko/20100101 Firefox/16.0
Host: www.iftin.co
Referer: https://www.iftin.co/DataEntry.aspx
Cookie:.hawAuth=SomethingZ; ASP.NET_SessionId=SomethingW
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Language: nl,en-us;q=0.7,en;q=0.3
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 25236
Expect: 100-continue
Content-Type: application/x-www-form-urlencoded;boundary=----------------------------5bb8ec4e5ecc
If people want to see some code, let me know. Thanks for reading and spending time on this.
Ater writing this, I noticed that the Content-Length for my curl request is substantially bigger than for my browsers request. I copied all parameters in the body of the Post of the browser to my code and passed it to curl in a array.
So curl has to add some data to the request by default. Is this the case?
The CURLOPT_SSL_VERIFYPEER is set to false, while the page uses SSL. Could this be it? Why?

Related

PayPal sandbox IPN validation always return INVALID

I googled this question and tried all suggestions but nothing works.
I tried this code: https://developer.paypal.com/docs/classic/ipn/ht_ipn/ but it don't work. Just copy-pasted it and deleted old magick_quotes routunes.
I tried this code: http://samples.geekality.net/view-source.php?file=ipn/ipn_handler.class.php and it also do not work.
In all cases I tried to do following:
$req = 'cmd=_notify-validate&' . file_get_contents('php://input');
To ensure I sent to IPN exactly what it sent to me. In addition I used the debug proxy (Fiddler) and saved what IPN sent to me and what I sent to IPN. The requests bodies are byte-to-byte identical except my request is prefixed by the cmd=_notify-validate& string.
Yes, I checked I use proper sandbox URL. Here are entire requests bodies:
What IPN sent to me: (I just replaced personal data to XXX)
POST http://localhost.loc/en/payment/success/1 HTTP/1.1
Host: localhost.loc
Connection: keep-alive
Content-Length: 921
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Origin: null
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.81 Safari/537.36
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip, deflate
Accept-Language: ru-RU,ru;q=0.8,en-US;q=0.6,en;q=0.4
mc_gross=1.00&protection_eligibility=Ineligible&payer_id=5XNKM66NSDKC4&tax=0.00&payment_date=05%3A34%3A11+Jun+01%2C+2015+PDT&payment_status=Completed&charset=utf-8&first_name=XXX&mc_fee=0.33&notify_version=3.8&custom=topup%3A262262%3A1%3A1433162020&payer_status=verified&business=XXX&quantity=1&payer_email=XXX&verify_sign=AG58dBsn5g2z8O8NEjotbuJGP14PAIpZ4k26VL8IyhaDPkcDRj002Keq&memo=hmgvjgjhgfjhfggjhfjtfgjh&txn_id=4CN141026K278934Y&payment_type=instant&last_name=XXX&receiver_email=XXX&payment_fee=0.33&receiver_id=DCMXPXGX4QX6J&txn_type=web_accept&item_name=Account+top+up&mc_currency=USD&item_number=Account+262262+top+up&residence_country=US&test_ipn=1&handling_amount=0.00&transaction_subject=topup%3A262262%3A1%3A1433162020&payment_gross=1.00&shipping=0.00&auth=ANSTBwT3znll-gJQZO2cLoV5QJFW9v8W.FqyWxffdtI0L-9mfsoe2xRL44M86Sn2XtYGtcqG4Fjjel1kdYZyxpQ
What I sent to IPN:
POST https://www.sandbox.paypal.com/cgi-bin/webscr HTTP/1.1
Host: www.sandbox.paypal.com
Accept: */*
Content-Length: 942
Content-Type: application/x-www-form-urlencoded
cmd=_notify-validate&mc_gross=1.00&protection_eligibility=Ineligible&payer_id=5XNKM66NSDKC4&tax=0.00&payment_date=05%3A34%3A11+Jun+01%2C+2015+PDT&payment_status=Completed&charset=utf-8&first_name=XXX&mc_fee=0.33&notify_version=3.8&custom=topup%3A262262%3A1%3A1433162020&payer_status=verified&business=XXX&quantity=1&payer_email=XXX&verify_sign=AG58dBsn5g2z8O8NEjotbuJGP14PAIpZ4k26VL8IyhaDPkcDRj002Keq&memo=hmgvjgjhgfjhfggjhfjtfgjh&txn_id=4CN141026K278934Y&payment_type=instant&last_name=XXX&receiver_email=XXX&payment_fee=0.33&receiver_id=DCMXPXGX4QX6J&txn_type=web_accept&item_name=Account+top+up&mc_currency=USD&item_number=Account+262262+top+up&residence_country=US&test_ipn=1&handling_amount=0.00&transaction_subject=topup%3A262262%3A1%3A1433162020&payment_gross=1.00&shipping=0.00&auth=ANSTBwT3znll-gJQZO2cLoV5QJFW9v8W.FqyWxffdtI0L-9mfsoe2xRL44M86Sn2XtYGtcqG4Fjjel1kdYZyxpQ
Can anyone help me what I do wrong?
Thanks.
AARRRRGH!!!!!!!! I have only dirty words to PayPal!!!!!!! The problem was in... (drumroll... tadam!) in the charset field! No, its value must be the same as IPN sent it to you, but... in UPPERCASE! IPN sends it in lowercase! So you MUST modify IPN data to verify it successfully regardless manual tells us to return data back "as-is". PayPal bug?
So my final working code is: (using HTTP_Request2)
protected function verifyPostData() {
$this->request->setBody('cmd=_notify-validate&' . str_replace('=utf-8', '=UTF-8', file_get_contents('php://input')));
$response = $this->request->send();
if ($response->getStatus() != 200) {
throw new \RuntimeException("Transaction data verification request failed with code {$response->getStatus()}");
}
$content = trim($response->getBody());
return ($content == 'VERIFIED');
}
How I did it: I sent the PDT request for this transaction and obtained transaction data. Then I made field to field comparison of PDT and IPN data. PDT have no some IPN fields such as auth, verify_sign and test_ipn. But all other fields seems must be the same. And the only difference was in the characters case of the charset field. Then I tried to verify modified data and unexpectedly it become successful!
This is a recent PayPal bug, when the customer completes a payment and clicks on "Click here to return.." instead of waiting a few seconds, the parameters passed to the PDT script on your site are sent in lowercase.
This also messes up things like a case-sensitive or encoded cm/custom parameter.
Apparently PayPal are aware of it.

MSXML2.XMLHTTP send data to PHP script

I have been looking into an option to send data read from an attached file in an Outlook message, directly to a PHP script that will then insert the date in a nice MySQL database.
The extraction of the file and the splitting of data all ok, but here is the trick...
From the internet (here) I found a nice post by Jeremy Slade who has managed to send some data to a cgi scipt, all good.
So, clever as I thought I was, I thought I could re-write this into dealing with a PHP script.
But then the works stopped.
I have shortened the code to below snippet;
Sub TestURL()
Set xhr = CreateObject("MSXML2.XMLHTTP")
URL = "http://somedomain.com/php/test.php"
data = "someVariable=Test"
With xhr
.Open "POST", URL, False
.setRequestHeader "Content-Type", "application/x-www-form-urlencoded"
.Send data
End With
End Sub
This should, in theory, open a MSXML2.XMLHTTP request at the given URL and send whatever data with it to the script.
Funny enough, the script is called, but no data is passed ?
I've tried setting the PHP script to both $_GET and $_POST for the [someVariable] element, yet on neither is there any response ?
When I set the PHP to $_GET I matched the VBA MSXML2.XMLHTTP object to "GET" as well and vice versa...
I've tried passing the 'data' variable as argument to the 'function' .send by including it in brackets
i.e.
.send (data)
But this doesn't work either...
I'm a bit at a loss, because the script is called, a dataline is added to the table yet there is not an actual transfer of the 'sent' data ??
I've tried connecting the data string to the URL that is passed to the HTTP object, essentially passing a 'GET' URL to the HTTP object.
i.e.
URL = URL & "?" & data
but to no avail...:-(
The php script works in itself properly, if I pass data directly from the browser
i.e.
http://somedomain.com/php/test.php?someVariable=Test
the data is correctly added and the variable is read...
Can some more enlightened spirits guide me in the right direction ?
20141016 ********** UPDATE **********
Ok, when digging into stuff I found there is also an option to refer to the XmlHttp object as "Microsoft.XmlHttp" ?
Funny enough, when setting the object like that,
i.e.
Set xhr = CreateObject("Microsoft.XMLHTTP")
The code works and the data is added to the table and the .responsText is a success message.
Yet if I return to the original code, I get a PHP error message that tells me that there is an error in my PHP syntax ?? This would imply that the actual 'data' that is being send differs between using "MSXML2.XMLHTTP" and using "Microsoft.XMLHTTP" ???
Have tried to dig out the difference between the two from internet but can't find any post that provides me with a full understanding of the subject ?
Despite the fact that my code now works, I still have the bothering question of not understanding the difference between the two and would appreciate a reply from someone who does :-) As I now have a code that works, but not an understanding of why it works...:-)
Or mroeover not an understanding of why the "MSXML2" option does NOT work...
Much appreciated,
Kindest regards
Martijn
This is not exactly an answer but more like a comment as I lack enough reputation to comment.
The issue can be analyzed using Fiddler which provides details of the requests and responses. I checked the same code as yours in my system with both MSXML2.XMLHTTP and Mirosoft.XMLHTTP objects and found no difference in teh requests. Both of them passed the POST request body containing someVariable=Test to the URL http://somedomain.com/php/test.php.
Here is the raw POST request in both cases:
POST http://somedomain.com/php/test.php HTTP/1.1
Accept: */*
Accept-Language: en-us
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; Trident/5.0; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.0.30729; .NET CLR 3.5.30729; .NET4.0C; .NET4.0E; MS-RTC LM 8)
Host: somedomain.com
Content-Length: 17
Proxy-Connection: Keep-Alive
Pragma: no-cache
someVariable=Test
And the response from the sample URL provided:
HTTP/1.1 405 Method Not Allowed
Server: nginx/1.7.6
Date: Thu, 08 Jan 2015 15:23:58 GMT
Content-Type: text/html
via: HTTP/1.1 proxy226
Connection: close
<html>
<head><title>405 Not Allowed</title></head>
<body bgcolor="white">
<center><h1>405 Not Allowed</h1></center>
<hr><center>nginx/1.7.6</center>
</body>
</html>
<!-- a padding to disable MSIE and Chrome friendly error page -->
<!-- a padding to disable MSIE and Chrome friendly error page -->
<!-- a padding to disable MSIE and Chrome friendly error page -->
<!-- a padding to disable MSIE and Chrome friendly error page -->
<!-- a padding to disable MSIE and Chrome friendly error page -->
<!-- a padding to disable MSIE and Chrome friendly error page -->
One question here would be whether the web server in question is expecting further data to be passed by the way of headers (User-Agent, Referer, Cookies etc) or as part of the request body (may be further input elements that are part of the webform)?

Google Flight RPC and building a JSON array

I am trying to use the Google Flight RPC but am having trouble building a json array to post to it. The only documentation I've found is here. It describes what needs to be sent but doesn't explain how to construct the json array. Specifically, using PHP, I'm not sure how to build and post a json array using the following example from part 1 of the linked documentation.
[,[[,"fs","[,[,[\"SJC\"]\n,\"2012-04-05\",[\"EWR\",\"JFK\",\"LGA\"]\n,\"2012-04-12\"]\n]\n"]
]
,[,[[,"b_ca","54"]
,[,"f_ut","search;f=SJC;t=EWR,JFK,LGA;d=2012-04-05;r=2012-04-12"]
,[,"b_lr","11:36"]
,[,"b_lr","1:1528"]
,[,"b_lr","2:1827"]
,[,"b_qu","3"]
,[,"b_qc","1"]
]
]
]
The above does not appear to be properly formatted to be put into a json array. Additionally, HTTP request headers are needed. I assume these are set via cURL? I'm not clear on the documentation on how to do this.
I've tried several attempts and I don't get back any of the responses that are shown in the documentation.
Edit: Awesome, I got a Tumbleweed badge for this question! I would appreciate help. Thanks.
I was not able to find any documentation either.
However, if you go to https://www.google.com/flights/, open chrome console or firebug, click the network tab, then run a search, you will see that headers and body it makes for the RPC POST.
Sample Headers from the transfer:
Request URL:(I BROKE THE LINK) GOOGLE(dot)com /flights/rpc
Request Method:POST
Status Code:200 OK
Request Headers
:host:(I BROKE THE LINK) GOOGLE (dot) com
:method:POST
:path:/flights/rpc
:scheme:https
:version:HTTP/1.1
accept:/
accept-encoding:gzip,deflate,sdch
accept-language:en-US,en;q=0.8
content-length:169
content-type:application/json; charset=UTF-8
cookie:PREF=ID=f472fc4bbb95bc2b:U=9da5b7e4c1d04bda:FF=0:LD=en:TM=1390684154:LM=1390749713:GM=1:S=orUAMb3qaxBh99PJ; HSID=AHlw351sj7B7Om0t_; SSID=AKycPxLzyXkc4_tZJ; APISID=xKH5zAdc9vfBtiDy/Ab5TlD_Z4w2nP64Wl; SAPISID=7awo9qDssc3wr-fN/AQYOdvCN-I-UwtXQ1; NID=67=XnUn_DGdQDaeczlvXe-qTy9vy8gnQwhFwfRi52TRFS-_Dg-J58CgTGUY6Tkn3cCJYCcVJhK8unOrdffpgzeKed2jPqSazVI4Xplo5fW8-6wXoNi97L2gdoaOms0dKj4iOODoZpzd4DG_8YdQQcH6fl5xY__N929CJr8pdcAUwgnKf8X_mI8sLSB7CKVyS4ZvbGMCAiMLwIs1gJJz-UbppSj; S=travel-flights=5OJmMrbJoqLfOFzkZy285A; SID=DQAAAM0AAAAIGD56aXyxAxrRCSROmPy8AEtV3DaEwKT48aaZ98S35Nss09ishDZ3RxNT6ksikfAOJo-MLYVodF3jr-6imwzC8tRd7cxe-OoyafCZiGaf0qhp-yza4VZlAMInxGPhVae7wSXCRXlqb-wbYHBCHUSz_K5kYpvKwqC8pWuQ_6AUZa3WWqB6OmYpxuihxn3UxSve95zpkziyaDX0JFzUjyWX-0O_iIWZiEztywwyKVWCVv27ByGjIYTYV1G2byExt5M9-kEFpE_v0x8KgU7vleT
dnt:1
origin:(I BROKE THE LINK) SSL GOOGLE(dot)com
referer:(I BROKE THE LINK) SSL GOOGLE(dot)com flights
user-agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.107 Safari/537.36
x-gwt-cctoken:ADS25WMm8S7W0MlpX1-Lf_yNzQCrke7t6OvH2kFLkBJIH_Q-YTuu8VSHmgIxzFtGaL87SsM5PcZECRBP7IqMCbM5QKFVdWrw9hRIkHoL5oiyCzEu2ZCnKuhqvv2sUKcg4Z_HnajCZmM7aQ9nYsVMQnFxqrkgB2Cz7rAIP47zPJ_rakoyxlGE4yJvcuUeiQ
x-gwt-module-base:https://www.google.com/flights/static/
x-gwt-permutation:C8210E5F468630F84E578D8EDE10A1A0
Request Payload
[,[[,"no","[]","1531191655318648",11]],[,[[,"b_al","no:74"],[,"b_ahr","no:s"],[,"b_ca","103:34541"],[,"b_pe","4F2F79B9E3FB0.A40E22A.71A7"],[,"b_qu","0"],[,"b_qc","1"]]]]
Response Headersview source
alternate-protocol:443:quic
cache-control:no-cache, no-store, max-age=0, must-revalidate
content-encoding:gzip
content-length:75
content-type:application/json; charset=utf-8
date:Sat, 22 Feb 2014 05:00:17 GMT
expires:Fri, 01 Jan 1990 00:00:00 GMT
pragma:no-cache
server:GSE
status:200 OK
version:HTTP/1.1
x-content-type-options:nosniff
x-frame-options:SAMEORIGIN
x-xss-protection:1; mode=block
So, what you will likely have to do to get this to work correctly is to make a GET on the path /flights and read the headers, then put the headers in your POST request. (I have had to do similar things in the past). In order to figure out what fields are what I would play around with selecting different fields and seeing what changes in the JSON data that it posts.

How to modify CakePHP Cart Plugin settings to use AJAX button?

I am using the CakePHP Cart Plugin to add shop functionality to a website. I want to add items to the shopping cart using an AJAXified button. Submitting the form works fine. I am having trouble to correctly return just the shopped item as JSON data in method captureBuy() on line 210.
Which settings do I need to use and where/how do I set them? Any hint appreciated!
This is the request which is being sent:
Request URL:http://shop.site/carts_items/buy
Request Method:POST
Status Code:302 Found
Request Headers
Accept:application/json, text/javascript, */*; q=0.01
Accept-Encoding:gzip,deflate,sdch
Accept-Language:de-DE,de;q=0.8,en-US;q=0.6,en;q=0.4
Connection:keep-alive
Content-Length:134
Content-Type:application/x-www-form-urlencoded; charset=UTF-8
Cookie:CAKEPHP=cd4611d0bf0100247928990a8efa72d0; rememberMe[User]=2....BmKdEs;
DNT:1
Host:shop.site
Origin:http://shop.site
Referer:http://shop.site/offer/1
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1553.0 Safari/537.36 SUSE/30.0.1553.0
X-Requested-With:XMLHttpRequest
Form Data
_method:POST
data[CartsItem][foreign_key]:72
data[CartsItem][model]:Product
data[CartsItem][quantity]:1
Response Headers
Connection:keep-alive
Content-Type:text/html; charset=UTF-8
Date:Mon, 23 Sep 2013 20:32:28 GMT
Location:http://shop.site/offer/1
Server:nginx/1.0.15
Transfer-Encoding:chunked
Define "not correctly". Paste your request and response from your network tab (in Chrome).
You don't have to set the items, the CartManager is already setting them to the view and serializes them. At least it should.
Make sure you have the RequestHandler component loaded as well.

Javascript + PHP $_POST array empty

While trying to send a POST request via xmlhttp.open("POST", "url", true) (javascript) to the server I get an empty $_POST array.
Firebug shows that the data is being sent. Here is the data string from Firebug: a=1&q=151a45a150.... But $_POST['q'] returns nothing.
The interesting thing is that file_get_contents('php://input') does have my data (the string above), but PHP somehow doesn't recognize it. Tried both $_POST and $_REQUEST, nothing works.
Headers being sent:
POST /test.php HTTP/1.1
Host: website.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us;q=0.7,en;q=0.3
Accept-Encoding: gzip,deflate
Accept-Charset: utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Referer: http://website.com/
Content-Length: 156
Content-Type: text/plain; charset=UTF-8
Pragma: no-cache
Cache-Control: no-cache
Thank you for any suggestions.
It looks like you're missing the correct Content-Type header. This is necessary for POST requests:
xmlhttp.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded');
Send a
Content-Type: application/x-www-form-urlencoded
header instead of text/plain
You have to write it like this:
xmlhttp.open("POST", "script.php", true);
xmlhttp.send("foo=bar&answer=42");
Just spent hours trying to find a fix for this very problem.
I made the idiotic mistake of concatenating several strings which I wanted to be the parameters, and THEN calling encodeURIComponent on the whole lot. This of course meant that
foo=bar&this=that
became
foo%3Dbar%26this%3Dthat
which of course is gibberish to a PHP script. While I doubt there can be many people who would do something as silly as this, I hope it saves someone the headache I just gave myself....

Categories