So I have a site that I am building and it will be using data in a database to fill out forms on different websites. Now I understand that this can easily be done with cURL or python however when I intercept and read the post data it is usually a huge mess. For instance on this form there is only a option for comment and rating yet contains all types of other garbage:-----------------------------122061295120255
Content-Disposition: form-data; name="StylesheetManager_TSSM"
-----------------------------122061295120255
Content-Disposition: form-data; name="ScriptManager_TSM"
-----------------------------122061295120255
Content-Disposition: form-data; name="__EVENTTARGET"
dnn$ctr459$viewNukeNews$ctl00$ctlViewNewsComments$lbSaveCommentsRating
-----------------------------122061295120255
Content-Disposition: form-data; name="__EVENTARGUMENT"
-----------------------------122061295120255
Content-Disposition: form-data; name="__VIEWSTATE"
e938BM4vVlpM3W+nCswl1rBoQfFc4nBTmKwOSecEbbEU2NDYATm/kCix79QXoW+tZGED7vRe6DIpcv+r8R5dwtCCtCTG5bBJ7EgOAC0HD20XVgmqgNfd3zwiVH3hhYjW9TTPTtwNgPigqfthwmEijBiu8KVc7j/UqHTHti3e5VnRy/YjRxYTGUW5OEr15UHsHDK3fsGAMjzXEcvnzHELCzioBR76DmTpQQECJrRBD8phmoghyUWFU/uFQAfx4r83Q0eHTGv8MevBfrcokPlIkokgOKGinW3fjvdcTxF4peXoaD5iBlBS2eFFZwLZEXcNsOx1ttQQPKYueISssK02TTEg5vhaSSXzwUuEdfYHPdlBZB8TrjEtSAq8Y1OB4FtWtMYtgbAM17052yz9r+eEljLDCfnEAJBQw+Z7OfDu4FLnxuScTeci7K9x85p5sX8ggLOGCufCeO8zSNLsktMa29mOipYLlHDevicer+6pUfZIAUJyJ8nqxzUUMTu3s85H9TG7Ct2fUqO+JMvRygLcTo1NS3DPaHzNaTnDHtreZJ6tnHeE5lMg/pdg4dqTgNg5yLiDhvmbu+N3NAHvbw4ES/fheVqFt32LZMPjZC++xgcOMIyWPHlOFYKY3pXzUrX4Alc2yT8uegdqSGWU/gA2bIoddkXDryrsAG3mXMtyr6nhhkNMJGOqNoUmdrjw9fLaOTq7qjzBEMo9mUYlyJCEFsVnHTztQ2ytQOmRwpPwRqxI0A/y5Q0WsWtRyskR5aXedXuqSu3Li4JPRjilPzj0bZVHCHbaMn75rzvOXBQoeahtK/e2uim9t/HAy0qQZ3R390+hm8La024a+1NDZFK93n4Ykd/qs73VFYTqmRMmPjZAtjXh4B3oF5YJHH5z18aXOD3l56g0Ny0tSRULOz5sR9MUJBd2j4QBiQ8T4Iga4llle15V8jL55Dcfn5DTl/UzfqlwElui09FlF+SgKBdvpezWtKTd3Ny6gY1MAvh77aAZOU8GFr0F7AgqMt60K8T7LDCjDH/FJYVSKkJ141iBlUFShWGgrQHrMbgkwU8v2oTAIGmPO4uqVkDsXovlccPtFy3UDxVY/xNRwoAtGbBGgRjvu5NeNJfn3HPiIxKnb8TbDOhciGdF8CN8fcAb+mOmc8pzct2pfZN5pTtraSw72pO/3POG774EsUe9JsmJBwmZdmVk5bYS9v4CP9VqciIsb80/PmCcW4DT9cTBa5k91AqLQFG2Xa13USRIi3HMjSuUEOnTIIdRjKv1+0VmdDeatRVRvaA/9sgzmuMLQeMV3+q7Cej+dDbw9zrupqcCx7z5mU1IkDoBS2jiCz1qleagU5B3kr3j8JBNa9csxnjeZUi3nqYn9mL3wjLANGsEOtisCcuseywo3/ZNC9SFctDdaLBGpF1ijqsTYWhcy5FhwT3iCxneTjPmoIns44/p3DeOE26CDuL429UqQmuwX1XLHZC/HUCxk5AiEP4y31DDykzKI77uToPGK/7IgzpLQ6KooJZgH0wBtbxASIoGK/rJB2veKPP5bXX3UMwjCNc3k0J0hZ6gz0WtZUFm44EZL8MEEDa3vfoW5I3iPzvIur0FtfVkVGlTTF+7POiK8+kJbYe1Ppm8I/ImcKHmPKnFhWhRiwVUwXsoNH1oTKOafIy/UYRIvt+QQK2ugDTAoEGcYq4d/NLDHQUBKIfHiJZ25SI6pLbw2NHrAoysrvm5TbfPHma6b62ISpNJDjqn1lD7jYOxOpBFlz+h8GlgDj7e/HNdQqzr/f3756gmwq3Y3JP5ghQv7f+6CdV2TGPg6iw4exmxkiCcRrabsCM0rA5AMED6kvOhHf5AOHWSf3L0+c9H8atL8342G4rPNRzXQ8LvgerSdiy34bjcjR2rDMRGCZjzMo6tqqQawlw5eaHoqIxOhCw8Z5gGJDpZAPjP4IBAvK/1TH6eczMO+oXCNDq600ZUPCb3KxyRL/6PFKfg16ojk7QXxuLPL9znYK/uMu6lSAtMRd5ELK4E6Ot7D7zHQ/G5nF8axqiGviMkfKqpxsP6VNXs4GcKWRukImD0ps080rBYwmiy927S3GK6EMGX05g4UERRYni7dfc/Q5caeDSgxEp4AXLKxdCmHQA5MWzwg4q+daArvYqyK6kAVq6rgp6rI1cpbzLXJwUwp1ytNBQccyuOjZfXiioidKVwKMybnO3tUn6TNkgYSyHOxBbzy1q2DkY6jIygKtvgD6izYdwamw2m61F60VVy9XBkXtDVsqdwCCRN4H6NyMYWQs++LAlShsHYXg+bbzMa8Whq5oC6Wx3rlD5/EfsalvU2ZKROoDggB7CznyalA+dAtC1ZqX6Xrj52LcSQU8EOibHrlx7iqIB7pOQ2MtZx4cDB/8P+7AZKssXFvs0SVWMXAXPa4mivwg3UN8l+XlArvsoCDfas5LnXxajia8kg2hwzMaVfN/puGrKb9NsHTyJEjF6sFLAXMx3KdygE324Dkr7xIwIuhghBunL9YvKLWfD0POGzKmQZolcFIrU2VqguF7nXjKHyzZLErYwCqzH+uP/JW+UEKh/nYIYvCha3+I8DqG9T3MpSQTiyLTNV3I4uAA8QtvPV55LzuHXfOdtNUmPLvN0nQ5FcGJi/TpzOToii3idXeY+14+LZHbLZuupailpGRRdoyVBwcGjouLQg800qGBphJV9IftlDt51CDD0Y2FmmO/VycOOPY1pPZ8TjdAhgwybhBO7fd2NqVEUltHAnSnrtxuJeJfQNIFHINC4pllJMIA3Z2qq5gpnMk/HAAz6u9aEm5L6XCEbsCydjPuWjenHa5zC6wq85akW8AdNr0GgSE96NiOmFGtsybLdP7mrjA9ipG1/mCk+mBNfDkecT1o2FYQUsqBdjiCKNxmAsDGA7ZhQiggvULZWEvKsRQuHWJ3XmKN8taU7DCdPYgdF8vxQ2E1jaZpPn/fAfHkNvdy1AEFyJ6PRz3Nid8z7lG1rJewbLNDF9gypH+SKUd+7JzcA597YYEaDsr81A9Lld6kTv7BzG1x/+fF6NXjlFl8sNqXwNObch6ga2CBxq16Cgd5ffzvqEvycPTA5PBDLeZVJ6KZzqtL6Qw37YXE2QJQDu6xxmLn+ciShSnbkIwS5s+nEjQ5EkoUK18R1/82R5RFzKjvYWpt0fVqrHc/UDVgraMo2Y/mXk9QWOfW52qXoMGfaEEYXOCj7dzRZvBQ2xOeSvZ4KzKHGPSJKGglRwqcz576mEhI938e/lZpgdJxgcHAuka4+X1EeHjjDDfZUqiMMP/1K15ruK9/QfJmUim9xOdtB131NOEUIF1eZE/H6Ja+Jnm7TcTKALdylxudikSsi5V31zfbz4dlqTAj5oamsBBcodVzdYa0FZ6KCCDMEFMyX4o4s0Ktn7we/Spy3rI0wU82Dzb2WnseICbIA+Yd7xzEX5H+u1qyUT6Aayt/8tt/FGdxx5B8yMm1LPohgJBb/lu8Xa4PWDK1+muhlNmPlUu2UxTr/jJcfSEnfuyKZX37zM7VTsXjZh0g2agrr6UigQC+6GY+ghMTTo5+H2vDayvc5IEdA0n86LNePsQXIgBx7GnBAiU0nHg+Giq/7MHu/LF8XvSvJIKl7t/OaY5RxKXRFRPlQkRcRqxGMmSWccxxepWPTAbNrBdqCaZojI4MtafE0EbVUPVcbKljEySy98C4N72XCARYTEyofeqnUqYh71ymEMwcke1L7wKGNCGMfjf1cJowUkUng1tDiQg/mDw300cdUbkxwbM9ko9epIdorZP+9w2SuxfoPv59Fy56VNcRODqf+IY81MFOXX6KNpIj2N1Vh3TC/QbmCYhQSxjiQLVpYkm//kunb40zB0LA+yAvVQd1Y03EDuH5cvpDCCQOzzbwW253eayPp2tnUwgySV+M5Z/XWpKErvV9f7Gxfgr/g8vVRWo+crm9wsMScI8Mxbc1pGGMQcy3C1bNGQ6X0Bm+vke7TMkRaH7rRsZh/SJGMQQg1ie3CYpFmHk+vP/OS7H/i4tHyblJ+yPCwcACIM72JaBYnqjfLMQuA34FT0M3Rvk9vZhsRaA6HnLj12EsSssENCYZWScf2XEx8JfTePyIvs5al+Nal1bm2On0o44ZMJ31pc8K0iKBc9ig==
-----------------------------122061295120255
Content-Disposition: form-data; name="__VIEWSTATEGENERATOR"
CA0B0334
-----------------------------122061295120255
Content-Disposition: form-data; name="__VIEWSTATEENCRYPTED"
-----------------------------122061295120255
Content-Disposition: form-data; name="__EVENTVALIDATION"
tA4eGr1Xgh239z393i4iChEPuFYs10biEg4Ym9fZu0aLDt7H4yWECsFXKjtzX7fHWn9WDNOm4a+nPf+qka4hzEpBfm3zRotMOrkEzCm61aM+pbZgaqhQjMPpsDhT3t6k8NkeqaSkUIyFKbXYkpx4GTyyCk0s3UPlqFR8klie6NTAkt0qPH5cjc0GzVRmMBZ5GTbA+L4oGOCgDFpCZ7SFU+/VS+37gRU3YarzwmelKqRNYutT9MwJc5beUUxCNBm6r2Zdeb8OnQnpZR2KlNT8EP+x5+Wsj9Q738H7jX5p2rCNEqmH6mK1wAVM5Rqzo8JTFdtQ6da7PAi9uMj89Vq+LXlf/6BR9vlpEk1cozY9Ny4xdZr8xKSVUYcuJYQ=
-----------------------------122061295120255
Content-Disposition: form-data; name="dnn$dnnSEARCH$txtSearch"
-----------------------------122061295120255
Content-Disposition: form-data; name="dnn$ctr459$viewNukeNews$ctl00$ctlViewNewsComments$rblRating"
3
-----------------------------122061295120255
Content-Disposition: form-data; name="dnn$ctr459$viewNukeNews$ctl00$ctlViewNewsComments$tbComments"
COMMENT GOES HERE
-----------------------------122061295120255
Content-Disposition: form-data; name="ScrollTop"
260
-----------------------------122061295120255
Content-Disposition: form-data; name="__dnnVariable"
{"__scdoff":"1","containerid_dnn_ctr459_ModuleContent":"459","cookieid_dnn_ctr459_ModuleContent":"_Module459_Visible","min_icon_459":"/Portals/_default/Containers/Apple-Orange/min.gif","max_icon_459":"/Portals/_default/Containers/Apple-Orange/max.gif","max_text":"Maximize","min_text":"Minimize"}
-----------------------------122061295120255--
This is not a website I would be posting to however it is a very good representation of what I'm dealing with for the kinds of sites I'll be working with. I understand how to post using the multipart/form-data however what do I do for fields such as "__EVENTVALIDATION"?Edit: Added code that will be used function post_data($site,$data){
$datapost = curl_init();
$headers = array("Content-Type: multipart/form-data; boundary=---------------------------86732602411937");
curl_setopt($datapost, CURLOPT_URL, $site);
curl_setopt($datapost, CURLOPT_TIMEOUT, 40000);
curl_setopt($datapost, CURLOPT_HEADER, TRUE);
curl_setopt($datapost, CURLOPT_HTTPHEADER, $headers);
curl_setopt($datapost, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
curl_setopt($datapost, CURLOPT_POST, TRUE);
curl_setopt($datapost, CURLOPT_POSTFIELDS, $data);
curl_setopt($datapost, CURLOPT_COOKIEFILE, "cookie.txt");
ob_start();
return curl_exec ($datapost);
ob_end_clean();
curl_close ($datapost);
unset($datapost);
}
I have dealt with these types of forms before. They are a pain. What I do is:
cURL the page, with no POST data or anything
Parse the HTML to get the form elements and their current values
Change the values for fields that you need to set
Format all that into an array for POST
Curl the page again with that POST data.
Oh and sometimes there are fields like __EVENTTYPE that need to be set to a certain string for the event you want. To help break down what your second curl should look like, use Chrome developer tools to look at the Request nicely parsed. You can even copy it as a cURL.
Related
So I'm trying to mime the following:
------WebKitFormBoundarygLmTBM5HATvn9tpA
Content-Disposition: form-data; name="name"
p1bieoilebde1ra71v4tm2rlb3h.png
------WebKitFormBoundarygLmTBM5HATvn9tpA
Content-Disposition: form-data; name="file"; filename="watch.png"
Content-Type: image/png
------WebKitFormBoundarygLmTBM5HATvn9tpA--
My code looks like this
$cfile = new CURLFile(realpath(dirname(__FILE__) . "/../images/watch.png"));
$to_post = array ('file' => $cfile);
curl_setopt($ch, CURLOPT_POSTFIELDS, $to_post);
And it's not working, any help?
I'd like to know if it's possible to issue the following multipart/form-data request by relying on PHP's curl automatic body creation from the CURLOPT_POSTFIELDS option. I'd like to avoid building the body string myself.
POST / HTTP/1.0
Host: example.com
Content-type: multipart/form-data, boundary=AaB03x
Content-Length: ...
--AaB03x
Content-Disposition: form-data; name="field"
foo
--AaB03x
Content-Disposition: form-data; name="field"
bar
--AaB03x
content-disposition: form-data; name="file"; filename="filename"
Content-Type: text/plain
Content-Transfer-Encoding: binary
...
Notice how "field" appear twice. The API I'm dealing with requires that array be specified as duplicate (e.g field=foo&field=bar) and doesn't accept the PHP way of serializing such structure (e.g field[0]=foo&field[1]=bar).
From what I understand the correct way of POSTing files with curl on PHP is using a CurlFile:
$ch = curl_init();
...
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt(
$ch, CURLOPT_POSTFIELDS,
array('file' => new CurlFile('file.txt', 'text/plain', 'file.txt'))
);
The thing is I can't specify a duplicate POST field this way. I tried providing an array of value but it fails miserably with an Array to string conversion exception.
$ch = curl_init();
...
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt(
$ch, CURLOPT_POSTFIELDS, array(
'fields' => array('foo', 'bar'),
'file' => new CurlFile('file.txt', 'text/plain', 'file.txt'),
)
);
...
Array to string conversion
Is there any way I can achieve what I'm after or do I have to build my multipart request by myself?
It looks like this is a PHP bug.
I guess I'd like to know if it is possible POST multipart/data-form content type containing json, files, txt, xml in the same post.
so request would look like this:
Content-Type: multipart/form-data; boundary=BOUNDARY
--BOUNDARY
Content-type:application/json
Content-Disposition:form-data
{{"SomeJsonObject":"valueOfObject"}}
--BOUNDARY
Content-type:application/xml
Content-Disposition:form-data
<node>SomeXML Nodes</node>
--BOUNDARY--
I know I can code this as a string, include boundaries manually, but I want to know if it is possible via
curl_setopt($curl, CURLOPT_POSTFIELDS, $data);
Thank you
You can pass Content-Type to each multipart boundary with this hack also:
$url = 'https://...'
$data = ["json\";\nContent-type:\"application/json\";\nContent-disposition:\"form-data" => '{"my":json}',
"xml\";\nContent-type:\"application/xml\";\nContent-disposition:\"form-data" => "<root/>"];
$resource = curl_init();
curl_setopt($resource, CURLOPT_URL, $url);
curl_setopt($resource, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($resource, CURLOPT_POSTFIELDS, $data);
$result = curl_exec($resource);
curl_close($resource);
The idea is to 'inject' all necessary headers to "name" option, like in SQL injection.
The code above will send multipart request with all necessary headers:
------------------------------b66e31048210
Content-Disposition: form-data; name="json";
Content-type:"application/json";
Content-disposition:"form-data"
{"my":json}
------------------------------b66e31048210
Content-Disposition: form-data; name="xml";
Content-type:"application/xml";
Content-disposition:"form-data"
<root/>
But be careful, this stuff is very bad documented.
there is no way to post STRING data in POST except building boundary yourself but curl can post files from disk so
file_put_contents('/tmp/fileForSend.json');
curl_setopt($curl, CURLOPT_POSTFIELDS, array(
'file' =? '#/tmp/fileForSend.json;type=application/json', // this is CURL integrared feature, curl will read file itself
));
so putting '#' sybmol means for CURL that it must read file and put its content into POST request
when I post the reply in a forum, I use live http header to view parameter which used to post the reply.
but, the headers no parameter. but, there are some header like this:
Content-Length: 1115
-----------------------------5959623329472
Content-Disposition: form-data; name="subject"
the title of reply
-----------------------------5959623329472
Content-Disposition: form-data; name="message"
the content of reply
how to post the headers with curl ? my code don't work
curl_setopt($ch, CURLOPT_HTTPHEADER, array('POST /post HTTP/1.1',
'Referer: http://*****.n-stars.org/post?t=4221&mode=reply',
'Content-Disposition: form-data; name="subject"
test lagi kk 2',
'Content-Disposition: form-data; name="message"
test lagi ya kk 8)' ));
please help me :D
These are not headers, If you are trying to make multipart post request, this should be content of your request body. In headers you should only inform endpoint about multipart request and boundary between parts:
// Headers
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
'Referer: http://*****.n-stars.org/post?t=4221&mode=reply',
'Content-Type: multipart/form-data, boundary=5959623329472',
'Content-Length: 1115'
));
// Body
curl_setopt($ch, CURLOPT_POSTFIELDS,
'--5959623329472
Content-Disposition: form-data; name="subject"
test lagi kk 2
--5959623329472
Content-Disposition: form-data; name="message"
test lagi ya kk 8)
--5959623329472--'
);
More about multipart requests: http://www.faqs.org/rfcs/rfc1867.html
wondering how I can set all this data in a curl session, via php:
POST /feeds/api/users/default/uploads HTTP/1.1
Host: uploads.gdata.youtube.com
Authorization: AuthSub token="DXAA...sdb8"
GData-Version: 2
X-GData-Key: key=adf15ee97731bca89da876c...a8dc
Slug: video-test.mp4
Content-Type: multipart/related; boundary="f93dcbA3"
Content-Length: 1941255
Connection: close
--f93dcbA3
Content-Type: application/atom+xml; charset=UTF-8
<?xml version="1.0"?>
<entry xmlns="http://www.w3.org/2005/Atom"
xmlns:media="http://search.yahoo.com/mrss/"
xmlns:yt="http://gdata.youtube.com/schemas/2007">
<media:group>
<media:title type="plain">Bad Wedding Toast</media:title>
<media:description type="plain">
I gave a bad toast at my friend's wedding.
</media:description>
<media:category
scheme="http://gdata.youtube.com/schemas/2007/categories.cat">People
</media:category>
<media:keywords>toast, wedding</media:keywords>
</media:group>
</entry>
--f93dcbA3
Content-Type: video/mp4
Content-Transfer-Encoding: binary
<Binary File Data>
--f93dcbA3--
I don't understand why have some headers, then the --f93dcbA3 more headers (what's the boundary?), some xml (why here ?), more headers and the content of a file.
I know how to make the request without the xml part and the 'boundary'.
Any help will be appreciated :D
The boundary is required because the form enctype is multipart/form-data, rather in this case multipart/related. The boundary is a unique string that cannot appear anywhere else in the request, and it is used to separate each element from the form, whether it is the value of a text input, or a file upload. Each boundary has its own content-type.
Curl cannot do multipart/related for you, so you will need to use a workaround, see this message from the curl mailing list for suggestions. Basically, you will have to construct most of the message yourself.
Note, the last boundary has an additional -- at the end.
This code should hopefully help get you started:
<?php
$url = 'http://uploads.gdata.youtube.com/feeds/api/users/default/uploads';
$authToken = 'DXAA...sdb8'; // token you got from google auth
$boundary = uniqid(); // generate uniqe boundary
$headers = array("Content-Type: multipart/related; boundary=\"$boundary\"",
"Authorization: AuthSub token=\"$authToken\"",
'GData-Version: 2',
'X-GData-Key: key=adf15....a8dc',
'Slug: video-test.mp4');
$postData = "--$boundary\r\n"
."Content-Type: application/atom+xml; charset=UTF-8\r\n\r\n"
.$xmlString . "\r\n" // this is the xml atom data
."--$boundary\r\n"
."Content-Type: video/mp4\r\n"
."Content-Transfer-Encoding: binary\r\n\r\n"
.$videoData . "\r\n" // this is the content of the mp4
."--$boundary--";
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postData);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
$response = curl_exec($ch);
curl_close($ch);
Hope that helps.