PHP Curl with a file attachment - php

I am trying to simulate a PHP cURL POST that requires a file upload.
Here is the HTML form from the website I am trying to POST TO: http://pastebin.com/X6Y0mmfP
The file I need to upload is "domains.txt" which can be found on the same directory as the script.
Using Live HTTP headers (firefox addon) I've retrieved this information:
POST to: http://www.majesticseo.com/reports/bulk-backlinks-upload
HTTP Headers:
Host: www.majesticseo.com
User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64; rv:23.0) Gecko/20100101 Firefox/23.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: https://www.majesticseo.com/reports/bulk-backlink-checker
Cookie: _pk_id.2.d6bc=a607157d494109d4.1382175578.4.1388174858.1384073229.; RURI=reports%2Fbulk- backlink-checker; _pk_ses.2.d6bc=*; STOK=Ox09WRWBeFCU3l3TAim86efmBa
Connection: keep-alive
Content-Type: multipart/form-data; boundary=---------------------------210646678590
Content-Length: 1106
POST Content:
-----------------------------210646678590\r\n
Content-Disposition: form-data; name="fileType"\r\n
\r\n
SingleColumn\r\n
-----------------------------210646678590\r\n
Content-Disposition: form-data; name="indexType"\r\n
\r\n
F\r\n
-----------------------------210646678590\r\n
Content-Disposition: form-data; name="ajaxLoadUrl"\r\n
\r\n
/reports/downloads/confirm-file-upload/backlinksAjax\r\n
-----------------------------210646678590\r\n
Content-Disposition: form-data; name="file"; filename="domains.txt"\r\n
Content-Type: text/plain\r\n
\r\n
facebook.com\n
twitter.com\n
google.com\n
youtube.com\n
wordpress.org\n
adobe.com\n
blogspot.com\n
wikipedia.org\n
wordpress.com\n
linkedin.com\n
yahoo.com\n
amazon.com\n
flickr.com\n
w3.org\n
pinterest.com\n
apple.com\n
tumblr.com\n
myspace.com\n
microsoft.com\n
vimeo.com\n
digg.com\n
qq.com\n
stumbleupon.com\n
baidu.com\n
addthis.com\n
miibeian.gov.cn\n
statcounter.com\n
bit.ly\n
feedburner.com\n
nytimes.com\n
reddit.com\n
delicious.com\n
msn.com\n
macromedia.com\n
bbc.co.uk\n
weebly.com\n
blogger.com\n
icio.us\n
goo.gl\n
gov.uk\n
cnn.com\n
yandex.ru\n
webs.com\n
google.de\n
mail.ru\n
livejournal.com\n
sourceforge.net\n
go.com\n
imdb.com\n
jimdo.com\n
\r\n
-----------------------------210646678590--\r\n
In the manual browser upload, I am using domains.txt - which is also the file on the server (in the same directory as the script).
My script first logs in to then it attempts to make this request.
This is what I have tried to do so far, however it is not being accepted:
$ch = curl_init();
$post = array('fileType' => 'SingleColumn',
'indexType' => 'F',
'ajaxLoadUrl' => '/reports/downloads/confirm-file-upload/backlinksAjax',
'file'=>'#'.realpath('./domains.txt') . ';filename=domains.txt'
);
$post = http_build_query($post);
curl_setopt($ch, CURLOPT_URL, "https://www.majesticseo.com/reports/bulk-backlinks-upload");
curl_setopt($ch,CURLOPT_USERAGENT,'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 5 );
curl_setopt($ch, CURLOPT_COOKIEJAR, 'majestic.txt');
curl_setopt($ch, CURLOPT_COOKIEFILE, 'majestic.txt');
curl_setopt($ch, CURLOPT_REFERER, 'https://www.majesticseo.com/reports/bulk-backlink-checker');
curl_setopt ($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
$result = curl_exec($ch);
curl_close($ch);

Curl doesn't work very well with relative paths, please provide the full path.
ex:
realpath('/home/user/public_html/domains.txt')

This is the function and how I generated the info for the request
function send_curl_request_with_attachment($method, $headers, $url, $post_fields) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$url);
if($headers != "" && count($headers) > 0){
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
} curl_setopt($ch, CURLOPT_CUSTOMREQUEST, $method);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_fields);
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch, CURLOPT_VERBOSE,true);
$result = curl_exec($ch);
curl_close($ch);
return $result;}
$token_slams = "Authorization: Bearer " . $access_token;
$authHeader = array(
$token_slams,
'Accept: application/form-data');
$schedule_path ='../../documents/' . $docs_record["document"];
$cFile = curl_file_create($schedule_path);
$post = array(
'old_record' => $old_record,
'employer_number' => $employer_number,
'payment_date' => $payment_date,
'fund_year' => $fund_year,
'fund_month' => $fund_month,
'employer_schedule'=> $cFile
);
send_curl_request_with_attachment("POST", $authHeader, $my_url, $post);

Related

How to properly retrieve POST data through cURL and PHP?

I am trying to extract the data from this web page:
https://portal.icuregswe.org/siri/report/corona.vtfstart
I have identified that this data can be retrieved from https://portal.icuregswe.org/siri/api/reports/GenerateHighChart (POST). My attempted solution in PHP using cURL is as follows:
$url = 'https://portal.icuregswe.org/siri/api/reports/GenerateHigh';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
'Host: portal.icuregswe.org',
'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:84.0) Gecko/20100101 Firefox/84.0',
'Accept: application/json, text/plain, */*',
'Accept-Language: sv-SE,sv;q=0.8,en-US;q=0.5,en;q=0.3',
'Accept-Encoding: gzip, deflate, br',
'Content-Type: application/x-www-form-urlencoded',
'Content-Length: 179',
'Origin: https://portal.icuregswe.org',
'Connection: keep-alive',
'Referer: https://portal.icuregswe.org/siri/report/corona.vtfstart',
'Cookie: _ga=GA1.2.1861236498.1591648329; _gid=GA1.2.1182050970.1608386303',
'Pragma: no-cache',
'Cache-Control: no-cache'));
$output = curl_exec ($ch);
print $output;
curl_close ($ch);
I am expecting to retrieve the raw data but instead I am getting HTTP Error 400. The header is generated by accessing the given url in Firefox. Any help would be appreciated how to solve this problem.
there's no need to lie on the useragent, they don't run a user-agent blacklist; but it does require a bunch of POST parameters, and in fact, you don't need ANY of those headers (well, except the Content-Type header, but curl will add that for you automatically when setting CURLOPT_POSTFIELDS to a string, so you don't need to add it yourself, where you might risk introducing typos, while curl won't, so it's better if you don't), this
<?php
declare(strict_types=1);
$ch = curl_init();
curl_setopt_array($ch, array(
CURLOPT_URL => 'https://portal.icuregswe.org/siri/api/reports/GenerateHighChart',
CURLOPT_POST => 1,
CURLOPT_POSTFIELDS => http_build_query(array(
'highChartUrl' => '/api/reports/GenerateHighChart',
'tableUrl' => '/api/reports/GenerateExcel',
'chartWidth' => '900',
'reportName' => 'corona.vtfstart',
'startdat' => '2020-01-01',
'stopdat' => '2020-12-19',
'sasong' =>
array(
0 => '2020',
),
))
));
curl_exec($ch);
outputs:
{"ReportName":"Antal nyinskrivna vårdtillfällen med Coronavirus \\n Period ###","ChartTitle":"Antal nyinskrivna vårdtillfällen med Coronavirus ","ChartSubTitle":" Period 2020-01-01 - 2020-12-19","ChartColors":["#53a8c3","#ee5773","#f0d86e","#f6854f","#15ceac","#5773ee","#f597a9","#df7f23","#10a085","#986ef0"],"YaxisTitle":"Antal vtf","YaxisTitleColor":"","HasY2Axis":false,"Y2axisTitle":null,"Y2axisTitleColor":null,"XaxisTitle":"Datum","XaxisColors":["","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","",""],"Credits":"Detta är en originalrapport från Svenska Intensivvårdsregistret","SqlString":"<b>-- SQL-sats 1</b><br />SELECT<br /> DATEADD(DAY, 0, DATEDIFF(DAY, 0, Xa.VtfStart)) AS vtfstartdag#0,<br /> SUM(CASE WHEN Xa.InfEnkatSiriLabfyndId=3 THEN 1 ELSE 0 END) AS antalcorona#1,<br /> COUNT(DISTINCT Xa.PersonId) AS antalpersoner#2<br />FROM<br /> vueSiri AS Xa<br />WHERE<br /> (Xa.InfEnkatSiriLabfyndId=3) AND ((Xa.VtfStart>='2020-01-01 00:00:00') AND (Xa.VtfStart<='2020-12-19 23:59:00'))<br />GROUP BY<br /> DATEADD(DAY, 0, DATEDIFF(DAY, 0, Xa.VtfStart))<br />ORDER BY<br /> vtfstartdag#0<br /><br />","ChartSeries":[{"Name":"Antal vtf","Stack":null,"Type":"column","Color":"","Data":[{"Value":1.0,"Name":"2020-03-06 00:00:00","Color":null},{"Value":1.0,"Name":"2020-03-07 00:00:00","Color":null},{"Value":1.0,"Name":"2020-03-08 00:00:00","Color":null},{"Value":1.0,"Name":"2020-03-09 00:00:00","Color":null},{"Value":3.0,"Name":"2020-03-10 00:00:00","Color":null},{"Value":1.0,"Name":"2020-03-11 00:00:00","Color":null},{"Value":3.0,"Name":"2020-03-13 00:00:00","Color":null},{"Value":7.0,"Name":"2020-03-14 00:00:00","Color":null},{"Value":6.0,"Name":"2020-03-15 00:00:00","Color":null},{"Value":6.0,"Name":"2020-03-16 00:00:00","Color":null},{"Value":3.0,"Name":"2020-03-17 00:00:00","Color":null},{"Value":15.0,"Name":"2020-03-18 00:00:00","Color":null},{"Value":13.0,"Name":"2020-03-19 00:00:00","Color":null},{"Value":20.0,"Name":"2020-03-20 00:00:00","Color":null},{"Value":19.0,"Name":"2020-03-21 00:00:00","Color":null},{"Value":33.0,"Name":"2020-03-22 00:00:00","Color":null},{"Value":45.0,"Name":"2020-03-23 00:00:00","Color":null},{"Value":44.0,"Name":"2020-03-24 00:00:00","Color":null},{"Value":43.0,"Name":"2020-03-25 00:00:00","Color":null},{"Value":45.0,"Name":"2020-03-26 00:00:00","Color":null},{"Value":40.0,"Name":"2020-03-27 00:00:00","Color":null},{"Value":34.0,"Name":"2020-03-28 00:00:00","Color":null},{"Value":51.0,"Name":"2020-03-29 00:00:00","Color":null},{"Value":40.0,"Name":"2020-03-30 00:00:00","Color":null},{"Value":44.0,"Name":"2020-03-31 00:00:00","Color":null},{"Value":60.0,"Name":"2020-04-01 00:00:00","Color":null},{"Value":55.0,"Name":"2020-04-02 00:00:00","Color":null},{"Value":54.0,"Name":"2020-04-03 00:00:00","Color":null},{"Value":49.0,"Name":"2020-04-04 00:00:00","Color":null},{"Value":56.0,"Name":"2020-04-05 00:00:00","Color":null},{"Value":59.0,"Name":"2020-04-06 00:00:00","Color":null},{"Value":58.0,"Name":"2020-04-07 00:00:00","Color":null},{"Value":62.0,"Name":"2020-04-08 00:00:00","Color":null},{"Value":52.0,"Name":"2020-04-09 00:00:00","Color":null},{"Value":45.0,"Name":"2020-04-10 00:00:00","Color":null},{"Value":61.0,"Name":"2020-04-11 00:00:00","Color":null},{"Value":45.0,"Name":"2020-04-12 00:00:00","Color":null},{"Value":59.0,"Name":"2020-04-13 00:00:00","Color":null},{"Value":56.0,"Name":"2020-04-14 00:00:00","Color":null},{"Value":45.0,"Name":"2020-04-15 00:00:00","Color":null},{"Value":51.0,"Name":"2020-04-16 00:00:00","Color":null},{"Value":54.0,"Name":"2020-04-17 00:00:00","Color":null},{"Value":43.0,"Name":"2020-04-18 00:00:00","Color":null},{"Value":42.0,"Name":"2020-04-19 00:00:00","Color":null},{"Value":37.0,"Name":"2020-04-20 00:00:00","Color":null},{"Value":52.0,"Name":"2020-04-21 00:00:00","Color":null},{"Value":64.0,"Name":"2020-04-22 00:00:00","Color":null},{"Value":44.0,"Name":"2020-04-23 00:00:00","Color":null},{"Value":66.0,"Name":"2020-04-24 00:00:00","Color":null},{"Value":37.0,"Name":"2020-04-25 00:00:00","Color":null},{"Value":33.0,"Name":"2020-04-26 00:00:00","Color":null},{"Value":38.0,"Name":"2020-04-27 00:00:00","Color":null},{"Value":40.0,"Name":"2020-04-28 00:00:00","Color":null},{"Value":35.0,"Name":"2020-04-29 00:00:00","Color":null},{"Value":53.0,"Name":"2020-04-30 00:00:00","Color":null},{"Value":30.0,"Name":"2020-05-01 00:00:00","Color":null},{"Value":43.0,"Name":"2020-05-02 00:00:00","Color":null},{"Value":36.0,"Name":"2020-05-03 00:00:00","Color":null},{"Value":33.0,"Name":"2020-05-04 00:00:00","Color":null},}
(...capped, but you get the idea)
You can just copy and paste to see the result. I am also saving cookies in case we need later.
$end_date= date('Y-m-d');//You dont have to use like this, you can write directly the date in the array below.
$start_date = date("Y-m-d", strtotime("-91 day"));
$data = array(
'highChartUrl'=> '/api/reports/GenerateHighChart',
'tableUrl'=> '/api/reports/GenerateExcel',
'chartWidth'=> '900',
'reportName'=> 'corona.vtfstart',
'startdat'=> $start_date,
'stopdat'=> $end_date,
'sasong[0]'=> '2020'
);
$url = 'https://portal.icuregswe.org/siri/api/reports/GenerateHighChart';
$ch = CURL_INIT();
CURL_SETOPT($ch, CURLOPT_URL, $url);
CURL_SETOPT($ch, CURLOPT_POST, true); //Post request
CURL_SETOPT($ch, CURLOPT_PROXYTYPE,CURLPROXY_SOCKS5);
CURL_SETOPT($ch, CURLOPT_POSTFIELDS, $data);
CURL_SETOPT($ch, CURLOPT_RETURNTRANSFER,True);
CURL_SETOPT($ch, CURLOPT_FOLLOWLOCATION,True);
CURL_SETOPT($ch, CURLOPT_COOKIEJAR, dirname(__FILE__) ."/cookie.txt");
CURL_SETOPT($ch, CURLOPT_COOKIEFILE, dirname(__FILE__) ."/cookie.txt");
CURL_SETOPT($ch, CURLOPT_FOLLOWLOCATION, true); //ALLOW REDIRECTION
CURL_SETOPT($ch, CURLOPT_CONNECTTIMEOUT,90);
CURL_SETOPT($ch, CURLOPT_TIMEOUT,90);
$result = CURL_EXEC($ch);
echo $result;

PHP Login REST API

I'm trying to create a REST API and looking for a way to login using PHP, the documentation provided a login example using Python but I don't have an idea how to do this using PHP. I'm thinking if there's a PHP version of the Python code below.
See below code:
def login():
global sessionID
req = urllib2.Request("https://<host>/appserver/j_spring_security_check")
req.add_data(urllib.urlencode({"j_username" : "admin","j_password" :"demoserver"}))
res = opener.open(req)
sessionID = getCookie("JSESSIONID",cookies)
# Get the value of JSESSIONID cookie
response = res.read()
return
What is the login script (PHP version) that I can use if I need to login to web service using PHP (considering the Python example)?
Additional information:
Logging into the web service requires a JSON object as the request body with user name and password:
Successful execution of the method will return a Cookie session Id
Example request JSON: {"j_username" : "username", "j_password":"*******"}
User needs to parse the cookies and extract cookie with key as JSESSIONID. This JSESSIONID value needs to be added manually in all headers of the Rest calls
“Cookie”: “JSESSIONID=“ + cookieValue
Another example using Python:
//Request for All Apps
global sessionID
sID = "JSESSIONID="+sessionID
uri = "https://<hostname>/appserver/portal/api/1.0/apps"
req = urllib2.Request(uri)
req.add_header("Content-Type", "application/json")
req.add_header("Cookie", sID) # Header
req.get_method = "lambda: GET” # Method Type
res = opener.open(req) # URL Call
response = res.read()
return response
Request headers:
Host: 192.168.100.100:444
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:51.0) Gecko/20100101 Firefox/51.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*\/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Referer: https://192.168.100.100:444/appserver/portal/login;jsessionid=6AD37194D43AB02BB79E26C71554958F
Cookie: JSESSIONID=6AD37194D43AB02BB79E26C71554958F
Connection: keep-alive
Upgrade-Insecure-Requests: 1
----------
When I tried curl using Linux, here's the code.
curl -k -i -H "Content-type: application/x-www-form-urlencoded" -c cookies.txt -X POST https://192.168.100.100:444/appserver/j_spring_security_check -d "j_username=admin&j_password=demoserver"
Here's the result of the linux curl, which I believe has succeed in connecting since I was routed to the welcome page.
HTTP/1.1 302 Found
Date: Thu, 16 Feb 2017 18:41:59 GMT
Server: Apache/2.2.26 (Unix) mod_ssl/2.2.25 OpenSSL/1.0.1e mod_jk/1.2.37
Set-Cookie: JSESSIONID=358446CC1F87B2D698D48AFECA373691; Path=/appserver/; HttpOnly
Location: https://192.168.100.100:444/appserver/portal/welcome;jsessionid=358446CC1F87B2D698D48AFECA373691
Content-Length: 0
Access-Control-Allow-Origin: *
Content-Type: text/plain
----------
But when I tried using PHP curl with the code, still could not connect though.
<?php
$ch = curl_init();
$url = "https://192.168.100.100:444/appserver/j_spring_security_chec‌​k";
$postData = 'j_username=admin&j_password=demoserver';
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_POST, 1); // -X
curl_setopt($ch, CURLOPT_POSTFIELDS,$postData); // -d
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
'application/x-www-form-urlencoded'
)); // -H
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookies.txt'); // -c
curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookies.txt'); // -c
curl_setopt($ch, CURLOPT_HEADER, true); // -i
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); // -k
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
echo curl_exec ($ch);
curl_close ($ch);
This is the resulting header in my browser.
Request URL: http://localhost/curl.php
Request method: GET
Remote address: 127.0.0.1:80
Status code: 200 OK
Version: HTTP
Response header:
Host: localhost
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:51.0) Gecko/20100101 Firefox/51.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/\*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
DNT: 1
Connection: keep-alive
Upgrade-Insecure-Requests: 1
Response headers:
Date: Thu, 16 Feb 2017 18:43:52 GMT
Server: Apache/2.2.26 (Unix) mod_ssl/2.2.25 OpenSSL/1.0.1e mod_jk/1.2.37
Content-Language: en-US
Content-Length: 4815
Access-Control-Allow-Origin: *
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html;charset=UTF-8
You need to use CURL
Request to url : "https://<host>/appserver/j_spring_security_check"
And post data : "j_username=admin&j_password=demoserver"
So your code would look like
<?php
$ch = curl_init();
$url = "https://<host>/appserver/j_spring_security_check";
$postData = "j_username=admin&j_password=demoserver";
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS,$postData);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$server_output = curl_exec ($ch);
echo $server_output;
curl_close ($ch);
?>
Maybe you need cookie support? Something like this:
<?php
// this is for cookie handling in the session
session_start();
$tmpFname = tempnam(sys_get_temp_dir(),"COOKIE");
if (isset($_SESSION['cookies'])) {
file_put_contents($tmpFname,$_SESSION['cookies']);
}
// the request
$ch = curl_init();
$url = "https://<host>/appserver/j_spring_security_check";
$postData = "j_username=admin&j_password=demoserver";
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS,$postData);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
'Content-Type: application/x-www-form-urlencoded'
// you may add more request headers here
));
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
// the next two options are for cookie handling
curl_setopt($ch, CURLOPT_COOKIEJAR, $tmpFname);
curl_setopt($ch, CURLOPT_COOKIEFILE, $tmpFname);
$server_output = curl_exec ($ch);
echo $server_output;
curl_close ($ch);
// this is for cookie handling in the session
$_SESSION['cookies'] = file_get_contents($tmpFname);
unlink($tmpFname);
I hope that helps.
Based on your edited question I think it would be best if you did:
<?php
$ch = curl_init();
$url = "https://<host>/appserver/j_spring_security_check";
$postData = '{"j_username":"admin","j_password":"demoserver"}';
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS,$postData);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
'Content-Type: application/json',
'Host: 192.168.100.100:444',
'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:51.0) Gecko/20100101 Firefox/51.0',
'Referer: https://192.168.100.100:444/appserver/portal/login;jsessionid=6AD37194D43AB02BB79E26C71554958F',
'Cookie: JSESSIONID=6AD37194D43AB02BB79E26C71554958F'
));
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true); // see comment
$server_output = curl_exec ($ch);
echo $server_output;
curl_close ($ch);
I think this suits your question, in it's current form, better.

How do I download a file from an .ashx page with php?

I am trying to download a file from this url in php: http://www.roblox.com/Asset/BodyColors.ashx?userId=36377783
The page returns a file your webbrowser automatically downloads.
I tried using cURL:
<?php
$uid = 36377783;
$xUrl = "http://www.roblox.com/Asset/BodyColors.ashx?userId=".$uid;
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $xUrl);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
$xml = curl_exec($ch);
curl_close($ch);
echo $xml;
?>
But it redirects me to an error page.
How do I download the file the .ashx url returns?
(Setting CURLOPT_USERAGENT doesn't work.)
There is a redirection - i use file_get_contents() (but why not curl) and $http_response_header:
$uid = 36377783;
$xUrl = "http://www.roblox.com/Asset/BodyColors.ashx?userId=".$uid;
$opts = array(
'http'=>array(
'method'=>"GET",
'follow_location' => true,
'header'=>
"Host: www.roblox.com\r\n" .
"User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:43.0) Gecko/20100101 Firefox/43.0\r\n" .
"Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\n" .
"Accept-Encoding: gzip, deflate\r\n" .
"DNT: 1\r\n"
)
);
$context = stream_context_create($opts);
$xml = file_get_contents($xUrl, false, $context);
#print_r($http_response_header);
$url_redirect = str_replace('Location: ',"",$http_response_header[5]);
#print $url_redirect;
$xml = file_get_contents($url_redirect);
#print_r($xml);
$roblox_responses = new SimpleXMLElement($xml);
print_r($roblox_responses);

Pinterest login with PHP and cURL not working

I have been trying to make cURL login into pinterest.com for the last 17 hours straight, have tried countless and countless different ways just with cURL but it does not work at all.
My current code only goes to the page but the data is not posted, so it does not login just takes me to the login page.
This first code is using USERPWD which is where it takes me to the login page but it does not login.
error_reporting(E_ALL);
ini_set("display_errors", 1);
$url = "https://www.pinterest.com/login/";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6");
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($ch, CURLOPT_AUTOREFERER, 1);
curl_setopt($ch, CURLOPT_MAXREDIRS, 5);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); // allow https verification if true
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2); // check common name and verify with host name
curl_setopt($ch, CURLOPT_SSLVERSION,3); //
curl_setopt($ch, CURLOPT_CAINFO, getcwd() . "pin.pem"); // allow ssl cert direct comparison
curl_setopt($ch, CURLOPT_COOKIESESSION, TRUE); // set new cookie session
curl_setopt($ch, CURLOPT_COOKIEJAR, "cookies.txt");
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookies.txt");
curl_setopt($ch, CURLOPT_USERPWD, "email:password");
curl_setopt($ch, CURLOPT_SSLVERSION,3);
// grab URL and pass it to the browser
curl_exec($ch);
// close cURL connection, save cookie file, free up system resources
curl_close($ch);
and if I switch it from CURLOPT_USERPWD to
curl_setopt($ch, CURLOPT_POSTFIELDS, 'username_or_email=$email&password=$password');
it just displays a blank page.
The pin.pem is the X.509 Certificate (PEM) file.
Any direction to make this work would be greatly appreciated it.
Edit
new code but leaves blank page and I got the output with a few arrays and displays this:
Array ( [url] => https://www.pinterest.com/login/ [content_type] => [http_code] => 0 [header_size] => 0 [request_size] => 0 [filetime] => -1 [ssl_verify_result] => 0 [redirect_count] => 0 [total_time] => 0.036169 [namelookup_time] => 3.3E-5 [connect_time] => 0.036186 [pretransfer_time] => 0 [size_upload] => 0 [size_download] => 0 [speed_download] => 0 [speed_upload] => 0 [download_content_length] => -1 [upload_content_length] => -1 [starttransfer_time] => 0 [redirect_time] => 0 [certinfo] => Array ( ) [redirect_url] => )
error_reporting(E_ALL);
ini_set("display_errors", 1);
$email = 'email';
$password = 'password';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://www.pinterest.com/login/');
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6');
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($ch, CURLOPT_AUTOREFERER, 1);
curl_setopt($ch, CURLOPT_MAXREDIRS, 5);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, TRUE);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
curl_setopt($ch, CURLOPT_SSLVERSION,3); //
curl_setopt($ch, CURLOPT_CAINFO, getcwd() . 'pin.pem');
curl_setopt($ch, CURLOPT_COOKIESESSION, TRUE);
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookies.txt');
curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookies.txt');
curl_setopt($ch, CURLOPT_POSTFIELDS, 'username_or_email=$email&password=$password');
curl_setopt($ch, CURLOPT_SSLVERSION,3);
curl_exec($ch);
$output=#curl_exec($ch);
$info = #curl_getinfo($ch);
echo $output;
print_r($info);
curl_close($ch);
The Pinterest login process isn't quite that simple. They use a CSRF token which you must extract and send with your login, along with the username and password in the POST body.
Here is what an actual login request to Pinterest looks like, so you will need to emulate this with cURL.
POST /resource/UserSessionResource/create/ HTTP/1.1
Host: www.pinterest.com
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:28.0) Gecko/20100101 Firefox/28.0
Accept: application/json, text/javascript, */*; q=0.01
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
DNT: 1
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
X-CSRFToken: 2rjgN4Qv67pN4wX91kTr4eIkgF54CzJH
X-NEW-APP: 1
X-APP-VERSION: 737af79
X-Requested-With: XMLHttpRequest
Referer: https://www.pinterest.com/login/
Content-Length: 300
Cookie: csrftoken=2rjgN4Qv67pN4wX91kTr4eIkgF54CzJH; _pinterest_sess="aPgJnrIBzvSKLUY/4H5UocshliA47GkkGtHLQwo1H4IcQv58vrdazclonByOb4fWCzb3a3nycKjQzDc6SkCB9eBKoejaLiCjkKLk/QAFRn2x1pvHFlFM+1EoD01/yFxmeQKlvULYU9+qf4D6Mkj8A=="; _track_cm=1;
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
source_url=%2Flogin%2F&data=%7B%22options%22%3A%7B%22username_or_email%22%3A%22YOU%40YOUROMAIN.COM%22%2C%22password%22%3A%22YOURPASSWORD%22%7D%2C%22context%22%3A%7B%7D%7D&module_path=App()%3ELoginPage()%3ELogin()%3EButton(class_name%3Dprimary%2C+text%3DLog+In%2C+type%3Dsubmit%2C+size%3Dlarge)
The source_url data in the request is the POST body (urlencoded). Take note that username_or_email is your login (I put YOU%40YOURDOMAIN.COM) and password is the password.
What you will have to do is make a GET request to /login/ to establish a session and cookies in the cURL session. Then using the same cURL handle, you can switch to a POST request, set CURLOPT_POSTFIELDS with the data from the source_url...... line.
You will probably also need to set the headers X-CSRFToken, X-NEW-APP, X-APP-VERSION, and X-Requested-With to match the above (except you will need to figure out how to get the correct CSRF Token value).
Unfortunately I don't have the time right now to make a working example, the next paragraph may help. You will need to use your browser to help you debug some of the HTTP requests to figure out all the requests you may need to make to get all the relevant data for your request.
If you check out this answer it shows curl login with PHP and links to a number of useful other related answers with examples.
EDIT:
Here is a working example of using PHP and cURL to log in to Pinterest.
This code is a Pinterest PHP login example (works as of 2014-05-11]. You may ask yourself, can what I want to do be done with the API instead of this hackish code which could break at any time???
As you can see I parse the CSRF_Token out of the headers, you should probably do this for the APP-VERSION as well since it can update almost daily. Right now it's hard coded.
<?php
error_reporting(E_ALL);
ini_set('display_errors', 1);
$username = 'you#yoursite.com'; // your username
$password = 'yourpassword'; // your password
// this is the http post data for logging in - username & password are substituted in later
$login_post = array(
'source_url' => '/login/',
'data' => '{"options":{"username_or_email":"%s","password":"%s"},"context":{}}',
'module_path' => 'App()>LoginPage()>Login()>Button(class_name=primary, text=Log In, type=submit, size=large',
);
$pinterest_url = 'https://www.pinterest.com/'; // pinterest home url
$login_url = $pinterest_url . 'login/'; // pinterest login page url
$login_post_url = $pinterest_url . 'resource/UserSessionResource/create/'; // pinterest login post url
// http headers to send with requests
$httpheaders = array(
'Connection: keep-alive',
'Pragma: no-cache',
'Cache-Control: no-cache',
'Accept-Language: en-US,en;q=0.5',
);
// http headers to send when logging in
$login_header = array(
'X-NEW-APP: 1',
'X-APP-VERSION: d2bb370', // THIS WILL UPDATE FREQUENTLY, CHANGE IT!!!
'X-Requested-With: XMLHttpRequest',
'Accept: application/json, text/javascript, */*; q=0.01');
// ----------------------------------------------------------------------------
// request home page to establish cookies and a session, set curl options
$ch = curl_init($pinterest_url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_AUTOREFERER, 1);
curl_setopt($ch, CURLOPT_ENCODING, 'gzip,deflate');
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Iron/31.0.1700.0 Chrome/31.0.1700.0');
curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookie.txt');
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookie.txt');
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_STDERR, fopen('/tmp/debug.txt', 'w+'));
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_HTTPHEADER, $httpheaders);
$data = curl_exec($ch);
// ----------------------------------------------------------------------------
// parse the csrf token out of the cookies to set later when logging in
list($headers, $body) = explode("\r\n\r\n", $data, 2);
preg_match('/csrftoken=(.*?)[\b;\s]/i', $headers, $csrf_token);
// next request the login page
curl_setopt($ch, CURLOPT_URL, $login_url);
$data = curl_exec($ch);
// ----------------------------------------------------------------------------
// perform login post
$login_header[] = 'X-CSRFToken: ' . $csrf_token[1];
$login_post['data'] = sprintf($login_post['data'], $username, $password);
$post = http_build_query($login_post);
curl_setopt($ch, CURLOPT_URL, $login_post_url);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
curl_setopt($ch, CURLOPT_HTTPHEADER, array_merge($httpheaders, $login_header));
curl_setopt($ch, CURLOPT_REFERER, $login_url);
curl_setopt($ch, CURLOPT_HEADER, 0);
$data = curl_exec($ch);
// check response and output status
if (curl_getinfo($ch, CURLINFO_HTTP_CODE) != 200) {
echo "Error logging in.<br />";
var_dump(curl_getinfo($ch));
} else {
$response = json_decode($data, true);
if ($response === null) {
echo "Failed to decode JSON response.<br /><br />";
var_dump($response);
} else if ($response['resource_response']['error'] === null) {
echo "Login successful, " . $response['resource_response']['data']['username'] . "<br /><br />";
echo "You have {$response['resource_response']['data']['follower_count']} followers, are following {$response['resource_response']['data']['following_count']} users. You have liked {$response['resource_response']['data']['like_count']} pins.";
}
}
My output:
Login successful, drew010
You have 0 followers, are following 0 users. You have liked 0 pins.
FYI, Pinterest has login rate limit so don't run this before every request.
Here is my Ruby implementation of the Pinterest login/session mechanism.
Run this once a day to save the headers (including csrftoken). Then use the saved headers to do requests that are not (yet) supported by the api (like ads reports).
class PinterestHeadersScheduler
include Sidekiq::Worker
sidekiq_options queue: :recurring, retry: 0
HOMEPAGE = 'https://ads.pinterest.com/'
LOGIN_URL = "#{HOMEPAGE}login/"
SESSION_URL = "#{HOMEPAGE}resource/UserSessionResource/create/"
LOGIN_DATA = {
source_url: '/login/',
data: { options: { username_or_email: ENV['PI_USERNAME'], password: ENV['PI_PASSWORD'] }, context: {} }.to_json
}
HEADERS = {
'Accept': 'application/json,text/html,image/webp,image/apng,*/*;q=0.8',
'Origin': 'https://ads.pinterest.com',
'Referer': 'https://ads.pinterest.com/',
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36',
'Connection': 'keep-alive',
'Cache-Control': 'no-cache',
'Accept-Charset': 'utf-8;ISO-8859-1q=0.7,*;q=0.7',
'Accept-Encoding': 'gzip, deflate',
'Accept-Language': 'en-US,en;q=0.8'
}
SESSION_HEADERS = HEADERS.merge({
'Accept': 'application/json',
'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8',
'X-Requested-With': 'XMLHttpRequest'
})
def perform
login = HTTParty.get(LOGIN_URL, { headers: HEADERS })
cjar = login.get_fields('Set-Cookie').each_with_object(HTTParty::CookieHash.new) { |cookie, jar| jar.add_cookies(cookie) }
headers = SESSION_HEADERS.merge({ 'Cookie': cjar.to_cookie_string, 'X-CSRFToken': cjar[:csrftoken] })
res = HTTParty.post(SESSION_URL, { headers: headers, body: LOGIN_DATA.to_param })
session = JSON.parse(ActiveSupport::Gzip.decompress(res.body))
raise "login error #{session['resource_response']['error']}" if session['resource_response']['error']
cjar = res.headers.get_fields('Set-Cookie').each_with_object(HTTParty::CookieHash.new) { |cookie, jar| jar.add_cookies(cookie) }
save_session_headers(HEADERS.merge({ 'Cookie' => cjar.to_cookie_string }))
end
def save_session_headers(headers)
# replace this with your cache/db
Utils::RedisUtil.set(:pinterest_session_headers, headers.to_json)
end
end

PHP script works fine until I send it a parameter from HTTPService in Flash Builder 4?

I'm using a PHP script to read an RSS feed in my Flex 4 app. The script works when I put the URL of the feed in the actual script, but I can't get it to work when I try to send the URL as a parameter from a HTTPService in Flex.
Here is the HTTPService from Flex 4 that I'm using:
<mx:HTTPService url="http://talk.6te.net/proxy.php"
id="proxyService" method="POST"
result="rssResult()" fault="rssFault()">
<mx:request>
<url>
http://feeds.feedburner.com/nah_right
</url>
</mx:request>
</mx:HTTPService>
This is the script that works:
<?php
$ch = curl_init();
$timeout = 30;
$userAgent = $_SERVER['HTTP_USER_AGENT'];
curl_setopt($ch, CURLOPT_URL, "http://feeds.feedburner.com/nah_right");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
curl_setopt($ch, CURLOPT_USERAGENT, $userAgent);
$response = curl_exec($ch);
if (curl_errno($ch)) {
echo curl_error($ch);
} else {
curl_close($ch);
echo $response;
}
?>
But this is what I actually want to use, but it doesn't work (only line 6 is different):
<?php
$ch = curl_init();
$timeout = 30;
$userAgent = $_SERVER['HTTP_USER_AGENT'];
curl_setopt($ch, CURLOPT_URL, $_REQUEST['url']);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
curl_setopt($ch, CURLOPT_USERAGENT, $userAgent);
$response = curl_exec($ch);
if (curl_errno($ch)) {
echo curl_error($ch);
} else {
curl_close($ch);
echo $response;
}
?>
Here is the request and response output of the HTTPService from the network monitor in Flash Builder 4 (using the PHP script that doesn't work):
Request:
POST /proxy.php HTTP/1.1
Host: talk.6te.net
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Content-type: application/x-www-form-urlencoded
Content-length: 97
url=%0A%09%09%09%09%09http%3A%2F%2Ffeeds%2Efeedburner%2Ecom%2Fnah%5Fright%0A%20%20%20%20%09%09%09
Response:
HTTP/1.1 200 OK
Date: Mon, 10 May 2010 03:23:27 GMT
Server: Apache
X-Powered-By: PHP/5.2.13
Content-Length: 15
Content-Type: text/html
<url> malformed
I've tried putting the URL in " " in the HTTPService, but that didn't do anything. Any help would be greatly appreciated!
The $_REQUEST['url'] value is being urlencoded as opposed to urlencoding just the querystring. Somewhere in your code and/or FLEX service is causing a "double urlencode". Simply url decode it and you should get the value you need. Also, I noticed line feeds and tabs so you might want to trim it as well.
<?php
$ch = curl_init();
$timeout = 30;
$userAgent = $_SERVER['HTTP_USER_AGENT'];
curl_setopt($ch, CURLOPT_URL, trim(urldecode($_REQUEST['url'])));
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
curl_setopt($ch, CURLOPT_USERAGENT, $userAgent);
$response = curl_exec($ch);
if (curl_errno($ch)) {
echo curl_error($ch);
} else {
curl_close($ch);
echo $response;
}
?>
That's it.

Categories