So i have been trying to get this working, however for the past 3 hours all i am still getting is the login page.
To start I have gone trough the source code of the page and gotten all the inputs and are listed below:
__EVENTTARGET
__EVENTARGUMENT
__VIEWSTATE
__EVENTVALIDATION
__VIEWSTATEGENERATOR
EditToken
sUserName
sPassword
sParentUID
hidIE11BugFix
The Code:
<?php
//Login url, if i hit the page without logging in it will ask for username, password, and key then then will go to the page i an looking for.
$url = "https://domain.com/7000/7000.aspx?filtertype=2";
$ckfile = tempnam("/tmp", "CURLCOOKIE");
$useragent = 'Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.2 (KHTML, like Gecko) Chrome/5.0.342.3 Safari/533.2';
$username = "username";
$password = "password";
$key = "key";
$f = fopen('/tmp/log.txt', 'w'); // file to write request header for debug purpose
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_COOKIEFILE, $ckfile);
curl_setopt($ch, CURLOPT_COOKIEJAR, $ckfile);
//I have tried to toggle these but does not make a difference
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, $useragent);
$html = curl_exec($ch);
curl_close($ch);
preg_match('~<input type="hidden" name="__VIEWSTATE" id="__VIEWSTATE" value="(.*?)" />~', $html, $viewstate);
preg_match('~<input type="hidden" name="__EVENTVALIDATION" id="__EVENTVALIDATION" value="(.*?)" />~', $html, $eventValidation);
preg_match('~<input type="hidden" name="__VIEWSTATEGENERATOR" id="__VIEWSTATEGENERATOR" value="(.*?)" />~', $html, $viewstategenerator);
preg_match('~<input type=\'hidden\' name=\'EditToken\' id=\'EditToken\' value =\'(.*?)\'/>~', $html, $edittoken);
$viewstate = $viewstate[1];
$eventValidation = $eventValidation[1];
$viewstategenerator = $viewstategenerator[1];
$edittoken = $edittoken[1];
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_COOKIEJAR, $ckfile);
curl_setopt($ch, CURLOPT_COOKIEFILE, $ckfile);
curl_setopt($ch, CURLOPT_HEADER, FALSE);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_REFERER, $url);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_STDERR, $f);
curl_setopt($ch, CURLOPT_USERAGENT, $useragent);
// Collecting all POST fields
$postfields = array();
$postfields['__EVENTTARGET'] = "";
$postfields['__EVENTARGUMENT'] = "";
$postfields['__VIEWSTATE'] = $viewstate;
$postfields['__EVENTVALIDATION'] = $eventValidation;
$postfields['__VIEWSTATEGENERATOR'] = $viewstategenerator;
$postfields['EditToken'] = $edittoken;
$postfields['sUserName'] = $username;
$postfields['sPassword'] = $password;
$postfields['sParentUID'] = $key;
$postfields['hidIE11BugFix'] = "";
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postfields);
$ret = curl_exec($ch); // Get result after login page.
print $ret;
?>
The log file: I am new to reading response codes i have googled around and i see a couple items that might be red flags but im not really sure.
* About to connect() to domain.com port 443 (#0)
* Trying 123.123.123.123... * connected
* Connected to domain.com (1123.123.123.123) port 443 (#0)
* warning: ignoring value of ssl.verifyhost
* skipping SSL peer certificate verification
* SSL connection using TLS_RSA_WITH_AES_256_CBC_SHA
* Server certificate:
* subject: CN=domain.com,OU=COMODO SSL,OU=Issued through Domain Neme E-PKI Manager,OU=Domain Control Validated
* start date: Dec 10 00:00:00 2015 GMT
* expire date: Jan 21 23:59:59 2018 GMT
* common name: domain.com
* issuer: CN=COMODO RSA Domain Validation Secure Server CA,O=COMODO CA Limited,L=Salford,ST=Greater Manchester,C=GB
> POST /7000/7000.aspx?filtertype=2 HTTP/1.1
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.2 (KHTML, like Gecko) Chrome/5.0.342.3 Safari/533.2
Host: domain.com
Accept: */*
Referer: https://domain.com/7000/7000.aspx?filtertype=2
Cookie: ASP.NET_SessionId=1j4tzm4gp15myudy4hgtc3ua; GRI%5FLOGIN=NextURL=%2f7000%2f7000.aspx%3ffiltertype%3d2
Content-Length: 5658
Expect: 100-continue
Content-Type: multipart/form-data; boundary=----------------------------4e03c9b32001
< HTTP/1.1 100 Continue
< HTTP/1.1 302 Found
< Cache-Control: private
< Content-Type: text/html; charset=utf-8
< Location: /Login_Entry.aspx
< X-dynaTrace: PT=19804;PA=-499723444;RS=VMS1/20160624103046_0.session;SP=VMS1;PS=-1294020462
< dynaTrace: PT=19804;PA=-499723444;RS=VMS1/20160624103046_0.session;SP=VMS1;PS=-1294020462
< X-dynaTrace: PT=19804;PA=-499723444;RS=VMS1/20160624103046_0.session;SP=VMS1;PS=-1294020462
< dynaTrace: PT=19804;PA=-499723444;RS=VMS1/20160624103046_0.session;SP=VMS1;PS=-1294020462
* Replaced cookie GRI%5FLOGIN="NextURL=%2f7000%2f7000.aspx%3ffiltertype%3d2" for domain domain.com, path /, expire 0
< Set-Cookie: GRI%5FLOGIN=NextURL=%2f7000%2f7000.aspx%3ffiltertype%3d2; path=/
* Replaced cookie ASP.NET_SessionId="" for domain domain.com, path /, expire 0
< Set-Cookie: ASP.NET_SessionId=; path=/
< X-dynaTrace: PT=19804;PA=-499723444;RS=VMS1/20160624103046_0.session;SP=VMS1;PS=-1294020462
< dynaTrace: PT=19804;PA=-499723444;RS=VMS1/20160624103046_0.session;SP=VMS1;PS=-1294020462
< X-U2: ATLVP1WEB02
< X-FRAME-OPTIONS: SAMEORIGIN
< Date: Sat, 25 Jun 2016 01:09:24 GMT
< Content-Length: 134
<
* Ignoring the response-body
* Connection #0 to host domain.com left intact
* Issue another request to this URL: 'https://domain.com/Login_Entry.aspx'
* Violate RFC 2616/10.3.3 and switch from POST to GET
* Re-using existing connection! (#0) with host domain.com
* Connected to domain.com (165.193.56.22) port 443 (#0)
> GET /Login_Entry.aspx HTTP/1.1
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.2 (KHTML, like Gecko) Chrome/5.0.342.3 Safari/533.2
Host: domain.com
Accept: */*
Referer: https://domain.com/7000/7000.aspx?filtertype=2
Cookie: ASP.NET_SessionId=; GRI%5FLOGIN=NextURL=%2f7000%2f7000.aspx%3ffiltertype%3d2
< HTTP/1.1 200 OK
< Cache-Control: no-cache, no-store
< Pragma: no-cache
< Content-Type: text/html; charset=utf-8
< Expires: -1
< X-dynaTrace: PT=19807;PA=-499723444;RS=VMS1/20160624103046_0.session;SP=VMS1;PS=-1294020462
< dynaTrace: PT=19807;PA=-499723444;RS=VMS1/20160624103046_0.session;SP=VMS1;PS=-1294020462
* Replaced cookie ASP.NET_SessionId="hzfobhtjmshwj2w45yiyappm" for domain domain.com, path /, expire 0
< Set-Cookie: ASP.NET_SessionId=hzfobhtjmshwj2w45yiyappm; path=/; HttpOnly
< X-U2: ATLVP1WEB02
< X-FRAME-OPTIONS: SAMEORIGIN
< Date: Sat, 25 Jun 2016 01:09:24 GMT
< Content-Length: 23876
< Vary: Accept-Encoding
<
* Connection #0 to host domain.com left intact
* Closing connection #0
The cookie: Im not really sure if is important however if it helps.
# Netscape HTTP Cookie File
# http://curl.haxx.se/rfc/cookie_spec.html
# This file was generated by libcurl! Edit at your own risk.
domain.com FALSE / FALSE 0 GRI%5FLOGIN NextURL=%2f7000%2f7000.aspx%3ffiltertype%3d2
#HttpOnly_domain.com FALSE / FALSE 0 ASP.NET_SessionId hzfobhtjmshwj2w45yiyappm
Ok i figured it out,
The login page is:
https://domain.com/login_entry.aspx
Looking at the source code, the line:
<form method="POST" action="login_confirm.ashx" id="form1">
https://domain.com/login_confirm.ashx
I changed:
$url = "https://domain.com/7000/7000.aspx?filtertype=2";
To:
$url = "https://domain.com/login_confirm.ashx";
Now i am seeing that i want.
Im not sure if this should be obvious or not.
Related
I've run manually the url from postman, and I've successfully connected and it printed it results.
But when I tried calling it from a method, no result is printing.
here's the curl code I used:
public function curlHandle($apiPath, $postArray)
{
$CI = & get_instance();
$conf = $CI->config->config;
$curlHandle = curl_init($apiPath);
curl_setopt($curlHandle, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curlHandle, CURLOPT_USERAGENT,"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.57 Safari/537.36");
curl_setopt($curlHandle, CURLOPT_TIMEOUT , 300);
curl_setopt($curlHandle, CURLOPT_POST, 1);
curl_setopt($curlHandle, CURLOPT_POSTFIELDS, $postArray);
curl_setopt($curlHandle, CURLOPT_VERBOSE, 1);
$fp = fopen('/var/www/html/logs/curl_log.txt', 'w');
curl_setopt($curlHandle, CURLOPT_STDERR, $fp);
$result = curl_exec($curlHandle);
print_r(curl_error($curlHandle));
if ($result === FALSE) {
die(curl_error($curlHandle));
} else {
$data = json_decode($result, TRUE);
}
curl_close($curlHandle);
return $data;
}
Here's the content of curl_log.txt
Hostname localhost/codeignitertest was found in DNS cache
Trying 127.0.0.1...
TCP_NODELAY set
Connected to alocalhost/codeignitertest (127.0.0.1) port 80 (#0)
POST /test/process HTTP/1.1 Host: localhost/codeignitertest User-Agent: Mozilla/5.0 (X11; Linux
x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.57
Safari/537.36 Accept: / Content-Length: 142 Content-Type:
multipart/form-data; boundary=------------------------78a116d61ad4939d
< HTTP/1.1 200 OK < Date: Tue, 30 Apr 2019 21:19:43 GMT < Server:
Apache/2.4.29 (Ubuntu) < Cache-Control: no-store, no-cache,
must-revalidate < Pragma: no-cache < Set-Cookie:
ci_session=ag85kv438n0ukb1rvrkb5cdfkrd8s2q6; expires=Tue, 30-Apr-2019
21:49:43 GMT; Max-Age=1800; path=/; HttpOnly < Expires: Thu, 19 Nov
1981 08:52:00 GMT < Content-Length: 14 < Content-Type: text/html;
charset=UTF-8 <
* Connection #0 to host localhost/codeignitertest left intact
also note that curlHandle() is from another project and localhost/codeignitertest but both are running in my localhost
You've issued this option:
curl_setopt($curlHandle, CURLOPT_RETURNTRANSFER, true);
Which means that curl returns the output from the remote server instead of printing it.
You then capture that output into a variable:
$result = curl_exec($curlHandle);
Which you then encode and return to your caller:
return $data;
This code shouldn't print the results. It returns the results.
I have a problem logging in to a website with CURL and PHP.
I test with the Firefox add-on HttpRequester and this worked.
Result login:
POST https://www.balatarin.com/sessions
Content-Type: application/x-www-form-urlencoded
session[login]=testeruni&session[password]=123456789&session[remember_me]=1&commit=%D9%88%D8%B1%D9%88%D8%AF&utf8=%E2%9C%93&authenticity_token[![httprequester][1]][1]
-- response --
200 OK
Server: shield
Date: Thu, 19 Jan 2017 13:51:54 GMT
Content-Type: text/html; charset=utf-8
status: 200 OK
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
x-ua-compatible: IE=Edge,chrome=1
Etag: W/"7418542e936fbdfe20002faf11876845"
Cache-Control: must-revalidate, private, max-age=0
Set-Cookie: _balat_session_new=BAh7C0kiDHVzZXJfaWQGOgZFRmkD964BSSIPc2Vzc2lvbl9pZAY7AEZJIiUzZGUxMzIyN2ZhZDVmMDUzOGE3OGY0YTRhZDkzNmUyMQY7AFRJIhZpbnB1dF9kZXZpY2VfdHlwZQY7AEZJIgpNT1VTRQY7AEZJIhRob3Zlcl9zdXBwb3J0ZWQGOwBGVEkiCmZsYXNoBjsARm86JUFjdGlvbkRpc3BhdGNoOjpGbGFzaDo6Rmxhc2hIYXNoCToKQHVzZWRvOghTZXQGOgpAaGFzaHsGOgtub3RpY2VUOgxAY2xvc2VkRjoNQGZsYXNoZXN7BjsKSSI22YbYrtiz2Kog2YXYtNiq2LHaqSDahtmG2K8g2KjYp9mE2KfahtmHINi02YjbjNivLgY7AFQ6CUBub3cwSSIQX2NzcmZfdG9rZW4GOwBGSSIxT3krNk5nM1NTM2IreXc4SUtxbW9yN2NmMXQrdUNLWWdubFRRYmpidmtNTT0GOwBG--2c2a72f8ec27564250ba084d97998aefba4af11a; path=/; secure; HttpOnly geo=0
X-Request-Id: 521288561d7cfff0ef8fe8d72080760c
X-Runtime: 0.188862
X-Rack-Cache: miss
Content-Encoding: gzip
Via: 1.1 google
Alt-Svc: clear
Expires: Thu, 19 Jan 2017 13:51:54 GMT
X-Firefox-Spdy: h2
but it does not login with curl in PHP. I tested all headers in my CURL but it does not login, only works with HttpRequester.
public function actionLoggin()
{
$url = 'https://www.balatarin.com/sessions';
$headers[] = 'Content-Type: application/x-www-form-urlencoded';
$headers[] = 'Host: www.balatarin.com';
$headers[] = 'User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:50.0) Gecko/20100101 Firefox/50.0';
$headers[] = 'Referer: https://www.balatarin.com/login';
$params = array(
'session[login]' => 'testeruni',
'session[password]' => '123456789',
'session[remember_me]' => '0',
'commit' => 'ورود',
'utf8' => '✓',
'authenticity_token' => '',
);
//open connection
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($params));
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_COOKIEJAR, 'bala_cookie.txt');
curl_setopt($ch, CURLOPT_COOKIEFILE, 'bala_cookie.txt');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$result = curl_exec($ch);
curl_close($ch);
echo $result;
}
Here is my cookie file:
# Netscape HTTP Cookie File
# https://curl.haxx.se/docs/http-cookies.html
# This file was generated by libcurl! Edit at your own risk.
www.balatarin.com FALSE / FALSE 0 logged_in 1
#HttpOnly_www.balatarin.com FALSE / TRUE 0 _balat_session_new BAh7CToOcmV0dXJuX3RvMDoMdXNlcl9pZGkDj60BOhJsb2dpbl9yZXRyaWVzMEkiD3Nlc3Npb25faWQGOgZFRkkiJTgwN2ZmMDRjMGUzMzkyMDIyZWY5YzBmZTQxN2FmZWMzBjsIVA%3D%3D--d47dd61bc9900449cca69ebd727041c3946a13ba
www.balatarin.com FALSE / FALSE 0 geo 0
www.balatarin.com FALSE / FALSE 1516368886 corr b8ed93fa279a469a637b
I'm trying to upload a file via cURL but something is missing. I forces this request to be HTTP 1.0 because cURL adds the Expect: 100 header if I use HTTP 1.1 so thats why the extra header. Here is a simple test code:
<?php
if(isset($_POST["id"])) {
$data = array("id" => $_POST["id"]);
$data["file"] = "#".realpath($_FILES["file"]["tmp_name"]);
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
'Authorization: Bearer 0e39ffba-66cd-4933-9e94-fcdf600c2453',
'Connection: keep-alive'
));
curl_setopt($ch, CURLOPT_URL, "http://localhost:8080/test-api/upload");
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
curl_setopt($ch, CURLOPT_VERBOSE, false);
curl_setopt($ch, CURLOPT_HTTP_VERSION, 1);
$response = curl_exec($ch);
var_dump($response);
exit;
}
?>
My Jersey based server picks it up, and I can see these headers:
INFO: 25 * Server has received a request on thread http-nio-8080-exec-1
25 > POST http://localhost:8080/test-api/upload
25 > authorization: Bearer 0e39ffba-66cd-4933-9e94-fcdf600c2453
25 > connection: keep-alive
25 > content-length: 261
25 > content-type: multipart/form-data; boundary=------------------------53f7ba34739b4d9e
25 > host: localhost:8080
See the content-length? It's way too short. When I send the same file and the same request via my Postman REST client, I get these headers:
INFO: 26 * Server has received a request on thread http-nio-8080-exec-3
26 > POST http://localhost:8080/test-api/upload
26 > accept-encoding: gzip, deflate
26 > accept-language: hu-HU,hu;q=0.8,en-US;q=0.6,en;q=0.4
26 > authorization: Bearer 0e39ffba-66cd-4933-9e94-fcdf600c2453
26 > cache-control: no-cache, no-cache
26 > connection: keep-alive
26 > content-length: 144954
26 > content-type: multipart/form-data; boundary=----WebKitFormBoundarye5Tg0kEqi10nEBwv
26 > cookie: ff_uvid=126143952; _ga=GA1.1.459454356.1439469592; CAKEPHP=9mffidqo8203ugktan4roc0u82
26 > host: localhost:8080
26 > origin: chrome-extension://fdmmgilgnpjigdojojpjoooidkmcomcm
26 > user-agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.80 Safari/537.36
The content-length now is set property. What could be wrong here?
It sounds like you're using PHP 5.6.0 or later. As of this release, the # prefix for file uploads is disabled by default. You can enable it with
curl_setopt($ch, CURLOPT_SAFE_UPLOAD, false);
This option was added in 5.5, but the default was false for backward compatibility; 5.6 changed the default incompatibly.
The preferred way to perform file uploads starting with 5.5 is with the CurlFile class.
$data["file"] = new CurlFile(realpath($_FILES["file"]["tmp_name"]));
You have to actually insert the filecontent, this differs from the cli-version of curl.
try:
$data["file"] = file_get_contents($_FILES["file"]["tmp_name"]);
I'm trying to implement a functionnality like facebook, when you paste a link it's grabbing some information (h1, desc, images, ...) from the page and display them.
I already face several issues that I managed to fix (gzip, cookies, user agent, ...) but on this one I'm not sure what is blocking my request.
The link in question is http://www.mixcloud.com
Here is my PHP script:
protected function getContent()
{
$ch = curl_init();
$headers = array(
'Accept: */*',
// 'Accept-Encoding: gzip,deflate,sdch',
// 'Accept-Language: en-US,en;q=0.8,es;q=0.6,fr;q=0.4,pt;q=0.2',
// 'Cache-Control: no-cache',
// 'Connection: keep-alive'
);
$debug = TRUE;
// Set the request type
curl_setopt($ch, CURLOPT_VERBOSE, $debug);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET');
curl_setopt($ch, CURLOPT_NOBODY, FALSE);
curl_setopt($ch, CURLOPT_URL, $this->url);
curl_setopt($ch, CURLOPT_USERAGENT, $this->userAgent);
curl_setopt($ch, CURLOPT_REFERER, $this->referrer);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_HEADER, $debug);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_ENCODING , 'gzip');
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 GTB5');
curl_setopt($ch, CURLOPT_COOKIEJAR, '/tmp/cookies.txt');
curl_setopt($ch, CURLOPT_COOKIEFILE, '/tmp/cookies.txt');
$data = curl_exec($ch);
var_dump($data);die;
return curl_exec($ch);
}
Here is the verbose response:
* Adding handle: conn: 0x7f937504e400
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x7f937504e400) send_pipe: 1, recv_pipe: 0
* About to connect() to www.mixcloud.com port 80 (#0)
* Trying 46.23.65.210...
* Connected to www.mixcloud.com (46.23.65.210) port 80 (#0)
> GET / HTTP/1.1
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 GTB5
Host: www.mixcloud.com
Accept-Encoding: gzip
Referer: https://www.google.com.au
Accept: */*
< HTTP/1.1 403 Forbidden
* Server nginx/1.5.8 is not blacklisted
< Server: nginx/1.5.8
< Date: Tue, 18 Feb 2014 06:39:45 GMT
< Content-Type: text/html
< Transfer-Encoding: chunked
< Connection: keep-alive
< Vary: Accept-Encoding
< Content-Encoding: gzip
<
* Connection #0 to host www.mixcloud.com left intact
string(376) "HTTP/1.1 403 Forbidden\r\nServer: nginx/1.5.8\r\nDate: Tue, 18 Feb 2014 06:39:45 GMT\r\nContent-Type: text/html\r\nTransfer-Encoding: chunked\r\nConnection: keep-alive\r\nVary: Accept-Encoding\r\nContent-Encoding: gzip\r\n\r\n<html>\r\n<head><title>403 Forbidden</title></head>\r\n<body bgcolor="white">\r\n<center><h1>403 Forbidden</h1></center>\r\n<hr><center>nginx/1.5.8</center>\r\n</body>\r\n</html>\r\n"
Now if I try to execute the curl command in the shell it's working fine:
$ curl -i 'http://www.mixcloud.com' -v
* Adding handle: conn: 0x7fe28b004000
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x7fe28b004000) send_pipe: 1, recv_pipe: 0
* About to connect() to www.mixcloud.com port 80 (#0)
* Trying 46.23.65.210...
* Connected to www.mixcloud.com (46.23.65.210) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.30.0
> Host: www.mixcloud.com
> Accept: */*
>
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Date: Tue, 18 Feb 2014 06:41:30 GMT
Date: Tue, 18 Feb 2014 06:41:30 GMT
< Content-Type: text/html; charset=utf-8
Content-Type: text/html; charset=utf-8
< Content-Length: 194847
Content-Length: 194847
< Connection: keep-alive
Connection: keep-alive
< Vary: Accept-Encoding
Vary: Accept-Encoding
* Server gunicorn/0.17.4 is not blacklisted
< Server: gunicorn/0.17.4
Server: gunicorn/0.17.4
< Vary: Cookie, User-Agent, X-Requested-With, X-Ignore-Block
Vary: Cookie, User-Agent, X-Requested-With, X-Ignore-Block
< x-xss-protection: 1; mode=block
x-xss-protection: 1; mode=block
< x-content-type-options: nosniff
x-content-type-options: nosniff
< Set-Cookie: csrftoken=ciOosbUNp5EL8t5tiQQzkoeaJIDJ3VfO; Domain=.mixcloud.com; expires=Tue, 17-Feb-2015 06:41:30 GMT; Max-Age=31449600; Path=/
Set-Cookie: csrftoken=ciOosbUNp5EL8t5tiQQzkoeaJIDJ3VfO; Domain=.mixcloud.com; expires=Tue, 17-Feb-2015 06:41:30 GMT; Max-Age=31449600; Path=/
< Set-Cookie: eventstream=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/
Set-Cookie: eventstream=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/
<
<!DOCTYPE html> ...
I know that the cURL for PHP and cURL are different, but I can't see what I am missing.
Anyone?
Cheers,
Maxime
Ok I've found what was the issue. It was the user-agent.
It's really weird. I was using this user-agent:
Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 GTB5
With this user agent I was getting a 403. I've updated it using the following one:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.152 Safari/537.36
And it's now working well. I can't believe that people are still rejecting request for specific user agent...
I'm almost 100% sure I have ready every post on the internet that contains the keywords asp login curl php .ASPXAUTH, but I have been unable to find a solution. I am more of a code hacker than elegant developer though, so I hope that someone can help me please.
I have a curl script that logs in to two other websites to submit forms from behind the login successfully. However, I've recently tried to use a variation of this script for a third website. It works as far as returning the first page after login but then it treats any further cURL calls as if I haven't logged in. I discovered (well I think) that it's to do with the .ASPXAUTH cookie not being set. I do have a cookiefile and cookiejar setup in my cURL code and it catches the .ASP.NET_SessionID successfully, but not the .ASPXAUTH cookie.
I noticed that I can see the .ASPXAUTH cookie value in the headers when I watch "Live HTTP headers" but I can't get my cURL script to return the header with this set-cookie very easily. It seems that the cookie is set on a 302 after login and cURL is not handling this correctly. So I turned off CURLOPT_FOLLOWLOCATION and was trying to handle the redirect myself but I still can't get it right (the server returns a really strange redirect url and I don't think I'm doing this part right)
But I would be very grateful if someone could please help me...
Here is my code:
//setup Curl
$cookiename = substr($from,4,5);
$cookiefile = $cookiename . ".txt";
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (Windows; MSIE 6.0; U; Windows NT 5.1)");
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookiefile);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookiefile);
//read login page
curl_setopt($ch, CURLOPT_URL, "Login.aspx");
$result = curl_exec ($ch);
echo $result;
// extract values for hidden form fields __REQUESTDIGEST __VIEWSTATE __EVENTVALIDATION fields
//extract __REQUESTDIGEST
$start = strpos($result,"id=\"__REQUESTDIGEST\" value=\"") + 28;
$end = $start + 157;
$rdigest = substr($result , $start , $end - $start );
//extract __VIEWSTATE
$start = strpos($result,"id=\"__VIEWSTATE\" value=\"") + 24;
$end = $start + 16300;
$vstate = substr($result , $start , $end - $start );
$vstate = urlencode($vstate);
//extract __EVENTVALIDATION
$start = strpos($result,"id=\"__EVENTVALIDATION\" value=\"") + 30;
$end = $start + 120;
$event = substr($result , $start , $end - $start );
$event = urlencode($event);
//set login form values and login
//curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_REFERER, 'Login.aspx');
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, '__REQUESTDIGEST=' . $rdigest . '&__VIEWSTATE=' . $vstate . '&__EVENTVALIDATION=' . $event . '&UserName=' . $from . '&Password=' . $password);
$result = curl_exec ($ch);
echo $result;
//extract __redirect
$start = strpos($result,"Location:") + 10;
$end = strpos($result,".aspx") +5;
$redirect = substr($result , $start , $end - $start );
$redirect = "https://www.domain.com/" . $redirect;
echo $redirect ."<br /><br />";
echo $result;
curl_setopt($ch, CURLOPT_URL, $redirect);
$result = curl_exec ($ch);
echo $result;
And here is the output:
//Login page headers
HTTP/1.1 200 OK Date: Tue, 30 Nov 2010 12:57:09 GMT Server: Microsoft-IIS/6.0 X-Powered-By: ASP.NET X-AspNet-Version: 2.0.50727 Cache-Control: no-cache Pragma: no-cache Expires: -1 Content-Type: text/html; charset=utf-8 Content-Length: 81835
//Login page body
Submit login page headers
HTTP/1.1 100 Continue HTTP/1.1 302 Found Date: Tue, 30 Nov 2010 13:40:30 GMT Server: Microsoft-IIS/6.0 X-Powered-By: ASP.NET X-AspNet-Version: 2.0.50727 Location: /(F(RZPDiDBb9OPbTuBnj2RAgH8KglRdj4B4u8trRMpa6QbBjff4evKMtHnOFNyX046Xdr33PZA3-6dHoZjxQpeZ7aNTevF75gArtpeScCjE9fI1))/default.aspx Set-Cookie: ASP.NET_SessionId=bhugr045cyybck45xvhpeb55; path=/; HttpOnly Cache-Control: no-cache Pragma: no-cache Expires: -1 Content-Type: text/html; charset=utf-8 Content-Length: 82196
//Redirect page body
//The login page body is displayed again
//More headers
HTTP/1.1 100 Continue HTTP/1.1 500 Internal Server Error Date: Tue, 30 Nov 2010 13:29:05 GMT Server: Microsoft-IIS/6.0 X-Powered-By: ASP.NET X-AspNet-Version: 2.0.50727 Cache-Control: private Content-Type: text/html; charset=utf-8 Content-Length: 3026
//Error message from server
Server Error in '/' Application.
Runtime Error
Description: An application error occurred on the server. The current custom error settings for this application prevent the details of the application error from being viewed remotely (for security reasons). It could, however, be viewed by browsers running on the local server machine.
I updated my useragent line to a different user agent and suddenly the .ASPXAUTH cookie was set correctly (and automatically) in the cookie file :)
In otherwords I changed this line:
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (Windows; MSIE 6.0; U; Windows NT 5.1)");
to this:
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 ( .NET CLR 3.5.30729)");
And now both cookies are set automatically by curl - no problem.
Hooray!