Logging into a website with CURL PHP not working - php

I have a problem logging in to a website with CURL and PHP.
I test with the Firefox add-on HttpRequester and this worked.
Result login:
POST https://www.balatarin.com/sessions
Content-Type: application/x-www-form-urlencoded
session[login]=testeruni&session[password]=123456789&session[remember_me]=1&commit=%D9%88%D8%B1%D9%88%D8%AF&utf8=%E2%9C%93&authenticity_token[![httprequester][1]][1]
-- response --
200 OK
Server: shield
Date: Thu, 19 Jan 2017 13:51:54 GMT
Content-Type: text/html; charset=utf-8
status: 200 OK
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
x-ua-compatible: IE=Edge,chrome=1
Etag: W/"7418542e936fbdfe20002faf11876845"
Cache-Control: must-revalidate, private, max-age=0
Set-Cookie: _balat_session_new=BAh7C0kiDHVzZXJfaWQGOgZFRmkD964BSSIPc2Vzc2lvbl9pZAY7AEZJIiUzZGUxMzIyN2ZhZDVmMDUzOGE3OGY0YTRhZDkzNmUyMQY7AFRJIhZpbnB1dF9kZXZpY2VfdHlwZQY7AEZJIgpNT1VTRQY7AEZJIhRob3Zlcl9zdXBwb3J0ZWQGOwBGVEkiCmZsYXNoBjsARm86JUFjdGlvbkRpc3BhdGNoOjpGbGFzaDo6Rmxhc2hIYXNoCToKQHVzZWRvOghTZXQGOgpAaGFzaHsGOgtub3RpY2VUOgxAY2xvc2VkRjoNQGZsYXNoZXN7BjsKSSI22YbYrtiz2Kog2YXYtNiq2LHaqSDahtmG2K8g2KjYp9mE2KfahtmHINi02YjbjNivLgY7AFQ6CUBub3cwSSIQX2NzcmZfdG9rZW4GOwBGSSIxT3krNk5nM1NTM2IreXc4SUtxbW9yN2NmMXQrdUNLWWdubFRRYmpidmtNTT0GOwBG--2c2a72f8ec27564250ba084d97998aefba4af11a; path=/; secure; HttpOnly geo=0
X-Request-Id: 521288561d7cfff0ef8fe8d72080760c
X-Runtime: 0.188862
X-Rack-Cache: miss
Content-Encoding: gzip
Via: 1.1 google
Alt-Svc: clear
Expires: Thu, 19 Jan 2017 13:51:54 GMT
X-Firefox-Spdy: h2
but it does not login with curl in PHP. I tested all headers in my CURL but it does not login, only works with HttpRequester.
public function actionLoggin()
{
$url = 'https://www.balatarin.com/sessions';
$headers[] = 'Content-Type: application/x-www-form-urlencoded';
$headers[] = 'Host: www.balatarin.com';
$headers[] = 'User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:50.0) Gecko/20100101 Firefox/50.0';
$headers[] = 'Referer: https://www.balatarin.com/login';
$params = array(
'session[login]' => 'testeruni',
'session[password]' => '123456789',
'session[remember_me]' => '0',
'commit' => 'ورود',
'utf8' => '✓',
'authenticity_token' => '',
);
//open connection
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($params));
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_COOKIEJAR, 'bala_cookie.txt');
curl_setopt($ch, CURLOPT_COOKIEFILE, 'bala_cookie.txt');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$result = curl_exec($ch);
curl_close($ch);
echo $result;
}
Here is my cookie file:
# Netscape HTTP Cookie File
# https://curl.haxx.se/docs/http-cookies.html
# This file was generated by libcurl! Edit at your own risk.
www.balatarin.com FALSE / FALSE 0 logged_in 1
#HttpOnly_www.balatarin.com FALSE / TRUE 0 _balat_session_new BAh7CToOcmV0dXJuX3RvMDoMdXNlcl9pZGkDj60BOhJsb2dpbl9yZXRyaWVzMEkiD3Nlc3Npb25faWQGOgZFRkkiJTgwN2ZmMDRjMGUzMzkyMDIyZWY5YzBmZTQxN2FmZWMzBjsIVA%3D%3D--d47dd61bc9900449cca69ebd727041c3946a13ba
www.balatarin.com FALSE / FALSE 0 geo 0
www.balatarin.com FALSE / FALSE 1516368886 corr b8ed93fa279a469a637b

Related

How to use oembed Facebook

I'm trying to embed Facebook's posts (e.g. video) using oEmbed format. According to Facebook documentation, oEmbed is now supported. I'm trying this PHP code:
$json_post = #file_get_contents('https://www.facebook.com/plugins/video/oembed.json/?url={MY VIDEO URL HERE}');
$oembed = json_decode($json_post);
var_dump($oembed);
I already used the same code for Instagram with success, now I'm getting a NULL result. oEmbed works good if i directly write the URL on the browser. Am i missing something?
Thanks.
Update
I tried with Curl:
$url='https://www.facebook.com/plugins/video/oembed.json/?url=https%3A%2F%2Fwww.facebook.com%2Ffacebook%2Fvideos%2F10153231379946729%2F';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, TRUE);
//curl_setopt($ch, CURLOPT_NOBODY, TRUE); // remove body
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
$page = curl_exec($ch);
$httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
print_r($page);
curl_close($ch);
Now i get:
HTTP/1.1 302 Found
Location: https://www.facebook.com/unsupportedbrowser
access-control-allow-method: OPTIONS
Access-Control-Expose-Headers: X-FB-Debug, X-Loader-Length
Access-Control-Allow-Origin: https://www.facebook.com
Vary: Origin
Access-Control-Allow-Credentials: true
Content-Type: text/html
X-FB-Debug: gGcZzyllZadlcn/6jz2HqqouIcDnhTzxzR+etWXhZEnOcditfsaIUw0WjgO3nELHzveRCYa1UM86D3LA/nLnNw==
Date: Wed, 11 Jan 2017 10:18:47 GMT
Connection: keep-alive
Content-Length: 0
HTTP/1.1 200 OK
X-XSS-Protection: 0
public-key-pins-report-only: max-age=500; pin-sha256="WoiWRyIOVNa9ihaBciRSC7XHjliYS9VwUGOIud4PB18="; pin-sha256="r/mIkG3eEpVdm+u/ko/cwxzOMo1bk4TyHIlByibiA5E="; pin-sha256="q4PO2G2cbkZhZ82+JgmRUyGMoAeozA+BSXVXQWB8XWQ="; report-uri="http://reports.fb.com/hpkp/"
Pragma: no-cache
Cache-Control: private, no-cache, no-store, must-revalidate
Expires: Sat, 01 Jan 2000 00:00:00 GMT
X-Content-Type-Options: nosniff
Strict-Transport-Security: max-age=15552000; preload
X-Frame-Options: DENY
Vary: Accept-Encoding
Content-Type: text/html
X-FB-Debug: zwArox8KyM3BtwLymhiARCTltrrcE/pDqSWdqbHgstXVBEbIXG57Od2MfDnqgqSX5Tj43qoe8uYhphzwoZcXeg==
Date: Wed, 11 Jan 2017 10:18:48 GMT
Transfer-Encoding: chunked
Connection: keep-alive
Still waiting for a reply.
Thank You.
Set the user agent with the curl and try,
$browser = 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.16 (KHTML, like Gecko) \Chrome/24.0.1304.0 Safari/537.16';
curl_setopt($ch, CURLOPT_USERAGENT, $browser);
Here is the answer with file_get_content,
$options = array(
'http'=>array(
'method'=>"GET",
'header'=>"Accept-language: en\r\n" .
"Cookie: foo=bar\r\n" . // check function.stream-context-create on php.net
"User-Agent: Mozilla/5.0 (iPad; U; CPU OS 3_2 like Mac OS X; en-us) AppleWebKit/531.21.10 (KHTML, like Gecko) Version/4.0.4 Mobile/7B334b Safari/531.21.102011-10-16 20:23:10\r\n" // i.e. An iPad
)
);
$context = stream_context_create($options);
$json_post = #file_get_contents('https://www.facebook.com/plugins/video/oembed.json/?url=https%3A%2F%2Fwww.facebook.com%2Ffacebook%2Fvideos%2F10153231379946729%2F', false, $context);
$oembed = json_decode($json_post);
var_dump($oembed)

Why curl charset don't change ?

I need to parse with simpl_html_dom german site.I have a problem with german Umlauts, because utf-8 don't support Umlauts. I know ,if convert text from UTF-8 to UTF-16 or ISO-8859-1 problem solved.I use CURL for get content page. This page have charset ISO-8859-1.I try to set CURLOPT_ENCODING ISO-8859-1 ,but Curl always return the utf-8 text.I don't know what do.
Code of this method.
public function testsec()
{
require_once DIR_SYSTEM.'library'.DIRECTORY_SEPARATOR.'simpleHtml'.DIRECTORY_SEPARATOR.'simple_html_dom.php';
$regexpSecond = "~Möglicherweise.*? Vielen Dank~su";
$headers = array(
"User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0",
"Accept: text/plain",
"Connection: keep-alive",
);
$fp = fopen(DIR_ADMIN.'logCurl.txt','w+');
$head = fopen(DIR_ADMIN.'headers.txt','w+');
$curl = curl_init("http://test.site.com/bla-bla-bla");
curl_setopt($curl, CURLOPT_RETURNTRANSFER,true);
curl_setopt($curl, CURLOPT_ENCODING , "UTF-16");
curl_setopt($curl, CURLOPT_VERBOSE, 1);
curl_setopt($curl, CURLOPT_STDERR, $fp);
curl_setopt($curl, CURLOPT_HEADER ,$headers);
curl_setopt($curl, CURLOPT_WRITEHEADER, $head);
$result = curl_exec($curl);
curl_close($curl);
fclose($fp);
fclose($head);
$html = str_get_html($result);
echo mb_detect_encoding($result); //utf-8
}
Headers response
HTTP/1.1 200 OK
Date: Sun, 03 Jul 2016 05:22:34 GMT
Server: Apache
Set-Cookie: JTLSHOP=c1qv3vafghmf3ih43g5m96epi4; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: max-age=1, private, must-revalidate
Pragma: no-cache
Vary: Accept-Encoding
X-Powered-By: PleskLin
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html; charset=iso-8859-1
UTF-8 support Umlauts.
http://www.periodni.com/unicode_utf-8_encoding.html#german_special_characters
If you want to convert charset, Use ICONV function.

PHP cURL doesn't set content-length properly

I'm trying to upload a file via cURL but something is missing. I forces this request to be HTTP 1.0 because cURL adds the Expect: 100 header if I use HTTP 1.1 so thats why the extra header. Here is a simple test code:
<?php
if(isset($_POST["id"])) {
$data = array("id" => $_POST["id"]);
$data["file"] = "#".realpath($_FILES["file"]["tmp_name"]);
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
'Authorization: Bearer 0e39ffba-66cd-4933-9e94-fcdf600c2453',
'Connection: keep-alive'
));
curl_setopt($ch, CURLOPT_URL, "http://localhost:8080/test-api/upload");
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
curl_setopt($ch, CURLOPT_VERBOSE, false);
curl_setopt($ch, CURLOPT_HTTP_VERSION, 1);
$response = curl_exec($ch);
var_dump($response);
exit;
}
?>
My Jersey based server picks it up, and I can see these headers:
INFO: 25 * Server has received a request on thread http-nio-8080-exec-1
25 > POST http://localhost:8080/test-api/upload
25 > authorization: Bearer 0e39ffba-66cd-4933-9e94-fcdf600c2453
25 > connection: keep-alive
25 > content-length: 261
25 > content-type: multipart/form-data; boundary=------------------------53f7ba34739b4d9e
25 > host: localhost:8080
See the content-length? It's way too short. When I send the same file and the same request via my Postman REST client, I get these headers:
INFO: 26 * Server has received a request on thread http-nio-8080-exec-3
26 > POST http://localhost:8080/test-api/upload
26 > accept-encoding: gzip, deflate
26 > accept-language: hu-HU,hu;q=0.8,en-US;q=0.6,en;q=0.4
26 > authorization: Bearer 0e39ffba-66cd-4933-9e94-fcdf600c2453
26 > cache-control: no-cache, no-cache
26 > connection: keep-alive
26 > content-length: 144954
26 > content-type: multipart/form-data; boundary=----WebKitFormBoundarye5Tg0kEqi10nEBwv
26 > cookie: ff_uvid=126143952; _ga=GA1.1.459454356.1439469592; CAKEPHP=9mffidqo8203ugktan4roc0u82
26 > host: localhost:8080
26 > origin: chrome-extension://fdmmgilgnpjigdojojpjoooidkmcomcm
26 > user-agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.80 Safari/537.36
The content-length now is set property. What could be wrong here?
It sounds like you're using PHP 5.6.0 or later. As of this release, the # prefix for file uploads is disabled by default. You can enable it with
curl_setopt($ch, CURLOPT_SAFE_UPLOAD, false);
This option was added in 5.5, but the default was false for backward compatibility; 5.6 changed the default incompatibly.
The preferred way to perform file uploads starting with 5.5 is with the CurlFile class.
$data["file"] = new CurlFile(realpath($_FILES["file"]["tmp_name"]));
You have to actually insert the filecontent, this differs from the cli-version of curl.
try:
$data["file"] = file_get_contents($_FILES["file"]["tmp_name"]);

$_SESSION variable empty on multiple cURL requests

In my first cURL request i upload a file and i set a $_SESSION variable with the name, extension etc. In my second cURL request i want to move the uploaded file from tmp folder to user folder but baddly the $_SESSION variable is empty. why?
first request code code looks like this:
$upload = curl_init();
curl_setopt($upload, CURLOPT_URL, "http://localhost/upload/" );
curl_setopt($upload, CURLOPT_POST, true );
curl_setopt($upload, CURLOPT_RETURNTRANSFER, true );
curl_setopt($upload, CURLOPT_USERAGENT, $_SERVER["HTTP_USER_AGENT"] );
curl_setopt($upload, CURLOPT_HTTPHEADER, array("Set-Cookie: data=" . urldecode($cookie) ));
curl_custom_postfields($upload, $fields, $files);
$res = curl_exec($upload);
curl_close($upload);
and the second request code: following the first request:
$submit = curl_init();
curl_setopt($submit, CURLOPT_URL, "http://localhost/" );
curl_setopt($submit, CURLOPT_RETURNTRANSFER, true );
curl_setopt($submit, CURLOPT_USERAGENT, $_SERVER["HTTP_USER_AGENT"] );
curl_setopt($submit, CURLOPT_POST, count($fields));
curl_setopt($submit, CURLOPT_POSTFIELDS, $fields_string );
curl_setopt($submit, CURLOPT_HEADER, true);
curl_setopt($submit, CURLOPT_HTTPHEADER, array("Cookie: data=" . urldecode($cookie) ));
$res = curl_exec($submit);
curl_close($submit);
is there any option to keep session alive? is the same problem i meet on AJAX requests when i start using javascript with AJAX i think.
my response header:
HTTP/1.1 200 OK
Date: Wed, 04 Mar 2015 02:24:22 GMT
Server: Apache/2.2.12 (Win32) DAV/2 mod_ssl/2.2.12 OpenSSL/0.9.8k mod_autoindex_color PHP/5.3.0 mod_perl/2.0.4 Perl/v5.10.0
X-Powered-By: PHP/5.3.0
Set-Cookie: PHPSESSID=g7hc328ij8lr63mps6ub44gat2; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Content-Length: 22
Content-Type: text/html
I think you need a cookie jar to keep track of your session:
curl_setopt($ch, CURLOPT_COOKIEJAR, "/tmp/cookies");

PHP curl return 403 but not the shell command

I'm trying to implement a functionnality like facebook, when you paste a link it's grabbing some information (h1, desc, images, ...) from the page and display them.
I already face several issues that I managed to fix (gzip, cookies, user agent, ...) but on this one I'm not sure what is blocking my request.
The link in question is http://www.mixcloud.com
Here is my PHP script:
protected function getContent()
{
$ch = curl_init();
$headers = array(
'Accept: */*',
// 'Accept-Encoding: gzip,deflate,sdch',
// 'Accept-Language: en-US,en;q=0.8,es;q=0.6,fr;q=0.4,pt;q=0.2',
// 'Cache-Control: no-cache',
// 'Connection: keep-alive'
);
$debug = TRUE;
// Set the request type
curl_setopt($ch, CURLOPT_VERBOSE, $debug);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET');
curl_setopt($ch, CURLOPT_NOBODY, FALSE);
curl_setopt($ch, CURLOPT_URL, $this->url);
curl_setopt($ch, CURLOPT_USERAGENT, $this->userAgent);
curl_setopt($ch, CURLOPT_REFERER, $this->referrer);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_HEADER, $debug);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_ENCODING , 'gzip');
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 GTB5');
curl_setopt($ch, CURLOPT_COOKIEJAR, '/tmp/cookies.txt');
curl_setopt($ch, CURLOPT_COOKIEFILE, '/tmp/cookies.txt');
$data = curl_exec($ch);
var_dump($data);die;
return curl_exec($ch);
}
Here is the verbose response:
* Adding handle: conn: 0x7f937504e400
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x7f937504e400) send_pipe: 1, recv_pipe: 0
* About to connect() to www.mixcloud.com port 80 (#0)
* Trying 46.23.65.210...
* Connected to www.mixcloud.com (46.23.65.210) port 80 (#0)
> GET / HTTP/1.1
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 GTB5
Host: www.mixcloud.com
Accept-Encoding: gzip
Referer: https://www.google.com.au
Accept: */*
< HTTP/1.1 403 Forbidden
* Server nginx/1.5.8 is not blacklisted
< Server: nginx/1.5.8
< Date: Tue, 18 Feb 2014 06:39:45 GMT
< Content-Type: text/html
< Transfer-Encoding: chunked
< Connection: keep-alive
< Vary: Accept-Encoding
< Content-Encoding: gzip
<
* Connection #0 to host www.mixcloud.com left intact
string(376) "HTTP/1.1 403 Forbidden\r\nServer: nginx/1.5.8\r\nDate: Tue, 18 Feb 2014 06:39:45 GMT\r\nContent-Type: text/html\r\nTransfer-Encoding: chunked\r\nConnection: keep-alive\r\nVary: Accept-Encoding\r\nContent-Encoding: gzip\r\n\r\n<html>\r\n<head><title>403 Forbidden</title></head>\r\n<body bgcolor="white">\r\n<center><h1>403 Forbidden</h1></center>\r\n<hr><center>nginx/1.5.8</center>\r\n</body>\r\n</html>\r\n"
Now if I try to execute the curl command in the shell it's working fine:
$ curl -i 'http://www.mixcloud.com' -v
* Adding handle: conn: 0x7fe28b004000
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x7fe28b004000) send_pipe: 1, recv_pipe: 0
* About to connect() to www.mixcloud.com port 80 (#0)
* Trying 46.23.65.210...
* Connected to www.mixcloud.com (46.23.65.210) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.30.0
> Host: www.mixcloud.com
> Accept: */*
>
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Date: Tue, 18 Feb 2014 06:41:30 GMT
Date: Tue, 18 Feb 2014 06:41:30 GMT
< Content-Type: text/html; charset=utf-8
Content-Type: text/html; charset=utf-8
< Content-Length: 194847
Content-Length: 194847
< Connection: keep-alive
Connection: keep-alive
< Vary: Accept-Encoding
Vary: Accept-Encoding
* Server gunicorn/0.17.4 is not blacklisted
< Server: gunicorn/0.17.4
Server: gunicorn/0.17.4
< Vary: Cookie, User-Agent, X-Requested-With, X-Ignore-Block
Vary: Cookie, User-Agent, X-Requested-With, X-Ignore-Block
< x-xss-protection: 1; mode=block
x-xss-protection: 1; mode=block
< x-content-type-options: nosniff
x-content-type-options: nosniff
< Set-Cookie: csrftoken=ciOosbUNp5EL8t5tiQQzkoeaJIDJ3VfO; Domain=.mixcloud.com; expires=Tue, 17-Feb-2015 06:41:30 GMT; Max-Age=31449600; Path=/
Set-Cookie: csrftoken=ciOosbUNp5EL8t5tiQQzkoeaJIDJ3VfO; Domain=.mixcloud.com; expires=Tue, 17-Feb-2015 06:41:30 GMT; Max-Age=31449600; Path=/
< Set-Cookie: eventstream=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/
Set-Cookie: eventstream=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/
<
<!DOCTYPE html> ...
I know that the cURL for PHP and cURL are different, but I can't see what I am missing.
Anyone?
Cheers,
Maxime
Ok I've found what was the issue. It was the user-agent.
It's really weird. I was using this user-agent:
Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 GTB5
With this user agent I was getting a 403. I've updated it using the following one:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.152 Safari/537.36
And it's now working well. I can't believe that people are still rejecting request for specific user agent...

Categories