cannot use cookies while curling gmail account - php

Question

I'm trying to authorize on gmail, but it isn't see cookies.
Error. Most likely, your browser does not set a cookie. Check this
setting, or open a new browser window.
That's my code:
$tmpfname = dirname(__FILE__).'/cookie.txt';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://accounts.google.com/ServiceLoginAuth");
curl_setopt($ch, CURLOPT_POSTFIELDS, "GALX=MS-tSuNi3pg&continue=https%3A%2F%2Fmail.google.com%2Fmail%2F&service=mail&hl=ru&_utf8=%E2%98%83&bgresponse=%21A0Lf0QLPRaBwlUTp7ftMkStPvwIAAAAWUgAAAAcqAQFYvp-abJNRjR3DH8MqLNMd2V1lZIUZ8WD7-V22z_v8Lc-TfjBVXX8E0ElzA2hSNiaMERRhArrPj3NR1EuQ7UUE7KbsJ3DPYmn7jsKtGklYfxzO3Uonm6nKj_cfATL8wXFt_ngIdwFI0rY8J_2Kb51KDoxtcx6eEYfD8P0m-t6NcAITwyy3_0EG-1R12MNb2Lc7uLcMW76sHRTt2vc1zV1SjofqaYf73xJ5r-uatz_VTHQ_mT2JBU-92L32nx8qu9JF5__SAcj3-2umIjEiQvqd7KVxuFrSpKHiOGWkzr7CG9DMwFJVYeNvaE0liWW549s7yNcWIu_ERgau0KR0wyIC9A&pstMsg=1&dnConn=&checkConnection=youtube%3A137%3A1&checkedDomains=youtube&Email=*******&Passwd=*******&signIn=%D0%92%D0%BE%D0%B9%D1%82%D0%B8&PersistentCookie=yes&rmShown=1");
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_COOKIESESSION, true );
curl_setopt($ch, CURLOPT_COOKIEJAR, $tmpfname);
curl_setopt($ch, CURLOPT_COOKIEFILE, $tmpfname);
$result = curl_exec($ch);

Answer 1

But Cookies are just a very minor issue.
Google does not make it easy to use curl to login. Because Google uses cookies buried in a 301 redirect, curl may not keep them. Sometimes you also have to grab HTML our of hidden fields <input type=hidden name=”_NAME” value=”_VALUE”>
You have some work ahead of you. It's not as simple as you may think. It certainly cannot be done with one curl HTTP GET. gMail is a nightmare.
Along with about 50 HTTP GET and POST Requests on top of the redirects, Google also uses over 100 JS XHR GET and POST requests and tons of JSON. Information is embedded as cookies, URL Query Strings, and POST Data.
The big hurdle is that gMail will not function without javaScript. Curl does not have built in JavaScript. Without JavaScript you are getting nothing from gMail.
It is not an impossible feat. With 100% certainty it can be done. How long will it take you? is the question. My guess is it will take you about a year to get in from log-in to retrieve and send mail. That is why I suggest you try one function first. Then you will get a taste of what is ahead of you.
What you may be able to do is go to the page where you want to post or scrape the data from, record current cookies then click the feature, then get all the HTML, JS, and XHR requests and responses. You may be able to duplicate that one function without JavaScript. But you have to replace some/most/all (not sure which) of the JS requests with one of your own using curl.
Be prepared to spend some time updating you code as Google is a moving target. They keep changing the way things are done and you'll have to keep up with them.
But the cookies is simple.
This is my work around logging into Google Voice
First I would go to https://www.google.com/voice/
Google puts the cookies in a 301 Redirect. Then four more 302 redirects a little further down the road.
So I do not use:
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
I use:
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);
Then I will need access to the headers
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLINFO_HEADER_OUT, true);
Grab Cookies from Response Header
$data = curl_exec($ch);
if (curl_errno($ch)){
$data .= 'Retrieve Base Page Error: ' . curl_error($ch);
}
else {
$skip = intval(curl_getinfo($ch, CURLINFO_HEADER_SIZE));
$head = substr($data,0,$skip);
$e = 0;
while(true){
$s = strpos($head,'Set-Cookie: ',$e);
if (!$s){break;}
$s += 12;
$e = strpos($head,';',$s);
$cookie = substr($head,$s,$e-$s) ;
$s = strpos($cookie,'=');
$key = substr($cookie,0,$s);
$value = substr($cookie,$s);
$cookies[$key] = $value;
}
Then create cookie for request header:
$cookie = '';
$delim = '';
foreach ($cookies as $k => $v){
$cookie .= "$delim$k$v";
$delim = '; ';
}
Then catch their redirect location url
$info = curl_getinfo($ch);
$url = $info['redirect_url'];
Look to see if it is a redirect.
if (strlen($url) < 8){
$url='https://accounts.google.com/ServiceLogin';
}
sleep(2);
Then put the cookie in the header:
$request = array();
$request[] = "Host: accounts.google.com";
$request[] = "Pragma: no-cache";
$request[] = "Cookie: $cookie";
$request[] = "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
$request[] = "User-Agent: MOT-V9mm/00.62 UP.Browser/6.2.3.4.c.1.123 (GUI) MMP/2.0";
$request[] = "Accept-Language: en-US,en;q=0.5";
$request[] = "Connection: keep-alive";
$request[] = "Cache-Control: no-cache";
curl_setopt($ch, CURLOPT_HTTPHEADER, $request);
And when there is a Referer (That is not how I spell Referrer, the guy that added it to HTTP spelled it wrong)
$request[] = 'Referer: https://accounts.google.com/ServiceLogin?service=grandcentral&passive=1209600&continue=https%3A%2F%2Fwww.google.com%2Fvoice&followup=https%3A%2F%2Fwww.google.com%2Fvoice&ltmpl=open';
Get the cookies the same way from the redirect page as previously
Then grab the GALX cookie. Then do the Next Request.
$galax = $cookies['GALX'];
$post = "GALX=$galax&continue=https://www.google.com/voice&followup=https://www.google.com/voice&service=grandcentral&ltmpl=open&_utf8=%E2%98%83&bgresponse=js_disabled&Email=assratbastard#gmail.com&Passwd=$password&signIn=Sign+in&PersistentCookie=yes&rmShown=1";
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
curl_setopt($ch, CURLOPT_POST, true);
Lots more of those before you get in.
An example of things further down the road. this is just one HTTP POST Request
Query String Data
continue=https://mail.google.com/mail/
service=mail
sarp=1
Cookies
Cookie: GAPS=1:tx6dl5mwyjNKgiOEtjcvTvzGSNZqQQ:X9TX1quYjhjQfjho; GALX=kBQZRL4MXuU; GMAIL_RTT=216; GMAIL_LOGIN=T1420606553375/1420606553375/1420606580476; NID=67=LbIeO3Xwxjs0nGgZaTOTLrhdJ5bb7_Ce-de10-rKYZVzKVdM4XoKVr3T18sb9NLg_ghRkDoa-G-6vb66FdMR6uIMstAPd0qdQa18s1zGTHtvSOv8lRXaAdDDzqp8p8mguo0xA6VZnz_vV1JnoHMfulS9yoO4PA; SID=DQAAAAkBAADu9krli4XZTP6IWYOSEsmDBjYazF_ywtDmORhqZ8OeVGaC_K-3lSy4cNosYYXfG_-hrMd31fLPbAljFRt3Z5tpOAMLUPmzluYZC0_y1NTWMJ4D7I_bpIgiAsZO5oT9EFobf0vX50KfHLVKTHCetrgckDmLtMd4EkrOqsLkAAK9prD440GMqgCRoICNxLRVu-kS_-5N9mRrIuC3xsOsdi27Qfk4wPOqYNcO5sT1RGGgv1y7jwLqvHzHtz5DmlfARHv9lDtnKM8Gy3jo2Ax_7u8OrwIUP7Tcmz_9FJcj_q_Cz1cu94DbMHDN_qiUIwL1xYzClsdu3Z8EFiHDiEc8esXLg5_HkXPOPOvy-iGO9gTdLQ; LSID=ss:DQAAAAsBAABw1hSyS55goXFvcpcXQZQALGca7K26kfQ6HBc4c_agj3DJe_qMBMzqh0WXc3KNQ8OwP0lCPauBEhr3AdD0DyhCZQDFuIoglHPiw91_r-KIEZ62KjSmuTepv1UYDDEDiZeB5rYEOw4L6l2sOpOBmgBOZOyLfum4azJBLpEYo9kvMsX-OPUlqEJF0z0UMKM-R8Wh1Oxydr0j5R97U_juccmU6DqVsm0DTrP7rjPfv7cfZJ1wdqVemacZdfWjabrExrsXC21fin8ZUtXQI1dL8twk7fM7vo4fvKNdKoACBRUZpxltL9sTtBV-6QcynJF6Km5J6ICynuU3rtZvQNOS5VPIeajbcea7MI5p85XgweiVnw; HSID=A_8tAVmju5qj5J98Y; SSID=A_mBRb5lH8DXaOmm7; APISID=iNCCKNUIqLSXwe-P/AY-19Si5OAZhIv1aj; SAPISID=otuPxzrzp-BltlGm/AKleRqZyVwfhwwCB0; ACCOUNT_CHOOSER=AFx_qI5lJUnyOaSRIf2vxUKACWjny3nvliEw3h7h6NlUUHsklUqbMGc5NH7u6m6u4OSw8s5QqcsmV_fYx7-szFy4TVyvuA6A_itoAFoG-6B9txvdhP2T9gXFJzeRVMKHCQlRie0vibTz
POST Data
GALX=kBQZRL4MXuU&continue=https%3A%2F%2Fmail.google.com%2Fmail%2F&service=mail&rm=false&ltmpl=default&scc=1&ss=1&_utf8=%E2%98%83&bgresponse=%21A0JLwawFPV34bUQ3xjl5OdgBcQIAAABfUgAAABcqAQSqazjYJpDg-kapblPmSujml011OygP0EUqjVds9Vk_fynd6-gmQ4WyRLVnd1EWIKp_M68OiYoQpy-BsmXpxQoIqbS7pIne_scYIkttMyj3BqWGjYqKEQBS0Ynb39G7n7gVBo_e406b1Ww7Ny9f3nouYPJbOG-kMRdGsuhzBAGwT9v-vMum2Z36_N8gThf12ZQ0gNa1hmEUALqwF0H5leXH7Ex7JhXtGppJ7SiuFjvJYgs0SO_L1ptI5o6eHgud_ti8178KC5KXi0WheHrl5kM2NK6Dn3HhH85-5FTD4P74_HKAbqgH72IeKOosril6qqWekPx_ChXOmSLr6itlnhZjdbEr7g&pstMsg=1&dnConn=&checkConnection=youtube%3A384%3A0&checkedDomains=youtube&Email=g%40assratbastard#gamil.com&$password&signIn=Sign+in&PersistentCookie=yes&rmShown=1