I am using the following code
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $real_url);
curl_setopt($ch, CURLOPT_COOKIE, $cookie);
curl_setopt($ch, CURLOPT_NOBODY, true);
curl_setopt($ch, CURLOPT_COOKIEFILE, $fscookiefile);
curl_setopt($ch, CURLOPT_COOKIEJAR, $fscookiefile);
curl_setopt($ch, CURLOPT_HEADER, TRUE);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, false);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, FALSE);
curl_setopt($ch, CURLOPT_TIMEOUT, 60);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.13) Gecko/20101206 Ubuntu/10.04 (lucid) Firefox/3.6.13 GTB7.1');
$r = curl_exec($ch) or die('WHY DOES THIS DIE!!!');
curl_close($ch);
Which always results in WHY DOES THIS DIE!!! or the page keeps loading for sometime and then shows nothing if I don't use
or die('WHY DOES THIS DIE!!!')
If I set CURLOPT_NOBODY to false
I get
HTTP/1.1 200 OK Date: Thu, 09 Jun 2011 01:35:07 GMT Pragma: public Expires: 0 Cache-Control: must-revalidate, post-check=0, pre-check=0, private=false Content-Type: application/force-download Content-Transfer-Encoding: binary Keep-Alive: 1000 Content-Disposition: attachment; filename*= UTF-8''Ever17.part3.rar Accept-Ranges: bytes Content-Length: 520000000 Server: Jetty(6.1.16) Rar!zs sdt�NP�(�&R�%G��65,��Ever17\Ever_17_-_The_Out_of_Infinity_CD3.iso��"j����M�K�U:� >�b�2Q�sT?G��+iM���qN�'��&�-d�z�����} xi�<�3\�d��Z:|�/���#�gl��$��~#��u�וۊ \��jմg�)�L��#&��;��5M�s}���(�Đ$�yGwӏ$�E����$�Z���:�/�����r͐�8�&������#Q�00��>�)�n�̌}o�H����2Z���y�S㞝�H��=�9�[������E�nЭ��\�9Ǘ!{&y�j/���Ȇg������FW��~G�5������F-&����ƭ��9<���^7����(�HY(a����%�ɚQYL_���ܽ�X|��&
Why am I not getting any output with CURLOPT_NOBODY set to true ????
I am using PHP 5.1.6 on Apache/2.2.3 and have curl-7.15.5-9.el5_6.2.i386 installed
(That's a seriously outdated curl version)
Try using the command line version (preferably from the same host running that PHP script) like 'curl -I [URL]' and make sure the web site truly responds nice to a HEAD request.
Add -v or --trace-ascii for full debug/trace output.
Related
I try curl a website for show it in mine but always I stuck in a white page. I think because it have a redirection to a login form but I dont sure if is the reason because I failed. You can access the url that I use without login.
Here the code
$url = "http://www.faf.es/pnfg/NPcd/NFG_CmpJornada?cod_primaria=1000120&CodCompeticion=16867461&CodGrupo=17910021&CodTemporada=15&CodJornada=26&Sch_Codigo_Delegacion=1&Sch_Tipo_Juego=2";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_TIMEOUT, 99999);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
$output = curl_exec($ch);
curl_close($ch);
htmlentities($output);
The html result:
<html>
<head>
<title></title>
</head>
<body>
<!--
<h1>No se ha aceptado el cookie</h1>
-->
</body>
</html>
Your request url only validate requests which have JSESSIONID cookie. So before you must grab valid cookie:
<?php
$url = "http://www.faf.es/pnfg/NPcd/NFG_CmpJornada?cod_primaria=1000120&CodCompeticion=16867461&CodGrupo=17910021&CodTemporada=15&CodJornada=26&Sch_Codigo_Delegacion=1&Sch_Tipo_Juego=2";
$ch = curl_init();
$useragent = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36';
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_REFERER, 'http://www.google.com/');
curl_setopt($ch, CURLOPT_USERAGENT, $useragent);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_TIMEOUT, 99999);
$output = curl_exec($ch);
curl_close($ch);
echo $output;
This gives you three headers for requests:
HTTP/1.1 302 Movido temporalmente
Server: Apache-Coyote/1.1
Set-Cookie: JSESSIONID=648967C5FC907B1225EC61E9A65443E5; Path=/pnfg
Location: http://www.faf.es/pnfg/NLogin
Content-Length: 0 Date: Mon, 22 Jun 2020 19:33:30 GMT
Connection: close
HTTP/1.1 302 Movido temporalmente
Server: Apache-Coyote/1.1
Cache-Control: no-cache
Pragma: no-cache
Set-Cookie: JSESSIONID=9973308FA3AF81BE66C2F8C124671870; Path=/pnfg
Location: http://www.faf.es/pnfg/NLogin?NSess=1
Content-Length: 0 Date: Mon, 22 Jun 2020 19:33:30 GMT
Connection: close
HTTP/1.1 200 OK Server: Apache-Coyote/1.1
Cache-Control: no-cache
Pragma: no-cache
Content-Type: text/html;charset=ISO-8859-15
Content-Length: 117
Date: Mon, 22 Jun 2020 19:33:30 GMT
Connection: close
We have three header result because we said, request has CURLOPT_FOLLOWLOCATION option. So, we made three request in curl.
I identified, second JSESSIONID cookie value is valid. So, for getting web site information we must use cookie as below:
<?php
$url = "http://www.faf.es/pnfg/NPcd/NFG_CmpJornada?cod_primaria=1000120&CodCompeticion=16867461&CodGrupo=17910021&CodTemporada=15&CodJornada=26&Sch_Codigo_Delegacion=1&Sch_Tipo_Juego=2";
$ch = curl_init();
$useragent = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36';
curl_setopt($ch, CURLOPT_URL, $url);
// Secondary request cookie.
curl_setopt($ch, CURLOPT_COOKIE, "JSESSIONID=9973308FA3AF81BE66C2F8C124671870");
curl_setopt($ch, CURLOPT_REFERER, 'http://www.google.com/');
curl_setopt($ch, CURLOPT_USERAGENT, $useragent);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_TIMEOUT, 99999);
$output = curl_exec($ch);
curl_close($ch);
echo $output;
When web site content comes, do whatever you want.
You should revisit the htmlentities() documentation again; it doesn't do what it appears you believe it does.
In short, htmlentities() returns a string. You aren't capturing or otherwise outputting this string result anywhere. Use echo (or similar) to make sure the result of htmlentities($output) is output to where you expect it.
$url = "http://www.faf.es/pnfg/NPcd/NFG_CmpJornada?cod_primaria=1000120&CodCompeticion=16867461&CodGrupo=17910021&CodTemporada=15&CodJornada=26&Sch_Codigo_Delegacion=1&Sch_Tipo_Juego=2";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_TIMEOUT, 99999);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
$output = curl_exec($ch);
curl_close($ch);
echo htmlentities($output);
I created a script for send the xml request using POST,It works perfectly on linux command line but not in php curl and this is my xml file and the command which I used.
<?xml version="1.0"?>
<methodCall>
<methodName>test.echo</methodName>
<params>
<param><value><string>nbb</string></value></param>
<param><value><string>health check: 1436777963</string></value></param>
</params></methodCall>
curl -H "Content-Type: text/xml" -d #test.xml -X POST http:my.host:5862 -v
This is my php script,
$url = 'http://my.host:5862';
#$requested = xml_gen($function_name,$epos,$time);
$requested = '<?xml version="1.0"?><methodCall><methodName>test.echo</methodName><params><param><value><string>nbb</string></value></param><param><value><string>health check: 1436777963</string></value></param></params></methodCall>';
#echo($requested);
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_ENCODING ,"ISO-8859-1");
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'POST');
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: text/xml'));
curl_setopt($ch, CURLOPT_POSTFIELDS, $requested);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
echo $response = curl_exec($ch);
curl_close($ch);
//Decoding the response to be displayed
echo xmlrpc_decode($response);
This is the error I get with PHP's cURL.
HTTP/1.1 400 Bad Request
Server: nginx
Date: Mon, 13 Jul 2015 15:26:05 GMT
Content-Type: text/XML
Content-Length: 0
Connection: keep-alive
Keep-Alive: timeout=20
I found the solution, Have to add the user agent on curl request.
curl_setopt($ch,CURLOPT_USERAGENT,'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13');
I used below function to get redirected (final) url of these links
http://iprice.my/coupons/zalora/
function curlFileGetContents($url)
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; rv:16.0) Gecko/20100101 Firefox/16.0');
$result = curl_exec($ch);
$info = curl_getinfo($ch);
$effectiveUrl = $info['url'];
curl_close($ch);
return $effectiveUrl;
}
but I couldn't get anything, I wonder why? Eg I did curlFileGetContents('http://iprice.my/coupons/zalora/#007b9a6024d19edec91d04c2e92e143e744c83b6');
The URL isn't being redirected - the links you are referring to (that only spawn on that page) are pop unders. A look at the curl header shows the URL is not being re-directed:
curl --head http://iprice.my/coupons/zalora/#007b9a6024d19edec91d04c2e92e143e744c83b6
HTTP/1.1 200 OK
Server: nginx/1.4.6 (Ubuntu)
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
Vary: Accept-Encoding
X-Powered-By: PHP/5.5.9-1ubuntu4.9
Cache-Control: no-cache
Date: Fri, 22 May 2015 15:06:31 GMT
X-iPrice-Cached: 1
X-iPrice-Cached-Type: Response
The approach you currently have won't work for what you want to do.
This is an old script that is no longer working that logins to Paypal, I had a working version but deleted it by accident now I'm trying to get an old back up working.
Initialize curl:
$ch = curl_init();
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_POST, 0);
curl_setopt($ch, CURLOPT_AUTOREFERER, 1);
curl_setopt($ch, CURLOPT_REFERER, '');
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.10) Gecko/2009042523 Ubuntu/9.04 (jaunty) Firefox/3.0.10');
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_COOKIESESSION, true);
curl_setopt($ch, CURLOPT_COOKIEFILE, PAYPAL_COOKIE_FILE);
curl_setopt($ch, CURLOPT_COOKIEJAR, PAYPAL_COOKIE_FILE);
curl_setopt($ch, CURLINFO_HEADER_OUT, true);
cp_post_page function
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $query_string);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
ob_start();
curl_exec($ch);
$response = ob_get_contents();
ob_end_clean();
return $response;
Rest of the script:
$response = cp_get_page($ch, 'https://www.paypal.com/ca/cgi-bin/webscr?cmd=_ship-now');
$query_string = "CONTEXT=" . $form_context. "&login_cmd=&login_params=&login_email=" . PAYPAL_EMAIL . "&login_password=" . PAYPAL_PASSWORD
. "&submit.x=Log%20In&form_charset=UTF-8&auth=$form_auth"
. "&browser_name=Firefox&browser_version=3&browser_version_full=3&operating_system=Linux";
$response = cp_post_page($ch, $form_action, $query_string); <--- Fails here
Response from the curl:
HTTP/1.1 200 OK Server: Apache Strict-Transport-Security: max-age=14400 Strict-Transport-Security: max-age=14400 Content-Type: text/html DC: slc-a-origin-www-1.paypal.com Date: Mon, 04 Nov 2013 23:45:19 GMT Content-Length: 54 Connection: keep-alive Set-Cookie: X-PP-SILOVER=name%3DLIVE5.WEB.1%26silo_version%3D880%26app%3Dslingshot%26TIME%3D2402383954; domain=.paypal.com; path=/; Secure; HttpOnly Set-Cookie: X-PP-SILOVER=; Expires=Thu, 01 Jan 1970 00:00:01 GMT
Fatal Failure
Headers being sent:
POST /ca/cgi-bin/webscr?cmd=_flow&SESSION=[removed]&dispatch=[removed] HTTP/1.1
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.10) Gecko/2009042523 Ubuntu/9.04 (jaunty) Firefox/3.0.10
Host: www.paypal.com
Accept: */*
Referer: https://www.paypal.com/ca/cgi-bin/webscr?cmd=_ship-now
Cookie: aksession=1383609018~id=cookieHTdyV99GKjx4ataOFl8HX+fgn4AgJSYxaLCcm+N/2KWZPsBAQbqDZ0ek9tQy9J9/gwTMcvHTSYGX65BNgg10oVSLJurTnMsIlySSW7uFaZjrVKxpCVApCbxyp5lfygq/QA1GvRvOk0k=; DC=slc-a-origin-www-1.paypal.com; X-PP-SILOVER=name%3DLIVE5.WEB.1%26silo_version%3D880%26app%3Dslingshot%26TIME%3D2385606738; Apache=10.73.8.47.1383608717943546; navlns=0.0; pNTcMTtQfrJuaJiwEnWXQ6yNxfq=cRcI945EJH9tChcYgpHw1EwU36Z-qza0kxriR6IYWTNRRqPyItIzb1qCgt5K-W3DjQPwjI8yCfYaNInqtDcheZgtxQX9L7xLZM8pY7bKHS_XWsWt759waXfATBCGKYeusuJuPFdeRH2_qHRlS6s31k4inXdD-TZnRI8OEdaArFLEBx3t4-5d4NV5aeqdVSL8TuDf-kqWJFvs4Xzs2wdBEmpoocMLXGm_igzYEYHmP9KqDIUaXAMPiZUeMmPfAJiBxC8-EN5zJqI7dqs3-BgIPpCi5Is5IQe_84xDMHVBIAAgDgSUByR3-FkmBtPlfDB6rLoItmY0kT9L7yUZFW48kP3yNWHhWQ1o-InAmm; navcmd=_ship-now; cookie_check=yes; KHcl0EuY7AKSMgfvHl7J5E7hPtK=kCH9bvOH2hK9miohP-LJoRVMGNwgry4awBca8g8fKl4vOhFrS5fM82dAIUPhBGuKkYwgeLhgsPV2tVS-; cwrClyrK4LoCV1fydGbAxiNL6iG=yGLUV2f-3wPoFZV-vMEGW6jlZr7im-N5EYbO1KWk2loskGrFWziDsIkn0xLI8kQo5MScMg9TWryYuevj3SLa07p2m_IyjtTAa2W_iF4rbbPYPPsEdbypbjQjxWgd3RZw9IzCAaPJ3ZLS1R1-kJWYRevJBbZaqTApiIlRA8ALAZlOJ1g9ft_FL2GsOERgVaKpjz_aZcVeaKInfcPRHoGc9EMQTz9bFsIardyUhnQxw4Zu19vefkGYYk-1NCtLctqJ1jQ5HVn-3d7clgyddNul7JockOlurWRgPjbfkDjQ7-eXuleFhb9LkfUgpnQXPyvYTUmWh5QYnOEr9q_cRNFCRHs4vIvmhvaziv8Eyg7gFmg8v6--xOQy-a-gBqj7JEgd0kOKOWIRbneb1mm1Icd_o3lkuISss1xKvcIzXrx2scz3fq6Ys8z1VNWDNBK
Content-Length: 376
Content-Type: application/x-www-form-urlencoded
I verified that that the query string contains the right data, it is not urlencoded because curl will encode it from my understanding. I compared the POST data to a form submit using Chrome and the same number of fields are being sent to Paypal.
From what I am seeing the correct post data is being sent, the cookie exists but the login fails because it is missing something (I think it is a cookie issue, it is not a javascript issue as I login with javascript disabled). I'm not sure what other trouble shooting steps I can do to help pin point the issue.
I've solved the issue, given that one input is shown below:
<input type="hidden" id="CONTEXT_CGI_VAR" name="CONTEXT" value="REMOVED">
Even after analyzing a sample response, the reference to CONTEXT should be changed to CONTEXT_CGI_VAR in order for the curl to work correctly. Why? No idea.
I am trying to login into Tesco Mobile, using PHP curl, I have the following code:
$postData='user=XXX&password=XXXXX';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://paymonthly.tescomobile.com/orderentry/LoginSubmit.do');
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, TRUE);
curl_setopt ($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6");
curl_setopt ($ch, CURLOPT_TIMEOUT, 60);
curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_COOKIEJAR, "cookies.txt");
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookies.txt");
curl_setopt($ch, CURLOPT_POST, TRUE);
curl_setopt($ch, CURLOPT_REFERER, 'https://paymonthly.tescomobile.com/orderentry/LoginSubmit.do');
curl_setopt ($ch, CURLOPT_POSTFIELDS, $postdata);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
$result = curl_exec ($ch);
curl_close($ch);
But have been unsuccessful, please can anyone help how I can achieve this. Please find the output from Live HTTP below:
Thanks in advance.
https://paymonthly.tescomobile.com/orderentry/LoginSubmit.do
POST /orderentry/LoginSubmit.do HTTP/1.1 Host:
paymonthly.tescomobile.com User-Agent: Mozilla/5.0 (Macintosh; Intel
Mac OS X 10.7; rv:18.0) Gecko/20100101 Firefox/18.0 Accept:
text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate
Referer:
https://paymonthly.tescomobile.com/orderentry/LoginSubmit.do;jsessionid=60af415cd448d5728a9fa98cdf99
Cookie: JSESSIONID=60af415cd448d5728a9fa98cdf99;
reevoomark_marker=349049566;
reevoo_utma=1.324660794.1360800582.1360800582.1360800582.1;
reevoo_utmb=1.5.6.1360800582; reevoo_utmc=1;
reevoo_utmz=1.1360800582.1.1.utmgclid=CK6Vp5nBtLUCFeXKtAodVGIAVA|utmccn=(not%20set)|utmcmd=(not%20set)|utmctr=tescomobile;
reevoomark_bh_session=0af0a16fc00f5fbc8b0d28c571721a72;
CUSTOMER_UUID=04fdc3e6-59e1-4038-81b7-ec7c5d63c9d4;
__utma=152930744.490151867.1360800710.1360800710.1360800710.1; __utmb=152930744; __utmc=152930744; __utmz=152930744.1360800710.1.1.utmccn=(referral)|utmcsr=phone-shop.tesco.com|utmcct=/tesco-mobile/my-tesco-mobile/|utmcmd=referral
Connection: keep-alive Content-Type: application/x-www-form-urlencoded
Content-Length: 32 user=dinoc83&password=c079378643 HTTP/1.1 200 OK
X-Powered-By: JSP/2.1 Server: Sun GlassFish Enterprise Server v2.1.1
Content-Type: text/html;charset=UTF-8 Transfer-Encoding: chunked Date:
Thu, 14 Feb 2013 00:15:27 GMT
Try to change :
$postData='user=XXX&password=XXXXX';
by $postData=array('user=XXX&password=XXXXX');
There are some code samples in : http://php.net/manual/en/function.curl-setopt.php
Regards,
Just checked form
Form attribute action have parameter jsessionid In your code you don't use it. I think you should in first load page with form, then parse response code and get correct URL for post. And this need for initialize cookies.
Try to this in this way it's working for us posting data in another server
$postData='user=XXX&password=XXXXX'
$url ="https://paymonthly.tescomobile.com/orderentry/LoginSubmit.do";
$ch = curl_init() or die(curl_error());
curl_setopt($ch, CURLOPT_POST,1);
curl_setopt($ch, CURLOPT_POSTFIELDS,$postData);
curl_setopt($ch, CURLOPT_URL,$url);// here the request is sent to payment gateway
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$result=curl_exec($ch) or die(curl_error());
curl_close($ch);
You can also set $postData value this way:
$postData = array( 'user'=>'XXX', 'password'=>'XXXX' );
And note what:
Passing an array to CURLOPT_POSTFIELDS will encode the data as multipart/form-data, while passing a URL-encoded string will encode the data as application/x-www-form-urlencoded.