I am looking to call the Twitter API to grab tweets (successfully achieved tweets on load) but now I am looking to update the page automatically, allowing the tweets to automatically load without reload/user interaction.
I know this type of functionality is possible (monitter.com) does it, but what is the technology used to do so? Can I do it with PHP?
Thanks
As #suresh.g said, you can use AJAX. The simplest way: use jQuery.
Also, you can use an iframe that reloads every 10 seconds with the setInterval() javascript function. The user will not have a reload of his entire page, but the twitter iframe.
Another type of technology is COMET or PUSH technology, but I don't think you need it right now, but it's good to know about it ;)
use curl
function curl_grab_page($url,$data,$secure="false",$ref_url="",$login = "false",$proxy = "null",$proxystatus = "false")
{
if($login == 'true') {
$fp = fopen("cookie.txt", "w");
fclose($fp);
}
$ch = curl_init();
curl_setopt($ch, CURLOPT_COOKIEJAR, "cookie.txt");
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
curl_setopt($ch, CURLOPT_TIMEOUT, 60);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
if ($proxystatus == 'true') {
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, TRUE);
curl_setopt($ch, CURLOPT_PROXY, $proxy);
}
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
if($secure=='true')
{
curl_setopt($ch, CURLOPT_SSLVERSION,3);
}
curl_setopt( $ch, CURLOPT_HTTPHEADER, array( 'Expect:' ) );
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, $ref_url);
curl_setopt($ch, CURLOPT_HEADER, TRUE);
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_POST, TRUE);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
ob_start();
return curl_exec ($ch); // execute the curl command
curl_getinfo($ch);
ob_end_clean();
curl_close ($ch);
unset($ch);
}
just call this function the way you want the data you want to set it can do every thing for you just dont forget to setup curl in you php.ini
thanks
Related
I'm having trouble understanding how CURL handles headers.
I have a site.com/page1 that I want to access with CURL it does a 308 redirect to site.com/page2/file.zip
What I need is to go through site.com/page1 with CURL but download site.com/page2/file.zip directly from site.com
I'm using this code but it does not work as expected. It accesses site.com/page1 redirects to site.com/page2/file.zip but opens the file in the browser
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_COOKIEJAR, $_cookie_file);
curl_setopt($ch, CURLOPT_COOKIEFILE, $_cookie_file);
curl_setopt($ch, CURLOPT_REFERER, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_exec($ch);
$error = curl_getinfo($ch);
curl_close($ch);
I guess if I can keep the response headers I'll fix the problem. But how do I do it ?? How do I use the same headers for the CURL visitor that the site I am accessing is sending me.
You want the cURL option RETURNTRANSFER set to true so what is returned comes back to you. Since you are trying to save a ZIP file you'll also need to open a file and use the CURLOPT_FILE option to tell cURL where to save your ZIP file.
curl_setopt ($ch, CURLOPT_RETURNTRANSFER,1);
curl_setopt ($ch, CURLOPT_TIMEOUT,'180'); # 3 minute timeout
$FileOut = fopen('MyZIP_File.zip','w') or die('Could not open the output data file');
curl_setopt ($ch, CURLOPT_FILE,$FileOut);
curl_exec ($ch);
fclose($FileOut) or die('We ran into a problem saving data file');
This solved the problem.
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_COOKIEJAR, $_cookie_file);
curl_setopt($ch, CURLOPT_COOKIEFILE, $_cookie_file);
curl_setopt($ch, CURLOPT_REFERER, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_NOBODY, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_HEADER, true);
$result = curl_exec($ch);
if (preg_match('~Location: (.*)~i', $result, $match)) {
$location = trim($match[1]);
header('Location:' . $location);
}
I have a curl set up like this:
function file_get_contents_curl($url)
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_TIMEOUT, 400);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch,CURLOPT_BINARYTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
$data = curl_exec($ch);
return $data;
}
When I call it with the following:
$html = file_get_contents_curl("https://training.sap.com/service-id/module.php/core/as_login.php?AuthId=sf-sp&ReturnTo=%2Fsuccessfactors-community%2Fpermission-check");
You can see that link has an authentication, and I do not want to spend time on it. I just want that curl session to die after maybe 3 seconds or x seconds, or determine before hand if it is going to be a never ending loop on the curl for one reason such a SSL or password requirement etc.
This is a part of the code of a function of a class that I did in early 2015. Today it is no longer working. The variables defined in my code, so that is not the problem...
Now that we're clear, let's get to the point:
The Problems:
Do not generate cookies
In the login page , I get a red sign with "
security policy " ...
I get redirected to
https://facebook.com/common/invalid_request.php when I 'm logged (
below I explain as I did for logging )
Poorly made solutions :
To generate cookies , all you did was use an extension for Chrome that shows you cookies.txt called cookies in Netscape format and copy the file with the facebook and log in because I did not allow log in with cookies without Log .
I could not fix it
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://www.facebook.com/');
curl_setopt($ch, CURLOPT_POST, 0);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_COOKIEJAR, $this->CONFIG['cookie']);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_USERAGENT, $this->CONFIG['useragent']);
curl_setopt($ch, CURLOPT_REFERER, 'http://www.facebook.com');
curl_exec($ch) or die(curl_error($ch));
curl_close($ch);
unset($ch);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://www.facebook.com/login.php');
curl_setopt($ch, CURLOPT_POSTFIELDS,'email='.urlencode($email).'&pass='.urlencode($pass).'&login=Login');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_COOKIEJAR, $this->CONFIG['cookie']);
curl_setopt($ch, CURLOPT_COOKIEFILE, $this->CONFIG['cookie']);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_USERAGENT, $this->CONFIG['useragent']);
curl_setopt($ch, CURLOPT_REFERER, 'http://www.facebook.com');
$page = curl_exec($ch) or die(curl_error($ch));
file_put_contents(getcwd().'/debug.html', $page);
preg_match('/<body class="([^<]+)" dir="ltr">/', $page, $web);
if(file_exists($this->CONFIG['token'])) unlink($this->CONFIG['token']);
if(preg_match('/home/i', $web[1])) {
$this->login = true;
return true;
} else {
return false;
}
curl_close($ch);
You can´t login with CURL, you need to implement a proper login process and you NEVER need the password for this: https://developers.facebook.com/docs/facebook-login
I suggest using the JavaScript SDK for this, it´s very easy to handle. Here are some links to get you started:
https://developers.facebook.com/docs/facebook-login/web
http://www.devils-heaven.com/facebook-javascript-sdk-login/
I'm learning regex, php & cUrl and would like to get Google Image html
(for example : https://www.google.fr/search?q=terrorist&biw=1920&bih=1008&source=lnms&tbm=isch&gws_rd=ssl#tbm=isch&q=love)
I tried a lot of different answers but I don't understand why, surprisingly, when I do
<?php
function curl_get_contents($url)
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
curl_setopt($ch, CURLOPT_COOKIEJAR, "cookie.txt");
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($conn2, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($conn2, CURLOPT_SSL_VERIFYHOST, false);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
$get_page = curl_get_contents("https://www.google.fr/search?q=terrorist&biw=1920&bih=1008&source=lnms&tbm=isch&gws_rd=ssl#tbm=isch&q=love");
echo $get_page;
?>
I've got a very different result than in browser. All images links are dead for example.
Does anyone know why ? And what could I do to fix it?
Thanx a lot !!
There are 2 search queries in this request as shown below
https://www.google.fr/search?q=terrorist&biw=1920&bih=1008&source=lnms&tbm=isch&gws_rd=ssl#tbm=isch&q=love
try
https://www.google.fr/search?q=love&biw=1920&bih=1008&source=lnms&tbm=isch&gws_rd=ssl#tbm=isch
It looks like its returning the first query in the browser and then running the second, but it wont do the second request via curl.
This works for me:
<?php
function curl_get_contents($url)
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
curl_setopt($ch, CURLOPT_COOKIEJAR, "cookie.txt");
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
$data = curl_exec($ch);
var_dump(curl_error($ch));
curl_close($ch);
return $data;
}
$get_page = curl_get_contents("https://www.google.fr/search?q=love&biw=1920&bih=1008&source=lnms&tbm=isch&gws_rd=ssl#tbm=isch");
echo $get_page;
?>
EDIT: After further research this is an unsupported way of doing this, You should be using Google Custom Search API. The way that you are doing it will result in Google detecting misuse and presenting you with Captcha requests or maybe even blocking you.
I want to open a page from a site and save the cookies. Then open another page with those cookies.(It doesn't let me to visit the second page without the first page's cookies)
I tried this code but it didn't work.
$ch = curl_init();
curl_setopt($ch, CURLOPT_COOKIESESSION, 1);
curl_setopt($ch, CURLOPT_COOKIEFILE, 'cook.txt');
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cook.txt');
curl_setopt($ch, CURLOPT_URL, 'url');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
curl_exec($ch);
curl_setopt($ch, CURLOPT_COOKIESESSION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_URL, 'nexturl');
$rs = curl_exec($ch);
echo $rs;
End of your each curl operation close the curl handle. Otherwise it will not save the cookie into the file.
$rs = curl_exec($ch);
curl_close($ch);
Now initiate second curl, and send the request. It should work now.