What I have in place, is a domain availability check, which connects up to an API and outputs "Available: and Unavailable:" from $tmp. Ths below code will only check the availability ONCE.
I would like to check the availability of the domain, multiple times (possibly on a loop?), without having to run restart cURL connection everytime (as it wastes time - 300ms to 1s per query).
I just don't know how I can connect to cURL once and run the loop (doing the check through the API). Help adjusting the code would be very much appreciated! Minimizing the time it takes to output "available/not available" and looping the checks is key.
Thank you.
Current code
<?php
function GetCurlPage ($pageSpec)
{
$ch = curl_init($pageSpec);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, FALSE);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
$tmp = curl_exec ($ch);
curl_close ($ch);
$tmp = preg_replace('/(?s)<meta http-equiv="Expires"[^>]*>/i', '', $tmp);
$tmp = explode('<br>', $tmp);
echo $tmp[0];
echo "<br>";
echo $tmp[1];
echo "<br>";
return $tmp;
}
$returnUrl = "http://www.mysite.com.au/check.php";
$url = "https://www.apisite.com.au/availability/check.php?domain=testdomain&suffixes=.com.au";
$output = GetCurlPage("$url");
?>
#Marc B
function getCurlPage($pageSpec) {
if (is_null($ch)) {
$ch = curl_init($pageSpec);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, FALSE);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
} else {
curl_setopt($ch, CURLOPT_URL, $pageSpec);
}
while ($i < 5) {
$tmp = curl_exec ($ch);
//curl_close ($ch);
$tmp = preg_replace('/(?s)<meta http-equiv="Expires"[^>]*>/i', '', $tmp);
$tmp = explode('<br>', $tmp);
echo $tmp[0];
echo "<br>";
echo $tmp[1];
echo "<br>";
echo udate('H:i:s:u');
echo "<br><br>";
$i++;
}
return $tmp;
}
This should answer your question: Persistent/keepalive HTTP with the PHP Curl library?
comment followup:
function getCurlPage($pageSpec) {
if (is_null($ch))
static $ch = curl_init($pageSpec);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, FALSE);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
} else {
curl_setopt($ch, CURLOPT_URL, $pageSpec);
}
$tmp = curl_exec($ch);
... do NOT close the curl handle, otherwise do the rest the same as before ...
}
Probably won't work as is, doing this off the top of my head and with only 2 hours sleep, but this should be enough to get you started.
And by the way, there's no need to do doublequotes for GetCurlPage("$url"), it's a waste of parser time, as PHP will have to create a new empty string, stuff $url into it, and pass the new string on down. Just do GetCurlPage($url).
Related
in my code, I am making curl call and after getting response I am calling current file if proper response does not get. like this,
<?php
include 'fauxapi_client.php';
$obj = new Fauxapi_client;
$url=$obj->base_url.'/Fauxapi_client/device_version_update';
$version = trim(file_get_contents('/usr/local/www/version.txt'));
$iso_version = trim(file_get_contents('/usr/local/www/iso_version.txt'));
$temp = array('device_ip'=>$obj->current_device_ip,'version'=>$version,'iso_version'=>$iso_version);
$temp = http_build_query($temp);
$headers[] = 'Content-Type: application/x-www-form-urlencoded; charset=utf-8';
$ch = curl_init();
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT ,0);
curl_setopt($ch, CURLOPT_TIMEOUT, 40);
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $temp);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER,$headers);
$output = curl_exec ($ch);
curl_close ($ch);
if ($output === false) {
echo 'Curl Call Fail';
}
else{
$response = json_decode($output,true);
if (isset($response['data']['next_version'])) {
file_put_contents('/usr/local/www/version.txt', trim($response['data']['next_version']));
if (floatval($response['data']['max_updatable_version']) > floatval($response['data']['next_version'])) {
shell_exec("/usr/local/bin/php /usr/local/www/version_update.php");
}
}
else{
echo 'next_version is not in response ';
}
}
var_dump($output);
?>
in the above code
shell_exec("/usr/local/bin/php /usr/local/www/version_update.php");
is a current file call.
when I echo something there it will echo but not calling a current file. so, how to call current file there?
Are you sure the extra space after php? bin/php /
And if /usr/local/bin/php /usr/local/www/version_update.php is current file you can use __FILE__ instead it.
And please make sure sufficient permissions.
I am trying to automate the configuration of x IP cameras from their embbeded web server (Self Signed Certificates). So if you try to connect to a camera through a browser in a normal way (no script), you'll have to add an exception, works fine.
I want to automate this, and all my scripts PHP are ran in a Powershell CLI.
I have the following PHP script :
<?php
include('C:\wamp64\bin\php\php7.0.10\run\Librairie\LIB_parse.php');
include('C:\wamp64\bin\php\php7.0.10\run\Librairie\LIB_http.php');
include('C:\wamp64\bin\php\php7.0.10\run\Librairie\LIB_resolve_addresses.php');
$TableauIP = fopen('C:\wamp64\bin\php\php7.0.10\run\x\Ipcamera.txt', 'r');
$count = 0;
while (($URLcamera = fgets($TableauIP, 4096)) !== false){
$IP_unparsed = $URLcamera;
$Ipcamera = return_between($IP_unparsed, "//", "/", EXCL);
echo("Automatic configuration for : ".$Ipcamera."\n");
echo("...............\n\n");
echo("Downloading page : ".$IP_unparsed."\n\n");
$web_page =http_get($IP_unparsed, $ref = "");
echo "ERROR \n";
var_dump($web_page['ERROR']);
$head_section = return_between($string=$web_page['FILE'], $start="<head>", $end="</head>", $type=EXCL);
$meta_tag_array = parse_array($head_section, $beg_tag="<meta", $close_tag=">");
for($xx=0; $xx<count($meta_tag_array); $xx++){
echo $meta_tag_array[$xx]."\n";
}
for($xx=0; $xx<count($meta_tag_array); $xx++){
$meta_attribute = get_attribute($meta_tag_array[$xx], $attribute="http-equiv");
if(strtolower($meta_attribute)=="refresh"){
$new_page = return_between($meta_tag_array[$xx], $start="URL", $end=">", $type=EXCL);
$new_page = trim(str_replace("", "", $new_page));
$new_page = str_replace("=", "", $new_page);
$new_page = str_replace("\"", "", $new_page);
$new_page = resolve_address($new_page, $IP_unparsed);
}
break;
}
echo "HTML Head redirection detected<br>\n\n";
echo "Redirect page = ".$new_page."\n";
$web_page2 = http_get($new_page, $ref = "");
//$web_page = http_get($IP_unparsed.'/login.cs', $ref = "");
echo "FILE CONTENT \n";
var_dump($web_page2['FILE']);
echo "FILE ERROR \n";
var_dump($web_page2['ERROR']);
// for($xx=0; $xx<count($web_page); $xx++){
// echo($web_page[$xx]);
// }
// echo "ERROR \n";
// var_dump($new_page['ERROR']);
//*******************************
// $web_page = file($new_page);
// for($xx = 0; $xx < count($web_page); $xx++)
// echo $web_page[$xx];
//********************************
// $file_handle = fopen($new_page, "r");
// while (!feof($file_handle))
// {
// echo fgets($file_handle, 4096);
// }
// fclose($file_handle);
$count++;
}
?>
(I left the comments, I've tried different way to display the webpage)
As you can see, I am using the engine WampServer_x64 on a basic Windows 7.
I'm following a redirection to the https://x.x.x.x/login.cs page.
The important part is the download of webpage2.
Here the LIB_parse library (just necessary lines), wrapping curl options in PHP functions :
function http_get($target, $ref)
{
return http($target, $ref, $method="GET", $data_array="", EXCL_HEAD);
}
function http($target, $ref, $method, $data_array, $incl_head)
{
# Initialize PHP/CURL handle
$ch = curl_init();
# Prcess data, if presented
if(is_array($data_array))
{
# Convert data array into a query string (ie animal=dog&sport=baseball)
foreach ($data_array as $key => $value)
{
if(strlen(trim($value))>0)
$temp_string[] = $key . "=" . urlencode($value);
else
$temp_string[] = $key;
}
$query_string = join('&', $temp_string);
}
# HEAD method configuration
if($method == HEAD)
{
curl_setopt($ch, CURLOPT_HEADER, TRUE); // No http head
curl_setopt($ch, CURLOPT_NOBODY, TRUE); // Return body
}
else
{
# GET method configuration
if($method == GET)
{
if(isset($query_string))
$target = $target . "?" . $query_string;
curl_setopt ($ch, CURLOPT_HTTPGET, TRUE);
curl_setopt ($ch, CURLOPT_POST, FALSE);
}
# POST method configuration
if($method == POST)
{
if(isset($query_string))
curl_setopt ($ch, CURLOPT_POSTFIELDS, $query_string);
curl_setopt ($ch, CURLOPT_POST, TRUE);
curl_setopt ($ch, CURLOPT_HTTPGET, FALSE);
}
curl_setopt($ch, CURLOPT_HEADER, $incl_head); // Include head as needed
curl_setopt($ch, CURLOPT_NOBODY, FALSE); // Return body
}
curl_setopt($ch, CURLOPT_COOKIEJAR, COOKIE_FILE); // Cookie management.
curl_setopt($ch, CURLOPT_COOKIEFILE, COOKIE_FILE);
curl_setopt($ch, CURLOPT_TIMEOUT, CURL_TIMEOUT); // Timeout
curl_setopt($ch, CURLOPT_USERAGENT, WEBBOT_NAME); // Webbot name
curl_setopt($ch, CURLOPT_URL, $target); // Target site
curl_setopt($ch, CURLOPT_REFERER, $ref); // Referer value
curl_setopt($ch, CURLOPT_VERBOSE, FALSE); // Minimize logs
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, FALSE);
curl_setopt($ch, CURLOPT_SSLVERSION, CURL_SSLVERSION_TLSv1_2);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE); // No certificate
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE); // Follow redirects
curl_setopt($ch, CURLOPT_MAXREDIRS, 4); // Limit redirections to four
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE); // Return in string
# Create return array
$return_array['FILE'] = curl_exec($ch);
$return_array['STATUS'] = curl_getinfo($ch);
$return_array['ERROR'] = curl_error($ch);
# Close PHP/CURL handle
curl_close($ch);
# Return results
return $return_array;
}
I do not know how to handle the TLS connection with cURL. I've been trying for hours with different stuff .. I have this issue : encrypted alert :
whireshark capture TCP and TLS exchange
I've add this line to the original library :
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, FALSE);
//curl_setopt($ch, CURLOPT_SSLVERSION, 6);
curl_setopt($ch, CURLOPT_SSLVERSION, CURL_SSLVERSION_TLSv1_2);
I can't get the web page.
Apparently, the SSL version is 1.0.2h.
I have tried many different things .. With many different error types, but always around the SSL certificate stuff..
I have no more ideas where to look..
If you guys can give me a trail ! That would be nice
I am developing a small script and I am bit messed up with using a couple of curl and while loop.
I want to stop processing curl at a point when one of the URL is giving me a information. Note: I have multiple curl requests.
So my concept is,
I have a couple of URLS, which I have to process and get information from. If information is found on a particular URL, it will be giving me a string. If no information is found, it will give me no value. So I have nearly 10 URLs to process approximately. In all cases, any one of the URL will be giving me information, so the remaining urls will be producing no value. Since processing there much URLS, latency is a issue. So suppose in the sample code below, if the url ends with value2.php gives me a result, then I immediately wanted to stop processing the other URLs. Because I already got the result and no point in running other curl. Then finally I have to print the result.
Also I have a condition where none of the URL produce any result and it will be great if someone shows me how to handle that also.
My sample code.
<?php
///functions here
do {
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"example[dot]com/value1.php?process=$param");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$combined = curl_exec ($ch);
curl_close ($ch);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"example[dot]com/value2.php?process=$param");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$combined = curl_exec ($ch);
curl_close ($ch);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"example[dot]com/value3.php?process=$param");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$combined = curl_exec ($ch);
curl_close ($ch);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"example[dot]com/value4.php?process=$param");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$combined = curl_exec ($ch);
curl_close ($ch);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"example[dot]com/value5.php?process=$param");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$combined = curl_exec ($ch);
curl_close ($ch);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"example[dot]com/value6.php?process=$param");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$combined = curl_exec ($ch);
curl_close ($ch);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"example[dot]com/value7.php?process=$param");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$combined = curl_exec ($ch);
curl_close ($ch);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"example[dot]com/value8.php?process=$param");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$combined = curl_exec ($ch);
curl_close ($ch);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"example[dot]com/value9.php?process=$param");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$combined = curl_exec ($ch);
curl_close ($ch);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"example[dot]com/value10.php?process=$param");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$combined = curl_exec ($ch);
curl_close ($ch);
} while (strlen($combined) != 0);
echo $combied;
///functions here
?>
Try the following:
<?php
function callCURL($url) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$combined = curl_exec ($ch);
curl_close ($ch);
return $combined;
}
function getResult($urls) {
$return = array();
foreach ($urls as $url) {
$response = callCURL($url);
if (strlen($response) !== 0) {
$return[] = $response;
break;
}
}
return $return;
}
$urls = array("example.com/value1.php?process=$param", "example.com/value2.php?process=$param", "example.com/value3.php?process=$param")
$result = getResult($urls);
Like you have it in your question and how the other answers have it, the problem is that for each request you have to send it, wait for a response, process the data and only then do you make your subsequent requests. This works fine, but it is highly time-inefficient. Say, for example, each request takes 100 ms to make (which probably not unrealistic). For 10 requests you're looking at 1 second of load time. Instead I would recommend forgetting about trying to stop making requests after you find your result and send all the requests... simultaneously. This can be accomplished with PHP's curl_multi_* functions.
// Put all of your URLs in here. I'm just using google for
// all as an example:
$urls[] = 'http://www.google.com';
$urls[] = 'http://www.google.com';
$urls[] = 'http://www.google.com';
$urls[] = 'http://www.google.com';
$urls[] = 'http://www.google.com';
$urls[] = 'http://www.google.com';
// Get cURL handles
foreach ($urls as $key => $url) {
$chs[$key] = curl_init();
// Set all your options for each connection here
curl_setopt($chs[$key], CURLOPT_URL, $url);
curl_setopt($chs[$key], CURLOPT_HEADER, 0);
}
//create the multiple cURL handle
$mh = curl_multi_init();
//add the handles
foreach ($chs as &$ch) {
curl_multi_add_handle($mh,$ch);
}
$active = null;
//execute the handles
do {
$mrc = curl_multi_exec($mh, $active);
} while ($mrc == CURLM_CALL_MULTI_PERFORM);
while ($active && $mrc == CURLM_OK) {
if (curl_multi_select($mh) != -1) {
do {
$mrc = curl_multi_exec($mh, $active);
} while ($mrc == CURLM_CALL_MULTI_PERFORM);
}
}
foreach ($chs as $url=>&$ch) {
$html = curl_multi_getcontent($ch);
// [do what you want with the HTML]
curl_multi_remove_handle($mh, $ch); // remove the handle (assuming you are done with it);
}
curl_multi_close($mh);
You can use something like this. I didn't check the code, after debug it should work.
function callcurl($url)
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"example[dot]com/value5.php?process=$param");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$ret = curl_exec ($ch);
curl_close ($ch);
}
$urls = array('url1', 'url2'); /// etc
$combined = ''
$cnt = 0;
do {
$combined = callcurl($urls[$cnt++]);
} while (strlen($combined) != 0 && $cnt < count($urls)); //
print $combined;
You can achieve this using a for loop.
$fileNames = array(
'0' => 'valueabc',
'1' => 'valuedef',
[...] => [...]
);
$combined = array();
for ($i = 1; $i < 10; $i++)
{
$ch = curl_init();
$url = "example[dot]com/" . $fileNames[$i] . ".php?process=" . $param;
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$combined[$i] = curl_exec ($ch);
curl_close ($ch);
}
To see all the responses you can foreach $combined.
foreach ($combined as $value=>$response)
{
// TODO: Work with the response
echo "[" . $value . "]" . $response;
}
I'm trying to use the SPtrans API (http://www.sptrans.com.br/desenvolvedores/APIOlhoVivo.aspx), which is supposed to provide public transport information for the Sao Paulo (Brazil) area.
I'm trying to get in using PHP and curl.
I'm able to put in the requests and can authenticate myself (with a post request to /Login/Autenticar?token={token}. The post request returns a 'true' (and only a 'true').
(It seems that I need to put the token both as a GET and a POST.)
However, if I then put in an information (GET) request, for example to /Linha/Buscar?termosBusca={termosBusca}, I get a consistent return of "Authorization has been denied for this request." message.
You can see this (not) working at:
http://00qq.com/sptrans/index.php
Any thoughts or ideas on this would be extremely helpful.
Here's the code that picks up the data:
function getResult($accesspoint, $page, $postData, $post = true) {
$ch = curl_init();
$t = http_build_query($postData);
$url = $accesspoint.$page."?".$t;
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
if ($post == true) {
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postData);
}
curl_setopt($ch, CURLOPT_HEADER, 0);
$output = curl_exec($ch);
curl_close($ch);
$output = object_to_array(json_decode($output));
return $output;
}
Update
#chesterbr put me on the right track: I had to create, collect and store cookies upon authentication and then use those upon subsequent requests. Below is a proof of concept.
$site["sptrans"]["accesspoint"] = "http://api.olhovivo.sptrans.com.br/v0";
$site["sptrans"]["page"]["Login"] = "/Login/Autenticar";
$site["sptrans"]["page"]["Parada"] = "/Parada/Buscar";
$site["sptrans"]["page"]["Linha"] = "/Linha/Buscar";
$site["sptrans"]["token"] = ""; //This should contain your token.
error_reporting(E_ALL);
ini_set('display_errors', 1);
function object_to_array($data) {
if (is_array($data) || is_object($data)) {
$result = array();
foreach ($data as $key => $value)
$result[$key] = object_to_array($value);
return $result;
}
return $data;
}
function getResult($accesspoint, $page, $postData, $cookie, $post = true) {
$ch = curl_init();
$t = http_build_query($postData);
$url = $accesspoint.$page."?".$t;
// print $url."<br />";
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($ch, CURLOPT_COOKIEJAR, $cookie);
curl_setopt ($ch, CURLOPT_COOKIEFILE, $cookie);
// curl_setopt($ch, CURLOPT_COOKIESESSION, true);
if ($post == true) {
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postData);
}
curl_setopt($ch, CURLOPT_HEADER, 0);
$output = curl_exec($ch);
curl_close($ch);
$output = object_to_array(json_decode($output));
return $output;
}
//Create a cookie for the duration of the page.
$ckfile = tempnam ("cache/cookies", "spt.");
print "Authentication<br />";
$postData["token"] = $site["sptrans"]["token"];
$output = getResult($site["sptrans"]["accesspoint"], $site["sptrans"]["page"]["Login"], $postData, $ckfile);
print_r ($output);
unset($postData);
print "<hr />";
print "Linha<br />";
$postData["termosBusca"] = "8000";
$output = getResult($site["sptrans"]["accesspoint"], $site["sptrans"]["page"]["Linha"], $postData, $ckfile, false);
print_r ($output);
unset($postData);
print "<hr />";
print "Parada<br />";
$postData["termosBusca"] = "Afonso";
$output = getResult($site["sptrans"]["accesspoint"], $site["sptrans"]["page"]["Parada"], $postData, $ckfile, false);
print_r ($output);
unset($postData);
print "<hr />";
//Delete the cookie
unlink($ckfile);
You can see this work at http://00qq.com/sptrans/index.php
The API requires you to store the cookies from the authentication call and include them in subsequent ones (otherwise, the server can't know those calls belong to the same session, since HTTP is stateless by default).
You can make that in PHP by configuring the cURL library as described in: https://stackoverflow.com/a/12885587. See also http://www.php.net/manual/en/function.curl-setopt.php for more information on such options (search for "COOKIE" options).
I surely confirm the result of curl_exec will be cleaned up by curl_close.
I have to comment out the curl_close line to get the result.My php version is 5.3.8.
How do I get result with curl_close?
Here is my code
function curl_get_contents($url)
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$r = curl_exec($ch);
//curl_close($ch);
return $r;
}
It has no effect on the return value, as long as the data from curl_exec(); is stored in $r you can return as you like.
This works normally.
function curl_get_contents($url){
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$r = curl_exec($ch);
curl_close($ch);
return $r;
}
$returnedValue = curl_get_contents($url); //Holds the contents
Edit as Marc B pointed out :
You don't need to do a curl close. PHP will clean up for you when the
function returns and $ch goes out-of-scope.
Hence there's no point of even closing it, but it shouldn't happen.