Retry function when string not found - php

I have a function that scrapes data using a list of proxies with curl. It selects a random proxy each time the function is called. However sometimes a proxy can fail or timeout.
When the connection fails/timeout I would like to repeat the function up to 3 times until the data is returned.
The way I would like to test if the connection is bad is by checking if a string exists in the output like this:
$check = stripos($page,'string_to_check');
if($check > 0){
return $page; //String found. Return scraped data.
}
else {
//String not found. Loop the script
}
How would I get the whole function code to repeat if the string doesn't exist?

$max_tries = 3;
$success = false;
//try 3 times
for( $i = 0; $i < $max_tries; $i++ ) {
$page = your_scrape_function();
$check = stripos($page,'string_to_check');
if($check > 0){
$success = true;
break; //String found. Break loop.
}
}
// double check that the string was actually found and you didn't just exceed $max_tries
if( ! $success ) {
die('Error: String not found or scrape unsuccessful.');
}

Related

Large processes in PhalconPHP

I have webapp that is logging application and I need backup/restore/import/export feature there. I did this successfully with laravel but have some complications with Phalcon. I don't see native functions in phalcon that would split on chunks execution of large php scripts.
The thing is that logs will be backed up and restored as well as imported by users in ADIF format (adif.org) I have parser for that format which converts file to array of arrays then every record should search through another table, containing 2000 regular expressions, and find 3-10 matches there and connect imported records in one table to those in another table (model relation hasMany) That means that every imported record should have quite some processing time. laravel did it somehow with 3500 records imported, I dont know how it will handle more. The average import will contain 10000 records and each of them need to be verified with 2000 regular expression.
The main issue is how to split this huge processing mount into smaller chunks so I wouldnt get timeouts?
Here is the function that could flawlessly do the job with adding 3862 records in one table and as a result of processing of every record add 8119 records in another table:
public function restoreAction()
{
$this->view->disable();
$user = Users::findFirst($this->session->auth['id']);
if ($this->request->isPost()) {
if ($this->request->isAjax()) {
$frontCache = new CacheData(array(
"lifetime" => 21600
));
$cache = new CacheFile($frontCache, array(
"cacheDir" => "../plc/app/cache/"
));
$cacheKey = $this->request->getPost('fileName').'.cache';
$records = $cache->get($cacheKey);
if ($records === null) {
$rowsPerChunk = 50;
$adifSource = AdifHelper::parseFile(BASE_URL.'/uploads/'.$user->getUsername().'/'.$this->request->getPost('fileName'));
$records = array_chunk($adifSource, $rowsPerChunk);
$key = array_keys($records);
$size = count($key);
}
for ($i = 0; $i < $size; $i++) {
if (!isset($records[$i])) {
break;
}
set_time_limit(50);
for ($j=0; $j < $rowsPerChunk; $j++) {
$result = $records[$i][$j];
if (!isset($result)) {
break;
}
if(isset($result['call'])) {
$p = new PrefixHelper($result['call']);
}
$bandId = (isset($result['band']) && (strlen($result['band']) > 2)) ? Bands::findFirstByName($result['band'])->getId() : null;
$infos = (isset($p)) ? $p->prefixInfo() : null;
if (is_array($infos)) {
if (isset($result['qsl_sent']) && ($result['qsl_sent'] == 'q')) {
$qsl_rcvd = 'R';
} else if (isset($result['eqsl_qsl_sent']) && ($result['eqsl_qsl_sent'] == 'c')) {
$qsl_rcvd = 'y';
} else if (isset($result['qsl_rcvd'])) {
$qsl_rcvd = $result['qsl_rcvd'];
} else {
$qsl_rcvd ='i';
}
$logRow = new Logs();
$logRow->setCall($result['call']);
$logRow->setDatetime(date('Y-m-d H:i:s',strtotime($result['qso_date'].' '.$result['time_on'])));
$logRow->setFreq(isset($result['freq']) ? $result['freq'] : 0);
$logRow->setRst($result['rst_sent']);
$logRow->setQslnote(isset($result['qslmsg']) ? $result['qslmsg'] : '');
$logRow->setComment(isset($result['comment']) ? $result['comment'] : '');
$logRow->setQslRcvd($qsl_rcvd);
$logRow->setQslVia(isset($result['qsl_sent_via']) ? $result['qsl_sent_via'] : 'e');
$logRow->band_id = $bandId;
$logRow->user_id = $this->session->auth['id'];
$success = $logRow->save();
if ($success) {
foreach ($infos as $info) {
if (is_object($info)) {
$inf = new Infos();
$inf->setLat($info->lat);
$inf->setLon($info->lon);
$inf->setCq($info->cq);
$inf->setItu($info->itu);
if (isset($result['iota'])) {
$inf->setIota($result['iota']);
}
if (isset($result['pfx'])) {
$inf->setPfx($result['pfx']);
}
if (isset($result['gridsquare'])) {
$inf->setGrid($result['gridsquare']);
} else if (isset($result['grid'])) {
$inf->setGrid($result['grid']);
}
$inf->qso_id = $logRow->getId();
$inf->prefix_id = $info->id;
$infSuccess[] = $inf->save();
}
}
}
}
}
sleep(1);
}
}
}
}
I know, the script needs a lot of improvement but for now the task was just to make it work.
I think that the good practice for large processing task in php is console applications, that doesn't have restrictions in execution time and can be setup with more memory for execution.
As for phalcon, it has builtin mechanism for running and processing cli tasks - Command Line Applications (this link will always point to the documentation of a phalcon latest version)

isset $_post issue with mysql query

I'm just working on the Backend of a project an have a small problem with this snippet
if (isset($_POST['id'])) {
$cat_delete = "DELETE FROM category WHERE categoryid='".$_POST['id']."' ";
$cat_delete_ex = mysql_query($cat_delete);}`
But if the id is set with post, nothing happens.
The mysql query is working when I delete the
if (isset($_POST['id']))
anyone have an idea ?
Well I am not sure if your method is safe or not, but I would do it like this, might even throw in a regex to check for just numbers if the id is numeric:
EDIT: I made a revision, since you are dealing with an ID, I will assume the ID is numeric only, so instead of escaping it, I just will strip out everything but numbers. This may be a better fit for your situation. I also converted the function to a class so you will be able to reuse the script for several types of sanitizing strings. Maybe its because I am an overachiever too, I don't know. ADD, OCD, etc. Blame it on that :)
$postID = isset($_POST['id']) ? sanitize::ID($_POST['id']) : '';
if (sanitize::email("test#example.com")){
echo "Real email";
} else {
echo "Fake email";
}
if ($postID != ''){
$cat_delete = "DELETE FROM category WHERE categoryid='".$postID."' ";
$cat_delete_ex = mysql_query($cat_delete);
}
class sanitize{
function ID($string){
$string = preg_replace('/[^0-9,]|,[0-9]*$/','',$string);
return $string;
}
# I added another sanitize function so you can see what you can do
# with it. Add phone numbers, domain names, etc... Each one could
# be called with sanitize::{FUNCTION}
function email($string){
if (!ereg("^[^#]{1,64}#[^#]{1,255}$", $string)) {
return false;
}
$email_array = explode("#", $string);
$local_array = explode(".", $email_array[0]);
for ($i = 0; $i < sizeof($local_array); $i++) {
if (!ereg("^(([A-Za-z0-9!#$%&'*+/=?^_`{|}~-][A-Za-z0-9!#$%&'*+/=?^_`{|}~\.-]{0,63})|(\"[^(\\|\")]{0,62}\"))$",$local_array[$i])) return false;
}
if (!ereg("^\[?[0-9\.]+\]?$", $email_array[1])) {
$domain_array = explode(".", $email_array[1]);
if (sizeof($domain_array) < 2) return false;
for ($i = 0; $i < sizeof($domain_array); $i++) {
if (!ereg("^(([A-Za-z0-9][A-Za-z0-9-]{0,61}[A-Za-z0-9])|([A-Za-z0-9]+))$", $domain_array[$i])) return false;
}
}
return true;
}
}
are you sure you are using post for the id?(asking because is the right way, but i have one too many times
<form action="action.php?id=hereistheid"
which will bring the id in the $_GET not $_POST.
next the checking
$id=(int)$_POST['id'];
if($id)
{
//do smth
}

Baffled as to why PHP is giving simple logic error on if statement

Like the title says, PHP is really confusing me on a simple if comparison statement that's returning the opposite of what it should be returning. I'm trying to compare 2 datetime's that are first converted to strings:
//Fetched db query, this returns 2012-06-23 16:00:00
$databaseDateTime = strtotime($row['time']);
//This now returns 1340481600
//today's date and time I'm comparing to, this returns 2012-06-22 17:14:46
$todaysDateTime = strtotime(date("Y-m-d H:i:s"));
//this now returns 1340399686
Great, everything works perfect so far. Now here's where things get hairy:
if ($databaseDateTime < $todaysDateTime) { $eventType = 'past'; }
And this returns 'past', which of course it shouldn't. Please tell me I'm missing something. My project kind of depends on this functionality being airtight.
**EDIT***
Thanks guys for taking the time to help me out. Let me post the entire code because a few of you need more context. The request is coming from an IOS5 to my backend code and json is being sent back to the phone.
<?php
//all included files including $link to mysqli_db and function sendResponse()
function getEvents($eventType, $eventArray) {
global $link;
global $result;
global $i;
global $todaysDateTime;
foreach ($eventArray as $key => $value) {
$sqlGetDeal = mysqli_query($link, "SELECT time FROM deals WHERE id='$value' AND active='y' LIMIT 1") or die ("Sorry there has been an error!");
while ($row = mysqli_fetch_array($sqlGetDeal)) {
//compare times to check if event already happened
$databaseDateTime = strtotime($row['time']);
if ($databaseDateTime < $todaysDateTime) { $eventType = 'past'; }
$result[$i] = array(
'whenDeal' => $eventType,
'time' => $databaseDateTime,
);
$i++;
}//end while
}//end foreach
}
if (isset($_GET['my'])) {
//$_GET['my'] comes in as a string of numbers separated by commas e.g. 3,2,6,3
$myDeals = preg_replace('#[^0-9,]#', '', $_GET['my']);
$todaysDateTime = strtotime(date("Y-m-d H:i:s"));
$result = array();
$kaboomMy = explode(",", $myDeals);
$i = 1;
if ($myEvents != "") {
getEvents('future', $kaboomMy);
}//end if
sendResponse(200, json_encode($result));
} else {
sendResponse(400, 'Invalid request');
} //end $_POST isset
?>
Found a quick hack around the issue. I just added a local variable to my function and rearranged my compare statement
//added local variable $eventTyppe to function
$eventTyppe;
changed compare from:
if ($databaseDateTime < $todaysDateTime) { $eventType = 'past'; }
to:
if ($todaysDateTime < $databaseDateTime ) {
$eventTyppe = $eventType;
} else {
$eventTyppe = 'past';
}
Notice if I rearrange compare:
if ($databaseDateTime < $todaysDateTime ) {
$eventTyppe = 'past';
} else {
$eventTyppe = $eventType;
}
I still get the same error. This is the weirdest thing I've ever seen and the first PHP bug I've run into (I'm assuming it's a PHP bug).
Could you print the values of the times right before this line?
if ($databaseDateTime < $todaysDateTime) { $eventType = 'past'; }
Since that one is declared as global I'm wondering if is it coming back incorrectly.

Telnet to cisco switch using php

I need to telnet to cisco switch using php and execute show interface status command and get results. I tried some php classes I found on internet but none of them could connect to device. So I tried to write the script myself, but I have the same problem, I cant connect to device.
The host sends me banner message and then new line with username:.
I send my username with \r\n, wait some time and tries to read data, but it looks to me like host is just ignoring my new line characters. This is response I got (explode('\n') on response):
Array
(
[0] => %
[1] => User Access Verification
[2] => Username: timeout expired!
)
Why didn't I get prompt on password? I tried it with sending telnet headers, and without, no change. Can anyone please help me?
Here is my code
<?
$host = "switchName";
$name = "name";
$pass = "pass";
$port = 23;
$timeOut = 15;
$connected = false;
$skipNullLines = true;
$timeout = 125000;
$header1=chr(0xFF).chr(0xFB).chr(0x1F).chr(0xFF).chr(0xFB).chr(0x20).chr(0xFF).chr(0xFB).chr(0x18).chr(0xFF).chr(0xFB).chr(0x27).chr(0xFF).chr(0xFD).chr(0x01).chr(0xFF).chr(0xFB).chr(0x03).chr(0xFF).chr(0xFD).chr(0x03).chr(0xFF).chr(0xFC).chr(0x23).chr(0xFF).chr(0xFC).chr(0x24).chr(0xFF).chr(0xFA).chr(0x1F).chr(0x00).chr(0x50).chr(0x00).chr(0x18).chr(0xFF).chr(0xF0).chr(0xFF).chr(0xFA).chr(0x20).chr(0x00).chr(0x33).chr(0x38).chr(0x34).chr(0x30).chr(0x30).chr(0x2C).chr(0x33).chr(0x38).chr(0x34).chr(0x30).chr(0x30).chr(0xFF).chr(0xF0).chr(0xFF).chr(0xFA).chr(0x27).chr(0x00).chr(0xFF).chr(0xF0).chr(0xFF).chr(0xFA).chr(0x18).chr(0x00).chr(0x41).chr(0x4E).chr(0x53).chr(0x49).chr(0xFF).chr(0xF0);
$header2=chr(0xFF).chr(0xFC).chr(0x01).chr(0xFF).chr(0xFC).chr(0x22).chr(0xFF).chr(0xFE).chr(0x05).chr(0xFF).chr(0xFC).chr(0x21);
function read_string()
{
global $fw,$host,$skipNullLines;
$string = "";
while( !feof($fw) )
{
$read = fgets($fw);
$string .= $read;
// Probably prompt, stop reading
if( strpos($read, ':') !== FALSE || strpos($read, '> (enable)') !== FALSE || strpos($read, $host.'#') !== FALSE)
{ break; }
}
$string = explode("\n", $string);
// Get rid of null lines
$ret = array();
for($i = 0; $i<count($string); $i++)
{
if( trim($string[$i]) == '' && $skipNullLines ) continue;
$ret[] = $string[$i];
}
return $ret;
}
function send_string($string, $force=false)
{
GLOBAL $timeout,$fw;
$string = trim($string);
// execute only strings that are preceded by "show" (if not forced)
if(!$force && strpos($string, 'show ') !== 0)
{
return 1;
}
fputs($fw, $string."\r\n");
echo("SEND:".$string."\r\n");
usleep($timeout);
}
$fw = fsockopen($host, $port, $errno, $errorstr, $timeOut);
if($fw == false)
{
echo("Cant connect");
}
else
{
echo("Connected<br>");
$connected = true;
stream_set_timeout($fw, $timeout);
// fputs($fw, $header1);
// usleep($timeout);
// fputs($fw, $header2);
// usleep($timeout);
print_r(read_string());
send_string("test", true);
print_r(read_string());
}
fclose($fw);
?>
UPDATE
If I send username at first, and then I read, I get password prompt. I dont understand it, why cant I firstly read messages from host and then send my response. The way it works to me now (send response and then read for prompt) is no-sense! (and I still got "% Authentication failed." message event with right password/name).
...
$connected = true;
stream_set_timeout($fw, $timeout);
send_string("name", true);
send_string("password", true);
print_r(read_string());
...
Okay, so I dont know what was the problem, but after "few" tests I was able to write this class that works for me. I dont know why other telnet classes dont work altough they do pretty much the same. So if anyone will have similar problem, you can try this:
class TELNET
{
private $host;
private $name;
private $pass;
private $port;
private $connected;
private $connect_timeout;
private $stream_timetout;
private $socket;
public function TELNET()
{
$this->port = 23;
$this->connected = false; // connected?
$this->connect_timeout = 10; // timeout while asking for connection
$this->stream_timeout = 380000; // timeout between I/O operations
}
public function __destruct()
{
if($this->connected) { fclose($this->socket); }
}
// Connects to host
// #$_host - addres (or hostname) of host
// #$_user - name of user to log in as
// $#_pass - password of user
//
// Return: TRUE on success, other way function will return error string got by fsockopen()
public function Connect($_host, $_user, $_pass)
{
// If connected successfully
if( ($this->socket = #fsockopen($_host, $this->port, $errno, $errorstr, $this->connect_timeout)) !== FALSE )
{
$this->host = $_host;
$this->user = $_user;
$this->pass = $_pass;
$this->connected = true;
stream_set_timeout($this->socket, 0, 380000);
stream_set_blocking($this->socket, 1);
return true;
}
// else if coulnt connect
else return $errorstr;
}
// LogIn to host
//
// RETURN: will return true on success, other way returns false
public function LogIn()
{
if(!$this->connected) return false;
// Send name and password
$this->SendString($this->user, true);
$this->SendString($this->pass, true);
// read answer
$data = $this->ReadTo(array('#'));
// did we get the prompt from host?
if( strtolower(trim($data[count($data)-1])) == strtolower($this->host).'#' ) return true;
else return false;
}
// Function will execute command on host and returns output
//
// #$_command - command to be executed, only commands beginning with "show " can be executed, you can change this by adding
// "true" (bool type) as the second argument for function SendString($command) inside this function (3rd line)
//
function GetOutputOf($_command)
{
if(!$this->connected) return false;
$this->SendString($_command);
$output = array();
$work = true;
//
// Read whole output
//
// read_to( array( STRINGS ) ), STRINGS are meant as possible endings of outputs
while( $work && $data = $this->ReadTo( array("--More--","#") ) )
{
// CHeck wheter we actually did read any data
$null_data = true;
foreach($data as $line)
{
if(trim($line) != "") {$null_data = false;break;}
}
if($null_data) { break;}
// if device is paging output, send space to get rest
if( trim($data[count($data)-1]) == '--More--')
{
// delete line with prompt (or "--More--")
unset($data[count($data)-1]);
// if second line is blank, delete it
if( trim($data[1]) == '' ) unset($data[1]);
// If first line contains send command, delete it
if( strpos($data[0], $_command)!==FALSE ) unset($data[0]);
// send space
fputs($this->socket, " ");
}
// ak ma vystup max dva riadky
// alebo sme uz nacitali prompt
// IF we got prompt (line ending with #)
// OR string that we've read has only one line
// THEN we reached end of data and stop reading
if( strpos($data[count($data)-1], '#')!==FALSE /* || (count($data) == 1 && $data[0] == "")*/ )
{
// delete line with prompt
unset($data[count($data)-1]);
// if second line is blank, delete it
if( trim($data[1]) == '' ) unset($data[1]);
// If first line contains send command, delete it
if( strpos($data[0], $_command)!==FALSE ) unset($data[0]);
// stop while cyclus
$work = false;
}
// get rid of empty lines at the end
for($i = count($data)-1; $i>0; $i--)
{
if(trim($data[$i]) == "") unset($data[$i]);
else break;
}
// add new data to $output
foreach($data as $v)
{ $output[] = $v; }
}
// return output
return $output;
}
// Read from host until occurence of any index from $array_of_stops
// #array_of_stops - array that contains strings of texts that may be at the end of output
// RETURNS: output of command as array of lines
function ReadTo($array_of_stops)
{
$ret = array();
$max_empty_lines = 3;
$count_empty_lines = 0;
while( !feof($this->socket) )
{
$read = fgets($this->socket);
$ret[] = $read;
//
// Stop reading after (int)"$max_empty_lines" empty lines
//
if(trim($read) == "")
{
if($count_empty_lines++ > $max_empty_lines) break;
}
else $count_empty_lines = 0;
//
// Does last line of readed data contain any of "Stop" strings ??
$found = false;
foreach($array_of_stops AS $stop)
{
if( strpos($read, $stop) !== FALSE ) { $found = true; break; }
}
// If so, stop reading
if($found) break;
}
return $ret;
}
// Send string to host
// If force is set to false (default), function sends to host only strings that begins with "show "
//
// #$string - command to be executed
// #$force - force command? Execute if not preceeded by "show " ?
// #$newLine - append character of new line at the end of command?
function SendString($string, $force=false, $newLine=true)
{
$t1 = microtime(true);
$string = trim($string);
// execute only strings that are preceded by "show"
// and execute only one command (no new line characters) !
if(!$force && strpos($string, 'show ') !== 0 && count(explode("\n", $string)) == 1)
{
return 1;
}
if($newLine) $string .= "\n";
fputs($this->socket, $string);
$t2 = microtime(true);
}
}
// EXAMPLE
$host = "hostname";
$name = "username";
$pass = "password";
$t = new TELNET();
echo("CONNECT:".$t->Connect($host, $name, $pass)."<br>");
echo("LOGIN:".(int)$t->LogIn());
echo("<br>OUTPUT:<br>");
print_r($t->GetOutputOf("show snmp"));
print_r($t->GetOutputOf("show users"));
print_r($t->GetOutputOf("show interface status"));
PS: my devices prompt is "hostname#", so you may need to edit Login function to make this code work with prompt of your device (so you may need in GetOutputOf() )

Always returning 0, Google SERP Checker

My function searches google for the specific keyword and then checks for the site and then returns the what position it is on google (its for my seo dashboard) but it always return's 0, hopefully some fresh eyes can find the faults
<?php
function GoogleSerp($searchquery, $searchurl){
if(!empty($searchquery) && !empty($searchurl))
{
$query = str_replace(" ","+",$searchquery);
$query = str_replace("%26","&",$query);
// How many results to search through.
$total_to_search = 50;
// The number of hits per page.
$hits_per_page = 10;
// Obviously, the total pages / queries we will be doing is
// $total_to_search / $hits_per_page
// This will be our rank
$position = 0;
// This is the rank minus the duplicates
$real_position = 0;
$found = NULL;
$lastURL = NULL;
for($i=0;$i<$total_to_search && empty($found);$i+=$hits_per_page)
{
// Open the search page.
// We are filling in certain variables -
// $query,$hits_per_page and $start.
// $filename = "http://www.google.co.uk/xhtml?q=$query&start=$i&sa=N";
$filename = "http://www.google.co.uk/m?q=$query&num=$hits_per_page&filter=0&start=$i&sa=N";
$file = fopen($filename, "r");
if (!$file)
{
return "error";
}
else
{
// Now load the file into a variable line at a time
while (!feof($file))
{
$var = fgets($file, 1024);
// Try and find the font tag google uses to show the site URL
if(eregi("<span class=\"c\">(.*)</span>",$var,$out))
{
// If we find it take out any <B> </B> tags - google does
// highlight search terms within URLS
$out[1] = strtolower(strip_tags($out[1]));
// Get the domain name by looking for the first /
$x = strpos($out[1],"/");
// and get the URL
$url = substr($out[1],0,$x);
$url = str_replace("/","",$url);
$position++;
// If you want to see the hits, set $trace to something
// if($trace)return($url."<br>");
// If the last result process is the same as this one, it
// is a nest or internal domain result, so don't count it
// on $real_position
if(strcmp($lastURL,$url)<>0)$real_position++;
$lastURL = $url;
// Else if the sites match we have found it!!!
if(strcmp($searchurl,$url)==0)
{
$found = $position;
// We quit out, we don't need to go any further.
break;
}
}
}
}
fclose($file);
}
if($found)
{
$result = $real_position;
}else{
$result = 0;
}
}
return $result;
}
?>
Try urlencode() instead of the two replaces on the query.

Categories