I have a problem that I've been trying to solve for the last couple of days.
I have a site where I crawl news and that works perfectly.
Recently however, I've encountered a problem with my analyzer_script as it appears to exceed the time limit my web host has set. Apparently there is an max_execution time at about 1 minute, and my script takes way longer than that. And I'm not able to adjust that in the php.ini script since I'm hosting my website on a public server.
What can I do? Do I need to rewrite my script?
I appreciate your help!
My script is below:
<?php
$array = array();
$sub_array = array();
$analyzer_ids = array();
$res5 = mysqli_query($con,"SELECT id,status FROM statuz ORDER BY id DESC LIMIT 1");
$row5 = mysqli_fetch_array($res5);
$status = $row5['status'];
$status_id = $row5['id'];
if($status == 2) {
$res1 = mysqli_query($con,"SELECT tag, id FROM tags");
while($row1 = mysqli_fetch_array($res1)) {
$tag = $row1['tag'];
$id = $row1['id'];
$res2 = mysqli_query($con,"SELECT sub_tag FROM sub_tags WHERE tag_id = '$id'");
while($row2 = mysqli_fetch_array($res2)) {
$sub_tag = $row2['sub_tag'];
$sub_tag = strtolower($sub_tag);
$sub_array[] = $sub_tag;
}
$array[] = array('tag_id' => $id, 'tag' => $tag, 'sub_tag' => $sub_array);
$sub_array = array();
}
mysqli_query($con,"INSERT INTO analyzer_queue (crawler_id, status)
(SELECT id,0 FROM crawlers)");
$initial_res = mysqli_query($con,"SELECT crawler_id,id FROM analyzer_queue WHERE status = '0'");
while($initial_row = mysqli_fetch_array($initial_res)) {
$analyzer_id = $initial_row['id'];
$start_crawler_id = $initial_row['crawler_id'];
mysqli_query($con,"UPDATE analyzer_queue SET status = '1' WHERE crawler_id = '$start_crawler_id' ORDER BY id DESC LIMIT 1");
$analyzer_ids[] = $analyzer_id;
$res = mysqli_query($con,"SELECT cr.title, cr.content, cr.id
FROM crawler_results cr
INNER JOIN crawlers c
ON c.newspaper_id = cr.newspaper_id
WHERE c.id = '$start_crawler_id'
AND status = '3'
LIMIT 10");
while($row = mysqli_fetch_array($res)) {
$article_id = $row['id'];
$title = $row['title'];
$content = $row['content'];
$content = strip_tags($content);
$content = strtolower($content);
$title = strtolower($title);
$count = array();
foreach ($array as $tag) {
$regex = '/(?:\b' . preg_quote($tag['tag'], '/');
foreach ($tag['sub_tag'] as $sub) {
$regex .= '\b)|(?:\b' . preg_quote($sub, '/');
}
$regex .= '\b)/i';
$count_content = preg_match_all($regex, $content, $count_content);
$count_title = preg_match_all($regex, $title, $count_title);
$count_total[$tag['tag']] = $count_content + $count_title;
$total_count = $count_total[$tag['tag']];
$tag_name = $tag['tag'];
$res5 = mysqli_query($con,"SELECT id FROM tags WHERE tag = '$tag_name'");
$row5 = mysqli_fetch_array($res5);
$tag_id = $row5['id'];
if($total_count != 0) {
mysqli_query($con,"INSERT INTO article_tags (article_id,tag_id,count_tags) VALUES('$article_id','$tag_id','$total_count')");
}
echo$count_total[$tag['tag']];
echo"<br /><br />";
}
echo"<pre>";
print_r($count_total);
echo"</pre>";
mysqli_query($con,"UPDATE crawler_results SET status = '2', analyzer_id = '$analyzer_id' WHERE id = '$article_id'");
}
mysqli_query($con,"UPDATE analyzer_queue SET status = '2' WHERE crawler_id = '$start_crawler_id' ORDER BY id DESC LIMIT 1");
}
mysqli_query($con,"UPDATE crawler_results SET status = '4' WHERE analyzer_id NOT IN (".implode(',',$analyzer_ids).")");
mysqli_query($con,"UPDATE statuz SET status = '3' WHERE id = '$status_id'");
print_r($analyzer_ids);
} else {
echo"Not ready yet";
}
?>
You can use ini_set at the top of your script like this:
ini_set('max_execution_time' , 300 );
The above call would set max execution time of the script to five minutes. Here's the doc page:
http://php.net/manual/en/function.ini-set.php
If your host allows, you might also consider running this script via cron or some other scheduling daemon. PHP scripts run in the cli context usually have no execution time limit, or a much higher one.
Try this:
ini_set('max_execution_time', 0);
The original Post
Related
Okay, this is really frustrating me and I don't know why. I have been scouring the internet for at least two days trying to figure out the problem. I am trying to pull xp amount for a certain skill for all the names stored in the database. It echoed the specified xp for all the users in the database, but was repeating the values with an undefined index. Even though that error is there it still prints the values. I researched the internet and found that break will stop the repeating, but when I add the break it then wont print the values and just gives me the undefined index. THe index does exist.
$sql = "SELECT rsn FROM users";
$qry = $conn->prepare($sql);
$qry->execute();
while($row = $qry->fetch(PDO::FETCH_ASSOC)) {
$rsn = $row['rsn'];
$hs = file_get_contents("http://hiscore.runescape.com/index_lite.ws?player=". $rsn);
$hs = explode("\n",$hs);
$skills = array("Overall","Attack","Defence","Strength","Constitution","Ranged","Prayer","Magic","Cooking","Woodcutting","Fletching","Fishing","Firemaking","Crafting","Smithing","Mining","Herblore","Agility","Thieving","Slayer","Farming","Runecrafting","Hunter","Construction","Summoning","Dungeoneering", "Divination", "Invention");
$i = 0;
foreach(array_unique($skills) as $value){
$hs[$i] = explode(",",$hs[$i]);
$stats[$value]["rank"] = number_format($hs[$i][0]);
$stats[$value]["level"] = number_format($hs[$i][1]);
$stats[$value]["xp"] = number_format($hs[$i][2]);
$i++;
/*require_once 'tourny_config_include.php';
$sql = "UPDATE active_tourny SET xp = :xp AND from_date = :from AND to_date = :to WHERE rsn = :rsn";
$qry = $conn->prepare($sql);
$qry->execute(array(':xp' => $xp, ':rsn' => $rsn, ':to' => $tournyTo, ':from' => $tournyFrom));*/
echo $stats["Attack"]["xp"] . '<br>';
break;
}
}
$qry = $conn->prepare($sql);
$qry->execute();
while($row = $qry->fetch(PDO::FETCH_ASSOC)) {
$rsn = $row['rsn'];
$hs = file_get_contents("http://hiscore.runescape.com/index_lite.ws?player=". $rsn);
$hs = explode("\n",$hs);
$skills = array("Overall","Attack","Defence","Strength","Constitution","Ranged","Prayer","Magic","Cooking","Woodcutting","Fletching","Fishing","Firemaking","Crafting","Smithing","Mining","Herblore","Agility","Thieving","Slayer","Farming","Runecrafting","Hunter","Construction","Summoning","Dungeoneering", "Divination", "Invention");
$i = 0;
foreach($skills as $skill){
$hs[$i] = explode(",",$hs[$i]);
$stats[$skill]["rank"] = number_format($hs[$i][0]);
$stats[$skill]["level"] = number_format($hs[$i][1]);
$stats[$skill]["xp"] = number_format($hs[$i][2]);
$i++;
/*require_once 'tourny_config_include.php';
$sql = "UPDATE active_tourny SET xp = :xp AND from_date = :from AND to_date = :to WHERE rsn = :rsn";
$qry = $conn->prepare($sql);
$qry->execute(array(':xp' => $xp, ':rsn' => $rsn, ':to' => $tournyTo, ':from' => $tournyFrom));*/
// display Attack xp if available
if isset($stats["Attack"]["xp"])
echo $stats["Attack"]["xp"] . '<br>';
}
}
Wow I cannot believe that it was this simple. All I had to do is move
echo $stats["Attack"]["xp"] . '<br>';
outside the foreach loop
There is a table like this:
id imageid path tourid defaultimage
1 12 asd134 1 1
2 12 asd212 1 0
3 12 asd354 1 0
4 15 qwe 2 0
I need to get all entries with imageid 12 and put all of them into different variables. I could run a query for each one, but I never know how many entries are in the table with same id. Sometimes it is 1 and sometimes it is a lot.
I have tried this:
$result3 = mysqli_query($connect,"SELECT * FROM tourimages WHERE tourid='$tourid' AND defaultimage = '1'");
while($row3 = mysqli_fetch_array($result3)) {
$imagepath = $row3['imagepath'];
echo $imagepath;
}
$result4 = mysqli_query($connect,"SELECT * FROM tourimages WHERE tourid='$tourid' AND defaultimage = '0' LIMIT 1");
while($row4 = mysqli_fetch_array($result4)) {
$imagepath2 = $row4['imagepath'];
$image2id = $row4['imageid'];
echo $imagepath2 ;
}
$result5 = mysqli_query($connect,"SELECT * FROM tourimages WHERE tourid='$tourid' AND imageid!='{$image2id}' AND defaultimage = '0' LIMIT 1 ");
while($row5 = mysqli_fetch_array($result5)) {
$imagepath3 = $row5['imagepath'];
echo $imagepath3 ;
}
$imagepath = array();
$result3 = mysqli_query($connect,"SELECT * FROM tourimages WHERE tourid='$tourid'");
while($row3 = mysqli_fetch_assoc($result3)) {
$imagepath[] = $row3['path'];
}
echo "<pre/>";print_r($imagepath);
Use this $imagepath array anywhere you want.It will be useful for future reference too.
Try this code if you need variable:-
$i=1
while($row3 = mysqli_fetch_assoc($result3)) {
$imagepath.$i = $row3['imagepath'];
$i++
}
but good approach is use array :-
while($row3 = mysqli_fetch_assoc($result3)) {
$imagepath[] = $row3['imagepath'];
}
Try this
$result = mysqli_query($connect,"SELECT * FROM tourimages WHERE imageid='$imageid'");
$imagepath = [];
while($row3 = mysqli_fetch_array($result))
{
$imagepath[] = $row3['imagepath'];
}
print_r($imagepath);
Following is my PHP code which is only giving i =0 though in a loop I am incrementing the $i but it always return i as 0 and while loop is only working one time, though my query SELECT * FROM events WHERE DATE(event_date) < CURDATE() is returning 7 records when exectuing in phpmyadmin. Let me know what i am doing wrong here ?
Code -
<?php
include_once $_SERVER['DOCUMENT_ROOT'].'/app/'."config.php";
error_reporting(E_ALL);
if( $_POST['number'] == 'all' ) {
$eventArr = array();
$myarray = array();
$query = "SELECT * FROM events WHERE DATE(`event_date`) < CURDATE()";
$result = mysql_query($query);
$i =0;
while($row = mysql_fetch_assoc($result)) {
$eventArr[$i] = array('event_data'=> $row);
// Get image For an event
$event_id = $row['id'];
$query = "SELECT * FROM event_images WHERE event_id = $event_id ORDER BY `uploaded_date` DESC LIMIT 0,1";
$result = mysql_query($query);
$eventImgArr = array();
while($row = mysql_fetch_assoc($result)) {
$eventImgArr[] = $row;
}
$eventArr[$i]['event_image'] = $eventImgArr;
// Get venue details for the event
$venue_id = $row['venue_id'];
$eventVenArr = array();
$query = "SELECT * FROM `venues` WHERE id = $venue_id";
while($row = mysql_fetch_assoc($result)) {
$eventVenArr[] = $row;
}
$eventArr[$i]['venue_detail'] = $eventVenArr;
echo $i, " -- ";
$i++;
}
$myarray = array('response'=>'1','message'=>'Event data', 'data'=>$eventArr);
echo json_encode($myarray);
return;
}
You are re-using the $result variable for the other queries, which is destroying its value needed for the main loop.
P.S. Also, you're not actually executing the query for the venue details.
I have PHP function which checks to see if variables are set and then adds them onto my SQL query. However I am don't seem to be getting any results back?
$where_array = array();
if (array_key_exists("location", $_GET)) {
$location = addslashes($_GET['location']);
$where_array[] = "`mainID` = '".$location."'";
}
if (array_key_exists("gender", $_GET)) {
$gender = addslashes($_GET["gender"]);
$where_array[] = "`gender` = '".$gender."'";
}
if (array_key_exists("hair", $_GET)) {
$hair = addslashes($_GET["hair"]);
$where_array[] = "`hair` = '".$hair."'";
}
if (array_key_exists("area", $_GET)) {
$area = addslashes($_GET["area"]);
$where_array[] = "`locationID` = '".$area."'";
}
$where_expr = '';
if ($where_array) {
$where_expr = "WHERE " . implode(" AND ", $where_array);
}
$sql = "SELECT `postID` FROM `posts` ". $where_expr;
$dbi = new db();
$result = $dbi->query($sql);
$r = mysql_fetch_row($result);
I'm trying to call the data after in a list like so:
$dbi = new db();
$offset = ($currentpage - 1) * $rowsperpage;
// get the info from the db
$sql .= " ORDER BY `time` DESC LIMIT $offset, $rowsperpage";
$result = $dbi->query($sql);
// while there are rows to be fetched...
while ($row = mysql_fetch_object($result)){
// echo data
echo $row['text'];
} // end while
Anyone got any ideas why I am not retrieving any data?
while ($row = mysql_fetch_object($result)){
// echo data
echo $row->text;
} // end while
I forgot it wasn't coming from an array!
I am building a Links voting site, the formula works ok after the link is voted the second time, the problem is that when the link just has 1 vote it shows up backwards, from oldest to newest.
What I want is for links with one vote to show from newest to oldest. This is the line that calls the links in the front page:
$articles = Article::getAll("order by ranking desc limit $offset, $num_items");
This is the getAll function code:
static function getAll($conditions = ' ')
{
/* Retrieve all the records from the
* database according subject to
* conditions
*/
$db = null;
$results = null;
$records = array();
$query = "select id, created, modified, username, url, title, description, points, ranking from articles $conditions";
try
{
$db = parent::getConnection();
$results = parent::execSql($query);
while($row = $results->fetch_assoc())
{
$r_id = $row['id'];
$r_created = $row['created'];
$r_modified = $row['modified'];
$r_title = $row['title'];
$r_description = $row['description'];
if(!get_magic_quotes_gpc())
{
$r_title = stripslashes($r_title);
$r_description = stripslashes($r_description);
}
$r_url = $row['url'];
$r_username = $row['username'];
$r_points = $row['points'];
$r_ranking = $row['ranking'];
$article = new Article($r_title, $r_description , $r_url, $r_username, $r_created, $r_modified);
$article->id = $r_id;
$article->points = $r_points;
$article->ranking = $r_ranking;
$records[] = $article;
}
parent::closeConnection($db);
}
catch(Exception $e)
{
throw $e;
}
return $records;
}
If anyone can help I would appreciate it.
What about adding the created date to the order clause?
$articles = Article::getAll("order by ranking desc, created desc limit $offset, $num_items");
I'll do what David says, just that if you want the links ordered by the newest first then you have to add the "created" column in descending order:
$articles = Article::getAll("order by ranking desc, created DESC limit $offset, $num_items");