I would like to echo records from a large data set of 300000.
echo first 5000 records than unset($data) and iterate until end of records in the mysql table.
Something like this,
1)
for ($i=0; $i < 5; $i++) {
$data = openssl_random_pseudo_bytes(1000000);
echo "peak_memory_usage = ” . memory_get_peak_usage(true) . “\n”;
doSomething($data);
//unset($data);
}
echo “for loop completed, memory now at ” . memory_get_usage(true) . “\n”;
function doSomething($data) {
echo “size:” . strlen($data) . “\n”;
}
or something like this?
2)
nRows = $pdo->query('select count(*) from employees')->fetchColumn();
$users = new ArrayIterator(range(1, nRows)); // nRows are 3000000 test records
foreach(new LimitIterator($users, 0, 50000) as $u) {
echo $u, "\n";
}
OR
3) #Sameer would you like to add your suggestion to the query below, I might be doing something wrong adding usleep-my coding flaws, which causes the issue of timeout when usleep is added.
$data = $DB->query("SELECT * FROM user_details")->fetchAll();
foreach ($data as $row) {
echo $row['username']." -- ID :" .$row['user_id']. " -- FirstName :" .$row['first_name']. "<br />\n";
}
The third 3) options works fine 50,000 records not much of load on RAM but CPU, is there a way to optimize this to reduce the load on CPU, imagine if 30 people run the same query it will maxed out the CPU? and if i add usleep(10) - it echos the records but with an error at the end saying timeout.
Any suggestions are highly appreciated please.
many thanks for reading my post.
Modified the original post as the objective is the same to reduce load on
the server.. If you like my lengthy post, please vote guys, I am way
behind and I would like to be a contributor in future.
I stumble across an amazing solution by (Dm4Web) data load - amazing solution - but need HTML tables/append o be added and append the results.
Splitting an AJAX call which returns 5000 rows into multiple AJAX calls with 100 rows
Getting error on line 49 Uncaught SyntaxError: Unexpected identifie, when tried to run the script below:
<!DOCTYPE html>
<html>
<head>
<title>SQL Batch List AJAX and jQuery</title>
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.11.0/jquery.min.js"></script>
</head>
<body>
<div id="mainform">
<h2>Fetch REcords 5000 at a time</h2>
<div id="listData">
<div>
<input id="load" name="load" type="button" value ="Load Data">
<input id="cancel" name="cancel" type="button" value ="Cancel">
</div>
</div>
</div>
</body>
<script>
// counter that allows you to get a new set of rows
var step = 0;
// set variable if you want to restrict the number of rows will be loaded
var maxStep = 0;//
// how many rows should be returned
var count = 5000;
// if the cancel button is pressed
var cancel = false;
$(function() {
$('#load').click(function(){
getData();
})
$('#cancel').click(function(){
cancel = true;
})
});
function getData()
{
step++;
//If cancel variable is set to true stop new calls
if(cancel == true) return;
// checks if the variable is set and limits how many rows to be fetched
if(maxStep >0 $$ step >= maxStep)
$.post('ajax.php'
,{
'step':step,
'count':count,
}
,function(data, textStatus, jqXHR){
if(textStatus == "success")
alert("Data: " + data);
/* foreach (data as $row) {
echo $row['username']." -- ID :" .$row['user_id']. " -- FirstName :" .$row['first_name']. "<br />\n";
} */
if(textStatus == "error")
alert("Error: " + jqXHR.status + ": " + jqXHR.statusText);
// when it finishes processing the data, call back function
getData();
}
,'json'
)
}
</script>
</html>
==== ajax.php =====
step = 0;
if(isset($_POST['step'])) $step = (int)$_POST['step'];
$count = 0;
if(isset($_POST['count'])) $count = (int)$_POST['count'];
if($step>0 and $count>0)
{
$offset = ($step-1) * $count;
$limit = $offset.','.$count;
// --------------
// your code here
// --------------
$data = $DB->query("SELECT * FROM user_details LIMIT .$limit")->fetchAll();
$result = mysql_query($sql);
$arr_result = array();
foreach ($data as $row) {
$arr_result[] = $row;
}
$arr_result_enc = json_encode($arr_result);
echo $arr_result_enc;
// echo rows
//echo json_encode($rows);
}
Method 4)
$query = "SELECT COUNT(*) as num FROM employees";
//$select_run = mysqli_query($conn, $select);
$result = mysqli_query($conn, $query) or die(mysql_error());
$row = mysqli_fetch_array($result);
$itemcount = $row['num']; // Roughly 300,000 total items
$batches = $itemcount / 2000; // Number of while-loop calls - around 120.
for ($i = 0; $i <= $batches; $i++) {
$offset = $i * 2000; // MySQL Limit offset number
$query = "SELECT first_name,last_name FROM employees LIMIT 2500, $offset ";
$result = mysqli_query($conn,$query) or die(mysqli_error($conn));
while ($row = mysqli_fetch_array($result)) {
echo $row['first_name'];
}
echo "<BR>";
echo "Run Number: ".$i."<br />";
echo "<BR>";
}
$data is already getting overwritten, so that's not the problem here.
Heavy loops create constant tension on the server, which increases the load.
You can add sleep of few microseconds to allow server free up resources and some breathing time, which will lower the server load. Use usleep and set optimal microseconds.
for ($i=0; $i < 5; $i++) {
usleep(100);
$data = openssl_random_pseudo_bytes(1000000);
}
Ideas to fix your third attempt:
Select the columns you need instead of selecting *.
Cache the result so only need to run it once for each person that loads the page.
Add a LIMIT (paginate) so you only select the first 100 or 1000 rows instead of 50,000. This will also stop the browser from exploding when you load 50k rows at the same time.
Related
I want to make the copy button copy the value of user, but the problem is the button copied only the value of first user. When move to second user, it remain copied the valued of the first user. Here are my code.
<?php
require_once('connection.php');
$sql = "SELECT ftname, menu, longlan FROM
ownerinfo";
$result = $con->query($sql);
if ($result) {
// output data of each row
while($row = $result->fetch_assoc()) {
$a=1;
$long[$a]=$row["longlan"];
echo "Food Truck: " . $row["ftname"]. "
<br>Menu: " . $row["menu"]. "
<br>Longitude Latitude: <input id='he '
value='".$long[$a]."'></input><button
onclick='myFunction()'>Copy</button><br>
<a
href='https://maps.google.com/'>Maps</a>
<hr>";
$a=$a+1;
}
} else {
echo "0 results";
}
?>
<script>
function myFunction() {
var copyText = document.getElementById("he");
copyText.select();
document.execCommand("copy");
alert("Copied the text: " + copyText.value);
}
Considering your loop outputs an ID, your JavaScript will only ever be able to target the first instance of that ID (as multiple IDs are considered invalid markup).
To correct this, output classes from your PHP instead:
<input class='he' value='".$long[$a]."'>
And loop over each of the classes in your JavaScript:
const copyText = document.getElementsByClassName("he");
for (let i = 0; i < copyText.length; i++) {
copyText[i].select();
document.execCommand("copy");
alert("Copied the text: " + copyText[i].value);
}
I have a new question about a project I had been working on. I was designing a grid with different colored cells. it has a hidden div which shows when a cell is clicked, however I realized that only one cell(the last one of it's type) will show. i.e. if I have 2 objects with the column "objaffinity" as 0 ("enemy") it will show both red cells on the grid, however only the last one will actually work.
how can I make it so that it will show the proper information for each cell?
here's my code:
mapgen.php:
<script src="//ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="cellinfo.js"></script>
<script src="cmenu.js"></script>
<?php
require("sql.php");
$sql = <<<SQL
SELECT *
FROM `maps`
WHERE `objpresent` = 1
SQL;
if(!$result = $db->query($sql)){
die('There was an error running the query [' . $db->error . ']');
} // ran the query
//$xobj = array();
//$yobj = array();
$otype = array();
$oname = array();
$xyobj = array();
while($row = $result->fetch_assoc()){
$xyobj[$row['x']][$row['y']] = true;
$otype[$row['id']]=$row['objaffinity'];
$oname[$row['id']]=$row['object'];
}
// get the rows
$cellid=1;
//find whether the row is obstructed
for ($y = 0; $y < 20; $y++) {
echo '<tr>';
for ($x = 0; $x < 25; $x++) {
echo "<td>";
//Detect what type of object it is
if (isset($xyobj[$x][$y])) {
if($otype[$cellid] == 2)
{
echo "<a href='#'> <div class='foe'> </div><div class='foepopup'>";
echo $oname[$cellid];
echo "</div></a>";
}
elseif($otype[$cellid] == 1)
{
echo "<a href='#'><div class='friend'></div><div class='friendpopup'>";
echo $oname[$cellid];
echo "</div></a>";
}
else
{
echo "<a href='#'> <div class='neutral'></div><div class='neutralpopup'>";
echo $oname[$cellid];
echo "</div></a>";
}
$cellid++;
}
echo '</td>';
}
echo '</tr>';
}
?>
Cellinfo.js:
$(document).ready(function(){
//initially hide all popups
$(".foepopup").hide();
$(".neutralpopup").hide();
$(".friendpopup").hide();
//foebutton selected
$(".foe").on("click",function(e){
$(".friendpopup").hide();
$(".neutralpopup").hide();
$(".foepopup").show();
});
//close foe when selected
$(".foepopup").on("click",function(e){
$(".foepopup").hide();
});
//neutral button pressed
$(".neutral").on("click",function(e){
$(".foepopup").hide();
$(".friendpopup").hide();
$(".neutralpopup").show();
});
//close neutral
$(".neutralpopup").on("click",function(e){
$(".neutralpopup").hide();
});
//friend button pressed
$(".friend").on("click",function(e){
$(".foepopup").hide();
$(".neutralpopup").hide();
$(".friendpopup").show();
});
//close friend
$(".friendpopup").on("click",function(e){
$(".friendpopup").hide();
});
});
In your functions you use selectors, so for the script it does not matter which div was clicked.
Let me show you some examples:
$(".foepopup").on("click",function(e){
$(".foepopup").hide();
});
It should be something like this rather:
$(".foepopup").on("click",function(e){
$(this).hide();
});
And another example:
$(".neutral").on("click",function(e){
$(".foepopup").hide();
$(".friendpopup").hide();
$(".neutralpopup").show();
});
Rewrite it like this:
$(".neutral").on("click",function(e){
var td_tag = $(this).parent().parent();
td_tag.children(".foepopup").hide();
td_tag.children(".friendpopup").hide();
td_tag.children(".neutralpopup").show();
});
Rewrite other code on your own. this is the element on which click was triggered. td_tag will contain parent cell of a div clicked. After that, children method will allow you to find needed elements already inside specific cell.
Good luck!
I am getting data from a MySql database, although I don't know if there could be 1 or more hits from the query.
So, i am echoing something from the database, for example like this:
$code = "
<div style='position:absolute; top:100px'>
'$data'
</div>";
echo $code;
Now, since it is absolute; it is overlapping.
How could I change the 'top:' attribute if there are more results?(For every hit from the database, 'top:' would increment by some amount)
$top = 100; // top default value
$query = mysqli_query($link, 'SQL query is going to be here');
while (null !== ($data = mysqli_fetch_assoc($query)))
{
$code = "
<div style='position:absolute; top:".$top."px'>
".$data."
</div>";
echo $code;
$top += 20; // increment top
}
I'm trying to make a virtual shelf type of thing which is populated via a MySQL database. The column shelfPos holds the position of the item on the shelf. Each row/'shelf' starts with <div class="shelfRow"> and obviously ends with </div> so it's styled and positioned correctly. Items on the shelves can be moved around using the jQuery UI droppable interaction.
The overall layout is this: http://jsfiddle.net/aRA5D/
Each shelf can hold 5 items (left to right).
I'm having trouble populating the shelves. At the moment I've got this: (This is in the place of the HTML)
<?php
$sql="SELECT * FROM shelf WHERE userID='$userID'";
$result=mysql_query($sql);
if (mysql_num_rows($result) == 0) {
// Show a message of some sort? (No items)
}
else {
$tries = 1;
$times = 10; // How many shelves. (10 = 2 shelves)
while(($row = mysql_fetch_array($result)) && ($tries <= $times)) {
while ($tries <= $times) {
if ($tries == $row['shelfPos']) {
echo '<div class="drop" id="drop'.$tries.'"><div class="boxArt" id="'.$row['gameID'].'">'.$row['gameID'].'</div></div>';
}
else {
echo '<div class="drop" id="drop'.$tries.'"></div>';
}
$tries = $tries + 1;
}
$times = $times + 5;
}
}
?>
There's several things wrong with it. It doesn't include the <div class="shelfRow"> html (didn't know how/where to put it, as it needs to be echoed after every 5 'blank' and real items - for loop maybe?) and it requires me to input the number of shelves (2 in this case). Would it be possible to determine how many shelves are required based on the item's position? It's awkward to do because it also needs to echo 'blank' .drop divs before and after them so that the items can be moved around.
Hope this all makes sense. Thanks for the help!
First u need to get data in order of ShelfPos
"SELECT * FROM shelf WHERE userID='$userID' order by shelfPos asc"
And try this code:
...
$i = 0;
while($row = mysql_fetch_array($result)) {
//Each 5
if($i % 5 == 0) echo '<div class="shelfRow">';
if ($i == $row['shelfPos']) {
echo '<div class="drop" id="drop'.$i.'"><div class="boxArt" id="'.$row['gameID'].'">'.$row['gameID'].'</div></div>';
}
else {
echo '<div class="drop" id="drop'.$i.'"></div>';
}
//close shelfrow div
if($i % 5 == 4) echo '</div>';
$i++;
}
//to complete the loop
$shelv_left = 5 - ($i % 5);
if($shelv_left < 5) {
for($j=0; $j < $shelv_left; $j++) {
echo '<div class="drop" id="drop'.($i+$j).'"></div>';
}
echo '</div>'; // end shelfrow div
}
...
Alright I've been trying to find an answer to this for hours already but I couldn't resolve it myself.
I'm trying to call a Javascript parent function from a PHP function, however, it is not getting called.
When using the onclick method onclick='parent.dosomething(); everything seems to work fine but if I try to call the function by echo'ing it out, it would just fail for some reason.
echo "<script>parent.reloadprofmessages();</script>"; //this is what is not getting called
Here's the PHP function:
function checkactivity($username)
{
//These are just queries being executed (irrelevant)
$querystats = "SELECT users.fullname, activity.id, activity.sender, activity.receiver, activity.type, activity.dateposted, activity.seen, activity.related FROM activity, users WHERE activity.receiver = '$username' && activity.seen = '0' ORDER BY id DESC LIMIT 1";
$resultstats = mysql_query($querystats);
$num_stats = mysql_num_rows($resultstats);
$rowactivity = mysql_fetch_assoc($resultstats);
//End of queries
if($num_stats > 0) //If there are registries
{
$user = $_SESSION['Username'];
$activity_date = $rowactivity["dateposted"];
$activity_type = $rowactivity["type"];
$activity_sender = $rowactivity["sender"];
$timeactivity = strtotime( "$activity_date" );
$actualtime = time();
$timetoseconds = $actualtime - $timeposted;
$timetominutes = floor($timepassedtoseconds/60);
if($timetominutes < 2)
{
if($activity_sender != $user)
{
if($activity_type == 1) //Messages
{
echo "<script>parent.reloadprofmessages();</script>"; //this is what is not getting called
}
}
}
}
}
And this is my Javascript function at the parent page:
function reloadprofmessages()
{
$('#friendrequests').load('showprofmessages.php?username=<?php echo $actualuser; ?>').fadeIn("slow");
} //refreshes messages
I pressed CTRL + Shift + I in Google Chrome to get to the developer tools, Network > page that does the request that calls the PHP function > Preview and this was what I received:
<script>parent.reloadprofmessages();</script>
However, the function is not getting called.
Resolving this would solve me a lot of problems, to me it is actually still a mystery to know why it doesn't work since it has worked in other cases.
Thank you for your help in advance.
It's not a good idea to fetch javascript and execute it with AJAX. What I would suggest is to firstly change your PHP to this:
if($activity_type == 1) //Messages
{
echo "1";
}
else {
echo "0";
}
Then change your Javascript to this:
function reloadprofmessages()
{
var can_reload = $.ajax({ url: "showprofmessages.php?username=<?php echo $actualuser; ?>" });
if (can_reload) {
parent.erloadprofmessages();
}
}
Hope that helps
Add the type attribute for script tag
echo "<script type='text/javascript' >parent.reloadprofmessages();</script>";
and remember to define the javascript function before this line
So here is what was wrong: (Showing errors)
function checkactivity($username)
{
//These are just queries being executed (irrelevant)
$querystats = "SELECT users.fullname, activity.id, activity.sender, activity.receiver, activity.type, activity.dateposted, activity.seen, activity.related FROM activity, users WHERE activity.receiver = '$username' && activity.seen = '0' ORDER BY id DESC LIMIT 1";
$resultstats = mysql_query($querystats);
$num_stats = mysql_num_rows($resultstats);
$rowactivity = mysql_fetch_assoc($resultstats);
//End of queries
if($num_stats > 0) //If there are registries
{
$user = $_SESSION['Username'];
$activity_date = $rowactivity["dateposted"];
$activity_type = $rowactivity["type"];
$activity_sender = $rowactivity["sender"];
$timeactivity = strtotime( "$activity_date" ); //$timeactivity was not being used
$actualtime = time();
$timetoseconds = $actualtime - $timeposted; //$timeposted doesn't even exist, in other words I wasn't even converting the $activity_date timestamp to time.
$timetominutes = floor($timepassedtoseconds/60);
if($timetominutes < 2)
{
if($activity_sender != $user)
{
if($activity_type == 1) //Messages
{
echo "<script>parent.reloadprofmessages();</script>"; //this was not the correct way of calling a function from the parent page.
}
}
}
}
}
About the Javascript function:
This is what I ended with:
var auto_refresh = setInterval(
function reloadstring()
{
$.get("checknewactivity.php?vprofile=<?php echo $actualuser; ?>", function(activity){
if (activity == 1)
{
$('#profcommentsdiv').load('showprofmessages.php?vprofile=<?php echo $actualuser; ?>').fadeIn("slow");
}
});
}, 1000); // refresh every 1000 milliseconds
And now it works, thank you for your help, I really appreciate it, and as usual, I always get to a safer solution after asking it here.