I have a question regarding PHP/MYSQL. I have a function which gets a list of colors in a MYSQL table and renders them into numerous HTML tables. Idea is that most of the will be hidden unless the customer chooses a certain option. When the user puts his mouse over a color it loads a more detailed picture. It works fine, however after a certain number of caracters the script stops executing, here is the code:
Request:
$sql = 'SELECT *, GROUP_CONCAT(description ORDER BY i.model_correspondance ASC) AS concat, GROUP_CONCAT(model_correspondance ORDER BY i.model_correspondance ASC) AS modell, GROUP_CONCAT(element) AS elem, GROUP_CONCAT(coordinates ORDER BY i.model_correspondance ASC) AS coord FROM design_tool_items AS i WHERE i.model='.$shoe.' GROUP BY i.element, i.category, i.subelement ORDER BY i.id ASC';
And the rest of the PHP:
if ($results = Db::getInstance()->ExecuteS($sql)) {
foreach ($results as $row) {
if($row['subelement']=='') {
if(isset($i)) $code .= '</div>';
$code .= '<div class="'.$display.' color_holder" rel="'.$row['style'].'" id="div'.$row['subelement'].'.'.$row['element'].'">';
$i=1;
} else {
$code .= '</div>';
$code .= '<div class="no color_holder" rel="'.$row['style'].'" id="div'.$row['subelement'].'.'.$row['element'].'">';
}
$sentinel = $row['element'];
$code .= '<div style="text-align:left;padding:10px">'.$row['category'].'</div>';
$concat = explode(',', $row['concat']);
$coordinate = explode(',', $row['coord']);
$model_corres = explode(',', $row['modell']);
foreach($concat as $item => $corres) {
if(($i==1)&&(array_search($sentinel, $array_elements)===false)) {
$array_elements[] = $sentinel;
$select = 'selected';
} else $select = '';
$coord = explode('#', $coordinate[$item]);
$clean_array = array($row['category'], $corres);
foreach($clean_array as &$text) {
$text = str_replace(' ', '_', $text);
$text = strtolower($text);
}
$code .= '<div style="width:60px;padding-left:10px;margin:auto;float:left;">';
$code .= '<div onmouseover="Tip(\'<div><img style=width:200px;height:200px; src='.$val.'folded_images/'.$clean_array[0].'/'.$clean_array[1].'.jpg alt=Image:'.$corres.' /><br>'.$corres.'</div>\')" onmouseout="UnTip()" rel="'.$row['style'].'" denom="'.$row['subelement'].'.'.$sentinel.$model_corres[$item].'" class="color '.$select.'" style="background-position:'.$coord[0].'px '.$coord[1].'px;"></div>';
$code .= '</div>';
if(is_int($i/3)) $code .= '<div style="clear:both;"></div>';
$i++;
}
$code .= '<div style="clear:both;"></div>';
}
}
return $code;
At the 100th line the script stops working in the middle and shows only the first three characters of the description. If I reduce the length of the previous descriptions it works further so the problem comes from the array $concat.
Do you think there is a size limit (I know about the 128Mb in PHP 5.2 and 8Mb before however the website runs PHP 5.2.3 and I think I am far from 128Mb)?
Sorry about the long topic, thank you in advance to everyone who will put attention on this.
Have a good day!
Its because the use of GROUP_CONCAT.
A helpfull link that might be useful. link
Could be related to max execution time and/or memory limit. Worth tweaking each separately (and then, if in doubt, together) and see if you get favorable results. You may also want to enable better debug logging for PHP.
Related
Basically I need to get a bunch of data from several different tables and display it, depending upon the values of a filter (drop down box, slider and tags)
Here is my query at the moment:
$query_string = 'SELECT DISTINCT * FROM users
JOIN user_subjects ON users.id = user_subjects.user_id
JOIN user_notifications ON users.id = user_notifications.user_id
';
if ($subjects != 'false') {
if (count($subjects) > 1)
{
// If there is more than 1 subject tag
$query_string .= ' WHERE user_subjects.subject_id = '.$subjects[0];
foreach ($subjects as $subject) {
$query_string .= ' OR user_subjects.subject_id = '.$subject;
}
}
else if (count($subjects) == 1)
{
// If there's only 1 subject tag, select it only
$query_string .= " WHERE user_subjects.subject_id = ".$subjects[0];
}
}
if ($subscription != 'All Subscriptions')
{
$query_string .= ' AND users.user_plan = "'.$subscription.'"';
}
So that part of the query is mostly working, but I also need to select data from another table, depending upon the filter.
the data in the extra table (named 'user_notifications') is as follows:
- user_id
- action
- entity_id
- exp_rewarded
- timestamp
So basically I need to link this table in with the user's info and get everything I need. Sorry if this isn't particularly clear. I'm still trying to figure it out myself. Let me know if it's not.
My guess is that the query not working due to the OR AND issue. Use parenthesis.
if (count($subjects) > 1)
{
// If there is more than 1 subject tag
$query_string .= ' WHERE (user_subjects.subject_id = '.$subjects[0];
foreach ($subjects as $subject) {
$query_string .= ' OR user_subjects.subject_id = '.$subject;
}
$query_string .= ')';
}
I'm having some issues returning values from a server with php + mysql.
This is my code
$result = mysql_query("SELECT * FROM Nicknames", $con);
if (mysql_real_escape_string($_POST['Create']) == "NICKNAME") {
$output;
while ($row = mysql_fetch_assoc($result)) {
if ($row['Taken'] == '0') $output = $output . $row['Nickname'] . ",";
}
echo substr($output, 0, -1);
}
If I add break; in the while loop, it works perfectly and I just get 1 row of my table.
If instead I want to return all 3000 rows, I just get an empty answer from the server.
If the table has 10 rows it works.
I was wondering if it is about the amount of rows, or it is because eventual special characters.
thanks
UPDATE
It works until 1330 rows, if I try to get more, I get an empty result
$counter = 0;
while ($row = mysql_fetch_assoc($result)) {
if ($row['Taken'] == '0') $output = $output . $row['Nickname'] . ",";
if ($counter == 1330) break;
$counter++;
}
echo substr($output, 0, -1);
Somewhere in the middle of your table's rows there may have been an invalid character.
Since you know which row it stops working at, try running the SELECT with different ORDER BY's to determine if this is the case. :)
Have you tried showing error messages?
error_reporting(E_ALL|E_STRICT);
ini_set('display_errors', true);
It is possible that $output is filling the memory_limit before getting to your echo, and if display_errors is false, you won't see the error stating out of memory.
Maybe it's too obvious, but ...
Have you checked the max_execution_time value in your php.ini ?
Maybe the script is dying before finishing the 3000 rows fetch.
Try setting
set_time_limit(0);
at the beginning of the script to avoid this problem
$output after the if condition should be $output = "";
UPDATE: Still can't seem to figure it out. If anyone can lend a hand, it would be appreciated ^^.
I am having a problem and I'm not sure where my code is breaking down. I have a 'follow' function where you can follow different registered users. Their userID's of who you followed are stored in an array (follower_array). It's retrieving each member from the array, but of each member that's followed instead of displaying all the content, it's only displaying the 1 latest one from each member.
$broadcastList= "";
if ( $follower_array != "" ) {
$followArray = explode(",", $follower_array);
$followCount = count($followArray);
$i = 0;
$broadcastList .= "<table>";
foreach( $followArray as $key => $value ) {
$i++;
$sqlName = mysql_query("
SELECT username, fullname
FROM members
WHERE id='$value'
LIMIT 1
") or die ("error!");
while ( $row = mysql_fetch_array($sqlName) ) {
$friendUserName = substr($row["username"],0,12);
}
$sqlBroadcast = mysql_query("
SELECT mem_id, bc, bc_date, device
FROM broadcast
WHERE mem_id='$value'
ORDER BY bc_date ASC
LIMIT 50
") or die ("error!");
while ( $bc_row = mysql_fetch_array($sqlBroadcast) ) {
$mem_id = $bc_row['mem_id'];
$bc = $bc_row['bc'];
$dev = $bc_row['device'];
$bc_date = $bc_row['bc_date'];
$broadcastList .= "<td><a href='member.php?id=' .$value. ''>
$friendUserName</a> • $when_bc • Via: $dev <b>$bc</b></td></tr>";
}
broadcastList .= "</table>";
}
}
So, it's retrieving the members entirely, but not more than their single latest "broadcast." Any insight would be appreciated.. thank you!
Here's what's happening:
while( $row = some_next_result_function() ){
$result = $row["thing"];
}
Is going to overwrite $result every time, so you'll only end up with the last result.
You need to make a list and append to it instead.
I'm gonna take a shot in the dark:
$broadcastList .= "<td><a href='member.php?id=' .$value. ''>
$friendUserName</a> • $when_bc • Via: $dev <b>$bc</b></td></tr>";
That line pretty much misses a starting "tr" element. Are you sure the content isn't shown simply because of the faulty html markup?
Also:
broadcastList .= "</table>";
You're missing the $ there ;).
I don't know if this fixes it or even helps you; but I sure hope so :). Alternatively; you can also check the HTML source to see if the entries really aren't there (see first remark).
I'm trying to make a simple alphabetical list to order items in my database. The thing I can't figure out is how to actually list it.
I would like it to be the same format as you have on miniclip.com
Here's an image
I looked around, but couldnt find an answer really.
(I would like it to finish even at the end of each vertical column, except the last one for sure)
Any help would be welcome!
In MySQL:
SELECT * FROM table ORDER BY name ASC
In PHP:
$fruits = array("lemon", "orange", "banana", "apple");
sort($fruits);
foreach ($fruits as $key => $val) {
echo "fruits[" . $key . "] = " . $val . "\n";
}
fruits[0] = apple
fruits[1] = banana
fruits[2] = lemon
fruits[3] = orange
Assuming that your result set already is sorted by using the ORDER BY clause, to group the results by their first character you just need to remember the first character of the previous entry and print out the first character of the current entry if they are different. So:
$prevLabel = null;
while ($row = mysql_fetch_assoc($result)) {
$currLabel = strtoupper(substr($row['name'], 0, 1));
if ($currLabel !== $prevLabel) {
echo $currLabel;
$prevLabel = $currLabel;
}
echo $row['name'];
}
This will print the first character as a label for each group that’s members have the same first character.
He doesn't seem to have an issue with the storting, but doing the column format and headers for each new letter.
Suppose $arr contains your alphabetically sorted list with numeric keys. each element has indexes 'name' and 'link'. This should be pretty safe assumption for data from a SQL query.
$firstLetter = -1;
$desiredColumns = 4; //you can change this!
$columnCount = (count($arr)+27)/$desiredColumns+1;
echo "<table><tr><td>";
foreach($arr as $key => $cur)
{
if ($key != 0 && $key % desiredColumns == 0) echo "</td><td>";
if ($cur['name'][0] !== $firstLetter)
{
echo "<strong>$firstLetter</strong> <br />"; $firstLetter = $cur['name'][0];
}
echo "".$cur['name']."<br />";
}
echo "</td><tr></table>";
You'll have to treat numbers as a special case, but this is the idea. If you are using a template engine there are obviously better ways of doing this, but I figure you would have mentioned that. This is a rough sketch, making pretty HTML isn't my thing.
--Query-- get table into $arr. I can't see your tables obviously, Im making assumptions if names nad stuff so you'll need to verify or change them
$sql = "SELECT * FROM table T ORDER BY name";
$conn = //you should have this
$res = mysql_query($sql, $conn);
$arr = array();
while($row = mysql_fetch_assc($res)
$arr[] = $row;
// start above code here. This isn't safe for empty query responses or other error but it works
I presume you're using MySQL (or another SQL) database, in which case you should simply retrieve the data in the required order using a SORT BY clause on the lookup SELECT. (Sorting this PHP is trivial via the sort function, but it makes sense to get the database to do this - that's pretty much what it's for.)
In terms of balancing the output of each of the columns, you could get a COUNT of the required rows in your database (or simply use the count of the resulting PHP array of data) and use this to ensure that the output is balanced.
As a final thought, if this is going to be output on a per-page basis, I'd highly recommend generating it into a static file when the structure changes and simply including this static file as a part of the output - generating this on the fly is needlessly resource inefficient.
The mysql option mentioned above is definitely the best bet. If the data comes out of the DM in order, that's the simplest way to go.
Your next option might be to look at the
asort and ksort functions in PHP to find the exact one you're looking for.
http://www.php.net/manual/en/array.sorting.php
How are you pulling the data?
<?php
$result = mysql_query("SELECT titles FROM gamelist ORDER BY title ASC");
while ($row = mysql_fetch_assoc($result)) {
echo "{$result['title']}<br/>";
}
?>
There are two ways to do it.
You could use your database and use the 'order' clause to pull them by a specific field alphabetically.
You could also use either a key sort or value sort on a PHP array.
The PHP functions are sort($array) and ksort($array).
http://php.net/manual/en/function.sort.php
http://php.net/manual/en/function.ksort.php
<?php
$list = $your_list_array_from_database
//if you need info on how to do this, just let me know
sort($list);
foreach($list as $item) {
echo $item;
}
?>
I found this post and had the same problem. I used the code below to output a list by category name with a header equal to the first letter. In my database table (category) I have name and category_letter. So, name = football and category_list = 'F'.
<section>
<?php
try {
$cats_sql = $dbo->prepare("SELECT name, category_list, FROM category WHERE category_list REGEXP '^[A-Z#]' GROUP BY category_list ASC");
$cats_sql->execute();
$results_cats = $cats_sql->fetchAll();
} catch(PDOException $e) {
include('basehttp/error');
}
$array_cats = $results_cats;
if(is_array($array_cats)) {
foreach($array_cats as $row_cats) {
$cat_var = $row_cats[category_list]; // Each Category list title
?>
<aside>
<h1><a name=""><? echo $cat_var ?></a></h1>
<?php
try {
$search_sql = $dbo->prepare("SELECT name, category_list FROM category WHERE category_list=:cat_var ORDER BY name ASC"); // Pulling a list of names for the category list
$search_sql->bindParam(":cat_var",$cat_var,PDO::PARAM_STR);
$search_sql->execute();
$results_search = $search_sql->fetchAll();
} catch(PDOException $e) {
include('basehttp/error');
}
$array_search = $results_search;
if(is_array($array_search)) { // Output list of names which match category
foreach($array_search as $row_search) {
?>
<h2><?php echo $row_search[name]; ?></h2>
<br class="clear">
<?php
}
}
?>
</aside>
<br class="clear">
<?php
}
}
?>
</section>
Its actually Simple....I did similar thing for my project once. I had to pull out all music albums name and categorize them in alphabetical order.
In my table, "album_name" is the column where names are stored.
$sql= "select * from album_table order by album_name ASC";
$temp_char= ""; // temporary variable, initially blank;
using while loop, iterate through records;
while($row= $rs->fetch_assoc())
{
$album_name= $row['album_name'];
$first_char_of_albm= $album_name[0]; // this will store first alphabet;
$first_char_of_albm= strtoupper($first_char_of_albm); // make uppercase or lower as per your needs
if($temp_char!=$first_char_of_albm)
{
echo $first_char_of_albm;
$temp_char= $first_char_of_albm; // update $temp_char variable
}
}
That's it....
I am posting my answer to this old question for 3 reasons:
You don't always get to write your queries to MySQL or another DBMS, as with a web service / API. None of the other answers address PHP sorting without query manipulation, while also addressing the vertical alphabetical sort
Sometimes you have to deal with associative arrays, and only a couple other answers deal with assoc. arrays. BTW, my answer will work for both associative and indexed arrays.
I didn't want an overly complex solution.
Actually, the solution I came up with was pretty simple--use multiple tags with style="float:left", inside of a giant table. While I was sceptical that having multiple tbody tags in a single table would pass HTML validation, it in fact did pass without errors.
Some things to note:
$numCols is your desired number of columns.
Since we are floating items, you may need to set the width and min-width of parent elements and/or add some <br style="clear: both" />, based on your situation.
for alternative sorting methods, see http://php.net/manual/en/array.sorting.php
Here's my full answer:
function sortVertically( $data = array() )
{
/* PREPARE data for printing */
ksort( $data ); // Sort array by key.
$numCols = 4; // Desired number of columns
$numCells = is_array($data) ? count($data) : 1 ;
$numRows = ceil($numCells / $numCols);
$extraCells = $numCells % $numCols; // Store num of tbody's with extra cell
$i = 0; // iterator
$cCell = 0; // num of Cells printed
$output = NULL; // initialize
/* START table printing */
$output .= '<div>';
$output .= '<table>';
foreach( $data as $key => $value )
{
if( $i % $numRows === 0 ) // Start a new tbody
{
if( $i !== 0 ) // Close prev tbody
{
$extraCells--;
if ($extraCells === 0 )
{
$numRows--; // No more tbody's with an extra cell
$extraCells--; // Avoid re-reducing numRows
}
$output .= '</tbody>';
}
$output .= '<tbody style="float: left;">';
$i = 0; // Reset iterator to 0
}
$output .= '<tr>';
$output .= '<th>'.$key.'</th>';
$output .= '<td>'.$value.'</td>';
$output .= '</tr>';
$cCell++; // increase cells printed count
if($cCell == $numCells){ // last cell, close tbody
$output .= '</tbody>';
}
$i++;
}
$output .= '</table>';
$output .= '</div>';
return $output;
}
I hope that this code will be useful to you all.
Long before I knew anything - not that I know much even now - I desgined a web app in php which inserted data in my mysql database after running the values through htmlentities(). I eventually came to my senses and removed this step and stuck it in the output rather than input and went on my merry way.
However I've since had to revisit some of this old data and unfortunately I have an issue, when it's displayed on the screen I'm getting values displayed which are effectively htmlentitied twice.
So, is there a mysql or phpmyadmin way of changing all the older, affected rows back into their relevant characters or will I have to write a script to read each row, decode and update all 17 million rows in 12 tables?
EDIT:
Thanks for the help everyone, I wrote my own answer down below with some code in, it's not pretty but it worked on the test data earlier so barring someone pointing out a glaring error in my code while I'm in bed I'll be running it on a backup DB tomorrow and then on the live one if that works out alright.
I ended up using this, not pretty, but I'm tired, it's 2am and it did its job! (Edit: on test data)
$tables = array('users', 'users_more', 'users_extra', 'forum_posts', 'posts_edits', 'forum_threads', 'orders', 'product_comments', 'products', 'favourites', 'blocked', 'notes');
foreach($tables as $table)
{
$sql = "SELECT * FROM {$table} WHERE data_date_ts < '{$encode_cutoff}'";
$rows = $database->query($sql);
while($row = mysql_fetch_assoc($rows))
{
$new = array();
foreach($row as $key => $data)
{
$new[$key] = $database->escape_value(html_entity_decode($data, ENT_QUOTES, 'UTF-8'));
}
array_shift($new);
$new_string = "";
$i = 0;
foreach($new as $new_key => $new_data)
{
if($i > 0) { $new_string.= ", "; }
$new_string.= $new_key . "='" . $new_data . "'";
$i++;
}
$sql = "UPDATE {$table} SET " . $new_string . " WHERE id='" . $row['id'] . "'";
$database->query($sql);
// plus some code to check that all out
}
}
Since PHP was the method of encoding, you'll want to use it to decode. You can use html_entity_decode to convert them back to their original characters. Gotta loop!
Just be careful not to decode rows that don't need it. Not sure how you'll determine that.
I think writing a php script is good thing to do in this situation. You can use, as Dave said, the html_entity_decode() function to convert your texts back.
Try your script on a table with few entries first. This will make you save a lot of testing time. Of course, remember to backup your table(s) before running the php script.
I'm afraid there is no shorter possibility. The computation for millions of rows remains quite expensive, no matter how you convert the datasets back. So go for a php script... it's the easiest way
This is my bullet proof version. It iterates over all Tables and String columns in a database, determines primary key(s) and performs updates.
It is intended to run the php-file from command line to get progress information.
<?php
$DBC = new mysqli("localhost", "user", "dbpass", "dbname");
$DBC->set_charset("utf8");
$tables = $DBC->query("SHOW FULL TABLES WHERE Table_type='BASE TABLE'");
while($table = $tables->fetch_array()) {
$table = $table[0];
$columns = $DBC->query("DESCRIBE `{$table}`");
$textFields = array();
$primaryKeys = array();
while($column = $columns->fetch_assoc()) {
// check for char, varchar, text, mediumtext and so on
if ($column["Key"] == "PRI") {
$primaryKeys[] = $column['Field'];
} else if (strpos( $column["Type"], "char") !== false || strpos($column["Type"], "text") !== false ) {
$textFields[] = $column['Field'];
}
}
if (!count($primaryKeys)) {
echo "Cannot convert table without primary key: '$table'\n";
continue;
}
foreach ($textFields as $textField) {
$sql = "SELECT `".implode("`,`", $primaryKeys)."`,`$textField` from `$table` WHERE `$textField` like '%&%'";
$candidates = $DBC->query($sql);
$tmp = $DBC->query("SELECT FOUND_ROWS()");
$rowCount = $tmp->fetch_array()[0];
$tmp->free();
echo "Updating $rowCount in $table.$textField\n";
$count=0;
while($candidate = $candidates->fetch_assoc()) {
$oldValue = $candidate[$textField];
$newValue = html_entity_decode($candidate[$textField], ENT_QUOTES | ENT_XML1, 'UTF-8');
if ($oldValue != $newValue) {
$sql = "UPDATE `$table` SET `$textField` = '"
. $DBC->real_escape_string($newValue)
. "' WHERE ";
foreach ($primaryKeys as $pk) {
$sql .= "`$pk` = '" . $DBC->real_escape_string($candidate[$pk]) . "' AND ";
}
$sql .= "1";
$DBC->query($sql);
}
$count++;
echo "$count / $rowCount\r";
}
}
}
?>
cheers
Roland
It's a bit kludgy but I think the mass update is the only way to go...
$Query = "SELECT row_id, html_entitied_column FROM table";
$result = mysql_query($Query, $connection);
while($row = mysql_fetch_array($result)){
$updatedValue = html_entity_decode($row['html_entitied_column']);
$Query = "UPDATE table SET html_entitied_column = '" . $updatedValue . "' ";
$Query .= "WHERE row_id = " . $row['row_id'];
mysql_query($Query, $connection);
}
This is simplified, no error handling etc.
Not sure what the processing time would be on millions of rows so you might need to break it up into chunks to avoid script timeouts.
I had the exact same problem. Since I had multiple clients running the application in production, I wanted to avoid running a PHP script to clean the database for every one of them.
I came up with a solution that is far from perfect, but does the job painlessly.
Track all the spots in your code where you use htmlentities() before inserting data, and remove that.
Change your "display data as HTML" method to something like this :
return html_entity_decode(htmlentities($chaine, ENT_NOQUOTES), ENT_NOQUOTES);
The undo-redo process is kind of ridiculous, but it does the job. And your database will slowly clean itself everytime users update the incorrect data.