speed up text based many-to-many updating - php

TL;DR;
Is there a faster way of updating m:m records looping one by one, when ids are not known.
This is about using tags.
On the front-end, I've got forms that use text based input to list and update tags.
When the form is submitted a user record needs to have the tags updated by deleting and creating records in the m:m table
There are multiple tag types, so there are multiple tag input fields, but all are stored in same table.
because the current system is using the tags as string, once the form passes the input data back to the server, it is a CSV string.
The plan (plan because I'm updating from storing as a string) is to
join all tags $tags = implode(',', array_merge($tagset1, $tagset2));
get individual tags $update_tags = explode(',',$tags);
delete all tags for user in m:m user_tag table
loop through $update_tags as $tag_name
find tag_id from $tag_name
add record to m:m user_tag table for user_id and tag_id combination
in the past I've seen issues where the deleting happens, but some un-handled error causes updates not to go through. I suspect this can be mitigated by implementing rollbacks.
While the tag count for updating is fairly low on both sides (user will have less than 15, and there are less than 1000 tags total) I'm concerned about the looping aspect of this, as past experience has been a great indicator that sql within a loop is a great thing to avoid.

this is what I ended up doing ...
it's a bit different from what I intended.
I explode the string and compare to the view result (as array) and do two array_diff operations, one to find deletes and one to find insert.
there is one query for each type of operation, lookup user tags, insert user tags, delete user tags.
The only duplicate is the query that looks up tag ids based on the tag name, in which case it's not looping anyway.
public function update_tags($user_id, $tags)
{
// convert tags string to array
$update_tags = explode(',',$tags);
// find user's tags
$user_tag_view_reslt = $user_tag_view->find(array("user_id = ?", $user_id));
// reduce to array with tag names only
$user_tag_view_reslt = array_column( $user_tag_view_reslt, 'tag_name');
$remove_tags = array_diff($user_tag_view_reslt, $update_tags);
$insert_tags = array_diff($update_tags, $user_tag_view_reslt);
$this->remove_user_tags($user_id, $remove_tags);
$this->insert_user_tags($user_id, $insert_tags);
}
public function remove_user_tags($user_id, $tags_array)
{
$tags_string = "'".implode("','",$tags_array)."'";
$tag_model = new \Model\Tag();
$rows = $db->exec("SELECT tag_id FROM tag WHERE tag_name in($tags_string) ");
$delete_tag_ids = implode(',',array_column($rows,'tag_id'));
// check if there are any rows to insert before doing so
if($delete_tag_ids != ''){
$exec = $db->exec("DELETE FROM `user_tag` WHERE user_id = $user_id AND tag_id in ($delete_tag_ids);");
}
}
public function insert_user_tags($user_id, $tags_array)
{
$tags_string = "'".implode("','",$tags_array)."'";
$tag_model = new \Model\Tag();
$rows = $db->exec("SELECT tag_id FROM tag WHERE tag_name in($tags_string) ");
$insert_tag_ids = implode(',',array_column($rows,'tag_id'));
$sql_vals = array();
foreach($rows as $row){
array_push($sql_vals,"($user_id, ".$row['tag_id']." )");
}
$sql_vals_str = implode($sql_vals,",");
// check if there are any rows to insert before doing so
if($insert_tag_ids != ''){
$exec = $db->exec("INSERT INTO `user_tag` (user_id, tag_id) VALUES $sql_vals_str;");
}
}
requires PHP 5.5 or grater (due to array_column), it also won't work without adjusting and accommodating for the models whose code is not shown

Related

PHP: Automated data insertion to two connected tables from one form

So, I've been looking for a solution to my case, but I've kept finding only partial and not quite solving-the-matter kind of answers.
First, let me describe what I'm trying to achieve.
In my database I have two tables: PLACES and PLACES_CATEGORIES which are connected by a third table PLACES_A_CATEGORIES in an entity many to many. That is because a PLACE can be characterised by one or more CATEGORIES (but it can also have no CATEGORIES at all).
I want to add data send in one form to two tables: PLACES and PLACES_A_CATEGORIES. The user has all the categories listed with checkboxes and he may (but doesnt have to) check one or more of them.
I automated the display of those checkboxes so it reacts accordingly to changes in database (like adding or removing categories). This part works just fine, but let me show the code for you as it may be useful in solving the real issue:
$query = "SELECT name FROM places_categories";
$result = $connection->query($query);
$category_no = $result->num_rows;
echo "Categories of places:";
for ($j = 0; $j < $category_no; ++$j)
{
$category = $result->fetch_assoc()['name'];
echo '<br><input type="checkbox" id="'.$category.'" name="places_categories" value="'.$category.'"><label for="'.$category.'">'.$category.'</label><br>';
}
So, let's return to the problem. I want to:
always add data (only one row) to the table PLACES
add as many rows of data to the table PLACES_A_CATEGORIES as many checkboxes have been checked
So, let me now show you how I've tried to solve the matter and below I'll explain what and why I've done.
if ($everything_OK==true)//Hurra, everything is ok, lets add the place to the database
{
mysqli_query($connection, "SET NAMES utf8");//need it for special characters
//Adding multiple rows of data to database
$query = "SELECT name FROM places_categories";
$result = $connection->query($query);
$category_no = $result->num_rows;
for ($j = 0; $j < $category_no; ++$j)
{
$category[$j] = $result->fetch_assoc()['name'];
if ($_POST['places_categories'] == $category[$j])
{
//counts number of records in table PLACES
$query1 = "SELECT name FROM places";
$result1 = $connection->query($query1);
$places_no = $result1->num_rows;
$places_no += 1;
//looks for category_id in table places_categories where the name matches the current value from form
$query2 = "SELECT category_id FROM places_categories WHERE name='$category[$j]'";
$result2 = $connection->query($query2);
$what_category_id = mysqli_fetch_array($result2);
$connection->query("INSERT INTO places_a_categories VALUES ('$places_no', '$what_category_id')");
}
}
if ($connection->query("INSERT INTO places VALUES (NULL, 0, 0, 0, '$name', '$wysokosc', '$zajawka', '$zatloczenie', '$data_dodania', '$data_edycji', '$szer_geo', '$dlu_geo', '$tytul', '$opis', '$adres', '$tresc')"))
{
echo "Test!";
}
else
{
throw new Exception($connection->error);
}
}
Okay, explanations:
The part which inserts data to the table PLACES works just fine. It
adds data to the database according to what user has added in a form.
No help needed here.
Because of the before-mentioned automation of
table CATEGORIES I want to check how many of categories actually are
in the database. The first part of the code was supposed to do this.
with instruction FOR I assign every existing category to an array with a value equal to the name of the category in the database
then with instruction IF I want to add ass many rows of data to the table PLACES_A_CATEGORIES as many checkboxes have been checked
first value $places_no equals to id of the place which is being added to another table
second value $what_category_id looks for category_id in table PLACES_CATEGORIES where the name matches the current value got from the checkbox
And what are the results? Data is added to the table PLACES with no problem at all. But there is nothing added to the second table. Furthermore, I get no error message of any kind. It's probably some stupid error I just can't see... Any ideas? What have I done wrong?

Splitting a string of values like 1030:0,1031:1,1032:2 and storing data in database

I have a bunch of photos on a page and using jQuery UI's Sortable plugin, to allow for them to be reordered.
When my sortable function fires, it writes a new order sequence:
1030:0,1031:1,1032:2,1040:3,1033:4
Each item of the comma delimited string, consists of the photo ID and the order position, separated by a colon. When the user has completely finished their reordering, I'm posting this order sequence to a PHP page via AJAX, to store the changes in the database. Here's where I get into trouble.
I have no problem getting my script to work, but I'm pretty sure it's the incorrect way to achieve what I want, and will suffer hugely in performance and resources - I'm hoping somebody could advise me as to what would be the best approach.
This is my PHP script that deals with the sequence:
if ($sorted_order) {
$exploded_order = explode(',',$sorted_order);
foreach ($exploded_order as $order_part) {
$exploded_part = explode(':',$order_part);
$part_count = 0;
foreach ($exploded_part as $part) {
$part_count++;
if ($part_count == 1) {
$photo_id = $part;
} elseif ($part_count == 2) {
$order = $part;
}
$SQL = "UPDATE article_photos ";
$SQL .= "SET order_pos = :order_pos ";
$SQL .= "WHERE photo_id = :photo_id;";
... rest of PDO stuff ...
}
}
}
My concerns arise from the nested foreach functions and also running so many database updates. If a given sequence contained 150 items, would this script cry for help? If it will, how could I improve it?
** This is for an admin page, so it won't be heavily abused **
you can use one update, with some cleaver code like so:
create the array $data['order'] in the loop then:
$q = "UPDATE article_photos SET order_pos = (CASE photo_id ";
foreach($data['order'] as $sort => $id){
$q .= " WHEN {$id} THEN {$sort}";
}
$q .= " END ) WHERE photo_id IN (".implode(",",$data['order']).")";
a little clearer perhaps
UPDATE article_photos SET order_pos = (CASE photo_id
WHEN id = 1 THEN 999
WHEN id = 2 THEN 1000
WHEN id = 3 THEN 1001
END)
WHERE photo_id IN (1,2,3)
i use this approach for exactly what your doing, updating sort orders
No need for the second foreach: you know it's going to be two parts if your data passes validation (I'm assuming you validated this. If not: you should =) so just do:
if (count($exploded_part) == 2) {
$id = $exploded_part[0];
$seq = $exploded_part[1];
/* rest of code */
} else {
/* error - data does not conform despite validation */
}
As for update hammering: do your DB updates in a transaction. Your db will queue the ops, but not commit them to the main DB until you commit the transaction, at which point it'll happily do the update "for real" at lightning speed.
I suggest making your script even simplier and changing names of the variables, so the code would be way more readable.
$parts = explode(',',$sorted_order);
foreach ($parts as $part) {
list($id, $position) = explode(':',$order_part);
//Now you can work with $id and $position ;
}
More info about list: http://php.net/manual/en/function.list.php
Also, about performance and your data structure:
The way you store your data is not perfect. But that way you will not suffer any performance issues, that way you need to send less data, less overhead overall.
However the drawback of your data structure is that most probably you will be unable to establish relationships between tables and make joins or alter table structure in a correct way.

text input (seperated by comma) mysql input as array

I have a form where I am trying to implement a tag system.
It is just an:
<input type="text"/>
with values separated by commas.
e.g. "John,Mary,Ben,Steven,George"
(The list can be as long as the user wants it to be.)
I want to take that list and insert it into my database as an array (where users can add more tags later if they want). I suppose it doesn't have to be an array, that is just what seems will work best.
So, my question is how to take that list, turn it into an array, echo the array (values separated by commas), add more values later, and make the array searchable for other users. I know this question seems elementary, but no matter how much reading I do, I just can't seem to wrap my brain around how it all works. Once I think I have it figured out, something goes wrong. A simple example would be really appreciated. Thanks!
Here's what I got so far:
$DBCONNECT
$artisttags = $info['artisttags'];
$full_name = $info['full_name'];
$tel = $info['tel'];
$mainint = $info['maininst'];
if(isset($_POST['submit'])) {
$tags = $_POST['tags'];
if($artisttags == NULL) {
$artisttagsarray = array($full_name, $tel, $maininst);
array_push($artisttagsarray,$tags);
mysql_query("UPDATE users SET artisttags='$artisttagsarray' WHERE id='$id'");
print_r($artisttagsarray); //to see if I did it right
die();
} else {
array_push($artisttags,$tags);
mysql_query("UPDATE users SET artisttags='$artisttags' WHERE id='$id'");
echo $tags;
echo " <br/>";
echo $artisttags;
die();
}
}
Create a new table, let's call it "tags":
tags
- userid
- artisttag
Each user may have multiple rows in this table (with one different tag on each row). When querying you use a JOIN operation to combine the two tables. For example:
SELECT username, artisttag
FROM users, tags
WHERE users.userid = tags.userid
AND users.userid = 4711
This will give you all information about the user with id 4711.
Relational database systems are built for this type of work so it will not waste space and performance. In fact, this is the optimal way of doing it if you want to be able to search the tags.

Insert Comma Separated Values into MYSQL + Additional Form Data

I have a form that allows users to tag an image, choose a location from a drop-down, & upload the image. The tagging takes place by allowing multiple values, separated by commas, to be entered into a field.
This code is successfully inputting the comma delimited list to individual rows:
$categories = $_POST['bib'];
$categories = explode(",", $categories);
foreach($categories as $category) {
$category = trim($category); // Remove possible whitespace
$sql = "INSERT INTO athletes (bib) VALUES ('%s')";
$sql = sprintf($sql, mysql_real_escape_string($category));
mysql_query($sql);
}
However, it is not adding the additional content (location from drop-down list & image filename). For query purposes I need to be able to use both the 'bib' tag and the 'location' to be attached to images to allow users to search.
Before implementing the comma-separated option, this code was working to insert all of the data:
mysql_query("INSERT INTO `athletes` VALUES ('$id', '$bib', '$race','$new_file_name')") ;
So, basically I'm trying to merge the functionality of the two.
You need to use MySQL's UPDATE statement to update data in an existing row.

Need help INSERT record(s) MySQL DB

I have an online form which collects member(s) information and stores it into a very long MySQL database. We allow up to 16 members to enroll at a single time and originally structured the DB to allow such.
For example:
If 1 Member enrolls, his personal information (first name, last name, address, phone, email) are stored on a single row.
If 15 Members enroll (all at once), their personal information are stored in the same single row.
The row has information housing columns for all 'possible' inputs. I am trying to consolidate this code and having every nth member that enrolls put onto a new record within the database.
I have seen sugestions before for inserting multiple records as such:
INSERT INTO tablename VALUES
(('$f1name', '$f1address', '$f1phone'), ('$f2name', '$f2address', '$f2phone')...
The issue with this is two fold:
I do not know how many records are
being enrolled from person to person
so the only way to make the
statement above is to use a loop
The information collected from the
forms is NOT a single array so I
can't loop through one array and
have it parse out. My information is
collected as individual input fields
like such: Member1FirstName,
Member1LastName, Member1Phone,
Member2Firstname, Member2LastName,
Member2Phone... and so on
Is it possible to store information in separate rows WITHOUT using a loop (and therefore having to go back and completely restructure my form field names and such (which can't happen due to the way the validation rules are built.)
If you form's structured so that all the fields are numbered properly, so that a "firstname #1" is matched up with all the other "#1" numbered fields, then a loop is the simplest solution.
start_transaction();
$errors = false;
for ($i = 1; $i <= 16; $i++) {
if (... all $i fields are properly filled in ...) {
$field = $_POST["field$i"];
$otherfield = $_POST["otherfield$i"];
etc...
... insert into database ...
} else {
... handle error condition here
$errors = true;
}
}
if (!$errors) {
commit_transaction();
} else {
rollback();
}
If they're numbered randomly, so that firstname1 is matched with lastname42 and address3.1415927, then you'd have to build a lookup table to map all the random namings together, and loop over that
followup per comment:
well, if you absolutely insist on maintaining this database structure, where each row contains 16 sets of repeated firstname/lastname/etc.. records, then you'd do something like this:
$first = true;
for ($i = 1; $i <= 16; $i++) {
if (fields at position $i are valid) {
$firstname = mysql_escape_real_string($_POST["F{$i}name"]);
$lastname = mysql_real_escape_string($_POST["F{$i}lastname"]);
if ($first) {
$dbh->query("INSERT INTO table (f{$i}name, f{$i}lastname) VALUES ($firstname, $lastname);"
$recordID = $dbh->query("SELECT last_insert_id();");
$first = false;
} else {
$dbh->query("UPDATE table SET f{$i}name=$firstname, f{$i}lastname=$lastname WHERE idfield=$recordID");
}
}
}
It's ugly, but basically:
loop through the form field sets until you find a valid set (all required fields filled in, valid data entered, etc..
Insert that data set into the database to create the new record
retrieve ID of that new record
continue looping over the rest of the fields
for every subsequent set of valid records, do an update of the previously created record and add in the new fieldset data.
Though, honestly, unless you've got some highly offbeat design need to maintain a single table with 16 sets of repeated columns, you'd be better off normalizing a bit, and maintain two seperate tables. A parent "enrollment" table, and a child "members" table. That way you can create the parent enrollment table, then just insert new children as you encounter them in the form.
update #2:
well, a simplified form of a normalized layout would be:
signups (id, name, etc...)
signup_members (id, signup_id, firstname, lastname)
and you'd pull the full signup record set with the following query:
SELECT signups.id, signups.name, signup_members.id, firstname, lastname
FROM signups
LEFT JOIN signup_members ON signups.id = signup_members.signup_id
ORDER BY ...
That would give you a series of rows, one for each 'member' signup. To build the CSV, a simple loop with some state checking to see if you've reached a new signup yet:
$oldid = null;
$csv = ... put column headers here if you want ...
while ($signup = $result->fetchrow()) {
if ($signup['signups.id'] != $oldid) {
// current signup doesn't match previous seen id, so got a new signup record
$csv .= "\n"; // start new line in CSV
$csv .= ... add first few columns to new csv row ...
$oldid = $signup['signups.id']; // store new record id
} else {
$csv .= ... add extra member columns to current csv row ...
}
}
What you're trying to do could be simpler, but to solve the problem, you can join the user information into one variable, separated by a char of your choice and send it to Mysql DB...
$user1 = $f1name . ';' . $f1address . ';' . $f1phone;
$user2 = $f2name . ';' . $f2address . ';' . $f2phone;
$user3 = $f3name . ';' . $f3address . ';' . $f3phone;
INSERT INTO table-name VALUES('$user1','$user2','$user3')
To extract, just "explode" the value by the ";".
If you use the same order for all users data, and if you send a verification string in case one user leaves a field blank, works just fine :)
humm... this work's just fine if the user isn't allowed to use ";" as "personal data" :)
Hope it helps U!
I think you might want to look at "variable variables":
http://php.net/manual/en/language.variables.variable.php
Then you could conceivably loop through from 1 to 15, without having to rename your form fields.

Categories