Hello everyone and thank you for viewing this question.
Since someone asked what i am doing this for, here is the answer:
An artist asked me to make him a web app to store all his new concerts etc.. Now, when it comes to add the Instruments, artists etc, i could have 10 instruments, or maybe 100.. Everything is set into a form.. Some data is fixed like location, time etc, but this other fields are added dynamically using DOM..
I am building a system in which the user set up a form to be stored on a database like:
Name,Surname,field_1
//Lets say that this is the "fixed" part of the form
//But the user should be able to add 'n' other fields with no limit
//Therefore my problem is that i would end up with a row made of, lets say,
//4 colums
//And another one of, maybe, 100 columns
//
//Then i will need to access these rows, and row one should have 4 cols, row two 100..
//This can't be done in a "traditional" way since each row should have the
//same amount of cols
//
//I thought to create a new table for each submission
//but this doesn't really make that much sense to me..
//
//Storing all the possible fields in a single one and then
//access them through an array ? That would require too much, even since my fields
//should have the possibility to be edited..
//Each field is a mixture of variables then, like
//field1:a=12,field2:b=18.. too complex
Any help would be very appreciated
I would go the one field approach. You could have three columns, Name, Surname, and field_values. In the field_values column, store a PHP serialized string of an array representing what would otherwise be your columns. For example, running:
array(
['col1'] => 'val',
['col2'] => 'val1',
['col3'] => 'val2',
['col4'] => 'val3'
)
through serialize() would give you:
a:4:{s:4:"col1";s:3:"val";s:4:"col2";s:4:"val1";s:4:"col3";s:4:"val2";s:4:"col4";s:4:"val3";}
and you can take this value and run it back through unserialize() to restore your array and use it however you need to. Loading/saving data within this array is no more difficult than changing values in the array before serializing it and then saving it to the field_values column.
With this method you can have as many or few 'columns' as you need with no need for a ton of columns or tables.
In this case I would personally create a new table for each user, with new row inserted for ever new custom field. You must have a master table containing table names of each user table to access the data within later.
Related
I need to make an import method that takes the CSV file and imports everything in the database.
I've done the parsing with one of Laravel's CSV addons and it works perfectly giving me a big array of values set as:
[
'col1_name' => 'col1 value',
'col2_name' => 'col2 value',
'col3_name' => 'col3 value,
'...' => '...'
]
This is also perfect since all the column names fit my model which makes the database inserts a breeze.
However - a lot of column values are strings that i'd like to set as separate tables/relations. For example, one column contains the name of the item manufacturer, and i have the manufacturer table set in my database.
My question is - what's the easy way to go through the imported CSV and swap the strings with the corresponding ID from the relationship table, making it compatible with my database design?
Something that would make the imported line:
[
'manufacturer' => 'Dell',
]
into:
[
'manufacturer' => '32',
]
I know i could just do a foreach loop comparing the needed values with values from the relationship models but I'm sure there's an easier and more clean way of doing it.
I don't think theres any "nice" way to do this - you'll need to look up each value for "manufacturer" - the question is, how many queries will you run to do so?
A consideration you need to make here is how many rows you will be importing from your CSV file.
You have a couple of options.
1) Querying 1 by 1
I'm assuming you're going to be looping through every line of the CSV file anyway, and then making a new model? In which case, you can add an extra database call in here;
$model->manufacturer_id = Manufacturer::whereName($colXValue)->first()->id;
(You'd obviously need to put in your own checks etc. here to make sure manufacturers exist)
This method is fine relatively small datsets, however, if you're importing lots and lots of rows, it might end up sluggish with alot of arguably unnecessary database calls.
2) Mapping ALL your Manufacturers
Another option would be to create a local map of all your Manufacturers before you loop through your CSV lines;
$mappedManufacturers = Manufacturer::all()->pluck('id', 'name');
This will make $mappedManufacturers an array of manufacturers that has name as a key, id as a value. This way, when you're building your model, you can do;
$model->manufacturer_id = $mappedManufacturers[$colXValue];
This method is also fine, unless you have tens of thousands of Manufacturers!
3) Where in - then re-looping
Another option would be to build up a list of manufacturer names when looping through your CSV lines, going to the database with 1 whereIn query and then re-looping through your models to populate the manufacturer ID.
So in your initial loop through your CSV, you can temporarily set a property to store the name of the manufacturer, whilst adding it to another array;
$models = collect();
$model->..... = ....;
$model->manufacturer = $colXValue;
$models->push($colXValue);
Then you'll end up with a collection of models. You then query the database for ONLY manufacturers which have appeared:
$manufacturers = Manufacturer::whereIn('name', $models->lists('manufacturer'))->get()->keyBy('name')->toArray();
This will give you array of manufacturers, keyed by their name.
You then loop through your $models collection again, assigning the correct manufacturer id using the map;
$model->manufacturer_id = $manufacturers[$model->manufacturer];
Hopefully this will give you some ideas of how you can achieve this. I'd say the solution mostly depends on your use case - if this was going to be a heavy duty ask - I'd definitely Queue it and be tempted to use Option 1! :P
I'm new to SQL and PHP. Maybe I don't understand what you will answer, so please explain it fully that I can understand so here's my question
I have 5 columns: Id, Name, UserName, Password, valuie
You can understand 4 columns, the fifth is valuie means what the user want to give in wish list. Now when user select a thing and add into wish list, that's good but when he/she adds 2 things in his wish list, how to put that in valuie? how can I display 1st and 2nd value? I mean if I want to display 1st one and if 2nd and if both, what I can do about it?
My PHP is good but MySQL is not good....
Code
Insert into user(Name, UserName, Password, Valuie)
Values("bla bla", "blabla", "blabla", "here's 1st value", "here's second");
This seems like something that should be handled by a one-to-many relationship. A user can have many items in his wishlist. This means that you will need to split your current table up into two. Example: A user table and a wish list table:
user: id, name, etc.
wishlist: id, item_name, user_id
Whenever the user adds a new wish list item, it should be added to the wishlist table, keyed by his/her user_id.
Seeing as you are new to MySQL, you should make sure that you read up on the concept of database normalization.
you can use explode function for it, here is a example below, which will make you easier to understand..
suppose you have a field in your db as value=(value1,value2)
now you can fetch both of the these values one by one as following..
$data=explode(',',value);
//An Explode function gives you an array,
//by which you can get any desired value just by passing it's index.
$data1=$data[0];
$data2=$data[1];
Hopefully this would help you.
Thanks.... :)
For that you can use simple technique. Create two columns Value1 and Value2. If user select first value then store it in Value1 and when user select second the store this value in Value2. Before performing this check if the Value1 field is Null in Database. If not then put that value to Value2.
You can also display both values on different location, if you want.
I've got a problem that I just can't seem to find the answer to. I've developed a very small CRM-like application in PHP that's driven by MySQL. Users of this application can import new data to the database via an uploaded CSV file. One of the issues we're working to solve right now is duplicate, or more importantly, near duplicate records. For example, if I have the following:
Record A: [1, Bob, Jones, Atlanta, GA, 30327, (404) 555-1234]
and
Record B: [2, Bobby, Jones, Atlanta, GA, 30327, Bob's Shoe Store, (404) 555-1234]
I need a way to see that these are both similar, take the record with more information (in this case record B) and remove record A.
But here's where it gets even more complicated. This must be done upon importing new data, and a function I can execute to remove duplicates from the database at any time. I have been able to put something together in PHP that gets all duplicate rows from the MySQL table and matches them up by phone number, or by using implode() on all columns in the row and then using strlen() to decide the longest record.
There has got to be a better way of doing this, and one that is more accurate.
Do any of you have any brilliant suggestions that I may be able to implement or build on? It's obvious that when importing new data I'll need to open their CSV file into an array or temporary MySQL table, do the duplicate/similar search, then recompile the CSV file or add everything from the temporary table to the main table. I think. :)
I'm hoping that some of you can point out something that I may be missing that can scale somewhat decently and that's somewhat accurate. I'd rather present a list of duplicates we're 'unsure' about to a user that's 5 records long, not 5,000.
Thanks in advance!
Alex
If I were you I'd give a UNIQUE key to name, surname and phone number since in theory if all these three are equal then it means that it is a duplicate. I am thinking so because a phone number can have only one owner. Anyways, you should find a combination of 2-3 or maybe 4 columns and assign them a unique key. Once you have such a structure, run something like this:
// assuming that you have defined something like the following in your CREATE TABLE:
UNIQUE(phone, name, surname)
// then you should perform something like:
INSERT INTO your_table (phone, name, surname) VALUES ($val1, $val2, $val3)
ON DUPLICATE KEY UPDATE phone = IFNULL($val1, phone),
name = IFNULL($val2, name),
surname = IFNULL($val3, surname);
So basically, if the inserted value is a duplicate, this code will update the row, rather than inserting a new one. The IFNULL function performs a check to see whether the first expression is null or not. If it is null, then it picks the second expression, which in this case is the column value that already exists in your table. Hence, it will update your row with as much as information possible.
I don't think there're brilliant solutions. You need to determine priority of your data fields you can rely on for detecting similarity, for example phone, some kind of IDs, of some uniform address or official name.
You can save some cleaned up values (reduced to the same format like only digits in phones, concatenated full address) along with row which you would be able to use for similarity search when adding records.
Then you need to decide on data completeness in any case to update existing rows with more complete fields, or delete old and add new row.
Don't know any ready solutions for such a variable task and doubt they exist.
So I've got this form with an array of checkboxes to search for an event. When you create an event, you choose one or more of the checkboxes and then the event gets created with these "attributes". What is the best way to store it in a MySQL database if I want to filter results when searching for these events? Would creating several columns with boolean values be the best way? Or possibly a new table with the checkbox values only?
I'm pretty sure selializing is out of the question because I wouldn't be able to query the selialized string for whether the checkbox was ticked or not, right?
Thanks
You can use the set datatype or a separate table that you join. Either will work.
I would not do a bunch of columns though.
You can search the set easily using FIND_IN_SET(), but it's not indexed, so it depends on how many rows you expect (up to a few thousand is probably OK - it's a very fast search).
The normal solution is a separate table with one column being the ID of the event, and the second column being the attribute using the enum datatype (don't use text, it's slower).
create separate columns or you can store them all in one column using bit mask
One way would be to create a new table with a column for each checkbox, as already described by others. I'll not add to that.
However, another way is to use a bitmask. You have just one column myCheckboxes and store the values as an int. Then in the code you have constants or another appropriate way to store the correlation between each checkbox and it's bit. I.e.:
CHECKBOX_ONE 1
CHECKBOX_TWO 2
CHECKBOX_THREE 4
CHECKBOX_FOUR 8
...
CHECKBOX_NINE 256
Remember to always use the next power of two for new values, otherwise you'll get values that overlap.
So, if the first two checkboxes have been checked you should have 3 as the value of myCheckboxes for that row. If you have ONE and FOUR checked you'd have 9 as the values of myCheckboxes, etc. When you want to see which rows have say checkboxes ONE, THREE and NINE checked your query would be like:
SELECT * FROM myTable where myCheckboxes & 1 AND myCheckboxes & 4 AND myCheckboxes & 256;
This query will return only rows having all this checkboxes marked as checked.
You should also use bitwise operations when storing and reading the data.
This is a very efficient way when it comes to speed. You have just a single column, probably just a smallint, and your searches are pretty fast. This can make a big difference if you have several different collections of checkboxes that you want to store and search trough. However, this makes the values harder to understand. If you see the value 261 in the DB it'll not be easy for a human to immeditely see that this means checkboxes ONE, THREE and NINE have been checked whereas it is much easier for a human seeing separate columns for each checkbox. This normally is not an issue, cause humans don't need to manually poke the database, but it's something worth mentioning.
From the coding perspective it's not much of a difference, but you'll have to be careful not to corrupt the values, cause it's not that hard to mess up a single int, it's magnitudes easier than screwing the data than when it's stored in different columns. So test carefully when adding new stuff. All that said, the speed and low memory benefits can be very big if you have a ton of different collections.
I have a page that retrieves records from 1 table in a database when a certain condition is met.
What I want to achieve is to provide the use to with an opportunity to update each record displayed using text boxes.
I am having trouble interpreting what logic to proceed with after the user hits the 'submit' button.
Normally, if I'm updating one record (or a static number of records), I will use the apporpriate amount of SQL statements.
Since the amount of records are dynamically generated, what is the best way to update all at once? How would I know which records were retrieved in the first place to update?
FOR EXAMPLE:
OK, We have a table with student ids (ID), names (SNAME), subjects (SUBJ), grade for each subject (GRADE) and general remarks (COMMENTS).
I want to retrieve information about all students that got an 'A', and write UNIQUE congratulatory remarks for each student (such as 'good job', or 'congratulations', or etc.)
I'd retrieve the records and lay them out on the page, with a text box next to each student record for the comments to be entered. Because I don't know how many text boxes to make, I give the text boxes dynamically generated names using the student ID. The user now enters unique comments for each student, and clicks on submit.
Now, how am I supposed to update these records with the values entered in each text box?
I wouldn't know which students were retrieved in the first place - how would I know what names to use? I'm trying to avoid having to execute the query again after submitting - but is there any other way?
Hope this question was not too confusing.
thanks
Further expanding earlier answers:
You need a loop (e.g. foreach) to display and save the textareas. If the names of the textareas include the students ID, you don't need to know the name, because the text is inserted to the database by the primary key (the students ID). You may name your form-elements as array, to iterate over them, for example (where the numbers are the IDs):
<textarea name="comment[2345234]"></textarea>
<textarea name="comment[8957485]"></textarea>
Read it out as described by #evan:
foreach((array)$_POST['comment'] as $studentId => $studentComment)
{
var_dump($studentId, $studentComment);
}
And if you implement this whole thing as self-requesting form (Affenformular in german), you may also use just one single loop to save and output the textareas.
"I don't think you're understanding what I'm trying to ask." Maybe you don't understand the answers, even you stated it. You don't need a students name to save a database record. But if you really want to submit it, you may also use hidden inputs.
Use foreach() to find the values you care about, put them in an array, and process the array.
Expanding on #Ignacio's answer to make it more easily understandable:
foreach($_POST as $name_of_input => $value_of_input)
{
// do stuff - here is something so you can see the results after the submit
echo "$name_of_input :: $value_of_input <br>";
}