Can I store an array such as [1,2,3,4] or some JSON like {"name":"John Johnson","street":"Oslo, "phone":"5551234567"} as one record in mysql? If yes, is it really a good approach or for better result, Is it better to separate those and store in individual column and define the relationship? What is the good method in php to handle this?
EDIT: I want to make clear about what I wanted to do. I am using jquery UI and want to store the position of draggable and droppable in array format. The draggable and droppable will be identified with their ID along with their position so that they can be queried back and be created in the next html page whenever user wants to. I am not sure whether I have to explode those arrays, say like, in separate field like FROM_TOP, FROM_LEFT or just one array storage is sufficient. I was also little concerned about the efficiency.
You can; it is a bad idea. This violates something called normalization. You should have things separated out into individual columns and tables (as needed). It would require a lot of work to maintain the data in the JSON. Just think, you would have to update the whole record in PHP then update the record.
Say I just want to update the phone number...
Bad:
UPDATE `table`
SET `Data` = '{"name":"John Johnson","street":"Oslo, "phone":"5551234567foo"}'
Good:
UPDATE `table`
SET `Phone` = '5551234567foo'
Another reason this would be bad... querying would be slow with LIKE and very hard to be deterministic. Say you have a fax number field... which one would it be with the bad example?
Bad:
SELECT * FROM `table` WHERE `Data` LIKE '%5551234567%'
Good:
SELECT * FROM `table` WHERE `Phone` = '5551234567'
PHP really doesn't deal with the schema or handling the data directly, so there is no method to directly handle that. It's all done in the SQL.
If you know how to encode and decode json you can store it as TEXT in your Mysql Database...
It goes against the norms as usually you have normalize then into different tables.... but if this is your only way then use TEXT
It really depends what you are going to do with the data...
You can serialize() your data and store it in one database field or you can separate it and make is accessible from MySQL connections (use it to perform searches etc). If it is always standard (i.e. always has the same fields) save it in it's own fields, if it's just a random array the first method is fine
Your idea is not wrong . Sometimes, you may have to keep data in such format . i have seen this in many applications . Ex : keeping a regular expression or pattern like this . But this is a bad idea , considering database design guidelines . If you want to do a update query on this field, or change a portion of text in the field etc, at that time it would be a difficult task . If possible, keep the data in separate table with many rows .
Thanks
I use
serialize($item);
to store array of data into one field
then
unserialize($row['field_name']);
tor retrieve it
I found this to be an easy way to store a copy of a data set for revision
i would not use this as a primary method of storing data in a table though one i put it the column it there i don't try to manipulate it again. I just use it for storage
Yes you can. Just remember that you should not try to search within those fields. Horrible performance would be expected. Is it perfect design following normalization? No, but it still works.
Doctrine2 has a field type to do this automatically and abstract the serialization logic from you. However, it means getting to use the whole ORM.
If you go this route, I would advise you handle database migrations using scripts. This way, if your requirements ever change and you need to search one of the values contained, you can always write a simple migration and update your data. Modifying a schema should not be a burden.
Yes you can, MySql supports JSON natively see- https://dev.mysql.com/doc/refman/5.7/en/json.html.
As to the "horrible" performance, its not really an issue now, we use this in our production servers storing and querying millions of rows an hour. MySql has also branched out into NoSql territory so the claim that normalization is always right is bs.
INSERT INTO TABLENAME(JSON_COLUMN_NAME) VALUES (
'{"name":"John Johnson","street":"Oslo, "phone":"5551234567"}'
);
Try using json_encode when putting the array in the database and json_decode when extracting from the database.
Related
I am creating an application with a click to call button on an html page.
There will be one person manning the phone. I want this person to be able to set a variable with a boolean value on my server: 1 is available, 0 is unavailable.
I could create a single field SQL table but this feels like overkill, or I could read and write to a text file containing just one character.
What is the most correct way to store a single value?
I know it seems like overkill to use a small database table for this.
If your application already uses a database, this is by far the best way to proceed. Your database technology has all kinds of support for storing data so it doesn't get lost. But, don't stand up a database and organize your application to use it just for this one data point; a file will be easier in that case.
(WordPress does something similar; it uses a table called wp_options containing a lot of one-off settings values.)
I suggest your table contain two columns (or maybe more), agent_id and available. Then, if you happen to add another person taking telephone calls, your app will be ready to handle that growth. Your current person can have agent_id = 0.
If you have a DB set up, I'd use it.
That's what DB's are for, persisting changeable data.. otherwise you are basically writing your own separate DB system for the sake of one setting, which would be uberkill in my eyes!
There is value in consistency and flexibility.. what if I suddenly need to store an expected return time? How do I do this in a text-file, how do I differentiate the column? How do I manipulate the data? MySQL already answers all these questions for you.
As a team member, I'd expect most of my dev colleagues (and new hires) to know how to use MySQL.. I wouldn't want them to have to work with, extend or debug a separate bespoke file persistence system that I had tacked on.
If you are worried about having lots of one row tables dotted about, you could use a single table for miscellaneous singular config variables which need updating regularly.
We have a table like this:
Table: `setting`
Columns: `key_string` VARCHAR, `value` VARCHAR
And could store your variable as
['key_string' => 'telephone_service_available', 'value' => '1']
In this specific case a simple file check (Exist a file or not) is probably the most simple way you can do here. And it also has the benefit to easily check if the file exist or not, you don't have to read file contents.
But if you need just one more information, you have to go a complete other way.
Depends on what you try to do afterwards with the information.
If you use it within a web-application store it in the session.
Or try a flatfile-database like SQLite (no active DBMS needed). Its easy and you can extend it very easy.
Or just a bipolar information with creating a file. If the file is not there is is off.
So far I store all the relevant parameters that I need for my website as variables in a php file; then I point to this file through a requirestatement in each page that needs those parameters.
This is most of times good and easy to mantain, but sometimes I need to change those variables "on the fly" and I feel the need of some sort of web-panel where I can change them even more easily (I'm thinking to a web page with a form to update the parameters).
So I've created a table in my MySQL database to store parameters (basically, this table has two columns: ParamName and ParamValue; I've stored the parameter names as a varchar without the $ sign at the beginning), in order to create the web-panel I've in my mind.
I was thinking to extract all the parameters names and values using this query:
$query=$mysqli->query('select * from parameters');
while($row=mysqli_fetch_assoc($query)) {
$ParamName=$row["ParamName"];
$$ParamName=$row["ParamValue"]; }
Is this a good or a bad idea?
What are the main issues I could encounter doing so (in terms of security, too)?
How deprecable and dangerous is the use of $$ParamName?
I'd better change my mind or can I proceed this way?
EDIT: I changed mysql_ syntax into mysqli_ as suggested.
Using arbitrary values in a database as variable references is highly risky. What you'd want to do is fetch the data from your key/value store into a self-contained associative array and use that instead.
You also do not want to be using mysql_query for anything these days. Please, put that thing away.
If the parameters are all constants that will not be changed over time, it is faster to use a single file that stores them as constants in a simple associative array.
If these parameters will change, using a database that specializes in key-value stores might be more useful for you instead of using a standard relational database.
If you cannot change your database from a relational database, or you are not changing the values often, it would be faster to cache them using something like memcached.
Iam currently unserializing the data retrieved from database then updating the value as per by requirement & then serializing the values & running update query.
Similarly for implementing search on the serialized columns, I have unserialize & compare the values and mine search is like google search. So you can imagine it will be serializing for each requests.
I want to optimize it. Some methods that can improve my current method.
Don't used a serialized column of a DB for storing those values, use a proper search engine instead. In my case, I always use Lucene, using Zend_Search_Lucene.
You can build a Lucene Index very easily, just by defining documents and the fields you wish to store of those documents, then searching will be very efficient. If you want some docs, here and here you have some tutorials on how to use it, altough the official docs pointer first I find them quite explicative.
Hope I can help!
Your approach sounds really unefficient. Have a look at solr/lucene for a real document search system.
If the data is not serialised in your database then you should be able to query specific values without the overhead of having to serialise and unserialise.
The best method for updating serialized columns in database is not to have serialized columns at all. Period.
Map your serialized data into related tables and update whatever value using SQL query
I am developing an URL bookmark application (PHP and MySQL) for myself. With this application I will store URLs in MySQL.
The question is, should I store URLs in a TEXT column or should I first parse the URL and store its components (host, path, query, fragment) in separate columns in one table? The latter one also gives me the chance of generating statistical data by grouping servers and etc. Or maybe I should store servers in a separate table and use JOIN. What do you think?
Thanks.
I'd go with storing them in TEXT columns to start. As your application grows, you can build up the parsing and analysis functionality if you really want to. From what it sounds like, it's all just pie-in-the-sky functionality right now. Do what you need to get the basic application up and running first so that you have something to work with. You can always refactor and go from there.
The answer depends on how you like to use this data in the future.
If you like to analyze the different parts of the URL splitting them is the way to go.
If not. the INSERT, as well, as the SELECT, will be faster, if you store them in just one field.
If you know the URLs are not longer then 255 Chars, varchar(255) will be better, than text, for performance reasons.
If you seriously thing that you're going to be using it for getting interesting data, then sure, do it as a series of columns. Honestly, I'd say it'd probably just be easier to do it as a single column though.
Also, don't forget that it's easy for you to convert back and forth if you want to later. Single to multiple is just a SELECT;regex;INSERT[into another table]; multiple to single is just a INSERT SELECT with CONCAT.
On the Facebook FQL pages it shows the FQL table structure, here is a screenshot below to show some of it (screenshot gone).
You will notice that some items are an array, such as meeting_sex, meeting_for current_location. I am just curious, do you think they are storing this as an array in mysql or just returning it as one, from this data it really makes me think it is stored as an array. IF you think it is, or if you have done similar, what is a good way to store these items as an array into 1 table field and then retrieve it as an array on a PHP page?
alt text http://img2.pict.com/3a/70/2a/2439254/0/screenshot2b187.png
The correct way to store an array in a database is by storing it as a table, where each element of the array is a row in the table.
Everything else is a hack, and will eventually make you regret your decision to try to avoid an extra table.
There are two options for storing as an array:
The first, which you mentioned, is to make one, or several, tables, and enumerate each possible key you intend to store. This is the best for searching and having data that makes sense.
However, for what you want to do, use serialize(). Note: DO NOT EVER EVER EVER try to search against this data in its native string form. It is much faster (and saner) to just reload it, call unserialize(), and then search for your criteria than to develop some crazy search pattern to do your bidding.
EDIT: If it were me, and this were something I was seriously developing for others to use (or even for myself to use, to be completely honest), I would probably create a second lookup table to store all the keys as columns; Heck, if you did that, mysql_fetch_assoc() could give you the array you wanted just by running a quick second query (or you could extract them out via a JOINed query). However, if this is just quick-and-dirty to get whatever job done, then a serialized array may be for you. Unless you really, really don't care about ever searching that data, the proper column-to-key relationship is, I think most would agree, superior.
I guarantee you that Facebook is not storing that data in arrays inside their database.
The thing you have to realize about FQL is that you are not querying Facebook's main data servers directly. FQL is a shell, designed to provide you access to basic social data without letting you run crazy queries on real servers that have performance requirements. Arbitrary user-created queries on the main database would be functional suicide.
FQL provides a well-designed data return structure that is convenient for the type of data that you are querying, so as such, any piece of data that can have multiple associations (such as "meeting_for") gets packaged into an array before it gets returned as an API result.
As other posters have mentioned, the only way to store a programming language structure (such as an array or an object) inside a database (which has no concept of these things), is to serialize it. Serializing is expensive, and as soon as you serialize something, you effectively make it unusable for indexing and searching. Being a social network, Facebook needs to index and search almost everything, so this data would never exist in array form inside their main schemas.
Usually the only time you ever want to store serialized data inside a database is if it's temporary, such as session data, or where you have a valid performance requirement to do so. Otherwise, your data quickly becomes useless.
Split it out into other tables. You can serialize it but that will guarantee that you will want to query against that data later. Save yourself the frustration later and just split it out now.
you can serialize the array, insert it, and then unserialize it when you retrieve it.
They might be using multiple tables with many-to-many relationships, but use joins and MySql's GROUP_CONCAT function to return the values as an array for those columns in one query.