I`m creating a simple data mapping system with PHP, PDO and Mysql.
One of its premises is to map arrays to entities by creating tertiary tables (not sure if the name is correct).
So, when I map an Array, I create the table with a works-for-all statement that uses the class name and method name passed, something like this:
"create table if not exists ".$tablename." (id_".$firstName." int unsigned not null, ".$secondName." ".$type.", constraint fk_".$tablename." foreign key (id_".$firstName.") references ".$firstName."(id) ".$secondReference.");"
The code is not the problem here.
What I wanted to know is if its a bad idea to TRY to create a table (if not exists) in every iteration (it does only create it for real in the first iteration of each element).
EDIT (explaining): As stated, creating inumerous tables is not the worry (wont happen), for this process is automated according to the classes (models) I`m using. The worry is if it is too costy memory and trafic-wise to check if the table exists at every iteration (this way for each item I would access the database twice, once for checking if the table exists and then again for inserting the new element into the table).
Another option would be to check if the table exists trough a select statement first, but it doesn`t seem much better.
One important information is that these methods used for mapping will olny be accessed through the objects DAO referencing each entity.
Edit: The link for the GitHub with the project is https://github.com/Sirsirious/SPDMap
to me it doesn't sound ideal to create a table each time. Might be better to reuse the same table (with an additional column as identifier between your current 'tables'
if you do create the table, don't see anything wrong with create table if not exist. this is a safe and good programming
I'd also consider using temp tables for this thing. if you create the table each time, it sounds like they are one-time usage as well. so if you don't need the data forever, temp can be a good way to go
Related
quick question.
In my user database I have 5 separate tables all containing different information. 4 tables are connected by foreign key to the primary key of the first table.
I am wanting to trigger row inserts on the other 4 tables when I do an insert on the first (primary). I thought that with ON UPDATE CASCADE would do this for me but after trying it I realised it did not...I know clue is in the name ON UPDATE!!!!!
I also tried and failed at multiple triggers on the same table but found this was not possible either.
What I am planning on doing is putting a trigger on the first to INSERT on the second and then putting a trigger on the second to insert on the third......etc
Would just like to know if this is a wise thing to do or not or if I am missing a better and simpler way of doing this.
Any help/advice much appreciated.
Based on the given information, it "feels" as if there might be a flaw in the database design if each of the child tables requires a row for every single row in the parent table. There is a reason that "ON INSERT CASCADE" does not exist; it is typically not considered meaningful.
The first thought that comes to mind is that the child tables should actually be part of the parent table; it sounds as if there is a one-to-one relationship. It still may make sense to have separate tables from an organizational standpoint (and size of records), but it is something to think about.
If there is not a one-to-one relationship, then the ability to add meaningful data beyond default values to the child tables would imply there might be a bit more normalization of data required. If the only values to be added are NULLs, then one could maybe argue that there is no real point in having the record because a LEFT JOIN could produce the same results without that record.
Having said all that, if it is required, I would think that it would be better to have a single trigger on the parent table add all the records to the child tables rather than chain them in several triggers. That way the logic would be contained in a single location.
Not understanding your structure (the information you need in each of these tables is pertinent to correctly answer), I can only guess that a trigger might not be what you want to do this. If your tables have other fields beyond what is in table 1 and they do not have default values, how will you get the value for those other fields inthe trigger? Personally I would use a stored proc to insert to table1 and get the id value back from the insert and then insert to the other tables with the additonal information needed and put it all in a transaction so that if one insert fails all are rolled back.
I decided back when I was coding to have different tables for each type of content. Now I am stuck solving this. Basically my notification system ranks the newest content by its timestamp currently. This is inaccurate however because there is a small chance that someone would submit content at the same time as another person, and incorrect ranking would occur.
Now if I had all my content in a single table, I would simply rank it by an auto-incrementing variable. Is there a way to implement this auto-increment integer across multiple tables (e.g. When something is inserted into table1, id=0, something is inserted into table2, id=1). Or do I have to recode all my stuff into a single table.
NOTE:
The reason I have content in multiple tables is because its organized and it would reduce load stress. I don't really care about the organization anymore, because I can just access the data through a GUI I coded, I'm just wondering about the load stress.
EDIT:
I'm using PHP 5 with MySQL.
Your question, particularly the need for ID spanning over multiple tables, is clearly signalizing that your database design needs change. You should make one table for all content types (as a generalization), with autoincrementing ID. Then, for each particular content type, you can define other table (equivalent of inheritance in OOP) with extra fields, and foreign key pointing to the basic table.
In other words, you need something like inheritance in SQL.
You can create a table with auto increment id just to keep track of ids. Your program would do an insert on that table, get the id, use it as necessary.
Something along the lines of:
function getNextId() {
$res = mysql_query("INSERT INTO seq_table(id) VALUES (NULL)");
$id = mysql_insert_id();
if ($id % 10 == 0) {
mysql_query("DELETE FROM seq_table");
}
return $id;
}
Where seq_table is a table that you've to create just to get the ids. Make it a function so it can be used whenever you need. Every 10 ids generated I delete all generated ids, anyway you don't need them there. I don't delete every time since it would slow down. If another insert happen in the meantime and I delete 11 or more records, it doesn't affect the behaviour of this procedure. It's safe for the purpose it has to reach.
Even if the table is empty new ids will just keep on growing since you've declared id as auto-increment.
UPDATE: I want to clarify why the ID generation is not wrapped in a transaction and why it shouldn't.
If you generate an auto id and you rollback the transaction, the next auto id, will be incremented anyway. Excerpt from a MySQL bug report:
[...] this is not a bug but expected behavior that happens in every RDBMS we know. Generated values are not a part of transaction and they don't care about other statements.
Getting the ID with this procedure is perfectly thread safe. Your logic after the ID is obtained should be wrapped in a transaction, especially if you deal with multiple tables.
Getting a sequence in this way isn't a new concept, for instance, the code of metabase_mysql.php which is a stable DB access library has a method called GetSequenceNextValue() which is quite similar.
In a single table, you could have a field for the content type and clustered index that includes the content type field. This effectively keeps all of one content type in one place on the disc, and another content type in another place, etc. (It's actually organised into pages, but this physical organisation is still true.)
Assuming that each content type has the same fields, this would likely meet your needs and behave similarly to multiple tables. In some cases you may even find that, with appropriate indexes, a single table solution can be faster, more convenient and maintainable, etc. Such as trying to create global unique identifiers across all content types.
If you're unable to merge these back into a single table, you could create a central link table...
CREATE TABLE content_link (
id INT IDENTITY(1,1), -- MS SQL SERVER syntax
content_type INT,
content_id INT -- The id from the real table
)
As you insert into the content tables, also insert into the link table to create your globally unique id.
More simply, but even more manually, just hold a single value somewhere in the database. Whenever you need a new id, use that centrally stored value and increment it by one. Be sure to wrap the increment and collection in a single transaction to stop race conditions. (This can be done in a number of ways, depending on your flavor of SQL.)
EDIT
A couple of MySQL example lines of code from the web...
START TRANSACTION;
INSERT INTO foo (auto,text)
VALUES(NULL,'text'); # generate ID by inserting NULL
INSERT INTO foo2 (id,text)
VALUES(LAST_INSERT_ID(),'text'); # use ID in second table
COMMIT TRANSACTION;
Personally, I'd actually store the value in a variable, commit the transaction, and then continue with my business logic. This would keep the locks on the tables to a minimum.
You could have a separate ID table, insert into that, and use the newly-inserted ID.
e.g.
CREATE TABLE ids (INT UNSIGNED AUTO INCREMENT PRIMARY KEY, timeadded DATETIME);
In the script:
<?php
$r = mysql_query('INSERT INTO ids (timeadded) VALUES (NOW())');
$id = mysql_insert_id();
mysql_query("INSERT INTO someOtherTable (id, data) VALUES ('$id', '$data)");
Add error checking etc. to taste.
The MySQL manual states:
The ID that was generated is maintained in the server on a
per-connection basis. This means that the value returned by the
function to a given client is the first AUTO_INCREMENT value generated
for most recent statement affecting an AUTO_INCREMENT column by that
client. This value cannot be affected by other clients, even if they
generate AUTO_INCREMENT values of their own. This behavior ensures
that each client can retrieve its own ID without concern for the
activity of other clients, and without the need for locks or
transactions.
(Source) So I don't think concerns about ACID complians are a problem.
I am trying to create a site where users can register and create a profile, therefore I am using two MySQL tables within a database e.g. users and user_profile.
The users table has an auto increment primary key called user_id.
The user_profile table has the same primary key called user_id however it is not auto increment.
*see note for why I have multiple tables.
When a user signs up, data from the registration form is inserted into users, then the last_insert_id() is input into the user_id field of the user_profile table. I use transactions to ensure this always happens.
My question is, is this bad practice?
Should I have a unique auto increment primary key for the user_profile table, even though one user can only ever have one profile?
Maybe there are other downsides to creating a database like this?
I'd appreciate if anyone can explain why this is a problem or if it's fine, I'd like to make sure my database is as efficient as possible.
Note: I am using seperate tables for user and user_profile because user_profile contains fields that are potentially null and also will be requested much more than the user table, due to the data being displayed on a public profile.
Maybe this is also bad practice and they should be lumped in one table?
I find this a good approach, I'd give bonus point if you use a foreign key relation and preferably cascade when deleting the user from the user table.
As too separated the core user data in one table, and the option profile data in another - good job. Nothing more annoying then a 50 field dragonish entry with 90% empty values.
It is generally frowned upon, but as long as you can provide the reasoning for the 1 to 1 relationship I'm sure it is fine.
I have used them when I have hundreds of columns (and it would be more logical to split them out into separate tables)
or I need a thinner table to speed up fullscans
In your case I would use a single table and create a couple of views.
see: http://dev.mysql.com/doc/refman/5.0/en/create-view.html
In general a single table approach is more logical, quicker, simpiler, and uses less space.
I don't think it's a bad practice. Sometimes it's quite useful, especially if you want one class to deal with authentication, and not load all profile data. You can then modify how your authentication works, build web services and so on, with little care about maintaining data structures about profiles information which is likely to change as your project evolves.
This is very good practice.
It's right at the core of writing good, modular, normalised relational database structures.
This seems to be a simple problem, but after a while of searching I can't figure out the answer.
I currently have a MySQL table in my local database used by a webapp, and them same table on a database in a remote server. Right now, I'm using the CREATE TABLE IF NOT EXISTS command through PHP to create the table on the databases:
CREATE TABLE IF NOT EXISTS users (
`id` int(10) NOT NULL AUTO_INCREMENT,
`username` varchar(18) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=2 ;
However, let's say I make a modification to the local database, adding a collumn, for example. It would be really annoying to have to go and change the remote database every time I change the local one. Is there an easier way to run code to create a table if it doesn't exist, and if it does exist, make sure it's structure matches that of the create table structure?
Here's an example, to make what I'm trying to convey a little clearer. Let's say on the local database I have a users table, and I decide that in my webapp I want to have another collumn, password. So I go to the local database and add a password collumn. Is there PHP/MySQL code I can run to check if the users table exists, and if it does, make sure it has a password collumn, and if not, add it?
What you are actually looking for are Migrations, e.g. you are looking for a Schema Management Tool that lets you manage your Database structure in versioned code diffs.
For instance, for your described scenario you would first create a script to create the table, e.g. 001_create_user_table.sql. Then you'd use the schema manager to connect and deploy these changes to your databases.
When you want to change or add something, you just write another script, for instance, 002_Add_Password_Column_To_User_Table.sql. Fill in just the code to do that change. Then run the schema manager again.
Typically, you tell the Schema Manager to go through all existing migrations files. On each run, the Schema manager will update a changelog table in the database, so when you run it, it will know which of your scripts it should apply.
The good thing is, you can add these migrations to your regular VCS, so you will always know which database schema you had at which version of your application. And you will have a proper changelog for them.
To directly answer your question you can create temporary procedures to detect field existence like using a query like this:
SHOW COLUMNS FROM table_name LIKE 'column_name';
However in the real world, database changes are general rolled into three scripts. A create script and two deltas one up and one down. Then the database is versioned so that you know at what state the database is in at any given time.
To specifically check for a password column you can use DESCRIBE:
$colExists = false;
$res = mysql_query('DESCRIBE `users`');
while ($row = mysql_fetch_assoc($res)) {
if ($row['Field'] == 'password') {
$colExists = true;
break;
}
}
if (!$colExists) {
// create column
}
However, you should check into replication or some other automated tool to see if they would be a better solution for you.
Follow these steps (you can easily implement this in PHP, I assumed that the name of the table is Foo)
1.) Run the following code:
desc Foo
2.) Based on the result of the first step you can make your create table command (and you should)
3.) Store your data from the existing table which will be replaced in a variable (Optional, you only need this if you can potentially use data from the old table)
4.) Modify the extracted rows from step 3.) so they will be compatible with your new definition (Optional, you only need this if you can potentially use data from the old table)
5.) Get the rows from your new Foo table
6.) Merge the results got in steps 4.) an 5.) (Optional, you only need this if you can potentially use data from the old table)
7.) Run a drop table for the old table
8.) Generate a replace into command to insert all your rows into the newly created Foo table (you can read more about this here)
After these steps, as a result, you will have the new version of the table. If your tables are too large, you can do a CREATE TABLE IF NOT EXISTS command and if that was not successful, run the alter command.
Also, you can make a library to do these steps and will use that in the future instead of solving the same problem several times.
EDIT:
You can connect the database using this function: mysql-connect (documentation here)
You can run a query using this function: mysql-query (documentation here)
Based on the first step you will get the field names (let's assume you store it in a variable called $bar) and you can use your result to generate your select command (connecting to the database where you have important data. It may be both):
$field_list = "1";
foreach ($bar as $key => $value)
$field_list.= ",".$bar[$key];
mysql_connect(/*connection data*/);
mysql_query("select ".$field_list." from Foo");
You can use your new resource to build up an insert command to insert all your important data after deletion recreation (about resources read more here, about how you can generate your insert you can read here, but I suggest that you should use replace into instead of insert which works like the insert, except that it replaces the row if it already exists, it's better here than an insert, read more here)
So, use mysql_connect and mysql_query, and the resource returned by the mysql_query function can be used for replace into later (I've linked now the URL's for everything you need, so I'm pretty sure you'll solve the problem.), apologies for being not specific enough before.
referring to this question, I've decided to duplicate the tables every year, creating tables with the data of the year, something like, for example:
orders_2008
orders_2009
orders_2010
etc...
Well, I know that probably the speed problem could be solved with just 2 tables for each element, like orders_history and order_actual, but I thought that once the handler code is been wrote, there will be no difference.. just many tables.
Those tables will have even some child with foreign key;
for example the orders_2008 will have the child items_2008:
CREATE TABLE orders_2008 (
id serial NOT NULL,
code character(5),
customer text
);
ALTER TABLE ONLY orders_2008
ADD CONSTRAINT orders_2008_pkey PRIMARY KEY (id);
CREATE TABLE items_2008 (
id serial NOT NULL,
order_id integer,
item_name text,
price money
);
ALTER TABLE ONLY items_2008
ADD CONSTRAINT items_2008_pkey PRIMARY KEY (id);
ALTER TABLE ONLY items_2008
ADD CONSTRAINT "$1" FOREIGN KEY (order_id) REFERENCES orders_2008(id) ON DELETE CASCADE;
So, my problem is: what do you think is the best way to replicate those tables every 1st january and, of course, keeping the table dependencies?
A PHP/Python script that, query after query, rebuild the structure for the new year (called by a cron job)?
Can the PostgreSQL's functions be used in that way?
If yes, how (an little example will be nice)
Actually I'm going for the first way (a .sql file containing the structure, and a php/python script loaded by cronjob that rebuild the structure), but i'm wondering if this is the best way.
edit: i've seen that the pgsql function CREATE TABLE LIKE, but the foreigns keys must be added in a second time.. or it will keep the new tables referencied tot he old one.
PostgreSQL has a feature that lets you create a table that inherits fields from another table. The documentation can be found in their manual. That might simplify your process a bit.
You should look at Partitioning in Postgresql. It's the standard way of doing what you want to do. It uses inheritance as John Downey suggested.
Very bad idea.
Have a look around partitioning and keep your eyes on your real goal:
You don't want table sets for every year, because this is not your problem. Lots of systems are working perfectly without them :)
You want to solve some performance and/or storage space issues.
I'd recommend orders and order_history... just periodically roll the old orders into the history, which is a read-only dataset now, so you add an index to cater for every single query you require, and it should (if your data structures are half decent) remain performant.
If your history table starts getting "too big" it's probably time to start think about data warehousing... which really is marvelous, but it certainly ain't cheap.
As others mentioned in your previous question, this is probably a bad idea. That said, if you are dead set on doing it this way, why not just create all the tables up front (say 2008-2050)?