I want to add a certain condition to every single query that is executed. So a certain column within a table needs to be a certain value. I would use defaultScope, but it applies only to SELECT queries. This certain column of course exist in every table throughout the whole database.
EDIT:
The following table:
CREATE TABLE `table` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(200) NOT NULL,
`assigned_user` int(11) NOT NULL,
PRIMARY KEY (`id`)
);
A certain user having ID 1 shall be able to see, edit etc. records that have the value 1 in assigned_user. Using defaultScope he cant see anything else than those, however he is still able to use other query types of the model on records that do not have assigned_user set to 1. This is exactly what I want to restrict, so there is no way he can access other records, by any URL change or different POST variable.
Thanks for every hint, answer and suggestion :)
Related
Well, I have this attendance system who mark an attendance every day, well what I am looking is for a restriction other than PHP code like a restriction that can restrict users to enter duplicate record over time.
For e.g I have already marked my attendance .myself
DATE 22-05-2018 and trackingid = 1
if I try to insert mark attendance one more time it should not insert the statement.
It can be done via php and its a long code and i mean like it is possible but is there any way around with MySQL , through which we can make 2 columns unique if they both already exist just dont let user insert .
Use unique_index on your columns
ALTER TABLE `tablename` ADD UNIQUE `unique_index`(`columnOneName`, `columnTwoName`);
You can also use the following like sql while creating your table:-
CREATE TABLE `tableName` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`columnOne` int(11) DEFAULT NULL,
`columnTwo` int(11) DEFAULT NULL,
`columnThree` varchar(128),
PRIMARY KEY (`id`),
UNIQUE KEY `columnOne_columnTwo_unique_index` (`id_box_elements`,`id_router`)
);
I am a bit stumped on this wierdness.
I have a gps tracking app that logs gps points into a track_log table.
When I do a basic query on the running log table it takes about 50 seconds to complete:
SELECT * FROM track_log WHERE node_id = '26' ORDER BY time_stamp DESC LIMIT 1
When I run the exact same query on the archived table where I copied most of the logs to to reduce the running table's logs to about 1.2 million records.
The archive table is 7.5 million records big.
The exact same query on the archive table runs for 0.1 seconds on the same server even though it's six times bigger!
What's going on?
Here's the full Create Table schema:
CREATE TABLE `track_log` (
`id_track_log` INT(11) NOT NULL AUTO_INCREMENT,
`node_id` INT(11) DEFAULT NULL,
`client_id` INT(11) DEFAULT NULL,
`time_stamp` DATETIME NOT NULL,
`latitude` DOUBLE DEFAULT NULL,
`longitude` DOUBLE DEFAULT NULL,
`altitude` DOUBLE DEFAULT NULL,
`direction` DOUBLE DEFAULT NULL,
`speed` DOUBLE DEFAULT NULL,
`event_code` INT(11) DEFAULT NULL,
`event_description` VARCHAR(255) DEFAULT NULL,
`street_address` VARCHAR(255) DEFAULT NULL,
`mileage` INT(11) DEFAULT NULL,
`run_time` INT(11) DEFAULT NULL,
`satellites` INT(11) DEFAULT NULL,
`gsm_signal_status` DOUBLE DEFAULT NULL,
`hor_pos_accuracy` double DEFAULT NULL,
`positioning_status` char(1) DEFAULT NULL,
`io_port_status` char(16) DEFAULT NULL,
`AD1` decimal(10,2) DEFAULT NULL,
`AD2` decimal(10,2) DEFAULT NULL,
`AD3` decimal(10,2) DEFAULT NULL,
`battery_voltage` decimal(10,2) DEFAULT NULL,
`ext_power_voltage` decimal(10,2) DEFAULT NULL,
`rfid` char(8) DEFAULT NULL,
`pic_name` varchar(255) DEFAULT NULL,
`temp_sensor_no` char(2) DEFAULT NULL,
PRIMARY KEY (`id_track_log`),
UNIQUE KEY `id_track_log_UNIQUE` (`id_track_log`),
KEY `client_id_fk_idx` (`client_id`),
KEY `track_log_node_id_fk_idx` (`node_id`),
KEY `track_log_event_code_fk_idx` (`event_code`),
KEY `track_log_time_stamp_index` (`time_stamp`),
CONSTRAINT `track_log_client_id` FOREIGN KEY (`client_id`) REFERENCES `clients` (`client_id`) ON DELETE NO ACTION ON UPDATE NO ACTION,
CONSTRAINT `track_log_event_code_fk` FOREIGN KEY (`event_code`) REFERENCES `event_codes` (`event_code`) ON DELETE NO ACTION ON UPDATE NO ACTION,
CONSTRAINT `track_log_node_id_fk` FOREIGN KEY (`node_id`) REFERENCES `nodes` (`id_nodes`) ON DELETE NO ACTION ON UPDATE NO ACTION
) ENGINE=InnoDB AUTO_INCREMENT=8632967 DEFAULT CHARSET=utf8
TL;DR
Make sure the indexes are defined in both tables, for this query node_id and time_stamp are good indexes.
Defragment your table: https://dev.mysql.com/doc/refman/5.5/en/innodb-file-defragmenting.html (This could help, but should not make this much of a difference).
Make sure your query is not being blocked by other queries. If data is being inserted in the track_log table at continuously, those queries might block your query. You can prevent this by changing the transaction isolation level, see https://dev.mysql.com/doc/refman/5.5/en/set-transaction.html for more information. Caution: be carefull with this!
Indexes
I'm guessing this has something to do with the indexes you defined on the tables. Could you post the SHOW CREATE TABLES track_log output and the output of your archive table as well? The query you are executing would require an index on node_id and time_stamp for optimal performance.
Defragmentation
Besides this indexes you defined on the table, this might have something to do with data fragmentation. I'm assuming you are using InnoDB as your table engine now. Depending on your settings, every table in a database is stored in a separate file or every table in the database is stored in a single file (innodb_file_per_table variable). Those files will never shrink in size. If your track_log table has grown to 8.7 million records, on disk, it still takes up space for all those 8.7 million records.
If you have moved records from your track_log table to your archive table, the data might still be at the beginning and the end of the physical file for track_log. If no index is defined at time_stamp, a full table scan is still required to order by the timestamp. This means: reading the complete file from disk. Because the records you deleted still take up space in the file, this could make a difference.
Edit:
Transactions
Other transactions might be blocking your SELECT query. This can happen with the InnoDB engine. If you continously insert a lot of data into your track_log table, those queries might block your query. It will have to wait until no other transactions are being performed at this table.
There is a way around this, but you should be careful with this. You are able to change to transaction isolation level of your query. By setting the transaction isolation level to READ UNCOMMITTED you will be able to read data, while the other inserts are running. But it might not always give you the latest data. If you want to sacrifice this depends on your situation. If you are going to alter the data and update the data later, you generally do not want to change the transaction isolation level. But, for example, when showing statistics which should not always be accurate and up to date, this could be something that really speeds up your query.
I use this myself sometimes when I need to show statistics from large tables which are updated regularly.
This is almost certainly because your archive table has superior indexing to your track_log table.
To satisfy this query efficiently you need a compound index on (node_id, time_stamp) Why does this work? Because InnoDB and MyISAM indexes are so-called BTREE indexes, which means our intuition about searching them in order will work. Your query looks for a specific value of node_id, which means it can jump to that value in the index efficiently. The query then calls for the highest possible value of time_stamp related to that node_id value. Now that's in the same index, and in the right order to access it quickly too. So the row you need can be random-accessed, and MySQL doesn't have to hunt for it by scanning the table row by row. That scanning is almost certainly what's taking the time in your query.
Three things to keep in mind:
One: lots of indexes on single columns can't help a query as much as well-chosen compound indexes. Read this http://use-the-index-luke.com/
Two: SELECT * is usually harmful on a table with as many columns as the one you have shown. Instead, you should enumerate the columns you actually need in your SELECT query. That way MySQL doesn't have to sling as much data.
Three: The DOUBLE datatype is overkill for commercial-grade GPS data. FLOAT is plenty of precision.
Let us analyze your query:
SELECT * FROM track_log WHERE node_id = '26' ORDER BY time_stamp DESC LIMIT 1
The above mentioned query first sorts all the data present in the table based on time_stamp and then returns the top row.
But, when this query is executed on archived table, order by clause might be ignored (based on compression and system setting) and hence it returns the first row it encountered in the table.
You may verify the output of archived table by comparing the result with actual latest row.
I have a problem while designing my database and I don't know how to solve it:
i have following table (relevant columns):
CREATE TABLE IF NOT EXISTS `prmgmt_tasks` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(300) NOT NULL,
`project_id` int(11) NOT NULL,
);
What I want: every task has a unique id (autoincrement). The name of the task is not unique, but it should be unique for each project. For example "Design userinterface" can occur in project with id 1 and 2, but not twice in project with id 1. Something like 'unique for: group by project_id'.
Of course, I could check that in every query, but I am looking for a way to model this in the database, so it will allways be consistent, no matter what queries are executed.
Thanks for help!
Create a unique composite index on the combined fields.
CREATE UNIQUE INDEX tasks_name_project
ON prmgmt_tasks (name, project_id);
I have a MySQL query that sometimes takes over 1 second to execute. The query is as follows:
SELECT `id`,`totaldistance` FROM `alltrackers` WHERE `deviceid`='FT_99000083426364' AND (`gpsdatetime` BETWEEN 1341100800 AND 1342483200) ORDER BY `id` DESC LIMIT 1
This query is run in a loop to retrieve rows on certain days of the month and such. This causes the page to take over 25 seconds to load sometimes...
The table structure is as follows:
CREATE TABLE IF NOT EXISTS `alltrackers` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`deviceid` varchar(50) NOT NULL,
`lat` double NOT NULL,
`long` double NOT NULL,
`gpsdatetime` int(11) NOT NULL,
`version` int(11) DEFAULT NULL,
`totaldistance` int(11) NOT NULL DEFAULT '0',
`distanceprocessed` tinyint(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`id`),
UNIQUE KEY `id_deviceid` (`id`,`deviceid`),
UNIQUE KEY `deviceid_id` (`deviceid`,`id`),
KEY `deviceid` (`deviceid`),
KEY `deviceid_gpsdatetime` (`deviceid`,`gpsdatetime`),
KEY `gpsdatetime_deviceid` (`gpsdatetime`,`deviceid`),
KEY `gpsdatetime` (`gpsdatetime`),
KEY `id_deviceid_gpsdatetime` (`id`,`deviceid`,`gpsdatetime`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 AUTO_INCREMENT=677242 ;
I have added all kinds of index combinations (please tell me which to remove) in order to try and get MySQL to use indices for the query, but to no avail.
Here is the EXPLAIN output:
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE alltrackers index deviceid_id,deviceid,deviceid_gpsdatetime,gpsdatet... PRIMARY 4 NULL 677238 Using where
The reason I'm using ORDER BY ASC/DESC LIMIT 1 is because I need the first and last rows of the query. Would it be faster to just run the query without LIMIT 1 and use PHP to retrieve the first and last rows?
Many thanks for your help!
I can't say for your exact case, but I have found that querying all rows and ignoring all but the first is faster than a limit statement (this was using SQL Server though). Its easy to do - remove your limit 1 clause and give it a try. You can then use the PHP to read the first and last thus reducing the load on the MySQL instance (ie running a single query rather than 2).
Incidentally, why do you have 2 unique keys with the same columns in them - id_deviceid and deviceid_id? Remove all the indexes and then add them back in again, you really want as few indexes as possible for fast DBs.
Couple things I can think of off the top of my head:
1.) I'm not a mySQL/DBA guru by any stretch, but the index there seems like a bit of overkill. Typically I'll make sure that columns that are either queried on or joined on are indexed; so you'd want one for deviceid and gpsdatetime. 1 second per query isn't horrendous, so your returns here might be limited.
2.) Try to eliminate the looping. If you're going back to the database 25 times; you're going to incur overhead simply opening/closing connections and such. It might be faster to go to the database once, and then process the results using PHP to get the final data you need.
I have the following tables;
CREATE TABLE IF NOT EXISTS `tags` (
`tag_id` int(11) NOT NULL auto_increment,
`tag_text` varchar(255) NOT NULL,
PRIMARY KEY (`tag_id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=9 ;
CREATE TABLE IF NOT EXISTS `users` (
`user_id` int(11) NOT NULL auto_increment,
`user_display_name` varchar(128) default NULL,
PRIMARY KEY (`user_id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=10 ;
CREATE TABLE IF NOT EXISTS `user_post_tag` (
`upt_id` int(11) NOT NULL auto_increment,
`upt_user_id` int(11) NOT NULL,
`upt_post_id` int(11) NOT NULL,
`upt_tag_id` int(11) NOT NULL,
PRIMARY KEY (`upt_id`),
KEY `upt_user_id` (`upt_user_id`),
KEY `upt_post_id` (`upt_post_id`),
KEY `upt_tag_id` (`upt_tag_id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=9 ;
CREATE TABLE IF NOT EXISTS `view_post` (
`post_id` int(11)
,`post_url` varchar(255)
,`post_text` text
,`post_title` varchar(255)
,`post_date` datetime
,`user_id` int(11)
,`user_display_name` varchar(128)
);
The idea is that I would like to use the most effective way to save tags, for a post and users. Simply once I add a post I pass few tags along that post and user. Later I would like to be able to count tabs for each user and post. Something very similar to Stack Overflow.
I suppose that the 'tag_text' should be unique? Is if effective that I run a function each time I submit a new post to go through the 'tags' table to check if a tag already exists, and if yes, return its 'tag_id' so I can insert it into 'user_post_tag' table.
Is this maybe a bad approach to tackle this kind of issue.
All suggestions are welcome.
Yes, what you are doing is the best way to do it. You created an n to m relationship, as a post can have multiple tags and the same tag can be on multiple posts. You do not want to store the tag name for each of the posts, so you store the id.
But, you should -NOT- have this redudancy of storing multiple times the same tag_id for the same user. It will hit hard your server if the users have multiple tags and you have to execute SELECT count(...) for each of these tags. Do you understand what I'm talking about here? Because right now, how would get how many times the user A has the tag B? You'd have to do SELECT count(*) FROM user_post_tag INNER JOIN tags ON (...) WHERE user_id=A and tag_id=B.
My suggestion is to split user_post_tag into two tables:
user_tags, to count how many times the user has this tag, primary key would be user_id and tag_id and you'd have a count field, which you would just update with count=count+1 everytime this user makes a new post with the tag. This way, you can simply do SELECT tag_text, count FROM user_tags INNER JOIN tags ON (...) WHERE user_id=A to select all tags (with number of times used) of a given user. You're using a fully indexed query. You're not asking MySQL to go over the table, look for a bunch of rows and count them, you're telling to MySQL, go this row at this table and at the other table, join them and give it to me, fast!
post_tags, to store the tags a certain post have, primary key would be post_id and tag_id, no additional fields needed.
I suppose that the 'tag_text' should
be unique? Is if effective that I run
a function each time I submit a new
post to go through the 'tags' table to
check if a tag already exists, and if
yes, return its 'tag_id' so I can
insert it into 'user_post_tag' table.
Yes, it should be unique. It's way better to check if a tag exists before inserting and inserting if it doesn't than having redundancy and having to do SELECT ... count(*) to know how much times the tag has been used. It will be much mess less frequent post creation than post selection, so if you have to pick between being query intensive on insertion and selection, certainly pick insertion.
By the way, if you'd like to have a count of how many posts have the same tag, like in stack overflow, you'd need another table, with primary key tag_id, and then, like on user_tags, you increment the count field everytime a post gets a certain tag.
Hmmm, if your tags are all unique, then you don't need tag_id and tag_text in the tags table. Just use tag_text and make it the primary key. Then look at REPLACE INTO (http://dev.mysql.com/doc/refman/5.0/en/replace.html) to handle new tags.
Associating tags with users or posts? user_tags table and post_tags table. no auto-increment values just a compound key with user_id and tag_text or post_id and tag_text. I don't know if you're looking at the user_post_tags table for a performance increase over joining a post_tags table with posts and users. Still, "replace into" should be your friend here too.