I have a performance and best-practice question concerning mysql tables.
I´m working on an application which connects to a database which gets filled by other programms.
This system is deployed in differnet locations, and from location to location the name of some databases-tables can change (but the fields in this tables etc stay the same).
As I don´t want to change all sql querys in my application for every location, I thought about creating a mysql view which simply mirrors the contents of this table to the normaly used table-name.
Is this a suitable solution, or could it get awfully slow with big tables?
Simple views (created as SELECT * FROM table) behave like the specified table performance wise.
It should be a suitable solution for your case.
mmm, this is tricky. If there are multiple tables then a quick and dirty version for this would be something like
SELECT * FROM (SELECT * FROM table1
Union
SELECT * FROM table2
Union
SELECT * FROM table3) t
Which I think will work. You will of course have problems with pagination, sorting and searching - because you will have to try and do this over 3 or more tables.
Another way would be this
Create a table with the table names and a counter
ImportTable
name
id
Now in this you can enter the names of the tables and the last id that you want to import from.
Create another table to import the records
TableRecords
source
id
field1
field2
etc
Now run something that goes through the tables in ImportTable grabs any new records and shoves them into `TableRecords.
Now this becomes really simply you can query TableRecords and have pagination sorting and searching with no of the previous troubles.
Make something that runs this every 2 minutes say so TableRecords will be 2 mins behind but everything will be really easy and run like a dream.
Related
I'm using PHP to access a MySql database with ~50 tables. Almost every table has a field for "YearID". My app will be starting its second year soon and I'll just keep adding records to the tables with the new YearID. Currently, my queries do not include YearID as a constraint because there is only a single year's data stored (and I wasn't thinking far enough ahead!).
I want users to be able to access previous years' data. One way to do this, obviously, is to update all my queries throughout the site to include the YearID.
Before I do that, I was wondering if there is some dynamic setting in MySql that would implicitly limit all queries to all tables to only records with a particular YearID.
For example, one query now looks like this:
SELECT distinct RubricPK, tas.DisplayOrder, s.IDENTIFIER
FROM Rubric r
JOIN TeacherAssessedStandards2 tas ON tas.STANDARDID=r.EL_StandardID AND tas.SLorEL="EL" AND
tas.SectionDCID = ?
JOIN Standards s ON s.StandardID = tas.ParentStandardID
ORDER BY tas.DisplayOrder, s.IDENTIFIER
There is no YearID included in this query.
Is there a MySQL setting I can change with PHP so that the query above will only pull from records in each table where YearID=2020?
Or should I get to work editing all my queries to include a YearID constraint?
There is no way to achieve that directly via a setting. What you can do is write a few lines of $scripting_language to find all the tables via the information_schema and create a VIEW based on each one in a separate database. For example:
CREATE VIEW DB2020.table_name AS SELECT * FROM MainDB.table_name WHERE YearID = '2020';
Then give your users SELECT access only to DB2020 database.
I have been trying to create a database for fun to get a better understanding of databases and using PHP to query them for a website I'm messing around with. Pretty much I have one database with 4 tables when a user enters a search term in a PHP search box my code searches the database for any entries containing the search term. Now I can easily get my code to search individual tables, but I cannot seem to get it to search all 4 tables and display the results on the same page.
info: making a database for skyrim
Table names: classes, powers, skills, shouts
column names: name, information
Here is a snippet of the code I have that works so far:
$raw_results = mysql_query("
SELECT *
FROM `xaviorin_skyrim`.`shouts` , `xaviorin_skyrim`.`classes`
WHERE (CONVERT(`UID` USING utf8) LIKE '%".$query."%' OR
CONVERT(`Name` USING utf8) LIKE '%".$query."%' OR
CONVERT(`Information` USING utf8) LIKE '%".$query."%')
") or die(mysql_error());`
Literally all I thought I would need to do is change the table name from "shouts" to say "classes" in a new raw_results line of code but that didn't work. I have attempted unions and joins and either keep screwing them up or just don't understand how to properly format them.
echo "<p><h3>".$results['Name']."</h3>".$results['Information']."</p>";
The code above this text is what displays the results on the page on my website... it works but I don't know how to combine the information from all 4 tables into one page. If I'm going about this in the wrong way and anyone can point me in the right direction I would GREATLY appreciate it... I've been trying to research the problem without finding a proper answer for near a month now.
The problem with your approach is that relational databases do a cross join when there are several query results from two different tables. So basically every match in one table will be combined with every match from the second table. When you have 3 entries in the first and 4 in the second table, you will get 3 * 4 = 12 entries in your query result. If you add more tables, you get even more results. You want to do a full text search in several tables that are totally unrelated, thus creating some kind of non-existing relation via cross joining them will not be useful.
What you actually want to do is a UNION ALL (UNION is slower because it prunes duplicates) of several queries:
SELECT name, information, 'shouts' AS tablename FROM shouts WHERE ...
UNION ALL
SELECT name, information, 'classes' AS tablename FROM classes WHERE ...
This will do search queries on every single table and then place the results in a single result. Also note that I added a third column to each query to ensure that the originating table is not lost after merging the results.
Unless you need to do some sorting afterwards, I would suggest that you do all statements separately. Combining them this way will most likely make the post-processing more complex. And several single queries will also be faster than one big query with UNION statements.
And as I mentioned in the comments: Don't use mysql_* functions!
I have a question related to a web app that I developed in PHP, MYSQL.
basically part 1 is :
I display results in the form of table say for software testing.
ID Prod_Name Set Date Result Platform
1 Alpha1 Pro1 01.01.01 PASS 2.3.1.2_OS
Now, I have divided the tables accordingly
Table Name: Results
ID, Name, Date, Result
Table Name : Set
ID, Set_Name, Prod_name
Table Name : Platform
ID, Platform_Name, Set_Name
Now, ID in each table is an incremented value and does not relate to anything else.
My php app, starts with fetching the results from 'Results' table. Since I want SET to be displayed for every row, I am making an another connection to the database and using the query
select Set_name
from Set
where Prod_name = row['Name'] // row['Name'] is fetched from the results table.
now I also want to display platform which I am extracting it from Platform table using the above method i.e making another connection and passing Set_Name = row['Set_Name'] from the Set table.
Now for my application is there any other way to achieve the same result ?
Typically, for large web based applications, if data is coming from a database server is making multiple connection to a DB server a feasible option?
Please do not consider the fact that with MySQL declaring a connection statement once will do the needful but what about MSSQL server? Do we need to write a long sql statement with several joins/selfjoins/unions and use those variables all over the application?
How is the application design for this case will be?
Can anyonce give me some ideas please?
Thanks.
For pretty much any flavour of database, a single SELECT statement which joins three tables will perform better than three separate statements querying a table apiece. Joining is what relational databases do.
I may not have understood everything, but here is something similar. First, let's make an ER model.
Now, because you don't seem to like joins, create a view in the database.
CREATE VIEW v_test AS
SELECT TestID, ProductName, TestName, Date, Result, PlatformName
FROM Product AS p
JOIN Test AS t ON t.ProductID = p.ProductID
JOIN Platform AS f ON f.PlatformID = t.PlatformID;
With this in place, you can simply use:
SELECT * FROM v_test WHERE ProductName = 'Alpha1'
You may also take a look at this question/answer with a similar scenario.
I'm running a sql query to get basic details from a number of tables. Sorted by the last update date field. Its terribly tricky and I'm thinking if there is an alternate to using the UNION clause instead...I'm working in PHP MYSQL.
Actually I have a few tables containing news, articles, photos, events etc and need to collect all of them in one query to show a simple - whats newly added on the website kind of thing.
Maybe do it in PHP rather than MySQL - if you want the latest n items, then fetch the latest n of each of your news items, articles, photos and events, and sort in PHP (you'll need the last n of each obviously, and you'll then trim the dataset in PHP). This is probably easier than combining those with UNION given they're likely to have lots of data items which are different.
I'm not aware of an alternative to UNION that does what you want, and hopefully those fetches won't be too expensive. It would definitely be wise to profile this though.
If you use Join in your query you can select datas from differents tables who are related with foreign keys.
You can look of this from another angle: do you need absolutely updated information? (the moment someone enters new information it should appear)
If not, you can have a table holding the results of the query in the format you need (serving as cache), and update this table every 5 minutes or so. Then your query problem becomes trivial, as you can have the updates run as several updates in the background.
I want to list the recent activities of a user on my site without doing too many queries. I have a table where I list all the things the user did with the date.
page_id - reference_id - reference_table - created_at - updated_at
The reference_id is the ID I need to search for in the reference_table (example: comments). If I would do a SELECT on my activity table I would then have to query:
SELECT * FROM reference_table where id = reference_id LIMIT 1
An activity can be a comment, a page update or a subscription. Depending which one it is, I need to fetch different data from other tables in my database
For example if it is a comment, I need to fetch the author's name, the comment, if it is a reply I need to fetch the orignal comment username, etc.
I've looked into UNION keyword to union all my tables but I'm getting the error
1222 - The used SELECT statements have a different number of columns
and it seems rather complicated to make it work because the amount of columns has to match and none of my table has the same amount of tables and I'm not to fond of create column for the fun of it.
I've also looked into the CASE statement which also requires the amount of columns to match if I remember correctly (I could be wrong for this one though).
Does anyone has an idea of how I could list the recent activities of a user without doing too many queries?
I am using PHP and MySQL.
You probably want to split out the different activities into different tables. This will give you more flexiblity on how you query the data.
If you choose to use UNION, make sure that the you use the same number of columns in each select query that the UNION is comprised of.
EDIT:
I was down-voted for my response, so perhaps I can give a better explanation.
Split Table into Separate Tables and UNION
I recommended this technique, because it will allow you to be more explicit about the resources for which you are querying. Having a single table for inserting is convenient, but you will always have to do separate queries to join with other tables to get meaningful information. Also, you database schema will be obfuscated by a single column being a foreign key for different tables depending on the data stored in that row.
You could have tables for comment, update and subscription. These would have their own data which could be queried on individually. If, say, you wanted to look at ALL user activity, you could somewhat easily use a UNION as follows:
(SELECT 'comment', title, comment_id AS id, created FROM comment)
UNION
(SELECT 'update', title, update_id as id, created FROM update)
UNION
(SELECT 'subscription', title, subscription_id as id, created
FROM subscription)
ORDER BY created desc
This will provide you with a listing view. You could then link to the details of each type or load it on an ajax call.
You could accomplish this with the method that you are currently using, but this will actually eliminate the need for the 'reference_table' and will accomplish the same thing in a cleaner way (IMO).
The problem is that UNION should be used just to get similar recordsets together. If you try to unify two different queries (for example, with different columns being fetched) it's an error.
If the nature of the queries is different (having different column count, or data types) you'll need to make several different queries and treat them all separately.
Another approach (less elegant, I guess) would be LEFT JOINing your activities table with all the others, so you'll end up with a recordset with a lot of columns, and you'll need to check for each row which columns should be used depending on the activity nature.
Again, I'd rather stick with the first one, since the second procudes a rather sparse recorset.
With UNION you don't have to get all of the columns from each table, just as long as all of the columns have the same datatypes.
So you could do something like this:
SELECT name, comment as description
FROM Comments
UNION
SELECT name, reply as description
FROM Replies
And it wouldn't matter if Comments and Replies have the same number of columns.
This really depends on the amount of traffic on your site. The union approach is a straightforward and possibly the correct one, logically, but you'll suffer on the performance if your site is heavily loaded since the indexing of a UNIONed query is hard.
Joining might be good, but again, in terms of performance and code clarity, it's not the best of ways.
Another totally different approach is to create an 'activities' table, which will be updated with activity (in addition to the real activity, just for this purpose). In old terms of DB correctness, you should avoid this approach since it will create duplicate data on your system, I, however, found it very useful in terms of performance.
[Another side note about the UNION approach if you decide to take it: if you have difference in parameters length, you can SELECT bogus parameters on some of the unions, for example.. (SELECT UserId,UserName FROM users) UNION (SELECT 0,UserName from notes)