I need to JOIN 2 tables (lets say User & Order table) for reporting module in my web app.
The problems are:
The User table is located on the different server & different
DBMS from the Order table. Technically it is a different system, so the User table is located on SQL Server DB, meanwhile the Order table is located on MySQL DB.
I couldn't use SQL Server's Linked Server because my company policy doesn't allow it. So, I coudn't JOIN them directly with SQL code. They want me to use Web Service instead of linked server.
The result of JOIN operation from those tables has a large number of rows (maybe more than 10,000 rows because the data aimed for reporting). So, I think it was a horrible thing to mapping them using Web Service.
So I came up with this:
I collected 2 query result from different models and join them with my app code (I'm using PHP with CodeIgniter) :
// First result
$userData = $this->userModel->getAllUser();
// Second result
$orderData = $this->orderModel->getAllOrder();
The $userData contains all user entities with the following columns:
[UserId, Username, Address, PhoneNumber, etc..]
And the $orderData contains all order entities with the following columns:
[OrderId, UserId, Date, etc..]
But is it possible to join those two query results in PHP / CodeIgniter?
How about the performance regarding the large amount of data?
Should I just use Web Service as suggested or there's another solution to accomplish this?
Thanks in advance :)
A few things to think about:
Do you actually need to return all user and order records in one single go
Do you actually want to return all rows for these two types of record
Would you be better off with a Report module for these report queries?
Would plain SQL syntax be a smarter move than trying to shim this into existence with the CodeIgniter "Active Record" (renamed Query Builder in 3.0)
Is JOIN really so bad? It is not a UNION, you want the data to be related.
I would recommend you limit your data returns, SELECT only the fields you actually require, make a new Report model to avoid trying to mess up your generic models and do this with raw SQL.
Complicated things get all the more complicated when you try too hard to stick to rules like "1 table = 1 model" and "User::getAllFoos + controller processing > Report::getMonthlyOrderStats()".
Related
I'm working on an existing application that uses some JOIN statements to create "immutable" objects (i.e. the results are always JOINed to create a processable object - results from only one table will be meaningless).
For example:
SELECT r.*,u.user_username,u.user_pic FROM articles r INNER JOIN users u ON u.user_id=r.article_author WHERE ...
will yield a result of type, let's say, ArticleWithUser that is necessary to display an article with the author details (like a blog post).
Now, I need to make a table featured_items which contains the columnsitem_type (article, file, comment, etc.) and item_id (the article's, file's or comment's id), and query it to get a list of the featured items of some type.
Assuming tables other than articles contain whole objects that do not need JOINing with other tables, I can simply pull them with a dynamicially generated query like
SELECT some_table.* FROM featured_items RIGHT JOIN some_table ON some_table.id = featured_items.item_id WHERE featured_items.type = X
But what if I need to get a featured item from the aforementioned type ArticleWithUser? I cannot use the dynamically generated query because the syntax will not suit two JOINs.
So, my question is: is there a better practice to retrieve results that are always combined together? Maybe do the second JOIN on the application end?
Or do I have to write special code for each of those combined results types?
Thank you!
a view can be thot of as like a table for the faint of heart.
https://dev.mysql.com/doc/refman/5.0/en/create-view.html
views can incorporate joins. and other views. keep in mind that upon creation, they take a snapshot of the columns in existence at that time on underlying tables, so Alter Table stmts adding columns to those tables are not picked up in select *.
An old article which I consider required reading on the subject of MySQL Views:
By Peter Zaitsev
To answer your question as to whether they are widely used, they are a major part of the database developer's toolkit, and in some situations offer significant benefits, which have more to do with indexing than with the nature of views, per se.
I'll spare the back story, but basically I have two different databases that I retrieve information from. 1 being Oracle (which we've only been given read-only access to) and the other being Mysql (which we have full access to). My problem is there is column in the oracle table that is missing that we would like to pull data from, however that column we need is in the mysql table. Is there a way that I can write a join statement to include this mysql column to the oracle table? I've seen instances online where people have used 'LinkedServers', but I wasn't sure how that worked working in PHP. The DB connections are stored in a php file and the function is called when we want to use that particular database.
tl;dr
Can a user use a join for a mysql and oracle table? if so, how can it be implemented in php when the db info is in a separate php file?
I imagine the sql query would look similar to this:
select
*
from
LocalTable,
[OtherServerName].[OtherDB].[dbo].[OtherTable]
but with some join logic
You cannot join 2 different database vendors.
A query is executed by a specific vendor query analyzer.
You can however do the join itself IN php, but would require to dump a lot of info into php. Should be okay for small data sets not okay for large ones...
What you would want to do is copy the data from one vendor into a temp table in the other then execute the join.
For the purpose of this example I will use PDO.
I am going to take the oracle data and put it into mysql so that I can use a join on that data...
$oracle_table_b = "select * from mytable";
$oracle_data = $pdo->fetchAll();
$stmt = $pdo_mysql->prepare("insert into mytemptable (mycols)");
foreach ($oracle_data as $row) {
$stmt->execute($row);
}
//Then run your join
I'm running a sort of forum modification code on my server. In my database, I have a HUGE table called core, which basically includes all the admin-editable settings, of which there are a lot. My question is, should I normalize this?
I've created a fiddle to show you how large this table is: http://sqlfiddle.com/#!2/f4536/1
You'll see some columns called gear_notifications, gear_bank, gear_* etc. These indicate whether a certain system is turned on. For example, if gear_bank=1, then the Bank System is turned on. At the moment, in the __construct of my DB file, I run the following query:
$settings = mysql_query("SELECT GROUP_CONCAT(d.domain) AS domains, GROUP_CONCAT(d.zb) AS zb_info, c.* FROM core c JOIN domains d ON d.cid = c.id WHERE c.id ='$cid' LIMIT 1");
Ignoring the JOIN you can straight away see a problem here; the query returns EVERY field from the core table, regardless of whether the corresponding system is turned on. For example, if gear_bank=0, the query will still return bank_name, bank_history_perpage, bank_* etc. While this doesn't present any physical problem with how the code runs (as it can just ignore any data it does not need), I'd much prefer if it didn't have to load that data.
Would I be better off creating one table called core which has all the gear_* values, then corresponding tables (core_bank, core_* etc) for their corresponding values?
I've been querying this (bad pun, sorry!) for a while now and I just don't know enough to work out whether this will provide a performance boost for my code. Note that the above query will be run on EVERY page.
If I were to revert to the new system, i.e the multiple tables, how would I get all the information I need? I don't want to run one query to core to work out which systems are turned on, then subsequently run an extra query on all corresponding tables. I'd much prefer one query which JOINs all necessary tables based on the values of gear_* in core. Would that be possible?
I've not yet released this forum modification so I can make as many changes as I like without any real-world impact :) .
If you split your core into multiple tables you can then use NULL to indicate if a particular entry has info for a particular system. The joins will then work as expected.
However, your table really is not that big and it's doubtful that you will actually notice any speed improvement at the application level by breaking it up.
And if you don't need to query on individual columns then just use one TEXT column, put all your attributes in an array and then serialize the array into the text column.
I have a question related to a web app that I developed in PHP, MYSQL.
basically part 1 is :
I display results in the form of table say for software testing.
ID Prod_Name Set Date Result Platform
1 Alpha1 Pro1 01.01.01 PASS 2.3.1.2_OS
Now, I have divided the tables accordingly
Table Name: Results
ID, Name, Date, Result
Table Name : Set
ID, Set_Name, Prod_name
Table Name : Platform
ID, Platform_Name, Set_Name
Now, ID in each table is an incremented value and does not relate to anything else.
My php app, starts with fetching the results from 'Results' table. Since I want SET to be displayed for every row, I am making an another connection to the database and using the query
select Set_name
from Set
where Prod_name = row['Name'] // row['Name'] is fetched from the results table.
now I also want to display platform which I am extracting it from Platform table using the above method i.e making another connection and passing Set_Name = row['Set_Name'] from the Set table.
Now for my application is there any other way to achieve the same result ?
Typically, for large web based applications, if data is coming from a database server is making multiple connection to a DB server a feasible option?
Please do not consider the fact that with MySQL declaring a connection statement once will do the needful but what about MSSQL server? Do we need to write a long sql statement with several joins/selfjoins/unions and use those variables all over the application?
How is the application design for this case will be?
Can anyonce give me some ideas please?
Thanks.
For pretty much any flavour of database, a single SELECT statement which joins three tables will perform better than three separate statements querying a table apiece. Joining is what relational databases do.
I may not have understood everything, but here is something similar. First, let's make an ER model.
Now, because you don't seem to like joins, create a view in the database.
CREATE VIEW v_test AS
SELECT TestID, ProductName, TestName, Date, Result, PlatformName
FROM Product AS p
JOIN Test AS t ON t.ProductID = p.ProductID
JOIN Platform AS f ON f.PlatformID = t.PlatformID;
With this in place, you can simply use:
SELECT * FROM v_test WHERE ProductName = 'Alpha1'
You may also take a look at this question/answer with a similar scenario.
I have an array of user ids in a query from Database A, Table A (AA).
I have the main user database in Database B, Table A (BA).
For each user id returned in my result array from AA, I want to retrieve the first and last name of that user id from BA.
Different user accounts control each database. Unfortunately each login cannot have permissions to each database.
Question: How can I retrieve the firsts and lasts with the least amount of queries and / or processing time? With 20 users in the array? With 20,000 users in the array? Any order of magnitude higher, if applicable?
Using php 5 / mysql 5.
As long as the databases are on the same server just use a cross database join. The DB login being used to access the data will also need permissions on both databases. Something like:
SELECT AA.userID, BA.first, BA.last
FROM databasename.schema.table AA
INNER JOIN databasename.schema.table BA ON AA.userID = BA.userID
In response to comments:
I don't believe I read the part about multiple logins correctly, sorry. You cannot use two different mySQL logins on one connection. If you need to do multiple queries you really only have three options. A) Loop through the first result set and run multiple queries. B) Run a query which uses a WHERE clause with userID IN (#firstResultSet) and pass in the first result set. C) Select everything out of the second DB and join them in code.
All three of those options are not very good, so I would ask, why can't you change user permissions on one of the two DBs? I would also ask, why would you need to select the names and IDs of 20,000 users? Unless this is some type of data dump, I would be looking for a different way to display the data which would be both easier to use and less query intensive.
All that said, whichever option you choose will be based on a variety of different circumstances. With a low number of records, under 1,000, I would use option B. With a higher number of records, I would probably use options C and try to place the two result sets into something that can be joined (such as using array_combine).
I think they key here is that it should be possible in two database calls.
Your first one to get the id's from database A and the second one to pass them to database B.
I don't know mysql, but in sqlserver I'd use the xml datatype and pass all of the ids into a statement using that. Before the xml datatype I'd have built up some dynamic SQL with the id's in an IN statement.
SELECT UserId FROM DatabaseA.TableA
Loop through id's and build up a comma separated string.
"SELECT FirstName, Surname FROM DataBaseB.TableA WHERE UserId IN(" + stringId + ")"
The problem with this is that wth 20,000 id's you may have some performance issues with the amount of data you are sending. This is where'd I'd use the XML datatype, so maybe look at what alternatives mysql has for passing lists of ids.