I have a database where two of the tables have columns called Building Code. It is set as VARCHAR(5) in both of them, and sometimes the value is purely digits (ex: 0001) and sometimes it has a letter in it (ex: 0170A)
The two different tables in this database, are called "buildings", and "newconnections", of which I am trying to perform an inner join on.
In one of the tables, when I search for 0001 in phpmyadmin I get no results, but if I search for 0170A I get the correct results. In the other table, both work correctly. This issue has been affecting me as I have been trying to do an inner join on these data sets.
I have tried using LTRIM, RTRIM (this did not work), I tried casting as VARCHAR(5) (although admittedly this query just wasn't producing results so I may had done it wrong).
Here is the inner join I am trying to perform if it helps:
"SELECT BUILDING_CODE, buildings.BuildingName, buildings.Latitude, buildings.Longitude, FLOOR1, FLOOR2, FLOOR3
FROM buildings LEFT JOIN
(SELECT BUILDING_CODE,
SUM(IF(FLOOR_NUMBER = '1', FLOORCONNECTIONS, 0) ) AS FLOOR1,
SUM(IF(FLOOR_NUMBER = '2', FLOORCONNECTIONS, 0) ) AS FLOOR2,
SUM(IF(FLOOR_NUMBER = '3', FLOORCONNECTIONS, 0) ) AS FLOOR3
FROM
(
SELECT
BUILDING_CODE, FLOOR_NUMBER, COUNT(*) AS FLOORCONNECTIONS
FROM newconnections GROUP BY BUILDING_CODE, FLOOR_NUMBER
) as totals
GROUP BY BUILDING_CODE
) as r ON RTRIM(BUILDING_CODE) = RTRIM(buildings.BuildingCode)";
For instance, on the table that is giving me troubles:
SELECT * FROM newconnections WHERE BUILDING_CODE = '0170A'
returns results but
SELECT * FROM newconnections WHERE BUILDING_CODE = '0001'
returns nothing.
On the working table:
SELECT * FROM newconnections WHERE BuildingCode= '0170A'
and
SELECT * FROM newconnections WHERE BuildingCode= '0001'
return results
EDIT: A little more information. I actually generated a lot of this data using a script that I created and was inserting it into a text file and then inserting it into a database from that file. I guess there must be some weird issue with how it was printing certain numbers - I put my data this time into excel and exported it into a table from there and I added that table to my database and the queries run fine now, so case closed (although I wish I could figure out how to actually fix this problem for future references).
Related
I have a database that is already in use and I have to improve the performance of the system that's using this database.
There are 2 major queries running about 1000 times in a loop and this queries have inner joins to 3 other tables each. This in turn is making the system very slow.
I tried actually to remove the query from the loop and fetch all the data only once and process it in PHP. But this is putting to much load on the memory (RAM) and the system is hanging if 2 or more clients try to use the system.
There is a lot of data in the tables even after removing the expired data .
I have attached the query below.
Can anyone help me with this issue ?
select * from inventory
where (region_id = 38 or region_id = -1)
and (tour_opp_id = 410 or tour_opp_id = -1)
and room_plan_id = 141 and meal_plan_id = 1 and bed_type_id = 1 and hotel_id = 1059
and FIND_IN_SET(supplier_code, 'QOA,QTE,QM,TEST,TEST1,MQE1,MQE3,PERR,QKT')
and ( ('2014-11-14' between from_date and to_date) )
order by hotel_id desc ,supplier_code desc, region_id desc,tour_opp_id desc,inventory.inventory_id desc
SELECT * ,pinfo.fri as pi_day_fri,pinfoadd.fri as pa_day_fri,pinfochld.fri as pc_day_fri
FROM `profit_markup`
inner join profit_markup_info as pinfo on pinfo.profit_id = profit_markup.profit_markup_id
inner join profit_markup_add_info as pinfoadd on pinfoadd.profit_id = profit_markup.profit_markup_id
inner join profit_markup_child_info as pinfochld on pinfochld.profit_id = profit_markup.profit_markup_id
where profit_markup.hotel_id = 1059 and (`booking_channel` = 1 or `booking_channel` = 2)
and (`rate_region` = -1 or `rate_region` = 128)
and ( ( period_from <= '2014-11-14' and period_to >= '2014-11-14' ) )
ORDER BY profit_markup.hotel_id DESC,supplier_code desc, rate_region desc,operators_list desc, profit_markup_id DESC
Since we have not seen your SHOW CREATE TABLES; and EXPLAIN EXTENDED plan it is hard to give you 1 answer
But generally speaking in regard to your query "BTW I re-wrote below"
SELECT
hotel_id, supplier_code, region_id, tour_opp_id, inventory_id
FROM
inventory
WHERE
region_id IN (38, -1)
AND tour_opp_id IN (410, -1)
AND room_plan_id IN (141, 1)
AND bed_type_id IN (1, 1059)
AND supplier_code IN ('QOA', 'QTE', 'QM', 'TEST', 'TEST1', 'MQE1', 'MQE3', 'PERR', 'QKT')
AND ('2014-11-14' BETWEEN from_date AND to_date )
ORDER BY
hotel_id DESC, supplier_code DESC, region_id DESC, tour_opp_id DESC, inventory_id DESC
Do not use * to get all the columns. You should list the column that you really need. Using * is just a lazy way of writing a query. limiting the columns will limit the data size that is being selected.
How often is the records in the inventory are being updates/inserted/delete? If not too often then you can use consider using SQL_CACHE. However, caching a query will cause you problems if you use it and the inventory table is updated very often. In addition, to use query cache you must check the value of query_cache_type on your server. SHOW GLOBAL VARIABLES LIKE 'query_cache_type';. If this is set to "0" then the cache feature is disabled and SQL_CACHE will be ignored. If it is set to 1 then the server will cache all queries unless you tell it not too using NO_SQL_CACHE. If the option is set to 2 then MySQL will cache the query only where SQL_CACHE clause is used. here is documentation about query_cache_type
If you have an index on those following column in this order it will help you (hotel_id, supplier_code, region_id, tour_opp_id, inventory_id)
ALTER TABLE inventory
ADD INDEX (hotel_id, supplier_code, region_id, tour_opp_id, inventory_id);
If possible increase sort_buffer_size on your server as most likely you issue here is that your are doing too much sorting.
As for the second query "BTW I re-wrote below"
SELECT
*, pinfo.fri as pi_day_fri,
pinfoadd.fri as pa_day_fri,
pinfochld.fri as pc_day_fri
FROM
profit_markup
INNER JOIN
profit_markup_info AS pinfo ON pinfo.profit_id = profit_markup.profit_markup_id
INNER JOIN
profit_markup_add_info AS pinfoadd ON pinfoadd.profit_id = profit_markup.profit_markup_id
INNER JOIN
profit_markup_child_info AS pinfochld ON pinfochld.profit_id = profit_markup.profit_markup_id
WHERE
profit_markup.hotel_id = 1059
AND booking_channel IN (1, 2)
AND rate_region IN (-1, 128)
AND period_from <= '2014-11-14'
AND period_to >= '2014-11-14'
ORDER BY
profit_markup.hotel_id DESC, supplier_code DESC, rate_region DESC,
operators_list DESC, profit_markup_id DESC
Again eliminate the use of * from your query
Make sure that the following columns have the same type/collation and same size. pinfo.profit_id, profit_markup.profit_markup_id, pinfoadd.profit_id, pinfochld.profit_id and each one have to have an index on every table. If the columns have different types then MySQL will have to convert the data every time to join the records. Even if you have index it will be slower. Also, if those column are characters type (ie. VARCHAR()) make sure they are of the CHAR() with a collation of latin1_general_ci as this will be faster for finding ID, but if you are using INT() even better.
Use the 3rd and 4th trick I listed for the previous query
Try using STRAIGHT_JOIN "you must know what your doing here or it will bite you!" Here is a good thread about this When to use STRAIGHT_JOIN with MySQL
I hope this helps.
For the first query, I am not sure if you can do much (assuming you have already indexed the fields you are ordering by) apart from replacing the * with column names (Don't expect this to increase the performance drastically).
For the second query, before you go through the loop and put in selection arguments, you could create a view with all the tables joined and ordered then make a prepared statement to select from the view and bind arguments in the loop.
Also, if your php server and the database server are in two different places, it is better if you did the selection through a stored procedure in the database.
(If nothing works out, then memcache is the way to go... Although I have personally never done this)
Here you have increase query performance not an database performance.
For both queries first check index is available on WHERE and ON(Join) clause columns, if index is missing then you have to add index to improve query performance.
Check explain plane before create index.
If possible show me the explain plane of both query that will help us.
I try to use different columns within different tables.
Like I want it to run the query If or Where [table.column]
users.username = 'ExampleUsername' AND users.cardnumber = ''
I don't think I can use NULL instead of '', because its an empty text string?
users.cardnumber = NULL
Anyways, I couldn't come further as this:
INSERT INTO users (cardnumber, hasone)
WHERE users.username = 'ExampleName' AND users.cardnumber = ''
SELECT number, sethasone
FROM cards
WHERE cards.used = '0'
LIMIT 1
I'm a bit of new with SQL, but after I got it right I could put the code into my php script.
-
SOLVED! :
I've used two queries for each column.
update users
set hasone=(select sethasone from cards where used='0' LIMIT 1)
where username='TestUser'
and
update users
set cardnumber=(select number from cards where used='0' LIMIT 1)
where username='TestUser'
then I just deleted the row from cards and I was done.
delete from cards
where used = '1'
LIMIT 1
I gave the user a cardnumber from the table cards and delete that row in cards.
I think you are trying to write a nested query but you didn't know how to write it. If you want to write select query within insert or update query so before doing this Click here to read about sub-query or nested query.
Well, I think that you're trying to re-create a JOIN between 2 table. What you need to do is to add a "card_id" field into the users table. Then to get the user AND the card you can do something like :
SELECT * FROM users u LEFT JOIN cards c ON c.id = u.card_id
I have to run this Mysql query on my website to fetch huge amount of data: (3 tables , each with 100,000 + records)
SELECT on_resume.*, on_users.subscribed, on_users.user_avatar, on_resume_page.*
FROM on_resume
LEFT JOIN on_users ON (on_resume.resume_userid = on_users.user_id )
LEFT JOIN on_resume_page ON ( on_resume.resume_userid = on_resume_page.resume_userid)
WHERE on_resume.active= '1'
GROUP BY on_resume.rid
ORDER BY on_resume.rid DESC
LIMIT 0,18
The time I run this at Phpmyadmin sql section, the whole mysqld service will be down and needs to be restarted.
Now I was testing this query and I found out if I don't use Group by and Order by conditions the query will be fine.
SELECT on_resume.*, on_users.subscribed, on_users.user_avatar, on_resume_page.*
FROM on_resume
LEFT JOIN on_users ON (on_resume.resume_userid = on_users.user_id )
LEFT JOIN on_resume_page ON ( on_resume.resume_userid = on_resume_page.resume_userid)
WHERE on_resume.active= '1'
LIMIT 0,18
Showing rows 0 - 17 ( 18 total, Query took 0.4248 sec)
Why is it like this and how can I fix it?...
NOTE : I have tested the SQL query with group by or Order by alone in either case , even with one of them still the query fails and hangs the server.
EDIT : This problem is solved by making column on_resume_page.resume_userid indexed.
This is what i was told, took a while to figure it out:
Look at #jer in Chicago comment
Remember, when there is a GROUP BY clause, there are certain rules that apply for grouping columns. One of those rules is "The Single-Value Rule" -- every column named in the SELECT list must also be a grouping column unless it is an argument for one of the set functions. MySQL extends standard SQL by allowing you to use columns or calculations in a SELECT list that don't appear in a GROUP BY clause. However, we are warned not to use this feature unless the columns you omit from the GROUP BY clause are not unique in the group because you will get unpredictable results.
How to make this code worked in Kohana3 framework? I have a syntax problem with it.
ORM::factory('table1')
->where ( 'id', '=', ceil( DB::expr('rand()') * [SELECT max(id) from table1] ) )
->find();
That's how I want to select a random row from the table.
This works fine for small tables (that contains than 1000 rows), but not for big tables:
ORM::factory('table1')
->order_by(DB::expr('RAND()'))
->find();
The standard mysql equivalent of what I want is something like this:
SELECT name
FROM table1 JOIN
(SELECT CEIL(RAND() *
(SELECT MAX(id)
FROM table1)) AS id
) AS r2
USING (id);
So, how to convert that into a working code for Kohana3 framework?
P.S. This method works fine if there are no holes (no deleted rows) in the table, and that's fine in my case.
What you are trying to do is really a problem with MySQL. You need the right query to have it perform for any size table. A more scalable implementation is to use PHP to generate a random number less than the max ID and then select on that.
Get the highest ID:
SELECT MAX(id) max_id FROM table1
In PHP get the "id" to select:
$rand_id = mt_rand(0, $max_id-1);
Then create your query to select a random record, using a LIMIT 1 so it works even if there are holes.
SELECT * FROM table1 WHERE id>=$rand_id ORDER BY id LIMIT 1
Doing it all in MySQL can be done, but it's not going to be easy to read or implement with a DB abstraction layer.
I'm trying to create a query that will select all dates between two dates
This is my query:
$query = "SELECT DISTINCT * FROM D1,D2
WHERE D1.DATE_ADDED BETWEEN '$date1' AND '$date2' AND D1.D1_ID = D2.D2_ID";
The trouble is, it is not returning anything, but not producing an error either
So I tried inputting it directly into phpMyAdmin like this
SELECT DISTINCT * FROM D1,D2
WHERE D1.DATE_ADDED BETWEEN '2011-01-01' AND '2011-12-12'
AND D1.D1_ID = D2.D2_ID`
then like this
SELECT DISTINCT * FROM D1,D2
WHERE D1.DATE_ADDED BETWEEN '2011-01-01' AND '2011-12-12'
and like this
SELECT * FROM D1
WHERE DATE_ADDED BETWEEN '2011-01-01' AND '2011-12-12'
and I just get
MySQL returned an empty result set (i.e. zero rows). ( Query took 0.0003 sec )
Yes, my tables exist, and so do the columns :)
In the first cases the lack of results could be because of the inner join. For a result to be in the set it would require a record in both tables, ie. a record from d1 would not appear unless d2 also had that id in the d2_id column. To resolve this, if that is correct for your business logic, use left join.
However, the last of your cases (without the join) suggests the reasons is a lack of matching records in the first (left) table d1.
Without the full dataset we can't really comment further, since all the code you are running is perfectly valid.
If you always want to select an entire year it is easer to select it like this:
SELECT * FROM D1 WHERE YEAR(DATE_ADDED) = 2011;
Please implement below code
SELECT DISTINCT * FROM D1,D2
WHERE D1.DATE_ADDED BETWEEN DATE_FORMAT('2011-01-01','%Y-%m-%d')
AND DATE_FORMAT('2011-12-12','%Y-%m-%d')
AND D1.D1_ID = D2.D2_ID`