my below query is taking more then 10 seconds. How I can optimize this query. Please suggest which field to add index.
This query is drupal view for distance search.
SELECT DISTINCT location.lid AS location_lid, node.nid AS nid, location.name AS location_name, location_phone.phone AS location_phone_phone, location_fax.fax AS location_fax_fax, node.title AS node_title, node.language AS node_language, location.additional AS location_additional, location.city AS location_city, location.latitude AS location_latitude, location.longitude AS location_longitude, location.country AS location_country, location.postal_code AS location_postal_code, location.province AS location_province, location.street AS location_street, field_data_field_showroom_level.field_showroom_level_value AS field_data_field_showroom_level_field_showroom_level_value, field_data_field_showroom_type.field_showroom_type_value AS field_data_field_showroom_type_field_showroom_type_value, field_data_field_showroom_inventory_records.field_showroom_inventory_records_value AS field_data_field_showroom_inventory_records_field_showroom_i, field_data_field_record_type_id.field_record_type_id_value AS field_data_field_record_type_id_field_record_type_id_value, field_data_field_toto_gallery.field_toto_gallery_value AS field_data_field_toto_gallery_field_toto_gallery_value, 'node' AS field_data_field_showroom_location_node_entity_type, 'node' AS field_data_field_day_of_operation_node_entity_type, 'node' AS field_data_field_day_of_operation_comments_node_entity_type, 'node' AS field_data_field_showroom_open_node_entity_type, 'node' AS field_data_field_showroom_close_node_entity_type, 'node' AS field_data_field_appointment_needed_node_entity_type, (COALESCE(ACOS(0.83308162381476*COS(RADIANS(location.latitude))*(0.098658826854837*COS(RADIANS(location.longitude)) + -0.99512131716873*SIN(RADIANS(location.longitude))) + 0.55315007734083*SIN(RADIANS(location.latitude))), 0.00000)*6371570.9190939) AS location_distance, 'node' AS field_data_field_salesforce_id_node_entity_type, 'node' AS field_data_field_multiple_product_sku_node_entity_type, 'node' AS field_data_field_product_types_node_entity_type, 'node' AS field_data_field_toto_gallery_node_entity_type, 'node' AS field_data_field_showroom_level_node_entity_type
FROM
node node
LEFT JOIN location_instance location_instance ON node.vid = location_instance.vid
LEFT JOIN location location ON location_instance.lid = location.lid
INNER JOIN field_data_field_toto_gallery field_data_field_toto_gallery ON node.nid = field_data_field_toto_gallery.entity_id AND (field_data_field_toto_gallery.entity_type = 'node' AND field_data_field_toto_gallery.deleted = '0')
LEFT JOIN field_data_field_multiple_product_sku field_data_field_multiple_product_sku ON node.nid = field_data_field_multiple_product_sku.entity_id AND (field_data_field_multiple_product_sku.entity_type = 'node' AND field_data_field_multiple_product_sku.deleted = '0')
LEFT JOIN location_phone location_phone ON location_instance.lid = location_phone.lid
LEFT JOIN location_fax location_fax ON location_instance.lid = location_fax.lid
LEFT JOIN field_data_field_showroom_level field_data_field_showroom_level ON node.nid = field_data_field_showroom_level.entity_id AND (field_data_field_showroom_level.entity_type = 'node' AND field_data_field_showroom_level.deleted = '0')
LEFT JOIN field_data_field_showroom_type field_data_field_showroom_type ON node.nid = field_data_field_showroom_type.entity_id AND (field_data_field_showroom_type.entity_type = 'node' AND field_data_field_showroom_type.deleted = '0')
LEFT JOIN field_data_field_showroom_inventory_records field_data_field_showroom_inventory_records ON node.nid = field_data_field_showroom_inventory_records.entity_id AND (field_data_field_showroom_inventory_records.entity_type = 'node' AND field_data_field_showroom_inventory_records.deleted = '0')
LEFT JOIN field_data_field_record_type_id field_data_field_record_type_id ON node.nid = field_data_field_record_type_id.entity_id AND (field_data_field_record_type_id.entity_type = 'node' AND field_data_field_record_type_id.deleted = '0')
WHERE (( (node.status = '1')
AND (node.type IN ('showrooms'))
AND (location.latitude > '32.859795378699'
AND location.latitude < '34.306986221301'
AND location.longitude > '-85.206632058424'
AND location.longitude < '-83.469478341576')
AND ((COALESCE(ACOS(0.83308162574562*COS(RADIANS(location.latitude))*(0.098658823381208*COS(RADIANS(location.longitude)) + -0.99512131751311*SIN(RADIANS(location.longitude))) + 0.55315007443282*SIN(RADIANS(location.latitude))), 0.00000)*6371570.9191628) < '80467.35')
AND (location.province = 'GA')
AND (location.city LIKE 'Atlanta' ESCAPE '\\') )
AND( (field_data_field_toto_gallery.field_toto_gallery_value = '1')
OR (field_data_field_multiple_product_sku.field_multiple_product_sku_value NOT LIKE 'NULL' ESCAPE '\\') ))
ORDER BY field_data_field_showroom_level_field_showroom_level_value DESC,
field_data_field_showroom_type_field_showroom_type_value DESC,
location_distance ASC,
field_data_field_showroom_inventory_records_field_showroom_i DESC,
field_data_field_record_type_id_field_record_type_id_value ASC,
field_data_field_toto_gallery_field_toto_gallery_value DESC
LIMIT 10 OFFSET 0
I would recommend, that you learn about the PROCEDURE ANALYSE function of MySQL: http://dev.mysql.com/doc/refman/5.0/en/procedure-analyse.html
It will tell you which joins might not have an index they can use. From only looking at your query but without knowing the structure or data of the tables, we can't give you a good adivce. Usually you should have an index on every column, you use in a JOIN and ORDER BY part of you query and you might have some indexes for the columns used in the WHERE clause.
Here are some things to try immediately to see if you can improve performance.
Try a compound index on the location table on (province, city, latitude, longitude, lid). That compound index will, hopefully, accelerate the WHERE criteria computation for your locations.
Make sure your latitude and longitude columns have the FLOAT or DOUBLE data type.
Try a compound index on the node table on (status, type, vid). That index should help locate the exact node records in your WHERE clause and accelerate your joins.
Eliminate the spherical geometry computation from your WHERE clause. You already have the bounding box range restriction, and you're already sorting by distance, so you shouldn't get too many extra locations this way. And, if you do, they shouldn't be too far away by a factor of more than 4/pi.
Can you get rid of the SELECT DISTINCT and just use SELECT? Finding distinct values is a data-intensive operation.
For some helpful information on the distance computation and the bounding box optimization, please see this. http://www.plumislandmedia.net/mysql/haversine-mysql-nearest-loc/
Related
I have the following PHP code that works perfectly ($qry_str is actually generated in the PHP):
$qry_str = <<<'QRY'
FIND_IN_SET('6-47', attributes)
AND FIND_IN_SET('4-176', attributes)
AND FIND_IN_SET('9-218', attributes)
QRY;
$pdo->query('DROP TEMPORARY TABLE IF EXISTS `temp_attr`');
$temp_sql = <<<"TEMP"
CREATE TEMPORARY TABLE IF NOT EXISTS `temp_attr` (
SELECT product_id, GROUP_CONCAT(CONCAT(group_id, '-', IF (custom_value != '', custom_value, value_id)) SEPARATOR ',') AS attributes
FROM `products_attributes`
GROUP BY `product_id`
HAVING $qry_str
);
TEMP;
$pdo->query($temp_sql);
$sql = "SELECT
m.recommended_price AS msrp,
m.purchase_price AS cost,
pp.USD AS regular_price,
pc.USD AS special_price,
pc.start_date AS start_date,
pc.end_date AS end_date,
pl.permalink AS permalink,
pi.name AS description,
m.sku AS sku,
m.default_category_id AS cat,
m.id AS prod_id
FROM `products` AS m
LEFT JOIN `products_prices` AS pp ON m.id = pp.product_id
LEFT JOIN `products_campaigns` AS pc ON m.id = pc.product_id
LEFT JOIN `permalinks` AS pl ON (m.id = pl.resource_id AND pl.resource = 'product')
LEFT JOIN `products_info` AS pi ON (m.id = pi.product_id)
LEFT JOIN `products_to_categories` AS ptc ON (m.id = ptc.product_id)
INNER JOIN `temp_attr` AS pa
WHERE ptc.category_id = :cat
AND m.status = 1
AND m.id = pa.product_id
LIMIT 55;
";
$data = $pdo->prepare($sql)
->bindValue('cat', 100)
->execute()
->fetchAll();
However, when I use a placeholder in the temp table code, i.e.
$temp_sql = <<<"TEMP"
CREATE TEMPORARY TABLE IF NOT EXISTS `temp_attr` (
SELECT product_id, GROUP_CONCAT(CONCAT(group_id, '-', IF (custom_value != '', custom_value, value_id)) SEPARATOR ',') AS attributes
FROM `products_attributes`
GROUP BY `product_id`
HAVING :qry_str
);
TEMP;
$sth = $pdo->prepare($temp_sql);
$sth->bindValue('qry_str', $qry_str, PDO::PARAM_STR);
$sth->execute();
I get the following error:
PHP Fatal error: Uncaught PDOException: SQLSTATE[22007]: Invalid datetime format: 1292 Truncated incorrect DOUBLE value: 'FIND_IN_SET('6-47', attributes)
AND FIND_IN_SET('4-176', attributes)
AND FIND_IN_SET('9-218', attributes)
AND FIND_IN_SET(...'
There are no datetime columns in this table.
group_id and value_id are integer columns
Since the code works fine without the placeholder, I'm at a loss as to why the use of a placeholder breaks the code. The placeholder in the main SQL works properly.
PHP 8.0
https://dev.mysql.com/doc/refman/8.0/en/prepare.html explains:
Parameter markers can be used only where data values should appear, not for SQL keywords, identifiers, and so forth.
In your case, you're apparently trying to bind an expression with the FIND_IN_SET() function. You can't do that. All expressions and other SQL syntax must be fixed in the query at the time you prepare it. You can use a parameter only in a place where you would otherwise use a scalar literal value. That is, a quoted string or a numeric literal.
A php search page uses an SQL query to search for properties based on set criteria, in the example below the street "Harrison Road" is chosen:
search.php?city=1&rooms=3&rooms_upper=7&price_lower=55&price_upper=100&streetname=Harrison%20Road
I am looking for a way to leave the streetname blank and still SELECT ALL records as below:
search.php?city=1&rooms=3&rooms_upper=7&price_lower=55&price_upper=100&streetname=
I tried using the ' * ' symbol as with other SQL ocmmands to specify SELECT ALL but it does not work in this instance.
The problem is the search does not display any results while the street option is left blank.
The reason I am looking to run a search with an empty street criteria is because the search.php loads before the user selects a particular street using the drop down option on the page.
I would like it to SEARCH ALL records using the first criteria specified:
rooms, rooms_upper, price_lower, price_upper
The standard search page load has been left with the widest search criteria possible ( 3 < rooms < 7 ) and (£55 < rent < £75) in order to display ALL records before the user narrows the search criteria specifying a particular 'streetname' if desired.
Many Thanks!
Jeanclaude
The full SQL is here:
$sql=mysql_query("SELECT main_table.housenumber, main_table.housenumber, main_table.streetname,
max(main_table.rent) AS reviews_rent, main_table.rooms AS reviews_rooms,main_table.average,
houses.houseID, houses.path, houses.rooms AS houses_rooms,houses.rent AS houses_rent
FROM main_table
LEFT JOIN houses ON main_table.housenumber = houses.housenumber
AND main_table.streetname = houses.streetname
WHERE main_table.streetname='$page_streetname'
AND main_table.city=$city
AND main_table.verify='1'
AND main_table.rooms>='$rooms'
AND main_table.rooms<='$rooms_upper'
AND main_table.rent>=$price_lower
AND main_table.rent<=$price_upper
GROUP BY main_table.housenumber, main_table.streetname
ORDER BY average DESC, houseID DESC, reviewID DESC;");
I want to keep streetname in the WHERE clause, but I don't want to restrict the search if it is left blank.
How about creating the query dynamically.
Set up the base query without the streetname in the WHERE clause but include a sprintf string tag.
Then if $page_streetname has a value add the streetname selection to the query dynamically or if not just add nothing.
$q = "
SELECT main_table.housenumber,
main_table.streetname,
Max(main_table.rent) AS reviews_rent,
main_table.rooms AS reviews_rooms,
main_table.average,
houses.houseid,
houses.path,
houses.rooms AS houses_rooms,
houses.rent AS houses_rent
FROM main_table
LEFT JOIN houses
ON main_table.housenumber = houses.housenumber
AND main_table.streetname = houses.streetname
WHERE %s
main_table.city = $city
AND main_table.verify = '1'
AND main_table.rooms >= '$rooms'
AND main_table.rooms <= '$rooms_upper'
AND main_table.rent >=$ price_lower
AND main_table.rent <=$ price_upper
GROUP BY main_table.housenumber,
main_table.streetname
ORDER BY average DESC,
houseid DESC,
reviewid DESC";
$q = isset($page_streetname)
? sprintf( $q, "main_table.streetname = '$page_streetname' AND " )
: sprintf( $q, '');
That's easily solved with PHP (no need for sprintf()):
<?php
$page_streetname = empty($page_streetname)
? null
: "AND main_table.streetname = '{$page_streetname}'"
;
$result = mysql_query(
"SELECT
main_table.housenumber,
main_table.housenumber,
main_table.streetname,
max(main_table.rent) AS reviews_rent,
main_table.rooms AS reviews_rooms,
main_table.average,
houses.houseID,
houses.path,
houses.rooms AS houses_rooms,
houses.rent AS houses_rent
FROM main_table
LEFT JOIN houses
ON main_table.housenumber = houses.housenumber
AND main_table.streetname = houses.streetname
WHERE
main_table.city = '{$city}'
AND main_table.verify = 1
AND main_table.rooms >= '{$rooms}'
AND main_table.rooms <= '{$rooms_upper}'
AND main_table.rent >= {$price_lower}
AND main_table.rent <= {$price_upper}
{$page_streetname}
GROUP BY
main_table.housenumber,
main_table.streetname
ORDER BY
average DESC,
houseID DESC,
reviewID DESC"
);
?>
I have a result set and need to be displayed as it is on the screen.But the problem is while displaying each row of the result set using echo command the order is getting changed.Can anyone say why this is happening and provide me a way to overcome it.Here are my actual and printed outputs.
Actual Result set:
JAIKE-ILENE-WACKI-MAZIE-REGLE-SBJ-KMMU
LVZ-HARTY-MUGZY-STW
MAZIE-SIXIE-SBJ-KMMU
PXT-LOUIE-GATBY-RAZER-BUZIE-JAIKE-ILENE-WACKI-MAZIE
SWANN-GATBY-RAZER-BUZIE-JAIKE-ILENE-WACKI-MAZIE
Output:
JAIKE-SBJ-ILENE-KMMU-WACKI-MAZIE-REGLE
MUGZY-STW-LVZ-HARTY
SBJ-KMMU-MAZIE-SIXIE
ILENE-GATBY-WACKI-RAZER-MAZIE-BUZIE-PXT-JAIKE-LOUIE
WACKI-RAZER-MAZIE-BUZIE-JAIKE-SWANN-ILENE-GATBY
Here is my code
$sql3="SELECT GROUP_CONCAT(l.fix_ident SEPARATOR '-') AS fix_seq,l.airport_ident,x.star_ident,x.transition_ident,
x.fix_ident from corept.std_star_leg l
JOIN
(SELECT DISTINCT c.airport_ident,c.star_ident,c.transition_ident,c.fix_ident
FROM corept.std_star_leg c
INNER JOIN
(SELECT star_ident,transition_ident,max(sequence_num) seq,route_type
FROM corept.std_star_leg
WHERE data_supplier='J'
AND airport_ident='KMMU'
GROUP BY star_ident,
transition_ident)b ON c.sequence_num=b.seq
AND c.star_ident=b.star_ident AND c.transition_ident=b.transition_ident
LEFT JOIN
(SELECT name,trans
FROM skyplan_deploy.deploy_stars
WHERE apt='KMMU'
AND name!=trans) d
ON d.name=c.star_ident
AND d.trans=c.fix_ident
WHERE c.data_supplier='J'
AND c.airport_ident='KMMU' AND d.name is null)x
where l.airport_ident='KMMU' and l.transition_ident=x.transition_ident
and l.star_ident=x.star_ident and l.data_supplier='J'
group by x.star_ident,x.transition_ident
order by l.star_ident,x.transition_ident,l.sequence_num";
$res3=mysqli_query($mysqli,$sql3);
if($res3)
{
while($newArray3=mysqli_fetch_array($res3,MYSQLI_ASSOC))
{
$apt=$newArray3['airport_ident'];
$star_ident=$newArray3['star_ident'];
$trans_ident=$newArray3['transition_ident'];
$fix_ident=$newArray3['fix_ident'];
$fix_seq=$newArray3['fix_seq'];
echo $apt.",".$star_ident.",".$trans_ident.",".$fix_ident.",COREPT,".$fix_seq;
echo "<br>";
}
}
else
{
printf("ERROR:%s\n",mysqli_error($mysqli));
}
Your query looks overly complex. It appears to be finding the groupwise maximum std_star_leg records by sequence_num (grouped on start_ident and transition_ident), excluding those for which there is already a matching non-self-referencing deploy_star, then returning the results grouped again with all matching fix_ident values concatenated into a string?
If so, the following greatly simplified query ought to achieve the same outcome:
SELECT GROUP_CONCAT(fix_ident SEPARATOR '-') AS fix_seq,
airport_ident,
star_ident,
transition_ident
FROM corept.std_star_leg l NATURAL JOIN (
SELECT star_ident, transition_ident,
data_supplier, airport_ident,
MAX(sequence_num) sequence_num
FROM corept.std_star_leg
WHERE data_supplier = 'J'
AND airport_ident = 'KMMU'
GROUP BY star_ident, transition_ident
) b
WHERE NOT EXISTS (
SELECT NULL
FROM skyplan_deploy.deploy_stars d
WHERE d.name != d.trans
AND d.name = l.star_ident
AND d.trans = l.fix_ident
AND d.apt = l.airport_ident
)
GROUP BY star_ident, transition_ident
Note that whereas you were previously selecting x.fix_ident in the outermost select list, I have omitted such column because its value would be indeterminately selected by the server from amongst those in the fix_seq.
Now, as to your problem (which appears to be related to the order in which fix_ident values appear within the GROUP_CONCAT() string fix_seq—although it's terribly hard to appreciate that from your question), perhaps you want to use the ORDER BY parameter to the GROUP_CONCAT() function? For example:
SELECT GROUP_CONCAT(fix_ident SEPARATOR '-' ORDER BY ...) AS fix_seq
However, it is not clear to me what ordering you require (the only ORDER BY clause in your original query was entirely redundant).
MySQL always sends a "default response" even when the data doesn't match the query!
How can I design my query to get an error code from MySQL if no data is matching the query?
I need an error code from MySQL to create a 404 or 410 page.
SELECT
place.ID AS id,
place.latitude AS lat,
place.longitude AS lng,
IF(
place.translationID IS NULL,
place.name,
placel10n.text
) AS cityname,
IF(
admcode.translationID IS NULL,
'',
statel10n.text
) AS statename,
IF (
countrycode.translationID IS NULL,
countrycode.name,
countryl10n.Text
) AS countryname,
IF(
place.textID IS NULL,
'',
`l10n-strings`.text
) AS description
FROM
places AS place
LEFT JOIN `l10n-strings` AS placel10n ON (place.translationID = placel10n.translationID AND placel10n.languageCode = 'de')
LEFT JOIN admin1codesascii AS admcode ON (place.admin1 = admcode.statecode AND place.country = admcode.country)
LEFT JOIN `l10n-strings` AS statel10n ON (admcode.translationID = statel10n.translationID AND statel10n.languageCode = 'de')
LEFT JOIN countries AS countrycode ON (place.country = countrycode.iso_alpha2)
LEFT JOIN `l10n-strings` AS countryl10n ON (countrycode.translationID = countryl10n.TranslationID AND countryl10n.LanguageCode = 'de')
LEFT JOIN texts ON (place.textID = texts.id)
LEFT JOIN `l10n-strings` ON (texts.translationID = `l10n-strings`.translationID AND `l10n-strings`.languageCode= 'de')
WHERE
place.id = '8'
AND
place.featurecode = 'A'
OR
place.featurecode = 'AB'
OR
place.featurecode = 'AAC'
LIMIT 0,1
This query always returns a result even when the where statements are wrong.
Yes, that's the way SQL works. If there are no rows that match your query, the result set will be empty. It's the job of the program (written in PHP or what have you) in which your SQL query is embedded to respond to the "no rows" case and change the response status (to 404 or what have you). SQL queries are not responsible for HTTP status codes -- it's chalk and cheese.
In MySQL, AND has higher operator precedence than OR, so the conditions in your WHERE clause will be interpreted as
(place.id = '8' AND place.featurecode = 'A') OR
place.featurecode = 'AB' OR
place.featurecode = 'AAC'
Use parentheses to be explicit about the order of operation you intend. Perhaps you meant this?
place.id = '8' AND
(place.featurecode = 'A' OR
place.featurecode = 'AB' OR
place.featurecode = 'AAC')
I need to fetch the following information about indices on a specific table:
index name
columns that are indexed
unique or not?
How can I do that in PostgreSQL 8.4?
NOTE: I have to be able to call this stuff with PHP. Just saying...
EDIT: I first had this query, but it only works starting with PostgreSQL 9.0:
SELECT t.relname AS table_name,
relname AS index_name,
a.attname AS column_name,
ix.indisunique
FROM pg_class t,
pg_class i,
pg_index ix,
pg_attribute a,
pg_constraint c
WHERE t.oid = ix.indrelid
AND i.oid = ix.indexrelid
AND a.attrelid = t.oid
AND i.oid = c.conindid
AND a.attnum = ANY(ix.indkey)
AND c.contype != 'p'
AND t.relkind = 'r'
AND t.relname = 'tablename'
ORDER BY t.relname, i.relname
You could simply use pg_indexes which will include the full CREATE TABLE statement (and therefor the information about the columns and the uniqueness).
Alternatively, the following should work:
select t.relname as table_name,
ix.relname as index_name,
array_to_string(array_agg(col.attname), ',') as index_columns,
i.indisunique
from pg_index i
join pg_class ix on ix.oid = i.indexrelid
join pg_class t on t.oid = i.indrelid
join (select ic.indexrelid,
unnest(ic.indkey) as colnum
from pg_index ic) icols on icols.indexrelid = i.indexrelid
join pg_attribute col on col.attrelid = t.oid and col.attnum = icols.colnum
where t.relname = 'tablename'
group by t.relname, ix.relname, i.indisunique
order by t.relname,
ix.relname
It doesn't return the columns in the correct order though. But I didn't have time to dig deeper into that.