Sphinx best way WHERE and UPDATE on MySql table - php

I kind used in MySQL query:
UPDATE
Tovar
Tovar INNER JOIN (
SELECT * FROM (.....
SELECT * FROM
....) as `tmp` ON
CONCAT(' ',Tovar.drugApt,' ') LIKE CONCAT('% ',tmp.`P`,' %')
OR IF(
tmp.line is not null,
CONCAT(' ',Tovar.drugApt,' ') LIKE CONCAT('% ',tmp.`line`,' %'),
0
) OR IF(
tmp.invert is not null,
CONCAT(' ',Tovar.drugApt,' ') LIKE CONCAT('% ',tmp.`invert`,' %'),
0
)
WHERE
Tovar.idPreparat IS NULL
AND Tovar.md5Drug <> 0xF7270DB295EE1770CAD49A0CB0C40D91
) tmpf ON tmpf.idT = Tovar.id
SET Tovar.idPreparat = tmpf.id;
As is well-known mysql doesn't use index:
WHERE LIKE '%....%'
Slow performance.
Then I have decided to try Fulltext index in mysql, but I don't accept performance time.
Now I have passed to Sphinx. Since I work in him for the first time, I ask to help.
table Tovar:
id, idPreparat, idCountry, idManufacturer, drugA, countryA, manfA
table Preparat:
id, prep
Tovar.idPreparat associated with Preparat.id
I create two Index in Sphinx:
source tovar : apt
{
sql_query = SELECT Tovar.id as id, Tovar.drugApt as drug,Tovar.countryApt as country,Tovar.manfApt as manf FROM Tovar
sql_field_string = drug
sql_field_string = country
sql_field_string = manf
sql_ranged_throttle = 100
}
source prep : apt
{
sql_query = SELECT id, prep FROM Preparat
sql_field_string = cn
sql_ranged_throttle = 100
}
I need to put down idPreparat where idPreparat IS NULL there. Check will be on the column drugA.
Example (everything is done by me, to make it clear):
What best way of realization of my problem? Thank!

Related

Mysql query IN clause return results

Good Day! All Fridays,
I have some problem in my sql query. I'm using IN class with subquery like this
SELECT
cm.category_id,
cd.name
FROM
category_master cm,
category_detail cd,
brand_to_categories b2c
WHERE
cm.category_id = b2c.category_id
AND
cd.category_id = cm.category_id
AND
cd.language_id = 1
AND
cm.status <> 2
AND
cm.category_id IN (SELECT DISTINCT sub_dd.categories FROM distribution_master bdm, distribution_detail bdd, subscription_category_to_brand_user sub_dd WHERE bdd.distribution_id = bdm.distribution_id AND bdm.distributor_id = 35 AND bdd.brand_id = 7191 AND sub_dd.sub_d_id = bdd.id)
AND
b2c.brand_id = 7191;
The following is the sub-query which is creating problem for me.
cm.category_id IN (
SELECT DISTINCT
sub_dd.categories
FROM
distribution_master bdm,
distribution_detail bdd,
subscription_category_to_brand_user sub_dd
WHERE
bdd.distribution_id = bdm.distribution_id
AND
bdm.distributor_id = 35
AND
bdd.brand_id = 7191
AND
sub_dd.sub_d_id = bdd.id)
the result of the sub-query is like this.
3913,4517,6059,7137,7138,7139,7140,7141,7144
this result is coming from only single row in the target table because I stored these ids as string in the filed.
Now the problem is this, I can not get results of the all categories. Main query final result only return one category information which category_id is 3913. But if I run this query manually with sub-query values instead of the sub-query then it returns all the categories results.
Manual query with sub-query values is like this
SELECT
cm.category_id,
cd.name
FROM
category_master cm,
category_detail cd,
brand_to_categories b2c
WHERE
cm.category_id = b2c.category_id
AND
cd.category_id = cm.category_id
AND
cd.language_id = 1
AND
cm.status <> 2
AND
cm.category_id IN (3913,4517,6059,7137,7138,7139,7140,7141,7144)
AND
b2c.brand_id = 7191;
Please help me regarding this problem.
Sorry I forget, I'm using Mysql
Assuming you are using MySQL, use FIND_IN_SET:
WHERE
...
FIND_IN_SET(cm.category_id,
(SELECT DISTINCT sub_dd.categories
FROM distribution_master bdm,
distribution_detail bdd,
subscription_category_to_brand_user sub_dd
WHERE bdd.distribution_id = bdm.distribution_id AND
bdm.distributor_id = 35 AND
bdd.brand_id = 7191 AND
sub_dd.sub_d_id = bdd.id)) > 0
If you are using SQL Server, then we have to do a bit more work:
WHERE ',' + (SELECT DISTINCT ...) + ',' LIKE '%,' + cm.category_id + ',%'
General comment: Avoid storing CSV data in your SQL tables. MySQL almost made the problem worse by offering FIND_IN_SET and making it easier to skirt good table design.

DECLARE syntax in MySQLi Query | When To DECLARE variables in Query

Apologies if posted already.
Searched a lot on google but I'm not understanding how to use DECLARE syntax in MySQL Query.
I'm not expert in MySQL. I'm seeing these codes on internet for DECLARE syntax. Not seen complete query anywhere.
DECLARE #destlist varchar(max)
SET #destlist = SELECT destinations FROM tbl_rings_to_groups WHERE id = '1' AND user_id = '1'
SELECT count( id ) AS totalDestination, group_concat( ring_to_number ) AS phoneNumbers
FROM tbl_destinations
WHERE id IN (CONCAT('SELECT ', REPLACE(#destlist), ',', ' UNION ALL SELECT '))
I'm using this for fetch results.
$result = $db->query("select from
table where something =
somethingElse order by something");
Now, how to use DECLARE in above code.
I'm trying this.
$result = $db->query(" DECLARE #destlist varchar(max)
SET #destlist = SELECT destinations FROM tbl_rings_to_groups WHERE id = '1' AND user_id = '1'
SELECT count( id ) AS totalDestination, group_concat( ring_to_number ) AS phoneNumbers
FROM tbl_destinations
WHERE id IN (CONCAT('SELECT ', REPLACE(#destlist), ',', ' UNION ALL SELECT '))");
Is this correct way to using MySQL DECLARE?

MySql Query - optimization with varchars, indexs, taking over an hour to run

So I need to run a query that I do not know the UUID - but need to find it... so I am using the street num, street name, and a company UUID to find it
I have a few million records, and this took query is taking around an HOUR!!
any advice to speed it up?
gisPoints
UUID Indexed Unique varchar(36)
street_num int(11)
street_name varchar(128)
geoPoint_temp
UUID Indexed Unique varchar(36)
street_num int(11)
street_name varchar(128)
gcomUUID Indexed varchar(36)
update geoPoint_temp as temp JOIN gisPoints as `prod` on prod.gcomUUID=temp.gcomUUIDand prod.street_num=temp.street_num and prod.street_name REGEXP(temp.street_name)
set temp.UUID=prod.UUID,temp.customerUUID=prod.customerUUID WHERE temp.`uploadstate` = '1'";
Assuming you have the following values (in PHP):
$street_num = ...;//something
$street_name = ...;//something
$gcomUUID = ...;//something
If you run the following sql code:
$sql = "SELECT * FROM (
SELECT * FROM (
SELECT * FROM geoPoint_temp WHERE gcomUUID = $gcomUUID)
WHERE street_name = $street_name)
WHERE street_num = $street_num;"
You should obtain a list of rows (0 or more) from geoPoint_temp that have matching values, and it should be relatively fast even in a big table.
After obtaining those rows, you can check if the row count is greater than zero, and if so update the rows. If your using MySQL (PDO), you could do something similar to the following:
$count = $stmt->rowCount();
if ($count>0)
{
$rows = $stmt->fetchAll();
foreach ($rows as $row)
{
$sql = "UPDATE geoPoint_temp SET ... WHERE UUID = ".$row['UUID'];
$stmt = $conn->prepare($sql);
$stmt->execute();
}
}
Let me know if that helped.
EDITED:
Try the following as well and let me know if it works:
$sql = "
UPDATE geoPoint_temp SET ... WHERE UUID IN
(SELECT * FROM (
SELECT * FROM (
SELECT * FROM geoPoint_temp WHERE gcomUUID = $gcomUUID)
WHERE street_name = $street_name)
WHERE street_num = $street_num);"
And replace ... with the values you want updated.
This runs in 1.5 seconds opposed to the hours it was taking before
Much help to #Webeng for pointing us in the right direction!
$custquery="UPDATE geoPoint_temp as temp
join
(
select prod.name, prod.street_num, prod.street_name, prod.UUID,prod.customerUUID, prod.gcomUUID
FROM gisPoints as `prod`
JOIN
(
select t1.gcomUUID , t1.street_num, t1.street_name
FROM geoPoint_temp as t1
) as sub1 on prod.gcomUUID =sub1.gcomUUID and prod.street_num=sub1.street_num
) as sub2 on sub2.gcomUUID =temp.gcomUUID
and sub2.street_num=temp.street_num
AND sub2.street_name LIKE (CONCAT('%',temp.street_name,'%'))
set temp.customerUUID = sub2.customerUUID, temp.UUID=sub2.UUID";
$custre=mysql_query($custquery);
if (!$custre) { echo 'Could not run custre query: ' . mysql_error(); exit; }

Fetch index information in PostgreSQL 8.4

I need to fetch the following information about indices on a specific table:
index name
columns that are indexed
unique or not?
How can I do that in PostgreSQL 8.4?
NOTE: I have to be able to call this stuff with PHP. Just saying...
EDIT: I first had this query, but it only works starting with PostgreSQL 9.0:
SELECT t.relname AS table_name,
relname AS index_name,
a.attname AS column_name,
ix.indisunique
FROM pg_class t,
pg_class i,
pg_index ix,
pg_attribute a,
pg_constraint c
WHERE t.oid = ix.indrelid
AND i.oid = ix.indexrelid
AND a.attrelid = t.oid
AND i.oid = c.conindid
AND a.attnum = ANY(ix.indkey)
AND c.contype != 'p'
AND t.relkind = 'r'
AND t.relname = 'tablename'
ORDER BY t.relname, i.relname
You could simply use pg_indexes which will include the full CREATE TABLE statement (and therefor the information about the columns and the uniqueness).
Alternatively, the following should work:
select t.relname as table_name,
ix.relname as index_name,
array_to_string(array_agg(col.attname), ',') as index_columns,
i.indisunique
from pg_index i
join pg_class ix on ix.oid = i.indexrelid
join pg_class t on t.oid = i.indrelid
join (select ic.indexrelid,
unnest(ic.indkey) as colnum
from pg_index ic) icols on icols.indexrelid = i.indexrelid
join pg_attribute col on col.attrelid = t.oid and col.attnum = icols.colnum
where t.relname = 'tablename'
group by t.relname, ix.relname, i.indisunique
order by t.relname,
ix.relname
It doesn't return the columns in the correct order though. But I didn't have time to dig deeper into that.

mysql select related terms

I have a table of dictionary terms and now I would like to create another table which would hold ID of the 'main' term and ID of related terms that have the 'main' term's Name included in their Definition -
table: terms
| id | name | definition |
Table: related
| term_id | related_term_id |
It was pretty easy to create INSERT statements in PHP:
$result = mysql_query("SELECT * FROM terms");
while ($row = mysql_fetch_array($result)) {
$name = $row['name'];
$result2 = mysql_query("SELECT id,name FROM terms WHERE description LIKE \"%$name%\" AND id!=".$row['id']);
while ($row2 = mysql_fetch_array($result2)) {
echo "INSERT INTO related(term_id,related_term_id) VALUES(".$row['id'].",".$row2['id'].");";
}
}
However, as I don't have too much experience with MySQL I was wondering if that can be done purely in SQL - let say with use of INSERT-SELECT statement?
Thanks!
INSERT INTO related (term_id, related_term_id)
SELECT a.id as term_id, b.id AS related_term_id
FROM terms a, terms b
WHERE a.id != b.id
AND b.description LIKE CONCAT('%', b.name, '%')
It's going to be real slow if the terms table is big (more than 1000-10000 rows):
INSERT INTO related (term_id, related_term_id)
SELECT describing.id, described.id
FROM terms describing
JOIN terms described
ON CONCAT(' ', described.description, ' ') LIKE CONCAT('% ', describing.name, ' %')
Assuming all descriptions have spaces separating words and no other marks like , or ? or ! or . If that's not the case, the code will be more complicated.
Change your INSERT query:
echo "INSERT INTO related(term_id,related_term_id) VALUES('". $row['id'] ."','". $row2['id'] ."');";

Categories