Combine multiple SQL queries into single query for speed - php

I am working in PHP with Laravel and the page I have been given contains 3 SQL queries on separate tables but using the same criteria. I have tried combining them but the combined results are slower than the original. What can I do to increase rather than decrease the speed of these queries?
Here are my queries:
$searchFor = 'debtor_name';
$searchForBKR = 'estate_name';
$searchForHypotech = 'hypleg_lot_credit_debtor_name';
$arraySearch = explode(",", $request->input('name'));
$search = $searchFor . ' like "%';
$bkrSearch = $searchForBKR . ' like "%';
$hypotechSearch = $searchForHypotech . ' like "%';
$searchCompany = $searchCompanyFor . ' like "%';
$searchPlaintiff = $searchPlaintiffFor . ' like "%';
$searchDefendantName = $searchDefendantNameFor . ' like "%';
$search = $search . $arraySearch[0] . '%"';
$bkrSearch = $bkrSearch . $arraySearch[0] . '%"';
$hypotechSearch = $hypotechSearch . $arraySearch[0] . '%"';
$searchCompany = $searchCompany . $arraySearch[0] . '%"';
$searchPlaintiff = $searchPlaintiff . $arraySearch[0] . '%"';
$searchDefendantName = $searchDefendantName . $arraySearch[0] . '%"';
for ($i = 1; $i < count($arraySearch); $i++) {
$search = $search . " or " . $searchFor . ' like "%' . $arraySearch[$i] . '%"';
$bkrSearch = $bkrSearch . " or " . $searchForBKR . ' like "%' . $arraySearch[$i] . '%"';
$hypotechSearch = $hypotechSearch . " or " . $searchForHypotech . ' like "%' . $arraySearch[$i] . '%"';
$searchCompany = $searchCompany . " or " . $searchCompanyFor . ' like "%' . $arraySearch[$i] . '%"';
$searchPlaintiff = $searchPlaintiff . " or " . $searchPlaintiffFor . ' like "%' . $arraySearch[$i] . '%"';
$searchDefendantName = $searchDefendantName . " or " . $searchDefendantNameFor . ' like "%' . $arraySearch[$i] . '%"';
}
$sis = DB::select('select debtor_name, count(*) as sis_total from equifax_sis_regions' . $search . ')
group by debtor_name');
$bkr = DB::select('select estate_name as debtor_name, count(*) as bkr_total from equifax_bkr_regions' . $bkrSearch . ')
group by estate_name');
$hypotech = DB::select('select hypleg_lot_credit_debtor_name as debtor_name, count(*) as hypotech_total from equifax_hypotech_regions' . $hypotechSearch . ')
group by hypleg_lot_credit_debtor_name');
I have tried replacing the 3 queries by using outer joins like this:
$other = 'select debtor_name, count(debtor_name) as sis_total,
estate_name as debtor_name, count(estate_name) as bkr_total
hypleg_lot_credit_debtor_name as debtor_name, count(hypleg_lot_credit_debtor_name) as hypotech_total
from equifax_sis_regions outer join equifax_bkr_regions on (equifax_sis_regions.debtor_name=estate_name)
outer join equifax_hypotech_regions on (equifax_sis_regions.debtor_name=hypleg_lot_credit_debtor_name)
WHERE (' . $search . ') OR ' . $bkrSearch . ') OR (' . $hypotechSearch . ')
GROUP BY debtor_name, estate_name, hypleg_lot_credit_debtor_name';
but that is a much slower query to run. The database contains over 4 million rows and the difference in API calls to the 2 query versions is ~53 seconds for the original and ~78 seconds for the combined form. Is there any way I can modify this combined query to be faster rather than slower?
Thanks in advance

You can use the UNION ALL operator to combine multiple SELECT statements. You could combine them like this:
SELECT debtor_name, SUM(sis_total), SUM(bkr_total), SUM(hypotech_total)
FROM (
SELECT debtor_name, COUNT(*) AS sis_total, 0 AS bkr_total, 0 AS hypotech_total
FROM equifax_sis_regions
-- add where clause here
GROUP BY debtor_name
UNION ALL
SELECT estate_name AS debtor_name, 0 AS sis_total, COUNT(*) AS bkr_total, 0 AS hypotech_total
FROM equifax_bkr_regions
-- add where clause here
GROUP BY estate_name
UNION ALL
SELECT hypleg_lot_credit_debtor_name AS debtor_name, 0 AS sis_total, 0 AS bkr_total, COUNT(*) AS hypotech_total
FROM equifax_hypotech_regions
-- add where clause here
GROUP BY hypleg_lot_credit_debtor_name
) derived
GROUP BY debtor_name;

Related

Error GroupBy PHP + MSSQL

$Read->FullRead("SELECT an_promo_page_tipo_blocos.title,"
. "an_promo_page_tipo_blocos.img,"
. "an_promo_bloco_anexo.status,"
. "an_promo_bloco_anexo.pg_id,"
. "an_promo_bloco_anexo.id,"
. "an_promo_bloco_anexo.promo_id,"
. "an_promo_bloco_anexo.tipo_bloco,"
. "an_promo_bloco_anexo.bloco_id "
. "FROM an_promo_bloco_anexo "
. "INNER JOIN an_promo_page "
. "ON "
. "an_promo_bloco_anexo.pg_id = an_promo_page.pg_id"
. " INNER JOIN an_promo_page_tipo_blocos ON an_promo_bloco_anexo.tipo_bloco = an_promo_page_tipo_blocos.id "
. "WHERE an_promo_bloco_anexo.promo_id = :pro AND an_promo_bloco_anexo.pg_id = :pi"
. " AND an_promo_bloco_anexo.pg_tipo = :pt ORDER BY an_promo_bloco_anexo.ordem, an_promo_page_tipo_blocos.title"
. " AND GROUP BY "
. "an_promo_page_tipo_blocos,"
. "an_promo_bloco_anexo"
. "", "pro={$PromoId}&pi={$p['pg_id']}&pt={$p['pg_tipo']}");
Result:
protected 'message' => string 'SQLSTATE[42000]: [Microsoft][ODBC Driver 11 for SQL Server][SQL Server]Column 'an_promobar.promo_title' is invalid in the select list because it is not contained in either an aggregate function or the GROUP BY clause.' (length=217)
Anyone can help me?
When you use GROUP BY you need to group by all the fields that you are not using an aggregate function against (ie: SUM, MAX, etc).
GROUP BY is also not part of a WHERE clause; it stands alone.
Given your query has no aggregate functions, I don't see why you're using GROUP BY at all.
Try this.
$Read->FullRead("SELECT an_promo_page_tipo_blocos.title,"
. "an_promo_page_tipo_blocos.img,"
. "an_promo_bloco_anexo.status,"
. "an_promo_bloco_anexo.pg_id,"
. "an_promo_bloco_anexo.id,"
. "an_promo_bloco_anexo.promo_id,"
. "an_promo_bloco_anexo.tipo_bloco,"
. "an_promo_bloco_anexo.bloco_id "
. "FROM an_promo_bloco_anexo "
. "INNER JOIN an_promo_page "
. "ON "
. "an_promo_bloco_anexo.pg_id = an_promo_page.pg_id"
. " INNER JOIN an_promo_page_tipo_blocos ON an_promo_bloco_anexo.tipo_bloco = an_promo_page_tipo_blocos.id "
. "WHERE an_promo_bloco_anexo.promo_id = :pro AND an_promo_bloco_anexo.pg_id = :pi"
. " AND an_promo_bloco_anexo.pg_tipo = :pt ORDER BY an_promo_bloco_anexo.ordem, an_promo_page_tipo_blocos.title"
. "", "pro={$PromoId}&pi={$p['pg_id']}&pt={$p['pg_tipo']}");

While doesnt give me more trainers

I'm trying to make a map with available gyms that also shows what exercises they have. So it looks like this:
Name Gym
Open spots: (number)
Logo image of Gym
Time of excercise.
Image of a dumbel for example
Trainer
The part from Time of exercise til Trainer needs to show more than one, I got like 6 exercises with 6 trainers, and 6 times and 6 different images of exercises to show. I store those in a database; but they don't show up.
Here my code:
$sql = "SELECT * FROM forts";
$result = $mysqli->query($sql);
if ($result->num_rows > 0) {
echo "[";
$output = array();
while($row = $result->fetch_assoc()) {
$url = preg_replace("/^http:/i", "https:", $row['url']);
$sqlmon = "SELECT * FROM gym_trainers WHERE fort_id = " . $row['id'];
$resultmon = $mysqli->query($sqlmon);
$mon = $resultmon->fetch_assoc();
$sqlextra = "SELECT * FROM extrainfo WHERE fort_id = " . $row['id'] . " ORDER BY last_modified DESC";
$resultextra = $mysqli->query($sqlextra);
$extra = $resultextra->fetch_assoc();
array_push($output, '
{
"id": "' . $row['id'] . '",
"name": "' . htmlspecialchars($row['name']) . '",
"image": "' . $url . '",
"team": "' . $team . '",
"spots": ' . $extra['spots'] . ',
"exer_time": ' . $mon['time_exerise'] . ',
"trainer": "' . htmlspecialchars($mon['trainer_name']) . '",
"exercise_id": ' . $mon['exercise_id'] . ',
"lat": ' . $row['lat'] . ',
"lng": ' . $row['lon'] . '
}
');
}
echo implode(",", $output);
echo "]";
I think something with this part needs to change, but don't know what:
"exer_time": ' . $mon['time_exerise'] . ',
"trainer": "' . htmlspecialchars($mon['trainer_name']) . '",
"exercise_id": ' . $mon['exercise_id'] . ',
The specific problem is that you fetch only 1 record from the gym_trainers table (the $resultmon->fetch_assoc() is not in a separate loop).
The wider problem is that you do not use joins. The 3 separate sql queries could be written as
select *
from forts f
left join gym_trainers g on f.id=g.fort_id
left join extrainfo e on f.id=g.fort_id
In this case you would need a single loop and within the single loop you track if the fort changes.

Adapting query to JDatabase

How can I adapt the following query to JDatabase?
SELECT
c.RAZONSOCIAL usu_razonSocial
,c.NIT usu_nit
,c.SEDE usu_sede
,c.EMAIL usu_email
,IF(IFNULL(u.block, 1) = 0, 'Activo', 'Inactivo') usu_estado
,COUNT(ct.ctf_id) nroCertificados
FROM
cargacliente c
INNER JOIN
cargas
ON (crg_id = cargas_id)
AND (crg_status = 'Ok')
INNER JOIN
certificados ct
ON (ctf_sede = SEDE)
AND (ctf_nit = NIT)
LEFT JOIN
database_1.bml_users u
ON (id = user_id)
GROUP BY
c.NIT
,c.SEDE
ORDER BY
usu_razonSocial
,usu_nit
,usu_sede;
This is Joomla 2.5.4.
I have read this post but I couldn't do it.
I tried to do it in this way:
<?php
$query
->select($db->quoteName(array('c.RAZONSOCIAL AS usu_razonSocial', 'c.NIT usu_nit', 'c.SEDE usu_sede', 'c.EMAIL usu_email', 'IF(IFNULL(u.block, 1) = 0, \'Activo\', \'Inactivo\') usu_estado', 'COUNT(ct.ctf_id) nroCertificados')))
->from($db->quoteName('database_2.cargacliente', 'c'))
->join('INNER', $db->quoteName('database_2.cargas','a') . ' ON (' . $db->quoteName('a.crg_id') . ' = ' . $db->quoteName('database_2.cargacert.cargas_id') . ') AND (' . $db->quoteName('a.crg_status') . ' = \'Ok\')')
->join('INNER', $db->quoteName('database_2.certificados','b') . ' ON (' . $db->quoteName('b.ctf_sede') . ' = ' . $db->quoteName('database_2.cargacliente.SEDE') . ') AND (' . $db->quoteName('b.ctf_nit') . ' = NIT)')
->join('LEFT', $db->quoteName('joomla_database.bml_users', 'u') . ' ON (' . $db->quoteName('database_2.usuario.usu_id') . ' = ' . $db->quoteName('joomla_database.bml_users.user_id') . ')')
->group(array('c.NIT', 'c.SEDE'))
->order(array('database_2.usuario.usu_razonSocial', 'database_2.usuario.usu_nit', 'database_2.usuario.usu_sede'));
?>
The error shown is the following:
500 - Ha ocurrido un error.
Unknown column 'c.RAZONSOCIAL usu_razonSocial' in 'field list' SQL=SELECT `c`.`RAZONSOCIAL usu_razonSocial`,`c`.`NIT usu_nit`,`c`.`SEDE usu_sede`,`c`.`EMAIL usu_email`,`IF(IFNULL(u`.`block, 1) = 0, 'Activo', 'Inactivo') usu_estado`,`COUNT(ct`.`ctf_id) nroCertificados` FROM `biochemical`.`cargacliente` AS `c` INNER JOIN `biochemical`.`cargas` AS `a` ON (`a`.`crg_id` = `biochemical`.`cargacert`.`cargas_id`) AND (`a`.`crg_status` = 'Ok') INNER JOIN `biochemical`.`certificados` AS `b` ON (`b`.`ctf_sede` = `biochemical`.`cargacliente`.`SEDE`) AND (`b`.`ctf_nit` = NIT) LEFT JOIN `biochemical_bml`.`bml_users` AS `u` ON (`biochemical`.`usuario`.`usu_id` = `biochemical_bml`.`bml_users`.`user_id`) GROUP BY c.NIT,c.SEDE ORDER BY biochemical.usuario.usu_razonSocial,biochemical.usuario.usu_nit,biochemical.usuario.usu_sede
The changes were the following:
Remove the first $db->quoteName() in the SELECT statement.
Modify ALIAS from tables.
LEFT JOIN modification.
The final code is the following:
$query
->select(array('c.RAZONSOCIAL AS usu_razonSocial', 'c.NIT AS usu_nit', 'c.SEDE AS usu_sede', 'c.EMAIL AS usu_email', 'IF(IFNULL(u.block, 1) = 0, \'Activo\', \'Inactivo\') AS usu_estado', 'COUNT(b.ctf_id) AS nroCertificados'))
->from($db->quoteName('database_2.cargacliente', 'c'))
->join('INNER', $db->quoteName('database_2.cargas','a') . ' ON (' . $db->quoteName('a.crg_id') . ' = ' . $db->quoteName('c.cargas_id') . ') AND (' . $db->quoteName('a.crg_status') . ' = \'Ok\')')
->join('INNER', $db->quoteName('database_2.certificados','b') . ' ON (' . $db->quoteName('b.ctf_sede') . ' = ' . $db->quoteName('c.SEDE') . ') AND (' . $db->quoteName('b.ctf_nit') . ' = c.NIT)')
->join('LEFT', $db->quoteName('joomla_database.bml_users', 'u') . ' ON (' . $db->quoteName('c.user_id') . ' = ' . $db->quoteName('u.id') . ')')
->group(array('c.NIT', 'c.SEDE'))
->order(array('usu_razonSocial', 'usu_nit', 'usu_sede'));
This is a bit difficult to say what the problem is without having the right environment.
I would suggest to debug through the driver to see where it fails.
You can of course also put the query direct through the "setQuery" method. It would be however compatible with MySQL only.
But, once again, please update your Joomla! to the latest version ASAP.

How to Sort users who voted by number of Published links

problems with a lot of users have the names of long in who liked for this Story
So How can I sort users by number of published links in who voted.
in libs/htm1.php
function who_voted($storyid, $avatar_size){
// this returns who voted for a story
// eventually add support for filters (only show friends, etc)
global $db;
if (!is_numeric($storyid)) die();
$sql = 'SELECT ' . table_votes . '.*, ' . table_users . '.* FROM ' . table_votes . ' INNER JOIN ' . table_users . ' ON ' . table_votes . '.vote_user_id = ' . table_users . '.user_id WHERE (((' . table_votes . '.vote_value)>0) AND ((' . table_votes . '.vote_link_id)='.$storyid.') AND (' . table_votes . '.vote_type= "links")) AND user_level<>"god" AND user_level<>"Spammer"';
//echo $sql;
$voters = $db->get_results($sql);
$voters = object_2_array($voters);
foreach($voters as $key => $val){
$voters[$key]['Avatar_ImgSrc'] = get_avatar($avatar_size, "", $val['user_login'], $val['user_email']);
}
return $voters;
I found these lines in topusers.php, but not for a friendly experience in writing function correctly
case 2: // sort users by number of published links
$select = "SELECT user_id, count(*) as count ";
$from_where = " FROM " . table_links . ", " . table_users . " WHERE link_status = 'published' AND link_author=user_id AND user_level NOT IN ('god','Spammer') AND (user_login!='anonymous' OR user_lastip) GROUP BY link_author";
$order_by = " ORDER BY count DESC ";
break;

Dynamic MySQL recordset query using join syntax

I can't get the JOIN syntax correct to alter this existing MySQL query in PHP to include a join from another table. I have another table specified as DB_TABLE2 that contains columns InvmNumr and InvmDesc. The InvmNumr and InvlNumr are the exact same value in each table and I need to display InvmDesc from Table 2 in this query?
$res = mysql_query('SELECT LocId, InvlNumr, InvlQuant FROM ' . DB_TABLE1
. ' WHERE LocId = \'' . mysql_real_escape_string($cType) . '\'
AND InvlNumr = \'' . mysql_real_escape_string($br) . "'");
$markup = '';
Read up on LEFT JOIN vs INNER JOIN depending on how your data is stored in your DB_TABLE2 table.
$res = mysql_query('SELECT ' . DB_TABLE1 . '.LocId, ' . DB_TABLE1 . '.InvlNumr, ' .
DB_TABLE1 . '.InvlQuant, ' . DB_TABLE2 . '.InvmDesc
FROM ' . DB_TABLE1 . ' LEFT JOIN ' . DB_TABLE2 . ' ON ' .
DB_TABLE1 . '.InvlNumr = ' . DB_TABLE2 . '.InvmNumr
WHERE ' . DB_TABLE1 . '.LocId = \'' . mysql_real_escape_string($cType) . '\'
AND ' . DB_TABLE1 . '.InvlNumr = \'' . mysql_real_escape_string($br) . "'");
try this
SELECT db1.LocId,db1.InvlNumr,db1.InvlQuant
FROM DB_TABLE1 db1
INNER JOIN DB_TABLE2 db2 ON db1.InvlNumr = db2.InvlNumr
WHERE dbq.LocId = $cType
AND db1.InvlNumr = $br

Categories