I use Codeigniter 3.1.11 and have some code which makes one big query to MySQL with an array. The time of a query with a limit of 30000 is about 9 seconds.
How can I reduce the request time? Maybe by using some indexes on my table, or do you know another method? If I need to use indexes, what indexes would I need to use and how can I use these indexes in my query on Codeigniter?
Code from model:
function rows_update() {
$query = $this->db->order_by('rating', 'DESC')->get_where('members', 'game_rating_and_balance_update_last_time <= now() - INTERVAL 1 DAY', '30000', '0');
$arrUpdateBatchData = [];
while ($row = $query->unbuffered_row('array'))
{
// some code here
$arrUpdateData = [
'UserID' => $row['UserID'],
'game_vault_balance' => $new_game_vault_balance,
'game_available_balance' => $new_game_available_balance,
'rating' => $rating_member,
'game_rating_and_balance_update_last_time' => date('Y-m-d H:i:s')
];
$arrUpdateBatchData[] = $arrUpdateData;
if (count($arrUpdateBatchData) > 500)
{
$this->db->update_batch('members', $arrUpdateBatchData, 'UserID');
$arrUpdateBatchData = [];
}
}
//update last items
if (count($arrUpdateBatchData) > 0)
{
$this->db->update_batch('members', $arrUpdateBatchData, 'UserID');
$arrUpdateBatchData = [];
}
return;
}
Raw query to MySQL with update_batch (I simply write only one row from an array):
UPDATE members SET game_vault_balance = CASE WHEN UserID = '9915075' THEN 803.60516004772 ELSE game_vault_balance END, game_available_balance = CASE WHEN UserID = '9915075' THEN 4.1253850908788 ELSE game_available_balance END, rating = CASE WHEN UserID = '9915075' THEN 0.24 ELSE rating END, game_rating_and_balance_update_last_time = CASE WHEN UserID = '9915075' THEN '2020-07-24 22:00:36' ELSE game_rating_and_balance_update_last_time END WHERE UserID IN('9915075')
Table structure:
CREATE TABLE `members` (
`id` int(10) UNSIGNED NOT NULL,
`UserID` varchar(64) NOT NULL,
`telegram_id` varchar(64) DEFAULT NULL,
`first_name` varchar(64) DEFAULT NULL,
`last_name` varchar(64) DEFAULT NULL,
`language` varchar(64) DEFAULT NULL,
`currency` varchar(64) DEFAULT NULL,
`status` varchar(64) DEFAULT NULL,
`rating` varchar(64) DEFAULT NULL,
`game_vault_balance` decimal(32,8) DEFAULT 0.00000000,
`game_available_balance` decimal(32,8) DEFAULT 0.00000000,
`game_rating_and_balance_update_last_time` datetime DEFAULT NULL,
`updated` datetime DEFAULT NULL,
`created` datetime DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Indexes of this table:
ALTER TABLE `members`
ADD PRIMARY KEY (`id`),
ADD UNIQUE KEY `UserID` (`UserID`) USING BTREE;
AUTO_INCREMENT for the members table:
ALTER TABLE `members`
MODIFY `id` int(10) UNSIGNED NOT NULL AUTO_INCREMENT;
COMMIT;
Related
A PDO prepared update statement is somehow updating the datetime column for the selected record in the table, even though that particular datetime column is not even in the query.
if(isset($_POST['editCriteria']))
{
$value = $_POST['editCriteria'];
$editusername = $value['editusername'];
$hiddenUsername = $value['hiddenUsername'];
$editfullname = $value['editfullname'];
$editemail = $value['editemail'];
$edituserlevel = $value['edituserlevel'];
$editdivision = $value['editdivision'];
$editdept = $value['editdept'];
$editphone = $value['editphone'];
try
{
$dbc->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$update = $dbc->prepare("UPDATE users_edi SET username = :uname,
fullname = :fname, userlevel = :ulevel, email = :uemail,
division = :udivision, dept = :udept, phone = :uphone WHERE username = :hname");
$update->execute([
'uname' => $editusername,
'fname' => $editfullname,
'ulevel' => $edituserlevel,
'uemail' => $editemail,
'udivision' => $editdivision,
'udept' => $editdept,
'uphone' => $editphone,
'hname' => $hiddenUsername
]);
if($update)
{
echo "Success: User has been updated.";
}
}
catch(PDOException $e)
{
echo "Error: " . $e->getMessage();
}
}
In the database table, there is a column called lastLoginDate that is being updated to the current datetime.
If you'll notice in the update statement above, the query does not include lastLoginDate.
How is lastLoginDate being updated when it's not even in the query?
Upon using the SHOW CREATE TABLE command, there was indeed a trigger on the lastLoginDate column.
CREATE TABLE `users_edi` (
`username` varchar(30) NOT NULL DEFAULT '',
`fullname` varchar(50) DEFAULT NULL,
`userlevel` tinyint(1) unsigned NOT NULL,
`ipaddress` varchar(30) DEFAULT NULL,
`email` varchar(150) DEFAULT NULL,
`entrydate` datetime DEFAULT NULL,
`division` varchar(35) DEFAULT NULL,
`password` varchar(32) DEFAULT NULL,
`userid` varchar(32) DEFAULT NULL,
`timestamp` int(11) unsigned NOT NULL,
`job_title` varchar(30) DEFAULT NULL,
`dept` varchar(50) DEFAULT NULL,
`phone` varchar(11) DEFAULT NULL,
`lastLoginDate` datetime DEFAULT NULL ON UPDATE CURRENT_TIMESTAMP, // <-- here
PRIMARY KEY (`username`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
I will have to ask another question on how to remove this trigger.
I just having some problem of if/else or somewhat logical things in here, I have a fullcalendar that shows all the date that being reserve, I limit the reserve by 5 per date, but when I having a 2 or more reserved on the date and having reserve by other date, It gives me the same result as 4.
global $db;
$data = array();
$query = "SELECT * FROM reserve_master
INNER JOIN reserve_details
on reserve_master.reserve_id = reserve_details.reserve_id
INNER JOIN reserve_indicator
on reserve_master.reserve_id = reserve_indicator.reserve_id
WHERE reserve_indicator.touserid = '$id'
AND reserve_master.type = 'Repair' ";
$res = mysqli_query($db,$query);
$count = mysqli_num_rows($res);
$count = 5 - $count;//count the available slot
$date_changed = "";
$reserve_id = 0;
foreach ($res as $row)
{
date_default_timezone_set('Asia/Manila');
$current_timestamp = strtotime($row["dateend"] . '+1 day');
$time = date("Y/m/d",$current_timestamp);
if($row["datestart"] == $date_changed)
{
//This is for avoiding repeating graph on fullcalendar
}
else
{
if(empty($count))
{
$count = '0';
}
else
{
$count;
}
$data[] = array(
'id' => $row["reserve_id"],
'title' => $count,
'start' => $row["datestart"],
'end' => $time,
'color' =>getColor($row["status"])
);
$date_changed = $row["datestart"];
$reserve_id = $row["reserve_id"];
}
}
echo json_encode($data);
This is image of the error with captions
Database
CREATE TABLE `reserve_master` (
`reserve_id` int(11) NOT NULL AUTO_INCREMENT,
`datestart` date NOT NULL,
`dateend` date NOT NULL,
`type` varchar(255) NOT NULL,
PRIMARY KEY (`reserve_id`)
) ENGINE=InnoDB AUTO_INCREMENT=47 DEFAULT CHARSET=latin
CREATE TABLE `reserve_indicator` (
`indicator_id` int(11) NOT NULL AUTO_INCREMENT,
`reserve_id` int(11) NOT NULL,
`touserid` int(11) NOT NULL,
`byuserid` int(11) NOT NULL,
PRIMARY KEY (`indicator_id`),
KEY `reserve_id` (`reserve_id`) USING BTREE,
CONSTRAINT `reserve_indicator_ibfk_1` FOREIGN KEY (`reserve_id`) REFERENCES `reserve_master` (`reserve_id`)
) ENGINE=InnoDB AUTO_INCREMENT=35 DEFAULT CHARSET=latin1
CREATE TABLE `reserve_details` (
`details_id` int(11) NOT NULL AUTO_INCREMENT,
`reserve_id` int(11) NOT NULL,
`title` varchar(255) NOT NULL,
`description` varchar(255) NOT NULL,
`status` varchar(255) NOT NULL,
`location` varchar(255) NOT NULL,
PRIMARY KEY (`details_id`),
KEY `reserve_id` (`reserve_id`) USING BTREE,
CONSTRAINT `reserve_details_ibfk_1` FOREIGN KEY (`reserve_id`) REFERENCES `reserve_master` (`reserve_id`)
) ENGINE=InnoDB AUTO_INCREMENT=35 DEFAULT CHARSET=latin1
All i had tried is to get the right available on the 2nd of febuary 2018 and not the others.
The code is mess, I'm very sorry;
Problem is solved already;
I use eventLimit on fullcalendar and set it to 1
Count = 5 not subtracted by the num_rows
and few set on if-statement
Few answer I used rows and res as it is easy to indicate for results and rows, and i used it same as other queries.
Thank you.
My business has a simple invoicing system written by me a few years ago in php with a MySQL database. When I wrote it, I knew just enough to be dangerous. While it does work, a lot of the code is very inefficient, slow and poorly conceived, so I'm re-writing some of it in the hopes of fixing it.
There are two tables: customers and transactions. When invoices need to be created, there are two queries, the second unfortunately in a while loop. Here is some simplified pseudo-code to show what I'm doing:
// Get list of customers who owe something
SELECT name, address
FROM customers
WHERE
(SELECT COUNT(*) FROM transactions
WHERE transactions.customer_id = customers.id
AND owed > 0)
> 0
ORDER BY address
//loop through that result and query the transactions table to get a
//list of charges for each customer. Like:
while ($row = customer_array()) {
echo name_and_address;
SELECT * FROM transactions WHERE id = $row['customer_id']
AND owed = TRUE ORDER BY date
while ($row = transactions_array()) {
echo each_transaction_row;
}
So obviously the nested loops, subqueries and queries in loops are bad, worse and slow. I've tried joins but can't seem to figure out how to make them work correctly in this context. What is the correct way to do something like this? Can it be done with a single query?
Table structure:
CREATE TABLE `customer_data` (
`account_id` int(6) NOT NULL AUTO_INCREMENT,
`service_address` varchar(255) DEFAULT NULL,
`service_zipcode` int(5) DEFAULT NULL,
`first_name` varchar(255) DEFAULT NULL,
`last_name` varchar(255) DEFAULT NULL,
`billing_address1` varchar(255) DEFAULT NULL,
`billing_address2` varchar(255) DEFAULT NULL,
`billing_city` varchar(255) DEFAULT NULL,
`billing_state` varchar(2) DEFAULT NULL,
`billing_zipcode` varchar(20) DEFAULT NULL,
`phone1` varchar(100) DEFAULT NULL,
`phone2` varchar(100) DEFAULT NULL,
`quoted_price` int(6) DEFAULT NULL,
`current_price` int(6) DEFAULT NULL,
`original_interval` varchar(10) DEFAULT NULL,
`current_interval` varchar(7) DEFAULT NULL,
`temp_interval` int(5) NOT NULL,
`email` varchar(100) DEFAULT NULL,
`origin_date` varchar(20) DEFAULT NULL,
`remarks` text,
`crew_remarks` text NOT NULL,
`customer_type` varchar(10) DEFAULT NULL,
`perm_crew_assign` int(5) NOT NULL DEFAULT '0',
`temp_crew_assign` int(5) NOT NULL,
`date_last_service` date DEFAULT NULL,
`next_scheduled_date` date DEFAULT NULL,
`excluded_days` varchar(255) NOT NULL,
`special_instructions` text NOT NULL,
`inactive` tinyint(1) NOT NULL DEFAULT '0',
`location` varchar(255) NOT NULL,
`sent_letter` date NOT NULL,
`date_added` datetime NOT NULL,
`email_notify` tinyint(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`account_id`)
) ENGINE=InnoDB AUTO_INCREMENT=1687 DEFAULT CHARSET=utf8
CREATE TABLE `transactions` (
`transaction_id` int(10) NOT NULL AUTO_INCREMENT,
`account_id` int(6) NOT NULL,
`crew_id` int(3) NOT NULL,
`date_performed` date NOT NULL,
`date_recorded` datetime NOT NULL,
`price` int(6) NOT NULL,
`amount_paid` int(6) NOT NULL,
`description` varchar(255) NOT NULL,
`user` varchar(255) NOT NULL,
`status` int(1) NOT NULL DEFAULT '1',
PRIMARY KEY (`transaction_id`)
) ENGINE=InnoDB AUTO_INCREMENT=69233 DEFAULT CHARSET=latin1
You can do this with one query and one loop. You can join the two tables and sort the result. Then when the name changes, you know you are starting a new customer.
SELECT c.name, c.address, t.*
FROM customers AS c
JOIN transactions AS t ON (t.customer_id = c.customers.id)
WHERE c.owed > 0
ORDER BY `c`.`name`
// Data looks like this
// | customer.name | customer.address | transaction.* |
$prevName = null;
$prevCustTransList = [];
$prevCustTransTotalOwed = 0;
while ($row = result_array()) { // pseudo code
if ( $row['c.name'] != $prevName ) {
if ( $prevName != null && $prevCustTransTotalOwed > 0 ) {
echo $prevName; // pseudo code
foreach($prevCustTransList as $t) {
echo $t; // pseudo code
}
}
$prevName = $row['c.name'];
$prevCustTransTotalOwed = 0;
$prevCustTransList = [];
}
// Keep the list of transactions to print out later.
$prevCustTransList[] = transaction data; // pseudo code
// Keep a running total of the amount owed
// Replace with actual calculation
$prevCustTransTotalOwed += transaction.price;
}
Yes you can do this in one query, ordering by account_id. In your 'transactions' table account_id should be indexed, since you'll be using it to join to 'customer_data'.
$st_date = '2016-01-01';
$end_date = '2016-02-01';
//select sql
SELECT c.name,c.account_id, ts.*
FROM customer_data c
INNER JOIN transactions ts ON (c.account_id = ts.account_id)
WHERE ts.price > 0 and ts.date_performed between $st_date and $end_data
$current_account_id = null;
while ($row = $results) {
if ($current_account_id != $row['account_id']) {
//print customer name, details, etc..
}
//print the transactions detail
echo $results['transaction_id']; //etc...
//keep track of the current customer
$current_account_id = $row['account_id']
}
I'm developing a page to edit board meetings and I want to display all board members who did not attend specific meeting as a checkox located below who attend as an edit in case of user want to add more so I did this:
My code:
$q = "SELECT * FROM `boardteam`";
$r = mysql_query($q);
while ($dbfield = mysql_fetch_assoc($r))
{
$member_id =$dbfield['nationalID'];
$query = "SELECT `attendance` FROM `meetingattendance` WHERE `meetingID` = '$mid' AND `attendance`!= '$member_id'";
$res = mysql_query($query);
if ($res)
{
$tname ="";
switch ($dbfield['titleName'])
{
case "Dr":
$tname .= "د.";
break;
case "Ms":
$tname .= "السيدة.";
break;
case "Mr":
$tname .= "السيد.";
break;
}
$At .= "<input type='checkbox' name='moreAttendence[]' dir='rtl' value=".$dbfield['nationalID']."><div class='styled-checkbox'>".$tname." ".$dbfield['fName']." ".$dbfield['sName']." ".$dbfield['lName']."</div><br>";
}
}
DB:
CREATE TABLE `boardteam` (
`nationalID` int(10) NOT NULL,
`titleName` char(2) NOT NULL,
`fName` char(20) NOT NULL,
`sName` char(20) NOT NULL,
`lName` char(20) NOT NULL,
`gender` char(1) NOT NULL,
`birthDate` date DEFAULT NULL,
`materialStatus` char(15) DEFAULT NULL,
`jobTitle` varchar(100) NOT NULL,
`jobLocation` varchar(20) DEFAULT NULL,
`employer` varchar(100) DEFAULT NULL,
`email` varchar(100) NOT NULL,
`photo` varchar(255) DEFAULT NULL,
`academicGrade` char(15) DEFAULT NULL,
`employmentStartDate` date NOT NULL,
`employmentEndDate` date NOT NULL,
`employmentType` char(20) DEFAULT NULL,
`employmentStatus` char(15) DEFAULT NULL,
`jobStartDate` date DEFAULT NULL,
`jobNumber` int(10) DEFAULT NULL,
`cv` varchar(255) DEFAULT NULL,
PRIMARY KEY (`nationalID`)
)
CREATE TABLE `meetingattendance` (
`meetingID` int(11) NOT NULL,
`attendance` int(10) DEFAULT NULL,
`absence` int(10) DEFAULT NULL,
`reason` varchar(255) DEFAULT NULL,
`additionalAttendance` varchar(255) DEFAULT NULL,
KEY `absence` (`absence`),
KEY `meeingID` (`meetingID`),
KEY `attendance` (`attendance`),
CONSTRAINT `meetingattendane_ibfk_1` FOREIGN KEY (`meetingID`) REFERENCES `boardmeetings` (`meetingID`),
CONSTRAINT `meetingattendane_ibfk_2` FOREIGN KEY (`attendance`) REFERENCES `boardteam` (`nationalID`),
CONSTRAINT `meetingattendane_ibfk_3` FOREIGN KEY (`absence`) REFERENCES `boardteam` (`nationalID`)
)
With my code I got all board members including who attend, How to fix that ??
You need to use a LEFT JOIN in order to find people in the boardTeam who were not in a specific meeting. eg:
SELECT b.*, m.attendance
FROM boardTeam b
LEFT JOIN meetingattendance m
ON b.nationalID = m.attendance AND m.meetingID = $mid
WHERE m.meetingID IS NULL
If you want to get ALL board members, and then determine within PHP if they attended the meeting or not, simply remove the m.attendance IS NULL clause, as such:
SELECT b.*, m.attendance as attendance
FROM boardTeam b
LEFT JOIN meetingattendance m
ON b.nationalID = m.attendance AND m.meetingID = $mid
and now when you loop through the response rows in php, you can test as such (assuming you fetch your rows one by one into a $row variable):
if($row['attendance'] != null)
{
// attended meeting
}
else
{
// did not attend meeting
}
Also, as mentioned in the comments, use mysqli, or pdo instead of pure mysql_ functions
Example fiddle here: http://sqlfiddle.com/#!9/ba7d4/6
I have a table with over 10 thousand registers right now, and they start to run so slow.
I have the following code:
COUNT
$SqlCount = "SELECT tabnews.New_Id
FROM tabnew WHERE New_Id <> '' AND New_Status = 1";
$QueryCount = mysql_query($SqlCount, $Conn) or die(mysql_error($Conn));
$NumCount = mysql_num_rows($QueryCount);
$recordCount = $NumCount;
PAGINATION
if (!$id) $p = 1;
else $p = $id;
$pageSize = 16;
$itemIni = ($pageSize*$p)-$pageSize;
$totalPage = ceil($recordCount/$pageSize);
SHOW
$Sql52 = "SELECT New_Id, New_Nome, New_Data, New_Imagem FROM tabnews WHERE New_Status = 1 ORDER BY New_Id DESC LIMIT $itemIni, $pageSize ";
$Query52 = mysql_query($Sql52, $Conn);
while($Rs52 = mysql_fetch_array($Query52)){
// ECHO RESULTS
}
MY DATABASE:
CREATE TABLE IF NOT EXISTS `tabnews` (
`New_Id` int(11) NOT NULL AUTO_INCREMENT,
`Franquia_Id` text NOT NULL,
`New_Slide` int(2) NOT NULL,
`Categoria_Id` int(2) NOT NULL,
`New_Nome` varchar(255) NOT NULL,
`New_Data` date NOT NULL,
`New_Imagem` varchar(75) NOT NULL,
`New_Status` int(11) NOT NULL,
PRIMARY KEY (`New_Id`),
KEY `idx_1` (`New_Status`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=10490 ;
Any ideas on how I can make this run faster?
I have a dedicated server running CENTOS.
This:
New_Id <> ''
What does this do? It casts every single one of your INT primary key to string to compare it to a string. Why would you compare it to a string? It cannot be '' by definition, omit that New_Id <> '' from your WHERE clause.
20 seconds is very weird for such a little table.
I have a very similar table with almost 4 million rows and your both SQL statements takes less than 0.002 sec.
CREATE TABLE IF NOT EXISTS `tasks` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`status` varchar(10) COLLATE utf8_unicode_ci NOT NULL DEFAULT 'open',
`method` varchar(10) COLLATE utf8_unicode_ci NOT NULL DEFAULT 'GET',
`url` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
`params` text COLLATE utf8_unicode_ci,
`response` text COLLATE utf8_unicode_ci,
`executed_by` varchar(50) COLLATE utf8_unicode_ci DEFAULT '',
`execute_at` datetime DEFAULT NULL,
`created` datetime NOT NULL,
`modified` datetime NOT NULL,
PRIMARY KEY (`id`),
KEY `status` (`status`),
KEY `modified` (`modified`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci AUTO_INCREMENT=3839270 ;
-
SELECT COUNT(id) FROM tasks WHERE status='done';
---> Query took 0.0008 sec
-
SELECT id, status, method, url FROM tasks WHERE status='done' ORDER BY id DESC LIMIT 200, 100;
---> Query took 0.0011 sec
Observations:
You should use SELECT COUNT(New_Id)...
New_id <> '' doesn't make sense. New_id can't be empty or NULL
Set the length of New_Status to something that match the values you store there
Try turning off logging: SET GLOBAL general_log = 'OFF';
Update your server packages (specially MySQL)
Is it a dedicated server only for the database?
Is the server running other things? (run 'top' and 'uptime' to check it status)