How to streamline this (rather large) MySQL query in PHP - php

I thought I was pretty good at this sort of thing but...
I have the following query from a customer table that is about 8,500 rows and an address table that is about the same number of rows.
This query takes about 10 seconds to complete and I can't figure how to get it to milliseconds.
What am I doing wrong?
<select>
<option value=""></option>
<?php // get the customers
$sql = "
SELECT `cust`.`custid`, `cust`.`custname`, `cust`.`custactive`, `address`.`addtype`, `address`.`address1`, `address`.`addcity`, `address`.`addstate`
FROM `cust`
LEFT Join `address`
ON `cust`.`custid` = `address`.`addcustid`
and `address`.`addtype` = 'b'
WHERE `cust`.`custactive` = 'y'"
;
$result = mysqli_query($con,$sql) or die('Query failed: Could not get list of CLIENTS: ' . mysqli_error($con)); // query
while ($row = mysqli_fetch_array($result)) {
$custid = $row['custid'];
$space="";
$endingspaces = 4-(2*strlen($prodid));
for($count=1; $count<$endingspaces; $count++){
$space .=" ";
}
$custid = $row['custid'];
$custname = substr($row['custname'],0,15);
$address1 = substr($row['address1'],0,15);
$addcity = substr($row['addcity'],0,15);
$addstate = $row['addstate'];
print "<option value=\"$custid\">$custid: $space$custname, $address1, $addcity, $addstate</option>";
}
?>
</select>
DDL:
CREATE TABLE `cust` (
`custid` int(11) DEFAULT NULL,
`custname` varchar(45) DEFAULT NULL,
`custactive` varchar(255) DEFAULT NULL,
`custcreated` varchar(255) DEFAULT NULL,
`custmodified` varchar(255) DEFAULT NULL,
`cust_dataease_custid` int(11) DEFAULT NULL,
`custtype` varchar(5) DEFAULT NULL,
`custrep` int(11) DEFAULT NULL,
`custsource` varchar(5) DEFAULT NULL,
`custdiscount` decimal(65,30) DEFAULT NULL,
`custrepcomm` varchar(255) DEFAULT NULL,
`custsurcharge` varchar(255) DEFAULT NULL,
`custterms` varchar(12) DEFAULT NULL,
`custups` varchar(255) DEFAULT NULL,
`custnotes` varchar(255) DEFAULT NULL,
`custbilldif` varchar(5) DEFAULT NULL,
UNIQUE KEY `custid` (`custid`),
KEY `cust_dataease_custid_idx` (`cust_dataease_custid`),
KEY `custrep_idx` (`custrep`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
CREATE TABLE `address` (
`addid` int(11) unsigned NOT NULL AUTO_INCREMENT,
`addcustid` int(11) unsigned NOT NULL,
`addname` varchar(200) DEFAULT NULL,
`addtype` enum('b','s') NOT NULL DEFAULT 'b',
`address1` varchar(150) NOT NULL,
`address2` varchar(150) DEFAULT NULL,
`addcity` varchar(150) DEFAULT NULL,
`addstate` varchar(150) DEFAULT NULL,
`addstateother` varchar(150) DEFAULT NULL,
`addzip` varchar(150) DEFAULT NULL,
`addcountry` varchar(150) DEFAULT NULL,
`addnote` tinyblob,
PRIMARY KEY (`addid`),
KEY `add_cust_id_idx` (`addcustid`)
) ENGINE=InnoDB AUTO_INCREMENT=13037 DEFAULT CHARSET=utf8

John Green got me to thinking about the way information was being sent to and from the server, which led me to find the problem. Thanks John.
This is how I fixed it: Apparently, when my loop was iterating, it was "printing" each of the 8500 OPTION lines, which action (i didn't know), is a server-client interaction. So for each row, the server and browser had to have a conversation, which slowed everything down.
I fixed it by having the server compile the entire SELECT box on the server side, including the opening SELECT tags and interacting with the browser once through echoing my "add-to" variable: $select_box.
Now it takes like 1.2 seconds. I can live with that.
Thanks everyone for helping me chase this rabbit.
<?php // get the products
$select_box = "<select name=\"customer\" id=\"customer\" onChange=\"getcustinfo();\" data-placeholder=\"Choose a Customer\" class=\"chosen-select\" style=\"width:227px;\" tabindex=\"1\">
<option value=\"\"></option>";
$sql = "
SELECT *
FROM `cust`
LEFT Join `address`
ON `cust`.`custid` = `address`.`addcustid`
and `address`.`addtype` = 'b'
WHERE `cust`.`custactive` = 'y'"
;
$result = mysqli_query($con,$sql) or die('Query failed: Could not get list of CLIENTS: ' . mysqli_error($con)); // query
while ($row = mysqli_fetch_array($result)) {
foreach ($row as $key => $value){ ${$key} = $value; }
$space="";
$custname = substr($row['custname'],0,15);
$address1 = substr($row['address1'],0,15);
$addcity = substr($row['addcity'],0,15);
$select_box .= "<option value=\"$custid\">$custid: $space$custname, $address1, $addcity, $addstate</option>";
}
$select_box .="</select>";
echo $select_box;
?>

Related

Why won't my array get inserted into my Database?

I have this code and for some reason it won't get inserted into my database. It's basically taking an array, turning it into a string and then submit the values.
(If you need me to edit to show my whole code, I will do so)
Code I am having issues with down below
$array = array($RaceNumber,$Track,$Num,$HorseName,$Odds,$Color,$Jockey,$Trainer,$PostTime,$Course,$RaceDistance,$Win,$Place,$Show);
for ($a=0; $a<$Num; $a++) {
$dataArray=array($RaceNumber[$a],$Track[$a],$Num[$a],$HorseName[$a],$Odds[$a],$Color[$a],$Jockey[$a],$Trainer[$a],$PostTime[$a],$Course[$a],$RaceDistance[$a],$Win[$a],$Place[$a],$Show[$a]);
$dataArray--;
for ($j=0; $j<$Num; $j++) {
$RaceNumber=$dataArray[0];
$Track=$dataArray[1];
$Num=$dataArray[2];
$HorseName=$dataArray[3];
$Odds=$dataArray[4];
$Color=$dataArray[5];
$Jockey=$dataArray[6];
$Trainer=$dataArray[7];
$PostTime=$dataArray[8];
$Course=$dataArray[9];
$RaceDistance=$dataArray[10];
$Win=$dataArray[11];
$Place=$dataArray[12];
$Show=$dataArray[13];
$sql="INSERT INTO `$Date` (RaceNumber,Track,HorseNum,HorseName,Odds,Color,JockeyName,TrainerName,PostTime,Course,RaceDistance,Win,Place,Show) VALUES ('$RaceNumber','$Track','$Num','$HorseName','$Odds','$Color','$Jockey','$Trainer','$PostTime','$Course','$RaceDistance','$Win','$Place','$Show')";
echo $sql;
mysqli_query($query2,$sql);
}
}
when I echo my $sql I get
INSERT INTO 2018-09-20 (RaceNumber,Track,HorseNum,HorseName,Odds,Color,JockeyName,TrainerName,PostTime,Course,RaceDistance,Win,Place,Show) VALUES ('1','FingerLakes','1','','','Red','','','','Dirt','','none','none','none')
But when I do my query, it isn't inserting into database.
Part of my code where I create the datatable
<?php
if(isset($_POST['submit'])) {
$Date = $_POST['date'];
$sql = "CREATE TABLE IF NOT EXISTS `$Date` (
`Id` int NOT NULL AUTO_INCREMENT PRIMARY KEY,
`RaceNumber` varchar(255) NOT NULL,
`Track` varchar(255) NOT NULL,
`HorseNum` varchar(255) NOT NULL,
`HorseName` varchar(255) NOT NULL,
`Odds` varchar(255) NOT NULL,
`Color` varchar(255) NOT NULL,
`JockeyName` varchar(255) NOT NULL,
`TrainerName` varchar(255) NOT NULL,
`PostTime` varchar(255) NOT NULL,
`Course` varchar(255) NOT NULL,
`RaceDistance` varchar(255) NOT NULL,
`Win` varchar(255) NOT NULL,
`Place` varchar(255) NOT NULL,
`Show` varchar(255) NOT NULL
) ENGINE=MyISAM DEFAULT CHARSET=utf8"
;
$query2 = mysqli_connect('localhost','root','','Races');
$z= mysqli_query($query2, $sql) or die("Table already exist.. please try again");
echo "Your Table ".$Date." is successfully created <br/>";
$RaceNum = $_POST['RaceNum'];
$i=1;
I am receiving in my error log of
2018-09-20 16:00:59 9444 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'max_value' at position 4 to have type varbinary(255), found type varchar(255).
You are using a column named Show that's a reserved keyword in mysql, add backquotes to it and the insert query should work.
It's worth noting that you shouldn't name your table with only digits and hyphens.
For reference here is the complete list of the reserved keywords:
https://dev.mysql.com/doc/refman/8.0/en/keywords.html

Creating an invoice from SQL query without putting query in a loop

My business has a simple invoicing system written by me a few years ago in php with a MySQL database. When I wrote it, I knew just enough to be dangerous. While it does work, a lot of the code is very inefficient, slow and poorly conceived, so I'm re-writing some of it in the hopes of fixing it.
There are two tables: customers and transactions. When invoices need to be created, there are two queries, the second unfortunately in a while loop. Here is some simplified pseudo-code to show what I'm doing:
// Get list of customers who owe something
SELECT name, address
FROM customers
WHERE
(SELECT COUNT(*) FROM transactions
WHERE transactions.customer_id = customers.id
AND owed > 0)
> 0
ORDER BY address
//loop through that result and query the transactions table to get a
//list of charges for each customer. Like:
while ($row = customer_array()) {
echo name_and_address;
SELECT * FROM transactions WHERE id = $row['customer_id']
AND owed = TRUE ORDER BY date
while ($row = transactions_array()) {
echo each_transaction_row;
}
So obviously the nested loops, subqueries and queries in loops are bad, worse and slow. I've tried joins but can't seem to figure out how to make them work correctly in this context. What is the correct way to do something like this? Can it be done with a single query?
Table structure:
CREATE TABLE `customer_data` (
`account_id` int(6) NOT NULL AUTO_INCREMENT,
`service_address` varchar(255) DEFAULT NULL,
`service_zipcode` int(5) DEFAULT NULL,
`first_name` varchar(255) DEFAULT NULL,
`last_name` varchar(255) DEFAULT NULL,
`billing_address1` varchar(255) DEFAULT NULL,
`billing_address2` varchar(255) DEFAULT NULL,
`billing_city` varchar(255) DEFAULT NULL,
`billing_state` varchar(2) DEFAULT NULL,
`billing_zipcode` varchar(20) DEFAULT NULL,
`phone1` varchar(100) DEFAULT NULL,
`phone2` varchar(100) DEFAULT NULL,
`quoted_price` int(6) DEFAULT NULL,
`current_price` int(6) DEFAULT NULL,
`original_interval` varchar(10) DEFAULT NULL,
`current_interval` varchar(7) DEFAULT NULL,
`temp_interval` int(5) NOT NULL,
`email` varchar(100) DEFAULT NULL,
`origin_date` varchar(20) DEFAULT NULL,
`remarks` text,
`crew_remarks` text NOT NULL,
`customer_type` varchar(10) DEFAULT NULL,
`perm_crew_assign` int(5) NOT NULL DEFAULT '0',
`temp_crew_assign` int(5) NOT NULL,
`date_last_service` date DEFAULT NULL,
`next_scheduled_date` date DEFAULT NULL,
`excluded_days` varchar(255) NOT NULL,
`special_instructions` text NOT NULL,
`inactive` tinyint(1) NOT NULL DEFAULT '0',
`location` varchar(255) NOT NULL,
`sent_letter` date NOT NULL,
`date_added` datetime NOT NULL,
`email_notify` tinyint(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`account_id`)
) ENGINE=InnoDB AUTO_INCREMENT=1687 DEFAULT CHARSET=utf8
CREATE TABLE `transactions` (
`transaction_id` int(10) NOT NULL AUTO_INCREMENT,
`account_id` int(6) NOT NULL,
`crew_id` int(3) NOT NULL,
`date_performed` date NOT NULL,
`date_recorded` datetime NOT NULL,
`price` int(6) NOT NULL,
`amount_paid` int(6) NOT NULL,
`description` varchar(255) NOT NULL,
`user` varchar(255) NOT NULL,
`status` int(1) NOT NULL DEFAULT '1',
PRIMARY KEY (`transaction_id`)
) ENGINE=InnoDB AUTO_INCREMENT=69233 DEFAULT CHARSET=latin1
You can do this with one query and one loop. You can join the two tables and sort the result. Then when the name changes, you know you are starting a new customer.
SELECT c.name, c.address, t.*
FROM customers AS c
JOIN transactions AS t ON (t.customer_id = c.customers.id)
WHERE c.owed > 0
ORDER BY `c`.`name`
// Data looks like this
// | customer.name | customer.address | transaction.* |
$prevName = null;
$prevCustTransList = [];
$prevCustTransTotalOwed = 0;
while ($row = result_array()) { // pseudo code
if ( $row['c.name'] != $prevName ) {
if ( $prevName != null && $prevCustTransTotalOwed > 0 ) {
echo $prevName; // pseudo code
foreach($prevCustTransList as $t) {
echo $t; // pseudo code
}
}
$prevName = $row['c.name'];
$prevCustTransTotalOwed = 0;
$prevCustTransList = [];
}
// Keep the list of transactions to print out later.
$prevCustTransList[] = transaction data; // pseudo code
// Keep a running total of the amount owed
// Replace with actual calculation
$prevCustTransTotalOwed += transaction.price;
}
Yes you can do this in one query, ordering by account_id. In your 'transactions' table account_id should be indexed, since you'll be using it to join to 'customer_data'.
$st_date = '2016-01-01';
$end_date = '2016-02-01';
//select sql
SELECT c.name,c.account_id, ts.*
FROM customer_data c
INNER JOIN transactions ts ON (c.account_id = ts.account_id)
WHERE ts.price > 0 and ts.date_performed between $st_date and $end_data
$current_account_id = null;
while ($row = $results) {
if ($current_account_id != $row['account_id']) {
//print customer name, details, etc..
}
//print the transactions detail
echo $results['transaction_id']; //etc...
//keep track of the current customer
$current_account_id = $row['account_id']
}

How to write nested query MySQL PHP

I'm developing a page to edit board meetings and I want to display all board members who did not attend specific meeting as a checkox located below who attend as an edit in case of user want to add more so I did this:
My code:
$q = "SELECT * FROM `boardteam`";
$r = mysql_query($q);
while ($dbfield = mysql_fetch_assoc($r))
{
$member_id =$dbfield['nationalID'];
$query = "SELECT `attendance` FROM `meetingattendance` WHERE `meetingID` = '$mid' AND `attendance`!= '$member_id'";
$res = mysql_query($query);
if ($res)
{
$tname ="";
switch ($dbfield['titleName'])
{
case "Dr":
$tname .= "د.";
break;
case "Ms":
$tname .= "السيدة.";
break;
case "Mr":
$tname .= "السيد.";
break;
}
$At .= "<input type='checkbox' name='moreAttendence[]' dir='rtl' value=".$dbfield['nationalID']."><div class='styled-checkbox'>".$tname." ".$dbfield['fName']." ".$dbfield['sName']." ".$dbfield['lName']."</div><br>";
}
}
DB:
CREATE TABLE `boardteam` (
`nationalID` int(10) NOT NULL,
`titleName` char(2) NOT NULL,
`fName` char(20) NOT NULL,
`sName` char(20) NOT NULL,
`lName` char(20) NOT NULL,
`gender` char(1) NOT NULL,
`birthDate` date DEFAULT NULL,
`materialStatus` char(15) DEFAULT NULL,
`jobTitle` varchar(100) NOT NULL,
`jobLocation` varchar(20) DEFAULT NULL,
`employer` varchar(100) DEFAULT NULL,
`email` varchar(100) NOT NULL,
`photo` varchar(255) DEFAULT NULL,
`academicGrade` char(15) DEFAULT NULL,
`employmentStartDate` date NOT NULL,
`employmentEndDate` date NOT NULL,
`employmentType` char(20) DEFAULT NULL,
`employmentStatus` char(15) DEFAULT NULL,
`jobStartDate` date DEFAULT NULL,
`jobNumber` int(10) DEFAULT NULL,
`cv` varchar(255) DEFAULT NULL,
PRIMARY KEY (`nationalID`)
)
CREATE TABLE `meetingattendance` (
`meetingID` int(11) NOT NULL,
`attendance` int(10) DEFAULT NULL,
`absence` int(10) DEFAULT NULL,
`reason` varchar(255) DEFAULT NULL,
`additionalAttendance` varchar(255) DEFAULT NULL,
KEY `absence` (`absence`),
KEY `meeingID` (`meetingID`),
KEY `attendance` (`attendance`),
CONSTRAINT `meetingattendane_ibfk_1` FOREIGN KEY (`meetingID`) REFERENCES `boardmeetings` (`meetingID`),
CONSTRAINT `meetingattendane_ibfk_2` FOREIGN KEY (`attendance`) REFERENCES `boardteam` (`nationalID`),
CONSTRAINT `meetingattendane_ibfk_3` FOREIGN KEY (`absence`) REFERENCES `boardteam` (`nationalID`)
)
With my code I got all board members including who attend, How to fix that ??
You need to use a LEFT JOIN in order to find people in the boardTeam who were not in a specific meeting. eg:
SELECT b.*, m.attendance
FROM boardTeam b
LEFT JOIN meetingattendance m
ON b.nationalID = m.attendance AND m.meetingID = $mid
WHERE m.meetingID IS NULL
If you want to get ALL board members, and then determine within PHP if they attended the meeting or not, simply remove the m.attendance IS NULL clause, as such:
SELECT b.*, m.attendance as attendance
FROM boardTeam b
LEFT JOIN meetingattendance m
ON b.nationalID = m.attendance AND m.meetingID = $mid
and now when you loop through the response rows in php, you can test as such (assuming you fetch your rows one by one into a $row variable):
if($row['attendance'] != null)
{
// attended meeting
}
else
{
// did not attend meeting
}
Also, as mentioned in the comments, use mysqli, or pdo instead of pure mysql_ functions
Example fiddle here: http://sqlfiddle.com/#!9/ba7d4/6

mysql MAX() returns 0 instead of actual value

The following code returns 0 instead of the biggest number from the row order_id
if ($result_oid = $link->prepare("SELECT MAX(order_id) AS order_id FROM $table")) {
$result_oid->execute();
$obj = $result_oid->get_result()->fetch_object();
$oid_o = $obj->id;
$result_oid->close();
$oid = $oid_o + 1;
}
Here is a working example using the PHP mysql instead of mysqli (with the same mysql database):
mysql_connect($host, $user, $pwd) or die ("Couldn't connect to MySQL database.");
mysql_select_db($db) or die ("No Database found!");
$query = mysql_query('SELECT MAX(order_id) FROM airsale_list');
$result = mysql_fetch_array($query, MYSQL_NUM);
$max_order_id = $result[0];
$max_order_id = (int)$max_order_id;
$oid = $max_order_id++;
echo "<h4>order_id: $oid</h4>";
mysql_close();
Table structure
CREATE TABLE IF NOT EXISTS `airsale_list` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`cat` int(11) NOT NULL,
`title` varchar(255) NOT NULL,
`desc_small` varchar(1000) DEFAULT NULL,
`name` varchar(255) NOT NULL,
`lastupdate` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`price` int(20) NOT NULL,
`currency` varchar(3) NOT NULL DEFAULT 'EUR',
`total_time` varchar(255) DEFAULT NULL,
`engine` varchar(1000) DEFAULT NULL,
`engine_time` varchar(255) DEFAULT NULL,
`prop` varchar(1000) DEFAULT NULL,
`prop_time` varchar(255) DEFAULT NULL,
`exterior` varchar(2000) DEFAULT NULL,
`interior` varchar(2000) DEFAULT NULL,
`avionics` varchar(5000) DEFAULT NULL,
`add_info` varchar(5000) DEFAULT NULL,
`order_id` int(20) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=103;
I think you should be using $obj->order_id instead of $obj->id since you are grabbing the maximum value as order_id in the query.
if ($result_oid = $link->prepare("SELECT MAX(order_id) AS order_id FROM $table")) {
$result_oid->execute();
$obj = $result_oid->get_result()->fetch_object();
$oid_o = $obj->order_id;
$result_oid->close();
$oid = $oid_o + 1;
}

MySQL delete troubleshooting

I restarted the MySQL service and I attempted to use my PHP programs delete function to delete an existing row but I'm finding although the delete queries were counted the row was not deleted. I tried applying on delete cascade to the foreign key of the child table but that did not seem to have an effect. I'm wondering why the delete would be doing nothing.
CREATE TABLE `customers` (
`idcustomers` int(11) NOT NULL AUTO_INCREMENT,
`firstname` varchar(45) DEFAULT NULL,
`lastname` varchar(45) DEFAULT NULL,
`address1` varchar(45) DEFAULT NULL,
`address2` varchar(45) DEFAULT NULL,
`city` varchar(45) DEFAULT NULL,
`state` varchar(45) DEFAULT NULL,
`zip` varchar(45) DEFAULT NULL,
`phone` varchar(45) DEFAULT NULL,
`email` varchar(45) DEFAULT NULL,
`cell` varchar(45) DEFAULT NULL,
PRIMARY KEY (`idcustomers`),
UNIQUE KEY `idcustomers_UNIQUE` (`idcustomers`)
) ENGINE=InnoDB AUTO_INCREMENT=54 DEFAULT CHARSET=latin1
CREATE TABLE `events` (
`idevents` int(11) NOT NULL AUTO_INCREMENT,
`title` varchar(250) DEFAULT NULL,
`start` datetime DEFAULT NULL,
`end` datetime DEFAULT NULL,
`allday` varchar(50) DEFAULT NULL,
`url` varchar(1000) DEFAULT NULL,
`customerid` int(11) NOT NULL,
`memo` longtext,
`dispatchstatus` varchar(45) DEFAULT NULL,
PRIMARY KEY (`idevents`),
KEY `FK_events` (`customerid`),
CONSTRAINT `FK_events` FOREIGN KEY (`customerid`) REFERENCES `customers` (`idcustomers`) ON DELETE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=18 DEFAULT CHARSET=latin1
Com_delete 2
The PHP looks like this:
<?php
session_start();
date_default_timezone_set("America/Los_Angeles");
if($_SESSION['loggedin'] != TRUE)
{
header("Location: index.php");
}
require_once('../php.securelogin/include.securelogin.php');
$mysqli = new mysqli($ad_host, $ad_user, $ad_password, "samedaycrm");
if ($mysqli->connect_errno) {
printf("Connect failed: %s\n", $mysqli->connect_error);
exit();
}
$customerid = $_SESSION['customer_id'];
$tSQL = "delete from events where customerid = \"$customerid\"";
$result = $mysqli->query($tSQL);
$tSQL = "delete from customers where idcustomers = \"$customerid\"";
$result = $mysqli->query($tSQL);
echo $mysqli->error;
?>
Assuming that the customerid and idcustomers columns are both numeric it should be fine. You should not need to quote the variables in those queries btw, then you wouldnt need to escape them. You may try:
$tSQL = "delete from events where customerid = $customerid";
but it should not be any different than what you used already. Of course if you are not sure of the type of the column you can use:
$tSQL = "delete from events where customerid = '".$customerid."'";
or you can get away with:
$tSQL = "delete from events where customerid = '$customerid'";
but I have always hated that for some reason.
if all of that fails troubleshoot by spitting out the $customerid (or even the whole $tSQL) variable and then trying the query manually in phpmyadmin or toad or whatever db client you use, and see what it tells you. If it just says 0 rows affected, then run it like a select instead. Tailor to fit.

Categories