PHP updating large database - php

I have a large database (28k entries in this particular table one table) and I need to append some HTML tags to the front and back of every column in a table.
Here is my code:
try
{
$conn = new PDO("mysql:host=$host;dbname=$dbname", $username, $password);
if(!$conn)
{
echo "Error in connecting to the database.";
}
$conn->setAttribute( PDO::ATTR_ERRMODE, PDO::ERRMODE_WARNING );
$query = $conn->query("SELECT `id`, `introtext` FROM *TABLE* WHERE id >= 41155");
$query->setFetchMode(PDO::FETCH_OBJ);
//For each row in the table
while($row = $query->fetch())
{
$introtext = '<span class="*SPAN CLASS*">' . $row->introtext . '</span>';
$update_query = $conn->prepare("UPDATE *TABLE* SET introtext = ? WHERE id = ?");
if ($query->execute(array($introtext, $row->id)))
echo $row->id . " Done <br>";
else
echo $row->id . " Err<br>";
}
} catch(PDOexception $e) {
echo $e->getMessage();
}
$conn = null;
When I run the script, it outputs 41155 Done 4132 times. I'm not sure the logic here, but any help to get this working is appreciated.

I agree with Dagon that the database is not the place for that (what if tomorrow you decide that <span> should wrap another HTML tag?).
Anyway, it sounds like a one-time operation, so I wouldn't use PHP. Just run a MySQL client (the command line mysql, or Workbench, and use a query like this:
UPDATE *TABLE*
SET introtext = CONCAT('<span class="*SPAN CLASS*">', introtext, '</span>')
WHERE id >= 41155
One note about your current code: you're never executing the UPDATE query! You just prepare the statement, then instead of executing $update_query, you're executing $query again! That's why you're always printing the same id.

Related

How to get the value of expression from LAST_INSERT_ID(`my_column`+1)?

DB Type: MariaDB
Table Engine: InnoDB
I have a table where inside it has a column with a value which is being incremented (not auto, no inserting happens in this table)
When I run the following SQL query in phpMyAdmin it works just fine as it should:
UPDATE `my_table`
SET `my_column` = LAST_INSERT_ID(`my_column` + 1)
WHERE `my_column2` = 'abc';
SELECT LAST_INSERT_ID();
The above returns me the last value for the my_column table when the query happened. This query was taken directly from the mysql docs on locking: https://dev.mysql.com/doc/refman/8.0/en/innodb-locking-reads.html (to the bottom) and this seems to be the recommended way of working with counters when you don't want it to be affected by other connections.
My PDO:
try {
$conn = new PDO("mysql:host=$servername;dbname=$dbname", $username, $password);
// set the PDO error mode to exception
$conn->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$sql = "UPDATE `my_table`
SET `my_column` = LAST_INSERT_ID(`my_column` + 1)
WHERE `my_column2` = 'abc';
SELECT LAST_INSERT_ID();";
// Prepare statement
$stmt = $conn->prepare($sql);
// execute the query
$stmt->execute();
$result = $stmt->fetchColumn(); // causes general error
$result = $stmt->fetch(PDO::FETCH_ASSOC);// causes general error
// echo a message to say the UPDATE succeeded
echo $stmt->rowCount() . " records UPDATED successfully";
} catch(PDOException $e) {
echo $sql . "<br>" . $e->getMessage();
}
$conn = null;
Exact error SQLSTATE[HY000]: General error, If I remove the lines where I try to get the result, it updates the column, but I still do not have a return result... how do I perform that update query and get the select result all in one go like I do when I run it in phpMyAdmin? This all needs to happen in one go as specified by the MySQL docs so I don't have issues where two connections might get the same counter.
There is no need to perform SELECT LAST_INSERT_ID();. PDO will save that value automatically for you and you can get it out of PDO.
Simply do this:
$conn = new PDO("mysql:host=$servername;dbname=$dbname;charset=utf8mb4", $username, $password, [
PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION
]);
$sql = "UPDATE `my_table`
SET `my_column` = LAST_INSERT_ID(`my_column` + 1)
WHERE `my_column2` = 'abc'";
// Prepare statement
$stmt = $conn->prepare($sql);
// execute the query
$stmt->execute();
$newID = $conn->lastInsertId();
lastInsertId() will give you the value of the argument evaluated by LAST_INSERT_ID().

How to use SET and INSERT in one php variable?

This is not duplicated question, since I am asking how to use SET and INSERT in one PHP variable, there no any questions about AUTO_INCREMENT...
I have below page:
<?php
function genWO(){
$dbtype = "MySQL";
$username = "user";
$password = "pass";
$hostname = "10.10.10.10";
$dbname = "TABLES";
//connection to the database
$conn = new mysqli($hostname, $username, $password);
if ($conn->connect_error) {
die("Connection failed: " . $conn->connect_error);
}
$insertNewWONum = "SET #MAX_WO = (SELECT max(WO_NUM) + 1 from GENERIC_TABLES.WO_NUMBERS); INSERT INTO GENERIC_TABLES.WO_NUMBERS (WO_NUM, WO_REQUESTOR) values (#MAX_WO, `test`)";
if ($conn->query($insertNewWONum) === TRUE) {
echo "New record created successfully". "<br>";
} else {
echo "Error: " . $insertNewWONum . "<br>" . $conn->error;
}
$getWONum = "SELECT LPAD(max(WO_NUM) ,6,0) as NEW_WO_NUM from GENERIC_TABLES.WO_NUMBERS";
$result = $conn->query($getWONum);
if ($result->num_rows > 0) {
// output data of each row
while($row = $result->fetch_assoc()) {
echo "New WO Number: " . $row["NEW_WO_NUM"]. "<br>";
}
} else {
echo "0 results";
}
//close the connection
$conn->close();
}
?>
Since it is not allowed to use INSERT and SELECT for the same table in one query, I am trying to set variable and use it in INSERT query:
$insertNewWONum = "SET #MAX_WO = (SELECT max(WO_NUM) + 1 from GENERIC_TABLES.WO_NUMBERS); INSERT INTO GENERIC_TABLES.WO_NUMBERS (WO_NUM, WO_REQUESTOR) values (#MAX_WO, `test`)";
But it doesnt work, though it works fine if I am using this query in terminal.
Can anyone let me know how to achieve it please?
Since it is not allowed to use INSERT and SELECT for the same table in one query
All issues with your approach aside of course you can use INSERT and SELECT in one statement
INSERT INTO WO_NUMBERS (WO_NUM, WO_REQUESTOR)
SELECT COALESCE(MAX(WO_NUM), 0) + 1, 'Test'
FROM WO_NUMBERS;
Here is SQLFiddle demo.
Now what you need to realize is that your approach is unsafe for concurrent access. If this code is executed from two or more sessions simultaneously some or all of them may generate the same number effectively breaking your application.
Use AUTO_INCREMENT instead.
CREATE TABLE WO_NUMBERS
(
WO_NUM INT PRIMARY KEY AUTO_INCREMENT,
WO_REQUESTOR VARCHAR(32)
);
INSERT INTO WO_NUMBERS (WO_REQUESTOR) VALUES ('Test');
Here is SQLFiddle demo.
If for some reason you can't use AUTO_INCREMENT directly (and I honestly don't see why you couldn't) i.e. you need to add prefixes or augment the generated id/code in some way you can look this answer.

PHP MySQL PDO TextArea Where clause with condition checks

I have a page as shown in the screenshot below. The idea is to enter the bus number and the list of all stops on a particular route, one per line.
The stops are already stored in a database table called 'stops' I need the ID of each stop from the textarea. My current code only gets the ID of the last stop in the textarea. I feel like I am missing something. 'busnumber' is my textfield and 'busroute' is my textarea. I would appreciate if anyone can point me out on what I need to change in order to get the ID of each stop entered in the textarea as an array. Thanks for your time in advance.
try {
$conn = new PDO("mysql:host=$host;dbname=$dbname", $username, $password);
if(isset($_POST["busnumber"]) && isset($_POST["busroute"])){
$stops = explode(PHP_EOL, $_POST["busroute"]);
$stopsArray = '"' . implode('","', $stops) . '"';
$sql = "SELECT * FROM stops WHERE stop_name IN ($stopsArray)";
echo $sql."</br>";
$query = $conn->prepare($sql);
$query->execute();
$query->setFetchMode(PDO::FETCH_ASSOC);
$results = $query->fetchAll();
foreach($results as $result){
echo $result['stop_id'].' '.$result['stop_name'].'</br>';
}
}
} catch (PDOException $pe) {
die("Could not connect to the database $dbname :" . $pe->getMessage());
}
UPDATE 1
I changed the code as follows
$conn = new PDO("mysql:host=$host;dbname=$dbname", $username, $password);
if(isset($_POST["busnumber"], $_POST["busroute"])){
$stops = explode(PHP_EOL, $_POST["busroute"]);
foreach($stops as $stop){
$sql = "SELECT * FROM stops WHERE stop_name = '".$stop."'";
$statement = $conn->query($sql);
echo $sql.'</br>';
$statement->setFetchMode(PDO::FETCH_ASSOC);
$results = $statement->fetchAll();
foreach($results as $result){
echo $result['stop_id'].' '.$result['stop_name'];
}
$statement = null;
}
}
and still get the same output, it only gives me the ID of the last item inside the textarea
I just realized that you have working code displayed above. I'm sorry for giving answers before (see history if ya want) that are already there above (*haha). Here, I've updated the code of yours (the first one). I changed the part where you display the result:
try {
$conn = new PDO("mysql:host=$host;dbname=$dbname", $username, $password);
if(isset($_POST["busnumber"]) && isset($_POST["busroute"])){
$stops = explode(PHP_EOL, $_POST["busroute"]);
$stopsArray = '"' . implode('","', $stops) . '"';
$sql = "SELECT * FROM stops WHERE stop_name IN ($stopsArray)";
$query = $conn->prepare($sql);
$query->execute();
if ($query->rowCount() > 0){
while ($row = $query->fetch(PDO::FETCH_ASSOC)){
echo '<br/>'.$row['stop_id'].' '.$row['stop_name'];
}
}else{
echo "No records found...";
}
}
} catch (PDOException $pe) {
die("Could not connect to the database $dbname :" . $pe->getMessage());
}
Note: As I have read some tutorials, using while loop is conventional than fetchAll().
Your db structure could help to give you a more precise info. Since it is lacking, I am going to speculate over it a little just to give you an idea.
Each bus (route) has several stops. That means there must be a foreign key defined in stops table and it points to route table.
In order to select all stops from stops table on a given route, what you need to do is to modify your select statement as follows:
Semantic code
SELECT * from stops where stops.routeId = <aGivenRouteId>
or
SELECT * from stops where stops.routeId in (an Array Of Route IDs)
Keep in mind that second form is slower.
I hope it makes sense to you.
__UPDATE__
If this is the case, there might be a many to many relationship. If this is the case, look for another table which connects stops and routes. That table should contain just route_id and stop_id in order to associate them to each other. From that table, you can select stop_ids on a given route, and then from stops table you can get names of the stops.
Hope it helps.
__UPDATE2__
Oh I see. You may need to modify your screen a bit. Something like this:
+Add Route-----------------------------------------+
|Bus Number |
|__________ |
| |
|Stops In This Route All Stops |
+--------------------------+-+--+------------------+
|Stop 2 |x| |Stop 1 |
|Stop 5 |x| |Stop 2 |
|Stop 9 |x|<<|Stop 3 |
| | |Stop 4 V|
+--------------------------+-+--+------------------+
|Add Route |
+----------------------------+--+------------------+
In All Stops part, you can show all the stops in DB (in your stops table). For stops in this route part, I suggest you to create another table where you associate stops and routes, basically a table containing stop_id and route_id.
Would it work for you this way?
I FIXED THE ERROR. The solution for anyone out there is simple, the PHP explode function was used to split the contents of a textarea into separate lines but it doesn't work if you use explode() with PHP_EOL. PHP EOL tells you the server's newline character from what I understand. I used the preg_split instead to perform the splitting and it works on both my localhost that runs on Windows and my server that runs on Linux. Thank you everyone for your help!!!
$conn = new PDO("mysql:host=$host;dbname=$db;charset=$charset", $user, $pass);
$conn->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
if(isset($_POST["busnumber"], $_POST["busroute"])){
$stops = preg_split("/\\r\\n|\\r|\\n/", $_POST['busroute']);
$sql = 'SELECT * FROM stops WHERE stop_name LIKE :stop';
$statement = $conn->prepare($sql);
foreach($stops as $stop){
$statement->bindValue(':stop', $stop);
$statement->execute();
while($result = $statement->fetch()){
echo $result['stop_id'].' '.$result['stop_name'].'</br>';
}
}
}
$conn = new PDO("mysql:host=$host;dbname=$db;charset=$charset",$user,$pass);
$conn->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
if(isset($_POST["busnumber"], $_POST["busroute"]))
{
$stops = preg_split("/\\r\\n|\\r|\\n/$_POST['busroute']);
$sql = 'SELECT * FROM stops WHERE stop_name LIKE :stop';
$statement = $conn->prepare($sql);
foreach($stops as $stop){
$statement->bindValue(':stop', $stop);
$statement->execute();
while($result = $statement->fetch())
{
echo $result['stop_id'].' '.$result['stop_name'].'</br>';
}
}

mysql pdo hit counter

I can't make the counter add 1 into the DB and the value increment every time when somebody go to the page... only works when is reloaded.
Where is the error in my code? Can you help me with this issue?
I don't know if you need my entire html page, if you want I page it. the url have the id is show like this: post.php?id_blog=4
THe code:
<?php
try {
$query = "SELECT id_blog, blog_titulo, blog, vistoblog FROM BLOG WHERE id_blog = ?";
$stmt = $conn->prepare( $query );
$stmt->bindParam(1, $_REQUEST['id_blog']);
$stmt->execute();
$row = $stmt->fetch(PDO::FETCH_ASSOC);
$id_blog = $row['id_blog'];
$blog_titulo = $row['blog_titulo'];
$blog = $row['blog'];
$vistoblog = $row['vistoblog'];
}catch(PDOException $exception){
echo "Error: " . $exception->getMessage();
}
try{
$visto = $vistoblog + 1;
$sql = "UPDATE BLOG SET
vistoblog = :vistoblog
WHERE id_blog = :id_blog";
$stmt = $conn->prepare($sql);
$stmt->bindParam(':vistoblog', $visto, PDO::PARAM_STR);
$stmt->bindParam(':id_blog', $id_blog, PDO::PARAM_INT);
$stmt->execute();
}catch(PDOException $exception){
echo "Error: " . $exception->getMessage();
}
?>
I recommend making your SQL just like this:
UPDATE BLOG SET
vistoblog = vistoblog + 1
WHERE id_blog = :id_blog
The reason is to avoid a race condition. What if two people visit the page simultaneously, and both PHP threads read vistoblog value 123, add 1, and both try to increment it to value 124?
By using the expression above, you don't have to read the current value, and you avoid the chance of a race condition like that.

PDO prepared statements in functions [duplicate]

This question already has answers here:
How do I loop through a MySQL query via PDO in PHP?
(3 answers)
Closed 9 years ago.
I am currently using MySQL with PHP but am looking to start MySQLi or PDO
I have while loops like:
$sql="select from ... ";
$rs=mysql_query($sql);
while($result=mysql_fetch_array($rs))
{
$sql2="select from table2 where id = $result["tbl1_id"] ";
}
If I put my MySQLi or PDO queries into a function how can I run things like the above? Doing while loops with queries inside the while loops?
Or is if easier to not do the functions at all and just run the prepared statements as normal?
You wouldn't. And to be honest.. Even in the old days you would not do it this way, but like this:
$sql="select from ... ";
$rs=mysql_query($sql);
$ids = array()
while($result=mysql_fetch_array($rs))
{
$ids[] = $result["tbl1_id"];
}
$sql2="select from table2 where id in ".implode(',', $ids) .";
Or even better, you use a join to run the query just once, on all the tables that need to provide info.
In PDO you can do the same thing. Get all the ID's and the execute a query
I usually take the approach of preparing the query and not using a function. Also I am not clear as to what exactly it is that you want. You want to make your queries as quick and efficient as possible so you should not look to run a while look within another while loop.
This is how my PDO queries usually look
My connection:
$host = "localhost";
$db_name = "assignment";
$username = "root";
$password = "";
try {
$connection = new PDO("mysql:host={$host};dbname={$db_name}", $username, $password);
}catch(PDOException $exception){ //to handle connection error
echo "Connection error: " . $exception->getMessage();
}
MY query:
$query = "SELECT * FROM Table";
$stmt = $connection->prepare( $query );
$stmt->execute();
while ($row = $stmt->fetch(PDO::FETCH_ASSOC))
{
extract($row);
}
It's a duplication question like oGeez say, you have to learn how to code PDO in PHP and other before asking question,
this is the answer:
$dbh = new PDO("mysql:host=" . HOST . ";dbname=" . BASE, USER, PASS, array(PDO::MYSQL_ATTR_INIT_COMMAND => "SET NAMES utf8"));
$dbh->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$query = 'SELECT * FROM table';
$stmt = $dbh->query($query);
$items = $stmt->fetchAll(PDO::FETCH_OBJ);
foreach($items as $item {
print_r($item);
}
the main reason to put it in a function would be if you use the query in multiple files. i have a web app with many queries and i like to keep them in a separate file so that they're easier to track down if i need to make changes. the main thing is that you 1) have to pass your database as a parameter and 2) return the results
function pdoquery($db, $parameter){
$query = "SELECT * FROM table WHERE column=?";
$stmt = $db->prepare($query);
$stmt->bindValue(1, $parameter, PDO::PARAM_STR); //or PARAM_INT
if (!$stmt->execute()) {
echo "Could not get results: (" . $stmt->errorCode . ") " . $stmt->errorInfo;
exit;
}
else
$result = $stmt->fetch();
$db = null;
return $result;
}
but as others have mentioned, if its only used once, there's no need for a function, and looping through the results is best done outside of the function as well. however, it is possible to do it inside the function if you want to.

Categories