I used to use
<?php
$sql = "insert into test (owner) values ('owen')";
$db->autocommit(false);
if (!$db->query($sql))
$db->rollback();
else
$db->commit();
$db->close();
?>
However, today I run two insert php files in a same tables, without any action. It is simple like:
<?php
$sql = "insert into test (owner) values ('owen')"; //the other php is the same but replacing 'owen' to 'huhu'
for ($i = 0; $i < 100 * 1000; $i++) {
$db->query($sql);
}
$db->close();
?>
I run two php files in two different consoles. Then I got 200,000 records without any error. Does that mean using transaction manually is really not needed. As there are no conflicts.
You do not need transactions for this.
Transactions exist to be able to roll back half-finished changes to the database. These only occur when you have a set of multiple statements changing the database which might be interrupted inbetween. Then often only some of the statements have been execute which might leave the database in a state which is not 'clean' from the applications point of view.
A simple and good example is a money transfer between two tables:
first a removal from one table
then it is added to a second table
If this process is interrupted inbetween the money has vanished. That is not intended, you probably want to be able to rollback.
In your case however all statements are 'atomic', meaning they succeed or fail, but the databases state is always 'clean'. It does not matter in this if it is a single or multiple clients running the statements.
Related
I have 4 queries that run on a post. I've noticed that not all the queries are working if one of the first queries have no work to do.
Here are my queries
mysql_query("UPDATE invoice SET company='$company' WHERE company='$oldcompanyname'") or die(mysql_error());
mysql_query("UPDATE bill SET customer='$company' WHERE customer='$oldcompanyname'") or die(mysql_error());
mysql_query("UPDATE project SET customer='$company' WHERE customer='$oldcompanyname'") or die(mysql_error());
mysql_query("INSERT INTO activity_log (item, date) VALUES ('Customer Editted', NOW()) ") or die(mysql_error());
To give you an example, the first one runs ok. But the second one has no work to do because the data in the field does not exist. The third and the fourth should run but they do not.
I have always been accustomed to appending my queries with "or die(mysql_error());" but I'm thinking now that is not the best choice. But shouldn't the remaining queries run even if the one in the middle has no work to do?
If there is work to be done in all 4, then it works fine.
#HamZa DzCyberDeV is right, if one of your first queries fails, the die() call will stop the rest of your script from executing just like exit(). You are much better off removing the die statement and using if/else in cases where you need to run a query only if another one completes.
$result = mysql_query("UPDATE invoice SET company='$company' WHERE company='$oldcompanyname'");
if ($result !== false) {
// do something here or execute the next query
}
Also, don't use mysql_* functions in new code Why shouldn't I use mysql_* functions in PHP?. They are no longer maintained and are officially deprecated https://wiki.php.net/rfc/mysql_deprecation. Learn about Prepared statements https://en.wikipedia.org/wiki/Prepared_statement instead, and use PDO http://php.net/pdo or MySQLi http://php.net/mysqli.
This sounds like you should just normalize your data. If you had your customer table linked based off of an ID, you could just do:
UPDATE `company` SET name='$new_name' WHERE company_id=$id
And then in your invoice/bill/project tables you would have a foreign key for company_id, instead of basing it off of company_name.
The is one php file that governs rotating ads that is causing serious server performance issues and causing "too many connections" sql errors on the site. Here is the php script. Can anyone give me some insight into how to correct this as I am an novice at php.
<?
require("../../admin/lib/config.php");
// connect to database
mysql_pconnect(DB_HOST,DB_USER,DB_PASS);
mysql_select_db(DB_NAME);
$i = 1;
function grab()
{
$getBanner = mysql_query("SELECT * FROM sponsor WHERE active='Y' AND ID != 999 AND bannerRotation = '0' ORDER BY RAND() LIMIT 1");
$banner = mysql_fetch_array($getBanner);
if ($banner['ID'] == ''){
mysql_query("UPDATE sponsor SET bannerRotation = '0'");
}
if (file_exists(AD_PATH . $banner['ID'] . ".jpg")){
$hasAd = 1;
}
if (file_exists(BANNER_PATH . $banner['ID'] . ".jpg")){
return "$banner[ID],$hasAd";
} else {
return 0;
}
}
while ($i <= 3){
$banner = grab();
if ($banner != 0){
$banner = explode(",",$banner);
mysql_query("UPDATE sponsor SET bannerView = bannerView + 1 WHERE ID='$banner[0]'");
mysql_query("UPDATE sponsor SET bannerRotation = '1' WHERE ID = '$banner[0]'");
echo "banner$i=$banner[0]&hasAd$i=$banner[1]&";
$i++;
}
}
?>
I see not mysqli
The problem is that mysql_pconnect() opens a persistent connection to the database and is not closed at end of execution, and as you are not calling mysql_close() anywhere the connection never gets closed.
Its all in the manual: http://php.net/manual/en/function.mysql-pconnect.php
Well, the good news for your client is that the previous developer abandoned the project. He could only have done more damage if he had continued working on it.
This script is using ext/mysql, not ext/mysqli. It would be better to use mysqli or PDO_mysql, since ext/mysql is deprecated.
It's recommended to use the full PHP open tag syntax (<?php), not the short-tags syntax (<?). The reason is that not every PHP environment enables the use of short tags, and if you deploy code into such an environment, your code will be viewable by anyone browsing to the page.
This script does no error checking. You should always check for errors after attempting to connect to a database or submitting a query.
The method of using ORDER BY RAND() LIMIT 1 to choose a random row from a database is well known to be inefficient, and it cannot be optimized. As the table grows to have more than a trivial number of rows, this query is likely to be your bottleneck. See some of my past answers about optimizing ORDER BY RAND queries, or a great blog by Jan Kneschke on selecting random rows.
Even if you are stuck using ORDER BY RAND(), there's no need to call it three times to get three distinct random sponsors. Just use ORDER BY RAND() LIMIT 3. Then you don't need the complex and error-prone update against bannerRotation to ensure that you get sponsors that haven't been chosen before.
Using SELECT * fetches all the columns, even though they aren't needed for this function.
If a sponsor isn't eligible for random selection, i.e. if it has active!='Y' or if its ID=999, then I would move it to a different table. This will simplify your queries, and make the table of sponsors smaller and quicker to query.
The UPDATE in the grab() function has no WHERE clause, so it applies to all rows in the sponsor table. I don't believe this is intentional. I assume it should apply only to the single row WHERE ID=$banner['ID'].
This code has two consecutive UPDATE statements against the same row of the same table. Combine these into a single UPDATE statement that modifies two columns.
The grab() function appends values together separated by commas, and then explodes that string into an array as soon as it returns. As if the programmer doesn't know that a function can return an array.
Putting the $i++ inside a conditional block makes it possible for this code to run in an infinite loop. That means this script can run forever. Once a few dozen of these are running concurrently, you'll run out of connections.
This code uses no caching. Any script that serves ad banners must be quick, and doing multiple updates to the database is not going to be quick enough. You need to use some in-memory caching for reads and writes. A popular choice is memcached.
Why is this client coding their own ad-banner server so inexpertly? Just use Google DFP Small Business.
Yikes!
grab() is being called from within a loop, but is not parameterized. Nor does there seem to be any rationale for repeatedly calling it.
A 200% speedup is easily realizable.
Under plain PHP 5.3, I have some code which uses MySQL to first deletes some old records, record a log, perform a few tiny operations and then adds new replacement records.
The delete command looks like this:
DELETE FROM `rtable` WHERE `UserName`='%s';
And the add commands looks like this:
INSERT INTO `table` (`UserName`,`Attribute`,`op`,`Value`) VALUES ('%s','%s','%s','%s');
Oddly though, the insert commands appear to not execute when running normally, however if I enable my debugger and step through one line at a time, it appears to work. Likewise, if I insert a sleep command of two seconds after the delete commands. It appears to work. I am therefor assuming that the insert commands are running -before- the delete commands and thus the delete commands are also erasing the new records.
How can I get PHP to wait for the delete operation to finish before continuing to the insert commands?
That sounds really odd.
Do you happen to have a replicated database cluster?
Also, do you check the return value of the mysql_query or whatever command and print the error message (which of course is not recommended for scripts in production)?
I am not totally certain how PHP deals with processes and how consecutive queries are run, but if you want to make certain to encapsulate the delete in a transaction, you can do so with PDO like this:
$dbh->beginTransaction();
$sth = $dbh->exec("DELETE FROM `rtable` WHERE `UserName`='%s'");
$dbh->commit();
// You could also pop a transaction around the inserts
// in case another page tries to do the same
$sth = $dbh->exec("INSERT INTO `table`
(`UserName`,`Attribute`,`op`,`Value`)
VALUES ('%s','%s','%s','%s')");
BTW: I took the liberty of correcting the single quotes to backticks in your queries.
I have to insert 50,000 to 16.000.000 rows in MySQL Table from php. It is taking at least 15-20 min to store in database.
To complete my project i have to do it more faster insertion. do anyone having any flexible opinion.
i am using this code
for($i=$d;$i<=$fd;$i++)
{ $j = $i-$d+1;
$sql1 = "INSERT INTO information(id, redirection, username, pc, date, time,
method,text, http, code, data, request, software) VALUES ('".$i."','"
.$_SESSION["redirection"][$j]."','".$_SESSION["username"]$j]."','"
.$_SESSION["pc"][$j]."','".$_SESSION["date"][$j]."','".
$_SESSION["time"][$j]."','".$_SESSION["method"][$j]."','"
.$_SESSION["text"][$j]."','".$_SESSION["http"][$j]."','"
.$_SESSION["code"][$j]."','".$_SESSION["data"][$j]."','"
.$_SESSION["request"][$j]."','".$_SESSION["software"][$j]."')";
mysql_query($sql1);
}
I agree with Cal. Try this:
$sql1 = "INSERT INTO information(id, redirection, username, pc, date, time,
method,text, http, code, data, request, software) VALUES ";
<?php
for($i=$d;$i<=$fd;$i++)
{
$j = $i-$d+1;
$sql1 .= "('".$i."','"
.$_SESSION["redirection"][$j]."','".$_SESSION["username"]$j]."','"
.$_SESSION["pc"][$j]."','".$_SESSION["date"][$j]."','"
.$_SESSION["time"][$j]."','".$_SESSION["method"][$j]."','"
.$_SESSION["text"][$j]."','".$_SESSION["http"][$j]."','"
.$_SESSION["code"][$j]."','".$_SESSION["data"][$j]."','"
.$_SESSION["request"][$j]."','".$_SESSION["software"][$j]."')";
if($i<=$fd){
$sql1 .= ", ";
}
}
mysql_query($sql1);
?>
In this case you will perform only one insertion instead multiple. Also, as variant, you can store your data into text file (.csv, for example) and then import it into your database. But this is exotic method.
Use extended insert syntax to insert multiple rows per statement:
INSERT INTO tbl_name (a,b,c) VALUES(1,2,3),(4,5,6),(7,8,9);
http://dev.mysql.com/doc/refman/5.5/en/insert.html
Another trick is to disable/delete indexes and enable/add them after all data has been inserted.
DISABLE KEYS;
INSERT ......
ENABLE KEYS;
If the table supports transactions (InnoDB), you can speed up bulk inserts by doing everything inside a single transaction. Query BEGIN TRANSACTION before all the inserts, and COMMIT after you're done.
Obviously, this is irrelevant if your table doesn't support transactions (MyISAM).
Prepared statements are another way to speed up bulk inserts (with added security benefits), but it's difficult to do that in a project that uses mysql_query() everywhere, so I'll leave that out.
You're not really clear what your limits are, and especially WHY. Other answers assume you are on some sort of time-schedule, and need the insert completed before a certain moment
But when we assume you want to have control over your program back as quickly as possible, but not neccesairily need the insert to be finished, you could use delayed inserts?
http://dev.mysql.com/doc/refman/5.5/en/insert-delayed.html
The DELAYED option for the INSERT statement is a MySQL extension to
standard SQL that is very useful if you have clients that cannot or
need not wait for the INSERT to complete. This is a common situation
when you use MySQL for logging and you also periodically run SELECT
and UPDATE statements that take a long time to complete.
When a client uses INSERT DELAYED, it gets an okay from the server at
once, and the row is queued to be inserted when the table is not in
use by any other thread.
I'm creating a PHP script that imports some data from text files into a MySQL database. These text files are pretty large, an average file will have 10,000 lines in it each of which corresponds to a new item I want in my database. (I won't be importing files very often)
I'm worried that reading a line from the file, and then doing a INSERT query, 10,000 times in a row might cause some issues. Is there a better way for me to do this? Should I perform one INSERT query with all 10,000 values? Or would that be just as bad?
Maybe I can reach a medium, and perform something like 10 or 100 entries at once. Really my problem is that I don't know what is good practice. Maybe 10,000 queries in a row is fine and I'm just worrying for nothing.
Any suggestions?
yes it is
<?php
$lines = file('file.txt');
$count = count($lines);
$i = 0;
$query = "INSERT INTO table VALUES ";
foreach($lines as $line){
$i++;
if ($count == $i) {
$query .= "('".$line."')";
}
else{
$query .= "('".$line."'),";
}
}
echo $query;
http://sandbox.phpcode.eu/g/5ade4.php
this will make one single query, which is multiple faster than one-line-one-query style!
Use prepared statements, suggested by the authors of High Performance MySQL. It saves a lot of time (saves from wasteful protocol and SQL ASCII code).
I would do it in one large query with all the values at once. Just to be sure, though, make sure you run START TRANSACTION; before and COMMIT; afterwards, so that if something goes wrong during the execution of the query (which is possible, since it will most likely run for a fairly long time), the database will not be affected.