Hello so I have a code that will check if a username exists and if yes, I want to set the variable $var to the present accountID of the existing username. If the username doesn't exist, I want the MAX of the latest accountID to increment. Here is my code:
$checkUname = $conn->prepare("SELECT accountID from accounts WHERE username=?");
$checkUname->execute(array($_POST['txt_un']));
$row = $checkUname->fetch(PDO::FETCH_ASSOC);
if($row > 0) {
$var = $row['accountID'];
} else {
$getMax = $conn->query("SELECT MAX(accountID) as maxAccountID FROM accounts");
$row = $getMax->fetch(PDO::FETCH_ASSOC);
$maxID = $row['maxAccountID'];
$maxID++;
$var = $maxID;
}
$UpdateTable = $conn->prepare("UPDATE otherTable SET someField=? WHERE otherTableID=?");
$UpdateTable->execute(array($var, $_POST['tableID']));
Now my problem here is, what if 2 or more users will click and run the code above at the same time? Will there be issues on the incrementing ID?
You could use this query to get data at once
IF EXISTS (SELECT accountID from accounts WHERE username=?) THEN
SELECT accountID from accounts WHERE username=?;
ELSE
SELECT MAX(accountID) as maxAccountID FROM accounts;
END IF;
That's why you should have accountID to be an autoincrement column https://dev.mysql.com/doc/refman/5.0/en/example-auto-increment.html
Then you can use LAST_INSERT_ID() to get the id of the inserted user. The page above have a link to this function.
PHP have specific functions to use LAST_INSERT_ID(), each library have its specific function:
PDO: http://php.net/manual/en/pdo.lastinsertid.php
MySQLi: http://php.net/manual/en/mysqli.insert-id.php
MySQL: http://php.net/manual/en/function.mysql-insert-id.php
Here is some extract from this link:click me
It is not exactly what you need, but it has an approach to deal with such problems. And it impies that you are using MySQL InnoDB. If you use MyISAM or MariadDB the solution might be different.
Extract:
Let us look at another example: We have an integer counter field in a table child_codes that we use to assign a unique identifier to each child added to table child. It is not a good idea to use either consistent read or a shared mode read to read the present value of the counter because two users of the database may then see the same value for the counter, and a duplicate-key error occurs if two users attempt to add children with the same identifier to the table.
In this case, there are two good ways to implement reading and incrementing the counter:
First update the counter by incrementing it by 1, and then read it.
First perform a locking read of the counter using FOR UPDATE, and then increment the counter.
The latter approach can be implemented as follows:
SELECT counter_field FROM child_codes FOR UPDATE;
UPDATE child_codes SET counter_field = counter_field + 1;
A SELECT ... FOR UPDATE reads the latest available data, setting exclusive locks on each row it reads. Thus, it sets the same locks a searched SQL UPDATE would set on the rows.
The preceding description is merely an example of how SELECT ... FOR UPDATE works. In MySQL, the specific task of generating a unique identifier actually can be accomplished using only a single access to the table:
UPDATE child_codes SET counter_field = LAST_INSERT_ID(counter_field + 1);
SELECT LAST_INSERT_ID();
The SELECT statement merely retrieves the identifier information (specific to the current connection). It does not access any table.
In General No Problem . why?
It depends on so many factors starting from the ability of DB for performing such queries , the serer processing ,internet connection etc ...
go through links below to get real and specific info
How many MySql queries/second can be handled by a server?
http://yoshinorimatsunobu.blogspot.com/2010/10/using-mysql-as-nosql-story-for.html
Important :
What is PDO?
PHP Data Objects, or PDO, is a database abstraction layer specifically for PHP applications. PDO provides a consistent API for your PHP application regardless of the type of database server your application will connect to. In theory, if you are using the PDO API, you could switch the database server you used, from say Firebird to MySQL, and only need to make minor changes to your PHP code.
Other examples of database abstraction layers include JDBC for Java applications and DBI for Perl.
While PDO has its advantages, such as a clean, simple, portable API, its main disadvantage is that it doesn't allow you to use all of the advanced features that are available in the latest versions of MySQL server. For example, PDO does not allow you to use MySQL's support for Multiple Statements.
http://php.net/manual/en/mysqli.overview.php
Related
I have found two different ways to, first, get the next invoice number and, then, save the invoice in a multi-tenant database where, of course, each tenant will have his own invoices with different incremental numbers.
My first (and actual) approach is this (works fine):
Add a new record to the invoices tables. No matter the invoice number yet (for example, 0, or empty)
I get the unique ID of THAT created record after insert
Now I do a "SELECT table where ID = $lastcreatedID **FOR UPDATE**"
Here I get the latest saved invoice number with "SELECT #A:=MAX(NUMBER)+1 FROM TABLE WHERE......"
Finally I update the previously saved record with that invoice number with an "UPDATE table SET NUMBER = $mynumber WHERE ID = $lastcreatedID"
This works fine, but I don't know if the "for update" is really needed or if this is the correct way to do this in a multi-tenant DB, due to performance, etc.
The second (and simpler) approach is this (and works too, but I don't know if it is a secure approach):
INSERT INTO table (NUMBER,TENANT) SELECT COALESCE(MAX(NUMBER),0)+1,$tenant FROM table WHERE....
That's it
Both methods are working, but I would like to know the differences between them regarding speed, performance, if it may create duplicates, etc.
Or... is there any better way to do this?
I'm using MySQL and PHP. The application is an invoice/sales cloud software that will be used by a lot of customers (tenants).
Thanks
Regardless of if you're using these values as database IDs or not, re-using IDs is virtually guaranteed to cause problems at some point. Even if you're not re-using IDs you're going to run into the case where two invoice creation requests run at the same time and get the same MAX()+1 result.
To get around all this you need to reimplement a simple sequence generator that locks its storage while a value is being issued. Eg:
CREATE TABLE client_invoice_serial (
-- note: also FK this back to the client record
client_id INTEGER UNSIGNED NOT NULL PRIMARY KEY,
serial INTEGER UNSIGNED NOT NULL DEFAULT 0
);
$dbh = new PDO('mysql:...');
/* this defaults to 'on', making every query an implicit transaction. it needs to
be off for this. you may or may not want to set this globally, or just turn it off
before this, and back on at the end. */
$dbh->setAttribute(PDO::ATTR_AUTOCOMMIT,0);
// simple best practice, ensures that SQL errors MUST be dealt with. is assumed to be enabled for the below try/catch.
$dbh->setAttribute(PDO::ATTR_ERRMODE_EXCEPTION,1);
$dbh->beginTransaction();
try {
// the below will lock the selected row
$select = $dbh->prepare("SELECT * FROM client_invoice_serial WHERE client_id = ? FOR UPDATE;");
$select->execute([$client_id]);
if( $select->rowCount() === 0 ) {
$insert = $dbh->prepare("INSERT INTO client_invoice_serial (client_id, serial) VALUES (?, 1);");
$insert->execute([$client_id]);
$invoice_id = 1;
} else {
$invoice_id = $select->fetch(PDO::FETCH_ASSOC)['serial'] + 1;
$update = $dbh->prepare("UPDATE client_invoice_serial SET serial = serial + 1 WHERE client_id = ?");
$update->execute([$client_id])
}
$dbh->commit();
} catch(\PDOException $e) {
// make sure that the transaction is cleaned up ASAP, then let the exception bubble up into your general error handling.
$dbh->rollback();
throw $e; // or throw a more pertinent error/exception of your choosing.
}
// both committing and rolling back will release the lock
At a very basic level this is what MySQL is doing in the background for AUTOINCREMENT columns.
Do not use MAX(id)+1. It will, someday, bite you. There will be two invoices with the same number, and it will take us a few paragraphs to explain why it happened.
Instead, use AUTO_INCREMENT the way it is intended.
INSERT INTO Invoices (id, ...) VALUES (NULL, ...);
SELECT LAST_INSERT_ID(); -- specific to the conne ction
That is safe even when multiple connections are doing the same thing. No FOR UPDATE, no BEGIN, etc is necessary. (You may want such for other purposes.)
And, never delete rows. Instead, use the standard business practice of invalidating bad invoices. Imagine being audited.
All that said, there is still a potential problem. After a ROLLBACK or system crash, an id may be "burned". Also things like INSERT IGNORE allocate the id before checking to see whether it will be needed.
If you can live with the caveats, use AUTO_INCREMENT.
If not, then create a 1-row, 2-column table to simulate a sequence number generator: http://mysql.rjweb.org/doc.php/index_cookbook_mysql#sequence
Or use MariaDB's SEQUENCE
Both the approaches do work, but each with its own demerits in high traffic situations.
The first approach runs 3 queries for every invoice you create, putting extra load on your server.
The second approach can lead to duplicates in events where two invoices are generated with very little time difference (such that the SELECT query return same max number for both invoices).
Both the approaches may lead to problems in high traffic conditions.
Two solutions to the problems are listed below:
Use generated columns: Mysql supports generated columns, which are basically derived using other column values for each row. Refer this
Calculate invoice number on the fly: Since you're using the primary key as part of the invoice, let the DB handle generating unique primary keys, and then generate invoice numbers on the fly in your business logic using the id for each invoice.
I'm trying to move to my project to PostgreSQL from MySQL. For MySQL, PHP offers good functionality to fetch SELECT query metadata.
mysqli_fetch_field or mysqli_fetch_field_direct functions can give the following values for each field in the query
name
orgname
table
orgtable
max_length
length
type
(and a few others I'm not interested in currently)
For instance, an example query and metadata for the field "id" frm the query will look like this:
Query:
SELECT id as id2 FROM table1 t1
Metadata for field "id" :
name: id2
orgname : id
table : t1
orgtable : table1
length: 765
type: 253
The good thing about mysqli_fetch_field is that it is capable of returning both alias and original name of the fields and tables in the query.
When I try to imitate this functionality for PostgreSQL, I observe that there is no equivalent of mysqli_fetch_field function for PostgreSQL, instead, there are several pg_field_* functions each of which give a single information for a field.
pg_field_name returns name only and for this example it gives "id2"
pg_field_table returns original table name, e.g. "table1"
pg_field_type returns the type (no problem here)
pg_field_size returns the length of the field (it is also OK)
Obviously, pg_field_* functions do not offer the programmer original names of the tables/fields if they are renamed in the query while mysqli_fetch_field does.
Is there a way that I can fetch "detailed" metadata for a query in PostgreSQL? It can be using an additional query or a tool that the DBMS offers like EXPLAIN and DESCRIBE (I tried these too) or anything else.
Are you invested into the Postgres specific function API? If not, check out PDO, which provides you one common interface to your database, no matter if it is MySQL or Postgres. It also provides you a way to fetch specific columns:
$pdo = new PDO(...);
$statement = $pdo->prepare("SELECT foo FROM bar WHERE baz = ?");
$statement->execute([42]);
$foo = $statement->fetchColumn(0);
// $foo is the column value or false
You can also fetch column values of all result rows:
$foos = $statement->fetchAll(\PDO::FETCH_COLUMN, 0);
// $foos is an array, possibly empty
Note that fetchAll incurs the usual memory penalty, as a possibly large result is loaded into your process. You should only use fetchAll if the result set is known to always be small. Otherwise, use fetch or fetchColumn in a loop and page the query with LIMIT n, m clauses.
Fetching a specific column is resource friendly if only one column is queried. When you have other columns in your query, they would still be fetched into your process, just to be thrown away. You should make sure to not include any unnecessary columns.
For column metadata, use getColumnMeta:
$meta = $statement->getColumnMeta(0);
// $meta is false or array{native_type, driver:decl_type, flags, name, table, len, precision, pdo_type}
To get more detailed metadata, you should query into information_schema.columns. Best to do it before the data query, because the data query might be executed in a loop. Metadata need to be fetched only once beforehand.
After some research, I am eventually convinced that neither PHP Postgres API nor PostreSQL itself provides such functionality. Therefore, I could manage to solve the problem by a combination of pg_field_* functions and an SQL Parser written in PHP, namely PHP-SQL-Parser
Even though the SQL parser is written MySQL in mind, the default functionality worked for my use case and it is extensible in any case.
My situation: My website will look at a cookie for a remember me token and a user ID. If the cookie exists it will unhash it and look up the user ID and compare the token. with a "WHERE userid = '' and rememberme = ''".
My question is: Will MySQL optimize this query on the unique userid so that the query does not scan the entire database for this 20+ character token? Or instead should I just select the token from the database and then use a php if comparison to check if the tokens are the same?
In short (tl;dr): Would it be better to check if a token matches in with a MySQL select query, or to grab all the tokens from a databases database and compare the values with a php if conditional?
Thanks!
Simple answer:
YES, the database will definitely optimism your search AS LONG AS THE variable you are searching in the WHERE ... portion is indexed! You definitely should not retrieve all the information via SQL and then do a PHP conditional if you are worried about performance.
So if the id column in your table is not indexed, you should index it. If you have let say... 1 million rows already in your table and run a command like SELECT * FROM user WHERE id = 994321, you would see a definite increase in performance.
Elaborating:
A database (like MySQL) is made to be much faster at executing queries/commands than you would expect that to happen in php for instance. In your specific situation, lets say you are executing this SQL statement:
$sql = "SELECT * FROM users WHERE id = 4";
If you have 1 million users, and the id column is not indexed, MySQL will look through all 1 million users to find all the rows with id = 4. However, if it is indexed, there is something called a b tree that MySQL makes (behind the scenes) which works similarly to how the indexing of a dictionary work.
If you try to find the world slowly in a dictionary, you might open the book in the middle, find words that start with the letter M and then look in the middle again of the pages on your right side hoping to find a letter closer to S. This method of looking for a word is much faster than looking at each single page from the beginning 1 by 1.
For that very reason, MySQL has created indexes to help performance and this feature should definitely be taken advantage of to help increase the speed of your queries.
Comparing it on MySQL-side should be fast. It should find the corresponding row by ID first (fast) and then compare the hash (also fast, since there will be only 1 row to check).
Try analyzing the query with EXPLAIN to find out the actual execution plan.
In my opinion it will be always faster to use WHERE clause no matter what (real) database server will be used. Database engines have strong algorithms for searching data written in language that is compiling to low-level code dedicated to platform, so it cannot be even compared with some loop written in interpreted PHP.
And remember that for PHP loop you will have to send all records from DB to PHP.
If you Data Base its on a separate server than you Apache PHP there is not doubt it would be faster if you write a query in MySQL.
If your PHP and MySQL server is on the same physical server probably PHP would be faster cause the comparison will be made on the RAM But have all the User Id array into RAM would be a waste of RAM so you can use Indexes that would speed up your query
ALTER TABLE table ADD INDEX idx__tableName__fieldName (field)
I have two tables called clients, they are exactly the same but within two different db's. Now the master always needs to update with the secondary one. And all data should always be the same, the script runs once per day. What would be the best to accomplish this.
I had the following solution but I think maybe theres a better way to do this
$sql = "SELECT * FROM client";
$res = mysql_query($conn,$sql);
while($row = mysql_fetch_object($res)){
$sql = "SELECT count(*) FROM clients WHERE id={$row->id}";
$res1 = mysql_query($connSecond,$sql);
if(mysql_num_rows($res1) > 0){
//Update second table
}else{
//Insert into second table
}
}
and then I need a solution to delete all old data in second table thats not in master.
Any advise help would be appreaciated
This is by no means an answer to your php code, but you should take a look # Mysql Triggers, you should be able to create triggers (on updates / inserts / deletes) and have a trigger (like a stored proceedure) update your table.
Going off the description you give, I would create a trigger that would check for changes to the 2ndary table, then write that change to the primary table, and delete that initial entry (if so required) form the 2ndary table.
Triggers are run per conditions that you define.
Hopefully this gives you insight into 'another' way of doing this task.
More references on triggers for mysql:
http://dev.mysql.com/doc/refman/5.0/en/triggers.html
http://www.mysqltutorial.org/create-the-first-trigger-in-mysql.aspx
You can use mysql INSERT ... SELECT like this (but first truncate the target table):
TRUNCATE TABLE database2.client;
INSERT INTO database2.client SELECT * FROM database1.client;
It will be way faster than doing it by PHP.
And to your notice:
As long as the mysql user has been given the right permissions to all databases and tables where data is pulled from or pushed to, this will work. Though the mysql_select_db function selects one database, the mysql statement may reference another if you use complete reference like databasename.tablename
Not exactly answering your question, but how about just using 1 table, instead of 2? You could use a fedarated table to access the other (if it's on a different mysql instance) or reference the table directly (like shamittomar's suggestion)
If both are on the same MySQL instance, you could easily use a view:
CREATE VIEW database2.client SELECT * FROM database1.client;
And that's it! No synchronizing, no cron jobs, no voodoo :)
I'm using PHP and MySQL. I need to do a query:
DELETE FROM db1.players WHERE acc NOT IN (SELECT id FROM db2.accounts)
The problem is, that db1 and db2 are located on different servers. What is the fastest solution for such problem?
To be precise: I'm using 2 connections, so I think I can't use one query for it.
You will either have to save the list you want to an array and compare it with the other database
or
You can make a federated table and make it seem like the query is running on 1 database.
I don't know if it is the fastest, but I would create a temporary table on db2 containing account ids, export that table to db1 and run the query there.
For the exporting part you can either use mysql builtin export/import functions, where you would have to consider finding an unused table name or use the CSV-export/import of mysql.
If the expected number of results from the inner query is reasonably small, you can transfer from within PHP using strings:
$ids = query($db1, "SELECT GROUP_CONCAT(id) FROM accounts");
query($db2, "DELETE FROM players WHERE acc NOT IN ($ids)");
The syntax database.table.field should actually work in MySQL.
But I guess you've to run the query directly on the MySQL server. Not sure about this.
Have you tried the following?
DELETE FROM db1.players WHERE db1.players.acc NOT IN (SELECT db2.accounts.id FROM db2.accounts)
Or did I get you wrong and the two databases are not on the same server?
Have a look here: http://www.dottedidesign.com/node/14
I'd try something like:
create temporary table for the ids on the local server
statement to fetch ids from remote server
prepared statement to insert ids into temp. table
transfering ids remote->local
DELETE using the temp.table
<?php
$pdoRemote = new PDO('mysql:host=somewhere.else;dbname=foo', '...', '...');
$pdoRemote->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$pdoLocal = new PDO('mysql:host=localhost;dbname=bar', '...', '...');
$pdoLocal->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$pdoLocal->exec('CREATE TEMPORARY TABLE remoteIds ( id int )'); // maybe engine=heap
$stmtLocal = $pdoLocal->prepare('INSERT INTO remoteIds (id) VALUES (:id)');
$stmtLocal->bindParam(':id', $id);
foreach( $pdoRemote->query('SELECT id FROM accounts') as $row ) {
$id = $row['id'];
$stmtLocal->execute();
}
unset($pdoRemote);
To query multiple databases, on different servers, using a single query you can use federated tables.
You could also use replication, or temporary tables. This way you're accessing the databases on the same server.
The only other alternative I can think of would be to use MySQL Proxy to parse your query into 2 or more queries.
I think I'm right in saying that you can't do it the way you are be able to with MSSQL using the sp_addlinkedserver procedure.
You might be better using two queries.