is there any chance to set in PDO settings that SELECT's will be executed on SLAVE DB server and Insert & Update & DELETE will be executed on MASTER DB server, or I need to create PHP handler to do that?
Situation:
We have Master - Master replication for MySQL. We are going to add two new servers so it will be - Master/Slave - Master/Slave.
I want to create some handling for SELECT queries. I want execute SELECT queries on SLAVE instead of MASTER and all UPADTE&INSERT&DELETE queries will be executed on MASTER. Is this possible with some setting?
Thanks!
No, you can't configure PDO or any of PHP's database extensions to do this. That is simply because each PDO (or MySQLi, etc.) instance represents a single connection, to a single server.
So yes, you'll need a handler that is aware of multiple connections to do that. Some popular ORMs and other database-abstraction layers do provide such functionality.
I recommend not doing it even if you could. Replication is "asynchronous". That is, when you insert into the Master, there is no assurance that it will arrive at the Slave before you try to read it. Nor even any guarantee that it will arrive today!
If you user posts a comment on a blog, and then goes to a page that shows the comment, they will be annoyed if the comment does not show. They may assume that the comment was lost and then repost it. This causes you grief when users complain about double-posting.
This is called "critical read". This simple way to avoid the mess is to be careful about what you send to the Slaves -- namely nothing that would lead to "disappearing" posts.
There are various "proxy" packages that allow from the read-write split you describe; some try to avoid the "critical read", but I don't trust them.
A Galera Cluster (see PXC, MariaDB), does synchronous reads, so it can avoid the critical read problem. (There is, however, a setting you need to apply.)
Related
I am using MariaDB in a PHP application. The problem is the following: using Doctrine DBAL with the MySQL adaptor I do an insert from one page and then redirect to another one, in which a SELECT is done. Both are very basic queries.
The problem is that the SELECT does not reflect the actual data, but older one. I am hosting this application on a shared hosting, so please consider that I won't have all DB configuration options/permissions available.
I have tried to flush after the first INSERT, but it does not work either, and it still shows outdated data. I believed that the Query Caches are invalidated if the data changes, and that they do not apply because, in fact, it is a different query.
I do not use transactions either, so the commit is supposedly done after the insert. Any idea on how to get the most recent data possible?
It sounds like you are doing Replication and the proxy for directing queries is oblivious to "Critical Reads".
In a replication setup (MariaDB or MySQL), there is one Master server and one Slave (at least). INSERTs (and other writes) must occur on the Master. Then they are replicated to the Slave(s). SELECTs, on the other hand, may be done on either server, but, for sharing the load, it is better to do them on the Slave(s).
Replication is "asynchronous". That is, the write is eventually sent to the Slave and performed there. Normally, the delay is sub-second. But, for a number of reasons, the delay could be arbitrarily large. One should not depend on how quickly writes are replicated.
But... There is a thing called a "Critical Read". This is when the SELECT needs to "see" the thing that was just written.
You have a "critical read".
I don't know what is deciding to direct your SELECT to a Slave.
If you are using the Galera clustering option of MariaDB, then you can protect yourself from the critical read problem by changing your select to
SET SESSION wsrep_sync_wait = 1;
SELECT ... (as before)
SET SESSION wsrep_sync_wait = 0;
However; the SETs must go to the same 'node' as the SELECT. Without knowing what kind of proxying is going on, I cannot be more specific.
I hope you are not reconnecting before each statement. That would be really bad.
More on Galera issues for developers
If you are using replication and Doctrine DBAL has nothing for critical reads, complain to them!
Note: I used Google Translator to write this
I've always done the following to work with MySQL:
-> Open Connection to the database.
-> see details
-> Insert Data
-> another query
-> close Connection
I usually use the same connection to do various things before closing.
A friend who studies this in the IPN of Mexico mentioned to me that the right way (for safety) is to make a new connection for each query, for example:
-> Open Connection to the database.
-> see details
-> close Connection
-> Open Connection to the database.
-> Insert Data
-> close Connection
-> Open Connection to the database.
-> another query
-> close Connection
My question is, what is the right thing to do? My method has been to make the least amount of queries to the database, and only make a connection and keep it until it no longer serves me.
Additionally, is it possible to make a double insertion to a table? For example:
insert into table1(relacion) values([insert into tablaRelacionada(id) values("dato")]);
and that "relacion" is the inserted ID from the first query in "tablaRelacionada".
No, it's not possible to insert rows into two different tables with a single INSERT statement. (You can use a trigger to get it done, but that trigger will need to issue a separate INSERT statement... from the client side it will look like one statement, but on the server, there would be two INSERT statements executed.
If performance and scalability aren't concerns, then "churning" connections is workable. There's nothing necessarily "wrong" with creating a separate connection for each statement, but it's resource intensive. There is a lot of overhead in creating a new session. (It looks rather simple from the client side, but it requires a lot of work on the server side, in addition to the codepath on the client.)
Reusing existing connections is a common pattern. It's one of the biggest benefits of implementing "connection pool", to make it easy to reuse connections without "churning", repeatedly connecting and disconnecting from the database.
In terms of a separate connection for each SQL statement somehow increasing "safety", that's a bit of a stretch.
But I can see some benefit of having a freshly initialized session.
For example, if you reuse an existing session, you may not know what changes have been made in the session state. Any changes made previously are still "in effect". This would be things like session variable settings (e.g. timezone, characterset, autocommit, user defined variables) which could have an impact on the current statement. But within a single script, where you've gotten a fresh connection, you should know what changes have been made, so that shouldn't really be an issue. (This would be more of an issue with using connections from a pool, where the connections are shared by multiple processes. One process mucking with the timezone or characterset could cause a slew of problems for other processes that reuse the connection.)
Using a separate connection per query is at best a great way to bog down both your application and database servers with needless overhead. There are three aspects I see raised here:
Efficiency
Application Security
Network Security
1. Efficiency
Short answer: Bad idea.
Oftentimes the overhead required to initialize the connection is far more than what is required to run the actual query. Your application is probably going to run orders of magnitude slower if you take a connection-per-query approach.
2. Application Security
Short answer: Generally a bad idea, but in the context of PHP completely unnecessary.
The only 'safety' issue I can think of here would be worrying about users accessing leftover temp tables, or session settings "bleeding" over. This is unlikely to happen unless you're using persistent connections which are not the default. As well, most temporary values in MySQL are stored per-connection, and unless you have some PHP code that written poorly [in a particular, strange, and seldom-recommended way, ie. sharing around DB singletons and accessing them strangely] then maybe if the planets align just right you might access some MySQL session-specific data in an unexpected way.
This is pretty much the same as preemptive optimization, and is not worth worrying about.
3. Network Security
Short answer: No. What? Just... no.
If you're worried about someone peeping in on your connections the solution is not to make more of them, it to make them securely. MySQL supports SSL, so use that if you're worried.
TL;DR No. Don't create separate connections per-query. Bad. Whoever told you this needs to go back to school.
Multi-Table Insert
What you've quoted is not possible, you would want to do something along the lines of the following:
$dbh->query("INSERT tablaRelacionada(id) values('dato')");
$lastid = $dbh->lastInsertId();
$dbh->query("INSERT INTO table1(relacion) values($lastid);");
Assuming that the table tablaRelacionada has an AUTO_INCREMENT column which is what you're trying to get from the first query.
See: lastInsertId()
I'm writing a program that runs (24/7) on a Linux server and adds entries to a MySQL database.
The contents of the database are presented on a web interface with PHP and the user should be able to delete entries using the web interface.
Is it possible to access the database from multiple processes at the same time?
Yes, databases are designed for this purpose quite well. You'll want to keep a few things in mind in your designs:
Concurrency and race conditions on database writes.
Performance.
Separate database permissions for separate applications.
Unless you're doing something like accessing the DB using a singleton, the max number of simultaneous mysql connections php will use is limited in your php.ini. I believe it defaults to 100.
Yes multiple users can access the database at the same time.
You should however take care that the data is consistent.
If you create/edit entry with many small sql statements and in the meantime someone useses the web interface this may lead to some errors.
If you have a simple db this should not be a problem, else you should consider using transactions.
http://dev.mysql.com/doc/refman/5.0/en/ansi-diff-transactions.html
Yes and there will not be any problems while trying to delete records in the presence of that automated program which runs 24/7 if you are using the InnoDb engine. This is because transactions happen one at a time, one starts after another has finished and the database is consistent everytime.
This answer How to implement the ACID model for a database has many relevant points.
Read about the ACID Properties of a database. A Mysql database with InnoDb engine will take care of all these things for you and you need not worry about that.
I am wanting to hear what others think about this? Currently, I make a mysql database connection inside of a header type file that is then included in the top of every page of my site. I then can run as many queries as I want on that 1 open connection. IF the page is built from 6 files included and there is 15 different mysql queries, then they all would run on this 1 connection.
Now sometimes I see classes that make multiple connections, like 1 for each query.
Is there any benefit of using one method over the other? I think 1 connection is better then multiple but I could be wrong?
Creating connections can be expensive (I don't have a reference for this statement as yet Edit: Aha! Here it is) so it seems as if the consensus is to use fewer connections. Using a single connection for all queries on a single page seems to be a better choice than multiple connections.
In PHP+MySQL usually there is no much sence to use multiple connections per page (just slower and a little more RAM consumed).
The only way it might be useful is when you alter connection paremters which might interfer with other pages (like collation). But good PHP programs usually never do that kind of stuff.
Also, it is a good idea to enable persistent connections, so that 1 MySQL connection would be reused across multiples page executions.
If really depends on the level of activity you suspect the site will generate - if it's a high traffic web site, you'll soon run out of connections (unless you set the adjust MySQLs max connections to a stupidly high level, but that'll eventually grind the server to a halt).
I'd generally recommend that the front end of a web site should use a shared database object (singleton is your friend), as it doesn't require a great deal of discipline to write with this is mind and you won't waste time making connections. If you require additional concurrent queries on the backend, it shouldn't be that much of a deal as this isn't likely to be a highly trafficked area.
Its not recommended to execute multiple small queries where the work can be done using just one query, you can use a single query to get data from multiple tables and ieven multiple databases. see the link below:
http://www.x-developer.com/php-scripts/sql-connecting-multiple-databases-in-a-single-query
I don't see any benefit of using multiple connections, I 'd rather think it is a sign of bad structure. These are the reasons I can think of against using multiple connections:
You have to initialize the database multiple times. Setting conection properties upon connection establishment (like SET NAMES UTF8) would have to be done on multiple line.
It is definitely slower than a single connection.
A non-technical reason: Someone working with your code will most probably not expect it and might spend hours debugging the connection properties he had set in another connection.
Having a global connection object (or a class providing one) is the much better approach in PHP.
Are you sure the classes that make multiple connections aren't just returning a reference to the already open connection when one is open? I've seen a lot of stuff structured that way. It really is better performance-wise to use only one connection per page.
I'm currently writing a PHP application and drivers (classes) for the database engines. I was wondering if I need to write a replication support (master-slave)? I'm a bit new to this, so, what kind of things should my project or classes worry about if I want to support load balancing/replication? Oh and this is about MySQL.
The way we use our master-slave db, is to use the master for all "active usage", and the slave for all reporting (where it doesn't matter if the data is still "catching up" slightly). Depending on your needs, you could have -all- data manipulation occur on the master, and -all- data reading occur on the slave. This especially helps when you have blocking inserts or updates. (Note: Also consider the "insert delayed" MySQL syntax where possible, which helps avoid blocking too.)
As far as the PHP support for this, all you really need is to keep clean handling for multiple (two) database connections, and use the master (read/write) or slave (ONLY READ) db connection as desired.
If you think you will use the slaves to read and the master to write, then your Class needs to support at least several connections at once.
I will show you the API I used, If you choose that way, I can send you the class.
ShusterDb::getInstance('read')->select($sql); //makes sure this is a SELECT in the method.
ShusterDb::getInstance('write')->scalar($sql);
Itay, if you are open to sending your class, I would be interested in seeing / possibly using it.