I've got a site that requires manual creation of the database tables on install. At the moment they are saved (along with any initial data) in a collection of .sql files.
I've tried to auto-create using exec() with CLI MySQL and while it works on a few platforms it's fairly flakey and I don't really like doing it this way, plus it is hard to debug errors and is far from bulletproof (especially if the MySQL executable isn't in the system path).
Is there a better way of doing this? The MySQL query() command only allows one sql statement per query (which is the sticking point).
MySQLi I've heard may solve some of these issues but I am fairly invested in the original MySQL library but would be willing to switch provided it's stable, compatible and is commonly supported on a standard server build and is an improvement in general.
Failing this I'd probably be willing to do some sort of creation from a PHP array/data structure - which is arguably cleaner as it would be able to update tables to match the schema in situ. I am assuming this may be a problem that has already been solved, so any links to any example implementation with pro's/con's would be useful!
Thanks in advance for any insight.
Apparently you can pass 65536 as client flag when connecting to the datebase to allow multi queries, e.g. making use of ; in one SQL string.
You could also just read in the contents of the SQL files, explode by ; if necessary and run the queries inside a transaction to make sure all queries execute properly.
Another option would be to have a look at Phing and dbdeploy to manage databases.
If you're using this to migrate data between systems, consider using the LOAD DATA INFILE syntax (http://dev.mysql.com/doc/refman/5.1/en/load-data.html) after having used SELECT ... INTO OUTFILE (http://dev.mysql.com/doc/refman/5.1/en/select.html)
You can run the schema creation/update commands via the standard mysql_* PHP functions. And if the query() command as you call it will allow only one statement, just call it many times.
I really don't get why do you require everything to be in the same call.
You should check for errors after each statement and take corrective actions if it fails (unless you are using InnoDB, in which case you can wrap all statements in a transaction and rollback if it fails.)
Related
I'm developing a project where I need to retrieve HUGE amounts of data from an MsSQL database and treat that data. The data retrieval comes from 4 tables, 2 of them with 800-1000 rows, but the other two with 55000-65000 rows each one.
The execution time wasn't tollerable, so I started to rewrite the code, but I'm quite inexperienced with PHP and MsSQL. My execution of PHP atm is in localhost:8000. I'm generating the server using "php -S localhost:8000".
I think that this is one of my problems, the poor server for a huge ammount of data. I thought about XAMPP, but I need a server where I can put without problems the MsSQL Drivers to use the functions.
I cannot change the MsSQL for MySQL or some other changes like that, the company wants it that way...
Can you give me some advices about how to improve the performance? Any server that I can use to improve the PHP execution? Thank you really much in advance.
The PHP execution should least of your concerns. If it is, most likely you are going about things in the wrong way. All the PHP should be doing is running the SQL query against the database. If you are not using PDO, consider it: http://php.net/manual/en/book.pdo.php
First look to the way your SQL query is structured, and how it can be optimised. If in complete doubt, you could try posting the query here. Be aware that if you can't post a single SQL query that encapsulates your problem you're probably approaching the problem from the wrong angle.
I am assuming from your post that you do not have recourse to alter the database schema, but if so that would be the second course of action.
Try to do as much data processing in SQL Server as possible. Don't do data joining or other type of data processing that can be done in the RDBMS.
I've seen PHP code that retrieved data from multiple tables and matched lines based on several conditions. This is just an example of a misuse.
Also try to handle data in sets in SQL (be it MS* or My*) and avoid, if possible, line-by-line processing. The optimizer will output a much more performant plan.
This is small database. Really. My advices:
- Use paging for the tables and get data by portions (by parts)
- Use indexes for tables
- Try to find more powerful server. Often hosters companies uses one database server for thousands user's databases and speed is very slow. I suffered from this and bought dedicated server finally.
Right now I do some project to make normal MySQL syntax in PHP can work with MongoDB.
I already done in some simple syntax(SELECT, CREATE) by create my own PHP library for user to pass their MySQL syntax to my function(Ex. MongoQuery("SELECT User FROM A=1")), it will parser MySQL to Mongo Syntax and also query data from MongoDB.
But my question is, have any possible ways to make php can capture SQL syntax real-time when web is running?
For example, user not need to edit any code in their PHP file, just simple put it in web server folder and my program will handle to change MySQL operation to MongoDB operation and also query data from MongoDB return to normal SQL return value variable. Thank you :)
You can not just change those things in the fly, unless you write a PHP extension.
It is also never a good thing to use a document database like MongoDB as it were a relational database like MySQL. They have totally different features and performance characteristics. For example, MongoDB does not have joins, but instead uses embedded or nested documents; MongoDB does not do transactions either, so you will need to do things in another atomic way; MongoDB also does not do "WHERE A=B", so you need to redo your query thoughts.
In short, transparently converting SQL DQL to MongoDB queries is not possible.
I am developing a Codeigniter (2.0.2) Application, which will utilise a Master database for all write operations (INSERT/UPDATE/DELETE) and a read replica for all read operations (SELECT).
Now I know I can access two different database objects within the code to route the individual requests to the specific database server, but i'm thinking there has a better way, automated way. I'll be using MySQL and Active Record, and also want to build in Memcache checking - although it won't be used immediately, I'd like the option there for the future, built in at this stage.
I'm thinking if its possible to add a hook/library of some kind to intercept the $this->db->query so that the following happens:
1) SQL Query received
2) Check if SELECT query
2a) If SELECT, see if Memcache is active, if so encode SQL and check Memcache for response.
2b) If no memcache response, or Memcache is not active, execute query as normal through READ MySQL server.
3) Query was NOT select, so execute query as normal through the WRITE MySQL server.
4) Return response.
I'm sure that looking at this, it should be quite simple to do, but no matter how I look at it i'm just not seeing a potential answer - but there's got to be one! Can anyone help/assist?
In addition, I also want the ability to be able to log all write SQL commands for troubleshooting, presumably the best way is to introduce 3a) Write SQL command to plain text file ... into the above scheme. I don't believe MySQL actually logs the non-SELECT queries in anyway ... does it?
That type of behavior is a little bit beyond the normal scope of CI. Unfortunately, your best bet is to manually extend the database drivers, specifically override the function simple_query or _execute (simple_query is a wrapper around _execute which simply ensures initialization). That is really the only place where you can guarantee that you can catch all of the queries and branch the logic accordingly. (You may also want to override close as that is the cleanup script)
(Personally, I would have a the SELECT DB load a secondary DB into itself and just call $write_db->simple_query conditionally, that seems like it would be the least trouble).
Is it preferred to create tables in mysql using a third party application (phpmyadmin, TOAD, etc...) instead of php?
The end result is the same, I was just wondering if one way is protocol.
No, there isn't a 'set-in-stone' program to manage your database and query to it.
However, I highly recommend MySQL Workbench.
It allows you to graphically design your database, query to your database server and do all kinds of administration tasks.
I'd say it is far easier to do so within an application created for that purpose. The database itself obviously doesn't care as it's just DDL to it. Using Toad or PHP MyAdmin would help you do the job quicker and allow you to catch syntax errors prior to execution or use a wizard where you're not writing it by hand in the first place.
usually a software project provides one or more text files containing the ddl statements to create the necessary tables. what tool you use to execute those statements doesn't really matter. some php projects alwo provide a installer wizard php file which can be executed directly in the browser, so you don't need any additional tools at all.
I'll try to only answer what your question is - "Is it preferred to create tables in mysql using a third party application (phpmyadmin, TOAD, etc...) instead of php?"...
Yes, it is preferred to create tables or alter them or delete them or perhaps do any DB-related activity that is outside the scope of what interfaces your application provides, in MySQL using any of the many available MySQL clients. And the reason is because these applications are designed to perform DB related tasks and are best at doing them.
Though you may as well use PHP for creating tables depending on the situations, like if the application uses dynamic tables or needs "temporary" tables for performing complex jobs or storing intermediary results/calculations. Or perhaps if the application provides interfaces to manage/control certain aspects, like assume that a certain application consists of various user-roles that have their respective columns in the table. If the application provides rights to the admin to delete or add new roles, which will need to delete or add new columns, it's best to do such queries from PHP.
So, putting it again, use MySQL for any DB work that is not related or affected by what functionality or interfaces your PHP code provides.
Sidenote: Though I've used phpMyAdmin, TOAD, WorkBench and a few others, I think nothing's as efficient and quick as the MySQL client itself, i.e. working directly on the MySQL prompt. If you've always used GUI clients, you might find it unattractive to work on the prompt initially but it's real fun and helps you keep syntaxes on your tips :-)
You question might have been misunderstood by some people.
Charles Sprayberry was saying there's no best practice as far as which 3rd party MySQL client (i.e. phpmyadmin, TOAD, etc.) to use to edit your database. It comes down to personal preference.
Abhay was saying (and I really think this was the answer to your question), that typically, your application does not do DDL (although exceptions exist). Rather, your application will usually be performing DML commands only.
DML is Data Manipulation Language. For example:
select
insert
update
delete
DDL is Data Definition Language. For example:
create table
alter table
drop table
Basic SQL statements: DDL and DML
I've seen this question around the internet (here and here, for example), but I've never seen a good answer. Is it possible to find the length of time a given MySQL query (executed via mysql_query) took via PHP?
Some places recommend using php's microtime function, but this seems like it may be inaccurate. The mysql_query may be bogged down by network latency, or a sluggish system which isn't responding to your query quickly, or some other unrelated cause. None of these are directly related to the quality of your query, which is the only thing I really want to test out here. (Please mention in the comments if you disagree!)
My answer is similar, but varied. Record the time before and after the query, but do it within your database query class. Oh, you say you are using mysql_query directly? Well, now you know why you should use a class wrapper around those raw php database functions (pardon the snark). Actually, one is already built called PDO:
http://us2.php.net/pdo
If you want to extend the functionality to do timing around each of your queries... extend the class! Simple enough, right?
I you are only checking the quality of the query itself, then remove PHP from the equation. Use a tool like the MySQL Query Browser or SQLyog.
Or if you have shell access, just connect directly. Any of these methods will be superior in determining the actual performance of your queries.
At the php level you pretty much would need to record the time before and after the query.
If you only care about the query performance itself you can enable the slow query log in your mysql server: http://dev.mysql.com/doc/refman/5.0/en/slow-query-log.html That will log all queries longer than a specified number of seconds.
If you really need query information maybe you could make use of SHOW PROFILES:
http://dev.mysql.com/doc/refman/5.0/en/show-profiles.html
Personally, I would use a combination of microtime-ing, the slow query log, mytop, and analyzing problem queries with the MySQL client (command line).