I have a database and a string with about 100,000 key / value-pair records and I want to create a function to find the value.
Is it better to use a string or a database, considering performance (page load time) and crash safety? Here are my two code examples:
1.
echo find("Mohammad");
function find($value){
$sql = mysql_query("SELECT * FROM `table` WHERE `name`='$value' LIMIT 1");
$count = mysql_num_rows($sql);
if($count > 0){
$row = mysql_fetch_array($sql);
return $row["family"];
} else {
return 'NULL';
}
}
2.
$string = "Ali:Golam,Mohammad:Offer,Reza:Now,Saber:Yes";
echo find($string,"Mohammad");
function find($string,$value){
$array = explode(",",$string);
foreach($array as $into) {
$new = explode(":",$into);
if($new[0] == $value) {
return $new[1];
break;
}
}
}
The database is pretty sure a good idea.
Databases are fast, maybe they are not as fast as basic String operations in PHP, but if there is a lot of data, databases will probably be faster. A basic select Query takes (on my current default Desktop Hardware) about 15ms, cached less than 1ms, and this is pretty much independend of the number of names in your table, if the indexes are correct. So your site will always be fast.
Databases won't cause a StackOverflow or an out of memory error and crash your site (this is very depending on your PHP-Settings and Hardware)
Databases are more flexible, imagine you want to add / remove / edit names after creating the first Data-Set. It's very simple with "INSERT INTO", "DELETE FROM" and "UPDATE" to modify the data, better than editing a string somewhere in your code with about 100.000 entries.
Your Code
You definitly need to use MySQLi or PDO instead, code maybe like this:
$mysqli = new mysqli("host", "username", "password", "dbname");
$stmt = $mysqli->prepare(
'SELECT string FROM table WHERE name = ? LIMIT 1'
);
$stmt->bind_param("s", $value);
$stmt->execute();
$stmt->store_result();
$stmt->bind_result($string);
$stmt->fetch();
$stmt->close();
This uses MySQLi and Prepared Statements (for security), instead of the depracated MySQL extension (in PHP 5.5)
Related
I'm developing a PHP script and I just want to know if I can make this code piece with better performance.
I have to make 2 mysql queries to complete my task. Is there any other way to complete this with better performance?
$language = "en";
$checkLanguage = $db->query("select id from language where shortName='$language'")->num_rows;
if($checkLanguage>0){
$languageSql = $db->query("select id from language where shortName='$language'")->fetch_assoc();
$languageId = $languageSql['id'];
}else{
$languageSql = $db->query("insert into language(shortName) values('$language')");
$languageId = $db->insert_id;
}
echo $languageId
You can improve your performance, by storing the stamtement object to a variable, this way it will be one less query:
$checkLanguage = $db->query("select id from language where shortName='$language'");
if($checkLanguage->num_rows >0){
$languageSql = $checkLanguage->fetch_assoc();
$languageId = $languageSql['id'];
}else{
$languageSql = $db->query("insert into language(shortName) values('$language')");
$languageId = $db->insert_id;
}
echo $languageId
or second option you add unique constraint to language and shortName.
If you insert a duplicate it will throw an error, if not it will insert, this way you keep only one query the INSERT one, but you might need a try catch for duplicates.
Why not just do something like this:
$language = "en";
$dbh = $db->query("INSERT IGNORE INTO `language` (`shortName`) VALUES ('{$language}');");
$id = $db->insert_id;
echo (($id !== 0) ? $id : FALSE);
This will perform your logic in a single query, and return the id, or false on a duplicate. It is generally better to resolve database performance issues in the SQL rather than in PHP, because about 65% of your overhead is in the actual connection to the database, not the query itself. Reducing the number of queries you run typically has a lot better impact on performance than improving your scripting logic revolving around them. People that consistently rely on ORM's often have a lot of trouble with this, because cookie cutter SQL is usually not very performant.
Does anyone have or can point to a good example of creating sqlite in-memory table using php pdo where the table columns formats are gathered directly from a PDO query result?
My use case is that I'd like to use SQLite3 to store multiple resultsets from a number of queries (all the same column order etc) that are returned from a number of different databases. The end goal is to use SQLite to "reduce" the data for finalization.
I understand I could write a custom function for each query result, but my goal is to implement a universal solution, not needing to know the table structure, passing x homogenous resultsets (PDO::FETCH_ASSOC array results for example) and return a temporary SQLite table name to be able to query the UNION of the resultset(s) with SQLite before discarding the data.
It seems like this would be out on the internet somewhere and the SQLite website even alludes to this being a use case, but all my searches point to basic SQLite howtos on creating and querying...thanks in advance!
Since nobody replied, here is a rough effort. This is based on the db handle being mySQL, but it may be possible to ignore types and let sqlite do some magic. If so, see this link:
https://www.sqlite.org/datatype3.html
try{
$link = connectToDB_PDO();
$strSQL="select * from website.customers";
$mySth=$link->prepare($strSQL);
$meta=array();
$mySth->execute();
foreach(range(0, $mySth->columnCount() - 1) as $column_index){
$meta[] = $mySth->getColumnMeta($column_index);
}
$result=$mySth->fetchAll(PDO::FETCH_ASSOC);
// Create new database in memory
$memdb = new PDO('sqlite::memory:');
$memdb->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION); // Set errormode to exceptions
$tablename='mytable';
$colnames=array(); // for the insert statement below
$tableSQL="CREATE TABLE $tablename ( ";
foreach($meta as $col){
$colnames[]=$col['name'];
$tableSQL.=" ".$col['name']. " ".$col['native_type']." ";
if(array_key_exists('flags',$col)){
foreach($col['flags'] as $flag){
switch($flag)
{
case 'not_null':
$tableSQL.=" NOT NULL ";
break;
case 'primary_key':
$tableSQL.=" PRIMARY KEY ";
break;
default:
} // end switch
} // end each flag
} // end if flags
$tableSQL.=", ";
} // end each column
$tableSQL=rtrim($tableSQL,", ");
$tableSQL.=")";
$memdb->exec($tableSQL);
$pragmaSQL="PRAGMA table_info($tablename)";
$st2=$memdb->prepare($pragmaSQL);
$st2->execute();
$pragres=$st2->fetchAll(PDO::FETCH_ASSOC);
$insert = "INSERT INTO $tablename (".implode(",",$colnames).") ".
"VALUES (:".implode(",:",$colnames).")";
$stmt = $memdb->prepare($insert);
foreach ($result as $row) {
$parms=array();
for($i=0;$i<count($colnames);$i++){
$parms[':'.$colnames[$i]]=$row[$colnames[$i]]; // Bind parameters to statement variables
}
$stmt->execute($parms);
}
$strSQL="SELECT sum(customers_id) FROM $tablename";
$sth=$memdb->prepare($strSQL);
$sth->execute();
$result2=$sth->fetchAll(PDO::FETCH_ASSOC);
foreach ($result2 as $m) {
echo print_r($m,true);
}
$memdb->exec("DROP TABLE $tablename");
$memdb = null;
} catch(PDOException $e) {
echo $e->getMessage();
error_log($e->getMessage());
}
I'm not sure if this is the best answer, but its a start - anxious to see other's input!
i'm trying to make a long mysql query and process and update the row founded:
$query = 'SELECT tvshows.id_show, tvshows.actors FROM tvshows where tvshows.actors is not NULL';
$result = mysql_query($query);
$total = mysql_num_rows($result);
echo $total;
while ($db_row = mysql_fetch_assoc($result))
{
//process row
}
but after 60 second give me a timeout request, i have try to insert these in my php code:
set_time_limit(400);
but it's the same, how i can do?
EDIT:
only the query:
$query = 'SELECT tvshows.id_show, tvshows.actors FROM tvshows where tvshows.actors is not NULL';
takes 2-3 second to perform, so i think the problem is when in php i iterate all the result to insert to row or update it, so i think the problem is in the php, how i can change the timeout?
EDIT:
here is the complete code, i don't think is a problem here in the code...
$query = 'SELECT tvshows.id_show, tvshows.actors FROM tvshows where tvshows.actors is not NULL';
$result = mysql_query($query);
$total = mysql_num_rows($result);
echo $total;
while ($db_row = mysql_fetch_assoc($result)) {
//print $db_row['id_show']."-".$db_row['actors']."<BR>";
$explode = explode("|", $db_row['actors']);
foreach ($explode as $value) {
if ($value != "") {
$checkactor = mysql_query(sprintf("SELECT id_actor,name FROM actors WHERE name = '%s'",mysql_real_escape_string($value))) or die(mysql_error());
if (mysql_num_rows($checkactor) != 0) {
$actorrow = mysql_fetch_row($checkactor);
$checkrole = mysql_query(sprintf("SELECT id_show,id_actor FROM actor_role WHERE id_show = %d AND id_actor = %d",$db_row['id_show'],$actorrow[0])) or die(mysql_error());
if (mysql_num_rows($checkrole) == 0) {
$insertactorrole = mysql_query(sprintf("INSERT INTO actor_role (id_show, id_actor) VALUES (%d, %d)",$db_row['id_show'],$actorrow[0])) or die(mysql_error());
}
} else {
$insertactor = mysql_query(sprintf("INSERT INTO actors (name) VALUES ('%s')",mysql_real_escape_string($value))) or die(mysql_error());
$insertactorrole = mysql_query(sprintf("INSERT INTO actor_role (id_show, id_actor, role) VALUES (%d, %d,'')",$db_row['id_show'],mysql_insert_id())) or die(mysql_error());
}
}
}
}
Should definitely try what #rid suggested, and to execute the query on the server and see the results/duration to debug - if the query is not a simple one, construct it as you would in your PHP script, and only echo the SQL command, don't have to execute it, and just copy that in to the server MySQL command line or whichever tool you use.
If you have shell access, use the top command after running the above script again, and see if the MySQL demon server is spiking in resources to see if it really is the cause.
Can you also try a simpler query in place of the longer one? Like just a simple SELECT count(*) FROM tvshows and see if that also takes a long time to return a value?
Hope these suggestions help.
There are so many problems with your code.
Don't store multiple values in a single column. Your actors column is pipe-delimited text. This is a big no-no.
Use JOINs instead of additional queries. You can (or could, if the above weren't true) get all of this data in a single query.
All of your code can be done in a single query on the server. As I see it, it takes no input from the user and produces no output. It just updates a table. Why do this in PHP? Learn about INSERT...SELECT....
Here are some resources to get you started (from Googling, but hopefully they'll be good enough):
http://www.sitepoint.com/understanding-sql-joins-mysql-database/
http://dev.mysql.com/doc/refman/5.1/en/join.html
http://dev.mysql.com/doc/refman/5.1/en/insert-select.html
What is Normalisation (or Normalization)?
Let me know if you have any further questions.
I was wondering if you think this is possible:
Ok so I have a database storing usernames and I would like to echo the admins which are inside a file called admins.php IF they match the usernames inside the database so far I have got:
admins.php;
$admins = array("username","username2","username3");
and
$users="SELECT username from usrsys";
$query_users=mysql_query($users);
while loop here.
The while loop should hopefully echo the users which matches the admins.php file. I assume I should use something like (inarray()), but I am really not sure.
You should definitely use IN clause in your SQL to do this. Selecting everything from the table in order to determine in PHP if it contains the user names you're looking for makes no sense and is very wasteful. Can you imagine what would happen if you had a table of 1 million users and you needed to see if two of them were on that list? You would be asking your DBMS to return 1 million rows to PHP so that you can search through each of those names and then determine whether or not any of them are the ones you're looking for. You're asking your DBMS to do a lot of work (send over all the rows in the table), and you're also asking PHP to do a lot of work (store all those rows in memory and compute a match), unnecessarily.
There is a much more efficient and faster solution depending on what you want.
First, if you only need to know that all of those users exist in the table then use SELECT COUNT(username) instead and your database will return a single row with a value for how many rows were found in the table. That way you have an all or nothing approach (if that's what you're looking for). Either there were 3 rows found in the table and 3 elements in the array or there weren't. This also utilizes your table indexes (which you should have properly indexed) and means faster results.
$admins = array("username","username2","username3");
// Make sure you properly escape your data before you put in your SQL
$list = array_map('mysql_real_escape_string', $admins);
// You're going to need to quote the strings as well before they work in your SQL
foreach ($list as $k => $v) $list[$k] = "'$v'";
$list = implode(',', $list);
$users = "SELECT COUNT(username) FROM usrsys WHERE username IN($list)";
$query_users = mysql_query($users);
if (!$query_users) {
echo "Huston we have a problem! " . mysql_error(); // Basic error handling (DEBUG ONLY)
exit;
}
if (false === $result = mysql_fetch_row($query_users)) {
echo "Huston we have a problme! " . mysql_error(); // Basic error handling (DEBUG ONLY)
}
if ($result[0] == count($admins)) {
echo "All admins found! We have {$result[0]} admins in the table... Mission complete. Returning to base, over...";
}
If you actually do want all the data then remove the COUNT from the SQL and you will simply get all the rows for those users (if any are found).
$admins = array("username","username2","username3");
// Make sure you properly escape your data before you put in your SQL
$list = array_map('mysql_real_escape_string', $admins);
// You're going to need to quote the strings as well before they work in your SQL
foreach ($list as $k => $v) $list[$k] = "'$v'";
$list = implode(',', $list);
$users = "SELECT username FROM usrsys WHERE username IN($list)";
$query_users = mysql_query($users);
if (!$query_users) {
echo "Huston we have a problem! " . mysql_error(); // Basic error handling (DEBUG ONLY)
exit;
}
// Loop over the result set
while ($result = mysql_fetch_assoc($query_users)) {
echo "User name found: {$result['username']}\n";
}
However, I really urge you to reconsider using the old ext/mysql API to interface with your MySQL database in PHP since it is deprecated and has been discouraged from use for quite some time. I would really urge you to start using the new alternative APIs such as PDO or MySQLi and see the guide in the manual for help with choosing an API.
In PDO, for example this process would be quite simple with prepared statements and parameterized queries as you don't have to worry about all this escaping.
There's an example in the PDOStatement::Execute page (Example #5) that shows you just how to do use the IN clause that way with prepared statements... You can then reuse this statement in other places in your code and it offers a performance benefit as well as making it harder for you to inadvertently expose yourself to SQL injection vulnerabilities.
// Connect to your database
$pdo = new PDO("mysql:dbname=mydb;host=127.0.0.1", $username, $password);
// List of admins we want to find in the table
$admins = array("username","username2","username3");
// Create the place holders for your paratmers
$place_holders = implode(',', array_fill(0, count($admins), '?'));
// Create the prepared statement
$sth = $dbh->prepare("SELECT username FROM usrsys WHERE username IN ($place_holders)");
// Execute the statement
$sth->execute($admins);
// Iterate over the result set
foreach ($sth->fetchAll(PDO::FETCH_ASSOC) as $row) {
echo "We found the user name: {$row['username']}!\n";
}
Your PHP code even looks so much better with PDO :)
Just include admins.php file and use the next construction in your loop:
while ($row = mysql_fetch_array($users)) {
if (in_array($users[0], $admins))
echo $users[0];
}
Try this:
<?php
# include admins.php file that holds the admins array
include "admins.php";
# join all values in the admins array using "," as a separator (to use them in the sql statement)
$admins = join(",", $admins);
# execute the query
$result = mysql_query("
SELECT username
FROM usrsys
WHERE username IN ($admins)
");
if ($result) {
while ($row = mysql_fetch_array($result)) {
echo $row["username"] . "<br>";
}
}
?>
If your looking for syntax to pull in only the users from your $admins array then you could use something like:
$users="SELECT username FROM usrsys WHERE username IN ('".join("','",$admins)."')";
Where the php function JOIN will print username,username2,username3. Your resulting MySQL statement will look like:
SELECT username FROM usrsys WHERE username IN ('username','username2','username3')
Alternatively, if your looking to iterate through your $query_vars array and separate your admins from non-admins then you could use something like:
<?php
while($row = mysql_fetch_assoc($query_users)){
if(in_array($row['username'],$admins)){
//do admin stuff here
}else{
//do NON-admin stuff here
}
}?>
I want to show all text messages from db where id=$e ($err is an array).
Inserted the query into the foreach loop, it works well but it does extra work (does query for every value of array).
Is there any other way to do it (i mean extract query from foreach loop)?
My code looks like this.
foreach ($err as $e)
{
$result = $db -> query("SELECT * from err_msgs WHERE id='$e'");
$row = $result -> fetch_array(MYSQLI_BOTH);
echo "<li><span>".$row[1]."</span></li>";
}
It is much more efficient to do this with implode() because it will only result in one database query.
if (!$result = $db->query("SELECT * FROM `err_msgs` WHERE `id`='".implode("' OR `id`='",$err)."'")) {
echo "Error during database query<br />\n";
// echo $db->error(); // Only uncomment this line in development environments. Don't show the error message to your users!
}
while ($row = $result->fetch_array(MYSQLI_BOTH)) {
echo "<li><span>".$row[1]."</span></li>\n";
}
Check the SQL IN clause.
Firstly, a bit of a lecture: embedding strings directly into your queries is going to cause you trouble at some point (SQL injection related trouble to be precise), try to avoid this if possible. Personally, I use the PDO PHP library which allows you to bind parameters instead of building up a string.
With regard to your question, I'm not sure I have understood. You say that it does extra work, do you mean that it returns the correct results but in an inefficient way? If so then this too can be addressed with PDO. Here's the idea.
Step 1: Prepare your statement, putting a placeholder where you currently have '$e'
Step 2: Loop through $err, in the body of the loop you will set the place holder to be the current value of $e
By doing this you not only address the SQL injection issue, you can potentially avoid the overhead of having to parse and optimise the query each time it is executed (although bear in mind that this may not be a significant overhead in your specific case).
Some actual code would look as follows:
// Assume that $dbdriver, $dbhost and $dbname have been initialised
// somewhere. For a mysql database, the value for $dbdriver should be
// "mysql".
$dsn = "$dbdriver:host=$dbhost;dbname=$dbname";
$dbh = new PDO($dsn, $dbuser, $dbpassword);
$qry = "SELECT * from err_msgs WHERE id = :e"
$sth = $dbh->prepare($qry);
foreach ($err as $e) {
$sth->bindParam(":e", $e);
$sth->execute();
$row = $sth->fetch();
// Prints out the *second* field of the record
// Note that $row is also an associative array so if we
// have a field called id, we could use $row["id"] to
// get its value
echo "<li><span>".$row[1]."</span></li>";
}
One final point, if you want to simply execute the query once, instead of executing it inside the loop, this is possible but again, may not yield any performance improvement. This could be achieved using the IN syntax. For example, if I'm interested in records with id in the set {5, 7, 21, 4, 76, 9}, I would do:
SELECT * from err_msgs WHERE id IN (5, 7, 21, 4, 76, 9)
I don't think there's a clean way to bind a list using PDO so you would use the loop to build up the string and then execute the query after the loop. Note that a query formulated in this way is unlikely to give you any noticable performance improvment but it really does depend on your specific circumstances and you'll just have to try it out.
You can do this much simpler by doing
$err_csv = implode("','",$err);
$sql = "SELECT FROM err_msgs WHERE id IN ('$err_csv')";
$result = $db -> query($sql);
while ($row = $result -> fetch_array(MYSQLI_BOTH))
{
echo "<li><span>".$row[1]."</span></li>";
}
That way you don't have to keep sending queries to the database.
Links:
http://php.net/manual/en/function.implode.php