I'm creating an API with PHP and PostgreSQL.
This is one of the query I use:
SELECT id, email, first_name, last_name, [...] FROM clients WHERE id = ?
The column "id" is an integer and I don't need a bigger type. ? is replaced by a value coming from the user.
But if the user sends a too big integer (9999999999999) PostgreSQL returns an error:
SQLSTATE[22003]: Numeric value out of range: 7 ERROR: value
\"99999999999999999999999\" is out of range for type integer
Should I check the overflow in the PHP logic ?
Can I turn this error into warning ?
The same query on MySQL doesn't fail...
Thank you
"The same query on MySQL doesn't fail" isn't a great recommendation (unless you're running MySQL in ANSI STRICT mode). MySQL accepts 0000-00-00 as a date, coerces invalid entries to nulls in places, etc, and often does so with at most a warning to tell you it's mangled your data.
That said, I do wish PostgreSQL offered versions of type conversion functions that returned an error code / returned null, so I could explicitly write:
INSERT INTO mytable(blah) VALUES (bigint_in_nullifinvalid('99999999999999999'));
It doesn't, though.
You can write such a function in PL/PgSQL but it's not efficient, and there's none built-in.
While I'm really happy that PostgreSQL won't mangle my data without being asked to, there are times I'd rather like it to mangle my data when I did ask it to.
In general it's better to:
Validate on the application side first;
Let the database enforce integrity; and
Trap errors and report them to the user. You can match the SQLSTATE return to find out details about the error.
PostgreSQL doesn't have any "insert invalid data with warnings" option.
Related
I'm having a problem trying to do an INSERT, it's succeeding even though it shouldn't.
My table structure:
Note: given1, given2 and given3 are required fields.
In my application I execute the following method (https://github.com/catfan/Medoo):
$this->medoo->insert('teste', ['dado2' => 11, 'dado3' => 'teste']);
This should not be accepted because data2 is of type SET, and data3 is of type INTEGER`. Even so, the insertion succeeds. Moreover, data1 is not passed as an argument, even though it is a required field.
I checked the MySQL log and got the following entries:
SET SQL_MODE=ANSI_QUOTES;
INSERT INTO "teste" ("dado2", "dado3") VALUES ('11', 'teste');
Running this SQL manually in the database, I discovered that the problem is in using ANSI_QUOTES. The database somehow accepts the insert, and instead of issuing an error message it issues a warning:
I think that is need to modify the Medoo source code or report this problem to MySQL. I do not know what to do.
MySQL version is 5.7.14
I use the MySQL Workbench 6.3.6
When mySQL server is set in STRICT mode, it produces error when you try to insert values that do not fit into a column. By calling SET SQL_MODE=ANSI_QUOTES; this strict mode is most likely disabled.
You can try to contact the author of Medoo framework or rather use another framework or just PDO to access your database.
See mysql docs on server modes for more information.
Please see the bottom of this post for newest information and current status
Following advise from posts like this one:
Using wildcards in prepared statement - MySQLi
I have my statement set up and it works with no errors. But it does not return the correct data.
My select statement has this for the WHERE:
WHERE `Name` LIKE ? order by `Name`
My string to set up the binding, and then the actual binding.
$whatToBind = '%'. $whatName .'%';
$stmt = $mysqli->prepare($selectStr);
$stmt->bind_param('s', $whatToBind);
$stmt->execute();
When I get my return, it will completely miss records that it should match.
Like, if I send in "Ken L", I get back records for "Ken Linton" but not "Ken Lawton". If I put in "Lawton", I get no return at all.
This is typical behavior across the board. If I search a phone number field, I get returns on "658", but no returns on "609-658".
If anyone can clue me in on what I'm missing, that would be great.
Example returns that show the exact examples I'm referring to:
Incomplete:
Empty, though it shouldn't be:
Returns all, including the record that should have been there with the other 2:
Questions to answer:
Some further things to check:
Check the MySQL / PHP interaction character set is set correctly, typically with: $mysqli->set_charset("utf8mb4"); right after database connection is established.
It is set to utf8. Although it behaved the same before this was set.
Can you show any output from $mysqli->error ?
There are no errors. Just incomplete returns
Can you show us your whole SQL query?
It's included in the screen grabs. Although, that's just a plain string. And it doesn't account for what the prepared statement looks like.
Can you show the Collation / MySQL structure of the Names column?
It is all utf8 as per GoDaddy's phpMyAdmin
Can you show what the value of $whatName is right before binding?
It's at the top of the screen grab. It's echoed back to show it before anything else happens.
At this point I am thinking that the issue lies in what happens when the field I'm searching has a space or other character that is not a letter. Not what I'm passing in exactly. But more like, once the statement is prepared what is prepared is not matching what is in the field it is searching. This doesn't happen when you search the field prior to where the space exists. This is why "Ken" works 100% of the time, but "Lawton" fails completely. It's after the space.
I have tried all manner of converting the encoding type. And I have tried the various methods of concatenating the string. The results I'm getting are either no better, of completely breaking it.
Still 21 hours left on this bounty, if anyone has any more ideas.
At this point, I'd be more happy to award 25 each to the two dudes that provided the best information. Seems unfair to reward one and not the other.
Please note that the details on this answer are unlikely to resolve the question on their own. With my own further testing I established that appending % or _ wildcards to strings before they are bound does not effect the way they are bound to the query.
What you are currently trying to do is Concatenate the data ($whatName) with an SQL instruction % each side of it, and the Prepared Statement parser is simply not having this, as it defeats the security purposes of prepard statements.
So, the solution is that you need to manually concatenate the variable into the LIKE statement only at the point of insertion, and not before, as you are doing at the moment.
The example below will do as you intend:
$selectStr = WHERE `Name` LIKE CONCAT('%',?,'%') ORDER BY `Name`
$whatToBind = $whatName;
$stmt = $mysqli->prepare($selectStr);
$stmt->bind_param('s', $whatToBind);
$stmt->execute();
Note that the data and the query never mix, until the prepared statement is executed.
A note on UTF-8 and character sets on MySQL:
Do not use the utf8_ MySQL character sets as they are an incomplete subset of true UTF-8 and so can still raise serious issues with character recognition. Instead you want to be using utf8mb4_ character sets/collations/etc.
Character encoding may be a related issue to your problem, and I highly recommend reading up on the excellent answers given on this Stack Overflow Question as well as using the PHP mb_ multibyte string functions.
Some further things to check:
Check the MySQL / PHP interaction character set is set correctly, typically with: $mysqli->set_charset("utf8mb4"); right after database connection is established.
Can you show any output from $mysqli->error ?
Can you show us your whole SQL query?
Can you show the Collation / MySQL structure of the Names column?
Can you show what the value of $whatName is right before binding? (while your question makes sense, having a specific example of a specific situation and a specific result as well as an intended result is very useful for us to debug.
Silly thing but ensure you don't have a LIMIT on your results! :-D
It probably is some sort of mixed up encoding issue in $whatName.
Check if your $whatName variable encoding, is UTF8.
mb_detect_encoding($whatName, 'UTF-8', true) // should return true
if not then you will have to use mb_detect_encoding and mb_convert_encoding on $whatName to convert it to utf8.
If you have not done this already
Set proper charset
$mysqli->set_charset('utf8mb4');
// if your MySQL version is lower than 5.5.3 then
// use $mysqli->set_charset('utf8');
before your prepared statement
$stmt = $mysqli->prepare($selectStr);
$stmt->bind_param('s', $whatToBind);
$stmt->execute();
Example:
$user_input = $_POST['input'];
'SELECT '.$user_input.' FROM table_name'
So it's selecting a column in a database based on a secure (this example isn't secure obviously) value. Is this practical/allowable code?
In SQL you simply send a string to the DBMS (like MySQL). Because of this you can build any string you want and submit it to the DBMS.
You just have to make sure the resulting SQL query is valid. That means that e.g. the columns exist and that no invalid symbol appears.
On it's face, this code is valid, assuming that $user_input must be a valid column name, which means that it must exist and must not contain any special characters, reserved words, etc. (unless they're escaped).
As you said, however, this code isn't secure, but as long as you plan to build the query securely and use PDO or MySQLi (no deprecated mysql_* functions...), you should be fine. If you need an example that doesn't use deprecated functions (including mysql_real_escape_string, which is also being deprecated) I'll provide one.
I know you stated that you know this code isn't secure, but here's another example if you're curious. As was discussed in the comments and this question, this input:
$user_input = '; DELETE FROM table_name ; *';
'SELECT '.$user_input.' FROM table_name'
will delete the entire contents of the table table_name. Even though this code raises a syntax error, MySQL will continue to execute it, thus effectively truncating table_name.
I've inherited a project which we are trying to migrate to MySQL 5 from MySQL 4.0(!) and from myISAM to InnoDB. Queries are now falling down because they are being constructed using an ADODB connection's ->qstr() on all parameters, including ints. Where no value is provided I end up with:
INSERT INTO tablename VALUES ('', 'stuff'...)
where the first column is an auto_increment. This causes an error (fair enough since '' isn't an int). Is there a switch in MySQL to make it behave as it used to (I assume it just silently converted to 0?)
Edit:
I just ran a few tests and what I wrote below won't help you at all. The error is because of the wrong datatype, and the SQL setting I suggested doesn't change that. I'll leave this answer here though, since it might be helpful to someone else.
Firstly, double check that the column really is auto increment - a couple of times I've had CREATE TABLE files where the fact that a column is auto_increment was sadly missing.
The other thing which might help is to check that NO_AUTO_VALUE_ON_ZERO is not turned on.
SET SQL_MODE='' should turn it off.
Is there any lightweight validation library available for PHP that easily can check if a specific string or value is valid for a known database type -
Something like this:
if (is_MEDIUMINT($var)) {
$this->db->insert($anothervar);
}
Thanks!
This isn't as simple as it seems, your is_MEDIUMINT() function could easily become:
is_MEDIUMINT()
is_MEDIUMINT_NULL()
is_MEDIUMINT_NOTNULL()
is_MEDIUMINT_UNSIGNED()
is_MEDIUMINT_UNSIGNED_NULL()
is_MEDIUMINT_UNSIGNED_NOTNULL()
Then you run into the problem of different databases types, SQLite for instance has only one INT type while MySQL has at least 5 (TINYINT, SMALLINT, MEDIUMINT, INT, BIGINT), not counting aliases (such as INTEGER, BOOL, BOOLEAN and SERIAL) and float types - which would be even harder to implement due to the variable precision argument. Bare in mind that I'm still ignoring several crucial features such as UNIQUE and Foreign Key Constrains which could only be validated on the DB.
I don't understand why you think such functions would be useful, because if you could set up your database to work in strict mode and then simply try to insert the values, if the query fails you know something is wrong, quoting the MySQL Manual:
In nonstrict mode, when an
out-of-range value is assigned to an
integer column, MySQL stores the value
representing the corresponding
endpoint of the column data type
range. If you store 256 into a TINYINT
or TINYINT UNSIGNED column, MySQL
stores 127 or 255, respectively.
What you be the point of validating the value prior to insertion anyway?
if (is_MEDIUMINT($var)) {
$this->db->insert($anothervar);
}
else {
// do what?
}
If you're trying to avoid errors run the query in a transaction or use the INSERT OR IGNORE syntax.
The INFORMATION_SCHEMA database is part of the ANSI 2003 specification, so you could use that across any DB vendor that supports it (MySQL, Postgresql, SQLite, MSSQL 2k5+).
/*public static*/ function is_dbtype($table, $column, $type) {
$db = (...)::getInstance(); // assuming PDO
$sql = 'SELECT COLUMN_TYPE FROM INFORMATION_SCHEMA.COLUMNS '.
'WHERE TABLE_NAME = :table AND COLUMN_NAME = :column';
$st = $db->prepare($sql);
$st->execute(array(':table' => $table, ':column' => $column));
return ($type == $st->fetchColumn());
}
Change COLUMN_TYPE to DATA_TYPE if you just want "varchar" instead of "varchar(64)". If you need more, there's plenty: IS_NULLABLE, NUMERIC_PRECISION, CHARACTER_SET_NAME, etc.
(Not sure I'd use personally use this though, the is_* functions usually do enough without an extra database call. More importantly, info_schema holds the structure of every database on the server, so granting read permissions to it might (should) be a big deal. If you're on a shared host you likely won't have access to it at all.)
MySQL-only alternate: do similar but with DESCRIBE [table]. It's pretty explicit though, you'll have to fish out the "bigint" in "bigint(21) unsigned" yourself if that's all you want.
Throw it into the database and see if it returns errors. If it doesn't, you types are good enough.
This also means that the dbms you are using will handle the valiation, which means you don't have to update all your validation functions when they decide to change theirs on a whim. And you will most likely not notice this until everything dies and you can't work out why. The less code you have to maintain yourself, the easier your life is :)
Not as far as I know.
You could, of course, create your own functions or class to do this, based on the rules of your specific database type.
Sorry I can't be more help. (And happy to be educated by any other users if there is a class/functions out there.)
Are you just looking for the is_* functions in PHP?
is_integer, is_float etc. ?
There is also get_type, but it shouldn't be used for type checking