PHP SQLite JSON Data Duplication - php

I have the following PHP code:
$testMessage = "TESTMESSAGE";
$db = new SQLite3('messages.sq3');
$db->exec('CREATE TABLE messages(id INTEGER PRIMARY KEY, message CHAR(255));');
$db->exec("INSERT INTO messages (message) VALUES ('$testMessage');");
$results = $db->query('SELECT * FROM messages ORDER BY id DESC LIMIT 5');
while ($row = $results->fetchArray()) {
print_r($row);
}
The resulting print_r:
Array ( [0] => 1 [id] => 1 [1] => TESTMESSAGE [message] => TESTMESSAGE )
Why is this data duplicated? Is this just the way the array is presented or are there really two copies of TESTMESSAGE string? Inspecting the sqlite file, I only see one actually stored there. I am trying to serialize the output via JSON and this duplication is carrying through to the serialization.

The default is to have the data with both numeric and string keys, merged in the same array.
You need to use $results->fetchArray(SQLITE3_NUM) or $results->fetchArray(SQLITE3_ASSOC) to get numeric and string keys respectively. The default is SQLITE3_BOTH, which I've always hated.

Both $row[1] and $row['message'] will give you the same data. This is because on technique uses the numerical index of the column and the other uses the name. They are both included in the column so that you can use either way to access them. It does not indicate any sort of duplication in the database itself.
Here you can see the documentation and how to tell PHP which version you want. By default it gives you both: http://php.net/manual/en/sqlite3result.fetcharray.php

Related

Truncated incorrect DOUBLE value: '12,11''

My POST form sends a value &messages=12,11
I get it using: $messages = $ep->remove($_POST["messages"]);
And my SQL string is:
$query = $db->prepare("DELETE FROM messages WHERE messageID IN ('".$messages."') AND accountID=:accountID");
$query->execute([':accountID' => $accountID]);
And the error appears....
<b>Fatal error</b>: Uncaught exception 'PDOException' with message 'SQLSTATE[22007]: Invalid datetime format: 1292 Truncated incorrect DOUBLE value: '12,11'' in /var/www/vhosts/xxx.xxx/xx/xxx/xx/xxxx/messages.php:17
This code deletes multiple messages from the database. But don't works for me. Any fix?
Remove the quotation marks:
$query = $db->prepare("DELETE FROM messages WHERE messageID IN (".$messages.") AND accountID=:accountID");
Otherwise the value you're sending is 12,11 (which isn't a number as per your database definition), as opposed to 12 and 11, which are both numbers.
Finally, this particular query structure is open to SQL injection. You may want to either sanitise the $messages variable (since it can only include numbers), or create a prepared statement.
For this example, sanitising could work as follows:
$messages = preg_replace('/[^0-9,]/', '', $messages);
//Removes all characters besides numbers and commas
You could also ensure that the list of numeric message IDs always matches the following regex pattern:
$\d+(?:,\d+)*$
That is, the parameter should always be some number, followed by an optional quantity of ,\d+ terms
if (!preg_match("/^\d+(?:,\d+)*$/", $messages)) {
// throw an exception, you are being injected
}
Just to add to the other answer, here is a way to properly prepare all the values
$array = explode(',', '1,2,3,4,5,6');
$keys = preg_replace('/(.+)/', ':v\1', array_keys($array));
print_r($keys);
$sql = "DELETE FROM messages WHERE messageID IN ( ".implode(',', $keys)." ) AND accountID=:accountID";
print_r($sql);
$params = array_combine($keys,$array);
$params['accountID'] = 'foo';
print_r($params);
//$stmt->execute($params);
Output
//print_r($keys);
Array
(
[0] => :v0
[1] => :v1
[2] => :v2
[3] => :v3
[4] => :v4
[5] => :v5
)
//print_r($sql);
DELETE FROM messages WHERE messageID IN ( :v0,:v1,:v2,:v3,:v4,:v5 ) AND accountID=:accountID
//print_r($params);
Array
(
[:v0] => 1
[:v1] => 2
[:v2] => 3
[:v3] => 4
[:v4] => 5
[:v5] => 6
[accountID] => foo
)
Sandbox
It was way too much for a comment.
Basically it takes the keys and using preg replace we can take the sequential number and add a string to it (placeholders have to start with a alpha). Technically the : in the array for execute is optional.
Then we can put that into the query, as it's all generated by PHP because the keys are made from explode. If you have keys from POST, don't use those instead use array_keys(array_values($array)) array values will make the array numerically indexed, then you use those keys.
Next using array combine we can merge those keys back into the original values and put that array into execute
In this case You can do it just with a regex, but I wanted to show something that was useful for more complex cases of IN.

Serialized multidimensional stored in MySQLi does not print past first array

Confusing title, the basics are that I'm saving a fully sorted and ordered multidimensional array from a script and into MySQL. I then, on another page, pull it from the database and unserialize it, and then proceed to print it out with this,
$s = "SELECT * FROM gator_historical_data WHERE channelid = '{$chanid}'";
$r = $link->query($s);
$comboarray = array();
while ($row = mysqli_fetch_assoc($r)) {
$comboarray[] = unserialize($row['dataarray']);
}
foreach ($comboarray as $item) {
$desc = $item['content']['description'];
$title = $item['content']['title'];
$datetime = $item['datetime'];
// ... ^^^ problems getting array data
}
The problem is that it doesn't take the full array from MySQL, only the first entry and thus only prints the first 'array'. So where the returned value from dataarray looks like this (var_dump): http://pastebin.com/raw.php?i=Z0jy55sM the data stored into the unserialized $comboarray only looks like this (var_dump): http://pastebin.com/raw.php?i=Ycwwa924
TL;DR: Pulling a serialized multidimensional array from a database, unserializing and it loses all arrays after the first one.
Any ideas what to do?
The string you've got is a serialized string plus something more at the end that is also a serialized string again and again:
a:3:{s:6:"source";s:25:"World news | The Guardian";s:8:"datetime ...
... story01.htm";}}a:3:{s:6:"source";s:16:"BBC News - World";
^^^
This format is not supported by PHP unserialize, it will only unserialize the first chunk and drop everything at the end.
Instead create one array, serialize it and store that result into the database.
Alternatively you can try to recover for the moment by un-chunking the string, however in case the paste was done right, there are more issues. But on the other hand the paste obvious isn't the done fully correct.

PHP MYSQL unserialize() issues

I have a "recruiter" table in my database which has different attributes and one of them is "Professions". "Professions" is a serialized array which I get from a multiple select form. And this works fine.
When I unserialize this attribute nothing is printed - no error, no text.
This is a code I was testing serialization with:
$sql = 'SELECT Company_name, Status, Size, Professions, Seniority_levels, Sector, Website, Location FROM Recruiter';
$query = mysql_query($sql, $con);
while($result = mysql_fetch_array($query, MYSQL_BOTH)){
$recruiters[] = array($result[0], $result[1], $result[2], $result[3], $result[4], $result[5], $result[6], $result[7]);
}
foreach($recruiters AS $recruiter){
$test = unserialize($recruiter[3]);
echo $test[0].'<br>';
}
So basically $test[0] prints nothing although the new lines are printed. Please help!
try printing the $test array and the $recruiters and the $recruiter arrays. See if the result is fine before the unserialisation of the data. If the query returns any data. Also try the while loop with mysql_fetch_assoc. Let me know of the results and if this solves the problem
test = unserialize($recruiter[3]); should become test = unserialize($recruiter[5]); since the sector field is the sixth column .
However what if somewhere in the future you might need to select rows where sectors equal smth ? serialize whont help you then so i suggest you have a look at a different implementation for the sector filed witch is called bitwize http://www.litfuel.net/tutorials/bitwise.htm
Edit
Asuming you hit the right column and the column contains a:1:{i:0;s:27: a:1:{i:0;s:27: a:38:{i:0;s:27: a:9:{i:0;s:39:, it looks like the serialized array is not fully saved in you're db, it's only part of it . So the unserialize function whont return you an array . Have a look at the length of the mysql field i assume you've set it smaller than you need so you're data is trimmed on insert/update .
Edit
a:1:{i:0;s:27: you're still missing the rest of the serialized array . s:27: means a string is following containint 27 characters, and you're serialized array stops there when it should look like
a:1:{i:0;s:27:"123456789012345678901234567";}
( a:1 stands for an array containing 1 value with it's content between {}, i:0; is the array key 0, s:27:""; stands for a string containing 27 characters as the value for the i:0 key ) .

mysql_fetch_array Returning Two Of Everything

I'm not exactly new to PHP but I haven't used MySQL that much, so I'll ask you guys about this one.
I have a database with one row of data, which consists of a date and two integers. However, when I query the database with this:
$query = "SELECT * FROM history";
$theQuery = mysql_query($query, $connect);
$array = mysql_fetch_array($theQuery);
... and I do a print_r, I get the following:
Array
(
[0] => 2010-08-17
[date] => 2010-08-17
[1] => 17454
[posts] => 17454
[2] => 1058
[members] => 1058
)
Am I doing something wrong? I plan on having many rows with a ton of data and printing it to a table, THEN how would I go about sorting through it?
First : You did everything right
You can use mysql_fetch_assoc or mysql_fetch_array(resource $result, MYSQL_ASSOC).
The default is to return MYSQL_BOTH (associative array + numeric array).
For more information, see mysql_fetch_array.
The second argument to mysql_fetch_array is optional but default is MYSQL_BOTH and will give you values both for indices and keys. You can use MYSQL_ASSOC instead.
The PHP manual on mysql_fetch_array:
Returns an array of strings that corresponds to the fetched row, or FALSE if there are no more rows. The type of returned array depends on how result_type is defined. By using MYSQL_BOTH (default), you'll get an array with both associative and number indices. Using MYSQL_ASSOC, you only get associative indices (as mysql_fetch_assoc() works), using MYSQL_NUM, you only get number indices (as mysql_fetch_row() works).
With mysql_fetch_array, you can run it in a loop ,
ex:
while($array = mysql_fetch_array($theQuery))
{
echo $array["date"];
echo $array["posts"];
echo $array["members"];
}

Save PHP array to MySQL?

What is a good way to save an array of data to a single mysql field?
Also once I query for that array in the mysql table, what is a good way to get it back into array form?
Is serialize and unserialize the answer?
There is no good way to store an array into a single field.
You need to examine your relational data and make the appropriate changes to your schema. See example below for a reference to this approach.
If you must save the array into a single field then the serialize() and unserialize() functions will do the trick. But you cannot perform queries on the actual content.
As an alternative to the serialization function there is also json_encode() and json_decode().
Consider the following array
$a = array(
1 => array(
'a' => 1,
'b' => 2,
'c' => 3
),
2 => array(
'a' => 1,
'b' => 2,
'c' => 3
),
);
To save it in the database you need to create a table like this
$c = mysql_connect($server, $username, $password);
mysql_select_db('test');
$r = mysql_query(
'DROP TABLE IF EXISTS test');
$r = mysql_query(
'CREATE TABLE test (
id INTEGER UNSIGNED NOT NULL,
a INTEGER UNSIGNED NOT NULL,
b INTEGER UNSIGNED NOT NULL,
c INTEGER UNSIGNED NOT NULL,
PRIMARY KEY (id)
)');
To work with the records you can perform queries such as these (and yes this is an example, beware!)
function getTest() {
$ret = array();
$c = connect();
$query = 'SELECT * FROM test';
$r = mysql_query($query,$c);
while ($o = mysql_fetch_array($r,MYSQL_ASSOC)) {
$ret[array_shift($o)] = $o;
}
mysql_close($c);
return $ret;
}
function putTest($t) {
$c = connect();
foreach ($t as $k => $v) {
$query = "INSERT INTO test (id,".
implode(',',array_keys($v)).
") VALUES ($k,".
implode(',',$v).
")";
$r = mysql_query($query,$c);
}
mysql_close($c);
}
putTest($a);
$b = getTest();
The connect() function returns a mysql connection resource
function connect() {
$c = mysql_connect($server, $username, $password);
mysql_select_db('test');
return $c;
}
Generally, yes, serialize and unserialize are the way to go.
If your data is something simple, though, saving as a comma-delimited string would probably be better for storage space. If you know that your array will just be a list of numbers, for example, then you should use implode/explode. It's the difference between 1,2,3 and a:3:{i:0;i:1;i:1;i:2;i:2;i:3;}.
If not, then serialize and unserialize work for all cases.
Just use the serialize PHP function:
<?php
$myArray = array('1', '2');
$seralizedArray = serialize($myArray);
?>
However, if you are using simple arrays like that you might as well use implode and explode.Use a blank array instead of new.
Serialize/Unserialize array for storage in a DB
Visit http://php.net/manual/en/function.serialize.php
From the PHP Manual:
Look under "Return" on the page
Returns a string containing a byte-stream representation of value that can be stored anywhere.
Note that this is a binary string which may include null bytes, and needs to be stored and handled as such. For example, serialize() output should generally be stored in a BLOB field in a database, rather than a CHAR or TEXT field.
Note: If you want to store html into a blob, be sure to base64 encode it or it could break the serialize function.
Example encoding:
$YourSerializedData = base64_encode(serialize($theHTML));
$YourSerializedData is now ready to be stored in blob.
After getting data from blob you need to base64_decode then unserialize
Example decoding:
$theHTML = unserialize(base64_decode($YourSerializedData));
The best way, that I found to myself is save array as data string with separator characters
$array = array("value1", "value2", "value3", "...", "valuen");
$array_data = implode("array_separator", $array);
$query = "INSERT INTO my_tbl_name (id, array_data) VALUES(NULL,'" . $array_data . "');";
You can then search data, stored in your array with simple query
$query = "SELECT * FROM my_tbl_name WHERE array_data LIKE '%value3%'";
use explode() function to convert "array_data" string to array
$array = explode("array_separator", $array_data);
note that this is not working with multidimensional arrays and make sure that your "array_separator" is unique and had not exist in array values.
Be careful !!! if you just will take a form data and put in database, you will be in trap, becous the form data isn't SQL-safe ! you must handle your form value
with mysql_real_escape_string or if you use MySQLi mysqli::real_escape_string
or if value are integer or boolean cast (int) (boolean) on them
$number = (int)$_POST['number'];
$checked = (boolean) $_POST['checked'];
$name = mysql_real_escape_string($db_pt, $_POST['name']);
$email = mysqli_obj->real_escape_string($_POST['email']);
Serialize and unserialize are pretty common for that. You could also use JSON via json_encode and json_decode for a less PHP-specific format.
As mentioned before - If you do not need to search for data within the array, you can use serialize - but this is "php only". So I would recommend to use json_decode / json_encode - not only for performance but also for readability and portability (other languages such as javascript can handle json_encoded data).
Uhh, I don't know why everyone suggests serializing the array.
I say, the best way is to actually fit it into your database schema. I have no idea (and you gave no clues) about the actual semantic meaning of the data in your array, but there are generally two ways of storing sequences like that
create table mydata (
id int not null auto_increment primary key,
field1 int not null,
field2 int not null,
...
fieldN int not null
)
This way you are storing your array in a single row.
create table mydata (
id int not null auto_increment primary key,
...
)
create table myotherdata (
id int not null auto_increment primary key,
mydata_id int not null,
sequence int not null,
data int not null
)
The disadvantage of the first method is, obviously, that if you have many items in your array, working with that table will not be the most elegant thing. It is also impractical (possible, but quite inelegant as well - just make the columns nullable) to work with sequences of variable length.
For the second method, you can have sequences of any length, but of only one type. You can, of course, make that one type varchar or something and serialize the items of your array. Not the best thing to do, but certainly better, than serializing the whole array, right?
Either way, any of this methods gets a clear advantage of being able to access an arbitrary element of the sequence and you don't have to worry about serializing arrays and ugly things like that.
As for getting it back. Well, get the appropriate row/sequence of rows with a query and, well, use a loop.. right?
You can save your array as a json.
there is documentation for json data type: https://dev.mysql.com/doc/refman/5.7/en/json.html
I think this is the best solution, and will help you maintain your code more readable by avoiding crazy functions.
I expect this is helpful for you.
Yup, serialize/unserialize is what I've seen the most in many open source projects.
I would suggest using implode/explode with a character that you know will not be contained in any of the individual array items. Then store it in SQL as a string.
you can insert serialized object ( array ) to mysql , example serialize($object) and you can unserize object example unserialize($object)
check out the implode function, since the values are in an array, you want to put the values of the array into a mysql query that inserts the values into a table.
$query = "INSERT INto hardware (specifications) VALUES (".implode(",",$specifications).")";
If the values in the array are text values, you will need to add quotes
$query = "INSERT INto hardware (specifications) VALUES ("'.implode("','",$specifications)."')";
mysql_query($conn,$query);
Also, if you don't want duplicate values, switch the "INto" to "IGNORE" and only unique values will be inserted into the table.
UPDATE
Warning
This extension was deprecated in PHP 5.5.0, and it was removed in PHP 7.0.0. Instead, the MySQLi or PDO_MySQL extension should be used. See also MySQL: choosing an API guide. Alternatives to this function include:
mysqli_query
PDO::query()
Instead of saving it to the database, save it to a file and then call it later.
What many php apps do (like sugarcrm) is to just use var_export to echo all the data of the array to a file.
This is what I use to save my configurations data:
private function saveConfig() {
file_put_contents($this->_data['pathtocompileddata'],'<?php' . PHP_EOL . '$acs_confdata = ' . var_export($this->_data,true) . ';');
}
I think this is a better way to save your data!

Categories