I recently switched from MySQL to PostgreSQL. I have one problem left however.
Previously, I would store small images in the BLOB format in MySQL.
PostgreSQL doesn't know such thing as a BLOB.
I tried using BYTEA field type instead. This actually inserts an large (hexadecimal?) string I guess, but now I'm stuck trying to get this string back to displaying an actual image in PHP..
Any ideas? Thanks in advance.
Here is a piece of code I use to save the image in the database:
$data = bin2hex(file_get_contents('php://input'));
if (!empty($data)) {
$sql = "UPDATE asset SET data = X'%s' WHERE uuid = '%s'";
$args = array($data, $asset_uuid);
}
psql (9.1.3) and php 5.3.6 are used
Bytea is a byte array. It's not a bit pattern. See section 4.2.1.5 of PostgreSQL Lexical Structure.
The correct way to enter bytea is '\x...' with hex values. So what you want is SET data = '\x%s'.
You might also want to look into prepared statements with pg_prepare.
Edit: I was able to insert a (text) file into a bytea with this:
$source = file_get_contents( 'hello.php' );
$insert = pg_prepare( $conn, '', 'insert into t (name, data) values($1,$2)' );
pg_execute( $conn, '', array( 'hello.php', $source ) );
3rd Edit: This works fine to insert the file into the database. However, the pgsql driver in PHP is quite impolite. The only way to retrieve the actual data back is using the old bytea escape mechanism, as detailed here: pg_unescape_bytea.
pg_query('SET bytea_output = "escape";');
$result = pg_query( 'select data from t' );
while ( $row = pg_fetch_row( $result ) ) {
echo pg_unescape_bytea( $row[0] );
}
I'm sorry about how annoying this is. The PostgreSQL interface in PHP can do with some major overhaul for binary values.
To insert bytea contents with the pg_* API, the binary value should always be run through the pg_escape_bytea() function, even if it's passed to the pg_execute or pg_query_params functions.
This is because the pg_* layer doesn't "know" that a particular parameter has binary contents, and it does not implement any real support for parameter types anyway. So the text representation must be used. It can either be in the escape form or the hex form, it doesn't matter to the PG server, and it's independant of the value of bytea_output, which is meaningful only for values read from the server.
Example:
$esc=pg_escape_bytea("\000\001\002");
pg_query_params('INSERT INTO some_table(some_col) VALUES($1)', array($esc));
To read bytea contents with the pg_* API, the value must be run through pg_unescape_bytea() after the fetch. Assuming the client library is not older than 9.0 (libq.so.5.3 or higher), it can decode the contents whether it's in hex form or escape form and it will autodetect it. Only with an older library would it be necessary to force bytea_output to escape for it to decode properly, either dynamically with SET or statically for the whole database (ALTER DATABASE SET bytea_output=escape) or in postgresql.conf for the whole instance.
Example:
$p=pg_query("SELECT some_col FROM some_table WHERE...");
$r=pg_fetch_array($p);
$contents = pg_unescape_bytea($r[0]);
Both answers posted here gave me some thoughts, but none were 100% of the answer.
So, I will explain in this answer what I did to get it to work.
When displaying the image, I used this:
header('Content-Type: image/jpeg');
$data = pack("H*", pg_unescape_bytea($data));
echo $data;
I'm running PHP 5.3.8, in PHP 5.4.0 it turns out you can use hex2bin instead of pack.
When adding the image to the database, I used this:
$data = pg_escape_bytea($data); // Escape input for PostgreSQL
$sql = "UPDATE asset SET data = '%s'WHERE uuid = '%s'";
I'm glad it is working now. Thank you both Daniel and Johann!
Related
I have a table for things properties.
Such as id, amount, owner, attributes.
The attributes is stores as blob, and could be any small information about the thing itself,.
Like, the thing may have text on it, or it may has something about it that makes it more unique.
When you export a table/row in PhpMyAdmin, I get these really nice values
INSERT INTO `user_things` (`owner_id`, `id`, `thingId`, `count`, `attributes`) VALUES
(1, 6, 101, 1, 0x10c8431200);
And what I'm trying to do, is being able get information from the blob in PHP that displays the blob the same way PhpMyAdmin displayed it, so I can with PHP build a string of query there the blob value stays at a 0x00000 value.
I've tried to use this code: (mysqlQuery is a simplified function I created for mysqli queries, and that one works perfect so don't worry about that part)
$thing = mysqlQuery("SELECT * FROM `user_things` LIMIT 1;");
$blob = $thing["attributes"];
but it doesn't work very well, and I've tried base64_encode and that doesn't give me the 0x000 either.
I would highly appreciate any help on this, how I can get the blob to the same type of value that PhpMyAdmin displays it as.
0x is just a convention for expressing hexadecimal values.
The 0x10c8431200 hexadecimal value is the same like 72079315456 decimal value.
You can try to convert binary data to hex with the bin2hex function.
Example:
$thing = mysqlQuery("SELECT * FROM `user_things` LIMIT 1;");
$blob = $thing["attributes"];
echo bin2hex($blog);
I "inherited" from a collegue a MySQL database with dozen of fields. Since I've worked very few times with databases, I'm looking for little advice here.
There's a binary(16) field named "IP": I suppose it is used to store user IPs. A typical stored value is, for example, 00000000000000000000ffff3d024463.
Using PHP (or even MySQL, if this is possibile), how can I convert this data to a plain IP address?
have you tried
$data = hex2bin($ip);
var_dump($data);
//not sure how your data was put into the column IP
on your sql query you can also try
$query = mysql_query("SELECT `HEX(IP)` FROM `database`");
$row = mysql_fetch_array($query);
foreach($row['ip'] as $ip)
{
$data = hex2bin($ip);
echo $data."<br />";
}
//code not tested
SQL Query with binary data (PHP and MySQL)
http://php.net/manual/en/function.hex2bin.php
http://php.net/manual/en/function.bin2hex.php
Solved: I had to convert that number using a MySQL function (INET6_NTOA).
I want to save (insert) a uploaded file to a database with PHP, which the type of the database filed is varbinary.
Finally I want to have the content of VarBinary (output) like when file is read in C# and then is stored in byte array and array is inserted to VarBinary.
Also my connection to the database is with sqlsrv.
The type of my files are just PDF and images.
I try this code but my output is different with the output of C#:
$handle=#fopen($_FILES["my_file"]["tmp_name"], 'rb');
$content= file_get_contents($_FILES["my_file"]["tmp_name"]);
$content = unpack("N*",$content);
$content= implode($content);
$sql = "INSERT INTO files (file_data) VALUES (CONVERT(varbinary(MAX)?)";
$params=array();
array_push($params,$content);
$table=sqlsrv_query( $conn, $sql, $params);
"$conn" is the name of my connection that works correctly.
PHP doesn't have a "byte array" data type. What it has is a string type, which is a byte array for all intents and purposes. To read the binary content of a file into a variable which is as close to a byte array as you'll ever get in PHP, do:
$content = file_get_contents($_FILES['my_file']['tmp_name']);
Yup, that's it. Nothing more to do.
I'm not particular familiar with the sqlsrv API, but perusing its documentation it appears that you can (need to?) set a flag this way to flag the data as being binary:
sqlsrv_query($conn, 'INSERT INTO files (file_data) VALUES (?)', array(
array($content, SQLSRV_PARAM_IN, SQLSRV_PHPTYPE_STRING, SQLSRV_SQLTYPE_BINARY)
));
I propose to always convert your binary data to base64.
So it can be stored in database easily and also it can be transfered from somewhere to anywhere with minimum headache!
$handle=#fopen($_FILES["my_file"]["tmp_name"], 'rb');
$elephantContent = file_get_contents($_FILES["my_file"]["tmp_name"]);
$rabbitContent = base64_encode($elephantContent);
//Now ...
$sql = "INSERT INTO files (file_data) VALUES (?)";
sqlsrv_query($conn , $sql , array($rabbitContent) );
file_data field in files table can be varchar, varbinary, blob, text ! :)
Now it can be invoked from database and be packed into a img tag directly.
<img src="data:image/jpeg;base64,PUT `file_data` CONTENTS HERE!" alt="..." />
You can store file type and alt properties in database and put them in tag.
You even can convert it to binary in database (if you insist on seeing binary data in db) (mysql 5.6+)
SELECT FROM_BASE64(`file_data`) as elephant_content from `files` WHERE ...
... And I'm pretty sure that there is equivalent method to do it so in SQL.
For example read this:
https://social.technet.microsoft.com/wiki/contents/articles/36388.transact-sql-convert-varbinary-to-base64-string-and-vice-versa.aspx
I'm having some problems when trying to unserialize an array in PHP.
I have two methods in a class, one to handle storing a serialized array in a mysql table and one to retrieve it.
I get the following error when executing the second method (when retrieving / unserializing):
Notice: unserialize(): Error at offset 50623 of 50647 bytes in /Users/benwaine/NetBeansProjects/SentimentEngineMk2/lib/TSE/Mapper/Filter.php on line 39
The string is stored ina blob field in MySQL. The array is multi-dimentional and composed of a token with an array of floats. The problem is intermittent perhaps suggesting the issue is with the tokens?
// Structure
keywordA - 0.234
- 0.234
KeywordB - 0.23
- 0.47
// Methods
public function insertFilterForKeyword($id, array $filterProbs)
{
$sql = "INSERT INTO filter (keyword_id, filter_probs, sample_size_pos, sample_size_neg)
VALUE (:id, :filterProbs, :pos, :neg)";
$stmt = $this->pdo->prepare($sql);
$meta = array_shift($filterProbs);
$stmt->bindParam(':id', $id);
$stmt->bindParam(':filterProbs',serialize($filterProbs));
$stmt->bindParam(':pos', $meta['sample_size_pos']);
$stmt->bindParam(':neg', $meta['sample_size_neg']);
$result = $stmt->execute();
return $result;
}
public function getFilterByKeyword($keyword)
{
$sql = 'SELECT f.id, f.keyword_id, f.filter_probs, f.sample_size_pos, f.sample_size_neg
FROM filter f
JOIN keyword k ON k.id = f.keyword_id
WHERE k.keyword = :keyword';
$stmt = $this->pdo->prepare($sql);
$stmt->bindParam(':keyword', $keyword);
$result = $stmt->execute();
$data = $stmt->fetch(PDO::FETCH_ASSOC);
$probArray = unserialize($data['filter_probs']);
$filter = new TSE_Filter($probArray);
return $filter;
}
I'm guessing the error has to do with characters in the data that are causing deserialization issues.
After googling I tried methods the deserialization line in the following ways, none of which worked:
1) base64_encode / decode the serialized string.
//to safely serialize
$safe_string_to_store = base64_encode(serialize($multidimensional_array));
//to unserialize...
$array_restored_from_db = unserialize(base64_decode($encoded_serialized_string));
2) Use a preg match to alter potentially difficult characters:
unserialize(preg_replace('/;n;/', ';N;', strtolower($data['filter_probs'])));
Any help would be much appreciated!
Thanks.
I'd've guessed length of the strin in the db being the issue. What happens if you echo out the serialised string once you've pulled it out of the database? Does it look like a finished string or does it break? I think blob is around 65k characters, you could try changing it to a longblob and then resave the data?
Also, if you have one that you know doesn't work, skip the database save and try to unserialise the string straight away. Then atleast you know if the issue is with unserialising, or the data in your db?
In your sample code, you're using urlencode on the serialized string. You're not decoding the urlencoding when you unserialize, which would cause the unserialization to fail.
Perhaps that's the problem?
(You should be able to store the serialized data without urlencoding)
My guess is that it has something to do with column, connection or client charsets.
From http://dev.mysql.com/doc/refman/5.1/en/charset-conversion.html:
If the column has a binary data type (BINARY, VARBINARY, BLOB), all the values that it contains must be encoded using a single character set (the character set you're converting the column to). If you use a binary column to store information in multiple character sets, MySQL has no way to know which values use which character set and cannot convert the data properly.
So I'd try setting the column to CHARACTER SET BINARY explicitly first. Remember that serialized strings contain null bytes and other weird stuff if you serialize objects (I realize you aren't doing that, but who knows what your array contains as keys and values).
Also, you're not flagging the serialized argument as a blob when preparing it, so PDO is probably applying some character conversion as well. For the insert, try using this:
$stmt->bindParam(':filterProbs', serialize($filterProbs), PDO::PARAM_LOB);
And maybe use the same approach when fetching as well, just with PDOStatement::bindColumn() (and remember fetching with PDOStatement::fetch(PDO::FETCH_BOUND) then).
Check database scheme definition for field filter.filter_probs if it is long enough to save serialized string.
Also you can validate the array before serialize and after unserialize containing same value.
Does the comment by getmequick http://php.net/manual/en/function.unserialize.php help? Search for 'error at offset'...
I am using a classified scripts and saves user_meta data in the wp_usermeta table.
The meta_key field is called user_address_info and in it there are all the data like below :
s:204:"a:7:{s:9:"user_add1";s:10:"my address";s:9:"user_add2";N;s:9:"user_city";s:7:"my city";s:10:"user_state";s:8:"my phone";s:12:"user_country";N;s:15:"user_postalcode";s:10:"comp phone";s:10:"user_phone";N;}";
I am not using all the fields on the script but user_add1, user_city, user_state and user_postalcode
I am having trouble to get the data using SQL like the example below (wordpress) :
$mylink = $wpdb->get_row("SELECT * FROM $wpdb->links WHERE link_id = 10", ARRAY_A);
I would like some help here so that I will display anywhere (I dont mind using any kind of SQL queries) the requested info e.g. the user_city of current author ID (e.g. 25)
I was given the following example but I want something dynamic
<?php
$s = 's:204:"a:7:{s:9:"user_add1";s:10:"my address";s:9:"user_add2";N;s:9:"user_city";s:7:"my city";s:10:"user_state";s:8:"my phone";s:12:"user_country";N;s:15:"user_postalcode";s:10:"comp phone";s:10:"user_phone";N;}"';
$u = unserialize($s);
$u2 = unserialize($u);
foreach ($u2 as $key => $value) {
echo "<br />$key == $value";
}
?>
Thank you very much.
No, you can't use SQL to unserialize.
That's why storing serialized data in a database is a very bad idea
And twice as bad is doing serialize twice.
So, you've got nothing but use the code you've given.
I see not much static in it though.
do you experience any certain problem with it?
Or you just want to fix something but don't know what something to fix? Get rid of serialization then
i have found that the serialize value stored to database is converted to some other way format. Since the serialize data store quotes marks, semicolon, culry bracket, the mysql need to be save on its own, So it automatically putting "backslash()" that comes from gpc_magic_quotes (CMIIW). So if you store a serialize data and you wanted to used it, in the interface you should used html_entity_decode() to make sure you have the actual format read by PHP.
here was my sample:
$ser = $data->serialization; // assume it is the serialization data from database
$arr_ser = unserialize(html_entity_decode($ser));
nb : i've try it and it works and be sure avoid this type to be stored in tables (to risky). this way can solve the json format stored in table too.