I know this question already asked, but I can't solve my problem, so I explained my problem here kindly help me to solve this.
I am getting data form this example URL by using file_get_contens()
$URL1 = 'abcd.com/xxx';
$URL2 = 'abcd.com/yyy';
$URL3 = 'abcd.com/zzz';
$response1 = file_get_contents($URL1);
$response2 = file_get_contents($URL2);
$response3 = file_get_contents($URL3);
And I compressed response data using gzencode because data too long and added prefix for my reference
then i save compressed data to DB
$arrayResponse['URL1'] = '_|_coMpResSed_|_' . gzencode($response1);
$arrayResponse['URL2'] = '_|_coMpResSed_|_' . gzencode($response2);
$arrayResponse['URL3'] = '_|_coMpResSed_|_' . gzencode($response3);
DB details
Storage Engine : InnoDB
Collation : utf8mb4_unicode_ci
Type : longblob or longtext (both i tried)
And I decompress the data by using gzdecode
$temp1 = explode('_|_coMpResSed_|_', $arrayResponse['URL1']);
$temp2 = explode('_|_coMpResSed_|_', $arrayResponse['URL2']);
$temp3 = explode('_|_coMpResSed_|_', $arrayResponse['URL3']);
if (!empty($temp1[1]) && !empty($temp2[1]) && !empty($temp3[1])) {
$arrayResponse['URL1'] = gzdecode($temp1[1]);//working fine
$arrayResponse['URL2'] = gzdecode($temp2[1]);// getting warning
$arrayResponse['URL3'] = gzdecode($temp3[1]);//working fine
}
And I am getting `Warning:
gzdecode(): data erroron line$arrayResponse['URL2'] = gzdecode($temp2[1]);`
Other lines are working fine . I dont know where I am making mistakes. Can any one help me to get this?
Having same problem, I just had a look at Mysql doc:
https://dev.mysql.com/doc/refman/5.5/en/encryption-functions.html
Many encryption and compression functions return strings for which the result might contain arbitrary byte values. If you want to store these results, use a column with a VARBINARY or BLOB binary string data type. This will avoid potential problems with trailing space removal or character set conversion that would change data values, such as may occur if you use a nonbinary string data type (CHAR, VARCHAR, TEXT).
I just change the column data type to VARBINARY and everything's OK.
Related
The title sounds confusing, but I don't know how to describe it better.
I've been given the task to do a database migration from program A to program B.
Program A uses a MSSQL database and stores all its files in the database.
Program B handles files with stored in the "normal" way, meaning in the file system.
Now I have to extract, transform and download the database stored files to file system files with PHP, but I was unsuccessful in my attempts to convert them.
For testing purposes, I created a simple .txt file with the content Test document for migration and program A stores it like this in the database:
0x5465737420646F63756D656E7420666F72206D6967726174696F6E'
What format is that, and how do I convert it to a normal document.txt file ?
Edit
Lot of thanks to #PanagiotisKanavos. This now does work with a stream:
$query = "select top(1) DESCRIPTION, FILETYPE, DOCUMENT from dbo.Documents;";
$stmt = sqlsrv_query($this->sqlsrv_conn, $query);
if (sqlsrv_fetch($stmt)) {
$document = sqlsrv_get_field($stmt, 2, SQLSRV_PHPTYPE_STREAM(SQLSRV_ENC_BINARY));
$fileName = sqlsrv_get_field($stmt, 0, SQLSRV_PHPTYPE_STRING(SQLSRV_ENC_CHAR));
$ext = sqlsrv_get_field($stmt, 1, SQLSRV_PHPTYPE_STRING(SQLSRV_ENC_CHAR));
file_put_contents(
$fileName . '.' . $ext,
stream_get_contents($document),
);
}
Now what is the most efficient way to do this with ALL the files? Do I have to execute a query for each and every row?
With PDO I could use $stmt->fetchAll(FETCH_ASSOC) which gave me a nice array of assoc arrays with data inside.
sqlsrv has a similar function sqlsrv_fetch_array which is explained php.net and [docs.microsoft] with the following example:
while( $row = sqlsrv_fetch_array( $stmt, SQLSRV_FETCH_ASSOC) ) {
echo $row['LastName'].", ".$row['FirstName']."<br />";
}
But as much as I could search, I couldn't find a way to loop over the result set without fetching it and then fetching each row individually with the stream and string type mixed. sqlsrv_fetch_array accepts only SQLSRV_FETCH_NUMERIC, SQLSRV_FETCH_ASSOC and SQLSRV_FETCH_BOTH and then the result set is already fetched, and I can't use sqlsrv_get_field to set the type of each field.
I can't be the first person ever that needs something like this, but I'm unable to find anything about it. Probably I'm searching wrongly, or I misunderstood a concept.
I found the solution!
I tested multiple online tool to decode strings and find out what the encoding was.
Even tried hex2bin() but all the tools told me that it wasn't a valid hex string.
Until I stumbled upon this godsent tool which translated the invalid hex to a ? but translated the rest of it which resulted in:
?Test document for migration
From then on it was easy to deduct that 0x was the troublemaker. After removing it, the conversion works like a charm and I could "convert" even complexer files like .doc. Here is the code:
file_put_contents(
// 'DESCRIPTION' is the file name
'files/' . $dbDocument['DESCRIPTION'] .
// 'FILETYPE' is the extension
'.' . mb_strtolower($dbDocument['FILETYPE']),
// 'DOCUMENT' is the document content hex encoded with prepended '0x'
hex2bin(str_replace('0x', '', $dbDocument['DOCUMENT']))
);
This isn't a format, it's how some database administration tools, eg SSMS, display binary data. The data is already binary and doesn't need to be converted.
Reading Large Objects (LOBs) as if they were numbers or strings is really slow though, caching the entire document in both server and client memory, even though the object won't be reused, and doesn't even need to be held in memory. After all, a BLOB in SQL Server can be 2GB and more. That's why almost all databases and data access libraries allow handling LOBs as file streams.
Microsoft's PHP doc examples show how to read LOBs as file streams both for PDO and SQLSRV.
Copying from the example, this parameterized query will search for a user's picture:
/* Get the product picture for a given product ID. */
$tsql = "SELECT LargePhoto
FROM Production.ProductPhoto AS p
JOIN Production.ProductProductPhoto AS q
ON p.ProductPhotoID = q.ProductPhotoID
WHERE ProductID = ?";
$params = array(&$_REQUEST['productId']);
/* Execute the query. */
$stmt = sqlsrv_query($conn, $tsql, $params);
Instead of reading the entire picture as a single value though, it's loaded as a file stream :
$getAsType = SQLSRV_PHPTYPE_STREAM(SQLSRV_ENC_BINARY);
if ( sqlsrv_fetch( $stmt ) )
{
$image = sqlsrv_get_field( $stmt, 0, $getAsType);
fpassthru($image);
}
else
{
echo "Error in retrieving data.</br>";
die(print_r( sqlsrv_errors(), true));
}
$getAsType = SQLSRV_PHPTYPE_STREAM(SQLSRV_ENC_BINARY); specifies that the data will be retrieved as a stream.
$image = sqlsrv_get_field( $stmt, 0, $getAsType); retrieves the 1st field using the specified type, in this case a stream. This doesn't load the actual contents.
fpassthru copies the stream contents directly to the output. The picture may be 2GB but it will never be held in the web server's memory.
I would like to start off by saying I am completely new to PHP CRUD API's.
I have ran into a problem where a certain MySQL table value "binds", which is stored as JSON in the database, are being outputted in Base64.
I can't seem to figure out why it is being outputted as Base64, here's the code that I use for the API
I apologize if this doesn't provide enough information, I am genuinely confused about what's causing this issue. Please comment if you guys need any more information, thanks.
This is the code from PHP-CRUD-API and I'm the author. If your column is identified to have a binary type then the output is base64 encoded. Whether or not it is of a binary type is determined by the code snippet:
return (($field->flags & 128) && (($field->type>=249 && $field->type<=252) ||
($field->type>=253 && $field->type<=254 && $field->charsetnr==63)));
from the isBinaryType($field) function, where:
128 = BINARY_FLAG
249 = TINY_BLOB
250 = MEDIUM_BLOB
251 = LONG_BLOB
252 = BLOB
253 = VAR_STRING
254 = STRING
63 = BINARY_CHARSET
These flags and types are determined by column type and collation. To resolve your issue you probably need to change the collation of the column from "utf8_bin" to "utf8_general_ci".
I am having an issue converting an image stored as base64 in a PostgreSQL database into an image to display on a website. The data type is bytea and I need to get the data via cURL.
I am working with an API to connect to a client's stock system which returns XML data.
I know storing images this way in a DB is not a great idea but that's how the client's system works and it can't be changed as it is a part of an enterprise solution provided by a 3rd Party.
I'm using the following to query the DB for the PICTURE field from the PICTURE table where the PART = 01000015
$ch = curl_init();
$server = 'xxxxxx';
$select = 'PICTURE';
$from = 'picture';
$where = 'part';
$answer = '01000015';
$myquery = "SELECT+".$select."+FROM+".$from.'+WHERE+'.$where."+=+'".$answer."'";
//Define curl options in an array
$options = array(CURLOPT_URL => "http://xx.xxx.xx.xx/GetSql?datasource=$server&query=$myquery+limit+1",
CURLOPT_PORT => "82",
CURLOPT_HEADER => "Content-Type:application/xml",
CURLOPT_RETURNTRANSFER => TRUE
);
//Set options against curl object
curl_setopt_array($ch, $options);
//Assign execution of curl object to a variable
$data = curl_exec($ch);
//Close curl object
curl_close($ch);
//Pass results to the SimpleXMLElement function
$xml = new SimpleXMLElement($data);
//Return String
echo $xml->row->picture;
The response I get from this is: System.Byte[]
Thus if I use base64_decode() in PHP I am obviously just decoding the string "System.Byte[]".
I am guessing that I need to use the DECODE() function in PostgreSQL to convert the data in the query? However, I've tried loads of combinations but I'm stuck. I've had a few downvotes for questions and I'm not too sure why so if this is a bad question I'm sorry, I just really need some help with this one.
(nb:I've replaced the IP and $server with xxxxx for security)
To explain further:
The client has a POS system which is based on ASP.NET and saves the data as XML files on the remote server. I have access to this data via an API which includes a SQL query function using HTTP/cURL defined as follows:
http://remoteserver:82/pos.asmx.GetSql?datasource=DATASOURCE&query=MYQUERY
So to get the field that contains the picture data I am currently usingthe above code.
The query is in the CURL URL i.e. http://remoteserver:82/pos.asmx.GetSql?datasource=12345&query=SELECT+*+FROM+picture+WHERE+part+=+'01000015'";
However, this returns System.Byte[] instead of encoded data which I can then decode in PHP.
Additional info:
PostgreSQL version: PostgreSQL 9.1.3 on i686-pc-linux-gnu, compiled by gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-51), 32-bit
Table Schema:
Available here: http://i.stack.imgur.com/sc8Gw.png
You should preferably have the server storing the data in PostgreSQL as a bytea field, then encoding to base64 to send to the client, but it sounds like you don't control the server.
The string System.Byte[] suggests it's an app using .NET, like ASP.NET or similar, and it's not correctly handling a bytea array. Instead of formatting it as base64 for output it's embedding the type name in the output.
You can't fix that on the client side, because the server is sending the wrong data.
You'll need to show the server-side tables and queries.
Update after query amended:
You're storing a bytea and returning it directly. The client doesn't seem to understand byte arrays and tries to output it naïvely, probably something like casting it to a string. Since the documentation says it expects "base64" you should probably provide that, instead of a byte array.
PostgreSQL has a handy function to base64-encode bytea data: encode.
Try:
SELECT
account, company, date_amended,
depot, keyfield, part,
encode(picture, 'base64') AS picture,
picture_size, source
FROM picture
WHERE part = '01000015'
The formating isn't significant, it just makes it easier to read here
I'm able to upload an image file and using PHP's file_get_contents put the binary into memory. From there I've attempted to use a basic update or insert query statement to then put that data into a blob. It keeps throwing errors.
In addition I've tried to use the addslashes PHP function as well, and it still doesn't work.
I've tried:
if (is_uploaded_file($_FILES['myFile']['tmp_name'])){
$fileData = file_get_contents($_FILES['myFile']['tmp_name']);
$fileData = unpack("H*hex",$fileData);
$content = "0x" . $fileData['hex'];
}
Then I try to insert into my database and it doesn't work either.
The datatype for my BLOB field is image.
The error I am most often getting is:
mssql_query(): message: Operand type clash: text is incompatible with image (severity 16)
Any help would be appreciated!
I'm creating a C# to PHP Data Connector to allow for a standardized connection to a web server to host data from a database to a C# WinForm application. Everything is working with this one small exception.
The basic of use is this.
C# sends an AES encrypted command to the server. The server parses the command and performs the SQL query and returns an AES encrypted string. This string is then converted to a DataTable in C#.
When the SQL contains a column that is a BLOB I'm only getting back a small part of the full data. It seems that the field is being limited to only the first 2792 bytes.
Is there a setting that is preventing the full contents of the BLOB to be returned?
I'm not sure if it will be helpful, but here is the code that does the work.
$DataConnection = new PDO('mysql:host=10.10.100.102;dbname=jmadata', "root", "nbtis01");
$DataConnection->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
if (isset($Parameters['SQLQuery'])) { // Default List
$SQLQuery = $Parameters['SQLQuery'];
unset($Parameters['SQLQuery']);
}
if (isset($Parameters['LimitOverride'])) {
if (!strpos(strtoupper($SQLQuery), "LIMIT"))
$SQLQuery = rtrim($SQLQuery, ';') . " LIMIT " . $Parameters['LimitOverride'];
unset($Parameters['LimitOverride']);
}
$QueryParams = array();
foreach ($Parameters as $key => $value)
if ($key !== '')
$QueryParams[$key] = $value;
$Query = $DataConnection->prepare($SQLQuery);
$Query->execute($QueryParams);
$ReturnArray = $Query->fetchAll(PDO::FETCH_ASSOC);
if (!$ReturnArray)
$ReturnArray[0] = array("NoResults" => "");
EDIT -- ANSWER
I found my issue. The problem had nothing to do with PDO, PHP or MySQL. I was taking the BLOB data and doing a Base64 to it before putting it in the array, as the split characters I was using to build the result string that would be converted to datatable in c# used non-printable characters and the binary data as a string might have included these characters. The issue was when I was doing a convert in c# to get the original string so that could convert that to a byte array. I was using System.Text.Encoding.ASCII.GetString to convert the Base64 byte array to the original string. This was working on everything but the binary data from the BLOB fields.
The suggestion that it might be a terminating character is what made me find it. Once the Base64 was converted to string using ASCII there was something that was turning into a terminator and it was stopping the convert at that point. Once I found this I changed to System.Text.Encoding.Default.GetString and now it works perfect.
Posted the answer in case anyone else might be trying to do this and having this same issue.
More details in the Edit of the question.
Changed from System.Text.Encoding.ASCII.GetString to System.Text.Encoding.Default.GetString and the issue was resolved.
Thank you crush for pointing me in the right direction to find the solution.