I have an automated PHP script which connects to an email box, reads emails and process them to create tickets. Some of these emails contain various types of file attachments. My script uses following code to save files directly to a postgress database. I'm using codeigniter.
public function saveFiles($filename, $fileurl, $jobid) {
$filedata = array();
$filedata['filename']= $filename;
$filedata['filedescription'] = 'Incoming attachment.';
$filedata['fileargid'] = $jobid;
$filedata['fileaddedon'] = date('Y-m-d H:i:s P');
$filedata['filedata'] = pg_escape_bytea(base64_encode(file_get_contents($fileurl)));
$results = $this->db->insert('file', $filedata);
if ($results)
return $this->db->insert_id();
else
return FALSE;
}
However, most of the files are saved without any issue. My problem is some pdf files get corrupted when I deployed this script. Script saves files to local disk before encoding to base64. All these files are healthy as well. I suspect something is happening during pg_escape_bytea(base64_encode(file_get_contents($fileurl))).
I developed this script using php 5.5.9/Ubuntu on my local PC and non of the files get corrupted there. But the script is deployed on a Ubuntu server with php 5.3.10 and files get corrupted there.
I tried to find out what is causing this but no lock so far. Is this because of different php versions?
Looks either:
you're encoding to database in the "escape" format and reading from it in hex-format.
You need to cast "when the client and backend character encoding does not match, and there may be multi-byte stream error. User must then cast to bytea to avoid this error." From pg_escape_bytea documentation, it also counts to unscape.
Check section 8.1 here
If isn't a problem I'd save bin2hex output to the field directly.
Related
I'm trying to send a tiff file from a controller in Kohana framework version 3.2 (I know it's a bit old) using the response->send_file() method, file is downloaded in browser and the size is ok. but when I try to view it I get an error showing the file is damaged. I download the same file using ssh and I can view it without any problems. when I compare the files in notepad++ encoding for the working file is ANSI but for the damaged one is utf-8-bom. this is the code for the method in my controller:
public function action_file() {
$this->auto_render = false;
$path = '/tmp/test.tiff'
$this->response->send_file($path);
}
I read the Kohana send_file source code and I see it is using:
echo fread(...)
to send the file to client.
How can I change the encoding on output buffer so the file be in ANSI (Windows-1252) format?
I tried
mb_http_output('Windows-1252');
mb_internal_encoding('Windows-1252');
with no suuccess
I added ob_clean() before sending file and problem is solved. It seems somewhere in my codes or the framework BOM was added to output buffer and the file was corrupted
public function action_file() {
$this->auto_render = false;
$path = '/tmp/test.tiff';
ob_clean();
$this->response->send_file($path);
}
thanks zerkms for mentioning comparing files in a hex editor
I've written a REST interface for my ownCloud app. I've method getFileFromRemote($path) which should return a JSON object with the file content.
Unfortunately this only works when the file that I've specified in $path is a plaintext file. When I try to call the method for an image or PDF the status code is 200 but the response is empty. For returning the file contents I use file_get_contents for retrieving the content.
Note: I know ownCloud has a WebDAV interface, but I want to solve this with REST only.
EDIT
This is the code server side (ownCloud):
public function synchroniseDown($path)
{
$this->_syncService->download(array($path));//get latest version
$content = file_get_contents($this->_homeFolder.urldecode($path));
return new DataResponse(['path'=>$path, 'fileContent'=>$content]);
}
The first line retrieves downloades the content on the ownCloud server and works completely.
You probably have to base64_encode your file content to make json_encode/decode handle it properly:
return new DataResponse([
'path'=>$path,
'fileContent' => base64_encode($content) // convert binary data to alphanum
]);
and then when receiving file via second side, you will have to always:
$fileContent = base64_decode($response['fileContent']);
It's only one, but ones of easiest way to handle that. Btw, sooner or later you will find that Mime-Type would be useful in response.
The problem is when I receive email attachments images/binary data becomes corrupted however when I upload files to the same database table they aren't corrupted.
On the left side is the normal image that works (when renamed to PHP and it's binary data is copied). The right side is the image after it's been corrupted. I figured I'd add the image because someone may see it and just know what is going on here. PHP code below...
So I have other files including images stored in the database and they work just fine...
$data = fread($fp, filesize($tmpName));
$data = addslashes($data);
//INSERT INTO table (data), ($data);//etc
However the error is occurring in the PHP code that I use to clean up emails and then store them in the database along with attached files. I know that the attached files are being targeted in PHP correctly so it's simply HOW I'm storing it which seems to be the problem. The data column in MySQL is longblob and uploading images before sending them works fine but when I RECEIVE emails this code that handles putting received attachments is what breaks (using the same database table).
$data = mysqli_real_escape_string($connection,base64_decode($email['attachments'][$t]['attachment']));
$data = mysqli_real_escape_string($connection,$email['attachments'][$t]['attachment']);
$data = base64_decode($email['attachments'][$t]['attachment']);
$data = mysql_real_escape_string(base64_decode($email['attachments'][$t]['attachment']));
$data = addslashes($email['attachments'][$t]['attachment']);
I have a DB system built in PHP/MySql. I'm fairly new at this. The system allows the user to upload an invoice. Others give permission to pay the invoice. The accounting person uploads the check. After check is uploaded, it generates a PDF as a cover, then uses PDFTK (using Ben Squire's PDFTK-PHP-Library) to combine all of the files together and present the user with a single PDF to download.
Some users upload PDF files which cause PDFTK to hang indefinitely when it tries to combine the PDF with others (but most of the time it works fine). No returned error, just hangs. In order to get back onto the sytem, user must clear cache and re-log in. There are no error messages logged by the server, it just freezes. The only difference I can find in the files that do or do not work in looking at them with Acrobat is that the bad files are legal sized (8.5 x 14) ... but if I create my own legal sized file and try that, it works fine.
Using Putty I've gone to command line and replicated the same problem, PDFTK can't read the file, it hangs on the command line as well. I tried using PDFMerge which uses FPDF to combine the files and get an error with the file as well (The error I get back from this is: FPDF error: Unable to find object (4, 0) at expected location). On the command line I was able to use ImageMagick to convert PDF to JPG, but it gives me an error: "Warning: File has an invalid xref entry: 2. Rebuilding xref table." and then it converts it to a jpg but gives a few other less helpful warnings.
If I could get PHP to check the PDF file to determine if is valid without hanging the system, I could use ImageMagick to convert the file and then convert it back to a PDF, but I don't want to do this to all files. How can I get it to check the validity of the file when uploaded to see if it needs to be converted without causing the system to hang?
Here is a link to a file that is causing problems: http://www.cssc-testing.org/accounting/school_9/20130604-a1atransportation-1.pdf
Thanks in advance for any guidance you can offer!
My Code (which I'm guessing is not very clean, as I'm new):
$pdftk = new pdftk();
if($create_cover) { $pdftk->setInputFile(array("filename" => $cover_page['server'])); }
// Load a list of attachments
$sql = "SELECT * FROM actg_attachments WHERE trans_id = {$trans_id}";
$attachments = Attachment::find_by_sql($sql);
foreach($attachments as $attachment) {
// Check if the file exists from the attachments
$attachment->set_variables();
$file = $attachment->abs_path . DS . $attachment->filename;
if(file_exists($file)){
// Use the pdftk tool to attach the documents to this PDF
$pdftk->setInputFile(array("filename" => $file));
}
}
$pdftk->setOutputFile($save_file);
$pdftk->_renderPdf();
the $pdftk class it is calling is from: https://github.com/bensquire/php-pdtfk-toolkit
You could possibly use Ghostscript using exec() to check the file.
The non-accepted answer here may help:
How can you find a problem with a programmatically generated PDF?
I wont say this is an appropriate/best fix, but it may resolve your problem,
In: pdf_parser.php, comment out the line:
$this->error("Unable to find object ({$obj_spec[1]}, {$obj_spec[2]}) at expected location");
It should be near line 544.
You'll also likely need to replace:
if (!is_array($kids))
$this->error('Cannot find /Kids in current /Page-Dictionary');
with:
if (!is_array($kids)){
// $this->error('Cannot find /Kids in current /Page-Dictionary');
return;
}
in the fpdi_pdf_parser.php file
Hope that helps. It worked for me.
I've got a webservice which expects a parameter of type "xs:base64Binary" - this is a file to store in the database.
I'm trying to consume the service using PHP 5's native webservice classes. I've tried a few things:
// Get the posted file
$file = file_get_contents($_FILES['Filedata']['tmp_name']);
// Add the file, encoding it as a base64
$parameters = array("fileBytes" => base64_encode($file));
// Call the webservice
$response = $client->attachFile($parameters);
The result is an error saying "Bad Request." If the file is a text file and I don't base64_encode, it works fine. Problem results when posting a binary file such as an image.
Anyone know the trick here?
EDIT 1
Also problematic is if I encode the text file, it seems to work but of course it's encoded and ends up being junk once downloaded and viewed again (i.e, the text is encoded and doesn't seem to get de-coded by the server).
As far as I know, base64_encode() should be doing the job.
Are you 100% sure $file contains something? Have you made a dump?
Ok, so it seems there is no need to use base64_encode. The file_get_contents already puts it into the required format.
Additionally, the problem was because I had the server side config setting for the maxArrayLength too low.