I am reading imap email with PHP and downloading attachments, using the core imap PHP functions. I am also connecting via the imap protocol (not POP). I would like to be able to know the filesize of attachments before I load them into memory, to reduce server resource usage.
For example I want a limit of 5mb per attachment, this way I can discard any attachments that are over the limit, before they consume any server resources. Finding the size of the email might help but the problem with that is I want the limit per attachment, not per email.
imap_fetchbody() is loading the attachment, which all works. I could probably read the size of the result of that function but that would mean loading the attachment into memory first.
Thanks
This question is just over 3 years old with no real answer, I'm surprised.
Loading the email structure using imap_fetchstructure(); will reveal all the information you'll need for this task. I will assume you know how to obtain the particular email you want to check before using imap_fetchstructure();.
Anyway, the function will return a list of "parts". These parts are of the IMAP4 specification. The results are given in an array, and one of the array indices is called bytes which indicates the number of bytes for that part. There is another index called encoding which indicates the method of encoding for that part.
As Paul answered, Base64 encoding is generally about 33% larger than the original size which should be good enough to determine the original data length of the attachment.
I believe you can avoid the hassle of loading the data to memory by writing the data directly to a file handle by the use of the imap_savebody(); function. You can overcome the encoding issue by adding a stream filter to that file handle before calling imap_savebody();. This is achieved by the stream_filter_append(); function.
I may edit my answer and add some working PHP code when I have more time.
I realize this is an older post, but in case it helps, the difference in size is due to the attachments being base64-encoded, which results in a roughly 33% expansion of the data. From RFC3501: "[Body size is] A number giving the size of the body in octets. Note that this size is the size in its transfer encoding and not the resulting size after any decoding."
Related
Sorry if it is trivial or obvious, but I could not find the answer by googling it.
From where does the size value in $_FILES['name'] array come from? Could you trust the value of it ($_FILES['name']['size']) or should you still check it using the filesize() function?
In other words, is it necessary to check actual file size by filesize function to notice if it is properly uploaded?
If the file is uploaded correctly and everything is fine, you can use the info provided by PHP superglobal $_FILES. Using filesize() adds small overhead since OS needs to inspect the file for the size. It's up to you, but checking PHP source on how it does all this indicates clearly that it correctly calculates the file size in the HTTP multipart request. Basically, you'd be doing the same job again if you were to filesize() the file.
The reason you can trust this directly from superglobal variable is the fact that multipart requests supply a boundary between which the data resides. By definition, it's not possible to obtain corrupt data if the protocol for extracting the data isn't followed. In other words, it means that browser sends a "delimiter" and PHP simply finds it and starts checking the text for data between that delimiter. To do this, it accurately allocates required memory and it can immediately cache the number allocated - and that number is the file size. If anything is wrong along the way, you will get errors. Therefore, if the file uploaded correctly, the information about the size is trusted.
PHP does seem to recalculate the size of the file after it is uploaded. Although the client does send a header specifying the content-length of the file, based on tests (with PHP 5.5) this header is simply ignored and, instead, the length is measured. Personally, I would always use filesize() to get the file size since you can be more confident about which measurement is being used, but it is ultimately up to you. Either way, $_FILES['file_name']['size'] appears to be a safe value to use.
You should rather check if the client-reported $_FILES['file_name']['size'] equals the value given by filesize(). A difference may indicate an error during transmission of the uploaded file.
I have a string to encrypt and then decrypt the encrypted string.
I can do this using the following function of php
string mcrypt_generic ( resource $td , string $data );
Successfully if $data size is 60MB . But I need to encrypt/decrypt file larger then 1GB. For larger database I got **Memory out of range ** error. I have the following configuration,
ini_set('memory_limit', '2048M'); // 2GB
But got same error.
Please suggest how can I do this.
Thanks.
That error message is not emitted by PHP, ext/mcrypt or libmcrypt ... not directly at least, it is likely caused by a PHP bug and not the result of a pre-set limit.
But regardless, such large files should be handled one small chunk at a time. You can read, for example, up to 4kb from the source file, encrypt that small portion of it, append the result to the target file - repeat that in a loop until you've processed all of it.
That is just an example of course, but you should take it as one for how to handle large files in general. If you want to encrypt something, I'd suggest you use an existing, proven tool that does all of it for you, otherwise there are just too many details to handle that you're not even aware of.
Let me first establish what I want to do:
My user is able to record voicenotes on my website, add tags to said notes for indexing as well as a title. When the note is saved I save the path of the note along with the other info in my DB.
Now, I have 2 choices to do the recording, both involve a .swf embedded in my site:
1) I could use Red5 server to stream the audio to my server and save the file and return the path to said file to my app to do the DB saving, seems rather complicated since I would have to convert the audio and move it to the appropriate folder that belongs to the user in a server side Red5 app, which I'm not very aware of how to build.
2) I could simply record the audio and grab its byte array, do a Base64 encoding on it and send it to PHP along with the rest of the data that is necessary (be it by a simple POST or an AJAX call), decode it on the server and make the file with the appropriate extension, audio conversion would also occur here using ffmpeg, this option seems simpler but I do not know how viable it is.
What option would you say is more viable and easier to develop? Thanks in advance
Depending on the planned duration of the recording, you may very well be able to use option number two. I recently used a similar approach successfully for a project, but recordings were only up to 30 seconds or so. Here's what I did differently from what you're suggesting though, and why I think it's better:
To capture the sound from the microphone and store it to a ByteArray, use the SAMPLE_DATA event which is dispatched whenever more sound data comes in from the microphone. There's an example in the documentation that should explain this well enough.
Because most users would be on normal home computers without any special recording equipment, it was safe to assume that the full fidelity of the recording is not necessary. I used just 2 bytes per sample, and only mono, instead of using the full 64 bit floats (AS3 Number) that you get from the microphone on the SAMPLE_DATA event. Simply read the Number and do myFloatSample * 0x7fff to convert to 16 bit signed integer.
Don't use the native 44.1kHz sampling rate if you're just recording speech or something else in that frequency range. You will likely get away just fine with 22.05kHz, which will cut the amount of data in half straight away. Just set the Microphone.rate property accordingly.
Don't use Base64 to encode your data. Send it as binary data, which will be significantly smaller. You can send it as raw POST data, or using something like AMF. Also, before you send it, use the native compress() or deflate() methods on the ByteArray to compress it. On the server, decompress using the ZLIB or raw DEFLATE (inflate) algorithms respectively, which PHP supports.
Once decompressed on the server, what you have is essentially what is called a raw 16-bit mono PCM stream. Incidentally, that should be one of the very input formats that ffmpeg (or lame) supports, so you should be able to encode it to mp3 without having to do any manual decoding first.
Obviously the Red5 solution will likely be better, because it's more tailored for the task. But if you don't have the resources to set up a Red5 server, or don't want to use Java, the above solution is proven to work well as long as you stay away from too long recordings.
To take a simple example, a 30 second recording at 22,050 samples per second, 2 bytes per sample will be ~1.3MB. Even once deflated, the transfer to the server will likely still be almost a megabyte for 30 seconds of audio. This may or may not be acceptable for your application.
I'm building a little script that will connect to an IMAP account and grab the content of the email and also the attachments. It works well for the most part, but when a really large file comes in, it causes the script to time out. Is there any way that I can check the file size before trying to grab it? I think that would be the simplest solution. Otherwise, I may have to upgrade to a server that has more memory.
Check out imap-fetch-overview() and imap-fetchstructure()
They look like they'll give you the size in bytes (depending on the server).
You can use imap_fetch_overview to retrieve information for one or more headers, including the size.
I'm creating something that includes a file upload service of sorts, and I need to store data compressed with zlib's compress() function. I send it across the internet already compressed, but I need to know the uncompressed file size on the remote server. Is there any way I can figure out this information without uncompress()ing the data on the server first, just for efficiency? That's how I'm doing it now, but if there's a shortcut I'd love to take it.
By the way, why is it called uncompress? That sounds pretty terrible to me, I always thought it would be decompress...
I doubt it. I don't believe this is something the underlying zlib libraries provide from memory (although it's been a good 7 or 8 years since I used it, the up-to-date docs don't seem to indicate this feature has been added).
One possibility would be to transfer another file which contained the uncompressed size (e.g., transfer both file.zip and file.zip.size) but that seems fraught with danger, especially if you get the size wrong.
Another alternative is, if the server uncompressing is time-expensive but doesn't have to be done immediately, to do it in a lower-priority background task (like with nice under Linux). But again, there may be drawbacks if the size checker starts running behind (too many uploads coming in).
And I tend to think of decompression in terms of "explosive decompression", not a good term to use :-)
If you're uploading using the raw 'compress' format, then you won't have information on the size of the data that's being uploaded. Pax is correct in this regard.
You can store it as a 4 byte header at the start of the compression buffer - assuming that the file size doesn't exceed 4GB.
some C code as an example:
uint8_t *compressBuffer = calloc(bufsize + sizeof (uLongf), 0);
uLongf compressedSize = bufsize;
*((uLongf *)compressBuffer) = filesize;
compress(compressBuffer + sizeof (uLongf), &compressedSize, sourceBuffer, bufsize);
Then you send the complete compressBuffer of the size compressedSize + sizeof (uLongf). When you receive it on the server side you can use the following code to get the data back:
// data is in compressBuffer, assume you already know compressed size.
uLongf originalSize = *((uLongf *)compressBuffer);
uint8_t *realCompressBuffer = compressBuffer + sizeof (uLongf);
If you don't trust the client to send the correct size then you will need to perform some sort of uncompressed data check on the server size. The suggestion of using uncompress to /dev/null is a reasonable one.
If you're uploading a .zip file, it contains a directory which tells you the size of the file when it's uncompressed. This information is built into the file format, again, though this is subject to malicious clients.
The zlib format doesn't have a field for the original input size, so I doubt you will be able to do that without simulating a decompression of the data. The gzip format has a "input size" (ISIZE) field, that you could use, but maybe you want to avoid changing the compression format or having the clients sending the file size.
But even if you use a different format, if you don't trust the clients you would still need to run a more expensive check to make sure the uncompressed data is the size the client says it is. In this case, what you can do is to make the uncompress-to-/dev/null process less expensive, making sure zlib doesn't write the output data anywhere, as you just want to know the uncompressed size.