I'm trying since several hours to implement the behaviour of PHP gzinflate() in C. In PHP it's just:
gzinflate($str);
In Python it's:
import zlib
...
return zlib.decompress( decoded_data , -15)
...
But I just don't manage to implement it in C. Can anybody help me with that? I'm really stuck.. I tried to do something with Zlib but it didn't work.. Anybody got a point?
Thanks in advance,
nemo
This zlib usage example is very thorough.
Note that you are closer to the bare metal here than in Python or PHP, so the usage isn't as simple.
Addendum:
The PHP gzinflate and gzdeflate functions perform input and output raw DEFLATE format. The zlib functions, on the other hand, work by default with zlib streams, which are the same with the addition of a 2 byte header and a 4 byte trailer.
You can either switch to using the PHP gzcompress and gzuncompress functions, which produce ZLIB format, or (if you have a recent version of zlib) use the deflateInit2 function instead of deflateInit and specify a negative value for windowBits, which requests raw DEFLATE format.
Related
I have a need to convert source data that I can't control into normalization form C. I am currently doing it, but by calling an external program (uconv). This is what my code snippet looks like:
$malayalam_books = preg_split("/\n/", shell_exec("uconv -f utf8 -t utf8 -x nfc book-names.txt"));
It works well, but obviously making calls to the system is not recommended. I know that PHP supports the ICU libraries, but it's so convoluted how to do a simple thing like this...
I've since discovered that the normalizer_normalize PECL function can handle this natively in PHP 5 >= 5.3.0.
How do I correctly compress a string, so PHP would be able to decompress?
I tried this:
public static byte[] compress(String string) throws IOException {
ByteArrayOutputStream os = new ByteArrayOutputStream(string.length());
DeflaterOutputStream gos = new DeflaterOutputStream(os);
// ALSO TRIED GZOutputStream, same results!
gos.write(string.getBytes());
gos.close();
byte[] compressed = os.toByteArray();
os.close();
return compressed;
}
But PHP does not recognize output as valid GZip compressed string...
The problem seems to be in some headers / footers being added by Android...
For example when I compress something word via PHP with gzcompress I got similar results as with Android, but not similar enough, so PHP could read it:
something (HEX DUMP):
Android: 1f8b08000000000000002bcecf4d2dc9c8cc4b0700fb31da0909000000
PHP: 789c2bcecf4d2dc9c8cc4b0700134703cf
The weirdest thing is that by changing GZOutputStream to DeflaterOutputStream it fixed the problem with something word, but the problem still appears with longer strings...
PS. Removing heading 10 characters from Android generated data does not help at all.
EDIT: I tried to decompress it in PHP with:
gzdecode() - this function does not exist in standard Debian PHP5
version.
gzdecompress() - does not work
And some functions to emulate gzdecode() from PHP site comments that don't really do much.
All above, with removing first 10 bytes and leaving them.
PS2. I tried every single solution from Stack Overflow, and other sources, and still nothing. It is not a duplicate.
EDIT2 (BINARY DUMP): Sample data generated with Android that can't be decomprssed by gzuncompress() or pseudo-gzdecode() functions from PHP.NET: data.compressed.
It supposed to be some JSON, after decompression.
The Android data that starts with 1f8b is a gzip stream. In php you use gzdecode() for that. gzencode() on php makes gzip streams.
The php data that starts with 789c is a zlib stream. You used gzcompress() to make that, and you would use gzuncompress() to decode it.
The compressed data contained within both of those streams, starting with 2bce is raw deflate data. You can use gzinflate() to decode that if you happened to make it somewhere, and you can use gzdeflate() to generate raw deflate.
Just to rant, gzencode(), gzcompress(), and gzdeflate() are some of the most misleading function names ever concocted, since only one of them is related to gzip yet all start with gz, and nothing in the name gzcompress() indicates zlib.
Update:
The "EDIT2" data is, for some reason, doubly compressed. It was compressed first to the zlib format, and then that zlib stream was compressed to the gzip format. (Though gzip couldn't compress the already compressed data, so it's a little bigger.)
You should repair the problem that made it doubly compressed. Or if you have no control over that, you can doubly decompress it, first stripping the gzip header using the RFC 1952 specification and then gzinflate() on the raw deflate data, and then using gzdecompress() on the result.
i faced PHP: Call to undefined function gzdecode() error
few days back so i have been using this in php
function gzdecode($data)
{
return gzinflate(substr($data,10,-8));
}
ref: PHP: Call to undefined function gzdecode()
i found this old code snippet while looking in old php files , so can i use above code instead of this one ?
function gzdecode($data) {
$g=tempnam('/tmp', 'php-gz');
#file_put_contents($g, $data);
ob_start();
readgzfile($g);
$d=ob_get_clean();
unlink($g);
return $d;
}
If you have access to your php.ini, simply enable the zlib extension to use gzencode. Each compression method has some slight differences.
Please read an abstract of this answer https://stackoverflow.com/a/621987/382536 by #thomasrutter
All of these are usable. I would use gzencode() because it is self-contained; its output is the same as if you had used the gzip command line tool.
There is not really much difference between the three.
gzencode() uses the fully self-contained gzip format, same as the
gzip command line tool
gzcompress() uses the raw ZLIB format. It is
similar to gzencode but with less header data, etc. I think it was
intended for streaming.
gzdeflate() uses the raw DEFLATE algorithm on
its own, which is the basis for both the other formats.
So I've got some data that's been compressed with PHP's gzcompress method:
http://us2.php.net/manual/en/function.gzcompress.php
How can I decode this data from node.js??
I've tried "compress", "zlib" and several other node compression libraries, but none of them seem to reckognize the data. For example, zlib just gives me "Error: incorrect header check"
Answer: Turns out that "zlib" is the way to go. We had an additional issue with binary data from memcache. If you have binary data in a node.js Buffer object and you call toString() instead of .toString('binary'), it get's all kinds of scrambled as stuff is escaped or escape sequences are interpreted or whatever. Unfortunately, all the memcache plugins I've tried to date assume string data from memcache, and are not disciplined about handling it properly.
Best ZLIB module I've found:
https://github.com/kkaefer/node-zlib
// first run "npm install zlib", then...
var zlib = require('zlib');
var gz = zlib.deflate(new Buffer("Hello World", 'binary')); // also another 'Buffer'
console.log(zlib.inflate(gz).toString('binary'));
FYI, this question is VERY similar to a related question about Java:
PHP's gzuncompress function in Java?
Stealing from another post (Which compression method to use in PHP?)
gzencode() uses the fully self-contained gzip format, same as the gzip command line tool
gzcompress() uses the raw ZLIB format. It is similar to gzencode but has different header data, etc. I think it was intended for streaming.
gzdeflate() uses the raw DEFLATE algorithm on its own, which is the basis for both the other formats.
Thus, "zlib" would be the correct choice. This is NOT cross-compatible with gzip.
Try https://github.com/kkaefer/node-zlib
php:
<?php
$data = 'HelloWorld';
$gzcompress = gzcompress($data);
$gzcompress_base64_encode = base64_encode($gzcompress);
echo "Compressing: {$data}\n";
echo $gzcompress."\n";
echo $gzcompress_base64_encode."\n";
echo "--- inverse ---\n";
echo gzuncompress(base64_decode($gzcompress_base64_encode))."\n";
nodejs:
const zlib = require('zlib');
var data = 'HelloWorld';
var z1 = zlib.deflateSync( Buffer.from(data));
var gzcompress = z1.toString();
var gzcompress_base64_encode = z1.toString('base64');
console.log('Compressing '+data);
console.log(gzcompress);
console.log(gzcompress_base64_encode);
console.log('--- inverse ---');
console.log(zlib.inflateSync(Buffer.from(gzcompress_base64_encode,'base64')).toString());
I want to compress a string in PHP and write it to a file without using the gzwrite function as I want to store the actual compressed string in a database first, but I am unsure as whether to use gzcompress, gzencode or gzdeflate as it's not very clear.
Any ideas?
Edit: the already compressed string will be written into a *.gz file from the database so it has to be compatible.
Use gzcompress if you just want to compress the string.
gzencode will also add gzip file headers so it can be uncompressed directly by gzip and similar tools.
gzdeflate uses the deflate algorithm which is very similar to the first one.
I think yo want to use gzencode in ths case since the data is going to be stored as a file.