In my app, I need to generate rather large zip archives, and to store them encrypted for security reasons.
I have found a way to do so using streams so as to reduce the amount of RAM required and so far this is working rather well. I've piped ZipStream's output to an OpenSSL process call and that gives me the result I want, using the following code:
$cmd = "openssl enc -e -aes-256-cbc -K $myKey -iv $myIv -out $myFile";
$pipesStructure = [0 => array("pipe", "r"), 1 => array("pipe", "w"), 2 => array("pipe", "w")];
$process = proc_open($cmd, $pipesStructure, $pipes);
if (is_resource($process)) {
$zipOptions = new Archive();
$zipOptions->setOutputStream($pipes[0]);
$zip = new ZipStream('archive.zip', $zipOptions);
foreach($myFiles as $f) {
$zip->addFile($f->name, $f->content);
}
$zip->finish();
fclose($pipes[0]);
fclose($pipes[1]);
fclose($pipes[2]);
$return_value = proc_close($process);
}
Now, I would like, while I do all that, to compute the hash of the unencrypted zip archive. The idea is to insert an additional "processing block" through which the stream would pass:
[ ZipStream ] -----> [ Hash computing ] -----> [ Encryption ] ------> Final output
But so far I'm unable to understand well enough how streams work and I'm failing to achieve that...
I'm pretty sure I need to use hash_init, hash_update_stream, and hash_final, but I can't seem to be able to "feed" the data to hash_update_stream.
I've initialized a hash context and added the following code before the loop that adds all files to the zip, hoping hash_update_stream would merely read what goes through $pipes[0], but apparently that's not the way to do it
...
hash_update_stream($hashContext, $pipes[0]);
...
Upon "finalizing" the hash, the output is the hash of an empty string ('')
Any pointers as to how to proceed?
Related
I use the following code to open a process with proc_open, and to save the handle and the pipes to a file:
$command = "COMMAND_TO_EXECUTE";
$descriptors = array(
0 => array("pipe", "r"), // stdin is a pipe that the child will read from
1 => array("pipe", "w"), // stdout is a pipe that the child will write to
2 => array("file", "error-output.txt", "a") // stderr is a file to write to
);
$pipes = array();
$processHandle = proc_open($command, $descriptors, $pipes);
if (is_resource($processHandle)) {
$processToSave = array(
"process" => $processHandle,
"pipes" => $pipes
);
file_put_contents("myfile.bin", serialize($processToSave) );
}
And in a second moment i need to retrieve this file handle from the file, i've used this code:
$processArray = unserialize(file_get_contents("myfile.bin"));
$processHandle = $processArray["process"];
$pipes = $processArray["pipes"];
But when I print a var_dump of $processHandle and $pipes after retrieving from file, I'll get integers instead of resource or process, but why??
var_dump($processHandle) -> int(0)
var_dump($pipes) - > array(2) { int(0), int(0) }
And at this point of course, if I try to close the pipes, i will get an error, resource expected, integer given.
How can I make this working? (NOTE: This is the solution I'm looking for)
But alternatively, I can get also the pid of process and then use this pid to stop or kill or do anything else with the process, but what about the pipes?
How can I read/write or save error from/to the process?
Thank you
Found the solution to myself, it's not to possible to serialize resource and when the script has done, those resource handler were free.
Solution was to create a daemon listening on a port, wich on request launch and stop process. Because the process is always running, it can maintain a list of handler process and stop when requested.
To my PHPeoples,
This is sort of a weird PHPuzzle, but I'm wondering if it's PHPossible.
The end goal is the equivalent functionality of
mysql mydb < file.sql
But with an API like this
./restore < file.sql
Where restore is a PHP script like this
#!/usr/bin/env php
$cmd = "msyql mydb";
passthru($cmd, $status);
However, I want to pass STDIN to the passthru command.
The clear benefit here is that I can put restore somewhere in a pipeline and everything works peachy. Here's an example
# would be pretty awesome!
ssh $remote "msyqldump $config mydb | gzip" | gzip -dc | ./restore
Anyway, I doubt it's possible using passthru, but perhaps with proc_open in some way?
As a last case resort, in the event of an unsolvable PHPredicament, I would do something like this
./restore file.sql
With script like this
#!/usr/bin/env php
$cmd = sprintf("mysql mydb < %s", $argv[1]);
passthru($cmd, $status);
PHPwnd
./restore
#!/usr/bin/env php
<?php
// set descriptors
$desc = array(
0 => array("file", "php://stdin", "r"),
1 => array("pipe", "w"),
2 => array("pipe", "w")
);
// open child proc
$proc = proc_open("mysql -uUSER -pPASS mydb", $desc, $pipes);
if (is_resource($proc)) {
// display child proc stdout
echo stream_get_contents($pipes[1]);
// close file streams
array_map('fclose', $pipes);
// get return value
$ret = proc_close($proc);
// display return value
printf("command returned %s\n", $ret);
}
Example use
cat file.sql.gz | gzip -dc | ./restore
What! PHProfound!
Turns out passthru already does this! PHenomenalP!
Check it out, PHPals:
gzip.php
#!/usr/bin/env php
<?php
passthru("gzip", $status);
if ($status !== 0) {
error_log("gzip exited with status %d", $status);
}
How PHProgressivist!
echo "hello" | php gzip.php | gzip -dc
Output
hello
I'm using proc_open with pdftk to pre-fill some forms with data, this works pretty well, but rather than save the result to a file and then read the file back, I'd like to print the file directly out. I've got this working so I'm not having any problems. But I'd like to direct the output of proc_open directly to the stream returned to the user so that I don't have to hold the value in memory in php. So far I have the following:
$descriptorspec = array(
0 => array("pipe","r"),
1 => array("pipe","w"),
2 => array("file","error.txt","a") );
$process = proc_open($command, $descriptorspec, $pipes);
if( is_resource( $process ) ) {
fwrite($pipes[0], $fdf);
fclose($pipes[0]);
echo stream_get_contents($pipes[1]));
fclose($pipes[1]);
proc_close($process);
}
I'd like to direct the result directly out to the use as you would get in the above code, without actually saving the value in a variable and then printing it out. Is there a way to direct a stream in php's output directly to the output. I'd like to do this to save memory and reduce the time it takes for the code to run.
Does anyone know of a function for a stream in php that prints the stream result directly out. Or is there a setting for proc_open that does this. Either way I fear this may not work at all, as I may need to add a content-length header to the output to display the PDF directly. If anyone knows of the function to print the stream directly out, is there also a way to get the byte length of the stream without actually reading it in.
Since no one has posted an answer this I may as well post the answer I've found.
Open a stream to php://output with fopen mode w, and use stream_copy_to_stream to send the data from the proc_open process to the user directly. There does not appear to be an easy method to get the length or size of the data in the stream however.
$descriptorspec = array(
0 => array("pipe","r"),
1 => array("pipe","w"),
2 => array("file","error.txt","a") );
$process = proc_open($command, $descriptorspec, $pipes);
if( is_resource( $process ) ) {
fwrite($pipes[0], $fdf);
fclose($pipes[0]);
/* New Code */
if( $fout = fopen("php://output","w") ) {
stream_copy_to_stream($pipes[1],$fout);
fclose($fout);
}
fclose($pipes[1]);
$result = proc_close($process);
}
I would like the output of one process created with proc_open to be piped to another one created with proc_open (in php). For example. In bash I can do:
[herbert#thdev1 ~]$ cat foo
2
3
1
[herbert#thdev1 ~]$ cat foo | sort
1
2
3
[herbert#thdev1 ~]$
I would like to simulate this in php using proc_open (instead of shell_exec) in order to have control over return-codes, pipes, etc. So I want something like this:
$catPipes=array();
$sortPipes=array();
$cwd = '/tmp';
$env = array();
$catProcess = proc_open("cat foo", array(
0 => array("pipe", "r"),
1 => array("pipe", "w")
), $catPipes, $cwd, $env);
$sortProcess = proc_open("sort", array(
0 => array("pipe", "r", $catPipes[1]),
1 => array("pipe", "w"),
), $sortPipes, $cwd, $env);
echo stream_get_contents($sortPipes[1]);
fclose($sortPipes[1]);
//proc_close(this) ... proc_close(that) ... etc
Would someone know how I can simulate the "|" of bash in php, i.e. connect the second descriptor of the cat-process to the first descriptor of the sort-process? Any help would be appreciated! But please do not redirect me to shell_exec, as I want to be able to check exit-codes and log errors :).
EDIT:
My needs-to-work-business-solution btw is:
while(!feof($searchPipes[1])) fwrite($lookupPipes[0], stream_get_line($searchPipes[1], 40000));
Which is basically what the OS would do, but I do not want to my own pipe-management, as I have a kernel/posix for that, and let's be honest, it's not 1976 :)
Yes, you can -- but i think you have to define this the way round. That you can use the STDIN of "sort" as STDOUT pipe for "cat". Have a look at the following, which works for me:
<?php
$txt = "a\nc\ne\nb\nd\n";
$fh = fopen('data://text/plain;base64,' . base64_encode($txt), 'r');
$sort_pipes = array();
$sort_proc = proc_open(
'sort',
array(
array('pipe', 'r'),
STDOUT
),
$sort_pipes
);
$cat_pipes = array();
$cat_proc = proc_open(
'cat',
array(
$fh,
$sort_pipes[0]
),
$cat_pipes
);
In the first two rows i defined a data stream from a text string that i do not have to rely on a file somewhere in the filesystem. Note, that i have a list of unsorted characters stored in the data stream (a, c, e, b, d). Running the script above should return a sorted list to STDOUT.
Note, that you can specify resources as descriptors, too. In this case you must omit the array notation, so:
STDOUT
instead of
array(STDOUT)
etc.
Btw.: you can even write directly to a file specified by a file name. You can find further information on the descriptor specification in the manual entry for proc_open at http://en.php.net/manual/de/function.proc-open.php
EDIT
The other way works too, of course: you can also write "cat" to an STDOUT pipe array('pipe', 'w') and use $cat_pipes[1] as STDIN for "sort". :)
I'm trying to get
`mysql -uroot`;
to enter the MySQL interactive client just as executing
$ mysql -uroot
from the shell does.
It's okay if the PHP script exists after (or before), but I need it to invoke the MySQL client.
I've tried using proc_open() and of course system(), exec() and passthru(). Wondering if anyone has any tips.
New solution:
<?php
$descriptorspec = array(
0 => STDIN,
1 => STDOUT,
2 => STDERR
);
$process = proc_open('mysql -uroot', $descriptorspec, $pipes);
Old one:
Save for tab completion (you could probably get it in there if you read out bytes with fread instead of using fgets), this gets you on your way, lots left to tweak:
<?php
$descriptorspec = array(
0 => array("pty"),
1 => array("pty"),
2 => array("pty")
);
$process = proc_open('mysql -uroot', $descriptorspec, $pipes);
stream_set_blocking($pipes[1], 0);
stream_set_blocking($pipes[2], 0);
stream_set_blocking(STDIN,0);
do {
echo stream_get_contents($pipes[1]);
echo stream_get_contents($pipes[2]);
while($in = fgets(STDIN)) fwrite($pipes[0],$in);
} while (1);
I guess it does work, but it's waiting for some input. Try sending some sql commands to it's stdin. Of course, since the backtick operator doesn't support IO remapping, you'll need more complex process handling.