I'm writing a simple http server and I'd like to use php-cgi to handle POST requests. I write the handlePOST as follows:
void KServer::handlePOST(int sockfd, string file, string pdata){
char* argvs[MAX_PARAMS];
argvs[0] = const_cast<char*>(CGI_PATH);
argvs[1] = const_cast<char*>(file.c_str());
stringstream ss;
ss<<pdata.length();
string clen, sname;
ss>>clen;
clen = "CONTENT_LENGTH=" + clen;
sname = "SCRIPT_FILENAME="+file;
char* env[] = {
"REQUEST_METHOD=POST",
"REDIRECT_STATUS=CGI",
const_cast<char*>(clen.c_str()),
const_cast<char*>(sname.c_str()),
"CONTENT_TYPE=application/x-www-form-urlencoded",
0
};
istringstream stream(pdata);
cin.rdbuf(stream.rdbuf());
execve(argvs[0], argvs, env);
...
I know that php-cgi get POST data from STDIN. And I put the POST data (var. pdata) into cin.rdbuf. However, the program fails to fetch data from STDIN when executing execve. But if I enter the string using console, the program can run correctly.
It is highly improbable that when your custom http server reads the socket, it always manages to read exactly the number of bytes in the header portion of the HTTP message before getting to this point; so that when it executes the external process, the body portion of the message, containing the POST data, is now waiting to be read, on its standard input.
More than likely, you are using the C++ I/O library, or maybe even the C stdio library, to read the standard input, or socket. Which means, of course, that the built-in input buffering, that's employed by iostreams or stdio, has already managed to swallow a good chunk of the POST data from the body of the HTTP message, that immediately follows its header. It's now waiting or you, to continue reading on your std::cin, stdin; or whatever your code is actually reading.
It therefore follows that, sadly, when you execute the external process, it will be dismayed to find that the POST data it expects has already been consumed, either in entirety, or in part, on its standard input. Instead, it's sitting right here, in your process, waiting for you to resume reading your std::cin, stdin, or whatever.
In other words, for this to work, you cannot just execute an external process, and wash your hands of the whole mess. You will need to set up a pipe to the external process's standard input, and it is your responsibility to proceed to meticulously read the body of the HTTP message, that forms the POST data, just like you've just now finished reading its header, and then dump it into the pipe.
That means parsing the Content-Length header, to know exactly how many bytes are in the body, and reading exactly that (because you're not going to get an EOF, and rely on it, since the socket will remain open for your expected HTTP reply).
And, of course, you will also then need to carefully handle everything that happens as a result of writing to a pipe, i.e. gracefully handling the reader shutting down before consuming the entirety of the piped data, resulting in a broken pipe, etc...
Related
I have some concerns about my PHP file, which is processing too long.
I'm using XAMPP.
The problem is, that when i use too many methods of my classes, the PHP file loads too slow or it executes too slow.
Here's my example code:
class Sample {
public function show() {
echo 'test';
}
}
$classObject = new Sample();
$classObject->show();
When i run the PHP code above, it takes only i think 1.5s, but if i add more method calls like this:
$classObject = new Sample();
$classObject->show();
$classObject->show();
the PHP code takes almost 3s while executing.
Is there a way to solve this problem?
I already found out what is the problem is when i used oop style php while fetching data from my database it takes long secs before it gonna execute but when I used procedural php style it executes faster.
Firstly, you should take a look at some PHP debugging techniques, because i dont see any attempt of debugging in your code sample.
To your problem:
What you've described is an issue, that can be but must not be a fault of PHP. The thing is, you have to understand how request processing works ... in your case (XAMPP + PHP) it goes like this:
WebServer start:
XAMPP starts Apache (or Tomcat/or whatever WebServer you're using)
Apache starts PHP binary
PHP loads the configuration (php.ini)
The request:
You'll start a web browser (doesn't matter which one)
You'll enter a web address (with or without a specific path - URI)
Then happens some DNS things - that's NOT important for you now
The browser sends a plain-text document on the server via the HTTP protocol, which communicates over TCP - also not so important for you at this time
The WebServer receives a HTTP request and begins the processing
The WebServer runs your PHP script in the already running PHP binary and awaits the ouput
The response
Then the WebServer takes the output of your script and sends it back to the browser
The browser decides (by the headers) how (and if) the content will be displayed to you
Then the browser begins displaying the response content - in your case a HTML or a plain-text document
While drawing the document, the browser begins processing all JavaScript and CSS on the way (up to down)
With this knowledge you need to find out WHERE on the way is the delay taking up time.
The first thing you should do, is to take a look on how long your script is being processed, so:
<?php
$start_time = microtime(true);
$classObject = Sample();
$classObject->show();
$classObject->show();
echo 'Processing took: ' . number_format(microtime(true) - $start_time, 6, '.', '') . ' seconds';
Then, you should look into some Developer Tool your browser provides, for example DevTools in Google Chrome (very good for debugging, though not the best) - hit F12 to open it.
You'll see something like this:
Time is the duration between the time, the request was fully sent and the time, the response was fully received
Load is the sum of durations of all the requests that were done at that moment
Finish is the duration between the time the first request was fully sent and the time the last response was fully received
DOMContentLoaded is the duration of rendering the entire document (browser/client side)
When you have all of this specific duartions, you can decide, where probably will be the problem :)
I have a python script (analyze.py) which takes a filename as a parameter and analyzes it. When it is done with analysis, it waits for another file name. What I want to do is:
Send file name as a parameter from PHP to Python.
Run analyze.py in the background as a daemon with the filename that came from PHP.
I can post the parameter from PHP as a command line argument to Python but I cannot send parameter to python script that already runs at the background.
Any ideas?
The obvious answer here is to either:
Run analyze.py once per filename, instead of running it as a daemon.
Pass analyze.py a whole slew of filenames at startup, instead of passing them one at a time.
But there may be a reason neither obvious answer will work in your case. If so, then you need some form of inter-process communication. There are a few alternatives:
Use the Python script's standard input to pass it data, by writing to it from the (PHP) parent process. (I'm not sure how to do this from PHP, or even if it's possible, but it's pretty simple from Python, sh, and many other languages, so …)
Open a TCP socket, Unix socket, named pipe, anonymous pipe, etc., giving one end to the Python child and keeping the other in the PHP parent. (Note that the first one is really just a special case of this one—under the covers, standard input is basically just an anonymous pipe between the child and parent.)
Open a region of shared memory, or an mmap-ed file, or similar in both parent and child. This probably also requires sharing a semaphore that you can use to build a condition or event, so the child has some way to wait on the next input.
Use some higher-level API that wraps up one of the above—e.g., write the Python child as a simple HTTP service (or JSON-RPC or ZeroMQ or pretty much anything you can find good libraries for in both languages); have the PHP code start that service and make requests as a client.
Here is what I did.
PHP Part:
<?php
$param1 = "filename";
$command = "python analyze.py ";
$command .= " $param1";
$pid = popen( $command,"r");
echo "<body><pre>";
while( !feof( $pid ) )
{
echo fread($pid, 256);
flush();
ob_flush();
}
pclose($pid);
?>
Python Part:
1. I used [JSON-RPC]: https://github.com/gerold-penz/python-jsonrpc to
create a http service that wraps my python script (runs forever)
2. Created a http client that calls the method of http service.
3. Printed the results in json format.
Works like a charm.
Well this is an old issue I've been dealing with and still no solution, so trying a new approach.
How can I send th SOAP response early (Before script execution ends)?
These issues are cause when the ACK file is not sent before 30 seconds as the process takes longer to complete then the allotted time.
flush() not working, get this error:
org.xml.sax.SAXParseException: XML document structures must start and end within the same entity.
without the flush() I get this
org.xml.sax.SAXParseException: Premature end of file.
The script process can takeover 180 seconds to complete and the server waiting for the response only wait for about 30 seconds before timing out (Which cause the above error).
any thoughts as to how I can fix this?
Here is some of the code: This is how I accept and send the ACK file for the imcoming SOAP request
$data = 'php://input';
$content = file_get_contents($data);
if($content) {
respond('true');
} else {
respond('false');
}
The respond function
function respond($tf) {
$ACK = <<<ACK
<?xml version = "1.0" encoding = "utf-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<soapenv:Body>
<notifications xmlns="http://soap.sforce.com/2005/09/outbound">
<Ack>$tf</Ack>
</notifications>
</soapenv:Body>
</soapenv:Envelope>
ACK;
print trim($ACK);
}
PHP uses a single thread processing approach and will not send back the ACK file until the thread has completed it's processing. Is there some way to close the socket after the ACK submission and continue the processing so I don't get these timeout issues on the sending server?
As far as I know we can't do anything about the 30 second limit and the output won't be flushed before the script finishes. Can you try splitting your processing logic into 2 pieces?
"Listener" script that accepts messages, logs the job to be done and sends ACK instantly, then quits.
Actual processing on your side which might take longer.
The "job to be done" could be a newly spawned process (check out the comments for the pcntl_fork() function but they don't look too promising to me) or some way to store the data in file or database and periodically process it with another script, for example scheduled to run every 5 minutes?
If 60 seconds would somehow save you, you could rewrite your outbound message into a callout from Apex. Basically you then write your own SOAP envelope and send it to any http address. You could put it in a trigger. See here for limitations of this approach though.
Other ranting:
I don't think your ACK really "matters" to salesforce. Outbound messages are just notifications. Do you rely somewhere else in your application on Ack = true/false? If not - listener that blindly sends ack=true & schedules the job might really be the way to go ;)
From the other queston I understood you basically just need to store updates in DB on your side. You realize you shouldn't use OM for audit purposes, right? (Link, search for "audit").
Wouldn't it be simpler to make PHP the active side? make it query Salesforce for [SELECT Id, Name FROM Account WHERE LastModifiedDate > :lastTimeYouQueried]? That way you can take all the time you want to process the results :)
a) (optional) you can use set_time_limit() for increase the time limit of the execution time.
b) You must increase the time to the wsdl client object, for example :
$clientwsdl->setOpt('timeout', 300); // if you are using PEAR:SOAP.
Most wsdl class allow to define the timeout.
c) and not, you can't response early using SOAP, not at least using one call. Take in consideration that SOAP return a XML, so a partial XML usually is invalid (it miss the closing tag).
d) Alternate, you can use other method than SOAP, for example reading a url:
$fp=fopen("http://www.mysite.com/url.php","r");
where url.php return the columns (or some kind of value without using xml :
30|50|70|80|20
30|50|70|80|20
30|50|70|80|20
I've got a chat program which pushes JSON data from Apache/PHP to Node.js, via a TCP socket:
// Node.js (Javascript)
phpListener = net.createServer(function(stream)
{
stream.setEncoding("utf8");
stream.on("data", function(txt)
{
var json = JSON.parse(txt);
// do stuff with json
}
}
phpListener.listen("8887", 'localhost');
// Apache (PHP)
$sock = stream_socket_client("tcp://localhost:8887");
$written = fwrite($sock, $json_string);
fclose($sock);
The problem is, if the JSON string is large enough (over around 8k), the output message gets split into multiple chunks, and the JSON parser fails. PHP returns the $written value as the correct length of the string, but the data event handler fires twice or more.
Should I be attaching the function to a different event, or is there a way to cache text across event fires, in a way that won't succumb to race conditions under heavy load? Or some other solution I haven't thought of?
Thanks!
You should try using a buffer, to cache the data, as Node.js tends to split data in order to improve performance.
http://nodejs.org/api.html#buffers-2
you can buffer all your request, and then call the function with the data stored at it.
TCP sockets don't handle buffering for you. How could it? It doesn't know what application layer protocol you are using and therefore has no idea what a "message" is. It is up to you to design and implement another protocol on top of it and handle any necessary buffering.
But, Node.js does have a built in application layer protocol on top of TCP that does automatically handle the buffering for you: the http module. If you use the http module instead of the tcp module for this you won't need to worry about packet fragmentation and buffering.
I first configure my script to run even after the HTTP request is over
ignore_user_abort(true);
then flush out some text.
echo "Thats all folks!";
flush();
Now how can I trick the browser into thinking the HTTP request is over? so I can continue doing my own work without the browser showing "page loading".
header(??) // something like this?
Here's how to do it. You tell the browser to read in the first N characters of output and then close the connection, while your script keeps running until it's done.
<?php
ob_end_clean();
header("Connection: close");
ignore_user_abort(true); // optional
ob_start();
echo ('Text the user will see');
$size = ob_get_length();
header("Content-Length: $size");
ob_end_flush(); // Will not work
flush(); // Unless both are called !
// At this point, the browser has closed connection to the web server
// Do processing here
echo('Text user will never see');
?>
Headers won't work (they're headers, so they come first)
I don't know of any way to close the http connection without terminating the script, though I suppose there's some obscure way of doing it.
Telling us what you want to do after the request is done would help us give better suggestions.
But generally, I'd be thinking about one of the following:
1) Execute some simple command-line script (using exec()) that looks like:
#!/bin/sh
php myscript.php <arg1> <arg2> .. <argN> &
Then kick that off from your http-bound script like:
<?PHP
exec('/path/to/my/script.sh');
?>
Or:
2) Write another program (possibly a continuously-running daemon, or just some script that is cronned ever so often), and figure out how your in-request code can pass it instructions. You could have a database table that queues work, or try to make it work with a flat file of some sort. You could also have your web-based script call some command-line command that causes your out-of-request script to queue some work.
At the end of the day, you don't want your script to keep executing after the http request. Assuming you're using mod_php, that means you'll be tying up an apache process until the script terminates.
Maybe this particular comment on php.net manual page will help: http://www.php.net/manual/en/features.connection-handling.php#71172
Theoretically, if HTTP 1.1 keep-alive is enabled and the client receives the amount of characters it expects from the server, it should treat it as the end of the response and go ahead and render the page (while keeping the connection still open.) Try sending these headers (if you can't enable them another way):
Connection: keep-alive
Content-Length: n
Where n is the amount of characters that you've sent in the response body (output buffering can help you count that.) I'm sorry that I don't have the time to test this out myself. I'm just throwing in the suggestion in case it works.
The best way to accomplish this is using output buffering. PHP sends the headers when it's good and ready, but if you wrap your output to the browser with ob_* you can control the headers every step of the way.
You can hold a rendered page in the buffer if you want and send headers till the sun comes up in china. This practice is why you may see a lot of opening <?php tags, but no closing tags nowadays. It keeps the script from sending any headers prematurely since there might some includes to consider.