I have a game site developed using flash and php. The php code contains 4000 lines and it will run as a cron. Inside the code, there is one while loop which will run infinitely for checking any data is written in the socket and call different functions accordingly and will send the results back to the sockets. From flash, it will get the results and will be displayed.
The problem Im facing is, somewhere from the php code, it is leaking memory. Since it is very big, I can not find out from where it is happening. Moreover it can be run only as a cron. Is there any tool to find out the memory leakage ? I have heard about xdebug but I didnt use. Any other ?
check.php (as cron)
$sock = fsockopen(IP_ADDRESS, PORT, $sock_error_code, $sock_error_string, 10); if (!$sock){
$message = "Server was down, restarting...\n\n";
$last_line = system("php -q gameserver/server.php", $retval);} else {
$message = "Server is up...";
$message .= $sock_error_string." (".$sock_error_code.")\n\n";}
server.php (only some part)
class gameserver {
var $server_running = true;
function gameserver() {
global $cfg, $db;
$this->max_connections = $cfg["server"]["max-connections"];
$this->start_socket();
echo "Gameserver initialized\n";
while ($this->server_running) {
$read = $this->get_socket_list();
$temp = socket_select($read, $null, $null, 0, 15);
if (!empty($read)) {
$this->read_sockets($read);
}
$db->reconnection();
$this->update_DB_records();
$this->check_games_progress();
if ($this->soft_shutdown && $this->active_games == 0) {
$this->server_running = false;
echo "soft shutdown complete\n";
}
}
$this->stop_socket();
echo "Server shut down\n";
}} $server = new gameserver();
Two things, first, ensure that you sleep at least once inside the loop, to ensure that you don't use 97% cpu.
Second, a trick I've found is, if there is any database activity, to call mysql_free_result (or it's equivalent for other DBMS') to free up the memory used to store the result of the query.
Are you starting never-ending programs from CRON? Cron will start a new instance according to the schedule you specify and you'll end up with several running programs doing the same thing.. Could this be your problem?
I am assuming that you are not starting a new instance using cron every minute, and having each run an infinite loop:
XDebug is probably your best bet. Other than that, you could use memory_get_usage() and log memory usage in specific points of your loop.
Could it simply be that your script accumulates data and doesn't clean it up properly on the end of each loop?
Related
I've read lots of answers but I haven't found a clear explanation. I'm trying to make an open port check tool in PHP.
When using fsockopen the server stops working while it checks every single port, so I cannot use this method to check the ports status.
Is there any efficient way of doing the same thing but in a non-blocking or asynchronous way?
This is my (non efficient) code:
$resultList = [];
foreach ($list as $key => $value) {
$object = new stdClass();
$object->id = $value["id"];
if (fsockopen($value["ip"], $value["port"])) {
$object->status = true;
} else {
$object->status = false;
}
array_push($resultList,$object);
}
A PHP script is always synchronous - you call a function, you wait for it to finish before your script moves on, however you can use PHP to trigger a second PHP script as a new process via the following generalised command:
shell_exec('/usr/bin/php -f /path/to/script.php &> /dev/null &');
Now this doesn't really help your problem I just included it to answer the exact words of your question - you can't have this second PHP script interact with the first in any way without involving AJAX, at which point you may as well just AJAX the entire thing.
To try and answer the spirit of your question I would like to suggest a different approach:
You have your parent page:
<?php
$hostname = '192.168.0.1';
?>
<h1>Ports for server <?php echo $hostname; ?>:</h1>
<iframe src="/path/to/my/post-scanner-script.php?hostname=<?php echo urlencode($hostname); ?>"></iframe>
Then your child page scans each port in sequence, and as it has finished scanning that port it uses flush()
<?php
// Stuff here to interpret $_GET['hostname'];
foreach ($list as $key => $value) {
$status = !!fsockopen($value["ip"], $value["port"]);
echo "<div>{$value['id']}: " . ($status ? 'OPEN' : 'CLOSED') . "</div>";
flush();
}
What flush() will do is immediatley send all rendered output to the client machine (it will not send the same content twice), so what you will get is a slowly appearing list of each port and its status, appearing as soon as your script knows the status of the port.
This provides some immediate feedback to the end user as well as a less frustrating wait (it's much more bearable when you can actually see progress being made).
What I've to do it's a bit complicated.
I've a python script and I want to run it from PHP, but in background. I had seen somewhere that for run a python script in background I have to use the PHP command exec(script.py) to run without wait for a return: and thus far no problem.
First question:
I have to stop this loop script with another PHP command, how to do this?
Second question:
I have to implement a server-side timer, who stops the script at the end of the time.
I found this code:
<?php
$timer = 60*5; // seconds
$timestamp_file = 'end_timestamp.txt';
if(!file_exists($timestamp_file))
{
file_put_contents($timestamp_file, time()+$timer);
}
$end_timestamp = file_get_contents($timestamp_file);
$current_timestamp = time();
$difference = $end_timestamp - $current_timestamp;
if($difference <= 0)
{
echo 'time is up, BOOOOOOM';
// execute your function here
// reset timer by writing new timestamp into file
file_put_contents($timestamp_file, time()+$timer);
}
else
{
echo $difference.'s left...';
}
?>
From this answer.
Then, there is a way to implement it in a MySQL database? (The integration with the script stop is not a problem)
That's actually pretty simple. You can use a memory object caching system. I would recommend memcached. Memory objects from memcached can be accessed literally from anywhere in your system. The only requirement is that a connection to the memcached backend server is supported. (PHP does, Python does, etc.)
Answer to your first question:
Create a variable called stopme with the value 0 in the memcached database.
Connect from your python script to the memcached database and read the variable stopme permanently. Let's say the python script is running when the variable stopme has the value 0.
In order to stop your script from PHP, make a connection from your PHP script to the memcached server and set stopme to 1.
The python script receives the updated value instantly and exits.
Answer to your second question:
It could be done like explained in my answer before through reading shared variables, but additionally I would like to mention that you also could use a cronjob to kill a running script.
I need a function that executes by itself in php without the help of crone. I have come up with the following code that works for me well but as it is a never-ending loop will it cause any problem to my server or script, if so could you give me some suggestion or alternatives, please. Thanks.
$interval=60; //minutes
set_time_limit(0);
while (1){
$now=time();
#do the routine job, trigger a php function and what not.
sleep($interval*60-(time()-$now));
}
We have used the infinite loop in a live system environment to basically wait for incoming SMS and then process it. We found out that doing it this way makes the server resource intensive over time and had to restart the server in order to free up memory.
Another issue we encountered is when you execute a script with an infinite loop in your browser, even if you hit the stop button it will continue to run unless you restart Apache.
while (1){ //infinite loop
// write code to insert text to a file
// The file size will still continue to grow
//even when you click 'stop' in your browser.
}
The solution is to run the PHP script as a deamon on the command line. Here's how:
nohup php myscript.php &
the & puts your process in the background.
Not only we found this method to be less memory intensive but you can also kill it without restarting apache by running the following command :
kill processid
Edit: As Dagon pointed out, this is not really the true way of running PHP as a 'Daemon' but using the nohup command can be considered as the poor man's way of running a process as a daemon.
You can use time_sleep_until() function. It will return TRUE OR FALSE
$interval=60; //minutes
set_time_limit( 0 );
$sleep = $interval*60-(time());
while ( 1 ){
if(time() != $sleep) {
// the looping will pause on the specific time it was set to sleep
// it will loop again once it finish sleeping.
time_sleep_until($sleep);
}
#do the routine job, trigger a php function and what not.
}
There are many ways to create a daemon in php, and have been for a very long time.
Just running something in background isn't good. If it tries to print something and the console is closed, for example, the program dies.
One method I have used on linux is pcntl_fork() in a php-cli script, which basically splits your script into two PIDs. Have the parent process kill itself, and have the child process fork itself again. Again have the parent process kill itself. The child process will now be completely divorced and can happily hang out in background doing whatever you want it to do.
$i = 0;
do{
$pid = pcntl_fork();
if( $pid == -1 ){
die( "Could not fork, exiting.\n" );
}else if ( $pid != 0 ){
// We are the parent
die( "Level $i forking worked, exiting.\n" );
}else{
// We are the child.
++$i;
}
}while( $i < 2 );
// This is the daemon child, do your thing here.
Unfortunately, this model has no way to restart itself if it crashes, or if the server is rebooted. (This can be resolved through creativity, but...)
To get the robustness of respawning, try an Upstart script (if you are on Ubuntu.) Here is a tutorial - but I have not yet tried this method.
while(1) means it is infinite loop. If you want to break it you should use break by condition.
eg,.
while (1){ //infinite loop
$now=time();
#do the routine job, trigger a php function and what no.
sleep($interval*60-(time()-$now));
if(condition) break; //it will break when condition is true
}
I'm trying to test a race condition in PHP. I'd like to have N PHP processes get ready to do something, then block. When I say "go", they should all execute the action at the same time. Hopefully this will demonstrate the race.
In Java, I would use Object.wait() and Object.notifyAll(). What can I use in PHP?
(Either Windows or linux native answers are acceptable)
Create a file "wait.txt"
Start N processes, each with the code shown below
Delete the "wait.txt" file.
...
<?php
while (file_exists('wait.txt')) {}
runRaceTest();
Usually with PHP file lock approach is used. One create a RUN_LOCK or similar file and asks for file_exists("RUN_LOCK"). This system is also used to secure potential endless loops in recursive threads.
I decided to require the file for the execution. Other approach may be, that existence of the file invokes the blocking algorithm. That depends on your situation. Always the safer situation should be the easier to achieve.
Wait code:
/*prepare the program*/
/* ... */
/*Block until its time to go*/
define("LOCK_FILE", "RUN_UNLOCK"); //I'd define this in some config.php
while(!file_exists(LOCK_FILE)) {
usleep(1); //No sleep will eat lots of CPU
}
/*Execute the main code*/
/* ... */
/*Delete the "run" file, so that no further executions should be allowed*/
usleep(1); //Just for sure - we want other processes to start execution phase too
if(file_exists(LOCK_FILE))
unlink(LOCK_FILE);
I guess it would be nice to have a blocking function for that, like this one:
function wait_for_file($filename, $timeout = -1) {
if($timeout>=0) {
$start = microtime(true)*1000; //Remember the start time
}
while(!file_exists($filename)) { //Check the file existence
if($timeout>=0) { //Only calculate when timeout is set
if((microtime(true)*1000-$start)>$timeout) //Compare current time with start time (current always will be > than start)
return false; //Return failure
}
usleep(1); //Save some CPU
}
return true; //Return success
}
It implements timeout. You don't need them but maybe someone else will.
Usage:
header("Content-Type: text/plain; charset=utf-8");
ob_implicit_flush(true);while (#ob_end_clean()); //Flush buffers so the output will be live stream
define("LOCK_FILE","RUN_FOREST_RUN"); //Define lock file name again
echo "Starting the blocking algorithm. Waiting for file: ".LOCK_FILE."\n";
if(wait_for_file(LOCK_FILE, 10000)) { //Wait for 10 secconds
echo "File found and deleted!\n";
if(file_exists(LOCK_FILE)) //May have been deleted by other proceses
unlink(LOCK_FILE);
}
else {
echo "Wait failed!\n";
}
This will output:
Starting the blocking algorithm. Waiting for file: RUN_FOREST_RUN
Wait failed!
~or~
Starting the blocking algorithm. Waiting for file: RUN_FOREST_RUN
File found and deleted!
PHP doesn't have multithreading. And its not planned to be implemented either.
You can try hacks with sockets though or 0MQ to communicate between multiple processes
See Why does PHP not support multithreading?
Php multithread
I have a list of data that needs to be processed. The way it works right now is this:
A user clicks a process button.
The PHP code takes the first item that needs to be processed, takes 15-25 secs to process it, moves on to the next item, and so on.
This takes way too long. What I'd like instead is that:
The user clicks the process button.
A PHP script takes the first item and starts to process it.
Simultaneously another instance of the script takes the next item and processes it.
And so on, so around 5-6 of the items are being process simultaneously and we get 6 items processed in 15-25 secs instead of just one.
Is something like this possible?
I was thinking that I use CRON to launch an instance of the script every second. All items that need to be processed will be flagged as such in the MySQL database, so whenever an instance is launched through CRON, it will simply take the next item flagged to be processed and remove the flag.
Thoughts?
Edit: To clarify something, each 'item' is stored in a mysql database table as seperate rows. Whenever processing starts on an item, it is flagged as being processed in the db, hence each new instance will simply grab the next row which is not being processed and process it. Hence I don't have to supply the items as command line arguments.
Here's one solution, not the greatest, but will work fine on Linux:
Split the processing PHP into a separate CLI scripts in which:
The command line inputs include `$id` and `$item`
The script writes its PID to a file in `/tmp/$id.$item.pid`
The script echos results as XML or something that can be read into PHP to stdout
When finished the script deletes the `/tmp/$id.$item.pid` file
Your master script (presumably on your webserver) would do:
`exec("nohup php myprocessing.php $id $item > /tmp/$id.$item.xml");` for each item
Poll the `/tmp/$id.$item.pid` files until all are deleted (sleep/check poll is enough)
If they are never deleted kill all the processing scripts and report failure
If successful read the from `/tmp/$id.$item.xml` for format/output to user
Delete the XML files if you don't want to cache for later use
A backgrounded nohup started application will run independent of the script that started it.
This interested me sufficiently that I decided to write a POC.
test.php
<?php
$dir = realpath(dirname(__FILE__));
$start = time();
// Time in seconds after which we give up and kill everything
$timeout = 25;
// The unique identifier for the request
$id = uniqid();
// Our "items" which would be supplied by the user
$items = array("foo", "bar", "0xdeadbeef");
// We exec a nohup command that is backgrounded which returns immediately
foreach ($items as $item) {
exec("nohup php proc.php $id $item > $dir/proc.$id.$item.out &");
}
echo "<pre>";
// Run until timeout or all processing has finished
while(time() - $start < $timeout)
{
echo (time() - $start), " seconds\n";
clearstatcache(); // Required since PHP will cache for file_exists
$running = array();
foreach($items as $item)
{
// If the pid file still exists the process is still running
if (file_exists("$dir/proc.$id.$item.pid")) {
$running[] = $item;
}
}
if (empty($running)) break;
echo implode($running, ','), " running\n";
flush();
sleep(1);
}
// Clean up if we timeout out
if (!empty($running)) {
clearstatcache();
foreach ($items as $item) {
// Kill process of anything still running (i.e. that has a pid file)
if(file_exists("$dir/proc.$id.$item.pid")
&& $pid = file_get_contents("$dir/proc.$id.$item.pid")) {
posix_kill($pid, 9);
unlink("$dir/proc.$id.$item.pid");
// Would want to log this in the real world
echo "Failed to process: ", $item, " pid ", $pid, "\n";
}
// delete the useless data
unlink("$dir/proc.$id.$item.out");
}
} else {
echo "Successfully processed all items in ", time() - $start, " seconds.\n";
foreach ($items as $item) {
// Grab the processed data and delete the file
echo(file_get_contents("$dir/proc.$id.$item.out"));
unlink("$dir/proc.$id.$item.out");
}
}
echo "</pre>";
?>
proc.php
<?php
$dir = realpath(dirname(__FILE__));
$id = $argv[1];
$item = $argv[2];
// Write out our pid file
file_put_contents("$dir/proc.$id.$item.pid", posix_getpid());
for($i=0;$i<80;++$i)
{
echo $item,':', $i, "\n";
usleep(250000);
}
// Remove our pid file to say we're done processing
unlink("proc.$id.$item.pid");
?>
Put test.php and proc.php in the same folder of your server, load test.php and enjoy.
You will of course need nohup (unix) and PHP cli to get this to work.
Lots of fun, I may find a use for it later.
Use an external workqueue like Beanstalkd which your PHP script writes a bunch of jobs too. You have as many worker processes pulling jobs from beanstalkd and processing them as fast as possible. You can spin up as many workers as you have memory / CPU. Your job body should contain as little information as possible, maybe just some IDs which you hit the DB with. beanstalkd has a slew of client APIs and itself has a very basic API, think memcached.
We use beanstalkd to process all of our background jobs, I love it. Easy to use, its very fast.
There is no multithreading in PHP, however you can use fork.
php.net:pcntl-fork
Or you could execute a system() command and start another process which is multithreaded.
can you implementing threading in javascript on the client side? seems to me i've seen a javascript library (from google perhaps?) that implements it. google it and i'm sure you'll find something. i've never done it, but i know its possible. anyway, your client-side javascript could activate (ajax) a php script once for each item in separate threads. that might be easier than trying to do it all on the server side.
-don
If you are running a high traffic PHP server you are INSANE if you do not use Alternative PHP Cache: http://php.net/manual/en/book.apc.php . You do not have to make code modifications to run APC.
Another useful technique that can work along with APC is using the Smarty template system which allows you to cache output so that pages do not have to be rebuilt.
To solve this problem, I've used two different products; Gearman and RabbitMQ.
The benefit of putting your jobs into some sort of queuing software like Gearman or Rabbit is that you have multiple machines, they can all participate in processing items off the queue(s).
Gearman is easier to setup, so I'd suggest poking around with it a bit first. If you find you need something more heavy duty with queue robustness; Look into RabbitMQ
http://www.danga.com/gearman/
http://pear.php.net/package/Net_Gearman (PEAR library)
You can use pcntl_fork() and family to fork a process - however you may need something like IPC to communicate back to the parent process that the child process (the one you fork'd) is finished.
You could have them write to shared memory, like via memcache or a DB.
You could also have the child process write the completed data to a file, that the parent process keeps checking - as each child process completes the file is created/written to/updated, and parent process can grab it, one at a time, and them throw them back to the callee/client.
The parent's job is to control the queue, to make sure the same data isn't processed twice and also to sanity check the children (better kill that runaway process and start over...etc)
Something else to keep in mind - on windows platforms you are going to be severely limited - I dont even think you have access to pcntl_ unless you compiled PHP with support for it.
Also, can you cache the data once its been processed, or is it unique data every time? that would surely speed things up..?