I need to write some service for my application. I want each client to have limited persistent connections (like only allow first 10 clients to connect).
I know I can listen on the port with PHP by socket_listen(). The parent process accept the connection, then pcntl_fork() the process to have children to handle connection.
But as far as I know, PHP resources doesn't persist when fork()ed. I wonder if it is possible to do this with PHP, or I have to do this in C?
1)
Why bother forking? Run your daemon as a single process and use socket_select() ( or stream_select) to listen for requests.
See Aleksey Zapparov's code here for a ready-written solution.
2)
Why bother with the pain of writing your own socket code - use [x]inetd to manage the servers and do al the communication on stdio (note that unlike solution 1, there will be a seperate process for each client - therefore the handling code will be non-blocking)
--
You are correct in saying that PHP resources should not be available in a forked process - but give no indication of how this relates to your current problem. Is it just so that you can count the number of connections? Or something else? In the case of the former, there are much easier ways of doing this. Using solution 1, just increment and decrement a counter variable when clients connect/disconnect. In the case of 2, the same approach but keep the variable in a datafile/database (you might also want to store info about the connections and run occasional audits). Alternatively limit the connections on the firewall.
C.
Maybe you could try share it using memcache (http://www.php.net/manual/en/book.memcache.php). (i never tried this, may it may works)
Related
Do I really need to do mysql_close()? Why or why not?
Is there a trigger that closes the link after mysql_connect even if I don't do mysql_close?
According to the documentation:
Using mysql_close() isn't usually necessary, as non-persistent open links are automatically closed at the end of the script's execution.
Personally, I always like to make sure I pedantically close anything that I open, but it's not required.
In most cases calling mysql_close will not make any difference, performance-wise. But it's always good practice to close out resources (file handles, open sockets, database connections, etc.) that your program is no longer using.
This is especially true if you're doing something that may potentially take a few seconds - for instance, reading and parsing data from a REST API. Since the API call is going over the line, negative network conditions can cause your script to block for several seconds. In this case, the appropriate time to open the database connection is after the REST call is completed and parsed.
To sum up my answer, the two big rules are:
Only allocate resources (file handles, sockets, database connections, etc.) when your program is ready to use them.
Free up resources immediately after your program is done with them.
what's the benefit of closing the link?
The benefit is that you can free the connection to the database, and corresponding resources in the database server, earlier than the PHP request cleanup would do it.
Say for example you query all the data your request will need in the first 20 milliseconds of the request. But then your PHP code spends another 80 milliseconds running code and formatting the results. Which means 80% of the time, the app is holding open a db connection without needing to, and on average, 8 out of 10 connection threads on the db server are idle and using resources.
The manual says:
Using mysql_close() isn't usually necessary, as non-persistent open links are automatically closed at the end of the script's execution.
So, no, not really. It is helpful to free up resources before attempting an operation that potentially consumes large amounts of resources though, but it probably won't make a big difference.
what;s the benefit of closing the link?
Normally, there is no benefit in closing the link yourself, as it will be automatically closed.
I can only think of a couple of benefits
If your script has a lot of processing to do after it has finished using the database, closing the database link prematurely may help free up a little bit of memory and other resources (such as MySQL connections) while your script continues with other things. This is very unlikely to be an issue in most scripts, since most scripts will terminate pretty quickly after it has finished with the database connection anyway, and the time the connection is held open before the PHP script terminates will be comparatively short.
Completeness and cleanliness of code. It can give you a good feeling, and is generally good code hygiene to close what you have opened, even though in this case it isn't technically required.
Suppose we have a user variable $_SESSION['variable'] that may or may not be modified as the user access a page.
Suppose the same user has several browser windows open and somehow makes simultaneous requests to the server that result on changes to the session variable.
Questions:
How does the server "queue" these changes, since they are targeted at
the same variable? Is there a potential for server error here?
Is there a way to "lock" the session variable for reading/writing in
order to implement some kind of status check before changing its
value?
EDIT
( thanks Unheilig for the cleanup)
Regarding the "queueing", I am interested in what happens if two requests arrive at the same time:
Change X to 1
Change X to 2
I know this doesn't seem a real world scenario, but it just came to my mind when designing something. It could become a problem if the system allows too many concurrent requests from the same user.
Each individual PHP Session is 'locked' between the call to session_start() and either the call to session_write_close() or the end of the request, whichever is sooner.
Most of the time you would never notice this behaviour.
However, if you have a site which does make many concurrent requests* then these requests would appear to queue in first-come-first-served order.
To be clear; in a typical Apache/PHP setup your requests will come in to the server and start your PHP executions concurrently. It is the session_start() call that will block/pause/wait/queue because it's waiting to gain the file-lock on the session file (or similar depending on your session_hander).
To increase request throughput or reduce waiting requests therefore:
Open and close the session (session_start(), do_work(), session_write_close()) as rapidly as possible; to reduce the time the session is locked for writing.
Make sure you're not leaving the session open on requests that are doing long work (accessing 3rd party APIs, generating or manipulating documents, running long database queries etc).. unless absolutely necessary.
Avoid, where possible, touching the session at all. Be as RESTful as possible.
Manage the queuing and debouncing of requests as elegantly as possible on the client side of the application
Hope that helps.
J.
*Ajax & Frame/iFrame heavy applications often exhibit this problem.
I'm writing a websocket server in PHP, that needs to be able to handle a large number of concurrent connections. I'm currently using the socket_select function to allow it to handle them, but this still blocks all other connections when sending a large block of data to a client. Is there a way for the master script to accept the incoming socket, and then start up a second PHP script (in a non-blocking fashion, obviously) and pass the client socket to that script for processing? I know this is possible in C, but the codebase is such that a migration is impossible, sadly.
*The server is running exclusively on a Unix stack, no need for a MS compatible solution.
I'm currently using the socket_select function to allow it to handle them, but this still blocks all other connections when sending a large block of data to a client.
Then don't send all the data at once. If you are going to do
socket_write ($mysocket, $mybuffer, 10000000);
then yeah, you'll have to wait until all 10 million bytes have been sent out. However, you can use the $write array of socket_select to check if you can write to the socket, in combination with non-blocking sockets. Each time socket_select says you have a 'go!' on the socket, write data until socket_write starts to complain (i.e. returns FALSE or less than the specified length). This will keep the socket's send buffer optimally filled.
The downside is that you must keep track of exactly where in your output buffer you are; also, turn off non-blocking on the socket after you've written al your data or socket_select will keep on firing (this assumes you want to send multiple large blobs of data).
The answer turns out to be the same answer you'd use in C - fork(). When you fork, the state of all open files and ports is preserved, so a child process can read a port that was opened by its parent (this is the same way that modern webservers spin off worker threads for each client connection that comes in) It does require using the pcntl (process control) module which is disabled by default and should be used sparingly, but it works:
if($verbose)
echo "Connected Client from $remoteaddy to $localaddy\n";
echo "Forking...";
$pid = pcntl_fork(); // you're bringing children into this world, just to kill them in a few seconds. You monster.
if($pid==0){
$p = posix.getpid();
echo "PID OF CHILD: $p\n";
//in child process. Send a handshake and wait for the callback from the WebSockets library
$this->send($client, "Thank you for contacting myAwesomeServer.com! I'm slave #{$p}, and I'll be your host today");
}else if($pid>0){
$childWorkers[]=$pid;
echo "[ OK ]\n";
$this->disconnect($client->socket, false); //disconnect the clients socket from the master thread, so only the child thread is talking to the client
}else if($pid==-1){
echo "[FAIL] unable to create child worker\n";
}
NOTE!! This approach is PURELY ACADEMIC, and should only be used on small, 'pet' projects when you don't have enough time to learn a more appropriate language (personally, I know C well enough to fork(), but my lack of knowledge of its string manipulation functions would no doubt leave a gaping security hole in the server). I'm not sure how the Zend engine is doing this pcntl_fork(), but I'd imagine that the memory image of this monstrosity is going to be many times the size of equivalent C code..
I have a PHP function that I want to make available publically on the web - but it uses a lot of server resources each time it is called.
What I'd like to happen is that a user who calls this function is forced to wait for some time, before the function is called (or, at the least, before they can call it a second time).
I'd greatly prefer this 'wait' to be enforced on the server-side, so that it can't be overridden by dubious clients.
I plan to insist that users log into an online account.
Is there an efficient way I can make the user wait, without using server resources?
Would 'sleep()' be an appropriate way to do this?
Are there any suggested problems with using sleep()?
Is there a better solution to this?
Excuse my ignorance, and thanks!
sleep would be fine if you were using PHP as a command line tool for example. For a website though, your sleep will hold the connection open. Your webserver will only have a finite number of concurrent connections, so this could be used to DOS your site.
A better - but more involved - way would be to use a job queue. Add the task to a queue which is processed by a scheduled script and update the web page using AJAX or a meta-refresh.
sleep() is a bad idea in almost all possible situations. In your case, it's bad because it keeps the connection to the client open, and most webservers have a limit of open connections.
sleep() will not help you at all. The user could just load the page twice at the same time, and the command would be executed twice right after each other.
Instead, you could save a timestamp in your database for when your function was last invoked. Then, before invoking it, you should check the database to see if a suitable amount of time has passed. If it has, invoke the function and update the timestamp in the database.
If you're planning on enforcing a user login, than the problem just got a whole lot simpler.
Have a record inn the database listing users and the last time they used your resource consuming service, and measure the time difference between then and now. If the time difference is too low, deny access and display an error message.
This is best handled at the server level. No reason to even invoke PHP for repeat requests.
Like many sites, I use Nginx and you can use it's rate-limiting to block repeat requests over a certain number. So like, three requests per IP, per hour.
I have never used memcache before now so please excuse my inexperience. Although it is pretty self explanatory, I would like to make sure I am using the built in functions correctly as I am creating a class that will be used commercially so it must be correctly coded and efficient.
I have several questions but as they are very basic I felt it would be alright to combine them into one Stackoverflow question.
If they require an essay answer, please dont bother and I will post it up as a separate question
When would I need to use memcache::addServer and what is the difference between this and memcache::connect?
Does memcache overwrite stored values if it runs out of memory, even if the item has not yet expired?
What would I use memcache::getExtendedStats for?
How do I check to see if a connection to memcache already exists and if not, create a connection?
If I have my usual memcache server of 'localhost' set up, how would I go about setting up another memcache server on my same dedicated server?
Apart from more memory, what is the benefit of having more than one memcache server?
Should I check for memcache server updates regularly?
Does it use a lot of memory to run memcache::connect at the beginning of each page, even if I am not using it?
When am I likely to return errors and how do I catch these?
Most importantly, if I am using memcache within another class that has several methods that may be called more then once per script, how should I go about initialising the object and connecting to the server within each method?
My guess for the last question would be to do it like so:
class test {
public function blah(){
// Make sure the memcache object is accessible
global $memcache;
// Do something ...
// Save result in memcache
$memcache->set(...);
}
public function foo(){
// Do something ...
// No use for memcache
}
}
// Initialise each class
$test = new test;
$memcache = new memcache;
$memcache->connect(...);
// Call some methods from the test class
$test->blah();
$test->foo();
$test->blah();
As you can see in the above example, I connect to the memcache server at the beginning of the script. If I was to include this at the beginning of every page, even on pages that do not use memcache, would this increase the response time a lot or minimal amounts? Hence, question 8!
You might need some coffee or something before you read this:
You'd want to use Memcache::addServer when you need to add more Memcached servers. For example, if you had a really busy website or web app... you'd probably want to have more than one Memcached server running1. Memcache::connect is used when you want to start a connection to one of your Memcached servers. Also, according to the Memcache::addServer docs, another difference between Memcache::addServer and Memcache::connect is that with Memcache::addServer, the connection is not established until actually needed2.
If Memcached runs out of RAM, it will discard the oldest values3.
Memcache::getExtendedStats is used to check information about your Memcached server. For example, if you need to find out how long the server has been up (uptime,) how many connections the server has, or general server usage4, this is a great tool.
Probably the easiest way to check if a connection to Memcached already exists is to check your $memcache connection variable to see if it returns TRUE5. If you need to have a persistent connection (that keeps on going even after your script ends,) there is the option to use Memcache::pconnect6.
If you want to have two Memcached servers going on... and your first server is already your localhost, you will most likely need to have a separate, distinct server for the second7.
At least one other benefit of having more than one Memcached server is the idea that whenever you diversify your data (or servers,) even when one server goes down... you still have however many other servers there to pick up the pieces. Memcached looks8 like it is distributed over however many servers you have running... so if a server goes down, you are still losing that part of the cache. But, you do still have other servers up and running to help keep going.
In general, it's not a bad idea to keep almost any type of software up to date. It looks like Memcached is still a highly active project9 so you may want to update it when you can. But the essence of Memcached doesn't seem to change a whole lot over past versions... so, it might not be as critical to update it compared to something like operating system software.
It sounds like the way that Memcached allocates memory for TCP connections (when you make a call to your Memcached server via Memcache::connect,) does end up costing you memory10. If you are sure you aren't going to need that connection on some of your pages, you may want to avoid making that connect call.
Hard to say what type of errors might come up in your code. But, with something like Memcached, you may find errors coming up when you are running out of memory11.
Like the answer to question eight, I would still recommend trying to only call that $memcache->connect() in areas where you absolutely need it. You might be using Memcached in a lot of your application or scripts; but there still will probably be places where you won't need it.
As far as your code idea for question 10 goes, it's really up to you as far as the implementation goes. In general, it's good to try to avoid global variables12 when possible, though. Instead, like that article (12) in the footnote talks about, it's easier to just use a singleton class call for a connection... and then just call that each time you want to make a connection.
Wow, my eyes are tired. I hope this helps, man...!
1 http://en.wikipedia.org/wiki/Memcached (see Architecture section)
2 http://www.php.net/manual/en/memcache.addserver.php
3 http://en.wikipedia.org/wiki/Memcached (see Architecture section)
4 http://www.php.net/manual/en/memcache.getextendedstats.php
5 http://www.php.net/manual/en/memcache.connect.php (see Return Values section)
6 http://www.php.net/manual/en/memcache.pconnect.php
7 http://www.php.net/manual/en/memcache.addserver.php#101194
8 Benefits of multiple memcached instances
9 http://code.google.com/p/memcached/
10 http://www.facebook.com/note.php?note_id=39391378919 (from Facebook's point of view)
11 http://groups.google.com/group/memcached/browse_thread/thread/9ce1e2691efb283b
12 How to avoid using PHP global objects?