I'm running a website which monitors some servers and there are more servers added daily.
Sadly my current solution is very slow (about 10 seconds loadtime for 31 servers). I'm using fsockopen which checks the IP and port. Because I parse a XML with the entries (host and port) I had to create a function and use this function in the parser so the visitors of the website can see the online or offline status of the server.
My current "checkserver" function looks like this:
function checkServer($ip, $port)
{
$fsockopen = #fsockopen($ip, $port, $errorNo, $errorStr, 1);
if (!$fsockopen)
return false;
else return true;
}
And in the parser the "if" rule for the server status looks like this:
if (checkServer((string)$server->host, (string)$server->port))
{
echo "SERVER ONLINE!";
}
else
{
echo "SERVER OFFLINE!";
}
Where $server is every single listed server in the XML <serverlist></serverlist> tag.
I already tried to change the timeout of fsockopen from 1 to 0.1 but some servers appear offline then and the loadtime is still at 8-10 seconds.
How could I speedup the loadtime? Could someone please help me with this?.. the project is very important for me. Thank you! I really appreciate every helpful answer!
First of all I would suggest caching. I am not sure how many users will open the page, but if you have multiple users per second opening the page you will have a lot of traffic to handle which on long term could create issues.
You have 2 solutions :
Using asynchronous events can allow you to do what you wish.T here are some libraries outhere doing this that can help you. I have used none sofar so I can't say which is best.
Using a library that cheats and uses exec & command lines : https://github.com/oliverde8/PHP-AsynchronousJobs
Using pthread library, this isn't a library coded in php, so it binaries you need to add to your http://pthreads.org/. You can add another library ontop to make the usage easier
Finally using javascript, so you open your page, then some ajax calls your php individually for each server & ask status.
I've tried a lot of ways but all of them are not working for me, I guess they are outdated and things changed. Maybe someone can advise me the direction to dig out?
I am the author of a PHP library called TorUtils that provides a number of classes related to Tor and relays.
One of the classes it provides is TorDNSEL example which gives you a simple interface to query the Tor DNS exit list to see if the remote IP is a Tor relay accessing your site.
The usage is simple (example updated 2022/02/04):
<?php
use Dapphp\TorUtils\TorDNSEL;
require_once 'src/TorDNSEL.php';
// Practical usage on a web server:
try {
if (TorDNSEL::isTor($_SERVER['SERVER_ADDR'])) {
// do something special for Tor users
} else {
// not using Tor, educate them! :-D
}
} catch (\Exception $ex) {
error_log("Tor DNSEL query failed: " . $ex->getMessage());
}
The TorDNSEL script doesn't require any of the other classes and can be downloaded and used as a standalone script. It makes a DNS request, but builds it directly so you also don't need any DNS functions or libraries available to PHP.
Alternatively, I use the ControlClient class to maintain very fresh exit lists. You can build your own lists using that (you'll need a Tor relay of your own to connect to, or you could use the Directory Authorities to build lists as well). Even easier, just periodically download and check the exit lists I export here. The list is updated every 10 minutes so please try not to download it more than once every 10 minutes or so.
Then save it to your site somewhere and check the client IP against the ones in the list.
You can download the source from GitHub or composer require dapphp/torutils to install to an existing project.
Hope that helps, let me know if I can answer anything further.
Tor browser or not checking in PHP code I am using currently:
<?php
namespace neoistone;
class TOR
{
public static function isTorRequest()
{
$ipServeur = $_SERVER['SERVER_ADDR'];
$ipUser = $_SERVER['REMOTE_ADDR'];
if(is_file(TMP.'torips.db') == false){
self::tor_ips_update();
}
if (self::match_ips($ipServeur)) {
return true;
} elseif (self::match_ips($ipUser)) {
return true;
} else {
return false;
}
}
public static function tor_ips_update()
{
file_put_contents(TMP.'torips.db',file_get_contents('https://check.torproject.org/torbulkexitlist'));
}
public static function match_ips($ip)
{
$lines = file(TMP.'torips.db', FILE_IGNORE_NEW_LINES | FILE_SKIP_EMPTY_LINES);
return in_array($ip, $lines);
}
}
?>
Live URL: https://reseller.neoistone.com/api/
The way check.torproject.org uses is
For every user that hits the website, we check if their IP matches a known exit node's IP, and that its exit policy would allow accessing this website on port 443(HTTPS). If it is, then we can make a good assumption that the user has successfully connected via the Tor network.
You can find its source at https://git.torproject.org/check.git. It is under MIT licence. You can use Exits.IsTor in datastore.go to determine whether the request comes from tor. You need to change
var DefaultTarget = AddressPort{"38.229.72.22", 443}
to your server's IP address and port. You will also need some interface written in Go if you are using PHP if you want to use their code.
If you use Cloudflare, it uses the 2-letter code "T1" for Tor, and you can have custom rules for Tor.
Note that it is hard to distinguish a request directly from a Tor exit node (not via Tor) from a request from Tor. Most places assume they are all from Tor.
I have implemented the long polling successfully using normal Apache server, PHP, AJAX and Javascript. I don't use the Jquery to communicate with the server.
The problem is that the Apache server capabilities are limited, the server is not able to serve more than 5 browser tabs.
I wonder if there is any customization for the Apache or for the PHP to make them handle more concurrent connections? Or if there is any new/smart technique to do that? What are the maximum threads can be handled by a robust web server specialized in long polling?
I am not interested in the Web Sockets because of the browsers compatibility. I need something easy and robust into PHP. What Facebook are doing? I wonder how can they handle all the dynamic updates for million of users! What products/techniques they use?
A sample of my code:
srv_polling.php
<?php
function getResults(){..... return result;}
// recursive function inside the server
function hasResultChanged($old,$timeStart){
// to avoid server timeout (in seconds) in case no change for results
if(round(abs(time() - $timeStart) / 60*60,2) > 50)
return;
$new = getResults();
if($new != $old) // get back to browser
return true;
else{
$old = getResults();
sleep(2);
return $hasResultChanged($old,$timeStart);
}
}
$timeStart = time();
$old = $getResults();
sleep(2);
$hasResultChanged($old,$timeStart);
?>
// Javascript code to be executed at browser end
alert('Result has changed');
// Send AJAX request again to same page(srv_polling.php):
ajax.call({......})
Thank you for your hints! Greatly appreciated.
I am using this in my project
public function getLPollData($user, $handlerName) {
set_time_limit (600);
date_default_timezone_set('Europe/Berlin');
$counterEnd = (int)$_REQUEST["counterEnd"];
$counterStart = (int)$_REQUEST["counterStart"];
$this->expireNotifications($counterStart, $counterEnd);
$secCount = IDLE_WAIT;
do {
sleep(IDLE_TIME);
$updates = $this->fetchAllNotifications($counterEnd);
} while (!$updates && ($secCount--)>0);
if($updates){
}
header("HTTP/1.0 200");
return sprintf ('{"time" : "%s", "counter" : "%d", start : %d, data : %s}'
, date('d/m H:i:s'), $counterEnd,$counterStart,json_encode($updates));
}
Its combination of IDLE_WAIT & IDLE_TIME (10*3=~30 secs).
But i dont think your problem is on server side, if you are opening 5-6 connections from a browser, then remember each browser has limitation on how many active connections it can have to some particular domain, at a time. try diff browser or better diff machines, max two tabs in one browser.
I am trying to implement a realtime chat application using PHP . Is it possible to do it without using a persistent data storage like database or file . Basically what I need is a mediator written in PHP who
accepts messages from client browsers
Broadcasts the message to other clients
Forgets the message
You should check out Web Sockets of html5. It uses two way connection so you will not need any database or file. Any chat message comes to the server will directly sent to the other users browser without any Ajax call. But you need also to setup web socket server.
Web sockets are used in many real time applications as well. I am shortly planing to write full tutorial on that. I will notify you.
Just tried something I had never done before in response to this question. Seemed to work but I only tested it once. Instead of using a Socket I had an idea of using a shared Session variable. Basically I forced the Session_id to be the same value regardless of the user therefore they are all sharing the same data. From a quick test it seems to work. Here is what I did:
session_id('12345');
session_start();
$session_id = session_id();
$_SESSION['test'] = $_SESSION['test'] + 1;
echo "session: {$session_id} test: {$_SESSION['test']} <br />";
So my thought process was that you could simply store the chat info in a Session variable and force everyone regardless of who they are to use a shared session. Then you can simply use ajax to continually reload the current Session variable, and use ajax to edit the session variable when adding a message. Also you would probably want to set the Session to never expire or have a really long maxlifetime.
As I said I just played around with this for a few minutes to see if it would work.
You will want to use Sockets. This article will cover exactly what you want to do: http://devzone.zend.com/209/writing-socket-servers-in-php/
When I tried to solve the same problem, I went with Nginx's Push Module. I chose to go this way since I had to support older browsers (that usually won't support WebSockets) and had no confidence in setting up an appropriate solution like Socket.io behind a TCP proxy.
The workflow went like this:
The clients connect through long-polling to my /subscriber location, which is open to all.
The /publisher location only accepts connections from my own server
When a client subscribes and talks, it basically just asks a PHP script to handle whatever data is sent.
This script can do validation, authorization, and such, and then forwards (via curl) the message in a JSON format to the /publisher.
Nginx's Push Module handles sending the message back to the subscribers and the client establishes a new long-polling connection.
If I had to do this all over again, then I would definitely go the Socket.io route, as it has proper fallbacks to Comet-style long-polling and has great docs for both Client and Server scripts.
Hope this helps.
If you have a business need for PHP, then adding another language to the mix just means you then have two problems.
It is perfectly possible to run a permanent, constantly-running daemonised PHP IRCd server: I know, because I've done it, to make an online game which ran for years.
The IRC server part I used is a modified version of WaveIRCd:
http://sourceforge.net/projects/waveircd/
I daemonised it using code I made available here:
http://www.thudgame.com/node/254
That code might be overkill: I wrote it to be as rugged as I could, so it tries to daemonise using PHP's pcntl_fork(), then falls back to calling itself recursively in the background, then falls back to perl, and so on: it also handles the security restrictions of PHP's safe mode in case someone turns that on, and the security restrictions imposed by being called through cron.
You could probably strip it down to just a few lines: the bits with the comments "Daemon Rule..." - follow those rules, and you'll daemonize your process just fine.
In order to handle any unexpected daemon deaths, etc, I then ran that daemoniser every minute through cron, where it checked to see if the daemon was already running, and if so either quietly died, or if the daemon was nonresponsive, killed it and took its place.
Because of the whole distributed nature of IRC, it was nicely rugged, and gave me a multiplayer browser game with no downtime for a good few years until bit-rot ate the site a few months back. I should try to rewrite the front end in Flash and get it back up again someday, when I have time...
(I then ran another daemonizer for a PHP bot to manage the game itself, then had my game connect to it as a java applet, and talk to the bot to play the game, but that's irrelevant here).
Since WaveIRCd is no longer maintained, it's probably worth having a hunt around to find if anyone else has forked the project and is supporting it.
[2012 edit: that said, if you want your front end to be HTML5/Javascript, or if you want to connect through the same port that HTTP connects through, then your options are more limited than when using Flash or Java. In that case, take the advice of others, and use "WebSockets" (poor support in most current browsers) or the "Socket.io" project (which uses WebSockets, but falls back to Flash, or various other methods, depending what the browser has available).
The above is for situations where your host allows you to run a service on another port. In particular, many have explicit rules in their ToS against running an IRCd.]
[2019 edit: WebSockets are now widely supported, you should be fine using them. As a relevant case study, Slack is written in PHP (per https://slack.engineering/taking-php-seriously-cf7a60065329), and for some time supported the IRC protocol, though I believe that that has since been retired. As its main protocol, it uses an API based on JSON over WebSockets (https://api.slack.com/rtm). This all shows that a PHP IRCd can deliver enterprise-level performance and quality, even where the IRC protocol is translated to/from another one, which you'd expect to give poorer performance.]
You need to use some kind of storage as a buffer. It IS plausable not to use file or db (which also uses a file). You can try using php's shared memory functions, but I don't know any working solution so you'll have to do it from scratch.
Is it possible to do it without using a persistent data storage like
database or file?
It is possible but you shouldn't use. Database or file based doesn't slows down chat. It will be giving additional security to your chat application. You can make web based chat using ajax and sockets without persistent data.
You should see following posts:
Is database based chat room bad idea?
Will polling from a SQL DB instead of a file for chat application increase performance?
Using memcached as a database buffer for chat messages
persistent data in php question
https://stackoverflow.com/questions/6569754/how-can-i-develop-social-network-chat-without-using-a-database-for-storing-the-c
File vs database for storage efficiency in chat app
PHP is not a good fit for your requirements (in a normal setup like apache-php, fastcgi etc.), because the PHP script gets executed from top to bottom for every request and cannot maintain any state between the requests without the use of external services or databases/files (Except e.g. http://php.net/manual/de/book.apc.php, but it is not intended for implementing a chat and will not scale to multiple servers.)
You should definitely look at Node.js and especially the Node.js module Socket.IO (A Websocket library). It's incredibly easy to use and rocks. Socket.IO can also scale to multiple chat servers with an optional redis backend, which means it's easier to scale.
Trying to use $_SESSION with a static session id as communication channel is not a solution by the way, because PHP saves the session data into files.
One solution to achieving this is by writing a PHP socket server.
<?php
// Set time limit to indefinite execution
set_time_limit (0);
// Set the ip and port we will listen on
$address = '192.168.0.100';
$port = 9000;
$max_clients = 10;
// Array that will hold client information
$clients = Array();
// Create a TCP Stream socket
$sock = socket_create(AF_INET, SOCK_STREAM, 0);
// Bind the socket to an address/port
socket_bind($sock, $address, $port) or die('Could not bind to address');
// Start listening for connections
socket_listen($sock);
// Loop continuously
while (true) {
// Setup clients listen socket for reading
$read[0] = $sock;
for ($i = 0; $i < $max_clients; $i++)
{
if ($client[$i]['sock'] != null)
$read[$i + 1] = $client[$i]['sock'] ;
}
// Set up a blocking call to socket_select()
$ready = socket_select($read,null,null,null);
/* if a new connection is being made add it to the client array */
if (in_array($sock, $read)) {
for ($i = 0; $i < $max_clients; $i++)
{
if ($client[$i]['sock'] == null) {
$client[$i]['sock'] = socket_accept($sock);
break;
}
elseif ($i == $max_clients - 1)
print ("too many clients")
}
if (--$ready <= 0)
continue;
} // end if in_array
// If a client is trying to write - handle it now
for ($i = 0; $i < $max_clients; $i++) // for each client
{
if (in_array($client[$i]['sock'] , $read))
{
$input = socket_read($client[$i]['sock'] , 1024);
if ($input == null) {
// Zero length string meaning disconnected
unset($client[$i]);
}
$n = trim($input);
if ($input == 'exit') {
// requested disconnect
socket_close($client[$i]['sock']);
} elseif ($input) {
// strip white spaces and write back to user
$output = ereg_replace("[ \t\n\r]","",$input).chr(0);
socket_write($client[$i]['sock'],$output);
}
} else {
// Close the socket
socket_close($client[$i]['sock']);
unset($client[$i]);
}
}
} // end while
// Close the master sockets
socket_close($sock);
?>
You would execute this by running it through command line and would always have to run for your PHP clients to connect to it. You could then write a PHP client that would connect to the socket.
<?php
$fp = fsockopen("www.example.com", 80, $errno, $errstr, 30);
if (!$fp) {
echo "$errstr ($errno)<br />\n";
} else {
$out = "GET / HTTP/1.1\r\n";
$out .= "Host: www.example.com\r\n";
$out .= "Connection: Close\r\n\r\n";
fwrite($fp, $out);
while (!feof($fp)) {
echo fgets($fp, 128);
}
fclose($fp);
}
?>
You would have to use some type of ajax to call with jQuery posting the message to this PHP client.
http://devzone.zend.com/209/writing-socket-servers-in-php/
http://php.net/manual/en/function.fsockopen.php
Better use a node.js server for this. WebSockets aren't cross-browser nowadays (except socket.io for node.js that works perfect)
in short answer, you can't.
the current HTTP/HTML implementation doesn't support the pushstate so the algorithm of your chat app should follow :
A: sent message
B,C,D: do while a new message has been sent get this message.
so the receivers always have to make a new request and check if a new message has been sent. (AJAX Call or something similar )
so always there are a delay between the sent event and the receive event.
which means the data must be saved in something global, like db or file system.
take a look for :
http://today.java.net/article/2010/03/31/html5-server-push-technologies-part-1
You didn't say it had to all be written it PHP :)
Install RabbitMQ, and then use this chat implementation built on top of websockets and RabbitMQ.
Your PHP is pretty much just 'chat room chrome'. It's possible most of your site would fit within the 5 meg limit of offline HTML5 content, and you have a very flexible (and likely more robust than if you did it yourself) chat system.
It even has 20 messages of chat history if you leave the room.
https://github.com/videlalvaro/rabbitmq-chat
If You need to use just PHP, then You can store chat messages in session variables, session could be like object, storing a lot of information.
If You can use jQuery then You could just append paragraph to a div after message has been sent, but then if site is refreshed, messages will be gone.
Or combining, store messages in session and update that with jQuery and ajax.
Try looking into socket libraries like ZeroMQ they allow for instant transport of the message, and are quicker than TCP, and is realtime. Their infrastructure allows for instant data send between points A and B, without the data being stored anywhere first (although you can still choose to).
Here's a tutorial for a chat client in ZeroMQ
I've written a little monitoring script in PHP, which should monitor a virtual directory and it's active directories. Everything works fine but when the virtual directory service freezes is my ldap_connect() not able to connect but also doesn't get an error back. So my whole script stands still. I think that the ldap_connect function gets a timeout back (like when you try to ping an IP and it's not reachable).
That's my connect command:
$connection = ldap_connect($hostname, $port) or die("Could not connect to {$hostname});
And I haven't found something in the manual for ldap_connect() (manual) about a timelimit parameter in which you could define how long the function should try to connect until it aborts.
How ever I wasn't quite able to come up with a solution with try and catch or something like this. I also didn't wanted to use the set_time_limit() function because my script needs to be run until the end.
I appreciate every help :)
Thanks and greetings
Tim
http://www.php.net/manual/en/function.ldap-set-option.php
particular the following options :-
LDAP_OPT_NETWORK_TIMEOUT
LDAP_OPT_TIMELIMIT
http://www.php.net/manual/en/function.ldap-set-option.php
try set LDAP_OPT_REFERRALS in 0
If you don't want your PHP program to wait XXX seconds before giving up in a case when one of your corporate DC's have failed,
and since ldap_connect() does not have a mechanism to timeout on a user specified time,
this is my workaround which shows excellent practical results.
function serviceping($host, $port=389, $timeout=1)
{
$op = fsockopen($host, $port, $errno, $errstr, $timeout);
if (!$op) return 0; //DC is N/A
else {
fclose($op); //explicitly close open socket connection
return 1; //DC is up & running, we can safely connect with ldap_connect
}
}
// ##### STATIC DC LIST, if your DNS round robin is not setup
//$dclist = array('10.111.222.111', '10.111.222.100', '10.111.222.200');
// ##### DYNAMIC DC LIST, reverse DNS lookup sorted by round-robin result
$dclist = gethostbynamel('domain.name');
foreach ($dclist as $k => $dc) if (serviceping($dc) == true) break; else $dc = 0;
//after this loop, either there will be at least one DC which is available at present, or $dc would return bool false while the next line stops program from further execution
if (!$dc) exit("NO DOMAIN CONTROLLERS AVAILABLE AT PRESENT, PLEASE TRY AGAIN LATER!"); //user being notified
//now, ldap_connect would certainly connect succesfully to DC tested previously and no timeout will occur
$ldapconn = ldap_connect($dc) or die("DC N/A, PLEASE TRY AGAIN LATER.");
Also with this approach, you get a real nice fail over functionality.
Take for an example a company with a dozen of DC-a distributed along distant places.
This way your PHP program will always have high availability if at least one DC is active at present.
You'll need to use an API that supports time-outs. Connection time-outs are not supported in a native fashion by LDAP (the protocol). The timelimit is a client-requested parameter that refers to how long the directory will spend processing a search request, and is not the same as a "connect time-out".