I'm trying to make my PHP server a bit more efficient.
I've built an object named Client which contains the connected client (which has an open socket connection with the server) information such as name, id etc.
For now I have one array of socket connections, and one array of Client objects.
When I'm referring a connection, I'm searching inside my Client array to find the right client who matches this connection.
It works great, but it's a bit inefficient.. For small amount of clients in the server you don't feel it, but I'm afraid that if I'll have thousands of connection it will slow down the server.
As a solution I thought about 2 dimensional array, but I have a logic problem designing it.
Can I do something like this:
$clients = array();
$temp = array($newsock, new Client());
$clients[] = $temp;
I want my $clients[] to be the socket and the $clients[][] to be the client object.
In each row of $client I will have only $client[$index][0] which will be my client object for that connection.
Will I be able to send this to the socket_select() function?
You say that you have within your client object an id attribute. Why not use that id as the key for both arrays?
Socket connections array
Client object array
You might even be able to hold the connection and the client object in one array, each in one object under the same key I talked about before - the clients id.
In any case, wherever you decide to store your clients connection object, you will be able to pass it to all the relevant socket functions -
socket_select();
socket_accept();
socket_write();
etc...
With regard to the efficiency of your server, I implemented some forking for broadcasting data to large amounts of clients (all of them in the example of a chat server).
This is the implementation that I used for forking the broadcasts -
function broadcastData($socketArray, $data){
global $db;
$pid = pcntl_fork();
if($pid == -1) {
// Something went wrong (handle errors here)
// Log error, email the admin, pull emergency stop, etc...
echo "Could not fork()!!";
} elseif($pid == 0) {
// This part is only executed in the child
foreach($socketArray AS $socket) {
// There's more happening here but the essence is this
socket_write($socket,$msg,strlen($msg));
// TODO : Consider additional forking here for each client.
}
// This is where the signal is fired
exit(0);
}
// The child process is now occupying the same database
// connection as its parent (in my case mysql). We have to
// reinitialize the parent's DB connection in order to continue using it.
$db = dbEngine::factory(_dbEngine);
}
The code above was lifted from a previous question of mine (that was self answered).
Terminating zombie child processes forked from socket server
Perhaps it might assist you if you chose to start forking processes.
Related
I am working on a browser/mobile game and I am trying to build a system that automatically ends queued tasks after a certain time has passed. It's the basic research schema used in most games.
Research A costs $100 and will take 1 hour to complete. Do I have to check every second for tasks that are at or past their completion time and trigger an event to clear them and increment the level number? Is there a better way or more optimum way? This idea works by itself but what happens if you need to run 5 or 6 different queues in the game design? show I abstract them enough to get them all in one table?
I apologize if I seem a little vague or erratic with my questions. I am trying to figure out where to start with this concept.
I'm not very familiar with it, but I believe you could use websockets or NodeJS to create a callback event, you could then call that callback with a PHP socket server. This kind of
You can base yourself off this tutorial: http://www.sanwebe.com/2013/05/chat-using-websocket-php-socket
Steps
First, identify the message type using the websocket.onmessage callback, something similar to this should work:
websockets.onmessage = function(ev)
{
var msg = JSON.parse(ev.data); //Assuming you'll encode the message components in JSON with PHP
if ( msg.type == "research_end" )
{
FinishResearch(msg.content); //Assuming that the content element of the JSON array contains the ID of the research
}
}
Secondly, make the server send the actual message. To not make this too complicated or long I'll just pretend that sendMessage($msg, $client) is a function that sends a message to a client.
However, as explained in the tutorial, each client socket is stored in an array called $clients, you'll have to add some kind of identifier to each research so it's easy to know which research belongs to what client.
Now, here's an important part. On the server there will be a variable called $research which will be structured as such:
$research['peername'][0]['time'] = 60000
$research['peername'][0]['type'] = 20
You can add the research by sending an outgoing message to the websocket server by using this:
var array = {message: '20', type: 'research', time : '300000'}; //Create the request array
websocket.send(JSON.stringify(msg)); //Send it to the socket server as a json string, decode with json_decode once it arrives
Then, when it gets to the server and is identified as a research request, we call a callback called doResearch which takes two arguments
//loop through all connected sockets
foreach ($changed as $changed_socket) {
//check for any incomming data
while(socket_recv($changed_socket, $buf, 1024, 0) >= 1)
{
$received_text = unmask($buf); // Unmask data
$msg_array = json_decode($received_text); // Decode the JSON string we sent to the server
doResearch($msg_array, $changed_socket); // Let's say this function contains all the procedures to do the research
}
}
doResearch would be similar to this:
function doResearch($msg_array, $socket)
{
$name = socket_getpeername($socket, $addr);
$count = count($research[$name]);
$research[$name][$count]['time'] = $msg_array['time'];
$research[$name][$count]['type'] = $msg_array['type'];
}
And finally, you would have to add a conditional like this inside the main server loop:
foreach ( $research as $i )
{
foreach ( $i as $i2 )
{
if ( time() <= $i2['time'] )
{
$sql->query("INSERT INTO researches('peer', 'researchid') VALUES ('".$i."', '".$i2['type']."')");
sendMessage('Research type '.$i2['type'].' has finished.', $i2['socket']);
}
}
}
Then, that would check if a research has been finished and insert it into the database.
Hope this helps.
I found this SMPP Transceiver implementation:
http://sourceforge.net/projects/php-smppv3-4/files/
The way I use smpp_transceiver.php, is the following:
File send.php
require_once "smpp_transceiver.php";
...
// Open socket
$tx = new SMPP('SOME IP HERE', $port); // (1)
$tx->debug = false;
$tx->system_type = $systemType;
$tx->addr_npi = 1;
// Login as transmitter
$bindResult = $tx->bindTransmitter($username, $password);
$tx->sms_source_addr_npi = 1;
$tx->sms_source_addr_ton = 0;
$tx->sms_dest_addr_ton = 0;
$tx->sms_dest_addr_npi = 1;
// Send SMS
$sendResult = $tx->sendSMS($from, $to, $msg);
// Close socket
$tx->close(); // (2)
$state2 = $tx->state;
// Delete object
unset($tx);
Very simple question:
Is it okay to constantly open (1) and close (2) sockets?
This send.php is supposed to act as a webservice.
So I will be calling this many consecutive times:
http://...../send.php?mobile=......&body=hey
http://...../send.php?mobile=......&body=blah
http://...../send.php?mobile=......&body=zort
http://...../send.php?mobile=......&body=troz
I was told that SMPP connection should be kept alive, and this is clearly not happening here.
So, two more questions:
How can I keep the connection alive? given that this is PHP and smpp_transceiver.php is a non static class. I want that every call to send.php uses the same socket connection.
Should I implement some kind of synchronized lock(o) to smpp_transceiver.php if the previous thing is not possible?
You should keep the connection alive when possible, but it's also a question of how frequently is frequent.
If "frequent" is a few times a minute, then, it's not the end of the world and carry on.
If "frequent" is a few times a second, then you may want to seek another approach for the actual SMPP portion. PHP isn't a great choice for services that need to be kept alive for long periods of time. Try python, node or ruby.
I've got a Minecraft Software written in C# that I want to send a heartbeat to my site. I've got the way to send the beat already written.
if (Server.Uri == null) return;
string uri = "http://GemsCraft.comli.com/Heartbeat.php";
// create a request
try
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
request.Method = "POST";
// turn request string into a byte stream
byte[] postBytes = Encoding.ASCII.GetBytes(string.Format("ServerName={0}&Url={1}&Players={2}&MaxPlayers={3}&Uptime={4}",
Uri.EscapeDataString(ConfigKey.ServerName.GetString()),
Server.Uri,
Server.Players.Length,
ConfigKey.MaxPlayers.GetInt(),
DateTime.UtcNow.Subtract(Server.StartTime).TotalMinutes));
request.ContentType = "application/x-www-form-urlencoded";
request.CachePolicy = new System.Net.Cache.RequestCachePolicy(System.Net.Cache.RequestCacheLevel.NoCacheNoStore);
request.ContentLength = postBytes.Length;
request.Timeout = 5000;
Stream requestStream = request.GetRequestStream();
// send it
requestStream.Write(postBytes, 0, postBytes.Length);
requestStream.Flush();
requestStream.Close();
/* try
{
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Logger.LogToConsole(new StreamReader(response.GetResponseStream()).ReadToEnd());
Logger.LogToConsole(response.StatusCode + "\n");
}
catch (Exception ex)
{
Logger.LogToConsole("" + ex);
}*/
}
Now, I want to be able to retrieve the heartbeat in PHP, upload it to the SQL database, and then display each user's server in a table that will be displayed on the webpage
How do I do this?
portforwardpodcast's answer isn't very well-suited for your purposes, here's a process for you to ponder
Server accesses the following page: heartbeat.php?port=25565&maxplayers=25&players=2&name=Cheese_Pizza_Palace
Your PHP script will then do the following...
Go through each value, making sure they're all the types you want them to be (integers/strings)
Connect to the database
Update the server in the database if it already exists, create it if it doesn't
Return some value so the server knows that it completed successfully.
And to display the servers
Fetch all 'active' servers
Loop through them and display each one.
Things you'll need to figure out:
How to determine uptime
How to determine "active" servers
How to update/create MySQL entries
How to (properly) connect to a database. I would suggest using PDO since you're using PHP. It's a bit difficult to learn, but it's much more secure than writing the queries directly.
How to loop through all the GET variables.
Good hunting!
I would create a simple php page accept a get variable. something like www.site.com/beat.php?lasttime=123456&serverid=1 where the number us the unix timestamp. Then you need to re-work your c# to do a simple get request on a website. Finally your php should insert into a mysql table with a column for id, timestamp, server_id etc.
First you need to pull the data from the request. The $_REQUEST variable in php is nice because it works for both GET and POST:
http://php.net/manual/en/reserved.variables.request.php
Start out by var_dump or echo the fields you want. Once you can get the needed data into variables you are done with the first part. For the next part you need to create a database and table in MySQL. The best tool for this is phpmyadmin. If you have a host like godaddy (or some others) you can get at this from the control panel. If not you may need to install upload the phpmyadmin files yourself. It's a pretty simple tool to use:
http://www.youtube.com/watch?v=xxQSFHADUIY
Once your database has the correct columns, you need to insert the data from your php file. This page should help:
http://www.w3schools.com/php/php_mysql_insert.asp
Note: This is not the same as this question which utilises MessageComponentInterface. I am using WampServerInterface instead, so this question pertains to that part specifically. I need an answer with code examples and an explanation, as I can see this being helpful to others in the future.
Attempting looped pushes for individual users
I'm using the WAMP part of Ratchet and ZeroMQ, and I currently have a working version of the push integration tutorial.
I'm attempting to perform the following:
The zeromq server is up and running, ready to log subscribers and unsubscribers
A user connects in their browser over the websocket protocol
A loop is started which sends data to the specific user who requested it
When the user disconnects, the loop for that user's data is stopped
I have points (1) and (2) working, however the issue I have is with the third one:
Firstly: How can I send data to each specific user only? Broadcast sends it to everyone, unless maybe the 'topics' end up being individual user IDs maybe?
Secondly: I have a big security issue. If I'm sending which user ID wants to subscribe from the client-side, which it seems like I need to, then the user could just change the variable to another user's ID and their data is returned instead.
Thirdly: I'm having to run a separate php script containing the code for zeromq to start the actual looping. I'm not sure this is the best way to do this and I would rather having this working completely within the codebase as opposed to a separate php file. This is a major area I need sorted.
The following code shows what I currently have.
The server that just runs from console
I literally type php bin/push-server.php to run this. Subscriptions and un-subscriptions are output to this terminal for debugging purposes.
$loop = React\EventLoop\Factory::create();
$pusher = Pusher;
$context = new React\ZMQ\Context($loop);
$pull = $context->getSocket(ZMQ::SOCKET_PULL);
$pull->bind('tcp://127.0.0.1:5555');
$pull->on('message', array($pusher, 'onMessage'));
$webSock = new React\Socket\Server($loop);
$webSock->listen(8080, '0.0.0.0'); // Binding to 0.0.0.0 means remotes can connect
$webServer = new Ratchet\Server\IoServer(
new Ratchet\WebSocket\WsServer(
new Ratchet\Wamp\WampServer(
$pusher
)
),
$webSock
);
$loop->run();
The Pusher that sends out data over websockets
I've omitted the useless stuff and concentrated on the onMessage() and onSubscribe() methods.
public function onSubscribe(ConnectionInterface $conn, $topic)
{
$subject = $topic->getId();
$ip = $conn->remoteAddress;
if (!array_key_exists($subject, $this->subscribedTopics))
{
$this->subscribedTopics[$subject] = $topic;
}
$this->clients[] = $conn->resourceId;
echo sprintf("New Connection: %s" . PHP_EOL, $conn->remoteAddress);
}
public function onMessage($entry) {
$entryData = json_decode($entry, true);
var_dump($entryData);
if (!array_key_exists($entryData['topic'], $this->subscribedTopics)) {
return;
}
$topic = $this->subscribedTopics[$entryData['topic']];
// This sends out everything to multiple users, not what I want!!
// I can't send() to individual connections from here I don't think :S
$topic->broadcast($entryData);
}
The script to start using the above Pusher code in a loop
This is my issue - this is a separate php file that hopefully may be integrated into other code in the future, but currently I'm not sure how to use this properly. Do I grab the user's ID from the session? I still need to send it from client-side...
// Thought sessions might work here but they don't work for subscription
session_start();
$userId = $_SESSION['userId'];
$loop = React\EventLoop\Factory::create();
$context = new ZMQContext();
$socket = $context->getSocket(ZMQ::SOCKET_PUSH, 'my pusher');
$socket->connect("tcp://localhost:5555");
$i = 0;
$loop->addPeriodicTimer(4, function() use ($socket, $loop, $userId, &$i) {
$entryData = array(
'topic' => 'subscriptionTopicHere',
'userId' => $userId
);
$i++;
// So it doesn't go on infinitely if run from browser
if ($i >= 3)
{
$loop->stop();
}
// Send stuff to the queue
$socket->send(json_encode($entryData));
});
Finally, the client-side js to subscribe with
$(document).ready(function() {
var conn = new ab.Session(
'ws://localhost:8080'
, function() {
conn.subscribe('topicHere', function(topic, data) {
console.log(topic);
console.log(data);
});
}
, function() {
console.warn('WebSocket connection closed');
}
, {
'skipSubprotocolCheck': true
}
);
});
Conclusion
The above is working, but I really need to figure out the following:
How can I send individual messages to individual users? When they visit the page that starts the websocket connection in JS, should I also be starting the script that shoves stuff into the queue in PHP (the zeromq)? That's what I'm currently doing manually, and it just feels wrong.
When subscribing a user from JS, it can't be safe to grab the users id from the session and send that from client-side. This could be faked. Please tell me there is an easier way, and if so, how?
Note: My answer here does not include references to ZeroMQ, as I am not using it any more. However, I'm sure you will be able to figure out how to use ZeroMQ with this answer if you need to.
Use JSON
First and foremost, the Websocket RFC and WAMP Spec state that the topic to subscribe to must be a string. I'm cheating a little here, but I'm still adhering to the spec: I'm passing JSON through instead.
{
"topic": "subject here",
"userId": "1",
"token": "dsah9273bui3f92h3r83f82h3"
}
JSON is still a string, but it allows me to pass through more data in place of the "topic", and it's simple for PHP to do a json_decode() on the other end. Of course, you should validate that you actually receive JSON, but that's up to your implementation.
So what am I passing through here, and why?
Topic
The topic is the subject the user is subscribing to. You use this to decide what data you pass back to the user.
UserId
Obviously the ID of the user. You must verify that this user exists and is allowed to subscribe, using the next part:
Token
This should be a one use randomly generated token, generated in your PHP, and passed to a JavaScript variable. When I say "one use", I mean every time you reload the page (and, by extension, on every HTTP request), your JavaScript variable should have a new token in there. This token should be stored in the database against the User's ID.
Then, once a websocket request is made, you match the token and user id to those in the database to make sure the user is indeed who they say they are, and they haven't been messing around with the JS variables.
Note: In your event handler, you can use $conn->remoteAddress to get the IP of the connection, so if someone is trying to connect maliciously, you can block them (log them or something).
Why does this work?
It works because every time a new connection comes through, the unique token ensures that no user will have access to anyone else's subscription data.
The Server
Here's what I am using for running the loop and event handler. I am creating the loop, doing all the decorator style object creation, and passing in my EventHandler (which I'll come to soon) with the loop in there too.
$loop = Factory::create();
new IoServer(
new WsServer(
new WampServer(
new EventHandler($loop) // This is my class. Pass in the loop!
)
),
$webSock
);
$loop->run();
The Event Handler
class EventHandler implements WampServerInterface, MessageComponentInterface
{
/**
* #var \React\EventLoop\LoopInterface
*/
private $loop;
/**
* #var array List of connected clients
*/
private $clients;
/**
* Pass in the react event loop here
*/
public function __construct(LoopInterface $loop)
{
$this->loop = $loop;
}
/**
* A user connects, we store the connection by the unique resource id
*/
public function onOpen(ConnectionInterface $conn)
{
$this->clients[$conn->resourceId]['conn'] = $conn;
}
/**
* A user subscribes. The JSON is in $subscription->getId()
*/
public function onSubscribe(ConnectionInterface $conn, $subscription)
{
// This is the JSON passed in from your JavaScript
// Obviously you need to validate it's JSON and expected data etc...
$data = json_decode(subscription->getId());
// Validate the users id and token together against the db values
// Now, let's subscribe this user only
// 5 = the interval, in seconds
$timer = $this->loop->addPeriodicTimer(5, function() use ($subscription) {
$data = "whatever data you want to broadcast";
return $subscription->broadcast(json_encode($data));
});
// Store the timer against that user's connection resource Id
$this->clients[$conn->resourceId]['timer'] = $timer;
}
public function onClose(ConnectionInterface $conn)
{
// There might be a connection without a timer
// So make sure there is one before trying to cancel it!
if (isset($this->clients[$conn->resourceId]['timer']))
{
if ($this->clients[$conn->resourceId]['timer'] instanceof TimerInterface)
{
$this->loop->cancelTimer($this->clients[$conn->resourceId]['timer']);
}
}
unset($this->clients[$conn->resourceId]);
}
/** Implement all the extra methods the interfaces say that you must use **/
}
That's basically it. The main points here are:
Unique token, userid and connection id provide the unique combination required to ensure that one user can't see another user's data.
Unique token means that if the same user opens another page and requests to subscribe, they'll have their own connection id + token combo so the same user won't have double the subscriptions on the same page (basically, each connection has it's own individual data).
Extension
You should be ensuring all data is validated and not a hack attempt before you do anything with it. Log all connection attempts using something like Monolog, and set up e-mail forwarding if any critical's occur (like the server stops working because someone is being a bastard and attempting to hack your server).
Closing Points
Validate Everything. I can't stress this enough. Your unique token that changes on every request is important.
Remember, if you re-generate the token on every HTTP request, and you make a POST request before attempting to connect via websockets, you'll have to pass back the re-generated token to your JavaScript before trying to connect (otherwise your token will be invalid).
Log everything. Keep a record of everyone that connects, asks for what topic, and disconnects. Monolog is great for this.
To send to specific users, you need a ROUTER-DEALER pattern instead of PUB-SUB. This is explained in the Guide, in chapter 3. Security, if you're using ZMQ v4.0, is handled at the wire level, so you don't see it in the application. It still requires some work, unless you use the CZMQ binding, which provides an authentication framework (zauth).
Basically, to authenticate, you install a handler on inproc://zeromq.zap.01, and respond to requests over that socket. Google ZeroMQ ZAP for the RFC; there is also a test case in the core libzmq/tests/test_security_curve.cpp program.
I have a script that is running on a shared hosting environment where I can't change the available amount of PHP memory. The script is consuming a web service via soap. I can't get all my data at once or else it runs out of memory so I have had some success with caching the data locally in a mysql database so that subsequent queries are faster.
Basically instead of querying the web service for 5 months of data I am querying it 1 month at a time and storing that in the mysql table and retrieving the next month etc. This usually works but I sometimes still run out of memory.
my basic code logic is like this:
connect to web service using soap;
connect to mysql database
query web service and store result in variable $results;
dump $results into mysql table
repeat steps 3 and 4 for each month of data
the same variables are used in each iteration so I would assume that each batch of results from the web service would overwrite the previous in memory? I tried using unset($results) in between iterations but that didn't do anything. I am outputting the memory used with memory_get_usage(true) each time and with every iteration the memory used is increased.
Any ideas how I can fix this memory leak? If I wasn't clear enough leave a comment and I can provide more details. Thanks!
***EDIT
Here is some code (I am using nusoap not the php5 native soap client if that makes a difference):
$startingDate = strtotime("3/1/2011");
$endingDate = strtotime("7/31/2011");
// connect to database
mysql_connect("dbhost.com", "dbusername" "dbpassword");
mysql_select_db("dbname");
// configure nusoap
$serverpath ='http://path.to/wsdl';
$client = new nusoap_client($serverpath);
// cache soap results locally
while($startingDate<=$endingDate) {
$sql = "SELECT * FROM table WHERE date >= ".date('Y-m-d', $startingDate)." AND date <= ".date('Y-m-d', strtotime($startingDate.' +1 month'));
$soapResult = $client->call('SelectData', $sql);
foreach($soapResult['SelectDateResult']['Result']['Row'] as $row) {
foreach($row as &$data) {
$data = mysql_real_escape_string($data);
}
$sql = "INSERT INTO table VALUES('".$row['dataOne']."', '".$row['dataTwo']."', '".$row['dataThree'].")";
$mysqlResults = mysql_query($sql);
}
$startingDate = strtotime($startingDate." +1 month");
echo memory_get_usage(true); // MEMORY INCREASES EACH ITERATION
}
Solved it. At least partially. There is a memory leak using nusoap. Nusoap writes a debug log to a $GLOBALS variable. Altering this line in nusoap.php freed up a lot of memory.
change
$GLOBALS['_transient']['static']['nusoap_base']->globalDebugLevel = 9;
to
$GLOBALS['_transient']['static']['nusoap_base']->globalDebugLevel = 0;
I'd prefer to just use php5's native soap client but I'm getting strange results that I believe are specific to the webservice I am trying to consume. If anyone is familiar with using php5's soap client with www.mindbodyonline.com 's SOAP API let me know.
Have you tried unset() on $startingDate and mysql_free_result() for $mysqlResults?
Also SELECT * is frowned upon even if that's not the problem here.
EDIT: Also free the SOAP result too, perhaps. Some simple stuff to begin with to see if that helps.