I am designing an app that will have to periodically check a web address for updates.
The updates will be in a MySQL database tables.
I know the idea way to do this, is to create a service that is constantly running and trying to check for an update periodically (lets say 10 seconds).
Below are some questions that are unclear to me as I start my quest to accomplish this task.
Do I need to manually do the check for an update everytime from the client to server ? (that is, take a value on the client side, send it to server side, and do a head to head comparison), OR, the php/MySQL server can send a notification that there is an update that took place.
I came across a RSS feature in several posts in SO, however those tend to be NEWS applications, mine is not anything like that. Does it help to use RSS feeds in here ?
I'm planning on doing the check every 10 seconds. This means that upon the install of the app and since the app is on the device (and as long as there is internet connectivity) I will keep on fetching the server. This tends to be bandwidth and ram/cpu consuming for the client side. How do big apps like Viber, WhatsApp manage to do so ?
Just thinking out loud - , in order to avoid such a hassle, I was thinking with each update on the server, send a notification to the user with a certain code, and do the math inside the onReceive, if the code was something related to an update, 1-not show the notification to the user, 2-run the server update check thing.
ideas ?
The trouble is traditional php/html systems are client centric (the client has to initiate all communications).
You might want to look into websockets, or node.js. These technologies will allow you to create a persistent connection between the server and its clients. This in turn will allow you to 'push' data to the client when appropriate, without the client having to ask for it every x amount of minutes.
Node.js Chat Example
Node.js info
Related
I am developing an picture sharing application with messages. Now want to know how to make it instant just like Viber or whatsapp. The timer of more than a minute is not a good idea the user will have to wait for a long time for the message to arrive. And HTTP request every second will have a lots of overload on the server.
I have heard about the socket programming but not sure if it is the best way. Also how to implement that?
So the question is What is the best way of implementing this kind of application?
I am using IOS and PHP as a server language.
There are two approaches:
You can use your own protocol over any kind of socket you want (most probably a TCP or SSL stream), and then you need a server at the other end that keeps these connections open and send notifications on the right connections when something happens (new message...). There are probably existing frameworks for this, though all I can think of right now are more geared towards integration with web applications rather than native ones. Note that this will only work as long as you app is active in the foreground. This also means you will get as many simultaneous connections to your server as there are apps running, so you may have a scalability issue.
or you can use Apple Push Notifications to send notifications to the app that there is something new happening. You may include all the relevant data in the payload, or you may trigger a connection by your app to fetch the rest of the data. This will work both in the background and the foreground, though slightly differently.
You may also mix both: use APNs when your app is in the background, and you own connection when it is in the foreground.
I am building a website in PHP that handles the sessions in Redis in JSON format.
This way the session can be accessed by both the PHP interpreter and a node.js server.
What I am trying to do is now to add notifications in said website; the procedure I was thinking of is the following: (just figure it as a simple friend request to simplify it all)
user A sends friend request.
PHP uses cURL to say to node.js service to send notification
user B gets a notification because he is connected to node.js via socket.io
What are the general guidelines to achieve this? Because it looks clear to me that if user A and B are in different servers, it will be impossible to scale horizontally;
Thanks in advance.
Sounds like a you could make use of Web Sockets here with a Publication / Subscription protocol, architecture.
You get Server client functionality with web sockets.
Node is a perfect choice for a websocket server, lots of small IO.
See http://en.wikipedia.org/wiki/Web_sockets
I'm wouldn't think if the shared session is required for php - node communication, just have your clients push requests through the socket and handle the reposes as needed.
I think the approach you propose sounds quite reasonable. However, instead of doing a direct request to the service, you could consider using a message queue (zeromq, rabbitmq, whatever) which would allow you to scale it more easily as you can easily add logic to the queue processing to pass the message to the correct node instance.
I managed to get 1400 concurrent connections with socket.io on a cheap VPS with no special configuration so even with no tricks it should scale quite well. In my case most of these connections were also sending and receiving data almost constantly. It could probably have handled more than that, 1400'ish was simply the max number of users I happened to get.
Though I'd worry more about getting all those users first ;)
Use Redis's built in pub-sub capability. When the PHP server gets a request, it publishes it to a channel set up for that purpose. Each of your node servers subscribes to that channel and checks if the user involved is connected to it. If so, it sends the notification via socket.io. As a bonus, you avoid an extra network connection and your overall logic is simplified.
simply setup ur database as per required then whenever a activity is made just tell ur node js to transfer the related information through redis to php and make a process and in make a response back from php to node via channel keep checking the notification from table and show
I have read a few good articles about coding a socket server but thought I would ask here to see if there is any further knowledge/ideas about what I actually need.
I run multiple websites for clients all running off the same server, connecting to the same DB etc. Each client website has a form where users can submit their details for services we offer. These users are spread out across the world but what I am wanting to build is a monitoring system where my interface displays the users IP address, client website they are on, the page they are on etc. From the IP I will do a country/state look up (I know its not 100% accurate but close enough is good).
I would like the visiting site to send a packet to the socket server which in turns sends the output information to my screen in real time (after I perform some actions). I guess you could say I am building a mini NOC to monitor website activity. I would also like the output information to be most recent activity at top of screen but also show a scroll bar to view all activity.
Are sockets the best mechanism for this system? Any other suggestions or tutorials on how to achieve the outcome?
Many thanks.
Before you begin down this road, have you checked out the realtime part of Google Analytics? It does most of what you are looking for.
When someone is visiting your site, you aren't typically going to have a persistent connection. I'd suggest that rather than creating one, parse your server logs or store user information in a database, and query for the last x minutes of visitors.
For updating the viewing page that you are on in realtime, Web Sockets are best for this if you need very fast response time, but are currently quite the hassle to do in PHP. In addition, browser support isn't very wide. If you insist on using PHP, I'd recommend polling over AJAX. Otherwise, look into using Node.JS with Socket.IO. Socket.IO wraps up a lot of similar methods to web sockets to get the same effect with little effort. Still use PHP for your application... just use the Node.JS/Socket.IO part for your monitor interface.
Finally, I'd suggest questioning again why you might want this. You can spend a lot of time on a project like this, and the truth is that your analytics data over time is far more valuable than a snapshot when you are looking at it.
I am creating a web app that requires a live notification system. How would I set up my server to pull data from a mySQL database and then push it to the browser. I have absolutely NO idea how to do this. If anybody can help, it would be much appreciated! Thank you so much!
EDIT: I should probably be more specific, I am pulling data as in XYZ recently created an account, XZY recently ... Thanks so much!
You cannot push data to a browser, but what you can do is set up your webpage to poll your server every few seconds for updates. An example setup would be:
From within your website, have a javascript function that runs on a timer every few seconds (or whatever interval works best for your situation).
Start that timer on page load.
That javascript function invokes an AJAX call to a web service on your web server (more on that in a second).
On the server side you'll need some sort of system that tracks these events and stores them somewhere such as in a database table with a timestamp. So for example when XYZ creates an account, that would be logged in this "event" table in the db.
The web service called by the AJAX call will then run a query on that table and retrieve all entries since the last time it was called. Then just update the webpage with those results.
It's obviously not 100% "live" as there will be a small delay depending on what time interval you set in the JS timer but it's pretty close.
You can create a push notification service for your website utilizing websockets and graceful degradation to a long polling fallback for browsers that don't support websockets. This does require a healthy amount of technical/programming knowledge.
Some good resources for this are:
http://socket.io (which uses a node.js backend for the web sockets and handles degradation)
http://pusherapp.com (a commercial solution if you don't want to roll and maintain your own service)
For a list of browsers that support websockets, you can search for "caniuse" -- which provides great details to features supported by browser versions
Note: For multi-million user applications like Facebook, I would assume they've weighed the advantages of running websockets for 50+ million simultaneous users and concluded that keeping data consistent across all the nodes connected to the millions of sockets would be too much. I can imagine it would be a logic nightmare to do that on a socket system instead of a basic action sql-like infrastructure. I can't speak on their behalf though, this is simply my assumption. However, you'd be surprised how many sites have been using push and polling systems lately :)
The challenge I'm facing is simulating a communication channel between two users of a website (e.g. gaming site), by using solely the technologies mentioned in the title.
Recently I've developed an online chess website where, the idea behind which was to give users the chance to play "live" matches too, not just lengthy games where you would make a move, then come back in 1-15 days to see if your opponent has responded. And the way this game engine works implies sending asynchronous requests to the server, both to update the info related to the game (in case you make a move), but also to verify if anything has changed (if you are waiting for the opponent to move).
To better explain this... the player whose turn it is (the browser of course) sends an async. request to update the game info, exactly when he makes his move. Meanwhile, the opponent sends PERIODIC requests, "asking" if anything has changed. If anything does change, the roles switch after the board updates.
Timers are behind the functionality of the engine, so my question to you is this: How would you go about simulating a com channel between two players, while trying not to put too much stress on the server, but also having games update as fast as possible, in order to maintain the "live" feeling to it. This is most important in 1 minute games (one of the available categories). Because in that case, requests NEED to take place VERY often (at least 1 second). But server responses could delay, there would be much stress on the server when having hundreds of games available at the same time etc... so you see my problem.
I look forward to hearing from you guys and picking your brain if you have any good ideas :)
Cheers!
Andrei
PS: In case you want to try it, the name is e-chess960.com
you would want to builds a socket server. node.js would be a good javascript based library to use for a socket server. each client would create a socket connection to the socket server, then when a client sends a message to the socket server, the socket server could immediately send the message back out to its subscribers without having to store it anywhere.
socket servers require socket access, which usually requires a vps instead of a shared server.
to make the solution work, you would require that the clients have the ability to create socket connections to your server. some browsers already can as part of html5, but not all yet. you could also use Flash as a way to create a socket connection.
What you actually want is not pulling the server every second, but instead keep a connection open and use the observer-pattern to distribute updates to the specific clients (in other words: push instead of pull). That way, your server resources will not get swamped by a new connection for every second and every client. A library to accomplish that is the Comet Library. Check out this nice tutorial for a sample application.