I am building a Mobile Application (Phone, Jquery Mobile) for Android.
It requires a user sending a message to the server side script using an Ajax request. I implemented a system such that messages sent are not sent immediately, instead they are stored in a SQLite database (this is so because, I want the user to think the message is sent even if the network is bad) and an iteration that runs every 15 seconds in the background picks it up and send it. So therefore, if 5 messages are in the SQLite database, they will be sent every 15 seconds apart.
The above system works fine when I tested it with my Android phone connected to my WAMP via Wi-Fi.
The problem I now foresee is that when I deploy on a production Server, the Ajax response from the server won't be that fast.
Is there a way to avoid a potential problem?
Note: The response from server is via json and it is essential because it will be used to delete the message from the phone SQlite Database.
Here is how I read your question:
A user interacts with the app and this generates actions that need to be updated on your backend. (for example, "like a post", "upvote a coment" etc.)
You have decided that the action does not need to complete synchronously and that the user does not need to wait around for the action to be completed/acknowledged; The user can continue on with new actions. At some point, the user will see that the actions have been processed but immediate updates aren't important.
Your polling loop in the app is responsible for eventually sending the actions to the backend to be completed. The backend acknowledges the action has been completed by sending a response back to the app to delete the pending action.
My answer based on above:
Since it is the app waiting on your custom protocol to complete and not the user waiting on a synchronous action to complete, you have a lot of flexibility in the server's response times.
More important is that your little protocol makes sure to eventually process the action on the backend and notify the client app. You are dealing with the concept of things becoming "eventually consistent" and thus you need to design your little protocol around that:
http://en.wikipedia.org/wiki/Eventual_consistency
Related
I've implemented realtime chat in Websockets and it works just like a charm. I did it side by side with AJAX polling (which existed before), as I didn't want to remove AJAX polling support for older browsers, so although Websockets will be preferred, I am keeping polling as an option.
The way I have the websocket code set up is to:
Use Websockets to send the messages to the appropriate connections
Log the message in the database
The way the AJAX send works is:
Log the message in the database
The way the AJAX poll works is:
Check when user last retrieved messages for a room, and retrieve all messages since then
I have more of a conceptual question about how to get the two to work with each other. Assume I have User A and User B. If User A and User B are either both using AJAX polling or both using websockets, there are no issues whatsoever. It works as you would expect. But take the case when User A is using websockets and User B is using AJAX polling.
A → B
User A sends message via websocket
Chat message gets sent to all relevant connections. In this case, it just echoes back to User A, since User B is not using websockets.
Chat messages gets logged into database
User B requests new messages, and User A's message is sent back to him
B → A
User B sends message via AJAX
Message gets saved in database
?
This is what I'm hung up on. Websocket -> AJAX user messages work because the message ends up in the database where it can be polled by an AJAX user. But websocket users don't poll the database at all, so messages from AJAX users have no way of getting into the pipeline. Currently, websocket users don't see any messages from AJAX users. They only see them if they do a full page reload, at which point all messages get retrieved directly from the database.
What would be the appropriate method to allow some way for messages from AJAX users to get sent out via websocket to relevant websocket users? Basically, how can I communicate in the other direction?
The only thing I've been able to find on this subject is this slideshow - however, I'm not using longpolling, and I'm not entirely sure what it means by "providing that" to the websocket app. Does that mean put the onus on the websocket server to check for new AJAX messages? Is there no way to "push" from the AJAX script to the websocket server?
Thanks to ADyson for some ideas on how to approach this. This is what I ended up doing:
I have my JS clients ping the server every few seconds. For AJAX users, it's more frequent because they need to poll for messages. For Websocket users, they don't need to poll for websocket messages, so they poll every 15s to "check in". This has the benefit of functioning as an infinite loop essentially for each client, which is exactly what the idea requires.
What I did was essentially add a column to the messages table that keeps track of the message source: 0 for AJAX and 1 for WebSocket. Then, I modified the function that retrieves messages to take in an $ajaxOnly parameter. When I call this on an AJAX poll, it's false. For websocket polls, I call it with true instead. The result is for all rooms a user is in, it polls the DB and checks if there are any new AJAX messages. If there are, it sends them back to the client.
There are 2 caveats:
Messages will appear out of order. Websocket messages are relayed in realtime, so an AJAX message sent before a Websocket message could appear earlier to another AJAX user but later to other Websocket users.
This is not realtime. It's as slow as your poll interval, which in my case is 15 seconds for websockets since polls are very DB-intensive. AJAX polls are more frequent because they need to be to get any messages at all.
This isn't a perfect solution, but it does meet the goal of allowing AJAX messages to show up for websocket users in near-realtime. For me, I wanted to fully support AJAX and WebSocket, but AJAX is more for compatability and ideally most people will use WebSockets, so the fact that this isn't a great solution doesn't concern me too much, since it does get the job done.
If you were hoping for a realtime solution (like I was), you're going to be disappointed by this, but this is at least a workable solution so I've adopted it until something better comes up. You can make it near-realtime by increasing the frequency with which clients ping your server.
Simple solution:
When you save the message into the Datebase, push it to the Websocket Clients as well.
I am currently creating a stock market simulation and am working on the moment that the user logs into the simulation. I have a PHP script that will generate a certain price for a company four times and update it into my MySQL database while running. I currently have the following code:
PHP:
if (isset($_SESSION['userId']))
{
$isPlaying = 0;
while ($isPlaying <= 3)
{
$priceTemp = (rand(3300, 3700) / 100);
$sql = "UPDATE pricestemp SET price = $priceTemp WHERE companyName = 'Bawden';";
mysqli_query($conn, $sql);
sleep(1);
$isPlaying++;
}
echo '<h1>Welcome to the simulation</h1>';
}
I am aiming for these updates to happen in the background once the user has logged into the simulation. When refreshing my database every second, the updated prices are shown which is one of my objectives. However, what I would like it to do is still load the HTML onto the page (to say "Welcome to the simulation") while updating the database with every second with an updated price.
So far, when I log in, I have to wait 4 seconds before the HTML will load. In the future, I hope to have it consisently updating until a certain condition is met but when I have set an infinite loop earlier the HTML never loaded.
What do I have to do to allow the HTML to load once logged in and have the prices being generated and updated in the MySQL database in the background with no delay in either of these tasks happening?
You have a fundamental misunderstanding of how web-based requests work.
What you need to understand is that PHP is a server-side language. PHP generates any combination of HTML, CSS, JavaScript, JSON, or any other forms of data you want and sends it to your web browser when it's finished. While it's doing that, it can also manage data within a database or perform any other number of actions, but it will never send what the web browser can make use of until it finishes setting everything up. So if you're within an infinite loop, it will never finish and therefore nothing will be sent back to the web browser.
To remedy this, you need to use something called "asynchronous JavaScript", more commonly referred to as "ajax". Specifically, you first send some initial HTML to the web browser in one request and let the request end immediately. This allows the user to see something without waiting around for an indefinite period of time. Then, on the web browser end, you can use JavaScript to automatically send a second request to the server. During this second request to the server, you can perform your data processing and send back some data when you're finished to display to the user.
If you want to periodically update what you show the user, then you would repeat that second request to refresh what is shown on the user's webpage.
Any time you see some kind of "real-time" updating on a website, it's not coming from a single, persistently open connection to the web server--it's actually a series of repeated, broken up requests that periodically refresh what you see.
Broken down, standard web request workflows look something like this:
Web browser asks the web server for the webpage. Web browser waits for a reply.
Web server generates the webpage and sends the webpage to the web browser. Web server is done.
Web browser receives the webpage and shows it to the user. Web browser stops waiting for a reply.
Web browser runs any JavaScript it needs to run and requests data from the web server. Web browsers waits for a reply.
Web server processes the request and sends the requested data back to the web browser. Web server is done.
Web browser receives the requested data and updates the HTML on the webpage so the user can see it. Web browser stops waiting for a reply.
As you can see, each series of requests is 1) initiated by the web browser, 2) processed by the web server, and 3) any replies from the web server are then handled by the web browser after the web server is finished up. So each and every request goes browser -> server -> browser. If we add steps 7., 8., and 9. to the above, we will see them repeat the exact same pattern.
If you want to avoid adding JavaScript into the mix, preferring to refresh the entire page every time, then keep your data processing short. Optimize your database calls, fix your infrastructure (make sure your server and database have a LAN connection, that your hardware is good enough, etc.), make your code more efficient... do whatever you need to do to keep the processing time to a minimum.
This is all incredibly simplified and not 100% accurate, but should hopefully help you with your specific problem. The short version of all of this is: you can't show your HTML and process your data at the same time the way you're doing things now. You need to fundamentally change your workflow.
You have to do this in 2 network calls. The first network call should fetch the html. Then you have to use Javascript to fire another call to update your data. Once that api call returns it will update the html.
The scheduling model to manage the frequency of a background operation based on the frequency of requests at the front end is a very difficult problem. It's also a problem you don't need to solve. The data doesn't need to be changed when nobody is looking at it. You just need to store when the data was last looked at and apply greater deltas to older data.
I have an php framework which supports me building websites and allows me to edit the content directly in the frontend via inline editing. I have kind of an api, which receives the updates via an ajax call.
So far i was the only one using it, but in the future there could me more people editing the pages the same time, so the page has to be locked for other users the moment someone starts editing (maybe displaying which people are viewing the page at the moment as well).
I have some experience in php, javascript and a rough understanding of http requests, node.js and so on, but I am not an expert.
Now I would think, that i simply had to setup up a websocket server, tell the websocket server, when a user is viewing and editing a single page and update the page for the other clients and lock the editing buttons via javascript. So the Page would be only marked as locked on the websocket server.
Maybe I could check the status of the page when updating the content of a page by sending another request to the websocket server, if the page is locked. Or should i update the database directly through the websocketserver and mark the page as locked in the database?
Can you tell me if I'am on the right track or if it's a completely wrong approach?
PS: Even if it may be an overkill I still would want to try it only to practice, as I haven't used this technology so far :)
The approach that I would take is as follows:
Problem: (restating to show my understanding)
Notify clients when content is being edited to prevent conflicts.
Hard requirements:
Implemented using WebSockets -- for educational purposes.
Assumptions:
Sending the edited content will continue to be done via AJAX calls.
Clients do not need to know about content on pages that they're not currently viewing.
Clients should learn about the lock status of content once the page is loaded, before (or at least as soon as) the ability to edit content is available to the user.
It may become a requirement for all clients to be notified that contents on a page has been updated so that they can request an updated version through AJAX calls.
Multiple pieces of content may potentially be edited on a page, and locks should only apply on a per-content chunk basis, not a per-page basis. (I.e., a page that lists 10 customer addresses, if editing 1 address, let the other 9 be available for others to edit.)
Approach:
Personally, I'd use a PHP-based WebSockets server, but that's because I'm biased towards the one I wrote. Because of that, I'm going to approach this answer from a PHP-WebSockets perspective, and I apologize for any server specific implementation details that might not translate. That being said, it is far more important for you to use the tools that you're most comfortable with than to use a tool that I recommend, and I am trying to write in as general terms as possible.
Client JS:
On connecting, send the URL of the page that is loaded.
On initiating and completing (committing or aborting) edits, send a message indicating which bit of content on that page to lock/unlock.
Client may request the lock status of any bit of content at any time.
WebSockets Server:
On new connection, store the URL that they're on. (The same user can have multiple pages open in multiple browser tabs, but the client's page to socket relationship should always be 1-to-1.) If the page has content that is locked, send that connection a message saying which is locked.
On a new lock, store the URL, client, and which piece of content is being locked. Send a message to all clients who are registered to that URL (including the originator, who will use that reply as confirmation) on what content is now locked. If desired, store the lock status in the DB.
On removing a lock, remove the record for the URL, client, and which piece of content was locked, sending a message to all clients registered to the URL, and clearing the flag from the DB. Leave room in this method to poll the database/framework on whether the content was changed or not, to potentially tell the clients registered to that URL to invalidate their view and fetch fresh content.
On a query about any locks, respond with all locks that currently exist for that page.
On client disconnect, remove any locks. If locks are removed, notify all clients registered to the URL. If the user re-connects, it will be on a separate socket so they'll have to establish a new, different lock anyways. Clean up the connection details as well (no need to try to send messages down a pipe that's closed, right?).
I want to create a bidding system where user can see the current price of items. And if any other user on any other location place a bid before me it should auto update bid in my browser.
I have read about autoupdate JS+Ajax functions but even if I place a 5 second timer to auto update the content on user's browser will it not put some extra load on server by making an ajax call every 5 second? Its a bidding system so user will have bids updating within 1-2 seconds so if i put an auto update ajax call for every 1-2 seconds it will put a lot of burden on server.
So I am wondering is there any better way to handle this type of stuff? how do twitter/facebook do update user's feeds?
AJAX or not, bidding systems always have high requests because people keeps refreshing the page to check for the latest bid information.
You can take a look and attempt long polling. Long polling a method where you "push" data from the server to the browser in response to the browser's HTTP request. It is a normal HTTP connection. This may reduce the number of requests sent from users to server, however you will still have many open and active connections between your users and your server.
You will want to look at long polling. In essence, this is how it works
On the server you need some sort of event mechanism (no probem with PHP)
Client (Browser) starts an AJAX request referencing a bidding item
Server checks for changes on the bid, if there is one, returns the request
If not, he waits for some time (minute range), waiting on an event concerning this bid
If such an event occurs, server returns the request with the info, if not he returns the request with "no bid" info
You might be able to get away with a streaming model...
Each JS client connects to the server once and keeps the conneciton open. As new events arrive at the server, they are broadcast to all the open connections in real time.
This is similar to the mechanism twitter uses to broadcast tweets.
I've created a small jquery and php chatroom with some .get and .post functions and php docs that read and write data to a sql server. It works fine, but the small problem is when someone posts something, it takes about half a second for it to appear (because of the lag).
I fear there's something wrong with my coding.
im using
setinterval (listen, 300)
as my continuous jquery function for reading new db entries, listen is a function with a .get inside. How does stackoverflow or facebook do it so that the user types something in and immediately it pops out?
Maybe try displaying the inputted chat message immediately to the user who posted it, prior to posting it to the database.
Like this:
User enters message, submits
Update users chat window so they see it immediately
POST message to database
GET from db and update all chat windows
This can be implemented using various techniques, which have many names: Long-polling, Server Sent Events, Comet, WebSockets, and others.
Basic idea is this:
Alice opens facebook. Her browser makes a request for updates ($.get, for example), but the server does not respond if there are no new updates and the request remains in 'waiting' state.
Bob opens facebook. He decides to comment on Alice's wall. His browser posts his comment to the server ($.post).
The server accepts this post, handles it properly (saves onto Alice's wall, etc)
But ALSO server checks if there is a waiting update request from Alice. If there is, server renders info about this update into response stream and closes the connection.
Alice's browser finally gets a response to this long hanging request and happily draws a red "1" in the notification area. It also immediately opens another update request (to not miss any).
Alice sees comment from Bob, which was delivered instantly.
The technique described is called "long polling" and it was first introduced by Google in Gmail.
You can use HTML5 sockets, however these are very much in their infancy and not widely supported (i.e. by IE).
Lots of systems use Flash as a middle-man as that can hold a connection open.
With either of these you can use your PHP code that stores the comment to the database to also push this out to every "listener". This will be the quickest way possible. If your system is super clever it'll incorporate all three - HTML5 sockets where it can, Flash where it can't, and your regular polling were there's no Flash either.
http://pusher.com/ might be a good starting point for further learning.
Most sites that have fast chat use a technique called comet. You can read more about it here: http://ajaxian.com/archives/comet-a-new-approach-to-ajax-applications
It essentially is a piece of modified server software that waits to return a response to the user until either a message is sent or it is about to timeout.