So I'm working on a simple Server/Client Android application where the Android app takes a picture and then sends it to a server.
After that the server has some backend code that will do some sort of processing to the image and generate a new image.
I currently have everything up to here working.
What I now need is a way for the server to let the client(android app) know when the image is ready.
Can somebody point me in the direction of the proper way of doing this? I was thinking something simple like a boolean value of whether or not the new image exists yet. My issue is that the processing may take a second or two so if I were to just simply do a GET there is a good chance the new image is not ready yet/does not exist.
Ultimately I want to display the new processed image in the android app. My current method is just to wait 1 second or two before doing the GET but that seems like a bad a way to do this.
Yes, the way you are doing it is completely useless.
You need to do some RestFul service there. When client requests to some certain api, you need to hold it there, do your image processing and respond in the same http request back.
RestFul API
Regarding to loader effect, when you implement the restful api on your server side it's easy to do it in the client. In Android you can use AsyncTask, which handles some background operation with onPreExecute and onPostExecute callbacks inside. So you can start your loader animation in onPreExecute and finish and replace the new image in onPostExecute methods.
You can also check out this discussion. SOF Question
Only a couple of ways, really ... simply have the client "call back" if the server process is anything that exceeds several seconds (a timer on the client). This works, but not the most elegant way of handling things.
My preferred way of tackling long latency server response issues is via a "push" notification to the handset. That way, the handset is free to continue forward with other tasks. But, once the server has something ready it sends the notification - handset receives notification, and "calls home" to pick up the waiting payload. I've been building mobile systems like this since BlackBerry came out with MDS back in 2002. It's an elegant approach, in my opinion. Amazon, Apple, and Google ... they all have their take on this infrastructure that models what RIM pioneered. Check them all out before making any decisions as they each have their own restrictions, pros and cons, costs, etc., etc.
Related
I'm building a prototype where on one page A I have a controller. Imagine a TV remote. On another page B, independent of form A -- it can be a different screen/computer -- with some elements X, Y, and Z that are going to be animated by the remote on page A.
My first idea would be:
Remote on page A saves an action wanted and sends JSON.
I create a listener on page B that reads the JSON to listen to what action to trigger.
I wonder if this the right direction.
It is just for a prototype so it doesn't have to be a production-perfect idea.
You can use web sockets for this.
Let's assume you're using 2 computers, both pointed at the website, as this technique could work in both scenarios.
If it's just a prototype, you could just simply have your page B poll the server every 5 seconds to look for updates that were submitted by page A.
However, in reality, for a production app with thousands of users, this could consume lots of bandwidth and put a heavy load on your server. To compensate for the load and bandwidth usage, you could increase the polling rate to 10 seconds, or 30 seconds, but in response to this change your users would experience delays while they wait for the browser to request an update from the server.
In a production app, many developers are turning to Comet as a solution. Comet is basically a term given by Alex Russell for a technique that involves using the request/response cycle to simulate server-push.
The general idea is that the browser makes a request to the server, and the server holds that connection open indefinitely. The connection remains open until another user posts an update to the server, in which case, the server then sends a response to the connected users.
The Dojo and Jetty team have demonstrations where they show that this technique can scale to 20,000 connections, when using continuations.
While I think you can carry out your experiment just fine with a database and/or some session variables, if your want to learn more about Comet on PHP, check out How to Implement Comet with PHP. Good luck!
UPDATE:
I also wanted to say that you definitely have the right idea for how to conceptually think about your message passing with JSON:
I create a listener on page B that reads the JSON to listen to what action to trigger.
I really like how you are thinking about passing a message that then tells the page what action to trigger. If you think about it, you could reuse your message passing concept to call other commands so that you avoid reinventing the wheel when a new command comes along that you need to call. Regardless of whether you poll, use Comet, or use WebSockets, it's a great idea to think about abstractions and generic, reusable data transports.
You could do this either with polling (having page B constantly poll for updates from the server) or use a server push technology like server sent events or websockets.
Yes, that would work. You could also just make it the same way you would make a vector line animation. Send the "commands" for movement to a server and record them (in a database, file, whatever) the client program can then request and redraw the movement smoothly any time and anywhere.
Using a cron job to execute Page B for every x unit time will make you check for any latest updated json (queried/returned output according your logic) from Page A. This way, you can use new updated json from page A and do your further task...
I'm trying to make a theoretical web chat application with php and jquery, I've read about long polling and http streaming, and I managed to apply most principles introduced in the articles. However, there are 2 main things I still can't get my head around.
With Long Polling
How will the server know when an update have been sent? will it need to query the databse continually or is there a better way?
With HTTP Streaming
How do I check for the results during the Ajax connection is still active? I'm aware of jQuery's success function for ajax calls, but how do I check the data while the connection is still ongoing?
I'll appreciate any and all answers, thanks in advance.
Yeah, the Comet-like techniques usually blowing up the brain in the beginning -- just making you think in a different way. And another problem is there are not that much resources available for PHP, cuz everyone's doing their Comet in node.js, Python, Java, etc.
I'll try to answer your questions, hope it would shed some light on this topic for people.
How will the server know when an update have been sent? will it need to query the databse continually or is there a better way?
The answer is: in the most general case you should use a message queue (MQ). RabbitMQ or the Pub/Sub functionality built into the Redis store may be a good choices, though there are many competing solutions on the market available such as ZeroMQ, Beanstalkd, etc.
So instead of continuous querying your database, you can just subscribe for an MQ-event and just hang until someone else will publish a message you subscribed for and MQ will wake you up and send a message. The chat app is a very good use case to understand this functionality.
Also I have to mention that if you would search for Comet-chat implementations in other languages, you might notice simple ones not using MQ. So how do they exchange the information then? The thing is such solutions are usually implemented as standalone single-threaded asynchronous servers, so they can store all connections in a thread local array (or something similar), handle many connections in a single loop and just pick a one and notify when needed. Such asynchronous server implementations are a modern approach that fits Comet-technique really great. However you're most likely implementing your Comet on top of mod_php or FastCGI, in this case this simple approach is not an option for you and you should use MQ.
This could still be very useful to understand how to implement a standalone asynchronous Comet-server to handle many connections in a single thread. Recent versions of PHP support Libevent and Socket Streams, so it is possible to implement such kind of server in PHP as well. There's also an example available in PHP documentation.
How do I check for the results during the Ajax connection is still active? I'm aware of jQuery's success function for ajax calls, but how do I check the data while the connection is still ongoing?
If you're doing your long-running polls with a usual Ajax technique such as plain XHR, jQuery Ajax, etc. you don't have an easy way to transmit several responses in a single Ajax request. As you mentioned you only have 'success' handler to deal with the response in whole and not with its part. As a workaround people send only a single response per request and process it in a 'success' handler, after that they just open a new long-poll request. This is just how HTTP-protocol works.
Also should be mentioned that actually there are workaround to implement streaming-like functionality using various techniques using techniques such as infinitely long page in a hidden IFRAME or using multipart HTTP-responses. Both of those methods are certain drawbacks (the former one is considered unreliable and sometimes could produce unwanted browser behavior such as infinite loading indicator and the latter one leaks consistent and straightforward cross-browser support, however certain applications still are known to successfully rely on that mechanism falling back to long-polling when the browser can't properly handle multipart responses).
If you'd like to handle multiple responses per single request/connection in a reliable way you should consider using a more advanced technology such as WebSocket which is supported by the most current browsers or on any platform that supports raw sockets (such as Flash or if you develop for a mobile app for instance).
Could you please elaborate more on message queues?
Message Queue is a term that describes a standalone (or built-in) implementation of the Observer pattern (also known as 'Publish/Subscribe' or simply PubSub). If you develop a big application, having one is very useful -- it allows you to decouple different parts of your system, implement event-driven asynchronous design and make your life much easier, especially in a heterogeneous systems. It has many applications to the real-world systems, I'll mention just couple of them:
Task queues. Let's say we're writing our own YouTube and need to convert users' video files in the background. We should obviously have a webapp with the UI to upload a movie and some fixed number of worker processes to convert the video files (maybe we would even need a number of dedicated servers where our workers only will leave). Also we would probably have to write our workers in C to ensure better performance. All we have to do is just setup a message queue server to collect and deliver video-conversion tasks from the webapp to our workers. When the worker spawns it connects to the MQ and goes idle waiting for a new tasks. When someone uploads a video file the webapp connects to the MQ and publishes a message with a new job. Powerful MQs such as RabbitMQ can equally distribute tasks among number of workers connected, keep track of what tasks had been completed, ensure nothing will get lost and will provide fail-over and even admin UI to browse current tasks pending and stats.
Asynchronous behavior. Our Comet-chat is a good example. Obviously we don't want to periodically poll our database all time (what's the use of Comet then? -- Not big difference of doing periodical Ajax-requests). We would rather need someone to notify us when a new chat-message appears. And a message queue is that someone. Let's say we're using Redis key/value store -- this is a really great tool that provides PubSub implementation among its data store features. The simplest scenario may look like following:
After someone enters the chat room a new Ajax long poll request is being made.
Request handler on the server side issues the command to Redis to subscribe a 'newmessage' channel.
Once someone enters a message into his chat the server-side handler publishes a message into the Redis' 'newmessage' topic.
Once a message is published, Redis will immediately notify all those pending handlers which subscribed to that channel before.
Upon notification PHP-code that keeps long-poll request open, can return the request with a new chat message, so all users will be notified. They can read new messages from the database at that moment, or the messages may be transmitted directly inside message payload.
I hope my illustration is easy to understand, however message queues is a very broad topic, so refer to the resources mentioned above for further reading.
How do I check for the results during the Ajax connection is still active? I'm aware of jQuery's success function for ajax calls, but how do I check the data while the connection is still ongoing?
Actually, you can. I've provided a revised answer for the above but I don't know if it's still pending or has been ignored. Providing an update here so that the correct information is available.
If you keep the connection between the client and the server open it is possible to push updates through which are appended to the response. As each update comes in the XMLHttpRequest.onreadystatechange event is fired and the value of the XMLHttpRequest.readyState will be 3. This means that the XMLHttpRequest.responseText continues to grow.
You can see an example of this here:
http://www.leggetter.co.uk/stackoverflow/7213549/
To see the JS code simply view source. The PHP code is:
<?php
$updates = $_GET['updates'];
if(!$updates) {
$updates = 100;
}
header('Content-type: text/plain');
echo str_pad('PADDING', 2048, '|PADDING'); // initial buffer required
$sleep_time = 1;
$count = 0;
$update_suffix = 'Just keep streaming, streaming, streaming. Just keep streaming.';
while($count < 100) {
$message = $count . ' >> ' . $update_suffix;
echo($message);
flush();
$count = $count + 1;
sleep($sleep_time);
}
?>
In Gecko based browsers such as Firefox it's possible to completely replaces the responseText by using multipart/x-mixed-replace. I've not provided an example of this.
It doesn't look like it's possible to achieve the same sort of functionality using jQuery.ajax. The success callback does not fire whenever the onreadystatechange event is fired. This is surprising since the documentation states:
No onreadystatechange mechanism is provided, however, since success, error, complete and statusCode cover all conceivable requirements.
So the documentation is potentially wrong unless I'm misinterpreting it?
You can see an example that tries to use jQuery here:
http://www.leggetter.co.uk/stackoverflow/7213549/jquery.html
If you take a look at the network tab in either Firebug or Chrome Developer tools you'll see the file size of stream.php growing but the success callback still isn't fire.
My setup: Currently running a dedicated server with an Apache, PHP, MYSQL.
My DB is all set up and stores everything correctly. I'm just trying to figure out how to best display things live in an efficient way.
This would be a live challenging system for a web based game.
User A sends a challenge to User B
User B is alerted immediately and must take action on whether to
Accept or Decline
Once User B accepts he and User A are both taken to a specific page
that is served up by the DB (nothing special happens on this
page,and they dont need to be in sync or anything)
The response from User B is a simple yes or no, no other parameters are set by User B, the page they are going to has already been defined when User A sends the challenge.
Whichever config I implement for this challenge system, I am assuming it will also work for instant sitewide notifications. The only difference is that notifications do not require an instant response from User B.
I have read up on long polling techniques, comet etc.. But im still looking for opinions on the best way to achieve this, and make it scalable.
I am open to trying anything as long as it will work with (or in tandem) to my current PHP and MYSQL set up. Thanks!
You're asking about Notifications from a Server to a Client. This can be implemented either by having the Client poll frequently for changes, or having the Server hold open access to the Client, and pushing changes. Both have their advantages and disadvantages.
EDIT: More Information
Pull Method Advantages:
Easy to implement
Server can be pretty naïve about who's getting data
Pull Method Disadvantages:
Resource intensive on the client side, regardless of polling frequency
Time vs. Resource debacle: More frequent polls mean more resource utilization. Less resource utilization means less immediate data.
Push Method Advantages:
Server has more control overall
Data is immediately sent to the client
Push Method Disadvantages:
Potentially very resource intensive on the server side
You need to implement some way for the server to know how to reach each individual client (for example, Apple uses Device UUIDs for their APNS)
What Wikipedia has to say (some really good stuff, actually): Pull, Push. If you are leaning toward a Push model, you might want to consider setting up your app as a Pushlet
I'm here to ask you if what I think is the right way to go around coding this.
I have a site that receives private messages and I wish a flag to show up the moment the person receives a message. Should I check for new messages every 3 seconds and show the flag if there is a new message or is there a better way?
If I did it in ajax, I was thinking check every 3-5 seconds for new messages, and once there's a flag, stop checking for more.
My only concern is, if it checks every 3-5 second, will it cause any lag or glitchyness for the person when they're typing? Lets say they're typing out a paragraph somewhere, I don't want their writing to glitch while it checks those 3-5 second intervals.
One of my coder friends mentioned there is a method with Ping(?) or something like that. Where the person is always connected to the server and when there's a change it notifies the user. I'm totally unsure of how this works.
Anyone know how facebook does it? haha.
Thank you!
If you have done the AJAX well, it should not lag/glitch while typing. Something like 3-5s is good as its fast enough but won't slow down server/browser.
Did he mean "push"? In push the messages are pushed to client in realtime, client is not asking if there is new messages. This is most likely the method Facebook is using.
One of my coder friends mentioned there is a method with Ping(?) or
something like that
To be honest I really don't like periodic refresh(polling at intervals), because tt has scaling problems(I got notice from hosting provider when using periodic refresh). You should use more efficient transports like for example:
WebSocket
Adobe® Flash® Socket
AJAX long polling
AJAX multipart streaming
Forever Iframe
JSONP Polling(cross domain)
To use this you could for example use:
hosted solution pusher with API and generous free plan. This gives you max 20 concurrent and 100,000 messages per day, but no SSL, so do not transmit sensitive information over the wire. They also provide third-party PHP client available at github implementing REST API.
socket.io (I like this a lot)
tornado
netty
Anyone know how facebook does it? haha.
For chat they use Erlang. They also have open-sourced tornado(see link above) which they required from friendfeed which they acquired in the past. Facebook is a PHP-shop, but they decided to not use PHP for this, because PHP can not yet do this efficiently. Anyway they are using one of the efficients transports above.
I'd like to create an application where when a Super user clicks a link the users should get a notification or rather a content like a pdf for them to access on the screen.
Use Case: When a teacher wants to share a PDF with his students he should be able to notify his students about the pdf available for download and a link has to be provided to do the same.
There are several ways you can accomplish this. The most supported way is through a technique called Comet or long-polling. Basically, the client sends a request to the server and the server doesn't send a response until some event happens. This gives the illusion that the server is pushing to the client.
There are other methods and technologies that actually allow pushing to the client instead of just simulating it (i.e. Web Sockets), but many browsers don't support them.
As you want to implement this in CakePHP (so I assume it's a web-based application), the user will have to have an 'active' page open in order to receive the push messages.
It's worth looking at the first two answers to this, but also just think about how other sites might achieve this. Sites like Facebook, BBC, Stackoverflow all use techniques to keep pages up to date.
I suspect Facebook just uses some AJAX that runs in a loop/timer to periodically pull updates in a way that would make it look like push. If the update request is often enough (short time period), it'll almost look realtime. If it's a long time period it'll look like a pull. Finding the right balance between up-to-dateness and browser/processor/network thrashing is the key.
The actual request shouldn't thrash the system, but the reply in some applications may be much bigger. In your case, the data in each direction is tiny, so you could make the request loop quite short.
Experiment!
Standard HTTP protocol doesn't allow push from server to client. You can emulate this by using for example AJAX requests with small interval.
Have a look at php-amqplib and RabbitMQ. Together they can help you implement AMQP (Advanced Message Queuing Protocol). Essentially your web page can be made to update by pushing a message to it.
[EDIT] I recently came across Pusher which I have implemented for a project. It is a HTML5 WebSocket powered realtime messaging service. It works really well and has a free bottom tier plan. It's also extremely simple to implement.
Check out node.js in combination with socket.io and express. Great starting point here