I need a kayak.com like, functionality.
That is, the user enters a keyword and I will need to display results as the become available. The important thing is that data should be displayed AS it BECOMES available. Kind of progressive display? Don't know if this is right term.
Kayak.com displays or gives the impression that data is displayed as they become available after an asynchronous call.
Can anyone give directions on this topic? (php on the server side) Is this a case of PUSH ?
Thank you.
Push is a nice way of keeping your clients up to date, however, the more users the push server is pushing data to, the more resource intensive it gets. If this is an application where you expect to have multiple users, pushing may be a bit intensive for your needs, whereas polling regularly might be better. But give push a try, these guys (icefaces) have a nice ajax push implementation.
Hope this helps!
It seems that the server is sending its response in paginated sections. This enables the client side to begin rendering much sooner. After the client receives the first page, it renders it and begins the request for the subsequent page, and so on, until there are no more pages remaining.
You may also be interested in reading up on Comet which can be thought of as an "AJAX Push". Not sure on any PHP implementations but I know there are several other solutions out there which you could potentially tie PHP into.
I've thought about this before but never really built anything. My thought process on this was to have your script grab the latest record and get some kind of unique identifier (ID or date). You log this id with javascript and have a request try to get results that are newer than the current ID every second (or longer if you don't need the data instantly). If results are returned, you once again log the latest result for the next request.
Related
We would like to make a realtime price change system to be able to see difference with competitor prices against our product prices.
We have no experience with Push notification / COMET system by Javascript and PHP. If you don't mind, I would like to get your experiences, suggestions about this technique. So here are my questions:
What is the best way doing like that system?
We are experienced PHP developers, so is PHP appropiate for this task?
If you know any project or solution ( open source or commercial ) that is able to do this, could you please share?
Here is the approach that we use. Javascript sends regular AJAX request to PHP file. PHP file makes database query and if noting found, just sleeps for 0.5 second (or 1 second), then makes database query again. If 30 second passed and still nothing new found in database (this is needed for giving output, before HTTP timeout occurs), then it outputs something (like noting found). Javascript starts another query immediately after it received output from the last query. Javascript always keeping track of last ID of the database table which comet is monitoring. This is used to query only database rows that is greater than last ID that we have seen.
Yes PHP is appropriate. Just remember one important thing! You need to close any sessions that are open before entering in comet loop. PHP uses session locking to prevent two threads writing into same session simultaneously. If you forget to close session all other threads will be locked (like browsing through website will be impossible).
I can advice open source PHP framework that we use. It called Stingle. It has solid and production ready Comet plugin.
try to use socket.io , no need to send request from client to server to get the data, on server side just send data using socket, client will get data.
just avoid make http request for notification since the notification is almost realtime.
I have a certain concept in mind, but I an unsure what my possibilities are.
User enters a name (html form) on a webpage
Webserver sends push notification to the Android device of the user
Application looks for this name in the contactslist and finds the corresponding phone number
Application instantly sends this information back to the webserver, where the user can use the returned phone number.
The only step I am unsure about is #4. I presume I'd have to use a temporary MYSQL database and use reverse AJAX or something alike to get the returned data to the user without him having to refresh the page (this entire process needs to be completed within a couple of seconds at most - the faster, the better).
This all sounds like a very hack-ish approach to me though, I was hoping any of you could hint me with better ways of tackling this - or maybe a good resource to the approach I described.
It's hard to find solutions when you're not sure what it is you're looking for.
Much obliged!
I can think of two ways of solving #4
poll "did I get the response yet?, did I get the response yet?..."
or use websockets. here's a post I found that might be useful: post
Here's one interesting approach to it all using node.js: nowjs
If you plan to use Androids C2DM, check out the min-max delivery times.
In an admin panel made with php, what is the best way for paging? ClientSide (jquery) or Serverside ?
Short answer, yes.
A bit longer answer, it depends on what you are pageing
If you are pageing large amounts of data, I'd go with a combination, using ajax to fetch the data and letting PHP sort out which data should be sent.
If you are simply pageing a couple of panels with controls I'd just do it with javascript, but thats my personal preference.
For 9/10 solutions, go with whatever you are more comfortable with.
Think again your question: what happens when (and how)?
Client side:
To achieve client side pagination you should serve all rows (data) to your client because at this time you haven't any info about which page needed. Then with the help of a (possibly) javascript solution splitting all of the served data into smaller chunks mimicking pagination.
Server side:
You should serve only the first (or actual) page.
I'd go with server side pagination of course.
Your decision will depend on how the data is likely to be used.
When a user is on that page, are they likely to stay the first page of results most of the time? In that case, server side paging works well, as you only go through the trouble of assembling that one page of data.
Or are they likely to constantly page back and forth through the results? In that case, you might as well build all the results efficiently in single shot and let the client do the paging, since you're going to need all the data eventually anyway.
It depends on several factors: (incomplete list)
How often do people want to see other pages than the first?
If they often browse around you'd want the interaction to be entirely on the client => instant feedback.
If, OTOH, the rest of the pages are seldom used there's no reason to send it to the client in the first place.
I.e., send what most users want, nothing more.
How big is the total data set?
How do you weight initial load time (higher with client side) vs. time to serve a new page (higher with server side)?
...
I'd go for server side, but:
Serve first page only, thus minimizing initial load time
Fetch other pages, when requested, via AJAX - serving only what's necessary in order to minimize the "page" load time
And it's an admin panel, so you should really follow #Kristoffer S Hansen's advice: Do whatever you're more comfortable with.
I have a large database of links, which are all sorted in specific ways and are attached to other information, which is valuable (to some people).
Currently my setup (which seems to work) simply calls a php file like link.php?id=123, it logs the request with a timestamp into the DB. Before it spits out the link, it checks how many requests were made from that IP in the last 5 minutes. If its greater than x, it redirects you to a captcha page.
That all works fine and dandy, but the site has been getting really popular (as well as been getting DDOsed for about 6 weeks), so php has been getting floored, so Im trying to minimize the times I have to hit up php to do something. I wanted to show links in plain text instead of thru link.php?id= and have an onclick function to simply add 1 to the view count. Im still hitting up php, but at least if it lags, it does so in the background, and the user can see the link they requested right away.
Problem is, that makes the site REALLY scrapable. Is there anything I can do to prevent this, but still not rely on php to do the check before spitting out the link?
It seems that the bottleneck is at the database. Each request performs an insert (logs the request), then a select (determine the number of requests from the IP in the last 5 minutes), and then whatever database operations are necessary to perform the core function of the application.
Consider maintaining the request throttling data (IP, request time) in server memory rather than burdening the database. Two solutions are memcache (http://www.php.net/manual/en/book.memcache.php) and memcached (http://php.net/manual/en/book.memcached.php).
As others have noted, ensure that indexes exist for whatever keys are queried (fields such as the link id). If indexes are in place and the database still suffers from the load, try an HTTP accelerator such as Varnish (http://varnish-cache.org/).
You could do the ip throttling at the web server level. Maybe a module exists for your webserver, or as an example, using apache you can write your own rewritemap and have it consult a daemon program so you can do more complex things. Have the daemon program query a memory database. It will be fast.
Check your database. Are you indexing everything properly? A table with this many entries will get big very fast and slow things down. You might also want to run a nightly process that deletes entries older than 1 hour etc.
If none of this works, you are looking at upgrading/load balancing your server. Linking directly to the pages will only buy you so much time before you have to upgrade anyway.
Every thing you do on the client side can't be protected, Why not just use AJAX ?
Have a onClick event that call's an ajax function, that returns just the link and fill it in a DIV on your page, beacause the size of the request an answer is small, it will work fast enougth for what you need. Just make sure in the function you call to check the timestamp, It is easy to make a script that call that function many times to steel you links.
You can check out jQuery, or other AJAX libraries (i use jQuery and sAjax). And I have lots of page that dinamicly change content very fast, The client doesn't even know is not pure JS.
Most scrapers just analyze static HTML so encode your links and then decode them dynamically in the client's web browser with JavaScript.
Determined scrapers can still get around this, but they can get around any technique if the data is valuable enough.
What is the best way to implement server push for more than one thing.
Lets say I want to just update user status, so I can periodicaly poll the server for status in like 1000ms and update the page.
The other way I can found is that server waits for like 30 sec, while it checks if there was any change and if one is found, server push response back to client, which update page and then make another poll.
But how would I implement this to check for like 10 things on website? For example if I want stackoverflow to refresh question votes when someone vote, but the only way how to do this that I can think of is
ask server for votes for each question -> server replies with votes of every question on the page
but how can I find out which question votes did change? I could send all current votes and then let server compare the values and reply only with those that did change, but I think that it would be very uneffective to do this while checking for like 30 values.
One example for all would be Facebook, where almost everything is refreshed by server push, but how can server find out what did change and what didnt?
Everything that I found (including my book "Ajax Patterns") explains only how to poll for one value, but I didn't find anything how to poll for many values at a time (like more than 10).
If you are prepared to use sessions, the easiest way would be to have a serial request ID, and let the server keep track of the latest information sent to the client on the session. Then the client asks if there is newer data than the last time it refreshed it.
e.g: Lets say the client sends ajax request #5001 after nothing has changed, the server might reply 'false'. Then someone posts a message, or there is a change of some kind, and so in request #5002 the server sends a list of changed elements (whatever these are). Then in request #5003 it would reply false again, because nothing has changed since request #5002.
A JSON client/server architecture would be perfect for this. It allows easy serialisation of object hierarchies/maps. I prefer jQuery for the client in javascript, and the server side is trivial to spit out.
I'd suggest to tag your data with revision numbers at the server side, so the client knows what revision of the data it has; create a composite query whereby the client can send a set of revisions, and the server can respond to that list of revisions with any updated revisions it may have for them. In that way, the client is only making one server query to see if there are any updates; you're just batching together all the data the client is interested in into one query. This method also has the strength of allowing the client to change the set of data that it's interested in; if your implementation server-side is flexible enough, you can use the same implementation for all your dynamic data needs.