I want to track a few hundred, maybe a few thousand people in real time.
Let's say that the hardware aspects are sorted out and I can get the data into a database.
Now, I want to get it out and show it, in real-time.
Weeeell ... "real-enough" time. Let's say that I want to draw a floorplan of a building and plot everyone every 1 to 5 seconds.
(I might want to show only certain "kinds" of people at the click of a button; I will need datamining, etc, but let's stick with the worse case scenario).
I am comfortable enough with PHP, though not this sort of thing. I personally would be happier with a windows app coded in Delphi, but the trend seems to be to make everything browser based.
So, the question, I guess is whether a browser can handle this and whether there are compelling arguments for a windows-based or browser-based solution.
If browser-based can handle this (displaying a few thousand data-points a second), and there are no overwhelming arguments for windows then I guess I will go for browser-based and learn a few new tricks. The obvious advantage being that I could also re-use a large part of my code for (vehicle) tracking on Google maps.
Most of your work will probably be done in your spatially enabled relational database. For example, PostGIS can select data points within a bounding box or more sophisticated spatial predicates (ST_Contains, ST_Crosses, ST_Intersects, ST_Touches, ...) as well as the usual SQL joins and WHERE conditions. Spatial selects should use a spatial index to speed things up.
If this is the case, your app will largely be a presentation layer. In this case, use whatever will be easiest for you. The advantage of browser-based is that it is cross-platform client-server by default, but this might not matter to you.
With respect to render speed, it really depends on how you are planning to render your map. There are speed tests available for Google Maps. However, I suspect that if you are planning on a google maps type interface several thousand points are going to turn into a blur of pins. Do you have an interface mockup?
If you'd be happier with one type of app versus another, then write that type of app. Don't make something a browser app just to make it a browser app.
First you should understand what is the target of your app (and its size, how many concurrent clients?), and then decide if a fat client or a thin/web one suits it best. Then you should check which kind of application is able to manage the load you forecast. Could you display a few thousand datapoint every second or so with the technology you master or can in your timeframe? Would multithreading help you to exploit current multicore processors to achieve your result? Which technology will let you take advantage of it?
Maybe seems a bit old hat, but if you do decide to go browser based, you could pre-render the points to an image and just show the image?
WebSockets
Take a look at HTML5 WebSockets, they are a new standard for server/browser data exchange:
WebSockets is a technology providing
for bi-directional, full-duplex
communications channels, over a single
Transmission Control Protocol (TCP)
socket, designed to be implemented in
web browsers and web servers.
The server can push new data to the client when it becomes available, removing the overhead of client pull requests.
The Wikipedia page lists available server-side implementations, including PHP versions (but not Delphi).
WebSockets are not bound to JavaScript, the websockets.org page says:
In addition, the Web Socket protocol
can be used to support a diverse set
of clients (e.g. JavaScript, Adobe
Flex, JavaFX, Microsoft Silverlight,
etc.). However, the HTML5
specification only defines support for
JavaScript, which is limited to
text-based protocols. To serve other
client-types and support binary
protocols you will need to look to
external offerings.
Message data can be exchanged in JSON format, which is supported by JavaScript and also available for Delphi (as open source implementations like SuperObject or lkJSON).
Ajax
Ajax based rich internet applications for Delphi can be built using Intraweb or ExtPascal. ExtPascal is an Object Pascal (Delphi, FreePascal/Lazarus) wrapper/binding for Ext JS, a complete GUI Ajax framework and offers transparent support for all main web browsers: IE 6+, Firefox 1.5+, Safari 3+, Opera 9+ and Chrome 2+ on any client side platform (PCs, SmartPhones, iPhone, PDAs, etc). Online demo applications: FishFacts, more.
Ajax Push (aka Reverse Ajax / Comet) offers "HTTP server push" of data (in XML or JSON format) which transfers data from the server to all connected web browser clients whenever data on the server changes. Clients can 'subscribe' to the information they are interested in.
Related
I'm trying to figure out a way for users of a website (say a student and teacher) to share a secure connection where real time updates on one page are viewed by both of them.
From research I've concluded that some of the real time updates could be performed using ajax and javascript.
But I'm stumped as to how users could share a connection where only the two users would be viewing the updates that take place on the website (such as flash animations of a drawing board.) I'm also confused how you would even begin to set up a connection like this.
I've looked intp php sessions and cookies but I'm not sure I'm doing the right research.
Any pointers as to how two specific users could share a secure connection where real time updates are viewed by the both of them only. I don't want a terse response please. I'm looking for specific details like functions and syntax specific to php. I appreciate the help and will rate you up if you give me good answers!
You cannot share a secure connection (e.g. HTTPS) its one client to one server.
If both clients are logged in and have a background AJAX task running in the browser, is it acceptable to have each client "pull" every few seconds the same data to display for both users?
This would require the "drawing board" updates to also be sent continuously back to the server to share the updated data with the other client. I'm sure there will be an event you can use to trigger the post of data (e.g. on mouse up).
If performance is an issue, you'd want to use a better server technology like Java that is able to keep session state between requests without having to persist to a database.
You can look at ajax push techniques. I used comet once where an administrator posted messages and everybody who was logged in saw that message appear on their screen. I don't know if comet supports PHP. I only used it with JSP. Just search for "ajax push" in Google.
Flash allows for connections between users, I think they refer to them as sockets.
If you want to use Ajax, et al, you need a server side technology that supports push.
Node is the standard in this regard, and you can set up a Heroku instance for free.
There are others, and you need to learn tools before even beginning to learn application.
Among the many overviews, this might interest you:
http://arstechnica.com/business/2012/05/say-hello-to-the-real-real-time-web/?1
A few good examples where this is happening:
Google Docs
Etherpad
HTML5 Games: Multi player
Techniques you can use (with varying browser support)
HTML5 WebSockets (Wikipedia; MDN; HTML5 Demo)
Comet (Wikipedia)
Really pushing data to a web browser client from a server (which would do that when it receives something from another client) is only possible with WebSockets as far as I know. Other mechanism would either require browser plugins or a stand-alone application.
However with Comet (through AJAX) you can get really close to pushing data by polling the server periodically for data. However contrary to traditional polling (e.g. where a clients asks for data every 5 seconds), with the Comet principle the server will hold that periodic request hostage for, say, up to 30 seconds. The server will not send back data until either it has data or the time out is reached. That way, during those 30 seconds, any data that the server receives can be instantly pushed back to the other clients. And right after that the client starts a new 30 second session, and so forth.
Although both Comet and WebSockets should work with a PHP backend served by Apache. I'd recommend looking into NodeJS (as server technology) for this.
There is a lot of information regarding Comet on the internet, I suggest you google it, maybe start at Wikipedia.
The great thing about Comet is that it is more a principle than a technology. It uses what we already have (simple HTTP requests with AJAX), so browser support is very wide.
You could also do a combination, where you use sockets if supported and fallback to Comet.
I'm sure you have looked into this. The opinion that this can be done via ajax is misleading to believe that two users of a website can communicate via javascript.
As you are aware, javascript happens on the client and ajax is essentially 'talking to the server without a page change or refresh'.
The communication between two users of the website has to happen via the server - php and some chosen datastore.
Hope that wasn't terse.
cheers, Rob
I am trying to add the feature that Google Docs has, which is real time collaboration, to an editable textarea in HTML. For example, this would allow 2 or 3 users can edit the same textarea collaboratively. How would one go about approaching this problem, or is there is a JavaScript library that can be used? (I use PHP, mySQL, and JavaScript/AJAX/jQuery).
In order to facilitate real-time updates between more than one Web client, you'll need to use a technology that either capitalizes on the request/response cycle of the Web by using a Comet or Websockets solution.
To keep the textarea updated, you'd need to establish a long-lived HTTP connection or a Websocket connection to your server from all 3 clients. Each textarea would need a keyup or keypress handler that, when invoked, sends the character across the stream to the server. When your server retrieves this data, it would then need to return a response to the other 2 connected clients.
The response would need to then be handled by updating the value property of the textarea with the most recent data.
I see you're using PHP, which I do know supports comet. You'll need to setup comet (or Websockets) in order to implement such a solution.
With that said, a more rudimentary alternative would be to use polling to achieve the desired effect. This would involve all 3 clients making requests to the server periodically to retrieve updates. As you can imagine, the faster the polling rate, the more real time the application would feel. However, with a faster the polling rate, your application would consume more bandwidth and resources.
For 3 clients, this may be feasible, but for any serious application that involved heavy usage, you would definitely want to look into Websockets or Comet.
To answer your question of JavaScript libraries, check out the Dojo Cometd library for a Comet solution on the client-side.
i cam across some older topics about this but since there is a lot of new techniques for creating web content i have to ask again.
My background:
I have good knowledge of xhtml/css/javascript/php/mysql. I have no trouble making a webshop from scratch including a comprehensive database and a cms to easily add/modify products. I am very comfortable using PHP and mysql to store data. A attempt at a tribal wars clone does not scare me at all. I also made a couple of games with c#/xna but i am still considering myself novice. I dabbled with flash and feel this could probably be used to get the result i want.
My project:
A 2D mmo tactical shooter. Players should be able to move across a fairly large maps and get position updates of other users very quickly. The game will hold a lot of maps and most maps should be able to handle just a couple of players. Hub maps and maps of interest however should be able to handle at least 100 players without lag.
It would be awesome if combat could be resolved real time on the map but i already thought of other ways to resolve this like opening an instance map where players could enter to join combat. I'm also considering a more tactical turn based approach on combat but it depends on the pro's and con's of real time combat.
To sum it up:
I am sure i can manage everything except for the graphical implementation. Mysql should not have much trouble querying the local player positions in a couple of milliseconds.
I cam across a game named Deeppolis that pretty much resembles graphically what i want. I'm still considering making this as a downloadable client but i am very interested what my options are if i'm going to do this browser based as browser based games tend to get more people attracted.
Thanks for reading this wall of text :D.
you would not want to "query mysql" to get player positions if this is real time. You will want to create a socket server that all clients connect to. the socket server would keep a subscriber list of all connected clients in memory. this way when one client sends a message to the socket server, the socket server can immediately push needed data to whichever clients need it. you would also want to keep a model of the game state in memory in your socket server to prevent cheating etc. you would only use a database for long term storage, everything else should be in memory. also keep in mind shared servers will likely not allow you access to sockets to create a socket server, so you will need a private server.
client side: javascript & socket.io (socket client)
server side: javascript & node.js (socket server)
I'm working on an in-house project management web based application that needs to support mobile devices as well as desktop.
It's built with Symfony2, jQuery, HTML5.
Are there any performance comparisons between using WURFL as opposed to responsive design, both on server and client side? Specifically I'm thinking about rendering times, HTTP calls (it's quite AJAX heavy).
Performance-wise, responsive design places the entire load on the client so you should ensure that this works adequately well by testing on many devices. Not all smart phones are created equal—some have slow CPUs that make JavaScript code and media queries painfully slow. Overall, using server side code can result in a much lighter experience for the client, while also allowing you to exercise a finer level of control over the experience.
But before you think about the performance aspects of this you should consider if this approach will deliver an adequate mobile experience at all. There are two important aspects of a mobile version of a site that you should aspire to:
A contextually appropriate experience—it should be able to deliver an
appropriate experience for someone using a mobile device. This may be
quite different to an appropriate experience of the same service on a
desktop. Note that use of a mobile device does not necessarily imply
mobility—mobile device users are often physically immobile but users
may nonetheless prefer to interact with your site or service in a
different way when using a mobile device. The importance of a
contextually appropriate experience is increasing dramatically as the
number of ways that we interact with the web is increasing: a
lean-forward experience that seems appropriate on a laptop may feel
entirely incorrect on a television browser that you interact with
from across a room.
A device-sensitive experience—it should be
capable of delivering an experience that works well on the devices
used by your website customers. This range of addressable devices is
increasing all the time, and growing more diverse, from feature
phones to televisions. Some are held close to the face, others are
interacted with from across a room. It is next to impossible to
deliver a satisfactory experience on such a wide range of devices,
each with their own input/output restrictions and conventions,
without tailoring the experience to the device. The major internet
brands are keenly aware of this and doing much more of it than may be
apparent—even the seemingly simple Google homepage masks vastly
different code behind the scenes served to different devices used to
achieve a useful experiences across the device landscape.
Used as a means to deliver both a desktop and mobile site, however, responsive design falls short on delivering both desired aspects of an ideal mobile site.
It cannot deliver a contextually appropriate experience because it
delivers the same experience regardless of the device that people are
using (this limitation may not be an issue for sites with restricted
use cases)
It can deliver a device-sensitive experience only to to a
limited range of devices, since the core technique limits the range
of devices that can be targeted to smartphones and other high-end
devices. The one-experience-fits-all issue and limited range of
addressable devices may not be a problem for all websites—some sites
don’t lend themselves well to mobile-specific experiences and equally
some site owners may not have a desire to serve a wide range of
devices.
It is worth noting that responsive design has an unknown impact on mobile SEO since it is not clear whether or not search engines will identify the content as being mobile-friendly and rank it accordingly in mobile searches.
Any logic that can be determined and acted upon on the server side can reduce data transfer and client side overhead. Reducing the size of the content sent - e.g. relevant CSS, JavaScript, HTML and optimised images - will clearly put less of a burden on the client.
A RESS based solution (i.e. Responsive Design + Server Side Components - http://www.lukew.com/ff/entry.asp?1392) will always have the opportunity to be faster that a responsive design solution on its own. You will always need to consider significance of "faster", but when I'm seeing (perhaps poorly designed) responsive design sites out there delivering 1Mb+ of content to a mobile device the performance optimisations they could gain from a little bit of server side intelligence are immense. Good white paper on why performance of web site matters from gomez # http://www.gomez.com/resources/whitepapers/why-web-performance-matters/ and why every second counts.
A few examples on how server side feature detection can help are listed at http://www.opendeviceknowledge.com/discovery including how responsive design and server side world can play together.
WURFL inventor, here. Ronan Cremin appears to have addressed the question rather extensively. At the end of the day, is a question uf usability vs cost (both development and maintenance).
The only other thing I want to point out is that WURFL and Responsive Web Design need not be separate worlds. This and this posts have my viewpoint on the subject.
More interestingly, we recently launched a service that make some of WURFL also available to Javascript developers free of charge. I advise you check out http://wurfl.io/ site.
In a nutshell, if you import a tiny JS file:
<script type='text/javascript' src="//wurfl.io/wurfl.js"></script>
you will be left with a JSON object that looks like:
{
"complete_device_name":"Google Nexus 7",
"is_mobile":true,
"form_factor":"Tablet"
}
(that's assuming you are using a Nexus 7, of course) and you will be able to do things like:
if(WURFL.form_factor == "Tablet"){
//dostuff();
}
This is what you are looking for.
Thanks
I am building a "multiplayer world" with jQuery and PHP. Here is a bit how it works:
User's character's positions are taken from a database, user is plotted accordingly (the position values are CSS values - left and top)
User is able to move about using the arrow keys on the keyboard, making their character move using jQuery animations. While this is happening (on each arrow press) the user's position values are inserted into a database and updated.
In order to make this "global" (so users see each other) as you could say, the values need to be updated all at once for each user using AJAX
The problem I am having is I need to continuously call a JavaScript function I wrote which connects to the MySQL server and grabs values from a database table. And this function needs to be called constantly via setInterval(thisFunction, 1000); however my host just suspended me for overloading the server's resources and I think this was because of all my MySQL queries. And even after grabbing values from my database repeatedly, I had to insert values every few seconds as well so I can imagine that would cause a crash over time if enough clients were to login. How can I reduce the amount of queries I am using? Is there another way to do what I need to do? Thank you.
This is essentially the same thing as a chat system with regards to resource usage. Try a search and you'll find many different solution, including concepts like long polling and memcached. For example, check this thread: Scaling a chat app - short polling vs. long polling (AJAX, PHP)
You should look into long polling - http://en.wikipedia.org/wiki/Push_technology. This method allows you to establish a connection with your server, and then update it only when you need to. However by the sounds of it, you have a pretty intensive thing going on if you want to update every time, you may want to look into another way of storing this data, but if your wondering how big companies do it, they tend to have mass amounts of servers to handle it for them, but they will also use a technique similar to long polling.
You could store all the positions in memory using memcached See http://php.net/manual/fr/book.memcached.php and save them all at once every few seconds into the database (if needed).
You could use web sockets to overcome this problem. Check out this nettuts tutorial.
There is another way, it's to emulate or use actual sockets. Instead of constantly pulling the data (refreshing to check if there are new records), you can push the data over WebSockets which works in Chrome at the moment (at least to my knowledge, didn't try it in FF4) or you can use Node.js for leaner long pooling. That way, the communication between players will be bi-directional without the need of MySQL for storing positions.
Checkout Tornado
From their site:
Tornado is an open source version of the scalable, non-blocking web server and tools that power FriendFeed. The FriendFeed application is written using a web framework that looks a bit like web.py or Google's webapp, but with additional tools and optimizations to take advantage of the underlying non-blocking infrastructure.
The framework is distinct from most mainstream web server frameworks (and certainly most Python frameworks) because it is non-blocking and reasonably fast. Because it is non-blocking and uses epoll, it can handle thousands of simultaneous standing connections, which means it is ideal for real-time web services. We built the web server specifically to handle FriendFeed's real-time features — every active user of FriendFeed maintains an open connection to the FriendFeed servers. (For more information on scaling servers to support thousands of clients, see The C10K problem.)