Currently I have a data file in dropbox that is uploaded every 15 seconds, I want to take this data, which has several different data types, and graph the real time data that the user selects on a website. I have a data server but my data is not on there. Is there any way for me to take this data from the file and graph it while also having a control panel that selects which data I want to graph.
You can refresh your web page using Ajax. Note that if your refresh is set to every 15 seconds and your data comes in every 15 seconds, worst-case is that you will show data that's almost 30 seconds old if the timing of the data update and the Ajax refresh is unfortunate.
You probably want to check for new data using Ajax more frequently, depending on your specific needs. On the server side, cache the result of the Ajax update to avoid too much duplicate processing.
To create the data that you return from the Ajax query, open and process the data file. No need for MySQL. You can use the timestamp of the file to invalidate the result cache I suggest in the previous paragraph.
There are many JavaScript based charting libraries that can update via Ajax. Here's a good starting point:
Graphing JavaScript Library
Related
What I would like to do is push JSON data from a PHP script to Vue (in the view of an MVC PHP framework), comparable to what you would see in an online exchange where the data updates in near real time. So without a page reload.
Now I understand you can use fetch to fetch data (and probably set an interval so it does that every x seconds), but I would like the UI to update in (near) real-time as the PHP script outputs the data. How to do this? Preferably without using a third party like pusher.com.
Some details about the UI:
It's about 10 to 30 rows in a <table>
In each row there is one status label and a few numbers that would get updated
There are a few other generic elements on the page that need an update (eg overall status label)
I've doubt regarding speed and latency for show real time data.
Let's assume that I want to show read time data to users by fire ajax requests at every second that get data from MySql table by simple collection query.
For that currently these two options are bubbling in my mind
MySql / Amazon Aurora
File system
Among these options which would be better? Or any other solution?
As I checked practically, if we open one page in browser then ajax requests gives response in less than 500ms using PHP, MySql, Nginx stack.
But if we open more pages then same ajax requests gives response in more than 1 second that should be less than 500ms for every visitors.
So in this case if visitors increase then ajax requests gives very poor response.
I also checked with Node.js+MySql but same result.
Is it good to create json files for records and fetch data from file? Or any other solution?
Indeed, you have to use database to store actual data but you can easily add memory cache (it could be internal dictionary or separate component) to track actual updates.
Than your typical ajax request will look something like:
Memcache, do we have anything new for user 123?
Last update was 10 minutes ago
aha, so nothing new, let's return null;
When you write data:
Put data into database
Update lastupdated time for clients in memcache
Actual key might be different - e.g. chat room id. Idea is to read database only when updates actually happened.
Level 2:
You will burn you webserver and also client internet with high number of calls. You can do something:
DateTime start = DateTime.Now;
while(Now.Subtract(30 seconds) < start)
{
if (hasUpdates) return updates;
sleep(100);
}
Than client will call server 1 time per 30 seconds.
Client will get response immediately when server notices new data.
Current situation: I have a web page that uses AJAX/JQUERY to refresh all the content on the page every 17 seconds. Every time this happens the server queries the database for data from two tables, one of which is large (450MiB in size, 11 columns) and processes all the data.
This is too resource intensive. I want:
The server queries the database only when one of the two tables have changed.
The page then reloads the page through AJAX only when the tables have been updated and the server has re-processed the data.
I think this falls under the category of comet programming. I'm not sure.
2 is easy. The webpage calls 'update.php' every 17 (or maybe less) seconds. The PHP script returns no data if no changes have been made. Only if data is returned then the current page is replaced with the new data. Please advise me if there is a better way.
As for 1 my googling tells that every time one of my two tables is updated I can put a notification in a table (or maybe just a single byte in a file) to indicate that I must query the database again and then the next time that the webpage sends an AJAX request I return the data.
The problem is that I'm working with a rather large code base I didn't write and I don't know of all the places that either of the two tables may be updated. Is there an easier way to check when the database is modified.
I'm working with PHP, Apache, Drupal and MYSQL.
You can chekout Server Sent Events
A server-sent event is when a web page automatically gets updates from a server.
there is an excellent article on HTML5rocks.com - Server Sent Events
All You have to do is create an object
var source = new EventSource('xyz.php'); //Your php files which will return the updates.
Once you create an object,you can listen to the events
source.addEventListener('message', function(e) {
console.log(e.data);
}, false);
I am wondering how to auto-save API data from http://api.bitcoinaverage.com/ticker/USD and https://crypto-trade.com/api/1/ticker/dvc_btc to create a chart using jqplot.
How can I make automated calls to each of the sites every 10 minutes and save data, and not have the data be overwritten by future calls?
Something like the chart here: vircurex.com/
You will need to use a combination of Database , Cache and a Cron (Scheduled Job) on your server to achieve what you want to do.
A high level approach would be:
1) Run a Backend Cron Job every 10 minutes. This will make a call to your Data Source i.e. HTTP Services and save the data in the database.
2) When the front-end makes a call, you should check if the results are present in the Cache. If they are present, return from the Cache itself so that Database calls (expensive) are avoided. If the results are not present in the Cache, retrieve from Database, put in Cache and return the data.
You might want to design your Cache and the items as per your requirements. Depending on the Caching libraries that you use, you could look for features like auto-expiring the cache items, reloading them automatically, etc.
I am running a sweepstakes like thing and want my users to all be able to load the page and have the exact same countdown and number generator running. I tried this in AS3 but each user caches their own swf file and gets a different result from the random number generator, and the as3 countdown is a few seconds off from each user. How would I go about making a countdown that is the exact same for each user looking at it at the same time and then a random number generator where every user will see the same result it gives? Is it even possible?
Sorry I wasnt clear on this. I would like to have it where the viewers could see the number being generated when the timer runs out. Kind of like watching the lotto on TV. Again, not sure if this is possible.
I have looked around, I know as3, some php, and some javascript. I have given up on doing this in flash.
Assuming I understand correctly what you want, store a random value (RV) every time someone accesses the page associated with a UNIX timestamp in seconds. Make time column unique, then if another request is made the same second, the random number is going to be taken out of the database.
Store the timer result in a table, show users the stored random number. Then when needed simply create another random number, store again, show user. Repeat.
Every user that come to that page , first print the time from the server so every user that will come and if your server time is 12:00 am for example , you will show him 12:00 am.
And then with Ajax refresh this time every X seconds ajax will send a request to the server and the server will tell you the time to display.
Keep the random logic on your server and when your application logic will say to change to a different number the server will return a different number and your clients will get it on the next pull ( ajax ) .
I hope that answering your question.
I'm not too familiar with flash, but what I would do is make the countdown happen client side using javascript's getUTCSeconds(), getUTCMinutes(), getUTCHours() etc to figure out current time verses end time of the counter (end time in universal time code).
Then you could use php to generate the random numbers (and a corresponding remaining time associated with it? how often do you want these?) and store it in a location for later retrieval (database, or file or some such). You could use ajax to grab the random number at the specified times
for more about js date/time functions w3schools has a pretty good resource:
http://www.w3schools.com/jsref/jsref_obj_date.asp
This is possible.
All logic should be stored on the server side. Use Flash only to show results.
Countdown: create it with php, store on the server side in the storage (database, memory, files, whatever). All clients (written on Flash) request counter value and display the counter on the client side starting from the value taken from server.
Lotto results are also generated on the server and passed to the clients. On the first hand you may generate intermediate results on the server and read them one by one by the clients, but I'd generate all results at once and pass them to the client.
Intermediate results can then be synchronized with the counter.