I want to plot some data into a web graph control(preferably javascript or php). The data is collected regularly from a microcontroller, however the data collection interval is not linear. For instance, I may collect 5 data points in one day, and then 2 data points the next, and at different intervals etc...
Is there a graphing tool that can automatically create a linear datetime axis such that the data is represented properly on the graph.
jquery's jqplot does the trick.
You can use jqplot as James Cotter says, but there are also :
Highcharts.js (apparently the best JS out there, but restrictive license)
Google Charts Tools (generate a graph from an URL, it returns a PNG image)
Depending on your needs, I'd definitely use Google Charts Tools as it'd decrease my bandwidth usage. However, if you need complex manipulation of your datas once the chart is displayed, use Highcharts.js (or jqPlot).
Related
I have 5 second data for a year in an ascii file
each line is a reading...
timestamp, value
for 6 million lines
I want to display this data in a chart or multiple charts in a web browser
I considered a choice of 3 charts
1 - last 500 data points at maximum (5 second resolution)
2 - last 500 points at 15 min resolution
3 - all data at various resolutions
etc
being wary of a) time to read file b) processing time c) amount and time to download data to browser for javascript plotting
Can php do direct access read from a file?
More to the point, these big dataset plotting problems must be quite common, bhoiw do people get around it?
Yes, PHP can read files on the filesystem. Databases may be very helpful along with reading data from file with PHP.
What would you like to do with the data - as in, what are you really looking for? Are you looking for peaks, averages, values greater than average etc.?
Generic analysis
Perhaps the answer is - I don't know; I want to look at the data and see what comes glaring out. Fair enough. In that case you could have a web page that uses something like a stock chart. Show the last 1000 records.
To get the last 1000 records, you could use a combination of commands such as top or head (on Linux-y systems) or powershell on Windows to get last 1000 rows and then parse them with PHP, shove them into an array or object and show them on screen using Javascript or PHP charting tools.
When the user changes selection, do a read of the file and display relevant records. This can be taxing because the file is constantly processed.
Non-PHP method
A faster non-PHP alternate would be to use something like an in-memory business intelligence tool like QlikView (free download for personal use, I think). The learning curve is not steep, ... and I have no affiliation with QlikView. Tableau and Spotfire are other tools that can be easy to use and can make analysis of large datasets relatively easy.
Specific analysis
If your interest is to find out number of days each month with sales of $1 million or above, you could do a single pass on the file and extract all lines with sales >= 1MM and store it in an array of date and sales. Pass through the array and output a file with Year, Month, Sales. That would be pre-processing of the data.
Then, the web application or your presentation layer can pull this data and show information in bar charts or whatever else. Javascript charting libraries like amcharts, d3, highcharts etc. can be used, or PHP charting libraries like jpGraph and such can be used to read pre-processed data on the fly and show them.
If data has to be looked from multiple angles such as - table showing top 10 products that are sold, scatter plot of # of orders vs. $ ordered etc., all this data could be shoved into a database and then pulled on the screen. As Mark Baker commented, appropriate indexes will be necessary to pull data efficiently.
Prepare specific datasets in batch
Some climate research centers in the US run programs that churn through millions of records, like you have, at night to create graphs, charts, maps etc. and then use web applications to display them. For example High Plains Regional Climate Center and Iowa Mesonet do that regularly. You could do something similar with PHP.
Databases are my favorite. I prefer massaging textual data, eliminate what I don't want and push the objective matter in databases. PHP can then utilize rollup, top n, group by etc. methods within dbs to extract data and present it on screen - primarily through a web interface.
If you have specific questions about toolset or so relevant to this question, feel free to comment. If a new question crops up in your mind, feel free to add a new question altogether to solicit diverse answers.
My client needs to use IE8. I'm using the flot graphing library and I'm hitting a limitation of the performance of javascript in IE.
When there are a thousand points the graph takes up to 10s to display. I have seen one possible solution to speeding it up, but not sure how well this will work.
Has anyone tried optimising flot for IE?
Failing that has anyone done some performance analysis with the various PHP javascript (not flash) graphing libraries if there is one that will out-perform flot in IE8 (i.e. without canvas).
You can try jqChart. The render speed of the Line Chart is optimized for handling a large set of data.
Take a look at this sample:
http://www.jqchart.com/samples/ChartPerformance/LineChart
You might want to take a look at Highcharts. It is compatible with IE 6 and the chart in this time series demo has 1096 points.
I have used this library in my applications with success, although you will have to evaluate it yourself to see if it meets your performance requirements.
Your only hope is to figure out a way to not plot all the points at once. For instance, if you try to plot 10,000 points on a graph that is 600x300, chances are that the majority of points overlap each other almost completely.
What most people do in these situations is pre-compute (server-side) averages, or whatever type of aggregation is necessary, then plot that instead. Then, use the selection plugin to allow them to zoom in on smaller areas of the graph and only there plot the complete data set for that area.
I am looking for cross-platform data visualization solution.
It should ideally work in web browsers and by extension on Android and iOS devices.
To go into more detail, I'd like for the server side to be able to accept data via HTTP POSTs (XML, JSON, etc), and then allow for themeable output of these data via widgets (Drupal-like) such as meters, graphs, plain text etc.
The data types should be configurable and easily created/modified.
Here is a quick example:
I want to show temperature from a bunch of sensors as a graph. The sensors can POST the data to a main server and the server can then be called from a browser to display a chewed-up and configurable graph.
Thanks for your input guys!
A little searching on google would yield a lot of libraries, My favorite being a html5 one called Rgraph http://www.rgraph.net/
High charts is another excellent solution. http://www.highcharts.com/
These are both html5 based.
You can check the combination of google spreadsheet for the DB/backend and Google Visualization for the frontend visualization.
Google GData API has a nice set of client libraries that make writing POST data into Google Spraedsheet a very simple task. Check out: http://code.google.com/apis/spreadsheets/data/3.0/developers_guide.html#CreatingListRows
ListEntry newEntry = new ListEntry();
newEntry.getCustomElements().setValueLocal(tag, value);
ListEntry insertedRow = service.insert(listFeedUrl, newEntry);
On the other end, It is rather easy to use the JavaScript libraries of Google Visualization to extract and present data from a published spreadsheet. It has a wide range of widgets ready to use. If you are missing any specific widget, you can use any JS library with some manipulation of the JSON data.
For simple example on the ability to turn a Google Spreadsheet to a live chart check: http://code.google.com/apis/chart/interactive/docs/spreadsheets.html#Google_Spreadsheets_as_a_Data_Source
I sampled a small wikipedia collection from 2008 and trying generate a graph similar to this: http://en.wikipedia.org/wiki/File:Wikipedia-n-zipf.png
from the processing I did. I have word frequencies and ranks of words. How can I generate the graph on the fly from the data I collected? Thanks.
If I am understanding your question correctly, you want to generate PNG charts on the fly?
This Page lists a few great chart engines for PHP. If you want to generate graphics using PHP for word frequencies, my personal favorite is PCHART It is a really nice library to generate charts like that.
Google Chart is completely free and doesn't use as much server power, but you have much less control over caching, styling, etc. The key difference here is that the actual image generation is done by Google.
If you want it to look exactly like that, you can use any of them and use the most basic settings to get what you need.
I think you want to use the google chart api / graph tools.
You can find it here
Line charts documentation and examples
I want to take data from a table that'll be almost exactly similiar to the one below, but have it in a line graph. The date values would be on the Y-axis, and it would plot the XP values on the X-axis. Since the numbers for each user vary, I'd need a way to make the distance between each point plotted "relative", I guess.
Example table http://img687.imageshack.us/img687/4175/tablel.png
Any suggestions?
If you want to generate a static image of the chart you could use Libchart or pChart for PHP.
It's better to get the data from the data source, not from the rendered table. Anyway it's two separate questions: how to take data from a table and how to draw a graph.
Drawing a chart in PHP is pretty easy. HTML/CSS can be used to draw a bar and PHP to calculate the bar length.
Or if you want to go the JavaScript route, maybe Raphaël.
I would go for HighCharts.
If you're using a linux/unix platform, you probably already have Gnuplot installed. Gnuplot can take a plot file and a datafile and generate an image, thus:
History.gnuplot, which needs to be generate first:
set title "Rawr_satch history"
set xlabel "Time"
set ylabel "Ranking"
set output "rawr_satch_graph.png"
set terminal png color
set xdata time
plot "rawr_satch_xphistory.dat" with linespoints
Which assumes a data file rawr_satch_xphistory.dat has already been generated, formatted thus:
2010-03-06 385581123
2010-03-05 384895430
2010-03-04 382983388
People have also written interfaces into Gnuplot for most languages, for example PHP-GNUPlot. Gnuplot scripts can be quite complex, and you could plot multiple variables, etc.
I use and fully recommend the Google Chart Tools API. If you're looking for a simple line-graph, they provide a very simple API that you can call that requires no installation or configuration. Some of the documentation is confusing, but I've been able to figure it out with a little bit of patience.
All that is required is an IMG tag and point the src to the Google URL with the right parameters.
They also have a more interactive Javascript library if you want to provide more functionality later on.
I've used the following from PHP: Google Charts, Open Flash Chart, YUI charts (Flash), and AMCharts (also Flash).
They're all pretty easy to use and all have their pros and cons. If you want to show more than a single graph on one page, don't use Flash-based charts. It turns out browsers have trouble displaying a dozen Flash charts all at once. AMCharts is probably the most feature rich, but the options available depend on which of their chart packages you select, and configuration is pretty complex.