I'm currently writing a web application in HTML5/Javascript and PHP.
Some employees of the company will need to use it to enter their work schedule. The application will calculate in real time (using Javascript) some complicated legal infos and display the result at the bottom of the page. When he's done, the user clicks the "save" button and everything is sent to the database.
The problem is that I need both the user to see the output in real time and the managers to get the same output from what was saved in the database. I also need to support the case where the user has Javascript disabled. In other words, I need to do the same calculus in both Javascript and PHP.
To make things more complicated, the formula is very complex and will change often (every month or so), so I'd like to avoid maintaining two different versions. Testing one will already be hard enough.
Also I'd like to avoid AJAX to ask the server for the output, because:
the user will often build its schedule according to the result of the real-time calculation, so even a 1-2 seconds lag could really be annoying for him
if possible I have to support HTML5's offline features, so the user can load the app on his mobile phone, fill its schedule while offline, and then upload it when online
For the moment the only solution I found is to write the formula in a language-agnostic way, then use some way to turn it into PHP code and JavaScript code, but that's not simple.
It sounds like you have essentially two approaches:
You can write a language-independent formula (in a database, or config file, since it changes so frequently) and two code generators (Javascript, PHP). This is actually less scary than it sounds, as long as you pick your input format sanely and you actually mean a formula, not some arbitrary computation.
You can use server-side JavaScript, and write the formula once in JavaScript. Wikipedia has a comparrison of server-side JavaScript implementations. There are even JavaScript interpreters written in PHP, including at least phpjs and J4P5.
Persisting data with language agnostic storage techniques and interpretative conversions are not the first thoughts that come to mind when the term "real-time" is mentioned. It is very possible, but usually much more challenging and time consuming to design infinitely scalable and easily maintainable software.
Personally, I weight the cost versus benefit associated with every known solution and decide on an approach that, first and foremost, benefits the end-user and meets or exceeds key objective deliverables. Once I've made a decision that satisfies major stakeholder's expectations, it is then my job to design and develop the most scalable and maintainable solution as possible within the confines of these expectations.
Based on the details you've provided I would absolutely write the equations using native JavaScript and PHP for optimal performance. If the software specifications literally (not figuratively) denote the necessity for real-time communications, I would then provide this by implementing WebSockets, otherwise long pulling would suffice.
Please note: Not all Web browsers support WebSockets, which may require end-users to conform to a supported Web browser. It is possible to avoid this conundrum by employing a seamless client-side failover (or altogether) by using something like Socket.IO.
This morning I got the idea to try XSLT for the calculation. This is really interesting, because it is supported by both Javascript and PHP, it's something reliable (contrary to a code generator I would have to write myself), and appropriate because the input data was already in XML.
Example input data :
<schedule>
<element start="2011-19-09 08:00:00" end="2011-19-09 17:00:00" type="work" />
</schedule>
Example XSLT (written by heart, may not work) :
<xsl:stylesheet>
<xsl:template match="/">
<output>
<out name="totalWorkHour"><xsl:value-of select="sum(/element[#type='work'](#end-#start))" /></out>
</output>
</xsl:template>
</xsl:stylesheet>
I'll try to code everything with XSLT, and I'll keep this solution if I manage to do that.
Related
Here is my situation. I am building an application that contains some heavy mathematical calculations where the formula needs to be editable by a sufficiently privileged, but untrusted, user.
I need a secure server side scripting language. I need to be able to access constants and values from 4+ database tables, the results of previous calculations, define user variables and functions, use if/then/else statements, and I'm sure more that I can't think of right now.
Some options I've considered:
I have considered using something like this matheval library but I would end up needing to extend it considerably for my use case. I would essentially be creating my own custom language.
PHP runkit sandbox. I've never used this before but am very concerned about the security issues involved. Considering the possible security issues, I don't think that this is a viable option.
One other idea that has crossed my mind that I don't know if it is possible would be to use something like javascript on the server side. I've seen js used as a scripting platform in desktop applications to extend functionality and it seems a similar approach may be feasible. I could ideally define the environment that things ran it, such as disabling filesystem access etc. Again, security seems like it would be an issue.
From the research I have done, it seems like #1 is probably my only option, but I thought I would check with a larger talent pool. :-)
If #3 is possible, it seems that it would be the way to go, but I can't seem to turn up anything that is helpful. On the other hand, there may not be much difference between #2 and #3.
Performance is another consideration. There will be roughly 65 some odd formulas each executing about 450 times. Each formula will have access to approximately 15 unique variables a hundred or so constants, and the results of previous formulas. (Yes, there is a specific order of execution.)
I can work with an asynchronous approach to calculation where the calculation would be initiated by a user event and stored in the db, but would prefer to not have to.
What is the best way to work with this situation? Are there any other third party libraries that I haven't turned up in my research? Is there another option in addition to my 3 that I should consider?
There's almost no reason to create a custom language today. There's so many available and hackable, writing your own is really a waste of time.
If you're not serving a zillion users (for assorted values of a zillion), most any modern scripting language is securable, especially if you're willing to take draconian measures to do so (such as completely eliminating I/O and system interfaces).
JavaScript is a valid option. Its straightforward to create mini-sandboxes within JS itself to run foreign code. If you want folks to be able to persist state across runs, simply require them store it in "JSON-like" JS structures that can be readily serialized from the system on exit, and just as easily reloaded. These can even be the results of the function.
If there's a function or routine you don't want them to use, you can un-define it before firing off of the foreign code. Don't want them using "read" to read a file? read = func(s) { }
Obviously you should talk to the mailing lists of the JS implementation you want to use to get some tips for better securing it.
But JS has good support, well documented, and the interpreters are really accessible.
You have two basic choices:
a) Provide your own language in which you completely control what is done,
so nothing bad can happen,
b) Use some other execution engine, and check everything it does to verify nothing bad happens.
My problem with b) is it is pretty hard to figure out all the bad things somebody might do in obscure ways.
I prefer a), because you only have to give them the ability to do what you allow.
If you have a rather simple set of formulas you want to process, it is actually pretty easy to write a parser/evaluator. See Is there an alternative for flex/bison that is usable on 8-bit embedded systems?
It isn't clear to me that you have a performance problem. yes, you want to execute something 450 times; but it includes database accesses, whose cost will dominate any computation involivng a 1000 arithmetic steps. You may find that your speed is limited by the DB access that that you need to cache the DB accesses to get it to go faster.
I am trying to build a very user-friendly user interface for my site. The standard right now is to use client side as well as server side validation for forms. Right? I was wondering if I could just forgo client side validation, and rely simply on server side. The validation would be triggered on blur, and will use ajax.
To go one step ahead, I was also planning to save a particular field in the database if it has been validated as correct. Something like a real-time form update.
You see, I am totally new to programming. So I dont know if this approach can work practically. I mean, will there be speed or connection problems? Will it take toll on the server in case of high traffic? Will the site slow down on HTTPS?
Are there any site out there which have implemented this?
Also, the way I see it, I would need a separate PHP script for every field! Is there a shorter way?
What you want to do is very doable. In fact, this is the out-of-the-box functionality you would get if you were using JSF with a rich component framework like ICEfaces or PrimeFaces.
Like all web technology, being able to do it with one language means you can do it with others. I have written forms like you describe in PHP manually. It's a substantial amount of work, and when you're first getting started it will definitely be easiest with one script per field backing the form. As you get better, you will discover how you can include the field name in the request and back it down to one script for Ajax interactions per form. You can of course reduce the burden even further.
PHP frameworks may be able to make this process less onerous, but I haven't used them and would recommend you avoid them initially until you get your bearings. The magic that a system like Cake or Rails provides is very helpful but you have to understand the tradeoffs and the underlying technology or it will be very hard to build robust systems atop their abstractions.
Calculating the server toll is not intuitive. On the one hand, handling large submissions is more work than handling smaller ones. It may be that you are replacing one big request with several tiny ones for a net gain. It's going to depend on the kind of work you have to do with each form field. For example, auto completion is much more expensive than checking for a username already being taken, which is more expensive than (say) verifying that some string is actually a number or some other obvious validation.
Since you don't want to repeat yourself it's very tempting to put all your validation on one side or the other, but there are tradeoffs either way, and it is true that server-side validation is going to be slower than client-side. But the speed of client-side validation is no substitute for the fact that it will introduce security problems if you count on it. So my general approach is to do validation on the server-side, and if I have time, I will add it to the client side as well so as to improve responsiveness. (In point of fact, I actually start with validation in the database as much as possible, then in the server-side code, then client-side, because this way even if my app blows up I don't have invalid data sticking around to worry about).
It used to be that you could expect your site to run about 1/3 as fast under SSL. I don't have up-to-date numbers but it will always be more expensive than unencrypted. It's just plain more work. SSL setup is also not a great deal of fun. Most sites I've worked on either put the whole thing under SSL, or broke the site into some kind of shopping cart which was encrypted and left the rest alone. I would not spend undue energy trying to optimize this. If you need encryption, use it and get on with your day.
At your stage of the game I would not lose too much sleep over performance. Since you're totally new, focus on the learning process, try to implement the features that you think will be gratifying and aim for improvement. It's easy to obsess about performance, but you're not going to have the kind of traffic that will squash you for a long time, unless half the planet is going to want to buy your product and your site is extremely heavy and your host extremely weak. When it comes, you should profile your code and find where you are doing too much work and fix that, and you will get much further than if you try and design up front a performant system. You just don't have enough data yet to do that. And most servers these days are well beyond equipped to handle fairly heavy load—you're probably not going to have hundreds of visitors per second sustained in the near future, and it will take a lot more than that to bring down a $20 VPS running a fairly simple PHP site. Consider that one visitor a second works out to about 80,000 hits a day, you'd need 8 million hits a day to reach 100/second. You're not going to need a whole second to render a page unless you've done something stupid. Which we all do, a few times, when we're learning. :)
Good luck on your journey!
I'm contemplating using http://pchart.sourceforge.net/ for our graphing / charting requirements but another developer suggested the use of a javascript/jquery based one like - http://dojotoolkit.org/
While the look-n-feel for both are different, and perhaps a javascript based one is more easily manipulatable - I'm not convinced it would be the faster solution.
Wouldnt a PHP based toolkit be faster anyday, for the end user, with less data going back and forth between our server and the client machine?
Our charting requirements are for reporting purposes - we dont require users to manipulate the charts 'live' at all.
Without considering presentation aspects, the 2 approaches generally have their pros and cons, whereas one's pro being the other's con.
The PHP approach:
arguably more consistent results.
resulting chart(s) could be saved if data is constant once entered,
this would avoid having the same process run multiple times,
resulting better overall performance.
The Javascript approach:
less computing power needed on the server.
resulting chart(s) could be generated dynamically delivering a more
interactive user experience.
I believe if the view is done at the client side it means less processing at the server. So i believe that client side graphing/charting would be better (using javascript)
with the requirement that you have stated the pHP one seems like a batter approach. Like you already mentioned if your chart is static and you are not sending over the data to the client side then it doesnt make sense to render the chart in js. Your php server will any day outperform the browser rendering speed as then your dependent on the client using the latest browsers in the market and they have adequate memory on client side to do the same.
Why is your teammate thinking of the JS approach. Does he have a reason for the same?
I have recently found jqPlot and used it in a corporate project to my boss' delight. A PHP solution will not give you a solution as dynamic as a Javascript one. Moreover, PHP graph approach are sometimes bulky and require more code.
Once the Javascript code is downloaded unto the client's machine, it is cached for further requests, so only the graph's initialization code is downloaded (a few bytes + the series data), this results in fewer bytes being sent, thus a lower bandwidth consumption.
As for the report data, using caching mechanisms is probably the best solution as you can reuse the report data for different views (downloadable as CSV, etc.)
All and all, I personally usually prefer to keep the server side processing for data, and client side processing for the view representation.
I want to develop an application that does a bunch of cool stuff. The first thing that I need in it is to get information about the page a person is browsing.
With that said, I need for example to know how long a user stayed in a page and where was the scrollbar. While getting that data, It's all saved to a database.
The thing is, I prefer doing that in Flash [although I have no experience in it] over Ajax since I want to hide the code - which as far as I know not possible in Javascript/Ajax.
So, can I do all that in Flash? - Read the content of the page, get the status of the scroll bar..
Plus, I then need to go threw the gathered information that is saved in the database. Since there could be many calculations i thought C++ .Net is better than PHP [which I know better].
Is that all possible or am I just crazy? :)
Thanks ahead.
Server side
I think it doesn't matter whether you'll use PHP/C++/Java/Ruby/Python/whatever... each of these is fast enough to do complex calculations, especially if we talk about pure math.
So if PHP is what you know the best, then it's obvious to use it.
Client side
Flash is pretty cool for animations and others visual effects, but for things like scroll position, time spend on website JavaScript is just better. It doesn't require third-parties plugins, it's integrated with DOM. Personally I just thing JavaScript is the most proper tool for this task.
I vote for JavaScript, you can do in JS all that you mention, using the DOM, and it has not a proprietary license.
Although you cannot compile JS code, obfuscation tools offers a decent level of protection. Closure is worth of mention too, YUY minifier etc.
Also check this ready made JS heatmap.
I advise against using C++ for server side programming. You'll be better off with Ruby/PHP etc.
I'm writing a shopping cart in PHP, and part of its function is to calculate shipping for a given set of products. For users with Javascript enabled, I want to provide the smoothest experience possible by having an accurate shipping calculation done on the client side so that changes in the cart do not require page reloads or AJAX calls.
Currently I have a function in PHP that calculates the shipping charge, and another function in Javascript that mirrors its functionality. I'm concerned that the formula may change frequently, and having to rewrite two separate functions in two different languages (and make sure their outputs match) could create maintainability problems.
What I'm looking for is a way of representing the formula for calculating shipping charges in some language agnostic way, so that it can be interpreted by both PHP and Javascript. Currently the shipping formula is only a function of a single variable (number of items) although I would like the capability to add more without too much rewriting.
Is there an accepted way to represent a fairly simple mathematical formula in a way that can be interpreted by both PHP and Javascript? Preferably without writing my own interpreter?
Edit: I don't need a whole function, just a fairly simple mathematical formula along the lines of "4 + 1.5n". The kind of thing you would find in a typical spreadsheet cell.
Generally, you are best off just to maintain two versions of the code. The most elegant solution, of course, is to write the server logic in JavaScript itself (see cappuccino.org); this would be a poor choice for you because it sounds like you have already written a ton of PHP code.
However, if you really feel the need to scratch this itch, consider an AJAX callback to the server. While slightly slower, most users will not notice.
I would say do it on server side, and user AJAX, It won't make much of difference for user, but maintaining two versions can make a difference when user after submitting the order sees the different calculation.
Anyway if you due to some reason do not want AJAX at all, best way would be to have single javascript library which is executed both at client and server side.
You may execute javascript from php on server side.
e.g. you may use following from linuxto execute javascript
http://www.mozilla.org/js/spidermonkey/
http://www.mozilla.org/rhino/
http://www.wxjavascript.net/
see http://en.wikipedia.org/wiki/Server-side_JavaScript for more options
This is a bad idea, and will become a worse idea the more complex the calculation becomes.
That being said, Javascript does support the dollar sign as legal in variable names. There's enough common syntax between the two languages that you could easily write code that parses and works the same in each, as long as you're only dealing with simple math. Arrays may be a bit tricker. Check out PHP.JS, a set of libraries to mimick PHP builtin functions in Javascript.
(Edit: I edited in the link to php.js without knowing someone else was going to post the same thing during my edit. Credit / horrors to him. ;) )
Maybe. Take a look at php.js. It gives you all the php functions but in JavaScript.
You could also write the whole validation in php, and use Ajax to query on the client side.
One would say this is the time to add some automated tests to your application, and run them at least each time you want to deploy your application to your production server -- and if some of those fail, cancel the deployment.
You'd have tests both for PHP and JS code, and some of those tests would calculate some total amount / charges / shipping costs ; and if something goes wrong because of some code change, you'd be able to detect the problem automatically, without your application breaking in production.
Of course, it requires a bit more work (both to write the tests, configure the building platform, and maintain the test data) ; but that would be a great security...
Another possible solution would be to write your calculation code only in Javascript, and run it both on the client side (this is not the hard part, obviously), and on the server side.
From PHP, you can execute JS code using the Spidermonkey PECL extension (Note it's still in beta, though ; and you'll need to be able to install PHP extensions, which will probably be possible only if you are admin of your server -- and not sure about stability).
Here is an article about it : Using JavaScript in PHP with PECL and SpiderMonkey.
With this solution, you could use JS for the code that only runs on the client ; PHP for the code that only run on the server... And JS for the code that runs on both sides.