I'm developing an app that is essentially going to pull data from a ton of different sources (twitter, facebook, github, basecamp, stackoverflow, etc.) and aggregate it all into a useful interface. Everything is going to need to be real-time.
I'm thinking that the majority of the app will avoid database usage - so the data I pull isn't going to be stored, it'll just be repulled and formatted everytime a user needs it. Obviously there is going to be a lot of AJAX involved and a lot of grabbing data from different APIs.
I'm most skilled in PHP, but I often times find it to be clunky, especially with this sort of thing. What language or framework would be the cleanest for pulling this kind of data so often, and quickly parsing it for the user? I know my way around Node.js and Ruby, but am very open to learning a new language/framework if need be.
all popular web languages are fine, but, i'd vote for node.js for the following reasons:
you know it.
easy for async networking to 3rd party services such as stackoverflow
your system won't be so snappy and node.js can handle many simultaneous clients.
php you know too and has curl that can do multi get to solve 2. but that won't help you with 3. ruby is in the same boat as php.
Related
So I have a simple jQuery Mobile site interfacing with a MySQL database via PHP. Users orders are queried initially, then updated using the mobile web interface. I am not using any PHP framework, just 7 or 8 PHP files I've written for retrieving, updating, creating, and deleting order records.
As I begin to develop the Android and iOS applications, I realized it would be beneficial to have a PHP backend framework that can handle requests to create/read/update/delete my orders and users records from any of the 3 sources without caring which one it is. I want to create a layer of abstraction between the server side database handling and each of the 3 mobile interfaces. I envision being able to make a POST call from the client side, passing a TYPE argument and a JSON array of the data to a generic page like /myserver/mRequest.php, which would determine what to do based on TYPE and handle the processing, returning a STATUS variable and a JSON array of data back to the client.
Is there a good (preferably free) PHP framework that will help layout the backend? I've looked at Zend and a few others, but they seem pretty heavy and intended for desktop sites. I want to minimize the number of calls to the server as well as the amount of data thats sent back and forth (we all know how slow Verizon 3G is).
I'm hoping theres some lightweight framework out there that can be a skeleton for me to organize CRUD functions, check for null values, handle MySQL errors, etc...
If not, a guide to developing a PHP mobile backend would be the next best thing.
Note: I apologize it there is a thread already asking this, but I couldn't find one that I thought fit.
Almost any of the major free frameworks would do the trick, Symfony for example handles regular page templating for your jQM pages and handles ajax nicely too.
However, if you only require the framework for the AJAX handling and something broader, you might be advised to develop your simple scripts, with less effort and less of a learning curve (of course, the learning may be part and parcel of your plan).
Another option is a Data Abstraction layer between your scripts and your PHP, this would give you greater flexibility in terms of your data changes, but not burden you with a hefty framework in between. Propel and Doctrine are worth a look and are often used as part of web frameworks.
I personally really like CodeIgniter, which is a very clean and powerfull lightweight PHP framework with easy database-access.
Just have a look at it.
Greetings,
Steffen
What I would propose would probably sound strange, but I have my reasons for it.
For a long time, we've had this Spring based API, which originated as an abstract set of REST services for CRUD functionality. With time however, we started building business and presentation layers on top until we reached a state of a dead end. Do not get me wrong, Spring/Hibernate are great frameworks, and being built on top of the JVM has its definite advantages, including performance over other web technologies, such as PHP. Compared to PHP, it gives us a much deeper way of controlling aspects such as transactioning, multithreading, dealing with byte data, control over native C++ applications, using JNI, etc.
Where the stack clearly hits a hard wall, is where the requirements get changed most often, namely the business and presentation layers. Turning the applciation into a modern, user-centered social applciation, we've experienced the toughest challenge in out careers. Java EE presentation technologies are tough to work with. Also, changing the business requirements became a tremendously long cycling because of the traditional hurdles in building testing and deploying massive Java applications.
It also felt like for a large part, we are trying to reinvent the wheel. In the PHP world, so many projects exist already, which give you a complete management system, agnostic to any kind of backend system (mapping hooks to REST/SOAP endpoints) . Many of them have all these functionalities ready, allow for admin-friendly change of scenarious, and rules, have templating, etc. Plus, being PHP based means absolutely no time wasted in building and deploying. Write the change, test, asert that it works, and switch.
Our idea now is to find a way to move the business and presentation layers in such a kind of front-server PHP-based application, and leave the pure backend stuff to Spring/Hibernate. We have a few concerns though, coming from our relative inexperience with Spring.
If we implement the business methods using PHP methods, how do we keep transactional security? I mean if a business method has to make three separate HTTP requests to the JAVA, how can we guarantee that they will all be executed in the same transaction, DB-wise?
Is there a way to use proxies or promise objects between both systems? For instance, if the PHP business method needs to call a Spring search method get a collection of objects from the database, and then pass it on to another spring method, this will mean that teh whole collection will have to be sent back and forth. Perhaps, one could store it in a session object on the JAVA side, and simply sent an empty proxy back to the frontend, which the frontend can bass back to another jav method.
A lot of our Spring based functionailities rely on a structure of plugins, using Spring events. How can we make so that our frontend server also gets notified on every ackend event that happens.
I have two ideas here: a post-processing-filter that simply makes a POST request to a controller on the frontend server, using some naming convention.
or ... using some kind of a message queue, such as JMS or RabbitMQ, or why not even something like Reddis, where one can watch data for changes
Anyone who has done that before? Is this a good idea in general? Any suggestions how to resolve the aforementioned issues?
This is not what you asked, but I think it's worth saying that some of the problems you are experiencing with Java are problems that most experienced Java developers faced some day, and most have found some solution. For instance, you can use Arquillian to do a "faster" TDD in Java. And you can use JRebel to do a "code-and-refresh" approach, like in PHP. Modern IDEs also helps in refactoring, something that helps in the "requirements change" scenario.
Granted, the presentation part is still a big problem with Java. I personally don't like JSF, and (to me), most of the other presentation technologies are either non-intuitive or they are cumbersome.
I would say that this is the reason why a lot of Java developers are becoming adopters of HTML5 and Javascript (backbone.js, underscore.js, jquery, ...) for the frontend, with REST in the backend. There's no need to have PHP in the middle.
I'm afraid I cannot answer your other questions, but perhaps a good start would be to see if PHP could be run from inside a Java EE container? I know this works for Ruby and Python apps, as JRuby and Jython would take care of the bridge between the two world.
Largely what you are trying to do is what we call: SOFEA
In this case PHP is your frontend and Java is your service layer.
Concern 1
Transactions over traditional REST or WS are a PITA.
Either consider aggregating your service calls to do more work so that the whole entire transaction is in one service calls or have Java handle your frontend. Otherwise see Concern 3 with Finagle.
Concern 2
I recommend you stay stateless and do REST or RCP.
It sounds like your concerned with speed which I think you shouldn't worry about till its a problem. However if you do want to do user based caching of objects I would use Redis and its pub/sub feature (or RabbitMQ + memcache).
Concern 3
Use RabbitMQ, or Redis, or/and Finagle.
MY final Opinion
I don't think you need PHP particularly if you know Java.
So much of the web app world today is moving towards Javascript (in the client) I don't see any benefit to using something like PHP serving pages when there are better options like Node.js or just sticking with Java.
And #jpkrohling is right... templating and frotend used to suck in Java but its gotten much better particularly since most of your UI code will be in Javascript anyway.
If you need an example of more modern web stuff in Java have a look at this project: MWA and also by the same author: Handlebars in Java.
To sum it up my opinion and go back to SOFEA: backend = Java and frontend = Javascript
Haven't done this before, not sure if it's a good idea, but here are some thoughts on your questions:
First of all, in the PHP world, each request (whether it uses keep-alive or not) is run as an isolated process, so this scenario is typically unheard of; it's possible to use persistent database connections, but any non-committed transactions are automatically rolled back when the request is done. Now, this is where my knowledge about Spring is lacking, because I'm not sure whether Java can keep a database connection in the same state between requests as long as the same HTTP connection is used; if so, you can use PHP + cURL to perform three HTTP requests while reusing the network connection.
Object (un)marshaling between PHP and Java will probably not work very well; a representation of the collection would have to be made in the form of perhaps a result identifier that can be fetched via REST by PHP and passed back to Java as an opaque string.
It should be noted that PHP requests are typically short-lived, which makes it a good candidate for horizontal scaling; long running processes, such as those for Pub/Sub systems, are therefore seen less often. Personally I prefer using the alternative of sending HTTP requests so that the frontend caching can be abstracted away from the Java service.
I'll add more when I think of anything else, do let me know if my answer gives you more questions :)
There is definitely no straightforward way out. You have to re-write your backend controller layer which exposes REST calls to PHP. You need to group some fine grained services and make them coarse grained.
A word of advise - you can make things work at the end day with tweaking here and there, introducing a queue or another framework or library etc - but you would have also introduced lot of Accidental complexity..
Things can get real nasty in future during maintenance, bug fixes, upgrades etc
If speed of development is a concern, explore other alternative like Play Framework 1.2.x framework, so that you can re-use your server side code as it is.
I want to try creating a canvas were multiple users can type and edit for brainstorming etc. I am thinking of php as backend. What are the technique used for it? For example, lets say its
pure text. How do we update the text like in the google wave(b4) and in google doc?
Is it just using websocket(or js pooling) and updating the text file? or are there any
better logical way to solve this issue?
Well, its not as easy as it is said in Brad's answer but he is partially right. You will definitely need WebSockets today to make it happen. But logically, it is not that simple to create a collaborative environment. There are issues like broadcasting changes of every user to every other user viewing the same document. Then you will have to make sure that every user gets to see exactly the same thing no matter who changes what and when. This gets difficult when multiple users are working on the same document collaboratively. There can be conflicts while merging changes of one another.
As far as PHP is concerned, language is not really a problem. Its how you handle the above stated problem.
Google released its research and engineering details on Google Wave (I guess after abandoning the project). This link might be some help to you: http://www.waveprotocol.org/whitepapers/
Websockets are the way to go for this. However, they are difficult to implement in PHP, and not supported by many browsers as of yet. You can do long-polling of AJAX requests pretty easily with PHP, but it can be quite a waste of resources.
This might not be what you are looking for, but I'd suggest changing your server-side technology. You can use Node.js with Socket.IO to easily use Websockets (or seemlessly emulate them for browsers that don't support them). This will provide the fastest method for pushing data to your users.
I've been developing a site to be able to curate sports media using Django, and that's going reasonably well, but my friend that I'm working with has some of our required functionality (some information display, page-level stuff) going with PHP. Is there an easy way to integrate those, like maybe running the php through the Django templates, or should we try and convert some the functions to one language or the other?
In such cases, I think, it makes sense to make some kind of internal interface through which your sites would communicate, and expose only one of them to public. That would make everything more maintainable.
For example, your friend can make his PHP pages to output information in JSON or YAML. In corresponding Django views, you'll have little to no logic, just making internal HTTP requests to these pages, and basically passing the data to templates.
This way, you'll have output via Django templates, and some logic still in PHP. If PHP code does some work with database or performs computation that can't be converted easily enough to Python, and you have limited time, this option may be the best.
Though, I guess, it depends a lot on the architecture of the project, especially the PHP part. There's not enough information to say what's the best option for you.
Don't mix languages if you have any other option. Honestly, I don't think the type of integration you're imagining is even possible. About the closest you would ever get would be two separate websites that shared a common look and feel and passed info back and forth to each other. At the end of the day, though, there would always be a separation of management and data.
There's some pretty robust PHP frameworks out there. Assuming your friend is using one of those, you'll need to decide which best fits with the time and skills you both have to devote to the project. If your friend is not using a framework, the decision is simple: move everything over to Django.
My question is in bold (see below) but I hope to provide some insight into my issues just in case it helps anyone who could answer my question.
I'm not sure which framework or language is the best for this job but I'd like to make an app that has two or three independently scrollable areas, similar to a 2 column template, that are all dynamically updated. Users can populate each list separately as they go and yah know ... I really don't want the app to move slowly in any way. It needs to be jerky with almost knee-jerk reactions to input.
I've read some interesting things about speeds for each language that I'm considering (php and python - possibly java or ruby) and well, I can't really decide for myself since I don't yet know what's going on to create any slowdowns.
The app would be very simple requiring basic information on a user and letting the user basically grow their own database of lists themselves. Some fancy things would happen on the site per the users input or list updates like color changes or pictures and numbers that change .. maybe a graph or something.
All that said, I hope my question(s) is simple and answerable:
What is a useful programming language and/or framework for making and handling user created hierarchical lists that would ultimately grow in complexity? And how does the language differ from the others for tasks like ones as described in this post?
Just trying to find the right/best/effective tool for this particular job.
By the way, I'm new to programming but have covered the basic tutorials for python (plus some django) and php via youtube mostly - I've got a few books in the queue. be gentle
I don't think you will conclusively be able to decide on a language for your task based on clear evidence of it outperforming others for similar task.
There's too many variables to factor in.
My take is, a well built app, whatever the technology would perform as well as the other one with different technologies, but built well. The difference wouldn't be much to make a fuss over.
Some things that are of note:
The client side code, HTML, CSS and Javascript would need to be top notch. You need to
Combine that with a good host, a nicely constructed and tuned database, a good lean method of communication back and forth between client and server.
Use gzip, caching, minifying and combining scripts, stylesheets, make fewer http requestes,
Architect the application with performance in mind from the get go.
If you are new to programming then the language/framework will be the least of your problems. You tried a little Python and some PHP. Which language did you find easier to grasp? Whichever one that was, pick it and just start writing.
As you work on your project and become a better programmer you can revisit the language/framework debate (although, to be honest, the programmer influences the site performance more than the framework).