I've developed a reporting application in PHP. The application is built with HTML, CSS, javascript libraries, charting library(Highcharts) & MySQL to store data. The user chooses some options in the front end & clicks a "Submit button". Then the PHP layer executes a bunch of required SQLs & sends json result back to the UI where the charting & data tables are drawn.
The requirement now is, to be able to plug in a big data solution, Apache Spark to the existing application. I've been researching for the last 2 weeks on if I can in someway plug in the PHP application using REST API or some sort of Spark SQL driver to connect to Spark SQL server & execute the same set of SQLs that I have now, on the Spark SQL. I haven't hit a solution yet. I've now started researching on java based technologies such as Spring, others such as Angularjs, Nodejs other MVC frameworks to rewrite the project from scratch. I'm not a big fan of java development as I'm not a hardcore developer.(I build some handy tools to get things done).
I did read this - https://cwiki.apache.org/confluence/display/Hive/HiveClient#HiveClient-PHP, but looks like it's for a standalone spark installation. I'm dealing with a huge cluster in my case.
I'd highly appreciate any direction here please.
Yes it can be done by using a hive context and spark sql thrift server in spark application.
you can run your spark application and do all the processing. After processing if you are using a Data frame you have to just register it as a temporary table.
Now you can start a thrift server from the spark application.
After starting the thrift server you can query the temporary table and get the results and insights using proper jdbc divers in PHP.
refer the link below for more details
https://medium.com/#anicolaspp/apache-spark-as-a-distributed-sql-engine-4373e254e0f9#.ekc3cs28u
This might not what you want. But if you consider using Scala to build it. Here is one possible solution.
Having a web server which either using Spark standalone or connect to a cluster.
Using spark-highcharts to plot Spark DataFrame with highcharts
Writing some code which accept some option from web and let it execute in the backend web server.
Are you using any specific cluster, like cloudera or hortonworks?
In case of Cloudera, you should use Impala and corresponding JDBC drivers.
In HDP, you should use Spark Thrift Server, with corresponding JDBC drivers.
Related
So, I've been taking a course on Vue.js.
Before, I've made small projects where I've used jQuery to make ajax calls to local PHP files, that then connects to a MySQL database in phpmyadmin. Just a simple CRUD application.
Using Vue, I'm struggling to figure out how to connect with PHP and MySQL in a similar way. The course has gone into the Vue CLI and using webpack to set up a project and then import components and so on. I understand when using "run dev" it sets up a node server and displays the app. XXAMP, which I've used before, just loads a bank page.
Is there a way I can harness the full extent of Vue but used with a locally hosted PHP / MySQL backend? If not, what technology do I need to use / learn for a full stack app?
I suggest you step a bit out of your comfort zone for this one.
Doing it basic like that via XAMPP, etc, is fine, but why not use something like Laravel together with Vue. Even officially the frameworks collaborate, and there are lots of examples of combined usage.
Also, Laravel makes sure that it sets you up with a local development environment without much fuss, and also utilizes PHP's own built-in development server, rendering your need for XAMPP obsolete. For starters check out this tutorial.
Good luck and have fun!
I have to begin a new project and that's 2 years ago that I'm working on SAPUI5, But for severals reasons I have to use the Laravel 5.1 Framework to handle the database logic and controller to get and put data from and to the database.
I appreciate the power of SAPUI5 and I would like to use it for the front-end of my app.
My question is : It is possible to handle a server application with those two technologies?
My second question is : Which hosting I have to choose, because that my client want a kind of desktop app, and for this I thought to use node webkit package. And for that i have to launch a node server. In my goDaddy shared hosting i don't have this possibility.
Thank you for your help!
You can definitely combine Laravel as back-end technology with SAPUI5/OpenUI5 as front-end technology. As Laravel provides the front-end with restful services, you're probably best-off leveraging the JSONModel in your UI5 application.
Laravel is a PHP framework, so you'll need to look for a PHP host. If a host is capable of running PHP, it is also able to serve the static UI5 content.
If you want to package your app as a desktop application, NW.js will work, but Electron will do the job too. A very nice comparison between the two frameworks can be found here: http://tangiblejs.com/posts/nw-js-electron-compared.
Please do note that both frameworks turn your application into a desktop application and will run on the desktop. You won't need to find a server that runs node.js. You do need a server that runs PHP for your back-end Laravel logic.
I'm a newbie to AngularJS, and I've created an web application by using it.
I would like to know whether it is possible to create a Web application in AngularJS with database connectivity and without using any framework (like Ruby on Rails or PHP or any other)?
You can use AngularJS to do as little or as much as you would like on your web application. However AngularJS is a client side scripting language, if you need to communicate with a database you will need a server side language in order to do this. You ask if you are able to do this without "frameworks" and the answer if yes, but you cannot do it without a physical language like Ruby or PHP.
If you are looking for a lightweight framework to do simple REST requests for your AngularJS application I would highly recommend a ruby framework called Sinatra it takes minutes to set up, if you would want to connect to a database like MySQL or PostgreSQL then I would also suggest using DataMapper as your ORM. Alternatively you can create your own PHP script to avoid the use of a framework.
I'm thinking in create a webapplication with cakephp but consuming python's appengine webservice. But, to install cakephp etc, I need to configure the database. Appengine uses another kind of datastorage, with is different from mysql, etc.
I was thinking in store the data in appengine, and using the python webservices, and with the cakephp application comunicating with the webservice, for insert and retrieve data.
Is there any good resource for this, or is it unpossible.
Obs: also opened for a possibility for developing the webapplicaiton completely in python running in appengine. If anyone has a good resource.
Thanks.
I think that you should try different solution: http://aws.amazon.com/simpledb/
It appears that CakePHP is a MVC framework that's very similar to django, which is included in app engine for python. I'm not sure why you would want to store your data in google app engine, unless you're dealing with an extremely large amount of data, in which case you're likely comfortable enough working in python to just make the app work entirely on GAE.
See the official docs for more info:
http://code.google.com/appengine/docs/python/overview.html
http://code.google.com/appengine/articles/django.html
What you can do is run your CakePHP app on a standard LAMP web host and access the GAE Data Store through a REST or RPC web service. This isn't such a bad idea if you already have a CakePHP front-end that deals with RPCs in the backend, but if your Cake app stores all it's Models in MySQL it could take considerable effort to adapt it.CakePHP Models abstract their storage method using the DataSource class. You might be able to find a DataSource class that uses REST or RPC. However, if you don't have a very considerable investment in CakePHP Controllers and Templates I would suggest simply building your app entirely in GAE
You can not run PHP on GAE. If you run PHP somewhere, it is a bad architecture to go over the internet for your data. It will be slooooow and a nightmare to develop in.
You should store your data where you run your php, unless you must have a distributed, globally scaling architecture, which afaiu not the case.
There's a detailed tutorial on getting CakePHP up using the PHP runtime that Google recently announced. http://aymanrb.blogspot.com/2013/05/cakephp-deployment-on-google-app-engine.html
I've designed my database diagram(11 entites) for my prototype application. I need to have an REST API on top of this that will allow me to build a prototype android tablet application that talks to it.
Considering this is only for prototype purposes. What would be the quickest way to get a API up and running that would allow me to get, put, delete etc..
Security at this point isn't an issue so even if all add edit delete were open that would be fine.
My initial idea is to build this with PHP and MySQL maybe using the Yii framework to help but I feel there might be something that might be faster for my purpose.
Any thoughts or recommendations or advice?
It is worth looking into API rapid-prototyping tools like Apify (tutorial) since you're starting from scratch. If you're planning to have both an API and a Web site accessing the same data, consider using one framework for both as it helps prevent duplicating logic. CakePHP and Yii are good examples of multipurpose frameworks that are suited for this type of development.
If Python is an option, it is trivial to use a light-weight http framework to build a REST API that accesses a database.
Here's a snippet for a fully functioning http server using the itty framework: https://github.com/toastdriven/itty/blob/master/examples/web_service.py and https://github.com/toastdriven/itty/blob/master/examples/posting_data.py
Look at this project which allows you to create an API REST in seconds.
https://github.com/GeekyTheory/Automatic-API-REST
Features
Creation a powerful API REST of your MySQL Data Base in Seconds.
Management of the API in situ, it is not neccesary an extra data base.
Private tables and fields.
Custom queries.
Installation
Clone the repo.
git clone https://github.com/GeekyTheory/Automatic-API-REST/
Place it in your web folder /var/www/YourWebPage/
Open the file config.php and complete all the fields with the server credencials.
Go to domain.com/AutomaticaApiRest