I'm looking for a simple framework that allow me to scrape some data on the web. In the past I used Slim to made a rest API, but now I'm going to make a web scraper. I already have experience with simple html dom parser, but I want to know if I Slim have a parser functionality.
In the documentation I can't find anything about it.
Most common frameworks are not really suited for this task and PHP is usually not a good choice as a language for scraping. There is a framework called Scrapy written in Python which is specific for scraping websites:
http://scrapy.org/
Slim offers just a quick and simple app skeleton: routes and middleware. There are no integrated tools for anything else.
I doubt there's a framework out there (as in web app framework: Zend/Laravel/Symfony etc.) providing anything that automates/helps with what you want.
If you really want to use PHP for web scraping, consider purchasing https://www.phparch.com/books/phparchitects-guide-to-web-scraping-with-php/.
Related
I'm attempting to build an application to aid engineers and designers in laying out o-ring grooves for sealing applications, a bit like this one.
I'd like to host it on my own website. The website is built on Joomla, and I'm trying to figure out the best way to integrate it. Ideally, the app shouldn't require Java or JS/ActiveX (as these are often blocked by modern browsers), but I'd also prefer to stay away from requiring a form submission to complete the calculations. I'm well versed in Python but have no experience with Django, and wonder if it might fit the bill for these requirements.
Right now the fallback solution is just to go with PHP and form submissions. If there is a way to get around that with Django or any other framework, I'd love to know about it.
python is not really capable of anything different than php. its just better organised. i think the solution to your wish of not having to use forms would be a heavy dose of javascript, html5 and ajax. also you need to think about how you want to include it in your joomla site. joomla is php, joomla components are written using php however it is possible to incorporate pretty much any web app with an iframe. but it tends to be ugly. so i'd recommend php over python in this case.
I want to develop an web application which i have Developed in c# previously.
Now I want to make the same banking application online. So please suggest me the process or Framework i should use to fulfill above.
I am new in web development but i have knowledge of JavaScript, PHP, CSS, HTML and more required for design and coding.
But i don't know how to use this knowledge please help me. Up-till now i have created UI for the project but i don't know where the code should reside how to call them etc.
Help will be Appreciated.
If you've already developed this in C# you can keep with C#. You can create C# web applications. I do believe, in fact, that this site is built using C#. :-)
If you want to learn another web language or technologies, there are plenty of tutorials, books, videos, and classes out there to help you.
Search around the web about developing web applications in .net. there should be plenty of articles covering that area. But basicly, i would say you should go for a Form based Web Application. That way you can make most use of the c# skills you already have. Read about that and the css html and javascript skills should give you a nice application. The deployment using IIS servers etc, you might need some help with if you dont have any experience with that. Maybe a nice article on the web could guide you through that as well.
Hope this helps!
Lookat sencha website,It has many products aimed on fast building web app, and you don't need to learn a lot about CSS,HTML,JS.
I would recommend you to try using Python(Django)or PHP(but I prefer the first one) on the server side + ExtJS for the UI(with ExtJS you can easily create great UIs).. But I don't really think that it's a good decision for business. For getting started - it's great but for business projects - I'm not sure..
As you already developed you application in C# then its better to use ASP.Net, so you can use your C# code as well, and if you don't have much knowledge in asp.net its really very easy to learn also, for that you have to go through some good books and some sites like asp.net etc
If you are already familiar with desktop development, you should take a look at Agile Toolkit, which is a web-based framework inspired by desktop development.
It is very helpful when you need a to produce a lot of inexpensive UI specifically for data management.
is there any way I can use Zend only for server process?. I want to have all my client process in javascript and the server process in php using Zend framework or any other framework.
Any ideas are welcome.
Zend is a component library that let you do php on the server side - So yes! you can use it to handle all your clients' process.
BTW, it's a good practice to leverage ZendFramework MVC so you will start with great concept/project structure etc'.
Good luck!
The answer is: of course!
ZF is a server-side framework. It's completely agnostic about it's output format. You'll find you have no problem outputting json, or xml, or whatever format you like.
While it's typical to have your application output HTML pages via Zend_View/Zend_Layout, there are other components available to facilitate sending non-HTML output..
If you're doing a fancy javascript-driven client, you might consider going the RESTful route with Zend_Rest_Route, and/or Zend_Rest_Controller.
In your comment, you mention concern about Zend_Form. You're free to ignore Zend_Form -- though you might want to reconsider. It's quite possible to leverage the filtering/validation functionality of Zend_Form (define a form, and just use populate(), isValid(), etc) without ever rendering a form. I did a project not long ago that was a single-page app using ExtJS on the client side, which which talked to a set of RESTful services in JSON provided by ZF. I used Zend_Form extensively just for filtering/validating data.
I'm writing an iPhone app as a hobby project and it will need a web service to provide it with data. It's not very different from what I do at work, but at work I only write views and controllers. Someone else is responsible for writing the model and usually the clients provide the web service.
I have done some web programming before, back when everyone were using MySQL and PHP, so my skills are a bit outdated, but I'm confident that I would be able to pull it of using the techniques I already know. However, I don't want to waste my time using obsolete tools. I've figured out that the state of the art would be to write a REST API. I was thinking that there should be some pretty good frameworks out there that pretty much just gives you a REST API with CRUD functionality as soon as you've defined a model.
I guess my question is: What would be the fastest way to get a REST API up and running? I really just want to focus on writing the iPhone app and not spend too much time on this API. It would be great if I could get web administration and revision history too. I should also add that the API isn't supposed to be public, so support for authentication would be great as well.
Just to be clear. I wouldn't mind a PHP framework. In fact it could possibly be better since I know that my current hosting supports it.
EDIT:
The links below which apparently were good for 3 years are no longer working so I went and found a couple of new tutorials that I think are going to stick around for a while. These are on the Ray Wenderlich site, a very well respected ios dev tutorial site. The first article actually references the broken links below but it is complete within itself:
How To Write A Simple PHP/MySQL Web Service for an iOS App
and the second one has a little twist to it. It used parse.com on the backend and AFNetworking. Both of which are quite excellent.
How To Synchronize Core Data with a Web Service – Part 1
I have fixed the broken links below by finding the articles in the way back machine. People seem to like the links so I will keep them. The links above should provide more food for thought.
I am doing exactly the same thing with my iphone app. I found this article on building a RESTful API in PHP:
https://web.archive.org/web/20130910164802/http://www.gen-x-design.com/archives/create-a-rest-api-with-php/
and there is also a followup article here:
https://web.archive.org/web/20130323001500/http://www.gen-x-design.com/archives/making-restful-requests-in-php/
with a link to source code at the bottom of the article.
I have programmed a REST API in ZEND Framework using the Zend_Rest_Controller, on the iPhone I used ASIHTTPRequest. My experience with both where good. At the beginning I had some trouble setting up ZEND and connecting it to mySQL, but once I figured out how to do it I was able to write the API very quickly. I can share more information with you if you have any further questions.
EDIT: There seems to be no official documentation on Zend_Rest_Controller. This link describes how to use it to create your API. You simply have to disable rendering in the init() of your subclass and implement the methods for each REST call.
Just to let you know:
I ended up using Ruby on Rails.
EDIT: Since this answer has been downvoted for not providing the reason behind choosing Ruby on Rails and also no instructions on how to write a REST API with it, I thought I would give you my motivation and some simple instructions.
I started reading a book about Ruby on Rails and realized that all I needed to do was to use scaffolding and I got a JSON REST API for free.
Here's a good guide to get you started: http://guides.rubyonrails.org/getting_started.html
When you have your Ruby on Rails environment up and running, creating your REST API isn't harder than running:
$ rails generate scaffold Post name:string title:string content:text
(Example from the above link.) I also found that Rails is very easy and free to deploy to heroku, which meant that I didn't have to pay for hosting for my very basic, low traffic, REST API. There are many other reasons why I am very happy to work with Ruby on Rails, but that's beyond the context of this question.
I followed a quite simple tutorial for creating RESTful APIs with PHP:
Corey Maynard - Creating a RESTful API with PHP
The main concept includes:
one abstract class that handles the parsing of the URI and returning the response, and
one concrete class that consists of just the endpoints for the API.
What about Python?
I'd use Python, Django and Piston.
I'd generate Django models from your
existent DB using inspectdb.
Add the Django admin to your models.
Add Django Piston to your app.
Profit.
With no experience with Python or Django probably it'll take you a day to develop this solution and all code is unit tested and proved to work.
If you want to use PHP I recommend using the CodeIgniter framework with Phil Sturgeon's REST server:
http://net.tutsplus.com/tutorials/php/working-with-restful-services-in-codeigniter-2
https://github.com/philsturgeon/codeigniter-restserver
Checkout the following PHP class that follows MVC.
http://www.phpclasses.org/package/5080-PHP-Implement-REST-Web-services-servers.html
Hope this helps.
If you already know PHP, there's nothing wrong with a PHP/MySQL backend. You can send all responses in iPhone-compatible plist xml format, and instantly turn the response into a NSDictionary/NSArray/NSNumber data structure with this short snippet of code:
NSString *response = [request responseString];
NSData* plistData = [response dataUsingEncoding:NSUTF8StringEncoding];
NSPropertyListFormat format;
NSString *errorStr;
NSDictionary* plist = [NSPropertyListSerialization propertyListFromData:plistData
mutabilityOption:NSPropertyListImmutable
format:&format
errorDescription:&errorStr];
I also use the ASIHTTP package for forming URLs, sending asynchronous requets, and receiving the responses, I highly recommend it:
http://allseeing-i.com/ASIHTTPRequest/
You should use whatever languages you are comfortable with for the web service. Any language that can formulate REST responses to requests is fine.
That said, if you want to get something running quickly, I suggest using Python on Google App Engine. It's free and you can use Java instead of Python if you so desire. App Engine supports authentication using OpenID and/or Google Accounts (not sure if they're mutually exclusive) so that should make things easier to code.
As far as making the requests on the iOS device, I suggest using ASIHTTPRequest.
Another option is restSQL, an ultra-lightweight persistence framework. See http://restsql.org. It supports MySQL and PostgreSQL and runs in a standard Java EE container, e.g. Apache Tomcat.
restSQL is a very unconventional data access layer. restSQL is not an object-oriented view of the database. It presents flat or hierarchical "views" of relational database tables. These views are query-able and updatable through a simple REST-based HTTP or Java API. The HTTP interface is based on REST principles, which use HTTP’s built-in features, rather than abstracting away from them.
You want a 'REST API with CRUD functionality' and that's exactly restSQL's sweet spot. You could do this with no code. Simply define your SQL Resources via XML files and start doing HTTP calls against them with full CRUD capability.
I need to implement a Web Service in PHP, but I've never written one in it (I nowadays use ASP.net which handles all that SOAP/WSDL/Disco stuff for me).
Are there any libraries/Frameworks for that? Should be as lightweight as possible as they wire into an existing system.
You might want to take a look at Zend Framework. It's an fully featured framework and it is basically just a bunch of loosely coupled components. You don't have to use anything you don't want to.
You can view the components here. Notice there is Zend_Soap and others.
Try NuSOAP it's lightweight and very easy to use.
PHP comes with a SOAP client built in:
http://us.php.net/soap
and there's always the ZF version:
Zend Framework: Zend_Soap