There are a bunch of HTML text extraction tools out there. Mostly for Java or Python. The one I come across most often is boilerpipe. There are a few APIs here and there, and some seem to work pretty well. Does anyone know of anything in PHP that does this?
You could try phpQuery:
http://code.google.com/p/phpquery/
DomDocument is a class available in PHP if you have libxml support that can parse HTML documents and let you iterate over them or issue XPath queries to find specific nodes in the DOM tree. This is the ideal method.
Or, if the text is simple enough and uniform, you can use preg_match() to extract text from the data using Regular Expressions.
Related
There are 365 codes that is
<sometag class=“day” date=“yyyy-mm-dd” count=“some Int”></sometag>
I have to parse date and count with php, beautiful soup or any parsing library which can be use with php and make JSON String
How can I parse? Every tags have same class name that is “day”
I will be waiting for more answers for more wide info. Thank you.
Agree with #Scuzzy. SimpleXML would be the easiest and cleanest way way for this very specific example. Problem with SimpleXML, I have found it to be very slow if you are parsing larger documents.
There's also this library for parsing html which I think you will find very useful if you're screen scraping with php which it looks like you are doing.
http://www.schrenk.com/nostarch/webbots/DSP_download.php
I'm trying to set up a translation tool to translate websites. What I want to do is import html-code and get all translatable texts from that site.
One idea would be to use strip_tags, but it would ignore strings that could be translated such as alt-texts, title-texts and probably others that I don't have on my mind yet. Is there a clean way to do this?
In this case you need to parse HTML and extract text yourself. As you, probably, already know, parsing HTML with regular expressions is A Bad Idea (tm). SO, the only right solution is to parse DOM of the document. On this step you are free to use any tools including standard DOMDocument class.
If you are looking for some libraries or scripts to help, i would suggest to look on html2text which could be used commercially. As i see, it doesn't support attributes for <img> tags, but it's very easy to fix (use <a> tag as example).
If you are looking for some automated text extraction, then you should definitely look on something like Bolierpipe.
I would personally use the DOM Crowler component from Symfony2, which is a nice wrapper around php DOM functions and start from there.
I've been trying to do some simple DOM parsing of HTML documents and am really shocked at how difficult it is to do.
I've looked into some of the many alternatives to PHP's DOM classes (like simple xml parser and simple HTML DOM). I found a very effective dom2array function too, which is useful for extremely basic parsing where you just want raw values of elements.
None of these alternatives is really compelling though.
PHP documentation of the DOM is typically lacking in detail and largely useless. A lot of the comments are actually really helpful though.
The tutorials I've found online typically cover only the very very basics like writing a 20 line XML document or parsing all the p tags in a document. Meh.
Are there any sites (or books) that go into detail specifically on working with the DOM using PHP's DOM libraries?
The DOM is a language-independent interface and documented in detail by the W3C.
That being said, if your aim is extremely simple parsing of (typically) structured information, XML may not be the correct format in the first place; XML includes a variety of advanced features (namespaces, DTDs, XSLT, distinction between attributes and text, markup instead of structured information). If that's the case, consider JSON, which is extremely easy to parse and generate.
Anything that says "DOM" in the name or claims to support it should support the DOM API as defined by the W3C, and you should consider their documentation normative for everything but the language-specific parts.
I should have titled my post, "Easiest way to parse HTML DOM in PHP". 'Easiest' is not a very good word, I know. It's all relative to what you're trying to do. What I'm doing is pretty straight-forward. I want to parse standalone HTML documents and present the content in a different context.
These are the things I wanted to do:
Parse basic properties like title and body
Alter all file references (images, links, css, js) to point to a valid location
Add/remove attributes from tags (dealing with 1995 HTML here)
Strip inline styles
I ended up going with Simple HTML DOM Parser
It has a very small learning curve and gives easy read/write access to the DOM. End of story. It does seem to choke on nested elements sometimes though.
Recently read this SO Post ...first answer is nutz. Basically it is theoretically impossible for large models because of Chomsky Grammars Types.
What it the alternative? I don't want to use a library object like DOMDocument, I want to understand what is the correct way to do this with pure code?
If you don't want to use DOMDocument (though I'd urge you to look into it again, it's not that bad - especially combined with DOMXPath), you can also use PHPQuery or Simple HTML DOM Parser.
I recently stumbled upon this:
PCRE Regex Syntax - Recursive Patterns
It appears to open up possibilities to "match" html tags, which regular expresisons were not good at. Can this experimental feature, in any way, be used to parse fragments of HTML? or the document, if possible?
It is highly recommended not to use regex for HTML parsing, recursion or no. People often use it because when you have a hammer, the world looks like a nail. The correct tool would be something more like PHP's DOMDocument class, which is fully designed for solving exactly this type of problem.
http://php.net/manual/en/class.domdocument.php