Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking for code must demonstrate a minimal understanding of the problem being solved. Include attempted solutions, why they didn't work, and the expected results. See also: Stack Overflow question checklist
Closed 9 years ago.
Improve this question
To give some background, I sell Lego parts online. The order total when you place the order is based on the price of the parts you purchased, and the shipping costs.
Shipping costs vary depending on the weight of the order, and the country of shipment.
I am not a techie buff, and thats why I need some help. I know the basics, but not much else, though I'd love to learn and I've been trying around with this for days before coming here.
The source code of an order page, the only place where you can see the weight is this:
<FONT CLASS="fv">Estimated Weight of Order:</FONT></TD><TD ALIGN="RIGHT"><FONT CLASS="fv">2.17oz 61.44g</FONT>
It is the same for every single order.
So, I know where the data I want is.
What I need help with is, coding something that pulls the data out of this webpage (say it's inside a webpage called order.com/order.asp and the document contains a bunch of other data apart from the weight) and exporting a shipping price based on the weight it inputed. I don't know whether you can do this with PHP or Python, etc.
I would have on my server a... say a table with the shipping costs based on weight. Now, what I needed, would be to take that bit of data from the order.com website into my own server. (On my own server process the weight data that I took, match it with the shipping cost, pull out invoices, etc). The weight data is in the order page, always on a line like the one I posted on the question. I just read about web scraping. Maybe some PHP that looks into the order page till it finds the line with the weight, and pulls out the weight?
Many, many, many thanks for your help, and I apologize in advance if I sound too uninformed, which I am. I really need a detailed explanation.
Gerald
*TL;DR*Two webpages. One is in my server and one isn't. The one that isn't in my server (order.asp), has this line:
<FONT CLASS="fv">Estimated Weight of Order:</FONT></TD><TD ALIGN="RIGHT"><FONT CLASS="fv">XX.XXoz XX.XXg</FONT>
I need something that I can put in my server, queries the weight from the page that isn't on my server (order.asp page) and matches the weight with a shipping price that I would have on my page (as a table or maybe with ifs).
There will be different order pages (order1.asp order2.asp order3.asp) with different weights. The script or whatever should do that for ea. wpage.
Thanks.
This would be the source code of an example page that I would need to take the weight of. Removed some sensitive info.
<SCRIPT TYPE="text/javascript" LANGUAGE="JavaScript">
function killImage(imgName){
if (document.images){
document.images[imgName].src="/images/noImage.gif"
}
}
function killImageM(imgName){
if (document.images){
document.images[imgName].src="/images/noImageM.gif"
}
}
</SCRIPT>
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=utf-8">
<META HTTP-EQUIV="IMAGETOOLBAR" CONTENT="NO">
<LINK REL="STYLESHEET" TYPE="text/css" HREF="/stylesheet.css?13">
<STYLE TYPE="text/css">body { margin: 15 auto; }</STYLE>
<SCRIPT TYPE="text/javascript" LANGUAGE="javascript" SRC="/js/getAjax.js"></SCRIPT>
<SCRIPT TYPE="text/javascript" LANGUAGE="javascript" SRC="/lytebox/lytebox.js?10"></SCRIPT>
<LINK REL="STYLESHEET" HREF="/lytebox/lytebox.css?13" TYPE="text/css" MEDIA="screen" />
</HEAD>
<BODY BGCOLOR="#666666">
<CENTER>
<TABLE WIDTH="680" CELLPADDING="10" CELLSPACING="0"><TR><TD BGCOLOR="#FFFFFF">
<TABLE BORDER="0" CELLPADDING="0" CELLSPACING="0"><TR>
<TD><IMG SRC="/images/logowhite.gif" WIDTH="200" HEIGHT="60" ALIGN="ABSMIDDLE" BORDER="0"> </TD>
<TD> <FONT SIZE="+3">Order #3953198</FONT></TD></TR></TABLE><P><FONT FACE="Tahoma,Arial" SIZE="2">
<HR NOSHADE SIZE="1" COLOR="#000000"><B>Order Summary</B><HR NOSHADE SIZE="1" COLOR="#000000">
<TABLE WIDTH="100%" CELLPADDING="5" CELLSPACING="0" BORDER="0" BGCOLOR="#EEEEEE"><TR><TD WIDTH="60%" VALIGN="TOP">
<TABLE WIDTH="100%" BORDER="0" CELLPADDING="1" CELLSPACING="0" CLASS="ta">
<TR>
<TD WIDTH="125">Order Date:</TD>
<TD>Nov 20, 2013 17:12</TD>
</TR>
<TR>
<TD>Payment By:</TD>
<TD>PayPal.com</TD>
</TR>
<TR>
<TD>Payment In:</TD>
<TD>Euro</TD>
</TR>
<TR VALIGN="TOP">
<TD>Order Status:</TD>
<TD>Shipped</TD>
</TR>
<TR>
<TD>Changed:</TD>
<TD>Nov 22, 2013 14:15</TD>
</TR>
<TR>
<TD NOWRAP>Total Items:</TD>
<TD>24</TD>
</TR>
<TR>
<TD NOWRAP>Unique Items (Lots):</TD>
<TD>2</TD>
</TR>
<TR>
<TD NOWRAP>Invoiced:</TD>
<TD>Nov 21, 2013 08:56</TD>
</TR>
<TR VALIGN="TOP">
<TD NOWRAP>Shipping Method:</TD>
<TD>Registered<BR><FONT CLASS="fv">By default, with tracking number and insured up to 30 euros only.</FONT></TD>
</TR>
</TABLE>
</TD><TD WIDTH="40%" VALIGN="TOP">
<TABLE WIDTH="100%" BORDER="0" CELLPADDING="1" CELLSPACING="0" CLASS="ta">
<TR>
<TD>Order Total:</TD>
<TD ALIGN="RIGHT">EUR 8.92</TD>
</TR>
<TR>
<TD>Shipping:</TD>
<TD ALIGN="RIGHT">EUR 4.85</TD>
</TR>
<TR>
<TD>Insurance:</TD>
<TD ALIGN="RIGHT">EUR 0.00</TD>
</TR>
<TR>
<TD>Additional Charges 1:</TD>
<TD ALIGN="RIGHT">EUR 0.00</TD>
</TR>
<TR>
<TD>Additional Charges 2:</TD>
<TD ALIGN="RIGHT">EUR 0.00</TD>
</TR>
<TR>
<TD>Credit:</TD>
<TD ALIGN="RIGHT">EUR 0.00</TD>
</TR>
<TR>
<TD>Grand Total:</TD>
<TD ALIGN="RIGHT"><B>EUR 13.77</TD>
</TR>
<TR>
<TD>Orders in this Store:</TD>
<TD ALIGN="RIGHT">1</TD>
</TR>
</TABLE>
</TD></TR>
</TABLE><HR NOSHADE SIZE="1" COLOR="#000000"><TABLE BORDER="0" CELLPADDING="0" CELLSPACING="0" WIDTH="100%" CLASS="ta"><TR><TD><B>Items in Order</B></TD></TR></TABLE><HR NOSHADE SIZE="1" COLOR="#000000"><TABLE WIDTH="100%" BORDER="0" CELLSPACING="1" CELLPADDING="3" CLASS="ta"><TR BGCOLOR="#C0C0C0"><TD><B>Image</B></TD><TD ALIGN="CENTER"><B>Condition</B></TD><TD><B>Item Description</B></TD><TD ALIGN="RIGHT"><B>Lots</B></TD><TD ALIGN="RIGHT"><B>Qty</B></TD><TD ALIGN="RIGHT"><B>Left</B></TD><TD ALIGN="RIGHT"><B>Price</B></TD><TD ALIGN="RIGHT"><B>Total</B></TD><TD ALIGN="RIGHT"><B>Weight</B></TD></TR><TR><TD COLSPAN="2" BGCOLOR="#C0C0C0"><B>Batch #1</B></TD><TD COLSPAN="7" BGCOLOR="#C0C0C0"><TABLE BORDER="0" CELLPADDING="0" CELLSPACING="0" WIDTH="100%"><TR><TD><FONT CLASS="fv">Submitted on Nov 20, 2013 17:12</TD><TD ALIGN="RIGHT"><IMG SRC="/images/printer16.png" WIDTH="16" HEIGHT="16" BORDER="0" ALT="Print Batch" TITLE="Print Batch"><IMG SRC="/images/dot.gif" WIDTH="5" HEIGHT="1"><IMG SRC="/images/invoice16YC.gif" WIDTH="16" HEIGHT="16" ALT="Batch Invoiced" TITLE="Batch Invoiced"></TD></TR></TABLE></TD></TR><TR BGCOLOR="FFFFFF"><TD HEIGHT="60"><CENTER><A ID='imgLink0' HREF='/catalogItemPic.asp?P=60208' REL='blcatimg'><IMG ALT="Lot ID: 48295541 Part No: 60208 Name: Wheel 31mm D. x 15mm Technic" TITLE="Lot ID: 48295541 Part No: 60208 Name: Wheel 31mm D. x 15mm Technic" BORDER='0' WIDTH='80' HEIGHT='60' SRC='http://img.bricklink.com/P/86/60208.gif' NAME='img0' ID='img0' onError="killImage('img0');"></A><BR><FONT FACE='Tahoma,Arial' SIZE='1'>*</FONT></TD><TD ALIGN="CENTER"><B>New</B></TD><TD><SPAN CLASS="u"><FONT COLOR="#000000">Light Bluish Gray Wheel 31mm D. x 15mm Technic </FONT></SPAN><BR><FONT CLASS="fv">AB4</FONT></TD><TD ALIGN="RIGHT"> </TD><TD ALIGN="RIGHT">12</TD><TD ALIGN="RIGHT">X</TD><TD ALIGN="RIGHT">EUR 0.11</TD><TD ALIGN="RIGHT">EUR 1.32</TD><TD ALIGN="RIGHT"><FONT CLASS="fv">38.16g</TD></TR><TR BGCOLOR="EEEEEE"><TD HEIGHT="60"><CENTER><A ID='imgLink1' HREF='/catalogItemPic.asp?P=6179' REL='blcatimg'><IMG ALT="Lot ID: 49014568 Part No: 6179 Name: Tile, Modified 4 x 4 with Studs on Edge" TITLE="Lot ID: 49014568 Part No: 6179 Name: Tile, Modified 4 x 4 with Studs on Edge" BORDER='0' WIDTH='80' HEIGHT='60' SRC='http://img.bricklink.com/P/86/6179.gif' NAME='img1' ID='img1' onError="killImage('img1');"></A><BR><FONT FACE='Tahoma,Arial' SIZE='1'>*</FONT></TD><TD ALIGN="CENTER"><B>New</B></TD><TD><SPAN CLASS="u"><FONT COLOR="#000000">Light Bluish Gray Tile, Modified 4 x 4 with Studs on Edge </FONT></SPAN><BR><FONT CLASS="fv">AJ2</FONT></TD><TD ALIGN="RIGHT"> </TD><TD ALIGN="RIGHT">12</TD><TD ALIGN="RIGHT">X</TD><TD ALIGN="RIGHT">EUR 0.633</TD><TD ALIGN="RIGHT">EUR 7.596</TD><TD ALIGN="RIGHT"><FONT CLASS="fv">23.28g</TD></TR><TR BGCOLOR="#DDDDDD"><TD COLSPAN="3"><B>Batch Total:</B></TD><TD ALIGN="RIGHT">2</TD><TD ALIGN="RIGHT">24</TD><TD></TD><TD> </TD><TD ALIGN="RIGHT">EUR 8.92</TD><TD ALIGN="RIGHT"><FONT CLASS="fv">61.44g</TD></TR><TR BGCOLOR="#C0C0C0"><TD COLSPAN="3"><B>Order Total:</B></TD><TD ALIGN="RIGHT">2</TD><TD ALIGN="RIGHT">24</TD><TD></TD><TD> </TD><TD ALIGN="RIGHT">EUR 8.92</TD><TD ALIGN="RIGHT"></TD></TR><TR><TD COLSPAN="10" ALIGN="RIGHT" BGCOLOR="#EEEEEE"><TABLE BORDER="0" CELLPADDING="0" CELLSPACING="0" WIDTH="100%"><TR><TD><FONT CLASS="fv">Estimated Weight of Order:</FONT></TD><TD ALIGN="RIGHT"><FONT CLASS="fv">2.17oz 61.44g</FONT></TD></TR></TABLE></TD></TR></TABLE><TABLE WIDTH="100%" BORDER="0" CELLPADDING="1" CELLSPACING="0" CLASS="ta"><TR><TD COLSPAN="2" CLASS="fv" ALIGN="RIGHT">Contact your buyer about this order<BR> </TD></TR></TABLE><HR NOSHADE SIZE="1" COLOR="#000000"><FONT CLASS="fv"><CENTER>This order will be purged from the BrickLink website on May 20, 2014.</CENTER></FONT></TABLE><FONT CLASS="fv"><P><CENTER><FONT COLOR="#FFFFFF">Back to Orders</FONT> | <FONT COLOR="#FFFFFF">Show Temporary Checkboxes</FONT> | <FONT COLOR="#FFFFFF">Show Categories</FONT> | <FONT COLOR="#FFFFFF">Consolidate Batches</FONT> | <FONT COLOR="#FFFFFF">My Settings</FONT><P><FONT COLOR="#FFFFFF">Hide Qty Left in My Inventory</FONT> | <FONT COLOR="#FFFFFF">Hide Item Weight</FONT> | <FONT COLOR="#FFFFFF">Show My Cost</FONT> | <FONT COLOR="#FFFFFF">Show Only Items in Order</FONT> | <FONT COLOR="#FFFFFF">Edit Order</FONT>
It's a little tough to write full-blown code without looking at the page you wish to scrape, but you should be able to use the following code to get what you want. The code below reads in a file called "html.txt", finds all orders in that text file, finds the total weight values in ozs and grams, and writes that data to an output file called foundWeights.txt. To run the code, just save your html in a text file called "html.txt", save the code below in a file called "findweights.py", and then put both of those files in the same folder. Then, open a shell or a terminal window, navigate to that folder, and type "python findweights.py" and momentarily a text file will appear in the same folder with your data in it.
html = open("html.txt").read()
out = open("foundWeights.txt", "w")
#split html on order number
legoOrders = html.split("Order #")
for order in legoOrders[1:]:
print order
orderNumber = order.split("<")[0]
weightString = order.split('Estimated Weight of Order:</FONT></TD><TD ALIGN="RIGHT"><FONT CLASS="fv">')[1]
splitWeightString = weightString.split(' ')
splitStringFinal = splitWeightString[1].split("<")
grams = splitStringFinal[0]
ozs = weightString.split(' ')[0]
out.write(str(orderNumber) + "\t" + str(grams) + "\t" + str(ozs) + "\n"
Outfile is tab-separated (Order #, grams, ozs):
3953198 61.44g 2.17oz
Related
Hello so basically I have an invoice email that gets sent out every night containing order information from todays orders. I am in the process of trying to automate this email with php and the mysql data but I am not quite sure what to do. Just an FYI I am new to html. I am used to bash and or python.
Below is a sample of the email html file and where I am trying to put order information
<!-- INSERT HERE Product quantity for Kit 1 Blue Dot -->
1</td>
<td width="30" class="wz2">
</td>
</tr>
<tr>
<td colspan="3" height="20" style="font-size:0;line-height:1;" class="va2">
</td>
</tr>
</table>
</th>
<th width="139" class="stack3" data-border-left-color="borderColor" data-
border-bottom-color="borderColor" style="border-left:1px solid
#dde5f1;border-bottom:1px solid #dde5f1;margin:0; padding:0;">
<table width="139" align="center" cellpadding="0" cellspacing="0" border="0"
class="table60033">
<tr>
<td colspan="3" height="20" style="font-size:0;line-height:1;" class="va2">
</td>
</tr>
<tr>
<td width="30" class="wz2">
</td>
<td class="RegularText5TD" data-link-style="text-decoration:none;
color:#67bffd;" data-link-color="RegularLink" data-color="RegularTXT"
style="color: #425065;font-family: sans-serif;font-size: 14px;font-weight:
lighter;text-align: center;line-height: 23px;">
<a href="#" target="_blank" data-color="RegularLink" style="text-decoration:
none;color: #67bffd;">
</a>
<!--INSERT HERE Total for Kit 1 Blue Dot-->
$22.00</td>
<td width="30" class="wz2">
</td>
</tr>
<tr>
<td colspan="3" height="20" style="font-size:0;line-height:1;" class="va2">
</td>
</tr>
What I would like to do is via php run the mysql query to populate and get the quantity of the product such as 'blue dot' shown here. Then take that quantity and multiply it by the price to get the total cost for the product. I got my queries and know what to run via php and grab the data. I just do not know how to get the data into this dynamic email template. I use phpmailer to mail this template. Any help would be great!
I am trying to pull each td element from the html table below and import each element into its own cell in a CSV file.
Here are the two html tables:
<table width="100%" border="0" cellspacing="1" cellpadding="0" bgcolor="#006699">
<tr align="center" class="tableRow1Font">
<td width="7%">WAITLIST</td>
<td width="5%">91630</td>
<td width="11%">
ACCY 2001
</td>
<td width="5%">10</td>
<td width="16%">Intro Financial Accounting</td>
<td width="6%">3.00</td>
<td width="8%"> Zou, Y</td>
<td width="8%"><A HREF="http://www.gwu.edu/~map/building.cfm?BLDG=DUQUES" target="_blank"
>DUQUES</a> 251</td>
<td width="13%">TR<br>09:35AM - 10:50AM</td>
<td width="14%">
01/13/14 - 04/28/14
</td>
<td width="7%">
</td>
</tr>
</table>
<table width="100%" border="0" cellspacing="1" cellpadding="0" bgcolor="#006699">
<tr align="center" class="tableRow2Font">
<td width="7%">WAITLIST</td>
<td width="5%">90003</td>
<td width="11%">
ACCY 2001
</td>
<td width="5%">11</td>
<td width="16%">Intro Financial Accounting</td>
<td width="6%">3.00</td>
<td width="8%"> Zou, Y</td>
<td width="8%"><A HREF="http://www.gwu.edu/~map/building.cfm?BLDG=DUQUES" target="_blank"
>DUQUES</a> 254</td>
<td width="13%">TR<br>11:10AM - 12:25PM</td>
<td width="14%">
1/13/14 - 04/28/14
</td>
<td width="7%">
</td>
</tr>
</table>
I have written code that goes through the tables and pulls the td elements:
foreach($html->find('tr[align=center] td') as $e)
$str .= strip_tags($e->innertext) . ', ';
echo $str;
So how can I extract these elements into a CSV file? In Excel I want it to look like this with each td element in its own cell, starting a new row for each html table:
WAITLIST 91630 ACCY 2001 10 Intro Financial Accounting 3.00 Zou, Y DUQUES 251 TR
WAITLIST 90003 ACCY 2001 11 Intro Financial Accounting 3.00 Zou, Y DUQUES 251 TR
There is a library exist for this. Goto http://phpexcel.codeplex.com/. Download the zip file and in example you would find 17html.php try this code. I hope this will help.
CSV means Comma Separated Values. Thus, as you echo out the data (after running it through your function to strip the <td> tags), put commas in between each piece of data (cell), and a new line where you want the next line to start.
So to use your example above, it should look like this:
WAITLIST,91630,ACCY,2001,10,Intro Financial Accounting,3.00,Zou,Y,DUQUES,251,TR
WAITLIST,90003,ACCY,2001,11,Intro Financial Accounting,3.00,Zou,Y,DUQUES,2,
Keep in mind that when you echo this, you shouldn't have any other html tags or anything.
i want to parse specific table for scrapping. the code of the table is given below..
<table class="NormalText" cellspacing="1" cellpadding="2" width="100%" border="0"
bgcolor="#eeeeee">
<tr>
<td width="108" align="center">
Stock No.
</td>
<td width="108" align="center">
<span id="invModule_grid_row18_lblMileage">Mileage</span>
</td>
<td width="108" align="center">
Color
</td>
<td width="76" align="center">
Interior
</td>
<td width="104" align="center">
Transmission
</td>
<td width="110" align="center">
Engine
</td>
</tr>
<tr>
<td width="108" align="center">
1204
</td>
<td width="108" align="center">
161,328
</td>
<td width="108" align="center">
Tan
</td>
<td width="76" align="center">
Leather
</td>
<td width="104" align="center">
Automatic
</td>
<td width="110" align="center">
3.5L V6 DOHC 16V
</td>
</tr>
<tr>
<td colspan="7" height="7">
</td>
</tr>
</table>
and the output i want is
1194 56,200 Blue Vinyl 5 Speed 6.8L V10 SOHC 30V
Questions
Which parsing Technique /Parser is best for this? PHPQuery, simplehtmlparse or xpath?
I am more familiar with domDocument, xpath and php, can it be done using xPath?
if yes, what will be xPath? (I am confused as my required data is in td and td tag has no id or class information attached. Also, on the uper row, which is basically a heading row, td are ther too)
Please guide me
XPath
The following example selects the text from all the td nodes in a table row in a table:
//table/tr[position()>1]/td/text()
You will have to know one of two things if there are other tables on the page:
Gets the last table:
//table[last()]/tr[position()>1]/td/text()
Gets the third table:
//table[2]/tr[position()>1]/td/text()
Gets a table based on an attribute, in this case, when class="NormalText":
//table[#class='NormalText']/tr[position()>1]/td/text()
I have something along the following lines in terms of HTML. I would like to extract the various contents of the table cells, however I discovered that there are some embedded divs occasionally in the cells and perhaps other oddities that I'm not sure of yet:
<p align="center">
<img src="some_image.gif" alt="Some Title">
</p>
<TABLE WIDTH=500 BORDER=1 class=textwhite ALIGN=center CELLPADDING=0 CELLSPACING=0>
<TR>
<TD colspan=4 ALIGN=center><b>Title</b></TD>
</TR>
<TR>
<TD ALIGN=center>Title</TD>
<TD ALIGN=center>date</TD>
<TD ALIGN=center>value</TD>
<TD ALIGN=center>value</TD>
</TR><TR>
<TD ALIGN=center>Title2</TD>
<TD ALIGN=center></TD>
<TD ALIGN=center><div class=redtext>----</div></TD>
<TD> </TD>
</TR><TR>
<TD ALIGN=center>Title3</TD>
<TD ALIGN=center><div class=yellowtext>value</div></TD>
<TD ALIGN=center><div class=redtext>value</div></TD>
<TD ALIGN=center>value<SUP>6</SUP></TD>
</TR><TR>
<TD ALIGN=center>Title4</TD>
<TD ALIGN=center><div class=bluetext>value</div></TD>
<TD ALIGN=center><div class=redtext>value</div></TD>
<TD> </TD>
</TR></TABLE>
<blockquote>
<p class="textstyle">
Text.
</p>
</blockquote>
My first impulse was to extract ALL element texts and just programmatically slice it up. I would watch for Title1, Title2, etc. to know when a row starts and then if a "----" is found meaning no value, just skip this row and move on. However, I realized that there is probably a better way of handling this with xpath directly.
How could this be solved with xpath so as to essentially give each cell's final child text content vs having to walk into each div if it exists? Or is there a more xpath like way to approach this?
Obviously I'm attempting to have the most flexible solution that will not be brittle if other unexpected elements crop up, even though they are unlikely.
The provided text isn't well-formed XML document, therefore XPath isn't applicable.
If you correct and covert it to a well-formed xml document as the one below, an expression like this might be useful:
/*/TABLE//TD//text()
or even:
//TABLE//TD//text()
Here is a wellformed XML document, constructed from the provided HTML:
<html>
<p align="center">
<img src="some_image.gif" alt="Some Title"/>
</p>
<TABLE WIDTH="500" BORDER="1" class="textwhite" ALIGN="center" CELLPADDING="0" CELLSPACING="0">
<TR>
<TD colspan="4" ALIGN="center">
<b>Title</b>
</TD>
</TR>
<TR>
<TD ALIGN="center">Title</TD>
<TD ALIGN="center">date</TD>
<TD ALIGN="center">value</TD>
<TD ALIGN="center">value</TD>
</TR>
<TR>
<TD ALIGN="center">Title2</TD>
<TD ALIGN="center"></TD>
<TD ALIGN="center">
<div class="redtext">----</div>
</TD>
<TD> </TD>
</TR>
<TR>
<TD ALIGN="center">Title3</TD>
<TD ALIGN="center">
<div class="yellowtext">value</div>
</TD>
<TD ALIGN="center">
<div class="redtext">value</div>
</TD>
<TD ALIGN="center">value
<SUP>6</SUP>
</TD>
</TR>
<TR>
<TD ALIGN="center">Title4</TD>
<TD ALIGN="center">
<div class="bluetext">value</div>
</TD>
<TD ALIGN="center">
<div class="redtext">value</div>
</TD>
<TD> </TD>
</TR>
</TABLE>
<blockquote>
<p class="textstyle"> Text. </p>
</blockquote>
</html>
So maybe you don't want to walk the divs, but here is my solution using lxml, which I highly recommend:
import re
from cStringIO import StringIO
from lxml import etree
def getTable(html, table_xpath, rows_xpath, cells_xpath):
"""Get a table on a webpage"""
parser = etree.HTMLParser()
# Build document tree and get table
root = etree.parse(StringIO(html), parser)
table = root.find(table_xpath)
if table == None:
print 'No table.'
return []
rows = table.findall(rows_xpath)
document = []
def cleanText(text):
"""Clean up text by replacing line breaks and tabs. """
return re.sub(r'[\r\n\t]+','',str(text).strip())
# iterate over the table rows and collect text from each cell.
for r in rows:
cells = r.findall(cells_xpath)
rowdata = []
for c in cells:
text = ''
it = c.itertext()
for i in it:
text += cleanText(i) + ' '
rowdata.append(text)
document.append(rowdata)
return document
html = """
<html><head><title></title></head><body>
<p align="center">
<img src="some_image.gif" alt="Some Title">
</p>
<TABLE WIDTH=500 BORDER=1 class=textwhite ALIGN=center CELLPADDING=0 CELLSPACING=0>
<TR>
<TD colspan=4 ALIGN=center><b>Title</b></TD>
</TR>
<TR>
<TD ALIGN=center>Title</TD>
<TD ALIGN=center>date</TD>
<TD ALIGN=center>value</TD>
<TD ALIGN=center>value</TD>
</TR><TR>
<TD ALIGN=center>Title2</TD>
<TD ALIGN=center></TD>
<TD ALIGN=center><div class=redtext>----</div></TD>
<TD> </TD>
</TR><TR>
<TD ALIGN=center>Title3</TD>
<TD ALIGN=center><div class=yellowtext>value</div></TD>
<TD ALIGN=center><div class=redtext>value</div></TD>
<TD ALIGN=center>value<SUP>6</SUP></TD>
</TR><TR>
<TD ALIGN=center>Title4</TD>
<TD ALIGN=center><div class=bluetext>value</div></TD>
<TD ALIGN=center><div class=redtext>value</div></TD>
<TD> </TD>
</TR></TABLE>
</body>
</html>
"""
tp = "//table[#width='500']"
rt = "tr"
cp = "td[#align='center']"
doc = getTable(html, tp, rt, cp)
print repr(doc)
I believe that your program is going to run into many problems as the input data is manipulated -- what if the case of 'title' changes, or there is a typo?
It's not really possible to make a rigorous solution to scraping someone else's website, as they can at no notice completely change everything. Better is normally to write tolerant and flexible code that at least tries to verify that its output is sane. In this case it's probably best to iterate over the results of '//table/tr', then inside this loop, process the td elements:
import lxml.etree
tree = lxml.etree.fromstring("<table><tr><td>test</td></tr><tr><td><div>test2</div></td></tr></table>")
stringify = lambda x : "".join(x.xpath(".//text()"))
for x in tree.xpath("//table/tr"):
print "New row"
for y in x.xpath("td"):
print stringify(y)
Output:
New row
test
New row
test2
The following code will, however, get the list you ask for:
print map(stringify, tree.xpath("//table/tr/td"))
Output:
['test', 'test2']
This will find all text elements which are at all descended from a td which is a direct descendant of a tr which is in turn a direct descendant of a table.
(Simply asking for all text() elements will create some funny bugs when run on HTML which contains "<td>Foo <b>bar</b></td>" or similar.)
I've trying to submit html emails to amazon's mechanical turk using the questionform xml data scheme. I'm having issues converting the html emails into well formed html data. I just input a script to grab it from my table and print the data inside the tags of the html email, but as you can see below, it's terribly formed and will not pass to mechanical turk - I've had to send the data as htmlentities() until now and this makes it difficult for HIT workers to easily solve my issues. Here's an example of how not well formed the data is - any tips on how to send this data through mechanical turk (php) or convert it to well formed html data would be appreciated.
<body text="#333333" bgcolor="#ffffff" link="#073064" vlink="#073064"
alink="#073064">
<a name="top"></a>
<table width="100%" cellspacing="0" cellpadding="10" bgcolor="#f4f2ee">
<tbody>
<tr>
<td valign="top" align="left">
<table width="600" cellspacing="0" cellpadding="0" bgcolor="#ffffff">
<tbody>
<tr>
<td style="background-color:#e8e6dd;background-image:none;background-repeat:repeat;background-position:top left;background-attachment:scroll;font-size:10px;color:#948765;line-height:200%;font-family:verdana;" >Email not displaying correctly?
<a
href="http://sitepointcom.cmail2.com/t/y/l/cvkit/ddktkrydd/r" style="color:#948765;" >View
it in your browser.</a></td>
</tr>
<tr>
<td height="93" bgcolor="#ff6501"
background="http://i1.cmail2.com/ei/y/4D/1C6/61A/231513/csimport/banner-tt_0.jpg">
<h1 style="font-size:30px;text-transform:lowercase;line-height:16px;color:#ffffff;font-family:Helvetica,Arial,sans-serif;text-indent:63px;margin-top:0;padding-top:29px;" >SitePoint <span
style="font-size: 17px; display: block; text-indent: 164px; color:
rgb(248, 255, 225); margin-top: 5px;">Tech Times</span></h1>
</td>
</tr>
<tr>
<td height="20" bgcolor="#C64F00" style="color:#e7fabd;font-family:arial;font-size:13px;" >
<span
id="Date" style="float:right;padding-left:5px;padding-right:5px;" ><strong>Issue 309:</strong> September 21,
2010 </span> Tips, Tricks, News and Reviews for Web Coders
</td>
</tr>
</tbody
</table>
<table width="600" cellspacing="0" cellpadding="0" bgcolor="#ffffff">
<tbody>
<tr>
<td colspan="5" height="10"></td>
</tr>
<tr>
<td width="10"></td>
<td rowspan="2" width="380" valign="top">
<table width="100%" border="0"
style="font-family:Verdana,Arial, Helvetica, sans-serif; font-size:13px;
color:#000">
<tr>
<td>
<a name='2'></a><h2 style="font-size:20px;font-weight:bold;color:#C64F00;font-family:arial;line-height:110%;" >
Introduction
</h2>
<p>
<img src="http://i2.cmail2.com/ei/y/4D/1C6/61A/231513/csimport/lisa-lang_1.jpg" height="119"
align="left" width="130" border="0" alt="Lisa Lang" /><strong><em>Sal
</em>Tech Timers! Every week we aim to provide you with a feast of tech
geekyness -- but this issue is particularly HUGE, with goodies for
everyone. This week, I'm proud to present our latest SitePoint release <a
href="http://sitepointcom.cmail2.com/t/y/l/cvkit/ddktkrydd/y"><em>Host Your Web Site in
the Cloud</em></a><em> </em>by web evangelist Jeff Barr. Everything you
need to know about cloud computing -- and how to make it work for you --
can be found in this book. </strong>
</p>
<p>
In celebration of this release, we'll be
running a live webinar with Jeff Barr, Kevin Yank, Lucas Chan, and Louis
Simoneau. The webinar will begin at 9:00 a.m. (Australian Eastern Standard
Time) on Wednesday, 22nd September. For those in the US, the meeting starts
at 4:00 p.m. (Pacific Standard Time) on Tuesday, 21st September. Places are
limited, so hurry to register now for free here!
</p>
<p>
In the meantime, to get you in the mood for the wonderful world of cloud
computing, have a read through Toby Tremayne's latest addition to his
series "What Cloud Computing Can Mean for Your Business." Toby
shows you how to get started, and introduces a wide range of handy (and
free) applications.
</p>
<p>
Next, the other big news of the week was the release of IE9. Craig
Buckler takes a look at its interface, including some new features and
development tools of this "Beauty of the Web."
</p>
<p>
And last but not least, James Edwards has some fun with shadows for
complex shapes. He shows you how to create a fancy solution in CSS, even
managing to make it work on all modern browsers.
</p>
<p>
That should keep you busy until next week. As always, feel free to come
over and join the discussions in our forums.
</p>
<p>
Keep rocking!
</p>
<hr color="#c5b172" size="1" />
<h2 style="line-height:1.2em;" ><a
href="http://sitepointcom.cmail2.com/t/y/l/cvkit/ddktkrydd/h"
style="color:red">Over 80% of Small Businesses Use Email Marketing ... But
Only a Handful Use It Effectively</a></h2><a
href="http://sitepointcom.cmail2.com/t/y/l/cvkit/ddktkrydd/k"><img
src="http://i3.cmail2.com/ei/y/4D/1C6/61A/231513/csimport/infusionsoft120x100-em20_2.jpg"
width="120" height="100" align="right" hspace="5" vspace="0" border="0"
/></a><p>Discover the secrets to effective, profitable email marketing
when you download the free report <em>"Email Marketing 2.0: the Three
Techniques That Will Actually Make a Difference In Your Email
Marketing."</em></p><p>Hint: this report does NOT cover subject line
suggestions, SPAM words to avoid, best time of day to send, or how to
address your contacts.</p><a
href="http://sitepointcom.cmail2.com/t/y/l/cvkit/ddktkrydd/u"
style="color:red"><strong>Don't wait -- Download your free copy
now!</strong></a><div
style="margin-bottom:2em;padding-bottom:1em;border-bottom: 1px dotted
#C5B172;"></div>
<p><strong>Summary</strong></p>
<ul style="font-size:110%;line-height:150%;" ><li>Introduction</li><li><a href="#5">What Cloud Computing Can
Mean for Your Business, Part II: Starting Your Cloud
Infrastructure</a></li><li>The IE9 Beta Review</li><li>Creating Shadows Around Polygons in CSS</li><li>New Technical Articles</li><li><a href="#12">Techy Forum
Threads</a></li><li>More Techy Blog Entries</li></ul>
<div style="margin-bottom:2em;padding-bottom:1em;border-bottom-width:1px;border-bottom-style:dotted;border-bottom-color:#C5B172;" ></div>
<a name='5'></a><h2 style="font-size:20px;font-weight:bold;color:#C64F00;font-family:arial;line-height:110%;" >
Starting Your Cloud Infrastructure
</h2>
<h3 style="font-size:16px;font-weight:bold;color:#C64F00;font-family:arial;line-height:110%;" >
What Cloud Computing Can Mean for Your Business, Part II
</h3>
<div>
Starting out, what do we need? We have to be able to communicate
with our customers and suppliers, so we need email, perhaps instant
messaging. If we have overseas or long distance clients, some kind of VOIP
phone would help to keep costs down.
</div>
<div>
We must ensure that anything we're working on is properly backed
up from the business plan to product concepts and beyond. Any loss
of data could be crucial when getting your product or service to market at
the right time. Keeping an eye on our schedule is vital to make sure
important events, tasks, and meetings are managed.<br />
</div>
<div>
There's a lot more to address, but this much is enough to get us
up and running so we can get about the business of doing business.
But if you're not an IT person and you don't know how to
setup email servers or backup systems, where do you begin?
</div>
<h3>
Email
</h3>
<div>
Google has been a provider of innovative products in the cloud for some
time, but many are unaware just how powerful these applications can be.
Gmail, for example, offers free email accounts with enormous amounts of
storage, and an easy-to-use interface. Your email can be accessed from
anywhere, you never have to delete as everything can be archived, and
backups are taken care of for you. The only issue is that you may want to
avoid sending emails to a big potential client from an anonymous, free
Gmail account; to make a proper impression, you need to be able to have
your own email address under your company name.
</div>
<div>
Google can still help you here, though; you can actually use the Gmail
system with your own domain name. As long as you have a <a
mce_href="http://sitepointcom.cmail2.com/t/y/l/cvkit/ddktkrydd/o"
href="http://sitepointcom.cmail2.com/t/y/l/cvkit/ddktkrydd/o">domain</a>, you can purchase a
Google Apps for Business account for the trifling sum of US$50 a year,
which lets you transfer your email hosting to Google's servers.
It's a very simple process, and once done you have full IMAP
access to your email from anywhere in the world, with a guarantee of
availability and uptime that few can compete with certainly not in a
small business.
</div>
<div>
The cost increases as you add more email accounts, but it's still
less than you'd pay to host a server with your own email software on
it. You can set up a normal email client like Outlook or Apple mail to use
the server, or use the user-friendly Gmail interface on the Web. This
means that no matter what happens, you'll always have access to email and
the ability to send from your own email address, even if your personal
computers fail. There are more great benefits to using Google Apps, but
we'll explore those in the next article. For now, check out <a
mce_href="http://sitepointcom.cmail2.com/t/y/l/cvkit/ddktkrydd/b" href="http://sitepointcom.cmail2.com/t/y/l/cvkit/ddktkrydd/b">Google Apps
for Business</a> to get up and running with your email.
</div>
<h3>
Messaging
</h3>
Note: I removed a lot of code here in the middle so that it would allow me to post the size of this document.
<hr color="#c5b172" size="1" /><h2
style="font-size:15px;font-weight:bold;color:#C64F00;font-family:arial;line-height:110%;"><a
name='11'></a>New Technical Articles</h2><h3><a
href="http://sitepointcom.cmail2.com/t/y/l/cvkit/ddktkrydd/tu"
style="color:#7B7B94"><strong>CSS3 Border Images for Beautiful, Flexible
Boxes</strong></a></h3>
<p><img src="http://i8.cmail2.com/ei/y/4D/1C6/61A/231513/csimport/author_louis_lazaris_17.jpg"
hspace="3" alt="Louis Lazaris" align="left" width="67" height="80" />Among
the raft of CSS3 features gaining increasing levels of browser support, the
border-image property is often overlooked. In this article, Louis gives us
the lowdown on what it is and how to use it.</p>
<p align="right"><a
href="http://sitepointcom.cmail2.com/t/y/l/cvkit/ddktkrydd/il"
style="color:#7B7B94">Full Story...</a></p><h3><a
href="http://sitepointcom.cmail2.com/t/y/l/cvkit/ddktkrydd/ir"
style="color:#7B7B94"><strong>HTML5 and Even Fancier
Forms</strong></a></h3>
<p><img src="http://i9.cmail2.com/ei/y/4D/1C6/61A/231513/csimport/author_tim_connell_18.jpg"
hspace="3" alt="Tim Connell" align="left" width="67" height="80" />Tim
Connell, co-author of SitePoint's Fancy Form Design, takes a look at
the new form input types available in HTML5, and gives you the skinny on
which ones you can start using right now.</p>
<p align="right"><a
href="http://sitepointcom.cmail2.com/t/y/l/cvkit/ddktkrydd/iy"
style="color:#7B7B94">Full Story...</a></p>
<h2
style="font-size:15px;font-weight:bold;color:#C64F00;font-family:arial;line-height:110%;margin-top:2em;padding-top:1em;border-top:
1px dotted #C5B172;"><a name='12'></a>Techy Forum Threads</h2><ul
class="forums" style="margin-left:18px;padding-left:0;" ><li><a
href="http://sitepointcom.cmail2.com/t/y/l/cvkit/ddktkrydd/ij"
style="color:#7B7B94">How do you organize your CSS?</a> in CSS</li><li><a
href="http://sitepointcom.cmail2.com/t/y/l/cvkit/ddktkrydd/it"
style="color:#7B7B94">Jack of all Trades...</a> in .NET</li><li><a
href="http://sitepointcom.cmail2.com/t/y/l/cvkit/ddktkrydd/ii"
style="color:#7B7B94">Personification of software: The contest</a> in
General Chat</li></ul>
<hr color="#c5b172" size="1" /><h2
style="font-size:15px;font-weight:bold;color:#C64F00;font-family:arial;line-height:110%;"><a
name='13'></a>More Techy Blog Entries</h2><p style="font-size:80%;color:#aea194;" >Web Tech</p><table cellpadding="0" cellspacing="0"
border="0" width="100%"><tbody><tr>
<td valign="top"><img
src="http://i10.cmail2.com/ei/y/4D/1C6/61A/231513/csimport/nlblog_19.gif" width="16"
height="19" /></td>
<td width="7"></td>
<td style="font-family:verdana;font-size:13px;" >
<a
href="http://sitepointcom.cmail2.com/t/y/l/cvkit/ddktkrydd/id"
style="color:#7B7B94">The Threat to Software Freedom</a>
</td>
</tr><tr>
<td colspan="2"></td>
<td><font size="-2" style="color:#AEA193;" >1 comment</td>
</tr><tr>
<td height="3"></td>
</tr><tr>
<td valign="top"><img
src="http://i10.cmail2.com/ei/y/4D/1C6/61A/231513/csimport/nlblog_19.gif" width="16"
height="19" /></td>
<td width="7"></td>
<td style="font-family:verdana;font-size:13px;" >
<a
href="http://sitepointcom.cmail2.com/t/y/l/cvkit/ddktkrydd/ih"
style="color:#7B7B94">Apple: Stuff Ups, Mistakes, and Finally Moving
Forward?</a>
</td>
</tr><tr>
<td colspan="2"></td>
<td><font size="-2" style="color:#AEA193;" >19 comments</td>
</tr><tr>
<td height="3"></td>
</tr><tr>
<td valign="top"><img
src="http://i10.cmail2.com/ei/y/4D/1C6/61A/231513/csimport/nlblog_19.gif" width="16"
height="19" /></td>
<td width="7"></td>
<td style="font-family:verdana;font-size:13px;" >
<a
href="http://sitepointcom.cmail2.com/t/y/l/cvkit/ddktkrydd/ik"
style="color:#7B7B94">Behind the Geek Out Scenes: Fancy Fonts and Jaunty
Input Fields</a>
</td>
</tr><tr>
<td colspan="2"></td>
<td><font size="-2" style="color:#AEA193;" >12 comments</td>
</tr><tr>
<td height="3"></td>
</tr></tbody></table><p style="font-size:80%;color:#aea194;" >JavaScript, CSS</p><table cellpadding="0" cellspacing="0"
border="0" width="100%"><tbody><tr>
<td valign="top"><img
src="http://i10.cmail2.com/ei/y/4D/1C6/61A/231513/csimport/nlblog_19.gif" width="16"
height="19" /></td>
<td width="7"></td>
<td style="font-family:verdana;font-size:13px;" >
<a
href="http://sitepointcom.cmail2.com/t/y/l/cvkit/ddktkrydd/iu"
style="color:#7B7B94">High-performance String Concatenation in
JavaScript</a>
</td>
</tr><tr>
<td colspan="2"></td>
<td><font size="-2" style="color:#AEA193;" >11 comments</td>
</tr><tr>
<td height="3"></td>
</tr></tbody></table><p style="font-size:80%;color:#aea194;" >Web
Design</p><table cellpadding="0" cellspacing="0" border="0"
width="100%"><tbody><tr>
<td valign="top"><img
src="http://i10.cmail2.com/ei/y/4D/1C6/61A/231513/csimport/nlblog_19.gif" width="16"
height="19" /></td>
<td width="7"></td>
<td style="font-family:verdana;font-size:13px;" >
<a
href="http://sitepointcom.cmail2.com/t/y/l/cvkit/ddktkrydd/dl"
style="color:#7B7B94">WordPress Trademark Transferred To WordPress
Foundation</a>
</td>
</tr><tr>
<td colspan="2"></td>
<td><font size="-2" style="color:#AEA193;" >2 comments</td>
</tr><tr>
<td height="3"></td>
</tr></tbody></table><p style="font-size:80%;color:#aea194;" >Community</p><table cellpadding="0" cellspacing="0" border="0"
width="100%"><tbody><tr>
<td valign="top"><img
src="http://i10.cmail2.com/ei/y/4D/1C6/61A/231513/csimport/nlblog_19.gif" width="16"
height="19" /></td>
<td width="7"></td>
<td style="font-family:verdana;font-size:13px;" >
<a
href="http://sitepointcom.cmail2.com/t/y/l/cvkit/ddktkrydd/dr"
style="color:#7B7B94">Important People With Things to Say</a>
</td>
</tr><tr>
<td height="3"></td>
</tr></tbody></table><hr color="#c5b172" size="1" />
</td>
<td width="10"></td>
</tr>
<tr>
<td width="10"></td>
<td width="20"
background="http://i5.cmail2.com/ei/y/4D/1C6/61A/231513/csimport/vertical-rule-bg_14.gif"></td>
<td width="180" valign="bottom" style="color:#000000;font-family:verdana;font-size:13px;" >
<div id="subscribe">
<h2 style="font-size:15px;font-weight:bold;color:#073064;font-family:Arial, Helvetica, sans-serif;line-height:110%;" >
Follow SitePoint on..
</h2>
<ul style="margin-left:5px;list-style-type:none;list-style-position:outside;list-style-image:none;padding-top:0;padding-bottom:0;padding-right:0;padding-left:0;" >
<li style="font-size:15px;" >
<img src="http://i1.cmail2.com/ei/y/4D/1C6/61A/231513/csimport/subs-nl_20.gif"
border="0" height="27" width="27" align="bottom" alt="Newsletters" /><a
style="margin-left:5px; text-decoration:none; font-weight:400"
href="http://sitepointcom.cmail2.com/t/y/l/cvkit/ddktkrydd/dy">Newsletters</a>
</li>
<li style="font-size:15px;" >
<img src="http://i2.cmail2.com/ei/y/4D/1C6/61A/231513/csimport/subs-tw_21.gif"
border="0" height="27" width="27" align="bottom" alt="Twitter" /><a
style="margin-left:5px; text-decoration:none; font-weight:400"
href="http://sitepointcom.cmail2.com/t/y/l/cvkit/ddktkrydd/dj" rel="nofollow">Twitter</a>
</li>
<li style="font-size:15px;" >
<img src="http://i3.cmail2.com/ei/y/4D/1C6/61A/231513/csimport/subs-m_22.gif" border="0"
height="27" width="27" align="bottom" alt="Mobile" /><a
style="margin-left:5px; text-decoration:none; font-weight:400"
href="http://sitepointcom.cmail2.com/t/y/l/cvkit/ddktkrydd/dt">Mobile</a>
</li>
<li style="font-size:15px;" >
<img src="http://i4.cmail2.com/ei/y/4D/1C6/61A/231513/csimport/subs-pod_23.gif"
border="0" height="27" width="27" align="bottom" alt="Podcast" /><a
style="margin-left:5px; text-decoration:none; font-weight:400"
href="http://sitepointcom.cmail2.com/t/y/l/cvkit/ddktkrydd/di">Podcast</a>
</li>
<li style="font-size:15px;" >
<img src="http://i5.cmail2.com/ei/y/4D/1C6/61A/231513/csimport/subs-rss_24.gif"
border="0" height="27" width="27" align="bottom" alt="RSS" /><a
style="margin-left:5px; text-decoration:none; font-weight:400"
href="http://sitepointcom.cmail2.com/t/y/l/cvkit/ddktkrydd/dd">RSS</a>
</li>
<li style="font-size:15px;" >
<img src="http://i6.cmail2.com/ei/y/4D/1C6/61A/231513/csimport/subs-fb_25.gif"
border="0" height="27" width="27" align="bottom" alt="Facebook" /><a
style="margin-left:5px; text-decoration:none; font-weight:400"
href="http://sitepointcom.cmail2.com/t/y/l/cvkit/ddktkrydd/dh" rel="nofollow">Facebook</a>
</li>
</ul>
</div>
<h2
style="font-size:15px;font-weight:bold;color:#C64F00;font-family:arial;line-height:110%;">Help
Your Friends Out</h2>
<p>People you care about can benefit from the wealth of information on
new
and maturing technologies available on the Internet. Help them learn
how to do it by forwarding them this issue of the Tech Times!</p>
<!--[if gte mso 0]><div style="display:none;" ><![endif]-->
<table cellspacing="0" cellpadding="0" bgcolor="#F9990C">
<tr>
<td colspan="3" height="3" bgcolor="#C2721C"></td>
</tr>
<tr>
<td colspan="3" height="10"></td>
</tr>
<tr>
<td width="10"></td>
<td style="color:#ffffff;font-family:verdana;font-size:13px;font-weight:bold;" >Send this to a friend</td>
<td width="20"></td>
</tr>
<tr>
<td colspan="3" height="5"></td>
</tr>
<tr>
<td width="10"></td>
<td style="font-family:verdana;font-size:13px;" >
<form autocomplete="on"
action="http://www.sitepoint.com/newsletter/forward" method="get"
style="margin: 0pt;">
<input name="newsletterid" value="3" type="hidden">
<input name="fromemail" value="rcavezza#gmail.com" type="hidden">
<input name="issuenum" value="309" type="hidden">
<input autocomplete="on" name="email" value="friend#example.com"
style="width: 120px;" type="text">
<br />
<input autocomplete="on" name="Send" value="Send" type="submit">
</form>
</td>
<td width="20"></td>
</tr>
<tr>
<td colspan="3" height="10"></td>
</tr>
</table>
<!--[if gte mso 0]></div><![endif]-->
</td>
<td width="10"></td>
</tr>
<tr>
<td colspan="5" height="20"></td>
</tr>
<tr>
<td colspan="5" height="10" bgcolor="#9999BC"></td>
</tr>
<tr>
<td bgcolor="#D7D7E5"></td>
<td bgcolor="#D7D7E5" colspan="4" style="font-family:verdana;font-size:10px;color:#5e5e91;" >
<table width="100%" cellspacing="0" cellpadding="0"
style="font-size:12px">
<tbody>
<tr>
<td width="100%" height="10" bgcolor="#D7D7E5"></td>
<td rowspan="3" bgcolor="#032a5c" style="color:#FFF;font-size:12px;text-align:center;" >
We send this newsletter using Campaign Monitor<br/>
<br/>
<a
href="http://sitepointcom.cmail2.com/t/y/l/cvkit/ddktkrydd/dk"><img
src="http://i7.cmail2.com/ei/y/4D/1C6/61A/231513/csimport/cm-passive-200x125_26.png" width="200"
height="125" hspace="10" border="0" alt="Campaign Monitor"/></a>
</td>
</tr>
<tr>
<td style="font-size:12x!important;font-family:arial,
verdana;" >
<p style="font-weight:bold;color:#353553;" >You are
subscribed as: <br>
<span style="font-size:13px;color:#CE6E11;font-weight:700;" ><code>rcavezza#gmail.com</code></span></p>
<ul>
<li><a href="http://sitepointcom.cmail2.com/t/y/u/cvkit/ddktkrydd/"
style="color:#7B7B94">Unsubscribe</a> from this list.</li>
<li><a href="http://sitepointcom.cmail2.com/t/y/l/cvkit/ddktkrydd/du"
style="color:#7B7B94">Manage your subscriptions</a>.</li>
<li><a href="http://sitepointcom.cmail2.com/t/y/l/cvkit/ddktkrydd/hl"
style="color:#7B7B94">View the newsletter archives</a>.</li>
</ul>
<p>
<span style="font-weight:bold;color:#353553;" >Mailing
Address:</span><br />
<span style="font-size:12px;" >48 Cambridge St, Collingwood, VIC,
3066 Australia</span>
</p>
<p><strong><span style="color:#353553;" >Phone:</span> +61 3
9090 8200</strong></p>
</td>
</tr>
<tr>
<td height="10" bgcolor="#D7D7E5"></td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
<img src="https://cmail2.com/t/y/o/cvkit/ddktkrydd/o.gif" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;" ></body>
EDIT: Just changed it to fix the issues below, and it's still not passing the validation test - any additional steps I should take/try?
It is not that badly formed. Just call quoted_printable_decode() on it first.
edit: well, it solves a few problems, but it is still misformed as *********. Whatever possessed them not to quote whole lists of style declarations?
edit2: Ah, Bob removed the quotes all on his own. I assume with leaving the quotes there & quoted printable decode it would be solved.
Yea that looks like a mess, have you looked into or tried something like htmlPurifier?
There are a few others, but I do not know them as the only one I ever used was the htmlPurifier, but you may want to look into that (if that is what you are asking for).
You can use tidy to repair your HTML. But it looks very bad so you should start with fixing the script that produces the HTML before.
On a windows machine you might have to add or uncomment the following line in your php.ini to be able to use it:
extension=php_tidy.dll
Some very basic example from the documentation:
$html = '<p>test</I>';
$tidy = tidy_parse_string($html);
$tidy->cleanRepair();
echo $tidy;
This will output the following:
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 3.2//EN">
<html>
<head>
<title></title>
</head>
<body>
<p>test</p>
</body>
</html>