I am trying to include a file which is scraping all my data from varrious websites, however its not working. Heres my code.
firstly, the scraping php file. named scrapedata.php
<?php
//Include the Simple HTML DOM to use its functions, used by other following scripts.
//navigate to the content of variable html and save it in price data variable
//get the whole html of the webpage into variable html
include 'simple_html_dom.php';
$html = file_get_html('http://www.play.com/Electronics/Electronics/4-/16230382/New-Apple-iPod-Touch-8GB-4th-Gen/Product.html?');
$price_data = $html->find('h6[id=bodycontent_0_middlecontent_1_ctl00_ctl00_product_ctl00__overview_ctl00__dataControl__price__headerSix',0)->plaintext;
//Amazon.co.uk scrape
$amazon_html = file_get_html('http://www.amazon.co.uk/New-Apple-iPod-touch-Generation/dp/B0040GIZTI/ref=br_lf_m_1000333483_1_1_img?ie=UTF8&s=electronics&pf_rd_p=229345967&pf_rd_s=center-3&pf_rd_t=1401&pf_rd_i=1000333483&pf_rd_m=A3P5ROKL5A1OLE&pf_rd_r=1ZW9HJW2KN2C2MTRJH60');
$amazon_pd = $amazon_html->find('b[class=priceLarge]',0)->innertext;
libxml_use_internal_errors(true);
$dom = new DOMDocument();
$dom->loadHTML($amazon_html);
$xpath = new DOMXpath($dom);
$expr = "/html/body/div[#id='divsinglecolumnminwidth']/form[#id='handleBuy']/table[3]/tr[3]/td/div/span";
$nodes = $xpath->query($expr); // returns DOMNodeList object
// you can check length property i.e. $nodes->length
//echo $nodes->item(0)->nodeValue; // get first DOMNode object and its value
$stock_data = $nodes->item(0)->nodeValue;
if ( $stock_data == "In stock." ) {
$stockyesno = "Yes";
} else {
$stockyesno = "No";
}
//Currys scrape
$currys_html = file_get_html('http://www.currys.co.uk/gbuk/apple-new-ipod-touch-8gb-4th-generation-07677427-pdt.html');
$currys_pd = $currys_html->find('p[class=prd-amount]',0)->plaintext;
$currys_stk = $currys_html->find('/html/body/div/div/div[2]/div/div/div[2]/div/ul[2]/li/span')->plaintext;
//span[class=icon icon-check]',0);
echo $currys_stk;
if ( $currys_stk == "Available for home delivery" ) {
$currys_stockyesno = "Yes";
} else {
$currys_stockyesno = "No";
}
//Argos scrape
$argos_html = file_get_html('http://www.argos.co.uk/static/Product/partNumber/9282197/Trail/searchtext%3EIPOD+TOUCH.htm');
$argos_pd = $argos_html->find('span[class=actualprice]',0)->plaintext;
//Ebuyer scrape
$ebuyer_html = file_get_html('http://www.ebuyer.com/product/237805');
$ebuyer_pd = $ebuyer_html->find('span[class=now]',0)->plaintext;
//PcWorld scrape
$pcworld_html = file_get_html('http://www.pcworld.co.uk/gbuk/apple-new-ipod-touch-8gb-4th-generation-07677427-pdt.html');
$pcworld_pd = $pcworld_html->find('p[class=prd-amount]',0)->plaintext;
?>
and then my page where its included, to then which its meant to access the data in the variables from the included file.
<?php include 'scrapedata.php';?>
<!-- MYSQL DATABASE CODE -->
<?php
$db_host = 'localhost';
$db_user = 'admin';
$db_pwd = '1admin';
$database = 'stock_checker';
$table = 'price_stock';
if (!mysql_connect($db_host, $db_user, $db_pwd))
die("Can't connect to database");
if (!mysql_select_db($database))
die("Can't select database");
?>
<!-- MYSQL DATABASE CODE END-->
//Insert the scraped data into the database.
mysql_query("INSERT INTO price_stock (retailer,price) VALUES('Play.com', '$play_pd' )")
or die(mysql_error());
mysql_query("INSERT INTO price_stock (retailer,price,stock) VALUES('Amazon', '$amazon_pd', '$stockyesno' )")
or die(mysql_error());
mysql_query("INSERT INTO price_stock (retailer,price,stock) VALUES('Currys', '$currys_pd', '$currys_stk' )")
or die(mysql_error());
mysql_query("INSERT INTO price_stock (retailer,price) VALUES('Argos', '$argos_pd' )")
or die(mysql_error());
mysql_query("INSERT INTO price_stock (retailer,price) VALUES('eBuyer', '$ebuyer_pd' )")
or die(mysql_error());
mysql_query("INSERT INTO price_stock (retailer,price) VALUES('PCWorld', '$pcworld_pd' )")
or die(mysql_error());
?>
<!-- MYSQL DATABASE % TABLE CREATION CODE -->
<?php
// sending query
$result = mysql_query("SELECT * FROM {$table}");
if (!$result) {
die("Query to show fields from table failed");
}
$fields_num = mysql_num_fields($result);
echo "<table width='650px'><tr>";
// printing table headers
for($i=0; $i<$fields_num; $i++)
{
$field = mysql_fetch_field($result);
echo "<td><b>{$field->name}</b></td>";
}
echo "</tr>\n";
// printing table rows
while($row = mysql_fetch_row($result))
{
echo "<tr>";
// $row is array... foreach( .. ) puts every element
// of $row to $cell variable
foreach($row as $cell)
echo "<td>$cell</td>";
echo "</tr>\n";
}
mysql_free_result($result);
?>
Have I done this correctly ? I don't know if they would automattically be included, or they have to be made GLOBAL ?
This is a very bad idea.
You should be using functions; passing and returning values. Include files at the beginning of the script, call functions when needed. Do not put any freestanding (non-function) code in files you include.
(By the way, the next step is OOP and autoloaders.)
And in case you're wondering why this is a very bad idea: I've looked over your code 5 times now (though with no in-depth analysis) and still haven't figured out what variables you want to share between the two files. If I went line-by-line, I'd find it, but I don't want to go line-by-line. Neither do you, in 3 months, when you're updating the code. Make our job easier.
--
From a point of view, an object is a collection of functions and the state they share; so it's the third step here: no functions -> some functions -> functions grouped together in classes. Don't even know if it's any good, but PHP has this to say about OOP. Autoloading is the mechanism PHP uses to load classes on-demand, usually saving you from includes.
They don't need to be imported explicitly.
Try this example:
<?php
// a.php
$myVar = 123;
and
<?php
// b.php
include 'a.php'
echo isset($myVar)?'Included':'Not included';
Related to your posted code: I don't see where you add data to the database. Shouldn't you do that as well?
It's a bad idea to use DOM Parsing for such requests. Amazon released a simple API - read here: http://aws.amazon.com/de/
PHP variables in the included file will appear in the main file by the same name.
But in your code I don't see variables being reused at all? The scrape generates information in variables, then your main file reads an unrelated database table. The two operations need to relate somehow.
Related
How to display this php code when php script is executed.
I only want to display these syntax not the result of the syntax.
This is for a making tutorial web site.
//sample code
$con = mysql_connect ("localhost", "root", "") or die(mysql_error ());
$db = mysql_select_db ("arraydb") or die(mysql_error ());
$i=0;
$sql = 'SELECT * FROM user_info WHERE i_id=130';
$query = mysql_query($sql);
foreach($result = mysql_fetch_assoc($query) as $key => $value)
{
echo '<pre>['.$key.' - '.$value.']</pre>';
}
echo '<hr/>%%<br><br><br>';
p.s i tried to type code without php tags then
1.html tags are not display
2.text indenting format is not visible in result
Assuming your code is in "sample.php" you could add the following line at the begining of your file or somewhere before the first "echo":
highlight_file('sample.php');
Note that you will see this line as well in the code sample.
You can try this http://micmap.org/php-by-example/en/function/highlight_string to get an idea of what it produces.
Did you try something like this ?
<?php
highlight_file(__FILE__);
echo '<br>';
// begin sample code
$a = 1;
$b = 2;
echo $a + $b;
// end sample code, result displayed below
use enclosed tags
< pre >
< /pre >
I can't get the below code to successfully prevent duplication when inserting data into the table:
<?php
include('simple_html_dom.php');
// Create DOM from URL
$html = file_get_html('http://thesite.com/example/');
// Find all article blocks
$con = mysql_connect("localhost","admin","password");
mysql_select_db("dataminer", $con);
if (!$con)
{
die(mysql_error());
}else {
foreach($html->find('li.search-result') as $sitelisting){
$escapedlisting = mysql_real_escape_string($sitelisting);
$debugquery = mysql_query("INSERT IGNORE INTO extract (listing) VALUES ('$escapedlisting')");
if (!$debugquery)
{
die(mysql_error());
}
}
}
mysql_close($con);
?>
Despite the INSERT IGNORE, it's still duplicating every entry every time the script is ran.
EDIT: Fixed this by changing:
foreach($html->find('li.search-result') as $sitelisting){
$escapedlisting = mysql_real_escape_string($sitelisting);
$debugquery = mysql_query("INSERT IGNORE INTO extract (listing) VALUES ('$escapedlisting')");
to:
foreach($html->find('li.search-result') as $sitelisting){
$item['address'] = mysql_real_escape_string($sitelisting->find('div.adr-flag', 0)->plaintext);
$item['desc'] = mysql_real_escape_string($sitelisting->find('ul.attributes', 0)->plaintext);
$debugquery = mysql_query("INSERT IGNORE INTO extract VALUES ('$item[address]', '$item[desc]')");
I'm trying to create an online database of the music I locally have on my computer. I already have a table of every song in my library and would like to create a different table with just the song ID's (auto-incremented when I added the songs to my main table) that act as one of my playlists.
From what I can tell, the only 100% certifiable method for matching the songs is by the location on my disk ( C:\Users\username\Music\iTunes\iTunes Media\Music\Jonathan Coulton and GLaDOS\Portal\128 128 Still Alive Duet.mp3 ) as an example. In my PHP code I get the song location into a variable and if I print it relative to the equivalent song location in my MYSql table, they match up exactly but when I try to run a select statement using that, it gives me an error. From what I can tell this is being caused by the backslashes in the location info.
This is the select statement I'm using,
SELECT id FROM itunes WHERE Location=$locationval
where $locationval is the current song's location, id is the autoincremented id in my main table, and itunes is my main table.
Is there any way around this? And as I am a beginner, is the backslashes really the issue?
For reference here is the full code for importing the playlist, its using the DB plugin for PEAR (a PHP extension).
<?php
// define table name
define('TABLE_NAME', 'playlist');
// create database connection
require_once('DB.php');
$dsn = 'mysql://username:password#localhost/itunes';
$DB =& DB::connect($dsn);
if (DB::isError($DB)) {
die($DB->getMessage());
}
$DB->setFetchMode(DB_FETCHMODE_ASSOC);
// load text file
$file = file_get_contents('Portal.txt');
// explode on new line
$file = explode("\r", $file);
set_time_limit(0);
// loop through each line in the file
foreach ($file as $key => $value) {
// explode on tab to get column list
$exploded = explode("\t", $value);
// check for first row, which contains column headers
if ($key == 0) {
}
else{
if(count($exploded)>3)
{
$locationval=$exploded[26];
echo $exploded[26];
echo "<br />";
$result = mysql_query("SELECT id FROM itunes WHERE Location=$locationval");
//$result = mysql_query("SELECT * FROM itunes WHERE id=8292");
set_time_limit(0);
$row = mysql_fetch_row($result);
//test statements to see if the query worked
echo "Test: ";
echo $row['id'];
echo $row['Location'];
echo "<br />";
}
}
}
?>
Which was modified from the code here: http://ericlondon.com/posts/208-exporting-itunes-data-into-mysql-and-creating-sql-to-show-top-rated-albums
If any more info is needed, please let me know.
You need to escape the backslashes when using it in mysql as a string literal, so just replace your following line:
$result = mysql_query("SELECT id FROM itunes WHERE Location=$locationval");
for this one:
$result = mysql_query("SELECT id FROM itunes WHERE Location='". str_replace('\\','\\\\',$locationval) . "'");
that should do what you want.
I've started making a Video script that loads the videos using MySql and I am using Mysqli.
However, There's 2 Rows that it should post, but it only post the second none, not the first one.
It loads the results using "Brand" so if there's 2 rows named "Test", it only loads the second one, but not the first one.
So, what is causing this? I've tried with 3 rows, and it did not include the first row.
Code
<?php
{ /* Global Data */
ini_set('display_errors', 0);
ini_set('error_reporting', -0);
$GetPath = $_GET['b'];
$SqlUser = "root";
$SqlPass = "**Private**";
$SqlHost = "localhost";
$SqlData = "heisteknikk";
}
{ /* Mysql Connect */
$Sql = new mysqli($SqlHost, $SqlUser, $SqlPass, $SqlData);
if ($Sql->connect_error) { die("Sorry, Could not connect (".$Sql->connect_errno.") ".$Sql->connect_error);}
}
{ /* Test */
$Brand = $Sql->real_escape_string($GetPath);
$SqlData = "SELECT * FROM videos WHERE Brand = '".$Brand."'";
$SqlQuery = $Sql->query($SqlData);
if (!$SqlQuery) {
echo $Sql->error;
}
if ($SqlQuery->num_rows == 0) {
die("Nothing was found");
}
$Data = $SqlQuery->fetch_array(MYSQLI_ASSOC);
echo "<table border='1'>";
while ($Heis = $SqlQuery->fetch_assoc()) {
echo "
<tr><td>".$Heis['Brand']."</td><td>".$Heis['Name']."</td><td>".$Heis['Location']."
";
}
echo "</table>";
$Sql->close();
}
?>
This line causes the bug:
$Data = $SqlQuery->fetch_array(MYSQLI_ASSOC);
With this line you effectively throw out the first row of the result set, as you don't process $Data at all in the code below. Just remove it, and your script should work fine (I assume that closing </td></tr> sequence was lost when pasting the code, am I right?)
Comment out (or better yet, remove) this line:
$Data = $SqlQuery->fetch_array(MYSQLI_ASSOC);
It is grabbing the first row...then you aren't doing anything with $Data and then grabbing the second (and consecutive rows) for display in your while loop. Commenting out that line above will make your loop grab and display them starting at the first one.
In your code
$Data = $SqlQuery->fetch_array(MYSQLI_ASSOC);
echo "<table border='1'>";
while ($Heis = $SqlQuery->fetch_assoc()) {
you're fetching a row (with fetch_array), throwing it away and then fetching another row (with fetch_assoc). That's probably why you're only seeing one row instead of two
I’m sorting a list and using Ajax to update a database. I need help parsing the string. This is the query string that I need to parse:
images_list[]=32&images_list[]=95&images_list[]=97&images_list[]=96&images_list[]=102&images_list[]=103&images_list[]=99&images_list[]=101&images_list[]=98&john=hi
I have put john=hi to test to see if the string is actually being sent via Ajax to the processor.php file. I am able to get the variable john=hi from the URL, so the query string is being sent perfectly fine. This is my code so far, but I can't seem to access the data I need, it appears as if nothing is there:
<?php
// Connect to the database
require_once('connect.php');
parse_str($_GET['images_list']);
for ($i = 0; $i < count($images_list); $i++) {
$id = $images_list[$i];
mysql_query("UPDATE images SET ranking = '$i' WHERE id = '$id'");
echo $images_list[$i];
}
?>
$_GET['images_list'] is an array of integers. There's nothing to parse there. PHP already did it for use. So, skip the parse_str part and easily use $_GET['images_list'] instead of $images_list.
Whole code:
<?php
//Connect to DB
require_once('connect.php');
foreach ($_GET['images_list'] as $i => $id) {
mysql_query("UPDATE images SET ranking = " .
mysql_real_escape_string($i) .
" WHERE id = " .
mysql_real_escape_string($id));
echo $id;
}
?>