i have php script it keep downloading feeds from the internet.
but unfortunately it keep duplicate the image feed into the database.
part from the image, the script working fine on other feed like description,link, title.
php script
<?php
require 'database.php';
$url = "http://www.albaldnews.com/rss.php?cat=24";
$rss = simplexml_load_file($url);
if($rss)
{
echo '<h1>'.$rss->channel->title.'</h1>';
echo '<li>'.$rss->channel->pubDate.'</li>';
$items = $rss->channel->item;
foreach($items as $item)
{
$enclosure = $item->enclosure[0]['url'];
$ch = curl_init ("$enclosure");
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_BINARYTRANSFER,1);
$rawdata=curl_exec ($ch);
curl_close ($ch);
$img=mysqli_real_escape_string($conn,$rawdata);
$query = "SELECT image from feedtable where link = '$img'";
$result= mysqli_query($conn, $query);
$num_rows = mysqli_num_rows($result);
if ($num_rows == 0)
{
$query1 = "INSERT INTO feedtable (image)VALUES ('$img')";
$result1= mysqli_query($conn, $query1);
echo "image added\n";
}
else
{
echo "duplicate entry\n";
}
}
}
?>
Related
I have a table where I want to run a PHP script during the maintenance.
I have the following code.
$sql = "SELECT `id`, `url` FROM `log_requests` WHERE `is_sent` = '0'";
$e = $conn->execute($sql);
My url array after fetching the table from DB.
Array
(
[0] => https://www.smsgatewaycenter.com/library/send_sms_2.php?UserName=username&Password=password&Type=Individual&To=9999999999&Mask=Senderid&Message=Hello+World+1
[1] => https://www.smsgatewaycenter.com/library/send_sms_2.php?UserName=username&Password=password&Type=Individual&To=9999999999&Mask=Senderid&Message=Hello+World+2
[2] => https://www.smsgatewaycenter.com/library/send_sms_2.php?UserName=username&Password=password&Type=Individual&To=9999999999&Mask=Senderid&Message=Hello+World+3
[3] => https://www.smsgatewaycenter.com/library/send_sms_2.php?UserName=username&Password=password&Type=Individual&To=9999999999&Mask=Senderid&Message=Hello+World+4
)
Then I run while loop
$ch = curl_init();
while($row = $e->FetchRow() ){
curl_setopt($ch, CURLOPT_URL, $row['url']);
curl_setopt($ch, CURLOPT_HEADER, 0);
$q = "UPDATE `log_requests` SET `is_sent` = '1' WHERE `id` = '".$row['id']."'";
$conn->execute($q);
}
curl_close($ch);
Issue is only one URL is executed using CURL and in my table only one row gets updated as 1 in is_sent column.
Why is it running single url in my while loop. What mistakes am I making here?
************************* EDIT ********************
Okay, now I came out with following and running successfully now. I just want to know whether am doing right or not.
$set = array();
while ($row = $executequery2->FetchRow()) {
$set[$row['url']][] = $row;
}
$curl_handles = array();
foreach($set as $url => $rows) {
$curl_handles[$url] = curl_init();
curl_setopt($curl_handles[$url], CURLOPT_URL, $url);
foreach($rows as $row) {
$query3 = "UPDATE `log_requests` SET `is_sent` = '1' WHERE `id` = '" . $row['id'] . "'";
$conn->Execute($query3);
}
}
$curl_multi_handle = curl_multi_init();
$i = 0;
$block = array();
foreach($curl_handles as $a_curl_handle) {
$i++;
curl_multi_add_handle($curl_multi_handle, $a_curl_handle);
$block[] = $a_curl_handle;
if (($i % 5 == 0) or ($i == count($curl_handles))) {
$running = NULL;
do {
$running_before = $running;
curl_multi_exec($curl_multi_handle, $running);
if ($running != $running_before) {
echo ("Waiting for $running sites to finish...<br />");
}
}
while ($running > 0);
foreach($block as $handle) {
$code = curl_getinfo($handle, CURLINFO_HTTP_CODE);
$curl_errno = curl_errno($handle);
$curl_error = curl_error($handle);
if ($curl_error) {
echo (" *** cURL error: ($curl_errno) $curl_error\n");
}
curl_multi_remove_handle($curl_multi_handle, $handle);
}
$block = array();
}
}
curl_multi_close($curl_multi_handle);
Updating the table column is the right way?
I had faced same issue and got this solution online. Just separate curl code from the loop and then try.
Something like this:
while($row = $e->FetchRow()) {
$result = call_curl($row['url']);
if($result === FALSE) {
die("Error");
}
else
{
$q = "UPDATE `log_requests` SET `is_sent` = '1' WHERE `id` = '".$row['id']."'";
}
}
function call_curl($url)
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0);
$result = curl_exec ($ch);
curl_close($ch);
return $result;
}
Hope this helps you as well.
I want to know how to explode or split xml and turn into array and then insert it to database. Because i have an api that need to hit every day and it return xml.
here's my xml sample:
<ArrayOfUnitPrice>
<UnitPrice>
<PriceAmount>1579.7080</PriceAmount>
<PriceDate>2016-09-02</PriceDate>
<PriceType>XWZ</PriceType>
</UnitPrice>
<UnitPrice>
<PriceAmount>1028.4137</PriceAmount>
<PriceDate>2016-09-02</PriceDate>
<PriceType>ABC</PriceType>
</UnitPrice>
...
</ArrayOfUnitPrice>
I'm using this code to extract the xml response:
$ch = curl_init("111.222.333.444:8080/code.asmx/Price");
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_POST,1);
curl_setopt($ch, CURLOPT_TIMEOUT, 60);
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_POSTFIELDS,$data);
$result=curl_exec ($ch);
//my code here
curl_close($ch);
SOLVED
I already done solve my code by using this code below guys.
$xml = simplexml_load_string($result);
for ($i=0; $i < count($xml) ; $i++) {
$arr[] = array (
'PriceAmount' => $xml->UnitPrice[$i]->PriceAmount,
'PriceDate' => $xml->UnitPrice[$i]->PriceDate,
'PriceType' => $xml->UnitPrice[$i]->PriceType
);
}
$data = json_decode(json_encode($arr), true);
$servername = "localhost";
$username = "root";
$password = "";
$dbname = "dailywork";
$conn = new mysqli($servername, $username, $password, $dbname);
if ($conn->connect_error) {
die("Connection failed: " . $conn->connect_error);
}
if(is_array($data)){
$check = "SELECT * FROM table_unit";
if($conn->query($check)->num_rows > 0){
// my stuff to here :-D
}else{
/*Insert data to DB*/
$sql = "INSERT INTO table_unit (PriceAmount, PriceDate, PriceType) values ";
$valuesArr = array();
foreach($data as $row){
$PriceAmount = $row[PriceAmount][0];
$PriceDate = $row[PriceDate][0];
$PriceType = $row[PriceType][0];
$valuesArr[] = "('$PriceAmount', '$PriceDate', '$PriceType')";
}
$sql .= implode(',', $valuesArr);
if ($conn->query($sql) === TRUE) {
echo "New record created successfully";
} else {
echo "Error: " . $sql . "<br>" . $conn->error;
}
/*End Insert data to DB*/
}
}
$conn->close();
This function will give you an array of your xml
function xml2php($xmlcontent)
{
$xml_parser = xml_parser_create();
xml_parse_into_struct($xml_parser, $xmlcontent, $arr_vals);
xml_parser_free($xml_parser);
return $arr_vals;
}
pass your $result like this and check it
xml2php($result);
let me know if it helps you
Hello there I've written some PHP that gets XML from a website, and will store it on a table on my database. It echo's Success, yet the table is blank? Heres my code:
<?php
$url ="http://rates.fxcm.com/RatesXML3";
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_URL, $url);
$data = curl_exec ($ch);
curl_close($ch);
$xml = simplexml_load_string($data);
$con = mysql_connect("localhost","username","password"); //this does contain proper information, just ive hidden it with the following
mysql_select_db("database", $con) or die(mysql_error()); //same for database too
foreach ($xml -> Rate as $row){
$Symbol = $row -> Symbol;
$Bid = $row -> Bid;
$Ask = $row -> Ask;
//performing sql query
$sql = "INSERT INTO 'FXCM_Rates' ('Symbol', 'Bid', 'Ask')"
. "VALUES ('$Symbol', '$Bid', '$Ask')";
$result = mysql_query($sql);
if (!result) {
echo 'MySQL ERROR';
} else{
echo 'SUCCESS';
}
}
?>
Now when I check the PHP, it has echoed success for all 63 Rates, yet when I go to check the table, it is blank? I just find it odd that it has worked for everyone else, but not me :(
Thanks to anyone that can help fix my code / point out my error.
You have '$' missing in the end.
if (!$result) {
Also now that I've done some tests, your variables are objects which I'm not sure that you can INSERT in database like that.
foreach ($xml -> Rate as $row){
$Symbol = (string)$row -> Symbol;
$Bid = (string)$row -> Bid;
$Ask = (string)$row -> Ask;
}
This will return strings that might be easier to use for you query
mysql is deprecated in newer version of php, you should be using mysqli(improved)..
<?php
$url ="http://rates.fxcm.com/RatesXML3";
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_URL, $url);
$data = curl_exec ($ch);
curl_close($ch);
$xml = simplexml_load_string($data);
$con = mysqli_connect("localhost","username","password","database") or die(mysqli_error($con));
foreach ($xml -> Rate as $row){
$Symbol = $row -> Symbol;
$Bid = $row -> Bid;
$Ask = $row -> Ask;
//performing sql query
$sql = "INSERT INTO 'FXCM_Rates' ('Symbol', 'Bid', 'Ask')"
. "VALUES ('$Symbol', '$Bid', '$Ask')";
$result = mysqli_query($con,$sql);
if (!$result) {
echo 'MySQL ERROR';
} else{
echo 'SUCCESS';
}
}
iam trying to make an php file to import xml to sql database, but i have this error-:
Warning: Invalid argument supplied for foreach() in /home/morrisse/public_html/xml_to_database.php on line 18
My code:
$url ="http://www.example/example.xml";
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_URL, $URL); //GET THE URL CONTENDS
$data = curl_exec ($ch); //execute the curl request
curl_close($ch);
$xml = simplexml_load_string($data);
foreach ($xml -> track as $row) {
$title = $row -> name;
$artist = $row -> artist;
$duration = $row -> duration;
// performing sql query
$sql = "INSERT INTO 'test_xml' ('title', 'artist', 'duration')"
. "VALUES ('$title', '$artist', '$duration')";
$result = mysql_query($sql);
if (!result) {
echo 'MYSQL ERROR';
} else {
echo 'SUCESSO';
}
}
my xml:
<lfm status="ok">
<track>
<id>602366984</id>
<name>Barbarism Begins At Home (BBC TV Tube performance)</name>
<mbid/>
<url>http://www.last.fm/music/The+Smiths/_/Barbarism+Begins+At+Home+(BBC+TV+Tube+performance)</url>
<duration>384000</duration>
<streamable fulltrack="0">0</streamable>
<listeners>10</listeners>
<playcount>24</playcount>
<artist>
<name>The Smiths</name>
<mbid>40f5d9e4-2de7-4f2d-ad41-e31a9a9fea27</mbid>
<url>http://www.last.fm/music/The+Smiths</url>
</artist>
<toptags>
</toptags>
</track>
</lfm>
help please?
The full code:
<?php
$url ="http://www.morrisseyradio.com/shoutcastinfo/cache/track.xml";
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_URL, $URL); //GET THE URL CONTENDS
$data = curl_exec ($ch); //execute the curl request
curl_close($ch);
$con=mysql_connect("localhost", "", ""); //connect to database
mysql_select_db("morrisse_forum", $con)
if (!$con) {
die('Could not connect: ' . mysql_error());
}; // select database
libxml_use_internal_errors(true);
$xml = simplexml_load_string($data);
if ($xml === false) {
$errors = libxml_get_errors();
foreach ($errors as $error) {
echo "XML Error at line $error->line: $error->message\n";
}
}
else {
foreach ($xml -> track as $row) {
$title = $row -> name;
$artist = $row -> artist;
$duration = $row -> duration;
// performing sql query
$sql = "INSERT INTO 'test_xml' ('title', 'artist', 'duration')"
. "VALUES ('$title', '$artist', '$duration')";
$result = mysql_query($sql);
if (!result) {
echo 'MYSQL ERROR';
} else {
echo 'SUCESSO';
}
}
}
?>
For anyone the answer is:
First thing:
in the line:
curl_setopt($ch, CURLOPT_URL, $URL);
the "$URL" must be Lower
Second:
In this line:
$sql = "INSERT INTO 'test_xml' ('title', 'artist', 'duration')"
the "'" must be "`"
It's pretty clear that your call to simplexml_load_string() failed because the xml was not well formed. When that happens you get back false rather than a usable xml element.
Here's what I suggest you do about this. Add some error-tracking code, as follows.
libxml_use_internal_errors(true);
$xml = simplexml_load_string($data);
if ($xml === false) {
$errors = libxml_get_errors();
foreach ($errors as $error) {
echo "XML Error at line $error->line: $error->message\n";
}
}
else {
foreach ( $xml->track as $row ) { /* your code continues here ... */
There's clearly something wrong with the xml you're getting back from that server you're using with curl. This code will help you figure out what is wrong.
I have three filed in my mysql table they are: id, url, status
How to check all urls from the column url and write either 1 (available) or 0 (unavailable) to the status column?
To just check urls manualy in php w/o mysql I could use this:
<?php
function Visit($url){
$agent = "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)";
$ch = curl_init();
curl_setopt ($ch, CURLOPT_URL,$url );
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($ch,CURLOPT_VERBOSE,false);
curl_setopt($ch, CURLOPT_TIMEOUT, 5);
$page=curl_exec($ch);
//echo curl_error($ch);
$httpcode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);
if($httpcode >= 200 && $httpcode < 300){
return true;
}
else {
return false;
}
}
if(Visit("http://www.google.com")){//maybe make it a variable from a result of a mysql select, but how to process it one by one?
echo "Website OK"; //maybe somesql here to wtite '1'
}
else{
echo "Website DOWN";//maybe somesql here to wtite '0'
}
?>
This is bad in terms of performance as doing queries in a loop is bad design, you should instead build a batch update and run the query in the end, but I've not enough time now to elaborate that. This is just to get you started:
$sql = "SELECT id,url FROM mytable";
$res = mysql_query($sql) or die(mysql_error());
if($res)
{
while($row = mysql_fetch_assoc($res))
{
$status = visit($row['url']) ? '1' : '0';
$id = $row['id'];
$update = "UPDATE mytable SET status = $status WHERE id = $id";
$res = mysql_query($update) or die(mysql_error());
}
}
echo mysql_num_rows($result) ? '1' : '0';
You can use sql queries for example:
select id, url from urltable;
update urltable set status=1 where id=99;
So, how about use this code with your Visit function:
$link = mysql_connect('localhost', 'test', 'pppppp');
$db_selected = mysql_select_db('test', $link);
$query = "select id, url from urltable";
$result = mysql_query($query);
while ($row = mysql_fetch_assoc($result)) {
$visitid = $row['id'];
$visiturl = $row['url'];
$visitstatus = Visit($visiturl)? 1: 0;
$upquery = sprintf("update urltable set status=%d where id=%d", $visitstatus, $visitid);
$upresult = mysql_query($upquery);
}