I have a script to get certain data from a page based on the content of a span ID
However there are 200+ pages of results to trawl through and it only displays 127 results on each page
The script I have does get data for the 127 elements that are on the first page however it wont then open the new page and continuing to get data
It just stops after the initial 127
Any help would be great
$end = 200;
$start = 1;
$stop = $start + 10;
$html = file_get_contents('http://example.com/res/'.$start);
$doc = new DOMDocument();
#$doc->loadHTML($html);
echo $stop;
$i = 0;
foreach($doc->getElementsByTagName('span') as $element ) { //Loops through all available span elements
if (!empty($element->attributes->getNamedItem('id')->value)) { // Discards irrelevant span elements based on their `ID`. A similar sorting is achieved with `empty()` as the target `span` doesn't have any associated `ID`.
echo "Record : ".$i.' '. $element->attributes->getNamedItem('id')->value."\n";
$i++;
$end = $start;
}
}
if($i == 127) {
$i = 0;
do {
$next = $start++;
$page = $next;
$html = file_get_contents('http://example.com/res/'.$page);
$doc = new DOMDocument();
#$doc->loadHTML($html);
foreach($doc->getElementsByTagName('span') as $element )
{
if (!empty($element->attributes->getNamedItem('id')->value))
{
echo "Record : ".$i.' '. $element->attributes->getNamedItem('id')->value."\n";
$i++;
$end = $start;
}
}
} while ($page != $stop);
//echo $i.' Records';
}
As stated in comments, since your last record displayed in the first loop is 127, the line after that echo increments $i from 127 to 128.
foreach($doc->getElementsByTagName('span') as $element ) { //Loops through all available span elements
if (!empty($element->attributes->getNamedItem('id')->value)) { // Discards irrelevant span elements based on their `ID`. A similar sorting is achieved with `empty()` as the target `span` doesn't have any associated `ID`.
echo "Record : ".$i.' '. $element->attributes->getNamedItem('id')->value."\n";
$i++; //At last iteration, $i = 128
$end = $start;
}
}
And then, if($i == 127) will be false.
I suggest you to change the condition to if($i >= 127)
Related
I have a code that removes only the xml tags that expired and list the other tags, but if one tag expired it will delete it and stop the foreach loop execution without listing the other tags. if I reload the page after the code finished removing the expired tag it will list the other tags normally without any problem. how can I make it continue to list the other tags?
php code:
$xml_file = simplexml_load_file("xml_file.xml");
foreach ($xml_file as $item)
{
$current_date = time();
$article_date = (int)$item->date;
$item_number = (int)str_replace("a" , "" ,$item->getName());
if ($current_date >= $article_date + 100000)
{
if ($item->children()->getName() == "a")
{
$dom = dom_import_simplexml($item);
$dom->parentNode->removeChild($dom);
$return = simplexml_import_dom($dom);
$xml_file->asXML('xml_file.xml');
unlink('file.html');
}
}
elseif ($current_date < $article_date + 100000)
{
echo 'hello';
}
}
xml code:
<articles>
<a1><a>gr</a><date>14</date></a1>
<a2><a>gr</a><date>1414141414141414</date></a2>
<a3><a>gr</a><date>1414141414141414</date></a3></articles>
this code should remove the first tag and print hello twice, but it just removes the first tag and stop the foreach loop execution without printing any thing, and if I reload the page after deleting the first tag it prints hello twice without any problem.
Some lines are commented as their purpose is unclear ... You can remove, but not with foreach loop and you have to start from the end... otherwise it is like removing the chair under yourself - loop is unclear, whether it should start with the 'new' $item, or jump over it.
$children = $xml_file->children();
for($i = count($children) - 1; $i >= 0; $i--)
{
$item = $children[$i];
$current_date = time();
$article_date = (int)$item->date;
$item_number = (int)str_replace("a" , "" ,$item->getName());
if ($current_date >= $article_date + 100000)
{
if ($item->children()->getName() == "a")
{
$dom = dom_import_simplexml($item);
$dom->parentNode->removeChild($dom);
// $return = simplexml_import_dom($dom);
// $xml_file->asXML('xml_file.xml');
// unlink('file.html');
}
}
elseif ($current_date < $article_date + 100000)
{
echo 'hello';
}
}
var_dump($xml_file);
Another way to remove child without conversion to DOM is
$children = &$xml_file->children();
// the rest is the same, but replace
// $dom = dom_import_simplexml($item);
// $dom->parentNode->removeChild($dom);
// with
unset($children[$i]);
I am trying to construct a php script that will page through an API. The api return ~25197 XML records. I am able to pass a start_offset and a end_offset to the API which will return a subset of the results.
The challenge I am having is that the for loop is not capturing the remaining records that are not within the 1000.
Example, the current for loop processes the records in blocks of 1000 (0-1000,1001-2000,2001-3000, etc.) I am not able to get the final block - 25,000 to 26,000. The for loop stop processing at 24,000 - 25,000. This leaves me with 197 unprocessed XML results.
<?php
//Set Start and Offset Parameters
$start_offset = 0;
$end_offset = 0;
$items_per_page = 1000;
$number = 0;
$counter = -2;
for ($count=0; $count<=100; $count++) {
$counter++;
//Validate that the counter is not null
if ($number != null){
echo "\n";
echo file_get_contents($static_url . "/sc_vuln_query-compliance.php?start=$start_offset&end=$end_offset&seq=$counter");
}
//Initialize the start and end offset variables
$end_offset = $number+=$items_per_page;
$start_offset = $number-$items_per_page+1;
//We want to start at record 0, reset start_offset back to 0 instead of 1
if($start_offset == 1) {
$start_offset = $number-$items_per_page;
}
// We are at the end of the total records, display the remaining
if ($number>$total_xml_records) {
$counter = $counter+1;
$padding = $end_offset + $items_per_page;
echo "\n";
echo file_get_contents($static_url. "/sc_vuln_query-compliance.php?start=$start_offset&end=$padding&seq=$counter");
break;
}
}
?>
Your code is at least buggy at this line $padding = $end_offset + $items_per_page; - here you raise the end for your last loop by another 1000 and therefor get ?start=25001&end=27000&seq=25.
Try $padding = $total_xml_records; instead, this will get you ?start=25001&end=25197&seq=25.
Anyway your code is quite complicated. Try this:
$total_xml_records = 25197; // index 0 .. 25196
$offset = 0;
$counter = 0;
while ($offset < $total_xml_records) {
echo "\n";
echo $static_url . "/sc_vuln_query-compliance.php?start=".($offset)."&end=".(min($offset+$items_per_page-1, $total_xml_records-1))."&seq=".($counter++);
$offset += $items_per_page;
}
I'm working on a printing a baseball team lineup, via php. I want to print a place holder for a missing Player 6 (or any missing position)
So if Player 1 -> Player 5 is OK print, NO Player #6 print place holder, Player 7 -> Player 9 is OK print. I tried to simplify the code. I have tried solving this every which way but I keep getting stuck.
CODE:
$rot = array();
$pos = array();
$jn = array();
$x = 1;
// loads up the arrays from the db
while ( $rot[$x], $pos[$x], $jn[$x])= $r->fetch_row() ) {
$x++;
}
// counts the actual number of players in linuep
// used for validation and error display
$num_players = mysqli_num_rows($r);
// controls the lineup position
for ($i = 1; $i <= 15; $i++){
if($rot[$i] == $i) {
//prints player
$lineup .= "<div data-fp='" . $pos[$i] . "'>" .$jn[$i]. "</div>";
} else {
// prints place holder
$text = "This Position needs to be filled before the next game.";
$lineup .= "<div id='pid' data-rel='".$text."' data-fp='' data-pid='' data-jn='' title=''>x</div>";
}
}
I also tried this to iterate through the array rot[] to find the matching position and print the line but it actually prints the holder repeatedly.
// controls the lineup position
for ($x = 1; $x <= 15; $x++){
for ($i = 1; $i <= ($num_players+1); $i++) {
if ($x == $i) {
//prints player
$lineup .= "<div data-fp='" . $pos[$i] . "'>" .$jn[$i]. "</div>";
} else {
// prints place holder
$text = "This Position needs to be filled before the next game.";
$lineup .= "<div id='pid' data-rel='".$text."' data-fp='' data-pid='' data-jn='' title=''>x</div>";
}
}
}
What about:
# index all players by position while taking them from the database
$players = array();
while ( $row = $r->fetch_row() ) {
list($rot, $pos, $jn) = $row;
$players[$pos] = compact(array('rot', $pos, $jn);
}
...
# line-up players
for ($pos = 1; $pos <= 15; $pos++)
{
$playerExists = isset($players[$pos]);
if ($playerExists)
{
# do this ...
}
else
{
# do that ...
}
}
I think you are creating an array where all numerical elements are filled (i.e. you'll always have a 1 thru 15) and your mistake is in the
if($rot[$i] == $i) {
When populating the arrays from the database, add this line:
$playertoid = array_flip($pos); # pos is the player number array?
i.e.
while ( ($rot[$x], $pos[$x], $jn[$x])= $r->fetch_row() ) {
$x++;
}
$playertoid = array_flip($pos);
Now you've created a reverse lookup table where the index is the player number.
Replace the
if($rot[$i] == $i) {
line with:
if (isset($playertoid[$i])) {
This script is supposed to to get a multidimensional array and iterate through the values.
The array size is 10 and each element should contain an associative array:
$games[0] => array('foo' => 'bar')
$games[1] => array('foo1' => 'bar1')
etc..
The while loop should iterate 5 times in this example. The for loop should iterate 10 times for each iteration of the while loop.
So I am expecting the echo to be:
countwhile = 5 countfor = 50 totalgames = 50
but im actually getting
countwhile = 5 countfor = 150 totalgames = 150
I believe $games array is not the problem because i have have made that call below before and have used print_r to view the contents and it is as expected.
This whole code is not in a function or class just as is on my index.php page, could the problem be to do with the variable scopes?
$totalruns = 5;
$endindx = 10;
$startindx = 0;
$countwhile = 0;
$countfor = 0;
$totalfilesize = 0;
$totalgames = 0;
$sizeof = 0;
while($totalruns > 0)
{
$games = $feedHandler->getGames($startindx, $endindx);
$sizeof = sizeof($games);
for($i=0; $i<$sizeof; $i++)
{
$totalfilesize += $games[$i]['swf_file_size'];
$countfor++;
}
$startindx += 10;
$endindx += 10;
$totalruns -= 1;
$totalgames += $sizeof;
unset($games);
}
echo'<p>' . ' countwhile = ' . $countwhile . ' countfor = ' . $countfor . '</p>';
problem 1:
$sizeof = sizeof($games)-1;
explain 1:
for($i=0, $sizeof = sizeof($games);$i<=$sizeof;$i++)
the above will execute 11 times is the sizeof($games) is 10
So, either
for($i=1, $sizeof = sizeof($games);$i<=$sizeof;$i++)
or
for($i=0, $sizeof=sizeof($games)-1;$i<=$sizeof;$i++)
problem 2 :
$e = sizeof($games);
explain 2 :
$e = count($games);
...
$e += $e;
If the final size of $games is 50, you just sum it to 100
so, it some kind of logic problem
I know the answer has been accepted, but thought I'd refactor and make this a little more clean.
function retrieveGamesInfo($limit, $start = 0)
{
$feedHandler = new FeedHandler(); // ignore this, just for testing to simluate your call
if ($start > $limit)
throw new Exception("Start index must be within the limit");
$result = Array(
'TotalGames' => 0,
'TotalFileSize' => 0
);
// iterate over the results in groups of 10
$range = $start;
while ($range < $limit)
{
$range_end = $range + 10; // change me to play with the grab amount
if ($range_end > $limit)
$range_end = $limit;
// grab the next 10 entries
$games = $feedHandler->getGames($range,$range_end);
$result['TotalGames'] += count($games);
foreach ($games as $game)
$result['TotalFileSize'] += $game['swf_file_size'];
$range = $range_end;
}
return $result;
}
var_dump(retrieveGamesInfo(50));
based one everything I've read and taken in, this should be a good supplement. The above provides the following result:
array(2) {
["TotalGames"]=>
int(50)
["TotalFileSize"]=>
int(275520)
}
As i said in my comment $e is overwritten at each loop, so what you have in $e at the end is just the last count of elements in $games *2.
Added with ajreal issues this means results are what your code is expected to render :-) and I'm quite sure your last $game is not just 10 elements but 50. Quiet sure... but it's hard to read.
I need to generate random number of divs with five items per div (and remaining items in the last div) from random number of $totalItems and also not all the items satisfy $OKItems... Hopefully the code explains better than me.
My problem is that this script generates empty divs with no content in them.
<?php
$OKItems = 0;
$totalItems = rand(2,30);
for ($i = 0; $i < $totalItems; $i++) {
echo ($OKItems == 0 || $OKItems % 5 == 0) ? 'div open<br />' : '';
$testValue = rand(0, 1);
if ($testValue != 0) {
echo '1';
$OKItems++;
}
echo ($OKItems % 5 == 0 || $i+1 == $totalItems) ? '<br />div close<br />' : '';
}
?>
This is what I might get:
div open
div close
div open
11111
div close
div open
div close
div open
div close
div open
11
div close
And this is what I would have wanted in this case:
div open
11111
div close
div open
11
div close
<?php
const N = 5;
$totalItems = rand(2,30);
$items = array() ;
for ($i = 0; $i < $totalItems; $i++) {
$testValue = rand(0, 1);
if ($testValue != 0) {
$items[] = 1 ;
}
if( N == sizeof($items) || (($i == $totalItems - 1) && 0 < sizeof($items)) ) {
echo "<div>" . join(",", $items) . "</div>";
$items = array() ;
}
}
I think you need a bit more structure to your code.
My approach would be to break it up into several stages, as opposed to trying to do all the logic in the loop that outputs data.
What I'd suggest:
Decide how many items to be tested
Test each item and only copy the ones that pass into a new array
Partition this new array into sets of 5
Output each partition as a div
Code (untested):
// Decide how many items to test
$totalItems = rand(2,30);
// Test these items and add them to an accepted array
$items = Array();
for ($i = 0; $i < $totalItems; $i++) {
$testValue = rand(0, 1);
if ($testValue != 0) { $items[] = "1" }
}
//Partition them into sections
$partitions = array_chunk($items,5);
//Output as divs
foreach($partitions as $partition):
echo 'div open <br />';
foreach($partition as $item):
echo $item . "<br />";
endforeach;
echo 'div close <br />';
endforeach;
When you split up the code into logical steps, it becomes much easier to maintain and debug.
<?php
$OKItems = 0;
$totalItems = rand(2,30);
for ($i = 0; $i < $totalItems; $i++) {
echo ($OKItems == 0 || $OKItems % 5 == 0) ? 'div open<br>' : '';
$testValue = rand(0, 1);
if ($testValue != 0) {
echo '1';
$OKItems++;
}
if($OKItems % 5 == 0 || $i+1 == $totalItems) {
echo '<br>div close<br>';
$OKItems = 0;
}
}
?>
That should be working ;)
I changed your check line for an if function that also resets your $OKItems. The problem you had (i think) was that you got a 0 as the random value and that would keep $OKitems on 5.