So I have a large table (701-ish rows, 19 columns). I need to extract the innertext in each td, and then I write it to a csv. The problem is, this takes forever. Doing just 100, takes 32 seconds. This is the code I have:
for ($j = 0; $j < 100; $j++)
{
$f = $html->find("td",$j); // get the td elements from the html
$rowArray[] = $f->innertext; // store that text inside the array
if(($j+1) % 19 == 0) // hit the end of the row
{
$txt .= implode(",", $rowArray) . "\r\n"; // format with comma's and throw it into $txt
unset($rowArray); // clear the array, for the next record
$rowArray = array(); // re-set the array
}
}
The 100 is a temporary value while I test, it really is closer to 13000. The biggest issue is finding the TD values. Is there a faster way for this or is this as good as I can get it?
Basically, looking for the quickest way to extract TD data from an HTML table so I can write it to a CSV.
Did a str_replace to get the stuff I didn't want out and was able to get the contents a lot quicker and faster.
Related
I want to print an array in spiral order. For arrays with sizes 3x3, 4x4, ...etc. my code works correctly, but for 3x5, 4x6 or 5x8 sizes the output is wrong, returning only the first iteration.
This is my simple code:
private function _spiral($rows, $cols, array $array) {
$offset = 0;
while($offset < ($rows - 1)){
for($col = $offset; $col <= $cols - 1; $col++){
print($array[$offset][$col] . ' ');
}
$offset++;
$cols--;
for($row = $offset; $row < $rows; $row++){
print($array[$row][$cols] . ' ');
}
$rows--;
for($col = $cols - 1; $col >= $offset; $col--){
print($array[$rows][$col] . ' ');
}
for($row = $rows; $row >= $offset; $row--){
print($array[$row][$offset - 1] . ' ');
}
}
}
Example with 3 rows and 4 columns:
$array = array(
array(00,01,02,03),
array(10,11,12,13),
array(20,21,22,23)
)
Expected result for this array is 0 1 2 3 13 23 22 21 20 10 11 12, but the output of my function stops after 10.
For 4 rows and 4 columns:
$array = array(
array(00,01,02,03),
array(10,11,12,13),
array(20,21,22,23),
array(30,31,32,33)
)
...it should return 0 1 2 3 13 23 33 32 31 30 20 10 11 12 22 21, and that is what my code returns.
But I want both cases to work with my code. How can I correct the code to also produce the correct output for the first, and other cases?
There are a few problems with your code:
it does not treat the four directions of traversal in the same way. You have four loops for these four directions, but in some you have <= as loop-end condition in others <, in some the condition is on something minus 1, in others not.
it has no provision for when all elements have been printed by the first or second inner loop, and thus the remaining loops will in some cases print already printed elements.
the outer loop condition does not check whether there are still columns that need traversal. It is not enough to test for such rows only.
Although you could try to fix your code, I think it is better to start from scratch, taking into account that the solution should be symmetric for all four directions. This is an important intuitive reaction to develop: spot symmetries. This will lead to less code and fewer bugs.
You want to traverse a dimension (row or column) in your array until you reach the border of the array or an element you already printed. Then you want to turn 90° to the right and repeat exactly the same logic over and over again. So if your code looks different for these different directions, something is not right.
I will share two implementations. Both will use the concept of the "current" cell, and let it move around in spiral motion.
The first solution treats going back or forward along a row with the same code, and similarly it has one piece of code for traversing a column forward or backward. So this solution has two inner loops, one for traversing along a row, and another for traversing along a column. The direction in which a row or column is traversed is kept in the $direction variable, which flips between 1 and -1 at each execution of the outer loop:
function _spiral(array $array) {
// No need to have the number of rows and columns passed as arguments:
// We can get that information from the array:
$rows = count($array);
$cols = count($array[0]);
// Set "current" cell to be outside array: it moves into it in first inner loop
$row = 0;
$col = -1;
$direction = 1; // Can be 1 for forward and -1 for backward
while ($rows > 0 and $cols > 0) {
// Print cells along one row
for ($step = 0; $step < $cols; $step++) {
$col += $direction;
print $array[$row][$col] . ' ';
}
// As we have printed a row, we have fewer rows left to print from:
$rows--;
// Print cells along one column
for ($step = 0; $step < $rows; $step++) {
$row += $direction;
print $array[$row][$col] . ' ';
}
// As we have printed a column, we have fewer columns left to print from:
$cols--;
// Now flip the direction between forward and backward
$direction = -$direction;
}
}
Note the perfect symmetry between the first inner loop and the second inner loop.
In a second solution, this use of symmetry is taken one step further, in order to replace the two inner loops with only one. For that to happen we must abandon the use of separate variables for rows and columns, and use the concept of a size related to a dimension:
function _spiral(array $array) {
// This version of the function aims to treat rows and columns in the same way,
// They are just another dimension, but all the logic is exactly the same:
// $size[] has the number of rows in $size[0] and number of columns in $size[1]
$size = Array(count($array), count($array[0]));
// $current[] has the current row in $current[0] and current column in $current[1]
$current = Array(0, -1);
// $direction[] has the current row-traversal direction in $direction[0]
// and column-traveral direction in $direction[1]
$direction = Array(1, 1);
$dimension = 0; // Which dimension to traverse along, can be 0 for row, 1 for column
while ($size[$dimension] > 0) {
// Switch dimension (row to column, column to row), to traverse along
$dimension = 1 - $dimension;
// Print one line along that dimension, in its current direction
for ($step = 0; $step < $size[$dimension]; $step++) {
$current[$dimension] += $direction[$dimension];
print $array[$current[0]][$current[1]] . ' ';
}
// As we have printed a line, we have fewer left to print from:
$size[1 - $dimension]--;
// Now flip the direction between forward and backward for this dimension:
$direction[$dimension] = -$direction[$dimension];
}
}
An extended version
Upon request more than one year later: here is a version that allows one to choose the corner to start from, and whether to do it counter-clockwise instead of clockwise. This function will not print the result, but return a 1D array, with the spiral sequence. This way you can decide yourself what to do with the result: print it, or ... whatever.
function spiral(array $array, $startRight = false, $startBottom = false,
$counterClockWise = false) {
// This version allows to select which corner to start from, and in which direction.
// $size[] has the number of rows in $size[0] and number of columns in $size[1]
$size = [count($array), count($array[0])];
// $direction[] has the current row-traversal direction in $direction[0]
// and column-traversal direction in $direction[1]
$direction = [$startBottom ? -1 : 1, $startRight ? -1 : 1];
// Which dimension to traverse along: false means row, true means column.
// Every one of the optional arguments will flip the first dimension to use:
$dimension = ($startBottom xor $startRight xor $counterClockWise);
// $current[] has the current row in $current[0] and current column in $current[1]
$current = [$startBottom * (count($array)-1), $startRight * (count($array[0])-1)];
// Go back one step, outside of the grid
$current[!$dimension] -= $direction[!$dimension];
while ($size[$dimension] > 0) {
// Switch dimension (row to column, column to row), to traverse along
$dimension = !$dimension;
// Print one line along that dimension, in its current direction
for ($step = 0; $step < $size[$dimension]; $step++) {
$current[$dimension] += $direction[$dimension];
$result[] = $array[$current[0]][$current[1]]; // store in new array
}
// As we have printed a line, we have fewer left to print from:
$size[!$dimension]--;
// Now flip the direction between forward and backward for this dimension:
$direction[$dimension] = -$direction[$dimension];
}
return $result; // Return the resulting spiral as a 1D array
}
See it run on eval.in
I'm pretty new to PHP but I have built some fairly simple WordPress themes in my time. I'm trying to make something unique and I'm not sure how to accomplish it (or if it's even possible).
My goal is to create a loop that will dynamically close rows when it reaches the column "12" count for bootstrap (IE: col-md-3 + col-md-3 + col-md-6 = 12). The look I'm ultimately trying to achieve and how I currently have my static file set up => https://jsfiddle.net/y2mLr3hd/7/embedded/result/. I'm only using "display: flex;" for right now to just demonstrate what I'm trying to achieve. I'd like it be just rows rather than a single row.
I'm use to the standard loop for WordPress using ULs and LIs but I have no idea on how to go about what I'm trying to do. I'd like the loop to figure a random number for the column size consisting of the column sizes "3, 4, 6, 8" and create rows with columns sizes equaling "12" like I stated before. THAT or find a way on how to make it work with the way I currently have it set up.
This is the closest thing to what I'm looking for but really isn't even that close =>https://stackoverflow.com/questions/16427962/twitter-bootstrap-spans-in-dynamic-websites#=. Here's the code from that link for quick reference:
$i=0;
foreach ($posts as $post):
if ($i%2==0) echo '<div class="row-fluid">';
echo '<div class="span6">'. $post->content .'</div>';
if ($i%2==1) echo '</div>';
$i++;
endforeach;
Any help on how I might be able to go about this would be GREATLY appreciated!
There are two parts to your question:
How to divide 12 in random parts using 3, 4, 6, and 8
Given an array of numbers that add up to 12, how to generate a row with posts?
The first one is more a mathematics question. Note that you can only combine 3s and 6s, or 4s and 8s. You cannot combine 3 and 4, and still get 12.
We'll devise a simple algorithm with this in mind:
function getRandomNumbers()
{
// We start with an empty array and add numbers until we hit 12.
$result = array();
// We choose either 3 or 4 as basic number.
$x = mt_rand(3, 4);
// Now, as long as we don't hit 12, we iterate through a loop,
// adding either the basic number, or 2 times the basic number:
while (array_sum($result) < 12) {
// Randomly decide
if (mt_rand(0, 1) > 0) {
$newElement = 2 * $x; // This is either 6 or 8
// However, always make sure not to exceed 12:
if (array_sum($result) + $newElement > 12) {
$newElement = $x;
}
} else {
$newElement = $x; // This is either 3 or 4
}
// Add the new number to the array:
$result[] = $newElement;
}
// Return the resulting array
return $result;
}
Now we need to use these arrays to create rows. We start by generating an array with random numbers with the function we wrote.
We'll simply iterate through the posts, use the numbers in the array until there's none left. We'll generate a new array with random numbers whenever we need a new one. Since that means we have added 12 width-worth of columns, that's the point where we need to start a new row as well.
// Get the initial array with random numbers
$randomArray = getRandomNumbers();
// Open the initial row.
echo '<div>';
foreach ($posts as $post):
if (count($randomArray) < 1) {
// Close the row and start a new one:
echo '</div><div>';
// Get a fresh array with random numbers:
$randomArray = getRandomNumbers();
}
$nextRandomNumber = array_pop($randomArray); // This takes the next number.
echo '<div class="col-md-' . $nextRandomNumber . '">'. $post->content .'</div>';
endforeach;
echo '</div>'; // Close the final row.
I have a file with ~ 10.000 lines inside. I want every time user access my website, it auto pick 10 lines randomly among them.
Code I currently used:
$filelog = 'items.txt';
$random_lines = (file_exists($filelog))? file($filelog) : array();
$random_count = count($random_lines);
$random_file_html = '';
if ($random_count > 10)
{
$random_file_html = '<div><ul>';
for ($i = 0; $i < 10; $i++)
{
$random_number = rand(0, $random_count - 1); // Duplicate are accepted
$random_file_html .= '<li>'.$random_lines[$random_number]."</li>\r\n";
}
$random_file_html .= '</ul>
</div>';
}
When I have < 1000 lines, every things is ok. But now, with 1000 lines. It make my website slow dow significantly.
That I'm thinking to other methods, like:
Divide file to 50 files, select randomly them, then select 10 lines randoms inside the selected file.
-- or --
I knew total lines (items). Make 10 numbers randomly, then read file use
$file = new SplFileObject('items.txt');
$file->seek($ranđom_number);
echo $file->current();
(My server does not support any type of SQL)
Maybe you have other methods that best suit for me. What is best method for my problem? Thank you very much!
The fastest way would be apparently not to pick 10 lines randomly out of a file with ~ 10.000 lines inside on every user's request.
It's impossible to answer more as we know no details of this "XY problem".
If it is possible to adjust the contents of the file then simply pad each of the lines so they have a common length. Then you can access the lines in the file using random access.
$lineLength = 50; // this is the assumed length of each line
$total = filesize($filename);
$numLines = $total/$lineLength;
// get ten random numbers
$fp = fopen($filename, "r");
for ($x = 0; $x < 10; $x++){
fseek($fp, (rand(1, $numLines)-1)*$lineLength, SEEK_SET);
echo fgets($fp, 50);
}
fclose($fp);
try:
$lines = file('YOUR_TXT_FILE.txt');
$rand = array_rand($lines);
echo $lines[$rand];
for 10 of them just put it in a loop:
$lines = file('YOUR_TXT_FILE.txt');
for ($i = 0; $i < 10; $i++) {
$rand = array_rand($lines);
echo $lines[$rand];
}
NOTE: ** the above code does not guarantee that **2 same lines wont be picked. In order to guarantee uniqueness you need to add extra while loop and an array that holds all randomly generated indexes, so next time it generates it and it already exists in an array, generate another one until its not in the array.
The above solution might not be fastest but might fulfill your needs. Since your server does not support any type of SQL, maybe switch to a different server? Because I am wondering how you are storing User Data? Are those stored in files also?
The following is a code block that's working fine as regards getting the data I want.
Don't laugh, it's probably inefficient, but I'm learning :)
What I want, is to use the $totalLength variable, to stop gathering data when the $totalLength is, say 1500 bytes/characters (ideally, ending on a full word, but I'm not looking for miracles!). Anyway, the code:
$paraLength = 0;
$totalLength = 0;
for ($k = 0; $k < $descriptionValue->length; $k++) { //define integer k as 0, get every description using ($k = 0; $k < $descriptionValue->length; $k++), increment the k loop (to get only 14 elements, use ($k <= 13))
$totalLength = $totalLength + $paraLength;
echo $totalLength." Total<br />";
$descNode = $descriptionValue->item($k)->nodeValue; //find each description element
$descNode = trim($descNode); //trim any whitespace around the element
$descPara = strip_tags($descNode); //remove any HTML tags from the elements
$paraLength = (strlen($descPara)); //find the length of each element
//if (preg_match('/^([0-9 ]+)$/', $descPara)) { //if element starts with numbers followed by a space, define it as a telephone number
// $number = $descPara;
// fwrite ($fh, "\t\t".'<div id="tel">'.$number."</div>\n"); //write a div with id tel, containing the number
//}
//else
if (preg_match('/[A-Z]{4,}/', $descPara)) { //if element starts with at least 4 uppercase characters, define it as a heading
$heading = $descPara;
$heading=ucfirst(strtolower($heading)); //convert the uppercase string to proper
fwrite ($fh, "\t\t".'<div id="heading"><h4>'.$heading."</h4></div>\n"); //write a div with id heading, containing the heading in h4 tags
}
else if (preg_match('/\d*\.\d{1,}[m x]/', $descPara)) { //if the element contains any number of digits followed by a dot, at least one further digit and the letters m x, define it as a heading based on it containing room measurements (this pattern matches at least two number after the dot \d*{2,}}
$room = $descPara;
fwrite ($fh, "\t\t".'<div id="roomheading"><h4>'.$room."</h4></div>\n"); //write a div with id roomheading, containing the heading in h4 tags
}
else if (preg_match('/^Disclaimer/i', $descPara)) { //if the element contains the word Disclaimer, define it as such
$disclaimer = $descPara;
fwrite ($fh, "\t\t".'<div id="disclaimer"><h4>'.$disclaimer."</h4></div>\n"); //write a div with id disclaimer, containing the heading in h4 tags
}
else if (strlen($paraLength<14 && $paraLength>3)) { //when all else fails, if the element is less than 14 but more than 3 characters, also define it as a heading
$other = $descPara;
fwrite ($fh, "\t\t".'<div id="other"><h4>'.$other."</h4></div>\n"); //write a div with id other and the heading in h4 tags
}
else {
fwrite ($fh, "\t\t\t<p>".$descPara."</p>\n"); //anything else is considered content, so write it out inside p tags
}
}
$totalLength counts nicely, but when I tried to put a while statement in there, it just hung. I tried putting the while statement before and after the for, but no joy. What am I doing wrong and how best to solve this one?
FYI $descriptionValue, is data parsed from HTML using DOM & xpath, the while I tried was while($totalLength <= 1500)
Maybe this is what You want:
if ($totalLength > 1500) {
break;
}
Just put a condition inside your for loop. It will jump outside the loop as soon as the condition evaluates to true.
// for () { ...
if ($totalLength > 1500) {
break;
}
// }
Basically, break ends execution of the current for, foreach, while, do-while or switch structure. You can find more about PHP's control structures in the manual.
You can also delete the for and add
$k = 0;
while ($totalLength <= 1500 || $k < $descriptionValue->length)
and inside the loop you increment the value ok $k
I'm trying to process a for loop within a for loop, and just a little wary of the syntax... Will this work? Essentially, I want to run code for every 1,000 records while the count is equal to or less than the $count... Will the syntax below work, or is there a better way?
for($x = 0; $x <= 700000; $x++) {
for($i = 0; $i <= 1000; $i++) {
//run the code
}
}
The syntax you have will work, but I don't think it's going to do exactly what you want. Right now, it's going to do the outer loop 700,001 times, and for every single one of those 700,001 times, it's going to do the inner loop.
That means, in total, the inner loop is going to run 700,001 x 1001 = about 700.7 million times.
If this isn't what you want, can you give a bit more information? I can't really work out what "I want to run code for every 1,000 records while the count is equal to or less than the $count" means. I don't see any variable named $count at all.
Well, essentially, I'm reading in a text file and inserting each of the lines into a db. I did originally try while(!feof($f)) [where $f = filename], but it keeps complaining of a broken pipe. I thought this would be another way to go
$f should be file-handle returned by fopen(), not a filename.
$file_handle = fopen($filename, 'r');
while(!feof($file_handle)) {
$line = fgets($file_handle);
$line = trim($line); // remove space chars at beginning and end
if(!$line) continue; // we don't need empty lines
mysql_query('INSERT INTO table (column) '
.'VALUES ("'.mysql_real_escape_string($line).'")');
}
Read through the documentation at php.net for fopen(), fgets(). You might also need explode() if you need to split your string.
If your file isn't big, you might want to read it into an array at once like this:
$filelines = file($filename, FILE_IGNORE_NEW_LINES | FILE_SKIP_EMPTY_LINES);
foreach($filelines as $line) {
do_stuff_with($line);
}
Hmm. Ok... Well, essentially, I'm reading in a text file and inserting each of the lines into a db. I did originally try while(!feof($f)) [where $f = filename], but it keeps complaining of a broken pipe. I thought this would be another way to go..
To read a text file line by line I usually:
$file = file("path to file")
foreach($file as $line){
//insert $line into db
}
Strictly answering the question, you'd want something more like this:
// $x would be 0, then 1000, then 2000, then 3000
for($x = 0; $x < 700000; $x += 1000) {
// $i would be $x through $x + 999
for($i = $x; $i < $x + 1000; $i++) {
//run the code
}
}
However, you should really consider one of the other methods for importing files to a database.