I am trying to optimize increase the speed of this loop. I would like it if anyone has any ideas to boost the speed as there is thousands of members and it does take time time to complete currently
SQL Stored PROCEDURE MembersMobileByVenue returns this->
Example
FirstName Phone Venue
Aaron 04******* 7272CD46D51F
Brad 04******* CF105BB0
Adam 04******* 7272CD46D51F
Craig 04******* CF105BB0
PHP
$venueIDS = isset($_POST['location']) ? $_POST['location'] : array();
$msg = $_POST['message'];
$response = array();
if(!empty($venueIDS)){
$Members = GoldCardMembers::MembersMobileByVenue();
foreach($Members as $Member){
if(in_array($Member->Venue, $venueIDS)){
$destination = $Member->Phone;
$text = 'Hi ' . $Member->FirstName . ' ' .$msg. '. Reply STOP to opt out';
$ref = 'Members';
$content = '&to='.rawurlencode($destination).
'&message='.rawurlencode($text).
'&ref='.rawurlencode($ref);
$smsbroadcast_response = sendSMS($content);
$response_lines = explode("\n", $smsbroadcast_response);
foreach( $response_lines as $data_line){
$message_data = "";
$message_data = explode(':',$data_line);
if($message_data[0] == "OK"){
array_push($response, "The message to ".$message_data[1]." was successful, with reference ".$message_data[2]."\n");
}elseif( $message_data[0] == "BAD" ){
array_push($response, "The message to ".$message_data[1]." was NOT successful. Reason: ".$message_data[2]."\n");
}elseif( $message_data[0] == "ERROR" ){
array_push($response, "There was an error with this request. Reason: ".$message_data[1]."\n");
}
}
}
}
}
foreach($response as $message){
echo $message;
}
$venueIDS = isset($_POST['location']) ? $_POST['location'] : array();
$msg = $_POST['message'];
$text = "";
$destination = "";
$content = "";
$response_lines = array();
$smsbroadcast_response = "";
$message_data = array();
if(!empty($venueIDS)){
$Members = GoldCardMembers::MembersMobileByVenue();
foreach($Members as $Member){ // O(n) size of $Members
if(in_array($Member->Venue, $venueIDS)){ // O(v) size of $venueIDs
$destination = rawurlencode($Member->Phone);
$text = rawurlencode($Member->FirstName . ' ' .$msg);
$content = '&to='. $destination .
'&message=Hi ' . $text . '. Reply STOP to opt out'.
'&ref=Members';
$smsbroadcast_response = sendSMS($content);
$response_lines = explode("\n", $smsbroadcast_response); // O(r) number of "\n" occurances
foreach( $response_lines as $data_line){ // O(r)
$message_data = explode(':',$data_line); // O(d) number of ":" occurances
if($message_data[0] == "OK"){
echo "The message to ".$message_data[1]." was successful, with reference ".$message_data[2]."\n";
}elseif( $message_data[0] == "BAD" ){
echo "The message to ".$message_data[1]." was NOT successful. Reason: ".$message_data[2]."\n";
}elseif( $message_data[0] == "ERROR" ){
echo "There was an error with this request. Reason: ".$message_data[1]."\n";
}
}
}
}
}
Here's a list of what I did:
removed the for loop from bottom and echo the $message directly from the main loop instead
removed $ref var
initialized $destination, $text, $content, $response_lines, $smsbroadcast_response and $message_data vars outside the for loop since declaring them inside the loop with each iteration is sub-optimal
That's all I can think about for now.
Notes for further improvements:
I'd investigate on the performance of sendSMS that you're using. It could be one of the components that are slowing things down.
In order to dig deeper, a helpful tool for analyzing performance is big Oh notation. So I've added comments to label loop components.
Because we're looking at multiple nested loops we generally have something like O(N^3) here.
However, we could write that equation to be more detailed such as this:
O( n * (v + 2r) * d)
(please check code comments to see what each variable represents)
Since you're asking for a high "n" (number of members) I'd investigate ways to make the other variables like "v", "r" or "d" smaller.
Eliminating any of these variables altogether like removing
$message_data = explode(':',$data_line);
have the highest positive impact on performance (given that is still achieves the same functionality).
Related
I was recently tasked to update some older sites from MySQL to MySQLi.
Though slow and steady, the update has been ok until I ran into an issue when exporting some data to an excel document.
This code was written by a previous developer. There's a lot going on in the file, and I hope I'm grabbing the part that is supposed to be creating the excel document:
<?php
$export = mysqli_query ( $session->connDB(),$sql ) or die ( "Sql error : " . mysqli_error( ) );
$fields = mysqli_num_fields ( $export );
$num_rows = mysqli_num_rows( $export );
$pattern = '/[A-Z0-9][A-Z0-9._-]+#[A-Z0-9][A-Z0-9.-]{0,61}[A-Z0-9]\.[A-Z.]{2,6}/i'; //email
$phonpat = '/(\(?([0-9]{3})+\)?[\-\s\.]?([0-9]{3})+[\-\s\.]?([0-9]{4})(( |-|\.)?[ext\.]+ ?\d+)|\(?([0-9]{3})+\)?[\-\s\.]?([0-9]{3})+[\-\s\.]?([0-9]{4}))/i'; //telephone
$phPat = '/([0-9]{3})([0-9]{3})([0-9]{4})/';
$vippatt = '/VIP/i';
for($f=0; $f<$fields; $f++){
$header.='"'.mysqli_fetch_fields($export, $f).'"'."\t";
}
for($i=0; $i<$num_rows; $i++){
for($x=0; $x<$fields; $x++){
$email = mysqli_fetch_assoc($export,$i,"EMAIL");
$phone = mysqli_fetch_assoc($export,$i,"PHONE");
$viprm = mysqli_fetch_assoc($export,$i,"VIP");
preg_match ($pattern, $email, $matches);
preg_match ($phonpat, $phone, $phoneno);
preg_match ($vippatt, $viprm, $vpmatch);
if(isset($matches[0])) {$emal=strtolower($matches[0]);} else {$emal="";}
if(isset($vpmatch[0])) {$vips=strtoupper($vpmatch[0]);} else {$vips="";}
if(isset($phoneno[0])) {$phne=preg_replace($phPat,'($1) $2-$3 ',formatPhone($phoneno[0],false,false));} else {$phne="";}
if(mysqli_fetch_fields($export, $x)=='EMAIL'){
$fld=$emal;
} else {
if(mysqli_fetch_fields($export, $x)=='PHONE'){
$fld=$phne;
} else {
if(mysqli_fetch_fields($export, $x)=='VIP'){
$fld=$vips;
} else {
if(mysqli_fetch_fields($export, $x)=='UNITS'){
$fld=1;
} else {
$fld = mysqli_fetch_assoc($export,$i,mysqli_fetch_fields($export, $x));
}
}
}
}
$data.= '"'.$fld.'"'."\t";
}
$data.="\n";
}
?>
Here is where the code checks if the data is blank or not, and then exports the spreadsheet:
<?php
if ($data == "") {
$data = "\nNo records found for your search parameters.\n\n".$sql;
} else {
echo "should show data";
}
global $time;
$time = time();
header("Content-Disposition: attachment; filename=CargoManagementCustomReport-$time.xls");
header("Pragma: no-cache");
header("Expires: 0");
print "$header\n$data";
?>
When the spreadsheet gets exported, I see "should show data". This tells me the $data variable obviously has data. It's just not getting into the spreadsheet.
If you'll notice in the above, I'm using mysqli_fetch_fields. This was used to replace mysql_field_name (in my attempt to update to MySQLi).
I also tried mysqli_fetch_field, but got the same results.
I am getting no errors, but the spreadsheet is still blank.
I can echo $sql to get the query, and I can run the query in the database and it returns data.
What am I doing wrong and how can I fix it?
That whole code is gibberish, so I hope I understood what it is that it was meant to do.
Here are the main problems:
mysqli_fetch_fields() takes only 1 argument and returns an array of objects. You can't cast an array to a string. I assume you wanted to get the field name.
mysqli_fetch_assoc() takes only 1 argument and returns an array of data in an associative array as the name suggests. It also moves the internal pointer to the next row every time it is called. You are trying to use it as if it was mysql_result().
Your nested loops are very messy. I replaced them with simple foreach loops and replaced the nested if statements with a switch. While I would normally stay away from such constructs, this is the easiest way to migrate this code.
After removing all the mysqli nonsense, the code is now readable. It iterates over every field of every row, applying some transformations to some fields and concatenating the result into a string.
Fixed code:
$conn = $session->connDB();
$export = mysqli_query($conn, $sql);
$pattern = '/[A-Z0-9][A-Z0-9._-]+#[A-Z0-9][A-Z0-9.-]{0,61}[A-Z0-9]\.[A-Z.]{2,6}/i'; //email
$phonpat = '/(\(?([0-9]{3})+\)?[\-\s\.]?([0-9]{3})+[\-\s\.]?([0-9]{4})(( |-|\.)?[ext\.]+ ?\d+)|\(?([0-9]{3})+\)?[\-\s\.]?([0-9]{3})+[\-\s\.]?([0-9]{4}))/i'; //telephone
$phPat = '/([0-9]{3})([0-9]{3})([0-9]{4})/';
$vippatt = '/VIP/i';
foreach (mysqli_fetch_fields($result) as $field) {
$header .= '"' . $field->name . '"' . "\t";
}
$data = '';
foreach ($export as $row) {
foreach ($rows as $fieldName => $value) {
switch ($fieldName) {
case 'EMAIL':
preg_match($pattern, $value, $matches);
$data .= '"' . (isset($matches[0]) ? strtolower($matches[0]) : '') . '"' . "\t";
break;
case 'PHONE':
preg_match($phonpat, $value, $phoneno);
$phne = "";
if (isset($phoneno[0])) {
$phne = preg_replace($phPat, '($1) $2-$3 ', formatPhone($phoneno[0], false, false));
}
$data .= '"' . $phne . '"' . "\t";
break;
case 'VIP':
preg_match($vippatt, $value, $vpmatch);
$data .= '"' . (isset($vpmatch[0]) ? strtolower($vpmatch[0]) : '') . '"' . "\t";
break;
case 'UNITS':
$data .= '"1"' . "\t";
break;
default:
$data .= '"' . $value . '"' . "\t";
break;
}
}
$data .= "\n";
}
I'm trying to code a website that takes the comments of a Reddit page and shows them. However, the comments have replies, and I want to show those too, but a comment can have none, one or more replies, and those replies can have replies. Is there a way to repeat the same code for all of the replies with minor differences (indentation)? I'm using the reddit json feature, and am getting the JSON from something like here: https://www.reddit.com/r/pcmasterrace/comments/dln0o3/foldinghome_and_pcmr_team_up_use_your_pc_to_help/.json.
I have:
$url = ('https://www.reddit.com/r/pcmasterrace/comments/dln0o3/foldinghome_and_pcmr_team_up_use_your_pc_to_help/.json');
$json = file_get_contents($url);
$obj = json_decode($json, true);
$comment_array = array_slice($obj[1]['data']['children'], 0, 50);
echo '<div class="comments">';
foreach ($comment_array as $c) {
echo "<p>(".$c['data']['author'].") ". $c['data']['score'] . " Points<br>".$c['data']['body']."</p>";
if (!($c['data']['replies'] == "")) {
$r1_array = $c['data']['replies']['data']['children'];
foreach ($r1_array as $r1) {
echo "<p> (".$r1['data']['author'].") ". $r1['data']['score'] . " Points<br> ".$r1['data']['body']."</p>";
if (!($r1['data']['replies'] == "")) {
$r2_array = $r1['data']['replies']['data']['children'];
foreach ($r2_array as $r2) {
echo "<p> (".$r2['data']['author'].") ". $r1['data']['score'] . " Points<br> ".$r2['data']['body']."</p>";
}
}
}
}
}
}
This produces the desired result, with replies to replies and such. However, it's a bit messy, and if there is a really long reply chain, it won't catch it. Is there a way to make it repeat somehow or should I just copy and paste it a bunch of times?
Thanks very much!
I think you're looking for a concept called recursion.
The basic idea is that a function will call itself as many times as needed (as opposed to using a fixed number of loops).
Something like this:
<?php
function output($data, $level = 0) {
$spaces = str_repeat(' ', $level);
foreach ($data as $post) {
echo "<p>(".$spaces.$post['data']['author'].") ". $post['data']['score'] . " Points<br>".$post['data']['body']."</p>\r\n";
if ($post['data']['replies']) {
// Notice that we are calling the function again, this time increasing the level
// This is the "recursive" part of the function
output($post['data']['replies']['data']['children'], $level + 1);
}
}
}
$url = ('https://www.reddit.com/r/pcmasterrace/comments/dln0o3/foldinghome_and_pcmr_team_up_use_your_pc_to_help/.json');
$json = file_get_contents($url);
$data = json_decode($json, true);
$comment_array = array_slice($data[1]['data']['children'], 0, 50);
output($comment_array);
?>
I have just learnt some basic skill for html and php and I hope someone could help me .
I had created a html file(a.html) with a form which allow students to input their name, student id, class, and class number .
Then, I created a php file(a.php) to saved the information from a.html into the info.txt file in the following format:
name1,id1,classA,1
name2,id2,classB,24
name3,id3,classA,15
and so on (The above part have been completed with no problem) .
After that I have created another html file(b.html), which require user to enter their name and id in the form.
For example, if the user input name2 and id2 in the form, then the php file(b.php) will print the result:
Class: classB
Class Number: 24
I have no idea on how to match both name and id at the same time in the txt file and return the result in b.php
example data:
name1,id1,classA,1
name2,id2,classB,24
name3,id3,classA,15
<?php
$name2 = $_POST['name2'];
$id2 = $_POST['id2'];
$data = file_get_contents('info.txt');
if($name2!='')
$konum = strpos($data, $name2);
elseif($id2!='')
$konum = strpos($data, $id2);
if($konum!==false){
$end = strpos($data, "\n", $konum);
$start = strrpos($data, "\n", (0-$end));
$row_string = substr($data, $start, ($end - $start));
$row = explode(",",$row_string);
echo 'Class : '.$row[2].'<br />';
echo 'Number : '.$row[3].'<br />';
}
?>
Iterate through lines until you find your match. Example:
<?php
$csv=<<<CSV
John,1,A
Jane,2,B
Joe,3,C
CSV;
$data = array_map('str_getcsv', explode("\n", $csv));
$get_name = function($number, $letter) use ($data) {
foreach($data as $row)
if($row[1] == $number && $row[2] == $letter)
return $row[0];
};
echo $get_name('3', 'C');
Output:
Joe
You could use some simple regex. For example:
<?php
$search_name = (isset($_POST['name'])) ? $_POST['name'] : exit('Name input required.');
$search_id = (isset($_POST['id'])) ? $_POST['id'] : exit('ID input required.');
// First we load the data of info.txt
$data = file_get_contents('info.txt');
// Then we create a array of lines
$lines = preg_split('#\\n#', $data);
// Now we can loop the lines
foreach($lines as $line){
// Now we split the line into parts using the , seperator
$line_parts = preg_split('#\,#', $line);
// $line_parts[0] contains the name, $line_parts[1] contains the id
if($line_parts[0] == $search_name && $line_parts[1] == $search_id){
echo 'Class: '.$line_parts[2].'<br>';
echo 'Class Number: '.$line_parts[3];
// No need to execute the script any further.
break;
}
}
You can run this. I think it is what you need. Also if you use post you can change get to post.
<?php
$name = $_GET['name'];
$id = $_GET['id'];
$students = fopen('info.txt', 'r');
echo "<pre>";
// read each line of the file one by one
while( $student = fgets($students) ) {
// split the file and create an array using the ',' delimiter
$student_attrs = explode(',',$student);
// first element of the array is the user name and second the id
if($student_attrs[0]==$name && $student_attrs[1]==$id){
$result = $student_attrs;
// stop the loop when it is found
break;
}
}
fclose($students);
echo "Class: ".$result[2]."\n";
echo "Class Number: ".$result[3]."\n";
echo "</pre>";
strpos can help you find a match in your file. This script assumes you used line feed characters to separate the lines in your text file, and that each name/id pairing is unique in the file.
if ($_POST) {
$str = $_POST["name"] . "," . $_POST["id"];
$file = file_get_contents("info.txt");
$data = explode("\n", $file);
$result = array();
$length = count($data);
$i = 0;
do {
$match = strpos($data[$i], $str, 0);
if ($match === 0) {
$result = explode(",", $data[$i]);
}
} while (!$result && (++$i < $length));
if ($result) {
print "Class: " . $result[2] . "<br />" . "Class Number: " . $result[3];
} else {
print "Not found";
}
}
I'm using PHP simple_html_dom and file_get_html in a foreach loop which loops about 4,000 times.
Here's what I'm doing. I'm extracting content from multiple urls on same host.
There are 4,000+ urls all on the same host. example.com
Example url: example.com/product.php?prod=00283
On the next pass-thru of the loop example.com/product.php?prod=03418
next pass-thru of loop example.com/product.php?prod=08192
etc, etc until all 4000+ urls are looped over.
Tested 100 urls took over 1 and a half minutes to parse and output content.
Is it possible to setup a multithreaded approach in the foreach loop to bring back the contents of all 4000 urls simultaneously?
In other words, do I use sockets or some other kind of multithreaded approach to iterate over all the urls in parallel?
Here is my sample code below so you can see what I'm doing here.
include 'simple_html_dom.php';
$partlist_file = 'parts.txt';
$partlist = file('partnumberlist.txt', FILE_IGNORE_NEW_LINES | FILE_SKIP_EMPTY_LINES);
$stock = '';
$isFirst = true;
$firstline = 'sku,qty,visibility,is_in_stock' . "\n";
$output_first = '';
foreach($partlist as $parts => $part) {
$html = file_get_html('example.com/product.php?prod=' . $part);
$ret = $html->find('span#cph_lb_stock_buy');
foreach($ret as $element) {
$stock = $element->plaintext;
$stock = preg_replace(array('/\\n/','/\\r/'),'',$stock);
$stock = trim($stock);
if($stock == 'Not in stock.') {
$stock = '0,1,0';
} elseif($stock == 'In Stock & Ready to Ship!') {
$stock = '4,4,4';
}
if ($isFirst) {
$output = $firstline;
$isFirst = false;
}
$output .= 'K' . $part . ',' . $stock . "\n";
}
}
//print_r (trim($output,"\n"));
file_put_contents($partlist_file, trim($output,"\n"));
I am having problem with the yahoo search API, sometimes it works and sometimes don't why I am getting problem with that
I am using this URL
http://api.search.yahoo.com/WebSearchService/rss/webSearch.xml?appid=yahoosearchwebrss&query=originurlextension%3Apdf+$search&adult_ok=1&start=$start
The code is given below:
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<? $search = $_GET["search"];
$replace = " "; $with = "+";
$search = str_replace($replace, $with, $search);
if ($rs =
$rss->get("http://api.search.yahoo.com/WebSearchService/rss/webSearch.xml?appid=yahoosearchwebrss&query=originurlextension%3Apdf+$search&adult_ok=1&start=$start")
)
{ }
// Go through the list powered by the search engine listed and get
// the data from each <item>
$colorCount="0";
foreach($rs['items'] as $item) { // Get the title of result
$title = $item['title']; // Get the description of the result
$description = $item['description']; // Get the link eg amazon.com
$urllink = $item['guid'];
if($colorCount%2==0) {
$color = ROW1_COLOR;
} else {
$color = ROW2_COLOR;
}
include "resulttemplate.php"; $colorCount++;
echo "\n";
}
?>
Sometimes it gives results and sometimes don't. I get this error usually
Warning: Invalid argument supplied for foreach() in /home4/thesisth/public_html/pdfsearchmachine/classes/rss.php on line 14
Can anyone help..
The error Warning: Invalid argument supplied for foreach() in /home4/thesisth/public_html/pdfsearchmachine/classes/rss.php on line 14 means the foreach construct did not receive an iterable (usually an array). Which in your case would mean the $rs['items'] is empty... maybe the search returned no results?
I would recommended adding some checks to the results of $rss->get("...") first, and also having an action for when the request fails or returns no results:
<?php
$search = isset($_GET["search"]) ? $_GET["search"] : "default search term";
$start = "something here"; // This was left out of your original code
$colorCount = "0";
$replace = " ";
$with = "+";
$search = str_replace($replace, $with, $search);
$rs = $rss->get("http://api.search.yahoo.com/WebSearchService/rss/webSearch.xml?appid=yahoosearchwebrss&query=originurlextension%3Apdf+$search&adult_ok=1&start=$start");
if (isset($rs) && isset($rs['items'])) {
foreach ($rs['items'] as $item) {
$title = $item['title']; // Get the title of the result
$description = $item['description']; // Get the description of the result
$urllink = $item['guid']; // Get the link eg amazon.com
$color = ($colorCount % 2) ? ROW2_COLOR : ROW1_COLOR;
include "resulttemplate.php";
echo "\n";
$colorCount++;
}
}
else {
echo "Could not find any results for your search '$search'";
}
Other changes:
$start was not declared before your $rss->get("...") call
compounded the $color if/else clause into a ternary operation with fewer comparisons
I wasn't sure what the purpose of the if ($rs = $rss->get("...")) { } was, so I removed it.
I would also recommend using require instead of include as it will cause a fatal error if resulttemplate.php doesn't exist, which in my opinion is a better way to detect bugs than PHP Warnings which will continue execution. However I don't know you whole situation so it might not be of great use.
Hope that helps!
Cheers