I have the following code which works great to do what it is meant to do. It scrapes bodys of all e-mails in the inbox at an e-mail address using IMAP and retrieves the unique code sent inside the body and stores it in a database with the amount paid. I'm hoping to make it so when a user purchases something they can send an Interac e-transfer and then enter the code that both of us receive via e-mail in the website and it will apply the credit of the e-transfer to their account/purchase.
However; after setting it up on a cron job to cycle every few minutes so the content in the database stays fresh it eventually exceeds the bandwidth for the account within a day or so (not positive on how long it took but it didn't take long). Now, like I said the code works it's just very resource intensive apparently.
I do not pipe the script because the e-mail account has already been set up a while ago and we do use the account for other e-transfers which I review manually for other transactions.
Is there anyway to clean it up so it works the same but uses less resources/bandwidth?
Would it be better to run it when a user enters their payment code? Depending on the number of users running it this could also lead to bandwidth troubles.
Is there any improvements or what ideas do you have?
define("MAX_EMAIL_COUNT", $_POST['maxcount']);
/* took from https://gist.github.com/agarzon/3123118 */
function extractEmail($content) {
$regexp = '/([a-z0-9_\.\-])+\#(([a-z0-9\-])+\.)+([a-z0-9]{2,4})+/i';
preg_match_all($regexp, $content, $m);
return isset($m[0]) ? $m[0] : array ();
}
function getAddressText(&$emailList, &$nameList, $addressObject) {
$emailList = '';
$nameList = '';
foreach ($addressObject as $object) {
$emailList .= ';';
if (isset($object->personal)) {
$emailList .= $object->personal;
}
$nameList .= ';';
if (isset($object->mailbox) && isset($object->host)) {
$nameList .= $object->mailbox . "#" . $object->host;
}
}
$emailList = ltrim($emailList, ';');
$nameList = ltrim($nameList, ';');
}
function processMessage($mbox, $messageNumber) {
global $db;
// get imap_fetch header and put single lines into array
$header = imap_rfc822_parse_headers(imap_fetchheader($mbox, $messageNumber));
$timestamp = strtotime($header->Date);
$fromEmailList = '';
$fromNameList = '';
if (isset($header->from)) {
getAddressText($fromEmailList, $fromNameList, $header->from);
}
$toEmailList = '';
$toNameList = '';
if (isset($header->to)) {
getAddressText($toEmailList, $toNameList, $header->to);
}
$body = imap_fetchbody($mbox, $messageNumber, 1);
//echo "<pre>".print_r($body,true)."</pre>";
/* Find Reference Number */
//echo "<pre style='background-color: #A2A2A2; border: 1px solid black'>$body</pre>";
$searchfor = 'Reference Number';
// get the file contents, assuming the file to be readable (and exist)
$contents = $body;
// escape special characters in the query
$pattern = preg_quote($searchfor, '/');
// finalise the regular expression, matching the whole line
$pattern = "/^.*$pattern.*\$/m";
// search, and store all matching occurences in $matches
if(preg_match_all($pattern, $contents, $matches)){
$reference = trim(str_replace('Reference Number : ','',$matches[0][0]));
}
else{
}
/* Find Amount Paid */
//echo "<pre style='background-color: #A2A2A2; border: 1px solid black'>$body</pre>";
$searchfor = 'has sent you a money transfer for the amount of';
// get the file contents, assuming the file to be readable (and exist)
$contents = $body;
// escape special characters in the query
$pattern = preg_quote($searchfor, '/');
// finalise the regular expression, matching the whole line
$pattern = "/^.*$pattern.*\$/m";
// search, and store all matching occurences in $matches
if(preg_match_all($pattern, $contents, $matches)){
$amount = trim(preg_replace("/[^0-9\.]/", "",$matches[0][0]),'.');
}
else{
}
$bodyEmailList = implode(';', extractEmail($body));
// Delete all messages older than one year (31557600 seconds). Divide it by two for six months.
if($timestamp < time()-31557600) {
if(imap_delete($mbox,$messageNumber)) {
/*echo "<strong>";
print_r($messageNumber . ' , ' . date("F j, Y g:i A",$timestamp).' , ' . 'Deleted' . "\n");
echo "</strong>";*/
}
}
else {
if(!empty($reference) && !empty($amount)) {
if($fromNameList == "catch#payments.interac.ca" && $toNameList!='etransfers#example.com') {
$query = "SELECT * FROM `".$db->prefix."payments_etransfer` WHERE `reference_id` = '".$reference."'";
$select = $db->select($query);
if($db->num_rows($select) > 0) {
}
else {
$do = $db->insert_sql("INSERT INTO `".$db->prefix."payments_etransfer` SET
`email_id` = '".$messageNumber."',
`timestamp` = '".$timestamp."',
`reference_id` = '".$reference."',
`amount` = '".$amount."',
`sender` = '".$fromEmailList."'");
if($do) {
}
else {
echo "Error<br><blockquote><pre>";
print_r($messageNumber . ',' . $timestamp. ',' . $reference . ',$' . $amount .
',' . $fromEmailList . ',' . $fromNameList
. ',' . $toEmailList . ',' . $toNameList
. ',' . $bodyEmailList . "\n"
);
echo "</pre></blockquote>";
}
}
}
}
}
}
// imap_timeout(IMAP_OPENTIMEOUT, 300);
// Open pop mailbox
if (!$mbox = imap_open($_POST['mailbox'], $_POST['login'], $_POST['password'])) {
die('Cannot connect/check pop mail! Exiting');
}
if ($hdr = imap_check($mbox)) {
$msgCount = $hdr->Nmsgs;
} else {
echo "Failed to get mail";
exit;
}
/* echo "<pre>";
echo 'emails count=' . $msgCount . "\n\n\n";
echo "record number,from emails list,from names list,to emails list, to names list,extracted from body\n";
*/
/* might improve performance according to
http://www.php.net/manual/en/function.imap-headerinfo.php#98809
imap_headers($mbox);
*/
for ($X = $msgCount; $X > 0; $X--) {
if($X > 0) {
processMessage($mbox, $X);
}
}
/*echo "</pre>";*/
imap_expunge($mbox);
imap_close($mbox);
/*
Related
I am making a simple visitor counter without MySQL in a WordPress plugin. To achieve this I make a simple text visitor.db where store visitor ip for a while and show the total number of IPs as current online visitor and after a time period the history log delete. But now I want to implement from that visitor.db to save them another.db where it count total visitor till today. Is it possible?
If I count while loop ? how can I save it to another.db
$counter = 0;
while(..) {
$counter++;
}
echo $counter;
For example now it show Online : 70
But I want to achieve Total : 145875458 Online : 70 something like this
define('VISITOR_DATA_PATH',plugin_dir_path( __FILE__ ));
$dbfile = VISITOR_DATA_PATH . "visitors.db";
$expire = 300; // average time in seconds to consider someone online before removing from the list
if(!file_exists($dbfile)) {
die("Error: Data file " . $dbfile . " NOT FOUND!");
}
if(!is_writable($dbfile)) {
die("Error: Data file " . $dbfile . " is NOT writable! Please CHMOD it to 666!");
}
function getIPs() {
if(isset($_SERVER['HTTP_X_FORWARDED_FOR'])) $ip = $_SERVER['HTTP_X_FORWARDED_FOR'];
elseif(isset($_SERVER['REMOTE_ADDR'])) $ip = $_SERVER['REMOTE_ADDR'];
else $ip = "0";
return $ip;
}
function CountVisitors() {
global $dbfile, $expire;
$cur_ip = getIPs();
$cur_time = time();
$dbary_new = array();
$dbary = unserialize(file_get_contents($dbfile));
if(is_array($dbary)) {
while(list($user_ip, $user_time) = each($dbary)) {
if(($user_ip != $cur_ip) && (($user_time + $expire) > $cur_time)) {
$dbary_new[$user_ip] = $user_time;
}
}
}
$dbary_new[$cur_ip] = $cur_time; // add record for current user
$fp = fopen($dbfile, "w");
fputs($fp, serialize($dbary_new));
fclose($fp);
$out = sprintf("%3d", count($dbary_new)); // format the result to display 3 digits with leading 0's
return $out;
}
function visitor_counter_shrocode(){
$visitors_online = CountVisitors();
ob_start();
$output='<span> Online: <b>'.$visitors_online.'</b></span>';
ob_end_clean();
return $output;
}
add_shortcode('onlinevisitor','visitor_counter_shrocode');
I'm writing a custom import script for SuiteCRM and I'm getting the error:
Warning: array_combine() expects parameter 2 to be array, boolean
given in /var/www/html/afscmedbv5test/custom/wimporter/newimporter.php
on line 162
My Script is as follows:
<?php
if (!defined('sugarEntry') || !sugarEntry) die ('Not a Valid Entry Point!');
$date = new DateTime();
echo '<H2>Wilderness Import Started</h2>';
echo $date->format('r').'<br>';
echo '-----------------------------------------------------------------------
--------------------------------------------------------------<br>';
require_once("include/utils/sugar_file_utils.php");
WildernessImportJob();
die();
function var_dump_ret($mixed = null) {
ob_start();
var_dump($mixed);
$content = ob_get_contents();
ob_end_clean();
return $content;
}
function time_elapsed()
{
static $first = null;
static $previous = null;
$now = microtime(true);
if ($first == null) $first = $now;
if ($previous != null)
echo '--- Partial ' . round(($now - $previous), 2) . ', Total ' . round(($now
- $first), 2) . ' ---'; // 109s
$ret = round(($now - $previous), 2);
$previous = $now;
return $ret;
}
function myLog ($str2log)
{
file_put_contents('./zlog_'.date("j.n.Y").'.txt', date("H:i:s", time())."
".$str2log.PHP_EOL, FILE_APPEND);
}
function calcDelta($a1, $a2)
{
//combine into a nice associative array:
$delta=Array();
foreach ($a1 as $key=>$value)
{
if ($a1[$key] != $a2->$key)
$delta[] = array($key => ("Was ". $a1[$key]. ", became " . $a2->$key));
}
$num = count($data);
if (empty($a1)) $delta[] = array("a1" => ("Was empty"));
if (empty($a2)) $delta[] = array("a2" => ("Was empty"));
return $delta;
}
require_once("include/utils/sugar_file_utils.php");
function fillPerson($record, &$person)
{
// $record is what is being imported from CSV
// $person is the bean about to be filled and going into the SuitCRM DB. It
may be new or not, depending on whether it exists in the DB previously.
// name: only updates if not existant yet, because it's the key we use for
search, and because names are more complex with parts
if ($person->full_name_c == "") {
$recordname = $record["FULL NAME"]; // != "") ? $record["FULL NAME"] : "
[To-be-filled]");
//echo $prefix;
$recordname = str_replace(" ", " ", $recordname);
echo $recordname;
$parts = explode(" ", $recordname);
$person->last_name = array_pop($parts);
$person->first_name = $parts[0];
$person->name = $person->first_name . " " . $person->last_name;
$person->full_name_c = $record["FULL NAME"]; // custom field created in
Studio
}
//$datanasc = DateTime::createFromFormat('!m/d/Y', $record["PPE"]);
// $datasnasc->setTime(0, 0);
// $person->ppe_date_c = ($datanasc == false) ? "" : $datanasc->format('m-d-
Y');
//$person->ppe_date_c = $record["PPE"];
//print_r($person);
var_dump($person);
}
// finish by making a complete analysis of what changed:
return calcDelta($person->fetched_row, $person);
function GetOrCreateMember ($the_name)
{
//Check if the fullname is null
if ($the_name != "")
{
$person = BeanFactory::getBean("locte_Members");
$person = $person->retrieve_by_string_fields(array('full_name_c' =>
$the_name));
if (is_null($person))
{
//get members bean
$person = BeanFactory::newBean("locte_Members");
//set full_name_c to the_name variable
$person->full_name_c = $the_name;
// $person->lcl_employee_id = $personEmployeeID;
$person_name = str_replace(" ", " ", $the_name);
$parts = explode(" ", $person_name);
$person->last_name = array_pop($parts);
$person->first_name = $parts[0];
//combine first and last name to populate the fullname field
$person->name = $person->first_name . " " . $person->last_name;
$person_id = $person->save();
// add new duespayment to member record
// $rosterDuesPayments = BeanFactory::getBean('Dues Payments')-
>retrieve_by_string_fields(array('name'=> $duesEmployeeID));
// $person->load_relationship('locte_Members_adues_dues'); //confirm
relationship name in cache
// $person->dues_payments->add($rosterDuesPayments->id);
}
return $person;
}
return null;
}
function WildernessImportJob()
{
try
{
time_elapsed();
$GLOBALS['log']->info('Wilderness Import');
$config = new Configurator();
$config->loadConfig();
$xmlDataDir = 'custom/wimporter/ToImport'; //$config->config['WildernessImporter_DataFilePath'];
$GLOBALS['log']->info("Wilderness Import: Scanning XML Data dir $xmlDataDir...");
echo("<h3>Wilderness Import: Scanning XML Data dir $xmlDataDir...<br /></h3>");
$directoryContent = scandir($xmlDataDir);
$GLOBALS['log']->info("Wilderness Import: Scanning XML Data dir $xmlDataDir... [Found " . count($directoryContent) . " files]");
echo("<h3>Wilderness Import: Scanning XML Data dir $xmlDataDir... [Found " . count($directoryContent) . " files]</h3><br />");
foreach ($directoryContent as $itemFile)
{
if (is_dir($xmlDataDir . DIRECTORY_SEPARATOR . $itemFile)) continue;
if (strcasecmp(substr($itemFile, -4), ".csv") != 0) continue;
$GLOBALS['log']->info("Wilderness Import: Processing $itemFile file...");
myLog("---------------------------------------------------");
myLog("Wilderness Import: Processing $itemFile file...");
myLog("----------------------------------------------------");
echo("<h4>---------------------------------------------------------------</h4>");
echo("<h4>Wilderness Import: Processing $itemFile file...</h4>");
echo("<h4>---------------------------------------------------------------</h4>");
$oFile = fopen($xmlDataDir . DIRECTORY_SEPARATOR . $itemFile, 'r');
if ($oFile !== FALSE)
{
// read entire file at once:
// expected separator is ",", expected encoding is UTF-8 without BOM (BOM is 3 weird characters in beginning of file)
while (($data[] = fgetcsv($oFile, 0, ',')) !== FALSE) { }
echo('File opened..... <br /> <br />');
fclose($oFile);
//combine into a nice associative array:
$arow=Array();
echo('Building CSV File Row Array <br /><br />');
$fields = array_shift($data);
echo('Building CSV Header Fields Array as shown below:<strong> <br /><br />');
echo implode(", ", $fields) . "</strong><br /><br />\n";
foreach ($data as $i=>$arow)
{
$GLOBALS['log']->info("Wilderness Import: array_combine " . $i);
$data[$i] = array_combine($fields, $arow);
}
unset($arow); // **********************************************! ! ! !! ! ! ! ! ! !! !
$num = count($data);
echo('Build Full Array of Roster to be Imported Complete. Entries to be imported are shown below <br /><br />');
for ($row=0; $row < $num - 1 ; $row++)
{ // normal bounds: from 0 to $num
//$num is the number of lines including header in csv file
echo "<strong>Filename: $itemFile | Roster Import, Row" . ($row + 1) . ":</strong><br />\n";
$GLOBALS['log']->info("Wilderness Import: Importing " . $data[$row]["FULL NAME"]);
// echo("<strong>Importing Roster Row #: ". ($row + 1) . "<br />" . "Local Number " . $data[$row]["AFFILIATE"] . "<br />" . "Employee: " . $data[$row]["FULL NAME"] . "</strong><br /><br />");
echo "<strong><table>\n";
foreach ($fields as $field) {
//echo "<tr><td>" . $field . "</td><td>" . $data[$row][$field] . "</td><td>" . $data[$row+1][$field] . "</td><td>" . $data[$row+2][$field] . "</td></tr>\n";
}
echo "</table>\n";
echo "File Row Data: ";
echo implode(", ", $data[$row]) . "</strong><br /><br />\n";
$Member = BeanFactory::getBean("locte_Members");
$FullName=$Member->full_name_c;
//$myfield_defs = $Member->getFieldDefinitions(); // just to help while developing
//foreach($myfield_defs as $def) echo $def["name"] . "<br />\n";
$Member=$Member->retrieve_by_string_fields(array('full_name_c' => $data[$row]["FULL NAME"]));
if (is_null($Member)) {
$Member = BeanFactory::newBean("locte_Members");
$delta = fillPerson($data[$row], $Member, ""); //->full_name_c, "FULL NAME");
}
if (count($delta)) {
$Member_id = $Member->save();
}
}
// Records have been saved: from this point on, only work on relationships:
$GLOBALS['log']->info('End: Wilderness Import');
myLog('End: Wilderness Import');
time_elapsed();
return true;
}
}
} catch (Exception $e)
{
$GLOBALS['log']->fatal("Wilderness Import: Exception " . $e->getMessage());
myLog("Wilderness Import: Exception " . $e->getMessage());
echo '\n\nCaught exception: ', $e->getMessage(), "\n";
return false;
}
}
It does return info from both the database and the csv file.
Image of error below.
Error - Capture from browser
Help always appreciated :)
This script is throwing error because the parameter $arow given to array_combine is not an array. So a check should be there to check whether $arow is an array or not.
Try code following:
foreach ($data as $i => $arow) {
$GLOBALS['log']->info("Wilderness Import: array_combine " . $i);
if (is_array($arow)) {
$data[$i] = array_combine($fields, $arow);
}
}
Read more about array_combine
Update
Code you are using to read csv need to be changed. Second parameter in fgetcsv must be greater than the longest line (in characters) to be found in the CSV file. So replace code
while (($data[] = fgetcsv($oFile, 0, ', ')) !== FALSE) {
}
with
while (($data[] = fgetcsv($oFile, 10, ', ')) !== FALSE) {
}
Read more about fgetcsv
This block is doing the saving of records using the beanfactory.... I think :|
$Member = BeanFactory::getBean("locte_Members");
$FullName=$Member->full_name_c;
//$myfield_defs = $Member->getFieldDefinitions(); // just to help while developing
//foreach($myfield_defs as $def) echo $def["name"] . "<br />\n";
$Member=$Member->retrieve_by_string_fields(array('full_name_c' => $data[$row]["FULL NAME"]));
if (is_null($Member)) {
$Member = BeanFactory::newBean("locte_Members");
$delta = fillPerson($data[$row], $Member->full_name_c, "FULL NAME");
var_dump($arow);
}
if (count($delta)) {
$Member_id = $Member->save();
}
}
// Records have been saved: from this point on, only work on relationships:
Here's the output of the script in the browser with field definition dump
Wilderness Import Started
Tue, 19 Jun 2018 02:05:56 -0400
Wilderness Import: Scanning XML Data dir custom/wimporter/ToImport...
Wilderness Import: Scanning XML Data dir custom/wimporter/ToImport... [Found 3 files]
Wilderness Import: Processing L1554v4.csv file...
File opened.....
Building CSV File Row Array
Building CSV Header Fields Array as shown below:
LAST NAME, FIRST NAME, FULL NAME
Build Full Array of Roster to be Imported Complete. Entries to be imported are shown below
Filename: L1554v4.csv | Roster Import, Row1:
LAST NAME BUTLER
FIRST NAME BRIANA
FULL NAME BUTLER BRIANA
File Row Data: BUTLER, BRIANA, BUTLER BRIANA
-id
-name
-date_entered
-date_modified
-modified_user_id
-modified_by_name
-created_by
-created_by_name
-description
-deleted
-securitygroup
-securitygroup_display
-created_by_link
-modified_user_link
-assigned_user_id
-assigned_user_name
-assigned_user_link
-additionalusers
-additionalusers_listview
-salutation
-first_name
-last_name
-full_name
-title
-photo
-department
-do_not_call
-phone_home
-email
-phone_mobile
-phone_work
-phone_other
-phone_fax
-email1
-email2
-invalid_email
-email_opt_out
-primary_address_street
-primary_address_street_2
-primary_address_street_3
-primary_address_city
-primary_address_state
-primary_address_postalcode
-primary_address_country
-alt_address_street
-alt_address_street_2
-alt_address_street_3
-alt_address_city
-alt_address_state
-alt_address_postalcode
-alt_address_country
-assistant
-assistant_phone
-email_addresses_primary
-email_addresses
-email_addresses_non_primary
-lcl_birthdate
-lcl_affiliate_number
-member_signature
-ssn
-staff_rep
-mem_join_date
-class_title
-steward
-olo_code
-enterprise_id
-member_card
-address_map
-member_status
-primary_language
-english_speaking
-annual_salary
-currency_id
-state_hire_date
-dues_frequency
-card_sent_date
-gender
-disabilities
-marital_status
-executive_board
-affiliate_type
-middle_name
-name_suffix
-mailableflag
-no_mailflag
-apt_number
-zip4
-infosrc
-lcl_phone_collected
-lcl_hasmobile
-lcl_hasemail
-lcl_member_status
-backofficeassistants
-backoffice_assistant_phonoe
-backoffice_assistant_email
-lcl_employercontact
-lcl_work_dept
-lcl_work_location
-lcl_jobclasstitle
-lcl_employee_id
-lcl_join_date
-lcl_sig_auth
-lcl_dues_status
-lcl_on_probation
-lcl_prob_end_date
-lcl_prob_term
-afs_signed_by
-afs_region_number
-afemp_employers_id_c
-afs_employer
-primary_address_county
-locte_members_adues_dues
-full_name_c
-locte_members_afcbu_cbu
-locte_members_afcbu_cbu_name
-locte_members_afcbu_cbuafcbu_cbu_idb
-ppe_date_c
--- Partial 0.01, Total 0.01 ---
I have a csv file containing about 3500 user's data. I want to import these users to my database and send them an email that they are registered.
I have the following code:
public function importUsers()
{
$this->load->model('perk/engagement_model');
$this->load->model('user/users_model');
$this->load->model('acl/aclUserRoles_model');
$allengagements = $this->engagement_model->getAll();
$filename = base_url() . 'assets/overdracht_users.csv';
$file = fopen($filename, "r");
$count = 0;
$totalImported = 0;
$importFails = array();
$mailFails = array();
while (($mappedData = fgetcsv($file, 10000, ";")) !== FALSE)
{
$count++;
//Skip first line because it is the header
if ($count > 1) {
if (!empty($mappedData[0])) {
$email = $mappedData[0];
$user = $this->users_model->getByEmail($email);
if (!$user) {
$user = new stdClass();
$user->email = $mappedData[0];
$user->first_name = $mappedData[1];
$user->family_name = $mappedData[2];
$user->address_line1 = $mappedData[3];
$user->address_postal_code = $mappedData[4];
$user->address_city = $mappedData[5];
$user->address_country = 'BE';
$user->volunteer_location = $mappedData[5];
$user->volunteer_location_max_distance = 50;
$user->phone = $mappedData[6];
if (!empty($mappedData[7])) {
$user->birthdate = $mappedData[7] . "-01-01 00:00:00";
} else {
$user->birthdate = null;
}
foreach ($allengagements as $eng) {
if ($eng->description == $mappedData[8]) {
$engagement = $eng->engagement_id;
}
}
$user->engagement = $engagement;
if (!empty($mappedData[9])) {
$date_created = str_replace('/', '-', $mappedData[9]);
$date_created = date('Y-m-d H:i:s', strtotime($date_created));
} else {
$date_created = date('Y-m-d H:i:s');
}
$user->created_at = $date_created;
if (!empty($mappedData[10])) {
$date_login = str_replace('/', '-', $mappedData[10]);
$date_login = date('Y-m-d H:i:s', strtotime($date_login));
} else {
$date_login = null;
}
$user->last_login = $date_login;
$user->auth_level = 1;
$user->is_profile_public = 1;
$user->is_account_active = 1;
$combinedname = $mappedData[1] . $mappedData[2];
$username = str_replace(' ', '', $combinedname);
if (!$this->users_model->isUsernameExists($username)) {
$uniqueUsername = $username;
} else {
$counter = 1;
while ($this->users_model->isUsernameExists($username . $counter)) {
$counter++;
}
$uniqueUsername = $username . $counter;
}
$user->username = $uniqueUsername;
$userid = $this->users_model->add($user);
if (!empty($userid)) {
$totalImported++;
//Add the user in the volunteer group in ACL
$aclData = [
'userID' => $userid,
'roleID' => 1,
'addDate' => date('Y-m-d H:i:s')
];
$this->aclUserRoles_model->add($aclData);
//Registration mail to volunteer
$mail_data['name'] = $user->first_name . ' ' . $user->family_name;
$mail_data['username'] = $user->username;
$this->email->from(GENERAL_MAIL, 'Test');
$this->email->to($user->email);
//$this->email->bcc(GENERAL_MAIL);
$this->email->subject('Test');
$message = $this->load->view('mail/register/registration',$mail_data,TRUE);
$this->email->message($message);
$mailsent = $this->email->send();
if (!$mailsent) {
array_push($mailFails, $mappedData);
}
} else {
array_push($importFails, $mappedData);
}
if ($count % 50 == 0) {
var_dump("count is " . $count);
var_dump("we are sleeping");
$min=20;
$max=40;
$randSleep = rand($min,$max);
sleep($randSleep);
var_dump("end of sleep (which is " . $randSleep . "seconds long)");
}
var_dump($user);
} else {
array_push($importFails, $mappedData);
}
}
}
}
var_dump("Totale aantal rijen in het bestand (met header) : " . $count);
var_dump("Totale aantal geimporteerd in de database : " . $totalImported);
var_dump("Totale aantal gefaalde imports in de database : " . count($importFails));
var_dump("Deze zijn gefailed : ");
var_dump($importFails);
}
If I do not add the users in the database, or send out a mail, and just var_dump() the $user, i can see all 3500+ users being correctly created in php objects (so they should be able to be inserted correctly).
The problem is that I want to add in a random sleep, between 20 and 40 seconds after every 50 mails that I sent.
So I started doing some testing and after commenting out the insert and mail code, I started running the script, noticing that after some amount (not 50 at all), it just stops for a bit, then continues and shows me the the var_dumps in my if case at the bottom, it can be shown here in the screenshots below.
The first screenshot shows the code stopping for a bit (note that I am only var_dumping stuff, i am not adding something in the database or sending out an email yet).
This screenshot shows what happens after the script reaches 200:
The script just completely stops from this point on. I have tried this 3 times, and every single time it stops exactly on 200.
What is happening here??
Most likely you're hitting default PHP limits. temporarily remove PHP default limits with:
ini_set('memory_limit',-1);
set_time_limit(0);
then, rerun the script and check your output.
There can be multiple reasons, but to know the exact one please enable error output with.
ini_set('display_errors',1);
ini_set('display_startup_errors',1);
and if limits are not the problem, the errors will tell you more.
HINT: using logs are still better than outputting errors to the visitors, but my guess is you're testing this on your computer.
I have two database (mysql) with the same structure. I want to:
compare data in two table. Table one - home and the second work,
send email with results,
update data in table work.
My query:
select id, code, quantity from wpx_products
I run this query in table home and work (two databases). And this is output:
"3";"home 005-07";"2"
"63";"home 033-12";"2"
"15";"home 005-19";"2"
and from work:
"1";"work 005-07";"2"
"2";"work 033-12";"5"
"3";"work 005-19";"2"
What I want to do ? What I mean by "compare" ? I want find record with excluding tag work or home in column 'code'. For example I want to find 033-12 and check quantity. If the difference copy value from home to work.
In first second I want to use trigger in mysql. But this is not solution for me because I cant run it by myself and I can't send email with results. What is the best way to achieve this functionality ? Thanks for help.
Kind regards
---------------------edit----------------------------
I check this code below (thanks #AntG). And I have one problem. When I print
foreach ($result_target AS $target) {
$code_target = substr($target['code'], 4);
if ($code_source === $code_target) {
if ($source['quantity'] !== $target['quantity']) {
print $source['quantity'] .' -> ' . $target['quantity']."<br /><br />";
$match[] = array('code' => $source['code'], 'quantity' => $source['quantity'], 'targetid' => $target['id'], 'sourceid' => $source['id']);
}
$found = true;
break;
}
}
I have this results: http://suszek.info/projekt1/ Like You see there is 50 values. When I print $match there is much more, duplicated value and I don't know why ?
$msg = '';
foreach ($match AS $entry) {
$msg .= 'Change identified: Home_ID=' . $entry['sourceid'] . ' code: ' . $entry['code'] . ' quantity:' . $entry['quantity'] . PHP_EOL . '<br />';
print $msg;
/* Perform DB updates using $entry['targetid'] and $entry['quantity'] */
}
I have this results: http://suszek.info/projekt1/index_1.php And all code:
$match = array(); $new = array();
foreach ($result_source AS $source) {
$found = false;
$code_source = substr($source['code'], 4);
foreach ($result_target AS $target) {
$code_target = substr($target['code'], 4);
if ($code_source === $code_target) {
if ($source['quantity'] !== $target['quantity']) {
print $source['quantity'] .' -> ' . $target['quantity']."<br /><br />";
$match[] = array('code' => $source['code'], 'quantity' => $source['quantity'], 'targetid' => $target['id'], 'sourceid' => $source['id']);
}
$found = true;
break;
}
}
if (!$found) {
$new[] = array('code' => $source['code'], 'quantity' => $source['quantity'], 'sourceid' => $source['id']);
} } $msg = ''; foreach ($match AS $entry) {
$msg .= 'Change identified: Home_ID=' . $entry['sourceid'] . ' code: ' . $entry['code'] . ' quantity:' . $entry['quantity'] . PHP_EOL
. '<br />';
print $msg;
/* Perform DB updates using $entry['targetid'] and $entry['quantity'] */ }
foreach ($new AS $entry) {
$msg .= 'New Entry: Home_ID=' . $entry['sourceid'] . ' code: ' . $entry['code'] . ' quantity:' . $entry['quantity'] . PHP_EOL . '<br
/>';
#print $msg;
/* Perform DB inserts using $entry['code'] and $entry['quantity'] if this is desired behaviour */ }
/* Send email with $msg */
If you are capturing both query results in a $results array, then I think substr() is the PHP function that you are after here:
$match=array();
$new=array();
foreach($results_home AS $home)
{
$found=false;
$code=substr($home['code'],5);
foreach($results_work AS $work)
{
if($code===substr($work,5))
{
if($home['quantity']!==$work['quantity'])
{
$match[]=array('code'=>$home['code'],'quantity'=>$home['quantity'],'workid'=>$work['id'],'homeid'=>$home['id']);
}
$found=true;
break;
}
}
if(!$found)
{
$new[]=array('code'=>$home['code'],'quantity'=>$home['quantity'],'homeid'=>$home['id']);
}
}
$msg='';
foreach($match AS $entry)
{
$msg.='Change identified: Home_ID='.$entry['homeid'].' code: '.$entry['code'].' quantity:'.$entry['quantity'].PHP_EOL;
/* Perform DB updates using $entry['workid'] and $entry['quantity'] */
}
foreach($new AS $entry)
{
$msg.='New Entry: Home_ID='.$entry['homeid'].' code: '.$entry['code'].' quantity:'.$entry['quantity'].PHP_EOL;
/* Perform DB inserts using $entry['code'] and $entry['quantity'] if this is desired behaviour */
}
/* Send email with $msg */
I am working on a php script to pull quest data from wowhead, particularly what starts and ends the quest, whether it is an item or a npc, and what its id or name is, respectively. This is the relevant portion of the whole script, with the rest involving database insertion. This is the completed snippet of code I came up with if anyone is interested. Also, seeing as this will run about 15,000 times, is this the best method of obtaining/storing the data?
<?php
$quests = array();
//$questlimit = 14987;
$questlimit = 5;
$currentquest = 1;
$questsprocessed = 0;
while($questsprocessed != $questlimit)
{
echo "<br>";
echo " Start of iteration: ".$questsprocessed." ";
echo "<br>";
echo " Attempting to process quest: ".$currentquest." ";
echo "<br>";
$quests[$currentquest] = array();
$baseurl = 'http://wowhead.com/quest=';
$fullurl = $baseurl.$currentquest;
$data = drupal_http_request($fullurl);
$queststartloc1 = strpos($data->data, 'quest_start');
$queststartloc2 = strpos($data->data, 'quest_end');
if($queststartloc1==false)
{$currentquest++; echo "No data for this quest"; echo "<br>"; continue;}
$questendloc1 = strpos($data->data, 'quest_end');
$questendloc2 = strpos($data->data, 'x5DDifficulty');
$startcaptureLength = $queststartloc2 - $queststartloc1;
$endcaptureLength = $questendloc2 - $questendloc1;
$quest_start_raw = substr($data->data,$queststartloc1, $startcaptureLength);
$quest_end_raw = substr($data->data, $questendloc1, $endcaptureLength);
$startDecoded = preg_replace('~\\\\x([A-Fa-f0-9]{2})~e', 'chr("0x$1")', $quest_start_raw);
$endDecoded = preg_replace('~\\\\x([A-Fa-f0-9]{2})~e', 'chr("0x$1")', $quest_end_raw);
$quests[$currentquest]['Start'] = array();
$quests[$currentquest]['End'] = array();
if(strstr($startDecoded, 'npc'))
{
$quests[$currentquest]['Start']['Type'] = "npc";
preg_match('~npc=(\d+)~', $startDecoded, $startmatch);
}
else
{
$quests[$currentquest]['Start']['Type'] = "item";
preg_match('~item=(\d+)~', $startDecoded, $startmatch);
}
$quests[$currentquest]['Start']['ID'] = $startmatch[1];
if(strstr($endDecoded, 'npc'))
{
$quests[$currentquest]['End']['Type'] = "npc";
preg_match('~npc=(\d+)~', $endDecoded, $endmatch);
}
else
{
$quests[$currentquest]['End']['Type'] = "item";
preg_match('~item=(\d+)~', $endDecoded, $endmatch);
}
$quests[$currentquest]['End']['ID'] = $endmatch[1];
//var_dump($quests[$currentquest]);
echo " End of iteration: ".$questsprocessed." ";
echo "<br>";
echo " Processed quest: ".$currentquest." ";
echo "<br>";
$currentquest++;
$questsprocessed++;
}
?>
These are called "escape sequences". Normally, they're used to represent characters not printable otherwise, but can encode any character. In php, you can decode them like this:
$text = '
quest_start\\x5DStart\\x3A\\x20\\x5Bitem\\x3D16305\\x5D\\x5B\\x2Ficon\\x5D\\x5B\\x2Fli\\x5D\\x5Bli\\x5D\\x5Bicon\\x20name\\x3Dquest_end\\x5DEnd\\x3A\\x20\\x5Burl\\x3D\\x2Fnpc\\x3D12696\\x5DSenani\\x20Thunderheart\\x5B\\x2Furl\\x5D\\x5B\\x2Ficon\\x5D\\x5B\\x2Fli\\x5D\\x5Bli\\x5DNot\\x20sharable\\x5B\\x2Fli\\x5D\\x5Bli
';
$decoded = preg_replace('~\\\\x([A-Fa-f0-9]{2})~e', 'chr("0x$1")', $text);
Which gives you a string similar to this:
quest_start]Start: [item=16305][/icon][/li][li][icon name=quest_end]End: [url=/npc=12696]Senani Thunderheart[/url][/icon][/li][li]Not sharable[/li][li
(obviously, some kind of BB-code). To remove all bbcodes, yet one replacement is necessary:
$clean = preg_replace('~(\[.+?\])+~', ' ', $decoded);