I'm trying to build a mail client with php and I get an issue when moving the e-mails from mail server to my database.
I'm using imap_fetchbody function. However, when I try to count(using imap_num_msg) or get the headers(including "0" on the 'section' parameter of imap_fetchbody) it does recognize all emails, without the body obviously. Database is local.
function getEmails(){
$query = "SELECT * FROM users";
$result = mysqli_query($this->connection, $query);
while($row = mysqli_fetch_assoc($result)) {
$imapResource = imap_open($row["mailbox"], $row["usernamemail"], $row["passwordmail"], NULL) or die ('IMAP connection error');
}
$body_count = imap_num_msg($imapResource);
echo "There is a total of " . $body_count . " emails in your inbox. <br>"; // This counts REAL total amount of emails
$emails = imap_search($imapResource, "ALL");
rsort($emails);
if(!empty($emails)){
foreach($emails as $email){
// Fetch mail content
$overview = imap_fetch_overview($imapResource, $email, 0);
$overview = $overview[0];
$structure = imap_fetchstructure($imapResource, $email);
$attachments = array();
$id = $overview->uid;
$date = $overview->date;
$from = $overview->from;
$to = $overview->to;
$subject = $overview->subject;
$seen = $overview->seen;
if ($seen == 0){
$seen = "Not Seen";
} else{
$seen = "Seen";
}
$body = imap_fetchbody($imapResource, $email, "1", FT_PEEK);
// Check duplicates before uploading
$checkQuery = "SELECT ID FROM emails";
$result = mysqli_query($this->connection, $checkQuery);
if(mysqli_num_rows($result) > 0) {
while($row = mysqli_fetch_assoc($result)){
$testID = (string)$id;
if($testID !== $row["ID"]){
// Input into DB
$query = "INSERT INTO emails (ID, Date_recieved, From_email, To_email, Subject, Body, Type, Seen) VALUES ('$id', '$date', '$from', '$to', '$subject', '$body', '$emailType', '$seen')";
mysqli_query($this->connection, $query);
} else {
break;
}
}
} else{
// Input into DB
$query = "INSERT INTO emails (ID, Date_recieved, From_email, To_email, Subject, Body, Type, Seen) VALUES ('$id', '$date', '$from', '$to', '$subject', '$body', '$emailType', '$seen')";
mysqli_query($this->connection, $query);
}
}
}
The goal is for all emails to be inserted into the database and then pulled out for display and managed by user
Fixed The problem was special characters in Email's Subject (single-quote).
Noticed after setting
mysqli_report(MYSQLI_REPORT_ERROR | MYSQLI_REPORT_STRICT);
To fix use mysqli_real_escape_string.
Fixed The problem was special characters in Email's Subject (single-quote). Noticed after setting
mysqli_report(MYSQLI_REPORT_ERROR | MYSQLI_REPORT_STRICT);
To fix use mysqli_real_escape_string.
Related
This question already has answers here:
How can I do 'insert if not exists' in MySQL?
(11 answers)
Closed 2 years ago.
I have a database that contains more than 640,000 records that I update every week with data from a JSON file. What I want to do is only load records into the database that do not currently exists. My script below works on small amounts of data but when I try to load a large file it times out (I get a 500 Internal Server Error). Is there a better way to do this?
<?php
set_time_limit(0);
ini_set('memory_limit','2000M');
$url = 'json/OERrecordstest.json';
$contents = file_get_contents($url);
$records = json_decode($contents, true);
include("../config.php");
echo "<div class='card card-body'>";
foreach($records as $record) {
$type = $record['type'];
$name = $record['title'];
$title = addslashes($name);
$creator = $record['author'];
$author = addslashes($creator);
$link = addslashes($record['link']);
$origin = $record['source'];
$source = addslashes($origin);
$description = addslashes($record['description']);
$base_url = $record['base_url'];
$isbn_number = $record['isbn_number'];
$e_isbn_number = $record['e_isbn_number'];
$publication_date = $record['publication_date'];
$license = $record['license'];
$subject = addslashes($record['subject']);
$image_url = $record['image_url'];
$review = $record['review'];
$language = $record['language'];
$license_url = $record['license_url'];
$publisher = addslashes($record['publisher']);
$publisher_url = $record['publisher_url'];
$query = $conn->prepare("SELECT * FROM oer_search WHERE title=:title AND author=:author AND source=:source");
$query->bindParam(":title", $name);
$query->bindParam(":author", $creator);
$query->bindParam(":source", $origin);
$query->execute();
if ($query->rowCount() == 0) {
$insert = $conn->prepare("INSERT INTO oer_search (type, title, author, link, source, description, base_url, isbn_number, e_isbn_number, publication_date, license, subject, image_url, review, language, license_url, publisher, publisher_url) VALUES ('$type', '$title', '$author', '$link', '$source', '$description', '$base_url', '$isbn_number', '$e_isbn_number', '$publication_date', '$license', '$subject', '$image_url', '$review', '$language', '$license_url', '$publisher', '$publisher_url')");
$insert->execute();
}
}
if($insert){
echo "<p><span class='recordInserted'><em>$name was successfully inserted into SOAR.</em></span></p>";
}
else {
echo "<p><span class='recordInserted'><em>Record(s) already exist in SOAR.</em></span></p>";
}
echo "</div>";
?>
I could not comment, I wrote as an answer because my score was not enough. can you change it like this and try it?
$query = $conn->prepare("SELECT id FROM oer_search WHERE title=:title AND author=:author AND source=:source limit 1");
or
<?php
if(!session_id()) session_start();
ini_set('memory_limit', '2000M');
$url = 'json/OERrecordstest.json';
$contents = file_get_contents($url);
$records = json_decode($contents, true);
include("../config.php");
echo "<div class='card card-body'>";
if (!$_SESSION["records"]) {
foreach ($records as $record) {
$_SESSION["records"][$record["id"]] = $records;
}
}
$i = 0;
foreach ($_SESSION["records"] as $record) {
$i++;
if ($i > 1000) break;
$type = $record['type'];
$name = $record['title'];
$title = addslashes($name);
$creator = $record['author'];
$author = addslashes($creator);
$link = addslashes($record['link']);
$origin = $record['source'];
$source = addslashes($origin);
$description = addslashes($record['description']);
$base_url = $record['base_url'];
$isbn_number = $record['isbn_number'];
$e_isbn_number = $record['e_isbn_number'];
$publication_date = $record['publication_date'];
$license = $record['license'];
$subject = addslashes($record['subject']);
$image_url = $record['image_url'];
$review = $record['review'];
$language = $record['language'];
$license_url = $record['license_url'];
$publisher = addslashes($record['publisher']);
$publisher_url = $record['publisher_url'];
$query = $conn->prepare("SELECT id FROM oer_search WHERE title=:title AND author=:author AND source=:source limit 1");
$query->bindParam(":title", $name);
$query->bindParam(":author", $creator);
$query->bindParam(":source", $origin);
$query->execute();
if ($query->rowCount() == 0) {
$insert = $conn->prepare("INSERT INTO oer_search (type, title, author, link, source, description, base_url, isbn_number, e_isbn_number, publication_date, license, subject, image_url, review, language, license_url, publisher, publisher_url) VALUES ('$type', '$title', '$author', '$link', '$source', '$description', '$base_url', '$isbn_number', '$e_isbn_number', '$publication_date', '$license', '$subject', '$image_url', '$review', '$language', '$license_url', '$publisher', '$publisher_url')");
$insert->execute();
unset($_SESSION["records"][$record["id"]]);
}
}
print "remaining data :". count($_SESSION["records"]);
?>
Tipps to speed up mass-imports:
Move your SQL prepare outside of the loop (you only have to do it once)
Collect data to insert into batches of 1000 (for example.. usually alot more possible)
Use transactions / disable Index calculation during insert
Find duplicates with a lookup array from existing data (don't query the database for each row of your import)
In general: Avoid SQL queries in Loops
hope that helps a bit
So it's probably a really stupid/basic question, but i have this simple PHP function (which works) and inserts data into a PostgreSQL DB.
My issue is when it encounters specific data;
function insertData($pg, $csvfile)
{
$x = 0;
foreach ($csvfile as $data)
{
$email = $csvfile[$x]['email'];
$name = $csvfile[$x]['name'];
$surname = $csvfile[$x]['surname'];
$query = "INSERT INTO users (email, name, surname) VALUES ('$email', '$name', '$surname')";
$result = pg_query($pg, $query);
$x++;
}
}
And while this works, it falls over with a surname such as:
O'hare
And obviously this occurs because then the PHP code comes out as:
...VALUES ('john#example.com', 'John', 'O'hare')";
but im not sure of how i should be structuring the PHP to allow for this.
Try this:
function insertData($pg, $csvfile) {
$nbr = count(file($csvfile));
for($i=0; $i<$nbr; $i++) {
$email = pg_escape_string( $csvfile[$i]['email'] );
$name = pg_escape_string( $csvfile[$i]['name'] );
$surname = pg_escape_string( $csvfile[$i]['surname'] );
$query = "INSERT INTO users (email, name, surname) VALUES ('$email', '$name', '$surname')";
$result = pg_query($pg, $query);
if (!$result) {
echo "Error while executing the query: " . $query;
exit;
}
}
}
You need to escape the string parameters. And it is much better if you can use PDO extension, because prepared statements can take care of escaping for you and also helps with preventing SQL injection and some other security concerns.
function insertData(PDO $dbh, $csvfile) {
$x = 0;
foreach ($csvfile as $data)
{
$query = "INSERT INTO users (email, name, surname) VALUES (?, ?, ?)";
$params = [
$csvfile[$x]['email'],
$csvfile[$x]['name'],
$csvfile[$x]['surname']
];
$statement = $pdo->prepare($query);
$statement->execute();
$x++;
}
}
PDO::prepare
PDOStatement::execute
Solution using prepared query
function insertData($dbname, $tbname, $csvfile)
{
$result = [];
// Connect to a database named "mary"
$dbconn = pg_connect("dbname=$dbname");
// Prepare a query for execution
$result = pg_prepare($dbconn, "my_query", 'INSERT INTO $1 (email, name, surname) VALUES ($2, $3, $4)');
// Execute the prepared query. Note that it is not necessary to escape
foreach ($csvfile as $data)
{
$email = $data['email'];
$name = $data['name'];
$surname = $data['surname'];
$query = "";
$result[] = pg_execute($dbconn, "my_query", array($tbname, $email, $name, $surname));
}
if (in_array(false, $result) )
return false;
else
return true;
}
$dbname = "your dbname";
$tbname = "name of table";
$csvFile = [];
if (insertData($dbname, $tbname, $csvFile))
echo "Data inserted";
else
echo "Data not inserted";
So i took note of the suggestions from #Karsten Koop and #TOH19, and came up with this code which is working;
function insertData($pg, $csvfile)
{
$x = 0;
foreach ($csvfile as $data)
{
$email = pg_escape_string($csvfile[$x]['email']);
$name = pg_escape_string($csvfile[$x]['name']);
$surname = pg_escape_string($csvfile[$x]['surname']);
$query = "INSERT INTO users (email, name, surname) VALUES ('".$email."', '".$name."', '".$surname."')";
$result = pg_query($pg, $query);
$x++;
}
}
I am using php mysql pdo in here and trying to concatenate fname and lname but nothing going right am encountering {"error":true,"error_msg":"Unknown error occurred in registration!"} ..plzz help me out,pardon me if am wrong
.php
<?php
/*
starts with database connection
and gives out the result of query
in json format
*/
require_once 'DB_Functions.php';
$db = new DB_Functions();
// json response array
$response = array("error" => false);
//proceed if fields are not empty
if (!empty($_POST['salutation']) && !empty($_POST['fname']) && !empty($_POST['mname']) && !empty($_POST['lname']) && !empty($_POST['pob']) && !empty($_POST['dob']) && !empty($_POST['qualification']) && !empty($_POST['pg']) && !empty($_POST['pgy']) && !empty($_POST['graduation']) && !empty($_POST['gy']) && !empty($_POST['schooling']) && !empty($_POST['sy']) && !empty($_POST['religion']) && !empty($_POST['caste']) && !empty($_POST['subcaste']) && !empty($_POST['familyname']) && !empty($_POST['fathername']) && !empty($_POST['mothername']) && !empty($_POST['brothers']) && !empty($_POST['sisters'])){
//reciving the post parameters
$salutation =$_POST['salutation'];
$fname = trim($_POST['fname']);
$mname = trim($_POST['mname']);
$lname = trim($_POST['lname']);
$pob = trim($_POST['pob']);
$dob = trim($_POST['dob']);
$qualification = trim($_POST['qualification']);
$pg = trim($_POST['pg']);
$pgy = trim($_POST['pgy']);
$graduation = trim($_POST['graduation']);
$gy = trim($_POST['gy']);
$schooling = trim($_POST['schooling']);
$sy = trim($_POST['sy']);
$religion = trim($_POST['religion']);
$caste = trim($_POST['caste']);
$subcaste = trim($_POST['subcaste']);
$familyname = trim($_POST['familyname']);
$fathername = trim($_POST['fathername']);
$mothername = trim($_POST['mothername']);
$brothers = trim($_POST['brothers']);
$sisters = trim($_POST['sisters']);
/*
validation process
begins from here
*/
// create a new user profile
$user = $db->storeUserProfile($salutation, $fname, $mname, $lname, $pob, $dob, $qualification, $pg, $pgy, $graduation, $gy, $schooling, $sy, $religion, $caste, $subcaste, $familyname, $fathername, $mothername, $brothers, $sisters);
if ($user){
// user stored successfully as post params passed
$response["error"] = false;
$response["uid"] = $user["id"];
$response["user"]["salutation"] = $user["salutation"];
$response["user"]["fname"] = $user["fname"];
$response["user"]["mname"] = $user["mname"];
$response["user"]["lname"] = $user["lname"];
$response["user"]["pob"] = $user["pob"];
$response["user"]["dob"] = $user["dob"];
$response["user"]["qualification"] = $user["qualification"];
$response["user"]["pg"] = $user["pg"];
$response["user"]["pgy"] = $user["pgy"];
$response["user"]["graduation"] = $user["graduation"];
$response["user"]["gy"] = $user["gy"];
$response["user"]["schooling"] = $user["schooling"];
$response["user"]["sy"] = $user["sy"];
$response["user"]["religion"] = $user["religion"];
$response["user"]["caste"] = $user["caste"];
$response["user"]["subcaste"] = $user["subcaste"];
$response["user"]["familyname"] = $user["familyname"];
$response["user"]["fathername"] = $user["fathername"];
$response["user"]["mothername"] = $user["mothername"];
$response["user"]["brothers"] = $user["brothers"];
$response["user"]["sisters"] = $user["sisters"];
$response["user"]["uuid"] = $user["unique_id"];
$response["user"]["created_at"] = $user["created_at"];
$response["user"]["updated_at"] = $user["updated_at"];
echo json_encode($response);
} else {
// user failed to store
$response["error"] = true;
$response["error_msg"] = "Unknown error occurred in registration!";
echo json_encode($response);
}
}else{
//missing the required fields
$response["error"] = true;
$response["error_msg"] = "Please fill all the required parameters!";
echo json_encode($response);
}
?>
this is the database part using pdo.
php
public function storeUserProfile($salutation, $fname, $mname, $lname, $pob, $dob, $qualification, $pg, $pgy, $graduation, $gy, $schooling, $sy, $religion, $caste, $subcaste, $familyname, $fathername, $mothername, $brothers, $sisters){
try {
$characters = '0123456789';
$uuid = '';
$random_string_length = 6;
for ($i = 0; $i < $random_string_length; $i++) {
$uuid .= $characters[rand(0, strlen($characters) - 1)];
}
$sql = "INSERT INTO profile_info(salutation, fname, mname, lname, fullname, pob, dob, qualification, pg, pgy, graduation, gy, schooling, sy, religion, caste, subcaste, familyname, fathername, mothername, brothers, sisters, unique_id, created_at) VALUES ( '$salutation', '$fname', '$mname', '$lname', '$fname'.', '.'$lname', '$pob', '$dob', '$qualification', '$pg', '$pgy', '$graduation', '$gy', '$schooling', '$sy', '$religion', '$caste', '$subcaste', '$familyname', '$fathername', '$mothername', '$brothers', '$sisters', '$uuid', NOW())";
$dbh = $this->db->prepare($sql);
if($dbh->execute()){
//concatenate the strings
$sql = "UPDATE profile_info SET fullname = CONCAT(fname, ', ', lname)";
$dbh = $this->db->prepare($sql);
$dbh->execute();
// get user details
$sql = "SELECT * FROM profile_info WHERE familyname = '$familyname' LIMIT 1";
$dbh = $this->db->prepare($sql);
$result = $dbh->execute();
$rows = $dbh->fetch();
$n = count($rows);
if($n){
return $rows;
}
}
}
catch (Exception $e) {
die('Error accessing database: ' . $e->getMessage());
}
return false;
}
The concatenation of first name and last name in your INSERT query is incorrect. Use a $fullname variable to specify full name of the person, and use that variable in your INSERT query. That way you won't have to update the row because you have already inserted the row with the correct full name.
Your code should be like this:
// your code
$fullname = $fname . ", " . $lname;
$sql = "INSERT INTO profile_info(salutation, fname, mname, lname, fullname, pob, dob, qualification, pg, pgy, graduation, gy, schooling, sy, religion, caste, subcaste, familyname, fathername, mothername, brothers, sisters, unique_id, created_at) VALUES ( '$salutation', '$fname', '$mname', '$lname', '$fullname', '$pob', '$dob', '$qualification', '$pg', '$pgy', '$graduation', '$gy', '$schooling', '$sy', '$religion', '$caste', '$subcaste', '$familyname', '$fathername', '$mothername', '$brothers', '$sisters', '$uuid', NOW())";
$dbh = $this->db->prepare($sql);
if($dbh->execute()){
// get user details
$sql = "SELECT * FROM profile_info WHERE familyname = '$familyname' LIMIT 1";
$dbh = $this->db->prepare($sql);
$result = $dbh->execute();
$rows = $dbh->fetch();
$n = count($rows);
if($n){
return $rows;
}
}
// your code
If I understand the issue properly, the values are not being inserted because you are executing, instead, a SELECT statement. SELECT statements do not modify table data. You would instead do something like this:
UPDATE profile_info SET fullname = CONCAT(fname, ', ', lname);
Note, this would update the entire table....
This will fill in a pre-existing column with the new concatenated value made from the fname and lname values of each row.
Of course, if your table does not currently have a column for fullname, add one:
ALTER TABLE profile_info ADD COLUMN fullname varchar(25);
UPDATE
Take this line out:
$sql = UPDATE profile_info SET fullname = CONCAT(fname, ', ', lname);
And change this line:
$sql = "INSERT INTO profile_info(salutation, fname, mname, lname, fullname, pob, dob, qualification, pg, pgy, graduation, gy, schooling, sy, religion, caste, subcaste, familyname, fathername, mothername, brothers, sisters, unique_id, created_at) VALUES ( '$salutation', '$fname', '$mname', '$lname', '$fname'.', '.'$lname', '$pob', '$dob', '$qualification', '$pg', '$pgy', '$graduation', '$gy', '$schooling', '$sy', '$religion', '$caste', '$subcaste', '$familyname', '$fathername', '$mothername', '$brothers', '$sisters', '$uuid', NOW())";
You'll see I added 'fullname' in the columns list, and this in the values list: '$fname'.', '.'$lname',
using PHP's concatenation operator .
The correct way to accomplish this is to simply concatenate the values and insert them at the very same time you insert the rest of the values. Let me know if that does it for you.
A side note, editing your original code does make the question more confusing for viewers who came in after the edits were made. Consider adding notes about any edits to the code, instead of editing the original example.
I'm trying to develop a E-Mail-Provider-Script.
With the following function the E-Mails should be saved in the MySQL database directly, but my efforts to save only the TEXT / HTML section failed. What am I doing wrong( likely with imap_fetchbody() )?
function getEmailsImap($mailserver, $port, $user, $pass)
{
$imap = imap_open( "{" . $mailserver . ":" . $port . "}INBOX", $user, $pass );
$check = imap_mailboxmsginfo($imap);
$totalrows = imap_num_msg($imap);
//iterate through all unread mails
for ($index = 0; $index < $totalrows; $index++)
{
$header = imap_header($imap, $index + 1);
//get mail subject
$subject = $header->subject;
//get mail sent date
$date = date(DateTime::ISO8601 , $header->udate);
//get email authors
$email = "{$header->from[0]->mailbox}#{$header->from[0]->host}";
//get body
$body = imap_fetchbody($imap, $index+1, "1.2"); /*** I think this might be the mistake **/
//get user
$to = $header->to[0]->mailbox;
$user = explode("#", $to)[0];
$id = (int)mysql_fetch_row(mysql_query("SELECT `id` FROM `fd_emails` ORDER BY `id` DESC LIMIT 1"))[0];
$new_id = $id+1;
$sql = mysql_query("INSERT INTO `fd_emails` (`id`, `subject`, `text`, `sender`, `user`, `date`)
VALUES ('$new_id', '$subject', '$body', '$email', '$user', '$date');");
imap_delete($imap, $index + 1);
}
//close connection to mailbox
imap_expunge($imap);
imap_close($imap);
return true;
}
Many thanks in advance.
As per the php documentation
string imap_fetchbody ( resource $imap_stream , int $msg_number , string $section [, int $options = 0 ] )
The number must have string type only. Using a integer type would raise an error.
I'm currently making a simple script that takes a user input named comments and putting it in a database. Every time I use the same email, I want it to overwrite their last entry. However, it keeps putting a new entry every time. Here is my code:
if($comments){
try{
echo "<img width=\"245\" height=\"130\" src=\"logo.png\"/><br/>";
echo "<h1>Thank you. You should receive your order on xx-xx-xx</h1>";
$TF = "TRUE";
if($numrows == 0){
$postquery = "INSERT INTO TTT25 (email,card,changes,comments) VALUES ('$email','$businesscard','$TF','$comments')";
$querythepost = sqlsrv_query($conn, $postquery);
}
else{
$postquery = "UPDATE TTT25 SET changes = '$TF', comments = '$comments' WHERE email = '$email'";
$querythepost = sqlsrv_query($conn, $postquery);
}
}
catch(Exception $e){}
}
elseif($optout=="false"){
echo "<img width=\"245\" height=\"130\" src=\"logo.png\"/><br/>";
echo "<h1>Thank you. You should receive your order on xx-xx-xx</h1>";
$TF = "FALSE";
$comments = "";
if($numrows == 0){
$postquery = "INSERT INTO TTT25 (email,card,changes,comments) VALUES ('$email','$businesscard','$TF','$comments')";
$querythepost = sqlsrv_query($conn, $postquery);
}
else{
$postquery = "UPDATE TTT25 SET changes = '$TF', comments = '$comments' WHERE email = '$email'";
$querythepost = sqlsrv_query($conn, $postquery);
}
}
Sorry it must have cut off:
my num rows and other variables defined before the conditional statements:
$optout = $_GET['opt'];
$encodedemail = $_GET['email'];
$email = base64_decode($encodedemail);
$originalcard = base64_decode($_GET['card']);
$businesscard = $originalcard;
$comments = $_POST['comments'];
//$primary = md5(uniqid(rand (), true)); no longer needed
$postquery;
$TF;
$sqlmatch = sqlsrv_query("SELECT * FROM TTT25 WHERE email = '".$email."'");
$numrows = sqlsrv_num_rows($sqlmatch);
echo $numrows;