Moving html table content to sql database - php

I've got a table (grabbed from another website using php) but now I want to import it's content into my database (instead of just printing out the table).
This is the printout (http://i.stack.imgur.com/NnbAo.png) from the table using the following code:
<?php
$uitvoer = getTable();
echo "<table">;
foreach ($uitvoer as $row) {
echo "<tr>";
foreach ($row as $col) {
echo "<td>" . $col . "</td>";
}
echo "</tr>";
}
echo"</table>";
?>
And when I try this nothing happens to my database:
<?php
$con = mysqli_connect("mysql1.000webhost.com","a2902119_lowavol","pswd")
$uitvoer = getTable();
foreach ($uitvoer as $row) {
foreach ($row as $col) {
$query += "'" . $col . "', ";
}
$sql="INSERT INTO test (cel1, cel2, cel3, cel4, cel5, cel6, cel7, cel8, cel9) VALUES ($query)";
mysqli_query($con, $sql);
}
?>
I have little to no knowledge about PHP. Thanks for helping me!

The concatenation operator for PHP is (.) and not (+)
<?php
$con = mysqli_connect("mysql1.000webhost.com","a2902119_lowavol","pswd")
$uitvoer = getTable();
foreach ($uitvoer as $row) {
$query = array();
$insert = "";
$insert_values = "";
foreach ($row as $col) {
$query[] = $col;
}
$insert_values = implode("','", $query) ;
$insert = "'".$insert_values."'";
$sql = "INSERT INTO test (cel1, cel2, cel3, cel4, cel5, cel6, cel7, cel8, cel9) VALUES ($insert)";
mysqli_query($con, $sql);
}
?>

your query will look like this:
INSERT INTO test (....) VALUES (val, val, val, )
which is obviously a syntax error... try and add the fields to an array instead:
foreach ($uitvoer as $row) {
$fields = array();
foreach ($row as $col) {
$fields[] = $col; // TODO: add escaping!
}
// now you can build the values list with implode
$values = implode(', ', $fields);
$sql="INSERT INTO test (...) VALUES ($values)";
See the docs about implode() for more info.

Use a prepared statement:
$sql="INSERT INTO test (cel1, cel2, cel3, cel4, cel5, cel6, cel7, cel8, cel9) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)";
$stmt = mysqli_prepare($con, $sql);
mysqli_stmt_bind_param($stmt, "sssssssss", $row[0], $row[1], $row[2], $row[3], $row[4], $row[5], $row[6], $row[7], $row[8]);
foreach ($uitvoer as $row) {
mysqli_stmt_execute($stmt);
}

Try echo-ing $sql. I think there is a comma in the end.
Use
$query = substr($query, 0, strlen($query) - 1)
in order to remove the last character of $query
(edited the command)

Related

How to print PDO CRUD Statements and Values of bindParams -PHP PDO

I am new to PDO PHP. I have created a post data function in order to achieve the insertion functionality. Here is my code.
public function PostData($conn, $table, $data)
{
try {
$query = "INSERT INTO " . $table . "(";
$arraylen = count($data);
$keyloopcount = 0;
foreach ($data as $key => $values) {
$query .= $key;
if ($keyloopcount != $arraylen - 1) {
$query .= ",";
}
$keyloopcount++;
}
$valloopcount = 0;
$query .= ") VALUES (";
foreach ($data as $key => $values) {
$query .= "':" . $key . "'";
if ($valloopcount != $arraylen - 1) {
$query .= ",";
}
$valloopcount++;
}
$query .= ")";
$stmt = $conn->prepare($query);
foreach ($data as $key => &$values) {
$stmt->bindParam(':'.$key, $values, PDO::PARAM_STR);
}
if ($stmt->execute()) {
$stmt->debugDumpParams();
echo "me";
} else {
echo "die";
}
} catch (PDOException $ex) {
echo $ex->getMessage();
}
}
When I execute this query. It doesn't show any exceptions. but when I look into my Database. It shows inserted records as follows.
I am wondering if there is a way I can debug what are the values in the binded params. I have also looked for $stmt->debugDumpParams(); as shown in the code. But it doesn't shows the values of params.
Need Guidance. TIA.
I sorted it out. I was making a silly mistake. The solution was so simple and easy.
Solution
The problem was lying in the part of the code.
foreach ($data as $key => $values) {
$query .= "':" . $key . "'";
if ($valloopcount != $arraylen - 1) {
$query .= ",";
}
$valloopcount++;
}
Changed the code to:
foreach ($data as $key => $values) {
$query .= ":" . $key;
if ($valloopcount != $arraylen - 1) {
$query .= ",";
}
$valloopcount++;
}
Resultant Query Before Amendment:
INSERT INTO cmp_categories(cat_name,cat_slug,cat_desc) VALUES (':cat_name',':cat_slug',':cat_desc')
Resultant Query AfterAmendment:
INSERT INTO cmp_categories(cat_name,cat_slug,cat_desc) VALUES (:cat_name,:cat_slug,:cat_desc)
More Clearly,
Removed ' from VALUES clause. ':cat_desc' => :cat_desc

Insert Multiple rows from loop query

I'm trying to get a list of customers from a table and then append data onto them each day. I can't figure out how to insert into the database multiple rows with the same name
I have 3 tables: Companies, jobs, reports. I'm doing a 2 loops to get the companies and jobs data from the tables into a form which is added to each day which will go into the repots table. I've tried doing it as an array and doing it in a while loop, foreach loop but each time hasn't worked. I'm going wrong somewhere i just dont know where
$companyquery = "SELECT * FROM companiestest WHERE (enabled != 'true') ORDER BY name";
$companies = $mysqli->query($companyquery);
while($company = $companies->fetch_assoc()) {
$cid = htmlspecialchars($company['id']);
$companyname = htmlspecialchars($company['name']);
echo "<tr>";
$jobquery = "SELECT * FROM jobstest WHERE (cid = " . $company['id'] . ") AND (enabled != 'true') ORDER BY jobname";
$jobs = $mysqli->query($jobquery);
while($job = $jobs->fetch_assoc()) {
$bid = htmlspecialchars($job['id']);
$product = htmlspecialchars($job['product']);
$server = htmlspecialchars($job['server']);
$medium = htmlspecialchars($job['medium']);
$jobname = htmlspecialchars($job['jobname']);
echo "<input type='hidden' name='backupid' value='$bid''>";
echo "<input type='hidden' name='companyname' value='$companyname''>";
echo "<input type='hidden' name='companyid' value='$cid'>";
echo "<td>$companyname - $cid</td>";
echo "<td>$product</td>";
echo "<td>$server</td>";
echo "<td>$medium</td>";
echo "<td>$jobname</td>";
echo "<td><select name='result' required>
<option disabled selected value>Result</option>
<option value='success'>Successful</option>
<option value='warn'>Warning</option>
<option value='fail'>Fail</option>
</select></td>";
echo "<td><input type='text' name='ticket' placeholder='Ticket Number'></td>";
echo "</tr>";
if(isset($_POST['submit'])){
$companyname = strip_tags(trim($_POST['companyname']));
$date = strip_tags(trim($_POST['date']));
$staffmember = strip_tags(trim($_POST['staff']));
$result = strip_tags(trim($_POST['result']));
$ticket = strip_tags(trim($_POST['ticket']));
$companyid = strip_tags(trim($_POST['companyid']));
$jobid = strip_tags(trim($_POST['jobid']));
$array = array();
array_push($array, $companyname, $date, $staffmember, $result, $ticket, $companyid, $jobid);
//$array = "('$companyname', '$date', '$staffmember', '$result', '$ticket', '$companyid')";
print_r($array);
$query = "INSERT INTO reporttest (company, date, staff, result, ticketnum, cid, bid) VALUES ($companyname, $date, $staffmember, $result, $ticket, $companyid, $jobid)";
//echo "$query";
}
The answer is a foreach loop with a key increment which inserts each variable into its own array. Then a foreach on the array to bind and execute into the database. Still a lot of modification to do with the data and restricting columns in the queries but so far it seems to be working without an issue.
No doubt there's a better way to do it but nobody has suggested anything to help as yet
$companyquery = "SELECT * FROM companiestest WHERE (enabled != 'true') ORDER BY name";
$companies = $mysqli->query($companyquery);
while($company = $companies->fetch_assoc()) {
$cid = htmlspecialchars($company['id']);
$companyname = htmlspecialchars($company['name']);
echo "<tr>";
$backupjobquery = "SELECT * FROM backupjobstest WHERE (cid = " . $company['id'] . ") AND (enabled != 'true') ORDER BY jobname";
$backupjobs = $mysqli->query($backupjobquery);
foreach($backupjobs as $key => $backupjob){
$key++;
$bid = htmlspecialchars($backupjob['id']);
$product = htmlspecialchars($backupjob['product']);
$server = htmlspecialchars($backupjob['server']);
$medium = htmlspecialchars($backupjob['medium']);
$jobname = htmlspecialchars($backupjob['jobname']);
echo "<input type='hidden' name='backupid[]' value='$bid'>";
echo "<input type='hidden' name='companyid[]' value='$cid'>";
echo "<td>$companyname - $cid</td>";
echo "<td>$product</td>";
echo "<td>$server</td>";
echo "<td>$medium</td>";
echo "<td>$jobname</td>";
echo "<td><select name='result[]' required>
<option disabled selected value>Result</option>
<option value='Successful'>Successful</option>
<option value='Warning'>Warning</option>
<option value='Failure'>Failure</option>
</select></td>";
echo "<td><input type='number' name='ticketnum[]' placeholder='Ticket Number'></td>";
echo "</tr>";
}
}
if(isset($_POST['submit'])){
$results = array();
$ticketnums = array();
$cids = array();
$bids = array();
$date = strip_tags(trim($_POST['date']));
$staffmember = strip_tags(trim($_POST['staff']));
foreach ($_POST['result'] as $key => $result) {
array_push($results, $result);
}
foreach ($_POST['ticketnum'] as $key => $ticketnum) {
array_push($ticketnums, $ticketnum);
}
foreach ($_POST['companyid'] as $key => $cid) {
array_push($cids, $cid);
}
foreach ($_POST['backupid'] as $key => $bid) {
array_push($bids, $bid);
}
$sql = "INSERT INTO reporttest (date, staff, result, ticketnum, cid, bid) VALUES (?, ?, ?, ?, ?, ?)";
$stmt = $mysqli->prepare($sql);
if ( !$stmt ) { die('prepare failed'); }
$stmt->bind_param("sssiii", $date, $staffmember, $result, $ticketnum, $cid, $bid);
foreach ($bids as $key => &$bid) {
$result = $results[$key];
$ticketnum = $ticketnums[$key];
$cid = $cids[$key];
$bid = $bids[$key];
if ($stmt->execute()) {
echo "Successfully updated the report";
} else {
echo "Error updating";
}
$stmt->fetch();
printf("Date: %s, Staff: %s, result: %s, Ticket: %s, cid: %s, bid: %s", $date, $staffmember, $result, $ticketnum, $cid, $bid);
echo "<br>";
}
$stmt->close();
}

Insert an array to MySQL database using PHP's prepared statements

I have an input array that I want to insert to database. I don't want to have 1 row in database for each item in the array. I want them all to go on same row. So this is my code:
<?php
session_start();
include('../../config/dbconf.php');
mysqli_select_db($conn, $webdb);
$stmt = $conn->prepare("INSERT INTO changelog (title, type, content, author, post_date) VALUES (?, ?, ?, ?, ?)");
$title = $_POST['title'];
$type = $_POST['type'];
$change = $_POST['change'];
$author = $_SESSION['waradmin'];
$date = time();
foreach($_POST['change'] as $key => $value) {
$changes = "<li>" . $change[$key] . "</li>";
}
$stmt->bind_param("sissi", $title, $type, $changes, $author, $date);
if($stmt->execute()) {
header('location: ../?p=dashboard');
}else{
echo "Error: " . $stmt->error;
}
$stmt->close();
?>
The query runs in database but only the first list item from the array... Do I need to use the implode function? I have never used it so if that is the only solution can someone show me how i use that in this query?
Instead of replacing variable value you have to concatenate
$changes = '';
foreach($_POST['change'] as $key => $value) {
$changes .= "<li>" . $change[$key] . "</li>";
}
And this is how to make a little more clean:
$changes = '';
foreach($_POST['change'] as $value) {
$changes .= '<li>' . $value . '</li>';
}

Build a batch query for MySQL insert each 1000 items

I need to perform a batch insert in MySQL/MariaDB but since data is dynamic I need to build the proper SQL query. In a few steps:
I should find whether the current row exists or not in table - this is the first SELECT inside the loop
Right now I have 1454 but have to insert around 150k later, is better a batch query than 150k INSERT per item on the loop
If record already exists I should update it if doesn't then I should insert ,I just not care about UPDATE yet and the code you're seeing is only for INSERT
So here is what I am doing:
// Get values from Csv file as an array of values
$data = convertCsvToArray($fileName);
echo "DEBUG count(data): ", count($data), "\n";
$i = 0;
$sqlInsert = "INSERT INTO reps(veeva_rep_id,first,last,email,username,lastLoginAt,lastSyncAt,display_name,rep_type,avatar_url,createdAt,updatedAt) ";
// Processing on each row of data
foreach ($data as $row) {
$sql = "SELECT id,lastSyncAt FROM reps WHERE veeva_rep_id='{$row['Id']}'";
echo "DEBUG: ", $sql, "\n";
$rs = $conn->query($sql);
if ($rs === false) {
echo 'Wrong SQL: '.$sql.' Error: '.$conn->error, E_USER_ERROR;
} else {
$rows_returned = $rs->num_rows;
$veeva_rep_id = "'".$conn->real_escape_string($row['Id'])."'";
$first = "'".$conn->real_escape_string(ucfirst(strtolower($row['FirstName'])))."'";
$last = "'".$conn->real_escape_string(ucfirst(strtolower($row['LastName'])))."'";
$email = "'".$conn->real_escape_string($row['Email'])."'";
$username = "'".$conn->real_escape_string($row['Username'])."'";
$display_name = "'".$conn->real_escape_string(
ucfirst(strtolower($row['FirstName'])).' '.ucfirst(strtolower($row['LastName']))
)."'";
// VALUES should be added only if row doesn't exists
if ($rows_returned === 0) {
// VALUES should be append until they reach 1000
while ($i % 1000 !== 0) {
$sqlInsert .= "VALUES($veeva_rep_id,$first,$last,$email,$username,NOW(),NOW(),$display_name,'VEEVA','https://pdone.s3.amazonaws.com/avatar/default_avatar.png',NOW(),NOW())";
++$i;;
}
// QUERY should be output to console to see if it's right or something is wrong
echo "DEBUG: ", $sqlInsert, "\n";
// QUERY should be executed if there are 1000 VALUES ready to add as a batch
/*$rs = $conn->query($sqlInsert);
if ($rs === false) {
echo 'Wrong SQL: '.$sqlInsert.' Error: '.$conn->error, E_USER_ERROR;*/
}
} else {
// UPDATE
echo "UPDATE";
}
}
}
But this line of code: echo "DEBUG: ", $sql, "\n"; is not outputting nothing to console. I must be doing something wrong but I can't find what. Can any help me to build the proper batch query and to execute it each 1000 values append?
Proper output should be:
DEBUG count(data): 1454
DEBUG: SELECT id,lastSyncAt FROM reps WHERE veeva_rep_id='00580000008ReolAAC'
DEBUG: SELECT id,lastSyncAt FROM reps WHERE veeva_rep_id='005800000039SIWAA2'
....
DEBUG: INSERT INTO reps(veeva_rep_id,first,last,email,username,lastLoginAt,lastSyncAt,display_name,rep_type,avatar_url,createdAt,updatedAt) VALUES(...), VALUES(...), VALUES(...)
Obtained result:
DEBUG count(data): 1454
DEBUG: SELECT id,lastSyncAt FROM reps WHERE veeva_rep_id='00580000008RGg6AAG'
DEBUG: INSERT INTO reps(veeva_rep_id,first,last,email,username,lastLoginAt,lastSyncAt,display_name,rep_type,avatar_url,createdAt,updatedAt)
DEBUG: SELECT id,lastSyncAt FROM reps WHERE veeva_rep_id='00580000008RQ4CAAW'
DEBUG: INSERT INTO reps(veeva_rep_id,first,last,email,username,lastLoginAt,lastSyncAt,display_name,rep_type,avatar_url,createdAt,updatedAt)
.... // until reach 1454 results
The table is empty so it should never goes through ELSE condition (UPDATE one).
EDIT
With help from the answer this is how the code looks now:
$data = convertCsvToArray($fileName);
echo "DEBUG count(data): ", count($data), "\n";
$i = 1;
$sqlInsert = "INSERT INTO reps(veeva_rep_id,first,last,email,username,lastLoginAt,lastSyncAt,display_name,rep_type,avatar_url,createdAt,updatedAt) VALUES";
foreach ($data as $row) {
$sql = "SELECT id,lastSyncAt FROM reps WHERE veeva_rep_id='{$row['Id']}'";
$rs = $conn->query($sql);
if ($rs === false) {
echo 'Wrong SQL: '.$sql.' Error: '.$conn->error, E_USER_ERROR;
} else {
$rows_returned = $rs->num_rows;
$veeva_rep_id = "'".$conn->real_escape_string($row['Id'])."'";
$first = "'".$conn->real_escape_string(ucfirst(strtolower($row['FirstName'])))."'";
$last = "'".$conn->real_escape_string(ucfirst(strtolower($row['LastName'])))."'";
$email = "'".$conn->real_escape_string($row['Email'])."'";
$username = "'".$conn->real_escape_string($row['Username'])."'";
$display_name = "'".$conn->real_escape_string(
ucfirst(strtolower($row['FirstName'])).' '.ucfirst(strtolower($row['LastName']))
)."'";
if ($rows_returned === 0) {
if ($i % 1000 === 0) {
file_put_contents("output.log", $sqlInsert."\n", FILE_APPEND);
$sqlInsert = "INSERT INTO reps(veeva_rep_id,first,last,email,username,lastLoginAt,lastSyncAt,display_name,rep_type,avatar_url,createdAt,updatedAt) VALUES";
} else {
$sqlInsert .= "($veeva_rep_id,$first,$last,$email,$username,NOW(),NOW(),$display_name,'VEEVA','https://pdone.s3.amazonaws.com/avatar/default_avatar.png',NOW(),NOW()), ";
}
$i++;
} else {
echo "UPDATE";
}
}
}
But still buggy because:
I have got a first empty INSERT query: INSERT INTO reps(veeva_rep_id,first,last,email,username,lastLoginAt,lastSyncAt,display_name,rep_type,avatar_url,createdAt,updatedAt) VALUES
I have got a second INSERT query with 1000 VALUES() append, but what happened with the rest? The remaining 454?
Can any give me another tip? Help?
Since it looks like you are trying to load data from a CSV file, you might want to consider using LOAD DATA INFILE functionality which is designed specifically for this purpose.
Here is link to documentation: https://dev.mysql.com/doc/refman/5.6/en/load-data.html
consider using INSERT IGNORE INTO table to check if the record already exists. How to 'insert if not exists' in MySQL?
if you haven't already done so, make veeva_rep_id a PRIMARY key so the INSERT IGNORE will work
also check out using PDO for transactions, prepared statements and dynamically generating queries using PDO
PDO Prepared Inserts multiple rows in single query
<?php
$sql = 'INSERT IGNORE INTO reps(veeva_rep_id,first,last,email,username,lastLoginAt,lastSyncAt,display_name,rep_type,avatar_url,createdAt,updatedAt) VALUES ';
$insertQuery = array();
$insertData = array();
/*
assuming the array from the csv is like this
$data = array(
0 => array('name' => 'Robert', 'value' => 'some value'),
1 => array('name' => 'Louise', 'value' => 'another value')
);
*/
foreach ($data as $row) {
$insertQuery[] = '(:veeva_rep_id' . $n . ', :first' . $n . ', :last' . $n . ', :email' . $n . ', :username' . $n . ', :lastLoginAt' . $n . ', :lastSyncAt' . $n . ', :display_name' . $n . ', :rep_type' . $n . ', :avatar_url' . $n . ', :createdAt' . $n . ', :updatedAt' . $n . ')';
$insertData['veeva_rep_id' . $n] = $row['name'];
$insertData['first' . $n] = $row['value'];
$insertData['last' . $n] = $row['name'];
$insertData['email' . $n] = $row['value'];
$insertData['username' . $n] = $row['name'];
$insertData['lastLoginAt' . $n] = $row['value'];
$insertData['lastSyncAt' . $n] = $row['value'];
$insertData['display_name' . $n] = $row['name'];
$insertData['rep_type' . $n] = $row['value'];
$insertData['avatar_url' . $n] = $row['value'];
$insertData['createdAt' . $n] = $row['name'];
$insertData['updatedAt' . $n] = $row['value'];
$n++;
}
$db->beginTransaction();
if (!empty($insertQuery) and count($insertQuery)>1000) {
$sql .= implode(', ', $insertQuery);
$stmt = $db->prepare($sql);
$stmt->execute($insertData);
}
$db->commit();
print $sql . PHP_EOL;
let me know if it helps.
You should have something like:
// Try fetching data from table 1
// If there is no record available, then fetch some data from table 2
// and insert that data inito table 1
You just wrote
$sql = "INSERT INTO reps(veeva_rep_id,first,last,email,username,lastLoginAt,lastSyncAt,display_name,rep_type,avatar_url,createdAt,updatedAt) ";
// Processing on each row of data
foreach ($data as $row) {
But from an insert no data is selected and second...you didn't run a select, where comes $data from?
update Use if ($i % 1000 === 0) { instead of while ($i % 1000 !== 0) {
$i = 0;
$sqlInsert = "INSERT INTO reps(veeva_rep_id,first,last,email,...) ";
// Processing on each row of data
foreach ($data as $row) {
$sql = "SELECT id,lastSyncAt FROM reps WHERE veeva_rep_id='{$row['Id']}'";
echo "DEBUG: ", $sql, "\n";
$rs = $conn->query($sql);
if ($rs === false) {
echo 'Wrong SQL: '.$sql.' Error: '.$conn->error, E_USER_ERROR;
} else {
$veeva_rep_id = ...;
$first = ...;
$last = ...;
$email = ...;
// ...
// VALUES should be added only if row doesn't exists
if($rs->num_rows == 0) {
// Insert some data
$i++;
if ($i % 1000 === 0) {
echo "DEBUG: ", $sqlInsert, "\n";
// execSql($sqlInsert);
$sqlInsert = "INSERT INTO reps(veeva_rep_id,first,last,email,...) "; // reset
} else {
$sqlInsert .= "VALUES($veeva_rep_id,$first,$last,$email,...) ";
}
} else {
echo "UPDATE";
}
}
}

array data to mysql

I need to parse the following code and process the resulting data.
foreach($job as $x=>$x_value)
{
echo "Key=" . $x . ", Value=" . $x_value;
echo "<br>";
}
The above code is returning the following as expected.
Key=vca_id, Value=20130<br>Key=uuid, Value=3c87e0b3-cfa<br>Key=originate_time, Value=2013-03-15 14:30:18<br>
What I need to do is to put the values in mysql database. So the insert statement would look something like this...
insert into test.master_table (vca_id, uuid, originate_time) values ('20130', '3c87e0b3-cfa', '2013-03-15 14:30:18')
What is the correct way to save the array values to mysql database?
<?php
mysql_query("insert into test.master_table(vca_id, uuid, originate_time)values('".$job['vca_id']."','".$job['uuid']."','".$job['originate_time']."')");
?>
Well i will recommend implode
$keys = array();
$values = array();
foreach($job as $x => $x_value)
{
$keys[] = $x;
$values[] = $x_value;
}
$query = 'INSERT INTO test.master_table' . '('.implode(',',$keys) .') VALUES (' .implode(',',$values) . ')';
You can try this
$temp_value_arr = array();
$query = "INSERT into test.master_table SET ";
foreach($job as $x=>$x_value)
{
$query .= "$x = '$x_value',";
}
$query = rtrim($query, ',');
mysql_query($query);

Categories