Automatically generate MySQL table after csv file upload - php

I am working on a PHP/MySQL project, it must verify the following tasks:
-> The user uploads multiple large CSV files at a time with the same column names (X,Y,Z) in MySQL tables
-> The web app must perform an arithmetic operation between each csv file's column
-> The user can download the csv files after the operation as Excel files
For the upload part, i need to find a way to auto generate a table in the database for each csv file uploaded -instead of creating it in advance-, because the user should be able to upload as many files as he wants.
i tried to set a while loop that contains a create table, the loop goes from 0 to $var which is the number of csv files the user wishes to upload, however it doesnt add any table, here's the code for that part :
$con= getdb();
$var=$_GET["quantity"];
mysql_query("set $i=0;
while $i<`".$var."` do
create table `couche".$var."` ( X float NOT NULL,Y float NOT NULL,Z float NOT NULL);
set $i= $i+1;
end while");
}

Hi you can use the following way to achieve it :
<?php
//database connection details
$link = mysqli_connect('localhost','root','password','db_name') or die('Could not connect to MySQL');
// path where your CSV file is located
define('CSV_PATH','/var/www/html/skerp_scripts/');
// Name of your CSV file
$csv_file = CSV_PATH . "importItems.csv";
if (($handle = fopen($csv_file, "r")) !== FALSE) {
$header = true; //If there is a header in first row then set it to true
while (($data = fgetcsv($handle, 100000, ",")) !== FALSE) {
if($header == true){
/* Here you can perform checks whether all the column are as expected
for Eg: in CSV1 : id, firstname, lastname, age
in CSV2 : firstname, age ,id
Than you can tell the user that there is a misatch
*/
$header = false;
continue;
}
$column1 = $data[0];
$column2 = $data[1];
$column3 = $data[2];
$calculation = $column1 * $column3;
$result = mysqli_query($link, "INSERT INTO table_name (column1, column2, column3)" VALUES($column1, $column2, $calculation));
}
}
echo "File data successfully imported to database!!";
mysqli_close($connect);

Related

Access denied for user 'dbtsorder'#'localhost' (using password: YES)

How to solve this message Access denied for user 'dbtsorder'#'localhost' (using password: YES) this message appear only online on localhost work fine
i try to load in file from csv file and this message appear only where this code is
database connection:
$host="localhost";
$user="dbtsorder";
$password="M#07765729s";
$db="tsorder";
$conn=mysqli_connect($host,$user,$password,$db);
mysqli_query($conn,"SET NAMES utf8");
mysqli_set_charset($conn,'utf8')
/*************************************/
Load in file code:
if(isset($_POST['submit'])){
$link = realpath('/var/www/order/projectmanagment/');
$checklist = $link.'checklist.csv';
$username=$_SESSION['username'];
$query=mysqli_query($conn,"select* from tbl_user where db_username='$username'")or die(mysqli_error($conn));
$res=mysqli_fetch_array($query);
$fname=$res['db_fname'];
$lname=$res['db_lname'];
$name=$res['db_fname'].' '.$res['db_lname'];
$projectname=$_POST['dep'];
$location=$_POST['cname'];
$psd=$_POST['txt_psd'];
$pdd=$_POST['txt_pdd'];
$past=$_POST['txt_past'];
$padd=$_POST['txt_padd'];
$duration=$_POST['duration'];
$aduration=$_POST['txt_aduration'];
$pnote=$_POST['txt_pnote'];
$transferredto=$_POST['txt_transferredto'];
$client=$_POST['txt_client'];
$cpercentage=$_POST['txt_cpercentage'];
$epercentage=$_POST['txt_epercentage'];
$mpercentage=$_POST['txt_mpercentage'];
$sum=$cpercentage+$epercentage+$mpercentage;
if($projectname=="" || $location=="" || $psd=="" || $pdd=="" || $client=="" ){
echo"Enter All Information.";
}
else{
if($sum==100){
$_SESSION['projectname']=$projectname;
$sql=mysqli_query($conn,"INSERT INTO tbl_project(db_projectname,db_location,db_transferredto,db_psd,db_pdd,db_duration,db_past,db_padd,db_aduration,db_pnote,db_user,db_client,db_cpercentage,db_epercentage,db_mpercentage)VALUES('$projectname','$location','$transferredto','$psd','$pdd','$duration','$past','$padd','$aduration','$pnote','$name','$client','$cpercentage','$epercentage','$mpercentage')")or die(mysqli_error($conn));
$import=mysqli_query($conn,"LOAD DATA INFILE '$checklist' INTO TABLE tbl_checklist FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '\"'
LINES TERMINATED BY '\n' set db_projectname='$projectname' ")or die(mysqli_error($conn));
header("location:allproject.php?msg=2");
}else{echo"Percentage should be equal to 100";}}
}
You can use this code to read csv file and insert data into database
$row=1;
if (($handle = fopen("test.csv", "r"))!== FALSE) {
while (($data = fgetcsv($handle, 1000, ",")) !== FALSE)
{
$num = count($data);
$row++;
if($row>2)
{
if($data[0]!="")
{
//write your inset query
}
}
}
fclose($handle);
}
user this code this will work for you,
$data[0] is the first column, for second use $data[1] etc.
So by looking into my Crystalball the issue seems to be with your Permissions.
In mySql you'll have the choice to whitelist certain clients, i suggest you take a read from http://dev.mysql.com/doc/refman/5.7/en/create-user.html to understand how your users work
The next step would be to read the following to learn more about the meaning of permissions http://dev.mysql.com/doc/refman/5.7/en/privileges-provided.html
What's the output of "SHOW GRANTS FOR dbtsorder"
Error is clearly showing that there is a connection problem, please check wheather dbtsorder user has access of tsorder or not. If not then first give this user access to particular database.

load data local infile user provided VALUES

My project requires external CSV files to be uploaded to a database every week and required outputs are extracted whenever required. Currently am using the below to browse through a HTML form and upload to the table. It works fine!
if(isset($_POST['submit']))
{
$file = $_FILES['file']['tmp_name'];
$handle = fopen($file,"r");
while(($fileop = fgetcsv($handle,1000,",")) !==false)
{
$placement_name = $fileop[2];
$statistics_date = $fileop[0];
$impressions = $fileop[5];
$clicks = $fileop[6];
$sql = mysql_query("INSERT INTO 'table'(source,advertiser,campaign,placement_name,statistics_date,impressions,clicks) VALUES ('xxx','yyy','zzz','$placement_name','$statistics_date','$impressions',' $clicks')");}
But now, we might need to do this on a daily basis and automated (which we hope to do with scheduled cron jobs), so decision was to use LOAD DATA LOCAL INFILE. As you could see the values inserted for source, advertiser and campaign are custom ones. How could we use the LOAD DATA... instead of the browse function?

PHP script export new entries to SQL table and overwrite file

I am trying to create a php script that will be run by a cron job on a nightly basis.
The database holds entries from a contact form.
The idea is to run the php script (via cron) and have it export any new entries from the database from that day into a csv file on the server, overwriting that file each time for housekeeping. It needs to only export the entries to the database from that day.
Here is what I have at the moment:
<?php
// Connect and query the database for the users
$conn = new PDO("mysql:host=localhost;dbname=dbname", 'username', 'password');
$sql = "SELECT * FROM enquiries ORDER BY firstname";
$results = $conn->query($sql);
// Pick a filename and destination directory for the file
// Remember that the folder where you want to write the file has to be writable
//$filename = "/tmp/db_user_export_".time().".csv";
$filename = "/home/domain/public_html/csv/db_user_export.csv";
// Actually create the file
// The w+ parameter will wipe out and overwrite any existing file with the same name
$handle = fopen($filename, 'wb');
// Write the spreadsheet column titles / labels
fputcsv($handle, array('FirstName','Email'));
// Write all the user records to the spreadsheet
foreach($results as $row)
{
fputcsv($handle, array($row['firstname'], $row['email']));
}
// Finish writing the file
fclose($handle);
?>
Your sql must by:
SELECT * FROM enquiries where DATE(DateField) = CURDATE() ORDER BY firstname

csv into mysql by php

Hello i'm trying to import data from a .csv file into mysql table. Below is the script i'm working with. After running it, only print_r($_FILES) was executed, and it didnt insert into the data base.
<?php session_start(); ?>
<?php require('includes/dbconnect.php'); ?>
<?php require 'includes/header.inc.php'; ?>
<?php
if(isset($_POST['SUBMIT']))
{
$fname = $_FILES['csv_file']['name']; //Acquire the name of the file
$chk_ext = explode(".",$fname);
$filename = $_FILES['csv_file']['tmp_name'];
$handle = fopen($filename, "r"); //Open the file for readability
while (($data = fgetcsv($handle,1000, ",")) !== FALSE)
{
$sql = "INSERT into biodata (student_number, fname, lname, level) values('$data[0]','$data[1]','$data[2]')";
mysql_query($sql) or die(mysql_error());
}
fclose($handle);
echo "Successfully Imported";
}
else
{
echo "Invalid File";
}
print_r($_FILES) ;
?>
Your query has problem.
Its expecting 4 columns (as you specified in column list) but you supllied only 3 columns.
$sql = "INSERT into biodata (student_number, fname, lname, level) values('$data[0]','$data[1]','$data[2]')";
First of all check whether file was opened successfully:
$handle = fopen($filename, "r");
if( !$handle){
die( 'Cannot open file fore reading');
}
That's actually only place you're not handling correctly (hope you have error reporting turned on, because this could only be problem with fgetcsv() and error report would be crucial).
Once you've worked out how to access the uploaded file, you could also look into using LOAD DATA INFILE. If you use the LOCAL keyword (LOAD DATA LOCAL INFILE), that should work even on shared hosting. For some examples, see: http://dev.mysql.com/doc/refman/5.0/en/load-data.html
This has the benefit of being much, much faster than large numbers of INSERTs, which is especially relevant for large CSV files. Plus, it's quite easy to use.

What is the best way to count page views in PHP/MySQL?

And by best I mean most efficient, right now placing this on my post.php file is the only thing I can think of:
$query = mysql_query(" UPDATE posts SET views + 1 WHERE id = '$id' ");
is there a better way, a method that would consume less server resources. I ask because if this was a small app I would have no problem with the above, but I am trying to build something that will be used by a lot of people and I want to be as query conscious as possible.
If you're interested in conserving resources and still using SQL for reporting, and precise # doesn't matter, you could try sampling like this (modify sample rate to suit your scale):
$sample_rate = 100;
if(mt_rand(1,$sample_rate) == 1) {
$query = mysql_query(" UPDATE posts SET views = views + {$sample_rate} WHERE id = '{$id}' ");
// execute query, etc
}
If memcache is an option in your server environment, here's another cool way to sample, but also keep up with the precise number (unlike my other answer):
function recordPostPageView($page_id) {
$memcache = new Memcached(); // you could also pull this instance from somewhere else, if you want a bit more efficiency*
$key = "Counter for Post {$page_id}";
if(!$memcache->get($key)) {
$memcache->set($key, 0);
}
$new_count = $memcache->increment($key);
// you could uncomment the following if you still want to notify mysql of the value occasionally
/*
$notify_mysql_interval = 100;
if($new_count % $notify_mysql_interval == 0) {
$query = mysql_query("UPDATE posts SET views = {$new_count} WHERE id = '{$page_id}' ");
// execute query, etc
}
*/
return $new_count;
}
And don't mind purists crying foul about Singletons. Or you could pass it into this function, if you're more purist than pragmatist :)
IMHO best solution is to have views_count stored inside memory (memcached, whatever),
and do updates in memory. (Of course updates have to be synchronized)
Then you can use cron script which will push those values to db. (after some time - seconds, minutes, whatever.)
in the database there is only one column ip with primary key defined and then store ip in database using PHP code below:
Connection file :
<?php
$conn = mysqli_connect("localhost","root","");
if (!$conn) {
die("Connection failed: " . mysqli_connect_error());
}
$db=mysqli_select_db($conn,"DB_NAME");
if(!$db)
{
echo "Connection failed";
}
?>
PHP file:
<?php
$ip=$_SERVER['REMOTE_ADDR'];
$insert="INSERT INTO `id928751_photography`.`ip` (`ip`)VALUES ('$ip');";
$result = mysqli_query($conn,$insert);
?>
show count :
<?php
$select="SELECT COUNT(ip) as count from ip;";
$run= mysqli_query($conn,$select);
$res=mysqli_fetch_array($run);
echo $res['count'];
?>
using this method in the database store all server ip
NOTE: only server ip can store or count not device ip
You could keep a counter-array in cache (like APC or Memcache) and increase the counter for certain posts in that. Then store the updates once a while. You might loose some views if a cache-reset occures
Other solution would be to keep a separate table for visits only (Field: postid, visits). That is the fasters you can get from mysql. Try to use InnoDB engine, since it provides row-level-locking!
You can also check these lines of code. I think it will be helpful because you can achieve your goal with just a text file. It does not require any database activity.
<?php
session_start();
$counter_name = "counter.txt";
// Check if a text file exists. If not create one and initialize it to zero.
if (!file_exists($counter_name)) {
$f = fopen($counter_name, "w");
fwrite($f,"0");
fclose($f);
}
// Read the current value of our counter file
$f = fopen($counter_name,"r");
$counterVal = fread($f, filesize($counter_name));
fclose($f);
// Has visitor been counted in this session?
// If not, increase counter value by one
if(!isset($_SESSION['hasVisited'])){
$_SESSION['hasVisited']="yes";
$counterVal++;
$f = fopen($counter_name, "w");
fwrite($f, $counterVal);
fclose($f);
}
echo "You are visitor number $counterVal to this site";
This way show how many actual people viewed your website not just how many times they viewed your website.
Step1: Connecting to MySQL
dbconfig.php
try
{
// Returns DB instance or create initial connection
$pdo = new PDO("mysql:host={$DB_host};port={$DB_port};dbname={$DB_name};charset=utf8mb4",$DB_user,$DB_pass);
$pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
}
catch(PDOException $e)
{
echo $e->getMessage();
}
Step2: Creating MySQL table
--
-- Table structure for table `unique_visitors`
--
CREATE TABLE `unique_visitors` (
`date` date NOT NULL,
`ip` text COLLATE utf8_unicode_ci NOT NULL,
`views` int(1) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
Step3: Create a visitor counter by using IP address.
<?php
require_once("dbconfig.php");
// Returns current date in YYYY-MM-DD format
$date = date("Y-m-d");
// Stores remote user ip address
$userIP = $_SERVER['REMOTE_ADDR'];
// Query for selecting record of current date from the table
$stmt = $pdo->prepare("SELECT * FROM unique_visitors WHERE date=:date");
$stmt->execute(['date' => $date]);
if(count($stmt->fetchAll()) === 0){
// Block will execute when there is no record of current date in the database table
$data = [
'date' => $date,
'ip' => $userIP,
];
// SQL query for inserting new record into the database table with current date and user IP address
$sql = "INSERT INTO unique_visitors (date, ip) VALUES (:date, :ip)";
$pdo->prepare($sql)->execute($data);
}else{
$row = $stmt->fetchAll(PDO::FETCH_ASSOC);
// Will execute when current IP is not in database
if(!preg_match('/'.$userIP.'/i',$row['ip'])){
// Combines previous and current user IP address with a separator for updating in the database
$newIP = "$row[ip] $userIP";
$data = [
'ip' => $newIP,
'date' => $date,
];
$sql = "UPDATE unique_visitors SET ip=:ip, views=views+1 WHERE date=:date";
$pdo->prepare($sql)->execute($data);
}
}
?>
<?php
session_start();
$counter_name = "counter.txt";
// Check if a text file exists. If not create one and initialize it to zero.
if (!file_exists($counter_name)) {
$f = fopen($counter_name, "w");
fwrite($f,"0");
fclose($f);
}
// Read the current value of our counter file
$f = fopen($counter_name,"r");
$counterVal = fread($f, filesize($counter_name));
fclose($f);
// Has visitor been counted in this session?
// If not, increase counter value by one
if(!isset($_SESSION['hasVisited'])){
$_SESSION['hasVisited']="yes";
$counterVal++;
$f = fopen($counter_name, "w");
fwrite($f, $counterVal);
fclose($f);
}
echo "You are visitor number $counterVal to this site";

Categories