import csv file from external FTP Site - php

I have the below script to import data from a csv file on my server, the script works correctly.
I need to change the location of the file from my server to an FTP server which requires authentication. The csv file name on the FTP server will cahnge according to the time stamp it was generated. After importing into the MySQL database, the file on the FTP server needs to be deleted.
I then need to schedule this job with cron jobs to run every 5 minutes.
Any assistance will be appreciated.
Thanks.
<?php
/********************************/
/* Code at http://legend.ws/blog/tips-tricks/csv-php-mysql-import/
/* Edit the entries below to reflect the appropriate values
/********************************/
$databasehost = "localhost";
$databasename = "dbname";
$databasetable = "dbtable";
$databaseusername ="username";
$databasepassword = "password";
$fieldseparator = ",";
$lineseparator = "\n";
$csvfile = "filenamewithtimestamp.csv";
/********************************/
/* Would you like to add an empty field at the beginning of these records?
/* This is useful if you have a table with the first field being an auto_increment integer
/* and the csv file does not have such as empty field before the records.
/* Set 1 for yes and 0 for no. ATTENTION: don't set to 1 if you are not sure.
/* This can dump data in the wrong fields if this extra field does not exist in the table
/********************************/
$addauto = 0;
/********************************/
/* Would you like to save the mysql queries in a file? If yes set $save to 1.
/* Permission on the file should be set to 777. Either upload a sample file through ftp and
/* change the permissions, or execute at the prompt: touch output.sql && chmod 777 output.sql
/********************************/
$save = 0;
$outputfile = "output.sql";
/********************************/
if(!file_exists($csvfile)) {
echo "File not found. Make sure you specified the correct path.\n";
exit;
}
$file = fopen($csvfile,"r");
if(!$file) {
echo "Error opening data file.\n";
exit;
}
$size = filesize($csvfile);
if(!$size) {
echo "File is empty.\n";
exit;
}
$csvcontent = fread($file,$size);
fclose($file);
$con = #mysql_connect($databasehost,$databaseusername,$databasepassword) or die(mysql_error());
#mysql_select_db($databasename) or die(mysql_error());
$lines = 0;
$queries = "";
$linearray = array();
foreach(split($lineseparator,$csvcontent) as $line) {
$lines++;
$line = trim($line," \t");
$line = str_replace("\r","",$line);
/************************************
This line escapes the special character. remove it if entries are already escaped in the csv file
************************************/
$line = str_replace("'","\'",$line);
/*************************************/
$linearray = explode($fieldseparator,$line);
$linemysql = implode("','",$linearray);
if($addauto)
$query = "insert into $databasetable values('','$linemysql');";
else
$query = "insert into $databasetable values('$linemysql');";
$queries .= $query . "\n";
#mysql_query($query);
}
#mysql_close($con);
if($save) {
if(!is_writable($outputfile)) {
echo "File is not writable, check permissions.\n";
}
else {
$file2 = fopen($outputfile,"w");
if(!$file2) {
echo "Error writing to the output file.\n";
}
else {
fwrite($file2,$queries);
fclose($file2);
}
}
}
echo "Found a total of $lines records in this csv file.\n All records imported";
?>

I thought I would just post the code that solved my issue; it downloads a csv file from an external FTP site onto my local server. It then imports the csv file into my local mysql database table. Once imported the file on the ftp server is deleted. I schedule this job in cpanel with cronjob.
I cannot take credit for the script, I have just modified a script found on another site. I'm sure there are better ways of doing this with some error checking. However, this is the best I could do with my limited skills.
I hope this assists future visitors with the same issue. Thanks to all those that assisted me.
Here is the code I used:
<?php
$source = "DespGoods.csv";
$target = fopen("DespGoods.csv", "w");
$conn = ftp_connect("ftp.server.com") or die("Could not connect");
ftp_login($conn,"ftpusername","ftppassword");
ftp_fget($conn,$target,$source,FTP_ASCII);
echo "file downloaded.\n";
/********************************/
/* Code at http://legend.ws/blog/tips-tricks/csv-php-mysql-import/
/* Edit the entries below to reflect the appropriate values
/********************************/
$dbhost = "localhost";
$dbname = "dbname";
$dbtable = "despgoods";
$dbusername ="dbusername";
$dbpassword = "dbpassword";
$fieldseparator = ",";
$lineseparator = "\n";
$csvfile = "DespGoods.csv";
/********************************/
/* Would you like to add an ampty field at the beginning of these records?
/* This is useful if you have a table with the first field being an auto_increment
/* integer and the csv file does not have such as empty field before the records.
/* Set 1 for yes and 0 for no. ATTENTION: don't set to 1 if you are not sure.
/* This can dump data in the wrong fields if this extra field does not
/* exist in the table.
/********************************/
$addauto = 0;
/********************************/
/* Would you like to save the mysql queries in a file? If yes set $save to 1.
/* Permission on the file should be set to 777. Either upload a sample file
/* through ftp and change the permissions, or execute at the prompt:
/* touch output.sql && chmod 777 output.sql
/********************************/
$save = 0;
$outputfile = "output.sql";
/********************************/
if(!file_exists($csvfile)) {
echo "File not found. Make sure you specified the correct path.\n";
exit;
}
$file = fopen($csvfile,"r");
if(!$file) {
echo "Error opening data file.\n";
exit;
}
$size = filesize($csvfile);
if(!$size) {
echo "File is empty.\n";
exit;
}
$csvcontent = fread($file,$size);
fclose($file);
$con = #mysql_connect($dbhost,$dbusername,$dbpassword) or die(mysql_error());
#mysql_select_db($dbname) or die(mysql_error());
$lines = 0;
$queries = "";
$linearray = array();
foreach(split($lineseparator,$csvcontent) as $line) {
$lines++;
$line = trim($line," \t");
$line = str_replace("\r","",$line);
/************************************
/* This line escapes the special character.
/* Remove it if entries are already escaped in the csv file
/************************************/
$line = str_replace("'","\'",$line);
/*************************************/
$linearray = explode($fieldseparator,$line);
$linemysql = implode("','",$linearray);
if($addauto) {
$query = "insert into $dbtable values('','$linemysql');";
}
else {
$query = "insert into $dbtable values('$linemysql');";
}
$queries .= $query . "\n";
#mysql_query($query);
}
#mysql_close($con);
if($save) {
if(!is_writable($outputfile)) {
echo "File is not writable, check permissions.\n";
}
else {
$file2 = fopen($outputfile,"w");
if(!$file2) {
echo "Error writing to the output file.\n";
}
else {
fwrite($file2,$queries);
fclose($file2);
}
}
}
echo "Found a total of $lines records in this csv file.\n";
$source = "DespGoods.csv";
$target = fopen("DespGoods.csv", "w");
$conn = ftp_connect("ftp.server") or die("Could not connect");
ftp_login($conn,"ftpusername","ftppassword");
ftp_delete($conn,$source);
ftp_close($conn);
echo "file deleted";
mysql_close($con);
?>

Is this what you're looking for?
http://www.php.net/manual/en/ftp.examples-basic.php

Related

How to get an array of all rows in column A in an Excel file using PHPExel?

I have a list on an Excel file of phone numbers (A row).
How do I get it as an array using PHPExel?
Can I get a whole example including a small explanation of which file from the 'Class' directory I should 'include_once'? How do I know which .php file to include? How do I scan the list?
<?php
/************************ YOUR DATABASE CONNECTION START HERE ****************************/
define ("DB_HOST", "localhost"); // set database host
define ("DB_USER", "root"); // set database user
define ("DB_PASS",""); // set database password
define ("DB_NAME","database Name here"); // set database name
$link = mysql_connect(DB_HOST, DB_USER, DB_PASS) or die("Couldn't make connection.");
$db = mysql_select_db(DB_NAME, $link) or die("Couldn't select database");
$databasetable = ""; // your table name
/************************ YOUR DATABASE CONNECTION END HERE ****************************/
set_include_path(get_include_path() . PATH_SEPARATOR . 'Classes/');
include 'PHPExcel-develop\Classes\PHPExcel\IOFactory.php';
$targetfolder = "";
$targetfolder = $targetfolder . basename( $_FILES['fileToUpload']['name']) ;
if(move_uploaded_file($_FILES['fileToUpload']['tmp_name'], $targetfolder))
{
echo "The file ". basename( $_FILES['fileToUpload']['name']). " is uploaded";
}
else
{
echo "Problem uploading file";
print $targetfolder . basename( $_FILES['fileToUpload']['name']) ;
}
$inputFileName = basename( $_FILES['fileToUpload']['name']);
try
{
$objPHPExcel = PHPExcel_IOFactory::load($inputFileName);
}
catch(Exception $e)
{
die('Error loading file "'.pathinfo($inputFileName,PATHINFO_BASENAME).'": '.$e->getMessage());
}
$allDataInSheet = $objPHPExcel->getActiveSheet()->toArray(null,true,true,true);
$arrayCount = count($allDataInSheet); // Here get total count of row in that Excel sheet
for($i=2;$i<=$arrayCount;$i++)
{
$value1 = trim($allDataInSheet[$i]["A"]);
$Value2 = trim($allDataInSheet[$i]["B"]);
$value3 = trim($allDataInSheet[$i]["C"]);
$query = "SELECT * FROM "; // your select query
$sql = mysql_query($query);
$recResult = mysql_fetch_array($sql);
$exist = $recResult["$value1"];
if($exist=="")
{
$insertTable= mysql_query("//your insert query");
$msg = 'Record has been added.</div>';
}
else
{
$msg = 'Record already exist. </div>';
}
}
?>

Content appended twice in the CSV file when CSV is generated using PHP button

This is my code to generate csv file.When I click php button to generate Csv file,which is filled withthe contents based on the category column from the database.But my problem here is when the contents are getting populated twice in the csv file as shown below.Please help to out where i have to modify the code so that i can get only one time populated content as shown below as expected.Thanks in advance.
createcsv.php
<?php
$servername = "localhost";
$username = "user";
$password = "";
$dbname = "stats";
define("DB_SERVER", "localhost");
define("DB_NAME", "stats");
define("DB_USER", "user");
define("DB_PASSWORD", '');
$dbconn = #mysql_connect(DB_SERVER, DB_USER, DB_PASSWORD);
$conn = #mysql_select_db(DB_NAME,$dbconn);
// Create connection
//$conn = new mysqli($servername, $username, $password, $dbname);
// Check connection
if ($conn->connect_error) {
die("Connection failed: " . $conn->connect_error);
echo "DB connection failed";
}
// Query DB to fetch hit count for each category and in turn create corresponding .csv file
function createCSVFile($type) {
$msql = "SELECT TRIM(TRAILING '.000000' from UNIX_TIMESTAMP(hitdate)*1000) as unixdate,count from h_stats where category='".$type."' order by unixdate asc";
$query = mysql_query($msql);
$type = str_replace(' ', '', $type);
$tmp_file = "data/tmp_".$type.".csv";
$fp = fopen("$tmp_file", "w");
// Write the query contents to temp file
while($row = mysql_fetch_array($query))
{
fputcsv($fp, $row);
}
fclose($fp);
// Modify the contents of the file as per the high chart input data format
$fp = fopen("$tmp_file", 'r+');
rewind($fp);
$file = "data/".$type.".csv";
$final = fopen("$file", 'w');
while($line = fgets($fp)){
trim($line);
$line = '['.$line.'],';
fputs($final,$line);
}
// Append var $type and remove the trailing ,
$final = file_get_contents($file);
$content = 'var '.$type .'= [' . rtrim($final, ","). ']';
file_put_contents("$file",$content);
}
// Query DB to fetch success/failure count for Hits and in turn create corresponding .csv file
function createHitOutcomeCSVFile($type,$category) {
$sql = "SELECT TRIM(TRAILING '.000000' from UNIX_TIMESTAMP(hitdate)*1000) as unixdate,".$type." from h_stats where category='".$category."' order by unixdate asc";
$query = mysql_query($sql);
$tmp_file = "data/tmp_".$type."_".$category.".csv";
$fp = fopen("$tmp_file", "w");
// Write the query contents to temp file
while($row = mysql_fetch_array($query)){
fputcsv($fp, $row);
}
fclose($fp);
// Modify the contents of the file as per the high chart input data format
$fp = fopen("$tmp_file", 'r+');
rewind($fp);
$category = str_replace(' ', '', $category);
$file = "data/".$type."_".$category.".csv";
$final = fopen("$file", 'w');
while($line = fgets($fp)){
trim($line);
$line = '['.$line.'],';
fputs($final,$line);
}
// Append var $type and remove the trailing ,
$final = file_get_contents($file);
$content = 'var '.$type.'_'.$category.'= [' . rtrim($final, ","). ']';
file_put_contents("$file",$content);
}
// Invoke function to create the Hits.csv file
createCSVFile('Hits');
// Invoke function to get Three Hits csv file
createHitOutcomeCSVFile('TCount','Hits');
// Invoke function to get O2 Hits csv file
createHitOutcomeCSVFile('BCount','Login');
echo "Generated successfully";
?>
not expected csv file with twice populated data:
var Login_Hits= [[1427826600000,1427826600000,8763,8763
]]
Expected csv file as per highcharts format:
var Login_Hits= [[1427826600000,8763
]]
Try to debug it...
it will be easier than seeing typo or so...
it looks like the tmp file is already corrupted...
try to display the $row variable and the $query...
the problem may come from here...
In while loop I have used mysql_fetch_assoc instead of mysql_fetch_array at both the functions
while($row = mysql_fetch_assoc($query))
{
fputcsv($fp, $row);
}
The content is not repeating twice in the Csv file.This works try it!

Writing a CSV File to SQL

Hope someone can help me with what I think will be something minor (I'm still learning...). I'm trying to write the entire contents of a CSV File server based to an SQL database here is the code I presently have. The line // out writes perfectly and generates a new record. The $ar0 values generate no entries into the table named order - even though the csv file is about 100 lines long I just get
Error: INSERT INTO order (Picker,Order_Number,Timestamp,System)values ('','','','')
$file = "Pal.ORD.csv";
$tbl = "order";
$f_pointer=fopen("$file","r"); // file pointer
while(! feof($f_pointer)){
$ar=fgetcsv($f_pointer);
//$sql="INSERT INTO `order` (Picker,Order_Number,Timestamp,System)values ('Me','9999','23-01-2015','ORD')";
$sql="INSERT INTO `order` (Picker,Order_Number,Timestamp,System)values ('$ar[0]','$ar[1]','$ar[2]','$ar[3]')";
echo $sql;
echo "<br>";
}
if ($connect->query($sql) === TRUE) {
echo "New records created successfully";
} else {
echo "Error: " . $sql . "<br>" . $conn->error;
}
What I think may be going on is that your file probably has an empty line/carriage return as the last line in the file and is using that to insert the data as blank entries.
I can't be 100% sure about this since you have not provided a sample of your CSV file, however that is what my tests revealed.
Based on the following CSV test model: (Sidenote: blank lines will be ignored)
a1,a2,a3,a4
b1,b2,b3,b4
c1,c2,c3,c4
Use the following and replace with your own credentials.
This will create a new entry/row for each line found in a given file based on the model I have provide above.
<?php
$DB_HOST = 'xxx';
$DB_USER = 'xxx';
$DB_PASS = 'xxx';
$DB_NAME = 'xxx';
$db = new mysqli($DB_HOST, $DB_USER, $DB_PASS, $DB_NAME);
if($db->connect_errno > 0) {
die('Connection failed [' . $db->connect_error . ']');
}
$file = "Pal.ORD.csv";
$delimiter = ',';
if (($handle = fopen("$file", "r")) !== FALSE) {
while (($data = fgetcsv($handle, 1000, $delimiter)) !== FALSE) {
foreach($data as $i => $content) {
$data[$i] = $db->real_escape_string($content);
}
// echo $data[$i].""; // test only not required
$db->query("INSERT INTO `order`
(Picker, Order_Number, Timestamp, System)
VALUES ('" . implode("','", $data) . "');");
}
fclose($handle);
}
if($db){
echo "Success";
}
else {
echo "Error: " . $db->error;
}
At a quick glance it seems like this:
$f_pointer=fopen("$file","r"); // file pointer
Should be this:
$f_pointer=fopen($file,"r"); // file pointer
You might not be reading anything from the file. You can try outputting the file contents to see if that part is working, since you've confirmed that you can insert into the DB.

Search through multiple directories for files PHP

New to php. I need to search through multiple directories on an ftp server and download files contained in them according to their date (which is contained in each file's name). So far I am running into issues with Array to string conversion - hopefully you can see what I am "trying" to do here and might be able to help me out.
<?php
$url = ' MY FTP SERVER';
$user = ' USERNAME ';
$pass = ' PASSWORD';
$target = 'LOCAL DIRECTORY';
$connection = ftp_connect($url) or die ("Could not connect to $url");
$loginBool = ftp_login($connection, $user, $pass);
if ($loginBool)
{
echo "Connected as $user at $url";
}else{
echo "Could not connect as $user";
}
ftp_chdir($connection, 'MOVE ONE FOLDER IN');
$ftpDirectories = ftp_nlist($connection, ".");
for($i=0; $i < (count($ftpDirectories)); $i++){
$directory = ("/MAIN DIR/$ftpDirectories[$i]/");
echo $directory;
ftp_chdir($connection, $directory);
echo ftp_nlist($connection, $directory);
}
?>
The logic, as I see it, works as follow:
Get a listing of all folders under the main folder which is stored in Array
Loop through each element of the array(folders on the ftp)and change into the directory
one by one
Print out the files (so I know its working)
What I still need to add:
After changing into a directory split the filenames into an array and compare them to
today's date -> download if the filenameDate[index] == today's date
THANK YOU FOR YOUR HELP!
EDIT
<?php
$url = 'pmike86.zxq.net';
$user = 'pmike86_zxq';
$pass = '******';
$target = 'C:\Folder\\';
$connection = ftp_connect($url) or die ("Could not connect to $url");
$loginBool = ftp_login($connection, $user, $pass);
if ($loginBool)
{
echo "Connected as $user at $url";
}else{
echo "Could not connect as $user";
}
ftp_chdir($connection, 'HW');
$ftpDirectories = ftp_nlist($connection, ".");
for($i=0; $i < (count($ftpDirectories)); $i++){
$directory = ("/HW/$ftpDirectories[$i]/");
echo $directory;
ftp_chdir($connection, $directory);
foreach (ftp_nlist($connection, $directory) as $item)
{
ftp_get($connection, $target, $item, FTP_BINARY);
}
}
?>

saving mysql query as csv file on remote ftp server using php

I am trying to export a MySQL query result to a remote ftp server.
I have the below code but I am currently getting an error:
Warning: ftp_put() [function.ftp-put]: Opening ASCII mode data connection. in /home/hulamyxr/public_html/kisv2/xmltest/export.php on line 50
I would think that my $file = $csv_filename; might be the issue as this is fetching the csv file that has just been created on my local server?
any ideas?
My syntax is:
<?php
$host = 'localhost';
$user = 'un';
$pass = 'pwd';
$db = 'dbname';
$table = 'v2ReportingTable';
$file = 'export';
$datetime=date("Y-m-d H:i:s");
$link = mysql_connect($host, $user, $pass) or die("Can not connect." . mysql_error());
//Create a CSV for
$result = mysql_query("SELECT * FROM dbname.v2ReportingTable");
if (!$result) die('Couldn\'t fetch records');
$num_fields = mysql_num_fields($result);
$headers = array();
for ($i = 0; $i < $num_fields; $i++)
{
$headers[] = mysql_field_name($result , $i);
}
$csv_filename = "export-" .$datetime.".csv";
$fp = fopen($csv_filename, 'w+');
if ($fp && $result)
{
fputcsv($fp, $headers);
while ($row = mysql_fetch_row($result))
{
fputcsv($fp, array_values($row));
}
}
//works till here
$ftp_server = "ftp.server.co.za";
$ftp_user_name = "un";
$ftp_user_pass = "pw";
$file = $csv_filename;
$remote_file = "/LocExports/".$file;
// set up basic connection
$conn_id = ftp_connect($ftp_server);
// login with username and password
$login_result = ftp_login($conn_id, $ftp_user_name, $ftp_user_pass);
// upload a file
if (ftp_put($conn_id, $remote_file, $file, FTP_ASCII)) {
echo "successfully uploaded $file\n";
} else {
echo "There was a problem while uploading $file\n";
}
// close the connection
ftp_close($conn_id);
?>
Thanks Again
and it creates the file in the correct location but is a 0kb file and all FTP commands thereafter fail. It is likely that the client is behind a firewall. To rectify this use:
<?php
ftp_pasv($resource, true);
?>

Categories