PHP read csv and load it into postgres with PDO - php

I have .csv file with 9k rows. When I use script from php manual :
<?php
$row = 1;
if (($handle = fopen("test.csv", "r")) !== FALSE) {
while (($data = fgetcsv($handle, 1000, ";")) !== FALSE) {
$num = count($data);
echo "<p> $num fields in line $row: <br /></p>\n";
$row++;
for ($c=0; $c < $num; $c++) {
echo $data[$c] . "<br />\n";
}
}
fclose($handle);
}
?>
I got on my output everysingle cell from the csv file.
Problem occurs when I want to treat every row as record create object from it and store it into database. I have modified it like:
if (($handle = fopen("test.csv", "r")) !== FALSE) {
while (($data = fgetcsv($handle, 1000, ";")) !== FALSE) {
$num = count($data);
$record= new Record($db);
echo "<p> $num fields in line $row: <br /></p>\n";
$row++;
for ($c=0; $c < $num; $c++) {
$record->$attributes[$c] = utf8_encode($data[$c]);
}
$record->Store();
}
fclose($handle);
}
In this case only about 2k records are store into mine PostgreSQL database, but no exception or error is shown. I have no idea why my loop stops. Everytime i run the script it loads into mine database different amount of records between 1,6k and 2,1k.
I have no limits set on my PostgreSQL (At least i don't knew of any..)
Can anyone explain me what am I doing wrong?

Two Things here I will change
A. change the !== FALSE) to !=false), don't ask me why , it sometimes work
B. do NOT NEW you record instance in the loop without unset it , you are creating 2000 new instance so that may be the reason.

You could try the copy statement. I belive it is similar to MySql LOAD DATA IN FILE
http://www.postgresql.org/docs/current/static/sql-copy.html
If it's just the data you need to add, it would be considerably faster that any other method.

Related

php fgetcsv can not get scientific notation value as it show in formula area

I hava a csv file with data like this:
enter image description here
My code to read this csv file is:
<?php
$row = 1;
if (($handle = fopen("test.csv", "r")) !== FALSE) {
while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) {
$num = count($data);
echo "<p> $num fields in line $row: <br /></p>\n";
$row++;
for ($c=0; $c < $num; $c++) {
echo $data[$c] . "<br />\n";
}
}
fclose($handle);
}
It outputs:
1 fields in line 1:
1.19617E+11
1 fields in line 2:
1.13953E+11
1 fields in line 3:
1.19501E+11
But what I expect is actual value in csv(value display in formula area):
119616896266
113953402648
119501389455
Can anyone help to solve this? Thank you!
(I came across this problem when I using phpspreadsheet, and I found it use fgetcsv to read file.)

rewrite content in excel file column php

I need to read an excel file in my codeigniter project. This is my example file.
I need to rewrite dynamic content(eg %VAR1 and %VAR2 ) inmsg field with the content in corresponding column.
ie The msg content in first row will be Dear (John) this I sa demo message. 8952 test.
The number of dynamic content may vary from 0 to n.(%VAR1, %VAR2, %VAR3 etc).
Please help me to do this
This is my code:
$filename=$_FILES["contactFile"]["tmp_name"];
if (($handle = fopen($filename, "r")) !== FALSE) {
while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) {
$num = count($data);
echo "<p> $num fields in line $row: <br /></p>\n";
$row++;
for ($c=0; $c < $num; $c++) {
echo $data[$c] . "<br />\n";
}
}
fclose($handle);
}
it displays content in each column. But how do i replace with another column value

Scan a csv file for certain value with php

I'm just wondering is it possible to have a csv file (with many values) and scan the file for a specific value with php?
Ideally if this is possible, Id also like to get the index of that value in the file (i.e what row number the value falls on).
I'm quiet new to php so I'm not sure where to start.
All help is greatly appreciated.
Sure,
Not tested:
<?php
$search = 'mysearchstring';
$row = 1;
if (($handle = fopen("test.csv", "r")) !== FALSE)
{
while (($data = fgetcsv($handle, 1000, ",")) !== FALSE)
{
$num = count($data);
$row++;
for ($c=0; $c < $num; $c++)
{
if ($data[$c] == $search)
{
echo "Found in row $row in column {$c+1} <br />\n";
}
}
}
fclose($handle);
}
?>

how to extend the length of time when fgetcsv function of php is reading csv file?

am having a problem here, i read a csv file using fgetcsv function then loop it, and write it into an xml file...when am using the script in a not so big data, everything works perfectly and writes the xml file....but when i use it in a huge csv, the xml can't finished being written and end up in an unclosed token thing
if (($handle = fopen("C:\\xampp\htdocs\yii\branch\seo_tagsdev.csv", "r")) !== FALSE) {
while (($data = fgetcsv($handle, 10000000, ",")) !== FALSE) {
$num = count($data);
// echo "<p> $num fields in line $row: <br /></p>\n";
$row++;
for ($c=0; $c < $num; $c++)
{
$url = $doc->createElement('url');
$root->appendChild($url);
$loc = $doc->createElement('loc',htmlentities($data[$c]));
$url->appendChild($loc);
$lastmod = $doc->createElement('lastmod',date('Y-m-d'));
$url->appendChild($lastmod);
$freq = $doc->createElement('changefreq','daily');
$url->appendChild($freq);
$priority = $doc->createElement('priority',0.9);
$url->appendChild($priority);
// echo $data[$c] . "<br />\n";
}
$doc->save("C:\\xampp\htdocs\yii\branch\seo_tagsdev.xml");
}
i even adjusted the length parameter of fgetcsv and yet it doesn't work at all

Tools / Scripts for importing csv to db table

In tried importing a csv into a table using phpMyAdmin and I'm getting permission errors so I'm looking for any php scripts available around that I can use.
What scripts are available? Can ane point to any free one's?
fgetcsv is the php function you should use.
For details see here.
Sample code here..
$row = 1;
if (($handle = fopen("test.csv", "r")) !== FALSE) {
while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) {
$num = count($data);
echo "<p> $num fields in line $row: <br /></p>\n";
$row++;
for ($c=0; $c < $num; $c++) {
// write insert query here
echo $data[$c] . "<br />\n";
}
}
fclose($handle);
}
Thanks.

Categories