Currently have a file that is set to read a CSV file. The CSV file contains 1600 api queries. Then each api query then returns more queries that need to be run. I am using Xampp v3.2.2 on windows 10 and running php v5.6.15. When running the file through my browser it ran fine for the first 800+ records in the CSV before timing out. When I rerun the file now I get an error "Site can't be reach ERR_CONNECTION_RESET". Not sure what could be causing this. abbreviated version of the code is included below
<?php
error_reporting(E_ERROR | E_PARSE);
set_time_limit (28800);
$csv = array_map('str_getcsv', file('file.csv'));
for($i = 0; $i < count($csv); $i++){
if($csv[$i][2] == 1){ //this item in csv is flag to check if this item has been run yet
if($csv[$i][3] != 'NULL' && trim($csv[$i][3]) != ''){ // check to make sure there is a URL
$return = file_get_contents(trim($csv[$i][3])); //Get the contents that links to new api calls
if($return){
$isCall = array(); // array to store all new calls
$data = array(); // array to store all data to put in csv
$doc = new DOMDocument('1.0'); // create new DOM object
$doc->loadHTML($return); // load page string into DOM object
$links = $doc->getElementsByTagName('a'); // get all <a> tags on the page
if($links->length > 0){ // if there is at least one <a> tag on page
for($j = 0; $j < $links->length; $j++){ // loop through <a> tags
$isCall[]= $links->item($j)->getAttribute('href'); // get href attribute from <a> tag and push into array
}
for($x = 0; $x < count($isCall); $x++){ // loop through all the calls and search for data
$string = file_get_contents($isCall[$x]);
if($string) {
$thispage = new DOMDocument('1.0');
$thispage->loadHTML($string);
$pagedata = $thispage->getElementsByTagName('div');
if ($pagedata->length > 0) {
for($j = 0; $j < $pagedata->length; $j++) {
$data[] = $pagedata->item($j)->C14N();
}
}
}
if(count($data) >= 5) break; // limiting to 5 data points to be added to csv
}
}
if(!empty($data)) $csv[$i] = array_merge($csv[$i], $data); // if we have data points lets add them to the row in the csv
}
}
$csv[$i][2] = 2; // set the flag to 2
$fp = fopen('file.csv', 'w'); // write the contents to the csv each time through loops so if it fails we start at last completed record
foreach ($csv as $f) {
fputcsv($fp, $f);
}
fclose($fp);
}
}
?>
Usually ERR_CONNECTION_RESET is an error that occurs when the site you are trying to connect to is unable to establish that connection. This usually happens due to reasons like firewall blocking on issues with ISP cache etc.
However in your case, I feel that the site you are connecting to is voluntarily closing connection attempts because what you are trying to do is loop over and hit that API site 1600 times continuously.
The API site is allowing the first 800-odd attempts but after that it gets worried that you are perhaps a malicious script trying to harm it. Like a classical example of DOS (Denial Of Service) attempt.
You should check if there is any restriction to the number of attempts a client can make to the API site with a fixed time (like say 500 hits every 24 hours) or you should try to sleep N seconds after each hit to the site or after each X number of hits to the site.
I want to delete sheets from an Excel2005/Excel5 file using PHP. I am using PHPExcel-1.7.9. I am using the following code to delete Excel2007 files as follows
$exceltype="Excel2007";
$excel = PHPExcel_IOFactory::createReader($exceltype);
$excel = $excel->load("ABC.xlsx");
$count = $excel->getSheetCount();
for($i = 0; $i < $count; $i++)
{
$excel->removeSheetByIndex(0);
}
When I use it for Excel5, I get an error
Fatal error: Call to undefined method PHPExcel_Reader_Excel5::getSheetCount()
Potential Issue #1
Use a different variable name for the Reader and for the object that you're loading from the Reader
$exceltype="Excel2007";
$excelReader = PHPExcel_IOFactory::createReader($exceltype);
$excel = $excelReader->load("ABC.xlsx");
Potential Issue #2
It's always sensible to let PHPExcel identify the filetype for you rather than trusting to the file extension
$excel = PHPExcel_IOFactory::load("ABC.xlsx");
Documentation
I am done with basic import to database. I have explained in the following example where I got stuck up ...
For example,
I have an xls file named project.xls and it has 6 sheets. I have populated all 6 sheet names in dropdown. If I select sheet2 and click button, it should import sheet2 data into db and sheet3 so on.
How can I do this ...? please help me...
You can access to differents sheets of a file with PHPExcelReader this way:
$filename = "path/to/filename.xls";
$reader = new Spreadsheet_Excel_Reader(); // Your PHPEXCELREDER Class
$reader->setOutputEncoding('UTF8');
// Read XLS File
$reader->read($filename);
$sheet = 2; // Your sheet
// Then walk trough your wanted sheet:
for ($i = 2; $i <= $reader->sheets[$sheet]['numRows']; $i++) {
// Do something
}
I dont know how to use filesize($filename) in this particular code that I need, the problem is I do not know how to represent the filename...
Could someone please tell me what should go after $tblDocs->Value["filesize"]=
global $dal;
$tblDocs = $dal->Table("doc_files");
$fileArray = my_json_decode($values["file"]);
for($i = 0; $i < count($fileArray); $i++)
{
$tblDocs->Value["parent_folder_id"]=$_SESSION["current_folder"];
$tblDocs->Value["file_type"]="file";
$tblDocs->Value["file"]=my_json_encode(array($fileArray[$i]));
$tblDocs->Value["hash"]=generatePassword(HASH_LENGTH);
$tblDocs->Value["name"]=$fileArray[$i]["usrName"];
$tblDocs->Value["ownerid"]=$_SESSION["user_id"];
$tblDocs->Value["created"]=now();
$tblDocs->Value["filesize"]=
$tblDocs->Add();
}
UPDATE: I have just exported a csv file form my database to see what is actually stored in the file field, and this is result
[{"usrName":"rootcause.jpg","name":"/home/sites/iso-drive.co.uk/public_html/portal/files/rootcause_48zewwp6.jpg","size":1323,"type":"image/jpeg"}]
So...how Do I capture the information in 3rd field "size":1323 - specifically the 1323?
Boldly guessing
$tblDocs->Value["filesize"]=$fileArray[$i]["size"];
I created a CSV parser that works fine for some CSV files I've found online, but one that I converted from XLS to CSV via Microsoft Excel 2011 does not work.
The ones that work are formatted as such:
"Sort Order","Common Name","Formal Name","Type","Sub Type","Sovereignty","Capital","ISO 4217 Currency Code","ISO 4217 Currency Name","ITU-T Telephone Code","ISO 3166-1 2 Letter Code","ISO 3166-1 3 Letter Code","ISO 3166-1 Number","IANA Country Code TLD"
"1","Afghanistan","Islamic State of Afghanistan","Independent State",,,"Kabul","AFN","Afghani","+93","AF","AFG","004",".af".........................etc...
The one that doesn't work is formatted like this:
Order Id,Date Ordered,Date Returned,Product Id,Description,Order Reason Code,Return Qty,Order Return Comment,Ship To Name,Ship To Address1,Ship To Address2,Ship To Address3,Ship To City,Ship To State,Ship To Zipcode,Ship To Country,Disposition,Ship To Email,ShipVia
5555555,2013-07-05 13:58:36.000,2013-08-16 00:00:00.000,5555-55,0555 - Some Test Thing,Refund,2,,jeric beatty,123 fake st,,,burke,NJ,55055,US,Discard,test#test.com,Super Fast Shipping
Is there anyway to get excel to export in the format as the first one? I would like to avoid doing this manually as the file is huge and I would have to manually edit lots of parts of it where I couldn't do a "replace all". Another issue could be that there are double and sometimes triple commas in some places. Though this does appear in both files.
Here is the parser:
function ingest_csv() {
$file_url = 'http://www.path.to/csv/file.csv';
$record_num = 0;
$records = array();
$header = array();
if (($handle = fopen($file_url, "r")) !== FALSE) {
$records['id'] = '';
while (($data = fgetcsv($handle)) !== FALSE) {
$records['id'][$record_num] = '';
$cell_num = 0;
foreach ($data as $cell) {
if($record_num == 0) {
$header = $data;
} else {
$current_key = $header[$cell_num];
$records['id'][$record_num][$current_key] = $cell;
}
$cell_num++;
}
$record_num++;
}
fclose($handle);
}
else {
echo 'could not open file.';
}
return array($record_num, $records);
}
function batch_csv() {
list($num_rows, $rows) = ingest_csv
print_r($num_rows);
print_r($rows);
}
As mentioned in the comments though you may be trying to reinvent the wheel here, though personally I've asked questions where I didn't want to give long rambling explanations of why I was forced to use unconventional approaches so should this be one of those situations here's an answer.
In OpenOffice Calculator (for example) and when you go to save as CSV you get a number of further options including the decision to double quote all fields.
Unfortunately Excel doesn't give you the choice, but Microsoft do offer up a workaround using a macro - http://support.microsoft.com/kb/291296/en-us