Efficien memory and cpu usage with progressive read and write [closed] - php

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
Suppose i have fetch more data rows in laravel and i have to write in csv file. And suppose my server execution time is out before all data rows are written in csv file. Then how can i proceed in order to be able to write all the data row in csv file ?
Actually i wanted to fetch data rows, clean memory and read and write data in the csv file but have no clue in laravel.
Please guys help me with any reference or ideas.

Create a file and having the file location stored somewhere.
Limit the records fetched once using the query.
e.g.
get the count of your required records and execute a loop according to that say 100 records at a time and write/append those records in your CSV file.
Alternative Way (Simplest Way)
use Maatwebsite/Laravel-Excel library to create CSV, excel etc.

Related

how would you create this database? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 1 year ago.
Improve this question
I'm trying to create a database to store nearly 5000 records per day (500 per user) and I was wondering if that can slow down the website.
I have an idea to store every 500 data for a user in a text file and save the .txt file into just one row. This way 5000 rows are reduced to 10 rows.
will this work?
PS: I will use PHP implode and explode functions to assemble and disassemble the .txt file.
If you can suggest any better ways that will be amazing.
5000, 50000, 500 000, 5 000 000, 50 000 000 records wont slow your website if you will set indexes right and definitely using .txt files are not best choice to store the data, just set your table in right way (correct column data type, indexes, good and optimized queries)

Include an array of 20k zip codes or call MySQL [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I have a list of about 20,000 zip codes that I need to check against. Should I store them in an PHP file as an array? How much memory would that occupy?
Or should I call MySQL every time to check against its database table to see if it exists? Which way is faster? I assume the first option should be faster? The connection to database alone may slow down the database call option quite significantly? I'm just a bit concerned about that memory problem if I do it by including PHP file on every call.
Databases are specifically designed to store and search through large amounts of data efficiently and quickly. If you were to put a 20,000 element array in every PHP file it would drastically slow down every page load, even when the array wasn't being used.

Daily import .csv files from sftp to a MySQL database [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I'm trying to automate a simple and daily task: There're multiple .csv files on a SFTP-server that I want to import in a MySQL-database.
The filenames have a structure like 20160601-1.csv, 20160601-2.csv, 20160601-3.csv, etc. The number of files divers from 8 to 20 per day. The layout of the .csv is always the same (10 columns with some empty fields).
I know this isn't very detailed, but i'm searching for some hours now and I don't know where to begin.
I think php is a good way to do this? Or maybe Python?
You can do this in PHP using fgetcsv() function,
Sample code to read csv file
//read file
$fp = fopen($filename,'r') or die("can't open file");
$csv_line=fgetcsv($fp);
$last=0;
$insert_count=0;
while($csv_line = fgetcsv($fp,1024))
{
//$csv_line is an indexed array for values on each line
//database update code goes here
}

How to make specific columns of a csv file read only with php [closed]

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 7 years ago.
Improve this question
I wrote a script to export some columns(id and name) of a mysql table so that scores of students can be inserted. But I want to make the id and name columns read only. I searched but could not find a solution. Please, how can I do this?
You cannot do this.
A file is just a stream of bytes stored (in HDD/memory). You read these bytes and interpret it as you want. In case of csv file you read one line and the split this based on separator.
The concept of read only comes from OS. Where you mark a file as read only to ensure users without proper permission do not edit the file.
While writing a program you open a file in read only mode using r option this is just a way of telling you do not intend to write anything to file and fwrite should not be allowed (throw exception) on this stream.
What you are asking is to make certain bytes of the files to be read only while others should be editable. OS or file stream have no way differentiating between the two.

How to write a Microsoft Excel file using PHP? [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
There are 8 columns that I want to save in the excel file. How do I use PHP to create, access and edit Microsoft Excel files?
I think this is what you want.
https://github.com/PHPOffice/PHPExcel#phpexcel---openxml---read-write-and-create-spreadsheet-documents-in-php---spreadsheet-engine
PHPExcel is a library that can handle Excel files, and some other file formats. If you only have this 8 columns you than just set the data with
$objPHPExcel->getActiveSheet()->SetCellValue('A1', 'Content');
...
$objPHPExcel->getActiveSheet()->SetCellValue('H1', 'Content');
And have a look at the hello word example, this will guide you how to create a simple xlsx file.
Check out https://phpexcel.codeplex.com/
It provides some very cool features to manipulate Excel files. You can find a couple of examples here: https://phpexcel.codeplex.com/wikipage?title=Examples&referringTitle=Home
PHPExcel is what you want, you can download from
https://phpexcel.codeplex.com/
Be careful when managing huge spreadsheets, because PHPExcel consumes lots of memory.
If you only need output a simple excel file with non formatted data, you can output the data in comma separated values (CSV)

Categories