Need to prevent parsing a huge CSV file repeatedly in PHP - php

I have a huge a CSV file which I parse to store the data in a PHP array. For different PHP files I have to parse it again and again to store it in the array. How can I prevent this by storing it in array and then have this array available to all PHP files?

apc_cache() is what you want. http://us.php.net/manual/en/function.apc-fetch.php

You can write it to a file. Use serialize() before. Then include the file each time you need it (include_once):
$my_csv_data = serialize($data);

Store the array in a file using serialize:
file_put_contents('file.txt', serialize($data));
Then when you need to access it again, use unserialize:
$data = unserialize(file_get_contents('file.txt'));

Related

Write object in JSON file [duplicate]

This question already has answers here:
How to add to JSON array in .json file
(2 answers)
Closed last year.
I have a project based on PHP with OOP structure. When I register a user, it creates a user object and writes that object to json file. The problem is, in this way each time I create a user, the json object stores separately in the file.(shown in the image)
So when I re-read this json file using json_decode() function, it doesn't read anything. I've used the following method to check the file reading.
$data = json_decode(file_get_contents($file), true);
$f = fopen("log", 'a');
fwrite($f, $data);
So how can I append the json objects as list of arrays(upon registering a single user) or read all the entries from the file?
Most of the solutions that I have seen are predefined arrays, but in my case the objects are being created at runtime and only one at a time.
I agree with #Konstantinos Gounaris. The json file is in wrong format. That's why the json_decode function returned nothing. Always use json_encode while writing the json file and write it in one flow. If you want to append to the file, just read the contents, decode it and write it again. Don't append to the file, because it will break the json format.
#Note: Saving confidential information to the file is wrong practice, which leads to security issues.
It is a better practice to use databases to store data(they are more secure than files). However. The first thing that i can see that is wrong is that you are decoding the json file into an php array and then you are writing this array into a file, this results into wrong Json format.
Bellow is an example of how you should do it:
$data_to_import = ['name'=>'John','surname'=>'Doe'];
$data_array = json_decode(file_get_contents($file), true);
array_push($data_array, $data_to_import);
$f = fopen("log", 'a');
fwrite($f, json_encode($data_array));

PHP sanitize strings used inside PHP PDO connection [duplicate]

I was wondering how to save PHP variables to a txt file and then
retrieve them again.
Example:
There is an input box, after submitted the stuff that was written in
the input box will be saved to a text file. Later on the results need
to be brought back as a variable. So lets say the variable is $text I
need that to be saved to a text file and be able to retrieve it back
again.
This should do what you want, but without more context I can't tell for sure.
Writing $text to a file:
$text = "Anything";
$var_str = var_export($text, true);
$var = "<?php\n\n\$text = $var_str;\n\n?>";
file_put_contents('filename.php', $var);
Retrieving it again:
include 'filename.php';
echo $text;
Personally, I'd use file_put_contents and file_get_contents (these are wrappers for fopen, fputs, etc).
Also, if you are going to write any structured data, such as arrays, I suggest you serialize and unserialize the files contents.
$file = '/tmp/file';
$content = serialize($my_variable);
file_put_contents($file, $content);
$content = unserialize(file_get_contents($file));
(Sorry I can't comment just yet, otherwise I would)
To add to Christian's answer you might consider using json_encode and json_decode instead of serialize and unserialize to keep you safe. See a warning from the PHP man page:
Warning
Do not pass untrusted user input to unserialize(). Unserialization can result in code being loaded and executed due to object instantiation and autoloading, and a malicious user may be able to exploit this. Use a safe, standard data interchange format such as JSON (via json_decode() and json_encode()) if you need to pass serialized data to the user.
So your final solution might have the following:
$file = '/tmp/file';
$content = json_encode($my_variable);
file_put_contents($file, $content);
$content = json_decode(file_get_contents($file), TRUE);
for_example, you have anyFile.php, and there is written $any_variable='hi Frank';
to change that variable to hi Jack, use like the following code:
<?php
$content = file_get_contents('anyFile.php');
$new_content = preg_replace('/\$any_variable=\"(.*?)\";/', '$any_variable="hi Jack";', $content);
file_put_contents('anyFile.php', $new_content);
?>
Use a combination of of fopen, fwrite and fread. PHP.net has excellent documentation and examples of each of them.
http://us2.php.net/manual/en/function.fopen.php
http://us2.php.net/manual/en/function.fwrite.php
http://us2.php.net/manual/en/function.fread.php
Use serialize() on the variable, then save the string to a file. later you will be able to read the serialed var from the file and rebuilt the original var (wether it was a string or an array or an object)
Okay, so I needed a solution to this, and I borrowed heavily from the answers to this question and made a library: https://github.com/rahuldottech/varDx (Licensed under the MIT license).
It uses serialize() and unserialize() and writes data to a file. It can read and write multiple objects/variables/whatever to and from the same file.
Usage:
<?php
require 'varDx.php';
$dx = new \varDx\cDX; //create an object
$dx->def('file.dat'); //define data file
$val1 = "this is a string";
$dx->write('data1', $val1); //writes key to file
echo $dx->read('data1'); //returns key value from file
See the github page for more information. It has functions to read, write, check, modify and delete data.

PHP read, stored php var in text file

I'm trying to figure out how to stored php values in a string/file.
I have a text file with "var1=foo&var2=foo2" etc in it, is there a way to read the values?
You could use file_get_contents() to open the file and parse_url() to parse the contents into an associative array.
$file = file_get_contents('file.txt');
parse_str($file, $params);
CodePad.
Or if you can change it, theres always serialize() and unserialize()

php how to write data into json?

I am newer for php. I want make php page cache, query data from mysql and store data into json format.
I have many questions:
which type of file should I store? .json or .txt or .cache? for I also need use json decode return datas into page.
I want use cron tab, make many mysql queries and write into one json file. what write code should I choose? fopen, fwrite or file_get_contents or other command? (do not cover the data, but continue write. I will deleted the file and renewer it at the next cron time)
If a multi write into a json data (10 or more mysql query at the same time and write into a same json file, each json child format like {name: ".$row['name']."}), how to completed a top { and bottom } to make a standad json data format?
{ //how to add this one
{name: ".$row['name']."}
{name: ".$row['name']."}
// many name from 10 more mysql queries
} //and this one
Thanks.
It's json_encode()
json_encode() — Returns the JSON representation of a value
<?php
$arr = array ('a'=>1,'b'=>2,'c'=>3,'d'=>4,'e'=>5);
echo json_encode($arr);
?>
which type of file should I store
It doesn't matter. There is no fixed extension, but I would pick .json just to make it clear what the file is supposed to contain.
what write code should I choose?
Just use file_put_contents to put the JSON string (see next section) into a file.
each json child format like
You really do not want to use that method. It might work for a while, but becomes very complex when you need to handle things like quoting and special-character escapes. Instead of re-inventing the wheel, use PHP's built-in JSON functions for this.
Create the data-structure you want using PHP's strings, numbers, and arrays, and then rely on json_encode to turn it into a string.
The main thing to be careful of is that depending on how your php array() looks, you might get JSON [] versus {}.
As far as saving the file as .txt or .json won't make a difference.
I think the focal point of this all lies in the json_encode page. Here's the example from that page:
This code:
<?php
$arr = array ('a'=>1,'b'=>2,'c'=>3,'d'=>4,'e'=>5);
echo json_encode($arr);
?>
Outputs like this:
{"a":1,"b":2,"c":3,"d":4,"e":5}
3 . You can use fopen and fwrite to write to your file. The second argument to fopen is the mode, you want to use 'a' for append.
Don't write your own cache because anything you write in PHP will be slower than can be supported by native extensions (like APC or memcached or even MySQL itself!!).
Don't cache as JSON. JSON is not a particulary 'fast' to serialize. If you're doing caching you don't want to do any serialization at all. Just store it as it is.
MySQL does query caching for you. If performance is a problem first tune your MySQL queries and database schema. Caching is one of the absolute last optimization you want to do.
If you want an easy way to cache, make a MySQL table called 'cache' and use that. If you want quick (small) file access, use MySQL (seriously). If you want an even faster cache access use an in-memory cache like APC or memcached.

Save PHP variables to a text file

I was wondering how to save PHP variables to a txt file and then
retrieve them again.
Example:
There is an input box, after submitted the stuff that was written in
the input box will be saved to a text file. Later on the results need
to be brought back as a variable. So lets say the variable is $text I
need that to be saved to a text file and be able to retrieve it back
again.
This should do what you want, but without more context I can't tell for sure.
Writing $text to a file:
$text = "Anything";
$var_str = var_export($text, true);
$var = "<?php\n\n\$text = $var_str;\n\n?>";
file_put_contents('filename.php', $var);
Retrieving it again:
include 'filename.php';
echo $text;
Personally, I'd use file_put_contents and file_get_contents (these are wrappers for fopen, fputs, etc).
Also, if you are going to write any structured data, such as arrays, I suggest you serialize and unserialize the files contents.
$file = '/tmp/file';
$content = serialize($my_variable);
file_put_contents($file, $content);
$content = unserialize(file_get_contents($file));
(Sorry I can't comment just yet, otherwise I would)
To add to Christian's answer you might consider using json_encode and json_decode instead of serialize and unserialize to keep you safe. See a warning from the PHP man page:
Warning
Do not pass untrusted user input to unserialize(). Unserialization can result in code being loaded and executed due to object instantiation and autoloading, and a malicious user may be able to exploit this. Use a safe, standard data interchange format such as JSON (via json_decode() and json_encode()) if you need to pass serialized data to the user.
So your final solution might have the following:
$file = '/tmp/file';
$content = json_encode($my_variable);
file_put_contents($file, $content);
$content = json_decode(file_get_contents($file), TRUE);
for_example, you have anyFile.php, and there is written $any_variable='hi Frank';
to change that variable to hi Jack, use like the following code:
<?php
$content = file_get_contents('anyFile.php');
$new_content = preg_replace('/\$any_variable=\"(.*?)\";/', '$any_variable="hi Jack";', $content);
file_put_contents('anyFile.php', $new_content);
?>
Use a combination of of fopen, fwrite and fread. PHP.net has excellent documentation and examples of each of them.
http://us2.php.net/manual/en/function.fopen.php
http://us2.php.net/manual/en/function.fwrite.php
http://us2.php.net/manual/en/function.fread.php
Use serialize() on the variable, then save the string to a file. later you will be able to read the serialed var from the file and rebuilt the original var (wether it was a string or an array or an object)
Okay, so I needed a solution to this, and I borrowed heavily from the answers to this question and made a library: https://github.com/rahuldottech/varDx (Licensed under the MIT license).
It uses serialize() and unserialize() and writes data to a file. It can read and write multiple objects/variables/whatever to and from the same file.
Usage:
<?php
require 'varDx.php';
$dx = new \varDx\cDX; //create an object
$dx->def('file.dat'); //define data file
$val1 = "this is a string";
$dx->write('data1', $val1); //writes key to file
echo $dx->read('data1'); //returns key value from file
See the github page for more information. It has functions to read, write, check, modify and delete data.

Categories