(PHP) Help debug tracking script to show only unique items - php

i have this script that i am trying to implement to work on my website and the script is suppose to track what games are being played at the moment but the only problem is that when i call the script to display the entries it would show duplicates and i want the script to actually explore the string and take only the $gamename and look for duplicates, not the entire string.
What the script does:
Records the gameID, gamename, gamethumb url, IP and time all separated by |.
Example: 1744|The Simpliest Snowboarding|The Simpliest Snowboarding|77.88.42.26|1264717552
Look for the IP and if already exists update the record with the new info.
If the IP does not exist already write a new line with the information.
If the record is older then 60min erase it.
Its just a simple script that i will use to show what people are currently playing on the website.
$dataFile = "visitors2.txt";
$numbergames = 30;
//Please do not edit bellow this line
error_reporting(E_ERROR | E_PARSE);
$users = array();
//getting
$fp = fopen($dataFile, "r");
flock($fp, LOCK_SH);
while(!feof($fp)) {
$users[] = fgets($fp, 4096);
}
flock($fp, LOCK_UN);
fclose($fp);
$i = 0;
echo '<ul>';
foreach(array_unique($users) as $key => $data) {
list($game2id , $gamename , $gamethumb , , ) = explode("|", $data);
//echo $game2id . $gamename;
if($gamename != "" && $i <= $numbergames) {
$newpageurl = str_replace(" ", "-", strip_tags(trim(str_replace($rplce, "", $gamename)))) ;
$url = $game2id .'-'. $newpageurl .'.html';
echo '<li><img src="./arcade/img/'.$gamethumb.'.png" width="35" height="35" border="0" />'.$gamename.'</li>';
}
$i++;
}
Please, help and thanks everyone in advance.

Your array_unique() call is only eliminating duplicate rows - entire rows, not just game names. You need to create an array of just the game names. Then, you can eliminate the dupes.
Something like:
$currentGames = array();
$max = 30;
$fp = fopen($dataFile, "r");
flock($fp, LOCK_SH);
while ((count($currentGames) < $max) && (($data = fgetcsv($fg, 0, '|')) !== FALSE)
{
if (!in_array($data[1], $currentGames)) $currentGames[$data[0]] = $data[1];
}
This will give you an associative array of unique game names with game ids as keys.

Related

replace rows from one csv file to another csv where id of is the samePHP?

I have one excel orginal.csv file
ID Name Price
1 Xblue 12
2 Yblue 32
3 Zblue 52
And another copy.csv file
ID Name Price
1 Xblue 89
2 Yblue 43
3 Zblue 45
I want to replace rows from orginal.csv to copy.csv where ID is the same.
Can I do this manually or maybe somehow using PHP?
I search for some options on the internet, but I only found getcsv and readcsv functions that can't help me in this case. Cause this is something like updating CSV file.
It may end up in request timeout in PHP because it requires so many loops to do it. If someone can reduce the time complexity of this program then it will work. if it even works it will take a lot of time to do it.
while(! feof($f_pointer)){ //open old csv to update loop1
$ar=fgetcsv($f_pointer); // getting first row
for($i=0;$i<count($ar);$i++){ //loop2 first row array
$f_pointer2=fopen("new.csv","r"); open new csv to get data
while(! feof($f_pointer2)){ // loop3 to find ID in new csv
$ar2=fgetcsv($f_pointer2); //getting each row in array
for($j=0;$j<count($ar2);$j++){ //loop4 to compare id of old csv to new csv and update data
if($ar[i] == $ar2[j]){
foreach ($ar2 as $fields) { //loop5
fputcsv($f_pointer, $fields);
}
}
}
}
}
}
?>
I've created a little soulution. If order is important you don't have to index the array and loop through the copied array.
<?php
if(file_exists('output.csv'))
{
unlink('output.csv');
}
function fputcsv_eol($handle, $array, $delimiter = ',', $enclosure = '"', $eol = "\n") {
$return = fputcsv($handle, $array, $delimiter, $enclosure);
if($return !== FALSE && "\n" != $eol && 0 === fseek($handle, -1, SEEK_CUR)) {
fwrite($handle, $eol);
}
return $return;
}
function scanFile($sFilename, $iIndexColumn)
{
$rFile = fopen($sFilename, 'r');
$aData = array();
while(($aLine = fgetcsv($rFile)) !== false)
{
$aData[$aLine[$iIndexColumn]] = $aLine;
}
fclose($rFile);
return $aData;
}
$iIndexColumn = 0;
$iValueColum = 2;
$aOriginalData = scanFile('original.csv', 0);
$aCopyData = scanFile('copy.csv', 0);
foreach($aOriginalData as $iID => $aOriginalDatum)
{
if(array_key_exists($iID, $aCopyData))
{
$aCopyData[$iID] = $aOriginalDatum;
}
}
$rFile = fopen('output.csv', 'w');
foreach($aCopyData as $aCopyDatum)
{
fputcsv_eol($rFile, $aCopyDatum, ',', '"',"\r\n");
}
fclose($rFile);

Read large data from csv file in php [duplicate]

This question already has answers here:
file_get_contents => PHP Fatal error: Allowed memory exhausted
(4 answers)
Closed 3 years ago.
I am reading csv & checking with mysql that records are present in my table or not in php.
csv has near about 25000 records & when i run my code it display "Service Unavailable" error after 2m 10s (onload: 2m 10s)
here i have added code
// for set memory limit & execution time
ini_set('memory_limit', '512M');
ini_set('max_execution_time', '180');
//function to read csv file
function readCSV($csvFile)
{
$file_handle = fopen($csvFile, 'r');
while (!feof($file_handle) ) {
set_time_limit(60); // you can enable this if you have lot of data
$line_of_text[] = fgetcsv($file_handle, 1024);
}
fclose($file_handle);
return $line_of_text;
}
// Set path to CSV file
$csvFile = 'my_records.csv';
$csv = readCSV($csvFile);
for($i=1;$i<count($csv);$i++)
{
$user_email= $csv[$i][1];
$qry = "SELECT u.user_id, u.user_email_id FROM tbl_user as u WHERE u.user_email_id = '".$user_email."'";
$result = #mysql_query($qry) or die("Couldn't execute query:".mysql_error().''.mysql_errno());
$rec = #mysql_fetch_row($result);
if($rec)
{
echo "Record exist";
}
else
{
echo "Record not exist";
}
}
Note: I just want to list out records those are not exist in my table.
Please suggest me solution on this...
An excellent method to deal with large files is located at: https://stackoverflow.com/a/5249971/797620
This method is used at http://www.cuddlycactus.com/knownpasswords/ (page has been taken down) to search through 170+ million passwords in just a few milliseconds.
After struggling a lot, finally i found a good solution, may be it help others also.
When i tried 2,367KB csv file containing 18226 rows, the least time taken by different php scripts were
(1) from php.net fgetcsv documentation named CsvImporter, and
(2) file_get_contents => PHP Fatal error: Allowed memory exhausted
(1) took 0.92574405670166
(2) took 0.12543702125549 (string form) & 0.52903485298157 (splitted to array)
Note: this calculation not include adding to mysql.
The best solution i found uses 3.0644409656525 total including adding to database and some conditional check also.
It took 11 seconds in processing a 8MB file.
solution is :
$csvInfo = analyse_file($file, 5);
$lineSeperator = $csvInfo['line_ending']['value'];
$fieldSeperator = $csvInfo['delimiter']['value'];
$columns = getColumns($file);
echo '<br>========Details========<br>';
echo 'Line Sep: \t '.$lineSeperator;
echo '<br>Field Sep:\t '.$fieldSeperator;
echo '<br>Columns: ';print_r($columns);
echo '<br>========Details========<br>';
$ext = pathinfo($file, PATHINFO_EXTENSION);
$table = str_replace(' ', '_', basename($file, "." . $ext));
$rslt = table_insert($table, $columns);
if($rslt){
$query = "LOAD DATA LOCAL INFILE '".$file."' INTO TABLE $table FIELDS TERMINATED BY '$fieldSeperator' ";
var_dump(addToDb($query, false));
}
function addToDb($query, $getRec = true){
//echo '<br>Query : '.$query;
$con = #mysql_connect('localhost', 'root', '');
#mysql_select_db('rtest', $con);
$result = mysql_query($query, $con);
if($result){
if($getRec){
$data = array();
while ($row = mysql_fetch_assoc($result)) {
$data[] = $row;
}
return $data;
}else return true;
}else{
var_dump(mysql_error());
return false;
}
}
function table_insert($table_name, $table_columns) {
$queryString = "CREATE TABLE " . $table_name . " (";
$columns = '';
$values = '';
foreach ($table_columns as $column) {
$values .= (strtolower(str_replace(' ', '_', $column))) . " VARCHAR(2048), ";
}
$values = substr($values, 0, strlen($values) - 2);
$queryString .= $values . ") ";
//// echo $queryString;
return addToDb($queryString, false);
}
function getColumns($file){
$cols = array();
if (($handle = fopen($file, 'r')) !== FALSE)
{
while (($row = fgetcsv($handle)) !== FALSE)
{
$cols = $row;
if(count($cols)>0){
break;
}
}
return $cols;
}else return false;
}
function analyse_file($file, $capture_limit_in_kb = 10) {
// capture starting memory usage
$output['peak_mem']['start'] = memory_get_peak_usage(true);
// log the limit how much of the file was sampled (in Kb)
$output['read_kb'] = $capture_limit_in_kb;
// read in file
$fh = fopen($file, 'r');
$contents = fread($fh, ($capture_limit_in_kb * 1024)); // in KB
fclose($fh);
// specify allowed field delimiters
$delimiters = array(
'comma' => ',',
'semicolon' => ';',
'tab' => "\t",
'pipe' => '|',
'colon' => ':'
);
// specify allowed line endings
$line_endings = array(
'rn' => "\r\n",
'n' => "\n",
'r' => "\r",
'nr' => "\n\r"
);
// loop and count each line ending instance
foreach ($line_endings as $key => $value) {
$line_result[$key] = substr_count($contents, $value);
}
// sort by largest array value
asort($line_result);
// log to output array
$output['line_ending']['results'] = $line_result;
$output['line_ending']['count'] = end($line_result);
$output['line_ending']['key'] = key($line_result);
$output['line_ending']['value'] = $line_endings[$output['line_ending']['key']];
$lines = explode($output['line_ending']['value'], $contents);
// remove last line of array, as this maybe incomplete?
array_pop($lines);
// create a string from the legal lines
$complete_lines = implode(' ', $lines);
// log statistics to output array
$output['lines']['count'] = count($lines);
$output['lines']['length'] = strlen($complete_lines);
// loop and count each delimiter instance
foreach ($delimiters as $delimiter_key => $delimiter) {
$delimiter_result[$delimiter_key] = substr_count($complete_lines, $delimiter);
}
// sort by largest array value
asort($delimiter_result);
// log statistics to output array with largest counts as the value
$output['delimiter']['results'] = $delimiter_result;
$output['delimiter']['count'] = end($delimiter_result);
$output['delimiter']['key'] = key($delimiter_result);
$output['delimiter']['value'] = $delimiters[$output['delimiter']['key']];
// capture ending memory usage
$output['peak_mem']['end'] = memory_get_peak_usage(true);
return $output;
}
Normally, "Service Unavailable" error will come when 500 error occurs.
I think this is coming because of insufficient execution time. Please check your log/browser console, may be you can see 500 error.
First of all,
Keep set_time_limit(60) out of loop.
Do some changes like,
Apply INDEX on user_email_id column, so you can get the rows faster with your select query.
Do not echo message, Keep the output buffer free.
And
I have done these kind of take using Open source program. You can get it here http://sourceforge.net/projects/phpexcelreader/
Try this.

PHP counter with flock

I have a problem with a counter. I need to count two variables, separated with a |, but sometimes the counter doesn't increase a variable's value.
numeri.txt (the counter):
6122|742610
This is the PHP script:
$filename="numeri.txt";
while(!$fp=fopen($filename,'c+'))
{
usleep(100000);
}
while(!flock($fp,LOCK_EX))
{
usleep(100000);
}
$contents=fread($fp,filesize($filename));
ftruncate($fp,0);
rewind($fp);
$contents=explode("|",$contents);
$clicks=$contents[0];
$impressions=$contents[1]+1;
fwrite($fp,$clicks."|".$impressions);
flock($fp,LOCK_UN);
fclose($fp);
I have another counter that is a lot slower but counts both values (clicks and impressions) exactly. Sometimes the counter numeri.txt counts more impressions than the other counter. Why? How can I fix this?
We're using the following at our high-traffic site to count impressions:
<?php
$countfile = "counter.txt"; // SET THIS
$yearmonthday = date("Y.m.d");
$yearmonth = date("Y.m");;
// Read the current counts
$countFileHandler = fopen($countfile, "r+");
if (!$countFileHandler) {
die("Can't open count file");
}
if (flock($countFileHandler, LOCK_EX)) {
while (($line = fgets($countFileHandler)) !== false) {
list($date, $count) = explode(":", trim($line));
$counts[$date] = $count;
}
$counts[$yearmonthday]++;
$counts[$yearmonth]++;
fseek($countFileHandler, 0);
// Write the counts back to the file
krsort($counts);
foreach ($counts as $date => $count) {
fwrite($countFileHandler, "$date:$count\n");
fflush($countFileHandler);
}
flock($countFileHandler, LOCK_UN);
} else {
echo "Couldn't acquire file lock!";
}
fclose($countFileHandler);
}
?>
The results are both daily and monthly totals:
2015.10.02:40513
2015.10.01:48396
2015.10:88909
Try performing a flush before unlocking. You're unlocking before the data might even be written, allowing another execution to clobber.
http://php.net/manual/en/function.fflush.php

Increment number in text file

I want to record downloads in a text file
Someone comes to my site and downloads something, it will add a new row to the text file if it hasn't already or increment the current one.
I have tried
$filename = 'a.txt';
$lines = file($filename);
$linea = array();
foreach ($lines as $line)
{
$linea[] = explode("|",$line);
}
$linea[0][1] ++;
$a = $linea[0][0] . "|" . $linea[0][1];
file_put_contents($filename, $a);
but it always increments it by more than 1
The text file format is
name|download_count
You're doing your incrementing outside of the for loop, and only accessing the [0]th element so nothing is changing anywhere else.
This should probably look something like:
$filename = 'a.txt';
$lines = file($filename);
// $k = key, $v = value
foreach ($lines as $k=>$v) {
$exploded = explode("|", $v);
// Does this match the site name you're trying to increment?
if ($exploded[0] == "some_name_up_to_you") {
$exploded[1]++;
// To make changes to the source array,
// it must be referenced using the key.
// (If you just change $v, the source won't be updated.)
$lines[$k] = implode("|", $exploded);
}
}
// Write.
file_put_contents($filename, $lines);
You should probably be using a database for this, though. Check out PDO and MYSQL and you'll be on your way to awesomeness.
EDIT
To do what you mentioned in your comments, you can set a boolean flag, and trigger it as you walk through the array. This may warrant a break, too, if you're only looking for one thing:
...
$found = false;
foreach ($lines as $k=>$v) {
$exploded = explode("|", $v);
if ($exploded[0] == "some_name_up_to_you") {
$found = true;
$exploded[1]++;
$lines[$k] = implode("|", $exploded);
break; // ???
}
}
if (!$found) {
$lines[] = "THE_NEW_SITE|1";
}
...
one hand you are using a foreach loop, another hand you are write only the first line into your file after storing it in $a... it's making me confuse what do you have in your .txt file...
Try this below code... hope it will solve your problem...
$filename = 'a.txt';
// get file contents and split it...
$data = explode('|',file_get_contents($filename));
// increment the counting number...
$data[1]++;
// join the contents...
$data = implode('|',$data);
file_put_contents($filename, $data);
Instead of creating your own structure inside a text file, why not just use PHP arrays to keep track? You should also apply proper locking to prevent race conditions:
function recordDownload($download, $counter = 'default')
{
// open lock file and acquire exclusive lock
if (false === ($f = fopen("$counter.lock", "c"))) {
return;
}
flock($f, LOCK_EX);
// read counter data
if (file_exists("$counter.stats")) {
$stats = include "$counter.stats";
} else {
$stats = array();
}
if (isset($stats[$download])) {
$stats[$download]++;
} else {
$stats[$download] = 1;
}
// write back counter data
file_put_contents('counter.txt', '<?php return ' . var_export($stats, true) . '?>');
// release exclusive lock
fclose($f);
}
recordDownload('product1'); // will save in default.stats
recordDownload('product2', 'special'); // will save in special.stats
personally i suggest using a json blob as the content of the text file. then you can read the file into php, decode it (json_decode), manipulate the data, then resave it.

what's the code meaning?

$file = fopen("test.txt","r");
while($line = fgets($file)) {
$line = trim($line);
list($model,$price) = preg_split('/\s+/',$line);
if(empty($price)) {
$price = 0;
}
$sql = "UPDATE products
SET products_price=$price
WHERE products_model='$model'";
// run the sql query.
}
fclose($file);
the txt file like this:
model price
LB2117 19.49
LB2381 25.99
1, what's the meaning of list($model,$price) = preg_split('/\s+/',$line);
i know preg_split like explode, but i don't know what't the parameter meaning of the above line
2, how to skip the first record.
it's taking the results of the preg_split and assigning them to the vars $model and $price. You're looking at a parsing algorithm. Sorry if this is not enough. I have a hard time understanding the question as it is written.
Also, if I read this correctly, there is no need to skip line 1 unless you have an item with the model defined as "model" in the database.
But if you wanted to for some reason, you could add a counter...
$i = 0;
while($line = fgets($file)) {
if($i > 0)
{
$line = trim($line);
list($model,$price) = preg_split('/\s+/',$line);
if(empty($price)) {
$price = 0;
}
$sql = "UPDATE products
SET products_price=$price
WHERE products_model='$model'";
// run the sql query.
}
$i++;
}
That is a language construct that allows you to assign to multiple variables at once. You can think of it as array unpacking (preg_split returns an array). So, when you do:
<?php
list($a, $b) = explode(".","a.b");
echo $a . "\n";
echo $b . "\n";
You will get:
a
b
Having less elements in list than the array is ok, excess elements in array are ignored, but having insufficent elements in array will give you an undefined index error. For example:
list($a) = explode(".","a.b"); // ok
list($a,$b,$c) = explode(".","a.b") // error
I don't know if you meant that by skip the first record but...
$file = fopen("test.txt","r"); // open file for reading
$first = true;
while($line = fgets($file)) { // get the content file
if ($first === true) { $first = false;}//skip the first record
else{
$line = trim($line); // remove the whitespace before and after the test
// not in the middle
list($model,$price) = preg_split('/\s+/',$line); // create two variable model and price, the values are from the preg_split \s means whitespace, tab and linebreak
if(empty($price)) { // if $price is empty price =0
$price = 0;
}
$sql = "UPDATE products // sql update
SET products_price=$price
WHERE products_model='$model'";
// run the sql query.
}
}
fclose($file); //close the file

Categories