PHP issue reading huge data from txt files - php

I'm running a php script that reads from some pipe-delimited txt files, those files contain around 22,000 records. I'm using PHP file() function to read those files, BTW the files have some inter-relation too, I'm pasting the code here for better understanding
public function getGames()
{
$resource = self::DATAFILES.'Product.txt';
$games = array_slice($this->readFile($resource), 1);
$data = array();
$count = 1;
foreach($games as &$records)
{
$game = new Game();
$attributes = explode($this->delimiter,$records);
$game->api = (int) $attributes[0];
$game->console_id = (string) $attributes[1];
$game->title = (string) $this->getTitle($attributes[2]);
$game->barcode = (string) $attributes[4];
$game->image = $this->getCoverImage($attributes[0]);
$game->releateDate = strtotime($attributes[8]);
$data[] = $game;
//if($count == 100): break; else: $count++; endif;
}
print '<pre>'; print_r($data);
}
public function getTitle($titleID)
{
$resource = self::DATAFILES.'Title.txt';
$titles = array_slice($this->readFile($resource), 1);
foreach($titles as $records)
{
$attributes = explode($this->delimiter,$records);
$pattern = '/^' . preg_quote($attributes[0], '/') . '$/';
if (preg_match($pattern, $titleID))
{
return $attributes[2];
break;
}
}
}
public function getCoverImage($gameID)
{
$resource1 = self::DATAFILES.'ProductImage.txt';
$coverImages = array_slice($this->readFile($resource1), 1);
foreach($coverImages as $img_records)
{
$image_attributes = explode($this->delimiter,$img_records);
$pattern1 = '/^' . preg_quote($image_attributes[0], '/') . '$/';
$pattern2 = '/^' . preg_quote($image_attributes[3], '/') . '$/';
if (preg_match($pattern1, $gameID) && preg_match($pattern2, 'Cover'))
{
return $image_attributes[2];
break;
}
}
So the problem I'm facing here is that - when I run that script it will just go on and on nothing happens i.e never throws any error neither print the returned array, however when I limit's the loop iteration up to 100 it works which means nothing wrong with the code so I thought perhaps, php script execution time making trouble so I changed it to max_execution_time = 0 - but still not getting the results. Any help or suggestion, would love hear that.. :)
BTW I'm using Xampp Apache for my local dev !

Related

PHP dynamic breadcrumb script creates wrong URL after switching to PHP 8.0

I'm using a PHP script to create a dynamic breadcrumb navigation. I found it several years ago in another forum where it was originally posted in 2006. I updated it by removing some deprecated functions and it worked well so far. It creates the crumbs from the page URL:
$path = $this->_homepath;
$html ='';
$html .= 'Start';
$parts = explode('/', $_SERVER['PHP_SELF']);
$partscount = count($parts);
for($i = 1; $i < ($partscount - 1); $i++) {
$path .= $parts[$i] . '/';
$title = $parts[$i];
if ($this->_usereplacements == true) {
reset($this->_replacements);
foreach($this->_replacements as $search => $replace) {
$title = str_replace($search, $replace, $title);
}
}
if ($this->_replaceunderlines == true) {
$title = str_replace('_', ' ', $title);
}
if ($this->_ucfirst == true) {
$title = ucfirst($title);
}
if ($this->_ucwords == true) {
$title = ucwords($title);
}
$html .= $this->_separator . '' . $title . '';
}
The current page title, i.e. the last element of the bradcrumb, is generated as follows:
$title = '';
if ($title == '') {
$title = $parts[$partscount-1];
if ($this->_usereplacements == true) {
reset($this->_replacements);
foreach($this->_replacements as $search => $replace) {
$title = str_replace($search, $replace, $title);
}
}
if ($this->_removeextension == true) {
$title = substr($title, 0, strrpos($title, '.'));
}
if ($this->_replaceunderlines == true) {
$title = str_replace('_', ' ', $title);
}
if ($this->_ucfirst == true) {
$title = ucfirst($title);
}
if ($this->_ucwords == true) {
$title = ucwords($title);
}
}
$html .= $this->_separator . '<b>' . $title . '</b>';
I only copied relevant parts of the script and spared eg. the function declarations.
After switching from PHP 7.4 to 8.0 I found that the script is messing up a bit. Consider a page with this url: https://www.myhomepage.com/site1/site2/site3
For the crumbs of site1 the script is now generating the URL https://www.myhomepage.com/site1/site2/site1 insted of https://www.myhomepage.com/site1/, whereas for site2 it shows https://www.myhomepage.com/site1/site2/site1/site2 instead of https://www.myhomepage.com/site1/site2/. As you can see, the whole path https://www.myhomepage.com/site1/site2/ is always added as a prefix before the actual path of the crumb.
I haven't found a solution yet despite looking over this thing for two days. I suppose there have been some changes in PHP 8.0 which causes this behaviour, but I haven't found any clues in the incompatibility list (https://www.php.net/manual/de/migration80.php). I printed §path and $title and they look like they should. When echoing $html after each iteration in the for loop it shows already the wrong URLs. That's why I think that probably the for loop is the problem here. Do you have any suggestions?
Any help on this would be very much appreciated.
I finally was able to figure out what causes the strange behaviour of the script. In the declaration of the breadcrumb constructing function there is $this->_homepath = '/';. Later on I assign $path = $this->_homepath;. When I echo $path its empty instead of showing the aforementioned "/". So I directly set $path = '/'; and now everything works again as it should.

PHP - proxy checker multi-threaded

I already made my personal single thread proxy checker using php,but I couldnt make it multi-thread,some days ago,I found one checker using multi-thread on github,can someone help to change it to save the good proxies into a file (ip:port format)?
https://raw.githubusercontent.com/samuel-allan/FastProxyChecker/master/checker.php
What i have tried:
original line 91:
echo json_encode($arr);
changed to:
$json = json_decode($arr);
$good_proxie = $json['arr']['result']['proxy']['ip'];
echo "$good_proxie";
I did not checked it, but think it'll work ^_^
function CheckMultiProxy($proxies, $timeout, $proxy_type)
{
$data = array();
foreach($proxies as $proxy)
{
$parts = explode(':', trim($proxy));
$url = strtok(curPageURL(),'?');
$data[] = $url . '?ip=' . $parts[0] . "&port=" . $parts[1] . "&timeout=" . $timeout . "&proxy_type=" . $proxy_type;
}
$results = multiRequest($data);
$holder = array();
foreach($results as $result)
{
$holder[] = json_decode($result, true)["result"];
}
$arr = array("results" => $holder);
foreach ($arr['results'] as $proxy) {
if ($proxy['success']) {
file_put_contents('YOUR_FILE_HERE', $proxy['proxy']['ip'].':'.$proxy['proxy']['port'].' '.$proxy['proxy']['type'].PHP_EOL, FILE_APPEND);
}
}
echo json_encode($arr);
}

Loading CSV from remote file

I am writing the following code from a CSV to get stock data, When I have downloaded the string it is splitting it in the following way
<COMPANY NAME>,<STOCK PRICE>,<STOCK CHANGE>
<COMPANY2 NAME>,<STOCK PRICE2>,<STOCK CHANGE2>
I have tried to split the array by using the /n character using the PHP function explode. However this did not split it properly. Here is my code:
public function getQuotes()
{
$result = array();
$format = $this->format;
$stockString = "";
foreach ($this->stocks as $stock)
{
$stockString = $stockString . $stock . "+";
}
//Remove the last "+"
$stockString = substr($stockString,0,strlen($stockString)-1);
$s = file_get_contents("http://finance.yahoo.com/d/quotes.csv?s=". $stockString . "&f=" . $format . "&e=.csv");
//The splitting is to be done here.
}
}
Thank you in advance
Use file function instead of file_get_contents - it will split the content for you, as php manual says:
Returns the file in an array. Each element of the array corresponds to
a line in the file, with the newline still attached. Upon failure,
file() returns FALSE.
Then you can use str_getcsv for each element of array to get field values.
public function getQuotes()
{
$result = array();
$format = $this->format;
$stockString = "";
foreach ($this->stocks as $stock)
{
$stockString = $stockString . $stock . "+";
}
//Remove the last "+"
$stockString = substr($stockString,0,strlen($stockString)-1);
$s = file("http://finance.yahoo.com/d/quotes.csv?s=". $stockString . "&f=" . $format . "&e=.csv");
foreach($s as $line) {
$values = str_getcsv($line);
}
}

Array to string conversion issue works before submitting form but not after

Hello I am having trouble with array to string conversion in the following script.
The problem is on the top if statement. For some reason it works fine in the else statement underneath but I keep getting a array to string conversion error when submitting my form and I'm not sure why it works before submitting the form but not after.
The problem is apparently in the first
$content[] = array.
It just returns the word array for $video variable. So where it states the videoid in data-videoID="" it just comes up as data-videoId="Array". Everything esle comes up fine. This would be on line 12 and 13. Here is the code I am using. Im sure it probably something simple that I am overlooking because I am in no way as experienced with php as I probably should be. I have looked over other posts and have tried taking off the brackets and several other things but still cant figure it out. Like I said it works fine if I take out the if statement and just use the else statement. Here is the code.
if(isset($_GET['q'])) {
$keyword_q = $_GET['q'];
$keyword = preg_replace('/\s+/', '/', $keyword_q);
$file = file_get_contents("https://www.googleapis.com/youtube/v3/search?videoEmbeddable=true&videoType=any&part=id%2Csnippet&q=$keyword&videoSyndicated=true&type=video&maxResults=50&key=$key");
$decoded = json_decode($file, true);
$entries = $decoded['items'];
if (!empty($entries)) {
for($i=0; $i<count($entries); $i++) {
$thumb = $entries[$i]['snippet']['thumbnails']['medium']['url'];
$videotitle = $entries[$i]['snippet']['title'];
$videodescription = $entries[$i]['snippet']['description'];
$video = $entries[$i]['id'];
$content[] = "<li><a href='#' data-videoID='$video' class='video-link'><img src='$thumb' alt='Play Video'></a><h3 class='video-title'>$videotitle</h3><p class='video-description'>$videodescription</p></li>";
}
}
$videos = "";
if (!empty($content)) {
foreach($content as $thumb){
$videos .= "$thumb";
}
}
}
else {
$filefeatured = file_get_contents("https://www.googleapis.com/youtube/v3/videos?chart=mostPopular&part=snippet&maxResults=30&key=$key");
$decoded = json_decode($filefeatured, true);
$entries = $decoded['items'];
if (!empty($entries)) {
for($i=0; $i<count($entries); $i++) {
$thumb = $entries[$i]['snippet']['thumbnails']['medium']['url'];
$videotitle = $entries[$i]['snippet']['title'];
$videodescription = $entries[$i]['snippet']['description'];
$video = $entries[$i]['id'];
$content[]= "<li><a href='#' data-videoID='$video' class='video-link'><img src='$thumb' alt='Play Video'></a><h3 class='video-title'>$videotitle</h3><p class='video-description'>$videodescription</p></li>";
}
}
$videos = "";
if (!empty($content)) {
foreach($content as $thumb){
$videos .= "$thumb";
}
}
}

Php simple algorithm for autoloader

Here's my "simple" algorithm:
if the class is named like 'AaaBbbCccDddEeeFff' loop like this:
include/aaa/bbb/ccc/ddd/eee/fff.php
include/aaa/bbb/ccc/ddd/eee_fff.php
include/aaa/bbb/ccc/ddd_eee_fff.php
include/aaa/bbb/ccc_ddd_eee_fff.php
include/aaa/bbb_ccc_ddd_eee_fff.php
include/aaa_bbb_ccc_ddd_eee_fff.php
if still nothing found, try to look if those files exist:
include/aaa/bbb/ccc/ddd/eee/fff/base.php
include/aaa/bbb/ccc/ddd/eee/base.php
include/aaa/bbb/ccc/ddd/base.php
include/aaa/bbb/ccc/base.php
include/aaa/bbb/base.php
include/aaa/base.php
include/base.php
If still not found then error.
I'm looking for a fast and easy way to convert this:
'AaaBbbCccDddEeeFff'
to this:
include/aaa/bbb/ccc/ddd/eee/fff.php
and then and easy way to remove latest folder (I guess I should look for explode()).
Any idea how to do this? (I'm not asking for the whole code, I'm not lazy).
Since you specifically asked not to have the whole code, here is some code to get you started. This takes the input and divides it into chunks delineated by changes in case. The rest you can work out as an exercise.
<?php
$input = "AaaBbbCccDddEeeFff";
$str_so_far = "";
$last_was_upper = 0;
$chunks = array();
while($next_letter = substr($input,0,1)) {
$is_upper = (strtoupper($next_letter)==$next_letter);
if($str_so_far && $is_upper && !$last_was_upper) {
$chunks[] = $str_so_far;
$str_so_far = "";
}
if($str_so_far && !$is_upper && $last_was_upper) {
$chunks[] = $str_so_far;
$str_so_far = "";
}
$str_so_far .= $next_letter;
$input = substr($input,1);
$last_was_upper = $is_upper;
}
var_dump($chunks);
?>
I think a regular expression would work. Something like preg_match_all('[A-Z][a-z][a-z]'
, $string); might work - that would match a capital letter, followed by a lowercase letter, and another lowercase letter.
As the other answers are regex, here's a non-regex way for completeness:
function transform($str){
$arr = array();
$part = '';
for($i=0; $i<strlen($str); $i++){
$char = substr($str, $i, 1);
if(ctype_upper($char) && $i > 0){
$arr[] = $part;
$part = '';
}
$part .= $char;
}
$arr[] = $part;
return 'include/' . strtolower(implode('/', $arr)) . '.php';
}
echo transform('AaaBbbCccDddEeeFff');
// include/aaa/bbb/ccc/ddd/eee/fff.php
This builds an array of the folders, so you can manipulate it as needed, for example remove a folder by unsetting the desired index, before it gets imploded.
Here is the first part of your algorithm:
AaaBbbCccDddEeeFff -> include/aaa/bbb/ccc/ddd/eee/fff.php
include/aaa/bbb/ccc/ddd/eee_fff.php
include/aaa/bbb/ccc/ddd_eee_fff.php
include/aaa/bbb/ccc_ddd_eee_fff.php
include/aaa/bbb_ccc_ddd_eee_fff.php
include/aaa_bbb_ccc_ddd_eee_fff.php
I think you can do last part independently based on my answer.
<?php
function convertClassToPath($class) {
return strtolower(preg_replace('/([a-z])([A-Z])/', '$1' . DIRECTORY_SEPARATOR . '$2', $class)) . '.php';
}
function autoload($path) {
$base_dir = 'include' . DIRECTORY_SEPARATOR;
$real_path = $base_dir . $path;
var_dump('Checking: ' . $real_path);
if (file_exists($real_path) === true) {
var_dump('Status: Success');
include $real_path;
} else {
var_dump('Status: Fail');
$last_separator_pos = strrpos($path, DIRECTORY_SEPARATOR);
if ($last_separator_pos === false) {
return;
} else {
$path = substr_replace($path, '_', $last_separator_pos, 1);
autoload($path);
}
}
}
$class = 'AaaBbbCccDddEeeFff';
var_dump(autoload(convertClassToPath($class)));

Categories