Check Files In Directory Are Modified - php

I need to deploy a PHP application written by CodeIgniter to client's web server (CentOS 5 or 6). As PHP is the scripting language, it does not need to compile to binary code for deployment. It has chances that client will modify the PHP program by themselves without a notice to me. If client has modified the program that made the application out of order, we need to take extra man power to find their modification and fix it.
So I would like to made something that can easy to let me know any files (php, css, html, etc.) of the application has been modification after my deployment. Is there any method suggested by anyone?
Thank you,

Use filemtime()
int filemtime ( string $filename )
This PHP function returns the time when the data blocks of a file were being written to, that is, the time when the content of the file was changed.
<?php
// outputs e.g. somefile.txt was last modified: December 12 2014 09:16:23.
$filename = 'somefile.txt';
if (file_exists($filename)) {
echo "$filename was last modified: " . date ("F d Y H:i:s.", filemtime($filename));
}
?>
To get the last modification time of a directory, you can use this:
<pre>
$getLastModDir = filemtime("/path/to/directory/.");
</pre>
Take note on the last dot which is needed to see the directory as a file and to actually get a last modification date of it.
This comes in handy when you want just one 'last updated' message on the frontpage of your website and still taking all files of your website into account.
To get the modification date of some remote file, you can use the fine function by notepad at codewalker dot com (with improvements by dma05 at web dot de and madsen at lillesvin dot net).
But you can achieve the same result more easily now with stream_get_meta_data (PHP>4.3.0).
However a problem may arise if some redirection occurs. In such a case, the server HTTP response contains no Last-Modified header, but there is a Location header indicating where to find the file. The function below takes care of any redirections, even multiple redirections, so that you reach the real file of which you want the last modification date.
<?php
// get remote file last modification date (returns unix timestamp)
function GetRemoteLastModified( $uri )
{
// default
$unixtime = 0;
$fp = fopen( $uri, "r" );
if( !$fp ) {return;}
$MetaData = stream_get_meta_data( $fp );
foreach( $MetaData['wrapper_data'] as $response )
{
// case: redirection
if( substr( strtolower($response), 0, 10 ) == 'location: ' )
{
$newUri = substr( $response, 10 );
fclose( $fp );
return GetRemoteLastModified( $newUri );
}
// case: last-modified
elseif( substr( strtolower($response), 0, 15 ) == 'last-modified: ' )
{
$unixtime = strtotime( substr($response, 15) );
break;
}
}
fclose( $fp );
return $unixtime;
}
?>

Related

edit to substr and strpos causes Error 500

I'm trying to make some edits to a piece of code but I get an error 500 when I do.
For this example, lets say that
basename(__FILE__)
is my_filename.php
The code is:
$HTTP_GET_VARS['feed_file'] = $_GET['feed_file']
= substr(
basename(__FILE__),
3,
strpos( basename(__FILE__), '.php') -3
);
$HTTP_GET_VARS['feed_file'] would echo as "filename"
Now, if
basename(__FILE__)
is aaa_filename.php
the original code would give $HTTP_GET_VARS['feed_file'] as "_filename"
I changed the code to
$HTTP_GET_VARS['export_feed'] = $_GET['export_feed']
= substr(
basename(__FILE__),
4,
strpos( basename(__FILE__), '.php') -3
);
$HTTP_GET_VARS['export_feed']now echos as "filename."
Ok, so I need to lose one more character from the end of the string. I change the -3 to -4 so that I have
$HTTP_GET_VARS['export_feed'] = $_GET['export_feed']
= substr(
basename(__FILE__),
4,
strpos( basename(__FILE__), '.php') -4
);
Only now the Error 500 is thrown.
Confusing the hell out of me as I thought it was going to be a simple change. Any suggestions on why I'm having problems simply changing the number of chars to drop from the beginning and end of a string?
I would use preg_match to dynamically get "filename":
$basename = 'ab_filename.php';
if(preg_match('/_(.+)\.php$/',$basename,$matches)):
$name = $matches[1]; // 'filename'
endif;
Now $name is just "filename"
Live demo
That aside, the code snippet you shared by itself would not cause a 500 server error. There is probably a domino effect and the error gets triggered elsewhere. Learn how to look up error logs for details.
Finally, if you'll keep using your current approach, don't hardcode the offset (your -3 or -4). Rather compute it dynamically like so:
$basename = 'abcd_filename.php';
$pos_ = strpos($basename,'_') + 1;
$len = strpos( $basename, '.php') - $pos_;
$name = substr($basename, $pos_, $len);
Now $name is "filename"

How can I get the time of a remote server?

I am currently working on a websever that will create a quick diagnose dashboard of other servers.
My requirement is to display the time of these remote servers, it seems that the NTP create some issues and I would like to see that.
I currently have a bat file on my desktop that simply send
net time \\SRV*******
I have also tried:
echo exec('net time \\\\SRV****');
=> result is '0'
But I would like to find a better solution in PHP so anybody of a team can read it on a webpage.
Any idea what I would do?
Note: this is not related to How to get date and time from server ad I want to get the time of a REMOTE server and not local server
You can use NTP protocol to retrieve datetime from a remote server.
Try this code:
error_reporting(E_ALL ^ E_NOTICE);
ini_set("display_errors", 1);
date_default_timezone_set("Europe/...");
function query_time_server ($timeserver, $socket)
{
$fp = fsockopen($timeserver,$socket,$err,$errstr,5);
# parameters: server, socket, error code, error text, timeout
if ($fp) {
fputs($fp, "\n");
$timevalue = fread($fp, 49);
fclose($fp); # close the connection
} else {
$timevalue = " ";
}
$ret = array();
$ret[] = $timevalue;
$ret[] = $err; # error code
$ret[] = $errstr; # error text
return($ret);
}
$timeserver = "10.10.10.10"; #server IP or host
$timercvd = query_time_server($timeserver, 37);
//if no error from query_time_server
if (!$timercvd[1]) {
$timevalue = bin2hex($timercvd[0]);
$timevalue = abs(HexDec('7fffffff') - HexDec($timevalue) - HexDec('7fffffff'));
$tmestamp = $timevalue - 2208988800; # convert to UNIX epoch time stamp
$datum = date("Y-m-d (D) H:i:s",$tmestamp - date("Z",$tmestamp)); /* incl time zone offset */
$doy = (date("z",$tmestamp)+1);
echo "Time check from time server ",$timeserver," : [<font color=\"red\">",$timevalue,"</font>]";
echo " (seconds since 1900-01-01 00:00.00).<br>\n";
echo "The current date and universal time is ",$datum," UTC. ";
echo "It is day ",$doy," of this year.<br>\n";
echo "The unix epoch time stamp is $tmestamp.<br>\n";
echo date("d/m/Y H:i:s", $tmestamp);
} else {
echo "Unfortunately, the time server $timeserver could not be reached at this time. ";
echo "$timercvd[1] $timercvd[2].<br>\n";
}
You can use a powershell script along with WMI (assuming your servers are windows machines)
This will be way faster than the old net time style.
The script needs to be executed with a Domain-Admin (or any other user that has WMI-query permissions)
$servers = 'dc1', 'dc2', 'dhcp1', 'dhcp2', 'wsus', 'web', 'file', 'hyperv1', 'hyperv2'
ForEach ($server in $servers) {
$time = "---"
$time = ([WMI]'').ConvertToDateTime((gwmi win32_operatingsystem -computername $server).LocalDateTime)
$server + ', ' + $time
}
You can achieve this with a scheduled task every 5 minutes.
If you want to display the time "with seconds / minutes" (I wouldn't query each server too often, as time does not change THAT fast), you could add a little maths:
When executing the script, don't store the actual time, but just the offset of each server, compared to your webservers time. (i.e. +2.2112, -15.213)
When your users are visiting the "dashboard" load these offsets and display the webservers current time +/- the offset per server. (Could be "outdated" by some microseconds then, but do you care?)
Why so complicated?
Just found that this is working:
echo exec('net time \\\\SRV****, $output);
echo '<pre>';
print_r($output);
echo '</pre>';

include files on date in directory

Allright, I have a overview of sites where I was/am working on. And for every site I have a php file. And in that php file I use this code to get the newest and oldest date of a file what is note a picture in the directory.
$date = "test/*.*";
$files = array_filter(glob($date), function($file) {
$ext = substr($file, strrpos($file, '.'));
return !in_array($ext, array('.jpg', '.bit', '.png', '.jpeg', '.gif', '.bmp'));
});
$latest = count($files)-1 ;
array_multisort(
array_map( 'filemtime', $files ),
SORT_NUMERIC,
SORT_ASC,
$files
);
$newestfile = date ("d F Y ", filemtime($files[0]));
$oldestfile = date ("d F Y ", filemtime($files[$latest]));
if($newestfile == $oldestfile) {
echo date ("d F Y ", filemtime($files[0]));
} else {
echo date ("d F Y ", filemtime($files[0]));
echo " - " ;
echo date ("d F Y .", filemtime($files[$latest]));
}
the output of this code would be like: 16 January 2013 - 25 October 2013 .
In my overview page I use a code to include all the php files (of the websites I've made) to that page. (btw. the php files are not big pages. just with a picture and a bit of text.)
$listy = glob("sites/*.php");
print_r ($listy) ;
array_multisort(
array_map( 'filemtime', $listy ),
SORT_NUMERIC,
SORT_DESC,
$listy
);
if (empty($listy)) {
include('includes/emptycontent.php');
} else {
foreach ($listy as $filename) {
include $filename;
}
}
the output of this array would be like:
Array (
[0] => sites/test.php
[1] => sites/test2.php
[2] => sites/test3.php
So far so good, no problems in that.
Now, I want to sort the included files not on time of the included file, like I did in the code above. But i want it to sort on the latest date of the files in the directory like in the php file. so actually I want to combine those to codes to one.
so I have a php file called test. and I want the date of the latest file in the directory also called test. Those have always the same name.
What I thought was to use the output of the second code and then get rid of the "sites/" and the ".php". Those names must be in a array I think. and then for each name get the newestfile and sort them from newest to oldest.
I think like this I get the sites on which I was working recently at the top and the older ones at the bottom of the page.
Maybe my approach is totally wrong but I have no idea how to do that in code.
Look at this bit:
array_map( 'filemtime', $listy ),
Here, you’re effectively converting a list of files to a list of modification dates. How about converting that into another function. One that finds a most recent file inside a directory:
array_map(function ($filename) {
// $filename = sites/test1.php
$dir = substr($filename, strlen("sites/"), - strlen(".php")); // cut those!
// or:
$dir = basename($filename, '.php');
// put the code listing files inside $dir
// then sort it (you did it in the first part)
// and then `return` the most or the least recent one
}, $listy),
HTH.

How do you get last some lines of file via SFTP in PHP

I need to login to a production server retrieve a file and update my data base with the data in this file. Since this is a production database, I don't want to get the whole file every 5 minutes since the file may be huge and this may impact the server. I need to get the last 30 lines of this file every 5 minutes interval and have as little impact as possible.
The following is my current code, I would appreciate any insight to how best accomplish this:
<?php
$user="id";
$pass="passed";
$c = curl_init("sftp://$user:$pass#server1.example.net/opt/vmstat_server1");
curl_setopt($c, CURLOPT_PROTOCOLS, CURLPROTO_SFTP);
curl_setopt($c, CURLOPT_RETURNTRANSFER, true);
$data = curl_exec($c);
curl_close($c);
$data = explode("\n", $data);
?>
Marc B is wrong. SFTP is perfectly capable of partial file transfers. Here's an example of how to do what you want with phpseclib, a pure PHP SFTP implementation:
<?php
include('Net/SFTP.php');
$sftp = new Net_SFTP('www.domain.tld');
if (!$sftp->login('username', 'password')) {
exit('Login Failed');
}
$size = $sftp->size('filename.remote');
// outputs the last ten bytes of filename.remote
echo $sftp->get('filename.remote', false, $size - 10);
?>
In fact I'd recommend an approach like this anyway since some SFTP servers don't let you run commands via the system shell. Plus, SFTP can work on Windows SFTP servers whereas tail is unlikely to do so even if you do have shell access. ie. overall, it's a lot more portable a solution.
If you want to get the last x lines of a file you could loop repeatedly, reading however many bytes each time, until you encounter 10x new line characters. ie. get the last 10 bytes, then the next to last 10 bytes, then the ten bytes before those ten bytes, etc.
An answer by #Sammitch to a duplicate question Get last 15 lines from a large file in SFTP with phpseclib:
The following should result in a blob of text with at least 15 lines from the end of the file that you can then process further with your existing logic. You may want to tweak some of the logic depending on if your file ends with a trailing newline, etc.
$filename = './file.txt'
$filesize = $sftp->size($filename);
$buffersize = 4096;
$offset = $filesize; // start at the end
$result = '';
$lines = 0;
while( $offset > 0 && $lines < 15 ) {
// work backwards
if( $offset < $buffersize ) {
$offset = 0;
} else {
$offset -= $buffer_size;
}
$buffer = $sftp->get($filename, false, $offset, $buffer_size));
// count the number of newlines as we go
$lines += substr_count($buffer, "\n");
$result = $buffer . $result;
}
SFTP is not capable of partial file transfers. You might have better luck using a fullblowin SSH connection and use a remote 'tail' operation to get the last lines of the file, e.g.
$lines = shell_exec("ssh user#remote.host 'tail -30 the_file'");
Of course, you might want to have something a little more robust that can handle things like net.glitches that prevent ssh from getting through, but as a basic starting point, this should do the trick.

Problem reading files greater than 1GB with XMLReader

Is there a maximum file size the XMLReader can handle?
I'm trying to process an XML feed about 3GB large. There are certainly no PHP errors as the script runs fine and successfully loads to the database after it's been run.
The script also runs fine with smaller test feeds - 1GB and below. However, when processing larger feeds the script stops reading the XML File after about 1GB and continues running the rest of the script.
Has anybody experienced a similar problem? and if so how did you work around it?
Thanks in advance.
I had same kind of problem recently and I thought to share my experience.
It seems that problem is in the way PHP was compiled, whether it was compiled with support for 64bit file sizes/offsets or only with 32bit.
With 32bits you can only address 4GB of data. You can find a bit confusing but good explanation here: http://blog.mayflower.de/archives/131-Handling-large-files-without-PHP.html
I had to split my files with Perl utility xml_split which you can find here: http://search.cpan.org/~mirod/XML-Twig/tools/xml_split/xml_split
I used it to split my huge XML file into manageable chunks. The good thing about the tool is that it splits XML files over whole elements. Unfortunately its not very fast.
I needed to do this one time only and it suited my needs, but I wouldn't recommend it repetitive use. After splitting I used XMLReader on smaller files of about 1GB in size.
Splitting up the file will definitely help. Other things to try...
adjust the memory_limit variable in php.ini. http://php.net/manual/en/ini.core.php
rewrite your parser using SAX -- http://php.net/manual/en/book.xml.php . This is a stream-oriented parser that doesn't need to parse the whole tree. Much more memory-efficient but slightly harder to program.
Depending on your OS, there might also be a 2gb limit on the RAM chunk that you can allocate. Very possible if you're running on a 32-bit OS.
It should be noted that PHP in general has a max file size. PHP does not allow for unsigned integers, or long integers, meaning you're capped at 2^31 (or 2^63 for 64 bit systems) for integers. This is important because PHP uses an integer for the file pointer (your position in the file as you read through), meaning it cannot process a file larger than 2^31 bytes in size.
However, this should be more than 1 gigabyte. I ran into issues with two gigabytes (as expected, since 2^31 is roughly 2 billion).
I've run into a similar issue when parsing large documents. What I wound up doing is breaking the feed into smaller chunks using filesystem functions, then parsing those smaller chunks... So if you have a bunch of <record> tags that you are parsing, parse them out with string functions as a stream, and when you get a full record in the buffer, parse that using the xml functions... It sucks, but it works quite well (and is very memory efficient, since you only have at most 1 record in memory at any one time)...
Do you get any errors with
libxml_use_internal_errors(true);
libxml_clear_errors();
// your parser stuff here....
$r = new XMLReader(...);
// ....
foreach( libxml_get_errors() as $err ) {
printf(". %d %s\n", $err->code, $err->message);
}
when the parser stops prematurely?
Using WindowsXP, NTFS as filesystem and php 5.3.2 there was no problem with this test script
<?php
define('SOURCEPATH', 'd:/test.xml');
if ( 0 ) {
build();
}
else {
echo 'filesize: ', number_format(filesize(SOURCEPATH)), "\n";
timing('read');
}
function timing($fn) {
$start = new DateTime();
echo 'start: ', $start->format('Y-m-d H:i:s'), "\n";
$fn();
$end = new DateTime();
echo 'end: ', $start->format('Y-m-d H:i:s'), "\n";
echo 'diff: ', $end->diff($start)->format('%I:%S'), "\n";
}
function read() {
$cnt = 0;
$r = new XMLReader;
$r->open(SOURCEPATH);
while( $r->read() ) {
if ( XMLReader::ELEMENT === $r->nodeType ) {
if ( 0===++$cnt%500000 ) {
echo '.';
}
}
}
echo "\n#elements: ", $cnt, "\n";
}
function build() {
$fp = fopen(SOURCEPATH, 'wb');
$s = '<catalogue>';
//for($i = 0; $i < 500000; $i++) {
for($i = 0; $i < 60000000; $i++) {
$s .= sprintf('<item>%010d</item>', $i);
if ( 0===$i%100000 ) {
fwrite($fp, $s);
$s = '';
echo $i/100000, ' ';
}
}
$s .= '</catalogue>';
fwrite($fp, $s);
flush($fp);
fclose($fp);
}
output:
filesize: 1,380,000,023
start: 2010-08-07 09:43:31
........................................................................................................................
#elements: 60000001
end: 2010-08-07 09:43:31
diff: 07:31
(as you can see I screwed up the output of the end-time but I don't want to run this script another 7+ minutes ;-))
Does this also work on your system?
As a side-note: The corresponding C# test application took only 41 seconds instead of 7,5 minutes. And my slow harddrive might have been the/one limiting factor in this case.
filesize: 1.380.000.023
start: 2010-08-07 09:55:24
........................................................................................................................
#elements: 60000001
end: 2010-08-07 09:56:05
diff: 00:41
and the source:
using System;
using System.IO;
using System.Xml;
namespace ConsoleApplication1
{
class SOTest
{
delegate void Foo();
const string sourcepath = #"d:\test.xml";
static void timing(Foo bar)
{
DateTime dtStart = DateTime.Now;
System.Console.WriteLine("start: " + dtStart.ToString("yyyy-MM-dd HH:mm:ss"));
bar();
DateTime dtEnd = DateTime.Now;
System.Console.WriteLine("end: " + dtEnd.ToString("yyyy-MM-dd HH:mm:ss"));
TimeSpan s = dtEnd.Subtract(dtStart);
System.Console.WriteLine("diff: {0:00}:{1:00}", s.Minutes, s.Seconds);
}
static void readTest()
{
XmlTextReader reader = new XmlTextReader(sourcepath);
int cnt = 0;
while (reader.Read())
{
if (XmlNodeType.Element == reader.NodeType)
{
if (0 == ++cnt % 500000)
{
System.Console.Write('.');
}
}
}
System.Console.WriteLine("\n#elements: " + cnt + "\n");
}
static void Main()
{
FileInfo f = new FileInfo(sourcepath);
System.Console.WriteLine("filesize: {0:N0}", f.Length);
timing(readTest);
return;
}
}
}

Categories