I've recently inherited a large PHP application with NO objects/modules/namespaces...only a lot of files containing functions.
Of course, there is a LOT of dependencies (and all files and almost always included).
I'm looking for a tool that could analyse the files and generate a dependencies graph. It would then be easier to detect independent files/set of files and re-factor the whole thing.
So far the best solution I've found would be to write a CodeSniffer sniff to detect all functions calls and then use that to generate the graph.
It seems something useful for other, so I'm sure tools already exists for it.
What would you recommend ?
I think that the best solution is use a doc generat + grapviz, PHPDocumentor looks to have a Grapviz extension at https://github.com/phpDocumentor/GraphViz
This is a example made with PHPDocumentor:
http://demo.phpdoc.org/Clean/graphs/classes.svg
Too you can use a hierarchical profiler like xhprof (https://github.com/facebook/xhprof), this can draw a tree of all call to functions from a execution.
A example form xhprof draw done by Graphviz
I could recommend a lightweight project I wrote few days ago. Basically I had a 300+ files PHP project and I wanted to detect what files do these files require/include and vice-versa. Moreover, I wanted to check for each individual file what files does this file requires/includes (directly or indirectly, ie. via file inheritance) and vice-versa: what are the files that include this particular file. For any combination of these I wanted an interactive dependency graph (base on file inclusion and not on class/function calls/usage).
Check out the project's sandbox and its source code.
Note that the whole thing was written in only 2 days so don't judge it
too harsh. What's important is that it's doing its job!
Related
If I would like to distribute PHP application with installer(package system of OS) how should I proceed? I don't want PHP files to be there, just working application, so when I type 'app' into console, it ends up being launching application, without need to install PHP on system(no php installation on host required). I would also like the application to have patch-able byte-code, so it's in parts, loaded when needed and only part needs to be replaced on update.
What I would do now is following:
->Compile PHP with extensions for specific platform.
->Make binary application which launches '/full/php app' when app is launched.
->Pack it in installer in a way, that there would be binary added to path when added, launching specific installation of PHP which is alongside the app with argument of start point->App would be running.
Problem is:
Maybe I don't want my PHP files to be exposed(in application, there will be available source anyway) is there some ready made stuff to do this? Is there some better way than I proposed?
Alternative: Modifying OP Cache to work with "packing" application to deliver byte codes to modified OP Cache which just reads the cache.
My suggestion would be a tiny tool I just finished, for almost exactly the same problem. (Oh yes I tried all the others but they're old and rusty, sometimes they're stuck with 4.x syntax, have no support, have no proper documentation, etc)
So here's RapidEXE:
http://deneskellner.com/sw/rapidexe
In the classical way, it's not a really-real compiler, just a glorified packer, but does exactly what you need: the output exe will be standalone, carrying everything with it and transparently building an ad-hoc runtime environment. Don't worry, it all happens very fast.
It uses PHP 7.2 / Win64 by default but has 5.x too, for XP compatibility.
It's freeware, obviously. (MIT License.)
(Just telling this because I don't want anyone to think I'm advertising or something. I just took a few minutes to read the guidelines about own-product answers and I'm trying to stay within the Code of the Jedi here.)
However...
I would also like the application to have patch-able byte-code, so it's in parts, loaded when needed and only part needs to be replaced on update.
It's easier to recompile the exe. You can extract the payload pieces of course but the source pack is one big zip; there seems to be no real advantage of handling it separately. Recompiling a project is just one command.
Maybe I don't want my PHP files to be exposed(in application, there will be available source anyway)
In this case, the exe contains your source compressed but eventually they get extracted into a temp folder. They're deleted immediately after run but, well, this is no protection whatsoever. Obfuscation seems to be the only viable option.
If something goes wrong, feel free to comment or drop me a line on developer-at-deneskellner-dot-com. (I mean, I just finished it, it's brand new, it may misbehave so consider it something like a beta for now.)
Happy compiling!
PHP doesn't do that natively, but here are a few ideas:
Self-extracting archive
Many archival programs allow you to create a self-extracting archive and some even allow to run a program after extraction. Configure it so that it extracts php.exe and all your code to a temp folder and then runs ir from there; deleting after the script has complete.
Transpilers/compilers
There's the old HPHC which translates PHP code to C++, and its wikipedia age also contains links to other, similar projects. Perhaps you can take advantage of those.
Modified PHP
PHP itself is opensource. You should be able to modify it withot too much difficulty to take the source code from another location, like some resource compiled directly inside the php.exe.
Use Zend Guard tool that compiles and converts the plain-text PHP scripts into a platform-independent binary format known as a 'Zend Intermediate Code' file. These encoded binary files can then be distributed instead of the plain text PHP. Zend Guard loaders are available for Windows and Linux platform that enables PHP to run the scripts encoded by Zend Guard.
Refer to http://www.zend.com/en/products/zend-guard
I would like to add another answer for anyone who might be Googling for answers.
Peach Pie compiler/runtime
There is an alternative method to run (and build apps from) .php source codes, without using the standard php.exe runtime. The solution is based on C#/.NET and is actually able to compile php source files to .NET bytecode.
This allows you to distribute your program without exposing its source code.
You can learn more about the project at:
https://www.peachpie.io/
You've got 3 overlapping questions.
1. Can I create a stand-alone executable from a PHP application?
Answered in this question. TL;DR: yes, but it's tricky, and many of the tools you might use are semi-abandoned.
2. Can I package my executable for distribution on client machines?
Yes, though it depends on how you answer question 1. If you use the .Net compiler, your options are different to the C++ option.
3. Can I protect my source code once I've created the application?
Again, depends on how you answer question 1. Many compilers include an "obfuscator" option which makes it hard to make sense of any information you get from decompiling the app. However, a determined attacker can probably get through that (this is why software piracy is possible).
I knew that PHP is able to read file content by different ways, for example: fread, file_get_contents, file, readfile, etc.
Currently, I am looking for an API that can read real index of files and folders in specific partition or folder, for example:
drive d:\ in windows contains three folders (folder1, folder2, folder3), and each folder contains some files, we can get these directory structure using PHP (opendir, scandir, readdir, etc) and list them as I want, however, windows saved file and folder names inside hard-disk with their attributes (size, last modified, created on, etc).
How I can read hard-disc using PHP and retrieving all file and folder attributes for a specific path?
for instance, if we consider last modified time we can use (filemtime()) function, but this attribute not saved inside the file, its saved some where else inside hard-drive, other attributes also saved in other location not inside the file.
When windows user copying file from flash-drive to local hard, windows will copy all file and folder attributes and saves them inside local hard drive. When using PHP for copying file, it depends on OS to handle this job, its not native support (as I think) for file and folder operations.
Do you have any idea?
There are many recovery program that uses this technology for reading hard-drive indexes, however, for PHP: I cant find any source for this problem.
Applications if I get correct answer:
I can check if such file securely deleted from my hard-drive? I can create secure delete application using PHP, or clearing hard-drive indexes for a given file.
Your help appreciated.
Problems with the proposition
The attributes of files, such as timestamps, permission flags etc, are stored in the file system (FAT, NTFS, Ext3 etc). As you say some of them can be read using PHPs different file and directory methods, but they all act through the OS file system abstraction and cant have access to block level information on the disk, such as what precise byte on disk stores the archive flag for file X. The whole point of the OS and FS is to abstract away this information from the user/client programs.
As suggested there are external tools, written in c or similar, that does have this access and that you can call from inside PHP. If you want a 'native' PHP way of doing this you'll have to compile a c extension for PHP that exposes these low level functions to you.
I'd say external tools is the way to go if you want to stick with PHP but for the task at hand, as far as we can see from your description, I'd go with another language that has more low level access. Like C or C++. PHP is a high level language for HTML pre processing and as such is a poor choice for low level system programming.
Practical advice
After looking through the PHP documentation and assorted third party libraries:
An of the shelf solution for reading file system information on a file allocation table level doesn't exist for PHP. The lowest level you get is the fstat() function, and that is not very far for what you want.
External tools
No mater exactly what you want to do there is probably a small binary that does it. PHP can be integrated with these programs, as suggested elsewhere, via the exec() function. This is probably the easiest approach for you unless you have serious amounts of time and/or development resources to devote to this problem.
Wrapping a library
There are libraries that solves this problem for you, written in low level languages. An open source library can be wrapped with SWIG to expose it to PHP. This will give you access to the low level methods you need, but it's a non trivial task. These kind of libraries also often require sole access to the device while they work on it, something that is difficult to achieve in most normal operating environments.
Note also that you will probably need a library per file system. Microsofts VFAT extension to FAT12/16/32 requiers a licens to use. So if you want to work with FAT and have files with long names (not 8.3 format) you'll have to fork up some dough to be legit.
Low level implementation
A last middle ground would be to write your own CLI tool that uses an external library to access the low level FS functions. You can then use exec() from inside PHP to interact with your own implementation.
This might be a reasonable path if you cant find an existing tool that solves your problem and you are not willing to spend the time to wrap a library.
In closing
You give a very narrow problem description with little to go on as for what the application is about. A broader discussion (in another forum) might yield better results since the problem might be better solved in another way entirely.
I found something on PHP.net which appears to do what you want:
http://php.net/manual/en/function.readdir.php#103418
Edit: I mis-understood the question. Attributes such as the last modified time, last accessed date and the like are stored in the file systems master file table. As far as I can tell, this isn't accessible with PHP, and if you were to write your own method to do this then you'd also have to account for different file systems as they all handle the storage of these attributes in their own unique way.
It could be that to get all of the information you're looking for is not possible with PHP without writing some form of extension to PHP itself.
Edit 2: Upon researching a little more...
http://php.net/manual/en/function.fileinode.php
This function could be an interesting one to look at.
Well if I understand correctly you just want to securely delete a file. You can just call [shred][1]
[1]: http://linux.die.net/man/1/shred via system or exec if you are on linux and you are good to go
Problem
I have a legacy codebase I need to analyze and determine dependencies. Particularly the dependencies on classes (internal/external) and extensions (Memcache, PDO, etc).
What I've Tried
I have reviewed the tools listed in Is there a static code analyzer for PHP files?. Unfortunately, this post is dated and most of the promising tools like phpCallGraph no longer work.
My thought was to analyze the code lexically and look for class tokens. Reviewing a list of these would at least allow me to visually determine dependencies. However finding OtherClass in the following code may be complex:
$classname = 'OtherClass';
echo $classname::doubleColon();
In the end, I doubt I'm the first to need this. I'm sure a tool or combination of tools exist to provide what I need. So I'm asking the SO community before writing this myself.
Update
Ideally this tool will analyze multiple files with complete code coverage. As such, tools like Xdebug, while great, are not solutions to this exact problem.
Instead of phpCallGraph you could use Gopal Vijayaraghavan's inclued extension which in combination with Graphviz gives you a nice looking graph of all included files for a certain execution path.
Example:
Moreover, I'd recommend Xdebug (a PHP debugger) which offers a profiler that outputs data consumable by Valgrind. Use the debugger with a compatible IDE to follow the execution path (which helped me a lot to wade thru e.g. Drupal's massive call-stack).
Combine both and you should get a fairly thourough overview.
EDIT
Searched the web and found nWire for PHP - an eclipse plugin that looks like it could be the right tool for you (30 day free trial which should be enough to give you a head start).
I think PhpCodeAnalyzer is exactly what you're looking for - https://github.com/wapmorgan/PhpCodeAnalyzer
It print list of all used external extensions in code base.
Is there any tool out there which could tell the useless files in the code base?
We have a big code base (PHP, HTML, CSS, JS files) and I want to be able to remove the not needed files. Any help would be appreciated.
I'm guessing deleting files and running your phpunit tests is a none starter.
If your files are not already in a version-control system - add them. Having the files in a version control system (such as svn or git) is crucial to allow you to recover from deleting any files that you thought were not being used but you later find out were.
Then, you can delete anything you think may not be being used, and if it doesn't affect the running of your application you can conclude that the files aren't used. If adverse effects show up - you can restore them from your repository with ease.
The above is most appropriate (probably) for frontend files (css, js, images). Any files you delete that are requested will show up in your webserver error log giving you a quick reference for files that nolonger exist that you need to restore.
For your php files, that's quite a bit more tricky, How did you arrive at a position where you have php files which you aren't using? Anyway you could for example:
Use xdebug
Enable profiling
Use append mode (one profile)
Use all the functions of your application
and you would then have a profile which includes all files you loaded. Scanning the generated profile for each php file in your codebase will give you some indication of which files you didn't use.
If you are only looking for unused files, don't be tempted to use code coverage analysis - it is very intensive and not the level of detail you're asking for.
A slightly less risky way would be to log whenever a file is loaded. e.g. put this at line one of each file:
<?php file_put_contents('/some/location/fileaccess.log', __FILE__, FILE_APPEND); ?>
and simply leave your application to be used for a while (days, weeks). Thereafter just scan that log, for any file that is named - remove the above line of code. For any that are not - delete (preferably after looking for the filename in your whole sourcecode and confirming it's nowhere).
OR: you could use a shutdown function which dumps the response of get_included_files() to a log file. This would allow you to achieve the same without editing all php files in your source tree.
Caveat: Be careful deleting your php files. Whereas a missing css/js/image will probably mean your application still works, a missing php file of course will have rather more impact :).
If it is in Git why not delete the local file and then do a git rm <file name> to remove it from that branch.
Agree with everything said by #AD7six.
What you might like to try with PHP is to log the use of the files in someway (logging to flat file or database).
This technique does not have to be in place for long you can do it with an include and require_once at the top of each file.
That technique also works for javascript functions you can just print to the console each function, and then unit test your site. You can probably clean out a lot of redundant code that way.
The rest is not so easy, but version tracking is the way to go.
This concept is a new one for me -- I first came across it at the YUI dependency configurator. Basically, instead of having multiple requests for many files, the files are chained into one http request to cut down on page load time.
Anyone know how to implement this on a LAMP stack? (I saw a similar question was asked already, but it seems to be ASP specific.
Thanks!
Update: Both answers are helpful...(my rep isn't high enough to comment yet so I'm adding some parting thoughts here). I also came across another blog post with PHP-specific examples that might be useful. David's build answer, though, is making me consider a different approach. Thanks, David!
There are various ways, the two most obvious would be:
Build a tool like YUI which builds a bespoke, unique version based on the components you ticked as required so that you can still serve the file as static. MooTools and jQuery UI all provide package-builders like this when you download their package to give you the most streamlined and effecient library possible. I'm sure a generic all purpose tool exists out there.
Create a simple Perl/PHP/Python/Ruby script that serves a bunch of JavaScript files based on the request. So "onerequest.js?load=ui&load=effects" would go to a PHP script that loads in the files and serves them with the correct content-type. There are many examples of this but personally I'm not a fan.
I prefer not to serve static files through any sort of script, but I also like to develop my code with 10 or so seperate small class files without the cost of 10 HTTP requests. So I came up with a custom build process that combines all the most common classes and functions and then minifies them into a single file like project.min.js and have a condition in all my views/templates that includes this file on production.
Edit - The "custom build process" is actually an extremely simple perl script. It reads in each of the files that I've passed as arguments and writes them to a new file, optionally passing the entire thing through JSMIN (available in all your favourite languages) automatically.
At the command like it looks like:
perl build-project-master.pl core.js class1.js etc.js /path/to/live/js/file.js
There is a good blog post on this # http://www.hunlock.com/blogs/Supercharged_Javascript.
What you want is Minify. I just wrote a walkthrough for setting it up.
Capistrano is a fairly popular Ruby-based web deployment tool. If you're considering it or already using it, there's a great gem that will figure out CSS and Javascript dependencies, merge, and minify the files.
gem install juicer
From the Juicer GitHub page, it can figure out which files depend on each other and merge them together, reducing the number of http requests per page view, thus improving performance.