I am using SVN to manage a copy of my web site. The site runs a typo3 installation, which uses PHP to create the backend.
The problem is, all the stupid .SVN folders show up in the directory listing of PHP. I DO NOT mean the build in apache listing. I mean a directoy listing created by the PHP backend.
So, is there any way to hide special directories from PHP?
[NOTE]
Changing the PHP source code is not an option. Typo3 is too big, and each extensions uses its own code. Would be much more effort than an SVN export script.
Chris
PS: I do not want to setup a svn export --> web_root just to get rid of the files. And I know that I can prevent apache from serving the .SVN directories, I did that. But they still show up in the backend, when browsing the directory tree (which is created by PHP). And they are very annoying...
This is difficult, since you will have to change behavior of something somewhere between the filesystem and Typo3. You have:
Filesystem → Operating System → PHP → Typo3
The files must stay in the filesystem and must stay visible by the operating system, so you can use SVN. Changing Typo3 is not an option for you, and changing PHP has many other major undesirable consequences that you should avoid. So, what you have left is to insert something in between OS→PHP or PHP→Typo3.
The first case is actually possible, depending on what operating system you use, and if you have administrator (root) access. FUSE is part of the Linux kernel, and is also available for many other operating systems. Then, with fuse, you may install a filter like rofs-filtered, that allows you to filter which files and directories are visible in a mounted volume. You use it to create a filesystem that mirrors your SVN checkout directory, filtering the .svn directories.
So, is there any way to hide special directories from PHP?
No.
As long as the user PHP is run under has read access to the directory it will always produce all the files/directories in that directory. There is no way to hide files from certain processes, were this possible writing a root kit to hide from ls and other file system tools would be a lot easier.
The option you would want/need is a way to define files that Typo3 ignores, and have it be system wide and thus used by the extensions as well. You have specified however that you do not want to change the source code, and do not want to do svn export.
You are thus stuck with the .svn directories.
The short answer is "Not easily, simply, or sanely".
Run the website from an export of SVN, not a checkout, instead.
Try this.
<locationmatch "/.svn/">
order allow,deny
deny from all
</locationmatch>
Btw in your loop in PHP you can do a logic check to see if the filename is not ".svn", usually PHP directory tools do that to exclude "." and ".." directories.
The problem is, all the stupid .SVN
folders show up in the directory
listing of PHP. I DO NOT mean the
build in apache listing. I mean a
directoy listing created by the PHP
backend.
What application is doing the directory listing? Have you considered looking into the code of the PHP backend and adding something to prevent the display of the .svn directories?
Just find or write a very simple application that will synchronize your current directory with a new directory that will be exposed to the Web. You could have a service that watches for changes or use something like an rsync with exclusions or what have you. This would be much simpler since, based on another question, you are on Windows.
ther's an extension called np_subversion which will take care of fileadmin changes via subversion. As a nice plus it will hide folders for you
I do not want to setup a svn export --> web_root just to get rid of the files
Are you sure? That’s how SVN is designed: you check code out of SVN to work on it, and export code from SVN to deploy it. If you don’t like that, then SVN probably isn’t the right choice. As gahooa said, maybe switch to Git?
It’s a bit like saying “I want to save my Word document, but I don’t want this stupid .doc file showing up on my computer.” That’s just how the software works.
Sara Golemon's Runkit can do this. You can remap functions like glob(). However, I am not sure if it's a good idea to run it in a production server.
If you don't need the .svn folders, you can just delete them.
find ./ -name ".svn" | xargs rm -f *.svn
Related
I have a PHP enabled site, with directory-listing turned off.
But, when I used Acunetix: (web vulnerability scanning software) to scan my site, and other high-profile websites, it was able to list all directories & files.
I don't know what this is happening, but I have this theory: maybe the software is using English words, trying to see if a folder exists by trying names like "include/", "css/", "/images", etc. Then, maybe it is able to list files that way.
Because, if directory listing is off, I don't know what more there is to do.
So, I devised this plan, that if I give my folders/files difficult names like I3Nc_lude, 11css11, etc., maybe it would be difficult for the software to find the names. What do you think?
I know, I could be dead-wrong about this, and the idea might be laughable but, that is why I am asking for help.
How do you Completely! Forbid directory listing??
Ensure all directories from the root of your site have directory
listings disabled. It is typically on by default when you setup a
new server.
Assuming that directory listing in your webserver is not your issue,
keep in mind that any resources you have in your site: CSS files, JS
sources, and of course HREFs can be traversed with little or no
effort (typically a few lines of javascript). There is no way to
hide anything that you've referenced. This is most likely what you
are seeing reflected in the scan.
Alternatively, if you use SVN or other version control systems to
deploy your site, often these can be used to determine the path of
every file in your codebase.
Probably the most common mistake people make when first creating sites is that they keep all their files in the webroot, and it becomes somewhat trivial to figure out where things are.
IMHO the best approach is have your code in a separate directory outside the webroot, and then load it as needed (this is how most MVC frameworks work). You can control entirely then what can and can not be accessed via the web. You can have 100s of classes in a directory and as long as they are not in the webroot, no one will ever be able to see them, even if directory listing were to become enabled.
The checkers aren't using some kind of language-based brute force attack, that would be far too costly and invasive even for the most inept hacker. Your internet file sharing service (Apache, IIS, whatever) is serving up the structure to anyone who asks.
I found this solution at - it should apply to you, I hope.
http://www.velvetblues.com/web-development-blog/dont-get-hacked-6-ways-to-secure-your-wordpress-blog/
Hide Your Directory Structure
It is also good practice to hide your directory structure. By default, many WordPress installations enable any visitors to snoop and see all files in folders lacking an index file. And while this might not seem dangerous, it really is. By enabling visitors to see what files are in each directory, they can better plot their attack.
To fix this problem, you can do one of two things:
Option 1: Use An Index File
For each directory that you want to protect, simply add an index file. A simple index.html file will suffice.
Option 2: Use An .htaccess File
The preferred way of hiding the directory structure is to use the following code in an .htaccess file.
Options -indexes
That just sounds like a nightmare to manage. Focus on securing the files the best you can with all preventative measures. Don't rely on security through obscurity. If someone wants in, some random directory names will just slow them down slightly
I'm looking to centralize a lot of my web applications code, so that multiple components have access to the same core functionality. This is how I have the website set up:
/var/www/website - domain.com
/var/www/subdomain1 - subdomain1.domain.com
/var/www/subdomain2 - subdomain2.domain.com
Naturally I've had a lot of trouble when it comes to the duplication of common functionality, as any changes made to one area would also need to be applied to other areas. My proposed solution is to create a new directory in /var/www which will contain all of the core scripts:
/var/www/code - core code
I would then set the PHP include directory to /var/www/code, so scripts can include these files without having to specify the absolute path.
Can you think of any more efficient ways of centralizing the code code?
Many thanks!
Your approach is good enough for this purpose.
Little suggestion:
store your front-end scripts in directory like /var/www/website/www instead of /var/www/website. There will be index file and ajax processors and scripts like that. But your project-based inclusions (as well as other miscellaneous stuff) would be stored in directory like /var/www/website/includes. It is simple yet efficient defense from hacker attacks on your inclusion files
so, your document roots will be in /var/www/website/www (domain) and /var/www/website/subdomain/www/ (subdomain)
It seems that you are thinking correctly :
Share Code between multiple PHP sites
It's only a suggestion, but you should put the public content in the /var/www/* which may end being publicly accessible—either because of your http server or because of some misconfiguration—and create some other directories for your shared code/libs like /usr/local/lib/php/*.
For more security you should frame it with open_basedir adding the private and public dirs—as well as upload and session dirs.
And don't forget to version your libs, e.g.:
/usr/local/lib/php/myLib-1.0
/usr/local/lib/php/myLib-1.2
etc.
Thus, you'll be able to make changes without breaking everything.
I have a rather large php project with all kinds of different third party includes. Its all managed with Netbeans PHP. I have a few problems i would like some oppinions on.
Does anyone build there php-project? When i upload to server via ftp i get all kinds of project files which doesnt belong to live environment.
Would be nice if a build solution could encode all files when building (actually copying files to new location and leaving out project files and files like that).
Let me hear how you handle your php projects (development, test, build/deploy) with OS X.
Thanks
Well you could write a shell (or scripting language of your choice) script to selectively copy the files you want to your remote server.
Alternately, you could put the code to be published (again, leaving out the undesired files) into a version control system (git, svn, etc) and pull the code to the remote machine that way. There are lots of other benefits of using VCS as well.
You could also use something like ant to write a "build" script which does any manipulation needed for deployment.
I use Eclipse PDT and mount server with Macfusion. Eclipse File Sync plugin then syncs everytihing to mounted drive (can exclude unwanted files/directories).
Also using a SVN and opening project directly from remote drive with Eclipse and including libraries from project properties. So there is two projects - one for local development and other is live in server - syncronized with SVN
(Eclipse plugin Clay for database diagrams is awsome too)
Though I am not an OSX user I am a Mac-in-the know. Basically I get invited to their house parties and politely decline to run amok with Windows and Linux/Unix (who is OSX's older sister, it is all so very confusing). In any case, Mac adds extra files to everything. I know when a designer is using a mac vs a pc. I get extra files and directories in their attachments if there is a folder. As far as project files, there needs to be a way to set up netbeans to keep the files for project in a different location from your working on files. If not then shame on your IDE. Well, if nothing works, since Linux/Unix is close to my heart I can write a script for you to pull all files of a list of extensions out and copy them to a new directory with subfolders in tact. Here ya are:
This should work
find . -name '*.php' | xargs rsync -avuzb '{}' exportdirectory/
I do not have a machine to test it on right now and Cygwin, for all it's effort just fails sometimes. Let me know how it goes, otherwise use Unix wizardy :D Because yes, as we know OSX uses Unix command line. The crude script is mainly to give you an idea of a way you can do it. I am sure there is php, html, css, js, and other files and on rare occasion (god lets hope not) just knowing extensions isn't going to be enough.
Personally I use gvim and refuse to use project managers, I use gvim and command line and though I have used both, I am highly preferential to the current way I am doing everything. Hey, it's a choice though. No criticism here if you walk another journey.
I currently ftp all my files to my website when i do an update (over a slowish adsl connection)
And I want to make things easier, so I just recently started using a hosted svn service, and i thought i could speed things up a bit by doing an svn export of my website directly onto my webserver
i have tried that a few times and it seems to work ok, however it does fetch the entire site everytime which is a bit slow for a 1 file update
so my questions are
is it possible to do an export and only get the changes since the last export (how will this handle deleted files ?)
OR will it be easier to do an svn checkout and svn update it all the time instead of svn export and just hide the .svn folders using apache htaccess
is this a good idea, or is there a better way to publish my website
i am trying to achieve the 1 click deploy type ideal
maybe there are some gotcha's i haven't thought of that someone else has run into
debian/apache/php
I would do an svn checkout, and have done so successfully on a live site for a number of years. You should add mod_rewrite rules to 404 the .svn directories (and files) though.
This is what I'm doing on my host:
For every project I have a structure that looks more less like this:
~/projects/myproj
~/public_html/myproj
First dir is a checkout from SVN, while second one is just svn export.
I have a small bash script
#!/bin/bash
SOURCE="$HOME/projects/"
TARGET="$HOME/public_html/"
for x in `ls $SOURCE`
do
if [ -d $SOURCE$x ]; then
svn update $SOURCE$x
svn export --force $SOURCE$x $TARGET$x
fi
done
Export is done from working copy so it's very fast.
It might not be exactly the answer you are looking for, but, if you have an SSH access to your webserver (it depends on your hosting service ; some "low cost" don't give such kind of access), you can use rsync to "synchronise" the remote site with what you have on your disk.
In the past, I was using something like the idea you are describing (fetching the svn log between last revision pushed to production and HEAD, analysing every lines, and in the end calculating what to send to the server) ; but it was not really a great process ; I now use rsync, and like it way better.
(Here too, you will have to exclude .svn directories, btw)
You can just accept having .svn directories in your website (generally not a problem esp. if you configure it not to permit access to these) - this is the easy option. Alternatively, do what RaYell does, and have two copies of your website on the webserver. One normal checkout outside of the web-directory, and one in your web-directory. When you update, simply export the svn (just a copy with .svn dirs deleted) into the web directory (and you should make sure to first delete old files if you wish to avoid files that have been removed from SVN from remaining on your website).
I do something like this, using robocopy set to mirror the svn checkout while excluding .svn directories, and get both the export and the old-file deletion in one step, thus minimizing downtime if the copy takes long. I'm sure this is easy on unix too, if that's your hosting environment. For example, you can use a local rsync: http://blog.gilluminate.com/2006/12/12/yes-you-can-rsync-between-two-local-directories/
Old topic but since this is what came up in Google during my research I thought I would add to this. I recommend doing an export to the site instead of checking out to it. I like to keep the repositories and the sites separate. I also don't recommend exporting the entire repo to the site each time, specially if only a few file change at a time. What you can do instead is do a diff on the repo to see what's changed from a release to another and only export those files. More info at: http://www.joeyrivera.com/2011/automate-svn-export-to-site-w-bash-script/
I've gotten comfy with SVN, and now I need a way to deploy my code to staging or live servers more easily. I'd also like some method for putting build info in the footer of this site to aid in testing. Site is PHP/MySQL.
First enable keyword substitution for a file where you wish to have the revision info:
svn propset svn:keywords "Rev" file.txt
The add to the file where you want the Revision info stored:
$Rev$
Further readings: http://svnbook.red-bean.com/en/1.4/svn.advanced.props.special.keywords.html
The properties methods will only give you the last revision number of the file you have the property in, not the head revision of the whole repository (a la the Stack Overflow footer). If you are wanting that, you'll need to use svnversion.
I recently started using Capistrano on a project and it superb and very flexible and powerful. I ended up deviating quite far from its normal usage, but it makes one "click" deployment much easier.
If you want to update the version number in a projects AssemblyInfo.cs you may be interested in this article:
CodeProject: Use Subversion Revision numbers in your Visual Studio Projects
If you enable SVN Keywords then every time you check in the project Subversion scans your files for certain "keywords" and replaces the keywords with some information.
For example, At the top of my source files I would create a header contain the following keywords:
'$Author:$
'$Id:$
'$Rev:$
When I check this file into Subversion these keywords are replaced with the following:
'$Author: paulbetteridge $
'$Id: myfile.vb 145 2008-07-16 15:24:29Z paulbetteridge $
'$Rev: 145 $
A script to svn update on an as needed basis.
SVN supports keywords. You can add which keywords you want to expand to the keywords property, and SVN will expand then. See $Id$ or $Rev$, or keywords described in the SVN book.
I'm a fan of using capistrano for pushes. Refer to here.
You could use the SVN $Rev$ property to get the revision number into your footer.
A really simple way to manage this is to setup your app in the following way:
Simply make your deployment app a working copy of your trunk (svn co the project to your /www root) and you run an svn up through an ssh console (ssh user#host.com svn up /path/to/project) when you need to update. You can also rollback with the appropriate checkout mechanisms. This is important: if you do this, add RewriteRules (or equivalent) to your .htaccess (or equivalent) to disallow access to .svn directories. If you can't do the above, run an svn export through ssh instead (so it won't be a 'working copy'), but this will naturally be slower than doing an up.
Also, you can look at what Ruby on Rails does with Capistrano.. it's the same basic concept but supports transactional backups if the update goes wrong in the middle by storing each checkout in a separate folder and symlinking the "latest" to your /www directory.
The keywords stuff will fail in plenty of cases -- like if you've modified the source before deploying, or if you check in from one directory in your project then a different directory in the same project will have different revision numbers. Check the docs carefully to make sure the keywords do what you think they do.
The better way is to use the svnversion program to generate information about your checked out directories at compile or deployment time. Svnversion will show information about the version of ALL of your directories as well as flagging whether or not the source was locally modified.
The method I've come up for my php projects, may not be the best method but after some time searching certainly seems to be, is to do a checkout, run a version check, wipe out the .svn folders, and move on. Here is a portion of shell script I've written:
(first, you need the script the checkout your repo)
# get the svn revision number and create a RELEASE file
svnvers=`svnversion .`
echo "version: $svnvers"
echo "<release><development>0</development><revision>$svnvers</revision></release>" > RELEASE
# remove all .svn directories
find . -name .svn -exec rm -rf {} \;