I am working on a project where composer (https://getcomposer.org) is used. My problem is, the destination server is equipped with only FTP access (no shell, ssh, rsync). Each time I run "composer update" or another "require" at my dev machine, I need to upload all the vendor directory. As for now, +16k files are not a big problem yet, but even a small change requires a transfer of relatively huge amount of data.
I have tried to git the vendor directory, and use "git ftp" to deploy only the changed files, but as some libraries are seen as subprojects and/or have their own .gitignore files included - this method doesn't work quite well.
Runnig a script exec("composer update") seems to be not working either. Probably, the "exec" function at the live server is disabled, which on other hand, doesn't seem to be a bad idea on a public server (security issues, to be investigated by me).
I would love to find something like "composer remote synch" or similar method to transfer only the new installed or updated libraries to the remote server (composer.json, autoload.php of course included, but these obvious files are not a problem). Any tips, ideas?
Related
As you can see here: Specifying Dependencies
Google App Engine "Composer runs automatically when you deploy a new version of your application..."
I would prefer it didn't. I have scripts that run prior to deployment to remove all of the unnecessities in composer dependencies that are not needed in a deployed application (documentation, testing, etc.).
My .gcloudignore file does not include the /vendor directory and this is confirmed by the file count on upload. When I checked the deployed source however, the composer dependencies do not match my cleaned local dependencies because App Engine is overriding them.
Not sure why this is a feature in the first place, it seems it would be better to leave this in the control of the developers.
I do have billing enabled and am full featured. Thanks!
EDIT
I added the composer.json and composer.lock files to the ignore file to prevent the update and then the vender directory was missing completely from the deployed source. The vendor directory is not in the ignored file and has been uploaded. If I add a depenency, for example, the upload file count reflects the file count for the added dependency files.
It makes me wonder if something is running after the upload during the deployment to remove the vendor directory. They do have it as being removed by default in the ignore file. I assumed though that by taking it out it would allow the upload and deployment of it (the vendor directory)
I've been working on reproducing your issue and trying to attain exactly everything you described as things you need to happen. And I've found some interesting things.
I wanted to see the exact behavior of a vendor/ folder when you deploy it to GAE and understand exactly what causes your issues, so i followed these steps:
Cloned a simple PHP project just to save time.
Installed composer locally so i can require the dependencies.
I required a simple dependency (so it creates the vendor/, composer.json, composer.lock
composer require phpunit/php-timer
This made me have locally the dependencies so it directly uploads them. Can you try to do that with your composer file before attempting to deploy?
I did gcloud app deploy and it uploaded everything, I still have my doubts as if this is my local folder or generated at run-time.
Then I made a simple .gcloudignore as a test, added test* into it, did a gcloud app deploy, and it uploaded everything except the test folders.
Here you can specify everything that you want to avoid uploading, i think this would be a much easier solution for all your troubles. These are the .gcloudignore formats you can use.
It worked for me like this, hope it can help you out.
Let me know.
I've been running a project written in Laravel which has been fun to use.
The setup I use is the vagrant box Homestead configuration so I do the majority of my work on my local machine and push up to the development server once its ready to go. During the installation I had to push up the logs & vendor folder for it to work properly but now I'm at a stage where every commit I do via the command line includes storage/logs/laravel.log which when I then pull down it asks me to stash/commit on the server because they're different.
I've added it to the .gitignore file in the root directory of the project and it looks like this:
/node_modules
/public/storage
/.idea
Homestead.json
Homestead.yaml
/storage/logs/
/vendor/
Vendor doesn't cause me any problems unless I make changes to it so its not much of a bother, its just the logs which will not stop going up. If I use a GUI tool, I can manually tell it not to go up but I want to get it to the point that I can use the terminal to push it up and not worry about logs need stashing on the server.
I believe this is the same for the .env so I imagine a solution will work for both. I have also noticed that PHPStorm have said they're ignored but tracked with git if that helps.
If you take a look at the Laravel repo on GitHub, you'll find the following .gitignore file in the storage directory:
https://github.com/laravel/laravel/blob/master/storage/logs/.gitignore
This comes with the default installation to mark the logs directory as ignored. If you've deleted this by mistake, you should be able to reinstate it and resolve the issue you're having.
Just as importantly though, your workflow isn't following best practice. With respect to "Vendor doesn't cause me and problems unless i make changes to it" - you should never make changes to your vendor directory. This folder is home to third-party packages and plugins, modifying them directly causes multiple issues, chief amongst them:
You can no longer update a modified package without breaking your application.
Other developers won't be able to replicate your installation easily.
In fact, the vendor directory shouldn't be versioned at all. The best way to handle the files within it is using a package manager, like Composer, to do it all for you. This means you can easily switch between different versions of your packages and, by versioning only the composer files, other developers can run composer install or composer update to synchronise their development environment to yours quickly and accurately.
Bit of a strange set up but have come across a project where Composer has been used in a local environment to get a project started. The original developer did not have ssh access to the production server therefore he used Composer locally and uploaded the 'vendor' directory from his desktop to the server using FTP.
I now need to add the PHPMailer package so have done the following locally on my Mac:
cd Desktop/
composer require phpmailer/phpmailer
This has created the following structure on my desktop:
Desktop/composer.json
Desktop/composer.lock
Desktop/vendor/autoload.php
Desktop/vendor/composer/*
Desktop/vendor/phpmailer/*
Which of these do I need to upload through FTP? I realise vendor/phpmailer/* is the package I want, so will need uploading.
What about the others? I already have an autoloader configured so guessing vendor/autoload.php is not required here?
composer.json I could add the package to what's already there, e.g.
"require": {
*other packages here*
"phpmailer/phpmailer": "^5.2"
}
But I wasn't sure if that's necessary because I'm not going to be using ssh/Composer on the server to run any updates?
The usual workflow would be:
Checkout the current version from the version control.
Add dependencies via command line composer require new/package.
This will download the new package and update the autoloading.
Test the result locally or on a test website environment.
If happy with the result, upload the whole folder to the production server.
There may be several exceptions from this general workflow:
ad 1: If there is no version control, you'd probably better of starting a local git repo right now, and download the current production state into it as the first commit. Not having version control will make things harder, especially going back to known working versions. And because the files on the production server are probably unmanaged, you'd also check in the vendor folder into your newly created version control just to avoid canceling any changes that had been made to these files.
ad 2: Manually editing the composer.json file sometimes is a faster way to get what you want if you know what you are doing, but you'd have to correctly edit the JSON. For me it usually is too much hassle if I already have a command line ready. The command will also select a matching version that fits into the already installed dependencies. Manual editing may lead to version conflicts that you'd have to untangle. Remember to only install dependencies that work with the PHP version in production. You probably should run composer config platform.php X.Y.Z in order to add the production version of PHP into the composer.json file, which prevents Composer from installing dependency versions based on your development PHP. Adding the -g switch will add this setting to your global (user) setting instead, which will affect all composer operations you start, also for other projects.
ad 3: Manual editing will require you to run composer update on the command line, so there's probably no reason to not do composer require instead.
ad 4: How this could be done is entirely dependent on what environment you have to work with.
ad 5: At this stage you have assembled all files necessary to create a working website. Uploading them to production will always result in a working website unless the upload fails somehow. You could also use some "upload first to temporary folder, then move on the server" approach if you fear FTP would be unreliable. Some people take a different approach: They have a git repository on the production server and they simply push the version that should go live onto that remote repo. Some post-push scripts will run composer install then. This automated approach will also work (but not using FTP), but has the higher risk of something failing during deployment, and probably has no easy way back to the previous situation.
So in the end I'd say that uploading the whole folder structure via FTP (well, that protocol is insecure itself, better replace it with FTPS (FTP with SSL), SFTP or SCP) is better compared to running Composer on the production server.
Your specific question regarding which folders to upload: All of them. Especially upload the whole vendor folder. It contains the current autoloader and all dependency packages the software needs. If you worked correctly, you downloaded the existing composer.json and composer.lock file together with everything else and added the new dependency to it. This has changed both these files, added the new package to the vendor folder and the classes to the autoloader.
Don't fiddle with uploading only parts of the vendor folder, or manually editing a component of the autoloading. You will only create surprises for the developer coming after you if you do some aspect incorrectly, and it also takes more time. Composer is a very good tool to manage dependencies - use it!
You could copy the composer.json from the server to your local server, add the requirements and run composer update.
After that you can copy all files (composer.json, composer.lock and vendor folder to your server)...
Or you can copy local vendor/phpmailer into servers vendor folder, search for the entry of phpmailer in vendor/composer/autoload_psr4.php and add it to your servers vendor/composer/autoload_psr4.php.
Using this method also add phpmailers dependencies the same way.
composer depends phpmailer/phpmailer
I'm getting my feet wet with dependencies. I recently wrote a small application in PHP and used Composer to get some dependencies (mainly Slim, twig, hassankhan/config and illuminate/database). So I have the project in my local computer and basically have:
public folder
app folder
vendor folder
composer.phar
composer.json
composer.lock
It's now time to upload to my web server (I work with a GoDaddy hosting account with Linux CPanel). The question is, what should I do? Do I upload the whole thing, with the vendor folder and composer files? Or am I supposed to use Composer in a different way when uploading to my hosting? Never done this before so any guidance will be deeply appreciated.
Thanks!
You are not required to use Composer in any particular way. I recommend using it in the way that works best for you. That will depend on the type of development and release process you use. It sounds like you are using a fairly short and simple process where you develop for a while, decide the code is ready, and upload it to your production server. With this process, you could simply upload everything, vendor directory and all - just as you state. This could lead to unused files/directories not being removed but that is a general problem with this type process anyway (you can work around it by first deleting everything, breaking your site temporarily).
Composer helps you (among other things) "lock" all of the required libraries at some specific version. This is very useful when used with version control, tagged releases, and multiple environments (such as your development environment, a QA/Testing environment, and a Production environment). This helps ensure that each environment uses the exact same versions of the required vendor libraries.
If you were to try to use composer directly on your hosted server (i.e composer update), you would have to make sure that the php cli is available and that all other prerequisites for composer (including composer itself) were available. This is usually hard to count on for generic hosting providers so, you really are better off not trying to execute composer there.
As your project matures and uptime is important, you will want your development and deployment processes to mature as well. That's when you will realize many of the benefits of Composer.
I've got a php-fpm 5.3.10 installation running on Ubuntu 12.04 on a remote server, and I'm trying to use PHPUnit from within PhpStorm 8 (as of yesterday). I don't have a local php installation.
I have installed PHPUnit on my server and can use it via the commandline.
I have (seemingly) successfully linked my PhpStorm to my remote server and remote interpreter, and configured the right deployment settings.
At first I had an issue where it would try to run my local paths remotely, but that issue has been sorted. However, I keep getting the error Cannot find PHPUnit in include path (.:/usr/share/php:/usr/local/bin). This is almost certainly a serverside error (PhpStorm also knows these include paths, it could be confusing them for local paths) as the paths change when I change my include path in my php.ini.
This is the same error it gives back if I don't install PHPUnit at all, so again it would seem this is a serverside issue.
I have tried installing PHPUnit through PEAR (which is deprecated and soon unsupported by PHPUnit but still mentioned in a lot of PhpStorm tutorials), through Composer (on the remote server, not in PhpStorm) and even just manually.
The server accepted these installation methods just fine (the PEAR method seems to install an older version of PHPUnit, 1.0.x instead of the apparently available 4.0.x) but they all give the same issue.
If I manually place PHPUnit in my include path (I'm unsure what kind of files it wants there, though. a .phar? Source files?) it does seem to pick up something, but I get an XDebug error instead and it tells me it can't find the necessary PHPUnit files so I assume it is not the way to go (it shouldn't have to be).
At this point I'm starting to run out of ideas and none of the Stackoverflow responses to similar issues or PhpStorm/PHPUnit documentations seem to be able to help me.
Edit 1: Included picture of settings, as requested.
(can't post images yet, not enough rep)
settings
Other than that it's probably worth noting I'm using a mounted folder for my deployment, which just refers to the filesystem of my server (it's in my local network)
Edit 2: Changed deployment to SFTP as suggested and updated my mappings, as PhpStorm is able to successfully validate my remote environment everything seems to be in order there.
After reinstalling PHPUnit via Composer and selecting the autoloader in my PHPUnit settings it was still giving the error about the include path (it shouldn't do when the autoloader is used, should it?) so I changed the include path to point to /usr/share/php/Composer/vendor/phpunit (and the two directories above it, for good measure).
It was still giving the error, however in the error it says it cannot find PHPUnit in the include path, while the installation is in a directory called phpunit (all lowercase). After changing the directory name to PHPUnit, it is no longer giving the include path error, but en XDebug error instead : Warning: include(/usr/share/php/Composer/vendor/phpunit/phpunit/PHPUnit/TextUI/ResultPrinter.php) [<a href='function.include'>function.include</a>]: failed to open stream: No such file or directory in /usr/share/php/Composer/vendor/composer/ClassLoader.php on line <i>382</i>
I'm unsure what this implies exactly, and I don't know whether the error is because I should not be changing the directory names (it seems as if that should not be necessary), because I haven't changed enough of them (other directories are likely still lowercase) or because these files should simply not be in the include path (composer should be managing my dependencies, shouldn't it?).
In some cases, this can be resolved extremely easily. PHPStorm looks for the folder PHPUnit in the include path(s), but I've found this is case sensitive! Installing through composer will install PHPUnit in a folder called /vendor/phpunit/phpunit/phpunit, this is all lower case. This means that PHPStorm will not be able to find the folder unless you change the folder name to 'PHPUnit'. The include path should therefore be: /usr/share/php/composer/vendor/phpunit/phpunit and in that folder, there should be another folder called 'PHPUnit' (case sensitive!). I'm sure this isn't the prefect solution, but it works.