aws s3 php: fails to upload directory using UploadSyncBuilder - php

Question

I am trying to upload a 14MB directory to s3 using aws php sdk. These are following scenarios and their results
Upload from my local system(india) to s3(us-east-1): success
Upload from ec2(ap-southeast-1) to s3(ap-southeast-1): success
Upload from ec2(ap-southeast-1) to s3(us-east-1): FAIL
The following code I have used
require('application/classes/vendor/autoload.php');
use Aws\S3\S3Client;
use Aws\Common\Credentials\Credentials;
use Aws\S3\Sync\UploadSyncBuilder;
use Guzzle\Log\ClosureLogAdapter;
use Guzzle\Plugin\Log\LogPlugin;
$newCredentials = new Credentials("API_KEY", "SECRET_KEY");
$s3 = S3Client::factory();
//$logPlugin = Guzzle\Plugin\Log\LogPlugin::getDebugPlugin();
$stream = fopen('out.log', 'w');
$logPlugin = new LogPlugin(new ClosureLogAdapter(function ($m) use ($stream) {
fwrite($stream, $m . PHP_EOL);
}), "# Request:\n{request}\n\n# Response:\n{response}\n\n# Errors: {curl_code} {curl_error}", true);
$s3->addSubscriber($logPlugin);
$s3->setCredentials($newCredentials);
$s3->setRegion("us-east-1");
UploadSyncBuilder::getInstance()
->setClient($s3)
->setBucket("BUCKET")
->setAcl("public-read")
->uploadFromDirectory("/mnt/1410503562Welcome to Leap!.zip_ext")
->setKeyPrefix("customs3/game10906/1410501964WelcometoLeap.zip_ext")
->setConcurrency(5)
->build()
->transfer();
Exception
PHP Fatal error: Uncaught exception 'Guzzle\Service\Exception\CommandTransferException' with message 'Errors during multi transfer (Guzzle\Http\Exception\RequestException) ./application/classes/vendor/guzzle/guzzle/src/Guzzle/Http/Message/Request.php line 569
Error completing request
#0 ./application/classes/vendor/guzzle/guzzle/src/Guzzle/Http/Message/Request.php(378): Guzzle\Http\Message\Request->processResponse(Array)
#1 ./application/classes/vendor/guzzle/guzzle/src/Guzzle/Http/Message/EntityEnclosingRequest.php(49): Guzzle\Http\Message\Request->setState('complete', Array)
#2 ./application/classes/vendor/guzzle/guzzle/src/Guzzle/Http/Curl/CurlMulti.php(286): Guzzle\Http\Message\EntityEnclosingRequest->setState('complete', Array)
#3 ./application/classes/vendor/guzzle/guzzle/src/Guzzle/Http/Curl/CurlMulti.php(244): Guzzle\Http\Curl\CurlMulti->processResponse(Object(Guzzle\Http\Message\EntityEnclosingRequest), Object(Guzzle\Http\Curl\CurlHandle), Array)
#4 ./application/classes/vendor/guzzle/guzzle in /vol/selfserve/application/classes/vendor/guzzle/guzzle/src/Guzzle/Service/Exception/CommandTransferException.php on line 25
I have setup log to debug and dig through Guzzle library as well. I found that when it fails, for a request no response comes. When I set concurrency to 1, it succeeds but takes much time. It fails even at concurrency of 2.
If any file fails, whole upload fails leaving it in partial uploaded state. I could not find any way to set options to retry if something fails.
Thanks for help!

Answer 1

After contacting with AWS support I was able to solve this with their help. Issue was in guzzle library. Following is the response from AWS support
This error occurs when cURL fails to rewind a stream and does not
associate an error with a curl handle. We've added checks to account
for this situation and automatically retry. Please ensure that they
are using the latest version of the SDK and version 3.9.2+ of Guzzle.