Does AWS PHP SDK automatically retry multipart uploads? - php

Based on the sdk code, the s3 client code uses retry logic, but the sample code from the docs suggest doing a loop until the multipart upload finishes correctly.
$s3Client = new S3Client([
'profile' => 'default',
'region' => 'us-east-2',
'version' => '2006-03-01'
]);
$bucket = 'your-bucket';
$key = 'my-file.zip';
// Using stream instead of file path
$source = fopen('/path/to/large/file.zip', 'rb');
$uploader = new ObjectUploader(
$s3Client,
$bucket,
$key,
$source
);
do {
try {
$result = $uploader->upload();
if ($result["#metadata"]["statusCode"] == '200') {
print('<p>File successfully uploaded to ' . $result["ObjectURL"] . '.</p>');
}
print($result);
} catch (MultipartUploadException $e) {
rewind($source);
$uploader = new MultipartUploader($s3Client, $source, [
'state' => $e->getState(),
]);
}
} while (!isset($result));
Is that MultipartUploadException being thrown after the standard 3 retries for it have happened? Or are multipart uploads not covered by the retry policy?

Related

Uploading Objects to AWS S3 with SDK 3 for PHP

For many years my PHP application has used the AWS SDK 2 for PHP and now we are considering switching to SDK 3.
However, looking in the SDK documentation we couldn't find any simple example, they all talk about multipart and other things that are very different from what we have today.
The code below is what we have for SDK 2, how would a simple object be uploaded to a bucket using SDK3?
<?php
define('S3_KEY', '');
define('S3_SECRET', '');
define('S3_BUCKET_NAME', '');
$s3 = S3Client::factory(array(
'key' => S3_KEY,
'secret' => S3_SECRET
));
$filename = 'example/001/1.jpg';
try {
$resource = fopen($origin, 'r');
$return = $s3->upload(S3_BUCKET_NAME, $filename, $resource, 'public-read');
return $return['ObjectURL'];
} catch (S3Exception $e) {
return false;
}
In AWS SDK for PHP Version 3, ObjectUploader can upload a file to Amazon S3 using either PutObject or MultipartUploader, depending on what is best based on the payload size.
Below is the sample code, you can use to upload objects into the S3 bucket:-
require 'vendor/autoload.php';
use Aws\S3\S3Client;
use Aws\Exception\AwsException;
use Aws\S3\ObjectUploader;
$s3Client = new S3Client([
'profile' => 'default',
'region' => 'us-east-2',
'version' => '2006-03-01'
]);
$bucket = 'your-bucket';
$key = 'my-file.zip';
// Using stream instead of file path
$source = fopen('/path/to/file.zip', 'rb');
$uploader = new ObjectUploader(
$s3Client,
$bucket,
$key,
$source
);
do {
try {
$result = $uploader->upload();
if ($result["#metadata"]["statusCode"] == '200') {
print('<p>File successfully uploaded to ' . $result["ObjectURL"] . '.</p>');
}
print($result);
} catch (MultipartUploadException $e) {
rewind($source);
$uploader = new MultipartUploader($s3Client, $source, [
'state' => $e->getState(),
]);
}
} while (!isset($result));
fclose($source);
From Version 3 of the SDK Key differences:
Use new instead of factory() to instantiate the client.
The 'version' and 'region' options are required during
instantiation.

PHP Uploading Zero Bytes To S3

I am trying to upload mp3 files to Amazon S3 but for some reason anything I upload is uploaded with a byte length of zero. I cannot seem to figure out why this is happening. I've tried following a tutorial and adjusting the code myself. Any help is appreciated.
<?php
error_reporting(0);
require '../../../../includes/aws-sdk-php/vendor/autoload.php';
use Aws\S3\S3Client;
use Aws\S3\Exception\S3Exception;
if (isset($_GET['test'])){
if($_GET['test'] == "true"){
include("../../depend.php");
include("../../../../includes/okta-jwt/okta-jwt-functions.php");
include("../../../../includes/aws-s3/functions.php");
$jwtToken = $_GET['token'];
$validateJWT = validateJWT($jwtToken);
echo "$validateJWT";
die();
}
}
include("../../depend.php");
include("../../../../includes/okta-jwt/okta-jwt-functions.php");
if(isset($_POST['episodeTitle'])){
$episodeTitle = $_POST['episodeTitle'];
}
if(isset($_POST['episodeDescription'])){
$episodeDescription = $_POST['episodeDescription'];
}
if(isset($_POST['explicitContent'])){
$explicitContent = $_POST['explicitContent'];
}
if(isset($_POST['episodeShowID'])){
$episodeShowID = $_POST['episodeShowID'];
}
if(isset($_POST['jwtToken'])){
$jwtToken = $_POST['jwtToken'];
}
if(isset($_POST['audioFile'])){
$episodeAudio = $_POST['audioFile'];
}
$validateJWT = validateJWT($jwtToken);
$payloadJSON = json_decode($validateJWT);
$payloadDecoded = $payloadJSON;
$payloadUserID = $payloadDecoded->userID;
//Check if JWT Token Is Valid
if($payloadUserID != 0){
$payloadUserOrgID = $payloadDecoded->userOrgID;
$payloadRole = $payloadDecoded->role;
$payloadExp = $payloadDecoded->exp;
$payloadState = $payloadDecoded->state;
$payloadFirstName = $payloadDecoded->firstName;
$payloadLastName = $payloadDecoded->lastName;
$payloadFullName = $payloadDecoded->fullName;
$episodeStateCode = bin2hex(random_bytes(25));
date_default_timezone_set('UTC');
$showUtcTimestamp = date("Y-m-d H:i:s");
// AWS Info
$bucketName = 'XXX';
$IAM_KEY = 'XXX';
$IAM_SECRET = 'XXX';
// Connect to AWS
try {
// You may need to change the region. It will say in the URL when the bucket is open
// and on creation.
$s3 = S3Client::factory(
array(
'credentials' => array(
'key' => $IAM_KEY,
'secret' => $IAM_SECRET
),
'version' => 'latest',
'region' => 'us-east-2'
)
);
} catch (Exception $e) {
// We use a die, so if this fails. It stops here. Typically this is a REST call so this would
// return a json object.
die("Error: " . $e->getMessage());
}
// $keyName = "test_example/" . basename($_FILES["audioFile"]['name']);
// $keyName = "org-$payloadUserOrgID/$episodeStateCode-" . basename($_FILES["audioFile"]['name']);
$keyName = "org-$payloadUserOrgID/$episodeStateCode.mp3";
$pathInS3 = 'https://s3.us-east-2.amazonaws.com/' . $bucketName . '/' . $keyName;
// Add it to S3
try {
// Uploaded:
$file = $_FILES["audioFile"]['tmp_file'];
$s3->putObject(
array(
'Bucket'=>$bucketName,
'Key' => $keyName,
'SourceFile' => $file,
'StorageClass' => 'STANDARD'
)
);
} catch (S3Exception $e) {
die('Error:' . $e->getMessage());
} catch (Exception $e) {
die('Error:' . $e->getMessage());
}
echo 'Done';
}
?>
I can confirm that the file is being uploaded to S3 but the data in the file is not there. Any help in this would be very much appreciated.

How to check if 2 images with different file names are identical with PHP

I am getting images from the WEB and sending them to an S3 bucket on AWS. Each image gets a random name. But I don't want to store identical twins. So, before storing the image on my server (to then sending it to S3), I need to check if an identical image already exists there (on S3).
I cannot do the check on my server, because files will be deleted after successfully sent to S3.
Here is my code:
$filenameIn = $url_img;
$nome = randomString();
$nome_img = "$nome.png";
$filenameOut = __DIR__ . '/img/' . $nome_img;
$contentOrFalseOnFailure = file_get_contents($filenameIn);
$bucket = 'bucket-img';
if( !$contentOrFalseOnFailure ) {
return "error1: ...";
}
$byteCountOrFalseOnFailure = file_put_contents($filenameOut, $contentOrFalseOnFailure);
if( !$byteCountOrFalseOnFailure ) {
return "error2: ...";
}
// Instantiate the S3 client with your AWS credentials
$aws = new Aws\S3\S3Client([
'version' => 'latest',
'region' => 'sa-east-1',
'credentials' => array(
'key' => 'omissis',
'secret' => 'omissis',
)
]);
/*
I BELIEVE AT THIS POINT I SHOULD MAKE A COMPARISON TO CHECK IF THE FILE THAT
IS ABOUT TO BE SENT TO S3 HAS AN IDENTICAL TWIN THERE, REGARDLESS OF THEIR
RANDOM NAME. IF SO, UPLOAD SHOULD BE ABORTED AND THE SCRIPT SHOULD PROVIDE
ME WITH THE URL OF THE TWIN.
*/
try{
// Send a PutObject request and get the result object.
$result = $aws->putObject([
'Bucket' => $bucket,
'Key' => $nome_img,
'SourceFile' => $filenameOut,
'ACL' => 'public-read'
]);
$r = $aws->getObjectUrl($bucket, $nome_img);
return $r;
} catch (S3Exception $e) {
return $e->getMessage() . "\n";
}
As I said in the code, I should get the url of the twin file, if it exists.
Is that possible?

Unable to open using mode r: fopen(): AWS Elastic Beanstalk

Error:Unable to open using mode r: fopen(): Filename cannot be empty I keep getting this error when I try to upload larger files (more than 5MB). I have uploaded the PHP app to AWS Elastic Beanstalk and I upload the files to AWS S3. I don't even have fopen() in the code.
Alson when I test the site using XAMPP I don't get this error.
This is the code I use to upload file to S3:
<?php
session_start();
require 'vendor/autoload.php';
use Aws\S3\S3Client;
use Aws\S3\Exception\S3Exception;
// AWS Info
$bucketName = 'tolga20.images';
$IAM_KEY = '******************';
$IAM_SECRET = '*************************';
$feedback = '';
$unqiue_num = mt_rand(1000, 9999);
if(isset($_FILES['fileToUpload'])) {
$user_set_id = $_POST['user_set_id'];
// Connect to AWS
try {
// You may need to change the region. It will say in the URL when the bucket is open
// and on creation.
$s3 = S3Client::factory(
array(
'credentials' => array(
'key' => $IAM_KEY,
'secret' => $IAM_SECRET
),
'version' => 'latest',
'region' => 'eu-west-2'
)
);
} catch (Exception $e) {
// We use a die, so if this fails. It stops here. Typically this is a REST call so this would
// return a json object.
die("Error: " . $e->getMessage());
}
$temp_name = explode(".", $_FILES["fileToUpload"]["name"]);
$newfilename = $unqiue_num . "-" . $user_set_id . '.' . end($temp_name);
// For this, I would generate a unqiue random string for the key name. But you can do whatever.
$keyName = 'images/' . basename($newfilename);
$pathInS3 = 'https://s3.eu-west-2.amazonaws.com/' . $bucketName . '/' . $keyName;
// Add it to S3
try {
// Uploaded:
$file = $_FILES["fileToUpload"]['tmp_name'];
$s3->putObject(
array(
'Bucket'=>$bucketName,
'Key' => $keyName,
'SourceFile' => $file,
'StorageClass' => 'REDUCED_REDUNDANCY'
)
);
} catch (S3Exception $e) {
die('Error:' . $e->getMessage());
} catch (Exception $e) {
die('Error:' . $e->getMessage());
}
//$feedback = 'File uploaded! Custom name: ' . '<b><i>' . $newfilename;
$_SESSION['newfilename'] = $newfilename;
header("Location: next.php");
}
?>
Try to increase the POST_MAX_SIZE and UPLOAD_MAX_FILESIZE values in your php.ini!
$file = $_FILES["fileToUpload"]['tmp_name'];
change to
$file = $_FILES["fileToUpload"]['name'];
Hope it will solve your problem.

Using PHP to upload to Amazon S3

I've spent the last few hours following tutorials for implementing file uploads to Amazon S3 using php. I uploaded the most recent version of Donovan Schönknecht's S3 class to my server (as S3.php) and I am trying to use the following code to test upload capability. I know this code will work because I've seen numerous examples in action.
<?php
require('S3.php');
$s3 = new S3('KEY', 'SECRET KEY');
//insert into s3
$new_name = time() . '.txt';
S3::putObject(
'upload-me.txt',
'bucketName',
$new_name,
S3::ACL_PUBLIC_READ,
array(),
array(),
S3::STORAGE_CLASS_RRS
);
?>
I get an error 500 server error when I attempt to load this page. Additionally, every other reputable tutorial of this nature has given me the same error 500.
I verified that my key and secret key are valid by connecting to S3 with Cyberduck.
Does anyone have a clue as to what I could be doing incorrectly?
Thanks,
Sean
As it turns out, I was missing the cURL extension for PHP and this was causing an issue as the S3 class I was using required the use of cURL. All is working now.
You should also consider using the official AWS SDK for PHP. Examples for using S3 with the SDK can be found in their S3 user guide.
You can download most recent version of Amazon PHP SDK by running following composer command
composer require aws/aws-sdk-php
Further configuration to upload file on Amazon s3 are following
// Include the SDK using the Composer autoloader
require 'vendor/autoload.php';
use Aws\S3\S3Client;
use Aws\S3\Exception\S3Exception;
// Set Amazon s3 credentials
$client = S3Client::factory(
array(
'key' => "your-key",
'secret' => "your secret key"
)
);
try {
$client->putObject(array(
'Bucket'=>'your-bucket-name',
'Key' => 'your-filepath-in-bucket',
'SourceFile' => 'source-filename-with-path',
'StorageClass' => 'REDUCED_REDUNDANCY'
));
} catch (S3Exception $e) {
// Catch an S3 specific exception.
echo $e->getMessage();
}
Get step by step details from here Amazon S3 File Upload Using PHP
Following example worked for me:
<?php
use Aws\S3\S3Client;
use Aws\S3\Exception\S3Exception;
$client = S3Client::factory([
'version' => 'latest',
'region' => 'us-west-1',
'credentials' => [
'key' => "<scret-key>",
'secret' => "<my-secret>"
]
]);
try {
$client->putObject([
'Bucket' =>'<my-bucket-name>',
'Key' => '<file-name>',
'SourceFile' => '<file-path-on-server>', // like /var/www/vhosts/mysite/file.csv
'ACL' => 'public-read',
]);
} catch (S3Exception $e) {
// Catch an S3 specific exception.
echo $e->getMessage();
}
Getting security credentials:
https://aws.amazon.com/blogs/security/wheres-my-secret-access-key/
https://console.aws.amazon.com/iam/home?#/security_credential
Getting region code
https://docs.aws.amazon.com/general/latest/gr/rande.html
Use this one to Upload Images using a form and it's working Fine for me
you may try using it with your code
$name = $_FILES['photo']['name'];
$size = $_FILES['photo']['size'];
$tmp = $_FILES['photo']['tmp_name'];
//////Upload Process
// Bucket Name
$bucket = 'bucket-name';
require_once('S3.php');
//AWS access info
$awsAccessKey = 'awsAccessKey';
$awsSecretKey = 'awsSecretKey';
//instantiate the class
$s3 = new S3($awsAccessKey, $awsSecretKey);
$s3->putBucket($bucket, S3::ACL_PUBLIC_READ);
//Rename image name.
$actual_image_name = time();
//Upload to S3
if($s3->putObjectFile($tmp, $bucket , $actual_image_name, S3::ACL_PUBLIC_READ) )
{
$image='http://'.$bucket.'.s3.amazonaws.com/'.$actual_image_name;
}else{
echo 'error uploading to S3 Amazon';
}
I never found a updated Script with Amazons latest sdk. i have made it by myself. it woks as a php commandline interpreter script. give it a try :
https://github.com/arizawan/aiss3clientphp
I'm not familiar with S3 API, but i used it as the storage with https://github.com/KnpLabs/Gaufrette. Gaufrette is a library that provides pretty nice abstraction layer above S3 and other file services/systems.
Here is sample code to upload images on Amazon S3.
// Bucket Name
$bucket="BucketName";
if (!class_exists('S3'))require_once('S3.php');
//AWS access info
if (!defined('awsAccessKey')) define('awsAccessKey', 'ACCESS_KEY');
if (!defined('awsSecretKey')) define('awsSecretKey', 'ACCESS_Secret_KEY');
$s3 = new S3(awsAccessKey, awsSecretKey);
$s3->putBucket($bucket, S3::ACL_PUBLIC_READ);
if($s3->putObjectFile($tmp, $bucket , $image_name_actual,S3::ACL_PUBLIC_READ) )
{
$message = "S3 Upload Successful.";
$s3file='http://'.$bucket.'.s3.amazonaws.com/'.$actual_image_name;
echo "<img src='$s3file'/>";
echo 'S3 File URL:'.$s3file;
}
else{
$message = "S3 Upload Fail.";
}
}
Below is the best solution. Its using multipart upload.Make Sure to install Aws SDK for PHP before using
require 'vendor/autoload.php';
use Aws\S3\S3Client;
use Aws\Exception\AwsException;
use Aws\S3\MultipartUploader;
use Aws\Exception\MultipartUploadException;
try {
$s3Client = new S3Client([
'version' => 'latest',
'region' => 'us-east-1',
'credentials' => [
'key' => 'AKIAUMIZJR5U5IO7M3',
'secret' => 'BmcA3vFso1bc/9GVK7nHJtFk0tQL6Vi5OoMySO',
],
]);
// Use multipart upload
$source = 'https://b8q9h6y2.stackpathcdn.com/wp-content/uploads/2016/08/banner-for-website-4.png';
$uploader = new MultipartUploader($s3Client, $source, [
'bucket' => 'videofilessandeep',
'key' => 'my-file.png',
'ACL' => 'public-read',
]);
try {
$result = $uploader->upload();
echo "Upload complete: {$result['ObjectURL']}\n";
} catch (MultipartUploadException $e) {
echo $e->getMessage() . "\n";
}
} catch (Exception $e) {
print_r($e);
}

Categories