Grabbing contents of Object from S3 via PHP SDK 2? - php

I have been trying to figure out how to grab contents from an S3 bucket to include in a ZipArchive for a client who is storing files on S3, they now need to create reports that hold the files that were pushed up to S3 by their customers. I have tried the following with the PHP SDK 2 API (Installed with PEAR):
require 'AWSSDKforPHP/aws.phar';
use Aws\S3\S3Client;
use Aws\Common\Enum\Region;
$config = array(
'key' => 'the-aws-key',
'secret' => 'the-aws-secret',
'region' => Region::US_EAST_1
);
$aws_s3 = S3Client::factory($config);
$app_config['s3']['bucket'] = 'the-aws-bucket';
$app_config['s3']['prefix'] = '';
$attach_name = 'hosted-test-file.jpg';
try {
$result = $aws_s3->getObject(
array(
'Bucket' => $app_config['s3']['bucket'],
'Key' => $app_config['s3']['prefix'].$attach_name
)
);
var_dump($result);
$body = $result->get('Body');
var_dump($body);
$handle = fopen('php://temp', 'r');
$content = stream_get_contents($handle);
echo "String length: ".strlen($content);
} catch(Aws\S3\Exception\S3Exception $e) {
echo "Request failed.<br />";
}
However, all it returns is an Guzzle\Http\EntityBody object, not sure how to grab the actual content so I can push it into the zip file.
Grabbing Object
object(Guzzle\Service\Resource\Model)[126]
protected 'structure' => object(Guzzle\Service\Description\Parameter)[109]
protected 'name' => null
protected 'description' => null
protected 'type' => string 'object' (length = 6)
protected 'required' => boolean false
protected 'enum' => null
protected 'additionalProperties' => boolean true
protected 'items' => null
protected 'parent' => null
protected 'ref' => null
protected 'format' => null
protected 'data' => array (size = 11)
'Body' => object(Guzzle\Http\EntityBody)[97]
protected 'contentEncoding' => boolean false
protected 'rewindFunction' => null
protected 'stream' => resource(292, stream)
protected 'size' => int 3078337
protected 'cache' => array (size = 9)
...
'DeleteMarker' => string '' (length = 0)
'Expiration' => string '' (length = 0)
'WebsiteRedirectLocation' => string '' (length = 0)
'LastModified' => string 'Fri, 30 Nov 2012 21:07:30 GMT' (length = 29)
'ContentType' => string 'binary/octet-stream' (length = 19)
'ContentLength' => string '3078337' (length = 7)
'ETag' => string '"the-etag-of-the-file"' (length = 34)
'ServerSideEncryption' => string '' (length = 0)
'VersionId' => string '' (length = 0)
'RequestId' => string 'request-id' (length = 16)
Returned from Body
object(Guzzle\Http\EntityBody)[96]
protected 'contentEncoding' => boolean false
protected 'rewindFunction' => null
protected 'stream' => resource(292, stream)
protected 'size' => int 3078337
protected 'cache' => array (size = 9)
'wrapper_type' => string 'php' (length = 3)
'stream_type' => string 'temp' (length = 4)
'mode' => string 'w+b' (length = 3)
'unread_bytes' => int 0
'seekable' => boolean true
'uri' => string 'php://temp' (length = 10)
'is_local' => boolean true
'is_readable' => boolean true
'is_writable' => boolean true
// Echo of strlen()
String length: 0
Any information would be high appreciated, thanks!
Solution
It me a while to figure it out but I was able to find a gist that pointed me in the right direction, in order to get the contents of the file you need to do the following:
require 'AWSSDKforPHP/aws.phar';
use Aws\S3\S3Client;
use Aws\Common\Enum\Region;
$config = array(
'key' => 'the-aws-key',
'secret' => 'the-aws-secret',
'region' => Region::US_EAST_1
);
$aws_s3 = S3Client::factory($config);
$app_config['s3']['bucket'] = 'the-aws-bucket';
$app_config['s3']['prefix'] = '';
$attach_name = 'hosted-test-file.jpg';
try {
$result = $aws_s3->getObject(
array(
'Bucket' => $app_config['s3']['bucket'],
'Key' => $app_config['s3']['prefix'].$attach_name
)
);
$body = $result->get('Body');
$body->rewind();
$content = $body->read($result['ContentLength']);
} catch(Aws\S3\Exception\S3Exception $e) {
echo "Request failed.<br />";
}

The body of the response is stored in a Guzzle\Http\EntityBody object. This is used to protect your application from downloading extremely large files and running out of memory.
If you need to use the contents of the the EntityBody object as a string, you can cast the object to a string:
$result = $s3Client->getObject(array(
'Bucket' => $bucket,
'Key' => $key
));
// Cast as a string
$bodyAsString = (string) $result['Body'];
// or call __toString directly
$bodyAsString = $result['Body']->__toString();
You can also download directly to the target file if needed:
use Guzzle\Http\EntityBody;
$s3Client->getObject(array(
'Bucket' => $bucket,
'Key' => $key,
'command.response_body' => EntityBody::factory(fopen("/tmp/{$key}", 'w+'))
));

This worked for me:
$tempSave = 'path/to/save/filename.txt';
return $this->S3->getObject([
'Bucket' => 'test',
'Key' => $keyName,
'SaveAs' => $tempSave,
]);

When calling getObject, you can pass in an array of options. In these options, you can specify if you want to download the object to your file system.
$bucket = "bucketName";
$file = "fileName";
$downloadTo = "path/to/save";
$opts = array( // array of options
'fileDownload' => $downloadTo . $file // tells the SDK to download the
// file to this location
);
$result = $aws_s3->getObject($bucket, $file, $opts);
getObject Reference

I am not that familiar with the version 2.00 SDK, but it looks like you have been passed a stream context on php://temp. From looking at your updated question and from a brief glance at the documentation, it seems the stream may be available as:
$result = $aws_s3->getObject(
array(
'Bucket' => $app_config['s3']['bucket'],
'Key' => $app_config['s3']['prefix'].$attach_name
)
);
$stream = $result->get('stream');
$content = file_get_contents($stream);

<?php
$o_iter = $client->getIterator('ListObjects', array(
'Bucket' => $bucketname
));
foreach ($o_iter as $o) {
echo "{$o['Key']}\t{$o['Size']}\t{$o['LastModified']}\n";
}

Related

Drupal 8 Custom Module - From dynamic data (API) to store at Fields (Content type) Drupal DB

I've created a custom module in Drupal 8 that grab some data from an API, and puts them in the Drupal DB creating a new table.
I want to add this data as the contents of a specific content type.
How can I do that?
here is my code :
<?php
/**
* Implements hook_cron().
*/
function ods_cron() {
$message = 'Cron run: ' . date('Y-m-d H:i:s');
$ods = \Drupal::service('ods.ods');
$conf = \Drupal::service('ods.ods_configuration_request');
if ($conf->isDevelopment()) {
// Development
$response_bond = beforeSendRequest($conf->devUrlExternalBond(), 'GET');
$response_mf = beforeSendRequest($conf->devUrlExternalMutualFund(), 'GET');
} else {
// Production
$parameters_bond = [
'headers' => $conf->headers(),
'authorization' => $conf->basicAuthorization(),
'data_post' => $conf->bodyBond(),
];
$parameters_mf = [
'headers' => $conf->headers(),
'authorization' => $conf->basicAuthorization(),
'data_post' => $conf->bodyMutualFund(),
];
$response_bond = beforeSendRequest($conf->urlExternalBond(), 'POST', $parameters_bond);
$response_mf = beforeSendRequest($conf->urlExternalMutualFund(), 'POST', $parameters_mf);
}
$raw_result_bond = json_decode($response_bond);
$raw_result_mf = json_decode($response_mf);
// Development
if ($conf->isDevelopment()) {
$raw_result_bond = json_decode($raw_result_bond[0]->field_bonds);
$raw_result_mf = json_decode($raw_result_mf[0]->field_api);
}
$BondsProductList = $raw_result_bond->BondsProductInqRs->BondsProductList;
$MFProductInqList = $raw_result_mf->MFProductInqRs->MFProductInqList;
// Bond data store to internal
if ($BondsProductList !== null) {
$bond_datas = [];
foreach ($BondsProductList as $row => $content) {
$bond_datas[] = [
'AskPrice' => number_format($content->AskPrice, 1, '.', ','),
'BidPrice' => number_format($content->BidPrice, 1, '.', ','),
'BuySettle' => number_format($content->BuySettle, 1, '.', ','),
'CouponFreqCode' => $content->CouponFreqCode,
'CouponFreqID' => number_format($content->CouponFreqID),
'CouponRate' => number_format($content->CouponRate, 2, '.', ','),
'IDCurrency' => $content->IDCurrency,
'LastCoupon' => $content->LastCoupon,
'MaturityDate' => $content->MaturityDate,
'MinimumBuyUnit' => number_format($content->MinimumBuyUnit),
'MultipleOfUnit' => number_format($content->MultipleOfUnit),
'NextCoupon' => $content->NextCoupon,
'Penerbit' => $content->Penerbit,
'ProductCode' => $content->ProductCode,
'ProductName' => $content->ProductName,
'ProductAlias' => $content->ProductAlias,
'RiskProfile' => $content->RiskProfile,
'SellSettle' => $content->SellSettle
];
}
$insert_data = $ods->setData(
'bond',
[
'AskPrice', 'BidPrice', 'BuySettle', 'CouponFreqCode', 'CouponFreqID', 'CouponRate', 'IDCurrency',
'LastCoupon', 'MaturityDate', 'MinimumBuyUnit', 'MultipleOfUnit', 'NextCoupon', 'Penerbit',
'ProductCode', 'ProductName', 'ProductAlias', 'RiskProfile', 'SellSettle'
],
$bond_datas
);
if ($insert_data) {
// make response as JSON File and store the file
$ods->makeJsonFile($bond_datas, 'feeds/bonds', 'bond.json');
}
}
// Mutual Fund data store to internal
if ($MFProductInqList !== null) {
$mf_datas = [];
foreach ($MFProductInqList as $row => $content) {
$mf_datas[] = [
'ProductCode' => $content->ProductCode,
'ProductName' => $content->ProductName,
'ProductCategory' => $content->ProductCategory,
'ProductType' => $content->ProductType,
'Currency' => $content->Currency,
'Performance1' => $content->field_1_tahun_mf,
'Performance2' => $content->Performance2,
'Performance3' => $content->Performance3,
'Performance4' => $content->Performance4,
'Performance5' => $content->Performance5,
'UrlProspektus' => $content->UrlProspektus,
'UrlFactSheet' => $content->UrlFactSheet,
'UrlProductFeatureDocument' => $content->UrlProductFeatureDocument,
'RiskProfile' => $content->RiskProfile,
'FundHouseName' => $content->FundHouseName,
'NAVDate' => $content->NAVDate,
'NAVValue' => $content->NAVValue
];
}
$insert_data_mf = $ods->setData(
'mutual_fund',
[
'ProductCode', 'ProductName', 'ProductCategory', 'ProductType', 'Currency', 'Performance1', 'Performance2', 'Performance3',
'Performance4', 'Performance5', 'UrlProspektus', 'UrlFactSheet', 'UrlProductFeatureDocument', 'RiskProfile', 'FundHouseName',
'NAVDate', 'NAVValue'
],
$mf_datas
);
if ($insert_data_mf) {
// make response as JSON File and store the file
$ods->makeJsonFile($mf_datas, 'feeds/mf', 'mutual_fund.json');
}
}
// console log
\Drupal::logger('ods')->notice($message);
}
So can I store the data to pristine drupal 8 table?
First, you need to create the content type in the Drupal 8 backend going to Structure > Content type.
Second you can add a node programmatically like this
use Drupal\node\Entity\Node;
$node = Node::create(array(
'type' => 'your_content_type',
'title' => 'your title',
'langcode' => 'en',
'uid' => '1',
'status' => 1,
'body'=> 'your body',
));
$node->save();

Trouble with PHP get contents + API

I am trying to create an Adfly shortener website for my associates to use using PHP. I thought this would be simple, but I am having some trouble getting this to work
$longURL = $_GET['longurl'];
$shortURL = shortAdfly($longURL);
// Print the result
print_r($shortURL);
// Adf.ly shortener
function shortAdfly($ToConvert) {
urlencode($ToConvert);
$apiUrl = 'http://api.adf.ly/api.php?' . http_build_query([
'key' => 'My API Key',
'uid' => 'My User ID',
'advert_type' => 'int',
'domain' => 'adf.ly',
'url' => $ToConvert ]);
$short_url = file_get_contents($apiUrl);
return $short_url;
}
Building URL query strings by hand is like encoding JSON by hand; you just shouldn't do it. Use the tools provided
$apiUrl = 'http://api.adf.ly/api.php?' . http_build_query([
'key' => 'your-api-key',
'uid' => 1234567,
'advert_type' => 'int',
'domain' => 'adf.ly',
'url' => $ToConvert
]);
return file_get_contents($apiUrl);

Storing key value dictionary in Elgg

I am working on a plugin for Elgg that keeps track of device ids that are sent to the application when logging in from your mobile phone. For this, I would like to store these device ids in the database and would like to use ElggObjects for this.
This is what I do now:
function initialize() {
$androidTokens = elgg_get_entities(array(
'type' => 'object',
'subtype' => 'androidTokens',
'limit' => 0
));
$iosTokens = elgg_get_entities(array(
'type' => 'object',
'subtype' => 'iosTokens',
'limit' => 0
));
if ($androidTokens == 0) {
$tokenObject = new ElggObject();
$tokenObject->subtype = 'androidTokens';
$tokenObject->tags = array();
$tokenObject->save();
}
if ($iosTokens == 0) {
$tokenObject = new ElggObject();
$tokenObject->subtype = 'iosTokens';
$tokenObject->tags = array();
$tokenObject->save();
}
}
So this generates two ElggObjects that hold ids for android and for ios devices, stored in the metadata field tags. This array of tags can however not be retrieved anymore. When I do:
$tokenObject = elgg_get_entities(array(
'type' => 'object',
'subtype' => $os.'Tokens',
'limit' => 0
));
$tokens = $tokenObject->tags
tokens remains empty. Does someone know what I am doing wrong? Am I using the Elgg objects wrong?
I think the reason you're running into issues there is that elgg_get_entities returns an array of entities.
Am I correct in assuming that you'll only ever have one of each token object subtype? (One for iOS and one for Android?)
If so, I would modify your code as follows:
function initialize() {
$androidTokens = elgg_get_entities(array(
'type' => 'object',
'subtype' => 'androidTokens',
'limit' => 1 // only expecting one entity
));
$iosTokens = elgg_get_entities(array(
'type' => 'object',
'subtype' => 'iosTokens',
'limit' => 1 // only expecting one entity
));
if (count($androidTokens) == 0) {
$tokenObject = new ElggObject();
$tokenObject->subtype = 'androidTokens';
$tokenObject->tags = array();
$tokenObject->save();
}
if (count($iosTokens) == 0) {
$tokenObject = new ElggObject();
$tokenObject->subtype = 'iosTokens';
$tokenObject->tags = array();
$tokenObject->save();
}
}
Later, when grabbing the entity:
$tokenObject = elgg_get_entities(array(
'type' => 'object',
'subtype' => $os.'Tokens',
'limit' => 1 // only grab one
));
$tokens = $tokenObject[0]->tags; // get tag data for first (and only) entity

Curl Exception 7 PHP and Guzzle with Elasticsearch

I am trying to index documents using the php client for elastic search which uses Guzzle. After compiling my php script I am getting an error that says Internal Server Error, code 500. After doing some research this seems to be an issue with connecting to a server but the strange part is that everything I'm trying to do is set up on the same machine. My instance of Elasticsearch, my documents I'm trying to index, and my php scripts are all saved and running on the same machine. This is my PHP Script:
<?php
require '/home/aharmon/vendor/autoload.php';
$client = new Elasticsearch\Client();
$root = realpath('/home/aharmon/elkdata/for_elk_test_2014_11_24/Agencies');
$iter = new RecursiveIteratorIterator(
new RecursiveDirectoryIterator($root, RecursiveDirectoryIterator::SKIP_DOTS),
RecursiveIteratorIterator::SELF_FIRST,
RecursiveIteratorIterator::CATCH_GET_CHILD);
$paths = array($root);
foreach ($iter as $path => $dir) {
if ($dir -> isDir()) {
$paths[] = $path;
}
}
//Create the index and mappings
$mapping['index'] = 'rvuehistoricaldocuments2009-2013'; //mapping code
$mapping['body'] = array (
'mappings' => array (
'documents' => array (
'_source' => array (
'enabled' => true
),
'properties' => array(
'doc_name' => array(
'type' => 'string',
'analyzer' => 'standard'
),
'description' => array(
'type' => 'string'
)
)
)
)
);
//Now index the documents
for ($i = 0; $i <= 10000; $i++) {
$params ['body'] [] = array(
'index' => array(
'_id' => $i
)
);
$params ['body'] [] = array(
'type' => 'documents',
'body' => array(
'foo' => 'bar'//Document body goes here
)
);
//Every 1000 documents stop and send the bulk request.
if($i % 1000) {
$responses = $client->bulk($params);
// erase the old bulk request
$params = array();
// unset the bulk response when you are done to save memory
unset($responses);
}
}
$client ->indices()->create($mapping)
?>
If anyone has seen this before or has an inclination as to what the issue the help would be greatly appreciated. I had a similar issue before when I tried to set up SSH but I got the firewall all configured and got SSH working so I'm not sure why this is happening.
check this link it is ok for me :
http://www.elastic.co/guide/en/elasticsearch/client/php-api/current/_index_operations.html#_put_mappings_api
<?php
// Set the index and type
$params['index'] = 'my_index';
$params['type'] = 'my_type2';
// Adding a new type to an existing index
$myTypeMapping2 = array(
'_source' => array(
'enabled' => true
),
'properties' => array(
'first_name' => array(
'type' => 'string',
'analyzer' => 'standard'
),
'age' => array(
'type' => 'integer'
)
)
);
$params['body']['my_type2'] = $myTypeMapping2;
// Update the index mapping
$client->indices()->putMapping($params);

post big data via Zend_Http_Client_Adapter_Curl

I use $site->setParameterPost and $site->request('POST')->getBody() for posting data to an action, one parameter of setParameterPost is very big data, and it doesn't send data via post method. What can I do?
$config = array('adapter' => 'Zend_Http_Client_Adapter_Curl' );
$site = new Zend_Http_Client('http://somewhere.tld/api/news', $config);
$site->setParameterPost(array(
'news' => $news, //very big data, without it data send properly
'modelName' => 'somemodel',
'method' => 'somemethod',
'key' => 'something',
'siteName' => $sitename,
));
$sitedata = $site->request('POST')->getBody();
I should use streaming requests, which are allowed only with PUT method.
$http_client = new Zend_Http_Client ('http://something.some/thing');
$http_client->setConfig (array (
'adapter' => 'Zend_Http_Client_Adapter_Curl',
'timeout' => 180
));
$file = fopen ('news.dat', 'r'); // put all your news to the file beforehand
$http_client->setRawData ($file);
$http_client->setParameterPost (array (
'modelName' => 'somemodel',
'method' => 'somemethod',
'key' => 'something',
'siteName' => $sitename
));
$response = $http_client->request ('PUT');
On the server-side you can access your big data through
fopen ("php://input", "r");

Categories