The google docs states
The GCS stream wrapper is built in to the run time, and is used when you supply a file name starting with gs://.
When I look into the app.yaml, I see where the runtime is selected. I have selected php runtime. However when I try to write to my bucket I get an error saying the wrapper is not found for gs://. But when I try to write to my bucket using the helloworld.php script that is provided by google here https://cloud.google.com/appengine/docs/php/gettingstarted/helloworld and modifying it so that it says
<?php
file_put_contents('gs://<app_id>.appspot.com/hello.txt', 'Hello');
I have to deploy the app in order for the write to be successful. I am not understanding why I have to deploy the app everytime to get the wrapper I need to write to my bucket. How come I can not write to my bucket from a random php script?
Google says
"In the Development Server, when a Google Cloud Storage URI is specified we emulate this functionality by reading and writing to temporary files on the user's local filesystem"
So, "gs://" is simulated locally - to actually write to GCS buckets using the stream wrapper, it has to run from App Engine itself.
Try something like this:
use google\appengine\api\cloud_storage\CloudStorageTools;
$object_url = "gs://bucket/file.png";
$options = stream_context_create(['gs'=>['acl'=>'private', 'Content-Type' => 'image/png']]);
$my_file = fopen($object_url, 'w', false, $options);
fwrite($my_file, $file_data));
fclose($my_file);
Related
My appengine project (running on App Engine Standard Environment), uses a media-converter service outside of appengine. I can easily start a new converting job and get notified whenever the job is done. The media-converter delivers a temporary url to retrieve the conversion-result (mp4 file).
Now i want, to start a background-job to download this converted file from the media-converter to my google-cloud storage.
Whatever i tried so far, i cannot download larger files that 32 mb.
These are my approaches so far:
First one by just copy with file_put_contents / file_get_contents as suggested on https://cloud.google.com/appengine/docs/standard/php/googlestorage/
$options = [
'Content-Type' => 'video/mp4',
'Content-Disposition' => 'inline',
];
$context = stream_context_create(['gs' => $options]);
file_put_contents($targetPath, file_get_contents($url), 0, $context);
Then i tried to work directly with streams:
$CHUNK_SIZE = 1024*1024;
$targetStream = fopen($targetPath, 'w');
$sourceStream = fopen($url, 'rb');
if ($sourceStream === false) {
return false;
}
while (!feof($sourceStream)) {
$buffer = fread($sourceStream, $CHUNK_SIZE);
Logger::log("Chuck");
fwrite($targetStream, $buffer);
}
fclose($sourceStream);
fclose($targetStream);
Then i was surprised that this actually worked (up to 32 mb)
copy($url, $targetPath);
Im running out of ideas now. Any suggestions? I kinda need the cp function of gutil in php.
https://cloud.google.com/storage/docs/quickstart-gsutil
I think this stackoverflow issue had a similar issue:
Large file upload to google cloud storage using PHP
There is a strict limit of 32MB of data for incoming requests.
Refer to Incoming Bandwidth for more details - https://cloud.google.com/appengine/quotas. This must be the reason why you are not able to go beyond the 32MB limit.
Possible Solution -
Can you modify the media-converter service?
If yes - Create an API in the media converter service and do the cloud storage upload at the media converter service itself by invoking the endpoint from your AppEngine application. Use the service account for cloud storage authentication (https://cloud.google.com/storage/docs/authentication#storage-authentication-php).
If no - You can do the same thing using Compute Engine. Create an API in the Compute Engine where you will be passing the URL of the file in response to the background job in AppEngine. Upload to cloud storage using the service account authentication.
I am new to coding world.I want to upload a simple txt file to a bucket that is in my cloud storage. I didn't find any useful cloud documentations regarding it.
here is the not working code:
<?php
require_once 'vendor/autoload.php';
use Google\Cloud\Storage\StorageClient;
define("PROJECT_ID", 'projectID');
define("BUCKET_NAME", 'bucketname');
$client = new Google_Client();
$client->setApplicationName("API_Cloud_Storage");
$client->useApplicationDefaultCredentials();
$client->setScopes(["https://www.googleapis.com/auth/cloud-platform"]);
$service = new Google_Service_Storage($client);
$request = $service->objects->listObjects(BUCKET_NAME);
foreach ($request["items"] as $object)
printf("%s\n", $object->getName());
printf("%s\n", $object->get);
$storage = new StorageClient();
$bucket = $storage->bucket(BUCKET_NAME); // Put your bucket name here.
$filePath="C:\users\useraccount\Desktop\address_file\textfile.txt";
$objectName="textfile.txt";
$object = $bucket->upload(file_get_contents($filePath),
['name' => $objectName]);
?>
It seems that the issue is in your code and there is an answer provided in a related question that you can refer to (it also has a link to another useful documentation) that can help you correct your code [1]. You can also rely on this sample code in GitHub [2].
In terms of the concept of uploading a local file to later provide public access to write to it. I think you may need to read a bit more about serving files uploaded with your app from the filesystem [3]. I also advise that you read through this entire documentation as it talks about serving files from a script or directly from GCS [4].
[1] https://stackoverflow.com/a/48559967
[2] https://github.com/GoogleCloudPlatform/php-docs-samples/blob/master/appengine/php55/storage/app.php#L79
[3] https://cloud.google.com/appengine/docs/standard/php/googlestorage/#is_there_any_other_way_to_read_and_write_files
[4]
https://cloud.google.com/appengine/docs/standard/php/googlestorage/public_access#top_of_page
Fortunately Google App Engine (GAE) provides a built-in Google Cloud Storage (GCS) stream wrapper that allows you to use many of the standard PHP filesystem functions to read and write files in an App Engine PHP app. I am also improving my coding skills and have found tips that worked for me in the GCP Documentation.
Taking from the code snippet you provided, you have to ensure you have done at least 2 things:
that you have properly created a storage bucket in Google Cloud Storage (GCS) [1].
Make sure to replace "BUCKET_NAME" in your code with the name of the GCS bucket you created, and indicate the filepath to the text file you want to upload.
that you have configured your PHP app for writing files to your app [2], or to allow user uploads if you want other users to be able to upload files to your GCS bucket [3].
These are standard how-to guides (Tutorials are at the bottom of navigation bar) so make sure to view the examples provided on Github to take a look at the complete codes and adapt the style to yours.
Hope this helps!
[1] https://cloud.google.com/storage/docs/creating-buckets
[2] https://cloud.google.com/appengine/docs/standard/php/googlestorage/#top_of_page
[3] https://cloud.google.com/appengine/docs/standard/php/googlestorage/user_upload
We have an PHP app, where for encryption of the connection with database we need to provide 3 files that shouldn't be publicly accessible, but should be present on the server to make the DB connection (https://www.cleardb.com/developers/ssl_connections)
Obviously we don't want to store them in the SCM with the app code, so the only idea that comes to my mind is using post-deploy action hook and fetch those files from storage account (with keys and URIs provided in the app parameters).
Is there a nicer/cleaner way to achieve this? :)
Thank you,
You can try to use Custom Deployment Script to execute additional scripts or command during the deployment task. So you can create a php script whose functionality is to download the certificate files from Blob Storage to server file system location. And then in your PHP application, the DB connection can use these files.
Following are the general steps:
Enable composer extension in your portal:
Install azure-cli module via npm, refer to https://learn.microsoft.com/en-us/azure/xplat-cli-install for more info.
Create deployment script for php via command azure site deplotmentscript --php
Execute command composer require microsoft/windowsazure, make sure you have a composer.json with the storage sdk dependency.
Create php script in your root directory to download flies from Blob Storage(e.g. named run.php):
require_once 'vendor/autoload.php';
use WindowsAzure\Common\ServicesBuilder;
use MicrosoftAzure\Storage\Common\ServiceException;
$connectionString = "<connection_string>";
$blobRestProxy = ServicesBuilder::getInstance()->createBlobService($connectionString);
$container = 'certificate';
$blobs = ['client-key.pem','client-cert.pem','cleardb-ca.pem'];
foreach($blobs as $k => $b){
$blobresult = $blobRestProxy->getBlob($container, $b);
$source = stream_get_contents($blobresult->getContentStream());
$result = file_put_contents($b, $source);
}
Modify the deploy.cmd script, add santence php run.php under the step KuduSync.
Deploy your application to Azure Web App via Git.
Any further concern, please feel free to let me know.
I'm trying to generate a PHP client for my Google Cloud Endpoints API using the Google APIs Client Generator but it just creates an empty directory instead.
The command I'm using looks like:
generate_library --language=php --language_variant=stable --output_dir=/path/php-client --input=/path/myApi-v1.json
It seems to work when I change the language to csharp and java. I turned on the verbose flag and don't see any errors, only tracing messages like:
DEBUG:codegen:Create: myMethod, parent=update
DEBUG:codegen:Schema.Create: updateRequestContent => MyMessage
DEBUG:codegen:DataTypeFromJson: add MyMessage to cache
Searching around I see someone at the AppEngine sub Reddit posted a similar issue with no response.
I used another approach:
downloaded the zip from
https://github.com/google/apis-client-generator
Extracted the zip file in a directory (i named it client-generator)
Executed the generate.sh script available in the extracted files.
/path/client-generator/generate.sh --input=/path/rest.json --language=php --output_dir=/path/output
The APICLassName.php file is generated
I tried again and changed the --language_variant argument from stable to 1.1.4 and it now works fine.
So ive followed the guide on how to use GCS on their site: https://developers.google.com/appengine/docs/php/googlestorage/
But the following code does not work. I cannot access my bucket, and the CloudStorageTools is not even found.
require_once 'google/appengine/api/cloud_storage/CloudStorageTools.php';
use google\appengine\api\cloud_storage\CloudStorageTools;
// check if GCS bucket if writable
$my_bucket = 'gs://my_bucket/';
$check1 = is_writable($my_bucket); //returns nothing
class_exists("CloudStorageTools"); //returns false
I also added google_app_engine.allow_include_gs_buckets into the php.ini file. Still no support for GCS.
Anyone have some running code they could share?
I think it may have something to do with permissions set to your bucket. Make sure you added your app account email Application Settings -> Service Account Name.
I ported my app with no issues and using Smarty to write into and include from the bucket. I did not have to declare:
require_once 'google/appengine/api/cloud_storage/CloudStorageTools.php';
use google\appengine\api\cloud_storage\CloudStorageTools;
and the app reads and writes to the bucket with no problems.
I have had however issues with some WordPress porting, when although the bucket existed and had properly setup access permission, for some reason app throws a Fatal Error and I can see it in the app logs, that application was trying to access the cloud storage via gs:// wrapper.
Hope that helps.
Hi dude make sure you bucket is Public Write enabled
use this gsutil command to do so (google how to configure gsutil utility)
$ gsutil acl ch -u AllUsers:W gs://example-bucket-name