Create .CRX (chrome extension / webapp ) file on demand using PHP - php

I need to create CRX file on the fly. It's for my CMS backend, so it will be just for authenticated users who can install CMS backend as webapp and offer some more privileges to the web app. The problem is, that the backend is used for many domains so creating CRX file for each of them is quite a work. So I figured that it would be easier to just create CRX file on demand which would be generated by PHP using its own domain and probably custom icon.

On the documentation page, they explain the CRX package format. There are many third party libraries that implemented that format. In the following page, you can learn the format and either download a Ruby / Bash script (you can find others too online), and if you want to implement your own packager, you can follow the format described there.
https://developer.chrome.com/extensions/crx
If you really don't want to follow the format, you can let your PHP script do one of the following:
Use Chrome binary chrome.exe --pack-extension=c:\myext --pack-extension-key=c:\myext.pem
Use the Ruby or Bash script from PHP (you can call system commands)
Hope that helps!

Also, for anyone still looking for a way to create a CTX in PHP, look at this question: Create Google Chrome Crx file with PHP

This works for me :D I just change from real path to null without that changes won't work on new chrome :D
/**
* Class CrxGenerator
*
* Create Chrome Extension CRX packages from
* folder & pem private key
*
* Based on CRX format documentation: http://developer.chrome.com/extensions/crx.html
*
* #author: Tomasz Banasiak
* #license: MIT
* #date: 2013-11-03
*/
class CrxGenerator {
const TEMP_ARCHIVE_EXT = '.zip';
private $sourceDir = null;
private $cacheDir = '';
private $privateKeyContents = null;
private $publicKeyContents = null;
private $privateKey = null;
private $publicKey = null;
/**
* #param $file Path to PEM key
* #throws Exception
*/
public function setPrivateKey($file) {
if (!file_exists($file)) {
throw new Exception('Private key file does not exist');
}
$this->privateKeyContents = file_get_contents($file);
$this->privateKey = $file;
}
/**
* #param $file Path to PUB key
* #throws Exception
*/
public function setPublicKey($file) {
if (!file_exists($file)) {
throw new Exception('Private key file does not exist');
}
$this->publicKeyContents = file_get_contents($file);
$this->publicKey = $file;
}
/**
* #param $cacheDir dir specified for caching temporary archives
* #throws Exception
*/
public function setCacheDir($cacheDir) {
if (!is_dir($cacheDir)) {
throw new Exception('Cache dir does not exist!');
}
$this->cacheDir = $cacheDir;
}
/**
* #param $sourceDir Extension source directory
*/
public function setSourceDir($sourceDir) {
$this->sourceDir = $sourceDir;
}
/**
* #param $outputFile path to output file
* #throws Exception
*/
public function generateCrx($outputFile) {
$basename = basename($outputFile);
// First step - create ZIP archive
$zipArchive = $this->cacheDir . DIRECTORY_SEPARATOR . $basename . self::TEMP_ARCHIVE_EXT;
$result = $this->createZipArchive(
$this->sourceDir,
$zipArchive
);
if (!$result) {
throw new Exception('ZIP creation failed');
}
$zipContents = file_get_contents($zipArchive);
// Second step - create file signature
$privateKey = openssl_pkey_get_private($this->privateKeyContents);
openssl_sign($zipContents, $signature, $privateKey, 'sha1');
openssl_free_key($privateKey);
// Create output file
$crx = fopen($outputFile, 'wb');
fwrite($crx, 'Cr24');
fwrite($crx, pack('V', 2));
fwrite($crx, pack('V', strlen($this->publicKeyContents)));
fwrite($crx, pack('V', strlen($signature)));
fwrite($crx, $this->publicKeyContents);
fwrite($crx, $signature);
fwrite($crx, $zipContents);
fclose($crx);
// Clear cache
unset($zipArchive);
}
/**
* #param $source - source dir
* #param $outputFile - output file
* #return bool - success?
*/
private function createZipArchive($source, $outputFile) {
if (!extension_loaded('zip') || !file_exists($source)) {
return false;
}
$zip = new ZipArchive();
if (!$zip->open($outputFile, ZIPARCHIVE::CREATE)) {
return false;
}
$source = str_replace('\\', '/', realpath($source));
if (is_dir($source) === true) {
$files = new RecursiveIteratorIterator(new RecursiveDirectoryIterator($source), RecursiveIteratorIterator::SELF_FIRST);
foreach ($files as $file) {
$file = str_replace('\\', '/', $file);
// Exclude "." and ".." folders
if( in_array(substr($file, strrpos($file, '/') + 1), array('.', '..')) ) {
continue;
}
$file = $file;
if (is_dir($file) === true) {
$zip->addEmptyDir(str_replace($source . '/', '', $file . '/'));
}
else if (is_file($file) === true) {
$zip->addFromString(str_replace($source . '/', '', $file), file_get_contents($file));
}
}
}
else if (is_file($source) === true) {
$zip->file_get_contents($source);
echo $source;
}
return $zip->close();
}
}

Looks like I have found exactly what I was looking for. Chrome team has made this option to create CRX-less web apps, just by using simple manifest file.
Much easier to create own webapp and publish it on website for installation. And it also solves my problem when I have many websites with a lot of domains and I don't have to create custom CRX file for each domain. I just create small PHP script which creates manifest files on the fly for each domain.

Related

manually creating an Symfony UploadedFile

I'm facing following problem and can't seem to figure this one out.
I wrote an API endpoint accepting a POST with binary data (header: content-type:image/jpeg).
I know i can read out the raw string with file_get_content('php://input') or Laravel's $request->getContent().
PHP also has a function createimagefromstring($string) which also seems to read the string in correctly.
What i'd like to do is create an UploadedFile from this raw data , so that I can handle it with already written functions.
Is this possible?
Thank you in advance
I think I found it... Still curious if there are improvements that can be made..
$imgRaw = imagecreatefromstring( $request->getContent() );
if ($imgRaw !== false) {
imagejpeg($imgRaw, storage_path().'/tmp/tmp.jpg',100);
imagedestroy($imgRaw);
$file = new UploadedFile( storage_path().'/tmp/tmp.jpg', 'tmp.jpg', 'image/jpeg',null,null,true);
// DO STUFF WITH THE UploadedFile
}
You can try to use base64 encoding. Symfony have some nice stuff for this.
Your code will be smthng like this:
$base64Content = $request->request->get('base64Content'); // this is your post data
$yourFile = new UploadedBase64EncodedFile(new Base64EncodedFile($base64Content)); // this is an instance of UploadedFile
Hope it helps!
As per Laravel 8
Just follow the constructor:
* #param string $path The full temporary path to the file
* #param string $originalName The original file name of the uploaded file
* #param string|null $mimeType The type of the file as provided by PHP; null defaults to application/octet-stream
* #param int|null $error The error constant of the upload (one of PHP's UPLOAD_ERR_XXX constants); null defaults to UPLOAD_ERR_OK
* #param bool $test Whether the test mode is active
$file = new UploadedFile(
$pathIncludingFilename,
$fileName,
'image/jpeg',
null,
false
);
There is no need to manually create it, Symfony parses received $_FILES array for you. Http Request object has a FileBag property called $files with a get() method which returns an UploadedFile instance.
/** #var UploadedFile $file */
$file = $request->files->get('user-pictures-upload')[0];
$cmd = new UploadPictureCmd($file, $this->getUser()->getId());
Here is the example of generating images files using fzaninotto/faker in Symfony 4 Fixtures
class FileFixtures extends Fixture
{
private $faker;
private $parameterBag;
public function __construct(ParameterBagInterface $parameterBag)
{
$this->faker = Factory::create();
$this->parameterBag = $parameterBag;
}
public function load(ObjectManager $manager)
{
$tempFixturesPath = $this->parameterBag->get('kernel.project_dir') . '/tmp';
if (!file_exists($tempFixturesPath)) {
mkdir($tempFixturesPath);
}
$fileName = $this->faker->image($tempFixturesPath, 640, 480, 'cats', false, true);
$file = new UploadedFile($tempFixturesPath . '/' . $fileName, $fileName, 'image/jpeg', null, null, true);
//do something with $file
}
}
If it counts for anything, this is how I did it in Laravel 5.4. In my case, I wanted to be able to easily resize an image and be able to do something like this.
request()->file('image')->resize(250, 250)->store('path/to/storage');
This is what I did to the UploadedFile class.
Illuminate\Http\UploadedFile.php ~this file ships with the Laravel framework
public function resize($w, $h) {
$image = Intervention::make($this)->fit($w, $h)->save($this->getPathname());
$filesize = filesize($this->getPathname());
return new static(
$this->getPathname(),
$this->getClientOriginalName(),
$this->getClientMimeType(),
$filesize,
null,
false
);
}
Using Intervention, I resized the image that is stored in the /tmp/ folder when files are uploaded and then I saved it in the same place. Now all I do after that is create an instance of UploadedFile so that I can keep using Laravel's methods on request()->file('image'). Hope this helps.

Google Drive SDK downloadFile - What does it do?

I am developing in Drupal and using the php code from the Google Drive SDK that is supposed to download a file:
/**
* Download a file's content.
*
* #param Google_DriveService $service Drive API service instance.
* #param File $file Drive File instance.
* #return String The file's content if successful, null otherwise.
*/
function downloadFile($service, $file) {
$downloadUrl = $file->getDownloadUrl();
if ($downloadUrl) {
$request = new Google_HttpRequest($downloadUrl, 'GET', null, null);
$httpRequest = Google_Client::$io->authenticatedRequest($request);
if ($httpRequest->getResponseHttpCode() == 200) {
return $httpRequest->getResponseBody();
} else {
// An error occurred.
return null;
}
} else {
// The file doesn't have any content stored on Drive.
return null;
}
}
It is returning a successful response code of 200 but I cannot tell that it has actually done anything. However, I can tell from observing my LAN connection status that something of the right size in bytes is being downloaded. I just don't know where it is ending up. I am using Acquia Dev Desktop so I am running the website on my PC, but I cannot find where in the website folder the downloaded file is ending up if it is indeed being stored. I then tried another function that I found here and got similar results. Can someone more familiar with this explain what is happening?
function GetFile($service, $fileId) {
$fileVars = null;
try {
/*
* Retrieve metadata for the file specified by $fileId.
*/
$file = $service->files->get($fileId);
$fileVars = get_object_vars($file);
/*
* Retrieve the file's content using download URL specified in metadata.
*/
$downloadUrl = $file->getDownloadUrl();
error_log('Download URL file from Drive: ' . $downloadUrl);
if ($downloadUrl) {
$request = new Google_HttpRequest($downloadUrl, 'GET', null, null);
$httpRequest = Google_Client::$io->authenticatedRequest($request);
error_log(print_r($httpRequest, 1));
if ($httpRequest->getResponseHttpCode() == 200) {
$content = $httpRequest->getResponseBody();
$fileVars['content'] = $content?($content):'';
} else {
// An error occurred.
return null;
}
} else {
// The file doesn't have any content stored on Drive.
return null;
}
} catch (apiServiceException $e) {
/*
* Log error and re-throw
*/
error_log('Error retrieving file from Drive: ' . $e->getMessage());
throw $e;
}
return json_encode($fileVars);
}
This method returns the following results:
{"alternateLink":"https:\/\/docs.google.com\/file\/d\/0B6ahcE1NPpIaMTJzcFRLMnNiYU0\/edit?usp=drivesdk","appDataContents":false,"createdDate":"2013-12-05T21:05:03.026Z","defaultOpenWithLink":null,"description":"Flashum mp4","downloadUrl":"https:\/\/doc-0g-50-docs.googleusercontent.com\/docs\/securesc\/apic76cj8gku48a9ogarn9khpku0s46q\/gc9ck66nc600ebdk0ak9f58gkthnnacm\/1386345600000\/08295678552528470768\/15372451357194678536\/0B6ahcE1NPpIaMTJzcFRLMnNiYU0?h=16653014193614665626&e=download&gd=true","editable":true,"embedLink":"https:\/\/video.google.com\/get_player?ps=docs&partnerid=30&docid=0B6ahcE1NPpIaMTJzcFRLMnNiYU0&BASE_URL=https:\/\/docs.google.com\/","etag":"\"maCRbN5nR56FjQPLihEiz9nzpho\/MTM4NjI3NzUwNDU1MA\"","explicitlyTrashed":null,"exportLinks":null,"fileExtension":"mp4","fileSize":"788456","headRevisionId":"0B6ahcE1NPpIaeVJ5b3RxdExvamlDdWNrcGFvWXdvaWg3VU5JPQ","iconLink":"https:\/\/ssl.gstatic.com\/docs\/doclist\/images\/icon_11_video_list.png","id":"0B6ahcE1NPpIaMTJzcFRLMnNiYU0","imageMediaMetadata":null,"indexableText":null,"kind":"drive#file","labels":{"hidden":false,"restricted":false,"starred":false,"trashed":false,"viewed":false},"lastModifyingUser":{"displayName":"C. David Young","isAuthenticatedUser":false,"kind":"drive#user","permissionId":"08295678552528470768","picture":{"url":"https:\/\/lh6.googleusercontent.com\/-tjPzr0pfd_4\/AAAAAAAAAAI\/AAAAAAAAAWc\/DZtQHXrxkgQ\/s64\/photo.jpg"}},"lastModifyingUserName":"C. David Young","lastViewedByMeDate":null,"md5Checksum":"00701d2dd7a1b99e8ebb68cf62305b0d","mimeType":"application\/video","modifiedByMeDate":null,"modifiedDate":"2013-12-05T21:05:04.550Z","openWithLinks":null,"originalFilename":"charlesdavidyoung#gmail.com#test.mp4","ownerNames":["C. David Young"],"owners":[{"displayName":"C. David Young","isAuthenticatedUser":false,"kind":"drive#user","permissionId":"08295678552528470768","picture":{"url":"https:\/\/lh6.googleusercontent.com\/-tjPzr0pfd_4\/AAAAAAAAAAI\/AAAAAAAAAWc\/DZtQHXrxkgQ\/s64\/photo.jpg"}}],"parents":[{"id":"0B6ahcE1NPpIaQlMyWFBPamZIeTA","isRoot":false,"kind":"drive#parentReference","parentLink":"https:\/\/www.googleapis.com\/drive\/v2\/files\/0B6ahcE1NPpIaQlMyWFBPamZIeTA","selfLink":"https:\/\/www.googleapis.com\/drive\/v2\/files\/0B6ahcE1NPpIaMTJzcFRLMnNiYU0\/parents\/0B6ahcE1NPpIaQlMyWFBPamZIeTA"}],"quotaBytesUsed":"788456","selfLink":"https:\/\/www.googleapis.com\/drive\/v2\/files\/0B6ahcE1NPpIaMTJzcFRLMnNiYU0","shared":true,"sharedWithMeDate":null,"thumbnail":null,"thumbnailLink":"https:\/\/lh4.googleusercontent.com\/Ja2DZ3vzXSTn7kY_xY8VmC2N7nCeBrbvnkr-lO4GbMuOmpETwLAYlaC7qr5fOpBTTQ=s220","title":"charlesdavidyoung#gmail.com#test.mp4","userPermission":{"additionalRoles":null,"authKey":null,"etag":"\"maCRbN5nR56FjQPLihEiz9nzpho\/8jq33BWTNkV1S9Wn1TTkFYEVpVc\"","id":"me","kind":"drive#permission","name":null,"photoLink":null,"role":"writer","selfLink":"https:\/\/www.googleapis.com\/drive\/v2\/files\/0B6ahcE1NPpIaMTJzcFRLMnNiYU0\/permissions\/me","type":"user","value":null,"withLink":null},"webContentLink":"https:\/\/docs.google.com\/uc?id=0B6ahcE1NPpIaMTJzcFRLMnNiYU0&export=download","webViewLink":null,"writersCanShare":true,"copyable":true,"content":null}
If you pass to your script a document id it should return you a download link under the field exportLinks. The fact that your exportLinks is null means that you cannot download the file (permissions?).

php zip include only new files

I've run into an issue with the php zip library causing an error 500 if the file is growing to larger than 500MB, probably memory related...
But I tried to cmd line the zip creation, which works well on my server.
<?php
set_time_limit(10000);
// Make Zip name
$zipname = "main_backup.zip";
// Make a zip file
$cmd = `zip -r $zipname * -x filelist.php -x $zipname`;
if($cmd){
echo 'zip created';
}
else{
echo 'failed';
}
unlink(__FILE__);
?>
I know how to exclude files and folders, but is there a way to zip only files based on the modified time using this approach?
I've googled for hours and came up empty.
for the sake of it, here's the code that was creating the error 500. My site is about 1.8GB, it always errors at 500MB~. I should note the error log is blank, so the cause of the error I'm just assuming to be RAM limit problems.
<?php
$zip = new ZipArchive;
$zip_name = "test.zip";
$res = $zip->open($zip_name, ZipArchive::CREATE);
/**
* Real Recursive Directory Iterator
*/
class RRDI extends RecursiveIteratorIterator {
/**
* Creates Real Recursive Directory Iterator
* #param string $path
* #param int $flags
* #return DirectoryIterator
*/
public function __construct($path, $flags = 0) {
parent::__construct(new RecursiveDirectoryIterator($path, $flags));
}
}
/**
* Real RecursiveDirectoryIterator Filtered Class
* Returns only those items which filenames match given regex
*/
class AdvancedDirectoryIterator extends FilterIterator {
/**
* Regex storage
* #var string
*/
private $regex;
/**
* Creates new AdvancedDirectoryIterator
* #param string $path, prefix with '-R ' for recursive, postfix with /[wildcards] for matching
* #param int $flags
* #return DirectoryIterator
*/
public function __construct($path, $flags = 0) {
if (strpos($path, '-R ') === 0) { $recursive = true; $path = substr($path, 3); }
if (preg_match('~/?([^/]*\*[^/]*)$~', $path, $matches)) { // matched wildcards in filename
$path = substr($path, 0, -strlen($matches[1]) - 1); // strip wildcards part from path
$this->regex = '~^' . str_replace('*', '.*', str_replace('.', '\.', $matches[1])) . '$~'; // convert wildcards to regex
if (!$path) $path = '.'; // if no path given, we assume CWD
}
parent::__construct($recursive ? new RRDI($path, $flags) : new DirectoryIterator($path));
}
/**
* Checks for regex in current filename, or matches all if no regex specified
* #return bool
*/
public function accept() { // FilterIterator method
return $this->regex === null ? true : preg_match($this->regex, $this->getInnerIterator()->getFilename());
}
}
foreach (new AdvancedDirectoryIterator('-R *') as $i){
//$fullpath = str_replace(',','',$i->getPath()).'/'.$i->getFilename();
//echo $fullpath.'<br />';
if ($i->isFile() && $i->getFilename()!='filelist.php') {
//echo $i->getFilename() . " " . $i->getMTime() . "<br />";
if($i->getMTime()>='0'){
$array[] = substr($i->getPathname(), 2);
}
}
};
// will output all php files in CWD and all subdirectories
foreach($array as $files) {
$zip_array[] = files;
$zip->addFile($files);
}
$zip->close();
echo 'done';
?>
You can use the -t or -tt option for the zip command and have your modifed date stored as a variable or just pass one in.
-t with the format mmddyyyy for from-date
-tt with the format mmddyyyy for before-date
//Zips up all files in current directory that were dated 08152013 or later
zip -r -t 08152013 $zipname * -x filelist.php -x $zipname

Why some classes are not loaded from my libraries on CodeIgniter

I'm trying to integrate Evernote SDK to my CodeIgniter web application and some classes from the library are loaded and others not, :-S I can't see why.
I have that simple piece of code:
$access_token = 'my validated access token ';
// Get User Store
$userStoreTrans;
try{
$userStoreTrans = new THttpClient(USER_STORE_HOST, USER_STORE_PORT, USER_STORE_URL, USER_STORE_PROTO);
}
catch(TTransportException $e)
{
print $e->errorCode.' Message:'.$e->parameter;
}
$userStoreProt = new TBinaryProtocol($userStoreTrans);
$userStoreClient = new UserStoreClient($userStoreProt, $userStoreProt);
While $userStoreTrans and $userStoreProt are created correctly, a THttpClient and TBinaryProtocol classes from Evernote SDK, $userStoreClient throws a Class 'UserStoreClient' not found in .../home.php
I don't understand why some classes are recognized and some others not, the main difference that I can see is that "TClasses" are under evernote-sdk-php/lib/transport/*.php and evernote-sdk-php/lib/protocol/*.php and UserStoreClient has an extra folder evernote-sdk-php/lib/packages/UserStore/*.php
I'll explain how I'm including evernote-sdk-php to my CodeIgniter installation:
This is my CodeIgniter config/autoload.php
$autoload['libraries'] = array('database','session','form_validation','security','tank_auth','Evernote_bootstrap');
This is my Evernote_bootstrap.php file
<?php if (!defined('BASEPATH')) exit('No direct script access allowed');
define("EVERNOTE_LIBS", dirname(__FILE__) . DIRECTORY_SEPARATOR . "evernote-sdk-php/lib");
// add ourselves to include path
ini_set("include_path", ini_get("include_path") . ":" . EVERNOTE_LIBS);
require_once("evernote-sdk-php/lib/autoload.php");
require_once("evernote-oauth/functions.php");
class Evernote_Bootstrap
{
function __construct()
{
// log_message('debug','Evernote_Bootstrap');
}
}
?>
The main purpose of Evernote_Bootstrap class is the require_once of evernote-sdk-php/lib/autoload.php, this class is auto-generated using the -phpa Thrift generator flag, I only add some logging to see the problem.
autoload.php:
<?php
/**
* Copyright (c) 2006- Facebook
* Distributed under the Thrift Software License
*
* See accompanying file LICENSE or visit the Thrift site at:
* http://developers.facebook.com/thrift/
*
* #package thrift
* #author Mark Slee <mcslee#facebook.com>
*/
/**
* Include this file if you wish to use autoload with your PHP generated Thrift
* code. The generated code will *not* include any defined Thrift classes by
* default, except for the service interfaces. The generated code will populate
* values into $GLOBALS['THRIFT_AUTOLOAD'] which can be used by the autoload
* method below. If you have your own autoload system already in place, rename your
* __autoload function to something else and then do:
* $GLOBALS['AUTOLOAD_HOOKS'][] = 'my_autoload_func';
*
* Generate this code using the -phpa Thrift generator flag.
*/
/**
* This parses a given filename for classnames and populates
* $GLOBALS['THRIFT_AUTOLOAD'] with key => value pairs
* where key is lower-case'd classname and value is full path to containing file.
*
* #param String $filename Full path to the filename to parse
*/
function scrapeClasses($filename) {
$fh = fopen($filename, "r");
while ($line = fgets($fh)) {
$matches = array();
if ( preg_match("/^\s*class\s+([^\s]+)/", $line, $matches)) {
if (count($matches) > 1)
$GLOBALS['THRIFT_AUTOLOAD'][strtolower($matches[1])] = $filename;
}
}
}
function findFiles($dir, $pattern, &$finds) {
if (! is_dir($dir))
return;
if (empty($pattern))
$pattern = "/^[^\.][^\.]?$/";
$files = scandir($dir);
if (!empty($files)) {
foreach ($files as $f) {
if ($f == "." || $f == "..")
continue;
if ( is_file($dir . DIRECTORY_SEPARATOR . $f) && preg_match($pattern, $f)) {
$finds[] = $dir . DIRECTORY_SEPARATOR . $f;
} else if ( is_dir($dir . DIRECTORY_SEPARATOR . $f) && substr($f, 0, 1) != ".") {
findFiles($dir . DIRECTORY_SEPARATOR . $f, $pattern, $finds);
}
}
}
}
function str_var_dump($object)
{
ob_start();
var_dump($object);
$dump = ob_get_clean();
return $dump;
}
// require Thrift core
require_once(dirname(__FILE__) . DIRECTORY_SEPARATOR . "Thrift.php");
if (! isset($GLOBALS['THRIFT_ROOT']))
$GLOBALS['THRIFT_ROOT'] = dirname(__FILE__);
log_message('debug','bootstrap autoload.php is executing');
// stuff for managing autoloading of classes
$GLOBALS['THRIFT_AUTOLOAD'] = array();
$GLOBALS['AUTOLOAD_HOOKS'] = array();
$THRIFT_AUTOLOAD =& $GLOBALS['THRIFT_AUTOLOAD'];
// only populate if not done so already
if (empty($GLOBALS['THRIFT_AUTOLOAD'])) {
//$allLibs = glob( dirname(__FILE__) . "/**/*.php"); // oh poor winblows users can't use glob recursively
$allLibs = array();
findFiles( dirname(__FILE__), "/\.php$/i", $allLibs);
log_message('debug',str_var_dump($allLibs));
if (!empty($allLibs)) {
foreach ($allLibs as $libFile) {
scrapeClasses($libFile);
}
log_message('debug','all scrapped classes: ' . str_var_dump($GLOBALS['THRIFT_AUTOLOAD']));
}
}else{
log_message('debug','$GLOBALS[THRIFT_AUTOLOAD] already defined');
}
// main autoloading
if (!function_exists('__autoload')) {
function __autoload($class) {
log_message('debug','__autoload');
global $THRIFT_AUTOLOAD;
$classl = strtolower($class);
if (isset($THRIFT_AUTOLOAD[$classl])) {
// log_message('debug','$THRIFT_AUTOLOAD[$classl] is set, do require_once');
//include_once $GLOBALS['THRIFT_ROOT'].'/packages/'.$THRIFT_AUTOLOAD[$classl];
require_once($THRIFT_AUTOLOAD[$classl]);
} else if (!empty($GLOBALS['AUTOLOAD_HOOKS'])) {
log_message('debug','$GLOBALS[AUTOLOAD_HOOKS]is no empty, lets foreach');
foreach ($GLOBALS['AUTOLOAD_HOOKS'] as $hook) {
// log_message('debug','iterate');
$hook($class);
}
} else {
log_message('debug','nothing to do');
}
}
}
Then I logged that library and seems to work fine. You can see the main important logs: log_message('debug',str_var_dump($allLibs)); and log_message('debug','all scrapped classes: ' . str_var_dump($GLOBALS['THRIFT_AUTOLOAD'])); and the output: http://pastebin.com/8w1MCKx9:
As you can see, UserStore class seems to be equally fine loaded than THttpClient or TBinaryProtocol... any idea about the problem?
I don't know if is important but I notice that $GLOBALS['THRIFT_ROOT'], defined on autoload.php, is not accesible from a CI Controller. Probably I'm missing something about CI architecture.
As of our lastest SDK updates, UserStoreClient (and the other SDK classes) are in appropriate namespaces. Assuming that you're using our generated code, have you imported the classes you're using? E.g.
use EDAM\UserStore\UserStoreClient;

PHP Thumbnail Image Generator Caching: How to set If-Last-Modified/Max-Age/Last-Modified HEADERS correctly in PHP?

Even after a very high score of Google PageSpeed(97) & Yahoo! YSlow(92) the PHP generated thumbnails don't seem to be coming passively from an old cache: they seem to be generated every time again...and again... freshly baked consuming lots of waisted time.
This question will focus only & specifically on how to solve the CACHE problem of the PHP Code that generates the thumbs:
Just have a look at these tiny puny little thumbnails measuring only 3 ~ 5 kb each!
Waterfall in detail: http://www.webpagetest.org/result/110328_AM_8T00/1/details/
Any & all suggestons are +1 help to me and warmly welcome, for I have grown quite desperate on this issue for the last months. Thanx a Thousand!
Using or not Modrewrite does not influence speed: both are same. I use these rewrite conditions: RewriteCond %{REQUEST_URI} ^/IMG-.*$ & RewriteCond %{REQUEST_FILENAME} !-f
Both the original default URL as well as the beautified rewritten URL produce the same delays!! So let us not point the fault to the lightning fast Apache: its the PHP Cache / headers that are somehow wrongly coded...
Warning by webpagetest.org: Leverage browser caching of static assets: 69/100
FAILED - (No max-age or expires): http://aster.nu/imgcpu?src=aster_bg/124.jpg&w=1400&h=100&c=p
After each refresh, you will see either of these two warnings appear on random at REDbot.org
Relevant Portions of The Code:
// Script is directly called
if(isset($_GET['src']) && (isset($_GET['w']) || isset($_GET['h']) || isset($_GET['m']) || isset($_GET['f']) || isset($_GET['q']))){
$ImageProcessor = new ImageProcessor(true);
$ImageProcessor->Load($_GET['src'], true);
$ImageProcessor->EnableCache("/var/www/vhosts/blabla.org/httpdocs/tmp/", 345600);
$ImageProcessor->Parse($quality);
}
/* Images processing class
* - create image thumbnails on the fly
* - Can be used with direct url imgcpu.php?src=
* - Cache images for efficiency
*/
class ImageProcessor
{
private $_image_path; # Origninal image path
protected $_image_name; # Image name string
private $_image_type; # Image type int
protected $_mime; # Image mime type string
private $_direct_call = false; # Is it a direct url call? boolean
private $_image_resource; # Image resource var Resource
private $_cache_folder; # Cache folder strig
private $_cache_ttl; # Cache time to live int
private $_cache = false; # Cache on boolean
private $_cache_skip = false; # Cache skip var boolean
private function cleanUrl($image){ # Cleanup url
$cimage = str_replace("\\", "/", $image);
return $cimage;
}
/** Get image resource
* #access private, #param string $image, #param string $extension, #return resource */
private function GetImageResource($image, $extension){
switch($extension){
case "jpg":
#ini_set('gd.jpeg_ignore_warning', 1);
$resource = imagecreatefromjpeg($image);
break;
}
return $resource;
}
/* Save image to cache folder
* #access private, #return void */
private function cacheImage($name, $content){
# Write content file
$path = $this->_cache_folder . $name;
$fh = fopen($path, 'w') or die("can't open file");
fwrite($fh, $content);
fclose($fh);
# Delete expired images
foreach (glob($this->_cache_folder . "*") as $filename) {
if(filemtime($filename) < (time() - $this->_cache_ttl)){
unlink( $filename );
}
}
}
/* Get an image from cache
* #access public, #param string $name, #return void */
private function cachedImage($name){
$file = $this->_cache_folder . $name;
$fh = fopen($file, 'r');
$content = fread($fh, filesize($file));
fclose($fh);
return $content;
}
/* Get name of the cache file
* #access private, #return string */
private function generateCacheName(){
$get = implode("-", $_GET);
return md5($this->_resize_mode . $this->_image_path . $this->_old_width . $this->_old_height . $this->_new_width . $this->_new_height . $get) . "." . $this->_extension;
}
/* Check if a cache file is expired
* #access private, #return bool */
private function cacheExpired(){
$path = $this->_cache_folder . $this->generateCacheName();
if(file_exists($path)){
$filetime = filemtime($path);
return $filetime < (time() - $this->_cache_ttl);
}else{
return true;
}
}
/* Lazy load the image resource needed for the caching to work
* #return void */
private function lazyLoad(){
if(empty($this->_image_resource)){
if($this->_cache && !$this->cacheExpired()){
$this->_cache_skip = true;
return;
}
$resource = $this->GetImageResource($this->_image_path, $this->_extension);
$this->_image_resource = $resource;
}
}
/* Constructor
* #access public, #param bool $direct_call, #return void */
public function __construct($direct_call=false){
# Check if GD extension is loaded
if (!extension_loaded('gd') && !extension_loaded('gd2')) {
$this->showError("GD is not loaded");
}
$this->_direct_call = $direct_call;
}
/* Resize
* #param int $width, #param int $height, #param define $mode
* #param bool $auto_orientation houd rekening met orientatie wanneer er een resize gebeurt */
public function Resize($width=100, $height=100, $mode=RESIZE_STRETCH, $auto_orientation=false){
// Validate resize mode
$valid_modes = array("f", "p");
}
// .... omitted .....
// Set news size vars because these are used for the
// cache name generation
// .... omitted .....
$this->_old_width = $width;
$this->_old_height = $height;
// Lazy load for the directurl cache to work
$this->lazyLoad();
if($this->_cache_skip) return true;
// Create canvas for the new image
$new_image = imagecreatetruecolor($width, $height);
imagecopyresampled($new_image, $this->_image_resource, $dst_x, $dst_y, $src_x, $src_y, $dst_w, $dst_h, $src_w, $src_h);
// .... omitted .....
$this->_image_resource = $new_image;
}
/* Create image resource from path or url
* #access public, #param string $location, #param bool $lazy_load, #return */
public function Load($image,$lazy_load=false){
// Cleanup image url
$image = $this->cleanUrl($image);
// Check if it is a valid image
if(isset($mimes[$extension]) && ((!strstr($image, "http://") && file_exists($image)) || strstr($image, "http://")) ){
// Urlencode if http
if(strstr($image, "http://")){
$image = str_replace(array('http%3A%2F%2F', '%2F'), array('http://', '/'), urlencode($image));
}
$image = str_replace("+", "%20", $image);
$this->_extension = $extension;
$this->_mime = $mimes[$extension];
$this->_image_path = $image;
$parts = explode("/", $image);
$this->_image_name = str_replace("." . $this->_extension, "", end($parts));
// Get image size
list($width, $height, $type) = getimagesize($image);
$this->_old_width = $width;
$this->_old_height = $height;
$this->_image_type = $type;
}else{
$this->showError("Wrong image type or file does not exists.");
}
if(!$lazy_load){
$resource = $this->GetImageResource($image, $extension);
$this->_image_resource = $resource;
}
}
/* Save image to computer
* #access public, #param string $destination, #return void */
public function Save($destination, $quality=60){
if($this->_extension == "png" || $this->_extension == "gif"){
imagesavealpha($this->_image_resource, true);
}
switch ($this->_extension) {
case "jpg": imagejpeg($this->_image_resource,$destination, $quality); break;
case "gif": imagegif($this->_image_resource,$destination); break;
default: $this->showError('Failed to save image!'); break;
}
}
/* Print image to screen
* #access public, #return void */
public function Parse($quality=60){
$name = $this->generateCacheName();
$content = "";
if(!$this->_cache || ($this->_cache && $this->cacheExpired())){
ob_start();
header ("Content-type: " . $this->_mime);
if($this->_extension == "png" || $this->_extension == "gif"){
imagesavealpha($this->_image_resource, true);
}
switch ($this->_extension) {
case "jpg": imagejpeg($this->_image_resource, "", $quality); break;
case "gif": imagegif($this->_image_resource); break;
default: $this->showError('Failed to save image!'); break;
}
$content = ob_get_contents();
ob_end_clean();
}else{
if (isset ($_SERVER['HTTP_IF_MODIFIED_SINCE'])) {
if (strtotime ($_SERVER['HTTP_IF_MODIFIED_SINCE']) < strtotime('now')) {
header ('HTTP/1.1 304 Not Modified');
die ();
}
}
// change the modified headers
$gmdate_expires = gmdate ('D, d M Y H:i:s', strtotime ('now +10 days')) . ' GMT';
$gmdate_modified = gmdate ('D, d M Y H:i:s') . ' GMT';
header ("Content-type: " . $this->_mime);
header ('Accept-Ranges: bytes');
header ('Last-Modified: ' . $gmdate_modified);
header ('Cache-Control: max-age=864000, must-revalidate');
header ('Expires: ' . $gmdate_expires);
echo $this->cachedImage($name);
exit();
}
// Save image content
if(!empty($content) && $this->_cache){
$this->cacheImage($name, $content);
}
// Destroy image
$this->Destroy();
echo $content;
exit();
}
/* Destroy resources
* #access public, #return void */
public function Destroy(){
imagedestroy($this->_image_resource);
}
/* Get image resources
* #access public, #return resource */
public function GetResource(){
return $this->_image_resource;
}
/* Set image resources
* #access public, #param resource $image, #return resource */
public function SetResource($image){
$this->_image_resource = $image;
}
/* Enable caching
* #access public, #param string $folder, #param int $ttl, * #return void */
public function EnableCache($folder="/var/www/vhosts/blabla.org/httpdocs/tmp/", $ttl=345600){
if(!is_dir($folder)){
$this->showError("Directory '" . $folder . "' does'nt exist");
}else{
$this->_cache = true;
$this->_cache_folder = $folder;
$this->_cache_ttl = $ttl;
}
return false;
}
}
The original author granted me permission for placing parts of code in here for solving this issue.
If I'm understanding the question correctly, this is entirely to be expected. Image manipulation is slow.
The yellow is your browser sending the request. The green is your browser waiting on the server to actually create the thumbnail, which takes a very significant amount of time, no matter what library the server is using. The blue is the server sending the response, which, unlike the previous steps, is affected by filesize.
There's not much to be done about the inherent slowness of image manipulation. It would be wise to cache these thumbnails so that they are only generated once and are then served statically. That way, very few of your users will ever have to sit through that green delay, and your server will be happy, too.
EDIT: If the issue is that the files exist at those URLs, but your RewriteRule is kicking in anyway, bear in mind that, by default, rules run without checking if the file exists.
Use the following condition above your RewriteRule to make sure the file exists.
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule # ...etc...
imgcpu.php?src=foo/foo.jpg&w=100&h=100
so imgcpu.php is running for every image request?
In that case, if you are worried about performance,:
the script needs to do some caching of the thumbnails it creates. If it resizes stuff on every request, that's your problem right there.
the script needs to send some caching headers to the browser - a pure PHP script won't do that, and will be refreshed on every page load
a session_start() call inside the PHP script could lead to concurrency issues because of session locking.
you will need to show some PHP code. Maybe in a separate question, though.
Apache can serve up files from your hard disk a lot faster than PHP can, and it appears that you're doing the latter to handle caching:
/**
* Get an image from cache
*
* #access public
* #param string $name
* #return void
*/
private function cachedImage($name){
$file = $this->_cache_folder . $name;
$fh = fopen($file, 'r');
$content = fread($fh, filesize($file));
fclose($fh);
return $content;
}
There's a better way of doing what that function is doing (passthru), but the best option is to setup a regex that'll only rewrite a request to your thumbnailing script if the file doesn't already exist:
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} -s [OR]
RewriteCond %{REQUEST_FILENAME} -l [OR]
RewriteCond %{REQUEST_FILENAME} -d
RewriteRule ^images/.*$ - [NC,L]
RewriteRule ^images/(.*)$ /imgcpu.php/$1 [NC,L]
And then introduce logic to parse the request to an image and format it accordingly. For example, you could say thumbs should be named after the original file and have the W x H dimensions appended like "stackoverflow_logo_100x100.jpg".
Make sense?
Per request (in the comment), the description of the "s", "l", and "d" flags is as follows (quoting the docs):
'-d' (is directory) Treats the
TestString as a pathname and tests
whether or not it exists, and is a
directory.
'-s' (is
regular file, with size) Treats the
TestString as a pathname and tests
whether or not it exists, and is a
regular file with size greater than
zero.
'-l' (is symbolic link) Treats
the TestString as a pathname and tests
whether or not it exists, and is a
symbolic link.
Your checking your HTTP_IF_MODIFIED_SINCE header and cache AFTER you generate the image so the image is getting generated and cached every time you load the page. You would get a considerable decrease in time if you move these checks closer to the start of execution, before you start processing the image.
Matchu gave you answer why. If you want to fix it, save the created thumbnails, so they are not recreated on each request. I use simple 404 page that catches request to thumbnails that haven't been created, that script figures out the required dimensions and file from url - eg /thumbs/100x100/cat.png means create 100x100 thumbnail from /images/cat.png.

Categories