http://static.ak.fbcdn.net/rsrc.php/117600/css/base.css
http://static.ak.fbcdn.net/rsrc.php/z4HBM/hash/3krgnmig.swf
http://b.static.ak.fbcdn.net/rsrc.php/z23ZQ/hash/3ls2fki5.xml
http://static.ak.fbcdn.net/rsrc.php/z7O0P/hash/4hw14aet.png
What does rsrc.php really does? I know that rsrc stands for resource and rsrc.php/z[random]/hash or css/file.extenstion loads a file from somehwere.
Assuming /hash/ or /css/ is a folder which keeps the files like .xml .png .swf but whats with z[random] thing and why they want to load a file from a php? Is it for something like version control for the file or what? If so how to do it (in a simpler way)?
rsrc.php is used by Facebook for version control of all static files, especially images, javascript, and stylesheets. This allows Facebook to apply changes to the main application stack including changes to static content files without breaking functionality for users who are running off an old cached version. It is built into the Facebook architecture as part of the Haste system.
Reference To Code Function Identification By Original Developer
Recommended Process For Managing Static Resources (phabricator.com)
I think that these files are stored in a database. Anything after the SELF (script name, in this case the script is rsrc.php) is passed to the script as a param for the database. I use myself on image files, you base64 the image, store it in the database and usually with a bit of mod_rewrite magic your can get the url of the image to be youtsite.com/images/fish-with-wings when it is really doing this: yoursite.com/some-script.php/fish-with-wings which is really telling the database to look look for get the image from the database where title is = fish-with-wings, and it spits out the base64 for that file.
The advantages of having everything in the database are that for content writers its easier to reference a file and you can delete or purge, or even modify with some cool AJAX and it's also useful to stop hotlinking, which facebook hasn't done here but you could say, if the url is the the full path the redirect to a hotlink warning.
There is a my version of rsrc.php
$request = basename($_SERVER[REQUEST_URI]);
$dotIndex = strrpos($request, ".");
$extension = substr($request, $dotIndex+1);
switch ($extension):
case 'js': $content_type="application/javascript"; break;
default: $content_type="text/css"; break;
endswitch;
$file = Gdecode($request);
$script_file = dirname(__FILE__)."/".$extension."/".$file.".".$extension;
$fp = #fopen($script_file, "r");
if($fp):
fclose($fp);
header('Content-type: '.$content_type);
echo file_get_contents($script_file);
endif;
Don't think it's related to CDN purposes, woulden't make sense running it through an "static" service to serve up dynamic generated content.
I do think however this might be used to hold an open connection, and push data through for facebook updates, ( that's where the xml would make sense for me ).
All of script/css files of Facebook are stored in database and Facebook uses rsrc.php to get them.
rsrc.php code may look like this:
$request = basename($_SERVER["REQUEST_URI"])
if($request != " ") {
$sql = "SELECT * FROM scripts";
$result = mysqli_query($conn, $sql);
if (mysqli_num_rows($result) > 0) {
while($row = mysqli_fetch_assoc($result)) {
header('Content-type: '.$row["type"]);
echo $row["script"];
}
}
}
Related
With this code I create new files with information from a database:
function getLocations() {
$query = mysql_query("SELECT DISTINCT `plaats` FROM `plaatsen` ORDER BY `plaats` ");
while ($row = mysql_fetch_assoc($query)){
$city = $row['plaats'];
$handle = fopen( "cities/$city.php",'w');
echo "<a href='cities/".$row['plaats'].".php' class='col-md-4'>"."<div class='steden'>".$row['plaats']."</div>"."</a>";
fwrite($handle, '<?php include("../stad.php") ?>');
fwrite($handle, $city);
}
}
The output are multiple links and files with different cities like: Amsterdam, Rotterdam, Den Haag. Now the content on these pages has to be different depending on the city it is. So basically the page should know which city it is and dan download content for this city from a database.
I tried to do this with writing a var to the pages which does write the city name to the page, but I cannot (as far as I know) do anything with this information.
Does someone know how to do this? Thanks!
$handle = fopen( "cities/$city.php",'w'); - ERK! This is really bad for security. Allowing your webserver uid write access to files which will then be processed by PHP is not in itself a vulnerability but provides a vector for escalation of an attack.
There may be a good reason why you want to cache content outside of your database - but use a caching reverse proxy or ESI or (as a last resort) implement caching of HTML fragments in your PHP code DO NOT USE PHP CODE AS A CACHE.
As to the question of identifying the name of a file PHP code is running in - PHP can get this from the __FILE__ constant. e.g.
<?php
print "I am in " . strtok(basename(__FILE__), '.');
To determine the filenames involved in a hierarchy of includes, have a look at debug_backtrace()
I'm trying to download a full library of photos from my university housing system.
The system runs on nginx, file parsing for that particular directory is unavailable, 404 returns all the time.
Anyway, every single photo is stored in a /res/up/250x300 directory, with a number, gender indicator and a random(?) hash, example here;
114057-f-95830a765f22b71ad5691adfdec6bfzfc222bb1c7.jpg
There's catalog of users which when called with specific user id returns file in EXACT same format as provided above, and is always stored in:
<div class="uwb-imgcover-photo-wrapper"/>
So, summarizing, it looks like this.
Calling for a specific user;
/kontroler.php?_action=katalog2/osoby/pokazOsobe&os_id=114057
class uwb-imgcover-photo-wrapper gives me a link to the photo:
/res/up/250x300/114057-f-95830a765f22b71ad5691adfdec6bfzfc222bb1c7.jpg
Wget magic doesn't work.
I was thinking about a loop or something like this. This is actually very first time I'm doing anything related to web file catching.
Try this
for($i=114050;$i<=114057;$i++){
$result = file_get_contents('http://domain.com/kontroler.php?_action=katalog2/osoby/pokazOsobe&os_id='.$i);
if($result){
preg_match('/\/res\/up\/250x300\/(.*?)\.jpg/',$result,$match);
if($match){
$filename = $match[1].'.jpg';
$imageURL = 'http://domain.com/res/up/250x300/'.$filename;
$image = file_get_contents($url);
file_put_contents($filename, $image);
}
}
}
I face a case I never did, and I dont know how to properly do it.
I have a php script which generate files for clients. At the end of the script, I echo the path for them to download the file, simply.
How can I do to provide the file - or the path or any what - for downloading it, and be sure to delete the file once downloaded.
Widely, I'd like to make the file available for one/unique download only. How to ?
EDIT
I cannot use headers
There are a few components to getting this to work. Without knowing which framework you use, I'll use comments as placeholders.
There is no way to do it without using the header function, though.
Here is the source for a file that outlines the process:
<?php
$fileid = $_GET['fileid'];
$key = $_GET['key'];
// find the file in the database, and store it in $file
if ($keyMatches) {
// it is important for security to only use file paths from the database
$actualPath = $file->getPathOnDisk();
$fileInfo = finfo_open(FILEINFO_MIME_TYPE);
$mime = finfo_file($fileInfo, $actualPath);
$fp = fopen($actualPath, 'rb');
header("Content-Type: " . $mime);
header("Content-Length: " . filesize($actualPath));
fpassthru($fp);
}
else
{
http_response_code(403); // forbidden
}
You'll use this by linking to download.php?fileid=1234&key=foobar, and generating the URL at the same time you generate the key and store it in the database.
For security, you'll keep the files outside of the web root, meaning they cannot be accessed through the web server without going through a script.
fpassthru is reasonably fast, and will not likely have a performance impact.
You must do a download file gateway, like download.php?id=XXX
Where XXX is the unique ID of each file you will store in DB. And of course, the file to be downloaded.
Then, each time a user will visit the page, you can :
- Check if he has already downloaded the file
- If no, redirect it to the real path of file
- If yes, display 403 message.
When a user download a file, update the DB, generate or copy the file to a new name, you play with headers, and delete file upon download or after a small timeout.
This question is more about methodology than actual code - lines
I would like to know how to implement a pseudo caching (for lack of a better name) for FILES in php . I have tried to read some articles, but most of them refer to the internal caching system of PHP , and not to what I need which is a FILE cache.
I have several scenarios where I needed such a system applied :
Scenario 1 :
While accessing a post and clicking a link, all the post attachments are collected and added to a zip file for download.
Scenario 2 :
Accessing a post , the script will scan all the content , extract all links, download some matching images for each link (or dynamically prepare one) and then serve those to browser . (but not after checing expiration period ?? )
( Those example uses "post" and "attachment" because i use wordpress and it is wordpress terminology, both currently work for me fine, except they generate the file over and over again. )
My doubts regarding the two scenarios (especially No.2) - How do I prevent the script to do the operation EVERY time the page is accessed ? (in other words , if the file exists , just serve it without looping the whole creating operation again)
My first instinct was call the file with some distinctive (but not load - unique like uniqueid() ) name and then check if it is already on the server , but that presents several problems (like it can already exists as naming , but of another post ..) and also - that should be very resource intensive for a server with 20,000 images .
The second thing I thought was to somehow associate a meta data for those files, but then again, How to implement it ? How to knwo which link is of what image ??
Also, in a case where I check for the file existence on the server , how can I know if the file SHOULD be changed (and therefor recreated ) ?
Since I am refering to wordpress, I thought about storing those images as base64 from binary directly to the DB with the transien_API - but it feels quite clumsy.
To sum up the question . How to generate a file, but also know if it exists and call it directly when needed ?? does my only option is store the file-name in DB and associate it somehow with the post ?? that seems so non efficient ..
EDIT I
I decided to include some example code , as it can help people to understand my dilemma .
function o99_wbss_prepare_with_callback($content,$width='250'){
$content = preg_replace_callback( '/(http[s]?:[^\s]*)/i', 'o99_wbss_prepare_cb', $content );
return $content;
}
function o99_wbss_prepare_cb($match){
$url = $match[1];
$url = esc_url_raw( $url );//someone said not need ??
$url_name = parse_url($url);
$url_name = $url_name['host'];// get rid of http://..
$param = '660';
$url = 'http://somescript/' . urlencode($url) . '?w=' . $param ;
$uploads = wp_upload_dir();
//$uniqid = uniqid();
$img = $uploads['basedir'] . '/tmp/' . $url_name .'.jpg' ; // was with $uniqid...
if(! # file_get_contents($url)){
$url = 'path ' .$url. ' doesn"t exist or unreachable';
return $url;
} else {
$file = file_get_contents( $url );
}
// here I will need to make some chck if the file already was generated , and
// if so - just serve it ..
if ( $file) {
file_put_contents( $img, $file );
// Do some other operations on the file and prepare a new one ...
// this produces a NEW file in the wp-uploads folder with the same name...
unlink($img);
}
return $url;
}
For Scenario 1:
Wordpress stored all post attachments as posts in the posts table. When a post is accessed run a function either in a created plugin or your themes functions.php. Use the pre_get_posts hook check if you have already created the zip file with function file_exists() using a unique name for each zip archive you create, post ID or permalink would be a good idea. Although you would need to make sure there was no user specific content. You can use filemtime() to check the time the file was created and if it is still relevant. If zip file does not exist create it, pre_get_posts will pass the query object which has the the post ID, just grab all the post attachments using get_posts and the parent ID being set to the ID passed in the query object. The GUID field contains the URL for each attachment then just generate a zip archive using ZipArchive() following this tutorial at.
For Scenario 2:
If your wordpress templates are set up to use the wordpress functions then replace the attachment functions to return their url and map that to the new url you have the cached content. For example the_post_thumbnail() would go to wp_get_attachment_thumb_url() copy the file to your cache and use the cache url as output. If you wanted to cache the DOM for the page as well use ob_start(). Now just run a check at the start of the template using file_exists and filetime(), if both are valid read in the cached DOM instead of loading the page.
I've recently started getting into the area of optimizing preformance and load times client side, compressing css/js, gzipping, paying attention to YSlow, etc.
I'm wondering, while trying to achieve all these micro-optimizations, what are the pros and cons of serving php files as css or javascript?
I'm not entirely sure where the bottleneck is, if there is one. I would assume that between an identical css and php file, the "pure" css file would be slightly faster simply because it doesn't need to parse php code. However, in a php file you can have more control over headers which may be more important(?).
Currently I'm doing a filemtime() check on a "trigger" file, and with some php voodoo writing a single compressed css file from it, combined with several other files in a defined group. This creates a file like css/groupname/301469778.css, which the php template catches and updates the html tags with the new file name. It seemed like the safest method, but I don't really like the server cache getting filled up with junk css files after several edits. I also don't bother doing this for small "helper" css files that are only loaded for certain pages.
If 99% of my output is generated by php anyways, what's the harm (if any) by using php to directly output css/js content? (assuming there are no php errors)
If using php, is it a good idea to mod_rewrite the files to use the css/js extension for any edge cases of browser misinterpretation? Can't hurt? Not needed?
Are there any separate guidelines/methods for css and javascript? I would assume that they would be equal.
Which is faster: A single css file with several #imports, or a php file with several readfile() calls?
What other ways does using php affect speed?
Once the file is cached in the browser, does it make a difference anymore?
I would prefer to use php with .htaccess because it is much simpler, but in the end I will use whatever method is best.
ok, so here are your direct answers:
no harm at all as long as your code is fine. The browser won't notice any difference.
no need for mod_rewrite. the browsers usually don't care about the URL (and often not even about the MIME type).
CSS files are usually smaller and often one file is enough, so no need to combine. Be aware that combining files from different directories affect images referenced in the CSS as they remain relative to the CSS URL
definitely readfile() will be faster as #import requires multiple HTTP requests and you want to reduce as much as possible
when comparing a single HTTP request, PHP may be slightly slower. But you loose the possibility to combine files unless you do that offline.
no, but browser caches are unreliable and improper web server config may cause the browser to unnecessarily re-fetch the URL.
It's impossible to give you a much more concrete answer because it depends a lot on your project details.
We are developing really large DHTML/AJAX web application with about 2+ MB of JavaScript code and they still load quickly with some optimizations:
try to reduce the number of Script URLs included. We use a simple PHP script that loads a bunch of .js files and sends them in one go to the browser (all concatenated). This will load your page a lot faster when you have a lot of .js files as we do since the overhead of setting up a HTTP connection is usually much higher that the actually transferring the content itself. Note that the browser needs to download JS files synchroneously.
be cache friendly. Our HTML page is also generated via PHP and the URL to the scripts contains a hash that's dependent on the file modification times. The PHP script above that combines the .js files then checks the HTTP cache headers and sets a long expiration time so that the browser does not even have to load any external scripts the second time the user visits the page.
GZIP compress the scripts. This will reduce your code by about 90%. We don't even have to minify the code (which makes debugging easier).
So, yes, using PHP to send the CSS/JS files can improve the loading time of your page a lot - especially for large pages.
EDIT: You may use this code to combine your files:
function combine_files($list, $mime) {
if (!is_array($list))
throw new Exception("Invalid list parameter");
ob_start();
$lastmod = filemtime(__FILE__);
foreach ($list as $fname) {
$fm = #filemtime($fname);
if ($fm === false) {
$msg = $_SERVER["SCRIPT_NAME"].": Failed to load file '$fname'";
if ($mime == "application/x-javascript") {
echo 'alert("'.addcslashes($msg, "\0..\37\"\\").'");';
exit(1);
} else {
die("*** ERROR: $msg");
}
}
if ($fm > $lastmod)
$lastmod = $fm;
}
//--
$if_modified_since = preg_replace('/;.*$/', '',
$_SERVER["HTTP_IF_MODIFIED_SINCE"]);
$gmdate_mod = gmdate('D, d M Y H:i:s', $lastmod) . ' GMT';
$etag = '"'.md5($gmdate_mod).'"';
if (headers_sent())
die("ABORTING - headers already sent");
if (($if_modified_since == $gmdate_mod) or
($etag == $_SERVER["HTTP_IF_NONE_MATCH"])) {
if (php_sapi_name()=='CGI') {
Header("Status: 304 Not Modified");
} else {
Header("HTTP/1.0 304 Not Modified");
}
exit();
}
header("Last-Modified: $gmdate_mod");
header("ETag: $etag");
fc_enable_gzip();
// Cache-Control
$maxage = 30*24*60*60; // 30 Tage (Versions-Unterstützung im HTML Code!)
$expire = gmdate('D, d M Y H:i:s', time() + $maxage) . ' GMT';
header("Expires: $expire");
header("Cache-Control: max-age=$maxage, must-revalidate");
header("Content-Type: $mime");
echo "/* ".date("r")." */\n";
foreach ($list as $fname) {
echo "\n\n/***** $fname *****/\n\n";
readfile($fname);
}
}
function files_hash($list, $basedir="") {
$temp = array();
$incomplete = false;
if (!is_array($list))
$list = array($list);
if ($basedir!="")
$basedir="$basedir/";
foreach ($list as $fname) {
$t = #filemtime($basedir.$fname);
if ($t===false)
$incomplete = true;
else
$temp[] = $t;
}
if (!count($temp))
return "ERROR";
return md5(implode(",",$temp)) . ($incomplete ? "-INCOMPLETE" : "");
}
function fc_compress_output_gzip($output) {
$compressed = gzencode($output);
$olen = strlen($output);
$clen = strlen($compressed);
if ($olen)
header("X-Compression-Info: original $olen bytes, gzipped $clen bytes ".
'('.round(100/$olen*$clen).'%)');
return $compressed;
}
function fc_compress_output_deflate($output) {
$compressed = gzdeflate($output, 9);
$olen = strlen($output);
$clen = strlen($compressed);
if ($olen)
header("X-Compression-Info: original $olen bytes, deflated $clen bytes ".
'('.round(100/$olen*$clen).'%)');
return $compressed;
}
function fc_enable_gzip() {
if(isset($_SERVER['HTTP_ACCEPT_ENCODING']))
$AE = $_SERVER['HTTP_ACCEPT_ENCODING'];
else
$AE = $_SERVER['HTTP_TE'];
$support_gzip = !(strpos($AE, 'gzip')===FALSE);
$support_deflate = !(strpos($AE, 'deflate')===FALSE);
if($support_gzip && $support_deflate) {
$support_deflate = $PREFER_DEFLATE;
}
if ($support_deflate) {
header("Content-Encoding: deflate");
ob_start("fc_compress_output_deflate");
} else{
if($support_gzip){
header("Content-Encoding: gzip");
ob_start("fc_compress_output_gzip");
} else{
ob_start();
}
}
}
Use files_hash() to generate a unique hash string that changes whenever your source files change and combine_files() to send the combined files to the browser. So, use files_hash() when generating the HTML code for the tag and combine_files() in the PHP script that is loaded via that tag. Just place the hash in the query string of the URL.
<script language="JavaScript" src="get_the_code.php?hash=<?=files_hash($list_of_js_files)?>"></script>
Make sure you specify the same $list in both cases.
You're talking about serving static files via PHP, there's really little point doing that since its always going to be slower than Apache serving a normal file. A CSS #import will be quicker that PHP's readfile() but the best performance will be gained by serving one minified CSS file that combines all the CSS you need to use.
If sounds like you're on the right track though. I'd advise pre-processing your CSS and saving to disk. If you need to set special headers for things like caching just do this in your VirtualHost directive or .htaccess file.
To avoid lots of cached files you could use a simple file-naming convention for your minified CSS. For example, if your main CSS file called main.css and it references reset.css and forms.css via #imports, the minified version could be called main.min.css
When this file is regenerated it simply replaces it. If you include a reference to that file in your HTML, you could send the request to PHP if the file doesn't exist, combine and minify the file (via something like YUI Compressor), and save it to disk and therefore be served via normal HTTP for all future requests.
When you update your CSS just delete the main.min.css version and it will automatically regenerate.
You can do the preprocessing with an ANT Build. Sorry, the post is german, but I've tried translate.google.com and it worked fine :-) So you can use the post as tutorial to achieve a better performance...
I would preprocess the files and save them to disk, just like simonrjones said. Caching-stuff etc. should be done by the dedicated elements, like Apache WebServer, Headers and Browser.
While slower, one advantage / reason you might have to do this is to put dynamic content into the files on the server, but still have them appear to be js or css from the client perspective.
Like this for example, passing the environment from php to javascript:
var environment = <?=getenv('APPLICATION_ENV');?>
// More JS code here ...