I'm trying to download a full library of photos from my university housing system.
The system runs on nginx, file parsing for that particular directory is unavailable, 404 returns all the time.
Anyway, every single photo is stored in a /res/up/250x300 directory, with a number, gender indicator and a random(?) hash, example here;
114057-f-95830a765f22b71ad5691adfdec6bfzfc222bb1c7.jpg
There's catalog of users which when called with specific user id returns file in EXACT same format as provided above, and is always stored in:
<div class="uwb-imgcover-photo-wrapper"/>
So, summarizing, it looks like this.
Calling for a specific user;
/kontroler.php?_action=katalog2/osoby/pokazOsobe&os_id=114057
class uwb-imgcover-photo-wrapper gives me a link to the photo:
/res/up/250x300/114057-f-95830a765f22b71ad5691adfdec6bfzfc222bb1c7.jpg
Wget magic doesn't work.
I was thinking about a loop or something like this. This is actually very first time I'm doing anything related to web file catching.
Try this
for($i=114050;$i<=114057;$i++){
$result = file_get_contents('http://domain.com/kontroler.php?_action=katalog2/osoby/pokazOsobe&os_id='.$i);
if($result){
preg_match('/\/res\/up\/250x300\/(.*?)\.jpg/',$result,$match);
if($match){
$filename = $match[1].'.jpg';
$imageURL = 'http://domain.com/res/up/250x300/'.$filename;
$image = file_get_contents($url);
file_put_contents($filename, $image);
}
}
}
Related
I will completely clarify my question, sorry to everybody.
I have code writed in files from a website that now is not working, the html code is on pages with php extension, in a folder of a Virtual Host in my PC using Wampserever. C:\wamp\1VH\PRU1, when the site was online there was a folder where was a file called image.php. This file was called from other pages inside the site like this: (a little code of a file, C:\wamp\1VH\PRU1\example.php)
"<div><img src="https://www.example.com/img/image.php?f=images&folder=foods&type=salads&desc=green&dim=50&id=23" alt="green salad 23"></div>"
And the result was that the images was showed correctly.
Now, like i have this proyect in local, and i HAVE NOT the code of that image.php file i must to write it myself, this way the images will be showed the same way that when the site was online.
When the site was online and i open a image returned by that image.php file the URL was, following the example, https://example.com/images/foods/salads/green_50/23.png.
Now how the site is only local and i have not that image.php file writed because i'm bot sure how to write it, the images obviously are not showed.
In C:\wamp\1VH\PRU1\example.php the code of files was changed deleting "https://www.example.com/img/image.php?" for a local path "img/image.php?".
And in the same folder there is anothers: "img" folder (here must be allocated the image.php file), and "images" folder, inside it /foods/salads/green_50/23.png, 24.png.25.png..............
So i have exactly the same folder architecture that the online site and i changed the code that i could only, for example replacing with Jquery "https://www.example.com/img/image.php?" for "img/image.php?" but wich i can not do is replace all the code after the image.php file to obtain a image file.
So i think that the easiest way to can obtain the images normally is creating that IMAGE.PHP file that i have not here in my virtual host.
I'd like to know how to obtain the parameters and return the correct URL in the image,php file.
The image of the DIV EXAMPLE must be C:/wamp/1VH/PRU1/images/foods/salads/green_50/23.png
I have in my PC the correct folders and the images, i only need to write the image.php file.
Note that there are "&" and i must to unite the values of "desc=green&dim=50&" being the result: green_50 (a folder in my PC).
TVM.
You probably want something like this.
image.php
$id = intval($_GET['id']);
echo '<div><img src="images/foods/salads/green_50/'.$id.'.png" alt="green salad '.$id.'"></div>';
Then you would call this page
www.example.com/image.php?id=23
So you can see here in the url we have id=23 in the query part of the url. And we access this in PHP using $_GET['id']. Pretty simple. In this case it equals 23 if it was id=52 it would be that number instead.
Now the intval part is very important for security reasons you should never put user input directly into file paths. I won't get into the details of Directory Transversal attacks. But if you just allow anything in there that's what you would be vulnerable to. It's often overlooked, so you wouldn't be the first.
https://en.wikipedia.org/wiki/Directory_traversal_attack
Now granted the Server should have user permissions setup properly, but I say why gamble when we can be safe with 1 line of code.
This should get you started. For the rest of them I would setup a white list like this:
For
folder=foods
You would make an array with the permissible values,
$allowedFolders = [
'food',
'clothes'
'kids'
];
etc...
Then you would check it like this
///set a default
$folder = '';
if(!empty($_GET['folder'])){
if(in_array($_GET['folder'], $allowedFolders)){
$folder = $_GET['folder'].'/';
}else{
throw new Exception('Invalid value for "folder"');
}
}
etc...
Then at the end you would stitch all the "cleaned" values together. As I said before a lot of people simply neglect this and just put the stuff right in the path. But, it's not the right way to do it.
Anyway hope that helps.
You essentially just need to parse the $_GET parameters, then do a few checks that the file is found, a real image and then just serve the file by setting the appropriate content type header and then outputting the files contents.
This should do the trick:
<?php
// define expected GET parameters
$params = ['f', 'folder', 'type', 'desc', 'dim', 'id'];
// loop over parameters in order to build path: /imagenes/foods/salads/green_50/23.png
$path = null;
foreach ($params as $key => $param) {
if (isset($_GET[$param])) {
$path .= ($param == 'dim' ? '_' : '/').basename($_GET[$param]);
unset($params[$key]);
}
}
$path .= '.png';
// check all params were passed
if (!empty($params)) {
die('Invalid request');
}
// check file exists
if (!file_exists($path)) {
die('File does not exist');
}
// check file is image
if (!getimagesize($path)) {
die('Invalid image');
}
// all good serve file
header("Content-Type: image/png");
header('Content-Length: '.filesize($path));
readfile($path);
https://3v4l.org/tTALQ
use $_GET[];
<?php
$yourParam = $_GET['param_name'];
?>
I can obtain the values of parameters in the image.php file tis way:
<?php
$f = $_GET['f'];
$folder = $_GET['folder'];
$type = $_GET['type'];
$desc = $_GET['desc'];
$dim = $_GET['dim'];
$id = $_GET['id'];
?>
But what must i do for the image:
C:/wamp/1VH/PRU1/images/foods/salads/green_50/23.png
can be showed correctly in the DIV with IMG SRC atribute?
I have a directory full of images (40,000 +) that I need sorted. I have designed a script to sort them into knew proper directories, however, I am having issues with the file name.
The images urls with the id they belong to are stored in a database, and I am using the database in conjunction with the script to sort the images.
My Problem:
The image url's in the database are shortened. An example of such corresponding images are like this:
dsc_0107-367.jpg
dsc_0107-367-5478-2354-0014.jpg
The first part of the filenames are the same, but the actual file contains more info. I'd like a way to move the file from the database with the known part of the file name.
I have a basic code:
<?php
$sfiles = mysqli_query($dbconn, "SELECT * FROM files WHERE gal_id = '$_GET[id']");
while($file = mysqli_fetch_assoc($sfiles)){
$folder = $file['gal_id'];
$fileToMove = $file['filename'];
$origDir = "mypath/to/dir";
$newDir = "mypath/to/new/dir/$file['gal_id']";
mkdir "$newDir";
mv "$fileToMove" "$newDir";
}
Im just confused on how to select the file based on the small part from the database.
NOTE: It's not as simple as changing the number of chars in the db, because the db was given to me from an external site thats been deleted. So this is all the data I have.
PHP can open files using the function glob() . Glob searches your server, or specified directory, for any files containing a "match" to a pattern you specify.
Using glob() like this will pull your images from a partial name.
Run this query separate from the second:
$update = mysqli($dbconn, "UPDATE files
SET filename = REPLACE(filename, '.info', ''));
filename should be the column in your database that contains the list of images. The reason we are removing the .jpg from the db columns is if your names are partial, the .jpg may not match with the given name in your directory. With it removed, we can search solely for the pattern of the name.
Build the query to select and move the folders:
$sfiles = mysqli_query($dbconn, "SELECT * FROM files");
while($file = mysqli_fetch_assoc($sfiles)){
$fileToMove = $file['filename'];
// because glob outputs the result set into an array,
// we will use foreach to run each result from the array individually.
foreach(glob("$fileToMove*") as filename){
echo "$filename <br>";
// I'm echoing this out to see that the results are being run
// one line at a time and to confirm the photo's are
// matching the pattern.
$folder = $file['gal_id'];
// pulling the id from the db of the gallery the photo belongs to.
// This will specify which folder to move the pic to.
// Replace gal_id with the name of your column.
$newDir = $_SERVER['DOCUMENT_ROOT']."/admin/wysiwyg/kcfinder/upload/images/gallery/old/".$folder;
copy($filename,$newDir."/".$filename);
// I would recommend copy rather than move.
// This will leave the original photo in its place.
// This measure is to ensure the photo made it to the new directory so you don't lose it.
// You could go back and delete the photos after if you'd prefer.
}
}
Your MySQL query is ripe for SQL Injection, and your GET statement needs to be sanitized, if I went to your page with something similar to :
pagename.php?id=' DROP TABLE; #--
this is going to end extremely badly for you.
So;
OVerall it's much better to use Prepared Statements. THere's LOTS and LOTS of data about how to use them all over SO and the wider internet. What I show below is only a stopgap measure.
$id = (int)$_GET['id'] //This forces the id value to be numeric.
$sfiles = mysqli_query($dbconn, "SELECT * FROM files WHERE gal_id = ".$id);
Also keep note of closing your ' and " quotes as your original doesn't close the array key wrapper quotes.
I never used mysqli_fetch_assoc and always used mysqli_fetch_array so will use that as it fits the same syntax :
while($file = mysqli_fetch_array($sfiles)){
$folder = $id //same thing.
$fileToMove = $file['filename'];
$origDir = "mypath/to/dir/".$fileToMove;
//This directory shold always start with Server['DOCUMENT_ROOT'].
//Please read the manual for it.
$newDir = $_SERVER['DOCUMENT_ROOT']."/mypath/to/new/dir/".$folder;
if(!is_dir($newDir)){
mkdir $newDir;
}
// Now the magic happens, copies the file to the new directory.
// Then (optionally) delete the original.
copy($origDir,$newDir."/".$fileToMove);
unlink($origDir); //removes original.
// Add a flag to your Database to know that this file has been copied,
// ideally you should resave the filepath to the correct new one.
//MySQL update saving the new filepath.
}
Read up on PHP Copy and PHP unlink.
And; please use Prepared Statements for PHP and Database interactions.!
How do I find the filename of an image on a MediaWiki site?
I don't want to put the filename in manually. I need PHP code which will fetch me the filename.
I can use $f = wfFindFile( '$filename' ); but HOW DO I GET $filename?
I've been looking at the FILE class but I can't figure out how to use File::getFilename(); I keep getting an error call to undefined method.
What am I doing wrong?
Explaining in more detail:
I would like to add the pin it button to my site so when you click on the button it post it on the pin it board with the image and description of the image. I need to use php to send the image information so it works on every page on my site. I can't code the image name manually each time.
So far I have the code:
<img border="0" src="//assets.pinterest.com/images/PinExt.png" title="Pin It" />
Which works great except I need to put in a value for $f (image name). My question is how do I get the value of $f without having to put in in eg $f = wfFindFile( 'Sunset.jpg' );
I would have thought this would be a really common request for anyone trying to add pinterest to their site.
Thanks
The $filename you are looking for is basically how it is named in MediaWiki when it got uploaded, for example Landscape-plain.jpg. You will just use the wfFindFile() helper function to get a File object. Then call the methods:
$ php maintenance/eval.php
> $file = wfFindFile( 'Landscape-plain.jpg' );
> print $file->getName();
Landscape-plain.jpg
> print $file->getPath();
mwstore://local-backend/local-public/b/b0/Landscape-plain.jpg
> print $file->getFullPath();
/path/to/images/b/b0/Landscape-plain.jpg
> print $file->getTitle();
File:Landscape-plain.jpg
> exit
API documentation:
http://svn.wikimedia.org/doc/classFile.html
http://svn.wikimedia.org/doc/classLocalFile.html
EDIT BELOW
The file informations are available through a File object, so you definitely need to use wfFindFile() to get such an object.
To actually find the filename for the page the user is browsing on, you want to use the query context and get its title:
$context = RequestContext::getMain();
$t = $context->getTitle();
if( $title->getNamespace == 'NS_FILE' ) {
$filename = $title->getPrefixedText;
// do your stuff.
}
http://static.ak.fbcdn.net/rsrc.php/117600/css/base.css
http://static.ak.fbcdn.net/rsrc.php/z4HBM/hash/3krgnmig.swf
http://b.static.ak.fbcdn.net/rsrc.php/z23ZQ/hash/3ls2fki5.xml
http://static.ak.fbcdn.net/rsrc.php/z7O0P/hash/4hw14aet.png
What does rsrc.php really does? I know that rsrc stands for resource and rsrc.php/z[random]/hash or css/file.extenstion loads a file from somehwere.
Assuming /hash/ or /css/ is a folder which keeps the files like .xml .png .swf but whats with z[random] thing and why they want to load a file from a php? Is it for something like version control for the file or what? If so how to do it (in a simpler way)?
rsrc.php is used by Facebook for version control of all static files, especially images, javascript, and stylesheets. This allows Facebook to apply changes to the main application stack including changes to static content files without breaking functionality for users who are running off an old cached version. It is built into the Facebook architecture as part of the Haste system.
Reference To Code Function Identification By Original Developer
Recommended Process For Managing Static Resources (phabricator.com)
I think that these files are stored in a database. Anything after the SELF (script name, in this case the script is rsrc.php) is passed to the script as a param for the database. I use myself on image files, you base64 the image, store it in the database and usually with a bit of mod_rewrite magic your can get the url of the image to be youtsite.com/images/fish-with-wings when it is really doing this: yoursite.com/some-script.php/fish-with-wings which is really telling the database to look look for get the image from the database where title is = fish-with-wings, and it spits out the base64 for that file.
The advantages of having everything in the database are that for content writers its easier to reference a file and you can delete or purge, or even modify with some cool AJAX and it's also useful to stop hotlinking, which facebook hasn't done here but you could say, if the url is the the full path the redirect to a hotlink warning.
There is a my version of rsrc.php
$request = basename($_SERVER[REQUEST_URI]);
$dotIndex = strrpos($request, ".");
$extension = substr($request, $dotIndex+1);
switch ($extension):
case 'js': $content_type="application/javascript"; break;
default: $content_type="text/css"; break;
endswitch;
$file = Gdecode($request);
$script_file = dirname(__FILE__)."/".$extension."/".$file.".".$extension;
$fp = #fopen($script_file, "r");
if($fp):
fclose($fp);
header('Content-type: '.$content_type);
echo file_get_contents($script_file);
endif;
Don't think it's related to CDN purposes, woulden't make sense running it through an "static" service to serve up dynamic generated content.
I do think however this might be used to hold an open connection, and push data through for facebook updates, ( that's where the xml would make sense for me ).
All of script/css files of Facebook are stored in database and Facebook uses rsrc.php to get them.
rsrc.php code may look like this:
$request = basename($_SERVER["REQUEST_URI"])
if($request != " ") {
$sql = "SELECT * FROM scripts";
$result = mysqli_query($conn, $sql);
if (mysqli_num_rows($result) > 0) {
while($row = mysqli_fetch_assoc($result)) {
header('Content-type: '.$row["type"]);
echo $row["script"];
}
}
}
I think I know the answer for this question allready, but just as curious I am, I'll ask it anyways.
I'm running a webshop which products come with a csv file. I can import all the objectsng without any trouble, the only thing is that images and thumbnail locations are not exported with the the database dump. (it's never perfect heh) You might say, do it manually then, that's what I did in the first place, but after 200 products and RSI, I gave it up and looked for a better more efficient way to do this.
I have asked my distributer and I can use their images for my own goals without any having copyright problems.
When I look at the location of the images, the url looks like this:
../img/i.php?type=i&file=1250757780.jpg
Does anyone have a idea how this problem can be tackled?
For scraping a website, I found this code:
<?php
function save_image($pageID) {
$base = 'http://www.gistron.com';
//use cURL functions to "open" page
//load $page as source code for target page
//Find catalog/ images on this page
preg_match_all('~catalog/([a-z0-9\.\_\-]+(\.gif|\.png|\.jpe?g))~i', $page, $matches);
/*
$matches[0] => array of image paths (as in source code)
$matches[1] => array of file names
$matches[2] => array of extensions
*/
for($i=0; $i < count($matches[0]); $i++) {
$source = $base . $matches[0][$i];
$tgt = $pageID . $matches[2][$i]; //NEW file name. ID + extension
if(copy($source, $tgt)) $success = true;
else $success = false;
}
return $success; //Rough validation. Only reports last image from source
}
//Download image from each page
for($i=1; $i<=6000; $i++) {
if(!save_image($i)) echo "Error with page $i<br>";
}
?>
For some reason it throws this error: Error with page 1, Error with page 2, etc
Well, you can either make the distributer to give you the image names in the CSV file and then you can construct the URLs directly, or you will have to scrap their website via a script and fetch the images (I'd ask them for permission before doing this).
That URL doesn't really tell you where the picture is located - only that a script i.php will be called and the file name is passed in as a parameter file on the query string.
Where the i.php script goes to actually find the image cannot be deduced from just the info you present here. You'd have to inspect the script to find out that information, me thinks.