I am using Scrapy to gather images. I would like to simulate a post onto a PHP script with multiple files. Similar to when someone uploads 10 files and they get processed by a PHP script using $_FILES['name']. I would also like to pass $_POST data as well.
Here is my Python.
post_array={
'parse':'listing'
}
files_array=response.xpath(root+'/photos//url/text()').extract()
returned=requests.post(php-script.php,data=post_array,files=files_array).text
pprint(returned)
So this is suppose to create a $_POST variable and a $_FILES variable with multiple files. How can I convert the list of URLs in files_array to become a $_FILES array in the php-script.php?
Python data input:
post_array={
'parse':'listing'
}
files_array=['https://example.co/123.jpg','https://example.co/124.jpg','https://example.co/125.jpg']]
into PHP data output inside php-script.php (desired result):
$_POST=['parse'=>'listing'];
$_FILES=['images'=>[
[0] => Array
(
[name] => 123.jpg
[type] => image/jpeg
[tmp_name] => /tmp/php/php6hst32
[error] =>
[size] => 98174
)
[1] => Array
(
[name] => 124.jpg
[type] => image/jpeg
[tmp_name] => /tmp/php/php6hst32
[error] =>
[size] => 98174
)
[2] => Array
(
[name] => 125.jpg
[type] => image/jpeg
[tmp_name] => /tmp/php/php6hst32
[error] =>
[size] => 98174
)
]];
I have also tried this:
returned=requests.post(triggers,data=post_array,files={'images':[url for url in files_array requests.get(url).content]}).text
pprint(returned)
The only way to convert list of URLs to a $_FILES array in PHP script is to actually upload these files (via POST request with enctype="multipart/form-data").
Here it's how you can do it with requests:
files_array = [('images', ('123.jpg', open('123.jpg', 'rb'), 'image/jpeg')),
('images', ('124.jpg', open('124.jpg', 'rb'), 'image/jpeg')),
('images', ('125.jpg', open('125.jpg', 'rb'), 'image/jpeg'))]
r = requests.post(url, data=post_array, files=files_array)
You can find detailed example in Advanced Usage documentation for Requests
Scrapy does not yet have file upload support, so you have to build such requests manually, which might not be trivial for you.
Adding file upload support to Scrapy has been requested, and there is an unfinished implementation that you could try out, or even try to finish.
Whatever approach you decide to follow, mind that you won’t be able to build such requests based on file URLs. To upload a file, you must have it in your computer; if you do not have it, you must download it.
Related
I'm working on someone else's page that has a simple form with a file upload. When I do a:
print_r($_FILES);
after the form submission I get the following:
Array
(
[input_41] => Array
(
[name] => Resources.pdf
[type] => application/pdf
[tmp_name] => /tmp/phpvVUox1
[error] => 0
[size] => 14563
)
)
I now need to get the name of the uploaded file - I'm familiar with the POST syntax of retrieving form field input values such as:
$_POST['input_32']
but not sure how to get the file name within an array.
As you can see in your print_r() output, you already have name property of the file in it, You can use this to get the name of the file.
$_FILES["input_41"]["name"]
Reference: http://php.net/manual/en/reserved.variables.files.php
I want to get the full path of image in PHP. I used <input> tag for image uploading. I'm unable to get the full path. when I alert the value of <input type="file">, I'm getting
c:/fake-path/image.jpg
Here is my code:
<input type="file" name="upload_captcha_background" id="upload_captcha_background" />
var file_path= jQuery("#upload_captcha_background").val();
alert(file_path);
and in PHP I'm fetching the value like:
$ux_image_path =$_FILES['upload_captcha_background'];
If you mean the path where the image was located on the user computer before he/she uploaded it to your form - you can never know it in php or javascript or anywhere else.
In PHP you can see the path on SERVER (usually in the temporary folder) where the file was stored so you can read or copy it. If may just happen that the server is as well your user computer (i.e. you are testing the website on http :// localhost) but don't confuse those two things.
You can never be sure of getting a full filepath or even a reliable filename or content-type submitted in a file upload file.
you should never do so....and i think trying it in latest browsers is useless(from what i know) .. all latest browsers on the other hand , will not allow this.
Some related Stackoverflow questions:
How to get the full path of the file from a file input
Full path from file input using jQuery
Retrieving the full path (server-side) of a file uploaded using Firefox
the structure for
$_FILES[]
as follows
/*
Array
(
[image] => Array
(
[name] => Array
(
[0] => 400.png
)
[type] => Array
(
[0] => image/png
)
[tmp_name] => Array
(
[0] => /tmp/php5Wx0aJ
)
[error] => Array
(
[0] => 0
)
[size] => Array
(
[0] => 15726
)
)
)
*/
if you are trying to upload a file and copy it to a directory
use the following code
$tmp_name = $_FILES['upload_captcha_background']["tmp_name"];
$name = $_FILES['upload_captcha_background']['name'];
move_uploaded_file($tmp_name, "$uploads_dir/$name");
I'm trying to upload a zip file and a csv file from HTML form.
On PHP, When I printed $_FILES (Actually $request->getFiles() in symfony), I got following.
Array
(
[zipfile] => Array
(
[name] => tempfiles.zip
[type] => application/octet-stream
[tmp_name] => C:\wamp\tmp\php5D42.tmp
[error] => 0
[size] => 850953
)
[csvfile] => Array
(
[name] => test.csv
[type] => application/vnd.ms-excel
[tmp_name] => C:\wamp\tmp\php5D52.tmp
[error] => 0
[size] => 312
)
)
I'm wondering with the type and tmp_name. I need to take few decisions based on type. Is it safe to take decisions on existing type? Will I get same result for similar files on Linux server?
Again tmp_name have .tmp extension. Is it consistent on both windows/linux? If not, is there any way that the code I write on windows (decision using type) will work on linux without any issue?
Using this type can be dangerous Because user can change the type of the files and can upload a php script.
You should validate the type first just like get_image_size() to validate a image file.I have no idea about .zip file
It is not safe to trust the type form $_FILES, you need to validate the file type in server side.
For .tmp extension, it is ok both on windows or linux.
while uploading MP4 video $_FILES array comes like this..
Array
(
[qqfile] => Array
(
[name] => video.mp4
[type] => video/mpeg4
[tmp_name] => /tmp/php74N9mR
[error] => 0
[size] => 199160
)
)
But while uploading .FLV file $_FILES array coming like this , why it is not coming proper?
Array
(
[qqfile] => Array
(
[name] => YouTube - My Youtube Contest Announcement.flv
[type] =>
[tmp_name] =>
[error] => 1
[size] => 0
)
)
please suggest.
The file you are trying to upload is too large. From the PHP manual's chapter on file uploads:
Since PHP 4.2.0, PHP returns an appropriate error code along with the file array. The error code can be found in the error segment of the file array that is created during the file upload by PHP. In other words, the error might be found in $_FILES['userfile']['error'].
...
UPLOAD_ERR_INI_SIZE
Value: 1; The uploaded file exceeds the upload_max_filesize directive in php.ini.
i think its related to the file name you upload ,
when you call to the php function wrap your file name with ''
can anyone shed some light on this:
Array
(
[video] => Array
(
[name] => 20051210-w50s.flv
[type] =>
[tmp_name] => /tmp/php38JFea
[error] => 0
[size] => 669036
)
)
I'm uploading an flv file, but the [type] is not being filled, is this common?
cheers in advance!
The type field is a MIME type set by the client. In this case, it just didn't know what a flv file is, and didn't set it. It is a good thing not to rely on this information, as it can be freely altered by an attacker.
If you want to get reliable info about an uploaded video file, you need to do a server-side check. The getid3 for example seems to be able to recognize flv files.
The built-in getimagesize() function can do the same for many image formats.