Writing in a file make my program load really slowly - php

I am trying to write csv files with fwrite/fputcsv in a loop but when I use those functions, they write in the first file and then the tab loads infinitely and doesn't do anything else.
I know those functions are the problem because when I remove them the rest of the code runs properly.
Here's my code :
<?php
$auth = '';
$date = getdate(); //Needed for the Trade lists files
$path = array(
'Some_folder/file1.csv',
'Some_folder/file2.csv',
'Some_folder/file3.csv',
'Some_folder/file4.csv'
);//Full path the files need to be in
$file = array(
pathinfo($path[0])['basename'],
pathinfo($path[1])['basename'],
pathinfo($path[2])['basename'],
pathinfo($path[3])['basename']
);//The names the files will have
$fp = array(
fopen($file[0], 'w+'),
fopen($file[1], 'w+'),
fopen($file[2], 'w+'),
fopen($file[3], 'w+')
);// Files that are create by fopen
for($i = 0; $i < 4; $i++) {
fputcsv($fp[$i], $file);
echo 'test';
$size = filesize($path[$i]);
$cheaders = array('Authorization: Bearer ' .$auth.'[Private access token]',
'Content-Type: application/octet-stream',
'Dropbox-API-Arg: {"path":"/'.$file[$i].'", "mode":"add"}');
$ch = curl_init('[Private information]');
curl_setopt($ch, CURLOPT_HTTPHEADER, $cheaders);
curl_setopt($ch, CURLOPT_PUT, true);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'POST');
curl_setopt($ch, CURLOPT_INFILE, $fp[$i]);
curl_setopt($ch, CURLOPT_INFILESIZE, $size);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$response = curl_exec($ch);
curl_close($ch);
fclose($fp[$i]);
}
?>
fputcsv and fwrite do write in the first file (file1.csv) but they make the program stuck in an infinite loading. The console doesn't say anything since I can't load the page and I don't know what else to try/do. I tried looking for a solution, unfortunately I didn't find it so here I am. I'll still look for an awnser but if someone has a solution, I'll take it too.
Thanks to all those who will try to help.
EDIT: It finally loaded but it took a really long time since it started loading approximately an hour ago. Is there something I did wrong for my program to be so slow to write so slowly in a file ?

I would suggest you to split the task in two sections. First write the files in your server using fopen in w mode. After writings, use another loop for reading the files using fopen in r mode for using in your cURL to upload in the server. Using the same handler for writing and uploading might cause the lagging problem.
You can use curl_multi_init for handling multiple cURL requests asynchronously. Here is also a post on it.

Related

Curl/PHP challenge transferring image and base64_encode/decode

I am transfering data from one linux-box to another. In generel this works fine, BUT I am having trouble when it comes to transferring images. I have tested all kinds of stuff. I hope some one may be able to help me out.
$filename = "/home/user/image.jpg";
$handle = fopen($filename, "r");
$data = fread($handle, filesize($filename));
$data = base64_encode($data);
# Transfer reading
#$arrIn['changeCharset'] = "true";
$arrIn['postFields'] = "action=test&data=$base64";
$test = curlServerPost($arrIn);
Here is my CURL-function:
function curlServerPost($arrIn)
{
$postFields = $arrIn['postFields'];
$url = "$GLOBALS[remoteSite]"; // Where you want to post data
$ch = curl_init(); // Initiate cURL
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, true); // Tell cURL you want to post something
curl_setopt($ch, CURLOPT_POSTFIELDS, $postFields); // Define what you want to post
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); // Return the output in string format
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); // Accepter alle certifikater, se: http://unitstep.net/blog/2009/05/05/using-curl-in-php-to-access-https-ssltls-protected-sites/
$res = curl_exec($ch); // Execute
curl_close($ch); // Close cURL handle
return($res);
}
Finally, here is the receiver (server) part
$data = $_POST[data];
$data = base64_decode($data);
$stmt = $GLOBALS[pdo]->prepare("INSERT INTO cameraImages (cameraImagesId) VALUES ('')");
$stmt->execute();
$cameraImagesId = $GLOBALS[pdo]->lastInsertId();
$stmt = $GLOBALS[pdo]->prepare("UPDATE cameraImages SET cameraImagesFile='$data' WHERE cameraImagesId='$cameraImagesId'");
$stmt->execute();
Some last remarks:
- If I don't base_decode on server-side I receive (but completely wrong format. Errors in image)
If I base_decode om server-side. Nothing is received.
I would like to don't encode/decode at all. If I do that. Only a small part of the image is stored (corrupted image)
My datafield is LONGBLOB (mysql)
PHP on server side is: PHP 5.3.3
PHP on client side is: PHP 7.0.27-0+deb9u1
I have tried all kinds of stuff. Followed all kinds of tutorials. It just won't work for me :-/
So if anyone can come up with ideas I am more than willing to test and try :)
Loooking forward to hear from you.
Succes. I just needed
urlencode($data);
And removed base64_encode($data); / base64_decode($data);
I have dropped putting images in database since many ppl don't recommend it.

Upload images to Imgur with Curl PHP

<form action="php.php" method="post" enctype="multipart/form-data">
Send these files:<br />
<input name="img[]" type="file" multiple="multiple" /><br />
<input type="submit" value="Send files" />
</form>
<?php
ini_set('display_errors', 1);
ini_set('display_startup_errors', 1);
error_reporting(E_ALL);
foreach ($_FILES['img']['tmp_name'] as $index => $tmpName) {
if( !empty( $tmpName ) && is_uploaded_file( $tmpName ) )
{
$handle = fopen($tmpName, "r");
$data = fread($handle, filesize($tmpName));
$client_id = "d5f419ef9aedf16";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://api.imgur.com/3/image.json');
curl_setopt($ch, CURLOPT_POST, TRUE);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Authorization: Client-ID ' . $client_id));
curl_setopt($ch, CURLOPT_POSTFIELDS, array('image' => base64_encode($data)));
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_STDERR, fopen('php://stderr', 'w'));
$reply = curl_exec($ch);
$info = curl_getinfo($ch);
var_dump($info);
if(curl_errno($ch))
echo 'Curl error: '.curl_error($ch);
curl_close($ch);
$reply = json_decode($reply);
printf('<img height="180" src="%s" >', $reply->data->link);
}
}
?>
I made this page, works perfectly on localhost but when I run it from my server it does not work, problem is I have no output at all, not even the curl_getinfo or errors. Dont know how to debug this since I can't get any info out of it.
Thank you hans for the suggestions, you were right, it was not mandatory to use base64, not in V3 API, it was on the previous versions, not anymore. Now it's much faster. This is my final code
if(!empty($_FILES['img']['tmp_name'])){
foreach ($_FILES['img']['tmp_name'] as $index => $tmpName) {
if( !empty( $tmpName ) && is_uploaded_file( $tmpName ) )
{
if ($handle = fopen($tmpName, "rb")) {
$data = stream_get_contents($handle, filesize($tmpName));
$client_id = "d5f619ef9aedf16";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://api.imgur.com/3/image.json');
curl_setopt($ch, CURLOPT_POST, TRUE);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Authorization: Client-ID ' . $client_id));
curl_setopt($ch, CURLOPT_POSTFIELDS, array('image' => $data));
curl_setopt($ch, CURLOPT_VERBOSE, true);
$reply = curl_exec($ch);
if(curl_errno($ch))
echo 'Curl error: '.curl_error($ch);
curl_close($ch);
$reply = json_decode($reply);
$screens .= $reply->data->link . " ";
fclose($handle);
}
}
}
}
The reason why it was not uploading them images was upload_max_filesize and max_post_size, I've increased them and I had no problems.
you do several things wrong.
this code:
foreach ($_FILES['img']['tmp_name'] as $index => $tmpName) { will be executed even on GET requests, which contains no files, thus $_FILES['img'] does not exist, and you get an Undefined index: img error. furthermore, on GET requests, this undefined index bug returns NULL to foreach, and when you tell foreach() to loop a NULL, it gives you this error: Invalid argument supplied for foreach(). the fact that you haven't already caught up on this, proves that you're not checking the php error logs. this is the first thing you should do, when debugging a malfunctioning php script. - to fix both of these, wrap the foreach in if(!empty($_FILES['img']['tmp_name']){} - next, find your PHP error logs, and read those error reports. (usually you can find it in phpinfo() under error_log - or in case its emptystring, its your web servers' error log, eg for nginx with an empty php error_log php setting, its in nginx's error log (like /var/log/nginx/error.log )
next issue, $handle = fopen($tmpName, "r"); this will corrupt binary data (like images) on some systems (famously, windows), change it to rb for safely reading binary data (and image formats are binary)
next issue, you believe that fread() reads as many bytes as you request it to read, that is incorrect. fread will read UP TO as many bytes as requested, but can stop for many reasons before it has read the requested number of bytes. you'll either have to loop calling fread until it has read all the bytes, or better yet, use stream_get_contents, which will essentially do that fread() loop for you.
next issue, this line curl_setopt($ch, CURLOPT_STDERR, fopen('php://stderr', 'w')); is completely unnecessary, PHP's STDERR is libcurl's default STDERR anyway.
next issue, you don't fclose() $handle. this is ok for short running scripts (becuase the OS fclose() it for you when the script finishes), but the bigger your code grows, the longer the resource leak will be there, its a good programming habit to always close it, so do that.
next issue, i can't believe imgur actually wants a base64 encoded copy of the image, that is such a cpu & bandwidth & ram waste... the POST encoding they're using, multipart/form-data is fully binary safe, so there's no reason to base64 encode it... are you sure they want it base64 encoded? you AND them can save about 33% bandwidth by NOT using base64 encoding here.
given all the things you are doing wrong with the file operations, it'd be better to use the file_get_contents function - it reads the entire file, it opens the file in binary read mode (as opposed to your code, where you open it in TEXT mode, potentially corrupting the image as you read it), and it reads the entire file, not just the chunk of it that the first fread() call will read (which, if you're lucky, is the entire file, but if you're not lucky, is just the first chunk of it), and it does fclose() for you (which you forget to do)
Why dont you use imgur php sdk and simple implement it with below code:
require_once APPPATH."/third_party/imgur/vendor/autoload.php";
$pathToFile = '../path/to/file.jpg';
$imageData = [
'image' => $pathToFile,
'type' => 'file',
];
$client->api('image')->upload($imageData);

check if file exist on remote website without knowing extension

I need to find a file name _template on more than one remote server.
The file will have one of two extensions:
_template.htm
or
_template.php
I can find a file using cURL on a remote server which is easy but cant wrap my head around how to find one or the other.
The code is in a loop and it will search on about 20 different sites for this file, if it finds it just needs to say: Found it if it doesn't find any of the two files it needs to say: None Found.
Doing cURL twice takes too long to load.
Searching through the ftp array also takes too long.
My current code below that doesnt work, in know why it doesnt work thanks guys i would like to know how i can make it work: (This is in a loop)
$template_file = glob("/_template.{htm,php}", GLOB_BRACE);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $client_link.$template_file);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_NOBODY, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_MAXREDIRS, 10);
$data = curl_exec($ch);
curl_close($ch);
preg_match_all("/HTTP\/1\.[1|0]\s(\d{3})/",$data,$matches);
$code = end($matches[1]);
if(!$data)
{
echo "No Files Found<br>";
}
else
{
if($code == 200)
{
$filefound = 'Found'.$template_file;
}
elseif($code == 404)
{
$filefound = '404 File not Found';
}
}
The problem is with the glob function - that function matches paths on the local filesystem. Add a print_r($template_file);' after theglob` and you'll see that it doesn't match anything (or it only matches things on your local system).
What you need to do is build the URLs one-by-one:
foreach(array("htm", "php") as $ext)
{
// build the URL string
$url = "http://example.com/_template." . $ext;
// now do whatever you need to with the URL...
}

Send cURL request from URL?

Greetings,
I'm looking for a way to send a curl request given a full url. All of the examples and documentation I can find look something like this:
$fullFilePath = 'C:\temp\test.jpg';
$upload_url = 'http://www.example.com/uploadtarget.php';
$params = array(
'photo'=>"#$fullFilePath",
'title'=>$title
);
$ch = curl_init();
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_URL, $upload_url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $params);
$response = curl_exec($ch);
curl_close($ch);
The problem is that the file, "test.jpg" is actually generated dynamically by a script on the server (so it doesn't exist on the file system).
How can I send the request, instead using $file = "http://www.mysite.com/generate/new_image.jpg"
One solution that came to mind was loading "new_image.jpg" into memory with fopen or file_get_contents() but once I get to that point I'm not sure how to send it as POST to another site
By far the easiest solution is going to be to write the file to a temporary location, then delete it once the cURL request is complete:
// assume $img contains the image file
$filepath = 'C:\temp\tmp_image_' . rand() . '.jpg'
file_put_contents($filepath, $img);
$params = array(
'photo'=>"#$filepath",
'title'=>$title
);
// do cURL request using $params...
unlink($filepath);
Note that I am inserting a random number to avoid race conditions. If your image is not particularly big, it would be better to use md5($img) in your filename instead of rand(), which could still result in collisions.

PUT string of data to XML server using PHP

I need to put a string of data like so: '< client>...<\client>' onto an XMl server (example url:'http://example.appspot.com/examples') using PHP.
(Context: Adding a new client's details to the server).
I have tried using CURLOPT_PUT, with a file and with just a string (since it requires CURLOPT_INFILESIZE and CURLOPT_INFILE) but it does not work!
Are there any other PHP functions that could be used to do such a thing? I have been looking around but PUT requests information is sparse.
Thanks.
// Start curl
$ch = curl_init();
// URL for curl
$url = "http://example.appspot.com/examples";
// Put string into a temporary file
$putString = '<client>the RAW data string I want to send</client>';
/** use a max of 256KB of RAM before going to disk */
$putData = fopen('php://temp/maxmemory:256000', 'w');
if (!$putData) {
die('could not open temp memory data');
}
fwrite($putData, $putString);
fseek($putData, 0);
// Headers
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
// Binary transfer i.e. --data-BINARY
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_URL, $url);
// Using a PUT method i.e. -XPUT
curl_setopt($ch, CURLOPT_PUT, true);
// Instead of POST fields use these settings
curl_setopt($ch, CURLOPT_INFILE, $putData);
curl_setopt($ch, CURLOPT_INFILESIZE, strlen($putString));
$output = curl_exec($ch);
echo $output;
// Close the file
fclose($putData);
// Stop curl
curl_close($ch);
since I haven't worked with cURL so far I can't really answer to that topic. If you'd like to use cURL I'd suggest looking at the server log and see what actually didn't work (so: Was the output of the request really what it's supposed to be?)
If you don't mind switching over to another technology/library I'd suggest you to use the Zend HTTP Client which is really straight forward to use, simple to include and should satisfy all your needs. Especially as performing a PUT Request is as simple as that:
<?php
// of course, perform require('Zend/...') and
// $client = new Zend_HTTP_Client() stuff before
// ...
[...]
$xml = '<yourxmlstuffhere>.....</...>';
$client->setRawData($xml)->setEncType('text/xml')->request('PUT');
?>
Code sample is from: Zend Framework Docs # RAW-Data Requests
Another way to add string body to the PUT request with CURL in PHP is:
<?php
$data = 'My string';
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'PUT'); // Define method type
curl_setopt($ch, CURLOPT_POSTFIELDS, $data); // Set data to the body request
?>
I hope this helps!

Categories