I am trying to implement methods discussed in this question to write a php function that downloads an audio file for a given string, but I can't seem to get around google's abuse protection. Results are sporadic, sometimes I get an audio file and other times it's an empty 2KB mp3 due to a response with "Our systems have detected unusual traffic from your computer network". Here is what I've got so far ( note the $file has a location in my code but for the purposes of this I've omitted it ) :
function downloadMP3( $url, $file ){
$curl = curl_init();
curl_setopt( $curl, CURLOPT_URL, $url );
curl_setopt( $curl, CURLOPT_RETURNTRANSFER, true );
curl_setopt( $curl, CURLOPT_REFERER, 'http://translate.google.com/' );
curl_setopt( $curl, CURLOPT_USERAGENT, 'stagefright/1.2 (Linux;Android 5.0)' );
$output = curl_exec( $curl );
curl_close( $curl );
if( $output === false ) {
return false;
}
$fp = fopen( $file, 'wb' );
fwrite( $fp, $output );
fclose( $fp );
return true;
}
$word = "Test";
$file = md5( $word ) . '.mp3';
if ( !file_exists( $file ) ) {
$url = 'http://translate.google.com/translate_tts?q=' . $word . '&tl=en&client=t';
downloadMP3( $url, $file );
}
Try another service, I just found one that works even better than Google Translate; Google Text-To-Speech API
Related
I am not sure if hang is the correct term, but once the file is 100% uploaded my code after the upload code is not ran.
set_time_limit( 0 );
$fp = fopen( $zip_file, 'r' );
$curl_handler = curl_init();
curl_setopt( $curl_handler, CURLOPT_URL, 'https://upload-site/' . $zip_name . '.zip' );
curl_setopt( $curl_handler, CURLOPT_HTTPHEADER, [
'requesttoken: ' . $request_token,
'authorization: Basic ' . $basic_authorization_token,
] );
curl_setopt( $curl_handler, CURLOPT_PUT, true );
curl_setopt( $curl_handler, CURLOPT_INFILESIZE, filesize( $zip_file ) );
curl_setopt( $curl_handler, CURLOPT_INFILE, $fp );
curl_setopt( $curl_handler, CURLOPT_NOPROGRESS, false );
curl_setopt( $curl_handler, CURLOPT_PROGRESSFUNCTION, [ $this, 'updateUploadProgress' ] );
curl_setopt( $curl_handler, CURLOPT_TIMEOUT, 0 );
curl_setopt( $curl_handler, CURLINFO_HEADER_OUT, true );
$curl_retval = curl_exec( $curl_handler );
curl_close( $curl_handler );
//This code here is never ran on large file uploads.
Progress function (works)
function updateUploadProgress( $curl_handler, $download_file_size, $downloaded, $upload_file_size, $uploaded ) {
$progress = round( ( $uploaded / $upload_file_size ) * 100 );
echo 'Uploading... ' . $progress . " / 100%\n";
}
When uploading small files around 5mb everything works fine. But uploading a 500mb file after hitting 100/100 on the upload progress none of the code after curl_close() gets ran.
The question is not really a [mre], but I tried to get it to run on my local system.
(My PHP coding skills are low...)
I change the line: $curl_retval = curl_exec( $curl_handler ); to:
//$curl_retval = curl_exec( $curl_handler );
if(curl_exec($curl_handler) === false)
{
echo 'Curl error: ' . curl_error($curl_handler);
}
else
{
echo 'Operation completed without any errors';
}
After updateUploadProgress was called multiple times, I did receive next message:
Curl error: Failed to connect to localhost port 443: Connection refused
This error is correct, because I am not allowed to upload to my local system (😉).
But I do ask meself which message you (#Midger) will be receiving ...
Hello I would like to get some data from a url. I already tried to make a request through the console, postman, browset - all of them worked correctly. But if I make a request using php (guzzle, symfony http client) it fails on SSL. Does anybody know how to get response from this url by curl? Thanks!
Please try the following code ( That's worked for me ):
<?php
function curl( $url, $data = array(), $headers = array(), $ssl_required = false ) {
$handle = curl_init( $url );
curl_setopt( $handle, CURLOPT_RETURNTRANSFER, true );
// Set post data if exist
if ( !empty( $data ) ) {
curl_setopt( $handle, CURLOPT_POST, true );
curl_setopt( $handle, CURLOPT_POSTFIELDS, $data );
}
// Set custom headers if exist
if ( count( $headers ) )
curl_setopt( $handle, CURLOPT_HTTPHEADER, $headers );
// If url was ssl, need to true
if ( $ssl_required )
curl_setopt($handle, CURLOPT_SSL_VERIFYPEER, false);
$output = curl_exec( $handle );
curl_close( $handle );
return $output;
}
echo curl("https://www.skroutz.gr/c/900/fakoi-epafhs.json");
I'm using CURL in PHP to download images from URLs from CSV file but if there are more than one images in a single line than some of the images downloaded corrupted and the size of that image is 0 byte.
Example:-
If the CSV file is like this then the second file always corrupted.
Image 1, "https://d2qx4k6xgs9iri.cloudfront.net/ProductImages/ce363947-f23a-46d6-b106-1201cdca37f0.jpg, https://homepages.cae.wisc.edu/~ece533/images/airplane.png"
But If I removed first or second image than the image successfully saved. Example:
Image 2, https://homepages.cae.wisc.edu/~ece533/images/arctichare.png
Here is my code that reads a CSV File
$file = fopen($file, "r");
while (!feof($file)) {
$data = fgetcsv($file);
$images = $data[1];
$images = explode(',', $images); //exploding images by ,
foreach ($images as $image) {
$milliseconds = md5(round(microtime(true) * 1000)) . '.jpg';
$imagename = saveImage($image, $milliseconds);
}
}
Below saveImage function
function saveImage($url,$image_name){
echo $url.'<br/>'; //URL is correct and have image. I have checked it manually
$ch = curl_init ($url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_BINARYTRANSFER,1);
$raw=curl_exec($ch);
curl_close ($ch);
$fp = fopen('assets/products/large/' . $image_name,'x');
fwrite($fp, $raw);
fclose($fp);
}
The Exact Sample for CSV File which I'm Using
Images 1, "https://d2qx4k6xgs9iri.cloudfront.net/ProductImages/ce363947-f23a-46d6-b106-1201cdca37f0.jpg, https://homepages.cae.wisc.edu/~ece533/images/airplane.png"
Images 2, https://homepages.cae.wisc.edu/~ece533/images/arctichare.png
Images 3, "https://homepages.cae.wisc.edu/~ece533/images/fruits.png, https://homepages.cae.wisc.edu/~ece533/images/girl.png"
Images 4, "https://homepages.cae.wisc.edu/~ece533/images/goldhill.bmp, https://homepages.cae.wisc.edu/~ece533/images/tulips.png"
I think the issue may well be that there is space in the url before the protocol - using trim to remove the space would help. Rather than use curl for testing I simply used file_get_contents and it downloaded all the files OK.
$dir = 'c:/temp/downloads/';
$file=__DIR__ . DIRECTORY_SEPARATOR . 'img.csv';
$file=fopen( $file, 'r' );
while( !feof( $file ) ){
$line = fgetcsv( $file );
if( !empty( $line[1] ) ){
$urls = explode( ',', $line[1] );
foreach( $urls as $url ){
$url=trim( $url );
$bytes = file_put_contents( $dir . basename( $url ), file_get_contents( $url ) );
printf('Saved %s - size: %sKb<br />',basename( $url ),$bytes / 1024 );
}
}
}
fclose( $file );
The curl function also needed a little tweak - as the urls are over SSL then you really should add additional parameters to the curl request. I modified the function like so:
function saveImage( $url, $image_path ){
global $cacert;
$fp = fopen( $image_path, 'w+' );
$ch = curl_init( $url );
curl_setopt($ch, CURLOPT_HEADER, 0 );
curl_setopt($ch, CURLOPT_TIMEOUT, 10 );
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true );
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true );
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true );
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, true );
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2 );
curl_setopt($ch, CURLOPT_CAINFO, $cacert );
curl_setopt($ch, CURLOPT_ENCODING, '' );
curl_setopt($ch, CURLOPT_FILE, $fp );
curl_exec($ch);
curl_close ($ch);
fclose($fp);
}
Where $cacert is defined elsewhere but essentially, on my system is c:\wwwroot\cacert.pem ~ you can download a copy from here - curl.haxx.se
I ran this code rather than the above like so:
while( !feof( $file ) ){
$line = fgetcsv( $file );
if( !empty( $line[1] ) ){
$urls = explode( ',', $line[1] );
foreach( $urls as $url ){
$url=trim( $url );
saveImage( $url, $dir . basename( $url ) );
}
}
}
fclose( $file );
Is it possible to download youtube video links through php with resumable. I can easily fetch the download links through youtube-dl.
In PHP resumable, it needs to be known the size of the video. But from the link i can't fetch out the size, so is it possible to make resumable through PHP.
Let say the link is: [mp4]
https://r10---sn-hp576n7z.googlevideo.com/videoplayback?mt=1415174349&itag=22&fexp=915516%2C930666%2C932404%2C934601%2C947209%2C947215%2C948124%2C952302%2C952901%2C953912%2C957103%2C957201&key=yt5&upn=xe1em66NCSc&sparams=gcr%2Cid%2Cip%2Cipbits%2Citag%2Cmime%2Cmm%2Cms%2Cmv%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&ipbits=0&ratebypass=yes&gcr=us&mv=u&id=o-ANmao-OacohTVjp7yijSePI5a3vvlERcSs6uL3WLfOA7&ms=au&expire=1415196179&sver=3&ip=176.576.184.114&mime=video%2Fmp4&requiressl=yes&source=youtube&mm=31&signature=29046D03C1EC013A78BD20E4CDD362BBA62ADF4B.A0E2E8E104769DD664202597A91F0888F9E554B3
Try this to get filesize:
<?php
/**
* Returns the size of a file without downloading it, or -1 if the file
* size could not be determined.
*
* #param $url - The location of the remote file to download. Cannot
* be null or empty.
*
* #return The size of the file referenced by $url, or -1 if the size
* could not be determined.
*/
function curl_get_file_size( $url ) {
// Assume failure.
$result = -1;
$curl = curl_init( $url );
// Issue a HEAD request and follow any redirects.
curl_setopt( $curl, CURLOPT_NOBODY, true );
curl_setopt( $curl, CURLOPT_HEADER, true );
curl_setopt( $curl, CURLOPT_RETURNTRANSFER, true );
curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, true );
curl_setopt( $curl, CURLOPT_USERAGENT, 'user_agent_string' );
$data = curl_exec( $curl );
curl_close( $curl );
if( $data ) {
$content_length = "unknown";
$status = "unknown";
if( preg_match( "/^HTTP\/1\.[01] (\d\d\d)/", $data, $matches ) ) {
$status = (int)$matches[1];
}
if( preg_match( "/Content-Length: (\d+)/", $data, $matches ) ) {
$content_length = (int)$matches[1];
}
// http://en.wikipedia.org/wiki/List_of_HTTP_status_codes
if( $status == 200 || ($status > 300 && $status <= 308) ) {
$result = $content_length;
}
}
return $result;
}
Usage:
$file_size = curl_get_file_size($url);
I have an AWS setup with Apache/PHP server on Port 80 and a REST Tomcat server on 8080.
If I try to access REST Services using IP Address A.B.C.D:8080/restapp from outside it works.
However if I try invoking from PHP code on the same box, it throws an internal error. Need your expert help in debugging this:
Checklist:
Security Profile:
8080 and 80 opened for 0.0.0.0/0
URL to be invoked: http://ec2-A-B-C-D.us-west-1.compute.amazonaws.com/myapp/ba-simple-proxy1.php?url=http://ec2-A-B-C-D.us-west-1.compute.amazonaws.com:8080/restapp/rest/user
ERROR RESPONSE:
"NetworkError: 500 Internal Server Error - http://ec2-A-B-C-D.us-west-1.compute.amazonaws.com/myapp/ba-simple-proxy1.php?url=http://ec2-A-B-C-D.us-west-1.compute.amazonaws.com:8080/restapp/rest/user"
Code Snippet from PHP - ba-simple-proxy1.php:
//print $url;
if ( !$url ) {
// Passed url not specified.
$contents = 'ERROR: url not specified';
$status = array( 'http_code' => 'ERROR' );
} else if ( !preg_match( $valid_url_regex, $url ) ) {
// Passed url doesn't match $valid_url_regex.
$contents = 'ERROR: invalid url';
$status = array( 'http_code' => 'ERROR' );
} else {
$ch = curl_init( $url );
if ( strtolower($_SERVER['REQUEST_METHOD']) == 'post' ) {
curl_setopt( $ch, CURLOPT_POST, true );
curl_setopt( $ch, CURLOPT_POSTFIELDS, $_POST );
}
if ( $_GET['send_cookies'] ) {
$cookie = array();
foreach ( $_COOKIE as $key => $value ) {
$cookie = array();
foreach ( $_COOKIE as $key => $value ) {
$cookie[] = $key . '=' . $value;
}
if ( $_GET['send_session'] ) {
$cookie[] = SID;
}
$cookie = implode( '; ', $cookie );
curl_setopt( $ch, CURLOPT_COOKIE, $cookie );
}
curl_setopt( $ch, CURLOPT_FOLLOWLOCATION, true );
curl_setopt( $ch, CURLOPT_HEADER, true );
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, true );
curl_setopt( $ch, CURLOPT_USERAGENT, $_GET['user_agent'] ? $_GET['user_agent'] : $_SERVER['HTTP_USER_AGENT'] );
list( $header, $contents ) = preg_split( '/([\r\n][\r\n])\\1/', curl_exec( $ch ), 2 );
//print $ch;
$status = curl_getinfo( $ch );
curl_close( $ch );
}
Turns out the php_curl lib was not part of PHP5 installation. I installed it and everything works fine now.