I get Error When Curl Data From URL ( Url From Array get in file ) .
When I use Normaly URL , It Work Correctly !
But When I get URL from Array and run a loop for it , and call function curl to run every line url , it not work ?
I don't know why , but i need someone help me .
I had turn on error report php , but nothing error !
Example File "link.txt"
https://www.fshare.vn/file/4UNG2NRPW5OQ
https://www.fshare.vn/file/CFTJYXKPMN7Q
https://www.fshare.vn/file/RMHD2XBFY93F
https://www.fshare.vn/file/I4TIUE6E9QOV
https://www.fshare.vn/file/5PV15EB38M7T
https://www.fshare.vn/file/ZDLJNDNWCNYM
https://www.fshare.vn/file/O1VYZUXXJNCI
https://www.fshare.vn/file/SD5C1VZ38IPN
And My PHP code
<?php
error_reporting(E_ALL);
$lines = file('link.txt', FILE_IGNORE_NEW_LINES); // Turn Every Line To Array From File , Or use other below Method
//$lines = explode("\n",fopen('link.txt',"r")); // Other Method Get Content To Array
//$lines = explode("\n",file_get_contents('link.txt')); // Other Method Get Content To Array
foreach($lines as $key => $value){
echo $value . "</br>";
echo get_data($value); // It not Work , Why is that ?
echo get_data("https://www.fshare.vn/file/ACKM6ONXFZJE"); // It Work Normally
die();
}
function get_data($url = null) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type' => 'application/x-www-form-urlencoded', 'charset' => 'utf-8'));
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:88.0) Gecko/20100101 Firefox/88.0");
curl_setopt($ch, CURLOPT_COOKIESESSION, true);
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
curl_setopt($ch, CURLOPT_COOKIEJAR, "cookie.txt");
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 5);
curl_setopt($ch, CURLOPT_REFERER, 'http://www.google.com.vn/');
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
Right now , i have tested again and it work correctly !
Don't know what happened !
Related
Should I be able to use PHP Curl to upload a file to Sharepoint. I have this logic that I am using at the moment to "try" and upload to "My Shared Document" for now.
$the_file = file_get_contents('index.html');
$user = '&&&&&&&';
$pass = '&&&&&&&';
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_VERBOSE, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array("Content-type: multipart/form-data"));
curl_setopt($ch, CURLOPT_USERPWD, "$user:$pass");
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (X11; Linux i686; rv:6.0) Gecko/20100101 Firefox/6.0Mozilla/4.0 (compatible;)");
curl_setopt($ch, CURLOPT_URL, 'https://&&&&.net/personal/&&&&&&&/Shared%20Documents/Forms/AllItems.aspx');
curl_setopt($ch, CURLOPT_POST, true);
$post = array('file_contents' => "the_file");
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
$response = curl_exec($ch);
if(curl_errno($ch)) {
echo json_encode('Error: ' . curl_error($ch));
}
else {
echo json_encode($response);
}
Any help is much appreciated.
Thanks
Martin
When you upload a file through curl, few things you need to consider.
use # with file path to upload the file directly. example:
$post = array('file_contents' => "#/home/xyz/the_file");
// if you have parameter with the file, then it will be,
$post = array('uid' => 110, 'file_contents' => "#/home/xyz/the_file");
2. You don't need to specify this header "Content-type: multipart/form-data". It will be added automatically when you use # with your file parameter.
I want to post some data including a image.I am using curl in php for this.My code is below-
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_VERBOSE, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array("Content-type: multipart/form-data"));
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (X11; Linux i686; rv:6.0) Gecko/20100101 Firefox/6.0Mozilla/4.0 (compatible;)");
curl_setopt($ch, CURLOPT_URL, 'http://postacity.co.uk:8080/shine/pp/upload/');
curl_setopt($ch, CURLOPT_POST, true);
$post = array(
'creatorid' => '119',
'userfile' => '#/temp/fgf.jpg',
'notice' => 'Image1',
'catid' => '1',
'title' => 'bookofzeus',
'boardid' => '332',
);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
$response = curl_exec($ch);
if(curl_errno($ch)) {
echo 'Error: ' . curl_error($ch);
}
else {
echo $response;
}
}
Every time i am getting the same error
Error: failed creating formpost data
Am i doing any mistake.I have searched for this problem but still no solution found.
The path to the file is the problem: '#/temp/fgf.jpg'
You should take a look at this bug gescription: Bug 50060 - failed creating formpost data if post array value starts with #
The solutions of this problem for example at php freaks.
And use absolute path to the files.
Refer this its working
<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_VERBOSE, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible;)");
curl_setopt($ch, CURLOPT_URL, _VIRUS_SCAN_URL);
curl_setopt($ch, CURLOPT_POST, true);
// same as <input type="file" name="file_box">
$post = array(
"file_box"=>"#/path/to/myfile.jpg",
"username"=>"foobar",
"password"=>"secret",
"submit"=>"submit"
);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
$response = curl_exec($ch);
?>
Also refer this url's
http://joshhighland.com/blog/2010/11/27/using-curl-and-php-to-upload-files-through-a-form-post/
http://blogs.digitss.com/php/curl-php/posting-or-uploading-files-using-curl-with-php/
I'm using this....
function cURL($url, $header=NULL, $p=NULL)
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_NOBODY, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:9.0.1) Gecko/20100101 Firefox/9.0.1");
curl_setopt($ch, CURLOPT_COOKIEFILE, "/tmp/TestCookies");
if ($p) {
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $p);
}
$result = curl_exec($ch);
if ($result) {
return $result;
} else {
return curl_error($ch);
}
curl_close($ch);
}
And I'm making a call like this...
$Headers = array(
"Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*\/*;q=0.5",
"Accept-Language: en-gb,en;q=0.5",
"Accept-Charset: windows-1251,utf-8;q=0.7,*;q=0.7");
$a = $this->cURL("https://www.mysite.com/",$Headers, null);
I'd expect this to receive a cookie and write it to the file /tmp/TestCookies.
The site definitely returns Set-Cookie headers - I can see them if I dump $a, however, the file in question is never created.
In case it's a permissions issue, I created it with touch and chmod 777'd it - The file now exists but it's empty.
What am I doing wrong?
The curl constants are split into
CURL_COOKIEFILE as the cookies.txt source which is read from
and CURL_COOKIEJAR as the datastore that is written back to
You do have to provide _COOKIEFILE with an empty string to enable it IIRC, but _COOKIEJAR if you want them stored. Normally set both options to the same name.
I am trying to make a API call to wikipedia through: http://en.wikipedia.org/w/api.php?action=parse&page=Petunia&format=xml, but the xml is full with html and css tags.
Is there a way to fetch only plain text without tags? Thanks!
*Edit 1:
$json = json_decode(file_get_contents('http://en.wikipedia.org/w/api.php?action=parse&page=Petunia&format=json'));
$txt = strip_tags($json->text);
var_dump($json);
Null displayed.
Question was partially answered here
$url = 'http://en.wikipedia.org/w/api.php?action=parse&page=Petunia&format=json&prop=text';
$ch = curl_init($url);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($ch, CURLOPT_USERAGENT, "TestScript"); // required by wikipedia.org server
$c = curl_exec($ch);
$json = json_decode($c);
var_dump(strip_tags($json->{'parse'}->{'text'}->{'*'}))
I was not able to use file_get_contents but it works fine with cURL.
it is possible to fetch info or description from wikipedia by using xml.
$url = "http://en.wikipedia.org/w/api.php?action=opensearch&search=".$term."&format=xml&limit=1";
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_HTTPGET, TRUE);
curl_setopt($ch, CURLOPT_POST, FALSE);
curl_setopt($ch, CURLOPT_HEADER, false); // Include head as needed
curl_setopt($ch, CURLOPT_NOBODY, FALSE); // Return body
curl_setopt($ch, CURLOPT_VERBOSE, FALSE); // Minimize logs
curl_setopt($ch, CURLOPT_REFERER, ""); // Referer value
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE); // No certificate
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE); // Follow redirects
curl_setopt($ch, CURLOPT_MAXREDIRS, 4); // Limit redirections to four
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE); // Return in string
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 6.1; he; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8"); // Webbot name
$page = curl_exec($ch);
$xml = simplexml_load_string($page);
if((string)$xml->Section->Item->Description) {
print_r(array((string)$xml->Section->Item->Text,
(string)$xml->Section->Item->Description,
(string)$xml->Section->Item->Url));
} else {
echo "sorry";
}
But curl must be install on server... have a nice day...
<?php
error_reporting(-1);
$config = array
(
"siteURL" => "http://domain.com",
"loginCheck" => "checkuser.php",
"userAgent" => "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.204 Safari/534.16"
);
$postFields = "username=user&password=pass&submit= Login ";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $config['siteURL'] . $config['loginCheck']);
curl_setopt($ch, CURLOPT_COOKIEJAR, "cookie.txt");
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postFields);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_USERAGENT, config['userAgent']);
curl_setopt($ch, CURLOPT_REFERER, config['siteURL']);
$content = curl_exec($ch);
curl_close($ch);
echo $content;
?>
I expect this to be echoing the result of the curl, but its not. am I doing anything wrong?
It does, if you set a complete URL with a '/' at the end (fix two other typos):
error_reporting(-1);
$config = array
(
"siteURL" => "http://www.apple.com/",
"loginCheck" => "checkuser.php",
"userAgent" => "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.204 Safari/534.16"
);
$postFields = "username=user&password=pass&submit= Login ";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $config['siteURL'] . $config['loginCheck']);
curl_setopt($ch, CURLOPT_COOKIEJAR, "cookie.txt");
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postFields);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_USERAGENT, $config['userAgent']);
curl_setopt($ch, CURLOPT_REFERER, $config['siteURL']);
$content = curl_exec($ch);
curl_close($ch);
echo $content;
If curl doesn't return anything then you have to check for the reason why that happen. Check for the last curl error message:
if(curl_exec($ch) === false)
{
echo 'Curl error: ' . curl_error($ch);
}
else
{
echo 'Operation completed without any errors, you have the response';
}
Basically it would be good to always check on that. You shouldn't assume that curl request succeeded.
Two of your variables are named badly:
curl_setopt($ch, CURLOPT_USERAGENT, config['userAgent']);
curl_setopt($ch, CURLOPT_REFERER, config['siteURL']);
Should be:
curl_setopt($ch, CURLOPT_USERAGENT, $config['userAgent']);
curl_setopt($ch, CURLOPT_REFERER, $config['siteURL']);
My guess is that PHP faulted to a blank page (since the variable is seen as a constant but constants must be scalar).
Another issue may be the user agent. I've seen servers that completely refuse to reply if no user agent was set.
You can use a class that I wrote that wraps around CURL. To install, see: https://github.com/homer6/altumo
It's much easier to use and can give you some easy to access debugging. For example:
try{
//load class autoloader
require_once( __DIR__ . '/loader.php' ); //you should ensure that is is the correct path for this file
//make the response; return the response with the headers
$client = new \Altumo\Http\OutgoingHttpRequest( 'http://www.domain.com/checkuser.php', array(
'username' => 'user',
'password' => 'pass',
'submit' => ' Login '
));
$client->setRequestMethod( \Altumo\Http\OutgoingHttpRequest::HTTP_METHOD_POST );
//send the request (with optional arguments for debugging)
//the first true will return the response headers
//the second true will turn on curl info so that it can be retrieved later
$response = $client->send( true, true );
//output the response and curl info
\Altumo\Utils\Debug::dump( $response, $client->getCurlInfo() );
//alternatively, you can get the response wrapped in an object that allows you to retrieve parts of the response
$http_response = $client->sendAndGetResponseMessage( true );
$status_code = $http_response->getStatusCode();
$message_body = $http_response->getMessageBody();
$full_http_response = $http_response->getRawHttpResponse();
\Altumo\Utils\Debug::dump( $status_code, $message_body, $full_http_response );
}catch( \Exception $e ){
//This will display an error if any exceptions have been thrown
echo 'Error: ' . $e->getMessage();
}
Hope that helps...