I have created a loop to extract specific links from google. but at the end when i am running this script it is creating empty file without any links.
for ($n=0;$n<500;$n+=10)
{
$country = $_GET['country'];
// Country Changing array to scan through different countries.
$google = array (
"default" => array("google.com" , "countryUS"),
"NZ" => array("google.co.nz" , "countryNZ"),
"UK" => array("google.co.uk" , "countryUK|countryGB"),
"AU" => array("google.com.au" , "countryAU")
);
// Variables for fopen
$URL = 'www.'.$google[$country][0];
$PORT = "80";
$TIMEOUT = 30;
$Q = $_GET['query'];
// Google String
$GET = '/search?q='.urlencode($Q).'&hl=en&cr='.$google[$country][1].'&tbs=ctr:'.$google[$country][1].'&start='.$n.'&sa=N';
$fp = fsockopen($URL, $PORT, $errno, $errstr, $TIMEOUT);
if (!$fp)
{
echo "Error connecting to {$URL}<br>\n";
} else
{
$out = "GET $GET HTTP/1.1\r\n";
$out .= "Host: $URL\r\n";
$out .= "Connection: Close\r\n\r\n";
fwrite($fp, $out);
$buffer = '';
while (!feof($fp))
{
$buffer .= fgets($fp, 128);
}
fclose($fp);
write($SITEs, filter($buffer));
}
}
please tell me if i am doing anything wrong.
Note: fopen is ON in my PHP.
Adding:
function write($A, $B) {
#touch($A);
$C = fopen($A, 'a');
fwrite($C, $B);
fclose($C);
}
You haven't opened your file you're going to write in!
Also, there's no PHP function write()
Related
This question already has answers here:
check if php file exists on server
(2 answers)
Closed 4 years ago.
how can I make sure that the file exists on the server and find out its size on the URL without first downloading the file
$url = 'http://site.zz/file.jpg';
file_exists($url); //always is false
filesize($url); //not working
Help eny one worked exemple pls
The function file_exists() only works on file that exists on the server locally.
Similarly; filesize() function returns the size of the file that exists on the server locally.
If you are trying to load the size of a file for a given url, you can try this approach:
function get_remote_file_info($url) {
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_HEADER, TRUE);
curl_setopt($ch, CURLOPT_NOBODY, TRUE);
$data = curl_exec($ch);
$fileSize = curl_getinfo($ch, CURLINFO_CONTENT_LENGTH_DOWNLOAD);
$httpResponseCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);
return [
'fileExists' => (int) $httpResponseCode == 200,
'fileSize' => (int) $fileSize
];
}
Usage:
$url = 'http://site.zz/file.jpg';
$result = get_remote_file_info($url);
var_dump($result);
Example output:
array(2) {
["fileExists"]=>
bool(true)
["fileSize"]=>
int(12345)
}
Without any libraries and file openning
$data = get_headers($url, true);
$size = isset($data['Content-Length'])?(int) $data['Content-Length']:0;
Open remote files:
function fsize($path) {
$fp = fopen($path,"r");
$inf = stream_get_meta_data($fp);
fclose($fp);
foreach($inf["wrapper_data"] as $v) {
if (stristr($v, "content-length")) {
$v = explode(":", $v);
return trim($v[1]);
}
}
return 0;
}
Usage:
$file = "https://zzz.org/file.jpg";
$inbytes = fsize($filesize);
Use sockets:
function getRemoteFileSize($url){
$parse = parse_url($url);
$host = $parse['host'];
$fp = #fsockopen ($host, 80, $errno, $errstr, 20);
if(!$fp){
$ret = 0;
}else{
$host = $parse['host'];
fputs($fp, "HEAD ".$url." HTTP/1.1\r\n");
fputs($fp, "HOST: ".$host."\r\n");
fputs($fp, "Connection: close\r\n\r\n");
$headers = "";
while (!feof($fp)){
$headers .= fgets ($fp, 128);
}
fclose ($fp);
$headers = strtolower($headers);
$array = preg_split("|[\s,]+|",$headers);
$key = array_search('content-length:',$array);
$ret = $array[$key+1];
}
if($array[1]==200) return $ret;
else return -1*$array[1];
}
You can't access to filesize of a distant file.
You have to check with your local filepath.
I need to get whois information. My function is working fine, but isn't returning "Administrative Contact, Registrant Contact, Administrative Contact, Technical Contact" information.
But when I run following command on my Mac it returns all information
"whois google.com"
Here is my php function to get information from whois server
function QueryWhoisServer($whoisserver, $domain) {
$port = 43;
$timeout = 10;
$fp = #fsockopen($whoisserver, $port, $errno, $errstr, $timeout) or die("Socket Error " . $errno . " - " . $errstr);
fputs($fp, $domain . "\r\n");
$out = "";
while(!feof($fp)){
$out .= fgets($fp);
}
fclose($fp);
return $out;
}
echo QueryWhoisServer("whois.verisign-grs.com", "google.com");
I found a possible solution, assuming (as checked by myself) registrar whois servers are returning contact infos.
To do this, it is needed for each domain to query related registrar whois server as per code below.
Check code comments for little explanation of what each function does.
function GetWhoisInfo($whoisserver, $domain){
$port = 43;
$timeout = 10;
$fp = #fsockopen($whoisserver, $port, $errno, $errstr, $timeout) or die("Socket Error " . $errno . " - " . $errstr);
stream_set_blocking($fp, true);
fputs($fp, $domain . "\r\n");
$out = "";
while(!feof($fp)){
$out .= fgets($fp);
}
fclose($fp);
return $out;
}
function GetRegistrarWhoisServer($whoisserver, $domain) {
$out = GetWhoisInfo($whoisserver, $domain);
$rws_string = explode("\r\n", $out);
$rws = explode("Registrar WHOIS Server: ", $rws_string[2])[1];
return $rws;
}
function WhoisToJson($winfo) {
$winfoarr = explode(PHP_EOL, $winfo);
$jsonarr = [];
foreach($winfoarr as $info){
$infodata = explode(": ", $info);
if($infodata[0] !== "")$jsonarr[$infodata[0]] = $infodata[1];
//avoid to process privacy info at the end of whois service output
if($infodata[0] === "DNSSEC")break;
}
return json_encode($jsonarr);
}
function QueryWhoisServer($whoisserver, $domain) {
//query to $whoisserver whois to get registrar whois server address only
$rws = GetRegistrarWhoisServer($whoisserver, $domain);
//query to registrar whois server (registrar whois servers are returning contact infos)
$out = GetWhoisInfo($rws, $domain);
//parsing infos and formatting to json
return WhoisToJson($out);
}
echo QueryWhoisServer("whois.verisign-grs.com", "google.com");
I have a script that makes 2-3 database queries , 2 inserts, and 1 update.
The issue I am having is that when I manually test the script by going to the web server page like www.domain.com/handler.php?a=1&b=2&c=3, I get the 3 updates perfectly. But when I use this function, it doesn't work... more specifically, the INSERT INTO doesn't work. I get the update in the database perfectly fine!, but no new entries are made.
while (($data = fgetcsv(STDIN)) !== false){
$url='http://domain.com';
$date = date('Y-m-d');
$type = trim($data[0]);
if ($type != ''){
$send_date = $data[2];
$email = $data[4];
$status = $data[7];
$error = $data[8];
$bounce = $data[10];
$ip = $data[14];
$job_id = $data[19];
$params = 'pmta_token='.$pmta_token.'&type='.$type.'&send_date='.$send_date.'&email='.$email.'&status='.$status.'&error='.$error.'&bounce='.$bounce.'&ip='.$ip.'&job_id='.$job_id;
$params= array('server'=>$_SERVER['SERVER_NAME'],'type'=>$data[0],'time'=>$data[1],'message'=>$data[8],'bouncecat'=>$data[10],'jobid'=>$data[19],'domip'=>$data[21]);
$line = $type.','.$email.','.$ip.','.$status.','.$error.','.$bounce.','.$job_id.','.date('c')."\n";
curl_post_async($url, $params);
}
}
function curl_post_async($url, $params){
foreach($params as $key => $val){
if (is_array($val)) $val = implode(',', $val);
$post_params[] = $key.'='.urlencode($val);
}
$post_string = implode('&', $post_params);
$parts=parse_url($url);
$fp = fsockopen($parts['host'],
isset($parts['port'])?$parts['port']:80,
$errno, $errstr, 30);
$out = "POST ".$parts['path']." HTTP/1.1\r\n";
$out.= "Host: ".$parts['host']."\r\n";
$out.= "Content-Type: application/x-www-form-urlencoded\r\n";
$out.= "Content-Length: ".strlen($post_string)."\r\n";
$out.= "Connection: Close\r\n\r\n";
if (isset($post_string)) $out.= $post_string;
fwrite($fp, $out);
fclose($fp);
}
I must be missing something! the reason I am using this function instead of curl, is because of the time curl takes connecting and waiting for a response... this script is handling about 2,000 requests a minute.
here is the handler.php
this is the one that is failing and not entering anything into the DB, no errors when I do it manually, it works like a charm, but using the socket fails, ONLY on the INSERT, the UPDATE works through the socket connect.?.?.?
mysql_connect('localhost','root','');
mysql_select_db('analyzer');
date_default_timezone_set('America/Los_Angeles');
if(isset($_POST['type'])){
$domip = #explode('/',$_POST['domip']);
$server = #strtolower($_POST['server']);
$type = #$_POST['type'];
$time = #$_POST['time'];
$message = #$_POST['message'];
$bouncecat = #$_POST['bouncecat'];
$jobid = #$_POST['jobid'];
$domain = #$domip[0];
$md5 = #$_POST['md5'];
$ip = #$domip[1];
if(isset($_POST['ip'])){$ip=$_POST['ip'];}
}else{
$domip = #explode('/',$_REQUEST['domip']);
$server = #strtolower($_REQUEST['server']);
$type = #$_REQUEST['type'];
$time = #$_REQUEST['time'];
$message = #$_REQUEST['message'];
$bouncecat = #$_REQUEST['bouncecat'];
$jobid = #$_REQUEST['jobid'];
$domain = #$domip[0];
$md5 = #$_REQUEST['md5'];
$ip = #$domip[1];
}
$now=date('Y-m-d H:i');
#mysql_query("INSERT INTO `log` (`type`,`time`,`message`,`cat`,`jobid`,`domain`,`ip`,`server`) VALUES('$type','$time','$message','$bouncecat','$jobid','$domain','$ip','$server')");
if(strpos($message,'ALERT'!==false)){
#mysql_query("INSERT INTO `alerts` (`type`,`time`,`message`,`level`,`value`,`page`) VALUES('error','$now','$message','5','$ip','apps/loganalyzer/incl/datatracker.php')");
}
if($type=='d'){
#mysql_query("UPDATE `$server` SET `delivered`=`delivered`+1 WHERE `time`='$now'");
}elseif(($type=='b')&&($bouncecat!='bad')){
#mysql_query("UPDATE `$server` SET `errored`=`errored`+1 WHERE `time`='$now'");
}elseif(($type=='b')&&($bouncecat=='bad')){
#mysql_query("UPDATE `$server` SET `bounced`=`bounced`+1 WHERE `time`='$now'");
}elseif($type=='c'){
#mysql_query("UPDATE `$server` SET `unsubscribed`=`unsubscribed`+1 WHERE `time`='$now'");
}
While I don't know exactly what the problem is, I have 2 suggestions for you to debug it:
Remove the # sign before the functions - that is hiding you the error you need to fix.
Make a small script to test the function with curl out of your system, so that you can actually get to see what is wrong with the INSERT query.
I have the following function that is supposed to open up a twitter stream for me
<?php
set_time_limit(0);
$query_data = array('track' => 'facebook');
$user = 'xxx'; // replace with your account
$pass = 'xxx'; // replace with your account
$fp = fsockopen("ssl://stream.twitter.com", 443, $errno, $errstr, 30);
if(!$fp){
print "$errstr ($errno)\n";
} else {
$request = "GET /1/statuses/filter.json?" . http_build_query($query_data) . " HTTP/1.1\r\n";
$request .= "Host: stream.twitter.com\r\n";
$request .= "Authorization: Basic " . base64_encode($user . ':' . $pass) . "\r\n\r\n";
fwrite($fp, $request);
while(!feof($fp)){
$json = fgets($fp);
$data = json_decode($json, true);
if($data){
//
// Do something with the data!
//
echo $data . "<br />";
}
}
fclose($fp);
}
?>
This is my first time ever to use php at all, so I'm completely oblivious as to what to do with this $data variable, this echo line does not produce any output on the browser. I'm sure this is trivial but a first-timer for me.
EDIT: The question is, what do I do with $data to make sure the code works? do I just shoot the file in a browser? it keeps saying "waiting for localhost..." forever
Why not just check to see if it's an array after the json_decode?
if( is_array( $data ) and count( $data ) ) // Ensures that json_decode created an array and has more than one element
{
// do your processing, it should be okay
}
else
{
// it's broken, error out
}
This is very strange, on some pages it will return the HTML fine, others it will add numbers to the beginning and end of the returned string ($out).
function lookupPage($page, $return = true) {
$fp = fsockopen("127.0.0.1", 48580, $errno, $errstr, 5);
if (!$fp) {
return false;
}
else {
$out = "";
$headers = "GET /" . $page . " HTTP/1.1\r\n";
$headers .= "Host: www.site.com\r\n";
$headers .= "Connection: Close\r\n\r\n";
fwrite($fp, $headers);
stream_set_timeout($fp, 300);
$info = stream_get_meta_data($fp);
while (!feof($fp) && !$info['timed_out'] && ($line = stream_get_line($fp, 1024)) !== false) {
$info = stream_get_meta_data($fp);
if ($return) $out .= $line;
}
fclose($fp);
if (!$info['timed_out']) {
if ($return) {
$out = substr($out, strpos($out, "\r\n\r\n") + 4);
return $out;
}
else {
return true;
}
}
else {
return false;
}
}
}
e.g...
3565
<html>
<head>
...
</html>
0
It is called Chunked Transfer Encoding
It is part of the HTTP 1.1 protocol and you're decoding it in a HTTP 1.0 way. You can just check for the values and trim them if you want. They only show the length of the response so the browser knows it has the complete response.
Also maybe look at file_get_contents
My guess would be that the server responds with chunked data.
Have a look at RFC2616 Transfer codings and its introduction.