Is it possible to check the response headers (200=OK) and download a file in a single CURL request?
Here is my code. The problem with this is that it makes 2 requests, and hence the second request can be different and the saved file will be overwritten. This is a problem with rate limited API. I searched here on Stackoverflow but most solutions still make 2 requests.
// Check response first, we don't want to download the response error to the file
$urlCheck = checkRemoteFile($to_download);
if ($urlCheck) {
// Response is 200, continue
} else {
// Do not overwrite existing file
echo 'Download failed, response code header is not 200';
exit();
}
// File Handling
$new_file = fopen($downloaded, "w") or die("cannot open" . $downloaded);
// Setting the curl operations
$cd = curl_init();
curl_setopt($cd, CURLOPT_URL, $to_download);
curl_setopt($cd, CURLOPT_FILE, $new_file);
curl_setopt($cd, CURLOPT_TIMEOUT, 30); // timeout is 30 seconds, to download the large files you may need to increase the timeout limit.
// Running curl to download file
curl_exec($cd);
if (curl_errno($cd)) {
echo "the cURL error is : " . curl_error($cd);
} else {
$status = curl_getinfo($cd);
echo $status["http_code"] == 200 ? "File Downloaded" : "The error code is : " . $status["http_code"] ;
// the http status 200 means everything is going well. the error codes can be 401, 403 or 404.
}
// close and finalize the operations.
curl_close($cd);
fclose($new_file);
# FUNCTIONS
function checkRemoteFile($url) {
$curl = curl_init($url);
//don't fetch the actual page, you only want to check the connection is ok
curl_setopt($curl, CURLOPT_NOBODY, true);
//do request
$result = curl_exec($curl);
$ret = false;
//if request did not fail
if ($result !== false) {
//if request was ok, check response code
$statusCode = curl_getinfo($curl, CURLINFO_HTTP_CODE);
if ($statusCode == 200) {
$ret = true;
}
}
curl_close($curl);
return $ret;
}
I am trying to update workspace name through geoserver rest api using put method.
I am getting "Can't change the name of a workspace." error.
This is my code.
$service = geoserver_url;
$data = "<workspace><name>testnew</name></workspace>";
$url = $service . "rest/workspaces/workspacename";
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$passwordStr = username:password
curl_setopt($ch, CURLOPT_USERPWD, $passwordStr);
curl_setopt($ch, CURLOPT_HTTPHEADER, array("Content-type: application/xml"););
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'PUT');
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
$result = curl_exec($ch);
$info = curl_getinfo($ch);
Any kind of help appreciated.
That is not an allowable operation (as mentioned in the manual). Any PUT that changes the name of a workspace returns a 403 Error.
All you can do is create a new workspace, copy the contents of the old one and then delete it.
According to geoserver REST API documentations (link), you can simply edit workspace name (ws_name) by putting a request which has an xml string data in its body.
Here I've given you an example. Because of using Express as my application sever I've implemented a request using javascript but it can be changed to your favorite syntax.
const options = {
url: 'http://localhost:8080/geoserver/rest/workspaces/{current_ws_sname}',
method: 'PUT',
headers: { 'Content-type': 'text/xml' },
body: '<workspace><name>{new_ws_name}</name></workspace>',
auth: {
user: {geoserver_username},
pass: {geoserver_password}
}
and for executing the request based on above option variable I've used request function in Express:
request(options, (error, response, body) => {
console.log('response status code:' , response.statusCode)
console.log('response status message: ', response.statusMessage)
const result = {}
if (!error) {
const statusCode = response.statusCode;
const statusMessage = response.statusMessage;
if (statusCode == 200) {
result.err = false
result.msg = `Modified`
} else if (statusCode == 404) {
result.err = true
result.msg = 'Workspace not found!'
} else if (statusCode == 405) {
result.err = true
result.msg = 'Forbidden to change the name of the workspace!'
//because of your username and/or password
} else if (statusCode == 500) {
result.err = true
result.msg = 'Server internal error or duplicate names'
}
} else {
result.err = true,
result.msg = error;
}
res.send(result);
})
Don't forget to replace {current_ws_sname}, {new_ws_sname}, {geoserver_username}, {geoserver_password} by your own values.
All the possible situations have been mentioned as above (i.e. 200, 404, 405, 500) and there is no error message like "Can't change the name of a workspace." in geoserver documentations!
What statusCode and statusMessage do you get in response? Can you confirm that your {new_ws_sname} isn't duplicate?
I'm trying to set up a telegram bot with a webhook. I can get it to work with getUpdates, but I want it to work with a webhook.
My site (that hosts the bot php script) has the SSL certificate working (I get the green lock in the address bar):
I set up the webhook with
https://api.telegram.org/bot<token>/setwebhook?url=https://www.example.com/bot/bot.php
And I got: {"ok":true,"result":true,"description":"Webhook was set"}
(I don't know if this matters, but I have given rwx rights to both the folder and the script)
The php bot: (https://www.example.com/bot/bot.php)
<?php
$botToken = <token>;
$website = "https://api.telegram.org/bot".$botToken;
#$update = url_get_contents('php://input');
$update = file_get_contents('php://input');
$update = json_decode($update, TRUE);
$chatId = $update["message"]["chat"]["id"];
$message = $update["message"]["text"];
switch($message) {
case "/test":
sendMessage($chatId, "test");
break;
case "/hi":
sendMessage($chatId, "hi there!");
break;
default:
sendMessage($chatId, "default");
}
function sendMessage ($chatId, $message) {
$url = $GLOBALS[website]."/sendMessage?chat_id=".$chatId."&text=".urlencode($message);
url_get_contents($url);
}
function url_get_contents($Url) {
if(!function_exists('curl_init')) {
die('CURL is not installed!');
}
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $Url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$output = curl_exec($ch);
curl_close($ch);
return $output;
}
?>
But when I write anything to the bot I receive no answers...
Any ideas why?
Thanks
In your question it's not clear the script location. Seeing your code, it seems that you try to load a request through url_get_contents to retrieve telegram server response. This is the correct method if your bot works without webhook. Otherwise, after setting webhook, you have to process incoming requests.
I.e., if you set webhook to https://example.com/mywebhook.php, in your https://example.com/mywebhook.php script you have to write something like this:
<?php
$request = file_get_contents( 'php://input' );
# ↑↑↑↑
$request = json_decode( $request, TRUE );
if( !$request )
{
// Some Error output (request is not valid JSON)
}
elseif( !isset($request['update_id']) || !isset($request['message']) )
{
// Some Error output (request has not message)
}
else
{
$chatId = $request['message']['chat']['id'];
$message = $request['message']['text'];
switch( $message )
{
// Process your message here
}
}
I need to create a function that returns if a URL is reachable or valid.
I am currently using something like the following to determine a valid url:
static public function urlExists($url)
{
$fp = #fopen($url, 'r');
if($fp)
{
return true;
}
return false;
}
It seems like there would be something faster, maybe something that just fetched the page header or something.
You can use curl as follows:
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_NOBODY, true); // set to HEAD request
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); // don't output the response
curl_exec($ch);
$valid = curl_getinfo($ch, CURLINFO_HTTP_CODE) == 200;
curl_close($ch);
You could check http status code.
Here is a code you could use to check that an url returns 2xx or 3xx http code to ensure the url works.
<?php
$url = "http://stackoverflow.com/questions/1122845";
function urlOK($url)
{
$url_data = parse_url ($url);
if (!$url_data) return FALSE;
$errno="";
$errstr="";
$fp=0;
$fp=fsockopen($url_data['host'],80,$errno,$errstr,30);
if($fp===0) return FALSE;
$path ='';
if (isset( $url_data['path'])) $path .= $url_data['path'];
if (isset( $url_data['query'])) $path .= '?' .$url_data['query'];
$out="GET /$path HTTP/1.1\r\n";
$out.="Host: {$url_data['host']}\r\n";
$out.="Connection: Close\r\n\r\n";
fwrite($fp,$out);
$content=fgets($fp);
$code=trim(substr($content,9,4)); //get http code
fclose($fp);
// if http code is 2xx or 3xx url should work
return ($code[0] == 2 || $code[0] == 3) ? TRUE : FALSE;
}
echo $url;
if (urlOK($url)) echo " is a working URL";
else echo " is a bad URL";
?>
Hope this helps!
You'll likely be limited to sending some kind of HTTP request. Then you can check HTTP status codes.
Be sure to send only a "HEAD" request, which doesn't pull back all the content. That ought to be sufficient and lightweight enough.
The best I could find, an if fclose fopen type thing, makes the page load really slowly.
Basically what I'm trying to do is the following: I have a list of websites, and I want to display their favicons next to them. However, if a site doesn't have one, I'd like to replace it with another image rather than display a broken image.
You can instruct curl to use the HTTP HEAD method via CURLOPT_NOBODY.
More or less
$ch = curl_init("http://www.example.com/favicon.ico");
curl_setopt($ch, CURLOPT_NOBODY, true);
curl_exec($ch);
$retcode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
// $retcode >= 400 -> not found, $retcode = 200, found.
curl_close($ch);
Anyway, you only save the cost of the HTTP transfer, not the TCP connection establishment and closing. And being favicons small, you might not see much improvement.
Caching the result locally seems a good idea if it turns out to be too slow.
HEAD checks the time of the file, and returns it in the headers. You can do like browsers and get the CURLINFO_FILETIME of the icon.
In your cache you can store the URL => [ favicon, timestamp ]. You can then compare the timestamp and reload the favicon.
As Pies say you can use cURL. You can get cURL to only give you the headers, and not the body, which might make it faster. A bad domain could always take a while because you will be waiting for the request to time-out; you could probably change the timeout length using cURL.
Here is example:
function remoteFileExists($url) {
$curl = curl_init($url);
//don't fetch the actual page, you only want to check the connection is ok
curl_setopt($curl, CURLOPT_NOBODY, true);
//do request
$result = curl_exec($curl);
$ret = false;
//if request did not fail
if ($result !== false) {
//if request was ok, check response code
$statusCode = curl_getinfo($curl, CURLINFO_HTTP_CODE);
if ($statusCode == 200) {
$ret = true;
}
}
curl_close($curl);
return $ret;
}
$exists = remoteFileExists('http://stackoverflow.com/favicon.ico');
if ($exists) {
echo 'file exists';
} else {
echo 'file does not exist';
}
CoolGoose's solution is good but this is faster for large files (as it only tries to read 1 byte):
if (false === file_get_contents("http://example.com/path/to/image",0,null,0,1)) {
$image = $default_image;
}
This is not an answer to your original question, but a better way of doing what you're trying to do:
Instead of actually trying to get the site's favicon directly (which is a royal pain given it could be /favicon.png, /favicon.ico, /favicon.gif, or even /path/to/favicon.png), use google:
<img src="http://www.google.com/s2/favicons?domain=[domain]">
Done.
A complete function of the most voted answer:
function remote_file_exists($url)
{
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_NOBODY, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); # handles 301/2 redirects
curl_exec($ch);
$httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);
if( $httpCode == 200 ){return true;}
}
You can use it like this:
if(remote_file_exists($url))
{
//file exists, do something
}
If you are dealing with images, use getimagesize. Unlike file_exists, this built-in function supports remote files. It will return an array that contains the image information (width, height, type..etc). All you have to do is to check the first element in the array (the width). use print_r to output the content of the array
$imageArray = getimagesize("http://www.example.com/image.jpg");
if($imageArray[0])
{
echo "it's an image and here is the image's info<br>";
print_r($imageArray);
}
else
{
echo "invalid image";
}
if (false === file_get_contents("http://example.com/path/to/image")) {
$image = $default_image;
}
Should work ;)
This can be done by obtaining the HTTP Status code (404 = not found) which is possible with file_get_contentsDocs making use of context options. The following code takes redirects into account and will return the status code of the final destination (Demo):
$url = 'http://example.com/';
$code = FALSE;
$options['http'] = array(
'method' => "HEAD",
'ignore_errors' => 1
);
$body = file_get_contents($url, NULL, stream_context_create($options));
foreach($http_response_header as $header)
sscanf($header, 'HTTP/%*d.%*d %d', $code);
echo "Status code: $code";
If you don't want to follow redirects, you can do it similar (Demo):
$url = 'http://example.com/';
$code = FALSE;
$options['http'] = array(
'method' => "HEAD",
'ignore_errors' => 1,
'max_redirects' => 0
);
$body = file_get_contents($url, NULL, stream_context_create($options));
sscanf($http_response_header[0], 'HTTP/%*d.%*d %d', $code);
echo "Status code: $code";
Some of the functions, options and variables in use are explained with more detail on a blog post I've written: HEAD first with PHP Streams.
PHP's inbuilt functions may not work for checking URL if allow_url_fopen setting is set to off for security reasons. Curl is a better option as we would not need to change our code at later stage. Below is the code I used to verify a valid URL:
$url = str_replace(' ', '%20', $url);
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_NOBODY, true);
curl_exec($ch);
$httpcode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);
if($httpcode>=200 && $httpcode<300){ return true; } else { return false; }
Kindly note the CURLOPT_SSL_VERIFYPEER option which also verify the URL's starting with HTTPS.
To check for the existence of images, exif_imagetype should be preferred over getimagesize, as it is much faster.
To suppress the E_NOTICE, just prepend the error control operator (#).
if (#exif_imagetype($filename)) {
// Image exist
}
As a bonus, with the returned value (IMAGETYPE_XXX) from exif_imagetype we could also get the mime-type or file-extension with image_type_to_mime_type / image_type_to_extension.
A radical solution would be to display the favicons as background images in a div above your default icon. That way, all overhead would be placed on the client while still not displaying broken images (missing background images are ignored in all browsers AFAIK).
You could use the following:
$file = 'http://mysite.co.za/images/favicon.ico';
$file_exists = (#fopen($file, "r")) ? true : false;
Worked for me when trying to check if an image exists on the URL
function remote_file_exists($url){
return(bool)preg_match('~HTTP/1\.\d\s+200\s+OK~', #current(get_headers($url)));
}
$ff = "http://www.emeditor.com/pub/emed32_11.0.5.exe";
if(remote_file_exists($ff)){
echo "file exist!";
}
else{
echo "file not exist!!!";
}
This works for me to check if a remote file exist in PHP:
$url = 'https://cdn.sstatic.net/Sites/stackoverflow/img/favicon.ico';
$header_response = get_headers($url, 1);
if ( strpos( $header_response[0], "404" ) !== false ) {
echo 'File does NOT exist';
} else {
echo 'File exists';
}
You can use :
$url=getimagesize(“http://www.flickr.com/photos/27505599#N07/2564389539/”);
if(!is_array($url))
{
$default_image =”…/directoryFolder/junal.jpg”;
}
If you're using the Laravel framework or guzzle package, there is also a much simpler way using the guzzle client, it also works when links are redirected:
$client = new \GuzzleHttp\Client(['allow_redirects' => ['track_redirects' => true]]);
try {
$response = $client->request('GET', 'your/url');
if ($response->getStatusCode() != 200) {
// not exists
}
} catch (\GuzzleHttp\Exception\GuzzleException $e) {
// not exists
}
More in Document : https://docs.guzzlephp.org/en/latest/faq.html#how-can-i-track-redirected-requests
You should issue HEAD requests, not GET one, because you don't need the URI contents at all. As Pies said above, you should check for status code (in 200-299 ranges, and you may optionally follow 3xx redirects).
The answers question contain a lot of code examples which may be helpful: PHP / Curl: HEAD Request takes a long time on some sites
There's an even more sophisticated alternative. You can do the checking all client-side using a JQuery trick.
$('a[href^="http://"]').filter(function(){
return this.hostname && this.hostname !== location.hostname;
}).each(function() {
var link = jQuery(this);
var faviconURL =
link.attr('href').replace(/^(http:\/\/[^\/]+).*$/, '$1')+'/favicon.ico';
var faviconIMG = jQuery('<img src="favicon.png" alt="" />')['appendTo'](link);
var extImg = new Image();
extImg.src = faviconURL;
if (extImg.complete)
faviconIMG.attr('src', faviconURL);
else
extImg.onload = function() { faviconIMG.attr('src', faviconURL); };
});
From http://snipplr.com/view/18782/add-a-favicon-near-external-links-with-jquery/ (the original blog is presently down)
all the answers here that use get_headers() are doing a GET request.
It's much faster/cheaper to just do a HEAD request.
To make sure that get_headers() does a HEAD request instead of a GET you should add this:
stream_context_set_default(
array(
'http' => array(
'method' => 'HEAD'
)
)
);
so to check if a file exists, your code would look something like this:
stream_context_set_default(
array(
'http' => array(
'method' => 'HEAD'
)
)
);
$headers = get_headers('http://website.com/dir/file.jpg', 1);
$file_found = stristr($headers[0], '200');
$file_found will return either false or true, obviously.
If the file is not hosted external you might translate the remote URL to an absolute Path on your webserver. That way you don't have to call CURL or file_get_contents, etc.
function remoteFileExists($url) {
$root = realpath($_SERVER["DOCUMENT_ROOT"]);
$urlParts = parse_url( $url );
if ( !isset( $urlParts['path'] ) )
return false;
if ( is_file( $root . $urlParts['path'] ) )
return true;
else
return false;
}
remoteFileExists( 'https://www.yourdomain.com/path/to/remote/image.png' );
Note: Your webserver must populate DOCUMENT_ROOT to use this function
Don't know if this one is any faster when the file does not exist remotely, is_file(), but you could give it a shot.
$favIcon = 'default FavIcon';
if(is_file($remotePath)) {
$favIcon = file_get_contents($remotePath);
}
If you're using the Symfony framework, there is also a much simpler way using the HttpClientInterface:
private function remoteFileExists(string $url, HttpClientInterface $client): bool {
$response = $client->request(
'GET',
$url //e.g. http://example.com/file.txt
);
return $response->getStatusCode() == 200;
}
The docs for the HttpClient are also very good and maybe worth looking into if you need a more specific approach: https://symfony.com/doc/current/http_client.html
You can use the filesystem:
use Symfony\Component\Filesystem\Filesystem;
use Symfony\Component\Filesystem\Exception\IOExceptionInterface;
and check
$fileSystem = new Filesystem();
if ($fileSystem->exists('path_to_file')==true) {...
Please check this URL. I believe it will help you. They provide two ways to overcome this with a bit of explanation.
Try this one.
// Remote file url
$remoteFile = 'https://www.example.com/files/project.zip';
// Initialize cURL
$ch = curl_init($remoteFile);
curl_setopt($ch, CURLOPT_NOBODY, true);
curl_exec($ch);
$responseCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);
// Check the response code
if($responseCode == 200){
echo 'File exists';
}else{
echo 'File not found';
}
or visit the URL
https://www.codexworld.com/how-to/check-if-remote-file-exists-url-php/#:~:text=The%20file_exists()%20function%20in,a%20remote%20server%20using%20PHP.