How to make PHP bot print the title - php

$instagram = "ugur2d";
$kaynak = file_get_contents("https://www.instagram.com/$instagram/");
preg_match('#<title>(.*?)</title>#si', $kaynak, $iglink);
echo $iglink[1];
Screen is empty. How can i run it?

After some checkings...
your url returns error 404 - file_get_contents will return false if 404 is returned by the server
(instagram returns a 404 error page if profile does not exist)
thats why you dont get any result!
anyway you should use curl instead!

Okey okey okey. Im fixed problem. Answer -> CURL :D
$ch = curl_init();
curl_setopt_array($ch, [
CURLOPT_URL => 'https://toosba.com/',
CURLOPT_USERAGENT => "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36",
CURLOPT_RETURNTRANSFER => true,
CURLOPT_FOLLOWLOCATION => true,
CURLOPT_SSL_VERIFYPEER => false,
CURLOPT_SSL_VERIFYHOST => FALSE
]);
$source = curl_exec($ch);
curl_close($ch);
preg_match('/<title>(.*?)<\/title>/', $source, $title);
print_r($title);
echo "<hr>".$title[1];
Thank you :)

Related

View Twitter home page with PHP cURL

I want to display some information on my website directly from Twitter. I know that there is their own API but what I need does not need all this since it is just information from the Twitter Home: https://twitter.com/explore
My code is just below I used to, but Twitter updated recently and now I can't.
$url = 'https://twitter.com/explore';
$options = [
CURLOPT_URL => $url,
CURLOPT_CUSTOMREQUEST => 'GET',
CURLOPT_HEADER => true,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_SSL_VERIFYPEER => false,
CURLOPT_SSL_VERIFYHOST => false,
CURLOPT_USERAGENT => 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36',
];
$ch = curl_init();
curl_setopt_array($ch, $options);
$response = curl_exec($ch);
curl_close($ch);
This code shows me the Twitter page but says it was not found.
Sorry, that page doesn’t exist!
If I remove the User-Agent it shows me a page with the old version of Twitter but I need the new one what should I do?

PHP Header Location response as empty

I have an URL like this one.
I'm trying to get location from header, but its not showing location as normal.
Here is my function to get header info.
function headerInfo ($service_url) {
$handle = curl_init($service_url);
curl_setopt_array($handle, array(
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_SSL_VERIFYPEER => 0,
CURLOPT_NOBODY => 0,
CURLOPT_HEADER => 1
));
$curl_response = curl_exec($handle);
return $curl_response;
}
Response from PHP:
Response when trying from browser (chrome developer tool result):
Try setting a valid User-Agent.
Like so:
function headerInfo ($service_url) {
$handle = curl_init($service_url);
curl_setopt_array($handle, array(
CURLOPT_USERAGENT => 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36',
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_SSL_VERIFYPEER => 0,
CURLOPT_NOBODY => 0,
CURLOPT_HEADER => 1
));
$curl_response = curl_exec($handle);
return $curl_response;
}
You can find a list of valid User-Agents here:
Valid User-Agents

GET curl request only works in postman but not in php

I'm trying to get html as response but it only works in postman but not in php. In php I get different response like "Something went wrong..."
What I'm missing ?
Here is code:
$curl = curl_init();
curl_setopt_array($curl, array(
CURLOPT_URL => "https://www.avnet.com/shop/SearchDisplay?searchTerm=LMK316BJ476ML-T&countryId=apac&deflangId=-1&storeId=715839038&catalogId=10001&langId=-1&sType=SimpleSearch&resultCatEntryType=2&searchSource=Q&searchType=100&avnSearchType=all",
CURLOPT_RETURNTRANSFER => true,
CURLOPT_ENCODING => "",
CURLOPT_MAXREDIRS => 10,
CURLOPT_TIMEOUT => 30,
CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
CURLOPT_CUSTOMREQUEST => "GET",
CURLOPT_HTTPHEADER => array(
"Cache-Control: no-cache"
),
));
$response = curl_exec($curl);
$err = curl_error($curl);
curl_close($curl);
if ($err) {
echo "cURL Error #:" . $err;
} else {
echo $response;
}
postman response
php response
Thanks in advance.
I played around the CURLOPT_HTTPHEADER and figured out the site is validating 2 headers which are User-Agent and Accept-Language. I sent it correctly and it worked. Please find the headers below
CURLOPT_HTTPHEADER => array(
"User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36",
"Accept-Language:en-US,en;q=0.5"
)
Enjoyed figuring out the issue.
I had the same problem. This might solve the problem:
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
Last night, I spent more than 3 hours to solve the similar problem and at the end, I noticed it was related with server configuration. My sysadmin solved the problem by updating configuration, as I understand, some server configuration doesn't let you send CURL request same server.
Check below ticket:
https://talk.plesk.com/threads/curl-dont-resolved-correctly-at-same-server.295173/
curl_setopt($ch, CURLOPT_HTTPHEADER, ["Accept: application/json","User-Agent: WooCommerce API Client-PHP/3.1.0"]);
After spending lots of time, above Header works for me.
remove this
CURLOPT_ENCODING => "",

curl request works from one server and give 403 on another

I am using curl to hit a url and get contents from it and below is the code
$url = 'http://uber.com';
$ch = curl_init();
curl_setopt_array($ch, array(
CURLOPT_URL => $url,
CURLOPT_HEADER => false,
CURLOPT_FOLLOWLOCATION => true,
CURLOPT_SSL_VERIFYPEER => false,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_USERAGENT => 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.16 (KHTML, like Gecko) \ Chrome/24.0.1304.0 Safari/537.16',
CURLOPT_CONNECTTIMEOUT => 180,
CURLOPT_TIMEOUT => 180 + 60,
CURLOPT_COOKIE => 'lang=en;'
));
$response = curl_exec($ch);
var_dump($response);
It gives me the content when I execute the script on local server but while executing this on staging and production server, situated in US, it give 403-forbidden error. I also tested the same from another server in US and it works fine.
Any help is appreciated.

curl_setopt_array throwing warning message

I am trying to login into a website through cURL and have it return to me the actual page. So far I can only get the footer of the page with the feedback link and this error:
Warning: curl_setopt_array() [function.curl-setopt-array]: Array keys
must be CURLOPT constants or equivalent integer values on line 18
Loading
line 18 is the ");" at the end of the curl array
So far I've been able to figure out that if I just put in the website name with the access_token in my url from my browser I'll automatically be logged in so I'm trying to get cURL to emulate that.
<?php
$ckfile =' __utma=173730677.1410450142.1370837396.1370843059.4; __utmz=173730677.1370843059.4.3.utmcsr=web.com|utmccn=(referral)|utmcmd=referral|utmcct=/ou; __utma=64278953.892306882.1370766510.1370838026.4; __utmz=64278953.4.3.utmcsr=.com|utmccn=(referral)|utmcmd=referral|utmcct=/signin; __utmc=64278953; __utmc=173730677; __utmb=173730677.5.10.1370843059; __utmb=64278953.1.10';
$ch = curl_init();
curl_setopt_array(
$ch,
array(
CURLOPT_SSL_VERIFYPEER => false,
CURLOPT_URL => 'https://web.com',
CURLOPT_COOKIEFILE=> $ckfile,
CURLOPT_USERAGENT =>'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:21.0) Gecko/20100101 Firefox/21.0',
CURLOPT_GET=> true,
CURLOPT_REFERER => 'https://.com/signin',
CURLOPT_GETFIELDS=>'#access_token=',
CURLOPT_RETURNTRANSFER => true
)
);
$response = curl_exec($ch);
curl_close($ch);
echo '<pre>';
print_r($response);
?>
CURLOPT_GET and CURLOPT_GETFIELDS are not a valid options for cURL. The options may exist for POST, but they do not for GET. Just add the query string to the URL instead.
Also, make sure all the options are set to what cURL expects them to be set to. CURLOPT_COOKIEFILE should be set to a filename, not a string of cookie values. You want CURLOPT_COOKIE instead.
curl_setopt_array($ch, array(
CURLOPT_SSL_VERIFYPEER => false,
CURLOPT_URL => 'https://web.com?access_token=abc',
CURLOPT_COOKIE=> $ckfile,
CURLOPT_USERAGENT =>'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:21.0) Gecko/20100101 Firefox/21.0',
CURLOPT_REFERER => 'https://web.com/signin',
CURLOPT_RETURNTRANSFER => true
));
Please see the PHP cURL docs for all the options: http://php.net/manual/en/function.curl-setopt.php

Categories