PHP Curl to get page contenct - php

Im trying to fetch content of a website about a tourament. I want to display the results on a temporary page.
I'm tying to fetch this page:
http://www.tournamentsoftware.com/sport/draw.aspx?id=600CA297-99CA-4420-AE1A-698BA10C39B0&draw=1
I want to return the content of this page, and afterwards fetch the specific table with the fixtures.
The script i'm using returns a 404 Not found error, while the url is present.
My script:
function nxs_cURLTest($url, $msg, $testText){
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSLVERSION,3); // Apparently 2 or 3
$response = curl_exec($ch);
$errmsg = curl_error($ch);
$cInfo = curl_getinfo($ch);
curl_close($ch);
echo "Testing ... ".$url." - ".$cInfo['url']."<br />";
if (stripos($response, $testText)!==false)
echo "....".$msg." - OK<br />";
else
{
echo "....<b style='color:red;'>".$msg." - Problem</b><br /><pre>";
print_r($errmsg);
print_r($cInfo);
print_r(htmlentities($response));
echo "</pre>There is a problem with cURL. You need to contact your server admin or hosting provider.";
}
}
nxs_cURLTest("http://www.tournamentsoftware.com/sport/draw.aspx?id=600CA297-99CA-4420-AE1A-698BA10C39B0&draw=1", "HTTPS to Toernooi.nl", 'link rel="canonical" href="http://www.tournamentsoftware.com/sport/draw.aspx?id=600CA297-99CA-4420-AE1A-698BA10C39B0&draw=1"');
Can anyone help me on this one?

I used the following on xampp win7 with chrome and it returned data
$url = 'http://www.tournamentsoftware.com/sport/draw.aspx?id=600CA297-99CA-4420-AE1A-698BA10C39B0&draw=1';
echo curl_scrap($url);
function curl_scrap($url) {
$ch = curl_init();
$timeout = 5;
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
It did not return the same source as just using chrome to hit url, as function doesn't set anything like user agent field etc, but returned the data you want, just a quick and dirty test.

Related

get page content through curl call

I'm trying to get content of a page through curl call. But I'm getting only an empty array. Here is my code
print_r(get_data('https://www.realestate.com.au/sold/in-9%2f20%3b/list-1'));
function get_data($url)
{
$ch = curl_init();
$timeout = 5;
// the url to fetch
curl_setopt($ch, CURLOPT_URL, $url);
// return result as a string rather than direct output
curl_setopt($ch, CURLOPT_USERAGENT,'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.52 Safari/537.17');
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
// set max time of cURL execution
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
I have also tried different method like file_get_contents function but always getting empty page.
given function is to complicated I think its a copy paste function, as I can see from
return $result; } which is not needed and you should get syntax error.
probably that code will work without that bracket.
I cleared and shortened code, and giving a working and (tested) solution.
NOTE: You can add the code I removed back
$url 'your url';
//Usaqe of function I recomend to use parse_url();
echo get_data($url);
function get_data($url) {
$ch = curl_init();
$timeout = 5;
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}

Problems loading Spotify URL using curl in PHP

When running on my localhost or my server I can’t load a particular external a page. However, if I load the page in my browser it loads or using Postman it loads fine.
How can I fix this and how is Spotify preventing this?
The URL of the content I want to load is this.
<?php
$url="https://embed.spotify.com/?uri=spotify:user:spotify:playlist:4hOKQuZbraPDIfaGbM3lKI";
$page = file_get_contents($url);
echo $page; //returns nothing
$page = get_data($url);
echo $page; //returns nothing with a http code of 0
$url="https://www.google.com";
$page = file_get_contents($url);
echo $page; //returns google
$page = get_data($url);
echo $page; //returns google with a http code of 200
/* gets the data from a URL */
function get_data($url) {
$ch = curl_init();
$timeout = 5;
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
curl_setopt($ch,CURLOPT_USERAGENT,'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13');
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
$data = curl_exec($ch);
echo curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);
return $data;
}
?>
Try setting CURLOPT_POSTFIELDS to true & set the URL parameters in CURLOPT_POSTFIELDS like this. Note the URL changes as a result since the parameters are now in CURLOPT_POSTFIELDS. I set the params as an array called $post_fields since I find that to be an easier way to read that way when debugging.
UPDATE: The post params didn’t work. But adding CURLOPT_SSL_VERIFYHOST set to false as well as CURLOPT_SSL_VERIFYPEER set to false seems to do the trick on my side.
Here is my cleaned up version of your code. Removed your tests & commented out the post param stuff I thought would help before:
// Set the `url`.
$url="https://embed.spotify.com/?uri=spotify:user:spotify:playlist:4hOKQuZbraPDIfaGbM3lKI";
// Set the `post` fields.
$post_fields = array();
// $post_fields['uri'] = 'spotify:user:spotify:playlist:4hOKQuZbraPDIfaGbM3lKI';
// Set the `post` fields.
$page = get_data($url, $post_fields);
// Echo the output.
echo $page;
// Gets the data from a `url`.
function get_data($url, $post_fields) {
$curl_timeout = 5;
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
// curl_setopt($ch, CURLOPT_POST, true);
// curl_setopt($ch, CURLOPT_POSTFIELDS, $post_fields);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $curl_timeout);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13');
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
$data = curl_exec($ch);
// echo curl_getinfo($ch, CURLINFO_EFFECTIVE_URL);
curl_close($ch);
return $data;
}

Trello API PHP Curl

I am trying to integrate/replace my web edit queue into Trello.
I made an organization which is not public but have created a token for read/write access.
I haven't seen a good PHP wrapper for the Trello API (have looked at the two available and couldn't really get them up and running for my purposes.)
Anyway, what I'd like to do is provide rather rudimentary access to read and insert cards to a particular list.
I have gotten as far as using the API to return the results of a List using the following:
https://api.trello.com/1/lists/[mylistID]/cards?key=[myappkey]&token=[mytoken]
I get exactly what I want as a result, json of the cards in the list.
Now I;m trying to recreate that in PHP using CURL and I'm getting an error response of unauthorized or bad request from whatever I try in the following code:
$url = "https://api.trello.com/1/lists/[myboardID]/cards";
$trello_key = 'mykey';
$trello_list_id = 'mylistid';
$trello_member_token = 'mytoken';
$fields = "key=$trello_key&token=$trello_member_token&name=$name&idList=$trello_list_id";
e
# init curl
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POSTFIELDS, $fields);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)");
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLINFO_HEADER_OUT, TRUE); // make sure we see the sended header afterwards
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_TIMEOUT, 0);
curl_setopt($ch, CURLOPT_POST, 1);
# dont care about ssl
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
# download and close
$output = curl_exec($ch);
$request = curl_getinfo($ch, CURLINFO_HEADER_OUT);
$error = curl_error($ch);
curl_close($ch);
So I'm just looking to see if anyone knows what I'm doing wrong. I feel like it should be simple but I've spent a couple hours on it and I think I need some help. Let me know if you have any ideas.
{i have left out obvious references to my API key, token, BoardID etc}
This actually seems to work for me. I was trying to use POST instead of the default GET with CURL. Still working on parsing the response but it seems like I'm on the right track. Got an "200 OK" in the response.
$url = 'https://api.trello.com/1/lists/[myListID]/cards?key=[MyApiKey]&token=[myToken]';
# init curl
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
//curl_setopt($ch, CURLOPT_POSTFIELDS, $encoded_fields);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)");
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLINFO_HEADER_OUT, TRUE); // make sure we see the sended header afterwards
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_TIMEOUT, 0);
//curl_setopt($ch, CURLOPT_POST, 1);
# dont care about ssl
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
# download and close
$output = curl_exec($ch);
$request = curl_getinfo($ch, CURLINFO_HEADER_OUT);
$error = curl_error($ch);
curl_close($ch);
echo 'This is output = '.$output .'<br />';
echo 'This is request = '.$request .'<br />';
echo 'This is error = '.$error .'<br />';
$url = 'https://api.trello.com/1/boards/[myListID]/cards?key=[MyApiKey]&token=[myToken]';
# init curl
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
//curl_setopt($ch, CURLOPT_POSTFIELDS, $encoded_fields);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)");
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLINFO_HEADER_OUT, TRUE); // make sure we see the sended header afterwards
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_TIMEOUT, 0);
//curl_setopt($ch, CURLOPT_POST, 1);
# dont care about ssl
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
# download and close
$output = curl_exec($ch);
$request = curl_getinfo($ch, CURLINFO_HEADER_OUT);
$error = curl_error($ch);
curl_close($ch);
echo 'This is output = '.$output .'<br />';
echo 'This is request = '.$request .'<br />';
echo 'This is error = '.$error .'<br />';

PHP Curl does not return a values from url

i am using above function to retrieve the two urls , second one works when i am retriving google or other websites, but below things is not getting any response. but when i enter the url in browser i am seeing the response.. can you guide me to fix this issue?
function file_get_contents_curl($url) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_AUTOREFERER, TRUE);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
$response=file_get_contents_curl('http://XX.XX.XX.XX:YYYYY/Request?Mob=999999999&Message=Rr+vodafone+999999999+10+1234&myTxId=10');
echo "<br/>respnse is...".$response;
$response=file_get_contents_curl('http://www.google.com');
echo "<br/>website respond is...".$response;
Edit:
Issue is with Port number i am trying to access 8090 is not enabled in my server. when i enabled it for outgoing it was working... Thanks every one for the support.
Can you try:
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 5.1; rv:12.0) Gecko/20100101 Firefox/12.0');
Try to find error give this code before curl close:--
echo "Curl Error :--" . curl_error($ch);
if no error found do like this:-
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$result = curl_exec($ch);
then
print_r($result);
exit;

Fetch the description from wikipedia from an article

I am trying to make a API call to wikipedia through: http://en.wikipedia.org/w/api.php?action=parse&page=Petunia&format=xml, but the xml is full with html and css tags.
Is there a way to fetch only plain text without tags? Thanks!
*Edit 1:
$json = json_decode(file_get_contents('http://en.wikipedia.org/w/api.php?action=parse&page=Petunia&format=json'));
$txt = strip_tags($json->text);
var_dump($json);
Null displayed.
Question was partially answered here
$url = 'http://en.wikipedia.org/w/api.php?action=parse&page=Petunia&format=json&prop=text';
$ch = curl_init($url);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($ch, CURLOPT_USERAGENT, "TestScript"); // required by wikipedia.org server
$c = curl_exec($ch);
$json = json_decode($c);
var_dump(strip_tags($json->{'parse'}->{'text'}->{'*'}))
I was not able to use file_get_contents but it works fine with cURL.
it is possible to fetch info or description from wikipedia by using xml.
$url = "http://en.wikipedia.org/w/api.php?action=opensearch&search=".$term."&format=xml&limit=1";
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_HTTPGET, TRUE);
curl_setopt($ch, CURLOPT_POST, FALSE);
curl_setopt($ch, CURLOPT_HEADER, false); // Include head as needed
curl_setopt($ch, CURLOPT_NOBODY, FALSE); // Return body
curl_setopt($ch, CURLOPT_VERBOSE, FALSE); // Minimize logs
curl_setopt($ch, CURLOPT_REFERER, ""); // Referer value
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE); // No certificate
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE); // Follow redirects
curl_setopt($ch, CURLOPT_MAXREDIRS, 4); // Limit redirections to four
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE); // Return in string
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 6.1; he; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8"); // Webbot name
$page = curl_exec($ch);
$xml = simplexml_load_string($page);
if((string)$xml->Section->Item->Description) {
print_r(array((string)$xml->Section->Item->Text,
(string)$xml->Section->Item->Description,
(string)$xml->Section->Item->Url));
} else {
echo "sorry";
}
But curl must be install on server... have a nice day...

Categories