Using curl to bring search results from external site - php

I have 2 sites, one main, one external. On the main site, I am using Lucene to search through it. The problem is, I am trying to also search through the external site.
The Form action for the external site:
<form action="https://secure.bcchf.ca/SuperheroPages/searchResults.cfm?Event=WOT" method="post" name="search_tribute" >
I've tried to use curl, but it only brings up the search form without actually doing the search (the field is empty as well).
<?php
$ch = curl_init("https://secure.bcchf.ca/SuperheroPages/searchResults.cfm?Event=WOT");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, tname='hello');
$output = curl_exec($ch);
echo $output;
curl_close($ch);
?>
Any tips?
I don't have access to the form action since it's on an external site. All i have is a form that links to it when I submit it.

<?php
$ch = curl_init("https://secure.bcchf.ca/SuperheroPages/searchResults.cfm?Event=WOT");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, array("teamName" => "hello", "searchType" => "team"));
$output = curl_exec($ch);
echo $output;
curl_close($ch);
?>
Can you try this?
I'm pretty sure it's supposed to be teamName instead of tName

Most search engine use GET and not POST .. you can try
// asumption
$_POST['search'] = "hello";
// Return goole Search Result
echo curlGoogle($_POST['search']);
function curlGoogle($keyword) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://www.google.com/search?hl=en&q=' . urlencode($keyword) . '&btnG=Google+Search&meta=');
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FILETIME, true);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
Or if you want post then
curl_setopt($ch, CURLOPT_POSTFIELDS, array("search"=>"hello"));

Your php code is not valid syntax, it does not compile.
So if this is really what you have, your problem is that your file generates a fatal error.
That being said, this question is hard to answer since we don't know the site you want to grab your search results from.
Try modifying your line like this:
curl_setopt($ch, CURLOPT_POSTFIELDS, "search=hello");
or alternatively
curl_setopt($ch, CURLOPT_POSTFIELDS, array("search" => "hello");
Maby it will work, however it may be that more post data is required or that the element name is not correct.
You have to look at the form or try making a request and look at it with chromes developer tools or firebug.
Also there are a number of ways for external sites to prevent what you are doing, altough evertything can be worked around somehow.
Assuming that is not the case, I hope i could help you.

Try just putting it into an array.
as that will be the variable the $_POST checks on the other side
and just checked your link, its teamName for the field
$fields = array("teamName"=>"julia");
Then..
curl_setopt($ch, CURLOPT_POSTFIELDS, $fields);
So your complete code is...
<?php
$ch = curl_init("https://secure.bcchf.ca/SuperheroPages/searchResults.cfm?Event=WOT");
$fields = array("teamName"=>"julia");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $fields);
$output = curl_exec($ch);
var_dump($output);
curl_close($ch);
?>

Related

PHP Curl GET request doesn't return data

This really has me perplexed and other examples on Stackoverflow have not helped.
If I type into the browser :
https://api.bitfinex.com/v2/book/tXMRUSD/P0
I will get an long array of data which is correct.
With the following simple code :
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"https://api.bitfinex.com/v2/book/tXMRUSD/P0");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HTTPGET, 1);
$result = curl_exec($ch);
curl_close($ch);
var_dump($result);
I get :
bool(false)
The only difference is my browser fetches the data directly, while the PHP code is on a local website hosted using IIS on windows 10.
I've tried everything and just can't see where I'm going wrong. Any help will be most appreciated. (Ironically, all the authenticated and encrypted POST code works fine and I'm stuck on the easy stuff!)
Try This code it will work!!!!
<?php
$url= "https://api.bitfinex.com/v2/book/tXMRUSD/P0";
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
$result = curl_exec($ch);
curl_close($ch);
print_r($result);
?>

How to make a call to .aspx https from php script from my localhost with xamp?

I am trying to send SMS from my localhost with xamp installed.
Requested page is on https and an .aspx page.
I am getting error: "HTTP Error 400. The request is badly formed." or blank page only in some cases.
Detaisl is as follows :
$url = 'https://www.ismartsms.net/iBulkSMS/HttpWS/SMSDynamicAPI.aspx';
$postArgs = 'UserId='.$username.
'&Password='.$password.
'&MobileNo='.$destination.
'&Message='.$text.
'&PushDateTime='.$PushDateTime.
'&Lang='.$Lang;
function getSslPage($url) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_REFERER, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$result = curl_exec($ch);
curl_close($ch);
return $result;
}
$response = getSslPage($all);
echo "<pre>";
print_r($response); exit;
I tried every possible solution/combination found on internet but could not resolve that. The API developers do not have a example for php script.
I tried httpful php library and file_get_contents function but getting empty page. Also tried every combination with curl_setup.
I need to call this url without any post data and see the response from it.
Instead getting a blank page.
Please note that when I execute the url with all details in browser it works fine.
Can anybody help me in this regard.
Thank you,
Usman
First do urlencode over your data as follows:
$postArgs = 'UserId='. urlencode($username.
'&Password='.urlencode($password).
'&MobileNo='.urlencode($destination).
'&Message='.urlencode($text).
'&PushDateTime='.urlencode($PushDateTime).
'&Lang='.urlencode($Lang);
After that two possible solutions. One is using GET.
curl_setopt($ch, CURLOPT_URL, $url . "?" . $postArgs);
Second option is using POST method.
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postArgs);

cURL stristr errors

I'm trying to figure out why this won't work for me. I'm a complete noob when it comes to cURL, today is my first day using it. I followed a tutorial for this but obviously failed.
It should check the page and if it sees "Skill Stats" on there, then return "Success", and return "Failure" if it spots "Member Rankings".
<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://services.runescape.com/m=hiscore/compare.ws?user1=Mercon185");
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_HTTPGET, TRUE);
curl_setopt($ch, CURLOPT_POST, FALSE);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$output = curl_exec($ch);
if (stristr($output,"Skill Stats")) {
echo 'Success';
}
if (stristr($output,"Member Rankings")) {
echo 'Failure';
}
curl_close($ch);
?>
`
You need to enable follow redirects. As I see currently, your URL redirects to http://services.runescape.com/m=hiscore/overall.ws?errorcode=1. Without follow redirects enabled, it only fetches the first page, which indeed is empty.
The final landing page though, contains the data you want, so if you add this line to your cURL options, it should work:
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);

How to get the URL of a download link

I am trying to parse a page which contains some links. These links, if followed, will redirect to some files to download.
For example, Download which redirects to <a href="http://example.com/1.pdf".
I don't want to download the file, I just want to get the file link (int this case http://example.com/1.pdf).
I am trying this:
curl_setopt($ch, CURLOPT_RETURNTRANSFER, FALSE); // Return in string
curl_setopt($ch, CURLOPT_URL, $url);
curl_exec($ch);
var_dump(curl_getinfo($ch));
But, it gives me the file contents.
Does anyone have any idea how to this?
==EDIT==
Thank you guys. I solved it like this:
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLINFO_HEADER_OUT, TRUE);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, TRUE);
curl_setopt($ch, CURLOPT_NOBODY, TRUE);
curl_exec($ch);
$info = curl_getinfo($ch);
Now, $info contains the header and I can the link from it.
The reason the output is being sent to the screen is because you're telling cURL to do so. If you want to store the response in a variable the following line:
curl_setopt($ch, CURLOPT_RETURNTRANSFER, FALSE);
should read:
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
Then, actually retrieve the returned output from curl_exec like so:
$output = curl_exec($ch);
Once you have the returned HTML content from the remote page in the $output variable you can use DOMdocs or regex (but preferably DOM) to parse out any information you want.
UPDATE
I can't tell because the question is vaguely worded: is there actually a Location header redirect happening? If so, you'll want to do as #heiko suggests to prevent cURL from following the redirect and retrieve the headers. Then you can easily parse the contents of the location header:
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, FALSE);
curl_setopt($ch, CURLINFO_HEADER, TRUE); // add header output
# make sure to not follow Location: Header
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, FALSE);
# add Response Header to Output, so that you can find the Location-Header in there!
curl_setopt($ch, CURLINFO_HEADER_OUT, TRUE);
Use RETURN TRANSFER as 1, also use htmlentities() if you want to display HTML source on your page , else just echo the variable ( to display the page [redirects to google] ).
<?php
$url = "http://www.google.co.in";
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); // Return in string
curl_setopt($ch, CURLOPT_URL, $url);
$varx = curl_exec($ch);
echo htmlentities($varx);
?>
With the $varx variable , use Regular Expressions to match which data you want.

getting results from a site using curl

I have been trying to post some variables to a site using POST method, using curl to get some results. I am posting to this link.
http://www.rasta.pk/Lhr/Lhr_Traffic.aspx
At this page you will see a drop down menu .. onchange some values are returned in "Yellow" colored table.
I have monitored this site and trying to get those results by making a post request to that link. But, I am getting "Bad Header" error. I have tried changing things but ubable to find a solution.
Here is my code:
"Canal Bank Rd",
"ScriptManager1 " => "UpdatePanelDDLRoads|DropDownRoads",
"__EVENTARGUMENT" => "",
"__EVENTTARGET" => "DropDownRoads"
);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headerz);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_data);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($ch, CURLOPT_REFERER, "http://www.rasta.pk/Lhr/Lhr_Traffic.aspx");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$output = curl_exec($ch);
curl_close($ch);
echo $output;
?>
There's too little details presented for us to know for sure.
A guess is that you're doing the wrong kind of post, since when you pass in a hash array to CURLOPT_POSTFIELDS it will do a multipart formpost which might not be what the server expects. Pass in a string instead to make a "normal" POST.
If that is not enough, use LiveHTTPHeaders or similar in a browser to figure out exactly what is sent in a "manual" session and then you make sure that your curl program mimics that operation as closely as possible.

Categories