i am trying to get the content of this json: http://steamcommunity.com/market/pricehistory/?country=DE¤cy=3&appid=730&market_hash_name=Chroma%20Case
This is my code:
$url = "http://steamcommunity.com/market/pricehistory/?country=DE¤cy=3&appid=730&market_hash_name=Chroma%20Case";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_COOKIE, 'steamLogin = 76561198075419487%7C%7C3F1A776553C4BE1D0F6DA83059052E79DB7EB3C7');
$output = curl_exec($ch);
$info = curl_getinfo($ch);
curl_close($ch);
$json_string = json_encode($output, JSON_PRETTY_PRINT);
When printing out $json_string it results in nothing, $output results in "‹ŠŽÿÿ)»L". I would like to grab the actual content on the website, the steamLogin-Cookie is needed for that. The cookie that's stored in my browser at the moment is the one I hardcoded in the source.
If you need any more info, feel free to ask.
Adding curl_setopt($ch, CURLOPT_ENCODING,""); made it :)
Related
I have a task: get by inputed keyword Wikipedia article, save it to database and then make a search inside them.
The problem is: how to access api and retrieve data from wikipedia, I've tried this url (at the begining i've tried json format):
$url = 'https://en.wikipedia.org/w/api.php?action=query&titles=Dog&prop=revisions&rvprop=content&format=xml';
and this php code:
$ch=curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
$res = curl_exec($ch);
if (!$res) {
echo 'cURL Error: '.curl_error($ch);
}
var_dump($res);
but nothing happend. Is it possible to access data with curl?
At the end one code worked with url above:
ini_set('user_agent','TestText');
$xmlDoc = new \DOMDocument();
$xmlDoc->load($url);
echo($xmlDoc->saveXML());
and then I get the text like this
{{about|the domestic dog|related species known as "dogs"|Canidae|other
uses|Dog (disambiguation)|}} {{Redirect|Doggie|the Danish
artist|Doggie (artist)}} {{pp-semi-indef}} {{pp-move-indef}} {{Taxobox
| name = Domestic dog | fossil_range = {{Fossil
range|0.033|0}}[[Pleistocene]] – [[Recent]] |
How can I handle it to be prettier (text with paragraphes or at liest plain text)?
So, There are two questions:
1. Is it possible to access wiki data with php curl and how I should improve my code?
2. How do I make wiki xml code prettier?
My question about code, especially about curl. Why it doesn't work?
And also, answer to another question says only about wikipedia api urls. By only changing url I can't solve problem.
I've found the solution, CURLOPT_SSL_VERIFYPEER was needed:
$url = 'http://en.wikipedia.org/w/api.php?action=query&prop=extracts&format=json&explaintext=&titles=Dog';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
$res = curl_exec($ch);
//$json_data = mb_substr($res, curl_getinfo($ch, CURLINFO_HEADER_SIZE));
curl_close($ch);
$json = json_decode($res);
$content = $json->query->pages;
$wiki_id = '';
foreach ($content as $key => $value) {
$wiki_id = $key;
}
echo $content = $content->$wiki_id->extract;
I'm doing a simple curl on this address: https://github.com/users/davidhariri/contributions_calendar_data
When i grab the result with this function:
function fetch_data($url){
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_TIMEOUT, 20);
$result = curl_exec($ch);
curl_close($ch);
print_r($result);
return $result;
}
The strings are correct, but the ints (the contributions) are wrong.
Results from curl
[...["2014/01/04",0],["2014/01/05",0],["2014/01/06",0],["2014/01/07",1],["2014/01/08",0]]
Results from just navigating to the address
[...["2014/01/04",0],["2014/01/05",0],["2014/01/06",1],["2014/01/07",5],["2014/01/08",5]]
Something during the curl process might be transforming ints to binary and back again? I have no idea what's happening here.
Check that you are not logged in in the browser. You may get different results if so.
I have 2 sites, one main, one external. On the main site, I am using Lucene to search through it. The problem is, I am trying to also search through the external site.
The Form action for the external site:
<form action="https://secure.bcchf.ca/SuperheroPages/searchResults.cfm?Event=WOT" method="post" name="search_tribute" >
I've tried to use curl, but it only brings up the search form without actually doing the search (the field is empty as well).
<?php
$ch = curl_init("https://secure.bcchf.ca/SuperheroPages/searchResults.cfm?Event=WOT");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, tname='hello');
$output = curl_exec($ch);
echo $output;
curl_close($ch);
?>
Any tips?
I don't have access to the form action since it's on an external site. All i have is a form that links to it when I submit it.
<?php
$ch = curl_init("https://secure.bcchf.ca/SuperheroPages/searchResults.cfm?Event=WOT");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, array("teamName" => "hello", "searchType" => "team"));
$output = curl_exec($ch);
echo $output;
curl_close($ch);
?>
Can you try this?
I'm pretty sure it's supposed to be teamName instead of tName
Most search engine use GET and not POST .. you can try
// asumption
$_POST['search'] = "hello";
// Return goole Search Result
echo curlGoogle($_POST['search']);
function curlGoogle($keyword) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://www.google.com/search?hl=en&q=' . urlencode($keyword) . '&btnG=Google+Search&meta=');
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FILETIME, true);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
Or if you want post then
curl_setopt($ch, CURLOPT_POSTFIELDS, array("search"=>"hello"));
Your php code is not valid syntax, it does not compile.
So if this is really what you have, your problem is that your file generates a fatal error.
That being said, this question is hard to answer since we don't know the site you want to grab your search results from.
Try modifying your line like this:
curl_setopt($ch, CURLOPT_POSTFIELDS, "search=hello");
or alternatively
curl_setopt($ch, CURLOPT_POSTFIELDS, array("search" => "hello");
Maby it will work, however it may be that more post data is required or that the element name is not correct.
You have to look at the form or try making a request and look at it with chromes developer tools or firebug.
Also there are a number of ways for external sites to prevent what you are doing, altough evertything can be worked around somehow.
Assuming that is not the case, I hope i could help you.
Try just putting it into an array.
as that will be the variable the $_POST checks on the other side
and just checked your link, its teamName for the field
$fields = array("teamName"=>"julia");
Then..
curl_setopt($ch, CURLOPT_POSTFIELDS, $fields);
So your complete code is...
<?php
$ch = curl_init("https://secure.bcchf.ca/SuperheroPages/searchResults.cfm?Event=WOT");
$fields = array("teamName"=>"julia");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $fields);
$output = curl_exec($ch);
var_dump($output);
curl_close($ch);
?>
I have a Affiliate URL Like http://track.abc.com/?affid=1234
open this link will go to http://www.abc.com
now i want to execute the http://track.abc.com/?affid=1234 Using CURL
and now how i can Get http://www.abc.com
with Curl ?
If you want cURL to follow redirect headers from the responses it receives, you need to set that option with:
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
You may also want to limit the number of redirects it follows using:
curl_setopt($ch, CURLOPT_MAXREDIRS, 3);
So you'd using something similar to this:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://track.abc.com/?affid=1234");
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_MAXREDIRS, 3);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$data = curl_exec($ch);
Edit: Question wasn't exactly clear but from the comment below, if you want to get the redirect location, you need to get the headers from cURL and parse them for the Location header:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://track.abc.com/?affid=1234");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_HEADER, true);
$data = curl_exec($ch);
This will give you the headers returned by the server in $data, simply parse through them to get the location header and you'll get your result. This question shows you how to do that.
I wrote a function that will extract any header from a cURL header response.
function getHeader($headerString, $key) {
preg_match('#\s\b' . $key . '\b:\s.*\s#', $headerString, $header);
return substr($header[0], strlen($key) + 3, -2);
}
In this case, you're looking for the value of the header Location. I tested the function by retrieving headers from a TinyURL, that redirects to http://google.se, using cURL.
$url = "http://tinyurl.com/dtrkv";
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$data = curl_exec($ch);
curl_close($ch);
$location = getHeader($data, 'Location');
var_dump($location);
Output from the var_dump.
string(16) "http://google.se"
How do i resolve the urls like the one below:
http://www.google.co.in/url?sa=t&source=newssearch&cd=1&ved=0CC4QqQIwAA&url=http%3A%2F%2Fwww.usatoday.com%2Fnews%2Fworld%2Fstory%2F2011-09-18%2Findia-earthquake-fatalities%2F50456078%2F1&ei=JkF2TriYPImGrAeHxdCFDQ&usg=AFQjCNEshh4QAZQlM_tVPoT_l7rJ0ag21Q
to it's final url
http://www.usatoday.com/news/world/story/2011-09-18/india-earthquake-fatalities/50456078/1
I've tried curl but it's resolving it to http://www.google.co.in/http
http://sandbox.phpcode.eu/g/fc7c1/1
$ch = curl_init('http://www.google.co.in/url?sa=t&source=newssearch&cd=1&ved=0CC4QqQIwAA&url=http%3A%2F%2Fwww.usatoday.com%2Fnews%2Fworld%2Fstory%2F2011-09-18%2Findia-earthquake-fatalities%2F50456078%2F1&ei=JkF2TriYPImGrAeHxdCFDQ&usg=AFQjCNEshh4QAZQlM_tVPoT_l7rJ0ag21Q');
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_NOBODY, true);
$response = curl_exec($ch);
$info = curl_getinfo($ch);
echo $info['url'];
All you are after is the value of the url parameter. You can preg_split the initial url by /&\?/, then take the element starting with url=, finally split it by = sign and use urldecode on the final value.