I have a task: get by inputed keyword Wikipedia article, save it to database and then make a search inside them.
The problem is: how to access api and retrieve data from wikipedia, I've tried this url (at the begining i've tried json format):
$url = 'https://en.wikipedia.org/w/api.php?action=query&titles=Dog&prop=revisions&rvprop=content&format=xml';
and this php code:
$ch=curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
$res = curl_exec($ch);
if (!$res) {
echo 'cURL Error: '.curl_error($ch);
}
var_dump($res);
but nothing happend. Is it possible to access data with curl?
At the end one code worked with url above:
ini_set('user_agent','TestText');
$xmlDoc = new \DOMDocument();
$xmlDoc->load($url);
echo($xmlDoc->saveXML());
and then I get the text like this
{{about|the domestic dog|related species known as "dogs"|Canidae|other
uses|Dog (disambiguation)|}} {{Redirect|Doggie|the Danish
artist|Doggie (artist)}} {{pp-semi-indef}} {{pp-move-indef}} {{Taxobox
| name = Domestic dog | fossil_range = {{Fossil
range|0.033|0}}[[Pleistocene]] – [[Recent]] |
How can I handle it to be prettier (text with paragraphes or at liest plain text)?
So, There are two questions:
1. Is it possible to access wiki data with php curl and how I should improve my code?
2. How do I make wiki xml code prettier?
My question about code, especially about curl. Why it doesn't work?
And also, answer to another question says only about wikipedia api urls. By only changing url I can't solve problem.
I've found the solution, CURLOPT_SSL_VERIFYPEER was needed:
$url = 'http://en.wikipedia.org/w/api.php?action=query&prop=extracts&format=json&explaintext=&titles=Dog';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
$res = curl_exec($ch);
//$json_data = mb_substr($res, curl_getinfo($ch, CURLINFO_HEADER_SIZE));
curl_close($ch);
$json = json_decode($res);
$content = $json->query->pages;
$wiki_id = '';
foreach ($content as $key => $value) {
$wiki_id = $key;
}
echo $content = $content->$wiki_id->extract;
Related
i am trying to get the content of this json: http://steamcommunity.com/market/pricehistory/?country=DE¤cy=3&appid=730&market_hash_name=Chroma%20Case
This is my code:
$url = "http://steamcommunity.com/market/pricehistory/?country=DE¤cy=3&appid=730&market_hash_name=Chroma%20Case";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_COOKIE, 'steamLogin = 76561198075419487%7C%7C3F1A776553C4BE1D0F6DA83059052E79DB7EB3C7');
$output = curl_exec($ch);
$info = curl_getinfo($ch);
curl_close($ch);
$json_string = json_encode($output, JSON_PRETTY_PRINT);
When printing out $json_string it results in nothing, $output results in "‹ŠŽÿÿ)»L". I would like to grab the actual content on the website, the steamLogin-Cookie is needed for that. The cookie that's stored in my browser at the moment is the one I hardcoded in the source.
If you need any more info, feel free to ask.
Adding curl_setopt($ch, CURLOPT_ENCODING,""); made it :)
I need a fresh EURUSD quote to show my product prices in a website.
I'm trying to read the quote from Yahoo finance with the following code:
$eurUsd = 1.3700; // Default value
$yahooUrl = 'http://finance.yahoo.com/d/quotes.csv?f=l1&s=EURUSD=X';
$handle = fopen($yahooUrl, 'r');
if ($handle) {
$result = fgetcsv($handle);
fclose($handle);
$eurUsd = $result[0];
}
This code works fine in my testing environment but not in my website since I have "allow_url_fopen=0". I do not want to change it for a security reason.
The request I'm using returns a string like:
1.3715
Do you know how I can read the string in other way?
Do you know an alternative way to access the EURUSD quote?
EDIT
Thanks to Epik's code I tried to use curl, but without success with Yahoo.
Finally I rid off Yahoo and succeed with Rate Exchange JSON/JSONP APIs
A working solution:
$url = 'http://rate-exchange.appspot.com/currency?from=EUR&to=USD';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$result = curl_exec($ch);
curl_close($ch);
$eurUsd = json_decode($result)->{'rate'};
$ch = curl_init();
$url = 'http://finance.yahoo.com/d/quotes.csv?f=l1&s=EURUSD=X';
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch,CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$result = curl_exec($ch);
I have 2 sites, one main, one external. On the main site, I am using Lucene to search through it. The problem is, I am trying to also search through the external site.
The Form action for the external site:
<form action="https://secure.bcchf.ca/SuperheroPages/searchResults.cfm?Event=WOT" method="post" name="search_tribute" >
I've tried to use curl, but it only brings up the search form without actually doing the search (the field is empty as well).
<?php
$ch = curl_init("https://secure.bcchf.ca/SuperheroPages/searchResults.cfm?Event=WOT");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, tname='hello');
$output = curl_exec($ch);
echo $output;
curl_close($ch);
?>
Any tips?
I don't have access to the form action since it's on an external site. All i have is a form that links to it when I submit it.
<?php
$ch = curl_init("https://secure.bcchf.ca/SuperheroPages/searchResults.cfm?Event=WOT");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, array("teamName" => "hello", "searchType" => "team"));
$output = curl_exec($ch);
echo $output;
curl_close($ch);
?>
Can you try this?
I'm pretty sure it's supposed to be teamName instead of tName
Most search engine use GET and not POST .. you can try
// asumption
$_POST['search'] = "hello";
// Return goole Search Result
echo curlGoogle($_POST['search']);
function curlGoogle($keyword) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://www.google.com/search?hl=en&q=' . urlencode($keyword) . '&btnG=Google+Search&meta=');
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FILETIME, true);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
Or if you want post then
curl_setopt($ch, CURLOPT_POSTFIELDS, array("search"=>"hello"));
Your php code is not valid syntax, it does not compile.
So if this is really what you have, your problem is that your file generates a fatal error.
That being said, this question is hard to answer since we don't know the site you want to grab your search results from.
Try modifying your line like this:
curl_setopt($ch, CURLOPT_POSTFIELDS, "search=hello");
or alternatively
curl_setopt($ch, CURLOPT_POSTFIELDS, array("search" => "hello");
Maby it will work, however it may be that more post data is required or that the element name is not correct.
You have to look at the form or try making a request and look at it with chromes developer tools or firebug.
Also there are a number of ways for external sites to prevent what you are doing, altough evertything can be worked around somehow.
Assuming that is not the case, I hope i could help you.
Try just putting it into an array.
as that will be the variable the $_POST checks on the other side
and just checked your link, its teamName for the field
$fields = array("teamName"=>"julia");
Then..
curl_setopt($ch, CURLOPT_POSTFIELDS, $fields);
So your complete code is...
<?php
$ch = curl_init("https://secure.bcchf.ca/SuperheroPages/searchResults.cfm?Event=WOT");
$fields = array("teamName"=>"julia");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $fields);
$output = curl_exec($ch);
var_dump($output);
curl_close($ch);
?>
Got a php script using cURL grabbing the contents of a url that has colons in the source name:
$url = 'http://www.awebsite.com/anxml:file:thatoddly:hascolons:allovertheplace:';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 5);
$data = curl_exec($ch);
if(curl_errno($ch)) {
echo 'Curl error: ' . curl_error($ch);
}
curl_close($ch);
I am getting the error.
Could not resolve host: http; nodename nor servname provided, or not known <url here>
I've double checked that the url is working fine otherwise, but I suspect cURL is choking on the colons in the filename. The source isn't mine, so I can't remove the colons.
Is there another way around this?
Provider fixed their files, so I don't have to deal with colons any longer. Turns out I was using cURL improperly after all and likely the urlencode() with the code below would have worked.
This DIDN'T WORK:
$url = urlencode($url);
$url = str_replace("http%3A","http:",$url);
$ch = curl_init();
$timeout = 5;
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
$data = curl_exec($ch);
This DID WORK:
$url = urlencode($url);
$url = str_replace("http%3A","http:",$url);
$c = curl_init($url);
curl_setopt($c, CURLOPT_RETURNTRANSFER, true);
$data = iconv("UTF-8","ISO-8859-1",curl_exec($c));
Hope that helps someone out.
Looking at the man page, cURL has a --data-urlencode flag.
If it's just one URL not being done via CLI but PHP, you could use PHP's urlencode().
I tried to use JSON decode to get the youtube API feed. However, when I paste the JSON result in http://www.jsonlint.com/ I noticed something like
"media$group": {
"media$category": [
Unfortunately some symbols are rejected by php. Here is my code, I tried to remove this $ symbol, but maybe not success. How do I solve this?
$url = 'http://gdata.youtube.com/feeds/api/videosq=football&orderby=published&v=2&alt=json';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, $url);
$body1 = curl_exec($ch);
$body = str_replace('$','', $body1);
curl_close($ch);
$data = json_decode($body);
foreach ($data->feed->entry as $result) {
...
}
Your problem is the usage of PHP identifiers to access the contents. The simplest solution here would be to get an array instead of an object:
$data = json_decode ( $json , $assoc = true );
This allows access to fields with:
echo $result['media$group']['media$description'];
If you want to keep the object syntax, that's possible with this kludge:
echo $result->{'media$group'}->{'media$category'};
(But arrays are safer here. You don't get a fatal error should the format change and properties be absent.)
This work:
<?php
$url = 'http://gdata.youtube.com/feeds/api/videos?q=football&orderby=published&v=2&alt=json';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, $url);
$body1 = curl_exec($ch);
$body = str_replace('$','', $body1);
curl_close($ch);
$data = json_decode($body);
foreach ($data->feed->entry as $result) {
var_dump($result);
}
?>