I am trying to make a "echo" prices for different objects I have got in an array that contains weapons:
I have read lots about how to get prices through through Steammarket through Steamapi:
(none of these sources did fit my requirements)
Sources:
Get steam item prices
How get a response of multiple price items on the market
Get the price of an item on Steam Community Market with PHP and Regex
I found finally a code snippet that works flawless
FILE marketprices.php
<?php
$items = array("Exalted Manifold Paradox","Kinetic Gem","Mercurial's Call");
foreach($items as $item)
{
$json = json_decode(file_get_contents("http://steamcommunity.com/market/priceoverview/?appid=570&market_hash_name=".rawurlencode($item)), true);
if($json["success"] == true OR !empty($json))
{
echo $item."'s lowest price is ".$json["lowest_price"]."";
}
elseif($json["success"] == false OR empty($json))
{
echo "Could not get data for ".$item;
}
}
output>
Exalted Manifold Paradox's lowest price is $28.49Kinetic Gem's lowest price is $50.00Mercurial's Call's lowest price is $0.16
Source:http://gamebanana.com/tuts/11942
When I am trying to implent this snippet to my code I get error in my result:
I have created an array that contains different weapons:
<?
foreach($S_W as $item) // Steam weapon
{
echo $item;
}
?>
output>
AWP | Worm God (Factory New)
FAMAS | Cyanospatter (Field-Tested)
G3SG1 | Green Apple (Factory New)
G3SG1 | Polar Camo (Field-Tested)
Glock-18 | Death Rattle (Field-Tested)
M249 | Gator Mesh (Field-Tested)
MAC-10 | Heat (Field-Tested)
This is good so far..
I get error in the result
Here is my code below:
foreach($S_W as $item)
{
$json = json_decode(file_get_contents("http://steamcommunity.com/market/priceoverview/?appid=570&market_hash_name=".rawurlencode($item)), true);
if($json["success"] == true OR !empty($json))
{
echo $item."'s lowest price is ".$json["lowest_price"]."";
}
elseif($json["success"] == false OR empty($json))
{
echo "Could not get data for ".$item;
}
}
I receive following result:
I see a part of the error..
in the call in the end of the URL the code adds:</br>
Could someone help me with a solution?
Thank you in advance
Best regards
Daniel
Priceoverview 'API' endpoint is rate-limited by Steam, and was even further rate-limited by Valve very recently. You are getting 429 HTTP Status code, which stands for 'Too many requests', you are sending requests to their site way too often.
Related
Using PHP, I'm looking to get an id passed via the url and then lookup some data in a JSON file... then display the data back on the page.
I'll set up the url to be http://mytestapp.php?id=12345678 and then use;
$id = $_GET['id'];
to set the id variable. I then have a JSON as below;
{
"ads":
[
{ "id":"12345678",
"hirername":"Test Hirer",
"hirercontact":"Rob A",
"role":"Ultra Sat Role",
"requirements": [
{"req":"Right to work in Australia"},
{"req":"Live locally"}],
"candidates": [
{"name":"John Smith","dist":"Nunawading (23km away)","exp1":"Pizza maker at Don Domenicos for 4 years","exp2":"Bakery Assistant at Woolworths for 4 years","req":"","avail1":"Mon to Fri | Morning, Evening & Night","avail2":"","call":"0413451007"},
{"name":"Jack Smith","dist":"Endeadvour Hills (35km away)","exp1":"Pizzaiolo (Pizza maker) at Cuor Di Pizza for 1 year","exp2":"","req":"","avail1":"Mon to Fri | Morning & Evening","avail2":"","call":"041345690"}]
},
{ "id":"12345679",
"hirername":"Test Hirer 2",
"hirercontact":"Jesse S",
"role":"Ultra Sat Role 2",
"requirements": [
{"req":"Right to work in Australia"},
{"req":"Live locally"}],
"candidates": [
{"name":"Jill Smith","dist":"Nunawading (23km away)","exp1":"Pizza maker at Don Domenicos for 4 years","exp2":"Bakery Assistant at Woolworths for 4 years","req":"","avail1":"Mon to Fri | Morning, Evening & Night","avail2":"","call":"0413451007"},
{"name":"Jenny Smith","dist":"Endeadvour Hills (35km away)","exp1":"Pizzaiolo (Pizza maker) at Cuor Di Pizza for 1 year","exp2":"","req":"","avail1":"Mon to Fri | Morning & Evening","avail2":"","call":"041345690"}]
}
]
}
Which i want to search for the id, and then be able to echo the contents of the data out.
I'm reading the JSON and decoding into an array as such;
$json = file_get_contents('data.json');
$arr = json_decode($json, true);
But i'm not sure how to now read the array, find the data i want based on the id, and then pull out the data so i can display it on the page as follows;
Hirer: Test Hirer
Contact: Rob A
Role: Ultra Sat Role
Requirements:
- Right to work in Australia
- Live Locally
John Smith Nunawading (23km away)
Pizza maker at Don Domenicos for 4 years
Bakery Assistant at Woolworths for 4 years
Mon to Fri | Morning, Evening & Night
0413451007
Jack Smith Endeadvour Hills (35km away)
Pizzaiolo (Pizza maker) at Cuor Di Pizza for 1 year
Mon to Fri | Morning & Evening
041345690
Any ideas?
Thanks Rob.
Borrowed the example from #RobbieAverill and modified to suit your needs, please check if this works.
<?php
$id = $_GET['id'];
$json = file_get_contents('data.json');
$foundAd = null;
$json = json_decode($json,true);
foreach ($json['ads'] as $ad) {
if ($ad['id'] == $id) {
$foundAd = $ad;
break;
}
}
echo "Hirer:".$foundAd['hirername']."<br/>";
echo "contact:".$foundAd['hirercontact']."<br/>";
echo "role:".$foundAd['role']."<br/><br/>";
echo "Requirements<br/>";
echo "<ul>";
foreach($foundAd['requirements'] as $req){
echo "<li>".$req['req']."</li>";
}
echo "</ul><br/>";
foreach($foundAd['candidates'] as $req){
echo $req['name']." ". $req['dist']."</br>";
echo $req['exp1']."</br>";
echo $req['exp1']."</br>";
echo $req['avail1']."</br>";
if($req['avail2']!=""){
echo $req['avail2']."</br>";;
}
echo $req['call']."</br></br>";
}
?>
In your current inplementation you need to loop over all the ads objects like
foreach ($arr['ads'] as $ad){
if ($ad['id'] == $id){
//do stuff;
}
}
A better implementation would be to use the value of the id as the key of the json object when you store the json. Using something like
$ads[id] = $yourjsonobject;
Then referencing would just be $arr['ads'][id];.
You can then use multiple foreach or if your keys are knows just use the keys to output the object you need like
echo $ad["hirername"];
Using the foreach loop to print the complete object:
foreach( $ad as $value){
print_r($value);
}
I have integrated Yelp reviews into my directory site with each venue that has a Yelp ID returning the number of reviews and overall score.
Following a successful MySQL query for all venue details, I output the results of the database formatted for the user. The Yelp element is:
while ($searchresults = mysql_fetch_array($sql_result)) {
if ($yelpID = $searchresults['yelpID']) {
require('yelp.php');
if ( $numreviews > 0 ) {
$yelp = '<img src="'.$ratingimg.'" border="0" /> Read '.$numreviews.' reviews on <img src="graphics/yelp_logo_50x25.png" border="0" /><br />';
} else {
$yelp = '';
}
} //END if ($yelpID = $searchresults['yelpID']) {
} //END while ($searchresults = mysql_fetch_array($sql_result)) {
The yelp.php file returns:
$yrating = $result->rating;
$numreviews = $result->review_count;
$ratingimg = $result->rating_img_url;
$url = $result->url;
If a venue has a Yelp ID and one or more reviews then the output displays correctly, but if the venue has no Yelp ID or zero reviews then it displays the Yelp review number of the previous venue.
I've checked the $numreviews variable type and it's an integer.
So far I've tried multiple variations of the "if ( $numreviews > 0 )" statement such as testing it against >=1, !$numreviews etc., also converting the integer to a string and comparing it against other strings.
There are no errors and printing all of the variables returned gives the correct number of reviews for each property with venues having no ID or no reviews returning nothing (as opposed to zero). I've also compared it directly against $result->review_count with the same problem.
Is there a better way to make the comparison or better format of variable to work with to get the correct result?
EDIT:
The statement if ($yelpID = $searchresults['yelpID']) { is not operating as it should. It is identical to other statements in the file, validating row contents which work correctly for their given variable, e.g. $fbID = $searchresults['fbID'] etc.
When I changed require('yelp.php'); to require_once('yelp.php'); all of the venue outputs changed to showing only the first iterated result. Looking through the venues outputted, the error occurs on the first venue after a successful result which makes me think there is a pervasive piece of code in the yelp.php file, causing if ($yelpID = $searchresults['yelpID']) { to be ignored until a positive result is found (a yelpID in the db), i.e. each venue is correctly displayed with a yelp number of reviews until a blank venue is encountered. The preceding venues' number of reviews is then displayed and this continues for each blank venue until a venue is found with a yelpID when it shows the correct number again. The error reoccurs on the next venue output with no yelpID and so on.
Sample erroneous output: (line 1 is var_dump)
string(23) "bayview-hotel-bushmills"
Bayview Hotel
Read 3 reviews on yelp
Benedicts
Read 3 reviews on yelp (note no var_dump output, this link contains the url for the Bayview Hotel entry above)
string(31) "bushmills-inn-hotel-bushmills-2"
Bushmills Inn Hotel
Read 7 reviews on yelp
I suspect this would be a new question rather than clutter/confuse this one further?
END OF EDIT
Note: I'm aware of the need to upgrade to mysqli but I have thousands of lines of legacy code to update. For now I'm working on functionality before reviewing the code for best practice.
Since the yelp.php is sort of a blackbox; the best explanation for this behavior would be that it only set's those variables if it finds a match. Updating your code to this should fix that:
unset($yrating, $numreviews, $ratingimg, $url);
require('yelp.php');
I also noticed this peculiar if-statement, do you realize that's an assignment or is this a copy/paste error? If you want to test (that's what if is for)
if ($yelpID == $searchresults['yelpID']) {
I'm trying to audit a vast amount of company data from companycheck.co.uk my current script appears to be looping the first 10 results from only the first page. I had the script gather more than 10 results at one point, but this caused a fatal error after around 600 results (not a timeout error, but a connection error of some sort), I need the script to be more reliable as I'm fetching over 40,000 results.
My code so far:
<?php
set_time_limit(0);
ini_set('max_execution_time', 0);
require 'vendor/autoload.php';
require "Guzzle/guzzle.phar";
// Add this to allow your app to use Guzzle and the Cookie Plugin.
use Guzzle\Http\Client as GuzzleClient;
use Guzzle\Plugin\Cookie\Cookie;
use Guzzle\Plugin\Cookie\CookiePlugin;
use Guzzle\Plugin\Cookie\CookieJar\ArrayCookieJar;
use Guzzle\Plugin\Cookie\CookieJar\CookieJarInterface;
$Pagesurl = 'http://companycheck.co.uk/search/UpdateSearchCompany?searchTerm=cars&type=name';
$pagesData = json_decode(file_get_contents($Pagesurl), true);
$resultsFound = $pagesData["hits"]["found"];
$pages = ceil($resultsFound / 10);
//echo $pages;
echo "<br>";
for ($p = 0; $p < $pages; $p++) {
$url = 'http://companycheck.co.uk/search/UpdateSearchCompany?searchTerm=cars&type=name&companyPage=' . $p . '';
$data = json_decode(file_get_contents($url), true);
for ($i = 0; $i < 11; $i++) {
$id = $data["hits"]["hit"][$i]["id"];
$TradingAddress = $data["hits"]["hit"][$i]["data"]["address"][0];
$companyName = $data["hits"]["hit"][$i]["data"]["companyname"][0];
$companyNumber = $data["hits"]["hit"][$i]["data"]["companynumber"][0];
$finalURL = "http://companycheck.co.uk/company/" . $id . "";
$httpClient = new GuzzleClient($finalURL);
$httpClient->setSslVerification(FALSE);
$cookieJar = new ArrayCookieJar();
// Create a new cookie plugin
$cookiePlugin = new CookiePlugin($cookieJar);
// Add the cookie plugin to the client
$httpClient->addSubscriber($cookiePlugin);
$httpClient->setUserAgent("Opera/9.23 (Windows NT 5.1; U; en-US)");
$request = $httpClient->get($finalURL);
$response = $request->send();
$body = $response->getBody(true);
$matches = array();
preg_match_all('/<table.*?>(.*?)<\/table>/si', $body, $table);
preg_match('/<meta name=\"keywords\" content=\"(.*?)\"\/>/si', $body, $metaName);
preg_match('/<p itemprop="streetAddress".*?>(.*?)<\/p>/si', $body, $regOffice);
echo "<table><tbody>";
echo "<tr><th>Company Name</th><td>";
echo $companyName;
echo "</td></tr>";
echo "<tr><th>Company Number</th><td>";
echo $companyNumber;
echo "</td></tr>";
echo "<tr><th>Registar Address</th><td>";
echo str_replace("<br>", " ", $regOffice[0]);
echo "</td></tr>
<tr><th>Trading Address</th><td>";
echo $TradingAddress;
echo "</td></tr>
<tr>
<th>Director Name</th>
<td>";
$name = explode(',', $metaName[1]);
echo $name[2];
echo "</td>
</tr></tbody></table>";
echo $table[0][1];
echo "<br><br><br>";
}
}
To get each page, I use http://companycheck.co.uk/search/UpdateSearchCompany?searchTerm=cars&type=name&companyPage=1 which returns json for each page from http://companycheck.co.uk/search/results?SearchCompaniesForm[name]=cars&yt1= and some data, but not all.
With this I can get the ID of each company to navigate to each link and scrape some data from the frontend of the site.
For example the first result is:
"hits":{"found":42842,"start":0,"hit":[{"id":"08958547","data":{"address":["THE ALEXANDER SUITE SILK POINT, QUEENS AVENUE, MACCLESFIELD, SK10 2BB"],"assets":[],"assetsnegative":[],"cashatbank":[],"cashatbanknegative":[],"companyname":["CAR2CARS LIMITED"],"companynumber":["08958547"],"dissolved":["0"],"liabilities":[],"liabilitiesnegative":[],"networth":[],"networthnegative":[],"postcode":["SK10 2BB"],"siccode":[]}}
So the first link is: http://companycheck.co.uk/company/08958547
Then from this I can pull table data such as:
Registered Office
THE ALEXANDER SUITE SILK POINT
QUEENS AVENUE
MACCLESFIELD
SK10 2BB
And information from the meta tags such as:
<meta name="keywords" content="CAR2CARS LIMITED, 08958547,INCWISE COMPANY SECRETARIES LIMITED,MR ROBERT CARTER"/>
An example of one of the results returned:
Company Name CAR2CARS LIMITED
Company Number 08958547
Registar Address
THE ALEXANDER SUITE SILK POINT QUEENS AVENUE MACCLESFIELD SK10 2BB
Trading Address THE ALEXANDER SUITE SILK POINT, QUEENS AVENUE, MACCLESFIELD, SK10 2BB
Director Name INCWISE COMPANY SECRETARIES LIMITED
Telephone No telephone number available.
Email Address No email address available.
Contact Person No contact person available.
Business Activity No Business Activity on record.
Each json page contains 10 company IDs to put into the URL to find the company, from each of these companies I need to scrape data from the full URL, then after these 10 move onto the next page and get the next 10 and loop this up until the last page.
It is almost certainly blocking you deliberately due to an excessive number of requests. Try putting a pause in between requests - that might help you fly under their radar.
The website you are intending to scrape appears to be a private company that is reformatting and republishing data from Companies House, the official record of company information in the UK. This company offers an API which allows 10K requests per month, and this is either free or costs GBP200/month, depending on what data you need. Since you want 40K results immediately, it is no wonder they operate IP blocks.
The rights and wrongs of scraping are complicated, but there is an important point to understand: by copying someone else's data, you are attempting to avoid the costs of collating the data yourself. By taking them from someone else's server, you are also adding to their operational costs without reimbursing them, an economic phenomenon known as an externality.
There are some cases where I am sympathetic to passing on costs in this way, such as where the scrape target is engaged in potential market abuse (e.g. monopolistic practices) and scraping has an alleviating effect. I have heard that some airline companies operate anti-scraping devices because they don't want price scrapers to bring prices down. Since bringing prices down is in the interest of the consumer one could argue that the externality can be justified (on moral, if not legal grounds).
In your case, I would suggest obtaining this data directly from Companies House, where it might be available for a much lower cost. In any case, if you republish valuable data obtained from a scrape, having dodged technical attempts to block you, you may find yourself in legal trouble anyway. If in doubt (and if there is no moral or public interest defence such as I outlined earlier) get in touch with the site operator and ask if what you want to do is OK.
I got these Json data:
[{"category":"Pizza","name":"Beef Pronto","desc":"Description of Beef Pronton here","price":"12"},
{"category":"Drink","name":"Cool Delight","desc":"Description of Coold Delight here","price":"5"},
{"category":"Drink","name":"Cola","desc":"Description of Cola","price":"4"}
]
With Javascript I have successfully managed data to present as follow:
Pizza
-Beef Pronto: Description of Beef Pronton here: 12
Drink
-Cool Delight: Description of Coold Delight here: 5
-Cola: Description of Cola: 4
any Idea how to do it with PHP ?
-->Ok guys, this is How I do it with PHP:
<?
$listedmenuJSON = '[{"category":"Pizza","name":"Beef Pronto","desc":"Description of Beef Pronton here","price":"12"},
{"category":"Drink","name":"Cool Delight","desc":"Description of Coold Delight here","price":"5"},
{"category":"Drink","name":"Cola","desc":"Description of Cola","price":"4"}
]';
$json_decoded = json_decode($listedmenuJSON);
foreach ($json_decoded as $categoryvalue){
//echo $categoryvalue->category."<br/>";
$tempcategoryvalue[] = $categoryvalue->category;
$arrunique = array_unique($tempcategoryvalue);
}
foreach ($arrunique as $tmpcategory){
echo '<br/><b>'.$tmpcategory.'</b></br>';
foreach ($json_decoded as $tempo){
if($tempo->category == $tmpcategory){
echo $tempo->name.'<br/>';
echo '<i>'.$tempo->desc.'.......</i>';
echo $tempo->price.'<br/>';
}
}
}
?>
It will generate as following:
Pizza
Beef Pronto
Description of Beef Pronton here.......12
Drink
Cool Delight
Description of Coold Delight here.......5
Cola
Description of Cola.......4
If you got json in PHP Code Use Following line to convert json to PHP array and manipulate array for desire output
$array = json_decode($json,TRUE);
In javascript user below code to convert your json to array.
var obj = jQuery.parseJSON(responce);
Try using the PHP function 'json_decode' (http://php.net/manual/en/function.json-decode.php). This will create an associative array in PHP. It should be pretty simple to access and manipulate data thereon.
If you use a SQL server then you can select form the table using "group by category"
Else, you can just verify many times the arrays.
SQL:
<?php
$query = "SELECT * FROM 'table'
GROUP BY category
having (category=\"Pizza\");"
--SELECT * FROM 'table'
--GROUP BY category
--having (category="Drink");
if you use this in php you must use sql_query("query")
This is my problem. I'm trying to retrieve a value from an XML url (from last.fm api). For example the biography of an artist.
I already know that I can do it simply like this (e.g. for Adele):
<?php
$xml = simplexml_load_file("http://ws.audioscrobbler.com/2.0/?method=artist.getinfo&artist=adele&api_key=b25b959554ed76058ac220b7b2e0a026");
$info = $xml->artist->bio->summary;
echo $info;
?>
This returns:
Adele Laurie Blue Adkins, (born 5 May 1988), is a Grammy Award-Winning English singer-songwriter from Enfield, North London. Her debut album, <a title="Adele - 19" href="http://www.last.fm/music/Adele/19" class="bbcode_album">19</a>, was released in January 2008 and entered the UK album chart at #1. The album has since received four-times Platinum certification in the UK and has sold 5,500,000 copies worldwide. The album included the hugely popular song <a title="Adele – Chasing Pavements" href="http://www.last.fm/music/Adele/_/Chasing+Pavements" class="bbcode_track">Chasing Pavements</a>. 19 earned Adele two Grammy Awards in February 2009 for Best New Artist and Best Female Pop Vocal Performance.
I would now like to put that result in a text file and store it on a folder on my website, for example: "http://mysite.com/artistdescriptions/adele.txt".
I've already tried:
<?php
$file_path = $_SERVER['DOCUMENT_ROOT'].'/artistdescriptions/adele.txt';
$xml = simplexml_load_file("http://ws.audioscrobbler.com/2.0/?method=artist.getinfo&artist=adele&api_key=b25b959554ed76058ac220b7b2e0a026");
$info = $xml->artist->bio->summary;
file_put_contents($file_path, serialize($info));
$file_contents = file_get_contents($file_path);
if(!empty($file_contents)) {
$data = unserialize($file_contents);
echo $data;
}
?>
But this unfortunately didn't work. Any help would be greatly appreciated!
Echo call magic method __toString(), but serialize doesn't, and in SimpleXMLElement isn't serialize allowed, so it throw an Exception. But you can call __toString() magic method by yourself, and this solve you problem.
Replace your line with mine
$info = $xml->artist->bio->summary->__toString();