PHP CURL data from table into CSV - php

I would like to gather the historical price data shown on this page and format the output as a csv file.
require_once('simple_html_dom.php');
$html = file_get_html("https://coinmarketcap.com/currencies/binance-coin/historical-data/?start=20180101&end=20180519"); // you don't need to use curl
$yourDesiredContent = $html->find('#historical-data .table', 0)->plaintext;
The problem is this gathers everything into one "string" but I need to seperate each td value in the table with a comma or whatever is appropriate for csv format.
Output:
Date Open High Low Close Volume Market Cap May 18, 2018 12.36 16.22 12.05 15.14 243,261,000 1,409,710,000 May 17, 2018 12.32 12.97 12.27 12.46 54,251,500 1,404,490,000 May 16, 2018 12.56 12.57 11.95 12.26 35,233,200 1,432,140,000 May 15, 2018 12.85 13.27 12.51 12.57 45,696,200 1,465,880,000 May 14, 2018 13.16 13.17 12.36 12.87 49,317,700 1,500,590,000 May 13, 2018 13.01 13.55 12.74 13.12 70,816,500 1,483,450,000 May 12, 2018 12.97 13.15 12.25 12.94 44,001,000 1,479,430,000 May 11, 2018 13.88 14.09 12.59 12.99 57,027,500 1,582,950,000 May 10, 2018 14.68 15.05 13.82 13.82 68,250,000
Is it possible I can do this?

Related

Steam Market API?

I know that there's a Steam API allowing me to use data from Steam Community.
My question is, does anyone know if there's a Steam Market API?
For example, I want to get the current price of an item in the Steam Market.
I've googled and haven't found anything yet.
I'd be glad to have your help.
I could not find any documentation, but I use:
http://steamcommunity.com/market/priceoverview/?appid=730&currency=3&market_hash_name=StatTrak%E2%84%A2 M4A1-S | Hyper Beast (Minimal Wear)
to return a JSON.
At time of writing, it returns:
{"success":true,"lowest_price":"261,35€ ","volume":"11","median_price":"269,52€ "}
You can change the currency. 1 is USD, 3 is euro but there are probably others.
A better search api that can give you all the results for a game, example using pubg which only has 272 items, if your game has more try changing the count parameter at the end
https://steamcommunity.com/market/search/render/?search_descriptions=0&sort_column=default&sort_dir=desc&appid=578080&norender=1&count=500
I indexed the available currencies steam uses for argument
&currency=3
as:
1 : $63.83
2 : £46.85
3 : 52,--€
4 : CHF 56.41
5 : 4721,76 pуб.
6 : 235,09zł
7 : R$ 340,80
8 : ¥ 6,627.08
9 : 534,70 kr
10 : Rp 898 383.24
11 : RM257.74
12 : P3,072.66
13 : S$84.47
14 : ฿1,921.93
15 : 1.474.136,93₫
16 : ₩ 69,717.79
17 : 468,47 TL
18 : 2 214,94₴
19 : Mex$ 1,557.75
20 : CDN$ 99.09
21 : A$ 100.40
22 : NZ$ 107.55
23 : ¥ 505.96
24 : ₹ 5,733.04
25 : CLP$ 55.695,47
26 : S/.283.03
27 : COL$ 271.637,06
28 : R 1 193.49
29 : HK$ 606.83
30 : NT$ 2,189.42
31 : 293.64 SR
32 : 287.51 AED
Python dictionary with currency abbreviations and their codes:
currencies = {
"USD": 1, # United States dollar
"GBP": 2, # British pound sterling
"EUR": 3, # The euro
"CHF": 4, # Swiss franc
"RUB": 5, # Russian ruble
"PLN": 6, # Polish złoty
"BRL": 7, # Brazilian real
"JPY": 8, # Japanese yen
"SEK": 9, # Swedish krona
"IDR": 10, # Indonesian rupiah
"MYR": 11, # Malaysian ringgit
"BWP": 12, # Botswana pula
"SGD": 13, # Singapore dollar
"THB": 14, # Thai baht
"VND": 15, # Vietnamese dong
"KRW": 16, # South Korean won
"TRY": 17, # Turkish lira
"UAH": 18, # Ukrainian hryvnia
"MXN": 19, # Mexican Peso
"CAD": 20, # Canadian dollar
"AUD": 21, # Australian dollar
"NZD": 22, # New Zealand dollar
"CNY": 23, # Chinese yuan
"INR": 24, # Indian rupee
"CLP": 25, # Chilean peso
"PEN": 26, # Peruvian sol
"COP": 27, # Colombian peso
"ZAR": 28, # South African rand
"HKD": 29, # Hong Kong dollar
"TWD": 30, # New Taiwan dollar
"SAR": 31, # Saudi riyal
"AED": 32 # United Arab Emirates dirham
}
To add to what the other people have said, the temporary ban on the JSON site happens if you try and request 20 items within a minute's time from the server. If you're creating a script to request those links, add a three second delay between each script.
Also, the ban only lasts for the remaining server-side minute (which may not be 60 seconds).
You can use SteamApis.com to acquire Steam market prices and item information. The data is returned in JSON. The service is not free but also not that expensive.
The documentation is available to view here. It has detailed information on what endpoints are available and what data is returned.
There is not such API for now. But this link may help you:
Get the price of an item on Steam Community Market with PHP and Regex
It's basically what you want with pure php DOM parsing instead of an API. The main drawback is that you may have to change your code if Steam update their html markup.
Script-scraper which maps search results from https://steamcommunity.com/market/search?q= to array of objects
Array.from(document.querySelectorAll('a.market_listing_row_link')).map(item => {
const itemInfo = item.children[0]
return {
isStatTrek: itemInfo.getAttribute('data-hash-name').startsWith('StatTrak™'),
condition: itemInfo.getAttribute('data-hash-name').match(/.*\((.*)\)/)[1],
priceUSD: Number(itemInfo.querySelector('.normal_price[data-price]').getAttribute('data-price')/100)
}
})
can be used with iframe and "weapon | skin name (condition)" search template

extracting field from twitter feed using json_decode

I am trying to grab my twitter feed using the following code:
// Make the request and get the response into the $json variable
$json = $twitter->setGetfield($getfield)
->buildOauth($url, $requestMethod)
->performRequest();
// It's json, so decode it into an array
$result = json_decode($json);
// Access the profile_image_url element in the array
echo $result->created_at;
?>
I get the result of:
Thu Oct 25 18:40:50 +0000 2012
If I try to get the text with:
echo $result->text;
I get this error:
Notice: Undefined property: stdClass::$text in /Library/WebServer/Documents/include/twitter_noformat/items.php on line 35
A partial var_dump of my data format includes this:
{"created_at":"Thu Aug 01 16:12:18 +0000 2013",
"id":362969042497175553,
"id_str":"362969042497175553",
"text":"A warm welcome to our new international students from China, Hong Kong and Germany! http:\/\/t.co\/GLvt3GynJV",
"source":"web",
"truncated":false,"in_reply_to_status_id":null,"in_reply_to_status_id_str":null,"in_reply_to_user_id":null,"in_reply_to_user_id_str":null,"in_reply_to_screen_name":null,"user":
My question is:
created_at gives me a value. id gives me a value. Why doesn't text? I know nothing about JSON btw. I'm not a very advanced programmer, but the pattern looks the same to me.
Edit: Well I found a cool snippet that converted my twitter array to something more readable. The function goes like this:
// It's json, so decode it into an array
$result = json_decode($json);
// Access the profile_image_url element in the array
$pretty = function($v='',$c=" ",$in=-1,$k=null)use(&$pretty){$r='';if(in_array(gettype($v),array('object','array'))){$r.=($in!=-1?str_repeat($c,$in):'').(is_null($k)?'':"$k: ").'<br>';foreach($v as $sk=>$vl){$r.=$pretty($vl,$c,$in+1,$sk).'<br>';}}else{$r.=($in!=-1?str_repeat($c,$in):'').(is_null($k)?'':"$k: ").(is_null($v)?'<NULL>':"<strong>$v</strong>");}return$r;};
echo $pretty($result);
The results now look like this:
statuses_count: 583
lang: en
status:
created_at: Thu Aug 01 21:10:10 +0000 2013
id: 363044004444651522
id_str: 363044004444651522
text: #CalStateEastBay AD Sara Lillevand Judd '86 honored for her work as an athletic adminstrator. http://t.co/WzOqjIDrBw
This is strange because that makes text look like it's part of an object?
I have determined that twitter kicks back an array of objects. Those objects can have a lot of items(?) As I mentioned previously though I can echo $result->created_at; but not text. They are both at the same level of the array.
thanks in advance for your help,
Donovan
Alright here was my solution after a day of research:
$result = json_decode( $json );
echo "Text:" . $result->status->text . "<br />";
Text was a child(?) of status. I could echo created_at because it was used at two levels of the array, which I hadn't seen before. Text was inside the status object I guess.
created_at: Thu Oct 25 18:40:50 +0000 2012
favourites_count: 1
utc_offset: -25200
time_zone: Pacific Time (US & Canada)
geo_enabled: 1
verified:
statuses_count: 583
lang: en
status:
created_at: Thu Aug 01 21:10:10 +0000 2013
id: 363044004444651522
id_str: 363044004444651522
text: #CalStateEastBay AD Sara Lillevand Judd '86 honored for her work as an athletic adminstrator. http://t.co/WzOqjIDrBw

Check for '04' at the start of a number

I need away to tell if a $_POST[phone] is starting with 04 if it is I want run a query
In javascript I know how to do this but for some reason on Ubuntu 8 (Yes I know its old) and Firefox the agents are able to still post 04 number in mobile field after testing my javascript query over and over again - I thought instead
You can do a substr like this:
if (substr($_POST['phone'], 0, 2) == '04') {
// ... do some code here
}
If you want to match multiple strings at the same time you could do this (it's faster than doing an in_array()):
$prefixes = array('02'=>true, '03'=>true, '07'=>true, '08'=>true);
$inputPrefix = substr($_POST['phone'], 0, 2);
if (isset($prefixes[$inputPrefix])) {
// ... do some code here
}
Based on your comment if you want to make sure it is only 03, 02, 07 or 08 you can do the following:
$telephonePrefix = substr($_POST['phone'], 0, 2);
$knowPrefixes = array('03','02','07','08');
if (in_array($telephonePrefix , $knowPrefixes)) {
// ... do some code here
}

fullCalendar "events: 'somepage.php'" JSON Results not displaying

I am trying to get the events for my fullCalendar implementation with the "events (as a JSON feed)" built in feature.
...
event: 'calendarFill.php',
...
The php file returns the following JSON data:
[{"id":"40","title":"test four","services":"Reflexology (40), foot-soak","start":"Tue Mar 26 2013 13:00:00","end":"Tue Mar 26 2013 13:40:00","color":"#e56b15","customerId":"21","phone":"555-0404","email":"test4","notes":"test two of update. this sentence added at update.","seat":"Side Couch 9","vip":"yes","confirmed":"yes","knows_policies":"yes","customer_here":"0","allDay":"false"},{"id":"41","title":"test four","services":"Foot-Soak, ","start":"Wed Mar 27 2013 19:00:00","end":"Wed Mar 27 2013 19:15:00","color":"#e56b15","customerId":"21","phone":"555-0404","email":"test4","notes":"test of addslashes. what's going to happen?","seat":"Front Chair 2","vip":"yes","confirmed":"yes","knows_policies":"yes","customer_here":"0","allDay":"false"}]
The above does not work - the events are not displayed on the calendar. I've cut and pasted the code directory in my Javascript code:
events: {"id":"40","title":"test four","services":"Reflexology (40), foot-soak","start":"Tue Mar 26 2013 13:00:00","end":"Tue Mar 26 2013 13:40:00","color":"#e56b15","customerId":"21","phone":"555-0404","email":"test4","notes":"test two of update. this sentence added at update.","seat":"Side Couch 9","vip":"yes","confirmed":"yes","knows_policies":"yes","customer_here":"0","allDay":"false"},{"id":"41","title":"test four","services":"Foot-Soak, ","start":"Wed Mar 27 2013 19:00:00","end":"Wed Mar 27 2013 19:15:00","color":"#e56b15","customerId":"21","phone":"555-0404","email":"test4","notes":"test of addslashes. what's going to happen?","seat":"Front Chair 2","vip":"yes","confirmed":"yes","knows_policies":"yes","customer_here":"0","allDay":"false"}],
And, it does not display. But, if I edit the above to remove all the quotes that are not surrounding text values (as below), IT WORKS.
events: [{id:40,title:"test four",services:"Reflexology (40), foot-soak",start:"Tue Mar 26 2013 13:00:00",end:"Tue Mar 26 2013 13:40:00",color:"#e56b15",customerId:21,phone:"555-0404",email:"test4",notes:"test two of update. this sentence added at update.",seat:"Side Couch 9",vip:"yes",confirmed:"yes",knows_policies:"yes",customer_here:0,allDay:false},{id:41,title:"test four",services:"Foot-Soak, ",start:"Wed Mar 27 2013 19:00:00",end:"Wed Mar 27 2013 19:15:00",color:"#e56b15",customerId:21,phone:"555-0404",email:"test4",notes:"test of addslashes. what's going to happen?",seat:"Front Chair 2",vip:"yes",confirmed:"yes",knows_policies:"yes",customer_here:0,allDay:false}],
I am using the documentation at this link: http://arshaw.com/fullcalendar/docs/event_data/events_json_feed/
What am I doing wrong? Should I remove the 'json_encode' from my php and just return a string formatted like the above? I'd like to do it the 'right' way and not some work-around.
Added the php as requested:
// for connection to db
include("../includes/config.php");
include("../includes/mysql_functions.php");
// connection with db
$linkID = db_connect();
$returnValue = array();
$getEventData = mysql_query("SELECT apt_num, services, apt_date_start, apt_date_end, therapist, customer_num, notes, seat, confirmed, knows_policies, here FROM appointments", $linkID);
if ($getEventData != FALSE && mysql_num_rows($getEventData) > 0)
{
while ($theEventData = mysql_fetch_array($getEventData))
{
$getCustomerString = "SELECT first_name, middle_name, last_name, phone, email, vip FROM customer WHERE customer_num = ".$theEventData['customer_num'];
$getCustomerData = mysql_query($getCustomerString, $linkID);
if ($getCustomerData != FALSE && mysql_num_rows($getCustomerData) > 0)
{
while($theCustomerData = mysql_fetch_array($getCustomerData))
{
$customerName = $theCustomerData['first_name']." ".$theCustomerData['middle_name']." ".$theCustomerData['last_name'];
$customerPhone = $theCustomerData['phone'];
$customerEmail = $theCustomerData['email'];
$customerVip = $theCustomerData['vip'];
}
}
else
{
$customerName = "error";
$customerPhone = "error";
$customerEmail = "error";
$customerVip = "error";
}
$rowArray['id'] = $theEventData['apt_num'];
$rowArray['title'] = $customerName;
$rowArray['services'] = $theEventData['services'];
$rowArray['start'] = date("D", $theEventData['apt_date_start'])." ".date("M", $theEventData['apt_date_start'])." ".date("d", $theEventData['apt_date_start'])." ".date("Y", $theEventData['apt_date_start'])." ".date("H", $theEventData['apt_date_start']).":".date("i", $theEventData['apt_date_start']).":".date("s", $theEventData['apt_date_start']);
$rowArray['end'] = date("D", $theEventData['apt_date_end'])." ".date("M", $theEventData['apt_date_end'])." ".date("d", $theEventData['apt_date_end'])." ".date("Y", $theEventData['apt_date_end'])." ".date("H", $theEventData['apt_date_end']).":".date("i", $theEventData['apt_date_end']).":".date("s", $theEventData['apt_date_end']);
$rowArray['color'] = $theEventData['therapist'];
$rowArray['customerId'] = $theEventData['customer_num'];
$rowArray['phone'] = $customerPhone;
$rowArray['email'] = $customerEmail;
$rowArray['notes'] = $theEventData['notes'];
$rowArray['seat'] = $theEventData['seat'];
$rowArray['vip'] = $customerVip;
$rowArray['confirmed'] = $theEventData['confirmed'];
$rowArray['knows_policies'] = $theEventData['knows_policies'];
$rowArray['customer_here'] = $theEventData['here'];
$rowArray['allDay'] = "false";
array_push($returnValue, $rowArray);
}
}
else
{
$returnValue[0] = "error";
}
print json_encode($returnValue);
Markus Vetter (from Google+) found the problem. In the original code returned from my php file there is an "'" that causes an issue. I needed to 'addslashes' to my php before returning it to the jQuery/Javascript. Once that was done, the jQuery / fullCalendar code was able to render my events as expected.
A second pair of eyes does wonders! Thank you Markus Vetter!
The original code returned:
[{"id":"40","title":"test four","services":"Reflexology (40), foot-soak","start":"Tue Mar 26 2013 13:00:00","end":"Tue Mar 26 2013 13:40:00","color":"#e56b15","customerId":"21","phone":"555-0404","email":"test4","notes":"test two of update. this sentence added at update.","seat":"Side Couch 9","vip":"yes","confirmed":"yes","knows_policies":"yes","customer_here":"0","allDay":"false"},{"id":"41","title":"test four","services":"Foot-Soak, ","start":"Wed Mar 27 2013 19:00:00","end":"Wed Mar 27 2013 19:15:00","color":"#e56b15","customerId":"21","phone":"555-0404","email":"test4","notes":"test of addslashes. what's going to happen?","seat":"Front Chair 2","vip":"yes","confirmed":"yes","knows_policies":"yes","customer_here":"0","allDay":"false"}]
After using 'addslashes' in my php for all text fields:
[{"id":"40","title":"test four","services":"Reflexology (40), foot-soak","start":"Tue Mar 26 2013 13:00:00","end":"Tue Mar 26 2013 13:40:00","color":"#e56b15","customerId":"21","phone":"555-0404","email":"test4","notes":"test two of update. this sentence added at update.","seat":"Side Couch 9","vip":"yes","confirmed":"yes","knows_policies":"yes","customer_here":"0","allDay":"false"},{"id":"41","title":"test four","services":"Foot-Soak, ","start":"Wed Mar 27 2013 19:00:00","end":"Wed Mar 27 2013 19:15:00","color":"#e56b15","customerId":"21","phone":"555-0404","email":"test4","notes":"test of addslashes. what\'s going to happen?","seat":"Front Chair 2","vip":"yes","confirmed":"yes","knows_policies":"yes","customer_here":"0","allDay":"false"}]
Actually solved this time! In my php file, the last value added to the $rowArray is: $rowArray['allDay'] = "false". Instead of being a text value, it needs to be the actual boolean value. This "$rowArray['allDay'] = false" fixed the problem.
I had the same issue and it was a formatting thing for me as well.
This is how my json is returning now and it is finally working.
[{"title":"Coke Trade Show","start":"2014-12-03"},{"title":"Michigan Show","start":"2014-12-04"}]
Definitely a good idea to paste your json return straight into the JS to make sure it is working.

create parent child array from inline data in php

actually i have simple problem but i forget how to solve it.. :D
i have data on table with following format
01 Johson 01 Craig
01 Johson 02 Daniel
01 Johson 03 Abbey
02 Dawson 01 Brown
02 Dawson 02 Agust
03 Brick 01 Chev
03 Brick 01 Flinch
so i want it to become an array like this
01 Johson => 01 Craig
``````````````02 Daniel
```````````````03 Abey
`
etc...
how to iterate trough the data and make it array like that...
i'm newby in PHP :))
There are a number of ways to get the outcome, though I am partial to using a mysql group concat:
select
col1,
group_concat(col2) as col2
from
tableName
group by
col1;
This will return that particular column of data in a comma delimted string, which you can then very easily explode inside your PHP code as it comes from the database.
Even though a comma is the default for group_concat, you can easily change it to concat the rows of data on a different string as well.
Is this what you want?
$results = array();
while ($row = mysqli_fetch_assoc($query_result)) {
$parent = $row['parentid'].' '.$row['parentname'];
if (!array_key_exists($parent, $results)) {
$results[$parent] = array();
}
$results[$parent][] = $row['childid'].' '.$row['childname'];
}

Categories