Grabbing data from steamapi (json) and importing it to mysql - php

I can't quite figure out how to import data from remote json (steamapi) into MySql.
<?php
$con = mysql_connect("localhost","XXXXXXXXXXXXXXXXXXXX","XXXXXXXXXXXXXXXXXXXXXXXXX");
mysql_select_db('XXXXXXXXXXXXXXXXX',$con);
$url = "http://api.steampowered.com/ISteamUser/GetPlayerSummaries/v0002/?key=XXXXXXXXXXXXXXXXXXXXXXXX&steamids=76561198033811393";
$json = file_get_contents($url);
$result = json_decode($json);
foreach($result as $key => $value)
{
if($value)
{
mysql_query("INSERT INTO steam (steamid,
communityvisibilitystate,
profilestate)
VALUES ($value->steamid,
$value->communityvisibilitystate,
$value->profilestate)");
}
mysql_close($con);
}
?>
Steam JSON Image(click for full size):

You can read each player using $data->response->players, it will be assigned to $player where then, you can directly access their object like $player->steamid. See below example:
$data = json_decode($json);
foreach($data->response->players as $player)
{
echo $player->steamid, "\n\n";
}
I also suggest you change your query like this:
$query = sprintf("INSERT INTO steam (steamid,
communityvisibilitystate,
profilestate) VALUES ('%s','%s','%s')",
mysql_real_escape_string($player->steamid),
mysql_real_escape_string($player->communityvisibilitystate),
mysql_real_escape_string($player->profilestate));
To prevent any further issues like broken strings or injections.
Resulting code for the SQL part would be:
// Build the query
$query = sprintf("INSERT INTO steam (steamid,
communityvisibilitystate,
profilestate) VALUES ('%s','%s','%s')",
mysql_real_escape_string($player->steamid),
mysql_real_escape_string($player->communityvisibilitystate),
mysql_real_escape_string($player->profilestate));
// Perform Query
$insert = mysql_query($query);
// Check result
// This shows the actual query sent to MySQL, and the error. Useful for debugging.
if (!$insert)
{
echo 'Invalid query: ' . mysql_error() . "\n";
echo 'Whole query: ' . $query . "\n";
echo "------------------\n";
}
Given your actual code it would look like this:
<?php
$con = mysql_connect("localhost","XXXXXXXXXXXXXXXXXXXX","XXXXXXXXXXXXXXXXXXXXXXXXX");
mysql_select_db('XXXXXXXXXXXXXXXXX',$con);
$url = "http://api.steampowered.com/ISteamUser/GetPlayerSummaries/v0002/?key=XXXXXXXXXXXXXXXXXXXXXXXX&steamids=76561198033811393";
$json = file_get_contents($url);
$data = json_decode($json);
foreach($data->response->players as $player)
{
// Build the query
$query = sprintf("INSERT INTO steam (steamid,
communityvisibilitystate,
profilestate) VALUES ('%s','%s','%s')",
mysql_real_escape_string($player->steamid),
mysql_real_escape_string($player->communityvisibilitystate),
mysql_real_escape_string($player->profilestate));
// Perform Query
$insert = mysql_query($query);
// Check result
// This shows the actual query sent to MySQL, and the error. Useful for debugging.
if (!$insert)
{
echo 'Invalid query: ' . mysql_error() . "\n";
echo 'Whole query: ' . $query . "\n";
echo "------------------\n";
}
}
mysql_close($con);

Use this
<?php
$id = $_GET['id'];
$key = 'xxx';
$link = file_get_contents('http://api.steampowered.com/ISteamUser/GetPlayerSummaries/v0002/?key=' . $key . '&steamids=76561198033811393&format=json');
$myarray = json_decode($link, true);
$result = $myarray['response']['players'][0];
echo $result->steamid;
?>
if you get more then one result use foreach format given below
$result = $myarray['response']['players'];
foreach($result as $key => $value)
{
echo $value->steamid."<br>";
}

Related

Json data getting read but not inserting into mysql using php

Im trying to insert json data using php into mysql,
I get success msg, but no records are inserted.
My json data is :
jsondata.json:
{"users": { "bert":6.44, "earnie":0.25, "bigbird":34.45 }}
My php code:
<?php
//First: read data
$fo=fopen("data.json","r");
$fr=fread($fo,filesize("data.json"));
$array=json_decode($fr,true);
//Second: create $values
$rows = array();
foreach ($array['users'] as $key => $value)
$rows[] = "('" . $key . "', '" . $value . "')";
$values = implode(",", $rows);
//To display all values from JSON file
echo '<pre>';print_r($array);
//Save to DB
$hostname = 'localhost';
$username = 'root';
$password = '';
try
{
$dbh = new PDO("mysql:host=$hostname;dbname=nodejs", $username, $password);
echo 'Connected to database<br />';
//$count = $dbh->exec("INSERT INTO USERSAMOUNTS(USERNAME, AMOUNT) VALUES " . $values) or die(print_r($dbh->errorInfo(), true));
$count = $dbh->exec("INSERT INTO json(firstName) VALUES " . $values) or die(print_r($dbh->errorInfo(), true));
echo $count;// echo the number of affected rows
$dbh = null;// close the database connection
echo 'Success<br />';
}
catch(PDOException $e)
{
echo $e->getMessage();
}
?>
I believe the problem could be the order in which the actions are performed.
<?php
//First: read data
$fo = fopen("jsondata.json", "r");
$fr = fread($fo, filesize("jsondata.json"));
$array = json_decode($fr, true);
//Second: create $values
$rows = array();
foreach ($array['users'] as $key => $value)
$rows[] = "('" . $key . "', '" . $value . "')";
$values = implode(",", $rows);
//Third: display
echo '<pre>';
print_r($array);
//Fourth: save to db
$hostname = 'localhost';
$username = 'root';
$password = '';
try {
$dbh = new PDO("mysql:host=$hostname;dbname=nodejs", $username, $password);
echo 'Connected to database<br />'; // echo a message saying we have connected
$count = $dbh->exec("INSERT INTO USERAMOUNTS(USERNAME, AMOUNT) VALUES " . $values);
echo $count; // echo the number of affected rows
$dbh = null; // close the database connection
echo 'Success<br />';
} catch (PDOException $e) {
echo $e->getMessage();
}
?>
Enables or disables emulation of prepared statements. Some drivers do not support native prepared statements or have limited support for them for more info please check - http://php.net/manual/en/pdo.setattribute.php
$dbh = new PDO("mysql:host=$hostname;dbname=nodejs", $username, $password);
$dbh->setAttribute(PDO::ATTR_EMULATE_PREPARES, false);
$count = $dbh->exec("INSERT INTO USERAMOUNTS(USERNAME, AMOUNT) VALUES " . $values);
Hope this help.
The issue is with how your are trying to insert data. I'm surprised you're not getting an error.
You should use a prepared statement. See the following... https://stackoverflow.com/a/4629088/2033178
Some things are funky here.
At first it looks like you're expecting the data to come magically from $data (unless that is passed somewhere?)
$array = json_decode($data, true);
$rows = array();
foreach($array['users'] as $key => $value)
$rows[] = "('" . $key . "', '" . $value . "')";
$values = implode(",", $rows);
And then it looks like you're opening a file and parsing the JSON (but not doing the above magic with $rows[])
$fo=fopen("jsondata.json","r");
$fr=fread($fo,filesize("jsondata.json"));
$array=json_decode($fr,true);
Why not insert on the for each loop?
$fo=fopen("jsondata.json","r");
$fr=fread($fo,filesize("jsondata.json"));
$array=json_decode($fr,true);
$count = 0;
$dbh = new PDO("mysql:host=$hostname;dbname=nodejs", $username, $password);
try {
foreach($array['users'] as $key => $value)
$count = $count + $dbh->exec("INSERT INTO USERAMOUNTS(USERNAME, AMOUNT) VALUES " . $key . " " . $value . ")";
} catch ...

PHP array implode keys and values to function

I'm not too familiar with PHP arrays, I have the following code that generates query to output the results needed.
$allstore = $_POST['store'];
function createSelect($allstore)
{
if (empty($allstore))
return "";
$querySelect = "";
$queryJoin = "";
$baseTable = "";
foreach ($allstore as $store => $value) {
if (!$querySelect) {
$baseTable = $store;
$querySelect = "SELECT " . $store . ".item_no, " . $store . ".actual_price, " . $store . ".selling_price, " . $store . ".qty as " . $store;
} else {
$querySelect .= ", " . $store . ".qty as " . $store;
$queryJoin .= "
INNER JOIN " . $store . " ON " . $baseTable . ".item_no = " . $store . ".item_no";
}
}
$querySelect .= " FROM " . $baseTable;
$query = $querySelect . $queryJoin;
return $query;
}
//Stores to be shown
$allstore = ['s_M9' =>0 , 's_M10' =>1];
$query = (createSelect($allstore));
$result = mysql_query($query);
//rest of code...
As you can see above, at the very top there is $allstore = $_POST['store']; Which collects values based from previous form POST method that has checkbox with the name=store[] .
Now According to the function shown, if I create my own keys and values like this
$allstore = ['s_M9' =>0 , 's_M10' =>1];
the output shows exactly what i'm looking for. But the problem goes on how to let $allstore implode those stores s_M9, s_M10 based on what the user has selected on the previous page ( checkbox )? I mean, the user can select either one of the stores or Both stores . How can I implode the checked results between those brackets without inserting them manually?
Thank You
Edit :
<?php
echo "<form action='somewhere.php' method='POST'>";
$query = "SELECT * from stores_list ORDER BY short Asc";
$result = mysql_query($query);
if(mysql_num_rows($result)>0){
$num = mysql_num_rows($result);
for($i=0;$i<$num;$i++){
$row = mysql_fetch_assoc($result);
echo "<input type=checkbox name=store[] value={$row['short']} style='width:20px; height:20px;'>{$row['short']}";
}
}
else{
//No Stores Available
echo "No Stores Found !";
}
echo "</td><input type='submit' value='Search'/></form>";
$allstore = [];
if (!empty($_POST['store'])) {
foreach ($_POST['store'] as $value) {
$allstore[$value] = 1; // or 0, it doesn't matter because your function adds all the keys
}
}
$query = (createSelect($allstore));
$result = mysql_query($query);
And of course you have to take care of your createSelect function to avoid SQL Injections, please read here

Build a batch query for MySQL insert each 1000 items

I need to perform a batch insert in MySQL/MariaDB but since data is dynamic I need to build the proper SQL query. In a few steps:
I should find whether the current row exists or not in table - this is the first SELECT inside the loop
Right now I have 1454 but have to insert around 150k later, is better a batch query than 150k INSERT per item on the loop
If record already exists I should update it if doesn't then I should insert ,I just not care about UPDATE yet and the code you're seeing is only for INSERT
So here is what I am doing:
// Get values from Csv file as an array of values
$data = convertCsvToArray($fileName);
echo "DEBUG count(data): ", count($data), "\n";
$i = 0;
$sqlInsert = "INSERT INTO reps(veeva_rep_id,first,last,email,username,lastLoginAt,lastSyncAt,display_name,rep_type,avatar_url,createdAt,updatedAt) ";
// Processing on each row of data
foreach ($data as $row) {
$sql = "SELECT id,lastSyncAt FROM reps WHERE veeva_rep_id='{$row['Id']}'";
echo "DEBUG: ", $sql, "\n";
$rs = $conn->query($sql);
if ($rs === false) {
echo 'Wrong SQL: '.$sql.' Error: '.$conn->error, E_USER_ERROR;
} else {
$rows_returned = $rs->num_rows;
$veeva_rep_id = "'".$conn->real_escape_string($row['Id'])."'";
$first = "'".$conn->real_escape_string(ucfirst(strtolower($row['FirstName'])))."'";
$last = "'".$conn->real_escape_string(ucfirst(strtolower($row['LastName'])))."'";
$email = "'".$conn->real_escape_string($row['Email'])."'";
$username = "'".$conn->real_escape_string($row['Username'])."'";
$display_name = "'".$conn->real_escape_string(
ucfirst(strtolower($row['FirstName'])).' '.ucfirst(strtolower($row['LastName']))
)."'";
// VALUES should be added only if row doesn't exists
if ($rows_returned === 0) {
// VALUES should be append until they reach 1000
while ($i % 1000 !== 0) {
$sqlInsert .= "VALUES($veeva_rep_id,$first,$last,$email,$username,NOW(),NOW(),$display_name,'VEEVA','https://pdone.s3.amazonaws.com/avatar/default_avatar.png',NOW(),NOW())";
++$i;;
}
// QUERY should be output to console to see if it's right or something is wrong
echo "DEBUG: ", $sqlInsert, "\n";
// QUERY should be executed if there are 1000 VALUES ready to add as a batch
/*$rs = $conn->query($sqlInsert);
if ($rs === false) {
echo 'Wrong SQL: '.$sqlInsert.' Error: '.$conn->error, E_USER_ERROR;*/
}
} else {
// UPDATE
echo "UPDATE";
}
}
}
But this line of code: echo "DEBUG: ", $sql, "\n"; is not outputting nothing to console. I must be doing something wrong but I can't find what. Can any help me to build the proper batch query and to execute it each 1000 values append?
Proper output should be:
DEBUG count(data): 1454
DEBUG: SELECT id,lastSyncAt FROM reps WHERE veeva_rep_id='00580000008ReolAAC'
DEBUG: SELECT id,lastSyncAt FROM reps WHERE veeva_rep_id='005800000039SIWAA2'
....
DEBUG: INSERT INTO reps(veeva_rep_id,first,last,email,username,lastLoginAt,lastSyncAt,display_name,rep_type,avatar_url,createdAt,updatedAt) VALUES(...), VALUES(...), VALUES(...)
Obtained result:
DEBUG count(data): 1454
DEBUG: SELECT id,lastSyncAt FROM reps WHERE veeva_rep_id='00580000008RGg6AAG'
DEBUG: INSERT INTO reps(veeva_rep_id,first,last,email,username,lastLoginAt,lastSyncAt,display_name,rep_type,avatar_url,createdAt,updatedAt)
DEBUG: SELECT id,lastSyncAt FROM reps WHERE veeva_rep_id='00580000008RQ4CAAW'
DEBUG: INSERT INTO reps(veeva_rep_id,first,last,email,username,lastLoginAt,lastSyncAt,display_name,rep_type,avatar_url,createdAt,updatedAt)
.... // until reach 1454 results
The table is empty so it should never goes through ELSE condition (UPDATE one).
EDIT
With help from the answer this is how the code looks now:
$data = convertCsvToArray($fileName);
echo "DEBUG count(data): ", count($data), "\n";
$i = 1;
$sqlInsert = "INSERT INTO reps(veeva_rep_id,first,last,email,username,lastLoginAt,lastSyncAt,display_name,rep_type,avatar_url,createdAt,updatedAt) VALUES";
foreach ($data as $row) {
$sql = "SELECT id,lastSyncAt FROM reps WHERE veeva_rep_id='{$row['Id']}'";
$rs = $conn->query($sql);
if ($rs === false) {
echo 'Wrong SQL: '.$sql.' Error: '.$conn->error, E_USER_ERROR;
} else {
$rows_returned = $rs->num_rows;
$veeva_rep_id = "'".$conn->real_escape_string($row['Id'])."'";
$first = "'".$conn->real_escape_string(ucfirst(strtolower($row['FirstName'])))."'";
$last = "'".$conn->real_escape_string(ucfirst(strtolower($row['LastName'])))."'";
$email = "'".$conn->real_escape_string($row['Email'])."'";
$username = "'".$conn->real_escape_string($row['Username'])."'";
$display_name = "'".$conn->real_escape_string(
ucfirst(strtolower($row['FirstName'])).' '.ucfirst(strtolower($row['LastName']))
)."'";
if ($rows_returned === 0) {
if ($i % 1000 === 0) {
file_put_contents("output.log", $sqlInsert."\n", FILE_APPEND);
$sqlInsert = "INSERT INTO reps(veeva_rep_id,first,last,email,username,lastLoginAt,lastSyncAt,display_name,rep_type,avatar_url,createdAt,updatedAt) VALUES";
} else {
$sqlInsert .= "($veeva_rep_id,$first,$last,$email,$username,NOW(),NOW(),$display_name,'VEEVA','https://pdone.s3.amazonaws.com/avatar/default_avatar.png',NOW(),NOW()), ";
}
$i++;
} else {
echo "UPDATE";
}
}
}
But still buggy because:
I have got a first empty INSERT query: INSERT INTO reps(veeva_rep_id,first,last,email,username,lastLoginAt,lastSyncAt,display_name,rep_type,avatar_url,createdAt,updatedAt) VALUES
I have got a second INSERT query with 1000 VALUES() append, but what happened with the rest? The remaining 454?
Can any give me another tip? Help?
Since it looks like you are trying to load data from a CSV file, you might want to consider using LOAD DATA INFILE functionality which is designed specifically for this purpose.
Here is link to documentation: https://dev.mysql.com/doc/refman/5.6/en/load-data.html
consider using INSERT IGNORE INTO table to check if the record already exists. How to 'insert if not exists' in MySQL?
if you haven't already done so, make veeva_rep_id a PRIMARY key so the INSERT IGNORE will work
also check out using PDO for transactions, prepared statements and dynamically generating queries using PDO
PDO Prepared Inserts multiple rows in single query
<?php
$sql = 'INSERT IGNORE INTO reps(veeva_rep_id,first,last,email,username,lastLoginAt,lastSyncAt,display_name,rep_type,avatar_url,createdAt,updatedAt) VALUES ';
$insertQuery = array();
$insertData = array();
/*
assuming the array from the csv is like this
$data = array(
0 => array('name' => 'Robert', 'value' => 'some value'),
1 => array('name' => 'Louise', 'value' => 'another value')
);
*/
foreach ($data as $row) {
$insertQuery[] = '(:veeva_rep_id' . $n . ', :first' . $n . ', :last' . $n . ', :email' . $n . ', :username' . $n . ', :lastLoginAt' . $n . ', :lastSyncAt' . $n . ', :display_name' . $n . ', :rep_type' . $n . ', :avatar_url' . $n . ', :createdAt' . $n . ', :updatedAt' . $n . ')';
$insertData['veeva_rep_id' . $n] = $row['name'];
$insertData['first' . $n] = $row['value'];
$insertData['last' . $n] = $row['name'];
$insertData['email' . $n] = $row['value'];
$insertData['username' . $n] = $row['name'];
$insertData['lastLoginAt' . $n] = $row['value'];
$insertData['lastSyncAt' . $n] = $row['value'];
$insertData['display_name' . $n] = $row['name'];
$insertData['rep_type' . $n] = $row['value'];
$insertData['avatar_url' . $n] = $row['value'];
$insertData['createdAt' . $n] = $row['name'];
$insertData['updatedAt' . $n] = $row['value'];
$n++;
}
$db->beginTransaction();
if (!empty($insertQuery) and count($insertQuery)>1000) {
$sql .= implode(', ', $insertQuery);
$stmt = $db->prepare($sql);
$stmt->execute($insertData);
}
$db->commit();
print $sql . PHP_EOL;
let me know if it helps.
You should have something like:
// Try fetching data from table 1
// If there is no record available, then fetch some data from table 2
// and insert that data inito table 1
You just wrote
$sql = "INSERT INTO reps(veeva_rep_id,first,last,email,username,lastLoginAt,lastSyncAt,display_name,rep_type,avatar_url,createdAt,updatedAt) ";
// Processing on each row of data
foreach ($data as $row) {
But from an insert no data is selected and second...you didn't run a select, where comes $data from?
update Use if ($i % 1000 === 0) { instead of while ($i % 1000 !== 0) {
$i = 0;
$sqlInsert = "INSERT INTO reps(veeva_rep_id,first,last,email,...) ";
// Processing on each row of data
foreach ($data as $row) {
$sql = "SELECT id,lastSyncAt FROM reps WHERE veeva_rep_id='{$row['Id']}'";
echo "DEBUG: ", $sql, "\n";
$rs = $conn->query($sql);
if ($rs === false) {
echo 'Wrong SQL: '.$sql.' Error: '.$conn->error, E_USER_ERROR;
} else {
$veeva_rep_id = ...;
$first = ...;
$last = ...;
$email = ...;
// ...
// VALUES should be added only if row doesn't exists
if($rs->num_rows == 0) {
// Insert some data
$i++;
if ($i % 1000 === 0) {
echo "DEBUG: ", $sqlInsert, "\n";
// execSql($sqlInsert);
$sqlInsert = "INSERT INTO reps(veeva_rep_id,first,last,email,...) "; // reset
} else {
$sqlInsert .= "VALUES($veeva_rep_id,$first,$last,$email,...) ";
}
} else {
echo "UPDATE";
}
}
}

array data to mysql

I need to parse the following code and process the resulting data.
foreach($job as $x=>$x_value)
{
echo "Key=" . $x . ", Value=" . $x_value;
echo "<br>";
}
The above code is returning the following as expected.
Key=vca_id, Value=20130<br>Key=uuid, Value=3c87e0b3-cfa<br>Key=originate_time, Value=2013-03-15 14:30:18<br>
What I need to do is to put the values in mysql database. So the insert statement would look something like this...
insert into test.master_table (vca_id, uuid, originate_time) values ('20130', '3c87e0b3-cfa', '2013-03-15 14:30:18')
What is the correct way to save the array values to mysql database?
<?php
mysql_query("insert into test.master_table(vca_id, uuid, originate_time)values('".$job['vca_id']."','".$job['uuid']."','".$job['originate_time']."')");
?>
Well i will recommend implode
$keys = array();
$values = array();
foreach($job as $x => $x_value)
{
$keys[] = $x;
$values[] = $x_value;
}
$query = 'INSERT INTO test.master_table' . '('.implode(',',$keys) .') VALUES (' .implode(',',$values) . ')';
You can try this
$temp_value_arr = array();
$query = "INSERT into test.master_table SET ";
foreach($job as $x=>$x_value)
{
$query .= "$x = '$x_value',";
}
$query = rtrim($query, ',');
mysql_query($query);

Retrieving values from MySQL

I have a very simple table that contains a list of 'victims' and the corresponding number of that type destroyed. I'm trying to make an output page of this information, using this code:
foreach( $victims as $vic )
{
$hits = mysql_query("SELECT amount
FROM victims
WHERE victim = ".$vic );
echo $hits;
print "$vic: $hits <br /><hr>";
}
However, hits comes out empty. What's wrong with my SQL query?
foreach($victims as $vic)
{
$hits = mysql_query('SELECT amount
FROM victims
WHERE victim = "' . mysql_real_escape_string($vic) . '"');
if($hits && mysql_num_rows($hits)>0) {
while($row = mysql_fetch_array($hits)) {
echo '<p>' . $row['amount'] . ' hits</p>';
}
} else {
echo '<p>' . mysql_error() . '</p>';
}
}
mysql_query() doesn't return the actual result of your query, but rather a resource with which you can then access the results.
This is a typical pattern:
$result = mysql_query(...);
$row = mysql_fetch_assoc($result);
print($row['amount']);
Each call to mysql_fetch_assoc returns the next row of the result set. If you were expecting multiple rows to be returned, you can call this in a while loop:
$result = mysql_query(...);
while ($row = mysql_fetch_assoc($result)) {
print($row['amount']);
}
Since there's no sane error checking in any of the answers, I'll put the whole thing in here:
foreach( $victims as $vic )
{
$sql = "SELECT amount
FROM victims
WHERE victim = '".mysql_real_escape_string($vic)."'";
$result = mysql_query($sql);
$result or die('Query Error: '.mysql_error() . ' - ' . $sql);
$hitsarray = mysql_fetch_assoc($result);
if ($hitsarray) {
$hits = $hitsarray['amount'];
} else {
// No row was found
$hits = 0;
}
echo $hits;
print "$vic: $hits <br /><hr>";
}
Oh, and this fixes the query error that caused the issue in the first place. Note the quotes wrapping the $vic variable in the string, as well as the proper escaping of the string...

Categories