Overwriting entries in a MYSQL database with new entries - php

I am trying to replace previous entries in MYSQL database each time new data is available, I have the following PHP code but it seems to add new entries each time. Please help, thanks.
I have tried using REPLACE but it still does not work, could anyone tell me what it is I am doing wrong?
<?php
header('Content-Type: application/json');
$data = json_decode(file_get_contents('php://input'), true);
$mysqli = new mysqli("localhost","dbuser","Pa55uu0Rd","iewdb");
if (mysqli_connect_errno())
{
echo json_encode(array('error' => 'Failed to connect to MySQL: ' . mysqli_connect_error() ));
return;
}
if(!$data)
{
echo json_encode(array('error' => 'Error input data'));
return;
}
$usernme = $data['usernme'];
$longitude = $data['longitude'];
$latitude = $data['latitude'];
$user = $mysqli->query("SELECT id FROM Users WHERE usernme = '$usernme' LIMIT 1");
$user_id = $user->fetch_object();
if(!$user_id)
{
$mysqli->query("INSERT INTO Users (usernme) VALUES ('$usernme');");
$user_id->id = $mysqli->insert_id;
}
if($longitude && $latitude)
{
$mysqli->query("REPLACE INTO Locations (User_id,Longitude, Latitude) VALUES ($user_id->id,$longitude,$latitude);");
}
$mysqli->close();
echo json_encode(array('user_id' => $user_id->id));

use update query something like this
UPDATE MyTable
SET User_id = 'USER_ID_VALUE', Longitude='LONGITUDE_VALUE', Latitude='LATITUDE_VALUE'
WHERE SomeOtherColumn LIKE '%PATTERN%'

Logic : Instead of replacing old entry you can delete that old entries and later add fresh entries into database will always good in case of performance..
So you will have to write one delete and insert query only...instead of 3 queries

Here is my solution to the problem and it works just fine. I decided to go with UPDATE as you can see below as I thought it was tidiest, thanks for the help.
<?php
header('Content-Type: application/json');
//get parameters
$data = json_decode(file_get_contents('php://input'), true);
// Create connection
$mysqli = new mysqli("localhost","dbuser","Pa55w0rd","ewdb");
// Check connection
if (mysqli_connect_errno())
{
echo json_encode(array('error' => 'Failed to connect to MySQL: ' . mysqli_connect_error() ));
return;
}
if(!$data)
{
echo json_encode(array('error' => 'Error input data'));
return;
}
$usernme = $data['usernme'];
$longitude = $data['longitude'];
$latitude = $data['latitude'];
$user = $mysqli->query("SELECT id FROM Users WHERE usernme = '$usernme' LIMIT 1");
$user_id = $user->fetch_object();
if(!$user_id)
{
$mysqli->query("INSERT INTO Users (usernme) VALUES ('$usernme');");
$user_id->id = $mysqli->insert_id;
$mysqli->query("INSERT INTO Locations (User_id) VALUES ($user_id->id);");
}
if($longitude && $latitude)
{
$mysqli->query("UPDATE Locations SET Longitude = $longitude, Latitude = $latitude WHERE User_id = $user_id->id;");
}
/* close connection */
$mysqli->close();
echo json_encode(array('user_id' => $user_id->id));

Related

Scraping GPU data seems taking too long

so in my spare time I wanted to make a web to track the GPU price on a e-commerce. I am using PHP and the library Simple HTML DOM to parse the target HTML and it happen every hour from CRON Job.
(Yes, I knew I can make it in Selenium or others to scrape data more efficiently, but in this case just to challenge myself while learning it).
How it work is : Grab data and store it into database. Next, in other table it matches data from database : When the new price of a GPU is the same as latest price, it just update the date and time; If the new price is different with the latest, it make the latest price into old price and update some other things.
The scraping things is coded for a specific e-commerce website;
These variables placement are still scattered a little bit because I tried other
things;
It grab data every hour and logs the seconds on average 40-50, so my assumption is this processing time.
My question is : How can I make the code more efficient compared to my current method?
This is the code to grab the data :
<?php
error_reporting(E_ALL ^ E_WARNING);
require_once 'simple_html_dom.php';
// Database variables here
// ...
try {
$conn = new PDO("mysql:host=$servername;$dbname", $username, $password);
// set the PDO error mode to exception
$conn->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
// Get the URL List
$stmt = $conn->prepare("SELECT id,url FROM url_list");
$stmt->execute();
$url_list = $stmt->fetchAll(PDO::FETCH_COLUMN|PDO::FETCH_UNIQUE);
} catch(PDOException $e) {
echo "Connection failed: " . $e->getMessage();
}
// Scrap the data from a website then return as array
function get_gpu_info(string $targeturl, int $gpu_id)
{
$results = array();
$html = new simple_html_dom();
$html->load_file($targeturl);
if (!empty($html)) {
$div_class = $price = $stock = "";
$div_class = $html->find("#main-pdp-container", 0);
$out_of_stock = $html->find(".css-1igct5v-unf-quantity-editor__input[disabled]", 0);
$price = $div_class->find(".price", 0)->innertext;
$price_int = intval(preg_replace('/[^\d\,]+/', '', $price));
$stock = ($div_class->find(".css-1a29oke p b", 0)->innertext) ?: 0;
if (!empty($price)) {
$results = array(
'GPUID' => $gpu_id,
'PRICE' => $price,
'PRICEINT' => $price_int,
'STOCK' => $stock
);
} else {echo "Price not found";}
} else {echo "URL Not Found";}
return $results;
}
// Scrap every single data from the URL list found
$gpu_data = array_map('get_gpu_info', array_values($url_list), array_keys($url_list));
try {
$time = date("H:i:s");
$date = date("Y-m-d");
$stmt = $conn->prepare("INSERT INTO price_history (gpu_id, price, price_int, stock, update_time, update_date)
VALUES (:insert_gpu_id, :insert_price, :insert_price_int, :insert_stock, :insert_update_time, :insert_update_date)");
$stmt->bindParam(':insert_gpu_id', $insert_gpu_id);
$stmt->bindParam(':insert_price', $insert_price);
$stmt->bindParam(':insert_price_int', $insert_price_int);
$stmt->bindParam(':insert_stock', $insert_stock);
$stmt->bindParam(':insert_update_time', $time);
$stmt->bindParam(':insert_update_date', $date);
foreach ($gpu_data as $data => $val) {
$insert_gpu_id = $val['GPUID'];
$insert_price = $val['PRICE'];
$insert_price_int = $val['PRICEINT'];
$insert_stock = $val['STOCK'];
$stmt->execute();
$stmt2 = $conn->prepare("SELECT COUNT(gpu_id) FROM gpu_data WHERE gpu_id = :gpu_id");
$stmt2->bindValue(':gpu_id', $val['GPUID'], PDO::PARAM_INT);
$stmt2->execute();
$count = (int)$stmt2->fetchColumn();
if($count) {
$stmt4 = $conn->prepare("SELECT old_price, old_price_int, latest_price, latest_price_int, latest_update_time, latest_update_date FROM gpu_data WHERE gpu_id = :gpu_id");
$stmt4->bindParam(':gpu_id', $val['GPUID']);
$stmt4->execute();
$old_data = $stmt4->fetch(PDO::FETCH_ASSOC);
$old_price_int = $old_data['old_price_int'];
$old_latest_price_int = $old_data['latest_price_int'];
$old_price = $old_data['old_price'];
$get_date = $old_data['latest_update_date'];
$get_time = $old_data['latest_update_time'];
$combined_old_date_time = date('Y-m-d H:i:s', strtotime("$get_date $get_time"));
if($old_price_int == $insert_price_int) {
//print_r("Same price");
$stmt3 = $conn->prepare("UPDATE gpu_data SET
stock = :stock,
latest_update_time = :update_time,
latest_update_date = :update_date
WHERE gpu_id = :gpu_id");
} else {
//print_r("Different price");
$stmt3 = $conn->prepare("UPDATE gpu_data SET
old_price = :old_price,
old_price_int = :old_price_int,
old_datetime = :old_datetime,
latest_price = :price,
latest_price_int = :price_int,
stock = :stock,
latest_update_time = :update_time,
latest_update_date = :update_date
WHERE gpu_id = :gpu_id");
$stmt3->bindParam(':old_price', $old_price);
$stmt3->bindParam(':old_price_int', $old_price_int);
$stmt3->bindParam(':old_datetime', $combined_old_date_time);
$stmt3->bindParam(':price', $insert_price);
$stmt3->bindParam(':price_int', $insert_price_int);
print_r("Old price updated");
}
$stmt3->bindParam(':update_time', $time);
$stmt3->bindParam(':update_date', $date);
$stmt3->bindParam(':stock', $insert_stock);
$stmt3->bindParam(':gpu_id', $val['GPUID']);
$stmt3->execute();
//print_r("GPU Data with the same record found and has been updated");
} else {//print_r("ERROR: No GPU Data with that GPU ID has been found");
}
}
//print_r("Price record/s updated successfully");
} catch(PDOException $e) {
echo $sql . "<br>" . $e->getMessage();}
$conn = null;
?>
Thanks in advance!
It's likely you're taking a lot of time to load each page you're scraping. Probably some pages are a lot slower than others. Try doing something like this, to time your load_file() operations, to figure that out.
$loadStartTime = date();
$html->load_file($targeturl);
$loadEndTime = date();
echo $targeturl . ': ' . $loadEndTime - $loadStartTime . ' seconds to load.';
Your dom-romping code looks straightforward enough.
It seems doubtful you have many thousands of rows in your table, so your database stuff should be fast enough.

How to select data from 'logged in user' from database to a graph

I need help with some code. I want to select data from mySQL to a graph on my web page.
The data must be from the current logged in user, and when I select data to a card it works fine, but when I select it to a graph the graph disappears.
I'm using this code for the cards, and it works fine:
$sql = "SELECT energyexpenditure FROM energy4project WHERE user_id = '{$_SESSION["user_id"]}' ORDER BY time_stamp DESC LIMIT 1;";
This is the code for my current graph that don't show data based on logged in user:
<?php
header('Content-Type: application/json');
$host = "localhost";
$user = "`blabla";
$pwd = "blabla";
$db = "blabla";
$conn = new mysqli($host, $user, $pwd, $db);
// Check connection
if ($conn->connect_error) {
die("Connection failed: " . $conn->connect_error);
}
$sql = "SELECT energyexpenditure, time_stamp FROM energy4project ORDER BY time_stamp DESC LIMIT 7;";
$result = $conn->query($sql);
$data = array();
foreach ($result as $row) {
$data[] = $row;
}
mysqli_close($conn);
echo json_encode($data);
?>
When I implement the code from the cards in the graph code it doesn't work.
Why does the SELECT WHERE user = '{$_SESSION["user_id"]} not work in the graphs?
If I understood good your Mysql query return an empty result. Try to modify your code as follows:
if(!$result = $conn->query($sql)) {
echo "query error.";
die();
}
$data = array();
while($row = $result->fetch_array(MYSQLI_ASSOC))
{
$data[] = $row;
}
/* free result set */
$result->free();
/* close connection */
$conn->close();
//uncomment the below line if you want to check de result of your mysql query because it seems be good.
//var_dump($data); die(); echo "<pre>";
echo json_encode($data);
So if you don't have any mysql error, verify your database name, table name are correctly and if it has data in your table.
Reference: https://www.php.net/manual/en/mysqli-result.fetch-array.php
Regards

php/pdo insert into database mssql with arrays

I need some help
Is there a way to make this in PDO? https://stackoverflow.com/a/1899508/6208408
Yes I know I could change to mysql but I use a mssql server and can't use mysql. I tried some things but I'm not as good with PDO as mysql... It's hard to find some good examples of inserting array's into database with PDO. So quickly said I have a PDO based code connected to a mssql webserver.
best regards joep
I tried this before:
//id
$com_id = $_POST['com_id'];
//array
$mon_barcode = $_POST['mon_barcode'];
$mon_merk = $_POST['mon_merk'];
$mon_type = $_POST['mon_type'];
$mon_inch = $_POST['mon_inch'];
$mon_a_date = $_POST['mon_a_date'];
$mon_a_prijs = $_POST['mon_a_prijs'];
$data = array_merge($mon_barcode, $mon_merk, $mon_type, $mon_inch, $mon_a_date, $mon_a_prijs);
try{
$sql = "INSERT INTO IA_Monitor (Com_ID, Barcode, Merk, Type, Inch, Aanschaf_dat, Aanschaf_waarde) VALUES (?,?,?,?,?,?,?)";
$insertData = array();
foreach($_POST['mon_barcode'] as $i => $barcode)
{
$insertData[] = $barcode;
}
if (!empty($insertData))
{
implode(', ', $insertData);
$stmt = $conn->prepare($sql);
$stmt->execute($insertData);
}
}catch(PDOException $e){
echo $sql . "<br>" . $e->getMessage();
}
$conn = null;
The code below should fix your problems.
$db_username='';
$db_password='';
$conn = new \PDO("sqlsrv:Server=localhost,1521;Database=testdb", $db_username, $db_password,[]);
//above added per #YourCommonSense's request to provide a complete example to a code fragment
if (isset($_POST['com_id'])) { //was com_id posted?
//id
$com_id = $_POST['com_id'];
//array
$mon_barcode = $_POST['mon_barcode'];
$mon_merk = $_POST['mon_merk'];
$mon_type = $_POST['mon_type'];
$mon_inch = $_POST['mon_inch'];
$mon_a_date = $_POST['mon_a_date'];
$mon_a_prijs = $_POST['mon_a_prijs'];
$sql = "INSERT INTO IA_Monitor (Com_ID, Barcode, Merk, Type, Inch, Aanschaf_dat, Aanschaf_waarde) VALUES (?,?,?,?,?,?,?)";
try {
$stmt = $conn->prepare($sql);
foreach ($mon_barcode as $i => $barcode) {
$stmt->execute([$com_id, $barcode, $mon_merk[$i], $mon_type[$i], $mon_inch[$i], $mon_a_date[$i], $mon_a_prijs[$i]]);
}
} catch (\PDOException $e) {
echo $sql . "<br>" . $e->getMessage();
}
}
$conn = null;

Delete MySQL PHP

mysql_connect('localhost', 'root', '')
or die(mysql_error());
mysql_select_db('shuttle_service_system')
or die(mysql_error());
$insert="INSERT INTO inactive (ID_No, User_Password, First_Name, Last_Name, Email, Contact_Number)
VALUES('". $ID_No ."','". $UserPassword ."','". $FirstName ."','". $LastName ."','". $Email ."','". $ContactNumber ."')";
$result=mysql_query($insert);
$sql="DELETE FROM users WHERE ID_No = '$ID_No'";
$result2=mysql_query($sql);
if($result && $result2){
echo"Successful!";
} else {
echo "&nbsp Error";
}
Hi guys I have been stuck in delete function of MySQL, I have tried searching the net but when I ran my code it always goes to the else part which means there is an error, the insert is already okay but the delete is not.
PHP variables are allowed in double quotes. Hence try this,
$sql="DELETE FROM users WHERE ID_No = $ID_No";
Your first query was not properly escaped. Rewrite like
$insert="INSERT INTO inactive (`ID_No`, `User_Password`, `First_Name`, `Last_Name`, `Email`, `Contact_Number`)
VALUES('$ID_No','$UserPassword','$FirstName','$LastName','$Email','$ContactNumber')";
This (mysql_*) extension is deprecated as of PHP 5.5.0, and will be removed in the future. Instead, the MySQLi or PDO_MySQL extension should be used. Switching to PreparedStatements is even more better to ward off SQL Injection attacks !
First, use PDO.
Make your connection Database like this:
function connectToDB(){
$host='localhost';
try {
$user = 'username';
$pass = 'password';
$bdd = 'databaseName';
$dns = 'mysql:host='.$host.';dbname='.$bdd.'';
$options = array(
PDO::ATTR_PERSISTENT => true,
PDO::MYSQL_ATTR_INIT_COMMAND => "SET NAMES utf8",
PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION
);
return $connexion = new PDO($dns, $user, $pass, $options);
}catch ( Exception $e ) {
echo "Fail to connect: ", $e->getMessage();
die();
}
}
To delete something, here is an example:
function deleteUserWithId($ID_No){
$connexion = connectToDB();
try{
$connexion->exec('DELETE FROM users WHERE ID_No = '.$ID_No);
}catch(Exception $e){
echo "Error: ".$e->getMessage();
}
}
To insert something:
function addInactiveUser($UserPassword,$FirstName ,$LastName ,$Email,$ContactNumber){
$connexion = connectToDB();
$insert = $connexion->prepare('INSERT INTO inactive VALUES(:ID_No,
:User_Password,
:First_Name,
:Last_Name,
:Email,
:Contact_Number
)');
try {
// executing the request
$success = $insert->execute(array(
'ID_No'=>'',
'User_Password'=>$UserPassword,
'First_Name'=>$FirstName ,
'Last_Name'=>$LastName ,
'Email'=>$Email,
'Contact_Number'=>$ContactNumber
));
if($success)
// OK
else
// KO
}
catch (Exception $e){
echo "Error: ".$e->getMessage();
}
}
To make a select:
// If you want to display X user per pages for example
function getAllInactiveUsers($page, $numberInactiveUserPerPage){
$connexion = connectToDB();
$firstInactiveUser = ($page - 1) * $numberInactiveUserPerPage;
$selectAllInactiveUsers = $connexion->prepare('SELECT * FROM inactive ORDER BY ID_No DESC LIMIT '.$firstInactiveUser.','.$numberInactiveUserPerPage);
return $selectAllInactiveUsers ;
}
To get the results of this methods, just do something like this:
$inactiveUsers= getAllInactiveUsers(1,15); // for page 1, display 15 users
$inactiveUsers->execute();
while($row = $inactiveUsers->fetch(PDO::FETCH_OBJ)){
$id = $row->ID_No;
$first_name = $row->First_Name;
// etc...
}
Hope that's help :)
I am not sure if this helps you, but as an alternative you could delete the last entry in the table:
$delQ = mysql_query("SELECT * FROM ph ORDER BY id DESC LIMIT 1" );
while(( $ar = mysql_fetch_array($delQ)) !== false){
mysql_query("DELETE FROM ph WHERE id= $ar[id]");
}

PDO/SQL insert not working

got the following code. All values are gotten through javascript and then sent through ajax. The var_dump($array) at the end works and display all the correct values. So they are all passed through correctly. The catch error for the try method also never pops up. The values are not being inserted into the sql table. Whats wrong?
Thanks in advance.
$name = $_GET['name'];
$category = $_GET['category'];
$subCategory = $_GET['subCategory'];
$date = $_GET['date'];
$address = $_GET['address'];
$city = $_GET['city'];
$state = $_GET['state'];
$host = $_GET['host'];
$imagePath = $_GET['imagePath'];
$info = $_GET['info'];
//turn into array
$array = array();
$array[0]=$name;
$array[1]=$category;
$array[2]=$subCategory;
$array[3]=$date;
$array[4]=$address;
$array[5]=$city;
$array[6]=$state;
$array[7]=$host;
$array[8]='j';//$imagePath;
$array[9]=$info;
try {
$con = new PDO('mysql:host=localhost;dbname=test');
$insert = $con->prepare(" INSERT INTO create
(name,category,subCategory,date,address,city,state,host,imagePath,info)
VALUES (?,?,?,?,?,?,?,?,?,?) ");
$insert->execute($array);
}
catch(PDOException $e) { //try
echo 'error';
//echo 'ERROR: ' . $e->getMessage();
}
var_dump($array);
create is a reserved word in mysql so you need to quote it in backticks:
INSERT INTO `create` ...
To have PDO throw exceptions, you need to add that after you open your connection:
$con = new PDO('mysql:host=localhost;dbname=test');
$con->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
By the way, I assume that you are logging into your database with a username and a password as well (the second and third parameter of the PDO constructor)...

Categories