Parse rows of data from string and insert into database - php

I am new to PHP apologize if this is an unworldly question. I am receiving a packet of data with dynamic length on the tcp/ip socket. The packet looks like this:
Palace1,radio,location1,location2,location3,location4,GSMId:Palace2,radio,location1,location2,location3,location4,GSMId:Palace3,radio,location1,location2,location3,location4,GSMId
You can see after the GSMId I have a colon to separate one reports. The length of the packet could be anything.
My task is that I want to chop this packet after every colon (:) and want to save each report in Database.
Right now what I am doing to chop each packet is:
$string = "Palace1,radio,location1,location2,location3,location4,GSMId:Palace2,radio,location1,location2,location3,location4,GSMId:Palace3,radio,location1,location2,location3,location4,GSMId";
$countString = substr_count($string, ":");
$NumberOfReports = $countString + 1;
echo $NumberOfReports."\n";
echo $countString."\n";
$chopPacket = explode(':' , $string);
foreach($chopPacket as $value)
{
$Report = $value;
echo $Report."\n";
writeToDataBase($Report);
}
DataBAse Code :
function writeToDataBase($Report)
{
date_default_timezone_set("Europe/London");
$date = date('Y-m-d H:i:s');
$counter = 0;
$DecodingData = explode("," , $Report);
if ($DecodingData > 0) {
$username = "user";
$password = "password";
$host = "localhost";
$connector = #mysql_connect($host, $username, $password) or die("Unable to connect");
$selected = #mysql_select_db("gsmdb", $connector) or die("Unable to connect");
$importSQL = "INSERT INTO gsmclient_test VALUES('".$counter."','".$DecodingData[0]."','".$DecodingData[1]."','".$DecodingData[2]."','".$DecodingData[3]."','".$DecodingData[4]."', '".$DecodingData[5]."','".$DecodingData[6]."','".$date."')";
mysql_query($importSQL) or die(mysql_error());
mysql_close($connector);
}
}
The code above is only saving the first report in database.

You will need to perform 2 separate e plosions to prepare the data.
Use a single, prepared statement to ensure that your query is stable and secure.
Here is a similar mysqli technique as a comparison.
I don't like the look of that $counter. Your database should have the row identifier as an autoincremented primary key and the date column should have a default value of CURRENT_TIMESTAMP so that that column doesn't need to be declared during insert queries.
I don't know what your column names are, so I cannot add them to my sql.
We shouldn't see mysql_ functions anymore; use mysqli_ or pdo functions.
Code:
$string = "Palace1,radio,location1,location2,location3,location4,GSMId:Palace2,radio,location1,location2,location3,location4,GSMId:Palace3,radio,location1,location2,location3,location4,GSMId";
$stmt = $mysqli->prepare("INSERT INTO gsmclient_test VALUES(?,?,?,?,?,?)";
$stmt->bind_param('ssssss', $pal, $rad, $loc1, $loc2, $loc3, $loc4);
foreach (preg_split('/,GSMId:?/', $string, -1, PREG_SPLIT_NO_EMPTY) as $rowString) {
[$pal, $rad, $loc1, $loc2, $loc3, $loc4] = explode(',', $rowString, 6);
$stmt->execute();
}

Related

PHP stock market analyser app will only insert first line of historic data into database

Heres basic connection:
<?php
$connect = mysql_connect('localhost', 'root', '');
if (!$connect) {
die('Could not connect to database!');
}
mysql_select_db('stockmarket', $connect);
?>
The main code reads each stock in tickerMaster.php which is:
YHOO (YAHOO)
F (FORD)
NFLX (NETFLIX)
ADBE (ADOBE)
GE (GE)
Then each stock is read and used to download each stock's history from yahoo finance in the main code of the app
This is the Netflix(NFLX) file that gets created for example when the main code runs. The other 4 stocks get created the same with obviously different numbers.
This is the Date, Open, High, Low, Close, Volume, Adj Close:
2015-08-14,124.959999,125.00,123.00,123.389999,8631900,123.389999
2015-08-13,120.989998,125.730003,119.339996,123.730003,15221000,123.730003
2015-08-12,121.470001,122.449997,118.660004,120.510002,13822500,120.510002
2015-08-11,120.010002,123.760002,120.00,122.739998,11047600,122.739998
2015-08-10,126.089996,126.50,121.510002,123.029999,13370600,123.029999
The main code below will create a table for each stock properly. However will
only insert the first row from up above into the table and not loop through and
insert all the rows for each stock.
MAIN CODE:
<?php
include('includes/connect.php');
function createURL($ticker) {
$currentMonth = date("n");
$currentMonth = $currentMonth - 1;
$currentDay = date("j");
$currentYear = date("Y");
return "http://real-chart.finance.yahoo.com/
table.csv?s=$ticker&d=$currentMonth&e=$currentDay&f=$currentYear
&g=d&a=7&b=10&c=2015&ignore=.csv";
//Aug. 10 2015
function getCVSFile($url, $outputFile) {
$content = file_get_contents($url);
$content = str_replace(
"Date, Open, High, Low, Close, Volume, Adj Close",
"",
$content
); //above string comes with stock data
//downloaded from yahoo. I don't want
$content = trim($content);
file_put_contents($outputFile, $content);
}
function fileToDatabase($txtFile, $tableName) {
$file = fopen($txtFile, "r");
while (!feof($file)) {
$line = fgets($file);
$pieces = explode(",", $line);
$date = $pieces[0];
$open = $pieces[1];
$high = $pieces[2];
$low = $pieces[3];
$close = $pieces[4];
$volume = $pieces[5];
$adj_clo = $pieces[6];
$amount_change = $close - $open;
$percent_change = ($amount_change/$open) * 100;
$sql = "SELECT * FROM $tableName";
$result = mysql_query($sql);
if (!$result) {
$sql_2 = "CREATE TABLE $tableName (date DATE, PRIMARY KEY(date), open FLOAT, high FLOAT, low FLOAT, close FLOAT, volume INT, amount_change FLOAT, percent_change FLOAT, adj_clo FLOAT)";
mysql_query($sql_2);
}
$sql_3 = "INSERT INTO $tableName (date, open, high, low, close, volume, amount_change, percent_change, adj_clo)
VALUES ({$date}, {$open}, {$high}, {$low}, {$close}, {$volume}, {$amount_change}, {$percent_change}, {$adj_clo} )";
mysql_query($sql_3);
}
fclose($file);
}
function main() {
$mainTickerFile = fopen("tickerMaster.txt", "r");
while(!feof($mainTickerFile)) {
$companyTicker = fgets($mainTickerFile);
$companyTicker = trim($companyTicker);
$fileURL = createURL($companyTicker);
$companyTxtFile = "txtFiles/".$companyTicker."txt";
getCVSFile($fileURL, $companyTxtFile);
fileToDatabase($companyTxtFile, $companyTicker);
}
}
main();
?>
Any help much appreciated on why only the first row is only getting inserted into database.
This is the summarized answer based on the comments above:
1) You have to check, if the while loops are iterating over all rows.
2) Check if there are any problems in your SQL queries. (Check the return values of mysql_query calls and check the return value of mysql_error function if mysql_query(...) returns false.
3) Check if keyword date can be used as a fieldname in a mysql table.
4) Finally mysql_error told us that new datasets cannot be inserted because there are rows with duplicate values of date column (the primary key of the table). You check why date is 0000-00-00 or define another primary key, if the values of date are valid.

Using $_GET method will not store the right value in the database

I've been trying to build a comments section for my website using PHP5, however when I try to use the $_GET method it stores the wrong value in the database column.Originally I just used $_GET['pageid'] but that would store NULL instead of an integer and that was because $_GET['pageid'] had a string datatype.
I then began to use (int)$_GET['pageid'] which would store an integer but the wrong value. It would store 0 every time. I know other people have had the same problem as me but I found no viable solution.
I am able to retrieve comments from my database and post them to their respective pages using $_GET['pageid'] after manually inserting integers in the pageID column.
If it makes any difference I am using Webmatrix 3 with MySQLi and PHP5. Thanks in advance!
if (isset($_POST['submit']))
{
$pageid = (int)$_GET['pageid'];
$username = $_SESSION['username'];
$comment = $_POST['comment'];
$query = "INSERT INTO comments (pageID, username, comment) VALUES (?, ?, ?)";
$statement = $databaseConnection->prepare($query);
$statement->bind_param('iss', $pageid, $username, $comment);
$statement->execute();
$statement->store_result();
if ($statement->error)
{
die('Database query failed: ' . $statement->error);
}
$creationWasSuccessful = $statement->affected_rows == 1 ? true : false;
if ($creationWasSuccessful)
{
header ("Location: index.php");
}
}
If you CAST a string to integer it will become 0 , if the string has numbers in the beginning , only those numbers will be converted to integer , an also if a string begins with a space Ex:- " 12word" , the result will be 12 .
check these examples
$str = 'Hello World 12';
$int = (int)$str;
echo $int; // 0
$str = 'Hello World';
$int = (int)$str;
echo $int; // 0
$str = 'Hello 12 World';
$int = (int)$str;
echo $int; // 12
$str = '12Hello World';
$int = (int)$str;
echo $int; // 12
Check this link to get more Ideas . http://www.phpf1.com/tutorial/php-string-to-int.html

PHP. Read some characters in some strings of a file to write in database

I have a file like this:
FG 09097612 DN 6575 HL 879797
BHC 09097613 DN 6576 HL 879798
FG 09097614 DN 6577 IOPPP 879799
FG 09097614 DN 6577 IOPPP 879800
with its logic that never changes line by line, it is always the same logic.
I would create an array taking the first 2 characters as a variable "nation", then the first 8 characters as a var "prize", then 2 other characters as "player" and so on and create a record in the database for each line.
I am using this code (THE CODE IN TEH EDIT ABOVE IS CHANGED), but not being a csv with delimitation with comma or tab I don't know how to do.
ini_set("auto_detect_line_endings", 1);
$current_row = 1;
$handle = fopen("upload/import.txt", "r");
while ( ($csv_data = fgetcsv($handle, 10000, "\t") ) !== FALSE )
{
$number_of_fields = count($csv_data);
if ($current_row == 1) {
}
else {
}
fclose($handle);
}
I want to put theese var in a record of database, each var in each column.
What do you recommend?
Obviously I can not change the original txt file.
HOW TO SAVE IN DATABASE?
If I use this code (from one answer above):
$lines = file("upload/import.txt");
foreach($lines as $lineNum => $line ) {
$nation = trim(substr($line, 0, 4)); // get first four characters as nation and remove spaces
$prize = trim(substr($line, 4, 8)); // get 5th-12th characters as prize and remove spaces
$player = trim(substr($line, 12, 2)); // get 13th-14th characters as player and remove spaces
I use this code:
$dbhandle = odbc_connect("Driver={SQL Server Native Client 11.0};
Server=$myServer;Database=$myDB;", $myUser, $myPass)
or die("Couldn't connect to SQL Server on $myServer");
$query = "INSERT INTO TEST (nation) VALUES ('".$nation."')";
echo "<br>Inserted: ".$nation."<br>";
$result = odbc_exec($dbhandle, $query);
But it seems to me that this code is too heavy to be done in a foreach? Is not it?
1.In this situation, delimiter changes because of the formatting so I suggest you treat it as a string but not a csv.
FG 09097612 DN 6575 HL 879797
BHC 09097613 DN 6576 HL 879798
FG 09097614 DN 6577 IOPPP 879799
FG 09097614 DN 6577 IOPPP 879800
$lines = file("upload/import.txt");
foreach($lines as $lineNum => $line ) {
$nation = trim(substr($line, 0, 4)); // get first four characters as nation and remove spaces
$prize = trim(substr($line, 4, 8)); // get 5th-12th characters as prize and remove spaces
$player = trim(substr($line, 12, 2)); // get 13th-14th characters as player and remove spaces
}
2.Or if you insist on using csv parser, you should make consecutive spaces into one before you actually use fgetscsv:
$tempfile = "tmp/temp".microtime()."csv"; // a temp folder where you have write authority, `microtime()` here is used to generate a unique filename.
$content = file_get_contents("upload/import.txt");
while(strpos($content," ") !== false) {
// while consecutive spaces exist
$content = str_replace(" ", " ", $content);
}
file_put_contents($tempfile, $content);
Then you can treat it as a normal csv file with space delimiter like this:
$handle = fopen($tempfile, "r");
$current_row = 1;
while ( ($csv_data = fgetcsv($handle, 10000, " ") ) !== FALSE )
{
$number_of_fields = count($csv_data);
// ...
}
fclose($handle);
After you finish this , delete the temp file like this:
unlink($tempfile);
And it is better to do the insert only once than create and run the insert query in a foreach loop, so instead of adding
$query = "INSERT INTO TEST (nation) VALUES ('".$nation."')";
$result = odbc_exec($dbhandle, $query);
in each loop, which produces
INSERT INTO TEST (nation) VALUES ('FG');
INSERT INTO TEST (nation) VALUES ('BHC');
INSERT INTO TEST (nation) VALUES ('FG');
INSERT INTO TEST (nation) VALUES ('FG');
...
, it is more reasonable to create the query like this
$lines = file("upload/import.txt");
$dbhandle = odbc_connect("Driver={SQL Server Native Client 11.0};
Server=$myServer;Database=$myDB;", $myUser, $myPass)
or die("Couldn't connect to SQL Server on $myServer");
$query = "INSERT INTO TEST (nation) VALUES \n";
$row = array(); // query for each line
foreach($lines as $lineNum => $line ) {
$nation = trim(substr($line, 0, 4)); // get first four characters as nation and remove spaces
$prize = trim(substr($line, 4, 8)); // get 5th-12th characters as prize and remove spaces
$player = trim(substr($line, 12, 2)); // get 13th-14th characters as player and remove spaces
$row []= "(".$nation.")";
}
$query .= implode(",\n",$row).";";
$result = odbc_exec($dbhandle, $query);
If you echo $query, you should get something like this:
INSERT INTO TEST (nation) VALUES
('FG'),
('BHC'),
('FG'),
('FG');
which is much more light to run insert query every line.
PS: Please be careful that mysql has a limited query length. If the query is longer than max length, you will get an error , something like 'Mysql has gone away...'. I haven't much experience in using ms-sql server but there should be a same limit.
In this situation, you should split the query in a proper way. For example, run and clear the query every 10000 lines like this:
$lines = file("upload/import.txt");
$dbhandle = odbc_connect("Driver={SQL Server Native Client 11.0};
Server=$myServer;Database=$myDB;", $myUser, $myPass) or die("Couldn't connect to SQL Server on $myServer");
$query = "INSERT INTO TEST (nation) VALUES \n";
$row = array(); // query for each line
foreach($lines as $lineNum => $line ) {
$nation = trim(substr($line, 0, 4)); // get first four characters as nation and remove spaces
$prize = trim(substr($line, 4, 8)); // get 5th-12th characters as prize and remove spaces
$player = trim(substr($line, 12, 2)); // get 13th-14th characters as player and remove spaces
$row []= "('".$nation."')";
if($lineNum % 10000 == 9999){
// run and reproduce the query every 10000 lines
$query .= implode(",\n",$row).";";
// put 'echo $query;' here would help you understand the design
$result = odbc_exec($dbhandle, $query);
// It is better to check the result here if it success
// The query has been run so it should be initialized again, so is $row.
$query = "INSERT INTO TEST (nation) VALUES \n";
$row = array(); // query for each line
}
}
$query .= implode(",\n",$row).";";
$result = odbc_exec($dbhandle, $query);
Answer to 'I need now to check if in the file there are lines with the same nation, in this example "FG" and sum the $prize of each "FG" and save only the total in prize and not all the lines of FG? '
Of course you can do that, since I don't know your table or how you would like to save your other data so I will just provide a sample with only nation and price:
$lines = file("upload/import.txt");
$dbhandle = odbc_connect("Driver={SQL Server Native Client 11.0};
Server=$myServer;Database=$myDB;", $myUser, $myPass)
or die("Couldn't connect to SQL Server on $myServer");
$query = "INSERT INTO TEST (nation, prize) VALUES \n";
$row = array(); // data for each line
foreach($lines as $lineNum => $line ) {
$nation = trim(substr($line, 0, 4)); // get first four characters as nation and remove spaces
$prize = trim(substr($line, 4, 8)); // get 5th-12th characters as prize and remove spaces
$row[$nation] = !empty($row[$nation])? $row[$nation] + (int)$prize : 0 ; // make sure prize is an interger in your file
}
// Since there are not so much nations in the world, I don't think it is necessary to worry about the max query length
$query_row = array(); // query for each line
foreach($row as $nation => $sum_prize){
$query_row []= "('".$nation."','".$sum_prize."')";
}
$query .= implode(",\n",$query_row).";";
$result = odbc_exec($dbhandle, $query);
Try this:
$handle = fopen("upload/import.txt", "r");
$current_row = 1;
while(!feof($handle)){
$str=fgets($handle);//read one line of the file
$str=str_replace(" "," ",$str);//reduce all consecutive spaces to single space
$arr=explode(" ",$str);
if ($current_row == 1) {
}
else {
}
}
fclose($handle);
Now the array $arr will contain the strings {"FG","09097612,"DN","6575","HL","879797"} (after reading the first line). You can use this array to access the values and insert them in DB.
EDIT:
I understand that you want to have an array called "nation" which will contain values {"FG","BHC","FG","FG"}, and the same for prize and other variables. Try this code:
$nation=array();
$prize=array();
$player=array();
$handle = fopen("upload/import.txt", "r");
$current_row = 1;
while(!feof($handle)){
$str=fgets($handle);//read one line of the file
$str=str_replace(" "," ",$str);//reduce all consecutive spaces to single space
$arr=explode(" ",$str);
//now insert values in respective arrays
array_push($nation,$arr[0]);
array_push($prize,$arr[1]);
array_push($player,$arr[2]);
//and so on
if ($current_row == 1) {
}
else {
}
}
fclose($handle);
Now you can use the arrays $nation, $prize and $player. You can create arrays for the other values in the same manner.
Hope this helps.

SUM lines of a file for total and make query with total. PHP MSSQL

I have a file like this
FG 09097612 DN 6575 HL 879797
BHC 09097613 DN 6576 HL 879798
FG 09097614 DN 6577 IOPPP 879799
FG 09097614 DN 6577 IOPPP 879800
and I import it in mysql with
$lines = file("upload/import.txt");
$dbhandle = odbc_connect("Driver={SQL Server Native Client 11.0};
Server=$myServer;Database=$myDB;", $myUser, $myPass)
or die("Couldn't connect to SQL Server on $myServer");
$query = "INSERT INTO TEST (nation) VALUES \n";
$row = array(); // query for each line
foreach($lines as $lineNum => $line ) {
$nation = trim(substr($line, 0, 4)); // get first four characters as nation and remove spaces
$prize = trim(substr($line, 4, 8)); // get 5th-12th characters as prize and remove spaces
$player = trim(substr($line, 12, 2)); // get 13th-14th characters as player and remove spaces
$row []= "(".$nation.")";
}
$query .= implode(",\n",$row).";";
$result = odbc_exec($dbhandle, $query);
What I need now is to check if in the file there are lines with the same nation, in this example "FG" and sum the $prize of each "FG" and save only the total in prize and not all the lines of FG?
If you need to do this in php I would save the rows in an array keyed by the nation, and each time check if that key already exists and if so just increment the prize.
Something like this:-
<?php
$lines = file("upload/import.txt");
$dbhandle = odbc_connect("Driver={SQL Server Native Client 11.0};
Server=$myServer;Database=$myDB;", $myUser, $myPass)
or die("Couldn't connect to SQL Server on $myServer");
$query = "INSERT INTO TEST (nation, prize) VALUES \n";
$row = array(); // query for each line
foreach($lines as $lineNum => $line )
{
$nation = trim(substr($line, 0, 4)); // get first four characters as nation and remove spaces
if (array_key_exists($nation, $row))
{
$row[$nation]['prize'] += $prize;
}
else
{
$prize = trim(substr($line, 4, 8)); // get 5th-12th characters as prize and remove spaces
$player = trim(substr($line, 12, 2)); // get 13th-14th characters as player and remove spaces
$row[$nation]= array('nation'=>$nation, 'prize'=>$prize);
}
}
array_walk($row, function($v, $k){return "(".$v['nation'].", ".$v['prize'].")";});
$query .= implode(",\n",$row).";";
$result = odbc_exec($dbhandle, $query);
?>
However if this was being done in MySQL I would be tempted to just have the nation as a unique key on the database and add ON DUPLICATE KEY SET prize=prize + VALUES(prize) to the end of the insert query.
You could write really complicated php code to do this. Or, you could change your strategy. First, let me point out that you can just use load data infile to read the data from a file.
The new strategy is to read the data into a staging table and then copy it to the final table. Basically:
read data into test_staging table (using php or `load data infile`)
insert into test(nation, prize)
select nation, sum(prize)
from test_staging
group by nation;
drop table test_staging;
So if you need it in PHP, I would suggest using a named array like this:
$insertLines = array();
foreach($lines as $lineNum => $line ) {
$nation = trim(substr($line, 0, 4)); // get first four characters as nation and remove spaces
$prize = trim(substr($line, 4, 8)); // get 5th-12th characters as prize and remove spaces
insertLines[$nation] += $prize;
}
foreach($insertLines as $lineNation => $linePrice ) {
// The creation of the insert into the database goes here.
}
And please check the lenghts.
I think the plazer is in 14 to 15 ?
$player = trim(substr($line, 13, 2));

PHP - editing a string from a database

I have a column in a database which presents as the following;
< channel\01273123456:d24gf3fm >
I need to export the number from this string, the first "< channel\" is always the same, but the end ID is always unique.
I currently have the following code, but cannot think, nor find what I need to export the number.
//connection
$dbhandle = mysql_connect($hostname, $username, $password)
or die ("Unable to connect to Database");
echo "Connected\n";
//select DB
$selected = mysql_select_db("asterisk", $dbhandle)
or die("Could not select DB");
$result = mysql_query("SELECT * FROM TABLE");
while($row = mysql_fetch_array($result)) {
echo echo "channel:".$row{'channel'}."\n";
}
Hope someone could help, this is driving me crazy.
Substr seems to be the function you are seeking for:
substr and strpos.
Try something like this (depends on if you need the whole last part of the string or only the part until the ':':
$yourString = substr($row['channel'], strpos($row['channel'], '\'));
Would give you the whole substring. If you do not need the part after the the ':' you need no split yourString again from 0 to position of ':'.
$yourString2 = substr($yourString, 0, strpos($subString, ':'));
This should do the trick and as requested by tadman and it's a function :)
public function cleanMyString($string)
{
// remove channel garbage
// 01273123456:d24gf3fm >
$string = substr($string, 10, (strlen($string) - 10));
// remove space and >
// 01273123456:d24gf3fm
$string = substr($string, 0, -2);
// split on colon
// $colons[0] = 01273123456
// $colons[1] = d24gf3fm
$colons = explode(':', $string);
// first item in array is the channel
echo 'Channel: '.$colons[0].'<br><br>';
// second item is ID
echo 'ID: '.$colons[1];
}
// string, yay!
cleanMyString('< channel\\01273123456:d24gf3fm >');
have you tried these?
SELECT Right (LEFT(`your_column`, 21),11) as numbers
from asterisk
or this
SELECT SUBSTRING_INDEX(SUBSTRING_INDEX(`your_column`,'\', -1),':', 1) as numbers
from asterisk

Categories