I have the following rows in the database inside url column:
http://some_url/something/34123122.json
http://some_url/something/53124322.json
http://some_url/something/22214322.json
And I want to retrieve them in some function, like this (pseudocode):
function retrieve($ids) {
return $this->fetchAll("SELECT * FROM table WHERE url IN $ids");
}
The problem is that $ids parameter MUST BE an array with ids from those urls only, like:
array(
[0] => 34123122
[1] => 22214322
)
So I have to do something in this function so that I can retrieve rows with urls that contain those ids. How can I do that? Urls can change, but the /******.json ending has always the same pattern.
I don't want to make another query selecting the beginning of the url, it will slow down the application too much.
The proper way to do this is to query only the part of the data that you are interested in - the number. So, you receive an instant +10 to intelligence from performing a quest nearby and you determine that you could create another column to save that number. Your table looks like this now:
CREATE TABLE mytable (
id int not null auto_increment,
url varchar(255) not null,
json_number int not null,
PRIMARY KEY(id),
INDEX(json_number)
) ENGINE = InnoDB;
Before inserting into the table, you use integer sanitizing filter to extract the number from the URL without wasting too much time
Given a URL like this: http://some_url/something/34123122.json you can easily extract the number like this:
$url = 'http://some_url/something/34123122.json';
$number = filter_var($url, FILTER_SANITIZE_NUMBER_INT);
echo $number; // echoes 34123122
And now your query is trivial, you check the json_number column which is also indexed at the same time.
Naturally, you can ignore all I wrote and try other answers which are ugly hacks and worst of all - they're all full table scans.
you will have to use regex in mysql, change your function:
function retrieve($ids) {
$regex = '('.implode('|', $ids).').json$';
return $this->fetchAll("SELECT * FROM table WHERE url REGEXP '$regex'");
}
Note: this is not an optimal solution for large tables. I would suggest you to create an id field in your table and if all ids are unique then you can make id a primary key. Also whenever you insert in that table take out the id part from url and insert it into the id field. In that way you can skip regex. If you are willing to create an id field, then you can execute the following query to update your current table id field:
mysql> update your_table_name set id=replace(substring_index(url, '/', -1), '.json', '');
I do not know if this is a neat solution, but it should work.
function getData($ids) {
foreach($ids as $item) {
$str[] = $item . ".json";
}
$where = "";
foreach($str as $item) {
$where .= "url LIKE '%$item' OR ";
}
return substr("SELECT * FROM table WHERE " . $where, 0, -4);
}
$ids = array(34123122, 53124322, 22214322);
echo getData($ids);
Result:
SELECT * FROM table WHERE url LIKE '%34123122.json' OR url LIKE '%53124322.json' OR url LIKE '%22214322.json'
I think this should do it. Of course you have to run the query aswell.
Related
I have a TXT file with no punctuation between them. I would like to shred this file by the table column widths in the database and save it.
Let me tell you this step by step…
I’m creating a table in the database with my tabloolustur.php page. The column numbers and column widths of the tables I create will not be the same.
There are no punctuation marks between the TXT file data. First, I want to split the TXT file rows by column width.
$result = $baglanti->prepare("SHOW COLUMNS FROM customers where Field NOT IN('id')");
$result->execute();
$colcount = $result->columnCount()-1;
$columLen = array();
foreach($result as $key => $col){
preg_match('/\d+/', $col['Type'], $len);
$len = (isset($len[0]))? $len[0] : '';
$fieldname = $col['Field'];
$columLen[$fieldname] = $len;
}
For this, I get the number of columns and column widths with the code.
Then, I separate the data with commas with the following function.
function txtBol($metin, $genislik){
$parcala=array();
foreach ($genislik as $sutunadi => $lenght)
{
$parcala[$sutunadi] = substr($metin, 0, $lenght);
$metin = substr($metin, $lenght);
}
return $parcala;
}
I also get column names with the following code. (ps: to use in a query)
$KolAdi = $baglanti->query("SHOW COLUMNS FROM customers where Field NOT IN('id')");
$KolAdi->execute();
$colonAdi= $KolAdi->fetchAll(PDO::FETCH_COLUMN);
$colonAdi=implode(',', $colonAdi);
It prints the data i split correctly when printing it to the screen. So far, so good. But I can’t create the right query with PDO. How should I create the query? (ps: Table column names and column widths are not the same. There will be different numbers and width columns for each table)
I would appreciate it if you could help. I proceeded by implementing some solutions from your site.
Table:
id
name
cev1
cev2
cev3
1
MARTIN EDEN
AAAAAA
BBBBB
CCCC
txt:
MARTIN EDEN........AAAAAABBBBBDDDD
Assuming a valid PDO connection in $PDO, you can do the whole job like this.
It reads the column data once and uses that to create a query to prepare and a regular expression to extract the data.
$table = 'customer';
$txtFile = 'cust.txt';
// No need to prepare this since there's no untrusted data here.
$result = $pdo->query("show columns from `$table` where Field <> 'id'");
// Get column names and widths in arrays
while($columns = $result->fetch(PDO::FETCH_ASSOC)) {
$colNames[] = $columns['Field'];
preg_match('/\d+/', $columns['Type'], $len);
$colLengths[] = $len[0]??'';
}
// Create regex to extract data from lines:
// Looks like this: /(.{20})(.{10})(.{5})/
$regex = '/(.{'.implode('})(.{',$colLengths).'})/u';
// Create skeleton query with table name and field names and placeholders
// Looks like this: INSERT customer (name,addr1,postcode) VALUES (?,?,?)
$query = "INSERT $table (`".implode('`,`', $colNames).'`) VALUES ('.str_repeat('?,', count($colNames)-1).'?)';
// Read text file
if ($fileData = file($txtFile)) {
// Prepare the query only once
$stmt = $pdo->prepare($query);
foreach ($fileData as $row) {
// Get the data using the regex from above
preg_match($regex, $row, $rowData);
// Remove the first row of the regex matches - see PHP manual for why
array_shift($rowData);
// Now execute the prepared query using the data extracted by the regex.
$stmt->execute($rowData);
}
}
Note that this codes assumes that the table consists of a series of contiguous columns from which it can extract column widths, and no other columns except id, which it ignores. If your actual table structure is different you'll need to modify the SHOW COLUMNS query to omit those columns, or modify the code that extracts the column data to extract only the relevant columns.
{Edit]
Updated the code to include /u UTF-8 modifier on the regex, and to wrap the column names in the INSERT query in back ticks.
you can create a table like this and you don't need to check a number of fields by rows
let me know if this useful
CREATE TABLE new_table (
id INT NOT NULL AUTO_INCREMENT,
row_id INT NULL DEFAULT 0,
row_field_name VARCHAR(50) NULL DEFAULT NULL COMMENT 'Index of ',
row_value VARCHAR(255) NULL DEFAULT NULL,
PRIMARY KEY (id));
I know that varchar type of columns are not incrementable, but I want to do something like this:
When a duplicated value is inserted, the new one will be tagged with a number at the end.
For example, we have post-name on our column in the database. Another value post-name is entered, so the next is going to be post-name-2, followed by post-name-3.
I have programmed something in php but its not very convenient.
$post_url_ = $post_url." %";
$stmt_check1 = $this->conn->prepare("SELECT * FROM post WHERE post_url LIKE :post_url ");
$stmt_check2 = $this->conn->prepare("SELECT * FROM post WHERE post_url = :post_url ");
$stmt_check1->bindparam(":post_url",$post_url_);
$stmt_check2->bindparam(":post_url",$post_url);
$stmt_check1->execute();
$stmt_check2->execute();
$rows1 = $stmt_check1->rowCount();
$rows2 = $stmt_check2->rowCount();
if($rows1<=0 && $rows2==1) {
$repeat_no = $rows1+1;
$post_url = $post_url."-$repeat_no";
}
if($rows1>0){
$repeat_no = $rows1+1;
$post_url = $post_url."-$repeat_no";
}
Instead of trying to create a complicate process to keep the correct name, just add separated field version
Then for UI proporse just concatenate both
SELECT CONCAT(post , '-', version)
You can use do...while loop to check next numbers. It isn't performance problem, becuase you will use that only on save new record.
Before loop, create counter (initial by 0).
In loop - if counter > 0, add number to end of url
Increase counter
Check if url exists in database (while results from db > 0)
I wrote a function which makes a random id makeid(); Just to ensure the id is unique I have a SQL statement which checks if the id already exists.
$does_id_exist = mysql_query("SELECT COUNT(*) AS count FROM signups WHERE affid='$affid'");
if(mysql_num_rows($does_id_exist) == 1)
{
#loop function and perform query again
}
else
{
#insert record
}
So I'm having trouble with looping the function. How do I loop my function makeid() and perform the $does_id_exist check to ensure that each ID is unique.
--UPDATE-- Just to clarify- My code makes an id like YES#281E But before I INSERT this id into the users record. I just need to verify IF any other user already has this id. IF another user has this id that event must trigger my function to create a new id e.g. WOW!29E3 and again check the sql/query to ensure no other user has that id. Continue to loop if fails or end and INSERT if the id is available.
You can either just use a primary key on your database table, or something like this:
<?php
// the id to insert
$newId = null;
// populate with results from a SELECT `aff_id` FROM `table`
$currentIds = array();
// prepopulate
for( $i=0; $i<100000; $i++ )
{
$currentIds[] = "STRING_" + rand();
}
// generate at least one id
do
{
$newId = "STRING_" + rand();
}
// while the id is taken (cached in $currentIds)
while( in_array($newId, $currentIds) );
// when we get here, we have an id that's not taken.
echo $newId;
?>
Output:
STRING_905649971 (run time 95ms);
I'd definitely not recommend running the query repeatedly. Perhaps a final check before you insert, if your traffic volume is high enough.
Do not do COUNT(*), because you do not need to know how many rows is there (it should be 0 or 1 as you need Id unique), so even DB finds your row it will still be checking for the whole table to count. You really care if you got 1 row, so just select for row with that ID and this sufficient. You should also avoid using rand() - this does not help as you see and you cannot predict how many loops you can do before you find "free slot". use something predictable, like date prefix, or prefix incremented each day. anything that would help you narrow the data set. But for now (pseudocode!):
$id = null;
while( $id == null ) {
$newId = 'prefix' . rand();
mysql_query("SELECT `affid` FROM `signups` WHERE `affid`='${newId}'");
if( mysql_num_rows() == 0) {
$id = newId;
break;
}
}
Ensure you got DB indexed, to speed things up.
EDIT: I do agree that any cache would be useful to speed things up (you can add it easily yourself based on #Josh example), still, I think this is fixing at wrong place. If possible rethink the way you generate your ID. It does not really need to be auto increment, but something more predictable than rand() would help you. If your ID does not need to be easily memorable and it is not any security concern to have them sequential, maybe use numbers with other base than 10 (i.e. using 26 would use all digits + letters so you'd end with PREFIX-AX3TK, so string as you want, and at the same time you would easily be able to quickly generate next Id
There's a mysql database that stores ids and names, when users are creating names with a form they can create existent names since unique ids are the ids such as:
d49f2c32f9107c87c7bb7b44f1c8f297 name
2fa9c810fbe082900975f827e8ed9408 name
what i want to do is saving the second "name" -> "name(1)" when inserting into database.
So far what I've got is this as the idea
lets say the name entered is 'name'
$input = 'name';
select the name we want to check from mysql database
mysql_query(SELECT * FROM `table` WHERE `name` = '$input');
if the result exists, then insert as $input.'(1)'
my question is what if name exists, and name(1) also exists, then how can i add the name(2) there...
You could return the number of people with that name in the database, then add 1 to that number.
SELECT COUNT(id) FROM table WHERE name LIKE '$input(%)');
$i = 1;
$sourceName = $name;
while( sql "SELECT COUNT(*) FROM table WHERE name = '$name'" ) {
$name = $sourceName.' ('.$i.')';
$i++;
}
At this point you have the final $name (with $i counting the iteration)
Something like this should do the trick:
SELECT * FROM table WHERE name = '$input' OR name LIKE '$input(%)'
Note that for this to work, you'd need to escape any percent signs in $input in the LIKE clause, otherwise they'll be treated as wildcards.
Use a regex that checks for integers between two parentheses at the end of the string. If there exists an integer, add 1 to it.
You could also attempt to do it the other way around, make name field unique and try to input it in a while loop, if it fails add ($i) and do $i++ every iteration.
//edit:
The problem with using solutions that do a like comparison is that you will get false positives, for instance $hi% will also count hippie. Which gives you unnecessary (1) additions!
My application requires the user to enter their business name, which the application will automatically create into a unique identifier to be used in URLs, ie
"Bob's Cafe" will become "bobs-cafe"
But if there are duplicate names I would like the application to add a number so if there is already a "bobs-cafe" we will use "bobs-cafe-1" and likewise if there is already a "bobs-cafe-1" we will use "bobs-cafe-2"
Ive used explode and also looked at a regular expressions but I dont know the best way to approach this.
Im stuck in being able to grab the number and incrementing it and returning the string
Adding to Sarfraz's answer, you might want to find it using a LIKE statement
SELECT `urlIdentifier` FROM `businesses` WHERE `urlIdentifier` LIKE `bobs-cafe%`
which will get all the bobs-cafe items - that way, if you get 5 rows you know you have
bobs-cafe
bobs-cafe-1
bobs-cafe-2
bobs-cafe-3
bobs-cafe-4
and that you'll need to add bobs-cafe-5
EDIT - Or this:
SELECT count(*) as `howMany` FROM `businesses` WHERE `urlIdentifier` LIKE `bobs-cafe%`
Now your result object ( or array ) will have the total number:
echo $resultObject->howMany; // number of bobs-cafe sql found
Why not to add an autoincrement number to every identifier in the URL?
Just like SO does:
stackoverflow.com/questions/2895334/php-application-check-name-is-unique-if-not-append
so, you have both unique identifier and a business name.
This is even better because they are free to change their business name, without changing an identifier.
As for your question it's very simple. Just for the PHP practice:
if (/* you've found the name is already non unique and have the max one in the $id */) {
$parts = explode("-",$id);
if (isset($parts[1])) $newid = $parts[0]."-".($parts[1]+1);
else $newid = $parts[0]."-1";
}
Assuming $user is already in the form bobs-cafe
function username_exists ( $user ) {
$result = mysql_query("SELECT name FROM table WHERE $name LIKE '$user%' ");
$count = mysql_num_rows($result);
if ( $result ) {
$num = $count+1;
return username_exists ( $user.'-'.$num ) ;
} else {
return $user;
}
}