I am working on a cron script (in PHP PDS for a MySQL db) that will delete data older than 30 days from the database. Ive tried to do joining and on delete cascaded on my tables and statements but they haven't worked out so I went with a simpler approach here to get it working however I know this isn't the most processor effective method.
Please note that some of these tables (especially the links table) have hundreds of thousands of rows so that's why I wanted to be a little more clever than what I have here.
The table structure is easy, there is a table key table that has a time stamp and an id. That id is repeated in all tables as tablekey_id.
My current cron job is as follows.
/* Cron script to delete old data from the database. */
if (#include dirname(dirname(dirname(__FILE__))) . '/dbconn/website_connections.php') {
$conn = $connectionFromOtherFile;
$keyTable = 'table_key';
$linksTable = 'links';
$domainsTable = 'link_domains';
$terms = 'searched_domains';
$query = $conn->query("SELECT `id` FROM `$keyTable` WHERE `timestamp` < (NOW() - INTERVAL 30 DAY)");
$query->setFetchMode(PDO::FETCH_ASSOC);
while($row = $query->fetch()) {
$conn->exec("DELETE FROM `$linksTable` WHERE `tablekey_id` = $row[id]");
$conn->exec("DELETE FROM `$domainsTable` WHERE `tablekey_id` = $row[id]");
$conn->exec("DELETE FROM `$terms` WHERE `tablekey_id` = $row[id]");
$conn->exec("DELETE FROM `$keyTable` WHERE `id` = $row[id]");
}
}
Is there any way to get this into one statement? Please and thank you for the assistance.
EDIT: Heres what I ended up with.
/* Cron script to delete old data from the database. */
if (#include dirname(dirname(dirname(__FILE__))) . '/dbconn/website_connections.php') {
$conn = $connectionFromOtherFile;
$keyTable = 'table_key';
$linksTable = 'links';
$domainsTable = 'link_domains';
$terms = 'searched_domains';
$delete = $conn->prepare("DELETE tablekey, linktable, domaintable, searched
FROM `$keyTable` AS tablekey
LEFT OUTER JOIN `$linksTable` AS linktable on tablekey.id = linktable.tablekey_id
LEFT OUTER JOIN `$domainsTable` AS domaintable on tablekey.id = domaintable.tablekey_id
LEFT OUTER JOIN `$terms` AS searched on tablekey.id = searched.tablekey_id
WHERE tablekey.id = :tablekey_id");
$query = $conn->query("SELECT `id` FROM `$keyTable` WHERE `timestamp` < (NOW() - INTERVAL 30 DAY)");
$query->setFetchMode(PDO::FETCH_ASSOC);
while($row = $query->fetch()) {
$delete->execute(array('tablekey_id' => $row['id']));
}
}
You should be able to delete with one query like this:
DELETE kt, lt, dt, t
FROM `$keyTable` AS kt
LEFT OUTER JOIN `$linksTable` AS lt on kt.id = lt.tablekey_id
LEFT OUTER JOIN `$domainsTable` AS dt on kt.id = dt.tablekey_id
LEFT OUTER JOIN `$terms` AS t on kt.id = t.tablekey_id
WHERE kt.id = $row[id]
Note that I used outer joins here as I wasn't sure if the tables other than keyTable would be guaranteed to have records with that tablekey_id. If they are, you could use an inner join.
If tablekey_id is indexed, this should be fine even with thousands of rows. If you want to keep your database schema simple, I would keep 4 queries, deleting is not that CPU intensive.
If you really want one statement, you will have to complicate things and use a cascading deletion with foreign keys constraints:
CASCADE: Delete or update the row from the parent table, and automatically delete or update >the matching rows in the child table. Both ON DELETE CASCADE and ON UPDATE CASCADE are supported. Between two tables, do not define several ON UPDATE CASCADE clauses that act on the same column in the parent table or in the child table.
Source: http://dev.mysql.com/doc/refman/5.5/en/innodb-foreign-key-constraints.html
And this is admitting that your database is innoDB
first you want to retrieve the list of IDs like you are doing now, then you want to construct 4 queries like this:
DELETE FROM 'table' where 'id' IN ('id1', 'id2', 'id4')
this should prove a LOT more efficient then doing separate deletes. It would reduce the amount of queries to 5 in stead of 1+ 4N
As far as I know PDO only accepts one statement...
You could look at a trigger on deleting from the first (or you're main) table...
Sometimes I use this to solve my problems. Put the ids in string first.
$ids = array();
while($row = $query->fetch()) {
$ids[] = $row[id];
}
$ids_str = implode(',', $ids);
$conn->exec("DELETE FROM `$linksTable` WHERE `tablekey_id` in ($ids_str) ");
$conn->exec("DELETE FROM `$domainsTable` WHERE `tablekey_id` in ($ids_str)");
$conn->exec("DELETE FROM `$terms` WHERE `tablekey_id` in ($ids_str)");
$conn->exec("DELETE FROM `$keyTable` WHERE `id` in ($ids_str)");
Related
I need a PHP function that deletes duplicate rows from MySQL database table. For example, I have a table like this:
I tried the following without success:
$find = mysql_query("SELECT * FROM users");
while ($row = mysql_fetch_assoc($find))
{
$find_1 = mysql_query("SELECT * FROM users ");
if (mysql_num_rows($find_1) > 0) {
mysql_query("DELETE FROM users WHERE ID =$row[num]");
}
I need to delete this duplicate rows and Keep only one of them only using PHP (and not my SQL database). Is there a way to do this?
You can do this with a single query, without a php loop (which, obviously, does not do what you want here):
delete t
from mytable t
inner join (select user, min(num) num from mytable group by user) t1
on t.user = t1.user and t.num > t1.num
This deletes rows that have the same user, while retaining the row with the smallest num.
I'm trying to generate a list of events that a user is attending. All I'm trying to do is search through columns and comparing the userid to the names stored in each column using LIKE.
Right now I have two different events stored in my database for testing, each with a unique eventID. The userid i'm signed in with is attending both of these events, however it's only displaying the eventID1 twice instead of eventID1 and eventID2.
The usernames are stored in a column called acceptedInvites separated by "~". So right now it shows "1~2" for the userid's attending. Can I just use %like% to pull these events?
$userid = $_SESSION['userid'];
echo "<h2>My Events</h2>";
$myEvents = mysql_query("select eventID from events where acceptedInvites LIKE '%$userid%' ");
$fetch = mysql_fetch_array($myEvents);
foreach($fetch as $eventsAttending){
echo $eventsAttending['eventID'];
}
My output is just 11 when it should be 12
Change your table setup, into a many-to-many setup (many users can attend one event, and one user can attend many events):
users
- id (pk, ai)
- name
- embarrassing_personal_habits
events
- id (pk, ai)
- location
- start_time
users_to_events
- user_id ]-|
|- Joint pk
- event id ]-|
Now you just use joins:
SELECT u.*
FROM users u
JOIN users_to_events u2e
ON u.id = u2e.id
JOIN events e
ON u2e.event_id = e.id
WHERE u.id = 11
I'm a bit confused by your description, but I think the issue is that mysql_fetch_array just returns one row at a time and your code is currently set up in a way that seems to assume $fetch is filled with an array of all the results. You need to continuously be calling mysql_fetch_array for that to happen.
Instead of
$fetch = mysql_fetch_array($myEvents);
foreach($fetch as $eventsAttending){
echo $eventsAttending['eventID'];
}
You could have
while ($row = mysql_fetch_array($myEvents)) {
echo $row['eventID'];
}
This would cycle through the various rows of events in the table.
Instead of using foreach(), use while() like this:
$myEvents = mysql_query("SELECT `eventID` FROM `events` WHERE `acceptedInvites` LIKE '".$userid."'");
while ($fetch = mysql_fetch_array($myEvents))
{
echo $fetch['eventID'];
}
It will create a loop like foreach() but simpler...
P.S. When you make a MySQL Query, use backticks [ ` ] to ensure that the string is not confused with MySQL functions (LIKE,SELECT, etc.).
I have the following 3 tables in the database.
Programs_Table
Program_ID (Primary Key)
Start_Date
End_Date
IsCompleted
IsGoalsMet
Program_type_ID
Programs_Type_Table(different types of programs, supports a dropdown list in the form)
Program_type_ID (Primary Key)
Program_name
Program_description
Client_Program_Table
Client_ID (primary key)
Program_ID (primary key)
What is the best way to find out how many clients are in a specific program (program type)?
Would the following SQL statement be the best way, or even plausible?
SELECT Client_ID FROM Client_Program_Table
INNER JOIN Programs_Table
ON Client_Program_Table.Program_ID = Programs_Table.Program_ID
WHERE Programs_Table.Program_type_ID = "x"
where "x" is the Program_type_ID of the specific program we're interested in.
OR is the following a better way?
$result = mysql_query("SELECT Program_ID FROM Programs_Table
WHERE Program_type_ID = 'x'");
$row = mysql_fetch_assoc($result);
$ProgramID = $row['Program_ID'];
$result = mysql_query("SELECT * FROM Client_Program_Table
WHERE Program_ID = '$ProgramID'");
mysql_num_rows($result) // returns how many rows of clients we pulled.
Thank you in advance, please excuse my inexperience and any mistakes that I've made.
Here is how you can do it:
<?php
// always initialize a variable
$number_of_clients = 0;
// escape the string which will go in an SQL query
// to protect yourself from SQL injection
$program_type_id = mysql_real_escape_string('x');
// build a query, which will count how many clients
// belong to that program and put the value on the temporary colum "num_clients"
$query = "SELECT COUNT(*) `num_clients` FROM `Client_Program_Table` `cpt`
INNER JOIN `Programs_Table` `pt`
ON `cpt`.`Program_ID` = `pt`.`Program_ID`
AND `pt`.`Program_type_ID` = '$program_type_id'";
// execute the query
$result = mysql_query($query);
// check if the query executed correctly
// and returned at least a record
if(is_resource($result) && mysql_num_rows($result) > 0){
// turn the query result into an associative array
$row = mysql_fetch_assoc($result);
// get the value of the "num_clients" temporary created column
// and typecast it to an intiger so you can always be safe to use it later on
$number_of_clients = (int) $row['num_clients'];
} else{
// query did not return a record, so we have no clients on that program
$number_of_clients = 0;
}
?>
If you want to know how many clients are involved in a program, you'd rather want to use COUNT( * ). MySQL (with MyISAM) and SQL Server have a fast way to retrieve the total number of lines. Using a SELECT(*), then mysql_num_rows leads to unnecessary memory ressources and computing time. To me, this is the fastest, though not the "cleanest" way to write the query you want:
SELECT
COUNT(*)
FROM
Client_Program_Table
WHERE
Program_ID IN
(
SELECT
Program_ID
FROM
Programs_Table
WHERE
Program_type_ID = 'azerty'
)
Why is that?
Using JOIN make queries more readable, but subqueries often prove to be computed faster.
This returns a count of the clients in a specific program type (x):
SELECT COUNT(cpt.Client_ID), cpt.Program_ID
FROM Client_Program_Table cpt
INNER JOIN Programs_Table pt ON cpt.Program_ID=pt.Program_ID
WHERE pt.Program_type_ID = "x"
GROUP BY cpt.Program_ID
I have 2 tables. One (artWork) with all the data I want to pull from, including 2 cols of id's. The other (sharedWork) has the same 2 id cols that are also in the first -- but none of the essential data I want to echo out. Objective: use the id's in both table to filter out row in the first (artWork). See below in the code what I tried that didn't work
I also tried to figure out an inner join that would accomplish the same. No luck there either. Wondering which would be the best approach and how to do it.
thanks
Allen
//////// Get id's first ///////////
$QUERY0="SELECT * FROM artWork WHERE user_id = '$user_id' ";
$res0 = mysql_query($QUERY0);
$num0 = mysql_num_rows($res0);
if($num0>0){
while($row = mysql_fetch_array($res0)){
$art_id0 = $row['art_id'];
}
}
$QUERY1="SELECT * FROM shareWork WHERE user_id = '$user_id' ";
$res1 = mysql_query($QUERY1);
$num1 = mysql_num_rows($res1);
if($num1>0){
while($row = mysql_fetch_array($res1)){
$art_id = $row['art_id'];
}
}
$art_id2 = array_merge($art_id0, $art_id1);
foreach ($art_id2 as $art_id3){
$QUERY="SELECT * FROM artWork WHERE art_id = '$art_id3' ";
// echo "id..".$art_id0;
$res = mysql_query($QUERY);
$num = mysql_num_rows($res);
if($num>0){
while($row = mysql_fetch_array($res)){
$art_title = $row['art_title'];
$art_id = $row['art_id'];
etc................and so on
.........to....
</tr>";
}
}
}
Don't query your database inside a loop unless you absolutely have to.
Everytime you query the database, you're using disk I/O to read through the database and return your record. Disk I/O is the slowest read on a computer, and will be a massive bottleneck for your application.
If you run larger queries upfront, or at least outside of a loop, you will hit your disk less often, improving performance. Your results from larger queries will be held in memory, which is considerably faster than reading from disk.
Now, with that warning out of the way, let's address your actual problem:
It seems you're trying to grab records from artWork where the user is the primary artist, or the user was one of several artists to work on a group project. artWork seems to hold the id of the primary artist on the project whereas shareWork is probably some sort of many-to-many lookup table which associates user ids with all art projects they were a part of.
The first thing I should ask is whether or not you even need the first query to artWork or if the primary artist should have a record for that art_id in shareWork anyway, for having worked on the project at all.
If you don't need the first lookup, then the query becomes very easy: just grab all of the users art_ids from shareWork table and use that to lookup the his or her records in the main artWork table:
SELECT artWork.*
FROM artWork
WHERE art_id IN
(SELECT art_id
FROM shareWork
WHERE user_id = $user)
If you do need to look in both tables, then you just add a check in the query above to also check for that user in the artWork table:
SELECT artWork.*
FROM artWork
WHERE
user_id = $user
OR art_id IN
(SELECT art_id
FROM shareWork
WHERE user_id = $user)
This will get you all artWork records in a single query, rather than.. well, a lot of queries, and you can do your mysql_fetch_array loop over the results of that one query and be done with it.
I have a MySQL database called "bookfeather." It contains 56 tables. Each table has the following structure:
id site votes_up votes_down
The value for "site" is a book title. The value for "votes_up" is an integer. Sometimes a unique value for "site" appears in more than one table.
For each unique value "site" in the entire database, I would like to sum "votes_up" from all 56 tables. Then I would like to print the top 25 values for "site" ranked by total "votes_up".
How can I do this in PHP?
Thanks in advance,
John
You can do something like this (warning: Extremely poor SQL ahead)
select site, sum(votes_up) votes_up
from (
select site, votes_up from table_1
UNION
select site, votes_up from table_2
UNION
...
UNION
select site, votes_up from table_56
) group by site order by sum(votes_up) desc limit 25
But, as Dav asked, does your data have to be like this? There are much more efficient ways of storing this kind of data.
Edit: You just mentioned in a comment that you expect there to be more than 56 tables in the future -- I would look into MySQL limits on how many tables you can UNION before going forward with this kind of SQL.
Here's a PHP code snip that should get it done.
I have not tested it so it might have some typos and stuff, make sure you replace DB_NAME
$result = mysql_query("SHOW TABLES");
$tables = array();
while ($row = mysql_fetch_assoc($result)) {
$tables[] = '`'.$row["Tables_in_DB_NAME"].'`';
}
$subQuery = "SELECT site, votes_up FROM ".implode(" UNION ALL SELECT site, votes_up FROM ",$tables);
// Create one query that gets the data you need
$sqlStr = "SELECT site, sum(votes_up) sumVotesUp
FROM (
".$subQuery." ) subQuery
GROUP BY site ORDER BY sum(votes_up) DESC LIMIT 25";
$result = mysql_query($sqlStr);
$arr = array();
while ($row = mysql_fetch_assoc($result)) {
$arr[] = $row["site"]." - ".$row["sumVotesUp"];
}
print_r($arr)
The UNION part of Ian Clelland answer can be generated using a statement like the following. The table INFORMATION_SCHEMA.COLUMNS has a column TABLE_NAME to get all tables.
select * from information_schema.columns
where table_schema not like 'informat%'
and column_name like 'VOTES_UP'
Join all inner SELECT with UNION ALL instead of UNION. UNION is doing an implicit DISTINCT (on oracle).
The basic idea would be to iterate over all your tables (using a SQL SHOW TABLES statement or similar) in PHP, then for every table, iterate over the rows (SELECT site,votes_up FROM $table). Then, for every row, check the site against an array that you're building with sites as keys and votes up as values. If the site is already in the array, increment its votes appropriately; otherwise, add it.
Vaguely PHP-like pseudocode:
// Build an empty array for use later
$votes_array = empty_array();
// Get all the tables and iterate over them
$tables = query("SHOW TABLES");
for($table in $tables) {
$rows = query("SELECT site,votes_up FROM $table");
// Iterate over the rows in each table
for($row in $rows) {
$site = $row['site'];
$votes = $row['votes_up'];
// If the site is already in the array, increment votes; otherwise, add it
if(exists_in_array($site, $votes_array)) {
$votes_array[$site] += $votes;
} else {
insert_into_array($site => $votes);
}
}
}
// Get the sites and votes as lists, and print out the top 25
$sorted_sites = array_keys($votes_array);
$sorted_votes = array_values($votes_array);
for($i = 0; $i < 25; $i++) {
print "Site " . $sorted_sites[$i] . " has " . $sorted_votes[$i] . " votes";
}
"I allow users to add tables to the database." - I hope all your users are benevolent and trustworthy and capable. Do you worry about people dropping or truncating tables, creating incorrect new tables that break your code, or other things like that? What kind of security do you have when users can log right into your database and change the schema?
Here's a tutorial on relational database normalization. Maybe it'll help.
Just in case someone else that comes after you wants to find what this could have looked like, here's a single table that could do what you want:
create database bookfeather;
create user bookfeather identified by 'bookfeather';
grant all on bookfeather.* to 'bookfeather'#'%';
use bookfeather;
create table if not exists book
(
id int not null auto_increment,
title varchar(255) not null default '',
upvotes integer not null default 0,
downvotes integer not null default 0,
primary key(id),
unique(title)
);
You'd vote a title up or down with an UPDATE:
update book set upvotes = upvotes + 1 where id = ?
Adding a new book is as easy as adding another row:
insert into book(title) values('grails in action')
I'd strongly urge that you reconsider.