MYSQL Queries - my brain is exploding - php

this is my first question here, sorry if i'm breaking any etiquette.
I'm kinda into coding, but sometimes my brain is hard to swallow path of logic steps.
Currently i'm working on my own small web-app, where i have public events, and i'm making my own guestlist.
By the time, i've solved these things:(i think so)
Getting from Facebook page all events;
From current event view get all attendees;
Live search function and ordering array alphabetically.
To Do:
Complete CHECK button function - when checked, one get removed from
lis;
Other analysis functions.
Problem:
Currently i'm getting all attendees from JSON string, then converting to array, putting it all in database. I can't decide on SQL logic.
I have all list with people - json->array->db then it reads from db and show wich is checked wich one not, like comparing with table that is from JSON.
Current algorithm is - getting json, and in foreach cycle, everytime i load it writes in DB, using INSERT IGNORE it ignores if it's same userid, so i have db of all atendees.
How to arrange my database? I'm thinking about making tables:
guests - USERID ; EVENT ID; NAME; [for huge list of all people]
checkins - USERID; CHECKEDEVENTID; DATETIME; [for getting stats]
My goal is to make "Checking In" door-app, so in the end i see, that those and those users are attending more on those kind of events, than these one...
So how could i make like stats, like - EVENT - attended Y people of X, and more global SQL queries, like, USER Y came to EVENTS A,B,C. Or, most checkings happening at timespan [probably some bars or chart]....
Should i make for each event new table to store all guest there to see all atendee statistics, and checking table for checkin stats?

For the what you refer to as the "Check" feature, it sounds like you want (roughly*) the following tables:
create table users
(
userid float NOT NULL,
username varchar(64)
);
create table events
(
eventid float NOT NULL,
eventname varchar(64),
eventstart date,
eventlength float
);
create table checkin_activity
(
userid float not null,
eventid float not null,
checkin_time date
);
* This is a highly simplified database schema. You'll want to make sure you add the necessary keys, constraints, etc., and make sure the data types on your columns are appropriate. I didn't give that much thought with this quick example.
Using the entries in the USERS and EVENTS tables, you'll populate the CHECKIN_ACTIVITY table with what you refer to as the "Check" button. You can join queries against these tables as needed to run reports and so on.
NOTE: You mention:
Current algorithm is - getting json, and in foreach cycle, everytime i load it writes in DB, using INSERT IGNORE it ignores if it's same userid, so i have db of all atendees
You should avoid writing to the database within a for loop (into a table I didn't account for above; let's call it the EVENT_ATTENDEES table). Instead, build an INSERT ALL query and executing it once so you're not hitting the database's transaction handler n times.
INSERT ALL
INTO event_attendees (eventid, name) VALUES (1, 'John')
INTO event_attendees (eventid, name) VALUES (1, 'Jane')
INTO event_attendees (eventid, name) VALUES (1, 'Colin')
SELECT * FROM dual;
This is especially important if this kind of load is something you'll be doing often.

Related

Most efficient way of storing daily page views, as well as a total count

There allot of discussion on storing page views for an article or video in a database, but I can't seem to find any information on storing daily page views. For example DeviantArt shows you the past 15 or so days and how many page views each one got in a little graph, as well as the total page view for the profile.
You can see above exactly what I'm trying to achieve, and DeviantArt do it on a huge scale as they get millions of hits.
I am using CodeIgniter on PHP and MySQL
Usually what you do in these cases is have a logging table with 1 record per view. You have a cron run at certain intervals (daily) to summarize the logging data into another summary table. At the most basic level, you would have 1 record per day in the summary table with the total view count. Since the historical data never changes, you don't need to do updates. That gives you at most 365 records per item per year. Not a lot of records, so it scales well.
This scales better than having a single record for each day and updating the count. You could run into contention/locking issues if you are trying to update the same record multiple times. By doing an INSERT DELAYED and using a MyISAM table, your logging table can scale very well.
You can then select from the summary table to get the daily counts, and you could select from the logging table if you want to display the current days count.
If you want to keep your logging table small, when your cron run, you can rename the logging table and create a new, empty one. Then process the renamed logging table.
RENAME TABLE logging TO logging_old;CREATE TABLE logging LIKE logging_old;
If you really need to scale, you can use a replication trick to log views extremely fast. You use a BLACK HOLE table type for logging data, which replicates to another server where the table is MyISAM or InnoDB.
You could have a row for each day then increment the count when it is viewed.
INSERT INTO views (day,views) VALUES (CURDATE(),1) ON DUPLICATE KEY UPDATE views=views+1;
And a PHP script...
mysql_query("INSERT INTO views (day,views) VALUES (CURDATE(),1) ON DUPLICATE KEY UPDATE views=views+1;");
This uses a table called 'views' and 2 rows 'day' and 'views'.
See this article for more information.
I don't use codeIgniter but here's how you can do this in straight php.
Basically, you can either use sessions to record each visit to your site and record their page requests or you can simply record each hit on the page. The first option gives you more information (how many unique hits were made, how many hits are from returning visitors, etc.) but takes more programming to set up and requires user session management. So, for your purposes I'll explain option 2.
SQL:
Table 'pageViews'
Id - Integer, auto Increment, primary key
page - varchar(150)
datetime - datetime
On the page you're monitoring, you would add:
$query2 = "INSERT INTO `pageViews` (page, datetime) VALUES ('".$_SERVER["SCRIPT_NAME"]."', ".date("Y-M-d H:I:s").")";
$result2 = //insert DB connection etc. to apply update to DB
You can then fetch the rows from the db based on the page and the datetime stamp, count the results and display it using html5 canvas. Or, because each hit has a datetime, you can break it down further into hours of the day etc.
I hope this helps.
To modify this for simple daily counters, add an extra column called 'views' that is just an integer. Then on the page put a check to see if there is a row for the current page for today in the db. If so, then get the current views value and increment it and resale to the DB.
For the total views count I would probably create another table with the same columns but not with the date-time column. Then on your page, you increment the views value for the current page in the new table too.
This would be lightening fast. The trade-off is the loss of detail. So, here's the MySQL and PHP for this 3rd option:
MySQL:
Table 'dailyViews'
Id - Integer, auto Increment, primary key
views - integer
page - varchar(150)
datetime - datetime
Table 'totalViews'
Id - Integer, auto Increment, primary key
views - integer
page - varchar(150)
PHP:
// Search the DB for an entry today for this page in dailyViews
$query = "SELECT views FROM `dailyViews` WHERE `page`='".$_SERVER["SCRIPT_NAME"]."' AND `datetime`=".date("Y-M-d H:I:s")."";
$result = //insert your db connection etc to execute the query
if (mysqli_num_rows($result)==1) {
$resultArray = mysqli_fetch_array($result, MYSQLI_ASSOC);
$views = $resultArray['views']+1;
$query2 = "UPDATE `dailyViews` SET `views`=".$views." WHERE `page`='".$_SERVER["SCRIPT_NAME"]."' AND `datetime`=".date("Y-M-d H:I:s")."";
$result2 = //insert DB connection etc. to apply update to DB
} else if (mysqli_num_rows($result)<1) {
$query2 = "INSERT INTO `dailyViews` (views, page, datetime) VALUES (1, '".$_SERVER["SCRIPT_NAME"]."', ".date("Y-M-d H:I:s").")";
$result2 = //insert DB connection etc. to apply update to DB
} else {
// there is more than one entry for this page and date in the DB. An error has ccurred
}
// Then just get the total views for this page and increase it by 1.

mySQL: index table of 2 other table ID's

Ok, here's what I wanna do.
Table1(globalID [primary key, autoincremented] (is a global id i would use in PHP))
Table2 & table3(ID* [foreign primary key, referencing to globalID], other columns after this, different for each of the tables 2 and 3);
What I wannt to know is if there is a way, and how, to make it so that when I insert a row in table2 or table3 the globalID in talbe1 gets populated and it's value also inserted in table2 or table3 accordingly as an ID. I assume it would be accomplished with a trigger, but triggers are not my forte so if that's the only way it can be done please provide an example.
I was also thinking of expanding it to this scenario:
Table1(globalID [primary key, autoincremented] (is a global id i would use in PHP),
OtherID [an UUID]);
Table2 & table3(ID* [foreign primary key, referencing to OtherID], other columns after this, different for each of the tables 2 and 3);
But in that case there is still the almost nonexistent possibility that 2 users might generate the same UUID while inserting a new row in some table. I was wondering if i could avoid that by making the keys generated by the sql server automatically without me having to code it PHP server side.
If anyone has had this problem worked out and can also point out other things i need to watch out for or take into account please point them out. Also please provide an example of the solution.
EDIT:
#John B.
EDIT BY TosheX
thank you for the reply. I have been reading up on this in the meantime and i already have a solution outside of SQL (in php). Basically I use Yii PHP framework in which I create a model of the table, and create an active record to populate a new row. now when I do that the ID in table1 is generated, and afterwards I still have a variable that points to that populated row and I can just read the generated ID since it's automatically fetched (without having to check for the last record in the database, which may return the record someone created milliseconds after me). Afterwards I just need to create a row inside the appropriate table2 or 3 and assign the already generated value for the ID.
i found the idea here:
http://www.yiiframework.com/forum/index.php/topic/14593-alternate-for-mysql-insert-id/
I really wanted an inbuilt solution in the database, but since I have experience with databases and do know enough about triggers (enough to not like them :D ) i know it's a very tricky code to get right so I was wondering if there are alternatives to auto-populate or something.
Still, I really appreciate your response, since you took your time and all. I have been working in msSQL more too, and I find logic in what you said. UUID in mySQL is the same as GUID in msSQL and I have considered that too as you can see.
Taking all this into account I will go with the solution i found, but I will accept your answer since you did try to help out, and you did bring some good ideas to the table.
Thanks again.
I'm a mssql person, but I dont think it is possible, unless you can create a "BEFORE INSERT" trigger for your db (and even then it will be problematic), because:
Whatever the ID of Tables 2 & 3 is, be it an integer or a guid, these are primary keys: Being a primary key, this column will not allow nulls, yet what you require is essentially to create new rows in these tables, with this column null, and then generate a new row in Table1, and use the new value in that table to update the new row in either Table2 or Table3.
Therefore, you'll probably need to either:
Do a solution in code (ie outside of SQL) whereby you cause the creation of a new row in Table1 first, then use this value when subsequently creating in Table2/3, or:
On tables2/3 have a autoincremented column for your primary key lets call it "PK", then have ID as a separate ordinary FK but which allows nulls, plus a trigger as follows (though you might need to fiddle with the syntax a little bit, because I am a mssql person, and I don't know your db's syntax) :-(
Also this example is designed to work with a multiple row insert, and to achieve this uses a cursor, and my syntax for cursors is dodgy at best even in mssql!
CREATE TRIGGER trg_Table2 ON dbo.Table2
FOR INSERT /* might be AFTER INSERT etc */
AS BEGIN
SET nocount on;
Declare ICursor Scroll For SELECT PK FROM Inserted;
Declare #PK int,
#GlobalID int,
#ID guid;
OPEN ICursor
FETCH Next From ICursor INTO #PK
WHILE ##Fetch_Status BEGIN
INSERT Table1 (OtherID) VALUES (NewID());
SELECT #GlobalID = ##Identity
SELECT #ID = OtherID FROM Table1 WHERE GlobalID = #GlobalID
UPDATE Table2 SET ID = #ID WHERE PK = #PK
FETCH Next FROM ICursor INTO #PK
END
Close ICursor
Deallocate ICursor
END
Hope this helps

How to implement a global incrementing id for multiple tables in SQL?

I decided back when I was coding to have different tables for each type of content. Now I am stuck solving this. Basically my notification system ranks the newest content by its timestamp currently. This is inaccurate however because there is a small chance that someone would submit content at the same time as another person, and incorrect ranking would occur.
Now if I had all my content in a single table, I would simply rank it by an auto-incrementing variable. Is there a way to implement this auto-increment integer across multiple tables (e.g. When something is inserted into table1, id=0, something is inserted into table2, id=1). Or do I have to recode all my stuff into a single table.
NOTE:
The reason I have content in multiple tables is because its organized and it would reduce load stress. I don't really care about the organization anymore, because I can just access the data through a GUI I coded, I'm just wondering about the load stress.
EDIT:
I'm using PHP 5 with MySQL.
Your question, particularly the need for ID spanning over multiple tables, is clearly signalizing that your database design needs change. You should make one table for all content types (as a generalization), with autoincrementing ID. Then, for each particular content type, you can define other table (equivalent of inheritance in OOP) with extra fields, and foreign key pointing to the basic table.
In other words, you need something like inheritance in SQL.
You can create a table with auto increment id just to keep track of ids. Your program would do an insert on that table, get the id, use it as necessary.
Something along the lines of:
function getNextId() {
$res = mysql_query("INSERT INTO seq_table(id) VALUES (NULL)");
$id = mysql_insert_id();
if ($id % 10 == 0) {
mysql_query("DELETE FROM seq_table");
}
return $id;
}
Where seq_table is a table that you've to create just to get the ids. Make it a function so it can be used whenever you need. Every 10 ids generated I delete all generated ids, anyway you don't need them there. I don't delete every time since it would slow down. If another insert happen in the meantime and I delete 11 or more records, it doesn't affect the behaviour of this procedure. It's safe for the purpose it has to reach.
Even if the table is empty new ids will just keep on growing since you've declared id as auto-increment.
UPDATE: I want to clarify why the ID generation is not wrapped in a transaction and why it shouldn't.
If you generate an auto id and you rollback the transaction, the next auto id, will be incremented anyway. Excerpt from a MySQL bug report:
[...] this is not a bug but expected behavior that happens in every RDBMS we know. Generated values are not a part of transaction and they don't care about other statements.
Getting the ID with this procedure is perfectly thread safe. Your logic after the ID is obtained should be wrapped in a transaction, especially if you deal with multiple tables.
Getting a sequence in this way isn't a new concept, for instance, the code of metabase_mysql.php which is a stable DB access library has a method called GetSequenceNextValue() which is quite similar.
In a single table, you could have a field for the content type and clustered index that includes the content type field. This effectively keeps all of one content type in one place on the disc, and another content type in another place, etc. (It's actually organised into pages, but this physical organisation is still true.)
Assuming that each content type has the same fields, this would likely meet your needs and behave similarly to multiple tables. In some cases you may even find that, with appropriate indexes, a single table solution can be faster, more convenient and maintainable, etc. Such as trying to create global unique identifiers across all content types.
If you're unable to merge these back into a single table, you could create a central link table...
CREATE TABLE content_link (
id INT IDENTITY(1,1), -- MS SQL SERVER syntax
content_type INT,
content_id INT -- The id from the real table
)
As you insert into the content tables, also insert into the link table to create your globally unique id.
More simply, but even more manually, just hold a single value somewhere in the database. Whenever you need a new id, use that centrally stored value and increment it by one. Be sure to wrap the increment and collection in a single transaction to stop race conditions. (This can be done in a number of ways, depending on your flavor of SQL.)
EDIT
A couple of MySQL example lines of code from the web...
START TRANSACTION;
INSERT INTO foo (auto,text)
VALUES(NULL,'text'); # generate ID by inserting NULL
INSERT INTO foo2 (id,text)
VALUES(LAST_INSERT_ID(),'text'); # use ID in second table
COMMIT TRANSACTION;
Personally, I'd actually store the value in a variable, commit the transaction, and then continue with my business logic. This would keep the locks on the tables to a minimum.
You could have a separate ID table, insert into that, and use the newly-inserted ID.
e.g.
CREATE TABLE ids (INT UNSIGNED AUTO INCREMENT PRIMARY KEY, timeadded DATETIME);
In the script:
<?php
$r = mysql_query('INSERT INTO ids (timeadded) VALUES (NOW())');
$id = mysql_insert_id();
mysql_query("INSERT INTO someOtherTable (id, data) VALUES ('$id', '$data)");
Add error checking etc. to taste.
The MySQL manual states:
The ID that was generated is maintained in the server on a
per-connection basis. This means that the value returned by the
function to a given client is the first AUTO_INCREMENT value generated
for most recent statement affecting an AUTO_INCREMENT column by that
client. This value cannot be affected by other clients, even if they
generate AUTO_INCREMENT values of their own. This behavior ensures
that each client can retrieve its own ID without concern for the
activity of other clients, and without the need for locks or
transactions.
(Source) So I don't think concerns about ACID complians are a problem.

Optimal query - any way to avoid hundreds of queries in loop?

I want to save top 100 results for a game daily.
I have two tables: Users, Stats.
Users db has two columns: userID(mediumint(8) unsigned AUTO_INCREMENT) and userName(varchar(500)).
Stats db has three columns: time(date), userID(mediumint(8) unsigned AUTO_INCREMENT), result(tinyint(3) unsigned).
Now, every time I execute query (daily) I have array of 100 results wit user name. So here's what I need to do:
For every result in array:
get user id from Users table - or if user doesn't exist in User
table than create entry and get id;
Insert to Stats table current date, user id and and result.
What would be the most optimal way to do this in php and mysql. Is there a way to avoid having 200 queries in 'for' loop.
Thanks for your time guys.
200 queries par day is nothing. You can leave everything as is and there will be not a single problem.
why do you have an array with user names where you ought to have user ids instead?
Mysql INSERT query support multiple VALUES statement. So, you could assemble a string like
VALUES (time,userid,result),(time,userid,result),(time,userid,result)
and run it at once.
Also note that userID should be int, not medium int and in the stats table it shouldn't be autoincremented.
Use a pair of prepared statements. You'll still be running each one 100 times, but the query itself will already be parsed (and even cached by the DB server), it'll just have 100 different sets of parameters to be run with, which is quite efficient.

Best way to INSERT autoincrement field? (PHP/MySQL)

I have to insert data into two tables, Items and Class_Items. (A third table, Classes is related here, but is not being inserted into).
The primary key of Items is Item_ID, and it's an auto-incrementing integer. Aside from this primary key, there are no unique fields in Items. I need to know what the Item_ID is to match it to Classes in Class_Items.
This is all being done through a PHP interface. I'm wondering what the best way is to insert Items, and then match their Item_ID's into Class_Items. Here are the two main options I see:
INSERT each Item, then use mysql_insert_id() to get its Item_ID for the Class_Items INSERT query. This means one query for every Item (thousands of queries in total).
Get the next Autoincrement ID, then LOCK the Class_Items table so that I can just keep adding to an $item_id variable. This would mean just two queries (one for the Items, one for the Class_Items)
Which way is best and why? Also, if you have an unlisted alternative I'm open to whatever is most efficient.
The most efficient is probably going to be to use parameterized queries. That would require using the mysqli functions, but if you're to the point of needing to optimize this kind of query you should think about being there anyway.
No matter how you cut it, you've got two inserts to make. Doing the first, grabbing the new ID value as you've described (which imposes insignificant overhead, because the value is on hand to mysql already,) and using it in the second insert is pretty minimal.
I would investigate using stored procedures and/or transactions to make sure nothing bad happens.
I'm working on a project with mysql and what I did is the following (without using autoincrement fields):
1- I created a table called SEQUENCE with one field of type BIGINT called VALUE with an initial value of 1. This table will store the id value that will be incremented each time you insert a new record.
2- Create a store procedure and handle the id increment inside it within a transaction.
Here is an example.
CREATE PROCEDURE `SP_registerUser`(
IN _username VARCHAR(40),
IN _password VARCHAR(40),
)
BEGIN
DECLARE seq_user BIGINT;
START TRANSACTION;
#Validate that user does not exist etc..........
#Register the user
SELECT value FROM SEQUENCE INTO seq_user;
UPDATE SECUENCE SET value = value + 1;
INSERT INTO users VALUES(seq_user, _username, SHA1(_password));
INSERT INTO user_info VALUES(seq_user, UTC_TIMESTAMP());
COMMIT;
END //
In my case I want to store the user id in two different tables (users and user_info)

Categories