I'm writing a piece of software that generates a pdf report out of a raw user-defined SQL query execution.
The pdf contains a simple table with rows containing SQL result rows. I'd like to add a table header with column names retrieved along with the SQL results.
The column headers in SQL may have various structures, e.g:
a) select * from users;
b) select name, surname, email from users;
c) select name as UserName, surname as UserSurname, email as UserEmail from users;
So far I fetch SQL results as association array, take the keys of the first row and treat them as column names.
It works only if there is at least 1 result in result set, so it's a heavy flaw in this approach.
I could generate pdf with "No results" label.
I could run a regex on a SQL query for named columns and execute describe table x, but this is plain ridiculous.
I also have even more ridiculous ideas, but that's not the way.
Is there anybody having any idea for solving this?
I use Doctrine on MySQL for this, but simple PDO approach would be just as good as Doctrine's one.
EDIT
Right after posting this question it came to my mind I could generate a view out of my SQL query, then run SHOW COLUMNS FROM randomViewName; and drop the view immediately afterwards.
It's hacky and needs some db security work (I can handle that), but it's a working candidate.
What do you think?
It may not be perfect solution, but I go along with the approach mentioned in the question.
I create a MySQL view. This allows me to have an db object which is queryable and have exact column names as I want.
describe nameOfViewYouJustCreated;
gives me exactly what I need.
The view is dropped afterwards.
I have a database with employees in it.
Since my employer finds it easy to input the data in a CSV file, I wrote a program that truncates my database and inserts the CSV data in my DB.
Employee: [ID, LAST_NAME, NAME, EMAIL, REMARKS, ...]
I use the field ID, (which is an auto_increment value) to make all my employee's unique. This works fine, however recently my employer has asked me too to include a functionality to mark favorites.
The only thing which makes my employees unique is the ID key thus when I update
the new CSV file the ID's go all broke and are shifted since I had to truncate my database and the favorites don't match up any more.
An example of what I mean (CSV file):
0, Carlton, John, john#gmail.com, "Great worker",
1, Awsome, Dude, awsomeDud#aol.com, "Not so great",
2, Random, Randy, rr#hotmail.com, "idk"
Suppose somebody deletes the record with ID 1.
And my favorite was 1, the csv file however will now look like this:
0, Carlton, John, john#gmail.com, "Great worker",
1, Random, Randy, rr#hotmail.com, "idk"
It points to the wrong person.
Keep in mind that the ID's I wrote are not part of the csv file itself
they are the auto_increment value.
I have given this problem a lot of thought and I cannot seem to find a simple way to accomplish this.
Any help would be appreciated.
Notes:
Emails are not unique, nor required.
The only real unique field is the ID field.
Solution 1 (easiest)
Have an int is_favorite column in your database containing 1 or 0, with a default value of 0 (meaning is not a favorite). Then ask your client to slightly change the format of the csv file as follows:
Employee: [ID, LAST_NAME, NAME, EMAIL, REMARKS, FAVORITE, ...]
Example CSV:
0, Carlton, John, john#gmail.com, "Great worker", 1
1, Awsome, Dude, awsomeDud#aol.com, "Not so great", 0
2, Random, Randy, rr#hotmail.com, "idk"
When you process the CSV file, depending on the FAVORITE column just set the same value in the database. This will eliminate the problem with the mismatched favorites. Unfortunately, if in the near feature, the client requires new features which depend on the favorites, you might have the same issue again.
Solution 2 (best)
Discuss a more mature solution with your client pointing out the current CSV solution is no longer a valid option due to the issue with matching the CSV users with the appropriate sub features (i.e. favorites)
A possible solution would be to never truncate your table. Ever.
Find out what makes the employees unique. E.g. EMAIL.
Then when you parse the next CSV's, you don't simply INSERT the employees. You update the current ones and insert the new ones.
This way, your IDs always stay the same (which they should).
I would have used something like this:
IF EXISTS (SELECT 1 FROM [User] WHERE [Email] = #UsersEmail)
BEGIN
UPDATE [User]
SET [Name] = #NewName
WHERE [Email] = #UsersEmail
END
ELSE
BEGIN
INSERT INTO [User] ([Email], [Name]) VALUES
(#UsersEmail, #NewName)
END
But since you've tagged it PHP, I'm guessing you're using MySQL. Which can do it differently (from here):
INSERT INTO subs
(subs_name, subs_email, subs_birthday)
VALUES
(?, ?, ?)
ON DUPLICATE KEY UPDATE
subs_name = VALUES(subs_name),
subs_birthday = VALUES(subs_birthday)
I would not truncate the table. I would then upload the csv into a temporary table. If the same ID is in both tables, do an update. If it is only in the old version, delete it (deleting out favorites as well for that ID), or, perhaps better, have a flag on the employees table that deactivates the row. If it is only in the new version, insert everything except the ID (which will probably be an empty string anyway). Then you can delete the temporary table.
If you want to be paranoid, you can double check names or emails and if you find a mismatch, flag them without updating. That would cause a manual operation if someone changed their name, but it would also save you the trouble if someone messed up your id numbers.
The simple and clean way to solve this would be to find a way to recognise unique employees on the flat data.
Is there no other unique identifier that could be added to the csv file? For example, a windows login name? A company employee No? Something that would be static.
That way it's simple:
1, Don't truncate.
2, If Windows LoginID / EmpNo exists, update.
3, If not, add.
Also I'm concerned that your "favourites" table is clearly not using referential integrity. It should have a FK pointing to your Employee.ID; preventing you accidentally deleting an employee that was marked as a favourite, amongst other things.
A messier, much less bullet proof way, would be to mark your favourites based on your employee names rather than IDs. There are obvious draw backs to this approach, so use as last resort.
You should never use the ID to identify a given user for the reasons you described in the question.
You could create a new reference ID field based on what you already have and create a unique identifier by chaining the required fields as a single string and then calculating the MD5 hash for example.
I have a question (sorry but I can't comment - rep): your employer adds only new employees via CSV file or even edit existing ones?
If only new employees are added you don't need to rebuild the table from scratch and you can make sure that your program generates a unique reference ID (that will remain unchanged) before inserting data into the db. Also your program can handle the editing of the employee, instead of changing data from CSV, leaving reference ID untouched.
This way all the fields like name, email, etc. can be edited and the link to favourites will stay correct. In that case the reference ID can also be calculated using not only data on the CSV but other like creation timestamp.
You could create MD5 hash from name , email and the comment , save and use that as unique identifier .
Make sure you store MD5 hash as binary
Can you modify database? If you can, add another field that you can call favourite. Set it to simple enum (1,0) and set 1 for favourites, 0 for others. So when, you truncate database, you'll still have your favourites by those fields. Of course if you have multi-level favourites, don't set field to enum, set it to something else, more suitable for you.
One solution is that the database becomes the defacto 'source' for IDs.
After the initial import, the next time your boss wants to update the file, create a CSV FROM the database (with the ID's intact) and ask your boss to update that and return it.
You could ask him to add new rows to the bottom of the file and leave out the ID.
Any row in the new spreadsheet without an ID is a new record. An extra field at the end of the row could be used by the boss to indicate rows to be deleted.
Repeat this process the next time the boss wants to update the file.
Add an extra field to your database table as well as to the CSV file named something like "EmployeeID" which should be unique for all employees.
I am creating a php application that is the front end for a database.
Database Server Type: MySQL version 5.5.32
Web Server Type: Apache/2.4.4
OpenSSL: 0.9.8y
PHP 5.4.16
When populating the DB any blank fields that would need to be addressed in the future were entered as "TBD". I would like to create a query that searches the entire DB for fields that are equal to "TBD". If possible I would like the output of the query to have the following information:
Table Name:
Field Name:
Primary Key:
The PKs for all of my tables are named with the following convention: table name + ID. For example if a table is named "client" it's primary key field is called "clientID". I have set up a version of this report using arrays of the table names and fields name to generate multiple sql queries but I have to believe that there is a SQL wiz out there who can get this done in if not 1, just a query per table? This would be extremely beneficial because it would not require me to update the report, every time I make a change to the DB. Thanks!
You can use a script called anywhereindb
Download it here:
https://code.google.com/p/anywhereindb/
You simply have to upload a php file to your server, add the DB credentials and it'll search for a string throughout the DB. It has been very helpful.
Remember to delete the file after using it if is publicly accesible.
Try constructing your queries like this:
SELECT *
FROM tbl,
tbl1,
tbl2...tblx
WHERE tabl.condition OR ...
I have an application which keeps database of all nearby restaurants.
When adding a new entry, there is an option to add a few images of your restaurant.
Now the problem:
I'd like to sort the different restaurant images into separated folders, named by restaurant id stored in mysql auto increment ID. The only problem here is, that i dont know that id in advance.
Form example:
text input - title
text input - address
text input - phone
file input - image
file input - image
So, what should I do now?
I. Get the last id, lock the table, create folder named by id, store images inside, store information to mysql database including image paths, unlock table.
or
II. Store all information excluding images paths to mysql database, use PHP mysql_insert_id, create folder named by id, store images inside and store images paths to mysql database.
or
III. Better solution?
This kind of thing is usually done with your option II. Store in the database the main row of information, get the last insert id via mysql_insert_id() or the native MySQL LAST_INSERT_ID() function, then proceed to store rows in any related tables if necessary and create the image directory to store filesystem data.
Assuming the image paths you intend to store in the database will go into a different table with a one-to-many relationship back to the main restaurant table, you'll need to know the last insert id to insert them anyway. Don't worry much about doing it in multiple operations -- that's exactly the reason most RDBMS have a function like LAST_INSERT_ID().
If you are using the autoincrement column. You can right after your insert statement call the last_insert_id() function to retrieve the id of the last inserted record.
See this link for documentation:
http://dev.mysql.com/doc/refman/5.0/en/information-functions.html#function_last-insert-id
Its important that you do this within the same transaction/connection otherwise the value might be erroneous.
I don't see a need to use the restaurant's id to organize the pictures - especially since you get into the timing issues you describe. I'd create another unique id - call it picture_folder_id or something - and use that to name the folder for the pictures. As long as you enforce uniqueness on that id, you won't get any collisions, and you won't have any timing problems or locks.
I'm using PHP to fetch data from a database.
Here is the code:
<?php
$db = new SQLiteDatabase("testDB.db");
$query = $db->query("SELECT * FROM blog ORDER BY blogdate DESC");
while($entry = $query->fetch(SQLITE_ASSOC)) { //only one blog entry per day
echo "".$entry['blogdate']."<br>";
}
?>
But for some reason it doesn't return an entry that I am certain is in the database. The reason I think it's in the db is that I can see the text of the entry when I view the *.db file.
Here are some specific questions that might help me better understand what's going on:
I CREATE a table called blog. I INSERT tuples into blog using the query function as I did with the SELECT calls. When I DELETE tuples using LIKE, are those tuples being deleted from the database, or are they being deleted from the table blog only? If it is the latter case, how do I get SQLite to delete the tuple from the database completely?
Finally, I've observed some odd behavior. A tuple is added to blog with blogdate as "2009-12-1" (which I treat as a string because there's not date or time type in SQLite). When I run the PHP file with the above code, the entry with 2009-12-1 as blogdate does not show up. I ran another PHP page that searches for tuples with blogdate LIKE 2009-12-1, and it showed up in the search results. Only then did the tuple for 2009-12-1 show up when I SELECT *d for it using the PHP above.
The text of a record may show up when you view a DB file in a text/hex editor even though it may have been marked as deleted in the database.
To fully remove these deleted records, try compacting the database.
Viewing the binary database file is insufficient to show that a record actually exists in a database. Many databases don't bother actually removing the data from the data file, as that would be a waste of time. Instead, they may flag the block as "deleted", and overwrite it later when saving additional data. So, you assertion that the record is in the database because you can see it in the .db file means nothing, you need to open the file in a program designed to browse the contents of the database and see if it shows up.
To check your assertion that the data is in the database, I would suggest opening the database in an database browser.