How can I compile the results of various calculations from data gathered from different tables ? Present summaries in php table. I have people in a table that is linked to two other tables. It will be sorted by department how many people are women and men in number and percentage . How much they earn in total per department in average. How big women's pay is a percentage of men and how great the wage distribution is the max and min values for men and women.
These code is used to get all the data.
I have added some value manually just for clarification in this picture.
I have struggled with this for some time now and see no solution. Grateful for the help !
Please see below, it's not exactly as you need, since I don't know a structure of tables, but you will get an idea:
CREATE TABLE `usergroup` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(100) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB;
CREATE TABLE `user` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`sex` enum('yes','no') NOT NULL DEFAULT 'yes',
`salary` decimal(10,2) DEFAULT NULL,
`userGroupId` int(11) DEFAULT NULL,
`name` varchar(20) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB;
insert into `usergroup`(`id`,`name`) values (1,'Sales');
insert into `usergroup`(`id`,`name`) values (2,'Support');
insert into `usergroup`(`id`,`name`) values (3,'Managment');
insert into `usergroup`(`id`,`name`) values (4,'Others');
insert into `user`(`id`,`sex`,`salary`,`userGroupId`,`name`) values (1,'yes',20000.00,1,'Scott');
insert into `user`(`id`,`sex`,`salary`,`userGroupId`,`name`) values (2,'yes',30000.00,1,'Peter');
insert into `user`(`id`,`sex`,`salary`,`userGroupId`,`name`) values (3,'no',20000.00,1,'Mike');
insert into `user`(`id`,`sex`,`salary`,`userGroupId`,`name`) values (4,'yes',100000.00,2,'Senior');
insert into `user`(`id`,`sex`,`salary`,`userGroupId`,`name`) values (5,'no',50000.00,2,'Junior');
insert into `user`(`id`,`sex`,`salary`,`userGroupId`,`name`) values (6,'yes',75000.00,2,'Middle');
insert into `user`(`id`,`sex`,`salary`,`userGroupId`,`name`) values (7,'yes',250000.00,3,'King');
insert into `user`(`id`,`sex`,`salary`,`userGroupId`,`name`) values (8,'yes',300000.00,3,'Ace');
insert into `user`(`id`,`sex`,`salary`,`userGroupId`,`name`) values (9,'no',200000.00,3,'Queen');
insert into `user`(`id`,`sex`,`salary`,`userGroupId`,`name`) values (10,'yes',100000.00,3,'Jack');
insert into `user`(`id`,`sex`,`salary`,`userGroupId`,`name`) values (11,'no',400000.00,3,'LadyJoker');
Query to get report. The subquery is selecting a data with stats values, which are used in the outer query for calculation:
select analytic.userGroupId,
analytic.name,
analytic.countMen,
analytic.countWomen,
round((analytic.countMen / analytic.countMisc)*100, 2) as percentMen,
round((analytic.countWomen / analytic.countMisc)*100, 2) as percentWomen,
round((analytic.maxSalaryMen + analytic.minSalaryMen)/2, 2) as basicSalaryMen,
round((analytic.maxSalaryWomen + analytic.minSalaryWomen)/2, 2) as basicSalaryWomen,
round(
(analytic.maxSalaryMen + analytic.minSalaryMen +
analytic.maxSalaryWomen + analytic.minSalaryWomen
)/4,
2)
as basicSalaryMisc,
if(analytic.maxSalaryMen + analytic.minSalaryMen = 0, 100,
round((analytic.maxSalaryWomen + analytic.minSalaryWomen)/(analytic.maxSalaryMen + analytic.minSalaryMen), 2)
) as salaryBasicWomenPercentOfMen,
analytic.maxSalaryMen,
analytic.minSalaryMen,
analytic.maxSalaryWomen,
analytic.minSalaryWomen
from (
select ug.id as userGroupId,
ug.name,
sum(if(u.sex='yes', 1, 0)) countMen,
sum(if(u.sex='no', 1, 0)) countWomen,
count(*) countMisc,
max(if(u.sex='yes', u.salary, null)) maxSalaryMen,
min(if(u.sex='yes', u.salary, null)) minSalaryMen,
max(if(u.sex='no', u.salary, null)) maxSalaryWomen,
min(if(u.sex='no', u.salary, null)) minSalaryWomen
from userGroup ug left join user u
on u.userGroupId = ug.id
group by ug.id, ug.name
order by ug.name
) analytic
Related
I am trying to develop a rating system with php/mysql.
I have a simple rating object like this:
(t is type of rating, r is value of rating)
[{"t":"1","r":2},{"t":"2","r":4},{"t":"3","r":1},{"t":"4","r":2},{"t":"5","r":2}]
In DB, I have a lot of rating records
Like this:
object1=> [{"t":"1","r":2},{"t":"2","r":4},{"t":"3","r":1},{"t":"4","r":2},{"t":"5","r":2}]
object2=> [{"t":"1","r":1},{"t":"2","r":5},{"t":"3","r":3},{"t":"4","r":3},{"t":"5","r":1}]
In short for output I need a new object like this (I need to calculate average rating, with same keys.)
objectAverageCalculated=> [{"t":"1","r":1.5},{"t":"2","r":4.5},{"t":"3","r":2},{"t":"4","r":2.5},{"t":"5","r":1.5}]
My sql:
CREATE TABLE `ratings` (
`id` int(11) NOT NULL,
`rating` text NOT NULL,
`item_id` varchar(16) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
INSERT INTO `ratings` (`id`, `rating`, `item_id`) VALUES
(6, '[{\"t\":\"1\",\"r\":2},{\"t\":\"2\",\"r\":4},{\"t\":\"3\",\"r\":1},{\"t\":\"4\",\"r\":2},{\"t\":\"5\",\"r\":2}]', 'ABC123'),
(7, '[{\"t\":\"1\",\"r\":1},{\"t\":\"2\",\"r\":5},{\"t\":\"3\",\"r\":3},{\"t\":\"4\",\"r\":3},{\"t\":\"5\",\"r\":1}]', 'ABC123');
--
ALTER TABLE `ratings`
ADD PRIMARY KEY (`id`);
ALTER TABLE `ratings`
MODIFY `id` int(11) NOT NULL AUTO_INCREMENT, AUTO_INCREMENT=8;
COMMIT;
My code
$result = mysqli_query($con, "SELECT * FROM ratings WHERE item_id='ABC123' ");
while ($row = mysqli_fetch_array($result)) {
$tempArray = json_decode($row['rating'], true);
array_push($ratingsRaw, $tempArray);
}
I can not save every object with new variable (like $item1,$item2, etc...)
How can I store every object in one array and how can I get average of every rating type in one output object?
You can use AVG() method in your MySQL query and retrieve average value directly from database.
SELECT AVG(rating) AS avg_rating FROM ratings WHERE item_id='ABC123'
Or when you don't specify ID and you want average value for all items.
SELECT AVG(rating) AS avg_rating, item_id FROM ratings GROUP BY item_id
I'm trying to create an mysql table with some data in it which are special items. For example we have item1(chanse: 1), item2(chanse: 1), item(chance: 20%) and
item3 (chance: 20) etc.. etc...
- Chances are in %
So I created a table with the following information:
CREATE TABLE `special_items` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`item_id` int(11) NOT NULL,
`item_name` varchar(255) DEFAULT NULL,
`item_type` enum('SPECIAL','SILVER','BRONZE','GOLD') NOT NULL DEFAULT 'BRONZE',
`item_ratio` int(11) NOT NULL DEFAULT '0',
PRIMARY KEY (`id`),
UNIQUE KEY `id` (`id`) USING BTREE,
UNIQUE KEY `item` (`item_id`) USING BTREE
) ENGINE=InnoDB AUTO_INCREMENT=8 DEFAULT CHARSET=utf8;
-- ----------------------------
-- Records of special_items
-- ----------------------------
INSERT INTO `special_items` VALUES ('1', '200', 'special_name1', 'BRONZE', '80');
INSERT INTO `special_items` VALUES ('2', '204', 'special_name2', 'BRONZE', '4');
INSERT INTO `special_items` VALUES ('3', '875', 'special_name3', 'BRONZE', '80');
INSERT INTO `special_items` VALUES ('4', '900', 'special_name4', 'BRONZE', '60');
INSERT INTO `special_items` VALUES ('5', '901', 'special_name5', 'SILVER', '90');
INSERT INTO `special_items` VALUES ('6', '968', 'special_name6', 'BRONZE', '65');
INSERT INTO `special_items` VALUES ('7', '777', 'special_name7', 'BRONZE', '30');
What we want to do now is select from 800 rows 5 random items by there ratio. So it needs to have the following requirements:
Always random rows.
Select rows by there ratio (chance in the table is percentage for example)
I also found this query which almost fits the solution but its don't know how I would do this for the random ratio (percentage)
SELECT item_name
FROM special_items AS r1 JOIN
(SELECT CEIL(RAND() *
(SELECT MAX(id)
FROM special_items)) AS id)
AS r2
WHERE r1.id >= r2.id
ORDER BY r1.id ASC
LIMIT 5
If this can be done through PHP it would be awesome.
I'm open to any and all suggestions. I'll also be trying to figure this out on my own in the meantime, but I'm still stuck.
Let me guess that by "chanse" [sic], you mean that each row has a weight, and you want this weight to contribute to the chance of a row being selected.
One method is to generate a random number for each row, multiply by the weight, and then return 5 rows with the highest generated number. It is unclear what you mean by the chance, so this might do what you want:
select si.*
from (select si.*, rand() * item_ratio as weight
from special_items si
) si
order by weight desc
limit 5;
Note: The subquery is needed so the weight is only calculated once per line. I think this does the same thing:
select si.*, rand() * item_ratio as weight
from special_items si
order by weight desc
limit 5;
But MySQL can be peculiar.
I have a large database with EAV structured data that has to be searchable and pageable. I tried every trick in my book to get it fast enough, but under certain circumstances, it still fails to complete in a reasonable time.
This is my table structure (relevant parts only, ask away if you need more):
CREATE TABLE IF NOT EXISTS `object` (
`object_id` bigint(20) NOT NULL AUTO_INCREMENT,
`oid` varchar(32) CHARACTER SET utf8 NOT NULL,
`status` varchar(100) CHARACTER SET utf8 DEFAULT NULL,
`created` datetime NOT NULL,
`updated` datetime NOT NULL,
PRIMARY KEY (`object_id`),
UNIQUE KEY `oid` (`oid`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
CREATE TABLE IF NOT EXISTS `version` (
`version_id` bigint(20) NOT NULL AUTO_INCREMENT,
`type_id` bigint(20) NOT NULL,
`object_id` bigint(20) NOT NULL,
`created` datetime NOT NULL,
`status` varchar(100) CHARACTER SET utf8 DEFAULT NULL,
PRIMARY KEY (`version_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
CREATE TABLE IF NOT EXISTS `value` (
`value_id` bigint(20) NOT NULL AUTO_INCREMENT,
`object_id` int(11) NOT NULL,
`attribute_id` int(11) NOT NULL,
`version_id` bigint(20) NOT NULL,
`type_id` bigint(20) NOT NULL,
`value` text NOT NULL,
PRIMARY KEY (`value_id`),
KEY `field_id` (`attribute_id`),
KEY `action_id` (`version_id`),
KEY `form_id` (`type_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
This is a sample object. I have around 1 million of those in my database. each object may have different number of attributes with different attribute_id
INSERT INTO `owner` (`owner_id`, `uid`, `status`, `created`, `updated`) VALUES (1, 'cwnzrdxs4dzxns47xs4tx', 'Green', NOW(), NOW());
INSERT INTO `object` (`object_id`, `type_id`, `owner_id`, `created`, `status`) VALUES (1, 1, 1, NOW(), NOW());
INSERT INTO `value` (`value_id`, `owner_id`, `attribute_id`, `object_id`, `type_id`, `value`) VALUES (1, 1, 1, 1, 1, 'Munich');
INSERT INTO `value` (`value_id`, `owner_id`, `attribute_id`, `object_id`, `type_id`, `value`) VALUES (2, 1, 2, 1, 1, 'Germany');
INSERT INTO `value` (`value_id`, `owner_id`, `attribute_id`, `object_id`, `type_id`, `value`) VALUES (3, 1, 3, 1, 1, '123');
INSERT INTO `value` (`value_id`, `owner_id`, `attribute_id`, `object_id`, `type_id`, `value`) VALUES (4, 1, 4, 1, 1, '2012-01-13');
INSERT INTO `value` (`value_id`, `owner_id`, `attribute_id`, `object_id`, `type_id`, `value`) VALUES (5, 1, 5, 1, 1, 'A cake!');
Now on to my current mechanism. My first try was the typical approach to Mysql. Do one huge SQL with loads of joins on anything I require. Complete desaster! Took way to long to load and even crashed the PHP and MySQL servers due to exhausted RAM.
So I split my queries up into several steps:
1 Determine all needed attribute_ids.
I can look them up in another table that references the type_id of an object. The result is a list of attribute_ids. (this table is not very relevant to the performance, so it's not included in my sample.)
:type_id contains all type_ids from any objects I want to include in my search. I already got this information in my application. So this is inexpensive.
SELECT * FROM attribute WHERE form_id IN (:type_id)
Result is an array of type_id integers.
2 Search for matching objects
A big SQL query is compiled that adds one INNER JOIN for each and every condition I want. This sounds horrible, but in the end, it was the fastest method :(
A typical generated query might look like this. The LIMIT sadly is necessary or I will potentially get so many IDs that the resulting array makes PHP explode or break the IN statement in the next Query:
SELECT DISTINCT `version`.object_id FROM `version`
INNER JOIN `version` AS condition1
ON `version`.version_id = condition1.version_id
AND condition1.created = '2012-03-04' -- Filter by version date
INNER JOIN `value` AS condition2
ON `version`.version_id = condition2.version_id
AND condition2.type_id IN (:type_id) -- try to limit joins to object types we need
AND condition2.attribute_id = :field_id2 -- searching for a value in a specific attribute
AND condition2.value = 'Munich' -- searching for the value 'Munich'
INNER JOIN `value` AS condition3
ON `version`.version_id = condition3.version_id
AND condition3.type_id IN (:type_id) -- try to limit joins to object types we need
AND condition3.attribute_id = :field_id3 -- searching for a value in a specific attribute
AND condition3.value = 'Green' -- searching for the value 'Green'
WHERE `version`.type_id IN (:type_id) ORDER BY `version`.version_id DESC LIMIT 10000
The result will contain all object_ids from any object I might need. I am selecting object_ids and not version_ids as I need to have all versions of matching objects, regardless of which version matched.
3 Sort and page results
Next I will create a query that sorts the objects by a certain attribute and then pages the resulting array.
SELECT DISTINCT object_id
FROM value
WHERE object_id IN (:foundObjects)
AND attribute_id = :attribute_id_to_sort
AND value > ''
ORDER BY value ASC LIMIT :limit OFFSET :offset
The result is a sorted and paged list of object ids from former search
4 Get our complete objects, versions and attributes
In the last step, I will select all values for any objects and versions the former queries found.
SELECT `value`.*, `object`.*, `version`.*, `type`.*
`object`.status AS `object.status`,
`object`.flag AS `object.flag`,
`version`.created AS `version.created`,
`version`.status AS `version.status`,
FROM version
INNER JOIN `type` ON `version`.form_id = `type`.type_id
INNER JOIN `object` ON `version`.object_id = `object`.object_id
LEFT JOIN value ON `version`.version_id = `value`.version_id
WHERE version.object_id IN (:sortedObjectIds) AND `version.type_id IN (:typeIds)
ORDER BY version.created DESC
The result will then be compiled via PHP into nice object->version->value array structures.
Now the question:
Can this whole mess be accelerated in any way?
Can I somehow remove the LIMIT 10000 restriction from my search query?
If all else fails, maybe switch database technology? See my other question: Database optimized for searching in large number of objects with different attributes
Real Life samples
Table sizes: object - 193801 rows, version - 193841 rows, value - 1053928 rows
SELECT * FROM attribute WHERE attribute_id IN (30)
SELECT DISTINCT `version`.object_id
FROM version
INNER JOIN value AS condition_d4e328e33813
ON version.version_id = condition_d4e328e33813.version_id
AND condition_d4e328e33813.type_id IN (30)
AND condition_d4e328e33813.attribute_id IN (377)
AND condition_d4e328e33813.value LIKE '%e%'
INNER JOIN value AS condition_2c870b0a429f
ON version.version_id = condition_2c870b0a429f.version_id
AND condition_2c870b0a429f.type_id IN (30)
AND condition_2c870b0a429f.attribute_id IN (376)
AND condition_2c870b0a429f.value LIKE '%s%'
WHERE version.type_id IN (30)
ORDER BY version.version_id DESC LIMIT 10000 -- limit to 10000 or it breaks!
Explain:
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE condition_2c870b0a429f ref field_id,action_id,form_id field_id 4 const 178639 Using where; Using temporary; Using filesort
1 SIMPLE action eq_ref PRIMARY PRIMARY 8 condition_2c870b0a429f.action_id 1 Using where
1 SIMPLE condition_d4e328e33813 ref field_id,action_id,form_id action_id 8 action.action_id 11 Using where; Distinct
objects search completed (Peak RAM: 5.91MB, Time: 4.64s)
SELECT DISTINCT object_id
FROM version
WHERE object_id IN (193793,193789, ... ,135326,135324) -- 10000 ids in here!
ORDER BY created ASC
LIMIT 50 OFFSET 0
objects sort completed (Peak RAM: 6.68MB, Time: 0.352s)
SELECT `value`.*, object.*, version.*, type.*,
object.status AS `object.status`,
object.flag AS `object.flag`,
version.created AS `version.created`,
version.status AS `version.status`,
version.flag AS `version.flag`
FROM version
INNER JOIN type ON version.type_id = type.type_id
INNER JOIN object ON version.object_id = object.object_id
LEFT JOIN value ON version.version_id = `value`.version_id
WHERE version.object_id IN (135324,135326,...,135658,135661) AND version.type_id IN (30)
ORDER BY quality DESC, version.created DESC
objects load query completed (Peak RAM: 6.68MB, Time: 0.083s)
objects compilation into arrays completed (Peak RAM: 6.68MB, Time: 0.007s)
Just try to add an EXPLAIN before your search query :
EXPLAIN SELECT DISTINCT `version`.object_id FROM `version`, etc ...
then check the results in the "Extra" column, it will give you some clues to speedup your query, like adding INDEX on the right fields.
Also some times you can removeINNER JOIN, get more results in your Mysql response and filter the big array by processing with PHP loops.
I would start by trying to have covering indexes (ie: all columns to match the criteria you are querying on and even pulling out as result). This way the engine does not have to go back to the raw page data.
Since you need the "object_id" from version, and using the "version_id" as join basis to the other tables. Your version table also has a WHERE clause on the TYPE_ID, so I would have an index on
version table -- (object_id, version_id, type_id)
For your "value" table, match there too for criteria
value table -- ( version_id, type_id, attribute_id, value, created )
I have a table and some rows, i want do fulltext search but fulltext col without space.
For Example:
My MySQL Table:
CREATE TABLE IF NOT EXISTS `members` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`username` varchar(255) NOT NULL DEFAULT '',
`fulltextcol` varchar(10) NOT NULL DEFAULT '',
PRIMARY KEY (`id`),
FULLTEXT KEY `fulltextcol` (`fulltextcol`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 AUTO_INCREMENT=5 ;
Inserts:
INSERT INTO `members` (`id`, `username`, `fulltextcol`) VALUES
(1, 'username1', 'YYYYYNYNYN'),
(2, 'username2', 'NNNNYNYNYN'),
(3, 'username3', 'YYNNYNYYYY'),
(4, 'username4', 'YNYNYNYNYN');
And now my mysql command:
SELECT *,MATCH(fulltextcol) AGAINST('YNYNYNYNYN') as Relevance
FROM members
WHERE MATCH(fulltextcol) AGAINST('YNYNYNYNYN' IN BOOLEAN MODE)
HAVING Relevance > 0.2
ORDER BY Relevance DESC
Yes everything is OK, bu this query is select only exact match (so select only 4th row) , but i want it must select all of them with scores/relevances.
For example first row have "YYYYYNYNYN" and it's not exact match but i want this query select it with low relevance, if 4th row's relevance is 1.18, 1st row's relevance can be 0.32.
I hope I could explain my problem.
How can i do it ?
I would like to know if its possible to get 2 sums from one query using the table values and then add them togther.
Here is some simple table & data which might help.
CREATE TABLE `cartcontents` (
`id` int(11) NOT NULL auto_increment,
`code` varchar(40) NOT NULL,
`qty` int(10) NOT NULL,
`price` decimal(30,2) NOT NULL,
`cart_id` int(11) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `zone` (`zone_code`,`cart_id`)
) ENGINE=MyISAM AUTO_INCREMENT=8 DEFAULT CHARSET=latin1;
-- ----------------------------
-- Records
-- ----------------------------
INSERT INTO `cartcontents` VALUES ('5', '011242077783866125432', '1', '36.00', '2');
INSERT INTO `cartcontents` VALUES ('4', '011242077793513596890', '3', '33.00', '4');
INSERT INTO `cartcontents` VALUES ('6', '011242077649557011493', '1', '110.00', '4');
INSERT INTO `cartcontents` VALUES ('7', '011242077724922511037', '1', '177.00', '5');
So I would like to be able collect the total qty & total value for a given cart_id.
So this would mean if I had 3 in the qty filed the sum would need to be (qty * price) for each zone then add them in total for the cart_id.
So in the above example if I was looking for the values for cart_id 4 The values I would hope I could return would be qty = 4 & total value = 209
Hope this makes sense and thanks if you can help.
Something like this should work:
SELECT SUM(qty) AS qty, SUM(qty * price) AS total
FROM cartcontents
GROUP BY cart_id