Optimize this query for 1000000+ rows - php

Question

I need to pull the data and write it to a csv file but its taking too much time and too much ram. What is wrong with it and what can I do? Also, I feel like there's a redundancy in the query itself. I'm doing this with PHP.
Here's the query
CREATE TEMPORARY TABLE temp1 SELECT * FROM vicidial_closer_log
USE INDEX(call_date)
WHERE call_date BETWEEN '1980-01-01 00:00:00' AND '2019-03-12 23:59:59';
CREATE TEMPORARY TABLE temp2 SELECT * FROM vicidial_closer_log
USE INDEX(call_date)
WHERE call_date BETWEEN '1980-01-01 00:00:00' AND '2019-03-12 23:59:59';
SELECT a.call_date,
a.lead_id,
a.phone_number
AS customer_number,
IF(a.status != 'DROP', 'ANSWERED', 'UNANSWERED')
AS status,
IF(a.lead_id IS NOT NULL, 'inbound', 'outbound')
AS call_type,
a.USER
AS agent,
a.campaign_id
AS skill,
NULL
AS campaign,
a.status
AS disposition,
a.term_reason
AS Hangup,
a.uniqueid,
Sec_to_time(a.queue_seconds)
AS time_to_answer,
Sec_to_time(a.length_in_sec - a.queue_seconds)
AS talk_time,
Sec_to_time(a.park_sec)
AS hold_sec,
Sec_to_time(a.dispo_sec)
AS wrapup_sec,
From_unixtime(a.start_epoch)
AS start_time,
From_unixtime(a.end_epoch)
AS end_time,
c.USER
AS
transfered,
a.comments,
IF(a.length_in_sec IS NULL, Sec_to_time(a.queue_seconds),
Sec_to_time(a.length_in_sec + a.dispo_sec))
AS duration,
Sec_to_time(a.length_in_sec - a.queue_seconds + a.dispo_sec)
AS handling_time
FROM temp1 a
left outer join temp2 c
ON a.uniqueid = c.uniqueid
AND a.closecallid < c.closecallid
GROUP BY a.closecallid
I've uploaded screenshot of table structure and the indices.
Table Structure
Indices of Table
Thanks.
UPDATE:
SHOW CREATE TABLE vicidial_closer_log
vicidial_closer_log CREATE TABLE `vicidial_closer_log` (
`closecallid` int(9) unsigned NOT NULL AUTO_INCREMENT,
`lead_id` int(9) unsigned NOT NULL,
`list_id` bigint(14) unsigned DEFAULT NULL,
`campaign_id` varchar(20) COLLATE utf8_unicode_ci DEFAULT NULL,
`call_date` datetime DEFAULT NULL,
`start_epoch` int(10) unsigned DEFAULT NULL,
`end_epoch` int(10) unsigned DEFAULT NULL,
`length_in_sec` int(10) DEFAULT NULL,
`status` varchar(6) COLLATE utf8_unicode_ci DEFAULT NULL,
`phone_code` varchar(10) COLLATE utf8_unicode_ci DEFAULT NULL,
`phone_number` varchar(18) COLLATE utf8_unicode_ci DEFAULT NULL,
`user` varchar(20) COLLATE utf8_unicode_ci DEFAULT NULL,
`comments` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`processed` enum('Y','N') COLLATE utf8_unicode_ci DEFAULT NULL,
`queue_seconds` decimal(7,2) DEFAULT 0.00,
`user_group` varchar(20) COLLATE utf8_unicode_ci DEFAULT NULL,
`xfercallid` int(9) unsigned DEFAULT NULL,
`term_reason` enum('CALLER','AGENT','QUEUETIMEOUT','ABANDON','AFTERHOURS','HOLDRECALLXFER', 'HOLDTIME','NOAGENT','NONE','MAXCALLS') COLLATE utf8_unicode_ci DEFAULT 'NONE',
`uniqueid` varchar(20) COLLATE utf8_unicode_ci NOT NULL DEFAULT '',
`agent_only` varchar(20) COLLATE utf8_unicode_ci DEFAULT '',
`queue_position` smallint(4) unsigned DEFAULT 1,
`called_count` smallint(5) unsigned DEFAULT 0,
`nopaperform` varchar(5) COLLATE utf8_unicode_ci NOT NULL DEFAULT 'NO',
`park_sec` int(3) DEFAULT 0,
`dispo_sec` int(3) DEFAULT 0,
`record_file` text COLLATE utf8_unicode_ci DEFAULT NULL,
PRIMARY KEY (`closecallid`),
KEY `lead_id` (`lead_id`),
KEY `call_date` (`call_date`),
KEY `campaign_id` (`campaign_id`),
KEY `uniqueid` (`uniqueid`),
KEY `phone_number` (`phone_number`),
KEY `date_user` (`call_date`,`user`),
KEY `closecallid` (`closecallid`)
) ENGINE=MyISAM AUTO_INCREMENT=1850672 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci
EXPLAIN QUERY(On third query only):
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE a ALL NULL NULL NULL NULL 664640 Using temporary; Using filesort
1 SIMPLE c ALL NULL NULL NULL NULL 662480 Using where; Using join buffer (flat, BNL join)
UPDATE(Updated Query):
SELECT a.call_date,
a.lead_id,
a.phone_number
AS customer_number,
IF(a.status != 'DROP', 'ANSWERED', 'UNANSWERED')
AS status,
IF(a.lead_id IS NOT NULL, 'inbound', 'outbound')
AS call_type,
a.user
AS agent,
a.campaign_id
AS skill,
NULL
AS campaign,
a.status
AS disposition,
a.term_reason
AS Hangup,
a.uniqueid,
Sec_to_time(a.queue_seconds)
AS time_to_answer,
Sec_to_time(a.length_in_sec - a.queue_seconds)
AS talk_time,
Sec_to_time(a.park_sec)
AS hold_sec,
Sec_to_time(a.dispo_sec)
AS wrapup_sec,
From_unixtime(a.start_epoch)
AS start_time,
From_unixtime(a.end_epoch)
AS end_time,
c.user
AS
transfered,
a.comments,
IF(a.length_in_sec IS NULL, Sec_to_time(a.queue_seconds),
Sec_to_time(a.length_in_sec + a.dispo_sec))
AS duration,
Sec_to_time(a.length_in_sec - a.queue_seconds + a.dispo_sec)
AS handling_time
FROM vicidial_closer_log a
LEFT OUTER JOIN vicidial_closer_log c
ON a.closecallid <> c.closecallid
AND a.uniqueid = c.uniqueid
AND a.closecallid < c.closecallid
WHERE a.call_date BETWEEN '2018-01-01 00:00:00' AND '2019-03-13 23:59:59'
EXPLAIN on updated query:
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE a ALL call_date,date_user NULL NULL NULL 662829 Using where
1 SIMPLE c ref PRIMARY,uniqueid,closecallid uniqueid 62 aastell_bliss.a.uniqueid 1 Using where
Updated Query Execution Result:
Number of rows present between given time range: 155016 rows
Time taken: 0.0149 secs
It works!

Answer 1

Summary of comments that lead to an answer:
CREATE TEMPORARY TABLE ... SELECT doesn't create indexes on the temporary table
Explicit use of a temporary table, particularly of a large size, will rarely give a performance gain.
Using table aliases in a join allows for a self join
Group by Primary Key on the left side of a join doesn't add much as its already unique and the JOIN had no aggregate expressions. GROUP BY adds an implicit ORDER BY so you expression could end up slower if a secondary index was used to join the table.
While the date range of the query was large, preparing for it to be a significant filter when small would make the call_date more favourable as an index. To make this more favorable, the join key is added to the end of the index so most of the work of the join can happen by just looking at the index.
When PK is on a column, a secondary index on the same column isn't needed.