PHP Script runs slow, but server CPU is idle - php

Question

I have a PHP script to import various data from text files.
The import is very complex and my test file has 32.000 entrys. These entrys have to be parsed and inserted into a mysql database.
If i will run my script it needs 30 minutes to finish... And at this time my server cpu is 99% idle.
Is there a chance to optimize php and mysql that they are using more power from the machine?
code:
if ($handle = #fopen($filename, "r")) {
$canceled = false;
$item = null;
$result = null;
while (!feof($handle)) {
$buffer = fgets($handle, 4096);
$line = $buffer;
if (substr($line, 0, 2) == '00') {
continue;
}
else if (substr($line, 0, 2) == '99') {
continue;
}
else if (strlen($line) < 75) {
continue;
}
Reread:
if ($canceled) {
break;
}
if ($item == null) {
$item = new DA11_Item();
$item->boq_id = $boq_id;
}
$result = $this->add_line_into_item($item, $line);
if ($result == self::RESULT_CLOSED) {
$this->add_item($item);
$item = null;
}
else if ($result == self::RESULT_REREAD) {
$this->add_item($item);
$item = null;
goto Reread;
}
else if ($result == self::RESULT_IGNORD) {
if (count($item->details()) > 0) {
$this->add_item($item);
}
$item = null;
}
}
if ($item !== NULL) {
$this->add_item($item);
}
fclose($handle);
}
add_item will perform a $item->save() and saves it to the database.
thx and kind regards,
viperneo

Answer 1

One problem you have is that every single insert is a separate request to your db-server including it's response. With 32.000 records you maybe get an idea, that this is a quite huge overhead. Use bulk inserts for (lets say) 1000 records at once
INSERT INTO foo (col1, col2) VALUES
(1,'2'),
(3,'4')
-- 997 additional rows
(1999, '2000');
Additional Transactions may help
Update, because you mentioned active-record: I recommend to avoid any additional abstraction layer for such mass import tasks.

Answer 2

As Florent says in the comment, the MySQL engine is too slow to handle the requests. Moving the MySQL database to a SSD (instead of a HDD) will improve speed significantly.

Answer 3

There are some things that you can optimize.
But i think the most performance take your MySQL Database. Sometimes its possible that some indexes on your joined tables bring a lot of speed. You should check that at first.
When you have a linux system you could use mysqltuner to optimize your database.
The second way is to optimize your code and cache some results from your database when you've done the same before.