I just want to import csv file data to the database table. All fields were inserted successfully except my custom field that is date field(added_date).
It is display the error that.
Error: SQLSTATE[23000]: Integrity constraint violation: 1048 Column 'added_date' cannot be null
And below is my code for insertion csv data.
while (($emapData = fgetcsv($file, 10000, ",")) !== FALSE)
{
$rs[$cnt]['client_id'] = $emapData[0];
$rs[$cnt]['name'] = $emapData[1];
$rs[$cnt]['address1'] = $emapData[3];
$rs[$cnt]['address2'] = $emapData[4];
$rs[$cnt]['county'] = $emapData[8];
$rs[$cnt]['city'] = $emapData[6];
$rs[$cnt]['country'] = $emapData[8];
$rs[$cnt]['postcode'] = $emapData[7];
$rs[$cnt]['email_label'] = $emapData[14];
$rs[$cnt]['time_spent_on_site'] = $emapData[20];
$rs[$cnt]['added_date'] = date('Y-m-d H:i:s'); <== gives error that integrity constrains violation.
$cnt++;
}
fclose($file);
if (count($rs) > 0)
{
$customers = TableRegistry::get('Customers');
$entities = $customers->newEntities($rs);
$customers->connection()->transactional(function () use ($customers, $entities) {
foreach ($entities as $entity)
{
$customers->save($entity);
}
});
}
And database table structure is as follows:
Thanks you so much.
Its concept introduce in Cakephp 3.x that we need to use
$rs[$cnt]['added_date'] = new \DateTime('now');
in place of
$rs[$cnt]['added_date'] = date('Y-m-d H:i:s'); //Deprecated in Cakephp 3.x
Thank you so much for your approach.
Related
I am using Laravel Excel and I need to get id of imported rows during import.
code
public function model(array $row)
{
$link = new Link([
'site_name' => $row['site_name'],
]);
$name = explode('-', $row['site_name']);
$site = Site::whereIn('name', $name)->pluck('id');
$link->sites()->sync($site, false); // this `$link` can't get id of imported row
return $link;
}
Error
SQLSTATE[23000]: Integrity constraint violation: 1048 Column 'link_id'
cannot be null (SQL: insert into link_sites (link_id, site_id)
values (?, 14))
Any idea?
Solved
I've changed my function to use onEachRow and it's working now.
public function onRow(Row $row)
{
$rowIndex = $row->getIndex();
$row = $row->toArray();
$link = Link::create([
'site_name' => $row['site_name'],
]);
$name = explode('-', $row['site_name']);
$site = Site::whereIn('name', $name)->pluck('id');
$link->sites()->sync($site, false);
}
Document
This question has been asked so many times , I have tried couple of way also but this time I am stuck since my requirement is bit specific . None of the generic methods worked for me .
Details
File Size = 75MB
Total Rows = 300000
PHP Code
protected $chunkSize = 500;
public function handle()
{
try {
set_time_limit(0);
$file = Flag::where('imported','=','0')
->orderBy('created_at', 'DESC')
->first();
$file_path = Config::get('filesystems.disks.local.root') . '/exceluploads/' .$file->file_name;
// let's first count the total number of rows
Excel::load($file_path, function($reader) use($file) {
$objWorksheet = $reader->getActiveSheet();
$file->total_rows = $objWorksheet->getHighestRow() - 1; //exclude the heading
$file->save();
});
$chunkid=0;
//now let's import the rows, one by one while keeping track of the progress
Excel::filter('chunk')
->selectSheetsByIndex(0)
->load($file_path)
->chunk($this->chunkSize, function($results) use ($file,$chunkid) {
//let's do more processing (change values in cells) here as needed
$counter = 0;
$chunkid++;
$output = new ConsoleOutput();
$data =array();
foreach ($results->toArray() as $row)
{
$data[] = array(
'data'=> json_encode($row),
'created_at'=>date('Y-m-d H:i:s'),
'updated_at'=> date('Y-m-d H:i:s')
);
//$x->save();
$counter++;
}
DB::table('price_results')->insert($data);
$file = $file->fresh(); //reload from the database
$file->rows_imported = $file->rows_imported + $counter;
$file->save();
$countx = $file->rows_imported + $counter;
echo "Rows Executed".$countx.PHP_EOL;
},
false
);
$file->imported =1;
$file->save();
echo "end of execution";
}
catch(\Exception $e)
{
dd($e->getMessage());
}
}
So the above Code runs really fast for the 10,000 rows CSV File.
But when I upload a larger CSV its not working .
My Only restriction here is that I have to use following logic to transform each row of the CSV to KeyPair value json data
foreach ($results->toArray() as $row)
{
$data[] = array(
'data'=> json_encode($row),
'created_at'=>date('Y-m-d H:i:s'),
'updated_at'=> date('Y-m-d H:i:s')
);
//$x->save();
$counter++;
}
Any suggestions would be appreciated , Its been more than and Hour now and still only 100,000 rows have been inserted
I find this is really slow
Database : POSTGRES
I'm working on this project that have foreign keys on two tables. By using a form I'm trying to insert a new record to the database. And there's also an image path in the db and I'm inserting the image path via the form. I'm using codeigniter file upload library. Other fields of the database table get updated when i submit the form even the foreign key field. But the image path is not updating. When I submit the form it shows this error.
A Database Error Occurred
Error Number: 1452
Cannot add or update a child row: a foreign key constraint fails (`bfh`.`products`, CONSTRAINT `category_fk` FOREIGN KEY (`category_id`) REFERENCES `categories` (`id`))
INSERT INTO `products` (`img`) VALUES ('assets/img//sakya.PNG')
Filename: C:/xampp/htdocs/CI-skeleton/system/database/DB_driver.php
Line Number: 691
Controller
public function add_product()
{
$result = 0;
$cat_id = $this->input->post("category_id");
$type_id = $this->input->post("type_id");
$pname = $this->input->post("p_name");
$images = $this->input->post("images");
$images = $_FILES['images']['name'];
$price = $this->input->post("price");
$this->load->model("Products_Model");
$product_id = $this->Products_Model->add_product( $cat_id, $type_id, $pname, $price);
if ($product_id != 0) {
$result = $this->Products_Model->add_product_images($images);
}
if ($result && $_FILES['images']['name'][0] != "") {
$this->load->model('Image_Upload');
$result = $this->Image_Upload->upload("assets/img");
}
$this->session->set_flashdata('result', $result);
redirect('Products');
}
Model
public function add_product( $cat_id, $type_id, $pname, $price)
{
$result = $this->db->get_where("products", array('name' => $pname));
if ($result->num_rows() != 0) {
return 0; // record already exists
} else {
$data = array(
'category_id' => $cat_id,
'type_id' => $type_id,
'name' => $pname,
'price' => $price
);
if( !$this->db->insert('products', $data)) {
return -1; // error
}else{
return $this->db->insert_id();
}
return 1; // success
}
}
public function add_product_images($images)
{
$path = "assets/img/";
foreach ($images as $image) {
// if no images were given for the item
if ($image == "") {
return 1;
}
$data = array(
'img' => $path."/".$image
);
if ( ! $this->db->insert('products', $data)) {
return 0; // if something goes wrong
}
}
return 1;
}
You should use "update" query on the behalf of insert in add_product_images().
Because "insert" will add a new record of the product and there is no any category id(Foreign Key) with this that's why shows this error.
So try to update image.
you have two variables with the same name and thats the problem.
You need have different variable name.
I hope this will help you.
there are some problems in this script
if model you wrote, function add product. if your query failed, returns -1 otherwise return product Id
but if product was in database it returns 0, then in controller: you check if product_id != 0 then insert images. so you don't care if product didn't exists in database, you failed you insert that in table or not
in add_product_images you check if ($image == "") . I don't understand how you get that image array ($images) is empty if this statement is true. you can check if (sizeof($images) == 0) before foreach to check emptiness of image array ($images)
Suppose that I have 6000 values and I am performing a for loop for cycle on them and perform INSERT or UPDATE actions using Symfony2 + Doctrine, which is the best/right way for execute those statements and keep good levels of performance?
First
for ($i = 0; $i < 6000; $i++) {
// SELECT HERE - need to find if Territory already exists
$entTerritory = $em->getRepository('PDOneBundle:Territory')->findOneBy(array('veeva_territory_id' => $soqlObj2['records'][$i]['Id']));
if (!$entTerritory) {
// if there is no territory, then we add
$newTerritory = new Entity\Territory();
// we set the values from veeva
if ($soqlObj2['records'][$i]['Id'] !== null || $soqlObj2['records'][$i]['Id'] !== "") {
$newTerritory->setVeevaTerritoryId($soqlObj2['records'][$i]['Id']);
$newTerritory->setName($soqlObj2['records'][$i]['Name']);
$em->persist($newTerritory);
$em->flush(); // ---> FLUSH HERE
}
$terrArr[] = $newTerritory->getId();
$terrFailArr[] = $soqlObj2['records'][$i]['Name'];
} else {
$lastModifiedDate = new \DateTime(
$soqlObj2['records'][$i]['LastModifiedDate']
);
if ($lastModifiedDate > $entTerritory->getUpdatedAt()) {
// obtained a territory, we update its data
$entTerritory->setName($soqlObj2['records'][0]['Name']);
}
$em->flush(); // ---> FLUSH HERE
$terrArr[] = $entTerritory->getId();
}
}
Second
for ($i = 0; $i < 6000; $i++) {
// SELECT HERE - need to find if Territory already exists
$entTerritory = $em->getRepository('PDOneBundle:Territory')->findOneBy(array('veeva_territory_id' => $soqlObj2['records'][$i]['Id']));
if (!$entTerritory) {
// if there is no territory, then we add
$newTerritory = new Entity\Territory();
// we set the values from veeva
if ($soqlObj2['records'][$i]['Id'] !== null || $soqlObj2['records'][$i]['Id'] !== "") {
$newTerritory->setVeevaTerritoryId($soqlObj2['records'][$i]['Id']);
$newTerritory->setName($soqlObj2['records'][$i]['Name']);
$em->persist($newTerritory);
}
$terrArr[] = $newTerritory->getId();
$terrFailArr[] = $soqlObj2['records'][$i]['Name'];
} else {
$lastModifiedDate = new \DateTime(
$soqlObj2['records'][$i]['LastModifiedDate']
);
if ($lastModifiedDate > $entTerritory->getUpdatedAt()) {
// obtained a territory, we update its data
$entTerritory->setName($soqlObj2['records'][0]['Name']);
}
$em->flush(); // ---> FLUSH HERE
$terrArr[] = $entTerritory->getId();
}
}
$em->flush(); // ---> FLUSH FOR INSERT HERE
Third
for ($i = 0; $i < 6000; $i++) {
// SELECT HERE - need to find if Territory already exists
$entTerritory = $em->getRepository('PDOneBundle:Territory')->findOneBy(array('veeva_territory_id' => $soqlObj2['records'][$i]['Id']));
if (!$entTerritory) {
// if there is no territory, then we add
$newTerritory = new Entity\Territory();
// we set the values from veeva
if ($soqlObj2['records'][$i]['Id'] !== null || $soqlObj2['records'][$i]['Id'] !== "") {
$newTerritory->setVeevaTerritoryId($soqlObj2['records'][$i]['Id']);
$newTerritory->setName($soqlObj2['records'][$i]['Name']);
$em->persist($newTerritory);
}
$terrArr[] = $newTerritory->getId();
$terrFailArr[] = $soqlObj2['records'][$i]['Name'];
} else {
$lastModifiedDate = new \DateTime(
$soqlObj2['records'][$i]['LastModifiedDate']
);
if ($lastModifiedDate > $entTerritory->getUpdatedAt()) {
// obtained a territory, we update its data
$entTerritory->setName($soqlObj2['records'][0]['Name']);
}
$terrArr[] = $entTerritory->getId();
}
}
$em->flush(); // ---> FLUSH FOR INSERT AND UPDATE HERE
I have also found a topic around this here and they says:
When you have to update multiple entities, retrieving them all from
the database and iterating over ORM entities is known as a bad
practice.
You should never do like:
$friend = $em->getReference('Octivi\Entity\User', $friendId); $users =
$this->findAll();
foreach ($users as $user) {
$user->setFriend($friend);
$em->persist($user); }
$em->flush();
Instead, you should rely on the UPDATE query:
$qb->update('Octivi:User', 'u')
->set('u.friend', $friendId)
->getQuery()->execute();
Thanks to it, we only execute one SQL UPDATE statement instead of N-updates for each User entity.
So, what is the best way and why?
In my opinion, using SQL is the best/performance way for batch process. Doctrine is known to take a lot of memory and might quickly reach the allowed memory size.
I'm currently trying to integrate magento into an ERP System using an self written middleware.
I get this message:
Integrity constraint violation: 1452 Cannot add or update a child row: a foreign key constraint fails (`magento`.`catalog_product_entity`, CONSTRAINT `FK_CAT_PRD_ENTT_ATTR_SET_ID_EAV_ATTR_SET_ATTR_SET_ID` FOREIGN KEY (`attribute_set_id`) REFERENCES `eav_attribute_set` (`a)
I don't really know where the problem lies.
The result Object of a Product get and Product write look similar.
I checked the web already for help, but couldn't find an solution for it.
I also don't really know where to search, because the message on top is the only one I get.
Any help is appreceated.
Well the code itself is split up into multiple sections
but I'll try to show as much as possible
$this->product = Mage::getModel('catalog/product')->loadByAttribute('sku',$this->handler['art_nr']);
if($this->product===false || $this->product->getId()<1){
$this->product = Mage::getModel('catalog/product');
$this->product->setSku($this->handler['art_nr']);
$this->newProduct = true;
}
$this->product->setStatus($this->shoparticle['products_status']);
$this->product->setName($this->handler['art_name']);
$categories = array();
if(!$this->isNewProduct()){
$categories = $this->product->getCategoryIds();
}
$categories = $this->handler['all_categories'];
$this->product->setCategoryIds($categories);
$crosssellingSet = array();
$upsellingSet = array();
$relatedSet = array();
if(is_array($this->handler['xselling']) && count($this->handler['xselling'])>0){
foreach($this->handler['xselling'] as $valueSet){
$product = Mage::getModel('catalog/product')->loadBySku($valueSet['art_nr']);
if((int)$valueSet['group']===1){
$crossselling[$product->getId()] = array('position'=>$valueSet['sort_oder']);
}else if((int)$valueSet['group']===2){
$upsellingSet[$product->getId()] = array('position'=>$valueSet['sort_oder']);
}else if((int)$valueSet['group']===3){
$relatedSet[$product->getId()] = array('position'=>$valueSet['sort_oder']);
}
}
}
$this->product->setCrossSellProductsData($crosssellingSet);
$this->product->setUpsellingProductsData($upsellingSet);
$this->product->setRelatedProductsData($relatedSet);
$importDir = Mage::getBaseDir('media') . DS . 'import' . DS;
//check if exists and add .htaccess file for protection
if(!is_dir($importDir)){
#mkdir($importDir,0775,true);
#chmod($importDir,0775);
}
if(!is_dir($importDir)){
throw new Connector_Model_Exception_Error('Could not create import Directory!');
}
if(!file_exists($importDir.'.htaccess')){
file_put_contents($importDir.'.htaccess','Order deny,allow'."\n".'Deny from all'."\n");
}
//clean direcotry
$dir = dir($importDir);
while(($e=$dir->read())!==false){
if(strpos($e,'.jpg')||strpos($e,'.png')||strpos($e,'.jepg')||strpos($e,'.gif')||strpos($e,'.tif')){
#unlink($importDir.$e);
}
}
//write images into directory
//and run Import
foreach($this->handler['images'] as $image){
file_put_contents($importDir.$image['image_name'],$image['image']);
$this->product->addImageToMediaGallery($importDir.$image['image_name'], array('image', 'small_image', 'thumbnail'), false, false);
}
$groups = Mage::getModel('customer/group')->getCollection()->getAllIds();
if((float)$this->handler['Bpreis'] > 0.00){
$this->product->setPrice((float)$this->handler['Bpreis']);
}
if((float)$this->handler['art']['products_pprices'] > 0.00){
$this->product->setMsrp((float)$this->handler['art']['products_pprices']);
}
//preapre the price data for ranges
$groupsets = array();
if(count($this->handler['PGROUP'])){
foreach($this->handler['PGROUP'] as $group){
if(in_array(((int)$group['gruppe']-250),$groups)){
$groupsets[((int)$group['gruppe']-250)][(float)$group['marge']] = (float)$group['PGPRICE'];
}
}
}
//Now run ageanst groupsets to set price range etc
$storeid = Mage::app()->getStore()->getWebsiteId();
foreach($groupsets as $groupid=>$rangeset){
if(count($rangeset)>0){
foreach($rangeset as $key=>$value){
if(count($rangeset)===1 && $key === (float)0){
$this->product->setData(
'group_price',
array(
'website_id'=>$storeid,
'cust_group'=>$groupid,
'price'=>$value,
)
);
}else{
$this->product->setData(
'tier_price',array(
'website_id'=>$storeid,
'cust_group'=>$groupid,
'price'=>$value,
'price_qty'=>$key
)
);
}
}
}
}
Mage::app()->getStore()->setId(Mage_Core_Model_App::ADMIN_STORE_ID);
if($this->isNewProduct()){
$this->product->setCreatedAt(strtotime('now'));
}
$this->product->save();
The error is telling you exactly what the problem is. You are trying to save a product with an incorrect attribute set id. By that it means that the attribute set id you have set or not set in this case is not in the eav_attribute_set table. I think it's because if you create a new product you are not setting it. If you are updating an existing one you dont need to set it.
if($this->product===false || $this->product->getId()<1){
$this->product = Mage::getModel('catalog/product');
$this->product->setSku($this->handler['art_nr']);
// Set Attribute Set. Should be numeric for simple, bundle, configurable, grouped etc
$this->product->setAttributeSetId($this->handler['art_attribute_set_id']);
$this->newProduct = true;
}
Please check are you using right Attribute set id with your code. Please share the code where you have written the code to update/save the product.