deleting inactive products from oscommerce - php

i am using latest oscommerce.
I got a huge amount of inactive products.I want to remove them.Going though admin one at a time is really slow.
I thought if i create a new temp category and move all inactive products to this temp category then using back end of oscommerce i can easily delete them.Doing this will also remove the associated image.
Products are associated via product id and categories association is done by product to category table. inactive products are set via products_status = 0;
CREATE TABLE IF NOT EXISTS `products` (
`products_id` int(11) NOT NULL AUTO_INCREMENT,
`products_quantity` int(4) NOT NULL,
`products_model` varchar(64) COLLATE utf8_unicode_ci DEFAULT NULL,
`products_ean` varchar(64) COLLATE utf8_unicode_ci NOT NULL,
`google_product_category` varchar(300) COLLATE utf8_unicode_ci DEFAULT NULL,
`products_image` varchar(64) COLLATE utf8_unicode_ci DEFAULT NULL,
`products_price` decimal(15,4) NOT NULL,
`products_date_added` datetime NOT NULL,
`products_last_modified` datetime DEFAULT NULL,
`products_date_available` datetime DEFAULT NULL,
`products_weight` decimal(5,2) NOT NULL,
`products_status` tinyint(1) NOT NULL,
`products_tax_class_id` int(11) NOT NULL,
`manufacturers_id` int(11) DEFAULT NULL,
`products_ordered` int(11) NOT NULL DEFAULT '0',
`products_last_import` datetime DEFAULT NULL,
`products_submit_google` smallint(6) NOT NULL DEFAULT '1',
`icecat_prodid` int(10) unsigned NOT NULL,
`vendors_id` int(11) DEFAULT '1',
`products_availability` smallint(6) NOT NULL DEFAULT '0',
PRIMARY KEY (`products_id`),
KEY `idx_products_model` (`products_model`),
KEY `idx_products_date_added` (`products_date_added`),
KEY `idx_icecat_prodid` (`icecat_prodid`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci AUTO_INCREMENT=292067 ;
CREATE TABLE IF NOT EXISTS `products_to_categories` (
`products_id` int(11) NOT NULL,
`categories_id` int(11) NOT NULL,
PRIMARY KEY (`products_id`,`categories_id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
i have tried using the following query but i get an error #1062 - Duplicate entry '276917-29240' for key 'PRIMARY'
Update products p ,products_to_categories pc
set pc.categories_id = 29598
where p.products_id = pc.products_id
and p.products_status = 0

You most likely have a product that is linked in one or more categories.
Example: Product ABC with products_id = 123 can exist twice in the products_to_categories table if it is in two categories (say 'categories_id` 222 and 333. So you have two entries in your table, 123-222 and 123-333.
When you run your update, the first time it encounters product/category 123-222, it will change it's category to 123-29598. When it encounters the product/category 123-333, it will also try to update the row to 123-29598, due to your primary key constraint, and would cause the problem you see.
Perhaps in your script you can check if the product (123) already exists in the category, and if so, then remove the second entry (123-333) rather than change it's category to (123-29598). See here for information on deleting entries with the same id from your table.

Related

Laravel Morph Many from two columns

I need to find a way to modify the morphMany relationship to morph two different columns. Here is the table create syntax that is being morphed:
CREATE TABLE `user_friendships` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`sender_type` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`sender_id` bigint(20) unsigned NOT NULL,
`recipient_type` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`recipient_id` bigint(20) unsigned NOT NULL,
`status` tinyint(4) NOT NULL DEFAULT '0',
`created_at` timestamp NULL DEFAULT NULL,
`updated_at` timestamp NULL DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `user_friendships_sender_type_sender_id_index` (`sender_type`,`sender_id`),
KEY `user_friendships_recipient_type_recipient_id_index` (`recipient_type`,`recipient_id`)
) ENGINE=InnoDB AUTO_INCREMENT=12332 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
Sender and recipient can both be a user. So I want to morph based on whatever column of the two contains the user_id.
This only looks at if the sender is the user, but not the recipient.
return $this->morphMany(Friendship::class, 'sender');
The output of the query from that is:
select * from `user_friendships` where `user_friendships`.`sender_id` = 5971 and `user_friendships`.`sender_id` is not null and `user_friendships`.`sender_type` = "App\\\User"
What we actually want is:
select * from `user_friendships` where (`user_friendships`.`sender_id` = 5971 and `user_friendships`.`sender_id` is not null and `user_friendships`.`sender_type` = "App\\\User") OR (`user_friendships`.`recipient_id` = 5971 and `user_friendships`.`recipient_id` is not null and `user_friendships`.`recipient_type` = "App\\\User")
How do I accomplish this?
https://github.com/staudenmeir/laravel-merged-relations
This composer package was able to solve it.

Speed up MySQL inner join with LIKE clause

I have the following 2 tables, api_analytics_data, and telecordia.
CREATE TABLE `api_analytics_data` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`upload_file_id` bigint(20) NOT NULL,
`partNumber` varchar(100) DEFAULT NULL,
`clei` varchar(45) DEFAULT NULL,
`description` varchar(150) DEFAULT NULL,
`processed` tinyint(1) DEFAULT '0',
PRIMARY KEY (`id`),
KEY `idx_aad_clei` (`clei`),
KEY `idx_aad_pn` (`partNumber`),
KEY `id_aad_processed` (`processed`),
KEY `idx_combo1` (`partNumber`,`clei`,`upload_file_id`)
) ENGINE=InnoDB CHARSET=latin1;
CREATE TABLE `telecordia` (
`tid` int(11) NOT NULL AUTO_INCREMENT,
`ProdID` varchar(50) DEFAULT NULL,
`Mfg` varchar(20) DEFAULT NULL,
`Pn` varchar(50) DEFAULT NULL,
`Clei` varchar(50) DEFAULT NULL,
`Series` varchar(50) DEFAULT NULL,
`Dsc` varchar(50) DEFAULT NULL,
`Eci` varchar(50) DEFAULT NULL,
`AddDate` date DEFAULT NULL,
`ChangeDate` date DEFAULT NULL,
`Cost` float DEFAULT NULL,
PRIMARY KEY (`tid`),
KEY `telecordia.ProdID` (`ProdID`) USING BTREE,
KEY `telecordia.clei` (`Clei`),
KEY `telecordia.pn` (`Pn`),
KEY `telcordia.eci` (`Eci`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Users upload data via a web interface using Excel/CSV files into api_analytics_data. The data contains EITHER the partNumbers or CLEIs. I then update the api_analytics_data table by joining the telecordia table. The telecordia table is the master list of partNumber and Cleis.
So if a user uploads a file of CLEIs, the update/join I use is:
update api_analytics_data aad
inner join telecordia t on aad.clei = t.Clei
set aad.partNumber = t.Pn
where aad.partNumber is null
and aad.upload_file_id = 5;
It works quickly, but not very thoroughly. The problem I have is that the CLEI uploaded may only be a substring of the CLEI in the telecordia table.
For example, the uploaded CLEI may be "5SC1DX0". In the telcordia table, the correct matching row is:
tid: 184324
ProdID: 472467
Mfg: PLSE
Pn: AUA58-2-REV-E
Clei: 5SC1DX04AA
Series: null
Dsc: DL SGL-PTY POTS CU RT
Eci: 205756
AddDate: 1994-03-18
ChangeDate: 1998-04-13
Cost: null
So obviously my update doesn't work in this case, even though 5SC1DX0 and 5SC1DX04AA are the same part.
What I need is a wildcard search. However, when I try this, it is crazy slow. With about 4500 rows uploaded into the api_analytics_data table, it runs for about 10 minutes, and then loses the connection with the server.
update api_analytics_data aad
inner join telecordia t on aad.clei like concat(t.Clei,'%')
set aad.partNumber = t.Pn
where aad.partNumber is null
and aad.upload_file_id = 5;
Is there a way to optimize this so that it runs quickly?
The correct answer is "no". The better course of action is to create a new column in telecordia with the correct Clei value in it, one that can be used for joining the tables. In the most recent versions of MySQL, this can even be a computed column and be indexed.
That said, you might be able to do something if the matching portion is always the same length. If so, try this:
update api_analytics_data aad inner join
telecordia t
on t.Clei = left(aad.clei, 7)
set aad.partNumber = t.Pn
where aad.partNumber is null and aad.upload_file_id = 5;
For this query, you want an index on api_analytics_data(upload_fiel_id, partNumber, clei) and telecordia(clei, pn).

How to speed up slow MySQL UPDATE queries with InnoDB tables

I have a very simple MySQL update query on my InnoDB table.
UPDATE `players_teams` SET t_last_active=NOW() WHERE t_player_id=11225 AND t_team_id=6912 AND t_season_id=2002 LIMIT 1
My table is structured as so:
CREATE TABLE `players_teams` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`t_player_id` int(11) DEFAULT NULL,
`t_team_id` int(11) DEFAULT NULL,
`t_league_id` int(11) DEFAULT NULL,
`t_season_id` int(11) DEFAULT NULL,
`t_div` varchar(64) DEFAULT NULL,
`t_player_number` varchar(3) DEFAULT NULL,
`t_player_jersey_size` enum('UNKNOWN','XS','S','M','L','XL','XXL','XXXL') DEFAULT 'UNKNOWN',
`t_player_registration_number` varchar(64) DEFAULT NULL,
`t_player_class` enum('ROSTER','SPARE','COACH','INJURED','HOLIDAY','SUSPENDED','SCOREKEEPER') DEFAULT 'ROSTER',
`t_access_level` enum('PLAYER','MANAGER','ASSISTANT') DEFAULT 'PLAYER',
`t_player_position` enum('ANY','FORWARD','DEFENCE','GOALIE','PITCHER','CATCHER','FIRST BASE','SECOND BASE','THIRD BASE','SHORTSTOP','LEFT FIELD','CENTER FIELD','RIGHT FIELD') DEFAULT 'ANY',
`t_spare_status` enum('INVITED','IN','OUT') DEFAULT NULL,
`t_drink_next` int(1) DEFAULT '0',
`t_no_fees` tinyint(1) DEFAULT '0',
`t_no_drink` tinyint(1) DEFAULT '0',
`t_auto_check_in` tinyint(1) DEFAULT '0',
`t_print_reminder` tinyint(1) DEFAULT '0',
`t_notes` text,
`t_last_chatter_id` int(11) DEFAULT NULL,
`t_last_history_id` int(11) DEFAULT NULL,
`t_last_active` timestamp NULL DEFAULT NULL,
`t_status` enum('ACTIVE','INACTIVE','ARCHIVED') DEFAULT 'ACTIVE',
`t_added` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
KEY `t_player_id` (`t_player_id`),
KEY `t_team_id` (`t_team_id`),
KEY `t_season_id` (`t_season_id`),
KEY `t_player_id_2` (`t_player_id`,`t_team_id`,`t_season_id`),
KEY `Team/Player ID` (`t_team_id`,`t_player_id`),
KEY `UpdatePlayersDiv` (`t_team_id`,`t_season_id`)
) ENGINE=InnoDB AUTO_INCREMENT=23454 DEFAULT CHARSET=latin1;
This simple update query takes on average, 3.5 seconds? I'm running this on a MediaTemple MySQL GRID container.
Here is the result of an EXPLAIN when switching the UPDATE to a SELECT.
Can someone provide some insight on how I'm not doing this correctly?
[EDIT: Added list of Indexes]
So is this a big mess of indexes? Do I have a bunch of redundant indexes in here?
Cheers,
Jon
It's using the wrong key instead of the index on the 3 columns. You can hint at indexes with USE KEY ... syntax.
https://dev.mysql.com/doc/refman/5.1/en/index-hints.html
You may also want to try reordering your key on the 3 columns. Generally, you want the most restricting column to be first in the index, then the next most restricting and so on.

MySql Properly Join Complex Data/Tables

Abstract:
Every client is given a specific xml ad feed (publisher_feed table). Everytime there is a query or a click on that feed, it gets recorded (publisher_stats_raw table) (Each query/click will have multiple rows depending on the subid passed by the client (We can sum the clicks together)). The next day, we pull stats from an API to grab the previous days revenue numbers (rev_stats table) (Each revenue stat might have multiple rows depending on the country of the click (We can sum the revenue together)). Been having a hard time trying to link together these three tables to find the average RPC for each client for the previous day.
Table Structure:
CREATE TABLE `publisher_feed` (
`publisher_feed_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`alias` varchar(45) DEFAULT NULL,
`user_id` int(10) unsigned DEFAULT NULL,
`remote_feed_id` int(10) unsigned DEFAULT NULL,
`subid` varchar(255) DEFAULT '',
`requirement` enum('tq','tier2','ron','cpv','tos1','tos2','tos3','pv1','pv2','pv3','ar','ht') DEFAULT NULL,
`status` enum('enabled','disabled') DEFAULT 'enabled',
`tq` decimal(4,2) DEFAULT '0.00',
`clicklimit` int(11) DEFAULT '0',
`prev_rpc` decimal(20,10) DEFAULT '0.0000000000',
PRIMARY KEY (`publisher_feed_id`),
UNIQUE KEY `alias_UNIQUE` (`alias`),
KEY `publisher_feed_idx` (`remote_feed_id`),
KEY `publisher_feed_user` (`user_id`),
CONSTRAINT `publisher_feed_feed` FOREIGN KEY (`remote_feed_id`) REFERENCES `remote_feed` (`remote_feed_id`) ON DELETE NO ACTION ON UPDATE NO ACTION,
CONSTRAINT `publisher_feed_user` FOREIGN KEY (`user_id`) REFERENCES `user` (`user_id`) ON DELETE NO ACTION ON UPDATE NO ACTION
) ENGINE=InnoDB AUTO_INCREMENT=124 DEFAULT CHARSET=latin1$$
CREATE TABLE `publisher_stats_raw` (
`publisher_stats_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`unique_data` varchar(350) NOT NULL,
`publisher_feed_id` int(10) unsigned DEFAULT NULL,
`date` date DEFAULT NULL,
`subid` varchar(255) DEFAULT NULL,
`queries` int(10) unsigned DEFAULT '0',
`impressions` int(10) unsigned DEFAULT '0',
`clicks` int(10) unsigned DEFAULT '0',
`filtered` int(10) unsigned DEFAULT '0',
`revenue` decimal(20,10) unsigned DEFAULT '0.0000000000',
PRIMARY KEY (`publisher_stats_id`),
UNIQUE KEY `unique_data_UNIQUE` (`unique_data`),
KEY `publisher_stats_raw_remote_feed_idx` (`publisher_feed_id`)
) ENGINE=InnoDB AUTO_INCREMENT=472 DEFAULT CHARSET=latin1$$
CREATE TABLE `rev_stats` (
`rev_stats_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`date` date DEFAULT NULL,
`remote_feed_id` int(10) unsigned DEFAULT NULL,
`typetag` varchar(255) DEFAULT NULL,
`subid` varchar(255) DEFAULT NULL,
`country` varchar(2) DEFAULT NULL,
`revenue` decimal(20,10) DEFAULT NULL,
`tq` decimal(4,2) DEFAULT NULL,
`finalized` int(11) DEFAULT '0',
PRIMARY KEY (`rev_stats_id`),
KEY `rev_stats_remote_feed_idx` (`remote_feed_id`),
CONSTRAINT `rev_stats_remote_feed` FOREIGN KEY (`remote_feed_id`) REFERENCES `remote_feed` (`remote_feed_id`) ON DELETE NO ACTION ON UPDATE NO ACTION
) ENGINE=InnoDB AUTO_INCREMENT=58 DEFAULT CHARSET=latin1$$
Context:
Each remote_feed has a specific subid/typetag given to it. So we need to match up the both the remote_feed_id and the subid columsn from the publisher_feed table to the remote_feed_id and typetag columns in the revenue stats table.
My current, non working, implementation:
SELECT
pf.publisher_feed_id, psr.date, sum(clicks), sum(rs.revenue)
FROM
xml_network.publisher_feed pf
JOIN
xml_network.publisher_stats_raw psr
ON
psr.publisher_feed_id = pf.publisher_feed_id
JOIN
xml_network.rev_stats rs
ON
rs.remote_feed_id = pf.remote_feed_id
WHERE
pf.requirement = 'tq'
AND
pf.subid = rs.typetag
AND
psr.date <> date(curdate())
GROUP BY
psr.date
ORDER BY
psr.date DESC
LIMIT 1;
The above keeps pulling the wrong data out of the rev_stats table (pulls the sum of the correct stats, but repeats it over because of a join). Any help with how I would be able to properly pull the correct data would be greatly helpful ( I could use multiple queries and PHP to get the correct results, but what's the fun in that!)
Figured out a way to get this accomplished. Its def not a fast method by any means, needing 4 selects to get it done, but it works flawlessly =)
SELECT
pf.publisher_feed_id,
round(
(
SELECT
SUM(rs.revenue)
FROM
xml_network.rev_stats rs
WHERE
rs.remote_feed_id = pf.remote_feed_id
AND
rs.typetag = pf.subid
AND
rs.date = subdate(current_date, 1)
),10)as revenue,
(
SELECT
MAX(rs.tq)
FROM
xml_network.rev_stats rs
WHERE
rs.remote_feed_id = pf.remote_feed_id
AND
rs.typetag = pf.subid
AND
rs.date = subdate(current_date, 1)
) as tq,
(
SELECT
SUM(psr.clicks)-SUM(psr.filtered)
FROM
xml_network.publisher_stats_raw psr
WHERE
psr.publisher_feed_id = pf.publisher_feed_id
AND
psr.date = subdate(current_date, 1)
) as clicks
FROM
xml_network.publisher_feed pf
WHERE
pf.requirement = 'tq';

Sphinx PHP One-to-Many Search

I have the following mysql tables:
CREATE TABLE `video` (
`video_id` int(11) unsigned NOT NULL auto_increment,
`title` varchar(255) NOT NULL default '',
`description` text NOT NULL,
PRIMARY KEY (`video_id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
CREATE TABLE `video_categories` (
`cat_id` int(11) unsigned NOT NULL auto_increment,
`name` varchar(255) NOT NULL default '',
PRIMARY KEY (`cat_id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
CREATE TABLE `video_category` (
`video_id` int(11) unsigned NOT NULL default '0',
`cat_id` int(11) unsigned NOT NULL default '0',
KEY `video_id` (`video_id`),
KEY `cat_id` (`cat_id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
CREATE TABLE `video_tags` (
`tag_id` int(11) unsigned NOT NULL auto_increment,
`video_id` int(11) unsigned NOT NULL default '0',
`name` varchar(255) NOT NULL default '',
KEY `video_id` (`video_id`),
KEY `name` (`name`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
I created a sphinx configuration file and i can search from PHP. The problem is when i want to search for related videos, a related video must be in the same category as the video i'm searching for. I can do this with MVA and and SetFilter('categories', array(3)) for example, however the total number of matches results is the global one (i need total to display pagination via ajax) not the one in the category.
Any ideas how i can search through videos (documents in sphinx) that are only in a specified category?
Thanks,
Adrian.
You can define for the category ID an integer attribute in Sphinx:
sql_attr_uint = cat_id
And then just add #cat_id=12345 to your query.

Categories