I need to export 20 million data in a pdf file. I have used the "WkHtmlToPdf" package
(https://github.com/mikehaertl/phpwkhtmltopdf)
in my yii2 application. I have used Mongo database, I have created a report module in which I need to export the pdf with large data.
Code is working fine but it is taking too much time to write 1,00,000 it takes approx 10 mins so for 20 million approx it takes 20 hours so we are looking for a solution to write 20 million data.
Below is my query to fetch data from mongo:
$query = CannedReport::find()->where(['isParent'=>1]);
$this->load($params);
if (in_array(Yii::$app->user->identity->user_type, ['tenant', 'subtenant'])) {
$query->andWhere(['tm_id' => (string)Yii::$app->user->identity->tm_id]);
// $match['tm_id'] =(string)Yii::$app->user->identity->tm_id;
}
$query->andWhere(['!=', 'callstatus', 'completed']);
$query->andWhere(['hangup' => 'ORIGINATOR_CANCEL']);
$dataProvider = new ActiveDataProvider([
'query' => $query,
'pagination' => false,
'sort' => [
'defaultOrder' => ['start_epoch' => SORT_DESC],
],
]);
if (isset($this->em_id) && !empty($this->em_id)) {
$query->andFilterWhere(['in', 'em_id', $this->em_id]);
}
foreach (['start_epoch', 'end_epoch'] as $epoch) {
if ($epoch == 'start_epoch') {
if (isset($this->start_epoch) && !empty($this->start_epoch)) {
$date = explode(' - ', $this->start_epoch);
} else {
$date[0] = date('Y-m-d H:i:s', strtotime('today - 31 days'));
$date[1] = date('Y-m-d') . " 23:59:59";
$this->start_epoch = implode(' - ', $date);
}
} else {
$date = explode(' - ', $this->{$epoch});
}
if (isset($date[0]) && isset($date[1]) &&
Yii::$app->helper->validateDate($date[0], 'Y-m-d H:i:s') && Yii::$app->helper->validateDate($date[1],
'Y-m-d H:i:s')
) {
$query->andFilterWhere([
'>=',
$epoch,
(string)Yii::$app->helper->currentTimezone($date[0], 'Y-m-d H:i:s')
]);
$query->andFilterWhere([
'<=',
$epoch,
(string)Yii::$app->helper->currentTimezone($date[1], 'Y-m-d H:i:s')
]);
}
unset($date);
}
return $dataProvider;
Pdf generation code
$pdf = new Pdf(array(
'binary' => '/usr/local/bin/wkhtmltopdf',
'ignoreWarnings' => true,
'commandOptions' => array(
'useExec' => true, // Can help on Windows systems
'procEnv' => array(
'LANG' => 'en_US.utf-8',
),
),
));
$options = array(
'dpi' => 96,
'image-quality' => 100,
'disable-smart-shrinking',
'no-outline',
);
$pdf->setOptions($options);
$pdf->addPage($tableHtml);
$detailHeadingHtml = '<div class="row" style="padding-top: 5px;"><h3 style="text-align: center; padding: 5px;">Detailed Records-Call Direction Report</h3><div>';
$pdf->addPage($detailHeadingHtml . '' . $abandonedCallsHtml . '' . $completedCallsHtml . '' . $failedCallsHtml . '' . $transferredCallsHtml . '<table><tr><td align="center"><b>Total Count:</b></td><td align="center"><b>' . $dataProvider->query->count() . '</b></td></tr></table>');
if (!$pdf->send($fileName)) {
$error = $pdf->getError();
echo $error;
}
exit();
Is there any optimized way to create a pdf with such large data using yii2 and mongo?
Related
public function storeActivity(Request $request)
{
$request['time']=date("Y-m-d h:i:s a", time());
$this->validate($request,[
'description1' => 'required',
'description2' => 'nullable',
'description3' => 'nullable',
'description4' => 'nullable',
]);
$emp_id = 'ims12933';
//$punch_in = $request['time']=date("Y-m-d h:i:s a", time());
$session_id = $_COOKIE["laravel_session"];
$post = new activity;
$post->emp_id = $emp_id;
$post->punch_in = $request['time']=date("Y-m-d h:i:s", time());
$post->punch_date = $request['time']=date("Y-m-d", time());
$post->sess_id = $session_id;
$post->description1 = $request->description1;
$post->description2 = $request->description2;
$post->description3 = $request->description3;
$post->description4 = $request->description4;
$post->save();
return redirect('home')->with('Successfully', 'Punch In');
}
This is I am submitting but can't take time out data in a single table of database.
DB::table('planned_activites')
->where('sess_id', $sess_id) // find your user by their email
->limit(1) // optional - to ensure only one record is updated.
->update(array('punch_out' => date("Y-m-d h:i:s", time()), 'ip_out' => \Request::getClientIp(true), 'description2' => $request->description2, 'description3' => $request->description3));
Mainly update function will update the same table.
I have one CSV file that is use to upload product in my site. So there I have image_url but when I try to upload the CSV the product gets added but the image is not uploaded. I'm using cURL to get the image from the url but it is not giving the UploadedFile Instance, So I think that could be the problem. So Can anyone give me any solution. Following is my code:-
public function import_product(Request $request)
{
$attributes = $request->all();
$path = $request->file('products_csv')->getRealPath();
$data = \Excel::toArray('', $path, null, \Maatwebsite\Excel\Excel::TSV)[0];
foreach ($data as $key => $value) {
if($key == 0) continue;
$checksku = $this->product->checksku($value[1]);
if($checksku == 1){
$notification = array(
'message' => 'SKU must be unique.',
'alert-type' => 'error'
);
return redirect('/products-section/import-view')->with($notification);
}
if($value[4] < 0){
$notification = array(
'message' => 'Price must be a positive number',
'alert-type' => 'error'
);
return redirect('/products-section/import-view')->with($notification);
}
if($value[5] < 0){
$notification = array(
'message' => 'Cost must be a positive number',
'alert-type' => 'error'
);
return redirect('/products-section/import-view')->with($notification);
}
if(!filter_var($value[6], FILTER_VALIDATE_INT)){
$notification = array(
'message' => 'Stock must be a positive integer',
'alert-type' => 'error'
);
return redirect('/products-section/import-view')->with($notification);
}
if($value[2] < 0){
$notification = array(
'message' => 'Shipping cost must be a positive number',
'alert-type' => 'error'
);
return redirect('/products-section/import-view')->with($notification);
}
$product = $this->product->create([
'title' => $value[0],
'sku' => $value[1],
'price' => round($value[4], 2),
'cost' => round($value[5], 2),
'stock_level' => $value[6],
'description' => '<p>' . $value[11] . '</p>',
'description_short' => '<p>' . $value[12] . '</p>',
'date_created' => date('Y-m-d'),
'date_updated' => date('Y-m-d'),
'supplier_id' => $attributes['supplier_id'],
'brand_id' => 1,
'status' => 1,
'taxable' => 0,
'shipping_cost' => round($value[2], 2),
'shipping_type' => $value[3],
'special_teamwear_category' => $value[7] == 'on' ? 1 : 0,
'message_artwork' => $value[8] == 'on' ? 1 : 0,
'extra_info' => $value[10] == 'on' ? 1 : 0,
'is_strike' => $value[9] == 'on' ? 1 : 0,
'is_deals' => $value[13] == 'on' ? 1 : 0
]);
ProductNew::saveproduct($product->id);
$product->categories()->create(['category_id' => $attributes['product_categories']]);
$url = $value[14];
$extension = pathinfo($url, PATHINFO_EXTENSION);
$im = ProductImage::create(['product_id' => $product->id]);
$path = 'product/' . (int)($product->id / 5000 + 1) . '/' . $product->id;
$image_name = str_slug($product->title) . "-" . $im->id . "." . $extension;
// $file = file_get_contents($url);
// file_put_contents()
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
$file = curl_exec($ch);
curl_close($ch);
$imageInfo = getimagesizefromstring($file);
$width = $imageInfo[0];
$height = $imageInfo[1];
if($width > config('const.image_size.product.width') && $height > config('const.image_size.product.height')){
$notification = array(
'message' => "Image must have dimensions:". config('const.image_size.product.width').' x '.config('const.image_size.product.height') .'px',
'alert-type' => 'error'
);
return redirect('/products-section/import-view')->with($notification);
}
$file_saved = Storage::disk('public')->putFileAs($path, $file, $image_name);
$im->path = $path;
$im->name = $image_name;
$im->copied = $file_saved;
$im->save();
}
$notification = array(
'message' => 'All of the products imported successfully.',
'alert-type' => 'success'
);
return redirect('/products-section/products/')->with($notification);
}
I get the following error :-
Please help me. Any help will be appreciated. thanks in advance.
Edit
I am using the below code instead of putFileAs
file_put_contents("http://outsourcingit.asia/clubmart_storage/storage/$path", $file);
But I'm having this error:-
I am running Active Collab 5.8.7 with PHP 5.6. I am using the API to create an invoice. I am following the API per this documentation. My issue is that the post requires an invoice number and I have no way of telling what the next invoice number should be.
I would like to know if there is a way to use the API to get the next invoice number inline or to add an invoice and let the system pick an invoice number for you.
Currently, I am using a random sequence of characters when I create an invoice through the API and then when we try to manually add an invoice, the invoice number field is blank. There has to be an easier and more consistent way to handle invoice numbers.
Thanks,
Larry
I figured this out on my own and wanted to post it here in case someone else needs it:
//get next invoice number
function get_next_ac_invoice_number($client) {
//get all invoices
$result = $client->get('/reports/run?type=InvoicesFilter')->getJson();
$invoices = $result['all']['invoices'];
usort($invoices,'sortbyInvoiceId');
$next_invoice_id = strval(intval(explode('-',$invoices[0]['number'])[0]) + 1) . '-' . date("Y");
return $next_invoice_id;
}
function sortbyInvoiceId($a,$b){
if ($a == $b) {
return 0;
}
return ($b < $a) ? -1 : 1;
}
EDIT FEB 22 '17
There is a bunch of cruft in here for Braintree but you should be able to get the gist of it. One thing I would make sure you consider is that you clear out AC's trash before posting since the next invoice id function will not return trashed invoices and will cause a duplicate invoice id error.
//get next invoice number
function get_next_ac_invoice_number($client) {
#get all invoices
$trashit = $client->delete('/trash');
$result = $client->get('/reports/run?type=InvoicesFilter')->getJson();
$invoices = $result['all']['invoices'];
usort($invoices,'sortbyInvoiceId');
$next_invoice_id = strval(intval(explode('-',$invoices[0]['number'])[0]) + 1) . '-' . date("Y");
return $next_invoice_id;
}
//creates an invoice in active collab
function create_ac_invoice($customer, $subscription, $transaction, $client) {
//get the next invoice ID
get_next_ac_invoice_number($client);
$plans = Braintree_Plan::all();
$plan;
foreach ($plans AS $myplan) {
if (strtolower($myplan->id) == strtolower($subscription->planId)) {
$plan = $myplan;
}
}
if (isset($transaction->discounts[0])) {
$result = $client->post('invoices', [
'company_id' => $customer['company_id'],
'number' => get_next_ac_invoice_number($client),
'items' => [
[
'description' => $plan->name . " - " . $plan->description,
'quantity' => 1,
'unit_cost' => $subscription->price
],
[
'description' => 'Promo Code Discount - ' . $transaction->discounts[0]->name,
'quantity' => 1,
'unit_cost' => (float) $transaction->discounts[0]->amount * -1
]
],
'private_note' => 'Auto-generated by Braintree'
]);
} else {
$result = $client->post('invoices', [
'company_id' => $customer['company_id'],
'number' => get_next_ac_invoice_number($client),
'items' => [
[
'description' => $plan->name . " - " . $plan->description,
'quantity' => 1,
'unit_cost' => $subscription->price
]
],
'private_note' => 'Auto-generated by Braintree'
]);
}
$invoice = $result->getJson();
if (isset($invoice['message'])) {
//we have an error, let's log and send email
$dump = print_r($invoice, true) . print_r($customer, true) . print_r($subscription, true);
logit('ERROR', $dump);
sendEmail($dump, 'Braintree Webhook Error Creating AC Invoice');
}
//mark the invoice as paid
$result = $client->post('payments', [
'parent_type' => 'Invoice',
'parent_id' => $invoice['single']['id'],
'amount' => getTotalCost($subscription, $transaction),
'comment' => 'Paid in full'
]);
$result = $result->getJson();
if (isset($result['message'])) {
//we have an error, let's log and send email
$dump = print_r($invoice, true) . print_r($customer, true) . print_r($subscription, true);
logit('ERROR', $dump);
sendEmail($dump, 'Braintree Webhook Error Creating AC Payment');
}
//send the invoice
$result = $client->put('invoices/' . $invoice['single']['id'] . '/send', [
'recipients' => [
$customer['email']
],
'subject' => "New Invoice",
'message' => "Thanks!",
'allow_payments' => 2
]);
$result = $result->getJson();
if (isset($result['message'])) {
//we have an error, let's log and send email
$dump = print_r($invoice, true) . print_r($customer, true) . print_r($subscription, true);
logit('ERROR', $dump);
sendEmail($dump, 'Braintree Webhook Error Sending AC Invoice Email');
}
return $invoice;
}
I have the following function :
public function check_daily_data() {
$month = $this->input->post('month');
echo 'Month :' . $month . '</br>';
$year = $this->input->post('year');
echo 'Year :' . $year . '</br>';
$user_id = $this->session->userdata('employee_id');
echo 'User Id : ' . $user_id . '</br>';
//load the cookie helper
$this->load->helper('cookie');
setcookie("month", "", time()-3600);
setcookie("year", "", time()-3600);
$time = time()+3600;
$cookie_month = array(
'name' => 'cookie_month',
'value' => $month,
'expire' => '3600',
'secure' => TRUE
);
$this->input->set_cookie($cookie_month);
$cookie_year = array(
'name' => 'cookie_year',
'value' => $year,
'expire' => '3600',
'secure' => TRUE
);
$this->input->set_cookie($cookie_year);
var_dump($this->input->cookie('cookie_month'));
$cookie_months = $this->input->cookie('cookie_month',TRUE);
echo 'Cookie Month : ' . $cookie_months . '</br>';
$cookie_years = $this->input->cookie('cookie_year',TRUE);
echo 'Cookie Year : ' . $cookie_years . '</br>';
}
When I run the script and check the cookie variable using FireBug, I can be able to view the cookie data but when I try to echo it, I get a blank value.
The var_dump returns a false, please can you advise what am I doing wrong?
well, why you are not getting value is because, you have set 'secure' => TRUE. you need to set it up to false. the true setting will work only with HTTPS
$cookie_month = array(
'name' => 'cookie_month',
'value' => $month,
'expire' => '3600',
'secure' => FALSE
);
$cookie_year = array(
'name' => 'cookie_year',
'value' => $year,
'expire' => '3600',
'secure' => FALSE
);
then you will get your value when you call
$cookie_months = $this->input->cookie('cookie_month',TRUE);
gform_after_submission is getting called in my page, but the $entry and $form objects are null. Is there any reason why that would be happening? Based on the output of the logs, I know the code runs, but can't figure out why the $entry argument is null.
add_action('gform_after_submission_2', 'post_to_third_party', 10, 2);
function post_to_third_party($entry, $form) {
error_log("Posting comments form");
$post_url = 'https://api.club-os.com/prospects?clubLocationId=686';
$body = array(
'first_name' => $entry['7.3'],
'last_name' => $entry['7.6'],
'email' => $entry['6'],
'mobilePhone' => $entry['8']
);
error_log('Before Post to' . $post_url);
$args = array(
'headers' => array('Authorization' => 'Basic ' . base64_encode( 'username' . ':' . 'password' )),
'body' => $body,
'sslverify' => false
);
foreach ($body as $key => $value) {
error_log($value . " in " . $key . ", ");
}
$request = new WP_Http();
$response = $request->post($post_url, $args);
}
Figured it out.
'first_name' => $entry['7.3'],
'last_name' => $entry['7.6'],
The 7.3 and 7.6 were incorrect indexes to the $entry object. I just used '7' and the split function to get it to work correctly.