I need to cache the results from Steam API parsing. And so the cached result lasts 15 minutes. I have a code:
public function load()
{
if (Auth::guest()) return response()->json(['success' => false, 'msg' => 'You need login!']);
$inventory = $this->getInventory(file_get_contents('http://steamcommunity.com/inventory/' . $this->user->steamid64 . '/570/2?l=russian&count=5000', true));
if (!$inventory) {
return response()->json(['success' => false, 'msg' => 'Error']);
}
$items = [];
$items_with_prices = json_decode(\Storage::get('prices.txt'));
$items_with_prices_by_key = [];
foreach ($items_with_prices->items as $item_price_key => $item_price_data) {
$items_with_prices_by_key[$item_price_key] = $item_price_data->price;
}
foreach ($inventory['rgInventory'] as $info) {
$item = $inventory['rgDescriptions'][$info['classid'] . '_' . $info['instanceid']];
if ($item['tradable'] == 0) continue;
$price = 0;//InventoryController::getItemPrice($item['market_hash_name']);
if (array_key_exists($item['market_hash_name'], $items_with_prices_by_key)) {
$price = $items_with_prices_by_key[$item['market_hash_name']];
}
if (!$price) continue;
if ($price < 1) $price = 0.64;
$type = $this->getType($item['type']);
$items[] = [
'id' => $info['id'],
'classid' => $item['icon_url'],
'price' => round($price, 2),
'type' => $type
];
}
usort($items, function($a, $b){
return ($b['price'] - $a['price']);
});
return response()->json(['success' => true, 'items' => $items]);
}
This code only works when a site user clicks on the "show my items" button and a request is sent to the list of user items in Steam Dota 2. Now if click constantly to get a list of items, Steam can block the server’s IP address for 24 hours. As I understand it, I need to throw the result of a $inventory variable into the cache. I create database table cache with fields id, user_id, items, date.
How can I now cache the result from a $inventory variable of 15 minutes?
Here is basic caching in laravel
$rememberTimeInSeconds = 3600;
$cachedResult = Cache::remember('name_of_your_cache', $rememberTimeInSeconds, function(){
// It can be any logic I just showing a simple query.
$users = User::all();
return $users;
});
I have built an Employee Management System using Laravel 5.7. Calculating salary works fine for 100-150 employees (but taking a long time to process the data) & for more than 150 employees it is showing a time out error.
I want to add the feature to calculate the salary of all employees in one go & also reduce the processing time for the same. Would it be appropriate to use chunk() for that? If yes, how do I implement that?
Note:
The web application is for a BPO so there are so many types of logic to apply while calculating a salary. The code is too lengthy to share here, but if anyone wants to see the code, I can share it.
Front-end form screenshot
Page after calculation
<?php
public function calculate_salary(Request $request){
$this->validate($request,[
'employee_id' => 'required',
'attendance_types' => 'required',
'month' => 'required_without:choose_date',
'choose_date' => 'required_without:month'
]);
$download_salary = false;
$submit_salary = false;
if ($request->has('submit_data'))
{
$submit_salary = true;
}
else
{
$submit_salary = false;
}
if ($request->has('download_data'))
{
$download_salary = true;
}
else
{
$download_salary = false;
}
$dept = $request->input('salary_department');
$role = $request->input('salary_role');
$process = $request->input('salary_process');
$salary_dept = Department::where('id', $dept)->pluck('dept_name')->first();
$salary_role = Role::where('id', $role)->pluck('role_name')->first();
$salary_process = Process::where('id', $process)->pluck('process_name')->first();
if ($submit_salary == true || $download_salary == true)
{
$months = $request->input('month');
$month = $months[0];
$employee_ids = $request->input('employee_id');
$attendance_types = $request->input('attendance_types');
$attendance_type_temp_insert = $attendance_types[0];
$attendance_type = explode(',', $attendance_type_temp_insert);
if (!empty($request->input('choose_date')))
{
$dates = $request->input('choose_date');
$date = $dates[0];
}
else
{
$date = "";
}
}
else
{
$month = $request->input('month');
$employee_ids = $request->input('employee_id');
$attendance_type = $request->input('attendance_types');
$attendance_type_temp_insert = implode(',', $request->input('attendance_types'));
$date = $request->input('choose_date');
}
// Get start & end dates
$data = $this->get_salary_dates($month, $date);
$start_date = $data['start_date'];
$end_date = $data['end_date'];
$salary_start_date = $start_date->toDateString();
$salary_end_date = $end_date->toDateString();
$working_days_by_month = $this->get_working_days_of_given_month($start_date, $end_date);
$working_days = $working_days_by_month['working_days'];
$no_days = $working_days_by_month['no_days'];
Schema::create('temp_salary', function (Blueprint $table) {
$table->increments('id');
$table->integer('employee_id');
$table->string('employee_name');
$table->integer('working_days');
$table->float('emp_working_days');
$table->float('basic');
$table->float('hra');
$table->float('conveyance');
$table->float('spcl_inc');
$table->float('gross_salary');
$table->float('pf');
$table->float('esi');
$table->integer('headphone');
$table->integer('id_card');
$table->float('deductions');
$table->integer('pl_balance');
$table->integer('combo_balance');
$table->float('net_payable');
$table->string('month')->nullable();
$table->string('attendance_types');
$table->string('choose_date')->nullable();
$table->string('department')->nullable();
$table->string('role')->nullable();
$table->string('process')->nullable();
$table->timestamps();
$table->temporary();
});
$csv[] = ['Employee ID', 'Name','Department','Role','Process','Bank','Bank Account Number', 'Basic', 'HRA', 'Conveyance', 'Spcl Inc', 'Gross Salary','PF', 'ESI', 'Deductions', 'Total Working Days', 'Net Payable','Salary Month', 'Salary Start Date','Salary End Date'];
$csv1[] = ['Employee ID', 'Name','Department','Role','Process','Bank','Bank Account Number', 'Basic', 'HRA', 'Conveyance', 'Spcl Inc', 'Gross Salary','PF', 'ESI', 'Deductions', 'P','H','A','WO','WOP','WOH','UAL','Paid Leaves','PL','Combo','ual_deduct','Total Working Days', 'Net Payable','Salary Month', 'Salary Start Date','Salary End Date'];
foreach($employee_ids as $id) {
if($submit_salary == true){
DB::table('attendances')->where('employee_id', '=', $id)->whereBetween('attendance_date', [$start_date, $end_date])->update(['salary_status' => 1]);
}
$employee = Employee::find($id);
$name = $employee->name;
$emp_role = $employee->empRole->role_name;
$emp_department = $employee->empDepartment->dept_name;
$emp_process = $employee->empProcess->process_name;
$bank_account_number = $employee->bank_account_number;
$bank_name = $employee->bank_name;
$salary = Salary::where('employee_id', $id)->first();
$pl_balance = $salary->pl_balance;
$combo_balance = $salary->combo_balance;
$attendances = DB::table('attendances')->where('employee_id', '=', $id)->get();
if ($attendances->count() > 0)
{
foreach($attendances as $attendance)
{
$att_id = $attendance->id;
$att_date = Carbon::parse($attendance->attendance_date);
$dialer_in = strtotime($attendance->dialer_in_time);
$dialer_out = strtotime($attendance->dialer_out_time);
$bio_in = strtotime($attendance->biometric_in_time);
$bio_out = strtotime($attendance->biometric_out_time);
$bio_diff = $bio_out - $bio_in;
$crm_in = strtotime($attendance->crm_in_time);
$crm_out = strtotime($attendance->crm_out_time);
$week_off = $attendance->week_off;
$combo = $attendance->combo;
$holiday = $attendance->holiday;
$ual = $attendance->ual;
$dialer_duration = $attendance->dialer_difference;
$bio_duration = $attendance->biometric_difference;
$crm_duration = $attendance->crm_difference;
$chkatt = $this->get_attendance($attendance_type, $attendance->dialer_in_time, $attendance->dialer_out_time,$attendance->crm_in_time,$attendance->crm_out_time, $dialer_duration, $bio_duration, $crm_duration);
$attendance_status = $this->attendance_status($id, $chkatt, $holiday, $week_off, $ual);
DB::table('attendances')->where('id', $att_id)->update(['attendance_status' => $attendance_status]);
//echo $attendance_status;
}
}
$totalWorkingDays = $this->get_work_days($id,$start_date,$end_date,$combo_balance,$pl_balance, $submit_salary);
$totalWorkingDays = number_format((float)$totalWorkingDays, 2, '.', '');
$salary_data = $this->get_calculated_salary_data($no_days, $totalWorkingDays, $id, $submit_salary);
$basic = $salary_data['basic'];
$hra = $salary_data['hra'];
$conveyance = $salary_data['conveyance'];
$spcl_inc = $salary_data['spcl_inc'];
$gross_salary = $salary_data['gross_salary'];
$pf = $salary_data['pf'];
$esi = $salary_data['esi'];
$hp_charges = $salary_data['hp_charges'];
$idcard_charges = $salary_data['idcard_charges'];
$deductions = $salary_data['deductions'];
$net_payable = $salary_data['net_payable'];
DB::table('temp_salary')->insert(['employee_id' => $id, 'employee_name' => $name, 'working_days' => $working_days, 'emp_working_days' => $totalWorkingDays, 'basic' => $basic, 'hra' => $hra, 'conveyance' => $conveyance, 'spcl_inc' => $spcl_inc, 'gross_salary' => $gross_salary, 'pf' => $pf, 'esi' => $esi, 'headphone' => $hp_charges, 'id_card' => $idcard_charges, 'deductions' => $deductions, 'pl_balance' => $pl_balance, 'combo_balance' => $combo_balance, 'net_payable' => $net_payable, 'month' => $month, 'attendance_types' => $attendance_type_temp_insert, 'choose_date' => $date, 'department' => $salary_dept, 'role' => $salary_role, 'process' => $salary_process]);
$csv[] = [$id, $name, $emp_department, $emp_role, $emp_process, $bank_name, $bank_account_number, $basic, $hra, $conveyance, $spcl_inc, $gross_salary, $pf, $esi, $deductions, $totalWorkingDays, $net_payable, $month, $start_date, $end_date];
$get_days_status = $this->get_days_status($id, $start_date, $end_date);
$pCount = $get_days_status['pCount'];
$HalfDaysCount = $get_days_status['hCount'];
$lateLeaveCount = $get_days_status['late_leave'];
$wopCount = $get_days_status['wopCount'];
$leaves_count = $get_days_status['leaves_count'];
$UalCount = $get_days_status['UalCount'];
$woCount = $get_days_status['woCount'];
$wohCount = $get_days_status['wohCount'];
$ual_deduct = ($UalCount * 1.5) - $UalCount;
$get_paid_leaves_row = $get_days_status['pl_leaves'];
$csv1[] = [$id, $name, $emp_department, $emp_role, $emp_process, $bank_name, $bank_account_number, $basic, $hra, $conveyance, $spcl_inc, $gross_salary, $pf, $esi, $deductions, $pCount,$HalfDaysCount,$leaves_count,$woCount,$wopCount,$wohCount,$UalCount,$get_paid_leaves_row,$pl_balance, $combo_balance,$ual_deduct, $totalWorkingDays, $net_payable, $month, $start_date, $end_date];
}
$datas = DB::table('temp_salary')->get();
if($submit_salary == true){
return Excel::create('Employee_salary_report', function($excel) use ($csv) {
$excel->sheet('Employee_salary_report', function($sheet) use ($csv) {
$sheet->fromArray($csv, null, 'A1', false, false)
->getStyle('A1')
->getAlignment()
->setWrapText(true);
});
})->download('csv');
}
if($download_salary == true){
return Excel::create('salary_report', function($excel) use ($csv1) {
$excel->sheet('salary_report', function($sheet) use ($csv1) {
$sheet->fromArray($csv1, null, 'A1', false, false)
->getStyle('A1')
->getAlignment()
->setWrapText(true);
});
})->download('csv');
}
Schema::drop('temp_salary');
$departments = Department::all();
$processes = Process::all();
$roles = Role::all();
return view('sys_mg.salaries.get-salary')->with(['datas'=>$datas,'departments' => $departments, 'processes' => $processes, 'roles' => $roles, 'salary_dept' => $salary_dept, 'salary_role' => $salary_role, 'salary_process'=>$salary_process, 'salary_month'=>$month,'salary_startDate' => $salary_start_date, 'salary_endDate' => $salary_end_date, 'attendance_check_type' => $attendance_type]);
}
This is a huge process, doing many things at once, some of them deeply nested in each other.
The function is very, very long, which means it probably needs to be abstracted into various methods and classes (better OOP).
You may also be running into the N+1 problem with some of your Laravel queries, although it's hard to say at a glance.
I would recommend using Laravel's Queues, and adding each employee's payroll calculation to the queue: https://laravel.com/docs/5.8/queues
You can then use a worker process to perform each one individually.
All the database updates and transactions means PHP is constantly going to-and-fro from your database. Try to perform as many operations in pure code as possible, and then once completed, write to the DB. (Where possible, this is not a blanket rule).
Initially I would say, consider creating a SalaryCalculator class with methods such as getAttendances() and calculateSalaryFromEmployeeAttendances().
Separately, create a CSV exporter class. Process all your payroll calculations first, store the results in the DB, and then convert to CSV on demand later.
You'll refactor this later once you are able to look at all the different parts, someone else may be able to suggest a better way to break it down, but in some ways there are no right answers... just start with some OOP and abstraction of methods, and it will get better.
Use microtime(true) to get and calculate the time differences between when you start and finish operations, to start to track how long each function runs for... and go looking for the big optimisations first. What is the slowest part? Why?
You could probably ask a dozen Stack Overflow questions to optimise each of those methods, and that's Ok!
So I've spent hours on trying to fix this one. every time I upload a file. all the data/id3 tags are being inserted twice on my database. before it was working properly now it has this bug and I badly need some help now.
In my controller I am executing update at the same time since I want also to update the data inserted into database with the details from the id3 tags.
Controller
public function save()
{
$id = $this->session->userdata('user_id');
$status = $this->input->post('status');
$original_file = $this->input->post('original_file');
$revised_file = $this->input->post('revised_file');
$song_data = array(
'user_id' => $id,
'song_info_status' => $status,
'song_info_file_name' => $original_file,
'song_info_revised_file_name' => $revised_file,
'song_info_date_added' => 'now()',
'song_info_date_updated' => 'now()'
);
$song_info_id = $this->song_info_model->save($song_data);
$composer_data = array(
'composer_name' => ''
);
$composer_id = $this->composer_model->save($composer_data);
$song_composer_data = array(
'song_info_id' => $song_info_id,
'composer_id' => $composer_id
);
$this->song_composer_model->save($song_composer_data);
$status_data = array(
'song_info_id' => $song_info_id
);
$status_id = $this->status_model->save($status_data);
$tags_info = array();
$tags_info = $this->getTagsInfo(FCPATH ."temp" . "/" .$revised_file);
if(isset($tags_info['Author']))
{
// echo iconv("UTF-8","EUC-JP", $tags_info['Author']);
//echo $str = mb_convert_encoding($tags_info['Author'], "UTF-8", "auto");
$status_data = array(
'status_artist_name' => $tags_info['Author']
);
$this->status_model->update($status_data, $status_id);
}
if(isset($tags_info['Title']))
{
$song_data = array(
'song_info_original_title' => $tags_info['Title']
);
$this->song_info_model->update($song_data, $song_info_id);
}
rename($this->source.$revised_file, $this->destination.$revised_file);
}
In my model I am just getting the data I have from my controller. there's no loop or anything and I really can't find the culprit.
Model
public function save($data)
{
$this->db->insert('song_info', $data);
return $this->db->insert_id();
}
public function update($data, $id)
{
$this->db->where('song_info_id', $id);
$this->db->update('song_info', $data);
}
I have a custom query in my repository that gets me results be accepting zipcodes or citynames as a value.
These values are served to an autocomplete text input.
public function findByZipOrCity($cz)
{
$qb = $this->createQueryBuilder('z');
if (substr($cz, 0, 1) == "0") {
$cz = substr($cz, 1);
$qb
->select('z')
->where($qb->expr()->orX(
$qb->expr()->like('z.city', ':czRequest'),
$qb->expr()->like('z.code', ':czRequest')
))
->andWhere('z.code <= :smaller')
->setParameter('czRequest', $cz . '%')
->setParameter('smaller', 9999);
} else {
$qb
->select('z')
->where($qb->expr()->orX(
$qb->expr()->like('z.city', ':czRequest'),
$qb->expr()->like('z.code', ':czRequest')
))
->setParameter('czRequest', $cz . '%');
}
return $qb->getQuery()->getArrayResult();
}
I work with this ArrayResult() to output JSON that the autosuggest plgin accepts:
$zipcodes = $this->getDoctrine()->getManager()->getRepository("AppBundle:Zip")->findByZipOrCity($zip);
$response = new JsonResponse();
$codes = array();
foreach ($zipcodes as $zipcode) {
$codes[] = array(
'id' => $zipcode['id'],
'country' => $zipcode['country'],
'city' => $zipcode['city'],
'code' => sprintf('%05d', $zipcode['code']),
);
}
$response->setData($codes);
return $response;
so querying for 'berli' gives the above result, the autosuggest looks like this:
This works fine, now I want to alter (any of the) function to have it displayed like this:
Which of the functions do I have to change? Can I group the query already (I need one valid zip though) or just do 'something' with the array afterwards?
Any hint highly appreciated!
You can change in array like follow:
foreach ($zipcodes as $zipcode) {
$codes[$zipcode['id']] = array(
'id' => $zipcode['id'],
'country' => $zipcode['country'],
'city' => $zipcode['city'],
'code' => sprintf('%05d', $zipcode['code']),
);
}
I'm trying to prevent the user to create the same username. well my real problem is how to loop a list of data from model in controller. Maybe we know how to loop it in view by using this -> data['user'] and in view we can call $user. but how can we do that in controller layer.
here's my code
Controller
$username = strtolower($this->input->post('name'));
$fixUsername = str_replace(" ",".",$username);
$counter = 1;
$list[] = $this->addusermodel->getAllUsername();
for($i=0;$i<sizeof($list);$i++) {
if($list[$i] == $fixUsername) {
$counter = 0;
}
}
if($counter == 0) {
$data['result'] = "The username has already been taken";
$this->load->view('adduserview',$data);
} else {
$data = array(
'Nama' => $this->input->post('name'),
'Username' => $fixUsername."#praba",
'Password' => md5($this->input->post('password')),
'created' => date("Y-m-d h:i:sa"),
'createdBy' => $createdBy,
'lastModified' => date("Y-m-d h:i:sa"),
'lastModifiedBy' => $lastModifiedBy
);
$this->addusermodel->saveUser($data);
//$data['Username'] = $listName;
$data['message'] = "New user successfully added.";
$data['messageContent'] = "The username: ".$fixUsername."#praba". $counter;
$this->load->view('successpageview',$data);
//redirect('successpageview','refresh');
}
my model (is like usual)
function getAllUsername() {
$this->db->select('Username');
$this->db->from('tbluser');
$query = $this->db->get();
return $query->result_array();
}
I think a better approach would be to create another function in your model, which searches your database by ID, or by email, or by another unique field. If the function returns a row - then the user exists. If it returns nothing - then add a new user.