Datatable with 50k + lines fails me - php

I am mounting a table and perfect until I want to show more data, I have read that I can use server side but I have not found any example how to consume an web service that returns json
html
<table id="datatable-buscador" class="display nowrap" style="width:100%">
<thead>
<tr>
<th>Nº reco.</th>
<th>Fecha</th>
<th>Id trabajador</th>
<th>Trabajador</th>
<th>Id empresa</th>
<th>Ref. Int.</th>
<th>Nombre empresa</th>
<th>Visado</th>
<th>Prop.</th>
</tr>
</thead>
</table>
javascript
let $tabla = jQuery("#datatable-buscador");
$tabla.dataTable().fnDestroy();
$tabla.dataTable({
"scrollX": true,
"lengthMenu": [[15, 25, 30, -1], [15, 25, 30, "All"]],
"lengthChange": false,
"order": [1, "desc"], //columna 0 y ordenar desc
"language": { "url": "/src_reconocimientos/includes/js/utils/spanish.json" },
"ajax": {
url: "/example/example.php",
type: "POST",
dataType: "JSON",
data: { "tipo": tipo, "dias_reco": diasReco },
timeout: 180000 //3 min
},
"createdRow": function (row, data, dataIndex) {
jQuery(row).attr("id", data.N_RECONOCIMIENTO);
},
"columnDefs": [
{
// targets: "_all",
targets: [0, 1, 2, 4, 5, 7, 8],
className: 'dt-body-center'
},
],
"columns": [
{ "data": "N_RECONOCIMIENTO" },// 0
{
"data": null,
render: function (data, type, full, meta) {
return formatFecha(data.FECHA_RECONOCIMIENTO);
}
}, //1
{ "data": "DNI" }, // 2
{ "data": "NOMBRE_TRABAJADOR" }, // 3
{ "data": "EMPRESA" }, // 4
{ "data": "REF_INTERNA" }, // 5
{ "data": "NOMBRE_CLIENTE" }, // 6
{ "data": "VISADO" }, // 7
{ "data": "PROP" }, // 8
]
});
json:
{
"data": [{
"N_RECONOCIMIENTO": 29457,
"FECHA_RECONOCIMIENTO": "2021-10-11",
"DNI": "28460Q",
"NOMBRE_TRABAJADOR": "CAMPS",
"EMPRESA": 112390,
"REF_INTERNA": "Ref_interna?",
"NOMBRE_CLIENTE": "example",
"VISADO": "N",
"PROP": "Prop?"
}, {
"N_RECONOCIMIENTO": 2907,
"FECHA_RECONOCIMIENTO": "2021-10-14",
"DNI": "414J",
"NOMBRE_TRABAJADOR": "CARLOS",
"EMPRESA": 103482,
"REF_INTERNA": "Ref_interna?",
"NOMBRE_CLIENTE": "(ACUERDO DE COLABORA)",
"VISADO": "N",
"PROP": "Prop?"
}, {
"N_RECONOCIMIENTO": 2619,
"FECHA_RECONOCIMIENTO": "2021-04-16",
"DNI": "72306W",
"NOMBRE_TRABAJADOR": "DIEZ",
"EMPRESA": 29514,
"REF_INTERNA": "Ref_interna?",
"NOMBRE_CLIENTE": "example 365, S.L.",
"VISADO": "S",
"PROP": "Prop?"
}]
}
when pointing to the url of the ajax what it returns me is a json that changes the content according to the data that I pass to it, but when I want to get more 50000 records, the table fails me, so I want to implement serverside, how can I implement it and keep pointing the same url? I don't see anything related in the datatables examples
add info:
It is a very simple company webservice that receives data through my php script with cURL, in this it receives a type of request (it can be 1, 2, 3) and receives parameters (variables), the web services does the queries, simple queries and returns the result as json, it cannot be interacted any more, that's why when I try to process 50000 the datatable gives me an error. my question is: how can I do an intermediate step to process so many lines?

Related

DataTables warning: table - Invalid JSON response

I know this must be one of the most popular questions. I can't see what's wrong with my data though. This is my php:
$sql = "SELECT eventid, event_type, date_display, date_from, date_to, location, done FROM events LIMIT 0, 30";
$get_result = mysql_query($sql);
$arr = Array();
while($res=mysql_fetch_assoc($get_result))
{
$arr['data'][] = $res;
}
echo json_encode($arr);
My JQuery:
function do_search()
{
var search_term=$("#search_term").val();
$.ajax
({
type:'post',
url:'get_results.php',
data:{
search:"search",
search_term:search_term
},
success:function(response)
{
console.log(JSON.parse(response))
$('#events').DataTable({
ajax: JSON.parse(response),
columns: [
{ data: 'eventid' },
{ data: 'event_type' },
{ data: 'date_display' },
{ data: 'date_from' },
{ data: 'date_to' },
{ data: 'location' },
{ data: 'done' },
],
"lengthMenu": [ [-1, 10, 25, 50, 100], ["All", 10, 25, 50, 100] ]
});
}
});
And my HTML:
<table id="events" class="display" style="width:100%">
<thead>
<tr>
<th>ID</th>
<th>Type</th>
<th>Date</th>
<th>From</th>
<th>To</th>
<th>Location</th>
<th>To do</th>
</tr>
</thead>
</table>
This is the data I get in the console:
{
"data": [
{
"eventid": "1",
"event_type": "Senate meeting",
"date_display": "61-60",
"date_from": "61",
"date_to": "60",
"location": "Unknown",
"done": "y"
},
{
"eventid": "2",
"event_type": "Legal hearing",
"date_display": "73-70",
"date_from": "73",
"date_to": "70",
"location": "Unknown",
"done": ""
},
[etc...]
]
}
As far as I can see I'm following the data expected by DataTables. What am I missing?
I am getting seven fields for each record, and the table has seven fields indeed, also mapped in the JQuery code.
In the example at the link you posted here: https://datatables.net/examples/ajax/objects.html, the ajax option in the datatables config is used to provide the URL of the file/script which returns the JSON data, so that datatables can initiate an AJAX request to fetch it.
Whereas what you've done is make your own AJAX request, and then pass the response to datatables. If you're going to do that, you should provide it to datatables via the data option instead. Your scenario, as coded now, is actually closer to this example: https://datatables.net/examples/data_sources/js_array.html
e.g.
$('#events').DataTable({
data: JSON.parse(response),
P.S. For a more complete discussion how to configure DataTables to use an AJAX request as a data source directly, see https://datatables.net/manual/ajax

how to reload or refresh data in treeTable in php

i have issue on use treeTable while reload data and after save and update data form
$(document).ready(function() {
const datajson = <?php echo $data_json ?>;
$('#tablejson').treeTable({
"data": datajson,
"collapsed": true,
"responsive": true,
"lengthChange": true,
"autoWidth": false,
"fnDrawCallback": function() {
$('[data-toggle="tooltip"]').tooltip();
},
"aLengthMenu": [
[10, 50, 100, -1],
[10, 50, 100, "All"]
],
"iDisplayLength": 10,
"columns": [{
"data": "nama_module",
},
{
"data": "controller",
},
{
"data": "function",
},
{
"data": "nm_group",
},
{
"data": "label",
},
{
"data": "btn"
}
],
"order": [],
});
$('#btn-reload').click(function() {
$('#tablejson').dataTable().api().ajax.reload();
});
});
i try to click button for reload tabel use $('#tablejson').dataTable().api().ajax.reload(); but not working, this is happen while i use treeTable, if i use datatable only its work for reload or refresh table.
does anyone have the same case with me ? thanks
finally, i found a solution in my case, i use code like this
$('#btn-reload').click(function() {
$('#tablejson').DataTable()
.order([2, 'desc'])
.draw();
});
thanks....

JSON data displayed but table not drawn (using Datables - server side processing)

I am using Yajra's DataTables server side processing. I can see the JSON data but the table is not being populated.
I managed to get DataTables working with client side processing, but as I will eventually have > 50,000 rows of data, I decided to try and implement server side processing by downloading Yajra's DataTables for Laravel 5.8.
When I call my route, I see the data in a JSON format, but I am not seeing the table at all. It says "draw: 0", so I guess there is an issue with drawing the table?
I have tried various solutions mentioned on stack overflow and the official DataTables website, however none seem to work for me. E.g.
- DataTables json is not processed (in Laravel)
- jQuery Datatables - Table not populated for Ajax response
- https://datatables.net/forums/discussion/45444/populate-table-body-with-ajax
The JSON data that I see when I call my route is as follows:
{
"draw": 0,
"recordsTotal": 3,
"recordsFiltered": 3,
"data": [
{
"id": "1",
"customerNr": "98764",
"orderNr": "23478",
"pallet_id": "66788",
"status_id": "2",
"created_by": "Sara",
"created_at": "04 Jul 2019",
"updated_at": "2019-07-09 07:23:20"
},
{
"id": "2",
"customerNr": "99999",
"orderNr": "22222",
"pallet_id": "22335",
"status_id": "1",
"created_by": "Sophie",
"created_at": "04 Jul 2019",
"updated_at": "2019-07-04 08:26:28"
},
{
"id": "3",
"customerNr": "54657",
"orderNr": "89856",
"pallet_id": "11228",
"status_id": "1",
"created_by": "Markus",
"created_at": "08 Jul 2019",
"updated_at": "2019-07-08 06:59:42"
},
],
"input": []
}
Here are the relevant files from my Laravel project:
web.php:
Route::get('returned-shipment', ['uses'=>'ReturnedShipmentController#index', 'as'=>'returned-shipment.index']);
ReturnedShipmentController:
public function index(
{
return DataTables::of(ReturnedShipment::all())->make();
}
index.blade.php:
<div class="row">
<div id="tbl" class="col-sm-12">
<table id="overview" class="cell-border display">
<thead class="tbl-top">
<tr>
<th>Retourennummer</th>
<th>Auftragsnummer</th>
<th>Datum</th>
<th>Zustand</th>
</tr>
</thead>
<tfoot class="tbl-bottom">
<tr>
<th>Retourennummer</th>
<th>Auftragsnummer</th>
<th>Datum</th>
<th>Zustand</th>
</tr>
</tfoot>
</table>
</div>
</div>
<script>
$(document).ready(function () {
var startingStatus = 'angelegt';
var table = $('#overview').DataTable({
"processing": true,
"serverSide": true,
"ajax": "{{ route('returned-shipment.index') }}",
"columns": [
{data: 'id'},
{data: 'orderNr'},
{data: 'created_at'},
{data: 'status_id'}
],
"search": {
"search": "angelegt"
},
"dom": "<'row'<'col-sm-4'l><'col-sm-8'f>>" +
"<'row'<'col-sm-12'tr>>" +
"<'row'<'col-sm-6'i><'col-sm-6'p>>",
"paging": true,
"info": true,
"searching": true,
"autoWidth": true,
"language": {
"paginate": {
"previous": "Vorherige Seite",
"next": "Nächste Seite"
},
"search": "Suche:",
"info": "Zeige _START_ - _END_ von insgesamt _TOTAL_ Einträgen",
"lengthMenu": 'Einträge pro Seite' + '<br>' +
'<select class="custom-select mr-sm-2" id="inlineFormCustomSelect">' +
'<option selected value="10">10</option>' +
'<option value="20">20</option>' +
'<option value="30">30</option>' +
'<option value="40">40</option>' +
'<option value="50">50</option>' +
'<option value="-1">Alle</option>' +
'</select>'
},
initComplete: function () {
/**
* Drop-down filter is created for the 4th column "status" in the header and populates it with
* the different status values
*/
this.api().columns([3]).every(function () {
var column = this;
var select = $('<select><option value="">alle</option></select>')
.appendTo($(column.header()))
.on('change', function () {
var val = $.fn.dataTable.util.escapeRegex(
$(this).val()
);
column
.search(val ? '^' + val + '$' : '', true, false)
.draw();
});
/**
* When clicking on drop-down next to status, the sorting function is not activated
*/
$(select).click(function (e) {
e.stopPropagation();
});
/**
* Once an option in the drop-down next to status has been selected, you can read the text in
* the drop-down
*/
column.data().unique().sort().each(function (d, j) {
if (startingStatus === d) {
select.append('<option SELECTED value="' + d + '">' + d + '</option>')
} else {
select.append('<option value="' + d + '">' + d + '</option>')
}
});
/**
* When drop-down is clicked on, search field is cleared. Otherwise search field must be
* manually cleared before using the drop-down.
*/
$(select).on('click', function () {
table.search(" ").draw();
});
});
}
});
});
</script>
I am expecting to see the table being populated with the data.
If I need to provide any more code or explain something further, please don't hesitate to ask. I am quite new to Laravel and DataTables, so I would greatly appreciate your help.
Thanks in advance! :)
I have checked your current code and checked your JSON. Your JSON file seems to be invalid.
{
"draw": 0,
"recordsTotal": 3,
"recordsFiltered": 3,
"data": [
{
"id": "1",
"customerNr": "98764",
"orderNr": "23478",
"pallet_id": "66788",
"status_id": "2",
"created_by": "Sara",
"created_at": "04 Jul 2019",
"updated_at": "2019-07-09 07:23:20"
},
{
"id": "2",
"customerNr": "99999",
"orderNr": "22222",
"pallet_id": "22335",
"status_id": "1",
"created_by": "Sophie",
"created_at": "04 Jul 2019",
"updated_at": "2019-07-04 08:26:28"
},
{
"id": "3",
"customerNr": "54657",
"orderNr": "89856",
"pallet_id": "11228",
"status_id": "1",
"created_by": "Markus",
"created_at": "08 Jul 2019",
"updated_at": "2019-07-08 06:59:42"
}
],
"input": []
}
This is the correct format. The comma after the last object in your data array needs to be removed. Can you confirm this solves your issue? I got datatables working without this comma.

Data Table request unknown parameter '0' from the data source for row 0

This is sample of my json response code :
[{"data":"{\"time\": \"2017-12-12 11:17:48\", \"user\": \"firman\", \"owner\": \"admin\", \"host_ip\": \"127.0.0.1\", \"user_ip\": \"192.168.3.235\", \"nas_port\": \"2162163717\", \"log_status\": \"accounting-success\", \"nas_port_id\": \"ether3-Hotspot\", \"nas_port_type\": \"wireless-802.11\", \"calling_station_id\": \"58:44:98:A0:A4:88\"}"},{"data":"{\"time\": \"2017-12-12 11:57:07\", \"user\": \"lenovo\", \"owner\": \"admin\", \"host_ip\": \"127.0.0.1\", \"user_ip\": \"192.168.3.236\", \"nas_port\": \"2162163713\", \"log_status\": \"accounting-success\", \"nas_port_id\": \"ether3-Hotspot\", \"nas_port_type\": \"wireless-802.11\", \"calling_station_id\": \"64:DB:43:BB:52:CF\"}"},{"data":"{\"time\": \"2017-12-12 04:59:19\", \"user\": \"endigo\", \"owner\": \"admin\", \"host_ip\": \"127.0.0.1\", \"user_ip\": \"192.168.3.240\", \"nas_port\": \"2160066562\", \"log_status\": \"accounting-success\", \"nas_port_id\": \"ether3-Hotspot\", \"nas_port_type\": \"wireless-802.11\", \"calling_station_id\": \"02:40:20:8F:F7:C0\"}"}]
When i load page , request unknown parameter '0' from the data source for row 0 show ,
And This is my datatable code :
<script type="text/javascript">
$("#radius_log").DataTable({
fixedColumns: true,
fixedHeader: true,
"pageLength": 10,
"paging": true,
"ajax": {
url: "./all_log",
type: "GET",
dataSrc: ""
},
"scrollX": true,
"columns": [
{"data": "owner", "title": "Owner"},
{"data": "user", "title": "User"},
{"data": "user_ip", "title": "IP Address"},
{"data": "host_ip", "title": "Radius Server"},
{"data": "nas_port_id", "title": "NAS Port"},
{"data": "nas_port_type", "title": "NAS Port Type"},
{"data": "time", "title": "Time"}
]
});
</script>
What your JSON is returning is:
What you want is:
Having had a fiddle would have made it easier, so here is one: https://jsfiddle.net/g5fa1mo0/
I actually got 2 error-messages, the second one was very clear and pointed out that ownerwas missing from the first record!

Free jqGrid - JSON encode example using PHP

Free jqGrid uses data in the following JSON name:value pairs format:
var data = {
"page": "1",
"records": "3",
"rows": [
{ "DataID": "1", "DataDesc": "Test 1", "DataTitle": "Test 1" }
]
};
I have the following in the PHP script:
$i=0;
while ($row = mysql_fetch_assoc($result)) {
$data->rows[$i]['cell']=array($row);
$i++;
}
print json_encode($data);
Which returns:
{"rows":[{"cell":[{"user_id":"00082563","first_name":"Peter","case_title":"Male with STI (urethritis)","case_started":"2017-06-02 10:52:10"}]}]}
Which looks to be OK. However, with the JSON parts of the code below, the grid doesn't display at all.
function loadFirstGrid() {
$("#FirstGrid").jqGrid({
url: "scripts/json_test.php?user=" + user,
dataType: "json",
mtype: "GET",
postData: {
json: JSON.stringify(data)
},
colModel: [{
name: "user_id",
label: "User ID",
width: 120
},
{
name: "first_name",
label: "Name",
width: 400
},
{
name: "case_title",
label: "Case Title",
width: 500
},
{
name: "case_started",
label: "Case Started",
width: 200
},
],
emtyrecords: "Nothing to display",
viewrecords: true,
sortable: true,
shrinkToFit: false,
autowidth: true,
caption: 'First Grid'
});
}
But if I remove the postData part to have the following, the grid displays, but of course no data.
function loadFirstGrid() {
$("#FirstGrid").jqGrid({
url: "scripts/json_test.php?user=" + user,
dataType: "json",
mtype: "GET",
colModel: [{...
Any ideas?
OK, finally worked through this and got it working with this
function loadFirstGrid() {
$("#FirstGrid").jqGrid({
url: "scripts/json_test.php?user=" + user,
dataType: "json",
mtype: "GET",
colModel: [
{name: "user_id", label:"User ID", width: 120},
{name: "first_name", label:"Name", width: 400},
{name: "case_title", label:"Case Title", width: 500},
{name: "case_started", label:"Case Started", width: 200},
],
emtyrecords: "Nothing to display",
viewrecords: true,
sortable: true,
shrinkToFit: false,
autowidth: true,
caption: 'First Grid',
});
}
To get the correct format JSON for jqGrid:
{"page":"1","total":"1","records":"1","rows":[{"user_id":"00082563","first_name":"Peter","case_title":"Male with STI (urethritis)","case_started":"2017-06-02 10:52:10"}]}
I used the following PHP script:
$page = '1';
$total_pages = '1';
$count = '1';
$data = (object) array('page' => $page, 'total' => $total_pages, 'records' =>$count, 'rows' => "");
$data->page = $page;
$data->total = $total_pages;
$data->records = $count;
$i=0;
while ($row = mysql_fetch_assoc($result)) {
$data->rows=array($row);
$i++;
}
print json_encode($data);
?>
(NOTE: I don't care about the number of pages, total_pages and count as my grids will only ever have one primary record and multiple subgrids with only one record). So hope this helps someone; there is not much in the documentation or examples that describes how to do this with Free jqGrid ;-(
First of all JavaScript is case sensitive language and jqGrid will ignore parameter dataType: "json". You should fix it to datatype: "json".
Seconds, you use exotic format of the JSON data:
{
"rows": [{
"cell": {
"user_id": "00082563",
"first_name": "Peter",
"case_title": "Male with STI (urethritis)",
"case_started": "2017-06-02 10:52:10"
}
}]
}
instead of
{
"rows": [ {
"user_id": "00082563",
"first_name": "Peter",
"case_title": "Male with STI (urethritis)",
"case_started": "2017-06-02 10:52:10"
}]
}
or
[ {
"user_id": "00082563",
"first_name": "Peter",
"case_title": "Male with STI (urethritis)",
"case_started": "2017-06-02 10:52:10"
}]
You don't use loadonce: true and it's unclear, whether you plan to implement server side paging, sorting and filtering of data or you want to return all the data at once and jqGrid should use client side paging, sorting and filtering.
Finally, you should use name properties of colModel corresponds to the properties of user_id. It's very important to understand, that jqGrid have to assign unique id to every row of the grid (see here). Thus you have to inform jqGrid, which property contains rowid. You can use either jsonReader: { id: "user_id" } or to include property key: true in the column user_id.
The demo https://jsfiddle.net/OlegKi/qgrwymuu/1/ contains an example of described above modifications. It uses Echo service of JSFiddle to simulate server, which responses with some JSON data.

Categories