I have a query that is very fast in finding the records it wants, but it takes too long to "extract" the data from the JSON field if I have a few hundred results. Is there a way to speed up the extraction process with this query and data structure? At about 500 results, the query takes around 3 seconds. If I remove the extract, it's basically instant.
The query returns exactly the data that I want, it's just too slow:
SELECT
unit_uid,
JSON_EXTRACT(rateplans, '$."2023-03-14"','$."2023-03-15"') AS rateplans
FROM
property_listings_rateplans
WHERE
unit_uid IN (1527,1639,5,9,17,20,24,27,1873,4279)
The structure of the data is like this but with a lot more values and more dates in the rateplans data:
unit_uid (int)
rateplans (JSON)
1527
{ "2023-03-14": { "date": "2022-12-02", "price": 17500, "online": 1 }, "2023-03-15": { "date": "2022-12-03", "price": 17500, "online": 1 }, }
1639
{ "2023-03-14": { "date": "2022-12-02", "price": 17500, "online": 1, }, "2023-03-15": { "date": "2022-12-03", "price": 17500, "online": 1, },}
Any ideas on how I should tackle this?
Edit:
I should also add that there will eventually be millions of unit_uids and approximately 2 years worth of dates in each JSON field as well as more key:value pairs within each date. This is just a sample of the live data.
Related
So, I have table ($data) with JSON records getting by select query.
for example:
"data": {
"id": "1",
"technology_name": "First",
"technology_info": "Something about first rec",
"created_at": null,
"updated_at": null
},
{
"id": "2",
"technology_name": "Second",
"technology_info": "Something about second rec",
"created_at": null,
"updated_at": null
}
}
...
Now I need to create filter by column (technology_name) by user request with 'like', something like
select * from "data" where technology_name = '%css%';
It is possible to do this with CodeIgniter/PHP code and JSON file, or is better way to do that? Can you give me some advice how to figure it out?
I think that I misspelled my problem.
I was able to find the correct solution.
So, I modified my script that return response as JSON representation with some instructions that SELECT data with WHERE by getting user request params.
I have this json value that I want to be sorty but for some reason it's not working.
[
{
"id": 15028,
"order_id": 342,
"user_id": 3,
"status": "1",
"priority": "1",
"donedate": null,
"user": {
"id": 3,
"name": "Max"
}
},
{
"id": 15030,
"order_id": 341,
"user_id": 4,
"status": "2",
"priority": "1",
"donedate": null,
"user": {
"id": 4,
"name": "Jon"
}
}
]
This jSon structure is the result of Laravel eloquent object conversion using $object->toJson();
Now I keep this output in my Redis cache. What I want is to when the status and or priority of any order gets changed then I want to sort this jSon and store it back in Redis.
$order_list = collect($json_decoded_with_updated_values);
$order_list = $order_list->sortBy('status')->sortBy('priority');
Redis::set(\GuzzleHttp\json_encode($stich_list_in_collection));
Redis::set("orders_list", $orders_list, 302400);
However, I don't get a sort list. What I want to achieve is that, just like I would run two to three orderBy on an eloquent model like orderBy('status')->orderBy('priority')->get() .. I want to run the same two sortings on this json list.
Thanks in advance.
I figured it out. Actually we don't need to have a call-back as suggested by #brokedid. We can do it like following.
$order_list->sortBy('status')->sortBy('priority')->values()->all();
So I was missing the "->values()->all()" part. I hope if any one runs into the same problem in future, they can get a hint from this.
If you want to sort by multiple Fields, then you could try to sort with a callback-method:
$orderedList = $unorderedList->sortBy(function($item) {
return $item->priority.'-'.$item->status;
});
I wonder what's the result when you choose a different sort direction.
$order_list = $order_list->sortByDesc('status');
I'm trying to get all the services with a code different than 0, I don't find any way to do it without doing multiple queries.
Here's the data i'm trying to query, thanks in advance for the help
"hits": [
{
"_index": "database_index",
"_type": "system",
"_id": "AV03gG7B3dd3_xKqi09H",
"_score": 1,
"_source": {
"name_id": "BOX",
"room_id": "63190",
"event": "SysMonitor",
"data": {
"ip": "192.168.60.25 \n",
"ns": "192.168.0.1\n",
"uptime": "3:12"
},
"services": {
"apache2": 1,
"gw-flash": 0,
"appmgr": 0,
"watchdog": 2,
"gui": 0,
"usb": 0,
"internet": 0
},
This is a classic fulltext search example, where one could write a very complicated query to cover this or you could optimize this on index time and thus have very cheap queries. If you change your data model to have a field that contains all the services that have a value of 0 or 1, you could easily search for that, i.e.
services_zero: [gwflash, appmgr]
services_one: [ apache2 ]
Suddenly your query can easily search or aggregate against those fields.
Not saying that this will perfectly fit your use-case, but I think it is worth to rethink the data modelling strategy here.
im trying to figure out what the easiest way would be. i have a to do an api call for data i get a json file in return but im trying to parse the data to php so that my website would just pull from the database. do i have to create all the same fields that the api call is using for instance name gender age height. and then do i name it something else so i can call from my webpage. because in order to get a nfl player it gives me a 16 digit code and i want it just to be say tom brady
i have used javascript to pull the data but dont know what to do from there
"players": [{
"name": "Kyle Rudolph",
"jersey": "82",
"last_name": "Rudolph",
"first_name": "Kyle",
"abbr_name": "K.Rudolph",
"preferred_name": "Kyle",
"birth_date": "1989-11-09",
"weight": 265.0,
"height": 78,
"status": "A01",
"id": "1059e9dc-97df-4643-9116-883a0573d8b1",
"position": "TE",
"birth_place": "Cincinnati, OH, USA",
"high_school": "Elder (OH)",
"college": "Notre Dame",
"college_conf": "Independent",
"rookie_year": 2011,
"draft": {
"year": 2011,
"round": 2,
"number": 43,
"team": {
"name": "Vikings",
"market": "Minnesota",
"alias": "MIN",
"id": "33405046-04ee-4058-a950-d606f8c30852"
}
},
John, you don't have to use the same names. Once your PHP receives data from the API, parse it into whatever names you want, ignore the values you don't need etc... For example, suppose the API gives you a first and last name, but in your database you only care about full name. As another example, if the API gives you player weight in pounds but you need it in kilograms:
$api_result = file_get_contents($url);
$api_data = json_decode($api_result);
$name = "$api_data->last, $api_data->first"
$weight = $api_data->weight * 0.454; //convert pounds to kg for storage
Now you can store $name and $weight in your DB as you like. When your website pulls data from your backend, the shape of the data produced by the API doesn't matter because you stored it in the shape that is most helpful to your application
Ok is there a quick way to remove the follwoing though PHP mongodb
here is our mongoDB row
{
"today":""
"session": "6266262626",
"products": [
{
"barcode": "27788822",
"item": "village day ticket",
"price": 1315,
"qty": "3"
},
{
"barcode": "8544122",
"item": "village night ticket",
"price": 1433,
"qty": "1"
}
]
}
I would like to delete the product
{
"barcode": "8544122",
"item": "village night ticket",
"price": 1433,
"qty": "1"
}
I know how to update, and insert but cant figure out how to delete it.
here is a mongo command to delete the item, given its barcode :
db.collection.update({session:'6266262626'},{ $pull: { products: { barcode : '8544122' } }})
If you want to delete multiple items from the products array, given an array of barcodes :
db.collection.update({session:'6266262626'},{ $pull: { products: { barcode : {$in : ['27788822','8544122'] } } }})
I don't know the PHP equivalent of those commands, but here is a related question using $pull and PHP which may help :
MongoDB pull array element from a collection
this question is pretty old, but deleting multiple array items in a single query was giving me trouble today, and in my case the above is working out, so maybe it will help somebody else, thanks.
Reading the docs is your friend. Use the positional operator
http://www.mongodb.org/display/DOCS/Updating#Updating-The%24positionaloperator