The aim is to save user entered JSON into a database. Now, before someone jumps at me, I know json, I know mysql and I know all the links inbetween.
The issue is: I need to safely store the ENTIRE JSON feed in a single cell in the table.
The background: this function will be a temp fix for a tool, that is needed asap but will require a lot of time. The temp fix will allow the system to go live with minimal code.
Users will create a GOOGLE maps style here ( http://gmaps-samples-v3.googlecode.com/svn/trunk/styledmaps/wizard/index.html)
and have the JSON made for them
[
{
"stylers": [
{ "visibility": "off" }
]
},{
"featureType": "water",
"stylers": [
{ "visibility": "on" }
]
},{
"featureType": "transit.line",
"elementType": "geometry.fill",
"stylers": [
{ "visibility": "on" },
{ "hue": "#ff3300" },
{ "color": "#ff0000" },
{ "weight": 0.7 }
]
},{
"featureType": "transit.station.rail",
"stylers": [
{ "visibility": "on" },
{ "color": "#0000ff" },
{ "weight": 4.6 }
]
},{
}
]
The site will then just call the JSON and apply it using jQuery later on. What would my ''best practice'' method be at doing this.
To answer your specific question, I would agree with the poster 'tom' that you should use a TEXT column.
However, I think for ease of use, you should also use prepared statements. If you create a prepared insert statement, you can then pass in the JSON directly. This will be the best representation in your database of the exact JSON (no annoying slashes) - AND be the safest. Please don't forget to do this step - its very important!
Since MySQL doesn't have a dedicated JSON column type, I would just store the JSON in an unbounded TEXT column. Just make sure you always check for valid JSON on write.
Related
My API call returns a pretty big JSON result and my initial thought was to parse out the two pieces of data I need for each event and create my own array. Does it make more sense to pass around the returned JSON or clean it up for my use throughout the application.
Which is more efficient?
Below is an example of one "Event" each result may have 20-50 events in the data. All I need is the ['resultsPage']['results']['event']['location']['lng'] and ['resultsPage']['results']['event']['location']['lat']:
{
"resultsPage": {
"results": {
"event": [
{
"id":11129128,
"type":"Concert",
"uri":"http://www.songkick.com/concerts/11129128-wild-flag-at-fillmore?utm_source=PARTNER_ID&utm_medium=partner",
"displayName":"Wild Flag at The Fillmore (April 18, 2012)",
"start": {
"time":"20:00:00",
"date":"2012-04-18",
"datetime":"2012-04-18T20:00:00-0800"
},
"performance": [
{
"artist": {
"id":29835,
"uri":"http://www.songkick.com/artists/29835-wild-flag?utm_source=PARTNER_ID&utm_medium=partner",
"displayName":"Wild Flag",
"identifier": []
},
"id":21579303,
"displayName":"Wild Flag",
"billingIndex":1,
"billing":"headline"
}
],
"location": {
"city":"San Francisco, CA, US",
"lng":-122.4332937,
"lat":37.7842398
},
"venue": {
"id":6239,
"displayName":"The Fillmore",
"uri":"http://www.songkick.com/venues/6239-fillmore?utm_source=PARTNER_ID&utm_medium=partner",
"lng":-122.4332937,
"lat":37.7842398,
"metroArea": {
"id":26330,
"uri":"http://www.songkick.com/metro_areas/26330-us-sf-bay-area?utm_source=PARTNER_ID&utm_medium=partner",
"displayName":"SF Bay Area",
"country": { "displayName":"US" },
"state": { "displayName":"CA" }
}
},
"status":"ok",
"popularity":0.012763
}, ....
]
},
"totalEntries":24,
"perPage":50,
"page":1,
"status":"ok"
}
}
My subjective answer is to just use the entire response in your application, grabbing only what you need when you need it. Taking the time to extract only the data you need might be an unnecessary optimization, and your time could be better spent elsewhere.
Optimize only what you measure. If you can measure your application execution time, perhaps with the help of a profiler, like this one with Xdebug, then you can use data to make an informed decision to optimize in this way. My guess is that your application could use optimizations elsewhere before you make this one, but again, without data, it's just a guess.
I am getting json array after getting applying query logic.
[
{
"id": "3",
"diag_name": "LT Diagnostics",
"test_name": "Alk PO4",
"booking_date": "2018-05-20"
},
{
"id": "3",
"diag_name": "LT Diagnostics",
"test_name": "CRP",
"booking_date": "2018-05-20"
},
{
"id": "4",
"diag_name": "Seepz Diagnostics",
"test_name": "Alk PO4",
"booking_date": "2018-05-21"
}
]
But i want a more justified json array written below.
[
{
"diag_name": "LT Diagnostics",
"test_name": [
{
"id": "3",
"name" : "Alk PO4"
},
{
"id": "3",
"name" : "CRP"
}
],
"booking_date": "2018-05-20"
},
{
"diag_name": "Seepz Diagnostics",
"test_name": [
{
"id": "4",
"name" : "Alk PO4"
}
],
"booking_date": "2018-05-21"
},
]
I am not getting it,How to do in php. I want a more consolidate json format.
Have you tried changing your SQL query to group by diag_name and booking_date? That would be the first step I’d employ to get the outer data.
Formatting the data in the nested manner you’re after could be a function of whatever record serializer you’re using — does it support nested JSON as a return type, or only flat JSON as your example return value shows?
If the record set -> JSON serializer only ever returns flat data, the comments above are correct that you will have to write your own formatter to change the shape of the JSON yourself...
The accepted answer of this other question may be of help:
Create multi-level JSON with PHP and MySQL
I'm not a PHP guy but this is a typical scenario to use functional programming by means of the monad Map.
Looking online I've found this article that could help you.
Changing datasource output is not always (seldom indeed) a viable option.
Enjoy coding
I'm using ElasticSearch's PHP client and I find really difficult to return results with scores whenever I want to search for a word that is "hidden" within a string.
This is an example:
I want to get all the documents where the field "file" has the word "anses" and files are named like this:
axx14anses19122015.zip
What I know about it
I know I should tokenize those words, can't realize how to do it.
Also I've read about aggregations but I'm really new to ES and I have to deliver a working piece ASAP.
What I've tried so far
REGEXP: using regular expressions is very expensive and does not return any scores, which is a must-to-have in order to shrink results and bring the user accurate information.
Wildcards: same thing, slow and no scores
Own script where I have a dictionary and search for critical words using regexp, if match, create a new field within that matched document with the word. The reason is to create a TOKEN so in future searches I can use regular match with scores. Negative side: the dictionary thing was totally denied by my boss so I'm here asking for any ideas.
Thanks in advance.
I suggest in your case nGram tokenizer see the example
I will create a analyzer and a mapping for a doc type
PUT /test_index
{
"settings": {
"number_of_shards": 1,
"analysis": {
"tokenizer": {
"ngram_tokenizer": {
"type": "nGram",
"min_gram": 4,
"max_gram": 4,
"token_chars": [ "letter", "digit" ]
}
},
"analyzer": {
"ngram_tokenizer_analyzer": {
"type": "custom",
"tokenizer": "ngram_tokenizer",
"filter": [
"lowercase"
]
}
}
}
},
"mappings": {
"doc": {
"properties": {
"text_field": {
"type": "string",
"term_vector": "yes",
"analyzer": "ngram_tokenizer_analyzer"
}
}
}
}
}
after that I`ll insert a document using your file name
PUT /test_index/doc/1
{
"text_field": "axx14anses19122015"
}
now I`ll just will use a query match
POST /test_index/_search
{
"query": {
"match": {
"text_field": "anses"
}
}
}
and will receive a reponse like this
{
"took": 8,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 0.10848885,
"hits": [
{
"_index": "test_index",
"_type": "doc",
"_id": "1",
"_score": 0.10848885,
"_source": {
"text_field": "axx14anses19122015"
}
}
]
}
}
What i did?
i just created a nGram tokenizer that will explode our string in 4 characters terms and will index this terms separated and they will be searched when I search a part of the string.
To see more, read this article https://qbox.io/blog/an-introduction-to-ngrams-in-elasticsearch
Hope it help!
Ok after trying -so- many times it worked. I'll share the solution just in case someone else needs it. Thank you so much to Waldemar, it was a really good approach and I still cannot see why it's not working.
curl -XPUT 'http://ipaddresshere/tokentest' -d
'{ "settings":
{ "number_of_shards": 1, "analysis" :
{ "analyzer" : { "myngram" : { "tokenizer" : "mytokenizer" } },
"tokenizer" : { "mytokenizer" : {
"type" : "nGram",
"min_gram" : "3",
"max_gram" : "5",
"token_chars" : [ "letter", "digit" ] } } } },
"mappings":
{ "doc" :
{ "properties" :
{ "field" : {
"type" : "string",
"term_vector" : "yes",
"analyzer" : "myngram" } } } } }'
Sorry for bad indentation, I'm really hurry but want to post the solution.
So, this will take any string from "field" and split it into nGrams with lenght 3 to 5. For example: "abcanses14f.zip" will result in:
abc, abca, abcan, bca, bcan, bcans, etc... until it reaches anses or a similar term which is matcheable and has a score related to it.
I am trying to develop a web application that can fetch data from Asana and generate custom spreadsheet reports. This wrapper class was very helpful in making things simple.
However, I am having a hard time in writing code that gets me the team/s that a particular task belongs to. Even when I export data as JSON through Asana's web application the 'teams' find no mention. From what I understand, Asana itself does not provide an association between teams and tasks. Please correct me if I am wrong.
But if I am right at my conclusion, is there a workaround I could use? Teams are an important part of my data rendering and I need them to be mapped correctly in my reports that I am trying to generate from Asana. The report I want to generate would be hierarchical in nature.
Organisation
Team
Projects
Tasks
Subtask
Can I do something to achieve this hierarchy? The only place I get stuck is getting the projects under a particular team.
Glad to hear that you found that wrapper useful. We will be releasing a PHP Library ourselves soon that you may be interested in. Stay tuned!
Below is some pseudo-code to derive the hierarchy you are looking for, I think. Let me know if it helps.
GET /workspaces
{
"data": [
{
"id": 1234,
"name": "Startup Inc"
}
]
}
GET /workspaces/1234
{
"data": {
"id": 1234,
"name": "Startup Inc",
"is_organization": true,
...
}
}
Because is_corganization is true, we can then continue...
GET /organizations/organization-id/teams
{
"data": [
{
"id": 9876,
"name": "Ninja Team"
}
]
}
GET /teams/9876/projects
{
"data": [
{
"id": 5678,
"name": "Stealth Project"
}
]
}
GET /projects/5678/tasks
{
"data": [
{
"id": 8675309,
"name": "Top secret video"
}
]
}
GET /tasks/8675309
{
"data": {
"id": 8675309,
"created_at": "2015-03-25T17:28:59.255Z",
"modified_at": "2015-05-15T03:13:28.754Z",
"name": "Top secret video",
"notes": "https://www.youtube.com/watch?v=6WTdTwcmxyo",
"completed": false,
... # All the task data
]
}
}
Ok is there a quick way to remove the follwoing though PHP mongodb
here is our mongoDB row
{
"today":""
"session": "6266262626",
"products": [
{
"barcode": "27788822",
"item": "village day ticket",
"price": 1315,
"qty": "3"
},
{
"barcode": "8544122",
"item": "village night ticket",
"price": 1433,
"qty": "1"
}
]
}
I would like to delete the product
{
"barcode": "8544122",
"item": "village night ticket",
"price": 1433,
"qty": "1"
}
I know how to update, and insert but cant figure out how to delete it.
here is a mongo command to delete the item, given its barcode :
db.collection.update({session:'6266262626'},{ $pull: { products: { barcode : '8544122' } }})
If you want to delete multiple items from the products array, given an array of barcodes :
db.collection.update({session:'6266262626'},{ $pull: { products: { barcode : {$in : ['27788822','8544122'] } } }})
I don't know the PHP equivalent of those commands, but here is a related question using $pull and PHP which may help :
MongoDB pull array element from a collection
this question is pretty old, but deleting multiple array items in a single query was giving me trouble today, and in my case the above is working out, so maybe it will help somebody else, thanks.
Reading the docs is your friend. Use the positional operator
http://www.mongodb.org/display/DOCS/Updating#Updating-The%24positionaloperator