There are tons of articles, blogs and API docs about REST API resource field expansion but none about how to implement an expansion in aspect of technique and data query in right way.
Simple example for a flat resource response:
GET /api/v1/customers
{
data: [
{
"id": "209e5d80-c459-4d08-b53d-71d11e216a5d",
"contracts": null
},
{
"id": "c641cb83-af29-485d-9be2-97925057e3b2",
"contracts": null
}
],
expandable: ["contract"]
}
Simple example for expanded resource:
GET /api/v1/customers?expand=contract
{
data: [
{
"id": "209e5d80-c459-4d08-b53d-71d11e216a5d",
"contracts": [
{......},
{......},
{......},
]
},
{
"id": "c641cb83-af29-485d-9be2-97925057e3b2",
"contracts": [
{......},
{......},
{......},
]
}
],
expandable: ["contract"]
}
Lets assume we use a api rest controller class which handles the enpoints and a read service (maybe cqs/cqrs based) which uses plain sql for read performance. At which point does the expansion logic start and what is the right way of handling queries without an exponential increase in queries?
Afaik, this is not possible in one or few SQL queries (except the dirty way of GROUP_CONCAT and separation of all data into one field). Should I query all customers and then iterate over all customers and query expanded data for each customer? This would cause an exponential increase of queries.
I was looking for the same thing and I would say that the correct answer is it depends on the infrastructure used.
In a general manner, the implementation should receive the expand or expandable' object in your endpoint and respond acordingly. So, if we have something like your example, when requesting /api/v1/customersviaGETpassing theexpandableobject, you should run another query to the database to get, as in the example, theuser contracts`.
There is not a unique, one size fits all answer for this question, specially if we are not talking about a specific language + framework.
Related
Assume you have a GraphQL type and it includes many fields.
How to query all the fields without writing down a long query that includes the names of all the fields?
For example, If I have these fields :
public function fields()
{
return [
'id' => [
'type' => Type::nonNull(Type::string()),
'description' => 'The id of the user'
],
'username' => [
'type' => Type::string(),
'description' => 'The email of user'
],
'count' => [
'type' => Type::int(),
'description' => 'login count for the user'
]
];
}
To query all the fields usually the query is something like this:
FetchUsers{users(id:"2"){id,username,count}}
But I want a way to have the same results without writing all the fields, something like this:
FetchUsers{users(id:"2"){*}}
//or
FetchUsers{users(id:"2")}
Is there a way to do this in GraphQL ??
I'm using Folkloreatelier/laravel-graphql library.
Unfortunately what you'd like to do is not possible. GraphQL requires you to be explicit about specifying which fields you would like returned from your query.
Yes, you can do this using introspection. Make a GraphQL query like (for type UserType)
{
__type(name:"UserType") {
fields {
name
description
}
}
}
and you'll get a response like (actual field names will depend on your actual schema/type definition)
{
"data": {
"__type": {
"fields": [
{
"name": "id",
"description": ""
},
{
"name": "username",
"description": "Required. 150 characters or fewer. Letters, digits, and #/./+/-/_ only."
},
{
"name": "firstName",
"description": ""
},
{
"name": "lastName",
"description": ""
},
{
"name": "email",
"description": ""
},
( etc. etc. ...)
]
}
}
}
You can then read this list of fields in your client and dynamically build a second GraphQL query to get the values of these fields.
This relies on you knowing the name of the type that you want to get the fields for -- if you don't know the type, you could get all the types and fields together using introspection like
{
__schema {
types {
name
fields {
name
description
}
}
}
}
NOTE: This is the over-the-wire GraphQL data -- you're on your own to figure out how to read and write with your actual client. Your GraphQL javascript library may already employ introspection in some capacity. For example, the apollo codegen command uses introspection to generate types.
2022 Update
Since this answer was originally written, it is now a recommended security practice to TURN OFF introspection in production. Reference: Why you should disable GraphQL introspection in production.
For an environment where introspection is off in production, you could use it in development as a way to assist in creating a static query that was used in production; you wouldn't actually be able to create a query dynamically in production.
I guess the only way to do this is by utilizing reusable fragments:
fragment UserFragment on Users {
id
username
count
}
FetchUsers {
users(id: "2") {
...UserFragment
}
}
I faced this same issue when I needed to load location data that I had serialized into the database from the google places API. Generally I would want the whole thing so it works with maps but I didn't want to have to specify all of the fields every time.
I was working in Ruby so I can't give you the PHP implementation but the principle should be the same.
I defined a custom scalar type called JSON which just returns a literal JSON object.
The ruby implementation was like so (using graphql-ruby)
module Graph
module Types
JsonType = GraphQL::ScalarType.define do
name "JSON"
coerce_input -> (x) { x }
coerce_result -> (x) { x }
end
end
end
Then I used it for our objects like so
field :location, Types::JsonType
I would use this very sparingly though, using it only where you know you always need the whole JSON object (as I did in my case). Otherwise it is defeating the object of GraphQL more generally speaking.
GraphQL query format was designed in order to allow:
Both query and result shape be exactly the same.
The server knows exactly the requested fields, thus the client downloads only essential data.
However, according to GraphQL documentation, you may create fragments in order to make selection sets more reusable:
# Only most used selection properties
fragment UserDetails on User {
id,
username
}
Then you could query all user details by:
FetchUsers {
users() {
...UserDetails
}
}
You can also add additional fields alongside your fragment:
FetchUserById($id: ID!) {
users(id: $id) {
...UserDetails
count
}
}
Package graphql-type-json supports custom-scalars type JSON.
Use it can show all the field of your json objects.
Here is the link of the example in ApolloGraphql Server.
https://www.apollographql.com/docs/apollo-server/schema/scalars-enums/#custom-scalars
A common way of passing parameters to a RESTful web service is in the URL:
website.com/action.php?table=myTable&key=myKey&values=myValues
Another way would be with JSON:
{
"data":
[
{
"parameters": {"table":"myTable", "key":"myKey", "values":"myValues"}
},
{
"content": {"data1":"dataVal1","data2":"dataVal2"}
}
]
}
What would be the pros and cons of these two methods:
When would I use one over the other
Benefits of each one
Weaknesses
Performance differences
To retrieve values using REST, you must use a GET request. There are no request bodies with GET requests, so your only option is the url.
When changing values in REST service, you typically use the PUT request. This PUT request should contain the new resource state in it's body.
So it's not an either/or matter. Where you place certain request parameters depends on what kind of operation you are doing, and what the meaning of the parameter is.
I've been looking into GraphQL as a replacement for some REST APIs of mine, and while I think I've wrapped my head around the basics and like most of what I see so far, there's one important feature that seems to be missing.
Let's say I've got a collection of items like this:
{
"id": "aaa",
"name": "Item 1",
...
}
An application needs a map of all those objects, indexed by ID as such:
{
"allItems": {
"aaa": {
"name": "Item 1",
...
},
"aab": {
"name": "Item 2",
...
}
}
}
Every API I've ever written has been able to give results back in a format like this, but I'm struggling to find a way to do it with GraphQL. I keep running across issue 101, but that deals more with unknown schemas. In my case, I know exactly what all the fields are; this is purely about output format. I know I could simply return all the items in an array and reformat it client-side, but that seems like overkill given that it's never been needed in the past, and would make GraphQL feel like a step backwards. I'm not sure if what I'm trying to do is impossible, or I'm just using all the wrong terminology. Should I keep digging, or is GraphQL just not suited to my needs? If this is possible, what might a query look like to retrieve data like this?
I'm currently working with graphql-php on the server, but I'm open to higher-level conceptual responses.
Unfortunately returning objects with arbitrary and dynamic keys like this is not really a first-class citizen in GraphQL. That is not to say you can't achieve the same thing, but in doing so you will lose many of the benefits of GraphQL.
If you are set on returning an object with id keys instead of returning a collection/list of objects containing the ids and then doing the transformation on the client then you can create a special GraphQLScalarType.
const GraphQLAnyObject = new GraphQLScalarType({
name: 'AnyObject',
description: 'Any JSON object. This type bypasses type checking.',
serialize: value => {
return value;
},
parseValue: value => {
return value;
},
parseLiteral: ast => {
if (ast.kind !== Kind.OBJECT) {
throw new GraphQLError("Query error: Can only parse object but got a: " + ast.kind, [ast]);
}
return ast.value;
}
});
The problem with this approach is that since it is a scalar type you cannot supply a selection set to query it. E.G. if you had a type
type MyType implements Node {
id: ID!
myKeyedCollection: AnyObject
}
Then you would only be able to query it like so
query {
getMyType(id: abc) {
myKeyedCollection # note there is no { ... }
}
}
As others have said, I wouldn't recommend this because you are losing a lot of the benefits of GraphQL but it goes to show that GraphQL can still do pretty much anything REST can.
Hope this helps!
Assume you have a GraphQL type and it includes many fields.
How to query all the fields without writing down a long query that includes the names of all the fields?
For example, If I have these fields :
public function fields()
{
return [
'id' => [
'type' => Type::nonNull(Type::string()),
'description' => 'The id of the user'
],
'username' => [
'type' => Type::string(),
'description' => 'The email of user'
],
'count' => [
'type' => Type::int(),
'description' => 'login count for the user'
]
];
}
To query all the fields usually the query is something like this:
FetchUsers{users(id:"2"){id,username,count}}
But I want a way to have the same results without writing all the fields, something like this:
FetchUsers{users(id:"2"){*}}
//or
FetchUsers{users(id:"2")}
Is there a way to do this in GraphQL ??
I'm using Folkloreatelier/laravel-graphql library.
Unfortunately what you'd like to do is not possible. GraphQL requires you to be explicit about specifying which fields you would like returned from your query.
Yes, you can do this using introspection. Make a GraphQL query like (for type UserType)
{
__type(name:"UserType") {
fields {
name
description
}
}
}
and you'll get a response like (actual field names will depend on your actual schema/type definition)
{
"data": {
"__type": {
"fields": [
{
"name": "id",
"description": ""
},
{
"name": "username",
"description": "Required. 150 characters or fewer. Letters, digits, and #/./+/-/_ only."
},
{
"name": "firstName",
"description": ""
},
{
"name": "lastName",
"description": ""
},
{
"name": "email",
"description": ""
},
( etc. etc. ...)
]
}
}
}
You can then read this list of fields in your client and dynamically build a second GraphQL query to get the values of these fields.
This relies on you knowing the name of the type that you want to get the fields for -- if you don't know the type, you could get all the types and fields together using introspection like
{
__schema {
types {
name
fields {
name
description
}
}
}
}
NOTE: This is the over-the-wire GraphQL data -- you're on your own to figure out how to read and write with your actual client. Your GraphQL javascript library may already employ introspection in some capacity. For example, the apollo codegen command uses introspection to generate types.
2022 Update
Since this answer was originally written, it is now a recommended security practice to TURN OFF introspection in production. Reference: Why you should disable GraphQL introspection in production.
For an environment where introspection is off in production, you could use it in development as a way to assist in creating a static query that was used in production; you wouldn't actually be able to create a query dynamically in production.
I guess the only way to do this is by utilizing reusable fragments:
fragment UserFragment on Users {
id
username
count
}
FetchUsers {
users(id: "2") {
...UserFragment
}
}
I faced this same issue when I needed to load location data that I had serialized into the database from the google places API. Generally I would want the whole thing so it works with maps but I didn't want to have to specify all of the fields every time.
I was working in Ruby so I can't give you the PHP implementation but the principle should be the same.
I defined a custom scalar type called JSON which just returns a literal JSON object.
The ruby implementation was like so (using graphql-ruby)
module Graph
module Types
JsonType = GraphQL::ScalarType.define do
name "JSON"
coerce_input -> (x) { x }
coerce_result -> (x) { x }
end
end
end
Then I used it for our objects like so
field :location, Types::JsonType
I would use this very sparingly though, using it only where you know you always need the whole JSON object (as I did in my case). Otherwise it is defeating the object of GraphQL more generally speaking.
GraphQL query format was designed in order to allow:
Both query and result shape be exactly the same.
The server knows exactly the requested fields, thus the client downloads only essential data.
However, according to GraphQL documentation, you may create fragments in order to make selection sets more reusable:
# Only most used selection properties
fragment UserDetails on User {
id,
username
}
Then you could query all user details by:
FetchUsers {
users() {
...UserDetails
}
}
You can also add additional fields alongside your fragment:
FetchUserById($id: ID!) {
users(id: $id) {
...UserDetails
count
}
}
Package graphql-type-json supports custom-scalars type JSON.
Use it can show all the field of your json objects.
Here is the link of the example in ApolloGraphql Server.
https://www.apollographql.com/docs/apollo-server/schema/scalars-enums/#custom-scalars
My CouchDB database is structured like this:
"custom_details": {
"user_education": [
{
"device_id": "358328030246627",
"college_name": "College",
"college_year": "2014"
},
]
}
"custom_details_1": {
"user_education": [
{
"device_id": "358328030246627",
"college_name": "College",
"college_year": "2014"
},
]
}
I have a lot of arrays within arrays. What I'm trying to do use Elasticsearch to search and find terms, regardless of where it's sitting in an array. Is that possible?
I've been going through the examples on here and haven't quite found what I'm looking for. I've tried using Elastica, the PHP Wrapper, but without fully understanding how to do this with REST, I'm lost. Is it even possible to search for data without knowing the field?
In Lucene, you could create a document instance for each device id:
public void indexRecord(CouchDBRecord rec) {
Document doc = new Document();
doc.add(new Field("device_id", rec.device_id, Store.YES, Index.NOT_ANALYZED));
doc.add(new Field("college_name", rec.college_name, Store.YES, Index.ANALYZED));
doc.add(new Field("college_year", rec.college_year.toString(), Store.YES, Index.NOT_ANALYZED));
this.writer.addDocument(doc);
}
This will allow you to search by keywords in the college name, or by exact device id or year, or some combination thereof.
If you are using Elastica, the whole REST thing is already done for you. If you want to search in all fields, you can define just the search term and it will search in all fields.
If you are having some troubles with Elastica or if some features are missing you need, let me know, as I'm the developer of Elastica.