☑ External Public API > Find outlines
This document describes the find API, intended for developers outside the company.
The find API is used to search for entities in the database. It allows nested filters on any entity served through the BasicLogicLayer.
The API should allow to filter on every visible field of the DTO api you want to search for.
To use the basic logic layer, you need to create a class that extends the BasicLogicLayer, every entity which needs the basic logic layer must implement a convertor and a secure db layer for entity segregation by owner. The dto should be marked with YubaLogicAnnotationProtected decorator with the option autoManageFindAnnotations set to true.
Every case presented in this document is tested in the e2e tests unless stated otherwise.
The format of low order queries are a target field, a hard value, and an operator type, it can be represented as follows:
{field key} {operator} {hard value}
as such it is not possible to compare 2 fields without a specific operator (ie something like field1 eq field2 is not supported, and should be implemented via a specific operator like 'equals another field' if needed)
Limitations
Search criteria values
The default maximum values usable to filter items based on identifiers and option lists is 150. It can be configured using the feature flag max_item_query_params
DB support
This service is only tested on MariaDb hosted entities, but some feature might work on other db, use at your own risk.
Foreign key support
Nested objects are not supported (ie objects related via foreign key, maria json fields are talked about in another section).
This is because this would need to implement a recursive query builder and decorator process, which is not trivial. It is however possible to join another entity by 'flattening' its field in the dto. To do so add a base query in the service layer, and add the field db name via the @YubaSearchable decorator.
Consider the following example with 2 entities an User and a UserSettings, where the UserSettings is related to the User
class UserSettings {
@YubaSearchable({ type: SearchableFieldType.number })
id: number;
@YubaSearchable({ type: SearchableFieldType.string })
name: string;
@YubaLogger({ type: LoggerFieldType.string })
value: string;
}
class User {
@YubaSearchable()
setting: UserSettings; // not supported
}2
3
4
5
6
7
8
9
10
11
12
13
class UserSettings {
name: string;
value: string;
}
// supported
class User {
@YubaSearchable({ type: SearchableFieldType.number, aka: ['userSetting.id'], dbName: 'userSetting.id' })
userSettingId: number;
@YubaSearchable({ type: SearchableFieldType.string, aka: ['userSetting.name'], dbName: 'userSetting.name' })
userSettingName: string;
@YubaSearchable({ type: SearchableFieldType.string, aka: ['userSetting.value'], dbName: 'userSetting.value' })
userSettingValue: string;
}2
3
4
5
6
7
8
9
10
11
12
13
14
It is strongly discouraged to add embedded fields (except for JSON fields) to the DTO because of the lack of support for nested fields in the find. It also allows to have a more flexible API, as the dev can choose to expose the field or not.
Maria json field support
Maria json fields are not supported as of now, but could be implemented with new json specific operators.
Query operators
This sections details the different types of queries that can be used to search for entities. For each query type, we detail the behavior as it is implemented in the current version of the API.
High order queries
AND
Both side of the operator must be true to return the result
OR
One side of the operator must be true to return the result
Low order queries
Always true
Returns all results no matter what
Always false
Returns no results no mater what
Equal
Equal is an operator that has a single value as a right operand, it is used to compare a field to a value.
On text types
Checks for strict equality between 2 texts, case-sensitive
| x is 'foo' | x is 'bar' | x is empty | |
|---|---|---|---|
| x eq 'foo' | 1 | 0 | 0 |
| x eq 'bar' | 0 | 1 | 0 |
| x eq 'FOO' | 0 | 0 | 0 |
| x eq 'f' | 0 | 0 | 0 |
| x eq 'fooo' | 0 | 0 | 0 |
| x eq 'o' | 0 | 0 | 0 |
On number types
Checks for strict equality between 2 numbers (excluding NaN). For floats the equality could behave strangely, when going into high decimal places
| x is 0 | x is 5 | x is empty | |
|---|---|---|---|
| x eq 0 | 1 | 0 | 0 |
| x eq 5 | 0 | 1 | 0 |
| x eq -1 | 0 | 0 | 0 |
| x eq -5 | 0 | 0 | 0 |
| x eq NaN* | type error | type error | type error |
* NaN is a special value in js that will be considered like 'empty' by the backend. See https://www.w3schools.com/jsref/jsref_number_nan.asp
Greater
Greater is an operator that has a single value as a right operand, it is used to find if the field is bigger than the value.
On text types
Check if the string value is greater than the right operand by lexicographical order.
| x is 'foo' | x is 'bar' | x is empty | x is '5' | |
|---|---|---|---|---|
| x gt 'foo' | 0 | 0 | 0 | 0 |
| x gt '1' | 1 | 1 | 0 | 1 |
| x gt '01' | 1 | 1 | 0 | 1 |
| x gt 'bar' | 1 | 0 | 0 | 0 |
| x gt '09' | 1 | 1 | 0 | 1* |
| x gt '9' | 1 | 1 | 0 | 0 |
* '09' is greater than '5' because ascii code of '0' is 48 and ascii code of '5' is 53
On number types
Checks if the number field is greater than the number value
| x is 0 | x is 5 | x is empty | |
|---|---|---|---|
| x gt 0 | 0 | 1 | 0 |
| x gt 5 | 0 | 0 | 0 |
Greater or equal
Same as greater, but also returns the matched dto if the field is equal to the value
On text types
Check if the string value is greater than or equal to the right operand by lexicographical order.
WARNING
Not tested in e2e use at your own risk
On number types
Checks if the number field is greater than or equal to the number value
| x is 0 | x is 5 | x is empty | |
|---|---|---|---|
| x gt 0 | 0 | 1 | 0 |
| x gt 5 | 1 | 1 | 0 |
In
In is an operator that has a list of values as a right operand. It functionally is equivalent to a series of OR operators.
WARNING
Empty arrays will yield an error. The type of values in the array must be the same as the type of the field. For example, if the field is a number, the array must contain only numbers. values like [], [1, 'foo'], [1, null] will yield an error.
On text types
Checks if the string value is in the list of values, case-sensitive
| x is 'foo' | x is 'bar' | x is empty | |
|---|---|---|---|
| x in ['foo'] | 1 | 0 | 0 |
| x in ['foo', 'bar'] | 1 | 1 | 0 |
| x in ['FOO'] | 0 | 0 | 0 |
| x in ['FOO','foo'] | 1 | 0 | 0 |
On number types
Checks if the number value is in the list of values
WARNING
Not tested in e2e use at your own risk
IsNull
IsNull is an operator that has a single value as a right operand, it is used to check if a field has a value.
| x is 'foo' | x is 'bar' | x is empty | |
|---|---|---|---|
| x isNull 'true' | 0 | 0 | 1 |
| x isNull 'false' | 1 | 1 | 0 |
Localized Search (Contains)
Localized search is an operator that has a two values as a right operand, it is used to find if the field contains a value in a specific language.
{
"type": "localizedSearch",
"field": "name",
"userLang": "fra",
"value": "%Distrib%"
}2
3
4
5
6
On text types
| x is 'foo' | x is 'bar' | x is empty | |
|---|---|---|---|
| x localizedSearch 'foo' | 1 | 0 | 0 |
| x localizedSearch 'f%' | 1 | 0 | 0 |
| x localizedSearch 'o%' | 0 | 0 | 0 |
| x localizedSearch '%o' | 0 | 0 | 0 |
| x localizedSearch '%foo%' | 1 | 0 | 0 |
| x localizedSearch 'f%o' | 1 | 0 | 0 |
Lower
See greater
On text types
Check if the string value is lower than the right operand by lexicographical order.
WARNING
Not tested in e2e use at your own risk
On number types
Checks if the number field is lower than the number value
WARNING
Not tested in e2e use at your own risk
Lower or equal
See greater or equal
On text types
Check if the string value is lower than or equal to the right operand by lexicographical order.
WARNING
Not tested in e2e use at your own risk
On number types
Checks if the number field is lower than or equal to the number value
WARNING
Not tested in e2e use at your own risk
NotIn
NotIn is an operator that has a list of values as a right operand. It functionally is equivalent to a series of OR operators.
WARNING
Empty arrays will yield an error. The type of values in the array must be the same as the type of the field. For example, if the field is a number, the array must contain only numbers. values like [], [1, 'foo'], [1, null] will yield an error.
On text types
Checks if the string value is in the list of values, case-sensitive
| x is 'foo' | x is 'bar' | x is empty | |
|---|---|---|---|
| x notIn ['foo'] | 0 | 1 | 0 |
| x notIn ['foo', 'bar'] | 0 | 0 | 0 |
| x notIn ['FOO'] | 0 | 0 | 0 |
| x notIn ['FOO','foo'] | 0 | 1 | 0 |
On number types
Checks if the number value is in the list of values
WARNING
Not tested in e2e use at your own risk
Search (Contains)
Search is an operator that has a single value as a right operand, it is used to find if the field contains the value.
On text types
| x is 'foo' | x is 'bar' | x is empty | |
|---|---|---|---|
| x search 'foo' | 1 | 0 | 0 |
| x search 'f%' | 1 | 0 | 0 |
| x search 'o%' | 0 | 0 | 0 |
| x search '%o' | 0 | 0 | 0 |
| x search '%foo%' | 1 | 0 | 0 |
| x search 'f%o' | 1 | 0 | 0 |
Type handling
Each searchable field is typed, the type of the field is used to determine the behavior of the operators, and the checks that are performed on the operands.
Empty or mistyped lest/right operand will yield an error. It is not possible if the field is typed as T to search for a value of type U, or null.
Example for equality operator:
| x is 'foo' | x is 5 | x is empty | |
|---|---|---|---|
| x eq 'foo' | 1 | type error | type error |
| x eq 5 | type error | 1 | type error |
| x eq empty | type error | type error | type error* |
* Empty cannot be considered like other types and complicates implementation of resolvers, we should instead create a nes 'is empty' operator
As of now, supported types are:
- string : simple json string
- number : simple json number
- json : simple json object (not supported yet)
TIP
Dates are handled like strings and are internally converted to a full iso 8601 string, allowing checks for equality, greater, lower, etc even if format are different. Ie if the db value is 2020-01-01T00:00:00+00:00 it is considered equal to 2020-01-01, 2020-01-01T00:00:00Z, 2020-01-01T00:00:00+01:00, etc.
Dates are compatible with the equals, greater, lower, greater or equal, lower or equal, in and search operators.
As of now malformed dates will not yield an error, but no results will be returned.
WARNING
Limitation on the types on low order query are the same as the DTO itself, see swagger for more details.
Multi tenant security and support
Multi tenant support is implemented automatically by the basic logic layer, the only thing that is required is to add the secure db layer with a base query that filters the entities by tenant id (ie most likely the account id).
It means that a correct token is required to access the entities. The secure db layer will automatically add base query in an AND clause with the find query that is passed to the API.
Some tests have been added to ensure that the multi tenant support is working as expected, and that sql injection is not possible. We should still be cautious about it and organise a regular security audits.
The basic logic layer has been upgraded with an ownership at the service layer that ensures that the user can only access the entities that are owned by the user. This is an extra security that should not be required, but it is better to be safe than sorry. This security is only added to the find api for now
Case sensitivity
All text operators are case-sensitive, ie 'foo' is not equal to 'FOO'. For a case-insensitive search, the boolean ' caseSensitive' could be set to false. This is not yet tested on all operators.
Pagination and sorting
Pagination is supported but not sorting (for now), limitation on the pagination are the same as the CRUD API. See the pagination page for more information.
Errors
A variety of errors with the code 'INVALID_QUERY' can be returned by the annotation system. The error message will contain the details of the error, including a ineligible message and the field that caused the error.
INVALID_QUERY are thrown if :
- an operator is unknown
- a field is unknown or not searchable
- a value is not of the correct type (including empty values, and arrays)
Additionally, the logic layer can throw usual errors like 'UNAUTHORIZED' if the user is not allowed to access, or an unknown error.
If a find do not yield any results, the logic layer will return an empty result and not an error.
Performance
It is recommended to use indexed fields to improve performance. Unfortunately, the indexed fields are not documented, but keys and ids are most likely indexed.
Since the query is user generated, it is possible to create a query that is not optimized and will yield poor performance. Additional indexes could be added to the database to improve performance.
The performance are affected by the number of entities that are returned, the complexity of the query, the needed joins to make the DTO, and the complexity of the additional business logic.
No performance tests have been done yet, but the logic layer, if not overridden with additional business logic, is designed to be as light as possible.
Manual matchers
Each query must implement the match method
abstract match(obj: any): boolean;This method allow for future extension of the system by allowing true agnostic search at the service layer. The service would fetch a page of entities, and then filter them with the match method. This would allow for a more flexible system but would be slower.
This is not tested and the behavior is not guaranteed to be consistent with maria entities
Examples
Full example are available in the monorepo (class TestLogicLayer)