Filter Mechanism

This chapter is dedicated to introducing users the filtering mechanism of UQL.

Introduction

uQL uses high-performance tree-structured filters that comprise logical operators and basic filtering conditions, the latter of which comprise filter item, conditional operator and value. Such a combination can implement varied kinds of filtering logic.

Ultipa filters are used in queries in all senses. With the aim of screening meta-data against their properties, Ultipa filters appear in node or edge like parameters, such as nodes(), edges(), node_filter(), edge_filter(), n(), e(), etc.

When filters are applied, subgraph can be dynamically extracted out of the whole graph.

Prerequisite

Before constructing filtering rules, make sure that you know the characteristics of your graph data (i.e., topology, data range), analyze and experiment with the data, and make sure your rules are reasonable and efficient.

We hereby offer the following tips:

  • Be specific, such as picking a specific id or property name
  • Put the highest hit-rate condition first and foremost
  • Use minimum (or less) filters
  • Avoid building a deep logic tree

Conditional Operator

Conditional operators are used when constructing the basic filtering conditions that are the leaves of the logic tree.

Syntax of basic filtering condition is:

{ <property> : { <operator> : <value> } }

Ultipa filters introduce the following commonly seen conditional operators:

Operator Equivalent Notation Meaning
$eq = returns true when property value is equal to the given value
$neq returns true when property value is not equal to the given value
$lt < returns true when property value is less than the given value
$gt > returns true when property value is greater than the given value
$lte returns true when property value is less than or equal to the given value
$gte returns true when property value is greater than or equal to the given value
$bt returns true when property value is between the two given values
$in returns true when property value is among the given values
$nin returns true when property value is not among the given values

$eq

Condition with $eq (equal):

{ <property> : { $eq : <string | number> } }

Operator $eq is one of the two most commonly used operators, the other being the operator $and. Operator $eq is usually omitted and the condition is abbreviated as:

{ <property> : <string | number> }

Under most circumstances in practical, the abbreviated form is recommended for its obvious simplicity.

When _id is the only filter item of the parameter, the item name is omitted and the condition becomes:

<id>

Example 1: Find nodes with 'industry' equal to 'Education'

find()
    .nodes( { industry: "Education" } )
    .limit(10).select(*)

Example 2: Find node with _id = 23

find().nodes(23).select(*)

$neq

Condition with $neq (not equal):

{ <property> : { $neq : <string | number> } }

Example: Find nodes with 'rating' not equal to 9.5

find()
    .nodes( { rating: { $neq : 9.5 } } )
    .limit(10).select(*)

$lt

Condition with $lt (less than):

{ <property> : { $lt : <number> } }

Example: Find nodes with rating < 9.0

find()
    .nodes( { rating: { $lt : 9.0 } } )
    .limit(10).select(*)

$lte

Condition with $lte (less than or equal):

{ <property> : { $lte : <number> } }

Example: Find nodes with rating ≤ 9.0

find()
    .nodes( { rating: { $lte : 9.0 } } )
    .limit(10).select(*)

$gt

Condition with $gt (greater than):

{ <property> : { $gt: <number> } }

Example: Find nodes with rating > 9.0

find()
    .nodes( { rating: { $gt : 9.0 } } )
    .limit(10).select(*)

$gte

Condition with $gte (greater than or equal):

{ <property> : { $gte: <number> } }

Example: Find nodes with rating ≥ 9.0

find()
    .nodes( { rating: { $gt : 9.0 } } )
    .limit(10).select(*)

$bt

Condition with $bt (between):

{ <property> : { $bt: [<number1>, <number2>] } }

Note that number1 can NOT be greater than number2

Example: Find nodes with 9.0 ≤ rating ≤ 9.5

find()
    .nodes( { rating: { $bt : [9.0, 9.5] } } )
    .limit(10).select(*)

$in

Operator $in checks whether the value of filter item <property> is equal to any of the given values.

Condition with $in:

{ <property> : { $in : [ <string | number>, ... ] } }

When _id is the only filter item of the parameter, the item name is omitted and the condition becomes:

<id>, <id>, ...

Example 1: Find nodes with industry of 'Education', 'IT' or 'Health'

find()
    .nodes( {industry: {$in: ["Education", "IT", "Health"] } })
    .limit(10).select(*)

Example 2: Find nodes with _id of 12, 23 and 45

find()
    .nodes( [12, 23, 45] ).select(*)

$nin

Operator $nin is exactly the opposite of operator $in, examining whether the item value is NOT equal to any of the given values.

Condition with $nin:

{ <property> : { $nin : [ <string | number>, ... ] } }

Example: Find nodes with 'year' that is NOT the same as any of the values from [1980, 1985, 2000]

find()
    .nodes( { year: { $nin : [ 1980, 1985, 2000 ] } } )
    .limit(10).select(*)

Logical Operator

Logical operators organize basic filtering conditions in a recursive way into compound filtering conditions, which ultimately represent the logic tree (structure) that runs from leaves all the way to the root.

The logical structure is illustrated here:


Figure: Logical Structure with Operators and Conditions

Syntax of compound filtering condition is:

{ <operator> : [<sub-condition>, <sub-condition>, ...] }

The sub-conditions can be either basic or compound filtering conditions.

Ultipa filters introduce two kinds of logical operators:

Operator Equivalent Notation Meaning
$and && returns true when all the sub-conditions are met
$or | | returns true when any of the sub-conditions is met

$or

Condition with $or:

{ $or : [<sub-condition>, <sub-condition>, ...] }

Example: Find nodes that are either year > 1999 or industry = 'IT'

find()
    .nodes( {$or: [{year: {$gt: 1999}}, {industry: "IT"}] })
    .limit(10).select(*)

$and

Condition with $and:

{ $and : [<sub-condition>, <sub-condition>, ...] }

When sub-conditions are all basic filtering conditions, namely the sub-conditions do not contain any logical operator, the operator $and is allowed to be omitted and the above condition becomes:

{
    <property> : { <operator> : <value> },
    <property> : { <operator> : <value> },
    ...
}

This abbreviated form can be seen as a basic filtering condition with multiple filter items, and is widely used in practice.

Example 1: Find nodes that are female, and either year > 1999 or industry = 'IT'

find()
    .nodes({
        $and: [
            {gender: "female"}, 
            {$or: [{year: {$gt: 1999}}, {industry: "IT"}] }
        ]
    })
    .limit(10).select(*)

Example 2: Find nodes that are female, year > 1999 and industry = 'IT'

find()
    .nodes( { gender: "female", year: {$gt: 1999}, industry: "IT" })
    .limit(10).select(*)

Example 3: Find edges that are

  • type = 'review' and value > 500
  • Or, type = 'comment' and value > 100
find()
    .edges({ 
        $or :[ 
            {type : "review", value: {$gt: 500}},
            {type : "comment", value: {$gt: 100}}
        ]
    })
    .limit(10).select(*)