Skip to content

Advanced Usage

Filter Variables

Arbitrary variables can be made available to filter expressions using the filter_context argument to findall() and finditer(). filter_context should be a mapping of strings to JSON-like objects, like lists, dictionaries, strings and integers.

Filter context variables are selected using a filter query starting with the filter context identifier, which defaults to _ and has usage similar to $ and @.

import jsonpath

data = {
    "users": [
        {
            "name": "Sue",
            "score": 100,
        },
        {
            "name": "John",
            "score": 86,
        },
        {
            "name": "Sally",
            "score": 84,
        },
        {
            "name": "Jane",
            "score": 55,
        },
    ]
}

user_names = jsonpath.findall(
    "$.users[?@.score < _.limit].name",
    data,
    filter_context={"limit": 100},
)

Function Extensions

Add, remove or replace filter functions by updating the function_extensions attribute of a JSONPathEnvironment. It is a regular Python dictionary mapping filter function names to any callable, like a function or class with a __call__ method.

Type System for Function Expressions

Section 2.4.1 of RFC 9535 defines a type system for function expressions and requires that we check that filter expressions are well-typed. With that in mind, you are encouraged to implement custom filter functions by extending jsonpath.function_extensions.FilterFunction, which forces you to be explicit about the types of arguments the function extension accepts and the type of its return value.

Info

FilterFunction was new in Python JSONPath version 0.10.0. Prior to that we did not enforce function expression well-typedness. To use any arbitrary callable as a function extension - or if you don't want built-in filter functions to raise a JSONPathTypeError for function expressions that are not well-typed - set well_typed to False when constructing a JSONPathEnvironment.

Example

As an example, we'll add a min() filter function, which will return the minimum of a sequence of values. If any of the values are not comparable, we'll return the special undefined value instead.

from typing import Iterable

import jsonpath
from jsonpath.function_extensions import ExpressionType
from jsonpath.function_extensions import FilterFunction


class MinFilterFunction(FilterFunction):
    """A JSONPath function extension returning the minimum of a sequence."""

    arg_types = [ExpressionType.VALUE]
    return_type = ExpressionType.VALUE

    def __call__(self, value: object) -> object:
        if not isinstance(value, Iterable):
            return jsonpath.UNDEFINED

        try:
            return min(value)
        except TypeError:
            return jsonpath.UNDEFINED


env = jsonpath.JSONPathEnvironment()
env.function_extensions["min"] = MinFilterFunction()

example_data = {"foo": [{"bar": [4, 5]}, {"bar": [1, 5]}]}
print(env.findall("$.foo[?min(@.bar) > 1]", example_data))

Now, when we use env.finall(), env.finditer() or env.compile(), our min function will be available for use in filter expressions.

$..products[?@.price == min($..products.price)]

Built-in Functions

The built-in functions can be removed from a JSONPathEnvironment by deleting the entry from function_extensions.

import jsonpath

env = jsonpath.JSONPathEnvironment()
del env.function_extensions["keys"]

Or aliased with an additional entry.

import jsonpath

env = jsonpath.JSONPathEnvironment()
env.function_extensions["properties"] = env.function_extensions["keys"]

Alternatively, you could subclass JSONPathEnvironment and override the setup_function_extensions method.

from typing import Iterable
import jsonpath

class MyEnv(jsonpath.JSONPathEnvironment):
    def setup_function_extensions(self) -> None:
        super().setup_function_extensions()
        self.function_extensions["properties"] = self.function_extensions["keys"]
        self.function_extensions["min"] = min_filter


def min_filter(obj: object) -> object:
    if not isinstance(obj, Iterable):
        return jsonpath.UNDEFINED

    try:
        return min(obj)
    except TypeError:
        return jsonpath.UNDEFINED

env = MyEnv()

Compile Time Validation

Calls to type-aware function extension are validated at JSONPath compile-time automatically. If well_typed is set to False or a custom function extension does not inherit from FilterFunction, its arguments can be validated by implementing the function as a class with a __call__ method, and a validate method. validate will be called after parsing the function, giving you the opportunity to inspect its arguments and raise a JSONPathTypeError should any arguments be unacceptable. If defined, validate must take a reference to the current environment, an argument list and the token pointing to the start of the function call.

def validate(
        self,
        env: JSONPathEnvironment,
        args: List[FilterExpression],
        token: Token,
) -> List[FilterExpression]:

It should return an argument list, either the same as the input argument list, or a modified version of it. See the implementation of the built-in match function for an example.

Custom Environments

Python JSONPath can be customized by subclassing JSONPathEnvironment and overriding class attributes and/or methods. Then using findall(), finditer() and compile() methods of that subclass.

Identifier Tokens

The default identifier tokens, like $ and @, can be changed by setting attributes a on JSONPathEnvironment. This example sets the root token (default $) to be ^.

import JSONPathEnvironment

class MyJSONPathEnvironment(JSONPathEnvironment):
    root_token = "^"


data = {
    "users": [
        {"name": "Sue", "score": 100},
        {"name": "John", "score": 86},
        {"name": "Sally", "score": 84},
        {"name": "Jane", "score": 55},
    ],
    "limit": 100,
}

env = MyJSONPathEnvironment()
user_names = env.findall(
    "^.users[?@.score < ^.limit].name",
    data,
)

This table shows all available identifier token attributes.

attribute default
filter_context_token _
keys_token #
root_token $
self_token @

Logical Operator Tokens

By default, we accept both Python and C-style logical operators in filter expressions. That is, not and ! are equivalent, and and && are equivalent and or and || are equivalent. You can change this using class attributes on a Lexer subclass and setting the lexer_class attribute on a JSONPathEnvironment.

This example changes all three logical operators to strictly match the JSONPath spec.

from jsonpath import JSONPathEnvironment
from jsonpath import Lexer

class MyLexer(Lexer):
    logical_not_pattern = r"!"
    logical_and_pattern = r"&&"
    logical_or_pattern = r"\|\|"

class MyJSONPathEnvironment(JSONPathEnvironment):
    lexer_class = MyLexer

env = MyJSONPathEnvironment()
env.compile("$.foo[?@.a > 0 && @.b < 100]")  # OK
env.compile("$.foo[?@.a > 0 and @.b < 100]")  # JSONPathSyntaxError

Keys Selector

The non-standard keys selector is used to retrieve the keys/properties from a JSON Object or Python mapping. It defaults to ~ and can be changed using the keys_selector_token attribute on a JSONPathEnvironment subclass.

This example changes the keys selector to *~.

from jsonpath import JSONPathEnvironment

class MyJSONPathEnvironment(JSONPathEnvironment):
    keys_selector_token = "*~"

data = {
    "users": [
        {"name": "Sue", "score": 100},
        {"name": "John", "score": 86},
        {"name": "Sally", "score": 84},
        {"name": "Jane", "score": 55},
    ],
    "limit": 100,
}

env = MyJSONPathEnvironment()
print(env.findall("$.users[0].*~", data))  # ['name', 'score']

Array Index Limits

Python JSONPath limits the minimum and maximum JSON array or Python sequence indices (including slice steps) allowed in a JSONPath query. The default minimum allowed index is set to -(2**53) + 1, and the maximum to (2**53) - 1. When a limit is reached, a JSONPathIndexError is raised.

You can change the minimum and maximum allowed indices using the min_int_index and max_int_index attributes on a JSONPathEnvironment subclass.

from jsonpath import JSONPathEnvironment

class MyJSONPathEnvironment(JSONPathEnvironment):
    min_int_index = -100
    max_int_index = 100

env = MyJSONPathEnvironment()
query = env.compile("$.users[999]")
# jsonpath.exceptions.JSONPathIndexError: index out of range, line 1, column 8

Subclassing Lexer

TODO:

Subclassing Parser

TODO:

Get Item

TODO:

Truthiness and Existence

TODO:

Filter Infix Expressions

TODO: