API Reference

pest.Parser

A pest parser.

This class provides methods to parse text using a pest grammar, generate parser source code, and inspect the grammar tree.

PARAMETER	DESCRIPTION
rules	Mapping of rule names to Rule objects. TYPE: Mapping[str, Rule]
doc	Optional list of grammar documentation lines. TYPE: list[str] | None DEFAULT: None
optimizer	Optional optimizer to apply to the rules. TYPE: Optimizer | None DEFAULT: None
debug	If True, enables debug output during optimization. TYPE: bool DEFAULT: False

ATTRIBUTE	DESCRIPTION
rules	A mapping of rule names to Rule instances, including built-ins. TYPE: dict[str, Rule]
doc	An optional list of grammar documentation lines.

from_grammar classmethod

from_grammar(
    grammar: str,
    *,
    optimizer: Optimizer | None = DEFAULT_OPTIMIZER,
    debug: bool = False,
) -> Parser

Parse a grammar definition and return a new Parser for it.

PARAMETER	DESCRIPTION
grammar	The grammar definition as a string. TYPE: str
optimizer	Optional optimizer to apply to the rules. TYPE: Optimizer | None DEFAULT: DEFAULT_OPTIMIZER
debug	If True, enables debug output during optimization. TYPE: bool DEFAULT: False

RETURNS	DESCRIPTION
Parser	A new parser instance for the given grammar. TYPE: Parser

RAISES	DESCRIPTION
PestGrammarSyntaxError	If grammar is invalid.

generate

generate() -> str

Return a generated parser as Python module source code.

RETURNS	DESCRIPTION
str	The generated Python source code for the parser. TYPE: str

parse

parse(
    start_rule: str, text: str, *, start_pos: int = 0
) -> Pairs

Parse text starting from the specified start_rule.

PARAMETER	DESCRIPTION
start_rule	The name of the rule to start parsing from. TYPE: str
text	The input string to parse. TYPE: str
start_pos	The position in the input string to start parsing from (default: 0). TYPE: int DEFAULT: 0

RETURNS	DESCRIPTION
Pairs	The parse tree as a Pairs object. TYPE: Pairs

RAISES	DESCRIPTION
KeyError	If start_rule is not a valid rule name.
PestParsingError	If the input text cannot be parsed according to the grammar.

tree_view

tree_view() -> str

Return a tree view for each non-built-in rule in this grammar.

RETURNS	DESCRIPTION
str	A string representation of the grammar's rule tree. TYPE: str

pest.Pair

A matching pair of Tokens and everything between them.

Represents a node in the parse tree, corresponding to a matched rule and its children.

PARAMETER	DESCRIPTION
input_	The input string. TYPE: str
start	Start position in the input. TYPE: int
end	End position in the input. TYPE: int
rule	The rule or rule frame this pair represents. TYPE: Rule | RuleFrame
children	List of child pairs (subrules). TYPE: list[Pair] | None DEFAULT: None
tag	Optional tag for this node. TYPE: str | None DEFAULT: None

inner_texts property

inner_texts: list[str]

The list of substrings pointed to by this pair's children.

text property

text: str

The substring pointed to by this token pair.

as_str

as_str() -> str

Return the substring pointed to by this token pair.

dump

dump() -> dict[str, object]

Return a pest-debug-like JSON structure representing this pair.

dumps

dumps(indent: int = 0, *, new_line: bool = True) -> str

Return a string representation of this token pair and all its children.

Translated from the format_pair function found in the source for pest.rs.

https://github.com/pest-parser/site/blob/master/src/lib.rs.

inner

inner() -> Pairs

Return inner pairs between this token pair.

line_col

line_col() -> tuple[int, int]

Return the line and column number of this pair's start position.

span

span() -> Span

Return the (start, end) span of this node.

stream

stream() -> Stream

Return inner pairs as a stream.

tokens

tokens() -> Iterator[Token]

Yield start and end tokens for this pair and any children in between.

pest.Pairs

Bases: Sequence[Pair]

A sequence of token pairs.

Provides sequence and utility methods for working with lists of Pair objects.

PARAMETER	DESCRIPTION
pairs	List of Pair objects. TYPE: list[Pair]

dump

dump() -> list[dict[str, object]]

Return pairs as a JSON-like list of dicts.

dumps

dumps(*, compact: bool = True) -> str

Return a JSON formatted string representation of this node.

PARAMETER	DESCRIPTION
compact	If True, returns a compact string; otherwise, pretty-prints JSON. TYPE: bool DEFAULT: True

RETURNS	DESCRIPTION
str	A string representation of the pairs.

find_first_tagged

find_first_tagged(label: str) -> Pair | None

Finds the first pair that has its node tagged with label.

PARAMETER	DESCRIPTION
label	The tag to search for. TYPE: str

RETURNS	DESCRIPTION
Pair | None	The first Pair with the given tag, or None if not found.

find_tagged

find_tagged(label: str) -> Iterator[Pair]

Iterate over pairs tagged with label.

PARAMETER	DESCRIPTION
label	The tag to search for. TYPE: str

RETURNS	DESCRIPTION
Iterator[Pair]	An iterator over all Pairs with the given tag.

first

first() -> Pair

Return the single root pair.

RETURNS	DESCRIPTION
Pair	The first Pair in the sequence.

flatten

flatten() -> Iterator[Pair]

Generate a flat iterator over all pairs and their descendants.

stream

stream() -> Stream

Return pairs as a stream that can be stepped through.

tokens

tokens() -> Iterator[Token]

Yield start and end tokens for each pair in the sequence.

pest.PrattParser

Bases: ABC, Generic[ExprT]

Generic Pratt parser base class operating on a pest Stream of Pairs.

Subclasses define how to construct AST nodes by overriding the four abstract parse_* methods and providing operator-precedence tables.

INFIX_OPS class-attribute

INFIX_OPS: dict[str, tuple[int, bool]] = {}

Mapping of infix operator rule names to (precedence, right_associative).

LEFT_ASSOC class-attribute instance-attribute

LEFT_ASSOC = False

An alias for False.

Use it as the second item in INFIX_OPS values, right_associative, for improved readability.

POSTFIX_OPS class-attribute

POSTFIX_OPS: dict[str, int] = {}

Mapping of postfix operator rule names to precedence levels.

PREFIX_OPS class-attribute

PREFIX_OPS: dict[str, int] = {}

Mapping of prefix operator rule names to precedence levels.

RIGHT_ASSOC class-attribute instance-attribute

RIGHT_ASSOC = True

An alias for True.

Use it as the second item in INFIX_OPS values, right_associative, for improved readability.

parse_expr

parse_expr(stream: Stream, min_prec: int = 0) -> ExprT

Parse an expression from a pest Stream using Pratt precedence rules.

parse_infix abstractmethod

parse_infix(lhs: ExprT, op: Pair, rhs: ExprT) -> ExprT

Build a node for an infix operator expression.

parse_postfix abstractmethod

parse_postfix(lhs: ExprT, op: Pair) -> ExprT

Build a node for a postfix operator expression.

parse_prefix abstractmethod

parse_prefix(op: Pair, rhs: ExprT) -> ExprT

Build a node for a prefix operator expression.

parse_primary abstractmethod

parse_primary(pair: Pair) -> ExprT

Parse a primary expression: literal, variable, or parenthesized.

pest.Stream

Step through pairs of tokens.

Provides a simple interface for sequential access to a list of Pair objects.

backup

backup() -> None

Go back one position in the stream, if possible.

next

next() -> Pair | None

Return the next pair and advance the stream.

RETURNS	DESCRIPTION
Pair | None	The next Pair, or None if at the end of the stream.

peek

peek() -> Pair | None

Return the next pair without advancing the stream.

RETURNS	DESCRIPTION
Pair | None	The next Pair, or None if at the end of the stream.

pest.Token

User-facing token stream element (start or end of a rule).

PARAMETER	DESCRIPTION
rule	name of the matched rule. TYPE: Rule | RuleFrame
pos	start position in Unicode code points. TYPE: int

pest.Start

Bases: Token

A token indicating the start of a rule.

pest.End

Bases: Token

A token indicating the end of a rule.

pest.Position

Bases: NamedTuple

A position in a string as a Unicode codepoint offset.

Provides utilities for determining line and column numbers.

line_col

line_col() -> tuple[int, int]

Return the line and column number of this position.

RETURNS	DESCRIPTION
tuple[int, int]	A tuple (line_number, column_number), both 1-based.

line_of

line_of() -> str

Return the line of text that contains this position.

pest.Span

Bases: NamedTuple

A half-open interval [start, end) into the input string.

Represents a substring of the input, along with its start and end positions.

as_str

as_str() -> str

Return the slice of the source corresponding to this span.

end_pos

end_pos() -> Position

Return this span's end position.

lines

lines() -> list[str]

Return a list of lines covered by this span.

Includes lines that are partially covered.

split

split() -> tuple[Position, Position]

Return a tuple of start position and end position.

start_pos

start_pos() -> Position

Return this span's start position.

pest.Rule

Bases: Expression

Base class for all rules.

children

children() -> list[Expression]

Return this expression's children.

generate

generate(
    gen: Builder, matched_var: str, pairs_var: str
) -> None

Emit Python source code that implements this grammar expression.

parse

parse(state: ParserState, pairs: list[Pair]) -> bool

Attempt to match this expression against the input at start.

with_children

with_children(expressions: list[Expression]) -> Self

Return a new instance of this expression with child expressions replaced.

pest.RuleFrame

Rule meta data for the generated rule stack.

Generated parser don't have access to complete Rule object. RuleFrame is a lightweight stand in.

ATTRIBUTE	DESCRIPTION
name	The rule's name. TYPE: str
modifier	A bit mask encoding modifiers applied to the rule. TYPE: int

pest.PestParsingError

Bases: Exception

An exception raised when an input string can't be passed by a pest grammar.

detailed_message

detailed_message() -> str

Return an error message formatted with extra context info.

expected

expected(
    expected: dict[str, list[str]],
    unexpected: dict[str, list[str]],
) -> str

Return the expected/unexpected part of a detailed error message.

expected_labels

expected_labels(
    expected: dict[str, list[str]],
    unexpected: dict[str, list[str]],
) -> str | None

Return a string representation of expected and unexpected labels.