Parser (xonsh.parser)

Implements the xonsh parser

class xonsh.parser.Location(fname, lineno, column=None)[source]

Location in a file.

Takes a filename, line number, and optionally a column number.

class xonsh.parser.Parser(lexer_optimize=True, lexer_table='xonsh.lexer_table', yacc_optimize=True, yacc_table='xonsh.parser_table', yacc_debug=False, outputdir=None)[source]

A class that parses the xonsh language.

Parameters:

lexer_optimize : bool, optional

Set to false when unstable and true when lexer is stable.

lexer_table : str, optional

Lexer module used when optimized.

yacc_optimize : bool, optional

Set to false when unstable and true when parser is stable.

yacc_table : str, optional

Parser module used when optimized.

yacc_debug : debug, optional

Dumps extra debug info.

outputdir : str or None, optional

The directory to place generated tables within.

currloc(lineno, column=None)[source]

Returns the current location.

expr(p)[source]

Creates an expression for a token.

p_ampersand_shift_expr(p)[source]

ampersand_shift_expr : AMPERSAND shift_expr

p_and_expr(p)[source]

and_expr : shift_expr ampersand_shift_expr_list_opt

p_and_not_test(p)[source]

and_not_test : AND not_test

p_and_test(p)[source]

and_test : not_test and_not_test_list_opt

p_arglist(p)[source]

arglist : argument comma_opt | argument_comma_list argument comma_opt | argument_comma_list_opt TIMES test comma_argument_list_opt | argument_comma_list_opt TIMES test COMMA POW test | argument_comma_list_opt TIMES test comma_argument_list COMMA POW test | argument_comma_list_opt POW test

p_argument(p)[source]

argument : test | test comp_for | test EQUALS test

p_argument_comma(p)[source]

argument_comma : argument COMMA

p_arith_expr(p)[source]

arith_expr : term | term pm_term_list

p_as_expr(p)[source]

as_expr : AS expr

p_as_name(p)[source]

as_name : AS NAME

p_assert_stmt(p)[source]

assert_stmt : ASSERT test comma_test_opt

p_atom(p)[source]

atom : LPAREN yield_expr_or_testlist_comp_opt RPAREN | LBRACKET testlist_comp_opt RBRACKET | LBRACE dictorsetmaker_opt RBRACE | NAME | number | string_literal_list | ELLIPSIS | NONE | TRUE | FALSE | REGEXPATH | DOLLAR_NAME | DOLLAR_LBRACE test RBRACE | DOLLAR_LPAREN subproc RPAREN | DOLLAR_LBRACKET subproc RBRACKET

p_attr_name(p)[source]

attr_name : NAME | NAME attr_period_name_list

p_attr_period_name(p)[source]

attr_period_name : PERIOD NAME

p_augassign(p)[source]

augassign : PLUSEQUAL | MINUSEQUAL | TIMESEQUAL | DIVEQUAL | MODEQUAL | AMPERSANDEQUAL | PIPEEQUAL | XOREQUAL | LSHIFTEQUAL | RSHIFTEQUAL | POWEQUAL | DOUBLEDIVEQUAL

p_break_stmt(p)[source]

break_stmt : BREAK

p_classdef(p)[source]

classdef : CLASS NAME func_call_opt COLON suite

p_classdef_or_funcdef(p)[source]

classdef_or_funcdef : classdef | funcdef

p_colon_test(p)[source]

colon_test : COLON test

p_comma_argument(p)[source]

comma_argument : COMMA argument

p_comma_dotted_as_name(p)[source]

comma_dotted_as_name : COMMA dotted_as_name

p_comma_expr_or_star_expr(p)[source]

comma_expr_or_star_expr : COMMA expr_or_star_expr

p_comma_import_as_name(p)[source]

comma_import_as_name : COMMA import_as_name

p_comma_item(p)[source]

comma_item : COMMA test COLON test

p_comma_name(p)[source]

comma_name : COMMA NAME

p_comma_opt(p)[source]

comma_opt : COMMA | empty

p_comma_pow_tfpdef(p)[source]

comma_pow_tfpdef : COMMA POW tfpdef

p_comma_pow_vfpdef(p)[source]

comma_pow_vfpdef : COMMA POW vfpdef

p_comma_subscript(p)[source]

comma_subscript : COMMA subscript

p_comma_test(p)[source]

comma_test : COMMA test

p_comma_test_or_star_expr(p)[source]

comma_test_or_star_expr : COMMA test_or_star_expr

p_comma_tfpdef(p)[source]

comma_tfpdef : COMMA | COMMA tfpdef equals_test_opt

p_comma_vfpdef(p)[source]

comma_vfpdef : COMMA | COMMA vfpdef equals_test_opt

p_comma_with_item(p)[source]

comma_with_item : COMMA with_item

p_comp_for(p)[source]

comp_for : FOR exprlist IN or_test comp_iter_opt

p_comp_if(p)[source]

comp_if : IF test_nocond comp_iter_opt

p_comp_iter(p)[source]

comp_iter : comp_for | comp_if

p_comp_op(p)[source]

comp_op : LT | GT | EQ | GE | LE | NE | IN | NOT IN | IS | IS NOT

p_comp_op_expr(p)[source]

comp_op_expr : comp_op expr

p_comparison(p)[source]

comparison : expr comp_op_expr_list_opt

p_compound_stmt(p)[source]

compound_stmt : if_stmt | while_stmt | for_stmt | try_stmt | with_stmt | funcdef | classdef | decorated

p_continue_stmt(p)[source]

continue_stmt : CONTINUE

p_decorated(p)[source]

decorated : decorators classdef_or_funcdef

p_decorator(p)[source]

decorator : AT attr_name NEWLINE | AT attr_name func_call NEWLINE

p_decorators(p)[source]

decorators : decorator | decorators decorator

p_del_stmt(p)[source]

del_stmt : DEL exprlist

p_dictorsetmaker(p)[source]

dictorsetmaker : test COLON test comp_for | test COLON test comma_item_list comma_opt | test COLON testlist | test comp_for | testlist

p_dotted_as_name(p)[source]

dotted_as_name : dotted_name as_name_opt

p_dotted_as_names(p)[source]

dotted_as_names : dotted_as_name comma_dotted_as_name_list_opt

p_dotted_name(p)[source]

dotted_name : NAME | NAME period_name_list

p_elif_part(p)[source]

elif_part : ELIF test COLON suite

p_else_part(p)[source]

else_part : ELSE COLON suite

p_empty(p)[source]

empty :

p_equals_test(p)[source]

equals_test : EQUALS test

p_equals_yield_expr_or_testlist(p)[source]

equals_yield_expr_or_testlist : EQUALS yield_expr_or_testlist

p_error(p)[source]
p_eval_input(p)[source]

eval_input : testlist newlines_opt

p_except_clause(p)[source]

except_clause : EXCEPT | EXCEPT test as_name_opt

p_except_part(p)[source]

except_part : except_clause COLON suite

p_expr(p)[source]

expr : xor_expr | xor_expr pipe_xor_expr_list

p_expr_or_star_expr(p)[source]

expr_or_star_expr : expr | star_expr

p_expr_stmt(p)[source]

expr_stmt : testlist_star_expr augassign yield_expr_or_testlist | testlist_star_expr equals_yield_expr_or_testlist_list_opt | testlist equals_yield_expr_or_testlist_list_opt | test_comma_list_opt star_expr comma_test_list equals_yield_expr_or_testlist | test_comma_list_opt star_expr comma_opt test_comma_list_opt equals_yield_expr_or_testlist

p_exprlist(p)[source]

exprlist : expr_or_star_expr comma_expr_or_star_expr_list comma_opt | expr_or_star_expr comma_opt

p_factor(p)[source]

factor : PLUS factor | MINUS factor | TILDE factor | power

p_file_input(p)[source]

file_input : file_stmts

p_file_stmts(p)[source]

file_stmts : newline_or_stmt | file_stmts newline_or_stmt

p_finally_part(p)[source]

finally_part : FINALLY COLON suite

p_flow_stmt(p)[source]

flow_stmt : break_stmt | continue_stmt | return_stmt | raise_stmt | yield_stmt

p_for_stmt(p)[source]

for_stmt : FOR exprlist IN testlist COLON suite | FOR exprlist IN testlist COLON suite else_part

p_func_call(p)[source]

func_call : LPAREN arglist_opt RPAREN

p_funcdef(p)[source]

funcdef : DEF NAME parameters rarrow_test_opt COLON suite

p_global_stmt(p)[source]

global_stmt : GLOBAL NAME comma_name_list_opt

p_if_stmt(p)[source]

if_stmt : IF test COLON suite elif_part_list_opt | IF test COLON suite elif_part_list_opt else_part

p_import_as_name(p)[source]

import_as_name : NAME as_name_opt

p_import_as_names(p)[source]

import_as_names : import_as_name comma_import_as_name_list_opt comma_opt

p_import_from(p)[source]

import_from : import_from_pre IMPORT import_from_post

p_import_from_post(p)[source]

import_from_post : TIMES | LPAREN import_as_names RPAREN | import_as_names

p_import_from_pre(p)[source]

import_from_pre : FROM period_or_ellipsis_list_opt dotted_name | FROM period_or_ellipsis_list

p_import_name(p)[source]

import_name : IMPORT dotted_as_names

p_import_stmt(p)[source]

import_stmt : import_name | import_from

p_lambdef(p)[source]

lambdef : LAMBDA varargslist_opt COLON test

p_lambdef_nocond(p)[source]

lambdef_nocond : LAMBDA varargslist_opt COLON test_nocond

p_newline_or_stmt(p)[source]

newline_or_stmt : NEWLINE | stmt

p_newlines(p)[source]

newlines : NEWLINE | newlines NEWLINE

p_nonlocal_stmt(p)[source]

nonlocal_stmt : NONLOCAL NAME comma_name_list_opt

p_not_test(p)[source]

not_test : NOT not_test | comparison

p_number(p)[source]

number : NUMBER

p_op_factor(p)[source]

op_factor : TIMES factor | DIVIDE factor | MOD factor | DOUBLEDIV factor

p_or_and_test(p)[source]

or_and_test : OR and_test

p_or_test(p)[source]

or_test : and_test or_and_test_list_opt

p_parameters(p)[source]

parameters : LPAREN typedargslist_opt RPAREN

p_pass_stmt(p)[source]

pass_stmt : PASS

p_period_name(p)[source]

period_name : PERIOD NAME

p_period_or_ellipsis(p)[source]

period_or_ellipsis : PERIOD | ELLIPSIS

p_pipe_xor_expr(p)[source]

pipe_xor_expr : PIPE xor_expr

p_pm_term(p)[source]

pm_term : PLUS term | MINUS term

p_power(p)[source]

power : atom trailer_list_opt | atom trailer_list_opt POW factor

p_raise_stmt(p)[source]

raise_stmt : RAISE | RAISE test | RAISE test FROM test

p_rarrow_test(p)[source]

rarrow_test : RARROW test

p_return_stmt(p)[source]

return_stmt : RETURN testlist_opt

p_semi_opt(p)[source]

semi_opt : SEMI | empty

p_semi_small_stmt(p)[source]

semi_small_stmt : SEMI small_stmt

p_shift_arith_expr(p)[source]

shift_arith_expr : LSHIFT arith_expr | RSHIFT arith_expr

p_shift_expr(p)[source]

shift_expr : arith_expr shift_arith_expr_list_opt

p_simple_stmt(p)[source]

simple_stmt : small_stmt semi_small_stmt_list semi_opt NEWLINE | small_stmt semi_opt NEWLINE

p_single_input(p)[source]

single_input : compound_stmt NEWLINE

p_sliceop(p)[source]

sliceop : COLON test_opt

p_small_stmt(p)[source]

small_stmt : expr_stmt | del_stmt | pass_stmt | flow_stmt | import_stmt | global_stmt | nonlocal_stmt | assert_stmt

p_star_expr(p)[source]

star_expr : TIMES expr

p_start_symbols(p)[source]

start_symbols : single_input | file_input | eval_input | empty

p_stmt(p)[source]

stmt : simple_stmt | compound_stmt

p_stmt_list(p)[source]

stmt_list : stmt | stmt_list stmt

p_string_literal(p)[source]

string_literal : STRING

p_string_literal_list(p)[source]

string_literal_list : string_literal | string_literal_list string_literal

p_subproc(p)[source]

subproc : subproc_atoms | subproc_atoms WS | subproc AMPERSAND | subproc subproc_special subproc_atoms | subproc subproc_special subproc_atoms WS

p_subproc_arg(p)[source]

subproc_arg : subproc_arg_part | subproc_arg subproc_arg_part

p_subproc_arg_part(p)[source]

subproc_arg_part : NAME | TILDE | PERIOD | DIVIDE | MINUS | PLUS | COLON | AT | EQUALS | TIMES | POW | MOD | XOR | DOUBLEDIV | ELLIPSIS | NONE | TRUE | FALSE | NUMBER

p_subproc_atom(p)[source]

subproc_atom : subproc_arg | string_literal | REGEXPATH | DOLLAR_NAME | AT_LPAREN test RPAREN | DOLLAR_LBRACE test RBRACE | DOLLAR_LPAREN subproc RPAREN | DOLLAR_LBRACKET subproc RBRACKET

p_subproc_atoms(p)[source]

subproc_atoms : subproc_atom | subproc_atoms WS subproc_atom

p_subproc_special(p)[source]

subproc_special : subproc_special_atom | WS subproc_special_atom | subproc_special_atom WS | WS subproc_special_atom WS

p_subproc_special_atom(p)[source]

subproc_special_atom : PIPE | GT | RSHIFT

p_subscript(p)[source]

subscript : test | test_opt COLON test_opt sliceop_opt

p_subscriptlist(p)[source]

subscriptlist : subscript comma_subscript_list_opt comma_opt

p_suite(p)[source]

suite : simple_stmt | NEWLINE INDENT stmt_list DEDENT

p_term(p)[source]

term : factor op_factor_list_opt

p_test(p)[source]

test : or_test | or_test IF or_test ELSE test | lambdef

p_test_comma(p)[source]

test_comma : test COMMA

p_test_nocond(p)[source]

test_nocond : or_test | lambdef_nocond

p_test_or_star_expr(p)[source]

test_or_star_expr : test | star_expr

p_testlist(p)[source]

testlist : test comma_test_list COMMA | test comma_test_list | test COMMA | test

p_testlist_comp(p)[source]

testlist_comp : test_or_star_expr comp_for | test_or_star_expr comma_opt | test_or_star_expr comma_test_or_star_expr_list comma_opt

p_testlist_star_expr(p)[source]

testlist_star_expr : test_or_star_expr comma_test_or_star_expr_list comma_opt | test_or_star_expr comma_opt

p_tfpdef(p)[source]

tfpdef : NAME colon_test_opt

p_trailer(p)[source]

trailer : LPAREN arglist_opt RPAREN | LBRACKET subscriptlist RBRACKET | PERIOD NAME | DOUBLE_QUESTION | QUESTION

p_try_stmt(p)[source]

try_stmt : TRY COLON suite except_part_list else_part finally_part_opt | TRY COLON suite except_part_list finally_part_opt | TRY COLON suite finally_part

p_typedargslist(p)[source]

typedargslist : tfpdef equals_test_opt comma_tfpdef_list_opt comma_opt | tfpdef equals_test_opt comma_tfpdef_list_opt comma_opt TIMES tfpdef_opt COMMA POW vfpdef | tfpdef equals_test_opt comma_tfpdef_list_opt comma_opt TIMES tfpdef_opt comma_tfpdef_list_opt | tfpdef equals_test_opt comma_tfpdef_list_opt comma_opt TIMES tfpdef_opt comma_tfpdef_list COMMA POW tfpdef | tfpdef equals_test_opt comma_tfpdef_list_opt comma_opt POW tfpdef | TIMES tfpdef_opt comma_tfpdef_list comma_pow_tfpdef_opt | TIMES tfpdef_opt comma_pow_tfpdef_opt | POW tfpdef

p_varargslist(p)[source]

varargslist : vfpdef equals_test_opt comma_vfpdef_list_opt comma_opt | vfpdef equals_test_opt comma_vfpdef_list_opt comma_opt TIMES vfpdef_opt COMMA POW vfpdef | vfpdef equals_test_opt comma_vfpdef_list_opt comma_opt TIMES vfpdef_opt comma_vfpdef_list_opt | vfpdef equals_test_opt comma_vfpdef_list_opt comma_opt TIMES vfpdef_opt comma_vfpdef_list COMMA POW vfpdef | vfpdef equals_test_opt comma_vfpdef_list_opt comma_opt POW vfpdef | TIMES vfpdef_opt comma_vfpdef_list comma_pow_vfpdef_opt | TIMES vfpdef_opt comma_pow_vfpdef_opt | POW vfpdef

p_vfpdef(p)[source]

vfpdef : NAME

p_while_stmt(p)[source]

while_stmt : WHILE test COLON suite | WHILE test COLON suite else_part

p_with_item(p)[source]

with_item : test | test as_expr

p_with_stmt(p)[source]

with_stmt : WITH with_item COLON suite | WITH with_item comma_with_item_list COLON suite

p_xor_and_expr(p)[source]

xor_and_expr : XOR and_expr

p_xor_expr(p)[source]

xor_expr : and_expr xor_and_expr_list_opt

p_yield_arg(p)[source]

yield_arg : FROM test | testlist

p_yield_expr(p)[source]

yield_expr : YIELD yield_arg_opt

p_yield_expr_or_testlist(p)[source]

yield_expr_or_testlist : yield_expr | testlist

p_yield_expr_or_testlist_comp(p)[source]

yield_expr_or_testlist_comp : yield_expr | testlist_comp

p_yield_stmt(p)[source]

yield_stmt : yield_expr

parse(s, filename='<code>', mode='exec', debug_level=0)[source]

Returns an abstract syntax tree of xonsh code.

Parameters:

s : str

The xonsh code.

filename : str, optional

Name of the file.

mode : str, optional

Execution mode, one of: exec, eval, or single.

debug_level : str, optional

Debugging level passed down to yacc.

Returns:

tree : AST

reset()[source]

Resets for clean parsing.

token_col(t)[source]

Gets ths token column

col[source]
lineno[source]
precedence = (('left', 'PIPE'), ('left', 'XOR'), ('left', 'AMPERSAND'), ('left', 'EQ', 'NE'), ('left', 'GT', 'GE', 'LT', 'LE'), ('left', 'RSHIFT', 'LSHIFT'), ('left', 'PLUS', 'MINUS'), ('left', 'TIMES', 'DIVIDE', 'DOUBLEDIV', 'MOD'), ('left', 'POW'))
xonsh.parser.binop(x, op, y, lineno=None, col=None)[source]

Creates the AST node for a binary operation.

xonsh.parser.call_split_lines(x, lineno=None, col=None)[source]

Creates the AST node for calling the ‘splitlines’ attribute of an object, nominally a string.

xonsh.parser.empty_list(lineno=None, col=None)[source]

Creates the AST node for an empty list.

xonsh.parser.empty_list_if_newline(x)[source]
xonsh.parser.ensure_has_elts(x, lineno=1, col_offset=1)[source]

Ensures that x is an AST node with elements.

xonsh.parser.ensure_list_from_str_or_list(x, lineno=None, col=None)[source]

Creates the AST node for the following expression:

[x] if isinstance(x, str) else x

Somewhat useful.

xonsh.parser.has_elts(x)[source]

Tests if x is an AST node with elements.

xonsh.parser.load_ctx(x)[source]

Recursively sets ctx to ast.Load()

xonsh.parser.store_ctx(x)[source]

Recursively sets ctx to ast.Store()

xonsh.parser.xonsh_call(name, args, lineno=None, col=None)[source]

Creates the AST node for calling a function of a given name.

xonsh.parser.xonsh_help(x, lineno=None, col=None)[source]

Creates the AST node for calling the __xonsh_help__() function.

xonsh.parser.xonsh_regexpath(x, lineno=None, col=None)[source]

Creates the AST node for calling the __xonsh_regexpath__() function.

xonsh.parser.xonsh_superhelp(x, lineno=None, col=None)[source]

Creates the AST node for calling the __xonsh_superhelp__() function.

Previous page

← Lexer (xonsh.lexer)

This Page