• Главная
• pQuery
• TokenizerBase

PHP Класс pQuery\TokenizerBase

Can convert any string into tokens. The base class only supports identifier/whitespace tokens. For more tokens, the class can be easily extended. Use like: next() !== $a::TOK_NULL) { echo $a->token, ': ',$a->getTokenString(), "
\n"; } ?> Показать файл Открыть проект

Открытые свойства

Свойство	Тип	Описание
$char_map	array	Automatically built character map. Built using {@link $identifiers}, {@link $whitespace} and {@link $custom_char_map}
$custom_char_map	array	All characters that should be mapped to a token/function that cannot be considered as whitespace or identifier
$doc	string	The document that is being tokenized
$errors	array	All errors found while parsing the document
$identifiers	array | string	List with all the character that can be considered as identifier
$line_pos		Current (Line/Column) position in document
$pos	integer	Current (character) position in the document
$size	integer	The size of the document (length of string)
$token	integer	Current token
$token_start	integer	Start position of token. If NULL, then current position is used.
$whitespace	array | string	List with all the character that can be considered as whitespace

Открытые методы

Метод	Описание
__construct ( string $doc = '', integer $pos )	Class constructor
addError ( string $error )	Add error to the array and appends current position
getDoc ( ) : string	Returns target document
getIdentifiers ( boolean $as_string = true ) : string | array	Returns identifier characters as string/array
getLinePos ( ) : array	Returns current position in document (Line/Char)
getPos ( ) : integer	Returns current position in document (Index)
getToken ( ) : integer	Returns current token
getTokenString ( integer $start_offset, integer $end_offset ) : string	Returns current token as string
getWhitespace ( boolean $as_string = true ) : string | array	Returns whitespace characters as string/array
mapChar ( string $char, integer | string $map )	Maps a custom character to a token/function
next ( ) : integer	Continues to the next token
next_no_whitespace ( ) : integer	Finds the next token, but skips whitespace
next_pos ( string $needle, boolean $callback = true ) : integer	Finds the next token by searching for a string
next_search ( string | array $characters, boolean $callback = true ) : integer	Finds the next token using stop characters.
setDoc ( string $doc, integer $pos )	Sets target document
setIdentifiers ( string | array $ident )	Sets characters to be recognized as identifier
setPos ( integer $pos )	Sets position in document
setWhitespace ( string | array $ws )	Sets characters to be recognized as whitespace
unmapChar ( string $char )	Removes a char mapped with {@link mapChar()}

Защищенные методы

Метод	Описание
buildCharMap ( )	Builds the {@link $map_char} array
expect ( string | integer $token, boolean | integer $do_next = true, boolean | integer $try_next = false, boolean | integer $next_on_match = 1 ) : boolean	Expect a specific token or character. Adds error if token doesn't match.
parse_identifier ( ) : integer	Parse identifiers
parse_linebreak ( )	Parse line breaks and increase line number
parse_whitespace ( ) : integer	Parse whitespace

Описание методов

Описание свойств