(Engine-Level Function)
This function works with streams. If you are interested in Control Tokens, refer to GetToken (Control Lock Module)
| Description: | Reads the next token from a stream and returns the token type. |
| Warning: | This function should be used only by advanced programmers. |
| Returns: | Numeric (see table in comments) |
Usage:  |
Script Only. |
| Function Groups: | Compilation and On-Line Modifications, Stream and Socket |
| Related to: | Compile |
| Format: |
GetToken(Stream, Token, ClassBuffer, NumClasses, StateBuffer, ActionBuffer, SkipWhite, LineCount, Column, CharCount) |
| Parameters: |
| Stream |
| Required. Any stream expression for the input stream. |
| Token |
| Required. A variable into which the next token read will be stored. |
| ClassBuffer |
| Required. Any text expression for the character classifier table. |
| NumClasses |
| Required. Any numeric expression for the number of character classes. |
| StateBuffer |
| Required. Any text expression for the tokenizer state table. |
| ActionBuffer |
| Required. Any text expression for the tokenizer action table. |
| SkipWhite |
| Required. Any logical expression. If true, all white spaced is skipped. Otherwise all continuous white space is treated as a token. |
| LineCount |
| Required. Must be a variable. The number of lines read is stored here. |
| Column |
| Required. Must be a variable. The current column is stored here. |
| CharCount |
| Required. Must be a variable. The number of characters read is stored here. |
| Comments: | The return value for this function is the token type. The tokens recognized by the Compile function are as follows (assuming the tables used by the compiler are those handed in to this function)
|
