Module token

Expand description

Tokens representing Rust punctuation, keywords, and delimiters.

The type names in this module can be difficult to keep straight, so we prefer to use the Token! macro instead. This is a type-macro that expands to the token type of the given token.

§Example

The ItemStatic syntax tree node is defined like this.

pub struct ItemStatic {
    pub attrs: Vec<Attribute>,
    pub vis: Visibility,
    pub static_token: Token![static],
    pub mutability: Option<Token![mut]>,
    pub ident: Ident,
    pub colon_token: Token![:],
    pub ty: Box<Type>,
    pub eq_token: Token![=],
    pub expr: Box<Expr>,
    pub semi_token: Token![;],
}

§Parsing

Keywords and punctuation can be parsed through the ParseStream::parse method. Delimiter tokens are parsed using the parenthesized!, bracketed! and braced! macros.

use syn::{Attribute, Result};
use syn::parse::{Parse, ParseStream};

// Parse the ItemStatic struct shown above.
impl Parse for ItemStatic {
    fn parse(input: ParseStream) -> Result<Self> {
        Ok(ItemStatic {
            attrs: input.call(Attribute::parse_outer)?,
            vis: input.parse()?,
            static_token: input.parse()?,
            mutability: input.parse()?,
            ident: input.parse()?,
            colon_token: input.parse()?,
            ty: input.parse()?,
            eq_token: input.parse()?,
            expr: input.parse()?,
            semi_token: input.parse()?,
        })
    }
}

§Other operations

Every keyword and punctuation token supports the following operations.

Peeking — input.peek(Token![...])
Parsing — input.parse::<Token![...]>()?
Printing — quote!( ... #the_token ... )
Construction from a Span — let the_token = Token![...](sp)
Field access to its span — let sp = the_token.span

Structs§

Abstract: abstract
And: &
AndAnd: &&
AndEq: &=
As: as
Async: async
At: @
Auto: auto
Await: await
Become: become
Box: box
Brace: {…}
Bracket: […]
Break: break
Caret: ^
CaretEq: ^=
Colon: :
Comma: ,
Const: const
Continue: continue
Crate: crate
Default: default
Do: do
Dollar: $
Dot: .
DotDot: ..
DotDotDot: ...
DotDotEq: ..=
Dyn: dyn
Else: else
Enum: enum
Eq: =
EqEq: ==
Extern: extern
FatArrow: =>
Final: final
Fn: fn
For: for
Ge: >=
Group: None-delimited group
Gt: >
If: if
Impl: impl
In: in
LArrow: <-
Le: <=
Let: let
Loop: loop
Lt: <
Macro: macro
Match: match
Minus: -
MinusEq: -=
Mod: mod
Move: move
Mut: mut
Ne: !=
Not: !
Or: |
OrEq: |=
OrOr: ||
Override: override
Paren: (…)
PathSep: ::
Percent: %
PercentEq: %=
Plus: +
PlusEq: +=
Pound: #
Priv: priv
Pub: pub
Question: ?
RArrow: ->
Raw: raw
Ref: ref
Return: return
SelfType: Self
SelfValue: self
Semi: ;
Shl: <<
ShlEq: <<=
Shr: >>
ShrEq: >>=
Slash: /
SlashEq: /=
Star: *
StarEq: *=
Static: static
Struct: struct
Super: super
Tilde: ~
Trait: trait
Try: try
Type: type
Typeof: typeof
Underscore: _
Union: union
Unsafe: unsafe
Unsized: unsized
Use: use
Virtual: virtual
Where: where
While: while
Yield: yield

Traits§

Token: Marker trait for types that represent single tokens.