Skip to content

Commit 2b587b0

Browse files
chrissimpkinsmark-i-m
authored andcommitted
[overview.md] add initial parser documentation
1 parent 74f5f88 commit 2b587b0

File tree

1 file changed

+27
-2
lines changed

1 file changed

+27
-2
lines changed

src/overview.md

+27-2
Original file line numberDiff line numberDiff line change
@@ -34,8 +34,33 @@ we'll talk about that later.
3434
[`librustc_parse`] to prepare for the next stage of the compile process. The
3535
[`StringReader`] struct is used at this stage to perform a set of validations
3636
and turn strings into interned symbols (_interning_ is discussed later).
37-
- (**TODO**: chrissimpkins - Expand info on parser) We then [_parse_ the stream
38-
of tokens][parser] to build an Abstract Syntax Tree (AST).
37+
- The lexer has a small interface and doesn't depend directly on the
38+
diagnostic infrastructure in `rustc`. Instead it provides diagnostics as plain
39+
data which are emitted in `librustc_parse::lexer::mod` as real diagnostics.
40+
- The lexer preseves full fidelity information for both IDEs and proc macros.
41+
- The parser [translates the token stream from the lexer into an Abstract Syntax
42+
Tree (AST)][parser]. It uses a recursive descent (top-down) approach to syntax
43+
analysis. The crate entry points for the parser are the `Parser.parse_crate_mod()` and
44+
`Parser.parse_mod()` methods found in `librustc_parse::parser::item`. The external
45+
module parsing entry point is `librustc_expand::module::parse_external_mod`. And
46+
the macro parser entry point is `rustc_expand::mbe::macro_parser::parse_nt`.
47+
- Parsing is performed with a set of `Parser` utility methods including `fn bump`,
48+
`fn check`, `fn eat`, `fn expect`, `fn look_ahead`.
49+
- Parsing is organized by the semantic construct that is being parsed. Separate
50+
`parse_*` methods can be found in `librustc_parse` `parser` directory. File
51+
naming follows the construct name. For example, the following files are found
52+
in the parser:
53+
- `expr.rs`
54+
- `pat.rs`
55+
- `ty.rs`
56+
- `stmt.rs`
57+
- This naming scheme is used across the parser, lowering, type checking,
58+
HAIR lowering, & MIR building stages of the compile process and you will
59+
find either a file or directory with the same name for most of these constructs
60+
at each of these stages of compilation.
61+
- For error handling, the parser uses the standard `DiagnosticBuilder` API, but we
62+
try to recover, parsing a superset of Rust's grammar, while also emitting an error.
63+
- The `rustc_ast::ast::{Crate, Mod, Expr, Pat, ...}` AST node returned from the parser.
3964
- macro expansion (**TODO** chrissimpkins)
4065
- ast validation (**TODO** chrissimpkins)
4166
- nameres (**TODO** chrissimpkins)

0 commit comments

Comments
 (0)