Skip to content

Commit d023ac7

Browse files
chrissimpkinsmark-i-m
authored andcommitted
[overview.md] add lexer updates, parser updates
includes feedback from matklad (lexer) and centril (parser)
1 parent 2b587b0 commit d023ac7

File tree

1 file changed

+11
-15
lines changed

1 file changed

+11
-15
lines changed

src/overview.md

+11-15
Original file line numberDiff line numberDiff line change
@@ -28,8 +28,8 @@ we'll talk about that later.
2828
to the rest of the compilation process as a [`rustc_interface::Config`].
2929
- The raw Rust source text is analyzed by a low-level lexer located in
3030
[`librustc_lexer`]. At this stage, the source text is turned into a stream of
31-
atomic source code units known as _tokens_. The lexer supports the Unicode
32-
character encoding.
31+
atomic source code units known as _tokens_. The lexer supports the
32+
Unicode character encoding.
3333
- The token stream passes through a higher-level lexer located in
3434
[`librustc_parse`] to prepare for the next stage of the compile process. The
3535
[`StringReader`] struct is used at this stage to perform a set of validations
@@ -47,25 +47,21 @@ we'll talk about that later.
4747
- Parsing is performed with a set of `Parser` utility methods including `fn bump`,
4848
`fn check`, `fn eat`, `fn expect`, `fn look_ahead`.
4949
- Parsing is organized by the semantic construct that is being parsed. Separate
50-
`parse_*` methods can be found in `librustc_parse` `parser` directory. File
51-
naming follows the construct name. For example, the following files are found
50+
`parse_*` methods can be found in `librustc_parse` `parser` directory. The source
51+
file name follows the construct name. For example, the following files are found
5252
in the parser:
5353
- `expr.rs`
5454
- `pat.rs`
5555
- `ty.rs`
5656
- `stmt.rs`
57-
- This naming scheme is used across the parser, lowering, type checking,
58-
HAIR lowering, & MIR building stages of the compile process and you will
59-
find either a file or directory with the same name for most of these constructs
60-
at each of these stages of compilation.
61-
- For error handling, the parser uses the standard `DiagnosticBuilder` API, but we
57+
- This naming scheme is used across many compiler stages. You will find
58+
either a file or directory with the same name across the parsing, lowering,
59+
type checking, HAIR lowering, and MIR building sources.
60+
- Macro expansion, AST validation, name resolution, and early linting takes place
61+
during this stage of the compile process.
62+
- The parser uses the standard `DiagnosticBuilder` API for error handling, but we
6263
try to recover, parsing a superset of Rust's grammar, while also emitting an error.
63-
- The `rustc_ast::ast::{Crate, Mod, Expr, Pat, ...}` AST node returned from the parser.
64-
- macro expansion (**TODO** chrissimpkins)
65-
- ast validation (**TODO** chrissimpkins)
66-
- nameres (**TODO** chrissimpkins)
67-
- early linting (**TODO** chrissimpkins)
68-
64+
- `rustc_ast::ast::{Crate, Mod, Expr, Pat, ...}` AST nodes are returned from the parser.
6965
- We then take the AST and [convert it to High-Level Intermediate
7066
Representation (HIR)][hir]. This is a compiler-friendly representation of the
7167
AST. This involves a lot of desugaring of things like loops and `async fn`.

0 commit comments

Comments
 (0)