File tree Expand file tree Collapse file tree 3 files changed +4
-4
lines changed Expand file tree Collapse file tree 3 files changed +4
-4
lines changed Original file line number Diff line number Diff line change @@ -39,8 +39,8 @@ The primary entry point is a :term:`generator`:
39
39
column where the token begins in the source; a 2-tuple ``(erow, ecol) `` of
40
40
ints specifying the row and column where the token ends in the source; and
41
41
the line on which the token was found. The line passed (the last tuple item)
42
- is the *physical * line; continuation lines are included . The 5 tuple is
43
- returned as a :term: ` named tuple ` with the field names:
42
+ is the *physical * line. The 5 tuple is returned as a :term: ` named tuple `
43
+ with the field names:
44
44
``type string start end line ``.
45
45
46
46
The returned :term: `named tuple ` has an additional property named
Original file line number Diff line number Diff line change @@ -346,7 +346,7 @@ def generate_tokens(readline):
346
346
column where the token begins in the source; a 2-tuple (erow, ecol) of
347
347
ints specifying the row and column where the token ends in the source;
348
348
and the line on which the token was found. The line passed is the
349
- physical line; continuation lines are included .
349
+ physical line.
350
350
"""
351
351
lnum = parenlev = continued = 0
352
352
contstr , needcont = '' , 0
Original file line number Diff line number Diff line change @@ -415,7 +415,7 @@ def tokenize(readline):
415
415
column where the token begins in the source; a 2-tuple (erow, ecol) of
416
416
ints specifying the row and column where the token ends in the source;
417
417
and the line on which the token was found. The line passed is the
418
- physical line; continuation lines are included .
418
+ physical line.
419
419
420
420
The first token sequence will always be an ENCODING token
421
421
which tells you which encoding was used to decode the bytes stream.
You can’t perform that action at this time.
0 commit comments