created compiletest.md #53

U007D · 2018-02-13T13:53:51Z

describe the steps required to add a test and a header command to compiletest

mark-i-m

Thanks! I think this chapter will nicely complement #47! (I think it might be a good idea to wait for #47 to merge so that we can update links between the chapters more easily)

I left a few comments below.

Also, could you update SUMMARY.md, so this appears in the ToC?

mark-i-m · 2018-02-13T16:38:38Z

src/compiletest.md

+The tests themselves are typically (but not always) organized into "suites"--for example, `run-pass`, a folder
+representing tests that should succeed, `run-fail`, a folder holding tests that should compile successfully, but return
+a failure (non-zero status), `compile-fail`, a folder holding tests that should fail to compile, and many more.  The various
+suites are defined in `src/tools/compiletest/src/common.rs` in the `pub struct Config` declaration.  And a very good 


Could we make this a link to GitHub? That way, the CI will check if the link breaks. (And the same for other references to specific files/folders in the repo)

mark-i-m · 2018-02-13T16:39:22Z

src/compiletest.md

+a failure (non-zero status), `compile-fail`, a folder holding tests that should fail to compile, and many more.  The various
+suites are defined in `src/tools/compiletest/src/common.rs` in the `pub struct Config` declaration.  And a very good 
+introduction to the different suites of compiler tests along with details about them can be found
+[here, at Brian Anderson's blog](https://brson.github.io/2017/07/10/how-rust-is-tested#s-ct).


I think we should also link to the chapter that will be added in #47

Do you want a link in the doc to the PR or to wait until it lands and can be linked to under master?

to wait until it lands and can be linked to under master

I think this would be best, unless you are under a time constraint.

mark-i-m · 2018-02-13T16:40:30Z

src/compiletest.md

+commands.  These commands can instruct `compiletest` to ignore this test, set expectations on whether it is expected to 
+succeed at compiling, or what the test's return code is expected to be.  Header commands (and their inline counterparts,
+Error Info commands) are described more fully 
+[here](https://github.com/rust-lang/rust/blob/master/src/test/COMPILER_TESTS.md).


I think this can be a link to the chapter from #47

Do you want a link in the doc to the PR or to wait until it lands and can be linked to under master?

to wait until it lands and can be linked to under master

I think this would be best, unless you are under a time constraint.

I think you missed this one

mark-i-m · 2018-02-13T16:43:21Z

src/compiletest.md

+property to its default value.
+
+#### Adding a new header command parser
+When `compiletest` encounters a test file, it parses the file a line at a time by calling every parsers defined in the


"every parsers" -> "every parser"

mark-i-m · 2018-02-13T16:45:38Z

src/compiletest.md

+Parsers will override a given header command property's default value merely by being specified in the test file as a header
+command or by having a parameter value specified in the test file, depending on the header command.
+
+Parsers defined in `impl Config` are typically named `parse_<header_command>` (note kebab-case `<header-command>` transformed


lol! I have never heard it called kebab-case before 😂

mark-i-m · 2018-02-13T16:47:18Z

src/compiletest.md

+to snake-case `<header_command>`).  `impl Config` also defines several 'low-level' parsers which make it simple to parse
+common patterns like simple presence or not (`parse_name_directive()`), header-command:parameter(s)
+(`parse_name_value_directive()`), optional parsing only if a particular `cfg` attribute is defined (`has_cfg_prefix()`) and
+many more.  The low-level parsers are found near the end of the `impl Config` block--be sure to look through them and their


Nit: everywhere else, we have been using - instead of --.

I just looked up em-dash vs en-dash to check for correct usage; it seems correct with two hyphens, but I've changed it to a ; to address your concern.

mark-i-m · 2018-02-13T16:48:24Z

src/compiletest.md

+
+As a concrete example, here is the implementation for the `parse_failure_status()` parser, in
+`src/tools/compiletest/src/header.rs`:
+```diff


This is kind of a cool use for diff formatting 👍

Yes, it's nice, eh?

mark-i-m · 2018-02-13T16:49:37Z

src/compiletest.md

+is that `compiletest` expects the failure code defined by the header command invoked in the test, rather than the default
+value.
+
+Although specific to `failure-status` (as every header ommand will have a different implementation in order to invoke


"ommand" -> "command"

mark-i-m · 2018-02-13T16:50:16Z

src/compiletest.md

+value.
+
+Although specific to `failure-status` (as every header ommand will have a different implementation in order to invoke
+behavior change) perhaps it is helpful to see the behavior hange implementation of one case, simply as an example.  To


"hange" -> "change"

fixed. weird; why am I leaving out so many first letters? :)

And they are both "c"...

changed file references to GitHub links directly to files

U007D · 2018-02-13T17:55:18Z

All feedback addressed (except link to PR#47, pending your reply).

mark-i-m · 2018-02-13T18:13:23Z

LGTM 👍

I think we should wait for #47 unless you are under time constraints because that will make it easier to link to the other chapter.

Thanks!

U007D · 2018-02-13T18:15:20Z

Agreed. All cool by me! And thanks for the proof-read!

mark-i-m · 2018-02-17T16:55:18Z

@U007D #47 has landed 🎉

A few comments

Perhaps we can move this chapter to be a subchapter in the SUMMARY?
Perhaps we can add links to this chapter in https://github.com/rust-lang-nursery/rustc-guide/blob/master/src/tests/adding.md and https://github.com/rust-lang-nursery/rustc-guide/blob/master/src/tests/intro.md
Could you update this chapter to link to those chapters?

Thanks!

U007D · 2018-02-18T09:06:14Z

Great, glad to hear it! Re: edits: Sure. I'll see what I can do to make the edits over the next few days, and will let you know once they're done.

…

-------- Original Message --------

On February 17, 2018 8:55 AM, Who? Me?! ***@***.***> wrote: ***@***.***(https://github.com/u007d) [#47](#47) has landed 🎉 A few comments - Perhaps we can move this chapter to be a subchapter in the SUMMARY? - Perhaps we can add links to this chapter in https://github.com/rust-lang-nursery/rustc-guide/blob/master/src/tests/adding.md and https://github.com/rust-lang-nursery/rustc-guide/blob/master/src/tests/intro.md - Could you update this chapter to link to those chapters? Thanks! — You are receiving this because you were mentioned. Reply to this email directly, [view it on GitHub](#53 (comment)), or [mute the thread](https://github.com/notifications/unsubscribe-auth/ACvebZE5zMNMwKNVYnkSgE0nHyvDwWG8ks5tVwR4gaJpZM4SDyMw).

…itional information

U007D · 2018-02-20T23:42:29Z

#61

mark-i-m

LGTM! Thanks :)

I left three minor comments, then r=me!

mark-i-m · 2018-02-21T03:57:39Z

src/compiletest.md

+a failure (non-zero status), `compile-fail`, a folder holding tests that should fail to compile, and many more.  The various
+suites are defined in [src/tools/compiletest/src/common.rs](https://github.com/rust-lang/rust/tree/master/src/tools/compiletest/src/common.rs) in the `pub struct Config` declaration.  And a very good 
+introduction to the different suites of compiler tests along with details about them can be found
+[here, at Brian Anderson's blog](https://brson.github.io/2017/07/10/how-rust-is-tested#s-ct).


I think we should also link to the "Adding tests" chapter here.

Seems a bit weird to me to provide the same link that we provide in the next paragraph... Plus, we're not even talking about adding tests in this paragraph.

LMK if you'd still like to link to Adding tests here, and if you'd like it alongside or instead of the link to @brson's blog.

As I think about it more, I suppose it's also weird to provide two different authoritative sources on a topic... ;) I think I'll just replace the brson link. LMK if you have other thoughts...

I think there sounds reasonable. There is already a link to brsons post in the other chapter, so let's just point to the other chapter everywhere.

mark-i-m · 2018-02-21T04:00:21Z

src/compiletest.md

+commands.  These commands can instruct `compiletest` to ignore this test, set expectations on whether it is expected to 
+succeed at compiling, or what the test's return code is expected to be.  Header commands (and their inline counterparts,
+Error Info commands) are described more fully 
+[here](https://github.com/rust-lang/rust/blob/master/src/test/COMPILER_TESTS.md).


I think you missed this one

mark-i-m · 2018-02-21T04:08:09Z

src/compiletest.md

+value.
+
+Although specific to `failure-status` (as every header command will have a different implementation in order to invoke
+behavior change) perhaps it is helpful to see the behavior hange implementation of one case, simply as an example.  To implement `failure-status`, the `check_correct_failure_status()` function found in the `TestCx` implementation block,


s/hange/change/

Wow. Something was definitely wrong with my 'c' finger that day! :)

U007D · 2018-02-21T15:45:21Z

Also improved some language under Adding a new header command.

mark-i-m · 2018-02-21T19:08:41Z

Thanks!

created compiletest.md

f79e0d0

describe the steps required to add a test and a header command to compiletest

U007D mentioned this pull request Feb 13, 2018

Relax termination_trait's error bound rust-lang/rust#47544

Merged

mark-i-m mentioned this pull request Feb 13, 2018

"Using the compiler testing framework" #10

Closed

8 tasks

mark-i-m suggested changes Feb 13, 2018

View reviewed changes

Xandkeeper added 3 commits February 13, 2018 09:22

Update compiletest.md

59485d2

changed file references to GitHub links directly to files

fixed typos

5723bce

Added 'How to add header commands to compiletest'

fc83b6f

mark-i-m added the S-blocked Status: this PR is blocked waiting for something label Feb 15, 2018

mark-i-m added S-waiting-on-author Status: this PR is waiting for additional action by the OP and removed S-blocked Status: this PR is blocked waiting for something labels Feb 17, 2018

Xandkeeper added 2 commits February 20, 2018 15:32

added compiletest to summary.md; linked to existing resources for add…

84c2cd5

…itional information

merged with upstream/master

8751dfe

U007D mentioned this pull request Feb 20, 2018

implemented changes... #61

Closed

mark-i-m suggested changes Feb 21, 2018

View reviewed changes

cleaned up some stragglers

572eaa6

changed to relative links, where possible

b1c4662

mark-i-m approved these changes Feb 21, 2018

View reviewed changes

mark-i-m merged commit 238b190 into rust-lang:master Feb 21, 2018

U007D deleted the compiletest branch February 21, 2018 19:13

created compiletest.md #53

created compiletest.md #53

Conversation

U007D commented Feb 13, 2018

mark-i-m left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

U007D Feb 13, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

U007D Feb 13, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

U007D commented Feb 13, 2018 • edited Loading

mark-i-m commented Feb 13, 2018

U007D commented Feb 13, 2018

mark-i-m commented Feb 17, 2018

U007D commented Feb 18, 2018 via email

U007D commented Feb 20, 2018

mark-i-m left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

U007D commented Feb 21, 2018 • edited Loading

mark-i-m commented Feb 21, 2018

U007D Feb 13, 2018 •

edited

Loading

U007D Feb 13, 2018 •

edited

Loading

U007D commented Feb 13, 2018 •

edited

Loading

U007D commented Feb 21, 2018 •

edited

Loading