=tck general tck/readme.md cleanup so it matches current code / spec

Resolves reactive-streams#99
Depends on reactive-streams#241

Author: ktoso

tck/README.md

@@ -13,8 +13,8 @@ not possible (or viable) to construct automated tests, thus the TCK does not cla
 The TCK is split up into 4 files JUnit 4 test classes which should be extended by implementers, providing their `Publisher` / `Subscriber` implementations for the test harness to validate them. The tests are split in the following way:
 
 * `PublisherVerification`
-* `SubscriberBlackboxVerification`
 * `SubscriberWhiteboxVerification`
+* `SubscriberBlackboxVerification`
 * `IdentityProcessorVerification`
 
 The next sections include examples on how these can be used and describe the various configuration options.
@@ -38,8 +38,9 @@ Since the TCK is aimed at Reactive Stream implementers, looking into the sources
 and should help during a libraries implementation cycle.
 
 In order to make mapping between test cases and Specification rules easier, each test case covering a specific
-Specification rule abides the following naming convention: `spec###_DESC` where:
+Specification rule abides the following naming convention: `TYPE_spec###_DESC` where:
 
+* `TYPE` is one of: `required`, `optional`, `stochastic` or `untested` which describe if this test is covering a Rule that MUST or SHOULD be implemented. The specific words are explained in detail below.
 * `###` is the Rule number (`1.xx` Rules are about Publishers, `2.xx` Rules are about Subscribers etc.)
 * `DESC` is a short explanation of what exactly is being tested in this test case, as sometimes one Rule may have multiple test cases in order to cover the entire Rule.
 
@@ -66,7 +67,7 @@ Explanations:
 @Test public void optional_spec104_mustSignalOnErrorWhenFails() throws Throwable
 ```
 
-... means that this test case is optional, it covers a *MAY* or *SHOULD* Rule of the Specification. This prefix is also used if more configuration is needed in order to run it, e.g. `@Additional(implement = "createErrorStatePublisher") @Test` signals the implementer that in order to include this test case in his test runs, (s)he must implement the `Publisher<T> createErrorStatePublisher()` method.
+... means that this test case is optional, it covers a *MAY* or *SHOULD* Rule of the Specification. This prefix is also used if more configuration is needed in order to run it, e.g. `@Additional(implement = "createFailedPublisher") @Test` signals the implementer that in order to include this test case in his test runs, (s)he must implement the `Publisher<T> createFailedPublisher()` method.
 
 ```java
 @Test public void stochastic_spec103_mustSignalOnMethodsSequentially() throws Throwable
@@ -142,7 +143,7 @@ public class RangePublisherTest extends PublisherVerification<Integer> {
   }
 
   @Override
-  public Publisher<Integer> createErrorStatePublisher() {
+  public Publisher<Integer> createFailedPublisher() {
     return new Publisher<Integer>() {
       @Override
       public void subscribe(Subscriber<Integer> s) {
@@ -375,7 +376,7 @@ public class MySubscriberTest extends BlackboxSubscriberVerification<Integer> {
   public static final long DEFAULT_TIMEOUT_MILLIS = 300L;
 
   public RangePublisherTest() {
-    super(new MySubscriberTest(DEFAULT_TIMEOUT_MILLIS));
+    super(new TestEnvironment(DEFAULT_TIMEOUT_MILLIS));
   }
 
   // ...
@@ -427,12 +428,14 @@ public class MyIdentityProcessorVerificationTest extends IdentityProcessorVerifi
   // ENABLE ADDITIONAL TESTS
 
   @Override
-  public Publisher<Integer> createErrorStatePublisher() {
-    // return error state Publisher instead of null to run additional tests
+  public Publisher<Integer> createFailedPublisher() {
+    // return Publisher that just signals onError instead of null to run additional tests
+    // see this methods JavaDocs for more details on how the returned Publisher should work.
     return null;
   }
 
   // OPTIONAL CONFIGURATION OVERRIDES
+  // override these only if you understand why you'd need to do so for your impl.
 
   @Override
   public long maxElementsFromPublisher() {
@@ -456,32 +459,65 @@ Since you inherit these tests instead of defining them yourself it's not possibl
 to skip tests inherited from the TCK's base classes:
 
 ```java
-package com.example.streams;
+package com.example;
 
+import org.reactivestreams.Processor;
+import org.reactivestreams.Publisher;
+import org.reactivestreams.Subscriber;
+import org.reactivestreams.Subscription;
 import org.reactivestreams.tck.IdentityProcessorVerification;
+import org.reactivestreams.tck.TestEnvironment;
+import org.testng.annotations.AfterClass;
+import org.testng.annotations.BeforeClass;
+
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+
+public class MyIdentityProcessorTest extends IdentityProcessorVerification<Integer> {
+
+  private ExecutorService e;
 
-public class SkippingIdentityProcessorTest extends IdentityProcessorVerification<Integer> {
+  @BeforeClass
+  public void before() { e = Executors.newFixedThreadPool(4); }
+
+  @AfterClass
+  public void after() { if (e != null) e.shutdown(); }
 
   public SkippingIdentityProcessorTest() {
-    super(new TestEnvironment(500, true), 1000);
+    super(new TestEnvironment());
+  }
+
+  @Override
+  public ExecutorService publisherExecutorService() {
+    return e;
+  }
+
+  @Override
+  public Integer createElement(int element) {
+    return element;
   }
 
   @Override
   public Processor<Integer, Integer> createIdentityProcessor(int bufferSize) {
-    return /* ... */;
+    return new MyProcessor<Integer, Integer>(); // return your implementation
   }
 
-  @Override // override the test method, and provide a reason on why you're doing so in the notVerified() message
-  public void spec999_mustDoVeryCrazyThings() throws Throwable {
-    notVerified("Unable to implement test because ...");
+  @Override
+  public Publisher<Integer> createFailedPublisher() {
+    return null; // skip error publisher tests
   }
 
 }
 ```
 
-## Upgrade story
+## Upgrading the TCK to newer versions
+While we do not expect the Reactive Streams specification to change in the forseeable future,
+it may happen that some semantics may need changing in the future. In this case you should expect test
+methods being phased out in terms of deprecation or removal, new tests may also be added over time.
 
-**TODO** - What is our story about updating the TCK? How do we make sure that implementations don't accidentally miss some change in the spec, if the TCK is unable to fail verify the new behavior? Comments are very welcome, discussion about this is under-way in [Issue #99 – TCK Upgrade Story](https://github.com/reactive-streams/reactive-streams/issues/99).
+In general this should not be of much concern to you, unless you override test methods in your test suite.
+We ask implementers who find the need of overriding provided test methods to reach out via opening tickets
+on the `reactive-streams/reactive-streams-jvm` project, so we can discuss the use case and, most likely, improve the TCK.
 
 ## Using the TCK from other languages
 
@@ -504,7 +540,7 @@ class IterablePublisherTest(env: TestEnvironment, publisherShutdownTimeout: Long
   def createPublisher(elements: Long): Publisher[Int] = ???
 
   // example error state publisher implementation
-  override def createErrorStatePublisher(): Publisher[Int] =
+  override def createFailedPublisher(): Publisher[Int] =
     new Publisher[Int] {
       override def subscribe(s: Subscriber[Int]): Unit = {
         s.onError(new Exception("Unable to serve subscribers right now!"))