Delete the documents related to Kokoro

Author: amin1377

README.developers.md

@@ -325,139 +325,6 @@ VTR also has a limited set of unit tests, which can be run with:
 $ make && make test
 ```
 
-## Running tests on Pull Requests (PRs) via Kokoro
-
-Because of the long runtime for nightly and weekly tests, a Kokoro job can be
-used to run these tests once a Pull Request (PR) has been made at
-https://github.com/verilog-to-routing/vtr-verilog-to-routing.
-
-Any pull request made by a contributor of the verilog-to-routing GitHub project
-on https://github.com/verilog-to-routing/ will get a set of jobs immediately.
-Non-contributors can request a contributor on the project add a label
-"kokoro:force-run" to the PR. Kokoro will then detect the tag, remove the tag,
-and then and issue jobs for that PR.  If the tag remains after being added,
-there may not be an available Kokoro runner, so wait.
-
-### Re-running tests on Kokoro
-
-If a job fails due to an intermittent failure or a re-run is desired, a
-contributor can add the label "kokoro:force-run" to re-issue jobs for that PR.
-
-### Checking results from Kokoro tests
-
-Currently there is not a way for an in-flight job to be monitored.
-
-Once a job has been completed, you can follow the "Details" link that appears on the PR status. 
-The Kokoro page will show the job's stdout in the 'Target Log' tab (once the job has completed).
-The full log can be downloading by clicking the 'Download Full Log' button, or from the 'Artifacts' tab.
-
-### Downloading logs from Google Cloud Storage (GCS)
-
-After a Kokoro run is complete a number of useful log files (e.g. for each VPR invocation) are stored to Google Cloud Storage (GCS).
-
-The top level directory containing all VTR Kokoro runs is:
-
-    https://console.cloud.google.com/storage/browser/vtr-verilog-to-routing/artifacts/prod/foss-fpga-tools/verilog-to-routing/upstream/
-
-PR jobs are under the `presubmit` directory, and continuous jobs (which run on the master branch) are under the `continuous` directory.
-
-Each Kokoro run has a unique build number, which can be found in the log file (available via the Kokoro run webpage).
-For example, if the log file contains:
-```
-export KOKORO_BUILD_NUMBER="450"
-```
-then the Kokoro build number is `450`.
-
-If build 450 corresponded to a PR (`presubmit`) build of the `nightly` regression tests, the resulting output files would be available at:
-
-    https://console.cloud.google.com/storage/browser/vtr-verilog-to-routing/artifacts/prod/foss-fpga-tools/verilog-to-routing/upstream/presubmit/nightly/450/
-
-where `presubmit/nightly/450/` (the type, test name and build number) have been appended to the base URL.
-Navigating to that URL will allow you to browse and download the collected log files.
-
-To download all the files from that Kokoro run, replace `https://console.cloud.google.com/storage/browser/` in the URL with `gs://` and invoke the [gsutil](https://cloud.google.com/storage/docs/gsutil) command (and it's `cp -R` sub-command), like so:
-
-```
-gsutil -m cp -R gs://vtr-verilog-to-routing/artifacts/prod/foss-fpga-tools/verilog-to-routing/upstream/presubmit/nightly/450 .
-```
-
-This will download all of the logs to the current directory for inspection.
-
-#### Kokoro runner details
-
-Kokoro runners are a standard
-[`n1-highmem-16`](https://cloud.google.com/compute/docs/machine-types#n1_high-memory_machine_types)
-VM with a 4 TB `pd-standard` disk used to perform the build of VPR and run the
-tests.
-
-#### What to do if Kokoro jobs are not starting?
-
-There are several reasons Kokoro jobs might not be starting.
-Try adding the "kokoro:force-run" label if it is not already added, or remove
-and add it if it already was added.
-
-If adding the label has no affect, check GCS status, as a GCS disruption will
-also disrupt Kokoro.
-
-Another reason jobs may not start is if there is a large backlog of jobs
-running, there may be no runners left to start.  In this case, someone with
-Kokoro management rights may need to terminate stale jobs, or wait for job
-timeouts.
-
-# Debugging Failed Tests
-
-If a test fails you probably want to look at the log files to determine the cause.
-
-Lets assume we have a failure in `vtr_reg_basic`:
-
-```shell
-#In the VTR root directory
-$ ./run_reg_test.py vtr_reg_strong
-#Output trimmed...
-regression_tests/vtr_reg_basic/basic_no_timing
------------------------------------------
-k4_N10_memSize16384_memData64/ch_intrinsics/common   failed: vpr
-k4_N10_memSize16384_memData64/diffeq1/common         failed: vpr
-#Output trimmed...
-regression_tests/vtr_reg_basic/basic_no_timing...[Fail]
- k4_N10_memSize16384_memData64.xml/ch_intrinsics.v vpr_status: golden = success result = exited
-#Output trimmed...
-Error: 10 tests failed!
-```
-
-Here we can see that `vpr` failed, which caused subsequent QoR failures (`[Fail]`), and resulted in 10 total errors.
-
-To see the log files we need to find the run directory.
-We can see from the output that  the specific test which failed was `regression_tests/vtr_reg_basic/basic_no_timing`.
-All the regression tests take place under `vtr_flow/tasks`, so the test directory is `vtr_flow/tasks/regression_tests/vtr_reg_basic/basic_no_timing`.
-Lets move to that directory:
-```shell
-#From the VTR root directory
-$ cd vtr_flow/tasks/regression_tests/vtr_reg_basic/basic_no_timing
-$ ls
-config  run001  run003
-latest  run002  run004  run005
-```
-
-There we see there is a `config` directory (which defines the test), and a set of run-directories.
-Each time a test is run it creates a new `runXXX` directory (where `XXX` is an incrementing number).
-From the above we can tell that our last run was `run005` (the symbolic link `latest` also points to the most recent run directory).
-From the output of `run_reg_test.py` we know that one of the failing architecture/circuit/parameters combinations was `k4_N10_memSize16384_memData64/ch_intrinsics/common`.
-Each architecture/circuit/parameter combination is run in its own sub-folder.
-Lets move to that directory:
-```shell
-$ cd run005/k4_N10_memSize16384_memData64/ch_intrinsics/common
-$ ls
-abc.out                     k4_N10_memSize16384_memData64.xml  qor_results.txt
-ch_intrinsics.net           odin.out                           thread_1.out
-ch_intrinsics.place         output.log                         vpr.out
-ch_intrinsics.pre-vpr.blif  output.txt                         vpr_stdout.log
-ch_intrinsics.route         parse_results.txt
-```
-
-Here we can see the individual log files produced by each tool (e.g. `vpr.out`), which we can use to guide our debugging.
-We could also manually re-run the tools (e.g. with a debugger) using files in this directory.
-
 # Evaluating Quality of Result (QoR) Changes
 VTR uses highly tuned and optimized algorithms and data structures.
 Changes which effect these can have significant impacts on the quality of VTR's design implementations (timing, area etc.) and VTR's run-time/memory usage.