VPR: Different QoR With The Same Seed

[amin1377]

I ran Titan benchmark multiple times with the same seed on the same machine. The results for some of the benchmarks differed from one run to the other.

Expected Behaviour

I expected QoR not to change when using the same seed on the same machine.

Current Behaviour

As shown in results, for two different runs, the CPDs are different.

I ran diff on *.net" files, and it shows that the packer output has not changed. Also, I ran the router with the same placement and packing, and the results were similar. When I ran diff on placement files, I realized the difference in the results is because of placement.

It is also important to mention that in Titan benchmark suit, blif files are given to the VPR. Thus, the inconsistency can not be based on other stages in VTR Flow.

Steps to Reproduce

I ran Titan benchmark 2-3 times with the same seed and machine, and CPDs were not similar.

@vaughnbetz

[amin1377]

After fixing this issue, I ran "titan_quick_qor" several times and got the same results. It seems that the problem is solved. I suggest that we add some CI tests to run the VPR flow multiple times each day, or week, to make sure that VPR's results remain deterministic.

@vaughnbetz