test_checks and test_interval have flaky tests

[MarcoGorelli]

Noticed on the first run of #4314

=================================== FAILURES ===================================
________________________________ test_interval _________________________________

    def test_interval():
        for a, b in [(-4, 5.5), (0.1, 0.7), (-10, 4.3)]:
            domain = Unit * np.float64(b - a) + np.float64(a)
            trans = tr.interval(a, b)
>           check_transform(trans, domain)

pymc3/tests/test_transforms.py:190: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pymc3/tests/test_transforms.py:53: in check_transform
    close_to(transform.forward_val(val), forward_f(val), tol)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

x = nan, v = array(4.5951228), bound = 1e-06, name = 'value'

    def close_to(x, v, bound, name="value"):
        assert np.all(np.logical_or(np.abs(x - v) < bound, x == v)), (
>           name + " out of bounds: " + repr(x) + ", " + repr(v) + ", " + repr(bound)
        )
E       AssertionError: value out of bounds: nan, array(4.5951228), 1e-06

pymc3/tests/checks.py:20: AssertionError
------------------------------ Captured log call -------------------------------
ERROR    theano.gof.opt:opt.py:188 SeqOptimizer apply <theano.tensor.opt.FusionOptimizer object at 0x7fb2d46135c0>
ERROR    theano.gof.opt:opt.py:189 Traceback:
ERROR    theano.gof.opt:opt.py:190 Traceback (most recent call last):
  File "/usr/share/miniconda/envs/pymc3-dev-py36/lib/python3.6/site-packages/theano/scalar/basic.py", line 358, in filter
    " (%s) and allow_downcast is not True" % self.dtype
TypeError: Value cannot accurately be converted to dtype (float64) and allow_downcast is not True

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/share/miniconda/envs/pymc3-dev-py36/lib/python3.6/site-packages/theano/gof/opt.py", line 249, in apply
    sub_prof = optimizer.optimize(fgraph)
  File "/usr/share/miniconda/envs/pymc3-dev-py36/lib/python3.6/site-packages/theano/gof/opt.py", line 89, in optimize
    ret = self.apply(fgraph, *args, **kwargs)
  File "/usr/share/miniconda/envs/pymc3-dev-py36/lib/python3.6/site-packages/theano/tensor/opt.py", line 7774, in apply
    new_outputs = self.optimizer(node)
  File "/usr/share/miniconda/envs/pymc3-dev-py36/lib/python3.6/site-packages/theano/tensor/opt.py", line 7596, in local_fuse
    tmp.tag.test_value = tv.flatten()[0]
  File "/usr/share/miniconda/envs/pymc3-dev-py36/lib/python3.6/site-packages/theano/gof/utils.py", line 277, in __setattr__
    obj = self.attr_filter(obj)
  File "/usr/share/miniconda/envs/pymc3-dev-py36/lib/python3.6/site-packages/theano/scalar/basic.py", line 364, in filter
    e,
TypeError: ("Could not convert <class 'numpy.float64'> (value=nan) to float64", TypeError('Value cannot accurately be converted to dtype (float64) and allow_downcast is not True',))

[MarcoGorelli]

these failed again here https://github.com/pymc-devs/pymc3/runs/1543128994

[MarcoGorelli]

and again here https://github.com/pymc-devs/pymc3/runs/1571170775

[ricardoV94]

Is the test_interval fixed by #4418 perhaps?

[ricardoV94]

The test_interval failures (another failure here) seem to be caused by precision issues in the numpy forward_val method, since that's the one that seems to always come back as nan in the runs:

https://github.com/pymc-devs/pymc3/blob/32b5c941c8f43b485afd3da1ae5544c3c50b9e77/pymc3/distributions/transforms.py#L289-L294

This is how a and b are created:

https://github.com/pymc-devs/pymc3/blob/32b5c941c8f43b485afd3da1ae5544c3c50b9e77/pymc3/tests/test_transforms.py#L186-L190

The enforcing of float64 is a bit strange, but I assume numpy does the right thing when on a 32-bit machine?

Somewhere we are losing precision. This can happen either in the a and b which are theano constants inside the trans object or when creating the domain object whose values are simple numpy arrays. However, I have no clue why this test would fail stochastically, unless there is something funny going on with theano.constants or the draw_values function :/ .

Edit: Could it be when a is an integer in trans but not in domain?

[ricardoV94]

The transforms methods have been refactored for V4 and among other things forward_val is no longer a thing. I don't think I've seen these fail since then, so I am closing this one tentatively