TST/CLN: legacy pickle testing

[jreback]

CLN:

• standardized testing data to live under tests/data (for that set of tests),
• added method util/testing.get_data_path to give you this path

TST:

• added io/tests/test_pickle.py and io/tests/generate_legacy_pickles.py
  to generate legacy pickle files and provide comparsions
  just run python path_to_generate_legacy_pickles.py

  it will create a version of pickle for that python version, arch, and system, and store in pandas/io/tests/data/legacy_pickle/VERSION

• sample pickles for windows-64/2.7/0.10.1 and linux/2.7/0.10.1 and linux/2.7/0.11.0 added

• need to run this on various combinations to generate the reference library

• allows backwards compatibility testing for these pickles

[jreback]

anybody know if py3 is supposed to be able to read py2 pickles?
(this is a pickle I created in py2.7 reading in 3.3)

Do we have to unicode any strings in the data first?

======================================================================
ERROR: test_read_pickles_0_10_1 (pandas.io.tests.test_pickle.TestPickle)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/virtualenv/python3.3_with_system_site_packages/lib/python3.3/site-packages/pandas-0.11.0.dev_a8cdf20-py3.3-linux-x86_64.egg/pandas/io/tests/test_pickle.py", line 42, in test_read_pickles_0_10_1
    self.compare(vf)
  File "/home/travis/virtualenv/python3.3_with_system_site_packages/lib/python3.3/site-packages/pandas-0.11.0.dev_a8cdf20-py3.3-linux-x86_64.egg/pandas/io/tests/test_pickle.py", line 26, in compare
    data = pickle.load(fh)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xf0 in position 14: ordinal not in range(128)

[jreback]

Turns out this does work, but you have to jump thru some hoops, so I updated the main pickle load method
Apparently numpy saves the dtype strings as strings and not unicode (I guess for compat)

from core/common

def load(path):
    try:
        with open(path,'rb') as fh:
            return pickle.load(fh)
    except:
        with open(path,'rb') as fh:
            return pickle.load(fh, encoding='latin1')

[ghost]

does the other way round work?

[jreback]

no, can't read a py3 pickle in py2 (its protocol is 3, which py2 doesn't know about), raises a ValueError (which I am now catching in the comparators). I added a py3 pickle as well. Cross platform is fine as well.

[wesm]

Merged. thanks!