Skip to content

Fix hard-coded pandas version to infer the current #24509

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

iamaziz
Copy link

@iamaziz iamaziz commented Dec 30, 2018

Fix hard-coded pandas version to infer the current version in pd.io.json.build_table_schema output.

>>> import pandas as pd
>>> pd.__version__
'0.23.4'
>>> df = pd.DataFrame({'A': [1,2], 'B': ['a','b']})
>>> # BEFORE
>>> pd.io.json.build_table_schema(df, index=0)['pandas_version']
'0.20.0'
>>> # AFTER
>>> pd.io.json.build_table_schema(df, index=0)['pandas_version']
'0.23.4'

>>> import pandas as pd
>>> pd.__version__
'0.23.4'
>>> df = pd.DataFrame({'A': [1,2], 'B': ['a','b']})
>>> # BEFORE
>>> pd.io.json.build_table_schema(df, index=0)['pandas_version']
'0.20.0'
>>> # AFTER
>>> pd.io.json.build_table_schema(df, index=0)['pandas_version']
'0.23.4'
@codecov
Copy link

codecov bot commented Dec 31, 2018

Codecov Report

Merging #24509 into master will increase coverage by <.01%.
The diff coverage is 50%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #24509      +/-   ##
==========================================
+ Coverage   31.89%    31.9%   +<.01%     
==========================================
  Files         166      166              
  Lines       52421    52422       +1     
==========================================
+ Hits        16722    16723       +1     
  Misses      35699    35699
Flag Coverage Δ
#multiple 30.29% <50%> (ø) ⬆️
#single 31.9% <50%> (ø) ⬆️
Impacted Files Coverage Δ
pandas/io/json/table_schema.py 11.86% <50%> (+0.75%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 100ffff...fd02c31. Read the comment docs.

1 similar comment
@codecov
Copy link

codecov bot commented Dec 31, 2018

Codecov Report

Merging #24509 into master will increase coverage by <.01%.
The diff coverage is 50%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #24509      +/-   ##
==========================================
+ Coverage   31.89%    31.9%   +<.01%     
==========================================
  Files         166      166              
  Lines       52421    52422       +1     
==========================================
+ Hits        16722    16723       +1     
  Misses      35699    35699
Flag Coverage Δ
#multiple 30.29% <50%> (ø) ⬆️
#single 31.9% <50%> (ø) ⬆️
Impacted Files Coverage Δ
pandas/io/json/table_schema.py 11.86% <50%> (+0.75%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 100ffff...fd02c31. Read the comment docs.

@@ -223,7 +224,7 @@ def build_table_schema(data, index=True, primary_key=None, version=True):
{'name': 'A', 'type': 'integer'},
{'name': 'B', 'type': 'string'},
{'name': 'C', 'type': 'datetime'}],
'pandas_version': '0.20.0',
'pandas_version': '0.23.4',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should also be dynamic if we change the other one.

You can use the Substitution decorator to do it.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be a good idea to use f-string instead of Substitution?

Copy link
Author

@iamaziz iamaziz Dec 31, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or if you know of a reference for Substitution decorator, I'm not familar with it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can't use f-strings, as we support python 2.7 and 3.5. I'd use Substitution for consistency with the rest of docstrings (and I'm not sure if you can call .format() on a docstring)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just do a grep in the code, it's a decorator implemented in pandas itself

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.format() is actually possible in docstring. Python 2.7 is gonna die soon :)

Copy link
Author

@iamaziz iamaziz Dec 31, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, not..format() does not seem to work with docstring

@datapythonista datapythonista added the IO JSON read_json, to_json, json_normalize label Dec 31, 2018
@datapythonista
Copy link
Member

you will also have to fix the imports, isort should be able to do it for you

@iamaziz
Copy link
Author

iamaziz commented Dec 31, 2018

Should isort be used on all the imports or just the one I added? Because isort suggested fixes on previous imports as well:
image

@datapythonista
Copy link
Member

you can leave all the changes that isort does, but as far as ci/code_checks.py is happy, it doesn't really matter

@TomAugspurger
Copy link
Contributor

I don't think we want to do this. I think the intent was to increment the version whenever we make an API breaking change to the schema. You might want to look back at the PR when JSON table schema writing was originally added to verify.

@TomAugspurger
Copy link
Contributor

In retrospect, using the pandas version at the time of the API break is not the clearest thing in the world.

@TomAugspurger
Copy link
Contributor

TomAugspurger commented Dec 31, 2018

Search for "pandas_version" in http://pandas-docs.github.io/pandas-docs-travis/io.html#table-schema. Again, not the clearest ("each revision" of what?) In my mind, it was each revision of the spec.

@iamaziz
Copy link
Author

iamaziz commented Dec 31, 2018

I see, thanks. I will go ahead and close the PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
IO JSON read_json, to_json, json_normalize
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants