Skip to content

sas7bdat: Check if the SAS file has zero variables #18184

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Nov 10, 2017
Merged

sas7bdat: Check if the SAS file has zero variables #18184

merged 1 commit into from
Nov 10, 2017

Conversation

AbdealiLoKo
Copy link
Contributor

@AbdealiLoKo AbdealiLoKo commented Nov 9, 2017 •
edited
Loading

If the given SAS file has 0 rows, throw an error for the EmptyData file.
When reading, check that the column information is available. If not,
throw an error.

  • closes #xxxx
  • tests added / passed
  • passes git diff upstream/master -u -- "*.py" | flake8 --diff
  • whatsnew entry

@pep8speaks
Copy link

pep8speaks commented Nov 9, 2017 •
edited
Loading

Hello @AbdealiJK! Thanks for updating the PR.

Cheers ! There are no PEP8 issues in this Pull Request. 🍻

Comment last updated on November 10, 2017 at 03:11 Hours UTC

jreback
jreback requested changes Nov 9, 2017
View reviewed changes
Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add a whatsnew note for 0.22 (can be in bug fixes). what did this do before with an empty file?

pandas/tests/io/sas/test_sas7bdat.py Outdated
@@ -174,3 +176,9 @@ def test_date_time():
df0 = pd.read_csv(fname, parse_dates=['Date1', 'Date2', 'DateTime',
'DateTimeHi', 'Taiw'])
tm.assert_frame_equal(df, df0)

def test_zero_variables():
dirpath = tm.get_data_path()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add the issue number (if no issue, then this PR number) as a comment

pandas/tests/io/sas/test_sas7bdat.py Outdated
def test_zero_variables():
dirpath = tm.get_data_path()
fname = os.path.join(dirpath, "zero_variables.sas7bdat")
with pytest.raises(EmptyDataError):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you test with the zero observations file as well (is it the same result)? you can parametrize if that'st the case

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The zero observations file gives a None value while reading an empty the csv file gives an empty dataframe. I think this is a bug but have not fixed it right now

pandas/tests/io/sas/test_sas7bdat.py Outdated
dirpath = tm.get_data_path()
fname = os.path.join(dirpath, "zero_variables.sas7bdat")
with pytest.raises(EmptyDataError):
df = pd.read_sas(fname)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't need the df = here

@jreback jreback added the IO SAS SAS: read_sas label Nov 9, 2017
@AbdealiLoKo
Copy link
Contributor Author

@jreback - Thanks for the comments. Updated.

@AbdealiLoKo
Copy link
Contributor Author

can you add a whatsnew note for 0.22 (can be in bug fixes). what did this do before with an empty file?

Can you help me with which file I should be updating for this ?

@jreback jreback mentioned this pull request Nov 9, 2017
Closed
@jreback
Copy link
Contributor

jreback commented Nov 9, 2017

doc/source/whatsnew/v0.22.0.txt

@AbdealiLoKo
Copy link
Contributor Author

@jreback Thanks. its done

@jreback jreback added this to the 0.22.0 milestone Nov 9, 2017
@jreback jreback added Bug Error Reporting Incorrect or improved errors from pandas labels Nov 9, 2017
jreback
jreback requested changes Nov 9, 2017
View reviewed changes
doc/source/whatsnew/v0.22.0.txt Outdated
@@ -90,6 +90,7 @@ Bug Fixes
- Bug in ``pd.read_msgpack()`` with a non existent file is passed in Python 2 (:issue:`15296`)
- Bug in ``DataFrame.groupby`` where key as tuple in a ``MultiIndex`` were interpreted as a list of keys (:issue:`17979`)
- Bug in :func:`pd.read_csv` where a ``MultiIndex`` with duplicate columns was not being mangled appropriately (:issue:`18062`)
- Bug in :func:`pd.read_sas` where a file with 0 variables gives an invalid error. Now it gives and ``EmptyDataError``
Copy link
Contributor

@jreback jreback Nov 9, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what was the previous error? can you re-word this a bit.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@AbdealiLoKo
sas7bdat: Check if the SAS file has zero variables
90d100c 
If the given SAS file has 0 rows, throw an error for the EmptyData file.
When reading, check that the column information is available. If not,
throw an error.
@codecov
Copy link

codecov bot commented Nov 10, 2017

Codecov Report

Merging #18184 into master will increase coverage by 0.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #18184      +/-   ##
==========================================
+ Coverage    91.4%   91.41%   +0.01%     
==========================================
  Files         163      163              
  Lines       50064    50067       +3     
==========================================
+ Hits        45759    45767       +8     
+ Misses       4305     4300       -5
Flag Coverage Δ
#multiple 89.22% <100%> (+0.02%) ⬆️
#single 40.35% <0%> (-0.07%) ⬇️
Impacted Files Coverage Δ
pandas/io/sas/sas7bdat.py 91.05% <100%> (+0.28%) ⬆️
pandas/io/gbq.py 25% <0%> (-58.34%) ⬇️
pandas/core/frame.py 97.8% <0%> (-0.1%) ⬇️
pandas/core/indexes/datetimes.py 95.48% <0%> (+0.09%) ⬆️
pandas/plotting/_converter.py 65.2% <0%> (+1.81%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ca737ac...90d100c. Read the comment docs.

@codecov
Copy link

codecov bot commented Nov 10, 2017 •
edited
Loading

Codecov Report

Merging #18184 into master will increase coverage by 0.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #18184      +/-   ##
==========================================
+ Coverage    91.4%   91.41%   +0.01%     
==========================================
  Files         163      163              
  Lines       50064    50067       +3     
==========================================
+ Hits        45759    45767       +8     
+ Misses       4305     4300       -5
Flag Coverage Δ
#multiple 89.22% <100%> (+0.02%) ⬆️
#single 40.35% <0%> (-0.07%) ⬇️
Impacted Files Coverage Δ
pandas/io/sas/sas7bdat.py 91.05% <100%> (+0.28%) ⬆️
pandas/io/gbq.py 25% <0%> (-58.34%) ⬇️
pandas/core/frame.py 97.8% <0%> (-0.1%) ⬇️
pandas/core/indexes/datetimes.py 95.48% <0%> (+0.09%) ⬆️
pandas/plotting/_converter.py 65.2% <0%> (+1.81%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ca737ac...90d100c. Read the comment docs.

@jreback jreback merged commit 150f6d4 into pandas-dev:master Nov 10, 2017
@jreback
Copy link
Contributor

jreback commented Nov 10, 2017

thanks @AbdealiJK

No-Stream pushed a commit to No-Stream/pandas that referenced this pull request Nov 28, 2017
@AbdealiLoKo @No-Stream
sas7bdat: Check if the SAS file has zero variables (pandas-dev#18184)
6bb6664
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jreback jreback jreback approved these changes

Assignees
No one assigned
Labels
Bug Error Reporting Incorrect or improved errors from pandas IO SAS SAS: read_sas
Projects
None yet
Milestone
0.23.0
Development

Successfully merging this pull request may close these issues.
3 participants
@AbdealiLoKo @pep8speaks @jreback