ENH: Support for reading SAS7BDAT files

closes #4052
closes #12015

Author: kshedden

LICENSES/SAS7BDAT_LICENSE

@@ -0,0 +1,19 @@
+Copyright (c) 2015 Jared Hobbs
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of
+this software and associated documentation files (the "Software"), to deal in
+the Software without restriction, including without limitation the rights to
+use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
+of the Software, and to permit persons to whom the Software is furnished to do
+so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

asv_bench/benchmarks/packers.py

@@ -318,6 +318,24 @@ def remove(self, f):
             pass
 
 
+class packers_read_sas7bdat(object):
+
+    def setup(self):
+        self.f = 'data/test1.sas7bdat'
+
+    def time_packers_read_sas7bdat(self):
+        pd.read_sas(self.f, format='sas7bdat')
+
+
+class packers_read_xport(object):
+
+    def setup(self):
+        self.f = 'data/paxraw_d_short.xpt'
+
+    def time_packers_read_xport(self):
+        pd.read_sas(self.f, format='xport')
+
+
 class packers_write_csv(object):
     goal_time = 0.2
 
@@ -854,4 +872,4 @@ def remove(self, f):
         try:
             os.remove(self.f)
         except:
-            pass
+            pass

doc/source/io.rst

@@ -2554,7 +2554,7 @@ both on the writing (serialization), and reading (deserialization).
    +----------------------+------------------------+
    | 0.18                 | >= 0.18                |
    +======================+========================+
-   
+
    Reading (files packed by older versions) is backward-compatibile, except for files packed with 0.17 in Python 2, in which case only they can only be unpacked in Python 2.
 
 .. ipython:: python
@@ -4198,7 +4198,7 @@ Authenticating with user account credentials is as simple as following the promp
 which will be automatically opened for you. You will be authenticated to the specified
 ``BigQuery`` account using the product name ``pandas GBQ``. It is only possible on local host.
 The remote authentication using user account credentials is not currently supported in Pandas.
-Additional information on the authentication mechanism can be found 
+Additional information on the authentication mechanism can be found
 `here <https://developers.google.com/identity/protocols/OAuth2#clientside/>`__.
 
 Authentication with service account credentials is possible via the `'private_key'` parameter. This method
@@ -4564,24 +4564,25 @@ easy conversion to and from pandas.
 
 .. _io.sas_reader:
 
-SAS Format
-----------
+SAS Formats
+-----------
 
 .. versionadded:: 0.17.0
 
-The top-level function :func:`read_sas` currently can read (but
-not write) SAS xport (.XPT) format files.  Pandas cannot currently
-handle SAS7BDAT files.
+The top-level function :func:`read_sas` can read (but not write) SAS
+`xport` (.XPT) and `SAS7BDAT` (.sas7bdat) format files (v0.18.0).
 
-XPORT files only contain two value types: ASCII text and double
-precision numeric values.  There is no automatic type conversion to
-integers, dates, or categoricals.  By default the whole file is read
-and returned as a ``DataFrame``.
+SAS files only contain two value types: ASCII text and floating point
+values (usually 8 bytes but sometimes truncated).  For xport files,
+there is no automatic type conversion to integers, dates, or
+categoricals.  For SAS7BDAT files, the format codes may allow date
+variables to be automatically converted to dates.  By default the
+whole file is read and returned as a ``DataFrame``.
 
-Specify a ``chunksize`` or use ``iterator=True`` to obtain an
-``XportReader`` object for incrementally reading the file.  The
-``XportReader`` object also has attributes that contain additional
-information about the file and its variables.
+Specify a ``chunksize`` or use ``iterator=True`` to obtain reader
+objects (``XportReader`` or ``SAS7BDATReader``) for incrementally
+reading the file.  The reader objects also have attributes that
+contain additional information about the file and its variables.
 
 Read a SAS XPORT file:
 
@@ -4602,6 +4603,8 @@ web site.
 
 .. _specification: https://support.sas.com/techsup/technote/ts140.pdf
 
+No official documentation is available for the SAS7BDAT format.
+
 .. _io.perf:
 
 Performance Considerations

doc/source/whatsnew/v0.18.0.txt

@@ -24,6 +24,7 @@ Highlights include:
   since 0.14.0. This will now raise a ``TypeError``, see :ref:`here <whatsnew_0180.float_indexers>`.
 - The ``.to_xarray()`` function has been added for compatibility with the
   `xarray package <http://xarray.pydata.org/en/stable/>`__, see :ref:`here <whatsnew_0180.enhancements.xarray>`.
+- The ``read_sas`` function has been enhanced to read ``sas7bdat`` files, see :ref:`here <whatsnew_0180.enhancements.sas>`.
 - Addition of the :ref:`.str.extractall() method <whatsnew_0180.enhancements.extract>`,
   and API changes to the :ref:`.str.extract() method <whatsnew_0180.enhancements.extract>`
   and :ref:`.str.cat() method <whatsnew_0180.enhancements.strcat>`.
@@ -403,6 +404,13 @@ For example, if you have a jupyter notebook you plan to convert to latex using n
 Options ``display.latex.escape`` and ``display.latex.longtable`` have also been added to the configuration and are used automatically by the ``to_latex``
 method. See the :ref:`options documentation<options>` for more info.
 
+.. _whatsnew_0180.enhancements.sas:
+
+SAS7BDAT files
+^^^^^^^^^^^^^^
+
+Pandas can now read SAS7BDAT files, including compressed files.  The files can be read in entirety, or incrementally.  For full details see :ref:`here <io.sas>`. (issue:`4052`)
+
 .. _whatsnew_0180.enhancements.other:
 
 Other enhancements

pandas/io/api.py

@@ -11,7 +11,7 @@
 from pandas.io.json import read_json
 from pandas.io.html import read_html
 from pandas.io.sql import read_sql, read_sql_table, read_sql_query
-from pandas.io.sas import read_sas
+from pandas.io.sas.sasreader import read_sas
 from pandas.io.stata import read_stata
 from pandas.io.pickle import read_pickle, to_pickle
 from pandas.io.packers import read_msgpack, to_msgpack