Skip to content

DOC: Expand on reference docs for read_json #14442

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Oct 27, 2016
Merged

DOC: Expand on reference docs for read_json #14442

merged 5 commits into from
Oct 27, 2016

Conversation

cswarth
Copy link
Contributor

@cswarth cswarth commented Oct 17, 2016

  • closes #xxxx
  • tests added / passed
  • passes git diff upstream/master | flake8 --diff
  • [] whatsnew entry

Continuation of closed PR #14284
Documentation update for read_json().
expanded reference document for pandas.read_json(), especially concentrating on the orient parameter. Also added some example usage code and explicitly mention to_json() as a source of valid JSON strings.

pandas.read_json

pandas.read_json(path_or_buf=None, orient=None, typ='frame', dtype=True, convert_axes=True, convert_dates=True, keep_default_dates=True, numpy=False, precise_float=False, date_unit=None, encoding=None, lines=False)[source]

Convert a JSON string to pandas object

Parameters:

path_or_buf : a valid JSON string or file-like, default: None

The string could be a URL. Valid URL schemes include http, ftp, s3, and file. For file URLs, a host is expected. For instance, a local file could be file://localhost/path/to/table.json

orient : string,

Indication of expected JSON string format. Compatible JSON strings can be produced by to_json() with a corresponding orient value. The set of possible orients is:

  • 'split' : dict like {index -> [index], columns -> [columns], data -> [values]}
  • 'records' : list like [{column -> value}, ... , {column -> value}]
  • 'index' : dict like {index -> {column -> value}}
  • 'columns' : dict like {column -> {index -> value}}
  • 'values' : just the values array

The allowed and default values depend on the value of the typ parameter.

  • when typ == 'series',
    • allowed orients are {'split','records','index'}
    • default is 'index'
    • The Series index must be unique for orient 'index'.
  • when typ == 'frame',
    • allowed orients are {'split','records','index', 'columns','values'}
    • default is 'columns'
    • The DataFrame index must be unique for orients 'index' and 'columns'.
    • The DataFrame columns must be unique for orients 'index', 'columns', and 'records'.

typ : type of object to recover (series or frame), default ‘frame’

dtype : boolean or dict, default True

If True, infer dtypes, if a dict of column to dtype, then use those, if False, then don’t infer dtypes at all, applies only to the data.

convert_axes : boolean, default True

Try to convert the axes to the proper dtypes.

convert_dates : boolean, default True

List of columns to parse for dates; If True, then try to parse datelike columns default is True; a column label is datelike if

  • it ends with '_at',
  • it ends with '_time',
  • it begins with 'timestamp',
  • it is 'modified', or
  • it is 'date'

keep_default_dates : boolean, default True

If parsing dates, then parse the default datelike columns

numpy : boolean, default False

Direct decoding to numpy arrays. Supports numeric data only, but non-numeric column and index labels are supported. Note also that the JSON ordering MUST be the same for each term if numpy=True.

precise_float : boolean, default False

Set to enable usage of higher precision (strtod) function when decoding string to double values. Default (False) is to use fast but less precise builtin functionality

date_unit : string, default None

The timestamp unit to detect if converting dates. The default behaviour is to try and detect the correct precision, but if this is not desired then pass one of ‘s’, ‘ms’, ‘us’ or ‘ns’ to force parsing only seconds, milliseconds, microseconds or nanoseconds respectively.

lines : boolean, default False

Read the file as a json object per line.

New in version 0.19.0.

encoding : str, default is ‘utf-8’

The encoding to use to decode py3 bytes.

New in version 0.19.0.

Returns:

result : Series or DataFrame, depending on the value of typ.

Examples

>>> df = pd.DataFrame([['a', 'b'], ['c', 'd']],
...                   index=['row 1', 'row 2'],
...                   columns=['col 1', 'col 2'])

Encoding/decoding a Dataframe using 'split' formatted JSON:

>>> df.to_json(orient='split')
'{"columns":["col 1","col 2"],
  "index":["row 1","row 2"],
  "data":[["a","b"],["c","d"]]}'
>>> pd.read_json(_, orient='split')
      col 1 col 2
row 1     a     b
row 2     c     d

Encoding/decoding a Dataframe using 'index' formatted JSON:

>>> df.to_json(orient='index')
'{"row 1":{"col 1":"a","col 2":"b"},"row 2":{"col 1":"c","col 2":"d"}}'
>>> pd.read_json(_, orient='index')
      col 1 col 2
row 1     a     b
row 2     c     d

Encoding/decoding a Dataframe using 'records' formatted JSON. Note that index labels are not preserved with this encoding.

>>> df.to_json(orient='records')
'[{"col 1":"a","col 2":"b"},{"col 1":"c","col 2":"d"}]'
>>> pd.read_json(_, orient='records')
  col 1 col 2
0     a     b
1     c     d

@codecov-io
Copy link

codecov-io commented Oct 17, 2016

Current coverage is 85.25% (diff: 100%)

Merging #14442 into master will decrease coverage by <.01%

@@             master     #14442   diff @@
==========================================
  Files           140        140          
  Lines         50631      50631          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          43167      43166     -1   
- Misses         7464       7465     +1   
  Partials          0          0          

Powered by Codecov. Last update 7cad3f1...bdc25b7

@jreback jreback added Docs IO JSON read_json, to_json, json_normalize labels Oct 19, 2016
@jreback jreback added this to the 0.19.1 milestone Oct 19, 2016
@jreback
Copy link
Contributor

jreback commented Oct 19, 2016

does this need corresponding updates in read_json?

@@ -197,7 +203,44 @@ def read_json(path_or_buf=None, orient=None, typ='frame', dtype=True,

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add a See Also section and reference read_json (and vice versa there)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jreback This is the docstring of read_json ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry, meant to_json for the reference in See Also

Copy link
Member

@jorisvandenbossche jorisvandenbossche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cswarth Very minor comment about whitespace (rst .. :-))

@@ -197,7 +203,44 @@ def read_json(path_or_buf=None, orient=None, typ='frame', dtype=True,

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jreback This is the docstring of read_json ?


* `Series`
- ``'split'`` : dict like
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is some indentation here compared to the base level of "The set of ..." (which is not needed)

@jorisvandenbossche
Copy link
Member

@cswarth Thanks a lot!
If you want to improve the to_json docstring with some examples as well, very welcome!

@jorisvandenbossche jorisvandenbossche changed the title Doc/read json DOC: Expand on reference docs for read_json Oct 27, 2016
@jorisvandenbossche jorisvandenbossche merged commit e7ac84d into pandas-dev:master Oct 27, 2016
jorisvandenbossche pushed a commit to jorisvandenbossche/pandas that referenced this pull request Nov 2, 2016
yarikoptic added a commit to neurodebian/pandas that referenced this pull request Nov 18, 2016
Version 0.19.1

* tag 'v0.19.1': (43 commits)
  RLS: v0.19.1
  DOC: update whatsnew/release notes for 0.19.1 (pandas-dev#14573)
  [Backport pandas-dev#14545] BUG/API: Index.append with mixed object/Categorical indices (pandas-dev#14545)
  DOC: rst fixes
  [Backport pandas-dev#14567] DEPR: add deprecation warning for com.array_equivalent (pandas-dev#14567)
  [Backport pandas-dev#14551] PERF: casting loc to labels dtype before searchsorted (pandas-dev#14551)
  [Backport pandas-dev#14536] BUG: DataFrame.quantile with NaNs (GH14357) (pandas-dev#14536)
  [Backport pandas-dev#14520] BUG: don't close user-provided file handles in C parser (GH14418) (pandas-dev#14520)
  [Backport pandas-dev#14392] BUG: Dataframe constructor when given dict with None value (pandas-dev#14392)
  [Backport pandas-dev#14514] BUG: Don't parse inline quotes in skipped lines (pandas-dev#14514)
  [Bacport pandas-dev#14543] BUG: tseries ceil doc fix (pandas-dev#14543)
  [Backport pandas-dev#14541] DOC: Simplify the gbq integration testing procedure for contributors (pandas-dev#14541)
  [Backport pandas-dev#14527] BUG/ERR: raise correct error when sql driver is not installed (pandas-dev#14527)
  [Backport pandas-dev#14501] BUG: fix DatetimeIndex._maybe_cast_slice_bound for empty index (GH14354) (pandas-dev#14501)
  [Backport pandas-dev#14442] DOC: Expand on reference docs for read_json() (pandas-dev#14442)
  BLD: fix 3.4 build for cython to 0.24.1
  [Backport pandas-dev#14492] BUG: Accept unicode quotechars again in pd.read_csv
  [Backport pandas-dev#14496] BLD: Support Cython 0.25
  [Backport pandas-dev#14498] COMPAT/TST: fix test for range testing of negative integers to neg powers
  [Backport pandas-dev#14476] PERF: performance regression in Series.asof (pandas-dev#14476)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Docs IO JSON read_json, to_json, json_normalize
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants