Skip to content

Add OANDA RestV20 Historical Instrument Reader #257

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Closed
Show file tree
Hide file tree
Changes from 53 commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
64e7e2a
Add OANDAPY as dependency for OANDA_REST datareader
davidandreoletti Nov 7, 2016
0fbc6a9
[OANDAREST] Added OANDARESTHistoricalInstrumentReader to read currenc…
davidandreoletti Nov 7, 2016
5d38d59
Rewrote OANDAHistoricalInstrumentReader to use OANDA REST V20 instead…
davidandreoletti Nov 8, 2016
c66ba1c
Skip OANDAHistoricalInstrumentReader tests when OANDA REST v20 API to…
davidandreoletti Nov 8, 2016
230d755
Replaced parameter granularity with freq. Freq accepts Panda TimeSeri…
davidandreoletti Nov 8, 2016
6d3bc7f
OANDAHistoricalInstrumentReader's freq parameter now accepts Pandas D…
davidandreoletti Nov 8, 2016
0c385aa
Removed duplicated DateOffset Aliases
davidandreoletti Nov 8, 2016
cf31164
Use DataOffset.freqstr to compute DateOffset string representation
davidandreoletti Nov 8, 2016
ee0c561
Restored access_key param inplace of custom param in DataReader function
davidandreoletti Nov 8, 2016
d1afab7
Fixed numerous python errors/warning raised by pyflake8
davidandreoletti Nov 8, 2016
23e44e9
Simplified convertion table between Panda DataOffset string represent…
davidandreoletti Nov 10, 2016
2543d81
Updated Dataframe ColumnsName to begin with uppercase character
davidandreoletti Nov 10, 2016
7a20979
Added OANDA_API_TOKEN env variable support
davidandreoletti Nov 10, 2016
772cf49
Allow list of instruments or a string representing a single instrument
davidandreoletti Nov 10, 2016
ef9f80e
Removed extraneous list comprehension. This is now done within the re…
davidandreoletti Nov 10, 2016
a89aca1
Imported os module
davidandreoletti Nov 11, 2016
1cef148
Fixed documentation for expected Panda Data Offset
davidandreoletti Nov 11, 2016
dcf86ff
Instruments are now stored in the same order as they are fetched
davidandreoletti Nov 12, 2016
2ed216f
Refactored JSON fields/columns constants
davidandreoletti Nov 12, 2016
784831a
Currency Pairs DataFrame returned are now MultiIndex enabled
davidandreoletti Nov 13, 2016
a5a3d39
Speed up unit test execution for OandaHistoricalInstrumentReader
davidandreoletti Nov 13, 2016
35036f6
Updated documentation. Gave a name to the different axes in the retur…
davidandreoletti Nov 13, 2016
ffeec50
Travis now exports OANDA_API_TOKEN
davidandreoletti Nov 14, 2016
76f8e4b
Use member instead of contructor parameter when deciding default value
davidandreoletti Dec 4, 2016
a6b8fa6
Panel returned now matches other readers dimensions order
davidandreoletti Dec 4, 2016
2c866a6
Reader fetches Midpoint, Ask and Bid candle prices by default instead…
davidandreoletti Dec 4, 2016
e659844
Verified panel returned contains expectated data structure
davidandreoletti Dec 4, 2016
a650368
Code cleanup
davidandreoletti Dec 4, 2016
005bbbc
Added example to retrieve price for a single currency and code cleanup
davidandreoletti Dec 5, 2016
3b55fe4
Moved start/end/currency1 string literal as object members to ease ma…
davidandreoletti Dec 5, 2016
9db2f72
Return DataFrame instead of Panel when a single symbol is requested (…
davidandreoletti Dec 6, 2016
1b7aa0f
Reduced memory usage by 50% (dataframe are concatenated as soon as th…
davidandreoletti Dec 6, 2016
57d17f5
Improved DataFrame layout display (for debugging)
davidandreoletti Dec 6, 2016
dba0b05
Merge 2 dataframes with DataFrame.append() instead of concat()
davidandreoletti Dec 7, 2016
cadfd8e
Remove duplicate date entries on the final dataframe
davidandreoletti Dec 7, 2016
55aee1d
Prevent copy when a renaming column
davidandreoletti Dec 7, 2016
526f4bb
Simplified DateTimeIndex creation
davidandreoletti Dec 7, 2016
dbe0d98
Code cleanup
davidandreoletti Dec 9, 2016
b4a5de6
Use datetime object to query a single price value
davidandreoletti Dec 9, 2016
b7b62b1
Fetch 2 currency pairs
davidandreoletti Dec 9, 2016
0cb0572
Test with 4H pricing data
davidandreoletti Dec 9, 2016
d26ed90
Code cleanup
davidandreoletti Dec 9, 2016
e720ba4
Updated Pandas Remote Data Documentation
davidandreoletti Dec 9, 2016
228f743
Code cleanup
davidandreoletti Dec 9, 2016
4a692a3
Fetch data for shorter duration for documentation example. Removed ex…
davidandreoletti Dec 11, 2016
29b0615
Fixed PEP8 error
davidandreoletti Dec 11, 2016
2fe4b2c
Convert pricing data from strings to float datatype. Added concurrent…
davidandreoletti Dec 11, 2016
15cb182
Enforce no more than 15 request/s per established connection as descr…
davidandreoletti Dec 11, 2016
7bf72c0
Fixed typos in documentation
davidandreoletti Dec 11, 2016
210d0c9
Added support for importing Python 2.x Queue module
davidandreoletti Dec 11, 2016
57957ec
Log OANDA specific errors
davidandreoletti Dec 11, 2016
257d108
Prevented condition where the calling thread (calling the PandaDataRe…
davidandreoletti Dec 11, 2016
0dfb503
Return None for each symbol where irrecoverable exception occured
davidandreoletti Dec 11, 2016
e6f01d8
Fixed documentation
davidandreoletti Dec 12, 2016
2b53d24
Fixed F841 error
davidandreoletti Dec 12, 2016
d063ab6
Show credentials data (for debugging purpose)
davidandreoletti Dec 12, 2016
56c28cf
Use logging.logger facility for logging statements instead of print
davidandreoletti Dec 18, 2016
0830029
Added OANDARestAccountInstrumentReader to get available instruments d…
davidandreoletti Dec 18, 2016
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ install:

script:
- export ENIGMA_API_KEY=$ENIGMA_API_KEY
- export OANDA_API_TOKEN=$OANDA_API_TOKEN
- nosetests -v --with-coverage --cover-package=pandas_datareader
- flake8 --version
- flake8 --ignore E501 pandas_datareader
Expand Down
37 changes: 37 additions & 0 deletions docs/source/remote_data.rst
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ Currently the following sources are supported:
- :ref:`Eurostat<remote_data.eurostat>`
- :ref:`Thrift Savings Plan<remote_data.tsp>`
- :ref:`Oanda currency historical rate<remote_data.oanda_curr_hist>`
- :ref:`Oanda REST currency historical rate<remote_data.oandarest_curr_hist>`
- :ref:`Nasdaq Trader symbol definitions<remote_data.nasdaq_symbols`

It should be noted, that various sources support different kinds of data, so not all sources implement the same methods and the data elements returned might also differ.
Expand Down Expand Up @@ -542,6 +543,42 @@ Download currency historical rate from `Oanda <https://www.oanda.com/>`__.

[153 rows x 3 columns]

.. _remote_data.oandarest_curr_hist

Oanda REST currency historical rate
==============================

Download currency historical rate from `Oanda <https://www.oanda.com/>`__.

.. code-block:: python

In [1]: import pandas_datareader.data as web
In [2]: start, end = "2016-01-01", "2016-02-01"
In [3]: symbols = ["EUR_USD"]
In [4]: credentials["accountType"]="practise"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

practice

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

empty credentials dict is required

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

In [5]: credentials["apiToken"]="Your OANDA API token"
In [6]: pn = web.DataReader(
symbols, data_source="oanda_historical_currency",
start=start, end=end,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might be able to pass the freq parameter

access_key=credentials
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this closing parenthesis is useless

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed


In [7]: ipdb> pn.transpose(2,1,0)["EUR_USD"].head(5)

Ask Bid Mid Ask Bid
Close High Low Open Close High Low Open Close High Low Open Volume Complete Volume Complete Volume Complete
Date
2014-03-19 09:00:00 1.39146 1.39148 1.39146 1.39146 1.39138 1.39140 1.39136 1.39136 1.39142 1.39144 1.39141 1.39141 5 True 5 True 5 True
2014-03-19 09:00:05 1.39147 1.39148 1.39147 1.39147 1.39138 1.39138 1.39137 1.39137 1.39142 1.39142 1.39142 1.39142 4 True 4 True 4 True
2014-03-19 09:00:10 1.39149 1.39149 1.39148 1.39149 1.39138 1.39141 1.39138 1.39141 1.39143 1.39145 1.39143 1.39145 3 True 3 True 3 True
2014-03-19 09:00:15 1.39153 1.39153 1.39149 1.39151 1.39143 1.39143 1.39139 1.39140 1.39148 1.39148 1.39144 1.39146 6 True 6 True 6 True
2014-03-19 09:00:20 1.39150 1.39154 1.39150 1.39154 1.39140 1.39144 1.39139 1.39143 1.39145 1.39149 1.39145 1.39148 10 True 10 True 10 True

[15438 rows x 18 columns]

In [8]: pn["Ask","Close"]["EUR_USD"][pandas._to_datetime("2014-03-19 09:00:00")]
Out[8]: 1.39147

.. _remote_data.nasdaq_symbols

Nasdaq Trader Symbol Definitions
Expand Down
24 changes: 21 additions & 3 deletions pandas_datareader/data.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@
from pandas_datareader.oanda import get_oanda_currency_historical_rates
from pandas_datareader.nasdaq_trader import get_nasdaq_symbols

from pandas_datareader.oandarest import OANDARestHistoricalInstrumentReader


def get_data_fred(*args, **kwargs):
return FredReader(*args, **kwargs).read()
Expand Down Expand Up @@ -57,7 +59,8 @@ def get_quote_google(*args, **kwargs):


def DataReader(name, data_source=None, start=None, end=None,
retry_count=3, pause=0.001, session=None, access_key=None):
retry_count=3, pause=0.001, session=None,
access_key=None):
"""
Imports data from a number of online sources.

Expand All @@ -67,11 +70,11 @@ def DataReader(name, data_source=None, start=None, end=None,
Parameters
----------
name : str or list of strs
the name of the dataset. Some data sources (yahoo, google, fred) will
the name of the dataset. Some data sources (yahoo, google, fred, oanda_historical_currency) will
accept a list of names.
data_source: {str, None}
the data source ("yahoo", "yahoo-actions", "yahoo-dividends",
"google", "fred", "ff", or "edgar-index")
"google", "fred", "ff", or "edgar-index", oanda_historical_currency)
start : {datetime, None}
left boundary for range (defaults to 1/1/2010)
end : {datetime, None}
Expand All @@ -83,6 +86,8 @@ def DataReader(name, data_source=None, start=None, end=None,
single value given for symbol, represents the pause between retries.
session : Session, default None
requests.sessions.Session instance to be used
access_key: dict of object
Reader specific credentials

Examples
----------
Expand All @@ -108,6 +113,13 @@ def DataReader(name, data_source=None, start=None, end=None,
# Data from EDGAR index
ed = DataReader("full", "edgar-index")
ed2 = DataReader("daily", "edgar-index")

# OANDA REST
oa = DataReader("EUR_USD", data_source="oanda_rest_historical_currency",
access_key={ "accountType"="practice",
"accountVersion="0"
"apiToken":"Your private API token" })
Copy link
Contributor

@femtotrader femtotrader Nov 8, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oa = DataReader("EUR_USD", data_source="oanda_rest_historical_currency",
    custom = {
        "accountType": "practice",
        "accountVersion": "0",
        "apiToken": "Your private API token"
    })

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@femtotrader Fixed


"""
if data_source == "yahoo":
return YahooDailyReader(symbols=name, start=start, end=end,
Expand Down Expand Up @@ -167,6 +179,12 @@ def DataReader(name, data_source=None, start=None, end=None,
raise ValueError("Only the string 'symbols' is supported for "
"Nasdaq, not %r" % (name,))
return get_nasdaq_symbols(retry_count=retry_count, pause=pause)
elif data_source == "oanda_historical_currency":
return OANDARestHistoricalInstrumentReader(
symbols=name, symbolsTypes=None,
start=start, end=end,
reader_compatible=True,
access_credential=access_key, session=session).read()
else:
msg = "data_source=%r is not implemented" % data_source
raise NotImplementedError(msg)
Expand Down
Loading