Skip to content

Commit 3d455d6

Browse files
shashank88shashank88
authored
Merge pull request pandas-dev#682 from shashank88/docs_more
More documentation changes
2 parents 02d0e7d + 7ed9fb9 commit 3d455d6

File tree

6 files changed

+396
-0
lines changed

6 files changed

+396
-0
lines changed

‎README.md

+3
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
# [![arctic](logo/arctic_50.png)](https://github.com/manahl/arctic) [Arctic TimeSeries and Tick store](https://github.com/manahl/arctic)
22

33

4+
[![Documentation Status](https://readthedocs.org/projects/arctic/badge/?version=latest)](https://arctic.readthedocs.io/en/latest/?badge=latest)
45
[![Travis CI](https://travis-ci.org/manahl/arctic.svg?branch=master)](https://travis-ci.org/manahl/arctic)
56
[![Coverage Status](https://coveralls.io/repos/github/manahl/arctic/badge.svg?branch=master)](https://coveralls.io/github/manahl/arctic?branch=master)
67
[![Code Health](https://landscape.io/github/manahl/arctic/master/landscape.svg?style=flat)](https://landscape.io/github/manahl/arctic/master)
@@ -65,7 +66,9 @@ VersionStore supports much more: [See the HowTo](howtos/how_to_use_arctic.py)!
6566
Plugging a custom class in as a library type is straightforward. [This example
6667
shows how.](howtos/how_to_custom_arctic_library.py)
6768

69+
## Documentation
6870

71+
You can find complete documentation at [Arctic docs](https://arctic.readthedocs.io/en/latest/)
6972

7073
## Concepts
7174

‎docs/chunkstore_api.md

+306
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,306 @@
1+
# [append](https://github.com/manahl/arctic/blob/master/arctic/chunkstore/chunkstore.py#L376)
2+
```
3+
append(symbol, item)
4+
Appends data from item to symbol's data in the database.
5+
6+
Is not idempotent
7+
8+
Parameters
9+
----------
10+
symbol: str
11+
the symbol for the given item in the DB
12+
item: DataFrame or Series
13+
the data to append
14+
```
15+
16+
Example usage:
17+
18+
```
19+
>>> df = DataFrame(data={'data': [100, 200, 300]},
20+
index=MultiIndex.from_tuples([(dt(2016, 1, 1), 1),
21+
(dt(2016, 1, 2), 1),
22+
(dt(2016, 1, 3), 1)],
23+
names=['date', 'id']))
24+
25+
26+
>>> lib.write('test', df, chunk_size='M')
27+
>>> lib.read('test')
28+
data
29+
date id
30+
2016-01-01 1 100
31+
2016-01-02 1 200
32+
2016-01-03 1 300
33+
34+
>>> lib.append('test', df)
35+
>>> lib.append('test', df)
36+
>>> lib.read('test')
37+
data
38+
date id
39+
2016-01-01 1 100
40+
1 100
41+
1 100
42+
2016-01-02 1 200
43+
1 200
44+
1 200
45+
2016-01-03 1 300
46+
1 300
47+
1 300
48+
49+
```
50+
51+
# [delete](https://github.com/manahl/arctic/blob/master/arctic/chunkstore/chunkstore.py#L102)
52+
```
53+
delete(symbol, chunk_range=None)
54+
Delete all chunks for a symbol, or optionally, chunks within a range
55+
56+
Parameters
57+
----------
58+
symbol : str
59+
symbol name for the item
60+
chunk_range: range object
61+
a date range to delete
62+
```
63+
Example usage:
64+
65+
```
66+
>>> lib.read('test')
67+
data
68+
date id
69+
2016-01-01 1 100
70+
2016-01-03 1 300
71+
72+
>>> lib.delete('test', chunk_range=pd.date_range('2016-01-01', '2016-01-01'))
73+
>>> lib.read('test')
74+
data
75+
date id
76+
2016-01-03 1 300
77+
78+
```
79+
80+
# [get_chunk_ranges](https://github.com/manahl/arctic/blob/master/arctic/chunkstore/chunkstore.py#L452)
81+
```
82+
get_chunk_ranges(symbol, chunk_range=None, reverse=False)
83+
Returns a generator of (Start, End) tuples for each chunk in the symbol
84+
85+
Parameters
86+
----------
87+
symbol: str
88+
the symbol for the given item in the DB
89+
chunk_range: None, or a range object
90+
allows you to subset the chunks by range
91+
reverse: boolean
92+
93+
Returns
94+
-------
95+
generator
96+
```
97+
98+
Example usage:
99+
100+
```
101+
>>> list(lib.get_chunk_ranges('new_name'))
102+
[('2016-01-01', '2016-01-01'), ('2016-01-03', '2016-01-03')]
103+
104+
```
105+
106+
107+
# [get_info](https://github.com/manahl/arctic/blob/master/arctic/chunkstore/chunkstore.py#L431)
108+
```
109+
get_info(symbol)
110+
Returns information about the symbol, in a dictionary
111+
112+
Parameters
113+
----------
114+
symbol: str
115+
the symbol for the given item in the DB
116+
117+
Returns
118+
-------
119+
dictionary
120+
```
121+
122+
# [iterator](https://github.com/manahl/arctic/blob/master/arctic/chunkstore/chunkstore.py#L477)
123+
```
124+
iterator(symbol, chunk_range=None):
125+
Returns a generator that accesses each chunk in ascending order
126+
127+
Parameters
128+
----------
129+
symbol: str
130+
the symbol for the given item in the DB
131+
chunk_range: None, or a range object
132+
allows you to subset the chunks by range
133+
134+
Returns
135+
-------
136+
generator
137+
```
138+
139+
# [list_symbols](https://github.com/manahl/arctic/blob/master/arctic/chunkstore/chunkstore.py#L138)
140+
```
141+
list_symbols()
142+
Returns all symbols in the library
143+
144+
Returns
145+
-------
146+
list of str
147+
148+
```
149+
150+
# [read](https://github.com/manahl/arctic/blob/master/arctic/chunkstore/chunkstore.py#L176)
151+
152+
```
153+
read(symbol, chunk_range=None, filter_data=True, **kwargs)
154+
Reads data for a given symbol from the database.
155+
156+
Parameters
157+
----------
158+
symbol: str
159+
the symbol to retrieve
160+
chunk_range: object
161+
corresponding range object for the specified chunker (for
162+
DateChunker it is a DateRange object or a DatetimeIndex,
163+
as returned by pandas.date_range
164+
filter_data: boolean
165+
perform chunk level filtering on the data (see filter in _chunker)
166+
only applicable when chunk_range is specified
167+
kwargs: ?
168+
values passed to the serializer. Varies by serializer
169+
170+
Returns
171+
-------
172+
DataFrame or Series
173+
```
174+
175+
Example usage:
176+
177+
```
178+
179+
>>> dr = pd.date_range(start='2010-01-01', periods=1000, freq='D')
180+
>>> df = DataFrame(data={'data': np.random.randint(0, 100, size=1000),
181+
'date': dr
182+
})
183+
184+
>>> lib.write('symbol_name', df, chunk_size='M')
185+
>>> lib.read('symbol_name', chunk_range=pd.date_range('2012-09-01', '2016-01-01'))
186+
data date
187+
0 61 2012-09-01
188+
1 69 2012-09-02
189+
2 96 2012-09-03
190+
3 23 2012-09-04
191+
4 66 2012-09-05
192+
5 54 2012-09-06
193+
6 21 2012-09-07
194+
7 92 2012-09-08
195+
8 95 2012-09-09
196+
9 24 2012-09-10
197+
10 87 2012-09-11
198+
11 33 2012-09-12
199+
12 59 2012-09-13
200+
13 54 2012-09-14
201+
14 48 2012-09-15
202+
15 67 2012-09-16
203+
16 73 2012-09-17
204+
17 72 2012-09-18
205+
18 6 2012-09-19
206+
19 24 2012-09-20
207+
20 8 2012-09-21
208+
21 50 2012-09-22
209+
22 40 2012-09-23
210+
23 45 2012-09-24
211+
24 8 2012-09-25
212+
25 73 2012-09-26
213+
214+
```
215+
216+
217+
# [rename](https://github.com/manahl/arctic/blob/master/arctic/chunkstore/chunkstore.py#L151)
218+
```
219+
rename(from_symbol, to_symbol)
220+
Rename a symbol
221+
222+
Parameters
223+
----------
224+
from_symbol: str
225+
the existing symbol that will be renamed
226+
to_symbol: str
227+
the new symbol name
228+
```
229+
230+
# [reverse_iterator](https://github.com/manahl/arctic/blob/master/arctic/chunkstore/chunkstore.py#L498)
231+
```
232+
reverse_iterator(symbol, chunk_range=None):
233+
Returns a generator that accesses each chunk in descending order
234+
235+
Parameters
236+
----------
237+
symbol: str
238+
the symbol for the given item in the DB
239+
chunk_range: None, or a range object
240+
allows you to subset the chunks by range
241+
242+
Returns
243+
-------
244+
generator
245+
```
246+
247+
248+
# [update](https://github.com/manahl/arctic/blob/master/arctic/chunkstore/chunkstore.py#L394)
249+
```
250+
update(symbol, item, chunk_range=None, upsert=False, **kwargs)
251+
Overwrites data in DB with data in item for the given symbol.
252+
253+
Is idempotent
254+
255+
Parameters
256+
----------
257+
symbol: str
258+
the symbol for the given item in the DB
259+
item: DataFrame or Series
260+
the data to update
261+
chunk_range: None, or a range object
262+
If a range is specified, it will clear/delete the data within the
263+
range and overwrite it with the data in item. This allows the user
264+
to update with data that might only be a subset of the
265+
original data.
266+
upsert: bool
267+
if True, will write the data even if the symbol does not exist.
268+
kwargs:
269+
optional keyword args passed to write during an upsert. Includes:
270+
chunk_size
271+
chunker
272+
```
273+
274+
275+
# [write](https://github.com/manahl/arctic/blob/master/arctic/chunkstore/chunkstore.py#L230)
276+
```
277+
write(symbol, item, chunker=DateChunker(), **kwargs)
278+
Writes data from item to symbol in the database
279+
280+
Parameters
281+
----------
282+
symbol: str
283+
the symbol that will be used to reference the written data
284+
item: Dataframe or Series
285+
the data to write the database
286+
chunker: Object of type Chunker
287+
A chunker that chunks the data in item
288+
kwargs:
289+
optional keyword args that are passed to the chunker. Includes:
290+
chunk_size:
291+
used by chunker to break data into discrete chunks.
292+
see specific chunkers for more information about this param.
293+
```
294+
295+
Example usage:
296+
297+
```
298+
>>> dr = pd.date_range(start='2010-01-01', periods=1000, freq='D')
299+
>>> df = DataFrame(data={'data': np.random.randint(0, 100, size=1000),
300+
'date': dr
301+
})
302+
303+
>>> lib.write('symbol_name', df, chunk_size='M')
304+
305+
```
306+

‎docs/quickstart.md

+47
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
## Quickstart
2+
3+
### Install Arctic
4+
5+
```
6+
pip install git+https://github.com/manahl/arctic.git
7+
```
8+
9+
### Run a MongoDB
10+
11+
```
12+
mongod --dbpath <path/to/db_directory>
13+
```
14+
15+
### Using VersionStore
16+
17+
```
18+
from arctic import Arctic
19+
import quandl
20+
21+
# Connect to Local MONGODB
22+
store = Arctic('localhost')
23+
24+
# Create the library - defaults to VersionStore
25+
store.initialize_library('NASDAQ')
26+
27+
# Access the library
28+
library = store['NASDAQ']
29+
30+
# Load some data - maybe from Quandl
31+
aapl = quandl.get("WIKI/AAPL", authtoken="your token here")
32+
33+
# Store the data in the library
34+
library.write('AAPL', aapl, metadata={'source': 'Quandl'})
35+
36+
# Reading the data
37+
item = library.read('AAPL')
38+
aapl = item.data
39+
metadata = item.metadata
40+
```
41+
42+
VersionStore supports much more: [See the HowTo](howtos/how_to_use_arctic.py)!
43+
44+
45+
### Adding your own storage engine
46+
47+
Plugging a custom class in as a library type is straightforward. [This example

‎docs/releasing.md

+26
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
## PyPI
2+
3+
Package is hosted here: https://pypi.python.org/pypi/arctic/
4+
5+
## General upload and packaging docs
6+
7+
http://peterdowns.com/posts/first-time-with-pypi.html
8+
9+
## Pre-requisites
10+
11+
* Ensure you have pypandoc installed for converting the `README.md` to rst for pypi.
12+
* Configure `.pypirc` to have appropriate credentials for upload. `@burrowsa` `@jamesblackburn` have access
13+
14+
```
15+
pip install pypandoc
16+
```
17+
18+
## Procedure
19+
20+
1. Confirm the version number is sane: `grep version= setup.py`
21+
1. Ensure the working directory is clean: `git status`
22+
1. Register the egg: `python setup.py register -r pypi`
23+
1. Upload the source-dist: `python setup.py sdist upload -r pypi`
24+
1. Upload the egg: `python setup.py build bdist_egg upload -r pypi`
25+
1. Tag the package: `git tag v1.0.0 -m "Tagging v1.0.0" && git push --tags`
26+
1. Update the version number in setup.py and push to master

0 commit comments

Comments
 (0)