Skip to content

Commit fb2fc9d

Browse files
committed
More FAQ docs
1 parent e97a4a5 commit fb2fc9d

File tree

1 file changed

+21
-8
lines changed

1 file changed

+21
-8
lines changed

docs/faq.md

+21-8
Original file line numberDiff line numberDiff line change
@@ -10,19 +10,28 @@ other data types and optional versioning.
1010

1111
Arctic can query millions of rows per second per client, achieves ~10x compression on network bandwidth,
1212
~10x compression on disk, and scales to hundreds of millions of rows per second per
13-
[MongoDB](https://www.mongodb.org/) instance.
13+
[MongoDB](https://www.mongodb.org/) instance.
14+
15+
Other benefits are:-
16+
* Serializes a number of data types eg. Pandas DataFrames, Numpy arrays, Python objects via pickling etc. so you don't have to handle different datatypes manually.
17+
* Uses LZ4 compression by default on the client side to get big savings on network / disk.
18+
* Allows you to version different stages of an object and snapshot the state (In some ways similar to git), and allows you to freely experiment and then just revert back the snapshot. [VersionStore only]
19+
* Does the chunking (breaking a Dataframe to smaller part* for you.
20+
* Adds a concept of Users and per User Libraries which can build on Mongo's auth.
21+
* Has different types of Stores, each with it's own benefits. Eg. Versionstore allows you to version and snapshot stuff, TickStore is for storage and highly efficient retrieval of streaming data, ChunkStore allows you to chunk and efficiently retrieve ranges of chunks. If nothing suits you, feel free to use vanilla Mongo commands with BSONStore.
22+
* Restricts data access to Mongo and thus prevents ad hoc queries on unindexed / unsharded collections
23+
1424

1525
## Differences between VersionStore and TickStore?
1626

17-
tickstore is for constant streams of data, version store is for working with data
18-
(i.e. playing around with it). It keeps versions so you can 'undo' changes and keep
19-
track of updates.
27+
Tickstore is for tick style data generally via streaming, VersionStore is for playing around with data. It keeps versions so you can 'undo' changes and keep track of updates.
2028

2129
## Which Store should I use?
2230

23-
* VersionStore: when ..
24-
* ChunkStore: when ..
25-
* TickStore: when ..
31+
* VersionStore: This is the default Store type. This gives you the ability to Version and Snapshot your objects while doing the serialization, compression etc alongside it. This is useful as you can basically play with your data and revert back to an older state if needed
32+
* ChunkStore: Use ChunkStore when you don't care about versioning, and want to store DataFrames into user defined chunks with fast reads.
33+
* TickStore: When you are storing constant tick data (eg. buy / sell info from exchanges). This generally plays well with Kafka / other message brokers.
34+
* BSONStore: For basically using raw Mongo operations via arctic. Can be used for storing adhoc data.
2635

2736
## Why Mongo?
2837

@@ -32,4 +41,8 @@ chose Mongo as the backend for Arctic.
3241
## I'm running Mongo in XXXX setup - what performance should I expect?
3342
We're constantly asked what the expected performance of Arctic is/should be for given configutations and Mongo cluster setups. Its hard to know for sure given the enormous number of ways Mongo, networks, machines, workstations, etc can be configured. MongoDB performance tuning is outside the scope of this library, but countless tutorials and examples are available via a quick search of the Internet.
3443

35-
... Work in Progress.
44+
45+
## Thread safety
46+
47+
VersionStore is thread safe, and operations that are interrupted should never corrupt the data, based on us writing the data segments first and then the pointers to it. This could leak data in cases though.
48+

0 commit comments

Comments
 (0)