Issue with offset

[jmdobry]

There is a problem with offset:
Suppose that resource has no injected items, then by calling

DS.findAll('resource', {offset: 10, limit: 10})

we will get requested items injected to data store and our query cached. But if we call the same code again, the DS.filter function kicks in and will offset these 10 items in data store, thus returning empty array.

See also #76

[aikoven]

One possible solution would be to store returned items by params hash alongside resource.completedQueries and return them from DS.findAll instead of calling DS.filter. A shortcoming of this approach is when offset is not a multiple of limit, we'd potentially get lots of cached arrays with lots of duplicate elements between them.

Another idea is to store offset value for each {where: ..., orderBy: ...} object for each item based on its index in returned array. Although it leads to much more complicated work to build an array from cache, this approach makes DS.filter and DS.findAll truly consistent.

[jmdobry]

I fixed this in js-data, so it will be available in js-data-angular (angular-data 2.0).

See js-data/js-data#30

[jmdobry]

One possible solution would be to store returned items by params hash alongside resource.completedQueries and return them from DS.findAll instead of calling DS.filter.

That was the solution.

[aikoven]

I've ran against an issue with this solution, when new items are added to storage, DS.findAll would still return old items. Also when item is ejected from storage, we only remove item from query data, although we should probably flush queryData completely. Here's an example:

Suppose that resource has 10 items on the server.

DS.findAll('resource', {limit: 5})  // get first 5 items
// ...
DS.eject('resource', 0)  // eject one item
DS.findAll('resource', {limit: 5})  // this will return 4 items instead of 5

[aikoven]

We can still optimise this a bit and only flush queryData when added/removed/changed item meets query criteria.

[jmdobry]

Here's how it works (assuming default settings). If the server has 10 items, then:

1. DS.findAll('resource', { limit: 5 }) // Injects 5 items into the store, and "remembers" that those particular 5 items were returned by the server for the "{limit:5}" query.
2. DS.findAll('resource', { limit: 5 }) // Resolves with whatever was cached for the "{limit:5}" query.
3. DS.findAll('resource', { limit: 5 }, { useFilter: true }) // Instead, delegate to DS#filter, passing it the params arg that was just passed to DS#findAll. Resolve with the result of the DS#filter call.
4. DS.eject('resource', 0) // Remove an item from the store, also removing it from any cached queries the item appears in. Any cached queries whose associated list of data becomes empty because of this action are automatically flushed/invalidated, otherwise, the store just assumes the updated cached query is what would be returned by the server if the query were to be made again.
5. DS.findAll('resource', { limit: 5 }) // Resolves with whatever was cached for the "{limit:5}" query (now 4 items).
6. DS.findAll('resource', { limit: 5 }, { useFilter: true }) // Instead, delegate to DS#filter, passing it the params arg that was just passed to DS#findAll. Resolve with the result of the DS#filter call (now 4 items).

Now, you can manually eject the remaining items and the cached query with be flushed/invalidated.

Or you can manually flush the query by doing this:

// Force a new query to be made
DS.findAll('resource', { limit: 5 }, { bypassCache: true })