Expose additional implementation details for stores

[WHenderson]

Describe the problem

The svelte store paradigm currently uses a breadth-first signaling pattern to solve the "diamond dependency problem".
This is a good solution, but the internal queue (subscriber_queue) is a hidden internal variable and this means that when writing a new store implementation, there will be issues with interoperability.

i.e. the signaling will no longer be perfectly "breadth first" and thus issues like the diamond dependency will be intractable when mixing store types.

See this REPL for an illustration of the problem
https://svelte.dev/repl/b9506f52bba44c158d4cddf996252ee0?version=3.42.1

Describe the proposed solution

I would recommend the internal signal queue (subscriber_queue) be exposed as a small API.
Internally the queue could be represented as an array of actions, allowing broader extensibility.
Notably svelte-immer-store internally signals with more than just new_value, and a queue of actions would allow authors to adopt their own requirements.

I would be happy to develop and submit a pull request to solve this issue. But I would like guidance to make sure the effort was not wasted.

Alternatives considered

The only other alternative I can find is to avoid mixing the stores, but that means no interoperability between libraries.

Importance

i cannot use svelte stores without it

Edit: should have said i cant use svelte stores. Svte itself is still good 😅

[DzmitryFil]

Is couple of excessive updates really that big of a deal that you can't use svelte though?
It seems like you should be able to write your derived stores to be correct in both cases, or am I missing something?

[Conduitry]

I'm really wary about adding anything to the official store contract, and I'm also wary about adding anything to the public APIs for the store implementations that isn't part of the contract.

[WHenderson]

Is couple of excessive updates really that big of a deal that you can't use svelte though?
The updates aren't simply excessive, they are wrong.
Derived stores derive the wrong values because they are operating on a mix of old and new data.

It seems like you should be able to write your derived stores to be correct in both cases, or am I missing something?

The svelte implementation solves the diamond dependency issue by sending signals breadth first. It does this by queuing signals in an internal list.

const run_queue = !subscriber_queue.length;
for (const subscriber of subscribers) {
	subscriber[1]();
	subscriber_queue.push(subscriber, value);
}
if (run_queue) {
	for (let i = 0; i < subscriber_queue.length; i += 2) {
		subscriber_queue[i][0](subscriber_queue[i + 1]);
	}
	subscriber_queue.length = 0;
}

Given that subscriber_queue is private, the 3rd party stores cannot add to it and are thus not executed in a breadth first manner.

[WHenderson]

I'm really wary about adding anything to the official store contract, and I'm also wary about adding anything to the public APIs for the store implementations that isn't part of the contract.

Any changes would need to be 100% backwards compatible and fall in line with the existing design.
The notable changes I would seek to make are:

1. Change the implementation of the subscriber_queue to be more generic:
   Instead of an array of alternating SubscribeInvalidateTuple/value elements, to change it to a simple array of () => void actions
2. Expose enough of this queue to allow for additional store types:
   Simplest would be to export queue directly. Safest (for future extensions) would be to separate out the code above into an exported function.
3. Export the Invalidator type since it is implicitly exported anyway and a necessary feature to interop with derived stores

Each of the above should carry very little risk.

[DzmitryFil]

Is couple of excessive updates really that big of a deal that you can't use svelte though?
The updates aren't simply excessive, they are wrong.
Derived stores derive the wrong values because they are operating on a mix of old and new data.

It seems like you should be able to write your derived stores to be correct in both cases, or am I missing something?

The svelte implementation solves the diamond dependency issue by sending signals breadth first. It does this by queuing signals in an internal list.

const run_queue = !subscriber_queue.length;
for (const subscriber of subscribers) {
	subscriber[1]();
	subscriber_queue.push(subscriber, value);
}
if (run_queue) {
	for (let i = 0; i < subscriber_queue.length; i += 2) {
		subscriber_queue[i][0](subscriber_queue[i + 1]);
	}
	subscriber_queue.length = 0;
}

Given that subscriber_queue is private, the 3rd party stores cannot add to it and are thus not executed in a breadth first manner.

Yeah i get that. I don't get what real world problem you can't solve with the current setup. Your sample just demonstrates order of internal updates, and isn't anything useful to the end user.

[WHenderson]

Yeah i get that. I don't get what real world problem you can't solve with the current setup. Your sample just demonstrates order of internal updates, and isn't anything useful to the end user.

The sample demonstrates that if I use a 3rd party store for some stores, then derived stores will emit incorrect signals.
In the REPL, each update should result in a value of axbx, where x is always the same number, but when mixing the stores, you'll see axby (where x is the new value and y is the old value).

[WHenderson]

I apologize if I am being obtuse.
I'm not sure the issue was coming across well in the original repl.
Are you saying that you don't see the issue, or that you see the issue but don't see how it would effect real-world usages?

If it's the former, I've made a clearer example of the issue:
https://svelte.dev/repl/17a1c4570af9434bb2deed80a493aad4?version=3.42.2

As you can see, when not using the built-in store, the derived store gets prematurely evaluated with a mix of old and new data.
The derived store is eventually evaluated with all new data, but it's that brief incorrect evaluation that is at issue.
Depending on what the derived store is being used for, that brief incorrect result could cause untold issues down the line in a complex application.

[jnordberg]

I'm also running into this issue when implementing a store that syncs across multiple window contexts using the Channel Messaging API.

[WHenderson]

I'm also running into this issue when implementing a store that syncs across multiple window contexts using the Channel Messaging API.

If it's of use, I've written a rather comprehensive replacement for the default store implementations stores-mono.

If you are after a drop-in replacement, try @crikey/stores-svelte.

As suggested by this ticket though, you'll need to ensure that all stores use this or similar replacement to solve the issue.
I also found a further dependency tracking/premature evaluation issue referenced here: premature-evaluation
This also contains a list of notable differences between these stores and the svelte default implementation.

I hope this helps.

[jnordberg]

Thanks @WHenderson that is really helpful!

[dummdidumm]

Closing as won't do since with Svelte 5, runes are the primary mechanism for universal reactivity which are superior with regards to fine grained reactivity and firing at the right times