Skip to content

Updated rows from batch should not be aggregated #440

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
lukaseder opened this issue Sep 1, 2021 · 10 comments
Closed

Updated rows from batch should not be aggregated #440

lukaseder opened this issue Sep 1, 2021 · 10 comments
Labels
type: bug A general bug
Milestone

Comments

@lukaseder
Copy link

Bug Report

Versions

  • Driver: 0.9.0.M2
  • Database: PostgreSQL 13.2 (Debian 13.2-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
  • Java: OpenJDK Runtime Environment (build 17-ea+31-2664)
  • OS: Microsoft Windows [Version 10.0.19043.1165]

Current Behavior

When running a batch with more than N statements, I would expect the driver to continue producing N update counts. However, it seems the driver now aggregates the update counts and produces only a single value.

Table schema

create table t (i int);

Steps to reproduce

System.out.println(
    Flux.from(cf.create())
        .flatMap(c -> c.createBatch()
            .add("insert into t values (1),(2)")
            .add("insert into t values (3)").execute())
        .flatMap(it -> it.getRowsUpdated())
        .collectList()
        .block()
    );

The output is [3]

Expected behavior/code

The expected output is [2, 1] (or [1, 2] if the sequence order cannot be maintained)

Additional context

Would be good to add expectations in the TCK for this behaviour to document it for other drivers:
https://github.com/r2dbc/r2dbc-spi/blob/47e25176a1e94736c004d20cd04315b463ddc289/r2dbc-spi-test/src/main/java/io/r2dbc/spi/test/TestKit.java#L292-L308

@mp911de
Copy link
Collaborator

mp911de commented Sep 1, 2021

This is a known limitation introduced with #401. The root cause is a weird behavior of the windowUntil operator that allows creation of substreams. In the context in which we used windowUntil, the demand dropped to zero and no frames were read from the connection. To avoid lockup we no longer split results into individual Result objects until the issue is fixed. That's a good opportunity to reach out to @simonbasle from the Reactor team.

@mp911de mp911de added status: blocked An issue that is blocked on an external project change type: bug A general bug and removed status: waiting-for-triage An issue we've not yet triaged labels Sep 1, 2021
@Squiry
Copy link
Collaborator

Squiry commented Sep 2, 2021

I think I should have provided an issue to Reactor project, but I forgot. Reproducer is here.

@mp911de
Copy link
Collaborator

mp911de commented Sep 10, 2021

Paging @simonbasle @OlegDokuka to sort out the windowUntil issue. You can find a reproducer at https://gist.github.com/mp911de/9b99e10f0e1211cd9d5d3fd1dd9479eb

Note that Reactor 3.3 can reproduce the issue while 3.4.9 seems fixed.

@OlegDokuka
Copy link
Contributor

OlegDokuka commented Sep 10, 2021

Paging @simonbasle @OlegDokuka to sort out the windowUntil issue. You can find a reproducer at https://gist.github.com/mp911de/9b99e10f0e1211cd9d5d3fd1dd9479eb

Note that Reactor 3.3 can reproduce the issue while 3.4.9 seems fixed.

We have done a couple of improvements in the few recent releases of 3.4.x, but we can see if those couled be back ported if these is critical for 3.3.x users

@mp911de
Copy link
Collaborator

mp911de commented Sep 10, 2021

That would be good to have it in 3.3.x as a few features are blocked by this issue.

@mp911de mp911de removed the status: blocked An issue that is blocked on an external project change label Sep 22, 2021
@mp911de mp911de added this to the 0.9.0.RC1 milestone Sep 22, 2021
@simonbasle
Copy link

simonbasle commented Sep 22, 2021

That would be good to have it in 3.3.x as a few features are blocked by this issue.

@mp911de do we still need to backport the changes in windowUntil from 3.4.x to 3.3.x ?
If so, can you open an issue in reactor-core so that we can track?

@mp911de
Copy link
Collaborator

mp911de commented Sep 22, 2021

Yes, the backport is needed, i filed reactor/reactor-core#2783.

@mp911de
Copy link
Collaborator

mp911de commented Sep 23, 2021

For the time being, I added a reflective guard to check when the driver is being used with Reactor 3.4, that we use windowUntil(…) as that is a safe operation.

@OlegDokuka
Copy link
Contributor

Paging @simonbasle @OlegDokuka to sort out the windowUntil issue. You can find a reproducer at gist.github.com/mp911de/9b99e10f0e1211cd9d5d3fd1dd9479eb

Note that Reactor 3.3 can reproduce the issue while 3.4.9 seems fixed.

Can you please try the same with reactor 3.3.19+. I just realized that we have them backported in exactly this version. 3.3.17 does not have them yet

@mp911de
Copy link
Collaborator

mp911de commented Sep 24, 2021

Thanks a lot. For some reason, we were stuck at Reactor 3.3.17. Reactor 3.3.20 no longer reproduces the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug A general bug
Projects
None yet
Development

No branches or pull requests

5 participants