Improve Documentation if an ItemWriter is used with a Database connection

[git9999999]

Hi

I have asked a Question https://stackoverflow.com/questions/76073523/spring-batch-with-faulttolerant-config-swallow-sql-exception-on-commit-of-a-item/76099432

Here is the same info again:

We make heavy use of Spring Batch (5.0.1) in our new Project (SpringBoot 3.0.5) and MS Sql Server. We like to use the "faultTolerant" feature. eg:

return new StepBuilder("abcStep", jobRepository)
     .<String, String>chunk(1, transactionManager)
     .reader(aItemReader)
     .writer(aItemWriter)
     .faultTolerant()
     .backOffPolicy(
         JobCommonConfig.milliSecondsBetweenRetiesPolicy(
            this.milliSecondsBetweenReties))
     .retryLimit(this.retryLimit)
     .retry(Throwable.class)
     .build();

Up to now, everything works fine. Now we have found a case where we process data in the ItemWriter that are not ok. Some data is too long for the DB field. We use JPA and do no flush.

WARN  o.h.engine.jdbc.spi.SqlExceptionHelper   : SQL Error: 2628, SQLState: S0001
ERROR o.h.engine.jdbc.spi.SqlExceptionHelper   : String or binary data would be truncated in table 'abc.dbo.T_.....', column 'FILE_NAME_BASE64'. Truncated value: '....'.
INFO  o.s.batch.core.step.tasklet.TaskletStep  : Commit failed while step execution data was already updated. Reverting to old version.
INFO  o.s.batch.core.step.AbstractStep         : Step: [aStep] executed in 124ms

My expectation would be, on an error, the Step stops with an exception. But this does not happen, it continues the Job. It also rollback my Business transaction.😱😱😱
For use, it looks like everything worked correct, but we just missed some data. 😱😱😱

If i add a flush() with the entitymanager, i see the exception and Spring Batch behaves as expected and stops the Job.

If i change the Step to this, it also works correct (stop the Job):

return new StepBuilder("abcStep", jobRepository)
     .<String, String>chunk(1, transactionManager)
     .reader(aItemReader)
     .writer(aItemWriter)
     .build();

I found two entries that could explain the problem:

#1189

#3950

What do i wrong? Is the Spring Batch faultTolerant broken?

---

1. I think the Documentation should be improved, telling you to flush when you use a DB connection without a correct Writer.

2. If possible Spring Batch should stop processing when this error occur:

WARN  o.h.engine.jdbc.spi.SqlExceptionHelper   : SQL Error: 2628, SQLState: S0001
ERROR o.h.engine.jdbc.spi.SqlExceptionHelper   : String or binary data would be truncated in table 'abc.dbo.T_.....', column 'FILE_NAME_BASE64'. Truncated value: '....'.
INFO  o.s.batch.core.step.tasklet.TaskletStep  : Commit failed while step execution data was already updated. Reverting to old version.
INFO  o.s.batch.core.step.AbstractStep         : Step: [aStep] executed in 124ms

[fmbenhassine]

We use JPA and do no flush.

What do i wrong? Is the Spring Batch faultTolerant broken?

No, the fault-tolerance feature is not broken. As mentioned in SO, the provided JpaItemWriter does the flush and things work as expected with that built-in writer.

Now if you use a custom writer, it is up to you to flush items. In SO, I said "This is probably a documentation issue", and after verifying, this is already in the documentation in the Database ItemWriters section. Here is the relevant excerpt:

Users can create their own DAOs that implement the ItemWriter interface [...]. Batching database
output does not have any inherent flaws, assuming we are careful to flush and there are no errors
in the data.

So since you are providing a custom JPA writer, it is assumed that you flush items yourself.