Skip to content

KafkaProducerMessageHandler send failure messages for immediate exceptions #8638

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
garyrussell opened this issue Jun 7, 2023 Discussed in #8637 · 0 comments
Closed

KafkaProducerMessageHandler send failure messages for immediate exceptions #8638

garyrussell opened this issue Jun 7, 2023 Discussed in #8637 · 0 comments
Assignees
@garyrussell
Labels
(EOL) for: backport-to-6.1.x in: Kafka type: enhancement
Milestone
6.2.0-M1

Comments

@garyrussell
Copy link
Contributor

Discussed in #8637

Originally posted by yevsh June 1, 2023
Sometimes there are "error" network/kafka cluster errors - that can be only detected when we see in logs something like:

"INFO","logger":"org.apache.kafka.clients.NetworkClient","message":"[Producer clientId=producer-1] Node 2 disconnected.

The problem KafkaTemplate/KafkaProducerMessageHandler can't identify it as error, and data can be lost.

Need some reliable solution for this.
@garyrussell garyrussell added this to the 6.2.x milestone Jun 7, 2023
@garyrussell garyrussell self-assigned this Jun 7, 2023
@garyrussell garyrussell modified the milestones: 6.2.x, 6.2.0-M1 Jun 8, 2023
garyrussell added a commit to garyrussell/spring-integration that referenced this issue Jun 8, 2023
@garyrussell
spring-projectsGH-8638: Kafka: Send All Fails to Failure Channel
95f58a8 
Resolveshttps://github.com/spring-projects/issues/8638

Previously, immediate failures (e.g. timeout getting metadata) were
only thrown as exceptions, and not sent to the failure channel, if present.

**cherry-pick to all supported branches**
garyrussell added a commit to garyrussell/spring-integration that referenced this issue Jun 8, 2023
@garyrussell
spring-projectsGH-8638: Kafka: Send All Fails to Failure Channel
9a2e195 
Resolveshttps://github.com/spring-projects/issues/8638

Previously, immediate failures (e.g. timeout getting metadata) were
only thrown as exceptions, and not sent to the failure channel, if present.

**cherry-pick to all supported branches**
@garyrussell garyrussell mentioned this issue Jun 8, 2023
Merged
artembilan pushed a commit that referenced this issue Jun 8, 2023
@garyrussell @artembilan
GH-8638: Kafka: Send All Fails to Failure Channel
01e7091 
Resolves #8638

Previously, immediate failures (e.g. timeout getting metadata) were
only thrown as exceptions, and not sent to the failure channel, if present.

**cherry-pick to all supported branches**
artembilan pushed a commit that referenced this issue Jun 8, 2023
@garyrussell @artembilan
GH-8638: Kafka: Send All Fails to Failure Channel
192bd7b 
Resolves #8638

Previously, immediate failures (e.g. timeout getting metadata) were
only thrown as exceptions, and not sent to the failure channel, if present.

**cherry-pick to all supported branches**
artembilan pushed a commit that referenced this issue Jun 8, 2023
@garyrussell @artembilan
GH-8638: Kafka: Send All Fails to Failure Channel
f9f1631 
Resolves #8638

Previously, immediate failures (e.g. timeout getting metadata) were
only thrown as exceptions, and not sent to the failure channel, if present.

**cherry-pick to all supported branches**
# Conflicts:
#	spring-integration-kafka/src/main/java/org/springframework/integration/kafka/outbound/KafkaProducerMessageHandler.java
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@garyrussell garyrussell

Labels
(EOL) for: backport-to-6.1.x in: Kafka type: enhancement
Projects
None yet
Milestone
6.2.0-M1
Development

No branches or pull requests
1 participant
@garyrussell