Skip to content

BulkIngester: Add pipeline to index and create operations #586

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
renatogm24 opened this issue May 26, 2023 · 1 comment
Closed

BulkIngester: Add pipeline to index and create operations #586

renatogm24 opened this issue May 26, 2023 · 1 comment
Assignees
Labels
Category: Bug Something isn't working

Comments

@renatogm24
Copy link
Contributor

renatogm24 commented May 26, 2023

Currently, when using the BulkIngester in the Elasticsearch Java client, the pipeline for individual index operations is not set up. Instead, the default pipeline on the index is used for all operations. This can limit flexibility and control when different documents in the same index require different processing.

For example, consider a scenario where we have multiple documents in the same index, but each document requires a different pipeline for processing. In the current implementation, all documents would be processed using the default pipeline on the index, which may not be suitable for all documents.

This issue can be addressed by allowing the pipeline to be set up for each individual index operation. This would involve copying the pipeline property from the IndexOperation to the IndexOperation.Builder for each operation.

I have created a pull request that introduces these changes: PR #587. Please review the PR and let me know if you have any questions or need further information.

@swallez swallez added the Category: Bug Something isn't working label Jun 9, 2023
@swallez swallez self-assigned this Jun 9, 2023
@swallez
Copy link
Member

swallez commented Jun 13, 2023

Fixed in PR #587 and backported to the 8.8 branch in PR #594

@swallez swallez closed this as completed Jun 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Category: Bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants