Skip to content

Search: index in small chunks #10914

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Nov 22, 2023
Merged

Search: index in small chunks #10914

merged 2 commits into from
Nov 22, 2023

Conversation

stsewd
Copy link
Member

@stsewd stsewd commented Nov 20, 2023

The default is 500 documents, our documents have nested objects, that can make the final document huge, and timeout ES.

closes #10911

Ref #10911 (comment)

The default is 500 documents, our documents have nested objects,
that can make the final document really big, and timeout ES.

Ref #10911 (comment)
@stsewd stsewd requested a review from a team as a code owner November 20, 2023 20:32
@stsewd stsewd requested a review from humitos November 20, 2023 20:32
Copy link
Member

@humitos humitos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are going down from 500 to 25, which seems a pretty big drop. Instead of reducing it in 25x I would start by a factor of 5x and use 100 documents.

@stsewd stsewd merged commit 6723e59 into main Nov 22, 2023
@stsewd stsewd deleted the index-in-small-chunks branch November 22, 2023 15:32
Copy link

sentry-io bot commented Nov 28, 2023

Suspect Issues

This pull request was deployed and Sentry observed the following issues:

  • ‼️ ConnectionTimeout: Connection timed out during request readthedocs.projects.tasks.search.index_build View Issue

Did you find this useful? React with a 👍 or 👎

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Duplicated search results
2 participants