Skip to content

lsremote fails if the output is bigger than DATA_UPLOAD_MAX_MEMORY_SIZE #9112

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
stsewd opened this issue Apr 14, 2022 · 2 comments
Closed
Labels
Accepted Accepted issue on our roadmap Bug A bug

Comments

@stsewd
Copy link
Member

stsewd commented Apr 14, 2022

When executing commands, we are sanitizing the output

self.output = self.sanitize_output(cmd_stdout)

sanitized = (
'.. (truncated) ...\n'
f'Output is too big. Truncated at {allowed_length} bytes.\n\n\n'
f'{truncated_output}'
)

but lsremote depends on the output as is. This is a problem for repositories with a lot of branches/tags

ref https://sentry.io/organizations/read-the-docs/issues/3193177679/?alert_rule_id=229005&alert_timestamp=1649941826119&alert_type=email&environment=production&project=148442

We should exclude cutting the output from that command or do this "sanitation" just when saving the command.

@stsewd stsewd added the Bug A bug label Apr 14, 2022
@agjohnson
Copy link
Contributor

agjohnson commented Apr 14, 2022

Raised in chat: I had added this separate implementation of API requests for users that were hitting request limits:

https://github.com/readthedocs/readthedocs.org/blob/main/readthedocs/doc_builder/environments.py#L238-L255

It might be good to get that out of feature flag, or potentially feature flag the truncation for now with the same API_LARGE_DATA feature at least.

@humitos humitos added the Accepted Accepted issue on our roadmap label Apr 19, 2022
@humitos
Copy link
Member

humitos commented Apr 19, 2022

This seems duplicated of #8990

@stsewd stsewd closed this as completed Apr 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Accepted Accepted issue on our roadmap Bug A bug
Projects
None yet
Development

No branches or pull requests

3 participants