Skip to content

Support for truncation_strategy, max_prompt_tokens and max_completion_tokens #1312

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
1 task done
passivebook opened this issue Apr 13, 2024 · 3 comments
Closed
1 task done

Comments

@passivebook
Copy link

Confirm this is a feature request for the Python library and not the underlying OpenAI API.

  • This is a feature request for the Python library

Describe the feature or improvement you're requesting

Can the python package be updated to support truncation_strategy, max_prompt_tokens and max_completion_tokens as seen in the OpenAI Documentation here?

Is it possible to do this already?

Additional context

No response

@CoderTonyB
Copy link

CoderTonyB commented Apr 15, 2024

I'm having the same problem. I need to upload a few thousand words as part of my workflow and it's truncating them. I see truncation_strategy as a header, but I can't figure out where to add them. As shown here:
https://platform.openai.com/docs/api-reference/runs/createRun#runs-createrun-truncation_strategy

I've tried using extra_headers:

assistant = client.beta.assistants.create(
    name="PDF Extract-O-Matic",
    instructions=instruction,
    tools=[{"type": "code_interpreter"}],
    model="gpt-4-turbo-2024-04-09",
    extra_headers={"truncation_strategy": "last_messages", "last_messages": "10"},
)

but when I add a message it still truncates the middle.

My workaround right now is just to upload the larger text as a file and reference it. It would be ideal if I didn't have to do that.

@rattrayalex
Copy link
Collaborator

I believe truncation_strategy is a body parameter, not a header.

This will be added to the SDKs shortly; thank you for your patience.

@pstern-sl
Copy link
Collaborator

This is available in version 1.18 released today #1315

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants