Skip to content

Improve debuggability during model load and inference failures #163

Closed
@namannandan

Description

@namannandan

Describe the feature you'd like
Enable logging errors with traceback during model load and inference to help with debugging.

Current implementation:

except Exception as e: # pylint: disable=broad-except
trace = traceback.format_exc()
if isinstance(e, BaseInferenceToolkitError):
return self.handle_error(context, e, trace)
else:
return self.handle_error(
context,
GenericInferenceToolkitError(http_client.INTERNAL_SERVER_ERROR, str(e)),
trace,
)

How would this feature be used? Please describe.
Errors during model loading and inference will be logged.

Describe alternatives you've considered
N/A

Additional context
This is useful in scenarios where there's no direct access to an endpoint where a model is deployed, for ex: Sagemaker Endpoint, where we only have access to logs.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions