message too long errors

[thehesiod]

Occasionally I'll get the following error from the udp_emitter:

2018-02-14 16:28:46,744 - aws_xray_sdk.core.emitters.udp_emitter - ERROR - failed to send data to X-Ray daemon.
Traceback (most recent call last):
  File "/Users/amohr/.pyenv/versions/3.6.3/lib/python3.6/site-packages/aws_xray_sdk/core/emitters/udp_emitter.py", line 55, in _send_data
    self._port))
OSError: [Errno 40] Message too long

my guess is that there's too much data in the trace. This sounds like a library problem, if there's too much data it should split it into multiple packets.

[haotianw465]

Hi, the SDK has thin layer modular design. The X-Ray recorder decides when to stream a segment (e.g. send out some closed children subsegments first) and the emitter simply executes what it is told to do. The default maximum number of subsegments it holds is set to 30: https://github.com/aws/aws-xray-sdk-python/blob/master/aws_xray_sdk/core/recorder.py#L46. The reason of using size of a subsegments array is that byte counting is expensive.

Depends on what do your segment look like 30 subsegments could still be large. You can modify it to a lower value so the streaming happens earlier and more frequent. This number is tested under UDP packet size 64KB but depend on your operating system configuration it might allow an even smaller packet pass through.

Hope it helps.

[haotianw465]

Close. Feel free to re-open for any additional comment.

[thehesiod]

I believe this is not configurable in xray-sdk, is this not the case? See https://github.com/aws/aws-xray-sdk-python/blob/master/aws_xray_sdk/core/recorder.py#L46

[thehesiod]

unfortunately I'm not allowed to re-open

[haotianw465]

Sorry about that. You are right. I confused it with our another SDK. The public interface configure() is missing a keyword argument for configuring this. Will add it to the next release. A PR is also welcome.

For workaround you can temporarily use xray_recorder._max_subsegments = 10 until the public interface is added.

[haotianw465]

Released as part of 0.96.

[MarSoft]

Just for record: looks like in current version _max_subsegments can be publically changed by assigning to streaming_threshold property:

xray_recorder.streaming_threshold = 10

Also it is exposed as streaming_threshold argument to .configure() method.