Skip to content

How to select a sampling rule with service name? #352

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
ohke opened this issue Aug 30, 2022 · 2 comments
Closed

How to select a sampling rule with service name? #352

ohke opened this issue Aug 30, 2022 · 2 comments

Comments

@ohke
Copy link

ohke commented Aug 30, 2022 •
edited
Loading

I am attempting to have the sampling rule selected with service name, but it does not appear to be selected correctly.

reproduction

The following sampling rules have already been created.
The scenario is intended to be that a rule matching the service and ServiceName of the trace is applied.

$ aws xray get-sampling-rules
{
    "SamplingRuleRecords": [
        {
            "SamplingRule": {
                "RuleName": "app-b",
                "RuleARN": "arn:aws:xray:ap-northeast-1:123456789012:sampling-rule/app-b",
                "ResourceARN": "*",
                "Priority": 11,
                "FixedRate": 1.0,
                "ReservoirSize": 1,
                "ServiceName": "app-b",
                "ServiceType": "*",
                "Host": "*",
                "HTTPMethod": "*",
                "URLPath": "*",
                "Version": 1,
                "Attributes": {}
            },
            "CreatedAt": 1661500922.0,
            "ModifiedAt": 1661826542.0
        },
        {
            "SamplingRule": {
                "RuleName": "app-a",
                "RuleARN": "arn:aws:xray:ap-northeast-1:123456789012:sampling-rule/app-a",
                "ResourceARN": "*",
                "Priority": 10,
                "FixedRate": 0.01,
                "ReservoirSize": 1,
                "ServiceName": "app-a",
                "ServiceType": "*",
                "Host": "*",
                "HTTPMethod": "*",
                "URLPath": "*",
                "Version": 1,
                "Attributes": {}
            },
            "CreatedAt": 1661233373.0,
            "ModifiedAt": 1661826536.0
        },
        {
            "SamplingRule": {
                "RuleName": "Default",
                "RuleARN": "arn:aws:xray:ap-northeast-1:123456789012:sampling-rule/Default",
                "ResourceARN": "*",
                "Priority": 10000,
                "FixedRate": 0.05,
                "ReservoirSize": 1,
                "ServiceName": "*",
                "ServiceType": "*",
                "Host": "*",
                "HTTPMethod": "*",
                "URLPath": "*",
                "Version": 1,
                "Attributes": {}
            },
            "CreatedAt": 0.0,
            "ModifiedAt": 0.0
        }
    ]
}

I have an X-Ray daemon process running on my local MacOS.

$ ./bin/xray_mac -v                    
AWS X-Ray daemon version: 3.3.3

$ ./bin/xray_mac -o -n ap-northeast-1

The following is the Python code that outputs the trace.
When creating the segment, "app-b" is set for service, but the sampling rules for "app-a" have been applied.

$ python --version
Python 3.9.0

$ pip list
Package         Version
--------------- -------
aws-xray-sdk    2.10.0
botocore        1.27.62
jmespath        1.0.1
pip             20.2.3
python-dateutil 2.8.2
setuptools      49.2.1
six             1.16.0
urllib3         1.26.12
wrapt           1.14.1

$ cat ./main.py
import logging
from time import sleep
from aws_xray_sdk.core import xray_recorder

def main():
    for _ in range(10):
        with xray_recorder.in_segment() as segment:
            sleep(0.4)
            print(f"trace_id: {segment.trace_id}")

if __name__ == "__main__":
    logging.basicConfig(level=logging.DEBUG)

    xray_recorder.configure(service="app-b", sampling=True)

    main()

$ AWS_XRAY_DAEMON_ADDRESS=127.0.0.1:2000 AWS_XRAY_TRACING_NAME=app-b python main.py
DEBUG:botocore.hooks:Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane
DEBUG:botocore.hooks:Changing event name from before-call.apigateway to before-call.api-gateway
DEBUG:botocore.hooks:Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict
DEBUG:botocore.hooks:Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration
DEBUG:botocore.hooks:Changing event name from before-parameter-build.route53 to before-parameter-build.route-53
DEBUG:botocore.hooks:Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search
DEBUG:botocore.hooks:Changing event name from docs.*.autoscaling.CreateLaunchConfiguration.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section
DEBUG:botocore.hooks:Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask
DEBUG:botocore.hooks:Changing event name from docs.*.logs.CreateExportTask.complete-section to docs.*.cloudwatch-logs.CreateExportTask.complete-section
DEBUG:botocore.hooks:Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search
DEBUG:botocore.hooks:Changing event name from docs.*.cloudsearchdomain.Search.complete-section to docs.*.cloudsearch-domain.Search.complete-section
DEBUG:botocore.loaders:Loading JSON file: /Users/kenta.onishi/chart/xray-sandbox/.venv/lib/python3.9/site-packages/botocore/data/endpoints.json
DEBUG:botocore.loaders:Loading JSON file: /Users/kenta.onishi/chart/xray-sandbox/.venv/lib/python3.9/site-packages/botocore/data/sdk-default-configuration.json
DEBUG:botocore.hooks:Event choose-service-name: calling handler <function handle_service_name_alias at 0x1032cb9d0>
DEBUG:botocore.loaders:Loading JSON file: /Users/kenta.onishi/chart/xray-sandbox/.venv/lib/python3.9/site-packages/botocore/data/xray/2016-04-12/service-2.json
DEBUG:botocore.hooks:Event creating-client-class.xray: calling handler <function add_generate_presigned_url at 0x103206ca0>
DEBUG:botocore.endpoint:Setting xray timeout as (60, 60)
DEBUG:botocore.loaders:Loading JSON file: /Users/kenta.onishi/chart/xray-sandbox/.venv/lib/python3.9/site-packages/botocore/data/_retry.json
DEBUG:botocore.client:Registering retry handlers for service: xray
DEBUG:botocore.hooks:Event before-parameter-build.xray.GetSamplingRules: calling handler <function generate_idempotent_uuid at 0x1032e7040>
DEBUG:botocore.hooks:Event before-call.xray.GetSamplingRules: calling handler <function add_recursion_detection_header at 0x1032e2ca0>
INFO:aws_xray_sdk.core.sampling.sampler:No effective centralized sampling rule match. Fallback to local rules.
DEBUG:botocore.hooks:Event before-call.xray.GetSamplingRules: calling handler <function inject_api_version_header_if_needed at 0x1032e88b0>
DEBUG:botocore.endpoint:Making request for OperationModel(name=GetSamplingRules) with params: {'url_path': '/GetSamplingRules', 'query_string': {}, 'method': 'POST', 'headers': {'Content-Type': 'application/json', 'User-Agent': 'Botocore/1.27.62 Python/3.9.0 Darwin/21.6.0'}, 'body': b'{}', 'url': 'http://127.0.0.1:2000/GetSamplingRules', 'context': {'client_region': 'us-west-2', 'client_config': <botocore.config.Config object at 0x1038cfd90>, 'has_streaming_input': False, 'auth_type': None}}
DEBUG:botocore.hooks:Event request-created.xray.GetSamplingRules: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x1038cfd30>>
DEBUG:botocore.hooks:Event choose-signer.xray.GetSamplingRules: calling handler <function set_operation_specific_signer at 0x1032e2ee0>
DEBUG:botocore.hooks:Event request-created.xray.GetSamplingRules: calling handler <function add_retry_headers at 0x1032e8f70>
DEBUG:botocore.endpoint:Sending http request: <AWSPreparedRequest stream_output=False, method=POST, url=http://127.0.0.1:2000/GetSamplingRules, headers={'Content-Type': b'application/json', 'User-Agent': b'Botocore/1.27.62 Python/3.9.0 Darwin/21.6.0', 'amz-sdk-invocation-id': b'd57dbf03-fd23-4807-bd1c-65630bf1dbd7', 'amz-sdk-request': b'attempt=1', 'Content-Length': '2'}>
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): 127.0.0.1:2000
DEBUG:urllib3.connectionpool:http://127.0.0.1:2000 "POST /GetSamplingRules HTTP/1.1" 200 1068
DEBUG:botocore.parsers:Response headers: {'Content-Length': '1068', 'Content-Type': 'application/json', 'Date': 'Tue, 30 Aug 2022 04:17:54 GMT', 'Strict-Transport-Security': 'max-age=31540000; includeSubDomains', 'X-Amzn-Requestid': '3fd0dde5-0007-460e-9762-5e3b3987870c', 'X-Content-Type-Options': 'nosniff'}
DEBUG:botocore.parsers:Response body:
b'{"NextToken":null,"SamplingRuleRecords":[{"CreatedAt":1.661500922E9,"ModifiedAt":1.661826542E9,"SamplingRule":{"Attributes":{},"FixedRate":1.0,"HTTPMethod":"*","Host":"*","Priority":11,"ReservoirSize":1,"ResourceARN":"*","RuleARN":"arn:aws:xray:ap-northeast-1:123456789012:sampling-rule/app-b","RuleName":"app-b","ServiceName":"app-b","ServiceType":"*","URLPath":"*","Version":1}},{"CreatedAt":1.661233373E9,"ModifiedAt":1.661826536E9,"SamplingRule":{"Attributes":{},"FixedRate":0.01,"HTTPMethod":"*","Host":"*","Priority":10,"ReservoirSize":1,"ResourceARN":"*","RuleARN":"arn:aws:xray:ap-northeast-1:123456789012:sampling-rule/app-a","RuleName":"app-a","ServiceName":"app-a","ServiceType":"*","URLPath":"*","Version":1}},{"CreatedAt":0.0,"ModifiedAt":0.0,"SamplingRule":{"Attributes":{},"FixedRate":0.05,"HTTPMethod":"*","Host":"*","Priority":10000,"ReservoirSize":1,"ResourceARN":"*","RuleARN":"arn:aws:xray:ap-northeast-1:123456789012:sampling-rule/Default","RuleName":"Default","ServiceName":"*","ServiceType":"*","URLPath":"*","Version":1}}]}'
DEBUG:botocore.hooks:Event needs-retry.xray.GetSamplingRules: calling handler <botocore.retryhandler.RetryHandler object at 0x1038cf310>
DEBUG:botocore.retryhandler:No retry needed.
trace_id: 1-630d8f72-32b7265a9103495bb9a41f38
DEBUG:aws_xray_sdk.core.emitters.udp_emitter:sending: {"format":"json","version":1}
{"id": "28a9fbef177a9cb0", "name": "app-b", "start_time": 1661833074.652981, "in_progress": false, "aws": {"xray": {"sdk": "X-Ray for Python", "sdk_version": "2.10.0"}}, "trace_id": "1-630d8f72-32b7265a9103495bb9a41f38", "service": {"runtime": "CPython", "runtime_version": "3.9.0"}, "end_time": 1661833075.056917} to 127.0.0.1:2000.
DEBUG:aws_xray_sdk.core.sampling.sampler:Rule app-a is selected to make a sampling decision.
trace_id: 1-630d8f73-813017f3aafc5dc1cc9b6b64
DEBUG:aws_xray_sdk.core.emitters.udp_emitter:sending: {"format":"json","version":1}
{"id": "07d58fa3c63246cd", "name": "app-b", "start_time": 1661833075.057573, "in_progress": false, "aws": {"xray": {"sdk": "X-Ray for Python", "sdk_version": "2.10.0", "sampling_rule_name": "app-a"}}, "trace_id": "1-630d8f73-813017f3aafc5dc1cc9b6b64", "service": {"runtime": "CPython", "runtime_version": "3.9.0"}, "end_time": 1661833075.46228} to 127.0.0.1:2000.
DEBUG:aws_xray_sdk.core.sampling.sampler:Rule app-a is selected to make a sampling decision.
trace_id: 1-00000000-000000000000000000000000
DEBUG:aws_xray_sdk.core.sampling.sampler:Rule app-a is selected to make a sampling decision.
trace_id: 1-00000000-000000000000000000000000
DEBUG:aws_xray_sdk.core.sampling.sampler:Rule app-a is selected to make a sampling decision.
trace_id: 1-630d8f74-c95387d8196161ec9bd65cbb
DEBUG:aws_xray_sdk.core.emitters.udp_emitter:sending: {"format":"json","version":1}
{"id": "8530c9dc398feb84", "name": "app-b", "start_time": 1661833076.271061, "in_progress": false, "aws": {"xray": {"sdk": "X-Ray for Python", "sdk_version": "2.10.0", "sampling_rule_name": "app-a"}}, "trace_id": "1-630d8f74-c95387d8196161ec9bd65cbb", "service": {"runtime": "CPython", "runtime_version": "3.9.0"}, "end_time": 1661833076.67341} to 127.0.0.1:2000.
DEBUG:aws_xray_sdk.core.sampling.sampler:Rule app-a is selected to make a sampling decision.
trace_id: 1-00000000-000000000000000000000000
DEBUG:aws_xray_sdk.core.sampling.sampler:Rule app-a is selected to make a sampling decision.
trace_id: 1-630d8f75-74b2720c662574a78476c09b
DEBUG:aws_xray_sdk.core.emitters.udp_emitter:sending: {"format":"json","version":1}
{"id": "790a9823ca73b034", "name": "app-b", "start_time": 1661833077.07854, "in_progress": false, "aws": {"xray": {"sdk": "X-Ray for Python", "sdk_version": "2.10.0", "sampling_rule_name": "app-a"}}, "trace_id": "1-630d8f75-74b2720c662574a78476c09b", "service": {"runtime": "CPython", "runtime_version": "3.9.0"}, "end_time": 1661833077.483422} to 127.0.0.1:2000.
DEBUG:aws_xray_sdk.core.sampling.sampler:Rule app-a is selected to make a sampling decision.
trace_id: 1-00000000-000000000000000000000000
DEBUG:aws_xray_sdk.core.sampling.sampler:Rule app-a is selected to make a sampling decision.
trace_id: 1-00000000-000000000000000000000000
DEBUG:aws_xray_sdk.core.sampling.sampler:Rule app-a is selected to make a sampling decision.
trace_id: 1-630d8f76-5b69640dd4404539427a9ea1
DEBUG:aws_xray_sdk.core.emitters.udp_emitter:sending: {"format":"json","version":1}
{"id": "48178b2211a495a8", "name": "app-b", "start_time": 1661833078.293151, "in_progress": false, "aws": {"xray": {"sdk": "X-Ray for Python", "sdk_version": "2.10.0", "sampling_rule_name": "app-a"}}, "trace_id": "1-630d8f76-5b69640dd4404539427a9ea1", "service": {"runtime": "CPython", "runtime_version": "3.9.0"}, "end_time": 1661833078.695625} to 127.0.0.1:2000.

I checked the sampling_rule_name of the raw data in the trace and it was "app-a" as well.

{
    "Id": "1-630d8f76-5b69640dd4404539427a9ea1",
    "Duration": 0.402,
    "LimitExceeded": false,
    "Segments": [
        {
            "Id": "48178b2211a495a8",
            "Document": {
                "id": "48178b2211a495a8",
                "name": "app-b",
                "start_time": 1661833078.293151,
                "trace_id": "1-630d8f76-5b69640dd4404539427a9ea1",
                "end_time": 1661833078.695625,
                "in_progress": false,
                "aws": {
                    "xray": {
                        "sampling_rule_name": "app-a",
                        "sdk_version": "2.10.0",
                        "sdk": "X-Ray for Python"
                    }
                },
                "service": {
                    "runtime": "CPython",
                    "runtime_version": "3.9.0"
                }
            }
        }
    ]
}
@srprash srprash mentioned this issue Sep 9, 2022
Merged
@srprash
Copy link
Contributor

srprash commented Sep 9, 2022

Hi @ohke
Thanks for bringing this issue to our attention.
I have opened a PR for the fix: #353

@ohke
Copy link
Author

ohke commented Sep 12, 2022

Hi @srprash
Thank you for the fix PR of this Issue.
With master branch code, I have confirmed that it is sampled by service name.

@ohke ohke closed this as completed Sep 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ohke @srprash