Skip to content

Commit ccb6672

Browse files
committed
fix: better logging docs
1 parent 8531e85 commit ccb6672

File tree

1 file changed

+4
-1
lines changed

1 file changed

+4
-1
lines changed
Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,8 @@
11
= Logging
22

3-
Logs produced by SparkApplications are automatically collected by vector agents and can be aggregated as described in xref:concepts:logging.adoc[roles]
3+
The Spark operator installs a https://vector.dev/docs/setup/deployment/roles/#agent[vector agent] as a side-car container in every application Pod except the `job` Pod that runs `spark-submit`. It also configures the logging framework to output logs in XML format. This is the same https://logging.apache.org/log4j/2.x/manual/layouts.html#XMLLayout[format] used across all Stackable products and it enables the https://vector.dev/docs/setup/deployment/roles/#aggregator[vector aggregator] to collect logs across the entire platform.
4+
5+
It is the user's responsibility to install and configure the vector aggregator, but the agents can discover the aggregator automatically using a discovery ConfigMap as described in the xref:concepts:logging.adoc[logging concepts].
46

57
NOTE: Only logs produced by the application's driver and executors are collected. Logs produced by `spark-submit` are discarded.
8+

0 commit comments

Comments
 (0)