| title | linkTitle | weight | slug |
|---|---|---|---|
Table-manager |
Table-manager |
3 |
table-manager |
Warning: the chunks storage is deprecated. You're encouraged to use the blocks storage.
The table-manager is the Cortex service responsible for creating the periodic tables used to store index and chunks, and deleting them once their data time range exceeds the retention period (if retention is enabled).
For more information about the schema config and periodic tables, please refer to the Schema config documentation.
The table-manager creates new tables slightly ahead of their start period, in order to make sure that the new table is ready once the current table end period is reached. The -table-manager.periodic-table.grace-period config option defines how long before a table should be created.
The retention - managed by the table-manager - is disabled by default, due to its destructive nature. You can enable the data retention explicitly via -table-manager.retention-deletes-enabled=true and setting -table-manager.retention-period to a value greater than zero.
The table-manager implements the retention deleting the entire tables whose data exceeded the retention period. This design allows to have fast delete operations, at the cost of having a retention granularity controlled by the table's period.
Given each table contains data for period of time and that the entire table is deleted, the table-manager keeps the last tables alive using this formula:
number_of_tables_to_keep = floor(retention_period / table_period) + 1
It's important to note that - due to the internal implementation - the table period and the retention period must be multiples of 24h in order to get the expected behavior.
For more information about the table-manager configuration, refer to the config file reference.
A table can be active or inactive.
A table is considered active if the current time is within the range:
- Table start period -
-table-manager.periodic-table.grace-period - Table end period +
-ingester.max-chunk-age
Currently, the difference between an active and inactive table only applies to the DynamoDB storage settings: capacity mode (on-demand or provisioned), read/write capacity units and autoscaling.
| DynamoDB | Active table | Inactive table |
|---|---|---|
| Capacity mode | enable_ondemand_throughput_mode |
enable_inactive_throughput_on_demand_mode |
| Read capacity unit | provisioned_read_throughput |
inactive_read_throughput |
| Write capacity unit | provisioned_write_throughput |
inactive_write_throughput |
| Autoscaling | Enabled (if configured) | Always disabled |
When configuring DynamoDB with the table-manager, the default on-demand provisioning capacity units for reads are set to 300 and writes are set to 3000. The defaults can be overwritten:
table_manager:
index_tables_provisioning:
provisioned_write_throughput: 10
provisioned_read_throughput: 10
chunk_tables_provisioning:
provisioned_write_throughput: 10
provisioned_read_throughput: 10