Skip to content

Commit ee35531

Browse files
jlnavjlnav
authored
Example manifests and config files - huge refactor of guide (#5)
completely refactors manifest/guide to include local examples. adds examples folders for manifests and nrmd configs. updates index, updates copyright
1 parent 6984973 commit ee35531

22 files changed

+572
-64
lines changed

‎LICENSE

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ file nrm/src/Codegen/Dhall.hs: see header.
33
file hsnrm/src/Nrm/Codegen.hs: see header.
44
other files and directories:
55

6-
Copyright (c) 2019, UChicago Argonne, LLC. All Rights Reserved.
6+
Copyright (c) 2021, UChicago Argonne, LLC. All Rights Reserved.
77

88
Redistribution and use in source and binary forms, with or without
99
modification, are permitted provided that the following conditions are met:

‎doc/conf.py

Lines changed: 3 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -12,34 +12,10 @@
1212
# add these directories to sys.path here. If the directory is relative to the
1313
# documentation root, use os.path.abspath to make it absolute, like shown here.
1414
#
15-
import os
16-
import requests
17-
outdir = './configfiles'
18-
base = 'https://raw.githubusercontent.com/anlsys/nrm-core/master/'
19-
configfiles = ['hsnrm/dhall/types/nrmd.dhall',
20-
'hsnrm/dhall/defaults/nrmd.dhall',
21-
'resources/defaults/nrmd.json',
22-
'examples/nrmd/control.yaml',
23-
'hsnrm/dhall/types/manifest.dhall',
24-
'hsnrm/dhall/defaults/manifest.dhall',
25-
'hsnrm/dhall/defaults/manifest.dhall',
26-
'resources/defaults/manifest.json',
27-
'examples/manifests/perfwrap.yaml'
28-
]
29-
30-
# Download each of the above for literal-includes in docs
31-
os.makedirs(outdir, exist_ok=True)
32-
os.chdir(outdir)
33-
for file in configfiles:
34-
if file not in os.listdir('.'):
35-
dirs = file.split('/')
36-
with open('_'.join([dirs[-2], dirs[-1]]), 'wb') as f:
37-
f.write(requests.get(base+file).content)
38-
39-
os.chdir('..')
4015

41-
# import sys
42-
# sys.path.insert(0, os.path.abspath('.'))
16+
import os
17+
import sys
18+
sys.path.append(os.path.abspath('../examples'))
4319

4420
# import os
4521
# import sys

‎doc/config.rst

Lines changed: 94 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1,47 +1,121 @@
11
NRM Configuration/Manifest Guide
22
================================
33

4+
The ``dhall`` and ``dhall-to-json`` utilities are available as convenience in
5+
this environment should you need them.
6+
7+
Types common to both daemon configuration and manifest scripts can be found here:
8+
9+
.. literalinclude:: ../examples/common_types.dhall
10+
:caption: /examples/common_types.dhall
11+
:linenos:
12+
413
Daemon Configuration
514
--------------------
615

16+
Types
17+
^^^^^
18+
719
``nrmd``'s configuration can be defined
820
in ``json``, ``yaml``, or `Dhall`_ formats. Admissible values are defined
921
via the following ``dhall`` file:
1022

11-
.. literalinclude:: ./configfiles/types_nrmd.dhall
23+
.. literalinclude:: ../examples/nrmd/nrmd_types.dhall
24+
:caption: /examples/nrmd/nrmd_types.dhall
25+
:linenos:
26+
27+
Defaults
28+
^^^^^^^^
29+
30+
Optional values are filled using defaults, here expressed in
31+
both ``dhall`` and ``json``:
32+
33+
.. literalinclude:: ../examples/nrmd/nrmd_defaults.dhall
34+
:caption: /examples/nrmd/nrmd_defaults.dhall
35+
:linenos:
36+
37+
.. literalinclude:: ../examples/nrmd/nrmd_defaults.json
38+
:caption: /examples/nrmd/nrmd_defaults.json
39+
:linenos:
40+
41+
Examples
42+
^^^^^^^^
1243

13-
Optional values are filled using defaults that can
14-
be found in either Dhall or ``json`` format:
44+
Attribute merging is always performed with these defaults. Example configurations
45+
are located in the `examples`_ folder, but we include each of them here. Note that
46+
Dhall is valid as a configuration language:
1547

16-
.. literalinclude:: ./configfiles/defaults_nrmd.dhall
48+
.. literalinclude:: ../examples/nrmd/control.yaml
49+
:caption: /examples/nrmd/control.yaml
50+
:linenos:
1751

18-
.. literalinclude:: ./configfiles/defaults_nrmd.json
52+
.. literalinclude:: ../examples/nrmd/control.dhall
53+
:caption: /examples/nrmd/control.dhall
54+
:linenos:
1955

20-
Attribute merging is always performed with those defaults. Example configurations
21-
are located in the `examples`_ folder.
56+
.. literalinclude:: ../examples/nrmd/extra-static-sensor.dhall
57+
:caption: /examples/nrmd/extra-static-sensor.dhall
58+
:linenos:
2259

23-
.. literalinclude:: ./configfiles/nrmd_control.yaml
60+
.. literalinclude:: ../examples/nrmd/extra-static-sensor.yaml
61+
:caption: /examples/nrmd/extra-static-sensor.yaml
62+
:linenos:
63+
64+
.. literalinclude:: ../examples/nrmd/extra-static-actuator.json
65+
:caption: /examples/nrmd/extra-static-actuator.json
66+
:linenos:
2467

2568
Manifest Configuration
2669
----------------------
2770

28-
Manifest files can also be defined either using Dhall, YAML, or JSON,
29-
and are in similar formats or locations:
71+
Types
72+
^^^^^
3073

31-
.. literalinclude:: ./configfiles/types_manifest.dhall
74+
Manifest files can also be defined either using Dhall, ``yaml``, or ``json``.
75+
Admissible values are defined via the following ``dhall`` file:
3276

33-
Under-specified manifests like the one in our ```workloads`` above (with missing
34-
optional fields from the schema) fill missing values with defaults, which are located
35-
here:
77+
.. literalinclude:: ../examples/manifests/manifest_types.dhall
78+
:caption: /examples/manifests/manifest_types.dhall
79+
:linenos:
3680

37-
.. literalinclude:: ./configfiles/defaults_manifest.dhall
81+
Defaults
82+
^^^^^^^^
3883

39-
The ``dhall`` and ``dhall-to-json`` utilities are available as convenience in
40-
this environment should you need them. Dhall is useful as a configuration language in itself:
84+
Under-specified manifests are also filled with defaults, specified here
85+
in both ``dhall`` and ``json``:
86+
87+
.. literalinclude:: ../examples/manifests/manifest_defaults.dhall
88+
:caption: /examples/manifests/manifest_defaults.dhall
89+
:linenos:
90+
91+
.. literalinclude:: ../examples/manifests/manifest_defaults.json
92+
:caption: /examples/manifests/manifest_defaults.json
93+
:linenos:
94+
95+
Examples
96+
^^^^^^^^
97+
98+
The following could be used with the `libnrm`_ interface:
99+
100+
.. literalinclude:: ../examples/manifests/libnrm.dhall
101+
:caption: /examples/manifests/libnrm.dhall
102+
:linenos:
103+
104+
.. literalinclude:: ../examples/manifests/libnrm.json
105+
:caption: /examples/manifests/libnrm.json
106+
:linenos:
107+
108+
The following could be used with the `perf`_ utility:
41109

42-
.. literalinclude:: ./configfiles/defaults_manifest.json
110+
.. literalinclude:: ../examples/manifests/perfwrap.dhall
111+
:caption: /examples/manifests/perfwrap.dhall
112+
:linenos:
43113

44-
.. literalinclude:: ./configfiles/manifests_perfwrap.yaml
114+
.. literalinclude:: ../examples/manifests/perfwrap.yaml
115+
:caption: /examples/manifests/perfwrap.yaml
116+
:linenos:
45117

46118
.. _`Dhall`: https://dhall-lang.org/
47-
.. _`examples`: https://github.com/anlsys/nrm-core/tree/master/examples
119+
.. _`examples`: https://github.com/anlsys/nrm-docs/tree/main/examples
120+
.. _`libnrm`: https://github.com/anlsys/libnrm
121+
.. _`perf`: https://www.man7.org/linux/man-pages/man1/perf.1.html

‎doc/index.rst

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -44,12 +44,6 @@ and the notebooks under :doc:`NRM-Python<nrm-python:index>` for python upstream
4444
NRM-Core <https://nrm.readthedocs.io/projects/nrm-core/en/master/>
4545

4646

47-
.. toctree::
48-
:maxdepth: 2
49-
:caption: Additional References:
50-
51-
release-notes
52-
5347
Indices and tables
5448
==================
5549

‎doc/quickstart.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -189,4 +189,4 @@ Set a node power target::
189189

190190

191191
.. _Singularity: https://singularity.lbl.gov/install-request
192-
.. _ GitHub: https://github.com/anlsys/nrm-core/releases
192+
.. _GitHub: https://github.com/anlsys/nrm-core/releases

‎doc/release-notes.rst

Lines changed: 0 additions & 9 deletions
This file was deleted.

‎examples/common_types.dhall

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
let Power =
2+
-- Base unit: power.
3+
{ microwatts : Double }
4+
5+
let Time =
6+
-- Base unit: time.
7+
{ microseconds : Double }
8+
9+
let Frequency =
10+
--Base unit: frequency.
11+
{ hertz : Double }
12+
13+
in { Frequency = Frequency, Time = Time, Power = Power }

‎examples/manifests/libnrm.dhall

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
let types = ./types_manifest.dhall
2+
3+
let default = ./defaults_manifest.dhall
4+
5+
in default
6+
⫽ { name = "libnrm-wrapped"
7+
, app =
8+
default.app
9+
⫽ { instrumentation = Some { ratelimit.hertz = 1000000.0 } }
10+
}
11+
: types.Manifest

‎examples/manifests/libnrm.json

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
{
2+
"name": "libnrm-wrapped",
3+
"app": {
4+
"instrumentation": {
5+
"ratelimit": {
6+
"hertz": 1000000
7+
}
8+
}
9+
}
10+
}

‎examples/manifests/manifest_defaults.dhall

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
-- ******************************************************************************
2+
-- Copyright 2021 UChicago Argonne, LLC.
3+
-- (c.f. AUTHORS, LICENSE)
4+
--
5+
-- SPDX-License-Identifier: BSD-3-Clause
6+
-- ******************************************************************************
7+
8+
let t = ./types_manifest.dhall
9+
10+
in { name = "default"
11+
, app =
12+
{ perfwrapper = None t.Perfwrapper
13+
, instrumentation = None t.Instrumentation
14+
, actuators = None (List t.AppActuatorKV)
15+
}
16+
}

‎examples/manifests/manifest_defaults.json

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
{
2+
"app": {
3+
"instrumentation": null,
4+
"perfwrapper": null
5+
},
6+
"name": "default"
7+
}

‎examples/manifests/manifest_types.dhall

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
-- ******************************************************************************
2+
-- Copyright 2021 UChicago Argonne, LLC.
3+
-- (c.f. AUTHORS, LICENSE)
4+
--
5+
-- SPDX-License-Identifier: BSD-3-Clause
6+
-- ******************************************************************************
7+
let types =
8+
-- This import defines a few base types that are common to manifest and
9+
-- configuration formats.
10+
../common_types.dhall
11+
12+
let Perfwrapper =
13+
-- Configuration for linux perf performance measurements. A fixed frequency
14+
-- is provided, along with a tentative upper bound on the maximum value of
15+
-- the sensor. This upper bound should be set to a low positive value if
16+
-- no a-priori information is known; NRM will then use a recursive-doubling
17+
-- strategy to maintain its own bound.
18+
{ perfFreq : types.Frequency
19+
, perfLimit : Integer
20+
, perfEvent : Text
21+
}
22+
23+
let Instrumentation =
24+
-- Activate LD_PRELOAD based `libnrm` instrumentation. The only attribute that configures this feature
25+
-- is a message rate limitation.
26+
{ ratelimit : types.Frequency }
27+
28+
let EnvVar =
29+
-- Key-value representation of environment variables
30+
{ envName : Text, envValue : Text }
31+
32+
let AppActuator =
33+
-- Configuration for an arbitrary actuator.
34+
{ actuatorBinary : Text
35+
, actuatorArguments : List Text
36+
, actuatorEnv : List EnvVar
37+
, actions : List Double
38+
, referenceAction : Double
39+
}
40+
41+
let AppActuatorKV =
42+
-- Key-value representation for an actuator.
43+
{ actuatorID : Text, actuator : AppActuator }
44+
45+
let App =
46+
-- Application configuration. Two features can be enabled or disabled:
47+
-- perfwrapper: an optional linux perf configuration
48+
-- instrumentation: an optional libnrm instrumentation configuration
49+
{ perfwrapper : Optional Perfwrapper
50+
, instrumentation : Optional Instrumentation
51+
, actuators : Optional (List AppActuatorKV)
52+
}
53+
54+
let Manifest =
55+
-- A manifest has a name, and an application configuration.
56+
{ name : Text, app : App }
57+
58+
in types
59+
⫽ { Perfwrapper = Perfwrapper
60+
, Instrumentation = Instrumentation
61+
, App = App
62+
, Manifest = Manifest
63+
, AppActuator = AppActuator
64+
, AppActuatorKV = AppActuatorKV
65+
, EnvVar = EnvVar
66+
}

‎examples/manifests/perfwrap.dhall

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
let types = ./types_manifest.dhall
2+
3+
let default = ./defaults_manifest.dhall
4+
5+
in default
6+
⫽ { name = "linuxperf-wrapped"
7+
, app =
8+
default.app
9+
⫽ { perfwrapper = Some { perfFreq.hertz = 1.0,
10+
perfLimit = +100000,
11+
perfEvent = "instructions" }
12+
}
13+
}
14+
: types.Manifest

‎examples/manifests/perfwrap.yaml

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
name: linuxperf-wrapped
2+
app:
3+
perfwrapper:
4+
perfFreq:
5+
hertz: 1
6+
perfLimit: 100000
7+
perfEvent: "instructions"

‎examples/nrmd/control.dhall

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
let types = ./types_nrmd.dhall
2+
3+
let default = ./defaults_nrmd.dhall
4+
5+
let action1 = { microwatts = 1.0e8 }
6+
7+
let action2 = { microwatts = 2.0e8 }
8+
9+
in default
10+
⫽ { raplCfg = Some
11+
{ raplPath = "/sys/devices/virtual/powercap/intel-rapl"
12+
, raplActions = [ action1, action2 ]
13+
, referencePower.microwatts = 2.5e8
14+
}
15+
, controlCfg =
16+
types.ControlCfg.ControlCfg
17+
{ minimumControlInterval.microseconds = 100000.0
18+
, minimumWaitInterval.microseconds = 100000.0
19+
, staticPower.microwatts = 2.0e8
20+
, learnCfg = types.LearnCfg.Contextual { horizon = +4000 }
21+
, speedThreshold = 1.1
22+
, referenceMeasurementRoundInterval = +6
23+
, hint = types.Hint.Full
24+
}
25+
, passiveSensorFrequency.hertz = 1.0
26+
}
27+
: types.Cfg

0 commit comments

Comments
 (0)