Reference: the CTLV specification language

Complete definition of the YAML inputs consumed by tools/ctlv.

1. Files

PathRole
Documentation/ctlv/ctlv-schema.yamlJSON Schema (draft 7) for a family specification.
Documentation/ctlv/ctlv-registry-schema.yamlJSON Schema for the family registry.
Documentation/ctlv/generated-files-schema.yamlJSON Schema for the output manifest.
Documentation/ctlv/specs/<family>.yamlSpecification of one family. The file stem is the family name.
Documentation/ctlv/families.yamlFamily registry.
Documentation/ctlv/generated-files.yamlOutput manifest.
Documentation/ctlv/abi/<family>.yamlABI snapshot of one family.
Documentation/ctlv/abi/families.registry.yamlABI baseline of the registry.
tools/ctlv/ctlvgen/cli.pyValidator, generator and checker entry point.
tools/ctlv/ctlv-regen.shRegenerates or checks every manifest output.
tools/ctlv/MakefileMake targets.

The specification of the root family ctlv is located relative to the meta-schema (specs/ctlv.yaml beside it). The specification of every other family is located in the directory of the specification being loaded.

Every registry entry has exactly one specification and one ABI snapshot. A specification or snapshot without a registry entry is an error.

2. Document syntax

YAML subset

Each file is a single YAML document whose top level is a mapping. The loader is yaml.SafeLoader with these additional rejections:

Required first line

The first line of a specification and of a registry is a comment matching #\s*SPDX-License-Identifier:\s*(\S.*?)\s*$. The captured identifier is the license of every generated file derived from the file.

Validation order

  1. YAML subset load.
  2. Meta-schema validation (syntax gate).
  3. Per-declaration normalization (spec.py).
  4. Chain resolution and cross-declaration checks (semantics.py).
  5. Generated-name collision check (cgen.symbols).

Every unknown key is an error at every level (additionalProperties: false).

3. Names and selectors

KindPatternAdditional constraint
name^[a-z0-9]+(-[a-z0-9]+)*$
family namenameAt most 63 characters.
selector^[a-z0-9]+(-[a-z0-9]+)*(\.[a-z0-9]+(-[a-z0-9]+)*)?$At most one dot.

A dot is reserved for selectors. A name does not contain one.

Selector forms

FormSelects
group.attrThe one attr attr of group group.
name (a group)Every attr of the group, in declaration order, as required placements unless selected optionally.
name (an attr set)The required and optional members of the set.

A selector without a dot is resolved as a group first and as an attr set second. Group names and attr set names share one namespace and are never equal. There are no tuple selectors and no family-qualified selectors.

A field typed selectors accepts one selector string or a non-empty array of selector strings.

Namespaces

NamespaceScopeMembers
DefinitionsInheritance chainenum names and flags names, together
Groups and setsInheritance chainattr group names and attr set names, together
AttrsOne groupattr names
OpsInheritance chainop names
ValuesOne enum or flags definitionvalue names

Redeclaring a name in the same namespace is an error, except that a descendant declares an inherited stub attr set once to fill it (stubs).

4. family

Required. Mapping with these keys.

KeyTypeDefaultDefinition
namefamily namerequiredRegistered name. Equals the file stem of the specification.
inheritsfamily namenoneThe single direct parent.
pre-initbooleanfalseThe family implements the generated pre-init callback, which translates its init information for its parent. Without it, the parent receives NULL.
post-initbooleanfalseThe family implements the generated post-init callback, run after every family priv is initialized.
pre-finibooleanfalseThe family implements the generated pre-fini callback, run before any family priv is finalized.

Rules

5. enums and flags

Each of enums and flags is an optional array with at least one element. An element is a mapping:

KeyTypeDefinition
namenameDefinition name.
valuesarray of name, at least oneValue names in declaration order. Names are unique within the definition.
enumflags
NumberingFrom 1, in declaration order. 0 is reserved.Bit 0 upward, in declaration order.
Maximum values429496729564
PayloadUnsigned 64-bitUnsigned 64-bit mask
MutationAppend-only.Append-only.

An attr references a definition by name with enum or flags. The referenced definition is declared in the same family or an ancestor, and has the matching kind.

6. attr-groups

Optional array, 1 to 65535 elements per family. An element is a mapping:

KeyTypeDefinition
namenameGroup name. Also the generated symbol prefix of its attrs.
attrsarray of attr, 1 to 4294967295Attr declarations in ID order. Attr names are unique within the group.

Attr keys

An attr declares name and exactly one primary form key out of type, enum, flags, attrs, blob. Zero or more than one primary form keys is an error.

KeyTypeApplies to
namenameall
typeuint | sint | bool | string | binaryprimary form
enumname of an enumprimary form
flagsname of a flags definitionprimary form
attrsselectorsprimary form (nest)
blobblob mappingprimary form
max-occurrencesinteger 1 to 4294967295, or unlimitedall
min, maxintegeruint, sint
unit, scaleunit, integeruint, sint
lengthinteger 0 to 4294967295binary
min-length, max-lengthinteger 0 to 4294967295string, binary
optional-attrsselectorsnest
recursivetruenest

A key that does not apply to the attr's form is an error. max-occurrences defaults to 1.

Payload forms

FormPayloadMaximum payload length
uint, sint, enum, flags64-bit integer8
boolOne byte, 0 or 11
stringCharacters and a terminating NULmax-length + 1
binaryBytesmax-length
nestChild attrsSum of the children
blobDescriptor nest3 × 24 for backing va

Integer attrs

KeyDefaultDefinition
mintype minimumInclusive lower bound.
maxtype maximumInclusive upper bound.
unitnoneOne of seconds, hertz, bytes, celsius.
scale0 when unit is presentDecimal exponent, −128 to 127. Requires unit.
TypeRange
uint0 to 18446744073709551615
sint−9223372036854775808 to 9223372036854775807

min and max lie in the type's range, and min ≤ max.

String and binary attrs

FormKeys
stringmax-length is required. min-length defaults to 0. Lengths exclude the terminator. min-length ≤ max-length.
binaryEither length alone, which sets minimum and maximum to the same value, or max-length with an optional min-length (default 0). length is not combined with min-length or max-length. One of length and max-length is required. min-length ≤ max-length.

Nest attrs

KeyDefinition
attrsSelectors of the required members.
optional-attrsSelectors of the optional members.
recursiveThe nest may contain its own attr again at any depth.

Recursive nests

Blob attrs

KeyTypeDefaultDefinition
directioninput | output | inoutrequiredHow the family uses the described memory.
allowed-backingsnon-empty unique array of varequiredAccepted backing types. va is the only value.
max-lengthinteger 1 to 18446744073709551615requiredMaximum external payload length. Independent of the inline message limit.
alignmentinteger 1 to 42949672951Required alignment of the described address or offset.

7. attr-sets

Optional array. An element is a mapping:

KeyTypeDefinition
namenameSet name.
stubtrueDeclares an extension point.
attrsselectorsRequired members.
optional-attrsselectorsOptional members.

Stubs

8. ops

Optional array, at most 4294967295 elements per family. An element is a mapping:

KeyTypeDefaultApplies toDefinition
namenamerequiredallOp name.
typeaction | query | eventrequiredallAn action is invoked once and does not fragment. A query may fragment its reply. An event is an unsolicited reply-only message.
fd-accessread | read-writeread-writeaction, queryread is executable through an observation fd. read-write requires a full-control fd. Events are always readable.
availabilityfixed | dynamicfixedaction, querydynamic permits the family to enable and disable the op per device. Every device starts with a dynamic op enabled. A disabled op remains in introspection and returns -EOPNOTSUPP.
optionalbooleanfalseaction, queryWhen true, the op exists on a device only if family initialization opts in before publication. An op not opted in is absent from introspection.
pre-postbooleanfalseaction, queryThe op has a generated pre/post hook pair implemented by its declaring family. Kernel implementation metadata, absent from UAPI and introspection.
requestplacementsroot with no childrenaction, queryRequest root.
replyplacementssee belowallReply root.
suggested-reply-lengthinteger 1 to 1048576see lengthsqueryInitial reply buffer length.

Placements

A mapping with at least one of the keys attrs and optional-attrs, each holding selectors. An empty mapping is an error.

Per-type rules

actionqueryevent
requestallowedallowednot allowed
reply omittedacknowledgement onlynot allowedroot with no children
fd-accessallowedallowednot allowed
availabilityallowedallowednot allowed
optionalallowedallowednot allowed
pre-postallowedallowednot allowed
suggested-reply-lengthnot allowedallowednot allowed

Message contexts

Constructrequestaction replyquery replyevent
max-occurrences: unlimitednonoyesno
re-entry of a recursive nestnonoyesno
blob attryesnonono

9. Composition rules

Expansion of a selection

Stub selection

Structural checks

10. Identifiers

IdentifierValue
Family IDZero-based index of the family name in families.yaml.
Group ID1-based position of the group in the family's attr-groups.
Local attr ID1-based position of the attr in its group.
Attr ID(family ID << 48) | (group ID << 32) | local attr ID
Local op ID1-based position of the op in the family's ops.
Op ID(family ID << 48) | local op ID
Set ID1-based counter per family; see attr-sets.
Enum value1-based position in values.
Flag bit0-based position in values.

An ID is part of the ABI for the life of the family.

11. Lengths

Constants

NameValue
Alignment (CTLV_ALIGNTO)8
Attr header length16
Event record header length24
CTLV_MAX_INLINE_MESSAGE_LEN0x100000
CTLV_MAX_ERROR_LEN0x4000
CTLV_MAX_NEST_DEPTH32
CTLV_DEFAULT_QUERY_REPLY_LEN0x10000
CTLV_MAX_EVENT_QUEUE_LEN0x1000000

The declared nest depth limit is CTLV_MAX_NEST_DEPTH − 1 = 31.

Formulas

align(n) rounds n up to a multiple of 8.

QuantityDefinition
Attr maximum length16 + payload maximum length
Root maximum length16 + sum of children
Children sumΣ max-occurrences × align(attr maximum length) over members
Blob payload maximum length(largest descriptor attr count over allowed-backings) × align(24); va counts 3
request-max-lenRoot maximum length of the request.
action-reply-max-lenRoot maximum length of the reply of an action with a reply.
event-record-max-len24 + align(root maximum length of the event reply)
query-reply-min-lenMaximum over the leaves of the reply tree of 16 × (number of ancestors including the root) + align(leaf attr maximum length). A tree that contains a re-entry uses 33 × 16 + align(attr maximum length) of the largest non-nest placement.
query-reply-suggested-lenWith suggested-reply-length declared: that value. Otherwise the larger of CTLV_DEFAULT_QUERY_REPLY_LEN and query-reply-min-len. For a composed family, the larger of the declared value and the composed minimum.

Checks

12. Limits

ItemLimit
Family name length63 characters
Registry entries65536 (family IDs 0 to 65535)
Attr groups per family65535
Attrs per group4294967295
Attr sets per family4294967295 new sets
Ops per family4294967295
Effective ops of a chain4294967295
Enum values4294967295
Flag bits64
max-occurrences4294967295
Declared nest depth31
Composed required attrs per level4294967295
Composed optional attrs per level4294967295
Schema nodes per tree4294967295
Stub references per tree4294967295
Composed schema nodes per op (request plus reply)4294967295
Stub fills per family4294967295
suggested-reply-length1 to 1048576

13. Family registry

Documentation/ctlv/families.yaml. Top-level key families: an array of 1 to 65536 elements.

Element formMeaning
name string (at most 63 characters, name pattern)A registered family.
mapping with name and deprecated: trueA deprecated family. Its ID stays reserved.

14. Output manifest

Documentation/ctlv/generated-files.yaml. Root keys, all required:

KeyTypeDefinition
version1Manifest version.
registrymappingsource (path of the registry) and outputs.uapi-header (path, required).
familiesarray, at least oneOne mapping per generated family.

A families element:

KeyRequiredDefinition
nameyesFamily name.
sourceyesPath of the specification.
outputsyesMapping with at least one of uapi-header, kernel-header, kernel-source.

Paths are non-empty, relative to the kernel tree root, and do not begin with /.

15. ABI snapshots

A snapshot records one family's own declarations: family flags, ID and parent; definitions with value numbers; groups with attr IDs and constraints; sets; stub fills; and ops with their request and reply trees. It is written by --abi-update and compared by --abi-check.

Compared members

DeclarationRule
FamilyName, ID, parent, pre-init, post-init, pre-fini are unchanged.
Enum, flagsNot removed. Kind unchanged. Each existing value remains present with the same number.
Attr groupNot removed. Group ID unchanged.
AttrNot removed. Local ID, kind, referenced definition, unit and scale unchanged.
Attr boundsmin and min-length do not increase. max and max-length do not decrease.
max-occurrencesDoes not decrease, and does not change from unlimited to a finite value.
Blobdirection and alignment unchanged. No allowed-backings removed. max-length does not decrease.
Attr setNot removed. Set ID and stub unchanged.
Stub fillNot removed. Compared as a node list.
OpNot removed. op-id, type, fd-access, availability, optional and pre-post unchanged.
Op request, replyPresent schema not removed. An action without a reply does not gain one. Compared as a node list.
Node listNo node removed. No node changes from optional to required. reenters unchanged. A node added since the snapshot is optional.

Adding an attr, group, set, op, enum value or flag bit, or making a required placement optional, is compatible.

Registry baseline

abi/families.registry.yaml records the ordered registry. The check fails when:

16. Generated names

Let F be the family name with dashes replaced by underscores (upper case in macros), and X-Y a YAML name rendered as X_Y.

DeclarationName
Family numberCTLV_FAMILY_<S>, where S is the family name without a leading ctlv-; CTLV_FAMILY_ROOT for ctlv
Group<F>_GROUP_<GROUP>
Attr<F>_ATTR_<GROUP>_<ATTR> (complete ID)
Set<F>_SET_<SET>
Op<F>_OP_<OP> (complete ID)
Enum value<F>_<ENUM>_<VALUE>
Flag bit<F>_<FLAGS>_<VALUE>_BIT
Flag mask<F>_<FLAGS>_<VALUE> and <F>_<FLAGS>_MASK

In the root family, F is CTLV.

17. Tooling

Dependencies: pyyaml 6.*, jsonschema 4.*.

Make targets

Run as make -C tools/ctlv <target>.

TargetAction
allvalidate and abi-check.
validateValidates every specification in specs/.
abi-checkChecks every specification against its snapshot.
abi-updateRewrites every snapshot.
regenRuns ctlv-regen.sh, which writes every manifest output.
check-regenFails when a checked-in output differs from generator output. Writes nothing.
lintRuns yamllint, pycodestyle and pyflakes.
cleanRemoves Python caches and build directories.

cli.py options

--spec PATH is required. Exactly one action is required.

OptionDefinition
--spec PATHSpecification to load.
--schema PATHMeta-schema. Default: ctlv-schema.yaml beside specs/.
--registry PATHRegistry. Default: families.yaml beside specs/.
--abi DIRSnapshot directory. Default: abi/ beside specs/.
--validateValidates the specification and its chain, and the generated-name check.
--abi-checkChecks the registry baseline and the family snapshot.
--abi-updateRewrites the family snapshot.
--tree-checkChecks registry, specifications, snapshots and manifest coverage.
--tree-check-updateChecks whole-tree inputs before an ABI update. Snapshots for appended registry entries may be missing.
--registry-abi-updateRewrites the registry baseline. Requires complete snapshots.
--outputs-checkChecks every manifest output.
--outputs-updateRegenerates every manifest output.
--mode uapi|kernelGenerates one file. Requires --header or --source.
--header, --sourceSelects the file kind of --mode. Mutually exclusive.
--schema-onlyWith --mode kernel, omits the family API.
-o, --output PATHWrites to a file instead of standard output. Requires --mode.
--cmp-outLeaves an unchanged output file untouched. Requires --mode and --output.
--output-root DIRAlternate destination root. Requires --outputs-check or --outputs-update. Not empty.

Exit status is 0 on success and 1 on a reported error. Each error is printed as ctlvgen: <location>: <message>.

ctlv-regen.sh

OptionDefinition
-fAccepted and ignored.
-cWrites nothing and fails on any difference.
-p DIRUses DIR as the output root. Not empty.

Whole-tree coverage

18. Root family

Family ctlv, family ID 0, specified in specs/ctlv.yaml. Its operations are inherited by every family. Its attr groups are placed inside the messages of other families.

Requirements on the root specification

Enums

EnumValues (1, 2, …)
op-typeaction, query, event
op-availabilityfixed, dynamic
blob-typeva
blob-directioninput, output, inout
validation-reasonmalformed-message, unknown-attr, unknown-flag, nesting-too-deep, missing-attr, too-many-occurrences, value-out-of-range, bad-length, bad-string, bad-blob-descriptor, stale-generation, noncontiguous-attr
op-fd-accessread, read-write

Attr groups

GroupAttrs
family (1)id, name (string, max-length 63), entry (nest of set family-entry, unlimited)
blob (2)type (enum blob-type), address, fd (max 2147483647), offset, length (min 1)
error (3)reason (enum validation-reason), message (string, max-length 255), bad-attr-offset, nest-offset, missing-attr-id, expected-generation, current-generation
op (4)id (min 1), type, availability, disabled (bool), request-max-len, action-reply-max-len, query-reply-min-len, query-reply-suggested-len, event-record-max-len, state-generation, entry (nest of op-entry, unlimited), fd-access
schema (5)attr-id, kind, max-occurrences, min, max, min-length, max-length, flags-mask, unit, scale (sint), blob-direction, blob-backings, blob-max-length, blob-alignment, reenters (min 1), attrs and optional-attrs (recursive nests of schema-node, unlimited), request, reply (nests of schema-node)
event (6)op-id (min 1, max-occurrences 256), queue-len (unit bytes, 1 to 16777216), queue-used (unit bytes), first-sequence, last-sequence, dropped-count

Unless noted, an attr is an unsigned integer (uint) with no bounds.

Attr sets

SetRequiredOptional
family-entryfamily.id, family.name
op-entryop.id, op.typeop.availability, op.disabled, op.request-max-len, op.action-reply-max-len, op.query-reply-min-len, op.query-reply-suggested-len, op.event-record-max-len, op.fd-access
op-schema-requestop.id
schema-nodeschema.attr-id, schema.kind, schema.max-occurrencesschema.min, schema.max, schema.min-length, schema.max-length, schema.flags-mask, schema.unit, schema.scale, schema.blob-direction, schema.blob-backings, schema.blob-max-length, schema.blob-alignment, schema.reenters, schema.attrs, schema.optional-attrs
op-schema-infoschema.request, schema.reply
op-change-infoop.id, op.disabled, op.state-generation
error-infoevery error.* attr
event-subscriptionevent.op-id
event-queue-limitevent.queue-len
event-queue-stateevent.queue-len, event.queue-used
event-overflow-infoevent.first-sequence, event.last-sequence, event.dropped-count

Ops

IDOpTypefd-accessRequestReply
1families-dumpqueryreadfamily.entry
2ops-dumpqueryreadop.entry
3op-schema-getqueryreadop-schema-requestop-schema-info
4subscriptions-addactionreadevent-subscriptionacknowledgement
5subscriptions-removeactionreadevent-subscriptionacknowledgement
6event-queue-getqueryreadevent-queue-state
7event-queue-setactionreadevent-queue-limitacknowledgement
8op-changedeventop-change-info
9event-overfloweventevent-overflow-info
10device-removedeventno children