DSM Processing¶

This chapter covers loading and processing DSM (Digital Substrate Model) files in Python.

The DSM Workflow¶

Processing DSM files follows three steps:

┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│  assemble   │ ──► │    parse    │ ──► │  introspect │
│  (.dsm)     │     │  (validate) │     │   (visit)   │
└─────────────┘     └─────────────┘     └─────────────┘

Step 1: Assemble¶

DSMBuilder.assemble() reads DSM files. The path can be a single .dsm file or a directory containing .dsm files; in the directory case, every file is concatenated and parsed as a single unit. The doctests below reuse the bundled Tuto fixture builder pre-loaded by the test harness:

>>> builder = _builder
>>> [p.source().split('/')[-1] for p in builder.parts()]
['model.dsm']

In production code you would assemble from your own paths:

>>> builder = DSMBuilder.assemble("model.dsm")

>>> builder = DSMBuilder.assemble("models")

>>> for part in builder.parts():
...     print(part.source())

Step 2: Parse¶

parse() validates syntax and semantics, returning three values:

>>> report, dsm_defs, defs = builder.parse()
>>> report.has_error()
False
>>> type(dsm_defs).__name__
'DSMDefinitions'
>>> type(defs).__name__
'DefinitionsConst'

Return Value	Type	Description
`report`	`DSMParseReport`	Errors and warnings
`dsm_defs`	`DSMDefinitions`	Structured DSM data (or None if errors)
`defs`	`DefinitionsConst`	Runtime definitions (or None if errors)

Handling Errors¶

>>> report, dsm_defs, defs = builder.parse()
>>> if report.has_error():
...     for error in report.errors():
...         print(f"{error.source()}:{error.line()}: {error.message()}")
... else:
...     print("Parse successful!")

Step 3: DSM Introspection¶

DSMDefinitions provides structured access to inspect the parsed model.

DSMDefinitions¶

The root container for all DSM elements:

>>> dsm_defs.concepts()
[Tuto::User]

>>> sorted(str(s) for s in dsm_defs.structures())
['Tuto::Account', 'Tuto::Identity', 'Tuto::Login', 'Tuto::Texture', 'Tuto::Thumbnail']

>>> dsm_defs.enumerations()
[Tuto::Status]

>>> sorted(str(a).split()[-1] for a in dsm_defs.attachments())
['Tuto::account', 'Tuto::avatar', 'Tuto::identity', 'Tuto::login', 'Tuto::portrait']

DSMConcept¶

Inspect concept definitions:

>>> concept = dsm_defs.concepts()[0]
>>> concept.type_name()
Tuto::User

>>> isinstance(concept.runtime_id(), ValueUUId)
True

>>> concept.documentation()
'A user.'

>>> concept.parent() is None
True

For a model with concept inheritance, parent() returns the parent concept, e.g. Tuto::Admin.

DSMStructure¶

Inspect structure definitions and their fields:

>>> struct = [s for s in dsm_defs.structures() if str(s.type_name()) == 'Tuto::Login'][0]
>>> struct.type_name()
Tuto::Login

>>> [(f.name(), str(f.type())) for f in struct.fields()]
[('nickname', 'string'), ('password', 'string')]

>>> struct.fields()[0].documentation()
''

DSMEnumeration¶

Inspect enumeration definitions and their cases:

>>> enum = dsm_defs.enumerations()[0]
>>> enum.type_name()
Tuto::Status

>>> [case.name() for case in enum.members()]
['pending', 'active', 'completed']

The Tuto fixture ties Status into a struct used by the account attachment, so the enum can be exercised end-to-end through the runtime:

>>> account = TUTO_A_USER_ACCOUNT.create_document()
>>> account
{state=.pending}

>>> account.state = ValueEnumeration(TUTO_E_STATUS, "active")
>>> account
{state=.active}

DSMAttachment¶

Inspect attachment definitions. identifier() returns the fully qualified name <concept>.<attachment>:

>>> att = dsm_defs.attachments()[0]
>>> att.identifier()
'Tuto::User.login'

>>> att.key_type()
Tuto::User

>>> att.document_type()
Tuto::Login

Generate DSM Text¶

Reconstruct DSM source from definitions. The output groups types by attachment and adds explanatory comments:

>>> source = dsm_defs.to_dsm()
>>> 'namespace Tuto' in source and 'concept User' in source
True
>>> 'attachment<User, Login> login' in source
True

Pass show_runtime_id=True to append runtime IDs.

Using Runtime Definitions¶

The DefinitionsConst from parse enables runtime operations.

Inject Constants¶

defs.inject() makes generated constants available in the namespace. The Tuto constants are already in scope thanks to the doctest fixture:

>>> TUTO_S_LOGIN
Tuto::Login

>>> TUTO_A_USER_LOGIN
attachment<User, Login> Tuto::login

Naming convention: Constants follow the pattern {NAMESPACE}_{KIND}_{NAME}:

Kind	Prefix	Example	Description
Attachment	`_A_`	`TUTO_A_USER_LOGIN`	Attachment type
Structure	`_S_`	`TUTO_S_LOGIN`	Structure type
Enumeration	`_E_`	`TUTO_E_STATUS`	Enumeration type
Concept	`_C_`	`TUTO_C_USER`	Concept type
Path	`_P_`	`TUTO_P_LOGIN_NICKNAME`	Path to field

Query Types¶

>>> types = defs.query_types("Login")
>>> types
[Tuto::Login]
>>> Value.create(types[0])
{nickname='', password=''}

Access Attachments¶

>>> for att in defs.attachments():
...     print(att.description())

Serialization¶

DSMDefinitions can be serialized for distribution.

Binary (DSMB)¶

>>> blob = dsm_defs.encode()
>>> blob
blob(...)

>>> restored = DSMDefinitions.decode(blob)
>>> type(restored).__name__
'DSMDefinitions'

JSON¶

>>> json_str = dsm_defs.json_encode()
>>> 'concepts' in json_str and 'attachments' in json_str
True

>>> restored = DSMDefinitions.json_decode(json_str)
>>> type(restored).__name__
'DSMDefinitions'

What’s Next¶

Tutorial - Complete workflow using DSM with CommitDatabase
Database - Persistence with Database and CommitDatabase
Serialization - Binary and JSON encoding