Processing
processing
Processing a feature collection by rules that are kept as data.
Each subpackage applies one rule file from xmas_core/resources to a feature collection:
transform migrates it between XPlanung versions, and
style derives stylesheetId and schriftinhalt for its
presentational objects. common reads and writes those files.
Nothing is imported here, so that reaching for one subpackage does not load the other's rules.
common
Reading and writing the rule files that transform and style are driven by.
Both rule files are YAML written by hand, and both are read the same way: through
load_yaml, which refuses what PyYAML would
otherwise read silently as something other than what was written.
The rest of this module is the part of scripts/normalize_migration_rules.py and
scripts/normalize_style_rules.py that does not depend on the document. A normalizer validates
the file into its model, has its own write turn the model back into YAML, and keeps the
result only if it validates to the same thing. It writes that result through
dump, so both files look alike:
- The comment block at the top of the file is kept as it is. A YAML comment anywhere below it
is refused, since it would not survive being written back; notes are
comment:fields. - Lists are indented under their key, strings are double-quoted where they need quoting, and an object shared by several places is anchored once under a name chosen by the caller.
Comment
Bases: str
A note, written folded when it does not fit on its key's line.
NormalizeError
Bases: Exception
The file cannot be normalized as it is.
RuleFileError
Bases: ValueError
A rule file does not describe the rules it claims to.
UniqueKeyLoader
Bases: SafeLoader
A loader that refuses a duplicate mapping key.
PyYAML keeps the last of a repeated key, which is the same silent shadowing the method dispatch had: the 4.1 rules defined four feature types twice each and only the later definition ever ran.
construct_mapping(node, deep=False)
Build the mapping, raising on any key that appears twice, or a merge key.
A merge key (<<: *template) is refused rather than resolved: a template is a whole
operation, rule or clause, used through a plain alias, so an operation always states
every field it has where it is written.
Source code in xmas_core/processing/common.py
dump(node, names=None)
Write node in the rule files' style.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
node
|
Any
|
Plain data: dicts, lists, strings, numbers and |
required |
names
|
dict[int, str] | None
|
Anchor names by |
None
|
Returns:
| Type | Description |
|---|---|
str
|
The YAML text. |
Source code in xmas_core/processing/common.py
fold_comments(node)
Mark every comment value below node as a Comment, in place.
In place, so an object shared by several places stays one object and keeps its anchor.
Source code in xmas_core/processing/common.py
load_yaml(text)
Read a rule file's YAML.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
text
|
str
|
The file's contents. |
required |
Raises:
| Type | Description |
|---|---|
RuleFileError
|
A key is written twice, is a merge key or reads as a boolean. |
YAMLError
|
|
Returns:
| Type | Description |
|---|---|
Any
|
The document, with every alias resolved to the object its anchor names. |
Source code in xmas_core/processing/common.py
normalize_document(text, *, validate, write, same=operator.eq)
Return text in canonical form, having checked it still says the same thing.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
text
|
str
|
The rule file's contents. |
required |
validate
|
Callable[[Any], D]
|
Builds the document's model from the loaded YAML. |
required |
write
|
Callable[[Any, D], str]
|
Writes the file below its header, given the loaded YAML and the model. |
required |
same
|
Callable[[D, D], bool]
|
Whether two models say the same thing. Equality unless the model holds something that normalizing is allowed to change. |
eq
|
Raises:
| Type | Description |
|---|---|
NormalizeError
|
The file is not valid, holds a YAML comment below its header, or would no longer say the same thing once written. |
Returns:
| Type | Description |
|---|---|
str
|
The canonical text, the header comment block included. |
Source code in xmas_core/processing/common.py
run_normalizer(argv, *, normalize, default, description)
Normalize the given files in place, or report with --check.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
argv
|
Sequence[str] | None
|
The command line, without the program name; None reads |
required |
normalize
|
Callable[[str], str]
|
Turns a file's text into its canonical form. |
required |
default
|
Path
|
The file to normalize when none is given. |
required |
description
|
str
|
The first line of the script's help. |
required |
Returns:
| Type | Description |
|---|---|
int
|
0 when nothing changed, 1 when a file was (or with |
int
|
2 on an error. |
Source code in xmas_core/processing/common.py
transform
Migrating a feature collection from one XPlanung version to the next.
result = migrate(collection, "6.1")
result.collection # the migrated collection
result.report.by_feature() # what changed, keyed by feature id
migrate walks the whole path - 4.1 to 6.1 is three hops -
and one report covers all of it. Every hop's rules live in one file,
resources/migration/rules.yaml, as one entry of its migrations: list.
MigrationReport
Bases: BaseModel
Every issue raised while migrating a collection, across all hops.
errors
property
The issues that failed the migration.
ok
property
Whether the migration produced no errors.
warnings
property
The issues the migration continued past.
add(severity, code, message, **fields)
Append an issue. Unknown keyword arguments land in meta.
Source code in xmas_core/processing/transform/report.py
by_feature()
Group the issues by the feature they belong to, in the order they were raised.
Issues with no feature - a collection-level failure - are not in the result.
Source code in xmas_core/processing/transform/report.py
error(code, message, **fields)
Record an error. An error aborts the migration once the current phase ends.
warning(code, message, **fields)
MigrationResult(collection, report)
dataclass
What migrate returns: the collection and the report.
A dataclass rather than a model, because the collection is the payload and nothing
should be tempted to serialise the two together - report dumps on its own.
migrate(collection, to_version, *, report=None)
Migrate a collection to to_version, however many hops that takes.
One report covers the whole run: every issue records the hop that raised it, so a 4.1 -> 6.1 migration still says which step dropped an attribute.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
collection
|
FeatureCollection
|
The collection to migrate. |
required |
to_version
|
str
|
The wanted XPlanung version. |
required |
report
|
MigrationReport | None
|
An existing report to collect into, if the caller is already holding one. |
None
|
Raises:
| Type | Description |
|---|---|
ValueError
|
|
MigrationError
|
A hop failed; the report says which features and why. |
UnsupportedFeatureError
|
The collection holds a feature with no mapping at all. |
Returns:
| Type | Description |
|---|---|
MigrationResult
|
The migrated collection and the report. |
Source code in xmas_core/processing/transform/__init__.py
migration_path(from_version, to_version)
The versions to migrate through to get from from_version to to_version.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
from_version
|
str
|
The collection's current version. |
required |
to_version
|
str
|
The wanted version. |
required |
Raises:
| Type | Description |
|---|---|
ValueError
|
|
Returns:
| Type | Description |
|---|---|
list[str]
|
The intermediate targets, in order, excluding |
list[str]
|
collection is already at |
Source code in xmas_core/processing/transform/__init__.py
report
The report a migration produces, and the error raised when one fails.
A migration is expected to change data in ways a downstream application has to look at:
an attribute the target version no longer has, a value with no equivalent code, a whole
feature the schema dropped. Every such finding is an Issue
carrying the feature_id it belongs to, so the consumer can put it next to the feature
rather than parse it out of a log line:
result = migrate(collection, "6.1")
for feature_id, issues in result.report.by_feature().items():
...
One report covers the whole run. A 4.1 -> 6.1 migration is three hops, and each issue
records the step that raised it, so the report stays legible after the collection has
been through all of them.
Issue
Bases: BaseModel
One finding about one feature.
code serialises to its name ("ATTRIBUTE_DROPPED"), the machine-readable token
xmas_core.codes documents consumers to branch on, and validates
back from it.
MigrationError(report)
Bases: BaseError
A migration failed. report holds every issue, including the errors that stopped it.
Carry the report of the run that failed.
Source code in xmas_core/processing/transform/report.py
MigrationReport
Bases: BaseModel
Every issue raised while migrating a collection, across all hops.
errors
property
The issues that failed the migration.
ok
property
Whether the migration produced no errors.
warnings
property
The issues the migration continued past.
add(severity, code, message, **fields)
Append an issue. Unknown keyword arguments land in meta.
Source code in xmas_core/processing/transform/report.py
by_feature()
Group the issues by the feature they belong to, in the order they were raised.
Issues with no feature - a collection-level failure - are not in the result.
Source code in xmas_core/processing/transform/report.py
error(code, message, **fields)
Record an error. An error aborts the migration once the current phase ends.
warning(code, message, **fields)
MigrationResult(collection, report)
dataclass
What migrate returns: the collection and the report.
A dataclass rather than a model, because the collection is the payload and nothing
should be tempted to serialise the two together - report dumps on its own.
ops
The two vocabularies a migration is written in.
A rule operation (single) works on one feature: it is
handed the plain dict that feature was dumped to and nothing else, so it cannot reach
another feature even by accident. A collection operation
(collection) is the exception, and it is a separate
module rather than a flag: pull and push are the only two, they run in their own phase,
and keeping them apart is what makes the rule vocabulary safe to read.
Both are data. Every operation is a pydantic model built from
resources/migration/rules.yaml, the vocabulary is closed, and there is no escape hatch back
into Python.
RuleContext(report, step, migration_record=dict(), removed=set(), feature=None, target_type=None, plan_prefix=None)
dataclass
What an operation is handed besides the feature dict.
One instance per hop. It deliberately does not hold the collection: a rule operates on one feature, and the phase that may look at others gets a context of its own.
feature = None
class-attribute
instance-attribute
The feature currently being migrated, as (id, type); set by the engine per feature.
A data type has no id of its own, so a rule that warns about one would otherwise produce an issue no consumer could attach to anything. This is what it falls back to.
plan_prefix = None
class-attribute
instance-attribute
The document's plan type (BP, FP, ...), computed once per hop by the engine.
A property of the collection rather than of any rule, which is why it is resolved here
for the by_plan of set and retype rather than stated in the rule file.
target_type = None
class-attribute
instance-attribute
The feature type the current feature will be validated as; retype writes it.
It is the feature's identity rather than one of its values, so it lives here instead of in the dict the operations mutate - where it sat beside the real attributes, could be read by a guard as though the feature declared such a property, and had to be popped back out before the dict could be validated. A data type has no type of its own here, so this is None while a rule reaches into one.
record(obj, key, value)
Note what happened to an attribute, so the art-xpath remap can follow it.
key and value are (name, index, ...) tuples in the shape
parse_art_xpath produces; a value of None
means the attribute is gone, and a list of tuples that it is several entries now.
The first entry of a repeatable attribute is filed under both spellings.
xplan:typ and xplan:typ[1] name the same entry and a presentation object may be
written either way, but parse_art_xpath keeps them apart as (typ, None) and
(typ, 0) while enrich_attr_tuple puts them back together on the way out - so a
record answering to only one of them silently left the other pointing at an
attribute the target class no longer has. That was 58 art strings across 20
documents of tests/data, every one of them a fold source named without an index.
Source code in xmas_core/processing/transform/ops/single.py
remove(obj)
Drop the feature from the migration.
Emptying the dict is the signal every caller already understands: apply_ops stops
applying rules to it and the engine never validates it into a target model.
Source code in xmas_core/processing/transform/ops/single.py
Staged(cls, obj)
dataclass
One feature between its source model and its target model.
obj is the data every operation works on, and cls is the class that data is to be
validated as - the source class until the rules have run, the target class afterwards.
A dict cannot answer each: or an is_a: guard, which is the whole reason the class
travels beside it rather than in it.
When
Bases: BaseModel
A guard on a single operation: one predicate over one or more attributes.
Roughly a seventh of the rules are one or two operations behind a condition, and this is what keeps them data. A clause names one subject - an attribute, a list of them, or a class - and carries at most one predicate over it. Naming no predicate is the common case and means the attribute has a value:
when: {attr: refText} # present
when: {attr: refText, not: true} # absent
when: {attr: art, in: [...], not: true} # none of these values
when: {attr: position, geom: [XP_Liniengeometrie]} # a value of this geometry type
when: {attrs: [a, b]} # both present
when: {attrs: [a, b], any: true} # at least one present
when: {attrs: [a, b], any: true, not: true} # at least one absent
Presence is is not None, so an attribute stated as an explicit false counts as
present - which is the whole reason it is not spelled as a value test.
not: negates the predicate per attribute and any: swaps the quantifier, which is
what lets six earlier keywords (set, not_in, all_set, any_set, all_unset,
any_unset) collapse into two modifiers. An operation may carry a list of clauses, which
are ANDed, and that is the whole of it: no or, no nesting. A conjunction is still a
table row; the moment a guard can be arbitrarily composed the rule file stops being
something anyone can read.
any_ = Field(default=False, alias='any')
class-attribute
instance-attribute
Quantify attrs with any rather than all.
geom = None
class-attribute
instance-attribute
Geometry types by their UML name (XP_Flaechengeometrie), as the source version's
properties declare them.
Not the model classes: the generated models spell a geometry type out as the union it
admits (Polygon | MultiPolygon) and keep its name only as the property's typename.
geom_types = None
class-attribute
instance-attribute
(loader) The model.appschema.definitions classes geom admits. Resolved from the
source appschema, and a load error if stated.
is_a = None
class-attribute
instance-attribute
Class predicate; only a collection operation can evaluate it, checked at load.
not_ = Field(default=False, alias='not')
class-attribute
instance-attribute
Negate the predicate, for each attribute separately.
predicate
property
Which predicate this clause carries, or None for the presence test.
subject
property
The attributes this clause tests. Empty for the class predicate.
matches(source, cls=None)
Evaluate the clause against a feature dict or a model - both are addressable.
cls is what an is_a clause tests, for a caller holding data rather than a model.
Only a collection operation passes it: a rule sees one feature's dict and the loader
refuses is_a there.
Source code in xmas_core/processing/transform/ops/single.py
apply_ops(ops, obj, ctx)
Run ops against obj in order, stopping if one removes the feature.
run_collection_ops(ops, staged)
Run every operation in order over the staged features, which are mutated in place.
Nothing is validated here, in either slot. A pre operation writes into source-shaped
data the rules are about to rewrite - one of them moves a Rasterdarstellung onto the
plan's texte, where it becomes a TextAbschnitt in the same hop, which the source schema
does not declare - and a post operation writes into target-shaped data the target
models are about to be built from. Either way the check that matters is the one at the
end of the hop, against the version the document is becoming.
Source code in xmas_core/processing/transform/ops/collection.py
single
The declarative vocabulary a migration rule is written in.
A rule is a list of operations against the plain dict a source feature was dumped to.
Every rule this package carries is one of eight - retype the feature, set, drop, move or map
an attribute, fold several into a nested object, report, or remove the feature - so
resources/migration/rules.yaml holds the whole migration as data. There is no escape hatch: an operation is
handed the dict and nothing else, so it cannot reach another feature even by accident. The
operations that legitimately need to, pull and push, live in
collection and run in their own phase.
What an operation does not say is anything the target schema already does. Whether a moved value lands in a list or replaces one, whether a fold builds one object or several, and whether a field wants text are read off the target classes - by the loader where the rule file is the only thing that knows which class a feature becomes, and by the engine just before validation otherwise. That is what keeps eight operations enough.
Every operation that removes or moves an attribute also writes the
migration record. That record is what the
art-xpath remap follows, and having the operation write it is the point of the vocabulary:
an earlier release had 55 hand-written bookkeeping calls next to the mutations they
described, and they drifted.
CodeTable
Bases: BaseModel
How codes change: each listed code becomes its value, an unlisted one is kept or dropped.
One class for both places a table is written. A hop's rule for an enumeration is one,
and the engine rewrites every property of that enumeration through it wherever it
appears; a map operation is one rule's table for one attribute, optionally moving the
result into another. A table value is one of:
- a code, which takes the old one's place;
- a list of codes, for an old code the new version covers with several, which take its place together;
null(or[]), for a code with no successor, which goes the way an unlisted code does underunmapped: drop.
A table that replaces any code by several keeps every code once - BP_Wegerecht's
1000 and 2000 are 2000, 2500 and 2000 otherwise. A table that does not keeps a list's
length, so a parallel detail list stays in step with it: two codes that became the
same one are both kept, since collapsing them would hand the second code's detail to
whatever came after it. That is also why the loader refuses paired beside the first
kind.
A removed code is recorded as gone, and every entry that moves is recorded where it went.
remap(obj, ctx, attr, to=None, paired=None)
Rewrite attr through the table, in place or into to.
Into another attribute a mapped value moves, and an unmapped one stays where it
was, so a drop after it reports what the table could not carry.
Source code in xmas_core/processing/transform/ops/single.py
Drop
Bases: _BaseOp
Remove attributes the target version does not have.
Losing a value is the one thing a migration does that a reader needs told about, so a
drop reports by default - once per attribute that actually held something, never for
one that was already absent. That also makes it the way to report what an earlier
operation left: a move with if_unset or a map that did not match leaves its
source in place, and the drop after it reports exactly those.
warn: null silences it, for the drop of a value that is not lost: one a pre:/post:
operation has already moved somewhere else.
Fold
Bases: _BaseOp
Collect several attributes into the nested object(s) the target version introduced.
A fold builds as many objects as its longest source has entries; a scalar source
counts as one. Each source contributes its entry i to object i, a scalar source
contributes to object [0] alone, and a source that runs out leaves the field unset.
Where 5.x kept a code list and a parallel detail list side by side, the target pairs each
code with its detail in one value, and pairing them by position is what those two lists
always meant - a shared scalar beside them, a roof pitch over several roof shapes, is
the same operation with one source that has nothing to pair past the first.
How many objects there are is therefore the data's answer, and whether to holds a list
of them is the target's, filled in by the loader. A field inside an object is written as
the value itself; the engine wraps it where the target declares the field 0..*.
Reports to when it built something.
attrs = Field(min_length=1)
class-attribute
instance-attribute
Source attribute -> its field name inside the object. Two sources may name the same field, whose stream is then their values in the order they are written.
many = False
class-attribute
instance-attribute
(loader) to holds a list of objects rather than one.
Resolved from the target's own multiplicity, and a load error if a rule states it. It says where the objects go, never how many there are.
Map
Bases: CodeTable, _BaseOp
Rewrite one attribute's codes through a table, in place or into another attribute.
Distinct from a hop's rule for an enumeration only in reach: that rewrites every value of the enumeration wherever it appears, this one attribute of one rule.
paired = None
class-attribute
instance-attribute
A parallel detail list whose entry at a code's index goes with it.
A code replaced by nothing takes its detail with it. The loader refuses a code replaced by several, since it has no single position left to pair with.
Move
Bases: _BaseOp
Move values from one or more attributes to another.
What the move does with them is the target's answer, not the rule's: into a list
they are appended after whatever to already holds, and into a single value the
first one wins and the rest are lost. That one distinction is what rename, merge and
take_first were three operations for. Reports the source whose entries were lost.
if_unset = False
class-attribute
instance-attribute
Leave an occupied to alone, and the source where it is.
The source is not consumed, so a drop after the move reports the value that did not
move - which is what a version that folded two attributes into one and kept the more
specific of them needs said.
many = False
class-attribute
instance-attribute
(loader) to holds a list. Resolved from the target, and a load error if stated.
set = None
class-attribute
instance-attribute
The field every moved value carries, with its value read from the from: table.
Written with item[field], because what a set: moves is a data type -
XP_ExterneReferenz - and the engine dumps every data-typed property to a mapping before
any rule runs. That is how attributes that each named one kind of document become one
reference list tagged by typ.
Remove
Bases: _BaseOp
Drop the whole feature: the target version has no equivalent.
A requirement the target places on a value is a guard on this: a geometry of a type the
target class has no place for is when: {attr: position, geom: [...], not: true}. A
requirement that only reports is not an operation at all - an attribute the target
declares 1..1 already fails validation, which says so.
abort = False
class-attribute
instance-attribute
Stop the whole migration: the document holds something with no mapping at all.
property = None
class-attribute
instance-attribute
The attribute that is the cause, checked at load like every other name a rule reads. Left out when the class itself is the cause, and the report then names the feature type.
warn = WarnCode.OBJECT_DROPPED
class-attribute
instance-attribute
null removes the feature without reporting, for one that is carried by another.
Retype
Bases: _BaseOp
Write the feature out as a different feature type, or one per plan type.
by_plan = None
class-attribute
instance-attribute
A target per plan type (BP, FP, ...), for a class the target split by plan.
targets
property
Every class this retype can leave the feature as.
RuleContext(report, step, migration_record=dict(), removed=set(), feature=None, target_type=None, plan_prefix=None)
dataclass
What an operation is handed besides the feature dict.
One instance per hop. It deliberately does not hold the collection: a rule operates on one feature, and the phase that may look at others gets a context of its own.
feature = None
class-attribute
instance-attribute
The feature currently being migrated, as (id, type); set by the engine per feature.
A data type has no id of its own, so a rule that warns about one would otherwise produce an issue no consumer could attach to anything. This is what it falls back to.
plan_prefix = None
class-attribute
instance-attribute
The document's plan type (BP, FP, ...), computed once per hop by the engine.
A property of the collection rather than of any rule, which is why it is resolved here
for the by_plan of set and retype rather than stated in the rule file.
target_type = None
class-attribute
instance-attribute
The feature type the current feature will be validated as; retype writes it.
It is the feature's identity rather than one of its values, so it lives here instead of in the dict the operations mutate - where it sat beside the real attributes, could be read by a guard as though the feature declared such a property, and had to be popped back out before the dict could be validated. A data type has no type of its own here, so this is None while a rule reaches into one.
record(obj, key, value)
Note what happened to an attribute, so the art-xpath remap can follow it.
key and value are (name, index, ...) tuples in the shape
parse_art_xpath produces; a value of None
means the attribute is gone, and a list of tuples that it is several entries now.
The first entry of a repeatable attribute is filed under both spellings.
xplan:typ and xplan:typ[1] name the same entry and a presentation object may be
written either way, but parse_art_xpath keeps them apart as (typ, None) and
(typ, 0) while enrich_attr_tuple puts them back together on the way out - so a
record answering to only one of them silently left the other pointing at an
attribute the target class no longer has. That was 58 art strings across 20
documents of tests/data, every one of them a fold source named without an index.
Source code in xmas_core/processing/transform/ops/single.py
remove(obj)
Drop the feature from the migration.
Emptying the dict is the signal every caller already understands: apply_ops stops
applying rules to it and the engine never validates it into a target model.
Source code in xmas_core/processing/transform/ops/single.py
Set
Bases: _BaseOp
Set an attribute: to a fixed value, another attribute's value, or a value per plan type.
Reports the attribute it set, for a value that is a guess a human should confirm.
by_plan = None
class-attribute
instance-attribute
A value per plan type (BP, FP, ...), for the few that differ by plan.
from_ = Field(default=None, alias='from')
class-attribute
instance-attribute
Copy this attribute's value, falling back to value when it is absent.
if_unset = False
class-attribute
instance-attribute
Leave an attribute that already holds a value alone.
Warn
Bases: _BaseOp
Report something about the feature without changing it.
When
Bases: BaseModel
A guard on a single operation: one predicate over one or more attributes.
Roughly a seventh of the rules are one or two operations behind a condition, and this is what keeps them data. A clause names one subject - an attribute, a list of them, or a class - and carries at most one predicate over it. Naming no predicate is the common case and means the attribute has a value:
when: {attr: refText} # present
when: {attr: refText, not: true} # absent
when: {attr: art, in: [...], not: true} # none of these values
when: {attr: position, geom: [XP_Liniengeometrie]} # a value of this geometry type
when: {attrs: [a, b]} # both present
when: {attrs: [a, b], any: true} # at least one present
when: {attrs: [a, b], any: true, not: true} # at least one absent
Presence is is not None, so an attribute stated as an explicit false counts as
present - which is the whole reason it is not spelled as a value test.
not: negates the predicate per attribute and any: swaps the quantifier, which is
what lets six earlier keywords (set, not_in, all_set, any_set, all_unset,
any_unset) collapse into two modifiers. An operation may carry a list of clauses, which
are ANDed, and that is the whole of it: no or, no nesting. A conjunction is still a
table row; the moment a guard can be arbitrarily composed the rule file stops being
something anyone can read.
any_ = Field(default=False, alias='any')
class-attribute
instance-attribute
Quantify attrs with any rather than all.
geom = None
class-attribute
instance-attribute
Geometry types by their UML name (XP_Flaechengeometrie), as the source version's
properties declare them.
Not the model classes: the generated models spell a geometry type out as the union it
admits (Polygon | MultiPolygon) and keep its name only as the property's typename.
geom_types = None
class-attribute
instance-attribute
(loader) The model.appschema.definitions classes geom admits. Resolved from the
source appschema, and a load error if stated.
is_a = None
class-attribute
instance-attribute
Class predicate; only a collection operation can evaluate it, checked at load.
not_ = Field(default=False, alias='not')
class-attribute
instance-attribute
Negate the predicate, for each attribute separately.
predicate
property
Which predicate this clause carries, or None for the presence test.
subject
property
The attributes this clause tests. Empty for the class predicate.
matches(source, cls=None)
Evaluate the clause against a feature dict or a model - both are addressable.
cls is what an is_a clause tests, for a caller holding data rather than a model.
Only a collection operation passes it: a rule sees one feature's dict and the loader
refuses is_a there.
Source code in xmas_core/processing/transform/ops/single.py
apply_ops(ops, obj, ctx)
Run ops against obj in order, stopping if one removes the feature.
guards_last(data)
data with when and then moved to the end, in that order.
Source code in xmas_core/processing/transform/ops/single.py
one_or_many_schema(schema)
Publish a when list as one clause or a list of them, as _one_or_many reads it.
Source code in xmas_core/processing/transform/ops/single.py
value_types(names)
cached
The value classes a geom: predicate was resolved to, looked up once.
Imported here rather than at module scope so the vocabulary stays independent of the model layer: every other part of it works on the dict alone.
Source code in xmas_core/processing/transform/ops/single.py
collection
The two operations that may look at more than one feature.
A migration rule works on one feature and is handed nothing but the dict it was dumped to,
which is what makes single safe to read: no rule can reach into
another feature, and none of them depends on the order features happen to appear in the
document. A handful of migrations genuinely do need the neighbours, though - a Bereich that
inherits verfahren from its plan, a TextAbschnitt that gains a role back to whatever
references it - and this is where they live.
They are the same shape in both directions. pull follows association roles away from a
feature and copies something back onto it; push follows the same roles and writes onto
what it finds. Because the roles are named rather than expressed, the loader can check every
one of them against the appschema, which is exactly what a path expression could not offer.
How either one writes is the written attribute's answer, not the rule's. Into a list they collect from every feature the roles reach, each value once; into a single value they fill it from the first feature that has one, and never overwrite what is there. The loader reads that off the appschema, so a rule states neither.
They run in two slots, declared as pre: and post: in the rule file, and both work on the
same dumped feature data every other operation is handed: pre after the source models are
dumped and before any rule runs, post after every rule has run and before the target models
are built. Nothing runs in between, so neither can observe a half-migrated document.
What they move is data - a string, a UUID, a mapping - never a model. That is not a style
preference: a live value carries its own class, and an enumeration member or a data type
copied out of a source-version feature is a different class from the identically named one
the target declares, so writing it onto a target model left the model holding a value its own
field did not declare. Handing the data to the receiving model instead lets validation do the
conversion, which it already knows how to do. It is also what lets a pre operation write an
attribute the source class has no field for - verfahren is stated on the plan in 5.4 and on
the Bereich in 6.0, and a dict has room for it where setattr on a 5.4 BP_Bereich raises.
Pull
Bases: _CollectionOp
Copy something from the features via reaches back onto the feature that owns it.
from: id copies the reached feature's own id, which is how a role pointing at it is
filled in. An attrs: group is filled as one: only when the feature holds none of it,
and all of it from the same reached feature.
attrs = None
class-attribute
instance-attribute
Shorthand for several attributes that keep their name on both sides.
with_ = Field(default_factory=dict, alias='with')
class-attribute
instance-attribute
Fixed fields added to each copied data type, rebuilt as the target role's type.
written
property
The attributes this operation writes on the owning feature.
Push
Bases: _CollectionOp
Write onto the features via reaches, from the feature that owns the reference.
set = None
class-attribute
instance-attribute
Fixed values, for a role whose meaning moved onto the referenced feature.
to = None
class-attribute
instance-attribute
The role to write the owning feature's id to, which is how an inverse role is filled in.
Staged(cls, obj)
dataclass
One feature between its source model and its target model.
obj is the data every operation works on, and cls is the class that data is to be
validated as - the source class until the rules have run, the target class afterwards.
A dict cannot answer each: or an is_a: guard, which is the whole reason the class
travels beside it rather than in it.
run_collection_ops(ops, staged)
Run every operation in order over the staged features, which are mutated in place.
Nothing is validated here, in either slot. A pre operation writes into source-shaped
data the rules are about to rewrite - one of them moves a Rasterdarstellung onto the
plan's texte, where it becomes a TextAbschnitt in the same hop, which the source schema
does not declare - and a post operation writes into target-shaped data the target
models are about to be built from. Either way the check that matters is the one at the
end of the hop, against the version the document is becoming.
Source code in xmas_core/processing/transform/ops/collection.py
rules
Loading and validating the migration rules.
resources/migration/rules.yaml holds the whole migration as data: one entry of
migrations: per hop, each with its rules - an operation list per feature or data type, a
code table per enumeration - and its two collection phases, plus the templates: those
entries share through YAML anchors.
load reads one hop and checks it against the
appschemas it claims to migrate between.
That check is the reason the rules are declared rather than discovered. An earlier release
dispatched by getattr(self, "_" + ClassName.replace("_", "").lower()), so a rule whose name
did not match any class was a silent no-op - which is what happened to
_lpschutzobjektbundsrecht, misspelled against LP_SchutzobjektBundesrecht and never
run in any release. Here that is a RuleFileError the moment the file loads.
RuleFileError
Bases: ValueError
A rule file does not describe the rules it claims to.
RuleSet(from_version, to_version, source, target, rules, enums, pre, post)
dataclass
Everything one hop needs: its rules, its enum tables and its collection passes.
step
property
The hop, as it is tagged onto every issue the hop raises.
build(text, from_version, to_version)
Validate rule-file text against the two appschemas and build the rule set.
Split out from load so the checks can be exercised
against a rule file that does not have to exist on disk.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
text
|
str
|
The rule file's contents. |
required |
from_version
|
str
|
The version being migrated away from. |
required |
to_version
|
str
|
The version being migrated to. |
required |
Raises:
| Type | Description |
|---|---|
RuleFileError
|
The file names a class, enumeration, role or attribute that does not exist, or holds the hop asked for not exactly once. |
Returns:
| Type | Description |
|---|---|
RuleSet
|
The hop's rules, ready to run. |
Source code in xmas_core/processing/transform/rules.py
151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 | |
load(from_version, to_version)
cached
Read, validate and return the rule set for one hop.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
from_version
|
str
|
The version being migrated away from. |
required |
to_version
|
str
|
The version being migrated to. |
required |
Raises:
| Type | Description |
|---|---|
RuleFileError
|
The file names a class, enumeration, role or attribute that does not exist, or holds the hop asked for not exactly once. |
Returns:
| Type | Description |
|---|---|
RuleSet
|
The hop's rules, ready to run. |
Source code in xmas_core/processing/transform/rules.py
engine
Running one migration hop.
Every source model is dumped once, up front, and everything after that works on the data:
pre- the hop's collection operations over the source-shaped data, for the few migrations that need a feature's neighbours before it is rewritten.- Per feature - apply the rules registered for every class in its inheritance chain, most abstract first, and point the feature at the class it is to be validated as.
post- collection operations over the target-shaped data, once every rule has run.- Repair - drop every reference to a feature the hop removed, or of a type its role no longer accepts.
- Validate - widen every scalar the target declares 0..* and render every value the target
declares as text, turn each dict into a model of the target version and build the
collection, then remap the presentation objects'
artxpaths.
The separation is the point, and phase 2 is where it is enforced rather than described: a
rule is handed the feature's dict and a RuleContext that does not hold the collection, so
it cannot read another feature. An earlier release let rules reach into the half-built
output while the loop was still running, so two of them carried a # order matters! comment
and correctness depended on the order features happened to appear in the source document.
Phases 1 and 3 are the ones that may look at a neighbour, and both of them move data rather
than models - see ops.collection. Nothing is
validated until phase 5, which is what lets a pre operation write an attribute the source
class has no field for and a post operation write one whose value only the target version
knows how to read.
run_hop(collection, rules, report)
Migrate collection one version forward.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
collection
|
FeatureCollection
|
The collection, at |
required |
rules
|
RuleSet
|
The hop's rules, from |
required |
report
|
MigrationReport
|
The report to collect into; shared across every hop of a run. |
required |
Raises:
| Type | Description |
|---|---|
MigrationError
|
A feature or the collection failed to validate against the target version. The report holds what went wrong with which feature. |
Returns:
| Type | Description |
|---|---|
FeatureCollection
|
The collection, at |
Source code in xmas_core/processing/transform/engine.py
style
Methods for handling XPlanung presentational objects.
Parse and serialize art XPath values and derive styling properties for presentational
objects by matching art references against the style rules in
resources/style/rules.yaml (see rules) to populate
stylesheetId and schriftinhalt.
PresentationObject
Bases: Protocol
The fields every version's presentational object classes declare beside FeatureType.
isinstance against it narrows a feature to one, which is how a checker learns what
the generated classes know and FeatureType does not.
add_style_properties(collection, *, to_text=False, always_populate_schriftinhalt=False)
Adds styling properties to every presentational object in a collection.
The collection-level counterpart of
add_style_properties_to_feature,
which it applies to each presentational object with the object that object presents.
A feature that names no object, names more than one, or carries no art is skipped
and logged; only a reference naming a feature the collection does not hold is an error,
since the collection's own reference check would already have refused it.
The features are replaced in place, so the collection is modified rather than copied.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
collection
|
FeatureCollection
|
The collection whose presentational objects are styled. |
required |
to_text
|
bool
|
Whether to convert symbolic presentational objects to textual ones. |
False
|
always_populate_schriftinhalt
|
bool
|
Populate |
False
|
Raises:
| Type | Description |
|---|---|
ValueError
|
A presentational object references a feature that is not in the collection, or more than one style rule matched it. |
Source code in xmas_core/processing/style/__init__.py
add_style_properties_to_feature(obj, ref_obj, to_text=False, always_populate_schriftinhalt=False)
Add styling properties to presentational objects.
This method parses object (dientZurDarstellungVon) and property (art) references from presentational objects and derives styling information (stylesheetId, schriftinhalt) based on a set of defined rules.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
obj
|
FeatureType
|
The presentational object. |
required |
ref_obj
|
FeatureType
|
The object referenced by the presentational object. |
required |
to_text
|
bool
|
Whether to convert symbolic presentational objects to textual ones. Defaults to False. |
False
|
always_populate_schriftinhalt
|
bool
|
Populate |
False
|
Source code in xmas_core/processing/style/__init__.py
184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 | |
enrich_attr_tuple(obj, art_tuple)
Return feature property information for construction of xpath expression.
The middle element of the result is whatever the last step of the walk landed on - a
leaf attribute value (an enum member, a Measure, a string), not necessarily a model -
so it is Any. Only the intermediate steps are models, and only a DataType can be
one of those.
Source code in xmas_core/processing/style/__init__.py
parse_art_xpath(xpath)
Parse an xpath expression into a tuple of feature property information and corresponding indices.
Source code in xmas_core/processing/style/__init__.py
serialize_art_xpath(t, prefix='xplan')
Construct xpath expression from tuple of feature property information.
Source code in xmas_core/processing/style/__init__.py
rules
Loading and validating the style rules.
resources/style/rules.yaml holds every style rule as data, keyed by the stylesheetId it
assigns. A rule states, per XPlanung version, which attributes a presentational object's art
has to reference - their names, types and values - and optionally a template for its
schriftinhalt. A version whose rule is the same as another version's is not written twice:
the first is anchored and the others alias it, so they load as the same rule.
The models are the file's canonical form as well as its validation. Dumped with
exclude_unset=True, they write fields in the order they declare them, only what was written,
versions in ascending order and rules by comment in natural order, which is what
scripts/normalize_style_rules.py writes back.
StyleRule
Bases: BaseModel
One stylesheetId: what it stands for and when it applies.
comment
instance-attribute
Which Planzeichen the rule represents, or another note for whoever reads the file.
StyleRules
Bases: RootModel[dict[UUID, StyleRule]]
The rule file: every style rule, keyed by the stylesheetId it assigns.
StyleSelector
Bases: BaseModel
What one attribute referenced by art has to be.
type
instance-attribute
Its type name in the version at hand.
value
instance-attribute
The values it may hold, or ["*"] for any value.
VersionedStyleRule
Bases: BaseModel
A style rule as it applies to one XPlanung version.
selector
instance-attribute
Keyed by attribute: art has to reference exactly these. A rule with no selector
matches nothing.
text = None
class-attribute
instance-attribute
A template for schriftinhalt, with the selected attributes as placeholders.
load()
cached
Read and validate the style rules, once.
Raises:
| Type | Description |
|---|---|
RuleFileError
|
A key is written twice. |
ValidationError
|
The file does not describe style rules. |
Returns:
| Type | Description |
|---|---|
StyleRules
|
The rules, with every alias resolved. |