Reading and expanding lattices¶

The parse_and_expand_pals entry point reads a PALS lattice file, resolves any files it includes, and expands the lattice line into an ordered list of elements. It returns a Lattices value with five independent views of the document:

original — the lattice exactly as written, one entry per file read.
combined — the lattice after its included and loaded files have been merged in.
full_expanded — the fully expanded root lattice, with lines resolved into a flat ordered sequence of elements and every dependent parameter computed.
expanded — the same lattice with the computed parameters pruned, leaving what the author wrote.
adjunct — everything else the document contained.

Each view is an ordinary YAMLNode, so everything in Parsing and writing YAML applies to it.

The expanded views and `adjunct`¶

Expansion picks one lattice — the root lattice — and resolves it. The two expanded views hold only that result, and are rooted at the lattice entry itself, without the PALS:/facility: scaffolding the lattice was written under:

lat1:
  kind: Lattice
  branches:
    - main_line:
        ...

so the lattice is reached as lat.full_expanded["lat1"], not through ["PALS"]["facility"].

Everything the root lattice did not absorb stays in adjunct, which does keep the full PALS:/facility: document: element and beamline definitions, use statements, constants and variables, Controllers, set commands, and any Lattice other than the one expanded. Definitions that expansion substituted into the lattice are copied rather than moved, so they appear in both views — the definition in adjunct, its inlined copy in the expanded lattice.

`full_expanded` and `expanded`¶

full_expanded is the lattice with everything it implies worked out: each element carries its ReferenceP, FloorP and s_position, the derived members of every parameter family it uses (Kn1L alongside Kn1, voltage alongside gradient, …) and the non-zero defaults of the groups it carries, and each branch is capped with a branch_end Placeholder holding its final reference and floor.

expanded is that same tree with all of it pruned: a parameter is kept only when the author wrote it (or a post-expand_lattice set wrote it). It is full_expanded with nodes removed rather than an earlier snapshot, so a parameter present in both holds the same value in both, with every set and ABSOLUTE controller applied.

Which to reach for:

full_expanded to ask what the lattice is — placement, reference energy, or any parameter derived from another. match_names and parameter_value search it for that reason.
expanded to see the inputs rather than their consequences, or to write a lattice back out without the computed values.

Basic use¶

import palsparserpy as pp

lat = pp.parse_and_expand_pals("ex.pals.yaml")

print(pp.to_yaml_string(lat.original))
print(pp.to_yaml_string(lat.combined))
print(pp.to_yaml_string(lat.full_expanded))
print(pp.to_yaml_string(lat.expanded))
print(pp.to_yaml_string(lat.adjunct))

To expand a single named lattice from a file that defines several, pass its name as the second argument:

lat = pp.parse_and_expand_pals("ex.pals.yaml", "main_ring")

Reporting problems¶

Expanding a lattice can hit problems that are not fatal but are worth knowing about: a line that references an element which was never defined, an inherit/repeat/Fork whose target is missing, or an expression that could not be evaluated (an unknown constant, a dangling element-parameter reference, a dependency cycle). Rather than abort, expansion keeps going — leaving the offending value as text — and collects a list of every such problem.

The problems argument controls what is done with that list:

# Default: print the problems to stderr (nothing prints when there are none).
lat = pp.parse_and_expand_pals("ex.pals.yaml")

# Write the problems to a file instead, printing nothing.
lat = pp.parse_and_expand_pals("ex.pals.yaml", problems="problems.txt")

# Say nothing at all.
lat = pp.parse_and_expand_pals("ex.pals.yaml", problems="none")

"print" and "none" are the two reserved names; any other value is taken as the path of the file to write.

A typical report looks like:

parse_and_expand_pals: 2 problem(s) encountered during lattice expansion:
  - ERROR: reference to undefined element or line 'NoSuchElement'
  - ERROR: could not evaluate expression for BendP.edge_int2: 0.02 * thingB>MagneticMultipoleP.NotThere

Reading the list programmatically¶

Whatever the reporting mode, the same list comes back in lat.problems as a list of Problem, so "none" still lets you inspect it. Each entry carries more than its message:

path — where it was found ("q1>ApertureP.shape"), empty when the problem is not tied to one spot.
severity — PROBLEM_ERROR when the trees can no longer be trusted around the fault, PROBLEM_WARNING when expansion produced a sound result anyway.
origin — PROBLEM_INPUT when your lattice is what needs fixing, PROBLEM_UNSUPPORTED when it is valid PALS that PALSParserCpp does not implement yet, and PROBLEM_UNSPECIFIED when the PALS standard does not define the case, so nothing was invented.

The last one is the one to filter on before failing a build, since editing the lattice can only ever clear a PROBLEM_INPUT:

lat = pp.parse_and_expand_pals("ex.pals.yaml", problems="none")

mine = [p for p in lat.problems if p.origin is pp.PROBLEM_INPUT]
if mine:
    raise SystemExit(f"{len(mine)} problem(s) to fix:\n" +
                     "\n".join(f"  {p}" for p in mine))

Only values that look like expressions (an operator, a parenthesis, an element-parameter > reference, or an explicit expr(...)) are flagged when they fail to evaluate; a plain name, label, or boolean that happens not to be a number is left alone.

Relative includes¶

Include paths inside a lattice file are resolved relative to the working directory when the C library opens them. If your lattice includes other files by relative path, change into the lattice directory first:

import contextlib
import os

@contextlib.contextmanager
def chdir(path):
    old = os.getcwd()
    os.chdir(path)
    try:
        yield
    finally:
        os.chdir(old)

with chdir("lattice_files"):
    lat = pp.parse_and_expand_pals("ex.pals.yaml")

(On Python 3.11 and newer, contextlib.chdir does this for you.)

Correspondence between the views¶

The derivation-chain trees describe the same lattice at successive stages of processing, so most of their nodes correspond: the constant a_const, for instance, exists in original, in combined, and — since it is not part of the lattice — in adjunct. node_correspondence builds that mapping: given any node, it returns the nodes it corresponds to in the other views.

lat = pp.parse_and_expand_pals("ex.pals.yaml")
corr = pp.node_correspondence(lat)

The result is a dict keyed by YAMLNode. Looking up a node returns a named tuple whose fields — original, combined, full_expanded, adjunct — are each a list of YAMLNode listing the corresponding nodes in that view:

a_const = lat.combined["PALS"]["facility"][0]["constants"]["a_const"]

corr[a_const].original        # [ the a_const node in the original tree ]
corr[a_const].adjunct         # [ the a_const node in the adjunct tree ]
corr[a_const].full_expanded   # [] -- the lattice never referenced it

The queried node is included in its own view’s list, so the four lists together form the complete set of nodes that correspond to one another. You can look a class up starting from any of those four trees and get the same result:

corr[corr[a_const].original[0]] == corr[a_const]   # True

Because expansion splits the document, a combined node can reach full_expanded, adjunct, or both. A beamline named by the root lattice is a good example: its definition stays in adjunct while a copy of it is inlined into the expanded lattice, and both belong to the same class.

The expanded view takes no part in the correspondence: it is a pruned copy of full_expanded rather than a step in the derivation chain, so a node in it is found by the path it sits at, not by a recorded link.

One-to-many correspondences¶

Expansion can turn a single node into several — a repeat unrolls a line, an inherit copies fields in, a bare element name is substituted with its full definition, and a fork spawns a new branch. The correspondence follows every copy, which is why each field is a list: one combined node can map to many full_expanded nodes.

# The sub-line repeated inside inj_line appears once in `combined`
# but several times in `full_expanded`.
for node, cls in corr.items():
    if len(cls.combined) == 1 and node == cls.combined[0] and \
            len(cls.full_expanded) > 1:
        print("combined node →", len(cls.full_expanded), "expanded copies")

A list is empty when a view has no corresponding node. For example, the destination_pointer scalar that expansion synthesises exists only in full_expanded, so its original and combined lists are empty; a constant the lattice never refers to exists only in adjunct, so its full_expanded list is empty.

Note

The mapping is exact, not heuristic. The correspondence is not recovered by re-matching the finished trees. The views are built as a derivation chain (original → combined → full_expanded and adjunct), and the provenance of every node is recorded as it is copied. node_correspondence reads back that recorded provenance, so the mapping is exact even where nodes are duplicated, merged, or renamed during expansion.

A runnable version of these examples is in examples/node_correspondence.py.

Matching constructs by name¶

Once a lattice is expanded, match_names finds every named construct that a PALS Name Matching string refers to — elements, parameter groups, parameters, constants, and variables — and returns them as a list of YAMLNode. The syntax is:

[{lattice}>>>][{branch}>>][{kind}::]{name}[>{group}.{subgroup}. … .{parameter}]

{lattice}, {branch}, and {name} are PCRE2 patterns matched against the whole name (anchored at both ends), so B1.* matches B1a and B1b but B1 on its own matches neither. {kind} is matched exactly, and the dotted parameter path after the single > is matched exactly, key by key. An omitted or empty pattern matches every name at that level, and {branch} matches an element if any enclosing BeamLine/Branch name matches — so elements in sub-lines are included.

lat = pp.parse_and_expand_pals("ex.pals.yaml")

# The `e1` bend parameter of every element whose name begins with `B1`:
pp.match_names(lat.full_expanded, "B1.*>BendP.e1")

# Restrict to an element kind with `::`:
pp.match_names(lat.full_expanded, "Quadrupole::.*>length")

# Restrict to a named beamline/branch (`>>`) or lattice (`>>>`):
pp.match_names(lat.full_expanded, "inj_line>>Q.*>length")
pp.match_names(lat.full_expanded, "ring>>>inj_line>>Q.*>length")

# Omit the parameter path to match the element itself, or the group:
pp.match_names(lat.full_expanded, "Q1a")             # the element node
pp.match_names(lat.full_expanded, "Q1a>BendP")       # a parameter-group node

Pass any node of the tree you want to search — lat.full_expanded for beamlines and elements, since those are only fully realised after expansion. The returned nodes belong to that same tree, so you can read or modify them in place:

for n in pp.match_names(lat.full_expanded, "B1.*>BendP.e1"):
    n.set_scalar("0.0")   # zero the entrance-face angle of each B1… bend

Constants and variables¶

Lattice parameters include constant and variable names. A bare name — no lattice/branch/kind qualifier and no parameter path — also matches every constant and variable defined directly under the PALS or facility node, in both the full (kind: constant / kind: variable) and compact (constants: / variables: list) forms.

Constants and variables are defined at facility level rather than inside the lattice, so they are found in lat.adjunct — searching lat.full_expanded for one matches nothing, as the PALS/facility node it lives under is not part of that tree:

pp.match_names(lat.adjunct, "a_const")   # one named constant
pp.match_names(lat.adjunct, "a_.*")      # every constant/var named a_…

For a compact-form entry the matched node is the name: value scalar; for a full-form entry it is the named node, underneath which kind/value live.

Note

Not yet implemented. The full Element Name Matching grammar also defines #N instance selection, {e1}:{e2} ranges, , unions, and & intersections. These are not yet handled by match_names.

Results are de-duplicated and returned in document order, and a malformed pattern yields an empty list. A runnable version of these examples is in examples/match_names.py.

Reading a parameter value¶

Where match_names returns the nodes a string refers to, parameter_value returns the single value a parameter holds. It takes the whole expanded lattice lat and the same Name Matching syntax, and returns a float, a str, or None. Like match_names, the string names either an element parameter (with a parameter path) or, as a bare name, a constant or variable:

lat = pp.parse_and_expand_pals("ex.pals.yaml")

pp.parameter_value(lat, "lat1>>>B1a>BendP.e1")        # 0.1    (from full_expanded)
pp.parameter_value(lat, "F1>ReferenceP.species_ref")  # '#3He' (a string)
pp.parameter_value(lat, "Q1>BendP.g")                 # 0.0    (unset → default)
pp.parameter_value(lat, "Q1")                         # None   (not a value)
pp.parameter_value(lat, "a_const")                    # a constant (from adjunct)

parameter_value searches only two of lat’s five views: lat.full_expanded, which holds the element parameters, and then, if the name is not found there, lat.adjunct, which holds the facility-level constants, variables, and any definitions not spliced into the lattice. The raw lat.original and lat.combined views are not searched, and neither is lat.expanded: a dependent parameter is a legitimate thing to ask for, and only full_expanded carries one.

Because both searched views are post-expansion, values come back already evaluated — a numeric value as a float, and a non-numeric one (a species name, or an expression expansion left unevaluated such as one using random()) verbatim as a str:

Element parameter, set — its value: a float, or a str when non-numeric.
Element parameter, unset — an element that exists but does not set the parameter yields the parameter’s default. That default is 0.0 for every parameter for now; real per-parameter defaults come later.
Constant or variable — a bare name yields its value, the same way.
Unidentified — None, when the name matches nothing in either view, is a bare element (an element has no single scalar value), stops on a whole parameter group rather than a single value, or several matches disagree on the value. (Matches that agree — the same element reused, or several that all take the default — collapse to the one shared value.)

Note

Defaults are provisional. Because there is no parameter schema yet, an unset parameter and a name that is not a real parameter are indistinguishable, so both return the 0.0 default rather than None. When defaults arrive, an unknown parameter name will return None instead.

Command-line driver¶

examples/read_pals.py is a small runnable program that wraps the above: it reads a lattice, expands it, and prints all five views.

python examples/read_pals.py

Place your lattice files under lattice_files/.

Reading and expanding lattices¶

The expanded views and adjunct¶

full_expanded and expanded¶

Basic use¶

Reporting problems¶

Reading the list programmatically¶

Relative includes¶

Correspondence between the views¶

One-to-many correspondences¶

Matching constructs by name¶

Constants and variables¶

Reading a parameter value¶

Command-line driver¶

The expanded views and `adjunct`¶

`full_expanded` and `expanded`¶