This post is number 4 in a sequence of 7. Click here to get to the beginning.
The ‘@@’ symbol – relative collection reference
The ‘@@’ symbol, as one might expect given the meaning of the ‘@’ symbol (see here), is used to denote a one-to-many relationship from a field within an ontological type, to a set of records of different type(s) that are referenced ‘internally’ via an embedded collection, and thus are ultimately stored along with the referencing structure in the same Mitopia® server responsible for the referencing structure itself. The structures within a ‘@@’ embedded collection cannot be directly referenced by unique ID, and cannot therefore be derived from Datum. However, it is permissible for structures within an ‘@@’ collection to reference external persistent items via ‘#’ (see here) or ‘##’ (see here) references on behalf of the record containing the ‘@@’ collection reference.
Structures within an ‘@@’ collection may have a ‘name’ field, but do not have a unique ID field. The $UniqueBy annotation can be associated with the type for the ‘@@’ collection elements in order to designate one or more of the fields within the record that can be used to check ‘@@’ elements for uniqueness. The field designated can be any string, integer, real, or persistent reference field within the type. If no $UniqueBy annotation is present, but the record has a ‘name’ field (‘@ char’ or char[]), that is used by default to determine uniqueness. If there is no $UniqueBy annotation, and no ‘name’ field, all new records added to the ‘@@’ collection are assumed to be unique, and there is no comparison with other elements of the ‘@@’ collection. The unique field (‘name’ or otherwise) is used by Mitopia® when matching records at the same level within such a sub-collection, and thus it is essential that these records have the unique field value within any given parent node of the ‘@@’ collection hierarchy, otherwise they will be merged with others in the same branch of the ‘@@’ tree.
Finally, ‘@@’ sub-collection element fields cannot be the explicit target of an echo field modifier (‘><‘) from another type, since only Datum-derived types can be so referenced. However, ‘#’ and ‘##’ fields within the elements of ‘@@‘ sub-collections directly referenced (but not within further nested ‘@@’ collections) from a Datum-derived type can serve as an ‘indirect‘ echo path for references to other records by use of the ‘><><‘ indirect-echo field specifier. See the next post (here) for complete details.
The only way to access these embedded structures is to fetch the referencing item itself from persistent storage and then navigate the sub-collection that comes along with it. The actual implementation of the relative collection reference field is identical to that of the ‘##’ field, but the ‘@@’ sub-collections are processed in a very different manner by all Mitopia® code.
Unlike the content of embedded ‘##’ collections, which is comprised of records of type ET_Hit that reference the outer collection (see here), the content of embedded ‘@@’ records contain the complete structure for instances of the type (or descendants) declared in the referencing ‘@@’ field. Moreover, the hierarchy within the ‘@@’ collection is not mandated by Mitopia® code in any way, and thus these collections can be organized in any manner that is advantageous. Contrary to the convention followed in all ‘persistent’ collections, in ‘relative’ collections, both leaf and non-leaf nodes can have associated values. It is quite possible for structures within an ‘@@’ embedded collection to contain ‘@@’ reference fields of their own, and this allows nested sub-collections to any number of levels, together with the ability to display and edit this data in the auto-generated UI. All such nested sub-collections are fetched whenever the referencing persistent structure and field is fetched from a server to the client, and all values in the sub-collection(s) maintain the appropriate ‘empty‘ flag settings.
All these features combined make the ‘@@’ collection a very useful way to describe elaborate sub-structure for a persistent record without having to consider the ontology, or how referenced items are shared between servers (since they are always held directly in the server collection for the persistent record that contains the ‘@@’ collection reference). Again we should stress how different and how much more powerful this behavior is from conventional databases (e.g., relational). The content of these ‘@@‘ sub-collections is fully searchable via the containing parent record and this allows for some very complex query behaviors.
The most obvious and simplest use of the ‘@@’ collection is within the ‘notes’ sub-structure for Datum viz:
Delta@@history; // time history of changes to this Datum
The Delta records hold a series of difference descriptions in reverse, much in the same way as a version control system such as CVS or Subversion does. These difference descriptions (held in the ‘modifyScript’ field) are tagged with the time at which the difference was caused, and thus by applying each difference in series to a record value, it is possible to re-create the state of the record at any given time in the past. This capability is one of the fundamental underpinnings of Mitopia’s handling of the time axis.
Clearly it is essential that these differences be stored along with the records to which they relate, and that they be fetched when the record is fetched in order to allow the client to examine the time axis if necessary. The ‘@@’ construct is therefore ideally suited to implementing this feature. See the “Role @@ roles ><>< players” of the type ‘Action’ for a more complex example, including specification of the echo field through use of the ‘><><‘ operator.
A more elaborate use of the ‘@@’ construct can be found in K.DEF where the description of a user interface window (or Canvas) for Mitopia’s ‘universal’ GUI renderer layer contains the “Control @@controls” field. If you examine the contents of K.DEF in detail, you will see that the description of the entire content and organization of the window is held within the “@@ controls” collection, including of course the nesting of sub-controls to arbitrary numbers of levels. Once again, the ‘persistent’ item is the window appearance itself (i.e., the Canvas), however by use of the ‘@@’ field, it is possible to save and persist the entire window layout and behavior and to manipulate the GUI directly using the ontology APIs rather than any specialized abstractions. The ‘@@’ construct effectively allows access the the full power of the abstraction layer in a manner that the constraints of persistent data organization through the ontology do not permit through the higher level Carmot abstractions. Through the ‘@@‘ construct it is possible to represent just about anything, regardless of complexity, within a single persistent record.
Obviously the differing semantics between the ‘##’ and ‘@@’ constructs require them to be treated very differently in all infrastructure code, and nowhere is this more true than in the GUI representation of the two constructs. This subject will be covered in more detail in later posts.
In both the servers and clients, the contents of the ‘@@’ sub collections are handled entirely for copying and merging by the MitoQuest™ plugin to the MitoPlex™ framework and thus any server that utilizes these ‘@@’ collections to contain specialized data that might for example be associated with some multimedia type handled by the server, does not have to supply specialized plugins to handle the data. MitoQuest™ itself is ‘responsible’ for ‘@@’ collection fields (like ‘##’ fields), and provides built-in support for server queries of these fields. As usual, custom media types can be referenced from ‘@’ fields within the ‘@@’ elements, and thus custom handlers and query capabilities can be registered on these ‘@’ fields.