In previous posts we have examined how the Carmot ontology for the system is leveraged by Mitopia® to auto-generate and handle persistent storage and query, thereby eliminating the ‘database’ portion of the Bermuda Triangle molecule discussed earlier. We have also discussed how the ontology is used to automatically generate and handle virtually all user interfaces (an adaptive UI) in a Mitopia®-based system, thereby eliminating the last remaining ‘GUI’ portion of the the triangle and creating an architecture that is fully adaptive to change (as required to solve any large problem in an unconstrained environment).
In this post I want to briefly examine Mitopia’s fundamental low level approach to the adaptive UI abstraction/isolation layer, particularly as it relates to adaptability issues. We should perhaps begin with a brief overview of the world of GUI frameworks and the current state of the art in this arena.
The list above shows some of the common high level GUI toolkits, for more details see the Wikipedia article on ‘Widget tookits‘. As can be seen, there are a vast number of commercial and free GUI development frameworks that are available. The list of free GUI toolkits contains around 120 distinct entries, while the commercial list has nearly 40. These figures are for C/C++ toolkits only. If other languages (particularly Java), are included, the lists are vastly larger. It is perhaps safe to say that more GUI toolkits are created than in any other functional area of software architectures, since it is one of the most enjoyable areas of software to dabble in, and also one of the few areas where OOP languages can be shown to best effect.
Historically, all operating systems tend to provide a low level (non-OOP) GUI toolkit which allows creation of arbitrary GUIs regardless of how the application code is organized or designed. This is of course essential since most applications have no formalism for the underlying data model and simply graft a custom UI onto the data in an ad hoc manner as required by the particular problem. For small to medium problem, this approach takes considerably less work than other approaches but of course it is highly susceptible to breaking when any aspect of system operation or data changes. This fact notwithstanding, the overwhelming majority of all GUI approaches are of this ad-hoc form, including many of the earlier GUI frameworks themselves.
Most modern GUI toolkits use the Model View Controller (MVC) design technique. In MVC, the Model represents the information (the data) of the application and the business rules used to manipulate the data; the View corresponds to elements of the user interface such as text, checkbox items, and so forth; and the Controller manages details involving the communication to the Model of user actions such as keystrokes and mouse movements.
Of course many applications use a persistent storage mechanism (such as a database) to store data. MVC does not specifically mention the data access layer because it is understood to be underneath or encapsulated by the Model, that is there is custom code in the Model that adapts the needs of the data repository to those of the application and the GUI framework API. In other words, the MVC approach makes no attempt to solve the Bermuda Triangle problem, it simply encourages isolating it into the ‘Model’. In reality, the MVC approach is another 3-atom molecule, identical to the molecule described for the Bermuda Triangle discussion, but labelled differently. Attempting to build a truly complex unconstrained application using the MVC technique will result in the same problems as all other existing techniques.
If we attempt to narrow our internet search to GUI toolkits that are in some way truly data-driven, that is the GUI framework discovers the ‘type’ of actual binary program data to be displayed at run-time and generates a suitable user interface on the fly (i.e., an adaptive UI), we find nothing. The closest matches to this kind of capability are the GUI generators available with high end databases for converting database schemas to UI, and the current efforts to generate GUI based on XML schema to allow XML data editing. Both of these technologies rely on a formalized schema to generate GUI, but this schema is unrelated to the actual types and structures in use by the application code, and is instead tied to the ‘database’ used to implement persistent storage. In other words, these approaches are ‘disjoint’ in the same manner that semantic ontologies are disjoint in that they do not address unifying the programming model for application code with that of the GUI or database. Without addressing this issue, we cannot truly solve the triangle problem.
The problem is that GUI frameworks tend to be created out of an interest in user interfaces and how to make them more functional or elaborate, not from an interest in making applications more adaptive and getting rid of UI related code entirely.
Mitopia’s takes a plugin approach to the GUI layer where specific implementations and delivery platforms (known as ‘renderers’) are registered with a ‘universal’ abstraction layer which is itself hidden behind an interpreted GUI language, based on Carmot-E so that direct access to system data is automatic. It performs this trick through a plug-in adapter to Mitopia’s data model and type management system. Virtually any conventional GUI framework could be registered behind these abstraction layers in order to target different GUI devices or environments. Indeed at this time within a standard Mitopia® local GUI environment, there are currently two very different renderers running, and communicating through, the same abstracted windowing environment. Which renderer is used by any given Mitopia® widget is a matter of widget and/or user choice.
The first such plugin is called Genesis. Genesis is a large and elaborate C++ cross-platform GUI framework and encapsulates all of the View and Controller logic within itself while providing a data-model interface for run-time data discovery (GUI-to-application). In other words from a GUI perspective, Genesis supports the discovery of the ‘Model’ in the DMV paradigm rather than just the registration of custom Model code. This is a more sophisticated approach to data connection than that used in conventional GUI toolkits. The interface between the ‘renderer’ and data residing in Mitopia® is provided by Mitopia’s low level DataModels.c package which supports the registration of all the necessary plug-in functions required to form a complete GUI/application connection. Genesis is the oldest of Mitopia’s current GUI plugins, predating Mitopia’s ‘Universal Renderer’ abstraction, and so it connects the data model directly to the GUI interpreter layer. All other renderers plug into the ‘Universal Renderer’ (UR) layer.
The ‘Universal’ renderer layer describes and manipulates the GUI through extensions to the ontology found in “K.DEF”. The universal renderer plugs into Mitopia® at the same level as does Genesis. The UR provides all the the data-discovery logic and most of the abstract GUI behaviors. The universal renderer requires two target-specific plugins, a ‘Canvas’ plugin (think Window handler) and a ‘Painter’ plugin (think Control handler), in order to direct rendered output to/from any given UI environment. This approach dramatically lowers the hurdles necessary to repurpose the GUI to any other target platform (e.g., a web page), since the canvas and painter plugins can be relatively ‘dumb’ and are easy to implement. Mitopia® itself provides platform specific canvas and painter plugins, so that most Mitopia® GUI is now handled through the UR. Currently a few visualizers remain to be implemented under the UR. The eventual goal is 100% UR layered GUI renderers since any GUI handled by the UR can be more easily repurposed to other GUI environments including target screens that are remote and elsewhere on the network.
The application-to-GUI interface between Mitopia® and it’s renderers (e.g., Universal and Genesis) is implemented though a single Mitopia® API function call UI_Command(). This API function is implemented as a parser, for which the interpreted language parsed is in fact Carmot-E, that is the run-time version of Carmot. The actual functional UI operations in GUI command scripts are implemented as a series of custom GUI-related Carmot-E procedures and $functions, and the link to the underlying data which is held in a Mitopia® collection and described by Carmot, is established by designating the collection node to be used at any given time. The node of course allows the interpreter to use Mitopia’s Carmot APIs to discover types, fields and all other information it needs to interpret the commands and data references within the GUI script. This use of the Carmot language to bridge the gap between Mitopia® code/data and the GUI framework, not only accomplishes the required degree of separation between GUI and application code, but also extends this separation all the way to the ‘discovered’ ontology, thereby further immunizing the compiled code from system-specific behaviors.
From the Mitopia® perspective, the ‘local’ renderers are simply the principal members of a number of potential ‘renderers’ which are dynamically registered with the architecture during application startup. Other renderers (via Universal Renderer plugins) might generate web pages, or might use completely different GUI frameworks without in any way impacting core architectural Mitopia® code. A Mitopia® widget generally does not know or care if its GUI is local or not – in any given Mitopia® process there may be a mixture of threads, some using local and some remote GUIs.
The Mitopia® ‘RendererPlugin’ package implements all the required renderer registries and resolution logic. In GUI discussions here and elsewhere, we give examples, and use screen shots, generated by the local renderers, however, it is important to bear in mind that alternate renderers can be registered simultaneously and might be used for different delivery platforms (e.g., a web browser interface) without impacting the functional thrust of the discussions.
GUI Environment Principles
Mitopia® requires and enforces a small number of functional assumptions and behaviors on any GUI environment provided by a renderer, and any framework that violates these requirements cannot be used in conjunction with Mitopia®. The requirements are made for the purpose of allowing Mitopia® to define an abstract UI model and registration interface that is capable of implementation by multiple renderers and does not allow renderer-specific assumptions to creep across the interface or to violate the assumptions of Mitopia’s cooperative threading model.
One of the most critical aspects of Mitopia’s programming model is the threaded nature of the code components known as widgets that are combined in order to create a complete Mitopia® application that can run effectively based on the laws of data-flow. This means that every widget must be functionally isolated from all others in its own thread. Each widget is allocated scheduling time by the Mitopia® scheduler according to data-flow and other scheduler-specific considerations. What this means for the GUI environment is that each widget’s GUI environment must be 100% isolated from that of any other widget. To fail to do this raises the potential for deadly embraces and lock/race conditions between widgets mediated through thread conflicts in the GUI layer.
The overwhelming bulk of all GUI frameworks tightly integrate the ‘Controller’ and ‘View’ aspects of the framework by providing their own internal event delivery mechanism. Unless the necessary plug-ins and callbacks are provided by the GUI framework to allow event delivery and all other GUI behaviors to be completely isolated so that Mitopia® retains control of which thread runs at any given time, the GUI framework is hard to adapt for use with Mitopia®.
At the fundamental level, this means that the GUI framework must perform all of these operations using an ‘application GUI context’ which can be switched by Mitopia™ as desired. This also effectively means no non-constant static global variables are allowed in the GUI framework, that the framework cannot assume a single thread or any framework control over thread switches, cannot assume a direct connection to underlying OS event delivery mechanisms, and cannot assume it has complete control of the entire screen or window layout. Most GUI frameworks do not support this level of customization and thus are unsuitable for Mitopia’s highly threaded and data-flow driven environment. These requirements placed on GUI frameworks by Mitopia® are in reality no more than requiring good framework design and coding principles, however, few actual frameworks implement this level of design or code rigor in practice.
Some frameworks attempt to address these issues by moving all GUI operations off to a separate GUI thread and passing messages to/from that thread (Java is an example). Unfortunately, such a strategy is rife with potential problems when combined with a data-flow based scheduling algorithm, since the GUI thread becomes an uncontrolled means of ‘entangling’ the operation of other widget threads and creating undetectable lock and race conditions between them. These ‘separate’ GUI thread frameworks can thus be eliminated from consideration, in fact GUI environments that assume a single application thread but follow good code design principles (like avoiding static globals), are far more easily adapted (by context switching) than those that attempt to address threading themselves. To illustrate how easily separate GUI threads can bring a highly threaded system to its knees, consider the following scenario if implemented using a separate GUI thread.
A user is in the process of using the File-Browser as initiated from UI from client widget ‘A’. For simplicity, most GUI frameworks and OS toolbox libraries would handle this as a ‘modal’ situation, that is they would hang in a ‘modal’ loop waiting for the user to make his pick and dismiss the dialog. Now suppose that running in another thread of the same application is a server which makes a harmless call to the GUI layer to ‘read’ the value in a UI control associated with the server’s UI. This ‘read’ must be sent to the GUI thread for implementation, however, the GUI thread is currently looping waiting for the user to pick a file (or any other modal situation) and is unable to service the request immediately. What this means is that the server is now locked up waiting for the user to pick his/her file. Perhaps the user has gone to get some coffee! This of course means the server is no longer able to process incoming server requests from other clients. This in turn will lead to communications timeouts and other errors with the clients. In other words, the GUI thread has managed to entangle multiple processes across the network and cause the entire application to malfunction by any modal behaviors. If fact, there are a number of other means of entangling external threads through a separate GUI thread that do not even involve modality, this is particularly true when ‘remote’ GUIs are involved.
Mitopia® provides its own internal event delivery system which extends the common UI style events (e.g., mouse down, key press etc.) to add a wide range of other specialized event types that are used by Mitopia® to perform various communications and housekeeping activities and are completely unrelated to but integrated with the UI events. For this reason, Mitopia® allows custom event plug-ins to be registered through the ‘platform’ registry API provided by the ‘RendererPlugin’ package. These platform plug-ins are responsible for capturing events from the OS event mechanism and inserting them into the internal Mitopia® event queues targeted for the appropriate widget(s), and retrieving them in the widget context. Mitopia® pre-registers a built-in platform plugin to capture platform events. It is desirable that all events, including GUI events, go through the Mitopia® event queues in this manner since the Mitopia® scheduler examines the internal queues for pending events in order to adjust its scheduling priorities to ensure that widgets with pending events get scheduling slots rapidly. It is therefore required that any registered GUI renderer is capable of operating normally when events have been routed in this manner rather than pulling events directly from the OS event queues.
Mitopia® logically breaks the executable widget code into a set of ‘views’. Each view can have a single ‘view window’ associated with it. Each view can contain one or more widgets within it, and each widget may have an associated ‘pane’ which is a nominally rectangular region of the view window. In the local GUI environment, panes within view windows in the platform context are implemented by Mitopia® using borderless windows that are locked in position relative to the view window and scale with it. This gives the appearance of a single window on the screen whereas in reality each widget has its own exclusive borderless pane window in which it displays its UI. From the perspective of the GUI framework, each pane window and widget represents a completely distinct application with no connections whatsoever to any other GUI in the Mitopia® application. Mitopia® is responsible for manipulating and ordering the view windows and panes, the GUI framework is responsible only for displaying and handling the content within a pane. Widgets can create daughter windows and dialogs which appear floating above the view window and widget/GUI code is responsible for handling these windows even though the events for these windows are routed through the Mitopia® event queues just like all other events. Only the daughter windows for the frontmost or ‘focus’ widget are showing at any given time, they are hidden by Mitopia® as soon as the widget loses focus.
Within any given widget pane or window, Mitopia® code assumes that the GUI renderer can provide a GUI environment that meets the following minimum criteria:
- Individual interactive areas in the window are known as controls. A full set of modern standard control types is provided such as text controls, style text, list controls, menus, popups, buttons, radio buttons, checkboxes, sliders, tabbed panels, image displays, etc.
- The GUI environment supports hierarchical nesting of controls within other controls which are referred to as ‘view controls’. The GUI environment handles all the implications of re-sizing within this hierarchical nesting to as many levels of nesting are required.
- The GUI environment understands and implements drag & drop (D&D) and provides the necessary plug-in hooks to allow D&D behavior to be modified within Mitopia®.
- The GUI environment provides some form of graphical GUI creation tool and also supports the creation and layout of all aspects of the UI through API calls.
- The GUI environment provides hooks to allow Mitopia® to potentially translate each and every bit of text that the GUI framework displays anywhere in the UI. This dynamic translation mechanism is critical to Mitopia’s ability to translate user interfaces on-the-fly to other languages.
- The GUI layer is capable of data-discovery through multiple registered data models (i.e., an adaptive UI).
- No operation performed by the GUI will result in an attempt to switch to another thread context or an attempt to render or access UI in any thread other than the one currently executing.
- The entire interface to the GUI environment can be expressed or encapsulated in C.
- The GUI environment never hangs in tight loops using OS native status APIs like ‘WhileMouseDown’. If they do, this must be patched.
- All renderers must support the concept that a control has associated with it a name string which can be any ASCII string. However, the majority of controls are assigned numeric values (as strings) for the name, and this is the control ‘label’ used throughout most Mitopia® calls from the GUI interpreter. Labels are used in preference to arbitrary name strings since they are far easier and more concise to manipulate in code, however for certain controls, the use of arbitrary strings may be necessary despite the fact that this makes the control harder to ‘find’ through the interpreter API.
- The renderer must support the ability to attach an arbitrary memory allocation to the controls within it. This allocation is used by Mitopia® to associated ‘tags’ with GUI elements in order to maintain and recover Mitopia®-related context in the renderer controls.
- All text displayed in the UI is encoded in UTF-8.