On UI Layout

Michael Lucas-Smith posted an interesting summary of UI layout models. I tried to leave the following as a comment, but the blog server swallowed it without posting. Fortunately, I half-expected this and copied the text before submitting. I’m posting it here instead, with some further edits.

Overall, this is a good summary, but there is a very important model it misses.

The model is a variant of constrained-based layout called linear constraints. In that model widget positions are expressed as a system of linear equations. This more formalized approach, unlike “intuitive” models with springs, wires and other pseudo-real life accessories, isn’t fiddly at all. There are algorithms (Cassowary is the most famous) to efficiently solve such constraint systems.

Also, importantly, many other models: fixed, relative, stacked and grid layouts are special cases of linear constraints. Only flow cannot be expressed this way. Thus, the models are not all incompatible. Also note that I’m listing “relative” as an additional model, which is the correct way to describe the VW approach (first proposed, most likely, by Luca Cardelli). (“Fixed” would be the case when the widget’s bounds are expressed as coordinates in some coordinate system, usually relative to the containing widget’s top left corner).

I’m not sure what it means to say that layout should not be part of the widget framework proper, without first defining the proper. One extreme is Windows which provides no automatic layout capabilities at all. That is indeed the pure fixed model. The other are frameworks with container visuals. In those frameworks the specific layout model is indeed dictated by the container. But still, the only difference between the two is that something like Qt has containers that do layout and Windows doesn’t. In neither of them layout is part of widgets (by widgets I mean atomic leaf components such as buttons or list views).

The important question is how hard or easy it is to introduce new containers with different layout models, but it is a matter of the overall framework design. The main complication here are performance constraints. The framework should reasonably quickly respond to incremental layout changes, and reconciling that need with pluggability of containers and their layout policies is where the devil lies. I’d estimate about 70% (if not more) of Brazil design effort has had something to do with layout management.

Which is Which in Newspeak UI – Brazil

Brazil, which is the foundation of Newspeak UI (and is indeed named after the movie), could be best described as a multi-platform logical UI and layout framework. It provides the familiar widgets (“visuals”) such as Labels, Buttons, and ListBoxes, and container visuals for combining widgets and managing their layout. Unlike the UI frameworks of Squeak or VisualWorks, Brazil does not attempt to actually implement the widgets by emulating them. Instead it functions as a “logical UI” layer that builds and drives the equivalent “physical UI” made of native artifacts of the host platform.

The first host platform of Brazil was Morphic–meaning the first “native” Brazil interfaces were in fact emulated. This trick allowed us to quickly build a minimal working implementation and polish the API without getting bogged down with FFI issues and native widget peculiarities early on. Since Morphic works on all OS platforms, so did Brazil from the start, allowing to use it on all platforms we need before we build real native mappings for those platforms.

Brazil also manages the layout of interfaces using a capability-based extensible model. Layout is fully managed by Brazil both on native platforms which usually provide rather weak layout facilities of their own, and in Morphic which provides a rich but baroque one. All that Brazil expects from the host are the abilities to position a widget within the specified rectangle, and to measure its natural (desired) extent.

Work on Brazil began in mid-March 2007 and in June a first real UI was built using it. I am currently working on a native Windows mapping. This work is at an advanced enough stage to be able to open all of the existing Brazil UIs as native in Windows with most functionality in place, without any changes in application code. Brazil windows can also be dynamically remapped, so that what appears as a Morphic window can “jump out” of Squeak and become a native window with native controls, and then go back to Morphic all the while retaining its state.

Unlike prior implementations of native UIs in Smalltalk, Brazil is implemented entirely at the image level. This means that there is not a single VM primitive to support UI operations. The mapping relies on “raw” OS functions accessed via the general-purpose Newspeak FFI to do everything, from Windows class registration to widget management and GDI+ painting.

Hopscotch Talk at Smalltalk Solutions

I’m presenting what I’ve been busy with for the past year and a half (well, almost) at Smalltalk Solutions in June. The title of the talk is “Interfaces without Tools”.

Programming environments are commonly built as sets of tools, where a tool has the shape of a pre-composed window displaying and manipulating a set of domain objects. This talk takes a critical look at that approach and presents Hopscotch, an application framework and development environment based on different concepts. Hopscotch is the IDE and the application framework of Newspeak, a new language and development platform inspired by Smalltalk and Self.

The talk is on the first day of the conference, June 19 at 1:30 pm.