A Taste of Nested Classes, part 2

(Continues from part 1)

Let’s now look at some message sends. Suppose we design the application so that the message respondToHit is sent to an asteroid when it’s hit by a missile. In Smalltalk, a typical response would look something like

respondToHit
    game
        replace: self with: self fragments;
        incrementScore.

game here is an instance variable holding a reference to the Game instance the asteroid belongs to, presumably initialized at the time the asteroid was created. How could we port this to Newspeak? Translating everything literally, we would define a slot named “game” in the Asteroid class and port its initialization logic. We would then port all the methods. The original respondToHit would remain unchanged, apart from the overall “packaging”:

respondToHit = (
    game
        replace: self with: self fragments;
        incrementScore. 
)

But is it all the same as before? Not quite, because just like in Self, Newspeak slots are accessible only through messages. What looks like a good old game variable reference has now become a send of the message game to (an implicit) self.

Let’s hold this thought and backtrack to the end of the previous post. We said that an asteroid always knows the game instance it belongs to, it’s the enclosing object. We don’t need the game slot we ported from the Smalltalk original because it holds onto the same thing!

We can drop it and any of its initialization logic because the language now keeps track of the enclosing game instance for us. Instead suppose we define this method in Asteroid:

game = (
    "Get and return the enclosing object. To avoid getting ahead
    of the presentation, we don't show how it's done yet."
)

As for respondToHit and any other methods, they are unchanged—a perfect example of the benefits of representation independence. When game is just a message send, nobody cares how it gets its result.

Now, instead of revealing how the method game is implemented, we are going to do something even better. We are going to get rid of it altogether.

Time to talk about implicit receiver sends. I’ve hinted more than once that an implicit receiver is not always self. If it’s not self, what else can it be? Is there any other object that is somehow implicitly “present” at the location where the message is sent? Of course, it’s the enclosing object. (Or objects, if there is more than one layer of nested classes). Think of it this way: the respondToHit method is contained not only in the Asteroid class, but also indirectly in the Game class. So, the code in the method runs not only in the context of the current receiver, an instance of Asteroid, but also in the context of the enclosing instance of Game.

Without further ado I’m going to show respondToHit rewritten to take advantage of implicit receiver sends, and then describe their exact behavior. This will also explain under what specific conditions such a rewrite would be possible.

respondToHit = (
    replace: self with: fragments.
    incrementScore.
)

This reads almost like plain English. replace:with: and incrementScore are now sent to the enclosing game via the implicit receiver mechanism. Note that fragments is also changed to have an implicit receiver, though in this case we expect that the receiver is self. How does it happen that in both cases the actual receiver is what we want it to be?

Let’s recap how these potential receivers are related. We have essentially a queue of them: first self, then its enclosing object. (With more than two levels of nested classes, it would be followed by the enclosing object of the enclosing object and so on). If we represent the situation as a picture of the classes of the objects involved and their inheritance, we get this comb-like structure (to make the picture a little more interesting, Asteroid here is subclassed from a hypothetical ScreenObject, even though our original code didn’t say that).

Classes potentially involved in message processing

Because a method with a matching selector might be defined by any class in the comb, it may seem that the most intuitively reasonable lookup strategy is the one adopted by NewtonScript. The policy there was to search the entire comb starting with the receiver and its superclasses, then the receiver’s enclosing object (called parent in NewtonScript) with its superclasses, and so on. In the example we would look first in Asteroid, ScreenObject and Object with the intent of the Asteroid instance being the receiver, then Game and Object with the intent of the Game instance being the receiver. The send would fail if none of the classes in the comb implemented a method to handle the message.

I emphasized “seem” because the strategy in Newspeak is different, for the reason I am outlining in the end of the post. But first, here is how it really works:

  • If one of the classes on the “trunk” of the comb implements a method with a matching selector—in other words, if a matching method is lexically visible at the send site—the message is sent to the instance of that class.
  • Otherwise, the message is sent to self.

Even more informally, we could say that when by “looking around and up,” “around” meaning the same class definition and “up”—the definitions it’s nested in, we can see one with a matching method, the instance of that class is what receives the message. Otherwise, it is sent to self (and results in an MNU if self does not understand it).

So, the lookup in Newspeak differs from NewtonScript-like approach in two important ways. First, it doesn’t traverse the entire comb. The only potential handlers of the message are the methods defined in the trunk classes and those in the superclasses of the class of self. Second, lexical visibility takes priority over inheritance. If a method of Asteroid sends the message redraw to an implicit receiver and a method named redraw is defined in both Game and ScreenObject, the ScreenObject implementation wins in NewtonScript, while the one in Game wins in Newspeak.

This explains how the correct receiver is chosen in our implementation of respondToHit. As long as replace:with: and incrementScore are defined in Game, they are sent to the enclosing game instance as lexically visible. As for fragments, it’s sent to the asteroid no matter whether it’s defined in Asteroid or inherited. If Asteroid defines it, it’s lexically visible and the asteroid receives it according to the lexical visibility rule. If ScreenObject defines it (and assuming Game doesn’t), it’s lexically invisible and the asteroid receives it because the asteroid is self.

What if Game also had a method named fragments? In that case the definition in Game would win, so in order to get the fragments of the asteroid we would need to write the send explicitly as self fragments. What if replace:with: is inherited by Game from the superclass? In that case we also need to make the receiver explicit, because otherwise it would go to self:

game replace: self with: fragments.

This brings us back to the problem of writing a method named game returning the enclosing instance of Game. Now we know enough to come up with a very simple solution: we add the method to the class Game (not Asteroid), defined as

game = (
    ^self
)

So, now that we know how it all works—why does it work this way? Why bring lexical scoping of messages into the picture? Is it Rightâ„¢ to give priority to methods of another object over inherited methods of the receiver? Why not just follow the NewtonScript example—it appears simple and reasonable enough.

The thing is that compared to NewtonScript, class nesting in Newspeak is intended to solve a very different problem.

NewtonScript is prototype-based, and its object nesting is motivated by the need to represent the nesting of UI elements and support communication within that structure. The nesting and the associated message processing is essentially a built-in Chain of Responsibility pattern, very common in the UI field. (As an aside, Hopscotch implements NewtonScript-like mechanism for sending a message up the chain of nested UI elements. This is implemented as a simple library facility with no more language magic than doesNotUnderstand:. Implicit receivers further help make this very unobtrusive).

In Newspeak, even though the “free” back pointer from Asteroid to Game in our example was nice to have, having it was not the primary reason for nesting one class in the other. In fact, class nesting is a rather unwieldy mechanism for implementing arbitrary parent-child relationships—imagine having to define a group of classes for the scenario of “buttons and list boxes in a window,” and an entirely different one for “buttons and list boxes in a tab control in a window”!

Newspeak-like class nesting and method lookup are designed to handle relationships that have more to do with the modular structure of code, and with the organization of information sharing within a module. A is nested in B when B is a larger-scale component as far as the system architecture is concerned. Even though such nesting does effectively create a parent-child relationship between Bs and As, there are parent-child relationships that do not justify nesting.

In nested Newspeak classes, a child calls onto the functionality provided by the parent not as a fallback case in chain-of-responsibility processing, but as a way of interacting with its architectural context. The lookup policy supports that and at the same time allows to modularize the context. That is the motivation behind putting lexical visibility before inheritance.

If this sounds somewhat vague, I hope some of the more realistic examples in the next part will help clarify this point.

Continues in part 3.

A Taste of Nested Classes, part 1

In the recent posts I mentioned that because of class nesting the receiver of an implicit-receiver message send is not necessarily “self”. The full story with a comparison to other approaches is told in Gilad’s paper On the Interaction of Method Lookup and Scope with Inheritance and Nesting. In this series of posts I provide an informal example-oriented introduction for someone familiar with Smalltalk, to help better understand the full story.

First an important terminology note. When we say “class” we can refer to either a (textual) class definition or a class metaobject. Because the relationship between the two in Smalltalk is normally one-to-one, we can pretty much ignore that ambiguity. The Newspeak situation is different, and not thinking of definitions and metaobjects as separate things is perhaps the easiest way to get confused. To avoid that, in this post I’ll stick to spelling out “class definition” or “class metaobject” fully, unless there is no danger of confusion.

Quite naturally, “nested classes” first of all mean nested class definitions. Why would we want to nest one class definition inside another? This, of course, largely depends on what exactly such nesting translates to and what benefits it brings, and that’s what this whole series is about. For starters (even though this by far is not the primary benefit) let’s assume that we do it as a matter of code organization. For example, suppose we are implementing the immortal Asteroids game and decide to nest the definition of the Asteroid class inside the Game class. In the current Newspeak syntax it would look something like this:

class Game = ()
(
    class Asteroid = () ()
)

This rather basic implementation will not bring us hours of gaming enjoyment, but at least we do now have one class definition nested inside the other. How do we work with this thing?

Starting from the outside, in the current Newspeak system hosted in Squeak, this definition would create a binding named “Game” in the Smalltalk system dictionary, holding onto the Game class metaobject. This means that in a regular Smalltalk workspace we could evaluate

Game new

and get an instance of Game.

I want to emphasize right away that despite what this example may suggest, Newspeak has no global namespace for top-level classes. This availability of Game as a global variable in Smalltalk alongside the regular Smalltalk classes is the scaffolding, useful here to write examples in plain Smalltalk to keep things more familiar for the time being.

So what about Asteroid—how could we instantiate that one? Nesting a class in another essentially adds a slot to the outer class definition, with an accessor to retrieve the value of the slot. That value is the metaobject of the inner class. So, we get an inner class (metaobject) by sending a message to an instance of its outer class (metaobject). Notice that it’s not “by sending a message to the outer class”—“an instance of” is an important difference:

game := Game new.
asteroidClass := game Asteroid.
asteroid1 := asteroidClass new.
asteroid2 := asteroidClass new

So far nothing really surprising, apart from having to go through an instance of one class to get at another class. Let’s now change the example:

class1 := Game new Asteroid.
class2 := Game new Asteroid.
asteroid1 := class1 new.
asteroid2 := class2 new

This is more interesting. asteroid1 and asteroid2 are both instances of Asteroid and behave the way the class definition says. But since the Asteroid class references come from different game instances, what can we expect beyond that? For example, what would these evaluate to:

asteroid1 class == asteroid2 class
asteroid1 isKindOf: class2

The answer is false, for both of them. The two instances of Game produced by Game new evaluated twice contain independent Asteroid class metaobjects. Their instances asteroid1 and asteroid2 are technically instances of different classes, even if those metaobjects represent the same class definition. (Relying on explicit class membership tests is a poor practice in Smalltalk, and this illustrates why it’s even more so in Newspeak).

Let’s now look at another variation:

game := Game new.
asteroid1 := game Asteroid new.
asteroid2 := game Asteroid new.

In this case

asteroid1 class == asteroid2 class

is true because the two sends of the message Asteroid retrieve the same Asteroid class metaobject held onto by the instance of Game.

Let’s now summarize the objects involved and their relationships. They are shown in the following picture

Nested classes illustration

There is a class metaobject for the Game class, contained by the system dictionary (shown in gray to emphasize that it’s not an “official” part of the scheme). An instance, connected to the class with a dotted line to represent the instance-of/class-of relationship, holds the Asteroid class metaobject. That metaobject has instances of its own, which are the asteroid1 and asteroid2 of our example.

If the Game class has additional instances (one such instance is shown small in the diagram), each instance has its own copy of the Asteroid class metaobject. If there were multiple levels of nested classes, the scheme would repeat for the deeper levels.

The diagram also illustrates a concept that will be central to the next part of this series: the enclosing object. In terms of our example, an enclosing object of an asteroid (instance) is the game (instance) that owns the class (metaobject) of the asteroid. This relationship is obviously quite important and useful: simply put, it means that an asteroid knows what game it is a part of. That’s simply by virtue of nesting the Asteroid class definition inside Game, without having to explicitly set up and maintain a back pointer from an asteroid to its game.

(Continued in part 2).