Archive for February, 2009

I’m feeling lucky

Saturday, February 28th, 2009

Just a few months ago I never would have thought I’d work on anything other than Newspeak for the foreseeable future. Working with Gilad, Peter, and others was a fantastic experience, and building a programming platform almost from scratch without political and marketing baggage attached was not a job you find every day on monster.com. But it turns out sometimes the foreseeable future runs out sooner than you expect.

The end of full-time Newspeak work is a sad thing, but I’m not writing this to cry over spilt milk. They say new beginnings bring new opportunities, and this is how I want to look at it. I have just finished my first week working for Google.

A Taste of Nested Classes, intermission

Sunday, February 22nd, 2009

(Continues from part 3).

This post is an intermission because it’s not specifically about nested classes. In fact, it’s not about anything that really exists in Newspeak. Instead, it reviews a few hypothetical examples to show off some of the expressive capabilities of implicit receiver sends.

Most expressions in Newspeak begin with an implicit receiver send, at least conceptually. In something like

foo size

or

OrderedCollection new

both foo and OrderedCollection really are message sends, for some definition of “really”.

Of course, in some cases we can’t do without real receivers. There are literals, as well as traditional pseudo-variables (“reserved words”), whose interpretation should be hard-coded into the language.

Or should it be?

Starting with literals, imagine that every occurrence of a literal is compiled as an implicit receiver send identifying the type of the literal, with the “primitive” value of the literal as the argument. For example,

'Hello, brave new world!'

would really mean

(literalString: 'Hello, brave new world!')

with literalString: implemented in Object as

literalString: primitiveValue <String> = (
    ^primitiveValue
)

At this point this is only a possibility, but one that is explicitly mentioned in the language spec (5.1). If implemented, this would allow user code to redefine the interpretation of what appears to be literals for greater flexibility when implementing internal DSLs.

As for pseudo-variables, consider this implementation of the class Object:

class Object = (
    ...

    self = (
        "empty - return the receiver"
    )

    true = (
        ^True soleInstance
    )

    false = (
        ^False soleInstance
    )

    nil = (
        ^UndefinedObject soleInstance
    )

    thisContext = (
        ^ActivationMirror forSenderContext
    )
)

With this, most of the reserved words of Smalltalk become regular messages. The expression

^self foo: nil

now really means

^σ self foo: σ nil

with σ denoting a reference to (usually!) the receiver unrepresentable in the language syntax. We use the fact that an empty method effectively reads as

to grab that elusive σ in the self method.

super is more complicated because it doesn’t denote a separate object but rather self with a special mode of message lookup. However, even that can be lifted from language magic into a reflective implementation at the library level:

super = (
    ^SuperDispatcher forReceiver: self
)

with SuperDispatcher implementing doesNotUndestand: to reflectively look for the method implementing the message starting from the superclass of the implementor of the sender context’s method. (A mouthful, but that really is what it should do).

In a similar way, outer, the new reserved word of Newspeak not found in Smalltalk could be implemented as a message send handled by reflective code at the image level.

The last two examples, super and outer, are even more hypothetical than the other pseudo-variables. While technically possible, a reflective implementation of those is quite expensive compared to the usual “language magic” solution.

Also, this scheme in general leaves the meaning of things like self or nil open to changes by a class for all its subclasses and nested classes. Perhaps there is such a thing as too much flexibility. But even if these ideas are too much or too expensive for the core language, all of the same mechanisms are available to a designer of an internal DSL, and that’s what makes them interesting. For example, the “dispatcher receiver” pattern illustrated with super and outer was conceived in Hopscotch and used there to support NewtonScript-like object hierarchy-bound message dispatch.

(Continues in part 4).

A Taste of Nested Classes, part 3

Sunday, February 15th, 2009

(Continues from part 2).

After an unplanned hiatus caused by various real life issues including the irritating surprise of becoming unemployed in these interesting times, here is the long-promised third part of the nested classes series.

Class nesting in our earlier example represented the natural nesting relationship of an asteroid object inside the game that contained it. Implicit receiver sends were a convenient mechanism of communicating with the game from inside an asteroid instance “for free”, without the need to maintain an explicit object reference. Today we look at a few examples with implicit sends of a slightly different flavor.

In a typical Smalltalk system, methods in the Object class define the behavior common to all objects. Some of them, such as class or identityHash are implemented to return some interesting information extracted from the receiver. Others, such as isNil are overridden in subclasses to vary their behavior polymorphically, again depending on the receiver object. In both of these cases, the receiver is involved one way or the other.

There is a third kind, methods like halt, error:, or flag: (in Squeak). They don’t extract any information from the receiver. Also, they are not overridden, so the implementation in Object tells the full story of their behavior. We send them to self and expect that they behave the same regardless of what self happens to be. As far as these messages are concerned, the receiver itself is unimportant.

These messages are utilities representing “ambient” functionality—something potentially useful everywhere in the code. We send them to self because it’s the most readily available receiver, and they are implemented in Object so that the sends work regardless of what self happens to be. Object in this case doubles as the container of these utility functions and inheritance makes them globally visible.

This global visibility is what also makes such utilities potentially problematic. Suppose I want to have a logError: utility that writes an entry into a log file, and I want to use it throughout my application. It’s fine to have it in Object only as long as another application doesn’t decide to use the same message but log the error to the system console instead.

Here is an alternative way to implement such utilities in Newspeak.

class AsteroidsGame = (...)
(
    ...
    class Space = (...)
    (
        moveObject: anObject to: position = (
            ...
            logError: 'invalid position'
        )
        ...
    )

    class Asteroid = GameObject (...)
    (
        respondToHit = (
            isLive ifFalse: [logError: 'invalid asteroid'].
            ...
        )
    )
    ...

    logError: message = (
        errorStream nextPutAll: message; cr.
    )
)

We put logError: in the top-level class representing our application. Any method of any nested class can invoke that utility through an implicit receiver send. The superclass of the nested class doesn’t matter—the utility is accessible because the class is nested in the one that provides it. Compared to the Smalltalk example, the utility is now available everywhere in our application and nowhere outside of it.

Here is another thing that’s interesting about the Newspeak alternative. In Smalltalk, logError: would have to be an extension method of Object. Extension methods are officially ungood in Newspeak, and the example shows the goodthink alternative.

In fact, such use of class nesting as a scoping mechanism for utilities solves an entire class of data conversion problems traditionally solved by extensions. Say, we want to convert an integer to a string with its representation as a Roman numeral. A common solution would be to add an asRomanNumeral extension method to Integer to do the conversion. This is convenient because we can write x asRomanNumeral anywhere, but has the disadvantage of the function leaking into the rest of the system. A more controlled alternative of implementing a convertToRomanNumeral: method in one of the application classes has the disadvantage of being easily accessible only in the implementing class and its subclasses. In other places we need to arrange for a reference to an instance that understands the message. In Newspeak, convertToRomanNumeral: can be a method of the application (module) so that it’s easily accessible, but only within the module.

Here is an interesting hypothetical example that continues the same theme. Suppose we implement the following method in a top-level class:

if: condition <Boolean> then: then <Block> else: else <Block> = (
    ^condition ifTrue: then ifFalse: else
)

Anywhere in the class and its nested classes we can now write conditionals in this more familiar to the general public style:

if: a < b
then: [^a]
else: [error: 'invalid a']

This example is hypothetical because it could only be fast enough on a very sophisticated VM with a good adaptive optimizer. But as far as less common and therefore less time-critical control messages go, they are no different from other utilities. We can define them in the top-level application class and use them throughout the application. For example:

^return: 5 if: #bar notUnderstoodIn: [foo bar]

defined in a top-level class as

return: value <Object> if: selector <Symbol> notUnderstoodIn: block <[]> = (
    ^block
        on: MessageNotUnderstood
        do: [:ex |
            ex message selector = selector
                ifTrue: [ex return: value]
                ifFalse: [ex pass]]
)

To sum up, some methods define the kind of ambient functionality that is used throughout an application. For that reason it’s best if it can be accessed implicitly, without arranging for an explicit reference to the object that provides it. In classic Smalltalk, the only mechanism to support that is inheritance, with the disadvantage that the functionality defined this way escapes the defining module. Nested classes and implicit receiver sends in Newspeak provide an alternative mechanism that keeps such ambient operations contained inside the module.

(Continues in intermission).