Message chains

On the squeak-dev mailing list, Fabio Filasieno raised the issue of “pipes” in Smalltalk, which generated quite a discussion. Since I’ve provided an implementation, I’m also writing down my thoughts on the subject.

In a nutshell, this is what the idea is about. While the “classic” Smalltalk syntax allows us to write chains of unary messages such as

x foo bar

without explicit parentheses, this is not the case with keyword messages.

(x foo: 1) bar: 2

is a chain similar to the one above, yet its syntactic form is somewhat further removed from the idea of a “smooth” linear progression of operations, especially so when there are more than two consecutive message sends:

(((x foo: 1) bar: 2) baz: 3) zork: 4

Imagine now that we have an operator which I will for now designate as “:>” allowing us to rewrite the above as

x foo: 1 :> bar: 2 :> baz: 3 :> zork: 4

In other words, it makes the result of the preceding expression the receiver of the following message send, so that in fact

x foo :> bar :> baz

and

x foo bar baz

are two syntactic forms of the same chain of messages.

One obvious example of the use of this operator is rewriting something like

(aCollection select: [:some | some foo]) collect: [:each | each bar]

as

aCollection select: [:some | some foo] :> collect: [:each | each bar]

An even more likely case, which I don’t believe was mentioned on squeak-dev, are complex conditions:

something thingsAt: aKey :> includes: aThing :> 
    ifTrue: [...]
    ifFalse [...]

Writing the “standard” form often involves going back to the beginning of the expression to add parentheses once you’ve written enough of the expression to realize what it will look like. The chain operator exactly follows the “do this, then this, then that” structure of such expressions and does not require going back.

The change set linked at the beginning of the post implements the operator for Squeak. The inevitable question now is, Is This A Good Thing?

Clearly it adds nothing new to the language—chaining messages was always possible—only a new ability to express something in a different syntactic form. Such things generally invite (justified) suspicion and tossing around the various “less is more” quotes. Without questioning the value of minimalism and structured approach, some prior “sugary” additions such as tuples (brace Array constructors) or ifNil:/ifNotNil: could be (and have been) criticized on the same grounds. Any Array constructor discussion usually involves someone pointing out how the ease of array creation is overrated because redesigning the code to replace arrays with structured objects or store them in self-documenting variables only improves the code. This is true, of course—and yet in some contexts, one of which are DSL-like Smalltalk expressions, syntactically lightweight array creation turns out quite useful. Perhaps there is a similar “killer use” out there waiting for lightweight message chains to become available. Or perhaps not. My take is that this operator might be a good thing, and only practice can show if there are use cases out there waiting for it.

As for less is more—perhaps the power of Smalltalk is not as much in having a small untouchable core (and if anything, Object and UndefinedObject protocols in Squeak live to prove that more is more) as in having a core small and malleable enough to support this sort of extensions and experimentation. I’d rather say Squeak core is still not small and malleable enough if something like this can only be done by modifying the compiler rather than tweaking the meta-level.

Having said this, I should comment on Bert Freudenberg’s elegant #asPipe implementation. There are two reasons why I consider it very neat but still not enough of the real thing. One is the look of the code. The value proposition of pipes is improving readability. Getting rid of parentheses is only one step towards that, the other is having the links of the chain stand out enough for the reader to easily see where the distinct steps are. This is partly an issue of formatting, but proper “graphic design” of a visually distinct special operator is still unbeatable.

In fact, in terms of visibility I think the operator even better than “:>” (which already looks quite like a smiley) is “:)”:

something thingsAt: aKey :) includes: aThing :)
    ifTrue: [...]
    ifFalse [...]

The other issue are minor anomalies that are always hard to hide with DNU tricks because of the need for the DNU handler to have some rudimentary behavior of its own, and because some things are not handled by messages. For example:

2 asPipe between: 1 and: 3; == true

evaluates to false.

2 asPipe between: 1 and: 3; ifTrue: [#foo]

fails to compile, ruling out the use of pipes to simplify conditions.

foo := [#nil].
bar := [#notNil].
nil asPipe value; ifNil: foo ifNotNil: bar

does compile because the branches are not literal blocks, but then the result of the “pipe” is #notNil.

NotNil, then what?

This is about a fix of a conceptual bug in the current version of Squeak, though it’s the detailed analysis why this was a bug that I think is worth a blog post.

A while ago when ifNil: and ifNotNil: were not standard in the commercial Smalltalks of the day, there was an occasional argument on c.l.s now and then whether they were a good thing or superfluous sugar not providing any new functionality. The typical case against ifNil: is something like

foo ifNil: [self doSomething]

which can trivially be rewritten in “classic” Smalltalk. The case that is less trivial, though, is

^self foo ifNil: [self defaultFoo]

It’s important that the value being tested is computed and not just fetched from a variable. For a computed value, we cannot in the general case reduce the above to

^self foo isNil ifTrue: [self defaultFoo] ifFalse: [self foo]

because this would evaluate self foo twice—something we would want to avoid if self foo had side effects. Avoiding double evaluation would call for a variable to hold onto the result of self foo:

| foo |
foo := self foo.
foo isNil ifTrue: [self defaultFoo] ifFalse: [foo]

which is significantly more verbose than the ifNil: alternative. So, while ifNil: in this case can still be reduced to “classic” Smalltalk, doing so takes us down an abstraction level—from a single message doing exactly what it says to messing around with variables, tests and branches. While in some languages such exposed plumbing is a fact of life, in Smalltalk we like to hide it when we don’t need to deal with it directly.

Now, looking at ifNotNil:, it’s important to note that its typical use case is different. In fact, this is what’s interesting about ifNil: and ifNotNil: in general—they are asymmetrical. While ifNil: allows us to provide a useful value in the cases when the “main” branch of the computation returns nil, it’s unlikely that we’d ever use ifNotNil: in a mirrored pattern as

^self foo ifNotNil: [nil]

This shows that the cause of the asymmetry is the obvious fact that typically nil is used as a token indicating the absence of a value. A non-nil object is interesting in its own right, while nil isn’t.

So, ifNotNil: is primarily useful not as a value-producing expression, but rather as a control statement that triggers computation when another expression produces a useful result, in the simplest case going as something like

foo ifNotNil: [self doSomethingWith: foo]

This use of ifNotNil: could again be reduced to isNil ifFalse:. The case when the receiver of ifNotNil: is computed is more interesting because of the same problem of avoiding multiple evaluation, the obvious solution to which would again make the code much bulkier:

| foo |
foo := self computeFoo.
foo ifNotNil: [self doSomethingWith: foo]

The way to hide the plumbing here is to have ifNotNil: accept a one-argument block and feed the receiver into it, allowing us to fold the above back into a single expression

self computeFoo ifNotNil: [:foo | self doSomethingWith: foo]

This illustrates another asymmetry of ifNotNil: and ifNil:—while ifNil: block needs no arguments because nil is not an “interesting” object, it’s often helpful for ifNotNil: to take the non-nil receiver as argument.

A number of years ago Squeak had ifNil: and ifNotNil: implemented exactly this way. The former would take a niladic block as the argument and the latter (as far as I can remember) would only accept a monadic one. When a few years ago Eliot and I were adding the two to VisualWorks we kept almost the same pattern, extending the ifNotNil: case to also accept a niladic block.

In Squeak the two messages have since then been reimplemented as special messages, expanded by the compiler into the equivalent == nil ifTrue:/ifFalse: forms. Inexplicably, in the process ifNotNil: was changed to only accept a niladic block—precisely the case that is less valuable! Also interestingly, the fallback implementation in ProtoObject still allows a monadic block in a “pure” ifNotNil: but not as the ifNotNil argument of ifNil:ifNotNil: and ifNotNil:ifNil:! Their classification under the ‘testing’ protocol is a minor nit in comparison.

And now for the fix. It is available as a zip file MonadicIfNotNil.zip with two change sets. One modifies the handling of ifNotNil: and related messages to allow monadic blocks. The second contains the tests and should obviously be filed in after the compiler changes in the first set.

The change was fairly straightforward. Most of the work was dancing around the original error checking code that assumes only niladic blocks are legal as arguments of macroexpanded messages. The expansion simply promotes the block argument into a method temp, expanding

self foo ifNotNil: [:arg | ...]

into

(arg := self foo) ifNotNil: [...]

In VisualWorks such treatment would count as incorrect, but this promotion of a block argument into a method temp is in fact the classic Smalltalk-80 trick still surviving in Squeak in the similar treatment of the to:do: message.