Gavin Grover's GROOVY Wikiblog

See later entries

5 July 2012

Groovy's Broken Closures

Groovy creator James Strachan wrote (see below): no other dynamic language I'm aware of has any concept of Groovy's dynamic name resolution; names are always statically bound to objects in a sensible way in all dynamic languages I'm aware of (lisp, smalltalk, python, ruby etc). Groovy threw away decades of language research and development with respect to name resolution across the language as a whole.

So how did the Codehaus implementation of the Groovy Language get its broken closures?

2nd Groovy conference

Groovy hero Jeremy Rayner, the primary author of Groovy's current parser, provides clues in his writeup of the 2nd Groovy Developers Conference on 28 Nov 2005. Present were Jeremy, James, Jochen Theodorou, John Wilson (a.k.a. Tug), and Guillaume Laforge. Wrote Jeremy:

The discussion soon turns into a heated debate over the treatment of vanilla names.

class MyScript{
  String toString(){
    return "foo"
  }
  void run(){
    myList.each{
      println(toString())
      //does this call MyScript.toString() or myList.toString() ?
    }
  }
}
Tug argues the enclosing object instance should listen for and consume names prior to the lexical scope. (i.e. above example would call myList.toString()). James argues that the default should be lexically bound to the current context (i.e. above example would call myScript.toString()), and if the object instance was needed to be referred to that some form of syntax could be constructed to allow this.

Day Two starts out more productive, ... ... ... However the discussion soon gets round to the topic of Builders (a.k.a. Markup) and again the name resolution and scoping bear comes up to bite us again.

We have Tug and Guillaume firmly on the side of Builders being in charge of everything defined within the markup delimiters. James argues that markup is fundamentally broken and some kind of syntactic clue that we are suddenly entering a different 'markup world' where normal resolution and scoping doesn't apply. For some reason, that I cannot fathom, Tug and Guillaume both seem vehemently opposed to any indication in the code that we are in a Builder section. This includes both the suggestion of a 'with' keyword or .{ notation to indicate to the reader of the code that something different is about to occur.

String toString() {
  return "foo"
}
  
swingBuilder.frame().{
  panel() {
    button(toString())
  }
}

//or the even clearer...
with( swingBuilder.frame() ){
  panel(){
    button( toString() )
  }
}
We were all fairly happy that the Builder could be responsible for name resolution inside it's own block (e.g. the swingBuilder instance would be responsible for how it treats toString() inside), but what seemed to really be a sticking point was the important fact that due to the dynamic nature of the name resolution/scoping within this block, some visual clue to the reader of the program was necessary.

James argued for, and I agree that, the use of some kind of syntactic indication was vital to ensure that the Groovy language was sane, predictable and maintainable. Thus implying to any compiler, IDE, or human reader that the block of code was not a true lexical closure, but some other construct which was dynamic in it's very nature, and thus warnings, code completion and everything else specification related did not apply, as the responsibility would lie with the implementation of the builder.

I'm sad to report that no agreement was reached on this matter. Not so much which syntax would be useful, but actually whether we should have any syntax denoting the difference between a true lexical Closure and one of these Builder blocks. The historical reasons go back to Builder blocks looking just like Closures, and I'm afraid this long standing mistake must be removed from the language before any true progress can be made, as no sensible specification rules can be applied while the dichotomy exists.

I headed back to London with a very disappointed James Strachan, the language which has some of the finest minds, and most exciting ideas now faces a pivotal point in its life, and I'll do everything I can to ensure that clarity over dynamism wins the day.

Strachan's final ever email

Two days later on the Groovy mailing list, James wrote in response to Laforge's statement that the upcoming MOP can solve the name resolution problems in closures:

> Especially since the MOP and the introspection API can solve all these problems without introducing another syntax change which would definitely break more code in production than bring more advantages.

The MOP and introspection APIs do NOT solve the horribly broken name resolution rules in the current RI of Groovy, which is why I created the name resolution proposal to fix name resolution rules for closures and builders - introducing a syntactic difference so we can have proper name resolution mechanism for each concept in the language. Though it seems you don't think we have any name resolution issues which is making it a bit hard to discuss the problem in the first place

Notice James was still refering to Codehaus Groovy as an "RI", a reference implementation for a Groovy Language standard, of which many implementations would be built, just like with Ruby and Python.

Later that day, James Strachan made his *last ever posting to the Groovy mailing list*:

for closures its only the outer class which should be used to resolve names - that's what closures are - their vanilla names are bound, always, to the outer lexical scope. Any other concept like dynamic vanilla name resolution is not a closure. Now I'm not saying we can't support other ideas and concepts; its just they are not closures.

And: no other dynamic language I'm aware of has any concept of dynamic name resolution in the way you suggest; names are always statically bound to objects in a sensible way in all dynamic languages I'm aware of (lisp, smalltalk, python, ruby etc). I see no argument yet for why we have to throw away decades of language research and development with respect to name resolution across the language as a whole - though I'm more than happy to innovate within a subset of the language (e.g. inside Markup sections). The only sensible solution I see is to ensure that the Markup name resolution mechanism is restricted to Markup sections.

And his very last sentence ever at the end: It just feels totally wrong to break Closures across the entire language just because of some use cases for Markup.

When Ruby copied the markup concept a year later, they did it correctly, so their closures are still real closures. Groovy's "closures" eventually got a name resolution field, with possible values like DELEGATE_THEN_OWNER, OWNER_THEN_DELEGATE, DELEGATE_ONLY, and OWNER_ONLY.

And despite there being many ex-developers and ex-despots in the Codehaus developers roll, James is unique in having been stripped totally of his commit privileges.

Why'd he do it?

What was the reason Laforge didn't want to fix Groovy's broken closures? He'd written that fixing the syntax would "definitely break more code in production than bring more advantages". Laforge was showing concern for existing production users, even though there weren't really any at that time because Grails 0.1 wasn't released until 3 months later. We can see more clues in his later postings...

A few months ago, Groovy hero Russel Winder, co-developer of Groovy's gpars, asked on the mailing list:

Looking at http://groovy.codehaus.org/Closures, the section on "Free variables" specifically states that Closures employ lexical binding, i.e. are closures. The examples work because the examples are such that they work. However Groovy doesn't actually create lexically scoped closures. The question is whether to make Groovy 2.0.0 do what the documentation (and the rest of the planet) expect of [Cc]losures or whether to correct the documentation for what Groovy actually does?

Laforge replied: Correct the documentation! Changing even just the naming of what people are used to use for 8+ years, that's more painful for everybody. Only perfectionists care of the few differences with "real closures" (TM). In the end, Groovy is for pragmatic people more than for language theorists.

Russel replies:

> Changing even just the naming of what people are used to use for 8+ years, that's more painful for everybody.

So fix the problem. Groovy 2.0 is the place to do it. There is an ideal opportunity. The MOP is changing in a breaking way between 1.8.x and 2.0.x so fix the problem. In any case I bet 99% of scripts won't notice but those 1% will make all the difference between Groovy still being around and having traction once the Java / Scala / Kotlin / Ceylon war really gets started.

> Only perfectionists care of the few differences with "real closures" (TM).

I do not believe that to be the case.

> In the end, Groovy is for pragmatic people more than for language theorists.

This I think is a very inappropriate stance. It translates to "we don't care about the rest of the world or CS theory we want to be a blue collar, insular ghetto of hackers". Still, it is your show.

Russel and Jochen continued that thread discussing the technical issues of how to fix the closures.

In his email discussions, both 6 years ago and nowadays, Laforge shows an intolerance to any changes to Groovy, just wanting to keep the status quo, even when moving Groovy from version 1.x to 2.0. I suspect he's Rocher's lackey, treating Groovy as just one part of Grails, and not changing anything in Groovy unless it's needed for Grails in some way.

But there weren't any Grails users in Dec 2005, the time of the 2nd Groovy developers meeting. Why else would Laforge be "vehemently opposed" to fixing closures in Groovy? This was about a year after I myself first discovered the Groovy Language. I believe he was trying to get rid of James Strachan, stirring up any arguments to wear him down. James wanted closures fixed, so Laforge wanted them to stay broken: it's that simple.

I've seen this behaviour many times in IT workplaces. It most often comes from "programmers" without aptitude, such as Laforge. He wanted to take over a growing open source project from someone else, so snuck in, stuck around, then struck. Et tu Laforge. The cost is broken closures in Codehaus Groovy, having "thrown away decades of programming language research and development", and "not caring about the rest of the programming world or Computer Science theory". Guillaume Laforge has ruined the Groovy Programming Language.


12 July 2012

Update to Groovy's Broken Closures

In this 4 yr old blog and comments, Guillaume Laforge and Cedric Champeau collude to influence Java's closure syntax. They want -> instead of => giving 2 reasons:
  • the => separator can be confusing e.g. {int x, int y => x >= y}
  • Groovy uses -> and more and more people are using Groovy

Java guru Neal Gafter replied: There is Groovy and Ruby (1.9+) and ML and Haskell on the one hand using ->; but C# and Scala on the other using =>. I don't think the number of users familiar with one notation or the other is a key metric for a decision.

Earlier this year, Oracle/Sun finally announced the decision to use C# and Scala's syntax, but they use the -> symbol in all their example code, e.g. (int x, int y) -> x >= y. Why was this? Did Laforge pester and pester and pester and pester and pester and pester and pester them, even dragging in "allies" like Champeau?

Groovy will probably need to support both its broken closures and Java 8's lambdas in its syntax. If Java 8 had copied C# and Scala's syntax and lexical symbol =>, Groovy's lambdas would be more visually distinct from its broken closures. Not only did Laforge cause the problem of Groovy's broken closures, but he could also be ruining the most viable solution to the problem.









Posts removed because they've been superceded.

Post removed because it's been superceded.






Post removed because it's been superceded.

Last edited Mar 29 at 6:05 AM by gavingrover, version 32

Comments

No comments yet.