Comments on Closures Aren't Easy

Some comments coming from the .NET world: Micros...

irmen · 2014-07-02T20:19:36.666000

Some comments coming from the .NET world:

Microsoft actually made a breaking change to their C# compiler to avoid this issue altogether. I think they concluded it was too easy a mistake to make when using lambda expressions within a loop.

See http://blogs.msdn.com/b/ericlippert/archive/2009/11/12/closing-over-the-loop-variable-considered-harmful.aspx


Resharper used to add a warning to your C# code if you did this in visual studio. I've just tried the following in PyCharm (also from JetBrains, the same guys that make resharper):

x = []
for i in range(100):
x.append(lambda: i*2)

and too bad, the same warning is not issued by PyCharm

The traditional remedy to this situation is to tak...

Brandon Rhodes · 2014-07-02T20:39:55.128000

The traditional remedy to this situation is to take advantage of the fact that keyword argument defaults are evaluated at definition-time rather than at call-time.

In fact, providing a remedy for situations like this might be the whole reason for that convention in the first place, since it otherwise seems to always be a surprise to all programmers everywhere that keyword argument expressions are evaluated during the definition.

In any case, try this in your first example:

def woofer(*args, func=func, **kw):
print("Woof")
return func(*args, **kw)

You will find that during each definition, Python grabs the particular object that "func" references and makes it the default for the keyword "func" of that particular version of the function.

Thanks, @Brandon. In point of fact I believe this ...

Steve · 2014-07-02T23:01:49.404000

Thanks, @Brandon. In point of fact I believe this used to be the recommended way to achieve "closure-like" behavior before the language formalized the relationship between lexically-nested scopes.

It's certainly a much more elegant way to transfer a value through to the inner namespace, but there is the possible danger that the keyword argument name will conflict with a legitimate keyword argument of the wrapped function.

I think this one is related: http://bugs.python.or...

Anonymous · 2014-07-02T23:26:24.194000

I think this one is related: http://bugs.python.org/issue7423

I stumbled upon the fact that this works as expected:

zip(*([(i, j) for j in 'ab'] for i in range(2)))

while this doesn't:

zip(*(((i, j) for j in 'ab') for i in range(2)))

@anonymous yes, the two do seem related. Thanks fo...

Steve · 2014-07-02T23:38:40.233000

@anonymous yes, the two do seem related. Thanks for your comment.

This is how it works in javascript closures as wel...

Anonymous · 2014-07-03T05:29:21.809000

This is how it works in javascript closures as well. When using in closures, the value is tied to the variable and not the copy of the variable (Javascript good parts - Douglas Crockford)

Basically it needs a new namespace and the hack you showed is what it does ("wrapper" which is invoked with the value to create a new namespace for that function).

Great article! didn't realize it stores it in __closure__ (though should have guessed by now. For all sorts of magic, check dunder methods first)

Sai
@agileseeker

The problem here is the semantics of the "for...

Cathal · 2014-07-03T10:30:10.619000

The problem here is the semantics of the "for" loop, rather than specifically closures. You can get obscure bugs from for loops anywhere that you don't duplicate the value of the internal reference. I imagine that'd be hard to clean up without seriously affecting the normal function of the for-loop, though, as it relates to the ability of for-loops to efficiently enclose scope and avoid leaking internal scoped variables.

Anyways, you can avoid this by never defining something in a for-loop scope that'll be strongly referential and re-used. I re-wrote the "barking" decorator as follows, and it works as intended:

def barking(cls):
def woofit(func):
def woofer(*a, **kw):
print("Woof")
return func(*a, **kw)
return woofer
for name in cls.__dict__:
if name.startswith("__"):
continue
func = getattr(cls, name)
setattr(cls, name, woofit(func))
return cls

(define (multiple-closures n) ; Scheme system ...

Anonymous · 2014-07-04T09:06:29.004000

(define (multiple-closures n) ; Scheme system : Racket v6.x
(define functions '())
(for ([i (in-range n)])
(define (inner x)
(+ x i))
(set! functions (cons inner functions)) ; reverse order, don't bother
(printf "~a returned ~a\n" inner (inner 10)))
functions)

(define functions (multiple-closures 3))
(for [(f functions)]
(printf "~a " (f 10))) ; 12 11 10 NOT THE SAME AS PYTHON.

Let me say I prefer Scheme behavior... Maybe due to the semantics of the for loop, in Scheme fresh locations are chosen at each iteration :

http://docs.racket-lang.org/reference/for.html?q=for#%28form._%28%28lib._racket%2Fprivate%2Fbase..rkt%29._for%29%29