Ian Woollard schrieb:
For example in C++ you can do things like:
{ FileHandle f = open("/adirectory/aFile", open);
f << "I went to the zoo";}
And the file may be automagically closed as f goes out of scope.
Yes, but you could also do this:
class ZooWriter { FileHandle mFH; public: void write_zoo() { mFH = open("/adir/afile", open); mFH << "I went to the zoo"; }
void after_writing_zoo_write_home() { mFH << ". Then I went home"; }
void essay() { write_zoo(); after_writing_zoo_write_home(); } }
And then you'll have a problem.
(Aside -- how would you implement the open function described in your example? Instead of creating an open function, I would have given FileHandle a two argument constructor and written FileHandle f("path", open), but there's always more I can learn about C++.)
But this is unreliable in Java/Self/Smalltalk- you don't know when the GC will get around to release it. I consider this to be a bug in these languages.
Yes, but if your runtime uses a two-space GC implementation and collects recently allocated objects more frequently, the file handle will be closed reasonably soon, and you get the added bonus that the example I provided won't cause any problems.
- sekhar
-- C. Ramakrishnan cramakrishnan@acm.org
Chandrasekhar Ramakrishnan wrote:
Yes, but you could also do this:
class ZooWriter { FileHandle mFH; public: void write_zoo() { mFH = open("/adir/afile", open); mFH << "I went to the zoo"; }
void after_writing_zoo_write_home() { mFH << ". Then I went home"; }
void essay() { write_zoo(); after_writing_zoo_write_home(); } }
And then you'll have a problem.
Well, because you've chosen to declare mFH in the object, it doesn't go out of scope in the above. However, if ZooWriter goes out of scope then it could (if you make appropriate changes to ZooWriter).
(Aside -- how would you implement the open function described in your example?
It's essentially optional reference counting; so returning it would increment it an extra time before it goes out of scope since that is a reference too.
Instead of creating an open function, I would have given FileHandle a two argument constructor and written FileHandle f("path", open), but there's always more I can learn about C++.)
It was dog C++ code; I was just trying to express the idea of closing a resource automagically; I think you got the idea.
My C++ is rusty, but IRC you can do it either way, but your way is cleaner.
But this is unreliable in Java/Self/Smalltalk- you don't know when the GC will get around to release it. I consider this to be a bug in these languages.
Yes, but if your runtime uses a two-space GC implementation and collects recently allocated objects more frequently, the file handle will be closed reasonably soon,
That's a big if; and GC rate depends on the rate of object creation in many cases. A generational technique is more popular these days I think, and can result in quite a long delay in the worst case.
and you get the added bonus that the example I provided won't cause any problems.
I'm not saying that finalisation functions are forbidden! I'm just saying that in quite a lot of cases this is a very useful pattern; and I've used it extensively in the past. No design patterns are universally applicable.
And although it is essentially the same technique as reference counting pointers, I don't see this as being a bottleneck on efficiency- it's unlikely that the reference/dereference calls would be as frequent.
- sekhar
-- C. Ramakrishnan cramakrishnan@acm.org
If I can paraphrase your position:
Reference counting is more deterministic when releasing resources that have semantics beyond bits in memory.
I would agree with that statement.
I think you are really arguing for the programer to have more control over the systemic decisions made by the VM developer (GC vs. Ref Counts, heap vs. stack allocation, etc).
I would agree with that in theory. In practice having too many knobs (look at J2EE) is also a problem.
This seems to again suggest some type of Aspect type system to apply flexible design decisions throughout a system is of interest, and a VM that allowed more control over the operation of the application.
Any design decision made by the VM limits the application's flexibility and ability to address design issues. As such the hard-coded design decisions should be kept as few as possible. This is where the Java VM has made some big mistakes.
Michael
On Jun 27, 2004, at 2:53 PM, Ian Woollard wrote:
Chandrasekhar Ramakrishnan wrote: Yes, but you could also do this:
class ZooWriter { FileHandle mFH; public: void write_zoo() { mFH = open("/adir/afile", open); mFH << "I went to the zoo"; }
void after_writing_zoo_write_home() { mFH << ". Then I went home"; } void essay() { write_zoo(); after_writing_zoo_write_home(); }}
And then you'll have a problem.
Well, because you've chosen to declare mFH in the object, it doesn't go out of scope in the above. However, if ZooWriter goes out of scope then it could (if you make appropriate changes to ZooWriter).
(Aside -- how would you implement the open function described in your example? It's essentially optional reference counting; so returning it would increment it an extra time before it goes out of scope since that is a reference too.
Instead of creating an open function, I would have given FileHandle a two argument constructor and written FileHandle f("path", open), but there's always more I can learn about C++.)
It was dog C++ code; I was just trying to express the idea of closing a resource automagically; I think you got the idea.
My C++ is rusty, but IRC you can do it either way, but your way is cleaner.
But this is unreliable in Java/Self/Smalltalk- you don't know when the GC will get around to release it. I consider this to be a bug in these languages.
Yes, but if your runtime uses a two-space GC implementation and collects recently allocated objects more frequently, the file handle will be closed reasonably soon, That's a big if; and GC rate depends on the rate of object creation in many cases. A generational technique is more popular these days I think, and can result in quite a long delay in the worst case.
and you get the added bonus that the example I provided won't cause any problems. I'm not saying that finalisation functions are forbidden! I'm just saying that in quite a lot of cases this is a very useful pattern; and I've used it extensively in the past. No design patterns are universally applicable.
And although it is essentially the same technique as reference counting pointers, I don't see this as being a bottleneck on efficiency- it's unlikely that the reference/dereference calls would be as frequent.
- sekhar
-- C. Ramakrishnan cramakrishnan@acm.org
Yahoo! Groups Links
• To visit your group on the web, go to: http://groups.yahoo.com/group/self-interest/ • To unsubscribe from this group, send an email to: self-interest-unsubscribe@yahoogroups.com • Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.
On Sun, Jun 27, 2004 at 08:20:43PM -0700, Michael Latta wrote:
If I can paraphrase your position:
Reference counting is more deterministic when releasing resources that have semantics beyond bits in memory.
I believe the team working on Parrot and Perl 6 has wrestled with these issues a bit. I'm not sure what the current status is, but I remember that at one point they were considering pure GC and no guarantees on object finalization. I believe that phase didn't last long - too many people made enough impassioned arguments in favor of deterministic resource release that a hybrid system has developed.
http://perlmonks.thepen.com/193313.html has a long discussion of GC vs. refcounting. The implementers keep saying, "Are you SURE you REALLY need deterministic finalization, 'cause we'd really like to be free of it. PLEASE!" And the users keep saying, "It can't be that painful, and yes, I'd much rather have it." The users never seem to understand how painful it is to implement and the performance tradeoffs, but the implementers never seem to understand the importance of having it be simple. The implementors keep coming up with approaches that increase the complexity for the user, but make things manageable for the implementors.
I wonder if this is a New Jersey vs. MIT school of thought thing. As a programmer, I deal with enough weirdness in the environment, without wanting to worry about weirdness in my language. In the end, I want a language that is _predictable_ to the greatest extent possible. I know when writing multi-threaded code that I have to worry about race conditions. But my instinct is to presume that in single-threaded code that I can forget about them, and that is no longer the case when deterministic finalization goes out the window.
Might take a look at http://www.sidhe.org/~dan/blog/archives/000199.html for what I think is something vaguely like a recent word on the subject.
--Toby Ovod-Everett
On Sunday 27 June 2004 22:54, Toby Ovod-Everett wrote:
On Sun, Jun 27, 2004 at 08:20:43PM -0700, Michael Latta wrote:
If I can paraphrase your position:
Reference counting is more deterministic when releasing resources that have semantics beyond bits in memory.
[Parrot notes]
Somewhat OT, but I believe that there is a blog entry or small whitepaper floating around on the Internet somewhere that shows that file operations are rarely as deterministic on Unix systems as people believe. Particularly close vs. file data being flushed out of per-process buffers etc.
So, while the language may make attempts to be deterministic, the OS may foil them. If we all (language designers) try to keep everything fully deterministic and the OS people pull the rug out from underneath us....
The point is that OSes and even processors are moving toward models that have less deterministic behavior, not more (see the IA-64 programming model for just how non-deterministic things are becoming). Languages need to plan for future use. If deterministic behavior is going to become harder and harder to get, then, like multithreading, it may be something that programmers need to keep in mind when programming.
For now most systems have some level of deterministic behavior, but it isn't something I think will last out the decade.
Best, Kyle
On Monday 28 June 2004 15:54, Toby Ovod-Everett wrote:
On Sun, Jun 27, 2004 at 08:20:43PM -0700, Michael Latta wrote: when writing multi-threaded code that I have to worry about race conditions. But my instinct is to presume that in single-threaded code that I can forget about them, and that is no longer the case when deterministic finalization goes out the window.
But surely not unless you're going to have a finalisation method (this assumes that the finalisation will be done on another thread - probably the gc thread). If you write single threaded code and have no need for finalisation then you still don't need to worry about race-conditions/non-determinism.
Can't most resources be managed via with-open-file type of calls (of scheme, lisp, dylan)?
Anyone know if this idiom made it into Self and/or Smalltalk?
Cheers, Steve.
On Jun 29, 2004, at 3:23 AM, Steven Shaw wrote:
On Monday 28 June 2004 15:54, Toby Ovod-Everett wrote:
On Sun, Jun 27, 2004 at 08:20:43PM -0700, Michael Latta wrote: when writing multi-threaded code that I have to worry about race conditions. But my instinct is to presume that in single-threaded code that I can forget about them, and that is no longer the case when deterministic finalization goes out the window.
But surely not unless you're going to have a finalisation method (this assumes that the finalisation will be done on another thread - probably the gc thread). If you write single threaded code and have no need for finalisation then you still don't need to worry about race-conditions/non-determinism.
Can't most resources be managed via with-open-file type of calls (of scheme, lisp, dylan)?
Anyone know if this idiom made it into Self and/or Smalltalk?
FileStream named:do: from my email is exactly such an idiom. Avi Bryant cooked it up for this discussion: http://discuss.fogcreek.com/newyork/default.asp?cmd=show&ixPost=1998
Self can do this in other ways, but this is the basic proof of concept.
-- Brian T. Rice LOGOS Research and Development http://tunes.org/~water/
On Wednesday 30 June 2004 04:24, Brian Rice wrote:
Anyone know if this idiom made it into Self and/or Smalltalk?
FileStream named:do: from my email is exactly such an idiom. Avi Bryant cooked it up for this discussion: http://discuss.fogcreek.com/newyork/default.asp?cmd=show&ixPost=1998
Self can do this in other ways, but this is the basic proof of concept.
Thanks, Brian. I am aware that it's possible to do in Self and Smalltalk (I certainly wasn't meaning it couldn't be done without macros - I am familiar with the Ruby way). I was meaning to ask if the idiom had become part of the standard library or a common library for Self or Smalltalk developers. I was also struggling to bring the thread back on topic :).
Cheers, Steve.
Slate has such an idiom, due to our general policy to collect and implement together all known good Smalltalk idioms (and some you might not have heard of because Smalltalk hasn't evolved to incorporate enough of the innovative library designs over the years).
Self doesn't even have a real stream library, so even more basic work has to be done!
On Jun 30, 2004, at 5:17 AM, Steven Shaw wrote:
On Wednesday 30 June 2004 04:24, Brian Rice wrote:
Anyone know if this idiom made it into Self and/or Smalltalk?
FileStream named:do: from my email is exactly such an idiom. Avi Bryant cooked it up for this discussion: http://discuss.fogcreek.com/newyork/default.asp?cmd=show&ixPost=1998
Self can do this in other ways, but this is the basic proof of concept.
Thanks, Brian. I am aware that it's possible to do in Self and Smalltalk (I certainly wasn't meaning it couldn't be done without macros - I am familiar with the Ruby way). I was meaning to ask if the idiom had become part of the standard library or a common library for Self or Smalltalk developers. I was also struggling to bring the thread back on topic :).
Cheers, Steve.
-- Brian T. Rice LOGOS Research and Development http://tunes.org/~water/
On Tue, Jun 29, 2004 at 08:23:00PM +1000, Steven Shaw wrote:
On Monday 28 June 2004 15:54, Toby Ovod-Everett wrote:
On Sun, Jun 27, 2004 at 08:20:43PM -0700, Michael Latta wrote: when writing multi-threaded code that I have to worry about race conditions. But my instinct is to presume that in single-threaded code that I can forget about them, and that is no longer the case when deterministic finalization goes out the window.
But surely not unless you're going to have a finalisation method (this assumes that the finalisation will be done on another thread - probably the gc thread). If you write single threaded code and have no need for finalisation then you still don't need to worry about race-conditions/non-determinism.
For instance, say I have a class that interacts with the Windows registry. In one block, I access a key and read through its values. The block closes and then I go to delete the key. If the finalizer for the object that interacts with the key doesn't run before I attempt to delete the key, I will have problems because the Windows registry system won't allow deletion of a key for which there are open handles. The requirement that I always explicitly close objects to ensure that finalizers run immediately begins to look an awful lot like the requirement that I hand back malloc'ed memory . . . :)
--Toby Ovod-Everett
The counter point is "don't design it that way". There is nothing in the language that requires you to wait for garbage collection to release resources. In particular if the registry requires you to close the accessor before doing something else, by all means close it explicitly. The use of blocks can make the open and/or close operations implicit in a custom control structure, or you can do it explicitly. The use of finalizers to release resources is a bad design practice in general.
Michael
On Jun 29, 2004, at 9:25 PM, Toby Ovod-Everett wrote:
On Tue, Jun 29, 2004 at 08:23:00PM +1000, Steven Shaw wrote:
On Monday 28 June 2004 15:54, Toby Ovod-Everett wrote:
On Sun, Jun 27, 2004 at 08:20:43PM -0700, Michael Latta wrote: when writing multi-threaded code that I have to worry about race conditions. But my instinct is to presume that in single-threaded code that I can forget about them, and that is no longer the case when deterministic finalization goes out the window.
But surely not unless you're going to have a finalisation method (this assumes that the finalisation will be done on another thread - probably the gc thread). If you write single threaded code and have no need for finalisation then you still don't need to worry about race-conditions/non-determinism.
For instance, say I have a class that interacts with the Windows registry. In one block, I access a key and read through its values. The block closes and then I go to delete the key. If the finalizer for the object that interacts with the key doesn't run before I attempt to delete the key, I will have problems because the Windows registry system won't allow deletion of a key for which there are open handles. The requirement that I always explicitly close objects to ensure that finalizers run immediately begins to look an awful lot like the requirement that I hand back malloc'ed memory . . . :)
On Wednesday 30 June 2004 14:25, Toby Ovod-Everett wrote:
For instance, say I have a class that interacts with the Windows registry. In one block, I access a key and read through its values. The block closes and then I go to delete the key. If the finalizer for the object that interacts with the key doesn't run before I attempt to delete the key, I will have problems because the Windows registry system won't allow deletion of a key for which there are open handles.
I agree with the other respondant - I don't think you need to design it this way. In this case (i.e your windows registry example), I'm sure you could use explicit resource management (hidden away appropriately) and have easy to understand, maintainable code. If it gets a little more complicated, it best interface may be "transactional".
The requirement that I always explicitly close objects to ensure that finalizers run immediately begins to look an awful lot like the requirement that I hand back malloc'ed memory . . . :)
This approaches a good argument against explicit resource management. We know that memory management works better when you don't have to manage it explicitly. However, in this case you generally have to be happy to have finalisation happen "sometime soon". Even with reference counting, you have cycles which are collected every now and then by a mark-sweep or something.
I wonder what makes most resources unlike memory? Maybe it's that most resources are held for short periods of time (although they say the same about memory...). Probably it's more something to do with the number of references to a resource. If anyone knows the answer please chime in.
Steven Shaw wrote:
I wonder what makes most resources unlike memory?
It's a very different emphasis.
There's a heck of a lot more bytes than copies of any other resource.
That means you don't need to be quite so paranoid about releasing memory- holding up deallocating an object for seconds, minutes or days may not matter at all. Failing to deallocate a semaphore for that long is likely to damage the system, perhaps fatally.
Maybe it's that most resources are held for short periods of time (although they say the same about memory...). Probably it's more something to do with the number of references to a resource. If anyone knows the answer please chime in.
Memory is often allocated/deallocated at extremely high rates. That means it *has* to be efficient. Other resources rarely call for the same degree of performance, but do need to be deallocated promptly.
On Thu, Jul 01, 2004 at 06:28:40PM +0100, Ian Woollard wrote:
Steven Shaw wrote:
I wonder what makes most resources unlike memory?
It's a very different emphasis.
There's a heck of a lot more bytes than copies of any other resource.
That means you don't need to be quite so paranoid about releasing memory- holding up deallocating an object for seconds, minutes or days may not matter at all. Failing to deallocate a semaphore for that long is likely to damage the system, perhaps fatally.
Maybe it's that most resources are held for short periods of time (although they say the same about memory...). Probably it's more something to do with the number of references to a resource. If anyone knows the answer please chime in.
Memory is often allocated/deallocated at extremely high rates. That means it *has* to be efficient. Other resources rarely call for the same degree of performance, but do need to be deallocated promptly.
Interesting. The problem is that as a programmer, I desire to use the same semantics that let me sweep memory management issues under the carpet (i.e. automatic memory management that lets me avoid issues of forgetting to release memory) to address similar issues I encounter with resources. In systems that have deterministic memory management (i.e. refcounting, barring issues with cycles), it is possible to hide the resource management issues by using the memory management system. In effect, we're repurposing the memory management stuff to handle the resource management stuff. The problem is that we're mixing metaphors, and while the two match up reasonably well in a refcounting system, garbage collection solves the memory management issues better than refcounting but doesn't solve the resource management issues.
Hmmm.
Points to consider:
* Under Unix and similar OSes (Win32, VMS I think, etc.) that have processes (not the only way to structure an OS, BTW), an exiting process cleans up all of the memory allocated to it as well as the resources allocated to it. This encourages people to think of them as similar things. It also complicates both memory management and resource management because leaks in either may not be evident in short running code and may later become an issue when the code runs for an extended period.
* Both memory and resources are shared resources across a system, and non-deterministic memory/resource managers may not play well with others. For instance, it would be good if the OS could say to processes, "Hey, can you release some memory - things are getting a bit tight." Similarly, it would be good if the OS could say similar things to processes regarding resources (open file handles, semaphors, etc.).
* Processes can share memory in modern OSes. Some sorts of resources they can share, some they can't.
* Many of the troubles we get into with "resources" like files, registry handles, etc. is because we're actually using them as a poor man's IPC system to send messages between systems, etc. Better, more secure, easy to use libraries with low overhead and so forth for dealing with these issues might help us because we'd be doing less misusing of metaphors.
* We really need to design completely new OSes from the ground up. Filesystems made sense 30 years ago. They're no longer an appropriate metaphor for dealing with and interacting with data, IMHO. Of course, Jecel groks all of this brilliantly, at least from the stuff I've seen about the design of Merlin.
* If we redesigned our OSes, perhaps we'd have fewer problems with our languages.
--Toby Ovod-Everett
On Sunday 27 June 2004 10:54 pm, Toby Ovod-Everett wrote:
I wonder if this is a New Jersey vs. MIT school of thought thing. As a programmer, I deal with enough weirdness in the environment, without wanting to worry about weirdness in my language.
Then why did you write (and use?) Class::Prototyped for Perl <g>?
An explanation for the rest of you on the list:
Toby is the author of a Self-like object model for Perl. This is a module that's available on CPAN and lets you combine regular old Perl 5 (either without objects, or using Perl's simple object model) with slot-based prototypes.
Since he may be the only user of this module, I should let him explain how it works in practice.
In the interests of full disclosure, I have to say that I helped writing Class::Prototyped, but I didn't inhale. And I quit working with Perl a long time ago.
On Tue, Jun 29, 2004 at 06:30:23AM -0700, Ned Konz wrote:
On Sunday 27 June 2004 10:54 pm, Toby Ovod-Everett wrote:
I wonder if this is a New Jersey vs. MIT school of thought thing. As a programmer, I deal with enough weirdness in the environment, without wanting to worry about weirdness in my language.
Then why did you write (and use?) Class::Prototyped for Perl <g>?
Because I like programming in Perl, but I needed access to the semantics of Self. :)
An explanation for the rest of you on the list:
Toby is the author of a Self-like object model for Perl. This is a module that's available on CPAN and lets you combine regular old Perl 5 (either without objects, or using Perl's simple object model) with slot-based prototypes.
Since he may be the only user of this module, I should let him explain how it works in practice.
Ned and I engaged in much deep Perl wizardry in order to get C::P to actually work. It is a bit frightening that it does. But in the interests of this discussion, I should state that a good portion of the test suite is devoted to ensuring that object finalization a) occurs at the proper time and place and b) happens in the proper order. This is a bit trickier that it may seem, because an object may inherit its destroy method from a parent object, and that parent object may exist only because the child object has a reference to it. It is important during the child object's destruction that the reference be held on to until the very end of the destruction process so that the child is not trying to call a destroy method that has disappeared . . .
In the interests of full disclosure, I have to say that I helped writing Class::Prototyped, but I didn't inhale. And I quit working with Perl a long time ago.
I think you inhaled deeply for at least one extended weekend :)
--Toby Ovod-Everett
Michael Latta wrote:
If I can paraphrase your position:
Reference counting is more deterministic when releasing resources that have semantics beyond bits in memory.
I would agree with that statement.
More generally, I would argue that Smalltalk/Java/Self rely on the programmer for deterministic resource release.
I think you are really arguing for the programer to have more control over the systemic decisions made by the VM developer (GC vs. Ref Counts, heap vs. stack allocation, etc).
I think you're very much overgeneralising. I was merely suggesting specific mechanisms.
I would agree with that in theory. In practice having too many knobs (look at J2EE) is also a problem.
I quite like Java, although I'm not entirely happy with it, in my experience, the C syntax impacts productivity and significantly increases software size. But compared with C++ it's a dream.
This seems to again suggest some type of Aspect type system to apply flexible design decisions throughout a system is of interest, and a VM that allowed more control over the operation of the application.
Perhaps.
Any design decision made by the VM limits the application's flexibility and ability to address design issues.
Not necessarily, although it may make the expression more or less difficult, and the performance higher or lower.
Michael
On Jun 28, 2004, at 12:55 PM, Ian Woollard wrote:
Michael Latta wrote:
If I can paraphrase your position:
Reference counting is more deterministic when releasing resources that have semantics beyond bits in memory.
I would agree with that statement.
More generally, I would argue that Smalltalk/Java/Self rely on the programmer for deterministic resource release.
I don't really care to get into this (read: reply to further posts in the same vein), but this seems truly ill-informed about Smalltalk or any language with extensible syntax and blocks (Ruby). I can define:
FileStream class>>named: filename do: aBlock | file | [file := FileStream open: filename. aBlock value: file] ensure: [file ifNotNil: [file close]]
and use it as such:
FileStream named: 'foo' do: [:file | self doSomethingWith: file. self doSomethingElseWith: file]
So now, we're not just talking about closing the file, but opening it as well, with the VM supporting ensure: semantics so that the finalizer is just a fallback. With ensure: or mechanisms like it, you can /alter/ your blocks into C++-style blocks without having to architect language support for it, or limit your block semantics to stack-allocation limits of semantics.
I am always suitably impressed by the number of people on this list who like Self but don't have a firm grasp of its much broader basis, Smalltalk-80. Self has quite poor libraries and as such is a demonstration and not a system that should be considered on its own unless you cite Smalltalk and really understand it.
-- Brian T. Rice LOGOS Research and Development http://tunes.org/~water/
self-interest@lists.selflanguage.org
-
Brian Rice -
Chandrasekhar Ramakrishnan -
Ian Woollard -
Kyle Hayes -
Michael Latta -
Ned Konz -
Steven Shaw -
Toby Ovod-Everett