[Ann] A simple streaming framework

[Russell Allen]

[This is a copy of the blogpost: https://blog.selflanguage.org ]

have long maintained that Self is the finest Yak shaving system in existence. Everything is possible and nothing is finished :)

So I was playing with building a website front end for the mailing list archives, which led me to improving the Self web server, which led me to making a new experimental stream framework for Self. This also led me to some interesting ideas on automatically checking for methods which should be present in an object but aren’t, but I’ll write about that later!

Self already has a readStream and writeStream prototypes, which are very basic entities modelled largely on Smalltalk-80. They’re mainly used in the context reading and writing to files and sockets.

What I’ve put together is a quick stream framework for putting together pipelines of filters etc and streaming objects through them. The general aim is to avoid the technical and conceptual overhead of copying collections multiple times as you handle them.

This isn’t particularly groundbreaking, lots of other languages do this sort of thing. It’s a bit like Smalltalk’s Xtreams but much more minimal. I’d like to hear suggestions on how to make what I’ve done more elegant and Self-ish.

The code is at https://github.com/russellallen/flow

Basically, streams are entities which understand either readIfFail: or write:IfFail:. Streams can only be read or write.

Flow streams are composable, which means you can link them in a chain with pipeable connectors.

The relevant parts are in the global object ‘flow’.

This isn’t optimised at all – ideally streams should cheat by copying larger chunks behind the scenes where appropriate for performance. Also file streams probably need to be more aware of reading and writing chunks with internal buffers, and string writing should be optimised to reduce copying!

Possible improvements include adding spitting and merging pipes, specific buffering pipes etc.

Using flow streams is a two or three step process:

1 Build structure.

| myPipeline |

myPipeline: 
      'This is a bit of text for us to play with' reading 
   |= (flow pipeable map copyOn: [|:c| c capitalize])
   |= (flow pipeable filter copyOn: [|:c| c isVowel])
   |= (flow pipeable gather copyOn: [|:c| (c & c) asList])
   |= flow writable string copy. 

2 Put pipeline into action

myPipeline flush.

3 (Optionally) convert your stream into something else

[myPipeline contents = 'IIIIAAIIOOEEOOUUOOAAII'] assert.

And that’s about it. I’ll play with this some more, any suggestions?

Russell