Yes, I seem to be drifting slowly off the main topic: Scheme-OSA = Scheme + OSA = Scheme + Open Scripting Architecture = Scheme + Applescript. Well, this is still the main goal for my current activities and this blog supposes to reflect it. However, I do get distracted from time to time for variety of reasons. So this post will be all about excuses.
Squeak and Seaside
A week ago I was asked to investigate this: given a XYZ program, which prints its data to a printer, redirect the printer input to a Squeak-based scheduling program UVW, parse and modify the data and then spit it out to the printer - using a similar format to the one used by the original XYZ program. Well, the UVW is not a Command Line Unix program -- it is a GUI universe on its own. You cannot just redirect stdin to UVW, then manipulate the data and finally print the results out to stdout and force UVW to exit. Well, you can do it in principle but there is no easy way to implemented it.
The obvious alternative solution is to embed a HTTP server within UVW - which brought me to Seaside framework -- a nice, several years old technology, combining a Comanche web server with a 'servlet' support, CSS and Ajax.
In my view Seaside has three major selling points: it understands and supports continuations, it completely separates flow control from user interface and it provides remote support for browsing and debugging. The latter means that you can connect to a live Seaside server, inspect and modify data using Squeak-like inspectors embedded in a web page, and browse or modify offending methods using tools, such as Class Hierachy Browser, which look almost identical as standard Squeak tools. When doing those modifications you do not need to kill any session or the server itself - all the operations can be made on the "living organism".
The Seaside's home page offers several demo videos. One such demo was evidently given to a Lisp user group; the Lispers had many technical questions and suggestions, but apparently the Seaside concept was taken seriously by them.
All of this has nothing to do with Scheme in general (with the exception of continuation jargon), and Scheme-OSA in particular. This was just a distraction, but since I was paid for that, I do not complain. Oh, yes, I used to program in Smalltalk, and I have some experience with Squeak.
Presentation of the world's first commercial quantum computer
Few days ago D-wavesys company from British Columbia, Canada announced its first public presentation of a 16-qubit quantum computer, Orion. Some details leading to the presentation are available at Geordie Rose's blog -- a technical person from D-wavesys. To the credit of Dr. Rose, his blog contains permanent links to the blogs of some active quantum physicists, and who are not always overly enthusiastic about the event. But in average the response of the QC community is ambivalent but not damning: "give us more data to analyze the system first". If you want to learn something about Orion read blogs of physicists, watch some U-tube videos produced by participants of the events (Vancouver and San Francisco), but avoid regular press as a plague: too much hype and very little understanding of QC concept.
What does it have to do with Scheme-OSA? Nothing, but it affects the author of this blog somehow. In 2001 I wrote Haskell Simulator of Quantum Computer, which is cited in several Haskell publication produced ever since. There are also several Scheme-based papers on the subject published in recent years. For some reasons I have not followed the topic of Quantum Computing, but I wish I had. Much has happened in the recent years in this field and it took me several days to familiarize myself with major accomplishements of QC.
Thursday, February 15, 2007
Thursday, February 1, 2007
Visualizing music
No, no, this is not about visualizing music in your iTunes by animating some pretty images. It is about visualization of structure of music.
Staking the ground
In this post I am going to prepare the background for another potential application of Scheme-OSA: transcription and visualization of different music formats. I should start with the link to MusicXML tutorial and advice you to look at their first lesson: "Hello World" in MusicXML. Surely, XML is the best way for exchanging information between incompatible systems. But XML is not designed for human consumption, it is designed for machine consumption.
Somewhere in between is a place for Scheme-based application, which would be able to handle both: human and machine interface. From the high level perspective, Scheme-OSA should be able to learn from two good applications: compo and Haskore. The first one is written in Lisp, the second in Haskell. The advantage of Scheme-OSA is in its ability in interfacing to variety of existing music notation programs available on Mac OS X platform.
I am particularly interested in various non-standard ways of music notation.
Folk music
The music score and the audio below have been produced by BarFly from a fragment of the wooodenflute.abc file.
The file woodenflute.abc comprises the ABC tunes posted to the woodenflute mailing list betweenOctober 1998 and April 2000. The list exists to discuss all aspects of the wooden flute - music, history, technique, players, making, workshops, festivals, new CDs, and gossip.
The ABC music language was originally invented by Chris Walshaw to transcribe folk music in plain ASCII. It has now developed to such an extent that it can be used for many other musical purposes.
BarFly is an integrated environment for Mac OS X, developed by Phil Taylor, for working with abc music files. It can play, check, transpose and display tunes in abc notation.
The original ABC code, from which the above was created, is shown below.
X:157
T:Sound Symposium Jig
R:Jig
C:Emile Benoit
M:6/8
L:1/8
Z:Gerry Strong
K:G
|:eaa eaa| eaa baf | gag fed | gag fed| eaa eaa| eaa baf | gag edB| 1A3 A3:| 2A3|
|:efe ~A3| efe A2 e| dBA GAB | dBA GBd| efe ~A3| efe A2 e| dBA GBg| edB A3:|
Polyphonic music
ABC notation can go way beyond simple folk music. Here is Johann Bach at the height of his powers in the stupendous Fugue in C#m from the first book of the Well-Tempered Clavier, converted from ABC to audio format. As Phil Taylor, designer of BarFly, writes:
So I did set the instrument to harpsichord - as the man adviced.
Music Animation Machine
Here are few first measures from Bach's Fugue No. 5 in D Major [Well Tempered Clavier, book II] shown in classical notation and visualized according to Stephen Malinowski's Music Animation Machine. If you have Apple's GarageBand installed on your Mac OS X Stephen's visualization should look familiar: notes are distributed vertically according to their pitch classes, note lengths are proportional to lengths of the pitch bars. But in addition to these two concepts, Music Animation Machine uses colors: each pitch class has its own color.
According to Stephen:
I applied the color wheel to the "circle of fifth", so that pitches which were closely related by perfect fifths would be close in color[...] This turned out to be useful for showing things about tonality. For example, all the notes in a major scale are grouped together. This means that if you're playing a piece that stays in one key, it stays in one area of the circle. If the key changes, the notes are in a new area. Also, notes which are not in the key stand out.
By the way, this method of coloring pitches is used in the Sibelius music notation software.
Here is Music Animation Machine in action; it visualizes Debussy's Clair de lune:
Critique of Classical Western Music Notation
Why is the standard music notation so complex - with all those clefs, sharps, flats, flags, scales, etc.? Serious articles would give you serious answers; I just say this: it is due to international conspiracy of great and lesser musicians of all times who had to support themselves by teaching dumb pupils. It was not in their interest to make their notation any simpler. :-)
But whatever the reason — serious or not — the CWMN does not have to be as complex as it is. When I was learning the basics of guitar I was lucky enough to come across one of those alternative notation methods, which — I know it now, after some googling — related somehow to some obscure Dutch school of music, known as Klavar, or Klavarscribo. The only difference was in direction of staves: in Klavar they are vertical, while in my source they were horizontal, more appropriate for a guitar student.
From there I learned that if I focus on relative pitches, rather than on absolute pitches, everything would become simple and mathematically pleasing.
If you took a look at the above link you should recognize some similarities between Klavar and the Music Animation Machine. Old ideas (1931) of Cornelis Pot die hard. :-)
From a programmer's perspective, scores of sophisticated music look like old-fashioned BASIC spaghetti code — with all those GOTO's statements and multi-level IF-ELSEs' thrown in. But even BASIC has become modernized, and there are many better languages than BASIC that people use nowadays.
Alternative Music Notations
It appears that I am not alone in my assessment. The Music Notation Modernization Association MNMA is in a process of elimination and selection of alternative proposals to Classical Western Music Notation. So far over 500 alternative music notation schemes have been considered, and a subset of those have been selected for further consideration. Many are somehow related to the good old Klavarscribo.
Will it ever be a consensus for a new notation? None of these proposals to MNMA has provided a sufficiently-compelling benefit to become broadly popular (except for guitar tablature). What would it take for a new notation method to become acceptable? One strong argument against any new music notation is the existing vast archive of CWMN. However, with all those score reading optical recognition programs this should not be a problem nowadays. Another argument against a new notation is lack of sheet music for the new style. Afficionados of Klavarscribo claim that there are 25,000 pieces of music transcribed into that format. Yet, Klavar did not make any dent in general attitude towards music notation.
When I was recently watching a video presentation of iPhone by Steve Jobs, one of his quotes of Allan Key (of the Smalltalk fame) struck me as applicable to this situation. He was saying something in effect that people will readily accept your new idea only if you offer them a totally unique solution, not just an alternative. In the iPhone context that meant a combination of unique sophisticated software and hardware.
And such a total solution for alternative music notation might be just around the corner, in the form of musical instrument Thummer from an obscure programmer-turned-businessman from Australia. After reading his Thummer documents the man did not struck me as one of those coocoos who want to replace List/Scheme parentheses by something else; he evidently has something to offer. Not that I like the look of his product - it would require Apple geniuses to style it to look really ergonomic and "pretty".
Thummer concepts are quite similar to those of iPhone in a way: simplify basic notation, layout the buttons to match the software, eliminate inconsistencies, add touch/gesture controls.
But predictions are just best guesses; nobody knows what would happen with the Thummer. People come with very good ideas all the time. For example, there is Paul Hirsch's interesting site dealing with instrument 'modernization' and music theory :Intuitive Instruments for Improvisors. Paul provides few interesting on screen instruments, which you can try on his web site. He also delivers several hardware gadgets that handle music in intuitive way.
You may also want to look at Blumberg's Music Theory Cipher — a chromatic approach to interval notation on guitar and other stringed instruments.
Not just simplicity
A need for a new music notation is not dictated by simplicity alone. Any introduction to the subject will point out that CWMN is not able to handle other scales, as invented by cultures other than Western European. Many modern composers, such as Arnold Schoenberg or John Cage, also felt a need for new kind of musical notation. Schoenberg is often quoted from his "A New Twelve-Tone Notation"[1924]:
The need for a new notation, or a radical improvement of the old, is greater than it seems, and the number of ingenious minds that have tackled the problem is greater than one might think.
Others seek new form of expressions in microtonal music. Here is an excerpt from Microtonal Projects Ltd:
Microtonal Music uses pitches other than the 12 equally spaced semitones normally heard in western music or their close relations (such as the ones found in the tmperaments used in Baroque music). Such music is not particularly rare. The ancient Greeks used structured intervals as small as a quarter-tone, so do many folk music from around the world and Indian classical music.
Staking the ground
In this post I am going to prepare the background for another potential application of Scheme-OSA: transcription and visualization of different music formats. I should start with the link to MusicXML tutorial and advice you to look at their first lesson: "Hello World" in MusicXML. Surely, XML is the best way for exchanging information between incompatible systems. But XML is not designed for human consumption, it is designed for machine consumption.
Somewhere in between is a place for Scheme-based application, which would be able to handle both: human and machine interface. From the high level perspective, Scheme-OSA should be able to learn from two good applications: compo and Haskore. The first one is written in Lisp, the second in Haskell. The advantage of Scheme-OSA is in its ability in interfacing to variety of existing music notation programs available on Mac OS X platform.
I am particularly interested in various non-standard ways of music notation.
Folk music
The music score and the audio below have been produced by BarFly from a fragment of the wooodenflute.abc file.
The file woodenflute.abc comprises the ABC tunes posted to the woodenflute mailing list betweenOctober 1998 and April 2000. The list exists to discuss all aspects of the wooden flute - music, history, technique, players, making, workshops, festivals, new CDs, and gossip.
The ABC music language was originally invented by Chris Walshaw to transcribe folk music in plain ASCII. It has now developed to such an extent that it can be used for many other musical purposes.
BarFly is an integrated environment for Mac OS X, developed by Phil Taylor, for working with abc music files. It can play, check, transpose and display tunes in abc notation.
The original ABC code, from which the above was created, is shown below.
X:157
T:Sound Symposium Jig
R:Jig
C:Emile Benoit
M:6/8
L:1/8
Z:Gerry Strong
K:G
|:eaa eaa| eaa baf | gag fed | gag fed| eaa eaa| eaa baf | gag edB| 1A3 A3:| 2A3|
|:efe ~A3| efe A2 e| dBA GAB | dBA GBd| efe ~A3| efe A2 e| dBA GBg| edB A3:|
Polyphonic music
ABC notation can go way beyond simple folk music. Here is Johann Bach at the height of his powers in the stupendous Fugue in C#m from the first book of the Well-Tempered Clavier, converted from ABC to audio format. As Phil Taylor, designer of BarFly, writes:
Five voices, in three subjects, coming and going all through with never a note out of place. My son, who is a keyboard player, calls this a "Real Man's Fugue", and would give his eye teeth to be able to play it. He says you would need hands the size of a bunch of bananas, for a start. I haven't set an instrument for this, and merging the voices onto two staves is still far beyond the capabilities of the program. Actually, it's much easier to read on five staves anyway. For greatest authenticity use a harpsichord, but piano sounds good too.
So I did set the instrument to harpsichord - as the man adviced.
Music Animation Machine
Here are few first measures from Bach's Fugue No. 5 in D Major [Well Tempered Clavier, book II] shown in classical notation and visualized according to Stephen Malinowski's Music Animation Machine. If you have Apple's GarageBand installed on your Mac OS X Stephen's visualization should look familiar: notes are distributed vertically according to their pitch classes, note lengths are proportional to lengths of the pitch bars. But in addition to these two concepts, Music Animation Machine uses colors: each pitch class has its own color.
According to Stephen:
I applied the color wheel to the "circle of fifth", so that pitches which were closely related by perfect fifths would be close in color[...] This turned out to be useful for showing things about tonality. For example, all the notes in a major scale are grouped together. This means that if you're playing a piece that stays in one key, it stays in one area of the circle. If the key changes, the notes are in a new area. Also, notes which are not in the key stand out.
By the way, this method of coloring pitches is used in the Sibelius music notation software.
Here is Music Animation Machine in action; it visualizes Debussy's Clair de lune:
Critique of Classical Western Music Notation
Why is the standard music notation so complex - with all those clefs, sharps, flats, flags, scales, etc.? Serious articles would give you serious answers; I just say this: it is due to international conspiracy of great and lesser musicians of all times who had to support themselves by teaching dumb pupils. It was not in their interest to make their notation any simpler. :-)
But whatever the reason — serious or not — the CWMN does not have to be as complex as it is. When I was learning the basics of guitar I was lucky enough to come across one of those alternative notation methods, which — I know it now, after some googling — related somehow to some obscure Dutch school of music, known as Klavar, or Klavarscribo. The only difference was in direction of staves: in Klavar they are vertical, while in my source they were horizontal, more appropriate for a guitar student.
From there I learned that if I focus on relative pitches, rather than on absolute pitches, everything would become simple and mathematically pleasing.
If you took a look at the above link you should recognize some similarities between Klavar and the Music Animation Machine. Old ideas (1931) of Cornelis Pot die hard. :-)
From a programmer's perspective, scores of sophisticated music look like old-fashioned BASIC spaghetti code — with all those GOTO's statements and multi-level IF-ELSEs' thrown in. But even BASIC has become modernized, and there are many better languages than BASIC that people use nowadays.
Alternative Music Notations
It appears that I am not alone in my assessment. The Music Notation Modernization Association MNMA is in a process of elimination and selection of alternative proposals to Classical Western Music Notation. So far over 500 alternative music notation schemes have been considered, and a subset of those have been selected for further consideration. Many are somehow related to the good old Klavarscribo.
Will it ever be a consensus for a new notation? None of these proposals to MNMA has provided a sufficiently-compelling benefit to become broadly popular (except for guitar tablature). What would it take for a new notation method to become acceptable? One strong argument against any new music notation is the existing vast archive of CWMN. However, with all those score reading optical recognition programs this should not be a problem nowadays. Another argument against a new notation is lack of sheet music for the new style. Afficionados of Klavarscribo claim that there are 25,000 pieces of music transcribed into that format. Yet, Klavar did not make any dent in general attitude towards music notation.
When I was recently watching a video presentation of iPhone by Steve Jobs, one of his quotes of Allan Key (of the Smalltalk fame) struck me as applicable to this situation. He was saying something in effect that people will readily accept your new idea only if you offer them a totally unique solution, not just an alternative. In the iPhone context that meant a combination of unique sophisticated software and hardware.
And such a total solution for alternative music notation might be just around the corner, in the form of musical instrument Thummer from an obscure programmer-turned-businessman from Australia. After reading his Thummer documents the man did not struck me as one of those coocoos who want to replace List/Scheme parentheses by something else; he evidently has something to offer. Not that I like the look of his product - it would require Apple geniuses to style it to look really ergonomic and "pretty".
Thummer concepts are quite similar to those of iPhone in a way: simplify basic notation, layout the buttons to match the software, eliminate inconsistencies, add touch/gesture controls.
But predictions are just best guesses; nobody knows what would happen with the Thummer. People come with very good ideas all the time. For example, there is Paul Hirsch's interesting site dealing with instrument 'modernization' and music theory :Intuitive Instruments for Improvisors. Paul provides few interesting on screen instruments, which you can try on his web site. He also delivers several hardware gadgets that handle music in intuitive way.
You may also want to look at Blumberg's Music Theory Cipher — a chromatic approach to interval notation on guitar and other stringed instruments.
Not just simplicity
A need for a new music notation is not dictated by simplicity alone. Any introduction to the subject will point out that CWMN is not able to handle other scales, as invented by cultures other than Western European. Many modern composers, such as Arnold Schoenberg or John Cage, also felt a need for new kind of musical notation. Schoenberg is often quoted from his "A New Twelve-Tone Notation"[1924]:
The need for a new notation, or a radical improvement of the old, is greater than it seems, and the number of ingenious minds that have tackled the problem is greater than one might think.
Others seek new form of expressions in microtonal music. Here is an excerpt from Microtonal Projects Ltd:
Microtonal Music uses pitches other than the 12 equally spaced semitones normally heard in western music or their close relations (such as the ones found in the tmperaments used in Baroque music). Such music is not particularly rare. The ancient Greeks used structured intervals as small as a quarter-tone, so do many folk music from around the world and Indian classical music.
Wednesday, January 24, 2007
Visualizing graphs
[All pictures in this article are reduced by half to speed-up loading of this page. Click on a picture to see its normal size]
To quote the first paragraph from Graphviz site:Graph visualization is a way of representing structural information as diagrams of abstract graphs and networks. Automatic graph drawing has many important applications in software engineering, database and web design, networking, and in visual interfaces for many other domains.
Graph visualization is a serious and hard business, and subject of very active research. There are some tools that do reasonably good layout and rendering job. One is Graphviz, which is based on the language DOT, developed in AT&T after 10 years of intensive research. The Mac OS X edition of Graphviz, by Glen Low, shown on the left, won two 2004 Apple Design Awards.
In this post I will demonstrate how to interface to this tool from Scheme-OSA. There is one problem though: Glen's tool is not scriptable. Fortunately Apple provides for more than one method to handle inter-application communication. One such method is Apple's Services, which use global Pasteboard for this purpose.
The second problem is the input language. Simple as it is, DOT is yet another language to learn, which is probably not what most of us would cherish. To spare you such effort I spent one afternoon reading documentation and writing a small translator from Scheme to DOT syntax. Now you can think and write Scheme when specifying visualization of your graphs.
The picture on the left is a capture of the Scheme-OSA window. Its upper portion, the input text view, contains a very simple graph example, written in Scheme. The code declares only few nodes and few paths, but nodes are individually colored, and one of them uses its own special set of parameters.
Node is a pair made of a name (symbol) and a quasi-record. Quasi-record is a list of properties. Property is a pair of two objects, where the first object must be a symbol - a key of the property. A value of a property can be one of: boolean, number, symbol or string. Path is a list of at least three symbols, where the first one is the right arrow, and the rest are names of nodes to be followed in order to form edges. This in essence describes the structure of a diagram. The DOT manual provides many other details, such as names of colors, names of node shapes (rectangle, triangle, etc. - about 20 of them), and other parameters, which are immaterial to this demo.
The last line of the input text view is the invocation of a command to display the graph specification in DOT language. Ready? Press the Perform button and you should see the DOT output in the output text view. Now select that view by pressing command-A,
then go to the Scheme-OSA menu and choose Scheme-OSA > Services > Graphviz > Open as Graph. The resulting picture will be displayed in Graphviz, as shown on the left. As you can see, one node (the one that says "hello world") is disconnected from the rest. This is what should be expected since the specification of paths above never refers to the node xyz, which is the "hello world" node. Plain and simple, but it could be even simpler if the Graphviz was scriptable. Maybe one day it will be as such. Concluding this demonstration, I should add that other editions of Graphviz (Linux, Windows) are not that simply to use as this one. Glen's tool prints its output to many, many graphic formats, and the rendering is excellent: no jagged lines, as in Linux, for example.
To quote the first paragraph from Graphviz site:Graph visualization is a way of representing structural information as diagrams of abstract graphs and networks. Automatic graph drawing has many important applications in software engineering, database and web design, networking, and in visual interfaces for many other domains.Graph visualization is a serious and hard business, and subject of very active research. There are some tools that do reasonably good layout and rendering job. One is Graphviz, which is based on the language DOT, developed in AT&T after 10 years of intensive research. The Mac OS X edition of Graphviz, by Glen Low, shown on the left, won two 2004 Apple Design Awards.
In this post I will demonstrate how to interface to this tool from Scheme-OSA. There is one problem though: Glen's tool is not scriptable. Fortunately Apple provides for more than one method to handle inter-application communication. One such method is Apple's Services, which use global Pasteboard for this purpose.
The second problem is the input language. Simple as it is, DOT is yet another language to learn, which is probably not what most of us would cherish. To spare you such effort I spent one afternoon reading documentation and writing a small translator from Scheme to DOT syntax. Now you can think and write Scheme when specifying visualization of your graphs.
The picture on the left is a capture of the Scheme-OSA window. Its upper portion, the input text view, contains a very simple graph example, written in Scheme. The code declares only few nodes and few paths, but nodes are individually colored, and one of them uses its own special set of parameters. Node is a pair made of a name (symbol) and a quasi-record. Quasi-record is a list of properties. Property is a pair of two objects, where the first object must be a symbol - a key of the property. A value of a property can be one of: boolean, number, symbol or string. Path is a list of at least three symbols, where the first one is the right arrow, and the rest are names of nodes to be followed in order to form edges. This in essence describes the structure of a diagram. The DOT manual provides many other details, such as names of colors, names of node shapes (rectangle, triangle, etc. - about 20 of them), and other parameters, which are immaterial to this demo.
The last line of the input text view is the invocation of a command to display the graph specification in DOT language. Ready? Press the Perform button and you should see the DOT output in the output text view. Now select that view by pressing command-A,then go to the Scheme-OSA menu and choose Scheme-OSA > Services > Graphviz > Open as Graph. The resulting picture will be displayed in Graphviz, as shown on the left. As you can see, one node (the one that says "hello world") is disconnected from the rest. This is what should be expected since the specification of paths above never refers to the node xyz, which is the "hello world" node. Plain and simple, but it could be even simpler if the Graphviz was scriptable. Maybe one day it will be as such. Concluding this demonstration, I should add that other editions of Graphviz (Linux, Windows) are not that simply to use as this one. Glen's tool prints its output to many, many graphic formats, and the rendering is excellent: no jagged lines, as in Linux, for example.
Monday, January 22, 2007
Processing sound via workflows
In my previous post, Sound engineering, I shortly reviewed Sound Studio scriptable application, which I consider a good candidate for interfacing from Scheme-OSA. Here is another one, with similar goals, but with dramatically different look and feel:
Sample Manager, version 3.01, from Audiofile Engineering.
According to their description:
The essence of Sample Manager
You can read all the blurp you want on the Audiofile Engineering site, but the essence of this software is, in my opinion, this:
Sample Manager is very extendable due to its support for user defined Automator actions, or Applescript support, where the latter, more or less, parallels the former.
Menus of Sample Manager are quite minimalistic but that should not fool you - you can gain a lot of power via built-in workflow templates, and you can easily built your own workflows by dragging Automator Actions from the drawer, on the right of the main windows. User interface is flexible, since you can easily expand or reduce the number of panes and drawers visible at any given time. The picture below shows just one such arrangement - where Details, Workflows Drawer and Actions Drawers have been hidden from view.
Comparison with Sound Studio
Compared to Sound Studio, reviewed in the previous post, GUI of Sample Manager seems a bit more complicated and it lacks few essential visual hints, which exist in the former. One example is "the fingerprint" (or overview) of entire sound channel. Sound Studio seems to have more traditional DSP power (Fourier spectrum and variety of filters) but Sample Manager seems to compensate on the file convertion/manipulation side.
I would be hard pressed to choose one over the other - especially since I have not used either one in a real project yet.
Comparison with my old software
I am a bit biased towards the Sample Manager since it reminds me my own software that I developed and commercially distributed long time ago. The domains are different (sound signals vs. power fault signals), macros are different (workflows vs. directed graphs), user interfaces are different (mine was not as pretty - considering dramatic changes that happened in meantime in GUI, graphics, processor speeds) but the concepts are generally similar.
I also started with a minimalistics browser - with a lot of manipulation and conversion abilities, but very little DSP power. But with time I added all those standard and non-standard transforms, convolutions, correlations, and some domains-specific algorithms. So if this may serve as any sign - I predict that Sample Manager will evolve into a very powerful product. :-)
Scriptability
From point of view of scriptability, both products seem to have similar functionality, and offer similar opportunities to Scheme-OSA.
The following is a very abbreviated digest of the functionality of Sample Manager. This is for the future reference for those willing to write Scheme wrappers around the commands listed below. You are excused, dear reader, from going down these lists with any attention whatsoever. Just skim them, please, to get some idea what this software is all about.
Main objects
There are four main objects in Sample Manager you can operate on:
Application object has a very large number of properties, but I will not list them here, since they are not essential for this digest.
Commands
Commands, however, are important because they describe functionality of this software. For a sake of simplicity I ommitted all parameters from this digest, but the direct ones.
Sample Manager, version 3.01, from Audiofile Engineering.
According to their description:
Sample Manager is the quintessential batch audio file processor for Mac OS X. Designed in Cocoa from the ground up, Sample Manager proudly takes advantage of CoreAudio, Quartz, and other solid OS X features [...]
Sample Manager is available now for a 15-day free trial. After the trial period, you'll need to purchase Sample Manager to continue using it. Sample Manager is available for only $90 from our online store [...]
For a limited time, Sample Manager 3 is available at the intro price of $70.
The essence of Sample Manager
You can read all the blurp you want on the Audiofile Engineering site, but the essence of this software is, in my opinion, this:
Sample Manager is very extendable due to its support for user defined Automator actions, or Applescript support, where the latter, more or less, parallels the former.
Menus of Sample Manager are quite minimalistic but that should not fool you - you can gain a lot of power via built-in workflow templates, and you can easily built your own workflows by dragging Automator Actions from the drawer, on the right of the main windows. User interface is flexible, since you can easily expand or reduce the number of panes and drawers visible at any given time. The picture below shows just one such arrangement - where Details, Workflows Drawer and Actions Drawers have been hidden from view.
Comparison with Sound Studio
Compared to Sound Studio, reviewed in the previous post, GUI of Sample Manager seems a bit more complicated and it lacks few essential visual hints, which exist in the former. One example is "the fingerprint" (or overview) of entire sound channel. Sound Studio seems to have more traditional DSP power (Fourier spectrum and variety of filters) but Sample Manager seems to compensate on the file convertion/manipulation side.
I would be hard pressed to choose one over the other - especially since I have not used either one in a real project yet.
Comparison with my old software
I am a bit biased towards the Sample Manager since it reminds me my own software that I developed and commercially distributed long time ago. The domains are different (sound signals vs. power fault signals), macros are different (workflows vs. directed graphs), user interfaces are different (mine was not as pretty - considering dramatic changes that happened in meantime in GUI, graphics, processor speeds) but the concepts are generally similar.
I also started with a minimalistics browser - with a lot of manipulation and conversion abilities, but very little DSP power. But with time I added all those standard and non-standard transforms, convolutions, correlations, and some domains-specific algorithms. So if this may serve as any sign - I predict that Sample Manager will evolve into a very powerful product. :-)
Scriptability
From point of view of scriptability, both products seem to have similar functionality, and offer similar opportunities to Scheme-OSA.
The following is a very abbreviated digest of the functionality of Sample Manager. This is for the future reference for those willing to write Scheme wrappers around the commands listed below. You are excused, dear reader, from going down these lists with any attention whatsoever. Just skim them, please, to get some idea what this software is all about.
Main objects
There are four main objects in Sample Manager you can operate on:
- application n : The Sample Manager application
- folder entry n : An entry in the files table relating to a folder
- group entry n : An entry in the files table that contains other entries
- smart group entry n : An entry in the files table that is the result of a search and contains other entries
Application object has a very large number of properties, but I will not list them here, since they are not essential for this digest.
Commands
Commands, however, are important because they describe functionality of this software. For a sake of simplicity I ommitted all parameters from this digest, but the direct ones.
- add to list v : Adds the files or folders to the file list
- remove from list v : Removes the entries from the file list
- play v : Starts playback of the selected entries
- stop v : Stops playback
- set v : Sets the value of the entries
- change gain of v : changes the level of the audio
- normalize v : sets the level of the audio files to a given amplitude - normalize entry or list of entry
- balance v : balances the channels of an audio file to an average level
- fade in v : fades in the audio from silence
- fade out v : fades out the audio to silence
- trim beginning of v : removes audio from the beginning
- trim end of v : removes audio from the end
- trim around loop of v : remove audio around a loop
- shorten v : removes audio from the file to make it a specific length
- pad v : adds silence to the beginning and/or end of an audio file
- extend v : adds silence to the file to make it a specific length
- convert format of v : converts the audio file format
- convert rate of v : converts the audio file sample rate
- convert range of v : converts the audio file bit range (depth)
- convert channels of v : converts the audio file channel format
- make labels at divisions for v : creates labels spaced evenly throughout an audio file
- make labels at tempo for v : creates labels on beat positions in an audio file
- make labels at thresholds for v : creates labels at audio thresholds in a file
- make labels at thresholds for entry or list of entry
- remove labels in v : removes all labels of the given types in the audio file
- extract at thresholds from v : creates new files from the between the given thresholds
- extract at labels from v : creates new files from the between the given labels
- extract channels from v : creates new mono files from the channels in the audio file
- mix v : Combines the audio files by mixing the audio
- append v : Combines the audio files by joining one after the other
- change speed of v : changes the speed of the audio by a given factor
- change length of v : changes the length of the audio to a given value
- change tempo of v : changes the tempo of the audio to a given value
- change pitch of v : changes the pitch of the audio
- transpose v : transposes the audio by a given interval
- filter DC of v : Filters the DC offset from the audio files
- reverse v : Reverses the audio files
- rename v : Renames the audio files
- export as aac v : exports the audio files to aac format
- export as ogg vorbis v : exports the audio files to ogg vorbis format
- export as vox v : exports the audio files to vox format
- compact v : removes unnecessary data from an audio file
- compress v : Compresses the audio files into a zip archive
- send via ftp v : Sends the audio files to an FTP server
- send to my idisk v : Sends the audio files to My iDisk
Sunday, January 21, 2007
Sound engineering
I have been busy last week exploring the world of notable scriptable applications for Mac OS X. This is, after all, the field targetted by Scheme-OSA. I found several ones worthy recommendation. In this post, I will shortly review Sound Studio 3, the essential tool for working with digital audio.
Programmed and copyrighted by Lucius Kwok, Felt Tip Software.
According to Lucius:
My first impression:
Commercial, price $80 but worth it, professionally done, simple and clean interface. Trial licence expires after 9 runs - plenty enough to explore.
Sound Studio is a scriptable application. It responds to following Applescript commands:
play, pause, unpause, stop, cut, copy, paste, clear, silence, crop, insert silence at, insert noise at, insert tone at, insert fm at, amplify, fade, normalize, compress dynamics, expand dynamics, noise gate, add noise, dc offset, interpolate, invert, swap channels, reverse, chorus, delay, flange, pitch, reverb, equalize, high pass, low pass.
From Scheme perspective: All those commands can be wrapped as Scheme procedures and used directly from Scheme-OSA in a batch mode for some repetetive tasks to be applied to a bunch of audio files.
Following is a digest of commands, as listed in Dictionary of Sound Studio. Details, such as arguments and optional parameters are omitted.
This is an impressive set of useful commands, that - I am certain - some user of Scheme-OSA (or myself) will nicely wrap in a practical Scheme module.
Programmed and copyrighted by Lucius Kwok, Felt Tip Software.
According to Lucius:
Sound Studio 3 is an easy-to-use sound editing application for recording and producing audio digitally on your Mac OS X computer. You can digitize records and tapes, record live events, create podcasts, tweak the audio in your iMovie or Final Cut project, and do many more things with Sound Studio.
Buy it, try it, cry it at Freeverse.
My first impression:
Commercial, price $80 but worth it, professionally done, simple and clean interface. Trial licence expires after 9 runs - plenty enough to explore.
Scriptability
Sound Studio is a scriptable application. It responds to following Applescript commands:
play, pause, unpause, stop, cut, copy, paste, clear, silence, crop, insert silence at, insert noise at, insert tone at, insert fm at, amplify, fade, normalize, compress dynamics, expand dynamics, noise gate, add noise, dc offset, interpolate, invert, swap channels, reverse, chorus, delay, flange, pitch, reverb, equalize, high pass, low pass.
From Scheme perspective: All those commands can be wrapped as Scheme procedures and used directly from Scheme-OSA in a batch mode for some repetetive tasks to be applied to a bunch of audio files.
Following is a digest of commands, as listed in Dictionary of Sound Studio. Details, such as arguments and optional parameters are omitted.
- play v : Play back audio.
- record v : Record audio at end of file.
- pause v : Pause playback or recording.
- unpause v : Unpause playback or recording.
- stop v : Stop playback or recording.
- resample v : Resample audio sample data.
- mixdown v : Mixdown audio tracks.
- cut v : Cut selection
- copy v : Copy selection
- paste v : Paste selection
- clear v : Delete audio selection
- silence v : Silence selection
- crop v : Crop selection
- split v : Split document by markers and save to files.
- insert silence at v : Insert silence at selection.
- insert tone at v : Insert tone at selection.
- insert fm at v : Insert FM Synthesis at selection.
- amplify v : Apply the Amplify/Volume filter to the audio. amplify
- normalize v : Apply the Normalize filter to the audio.
- compress dynamics v : Apply the Dynamics Compressor filter to the audio.
- expand dynamics v : Apply the Dynamics Expander filter to the audio.
- add noise v : Apply the Add Noise filter to the audio.
- interpolate v : Apply the Interpolate filter to the audio; replaces audio with a straight line between the first and last samples.
- invert v : Apply the Invert Signal Polarity filter to the audio; flips audio vertically.
- swap channels v : Apply the Swap Left and Right Channels filter to the audio; swaps audio data between even and odd numbered tracks.
- reverse v : Apply the Backwards/Reverse Audio filter to the audio; makes audio sound like it is played backwards.
- chorus v : Apply the Chorus filter to the audio.
- delay v : Apply the Delay and Echo filter to the audio.
- flange v : Apply the Flanger filter to the audio.
- pitch v : Apply the Pitch and Tempo filter to the audio.
- pitch specifier : the item(s) to filter
- equalize v : Apply the Graphic Equalizer filter to the audio.
- high pass v : Apply the High Pass filter to the audio.
- low pass v : Apply the Low Pass filter to the audio.
- expand tilde v : Expand tilde in POSIX path to full absolute path.
This is an impressive set of useful commands, that - I am certain - some user of Scheme-OSA (or myself) will nicely wrap in a practical Scheme module.
Sunday, January 14, 2007
How about some music?
In this post I will continue exploration of multimedia with Scheme-OSA. So far we have learned how to invoke all sorts of dialog boxes and choices, how to plot in two and three dimensions, how to translate text to speech and how to play short sounds. Actually, I use these sounds for audio feedback as a help in debugging and diagnostics of Scheme-OSA. For example, the "Submarine" sound tells me that the application just entered into the callback mode, while the "Frog" sound informs me that the interpreter has been just loaded and is running, etc. A simple beep would not do the job.
So how about some real music? A fair question would be: Why on Earth would you need scripting if you have the iTunes browser, which is such an excellent interactive tool? Apparently there are some reasons for scripting iTunes. Take a look at Doug's Applescripts for iTunes. There are 419 scripts, organized in 17 or so categories: Managing Tracks, Managing Track Info, Managing Artwork, Managing Playlists, Controlling iTunes, Exporting Info, Managing Files, Networking, Internet, iPod .. - to name the few.
Application iTunes is obviously scriptable. It is also well designed, well described, and most of its functionality is model-based, rather than view-based, which means that it is easy to use. But iTunes is not the only game in town. If you need something five times smaller than iTunes, and which also runs on background you might consider Play Sound by Microcosm Software. The author of Play Sound claims that
This excerpt from possibly much larger library of music wrappers demonstrates how to control music from Scheme-OSA: how to play a single sound file, how to play a list of sound files, how to choose a bunch of files interactively and then play them, and how to stop playing them.
Sorry that I cannot demonstrate it in real time from this blog, but believe me - it works!
So how about some real music? A fair question would be: Why on Earth would you need scripting if you have the iTunes browser, which is such an excellent interactive tool? Apparently there are some reasons for scripting iTunes. Take a look at Doug's Applescripts for iTunes. There are 419 scripts, organized in 17 or so categories: Managing Tracks, Managing Track Info, Managing Artwork, Managing Playlists, Controlling iTunes, Exporting Info, Managing Files, Networking, Internet, iPod .. - to name the few.
Application iTunes is obviously scriptable. It is also well designed, well described, and most of its functionality is model-based, rather than view-based, which means that it is easy to use. But iTunes is not the only game in town. If you need something five times smaller than iTunes, and which also runs on background you might consider Play Sound by Microcosm Software. The author of Play Sound claims that
Play Sound is a simple, no-fuss sound player for Mac OS X. It supports QuickTime sound files (for example, AIFF or MP3 files), Classic Mac OS System 7 sound files (files with a file type of 'sfil' that contain 'snd ' resources), or any 'snd ' resources embedded into any file. You can play an unlimited number of sounds concurrently or one at a time. You can loop sounds, repeat sounds, play specified portions of sounds, control the volume of sounds played, pause and continue sound play, and press the Escape or Command-Period keys to abort sound play.
This excerpt from possibly much larger library of music wrappers demonstrates how to control music from Scheme-OSA: how to play a single sound file, how to play a list of sound files, how to choose a bunch of files interactively and then play them, and how to stop playing them.
Sorry that I cannot demonstrate it in real time from this blog, but believe me - it works!
Saturday, January 13, 2007
Scalar, Contour and Surface maps
Here are some more examples of plots indirectly produced by Scheme-OSA. The direct, real producer of these pictures is Smile by Satimage Software. From Scheme-OSA perspective Smile is just an intelligent input/output device, capable of doing some outstanding visualization work. Average user of Scheme-OSA does not need to know anything about Smile - with the exception of the fact that it is a Mac OS X application and that it has been installed on the system. Few basic Scheme wrappers around Smile libraries should suffice for most of his/her needs. However, Smile also provides an excellent documentation, and this is what users should consult before writing their own wrappers.
This picture is a combination of two maps: Scalar Map and Contour Map. Three lines of Applescript code are needed to produce the picture itself. Few more lines are needed to add title and labels. And few more lines are required to wrap it up in a Scheme code.
Although this map has polar symmetry it has been generated using rectangular coordinates. User had requested 12 equidistant contours to be overlayed on a top of scalar map.
This is just a scalar map; no contours are overlayed on a top of it. Notice that this picture was generated using polar coordinates - hence it looks clipped ouside the circle.
And this how how the previous map looks in three dimensions.
This picture is a combination of two maps: Scalar Map and Contour Map. Three lines of Applescript code are needed to produce the picture itself. Few more lines are needed to add title and labels. And few more lines are required to wrap it up in a Scheme code.Although this map has polar symmetry it has been generated using rectangular coordinates. User had requested 12 equidistant contours to be overlayed on a top of scalar map.
This is just a scalar map; no contours are overlayed on a top of it. Notice that this picture was generated using polar coordinates - hence it looks clipped ouside the circle.
And this how how the previous map looks in three dimensions.
Friday, January 12, 2007
Plotting formulas with Smile
Smile is a scriptable visualization application, which is ideally suited for demonstration of some of the capabilities of Scheme-OSA. Smile is described at the bottom of this page. Shown below is a capture of Scheme-OSA window with a very simple example of formula plotting, which takes advantage of Smile capability of plotting.
Note that we are using here the interpreter MzScheme. Although bare bones MzScheme does not have its own plotting capability, but DrScheme - which uses MzScheme - does. Some could therefore argue that Scheme-OSA and its interface to Smile is redundant in such setting. Fair enough, but consider this: as long as you are on Mac OS X platform you can use the same code, irrespectively of interpreter of your choice: MzScheme, Petite Chez Scheme, Bigloo, Gambit, Gauche, Chicken, etc. And no matter what interpreter you are using the pictures will be always the same, as presented below.
We constrained ourselves to one specific set of visualization parameters for this picture. Smile application let you change any of those - before or after your data has been plotted. Frame size and type; size and thickness of ticks; grid size, pattern and color; location of curve labels; thickness, color and pattern of the curves; or type of curves themselves - these are the parameters changeable at ease.
Multi-plot Scheme wrapper
As said before, for a simplicity sake we using here one selected choice of parameters for data presentation.
Application Smile
Smile is an Applescript-based sophisticated software from Satimage Software. It comes in three versions:
1. Regular Smile - free version, with GUI builder, sophisticated 2D drawing engine, and many other goodies
2. Full Smile = Regular Smile + SmileLab, with 2D and 3D visualization engine. Commercial
3. Smile Server - commercial
Additionally, Satimage distributes free OSA extensions ("osaxen"): numerics.osax and satimage.osax, with many useful extensions to standard Applescript.
Smile is a scriptable application. This means that it can be talked to via OSA framework, as we do it from our Scheme-OSA. There are two types of scriptable applications: the sophisticated ones are model-based, less sophisticated are view-based. The latter force the user to explore how the application is assembled: what sort of views it has, how the buttons are named or indexed, etc.
Although with some patience the user can deduce the structure of the objects forming the view-based scriptable applications , they are not as adaptable and easy to use as model-based scriptable applications. Ironic as it might sound - most Applescript Studio applications are view-based. Many Cocoa applications are model-based. Scheme-OSA is still view-based, but I hope to convert it to a model-based scriptable application. Smile is model-based, even though most of its code is Applescript. That shows how sophisticated Smile is.
Note that we are using here the interpreter MzScheme. Although bare bones MzScheme does not have its own plotting capability, but DrScheme - which uses MzScheme - does. Some could therefore argue that Scheme-OSA and its interface to Smile is redundant in such setting. Fair enough, but consider this: as long as you are on Mac OS X platform you can use the same code, irrespectively of interpreter of your choice: MzScheme, Petite Chez Scheme, Bigloo, Gambit, Gauche, Chicken, etc. And no matter what interpreter you are using the pictures will be always the same, as presented below.
We constrained ourselves to one specific set of visualization parameters for this picture. Smile application let you change any of those - before or after your data has been plotted. Frame size and type; size and thickness of ticks; grid size, pattern and color; location of curve labels; thickness, color and pattern of the curves; or type of curves themselves - these are the parameters changeable at ease. Multi-plot Scheme wrapper
As said before, for a simplicity sake we using here one selected choice of parameters for data presentation.
;; Plot a list of formulas for a given enumerator and descriptor, where
;; + formulas is a list of records: (('recipe atring) ('name string))
;; + enumerator is a list describing a range of abscissas: (first second last)
;; + descriptor is a list of strings: (title xlabel ylabel).
(define (multi-plot formulas enumerator descriptor)
(define handler (string->symbol (string-append
"on multi_plot of recipes for abscissas ";; no line break
" given descriptor:ds\r"
" tell application \"Smile\"\r"
" set v to \"\"\r"
" repeat with i in recipes\r"
" set y to evalformula (recipe of i) with {x:abscissas}\r"
" set c to QuickCurve(abscissas, y, v)\r"
" set name of c to (name of i)\r"
" set v to container of c\r"
" end repeat\r"
" set properties of v to ";;no line break
" {name:(item 1 of ds), legend abscissa:8, ";;no line break
" legend kind:2, xlabel:(item 2 of ds), ylabel:(item 3 of ds)}\r"
" draw window of v\r"
" end tell\r"
"end multi_plot\r")))
(define abscissas
(enumerate-until (car enumerator) (cadr enumerator) (caddr enumerator)))
(osa (make-osa-string
(list handler 'multi_plot 'of formulas 'for abscissas
'given 'descriptor: descriptor ))))
Application Smile
Smile is an Applescript-based sophisticated software from Satimage Software. It comes in three versions:
1. Regular Smile - free version, with GUI builder, sophisticated 2D drawing engine, and many other goodies
2. Full Smile = Regular Smile + SmileLab, with 2D and 3D visualization engine. Commercial
3. Smile Server - commercial
Additionally, Satimage distributes free OSA extensions ("osaxen"): numerics.osax and satimage.osax, with many useful extensions to standard Applescript.
Smile is a scriptable application. This means that it can be talked to via OSA framework, as we do it from our Scheme-OSA. There are two types of scriptable applications: the sophisticated ones are model-based, less sophisticated are view-based. The latter force the user to explore how the application is assembled: what sort of views it has, how the buttons are named or indexed, etc.
Although with some patience the user can deduce the structure of the objects forming the view-based scriptable applications , they are not as adaptable and easy to use as model-based scriptable applications. Ironic as it might sound - most Applescript Studio applications are view-based. Many Cocoa applications are model-based. Scheme-OSA is still view-based, but I hope to convert it to a model-based scriptable application. Smile is model-based, even though most of its code is Applescript. That shows how sophisticated Smile is.
Monday, January 8, 2007
User Interaction Suite
At the bottom of this post is a collection of Scheme wrappers for selected handlers from User Interaction Suite of StandardAdditions OSAX[1].
Contents of User Interaction Suite
beep v : Beep 1 or more times
choose application v : Choose an application on this machine or the network
choose color v : Choose a color
choose file v : Choose a file on a disk or server
choose file name v : Get a new file reference from the user, without creating the file
choose folder v : Choose a folder on a disk or server
choose from list v : Choose one or more items from a list
choose remote application v: Choose a running application on a remote machine or on this machine
choose URL v : Choose a service on the Internet
delay v : Pause for a fixed amount of time
display alert v : Display an alert
display dialog v : Display a dialog box, optionally requesting user input
say v : Speak the given text
Usage
The wrappers isolate Scheme user from the details of bridging between Scheme and Applescript. The user just sees Scheme expressions like this:
which - when executed in Scheme-OSA environment - produces a dialog panel asking for a user name. It is provided with default answer "Peter Brown", two buttons: "Cancel" and "OK", default button "OK" and a timeout of 12 seconds.
The dialog returns the answer in the form of associative list (a simulated record, before real Scheme records become standard), such as:
which means a button returned: (bhit "OK"), a text returned: (ttxt "John Brown") and boolean indicating whether or not the dialog gave up waiting for user response: (gavu #f).
The symbols 'bhit, 'ttxt and 'gavu have been directly translated from Applescript; I have made no attempt to make them more user friendly. Fortunately, there are not that many such criptic names. Apple insists on four-character abbreviations for public symbols, but anything that you make for your own use stays non-abbreviated. Hence, your private records, such as
will preserve its readability when crossing the border between Scheme and Applescript.
Under the hood
Underneath this presentation layer are translation services: from Scheme to Applescript and back to Scheme.
Firstly, a Scheme expression, such as this one:
is send to Scheme interpreter for evaluation. The interpreter converts it to the following Applescript command
then wraps it into string "(osa)", prints it out to standard output and enters into port reading mode - waiting for an answer from the Applescript. After reception of the osa-string the bridge recognizes the header "(osa " as a signal not to print it out but to submit the tail of such string for compilation and execution as an Applescript command. Consequently, the dialog is displayed, data collected from the user and the response is prepared in the form of a record with several properties, as in:
The bridge intercepts it, translates it back to Scheme as a alist:
and returns it to Scheme interpreter which was waiting for such answer.The results are then used for further computations, or just sent back to the bridge for display.
References
[1] Full documentation of StandardAdditions is provided in its dictionary of definitions. The dictionary is conveniently browsable using dictionary browser, which can be opened on StandardAdditions either by double clicking on /System/Library/ScriptingAdditions/StandardAdditions.osax or by invoking Script Editor application, opening a list of scriptable applications and additions from its menu File -> Open Dictionary ... and finally selecting the Standard Additions from that list.
[2] 'AppleScript Scripting Additions Guide: English Dialect', a document from Apple Computers
Scheme wrappers
Contents of User Interaction Suite
beep v : Beep 1 or more times
choose application v : Choose an application on this machine or the network
choose color v : Choose a color
choose file v : Choose a file on a disk or server
choose file name v : Get a new file reference from the user, without creating the file
choose folder v : Choose a folder on a disk or server
choose from list v : Choose one or more items from a list
choose remote application v: Choose a running application on a remote machine or on this machine
choose URL v : Choose a service on the Internet
delay v : Pause for a fixed amount of time
display alert v : Display an alert
display dialog v : Display a dialog box, optionally requesting user input
say v : Speak the given text
Usage
The wrappers isolate Scheme user from the details of bridging between Scheme and Applescript. The user just sees Scheme expressions like this:
(display-dialog "Your name?" "Peter brown" '("Cancel" "OK") "OK" 12)
which - when executed in Scheme-OSA environment - produces a dialog panel asking for a user name. It is provided with default answer "Peter Brown", two buttons: "Cancel" and "OK", default button "OK" and a timeout of 12 seconds.
The dialog returns the answer in the form of associative list (a simulated record, before real Scheme records become standard), such as:
((bhit "OK") (ttxt "John Brown") (gavu #f))
which means a button returned: (bhit "OK"), a text returned: (ttxt "John Brown") and boolean indicating whether or not the dialog gave up waiting for user response: (gavu #f).
The symbols 'bhit, 'ttxt and 'gavu have been directly translated from Applescript; I have made no attempt to make them more user friendly. Fortunately, there are not that many such criptic names. Apple insists on four-character abbreviations for public symbols, but anything that you make for your own use stays non-abbreviated. Hence, your private records, such as
'((age 19) (factors '(3 5 7)) (speed 120))
will preserve its readability when crossing the border between Scheme and Applescript.
Under the hood
Underneath this presentation layer are translation services: from Scheme to Applescript and back to Scheme.
Firstly, a Scheme expression, such as this one:
(display-dialog "Your name?" "Peter Brown" '("Cancel" "OK") "OK" 12)
is send to Scheme interpreter for evaluation. The interpreter converts it to the following Applescript command
display dialog "Your name?" default answer "Peter Brown" ¬
buttons {"Cancel", "OK"} default button "OK" giving up after 12
then wraps it into string "(osa
{bhit: "OK", ttxt: "John Green", gavu: false}.
The bridge intercepts it, translates it back to Scheme as a alist:
((bhit "OK") (ttxt "John Green") (gavu #f))
and returns it to Scheme interpreter which was waiting for such answer.The results are then used for further computations, or just sent back to the bridge for display.
References
[1] Full documentation of StandardAdditions is provided in its dictionary of definitions. The dictionary is conveniently browsable using dictionary browser, which can be opened on StandardAdditions either by double clicking on /System/Library/ScriptingAdditions/StandardAdditions.osax or by invoking Script Editor application, opening a list of scriptable applications and additions from its menu File -> Open Dictionary ... and finally selecting the Standard Additions from that list.
[2] 'AppleScript Scripting Additions Guide: English Dialect', a document from Apple Computers
Scheme wrappers
;; procedure: (display-dialog title answer button-list default-button timeout)
;; return: aList
;; display-dialog:: String -> String-> List of String -> String -> Integer-> Alist
;;
;; Display a titled dialog window with title, default answer, at most three buttons,
;; one of which is declared a default and a timeout. Return a record with three
;; properties:
;; ((bhit name-of-a-button-pressed)
;; (ttxt string-from-the-text-field)
;; (gavu true-if-dialog-gaved-up-due-to-timeout))
;;
;; Example:
;; (display-dialog "Your name?" "Peter Brown" '("Cancel" "OK") "OK" 0)
;; ==> ((bhit "OK") (ttxt "Juan de Silva") (gavu #f))
(define (display-dialog text
default-answer buttons default-button timeout)
(define str (make-osa-string
(list 'display 'dialog text
'default 'answer default-answer
'buttons buttons
'default 'button default-button
'giving 'up 'after timeout)))
(cond
((not (member default-button buttons))
(error 'display-dialog
"Default button should belong to provided list of buttons"))
(else (osa str))))
;; procedure: (display-message text buttons)
;; Produce a message with several buttons
;; (display-message "Do you want to proceed?" '("No" "Yes"))
(define (display-message text buttons)
(define str (make-osa-string
(list 'display 'dialog text
'buttons buttons
)))
(osa str))
;; procedure: (display-alert text)
;; Display an alert
;; Return: ((bhit "OK"))
;; Example:
;; (display-alert "Beware of dog!")
(define (display-alert text)
(osa (make-osa-string (list 'display 'alert text))))
;; procedure: (ask-for-number title default)
;; Ask for a number, providing dialog title and default numerical value
;; Return a number
;; Example:
;; (* (ask-for-number "first number" 10) (ask-for-number "second number" 20))
;; ==> 200
(define (ask-for-number title default)
(string->number
(cadr (assq 'ttxt
(display-dialog
title
(number->string default)
'("OK") "OK" 0)))))
;; procedure: (choose-from-list aList prompt)
;; Choose one or more items from a list
;; Example:
;; (choose-from-list '("Eva" "Bob" "John") "Select a winner")
(define (choose-from-list aList prompt)
(osa (make-osa-string (list
'choose 'from 'list aList
'with 'prompt prompt
'with 'multiple 'selections 'allowed))))
;; procedure: (choose-files directory-path)
;; Choose one or more files (but not directories) from a specified directory path.
;; Return list of paths.
;; Example:
;; (choose-files "/Users/jans")
(define (choose-files directory-path)
(define handler (string->symbol (string-append
"on choose_files from directory\r"
" set dirAlias to POSIX file directory\r"
" set xs to choose file default location alias dirAlias"; no line break!
" with multiple selections allowed\r"
" set ys to {}\r"
" repeat with x in xs\r"
" set ys to ys & (POSIX path of x)\r"
" end repeat\r"
" return ys\r"
"end choose_files\r")))
(osa (make-osa-string
(list handler 'choose_files 'from directory-path))))
;; procedure: (choose-folders root-folder)
;; Choose one or more folders from a specified root-folder
;; Return a list of folders.
;; (choose-folders "/Users/jans")
(define (choose-folders root-folder)
(define handler (string->symbol (string-append
"on choose_folders from root_folder\r"
" set dirAlias to POSIX file root_folder\r"
" set xs to choose folder default location alias dirAlias"; no line break!
" with multiple selections allowed\r"
" set ys to {}\r"
" repeat with x in xs\r"
" set ys to ys & (POSIX path of x)\r"
" end repeat\r"
" return ys\r"
"end choose_folders\r")))
(osa (make-osa-string
(list handler 'choose_folders 'from root-folder))))
;; procedure: (choose-color default-rgb-list)
;; Choose a color from the color chooser
;; Example:
;; (choose-color '(0 65535 0))
(define (choose-color default-rgb-list)
(osa (make-osa-string (list
'choose 'color 'default 'color default-rgb-list))))
;; procedure: (beep n)
;; Beep 1 or more times
;; Return: undefined
(define (beep n)
(define str (make-osa-string (list 'beep n)))
(osa str))
;; procedure: (say text)
;; Speak the given text
;; Return: undefined
(define (say text)
(osa (make-osa-string (list 'say text))))
;; procedure: (play-standard-sound sound-name volume)
;;
;; Play a named sound at some volume level (0-7)
;; Avaulable sounds are taken (from /System/Library/Sounds/):
;; "Basso", "Blow", "Bottle", "Frog", "Funk", "Glass", "Hero", "Morse", "Ping",
;; "Pop", "Purr", "Sosumi", "Submarine", "Tink"
;;
;; Example:
;; (play-standard-sound "Frog" 7)
(define (play-standard-sound sound-name volume)
(define handler (string->symbol (string-append
"on play_standard_sound for soundName at someVolume\r"
" try\r"
" tell application \"Scheme-OSA\"\r"
" set volume someVolume\r"
" set theSound to load sound soundName\r"
" play theSound\r"
" delete theSound\r"
" end tell\r"
" on error msg number n\r"
" error soundName & \" is not a standard sound\" number n\r"
" end try\r"
"end play_standard_sound\r")))
(osa (make-osa-string
(list handler 'play_standard_sound 'for sound-name 'at volume))))
Sunday, January 7, 2007
Building Libraries
A picture above is a snapshot of Apple's Script Editor: at the input text view there is a definition of the handler choose_folders and below it is a "just evaluated" expression. The result is shown below, in the result text view, as a list of three chosen directories. All of this is of course written in Applescript.
This code is a wrapper around the choose folder handler, defined in User Interaction Suite of the Standard Additions OSAX, which enriches standard Applescript with several suites of commonly used handlers.
The formal description of the choose folder is shown below:
choose folder v : Choose a folder on a disk or server
choose folder
[with prompt string] :
the prompt to be displayed in the dialog box
[default location alias] :
the default folder location
[invisibles boolean] :
Show invisible files and folders? (default is false)
[multiple selections allowed boolean] :
Allow multiple items to be selected? (default is false)
[showing package contents boolean] :
Show the contents of packages?
(Packages will be treated as folders.
Default is false.)
→ alias : the chosen folder
The items in square brackets are optional. The one which says [default location alias], means that user can specify a default location, on which a browser of folders will be open. The world alias is a fancy name for existing file or directory, immutable with respect to its repositioning into other location of your directory structure. Once you make an alias and move it somewhere else the Applescript will find it for you anyway. But what is really important here is the format of this alias; it must be something like this: alias "ibook:Users:jans".
But what if you wish to use the standard POSIX directory paths instead? Well you would have to specify it as: alias POSIX file "/Users/jans" - with or without the terminating forward slash. The wrapper handler choose_folders customizes the original handler: it will accept "root folder" in POSIX format, it will allow to select more than one folder, and it will represent the result as a list of chosen folders in POSIX format. Other options are simply ignored.
This window, in turn, is Scheme-OSA window. There is an equivalent definition of the procedure choose-folders, this time of course written in legal Scheme.
Below it is "just evaluated" Scheme expression (choose-folders "/Users/jans"). The result is displayed in output text view as a Scheme list of three directories in POSIX format.
Notice that the reddish-brownish text on the left is almost the exact replica of the handler in top window. Well, not exactly. We need to preserve the formal syntax of Applescript, which does not like if a single command line is broken in half. It also likes each command on a separate line. And it prefers return rather than newline line terminators. Accordingly we assemble all of this from a list of strings, and then combine everything together with a subroutine invocation. At the end of this procedure is a little magic, which will announce it to Scheme-OSA bridge as a callback, rather than normal Scheme procedure.
Conclusions
So this is how we build new Scheme procedures, which are destined to talk to Applescript world.
1. We first read appropriate documentation.
2. We then start experimenting with Applescript using Script Editor, or other such tools, writing some Applescript code.
3. Once we are happy with the result we port such code to Scheme.
Those who know nothing about Applescript do not need to despair. For starters, I am providing a small library of useful procedures from User Interaction Suite of Standard Additions. You can go quite far with just this library.
Secondly, Applescript is not that difficult to learn, so you can start writing your own Scheme wrappers in few days.
Saturday, January 6, 2007
Introducing Scheme-OSA
The prototype
The picture above is a snapshot of window of working Scheme-OSA. Going from left to right along the window toolbar we can see that:
1. "Interpreter" has been installed (because of the green dot in the on/off button);
2. The name of interpreter is "Petite Chez ...";
3. The interpreter is ready to "Perform", that is, evaluate next command;
4. The "Break" button is disabled because there is nothing to break from yet;
5. The "Decorate" button is (always) enabled. In fact we already used it to colorize the Scheme code in the input text view: comments are green, strings are brownish and keywords are blue.
After pressing the "Perform" button (or command-return keys), Scheme-OSA will initialize evaluation of any selection in the input text view (none selected here), or of all the code in this view.
Consequently, a friendly voice, which you can pre-select from a list of several voices, will ask you to enter your name, then a dialog box will pop up asking you to enter your name (default name "Peter Fonda"). The box will be discharged after pressing "OK", and the voice will ask you again whether your real name is indeed "Peter Fonda", or whatever name you entered. If you happened to press "Cancel", rather than "OK", the Scheme error will be generated instead.
Any time the evaluation takes place the red "Break" button becomes enabled, giving you a chance to cancel longish loops of other such events. This button plays the role of Control-C in Unix terminal.
The goals
Scheme-OSA is conceived as Interactive Development Environment for Scheme on Mac OS X platform. It is also a framework, which will hopefully deliver access from Scheme to many facilities and services of that platform, and to many specialized scriptable applications. Because it piggy-backs on Applescript, it should be able to do what regular applescripts do. Play music, talk, create 2D and 3D scientific plots, animate OpenGL goodies, exchange data with professional editors, spreadsheets, etc. -- short of dancing and cooking...
In other words, it should be able to behave as regular Scheme interpreter - but with all those additional goodies available to Applescript. It is still under development, hence many of those promisses seem too far fetched. Some however are already working, other features will, hopefully, appear in the near future.
Take a note: this is just a prototype, and I am not going to distribute any code as yet - with the exception of some Scheme modules to anyone interested, or the Xcode bundles to those who wish to help with reviews, development, testing, documentation or porting Scheme-OSA to the dual universal platform.
The non-goals
I do not plan to build yet another programming editor. Instead I plan to take advantage of the best existing editors in Mac OS X world. Let the others do your job if they can do it better than you can.
Let me explain. More than a year ago I wrote my first predecessor of Scheme-OSA. It did not rely on Applescript, but it provided two services, which one might call "glorified copy" and "glorified paste". It worked as follows: after invocation of that application I would load an interpreter of my choice (usually Petite Chez Scheme or MzScheme), then hide the application away - doing all my work in a programmer's editor of my choice, which happens to be TextWrangler[3].
After selecting a portion of a Scheme code, or entire module, or Scheme's load command, I will copy it directly to running Scheme interpreter via command-9 (visual association with left parenthesis) for evaluation. If I were interested in any Scheme response I would wait a bit then hit command-0 (visual association with right perenthesis), which would copy Scheme response right below the original selection.
That was good enough for me with respect to interaction with a Scheme interpreter. Any other perks that Emacs users love to have are provided by TextWrangler: text coloring, code balancing, extensive search/grep/regex facilities, diffs, buffers vs. files, text formatting, built-in shell scripts, etc. In addition, TextWrangler has fine support for AppleScript, which you can take advantage of to significantly extent its functionalities.
Although I incorportated some of the standard features of programmers' editor to Scheme-OSA, such as text coloring and code balancing, they are not there to completely eliminate a good text editor. I still envision myself doing most of my editing work in TextWrangler. But when I switch to Scheme-OSA I want to have some of the good perks too.
A year has passed and I suddenly realized that I can build a much more powerful tool than my old (still in use) prototype. I have never appreciated Applescript - thinking in terms: "Oh, this must be one of those Mickey Mouse languages". I was wrong - it is a very powerful means for taking advantage of the best applications that Mac OS X can offer today.
The implementation
OSA stands for Open Scripting Architecture, which provides framework for one of the forms of inter-application communication on Mac OS X platform and the set of libraries open to anyone wishing to plug her programming language of choice to OSA. The primary OSA language is Apple's AppleScript. There are few other languages that are already OSA-compliant: Python for one, Perl in some form, JavaScript under development.
Scheme-OSA does not plug directly to OSA libraries, but indirectly by riding on the back of AppleScript. I am not sure whether this is the best way to do, but this approach offers several advantages.
1. Short time to delivery. I did not have to go through typical hoops: marshalling of primitives, objects, pointers, then binding to OSA functions and all of that stuff. Instead few very simple conversions were required - more or less the same stuff that is required when implementing XML-RPC.
2. Safety. Communication with Applescript is easy: there are no pointer-related issues, dangerous crashes, and so on. Worst what can happen is its occasional refusal "I do not understand your message" - as in Smalltalk.
3. Ease of development. I can use Applescript to script Scheme-OSA any time either under control of Xcode or outside it. Scripting the product helps in evaluating approaches, trying "what if scenarios", testing, and so on.
Scheme-OSA is being built as "Applescript Studio" product, with significant support from Scheme itself. Most of the code is Applescript, 10-20% of the code is Cocoa (Objective-C). The only third party library is Satimage[1], a freely available product, which I use for regexp oriented tasks.
No apologies for ignoring Emacs
I should have loved Emacs, because I used to love Smalltalk, which - similarly to Emacs - tends to behave as an operating system on its own. But I have few reservation with respect to Emacs. (OOPS, I am feeling like loosing few potential friends right now:-)). Obsolete user interface and bloated code aside, the real problem is my own memory: I dislike memorizing all those triple key combinations. This was also a reson I never liked Vi editor (OOPS, another bunch of sympathizers is gone! :-)). I know some easy to remember key sequences: Command-C and Command-V for copy and paste, Command-B for bold, Command-I for italic, but I will be certainly lost in the maze of Emacs shortcuts.
I was happy to discover Aquamacs[5] a year ago. As they claim on their web page:
We support the standard Mac user interface that you've come to love. For instance, in addition to traditional Emacs shortcuts like C-x C-f (open a new file), Aquamacs understands Apple-O. Aquamacs behaves like a modern application on Mac (or Windows) when it comes to selecting, copying, pasting texts within Aquamacs or in between applications. Aquamacs offers nice, smooth fonts. Asian input methods work. It's easy to install and runs out-of-the box with no configuration. And all is built on GNU Emacs, so you can use your favorite Emacs packages!
I did spend some time doing some extra customization of Aquamacs, but once I noticed that it becomes more and more obese with each new release, I decided that this tool was not for me. With all due respect, I am really happy with TextWrangler - as it comes to text editing.
No apologies for choosing Mac OS X
Mac OS X is my platform of choice of the last two years. Mostly as a programmer but also as a user - I crawled on DOS in Basic, C and MASM; managed to program in C on Unix graphic workstations; worked with passion in several self-contained, host-dependent and host-independent Smalltalks; did some angry programming in MS-Windows; then discovered Linux and stayed with it for at least 10 good years - programming in Eiffel, Java, C, Objective-C, Smalltalk and a bunch of other esoteric languages, Haskell including.
I like the Linux's bazaar approach to sharing the knowledge and building software for the common goals. I learned a lot from Linux community, and I also contributed back a bit. But I dislike randomness in delivered goods: although there are plenty of high quality software packages in the Linux world, there are also many poorly documented and buggy utilities and applications. I have been growing impatient and started valuing my time; I have become tired of fixing somobedy else's bugs and going through countless hours of installations, upgrades, and often fruitless evaluations of third party packages.
So when the NextStep for Intel became available I jumped off the Linux ship for a while; I really liked what I saw and experienced: dazzling user interface - reminescent of Smalltalk; excellent inter-process communication facilities, such as Services; very good documentation, very good software development tools, Unix and Mach kernel under the hood, bug-free applications and friendly community. But there was one big problem: the Company and the Community were too small to keep up with either Microsoft or Linux. There were too few drivers available and because of it installations or upgrades of NextStep OS were tough - I once spent good three days swapping cards until I finally succeded in getting my machine running the NextStep. Software-wise, important applications, such as the web browser, were also lagging behind Netscape or Explorer.
So, for all the goodies I saw in the NextStep OS, I realized that there was not much future in that platform. With all regrets, I moved back to Linux. Sadly, I was right - NextStep went down. Happily, it got reincarnated as Mac OS X. Two years ago I discovered the old friend, and I have not been looking back ever since. I am not going to gloss over advantages of this platform: suffices to say that it suits me well, and I am not going to be apologetic about it.
References:
[1] "Satimage Osax"[2] by Satimage-software, www.satimage-software.com
[2] OSA - Open Scripting Architecture
[3] TextWrangler - free Mac OS X editor, by Bare Bones Software,
http://www.barebones.com
[4] BBEdit - commercial Mac OS X editor, by Bare Bones Software,
http://www.barebones.com
[5] Aquamacs - "an easy-to-use, Mac-style Emacs for Mac OS X", http://aquamacs.org/
balance.scm
#|
file: balance.scm
author: Jan Skibinski
version: 0.1
initialized: 2007-01-05
license: at the bottom of this file
code type: Scheme and some embedded Applescript
For description see the previous post:
Balancing expressions containing parentheses, brackets and braces.
|#
;; procedure: (balance-range k)
;; Given the integer k representing cursor position just
;; before any of the "open" characters; that is, one of
;; '(open-paren open-bracket open-brace);
;; within the input text view of Scheme-OSA application:
;; 1. Produce range of integers bracketing a balanced
;; portion of a text that follows the cursor, such as
;; (13 56), where
;; + the first item is a copy of insertion point
;; + the second item is the position past the matching
;; "close" character; that is, one of
;; '(close-paren close-bracket close-brace)
;; 2. Call Applescript to select the computed range,
;; by executing attached applescript code
;; 3. Wait for the response from Applescript and display
;; selected range in output text view to show those
;; numbers to the user.
;; Failure to balance some portion of 'input-string
;; is indicated by identical values of both items in
;; computed range.
;;
(define (balance-range k)
(define result (balance-stream
(current-input-port)))
(define range
(cond
((positive? result)
(list k (+ result k 1)))
(else (list k k))))
(define handler (string->symbol
(string-append
"on select_range from range\r"
" try\r"
" tell application \"Scheme-OSA\"\r"
" set inputTextView to text view 1 of scroll view 1 of split view 1 of window 1\r"
" call method \"setSelectedRange:\" of inputTextView with parameter range\r"
" end tell\r"
" return range\r"
" on error msg number n\r"
" return msg\r"
" end try\r"
"end select_range\r")))
(osa (make-osa-string
(list handler 'select_range 'from range))))
;; procedure: (balance-stream port)
;; balance-stream :: input-port -> integer
;;
;; Providing that a stream begins with one of
;; the following characters:
;; '(open-paren open-bracket open-brace)
;; find index of one of
;; '(close-paren close-bracket close-brace), which
;; balances the opening character.
;; Ignore matching within strings or within line
;; comments.
;; [To do: provide similar escape mechanism from
;; block comments as well]
;;
;; On success: character position of matching
;; closing character
;; On failure: -1
;;
;; Implementation notes:
;; The balancer is not greedy: it exits on the
;; first match and discards the remaining characters
;; waiting in pipe.
;; This procedure maintains internally two stacks:
;; state stack and balance stack and two co-procedures:
;; balance/1 and test which manipulate those stacks.
;; Testing for balancing takes place only when the top
;; of the state stack is 'normal; that is,
;; not 'commenting and not 'quoting.
;;
(define (balance-stream port)
; Define open/close characters, quote and semicolon:
; Defined indirectly via integer->char to avoid
; confusion when attempting to
; balance this source code in TextWrangler
; or other ascii editors.
; Seems that not everyone is always perfect,
;not even TextWrangler.
(define open-paren (integer->char 40))
(define close-paren (integer->char 41))
(define open-bracket (integer->char 91))
(define close-bracket (integer->char 93))
(define open-brace (integer->char 123))
(define close-brace (integer->char 125))
(define double-quote (integer->char 34))
(define semicolon (integer->char 59))
; True if character x is a marker of End OF Data
; Our pipe should never be broken, because this
; would force the interpreter to exit.
; Consequently, no EOF is ever sent during normal
; transmission of data. To signal the end of data,
; a special marker must be appended
; to the end of data stream and then watched
;for within this procedure.
(define (end-of-data? x)
(char=? x #\@))
; Read and discard all characters till
;the end of the stream.
(define (skip-to-end port)
(define c (read-char port))
(if (not (end-of-data? c))
(skip-to-end port)))
;True if character x-open is a mate of y-close
(define (mate? x-open y-close)
(or
(and (char=? x-open open-paren)
(char=? y-close close-paren))
(and (char=? x-open open-bracket)
(char=? y-close close-bracket))
(and (char=? x-open open-brace)
(char=? y-close close-brace))))
;True if character x is one of
;(open-paren open-bracket open-brace)
(define (open? x)
(or
(char=? x open-paren)
(char=? x open-bracket)
(char=? x open-brace)))
;True if character x is one of
; (close-paren close-bracket close-brace)
(define (close? x)
(or
(char=? x close-paren)
(char=? x close-bracket)
(char=? x close-brace)))
; Recursive co-procedure:
; (balance/1 port n balance-stack state-stack)
; Exit with failure if EOF.
; Call the 'test co-procedure if in 'normal state.
; Otherwise, first decide whether or not to pop up
; and dismiss the current state from the state stack,
;and then recurse.
(define (balance/1 port n balance-stack state-stack)
(define c (read-char port))
(cond
; End of stream:
; break signaling the failure
((end-of-data? c) (list #f n c))
; Normal state:
;call co-procedure 'test
((eq? (car state-stack) 'normal)
(test c port (+ n 1) balance-stack state-stack))
; We are in 'quoting state
;and first char is double-quote:
; recurse with popped state-stack
((and
(eq? (car state-stack) 'quoting)
(char=? c double-quote))
(balance/1 port (+ n 1) balance-stack (cdr state-stack)))
; Commenting state and the first char
;is newline or return:
; recurse with popped state-stack
((and (eq? (car state-stack) 'commenting)
(or
(char=? c #\newline)
(char=? c #\return)))
(balance/1 port (+ n 1) balance-stack (cdr state-stack)))
; Otherwise: recurse with both
;stacks unchanged
(else (balance/1 port (+ n 1) balance-stack state-stack))
))
;Recursive co-procedure:
'(test x port n balance-stack state-stack).
;Test current character x and behave accordingly
(define (test x port n balance-stack state-stack)
(cond
;X is open char: push x on balance stack
; and recurse
((open? x)
(balance/1 port n
(cons x balance-stack) state-stack))
;X is double-quote: push 'quoting on
;state stack and recurse
((char=? x double-quote)
(balance/1 port n balance-stack
(cons 'quoting state-stack)))
;X is semicolon: push 'commenting on
;state stack and recurse
((char=? x semicolon)
(balance/1 port n balance-stack
(cons 'commenting state-stack)))
;X is not close char: ignore x and recurse
((not (close? x))
(balance/1 port n balance-stack state-stack))
;State stack is empty: break signaling failure
((null? balance-stack) (list #f n x))
;X is close char but is not a mate
; of top of balance stack:
; break signaling failure
((not (mate? (car balance-stack) x)) (list #f n x))
;X is close char and balance stack
;has just 1 element:
; Hurrah! Break signaling success
((= (length balance-stack) 1) (list #t n x))
;Otherwise, x is a matching close char:
; Partial match, pop up balance
; stack and recurse
(else (balance/1 port n
(cdr balance-stack) state-stack))))
; Peek the first character
(define c (peek-char port))
(cond
; Empty string?: discard end-of-data
; marker and fail
((end-of-data? c) (begin (skip-to-end port) -1))
; String does not start with "open" char?:
; discard everything and fail
((not (open? c)) (begin (skip-to-end port) -1))
(else
; Try the real balancing
(let ((result (balance/1 port -1 '() (list 'normal))))
(begin
; Some characters waiting in pipe?
; Discard them.
(if (not (end-of-data? (caddr result)))
(skip-to-end port))
(if (car result) (cadr result) -1))))))
#|
Copyright (C) Jan Skibinski (2006). All Rights Reserved.
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|#
Friday, January 5, 2007
Balancing code in Scheme-OSA
Balancing Scheme expressions containing parentheses, brackets and braces.
A practical application of callbacks from Scheme to Applescript - as implemented in Scheme-OSA application
Callbacks
A Scheme callback, as defined for our purposes, is a means by which a Scheme interpreter asks the Scheme-OSA bridge to evaluate some piece of foreign code on its behalf. The mechanism involves marking the Scheme response as a 'callback, attaching a foreign code (directly or indirectly) to be evaluated by Applescript or indirectly by Cocoa, and putting itself in a wait mode in order to read possibly useful results from the foreign calculations.
The normal mode of the bridge operation is simple: user types some Scheme expression into the input text view of Scheme-OSA application, the expression is sent via input pipe to Scheme interpreter, the interpreter computes the results and sends them back via its output pipe to the bridge, and the bridge displays the results in the output text view.
The callback mode is more complicated, since there are extra round trips to complete the transaction, and some data marshalling - simple as it is - from Scheme to Applescript and back. For example, this Scheme expression:
(*
(ask-for-number "enter odd number" 3)
(ask-for-number "enter even number" 20))
uses a wrapper of an applescript code from our tiny GUI library to prompt a user for some numeric data. As usually, this code is initiated by a user on the bridge side, but once it gets to the Scheme interpreter it causes the latter to issue two callbacks. Both callbacks invoke GUI dialogs, a user enters requested numbers into text fields of dialog boxes (defaults: 3 and 20 are for lazy users, :-)), and both numbers are returned to Scheme. From then on everything looks like in the normal mode of operation: Scheme mutiplies the received numbers and returns the final result (default: 60) to the bridge - which then displays it in output text view.
[A side note: Practical implementation of the above mechanism is more complicated than that, due the fact that the Scheme-OSA interaction is asynchronous: it is sequential, but not synchronous. In other words, bridge does not wait for Scheme interpreter to complete its computations, but goes away letting other appplescript applications to do their jobs. The GUI thread of Scheme-OSA but will be notified by a special event when Scheme response is to be read from a buffer. In meantime a background thread collects all characters that Scheme sends through its output pipe -- until it senses that it is time to send a "time-to-consume" notification to the main thread.]
This script provides some Scheme utilities to be used by Scheme-OSA bridge to perform paren/bracket/brace balancing of Scheme code, which is either typed into, dragged to, or copied to the input text view from other editors.
Description of procedures
Below are two procedures:
1. (balance-stream port)
2. (balance-range k)
The first one is a pure Scheme procedure, which you can test in any environment and any input port, including Scheme-OSA.
Examples:
(define p (current-input-port))
(balance-stream p)()rubbish@ ==> 1
(balance-stream p)(string-append "abc" "efg")rubbish@ ==> 26
(balance-stream p)(string-append "ab[c" "efg")rubbish@ ==> 27
(balance-stream p)((string-append "ab[c" "efg")rubbish@ ==> -1
The second one, the wrapper around the first one, is designed for consumption by Scheme-OSA bridge, to set a new text selection that visualizes the user-activated paren/bracket/brace balancing, or lack of such.
It is used as follows:
1. User moves the insertion point just in front of any "open" character in input text view; that is, one of '(open-paren open-bracket open-brace) and then presses the option-command-B keys, or selects the "Balance" commmand from the Text menu in the menu bar.
2. Scheme-OSA bridge passes the command "(balance-range k1)" and a stream of characters following the insertion point k1 to Scheme interpreter.
3. Scheme interpreter computes a list '(k1 k2) indicating a range of characters that are balanced by a "close" character; that is, one of '(close-paren close-bracket close-brace) characters, which match the opening character. After marking it as a callback and attaching to it an Applescript code the range is sent back to bridge for evaluation.
4. The bridge recognizes it as callback, and passes it to AppleScript runtime for compilation and evaluation. The runtime colorizes the computed range as a new selection and the computed range is returned back to Scheme.
5. Scheme, having no further use for this result, just echoes it back to the bridge.
6. Bridge displays it - as any other Scheme response destined for the direct consumption (i.e., not a callback) - in the output text view.
Why calling Scheme for balance computation?
It would seem that the problem of paren/bracket/brace balancing could be easily solved via regular expressions mechanism. And since we have access to one such regex library for Applescript, "Satimage Osax"[1], we could handle the balancing at the Applescript level with the help of Satimage -- without a need for calling back to Scheme. After all, we use Satimage's regular expressions for text coloring.
Unfortunately, Satimage - as most other regex libraries - does not support recursive patterns in regular expressions.
Let us see what others have to say on a subject of a code balancing. The following is a quote from the TextWrangler[3] editor manual:
Without the use of recursion, the best that can be done is to use a pattern that matches up to some fixed depth of nesting. It is not possible to handle an arbitrary nesting depth. Perl 5.6 has provided an experimental facility that allows regular expressions to recurse (among other things). It does this by interpolating Perl code in the expression at run time, and the code can refer to the expression itself.
Obviously, Scheme-OSA cannot support direct interpolation of Perl code at the Applescript level. There are two choices however: one can escape to shell via "do shell script" and then call the Perl engine, or one can call Scheme to perform the recursive balancing via a proper tail-recursive procedure, such as the 'balance-range. Since Scheme is the integral part of Scheme-OSA, we have chosen the latter approach for obvious resons...
Perceived efficiency
No noticeable delays are felt when running this code on G4 laptop, even when the input stream (a Scheme source code following the insertion point in input text view) becomes huge.
Why combination of keys: option-command-B for "Balance" command?
Two of the best ASCII editors for Mac OS X, the commercial BBEdit[4] and its free cousin TextWrangler[3] use command-B as the key sequence for the invocation of the "Balance" command. However, since one of our goals is to provide IDE working in a rich text environment, not just the ASCII one, we need to preserve the command-B for invocation of the "Bold" action, typical in any rich-text editing environment. Consequently, we choose option-command-B for the "Balance" command. If you do not like this, you can easily remap some of the keys of the main menu of the Scheme-OSA application, including option-command-B.
Limitations
There are several limitation to the algorithms presented here.
1. The algorithms work only in the "forward" mode, which means that the insertion point must be positioned just before an "open" character. A good balancer should also work "backwards" - from the last "close" character to the first matching "open" character, and "from the middle", where insertion point is placed somewhere between an "open" and a "close" characters.
I'll work on it later.
2. Current implementation correctly ignores any open/close characters that appear within strings or within line comments. However, it does not ignore balancing within the Scheme block comments. Working on it...
3. A temporary End-Of-Data hack, described below
A temporary End-Of-Data hack
In a rush to get a prototype working, I have introduced #\@ character as a End-Of-Data marker. Some sort of a marker is needed since the pipe between Applescript and Scheme is always open. In normal circumstances the marker is not actually needed since we can use Scheme prompts to decide whether or not the data is complete. In the postfix i/o cases, such as this one, the data marker is needed, because we have no other means to signal End-Of-Data; the open pipe will never send End-Of-File marker.
But we have to keep the input pipe to Scheme open, otherwise a Scheme interpreter would exit prematurely, if we were to send "close-pipe" command after writing data to Scheme. Consequently, no EOF object is ever to be expected on the pipe. But using character #\@ as a EOD marker is not a good choice: after all #\@ is as legal as any other ASCII character: it can be used in strings, symbols or identifiers.
Possible other solutions are:
1. Marking EOD by a sequence of ASCII characters, rather than by a single one. Such algorithms are known, but efficiency would somewhat suffer due to look ahead and backtracking.
2. Using a higher order character with a code point x, such that 127 < x < 256. Not every Scheme implementation supports this. Petite Chez Scheme for example, although happy with Unicode and UTF8 when reading strings or symbols, will crash when individual characters are read or examined.
3. Redesigning read-char, so
(read-char (current-input-port))#\bel
would indeed recognize #\bel (or other such control character) as a single character, and not as # followed by \bel.
4. Making exception to this and other i/o procedures and using temporary files or named fifo pipes, rather than the one standard input pipe. I'd rather not to introduce any exceptions to this simple model.
5. Redesigning the signature of the "balance-range" function to accept explicit strings, rather than implicit streams. This means that we would have to prepare a string like this: "(balance-string \"quoted-text-of-input-text-view\" k1)".
This works, but onus is on Applescript to double-quote every string that might appear in the text of the input text view. It follows that the text
a. must be surrounded by "\"" substrings
b. must have every double-quote character preceded by \\.
It is the point b. that makes such algorithm very inefficient, since such quoted string must be assembled on character-by-character basis. Although acceptable for short pieces of code, the performance is not acceptable when balancing even the smallest pieces of code in typically-sized Scheme modules.
References:
[1] "Satimage Osax" by Satimage-software, www.satimage-software.com
[2] osax - (Open Scripting Architecture eXtension) is the nickname for the Scripting Additions', the plug-ins for AppleScript.
[3] TextWrangler - free Mac OS X editor, by Bare Bones Software, http://www.barebones.com
[4] BBEdit - commercial Mac OS X editor, by Bare Bones Software, http://www.barebones.com
Subscribe to:
Posts (Atom)