Visualizing graphs

[All pictures in this article are reduced by half to speed-up loading of this page. Click on a picture to see its normal size]

To quote the first paragraph from Graphviz site:Graph visualization is a way of representing structural information as diagrams of abstract graphs and networks. Automatic graph drawing has many important applications in software engineering, database and web design, networking, and in visual interfaces for many other domains.
Graph visualization is a serious and hard business, and subject of very active research. There are some tools that do reasonably good layout and rendering job. One is Graphviz, which is based on the language DOT, developed in AT&T after 10 years of intensive research. The Mac OS X edition of Graphviz, by Glen Low, shown on the left, won two 2004 Apple Design Awards.

In this post I will demonstrate how to interface to this tool from Scheme-OSA. There is one problem though: Glen's tool is not scriptable. Fortunately Apple provides for more than one method to handle inter-application communication. One such method is Apple's Services, which use global Pasteboard for this purpose.
The second problem is the input language. Simple as it is, DOT is yet another language to learn, which is probably not what most of us would cherish. To spare you such effort I spent one afternoon reading documentation and writing a small translator from Scheme to DOT syntax. Now you can think and write Scheme when specifying visualization of your graphs.

The picture on the left is a capture of the Scheme-OSA window. Its upper portion, the input text view, contains a very simple graph example, written in Scheme. The code declares only few nodes and few paths, but nodes are individually colored, and one of them uses its own special set of parameters.
Node is a pair made of a name (symbol) and a quasi-record. Quasi-record is a list of properties. Property is a pair of two objects, where the first object must be a symbol - a key of the property. A value of a property can be one of: boolean, number, symbol or string. Path is a list of at least three symbols, where the first one is the right arrow, and the rest are names of nodes to be followed in order to form edges. This in essence describes the structure of a diagram. The DOT manual provides many other details, such as names of colors, names of node shapes (rectangle, triangle, etc. - about 20 of them), and other parameters, which are immaterial to this demo.

The last line of the input text view is the invocation of a command to display the graph specification in DOT language. Ready? Press the Perform button and you should see the DOT output in the output text view. Now select that view by pressing command-A,
then go to the Scheme-OSA menu and choose Scheme-OSA > Services > Graphviz > Open as Graph. The resulting picture will be displayed in Graphviz, as shown on the left. As you can see, one node (the one that says "hello world") is disconnected from the rest. This is what should be expected since the specification of paths above never refers to the node xyz, which is the "hello world" node. Plain and simple, but it could be even simpler if the Graphviz was scriptable. Maybe one day it will be as such. Concluding this demonstration, I should add that other editions of Graphviz (Linux, Windows) are not that simply to use as this one. Glen's tool prints its output to many, many graphic formats, and the rendering is excellent: no jagged lines, as in Linux, for example.

Processing sound via workflows

In my previous post, Sound engineering, I shortly reviewed Sound Studio scriptable application, which I consider a good candidate for interfacing from Scheme-OSA. Here is another one, with similar goals, but with dramatically different look and feel:

Sample Manager, version 3.01, from Audiofile Engineering.
According to their description:

Sample Manager is the quintessential batch audio file processor for Mac OS X. Designed in Cocoa from the ground up, Sample Manager proudly takes advantage of CoreAudio, Quartz, and other solid OS X features [...]

Sample Manager is available now for a 15-day free trial. After the trial period, you'll need to purchase Sample Manager to continue using it. Sample Manager is available for only $90 from our online store [...]
For a limited time, Sample Manager 3 is available at the intro price of $70.

The essence of Sample Manager
You can read all the blurp you want on the Audiofile Engineering site, but the essence of this software is, in my opinion, this:
Sample Manager is very extendable due to its support for user defined Automator actions, or Applescript support, where the latter, more or less, parallels the former.
Menus of Sample Manager are quite minimalistic but that should not fool you - you can gain a lot of power via built-in workflow templates, and you can easily built your own workflows by dragging Automator Actions from the drawer, on the right of the main windows. User interface is flexible, since you can easily expand or reduce the number of panes and drawers visible at any given time. The picture below shows just one such arrangement - where Details, Workflows Drawer and Actions Drawers have been hidden from view.


















Comparison with Sound Studio
Compared to Sound Studio, reviewed in the previous post, GUI of Sample Manager seems a bit more complicated and it lacks few essential visual hints, which exist in the former. One example is "the fingerprint" (or overview) of entire sound channel. Sound Studio seems to have more traditional DSP power (Fourier spectrum and variety of filters) but Sample Manager seems to compensate on the file convertion/manipulation side.
I would be hard pressed to choose one over the other - especially since I have not used either one in a real project yet.

Comparison with my old software
I am a bit biased towards the Sample Manager since it reminds me my own software that I developed and commercially distributed long time ago. The domains are different (sound signals vs. power fault signals), macros are different (workflows vs. directed graphs), user interfaces are different (mine was not as pretty - considering dramatic changes that happened in meantime in GUI, graphics, processor speeds) but the concepts are generally similar.
I also started with a minimalistics browser - with a lot of manipulation and conversion abilities, but very little DSP power. But with time I added all those standard and non-standard transforms, convolutions, correlations, and some domains-specific algorithms. So if this may serve as any sign - I predict that Sample Manager will evolve into a very powerful product. :-)

Scriptability
From point of view of scriptability, both products seem to have similar functionality, and offer similar opportunities to Scheme-OSA.
The following is a very abbreviated digest of the functionality of Sample Manager. This is for the future reference for those willing to write Scheme wrappers around the commands listed below. You are excused, dear reader, from going down these lists with any attention whatsoever. Just skim them, please, to get some idea what this software is all about.

Main objects
There are four main objects in Sample Manager you can operate on:

• application n : The Sample Manager application


• folder entry n : An entry in the files table relating to a folder


• group entry n : An entry in the files table that contains other entries


• smart group entry n : An entry in the files table that is the result of a search and contains other entries

Application object has a very large number of properties, but I will not list them here, since they are not essential for this digest.

Commands
Commands, however, are important because they describe functionality of this software. For a sake of simplicity I ommitted all parameters from this digest, but the direct ones.

• add to list v : Adds the files or folders to the file list


• remove from list v : Removes the entries from the file list


• play v : Starts playback of the selected entries


• stop v : Stops playback


• set v : Sets the value of the entries


• change gain of v : changes the level of the audio


• normalize v : sets the level of the audio files to a given amplitude - normalize entry or list of entry


• balance v : balances the channels of an audio file to an average level


• fade in v : fades in the audio from silence


• fade out v : fades out the audio to silence


• trim beginning of v : removes audio from the beginning


• trim end of v : removes audio from the end


• trim around loop of v : remove audio around a loop


• shorten v : removes audio from the file to make it a specific length


• pad v : adds silence to the beginning and/or end of an audio file


• extend v : adds silence to the file to make it a specific length


• convert format of v : converts the audio file format


• convert rate of v : converts the audio file sample rate


• convert range of v : converts the audio file bit range (depth)


• convert channels of v : converts the audio file channel format


• make labels at divisions for v : creates labels spaced evenly throughout an audio file


• make labels at tempo for v : creates labels on beat positions in an audio file


• make labels at thresholds for v : creates labels at audio thresholds in a file


• make labels at thresholds for entry or list of entry


• remove labels in v : removes all labels of the given types in the audio file


• extract at thresholds from v : creates new files from the between the given thresholds


• extract at labels from v : creates new files from the between the given labels


• extract channels from v : creates new mono files from the channels in the audio file


• mix v : Combines the audio files by mixing the audio


• append v : Combines the audio files by joining one after the other


• change speed of v : changes the speed of the audio by a given factor


• change length of v : changes the length of the audio to a given value


• change tempo of v : changes the tempo of the audio to a given value


• change pitch of v : changes the pitch of the audio


• transpose v : transposes the audio by a given interval


• filter DC of v : Filters the DC offset from the audio files


• reverse v : Reverses the audio files


• rename v : Renames the audio files


• export as aac v : exports the audio files to aac format


• export as ogg vorbis v : exports the audio files to ogg vorbis format


• export as vox v : exports the audio files to vox format


• compact v : removes unnecessary data from an audio file


• compress v : Compresses the audio files into a zip archive


• send via ftp v : Sends the audio files to an FTP server


• send to my idisk v : Sends the audio files to My iDisk

Sound engineering

I have been busy last week exploring the world of notable scriptable applications for Mac OS X. This is, after all, the field targetted by Scheme-OSA. I found several ones worthy recommendation. In this post, I will shortly review Sound Studio 3, the essential tool for working with digital audio.

Programmed and copyrighted by Lucius Kwok, Felt Tip Software.
According to Lucius:

Sound Studio 3 is an easy-to-use sound editing application for recording and producing audio digitally on your Mac OS X computer. You can digitize records and tapes, record live events, create podcasts, tweak the audio in your iMovie or Final Cut project, and do many more things with Sound Studio.
Buy it, try it, cry it at Freeverse.

My first impression:
Commercial, price $80 but worth it, professionally done, simple and clean interface. Trial licence expires after 9 runs - plenty enough to explore.

Scriptability

Sound Studio is a scriptable application. It responds to following Applescript commands:
play, pause, unpause, stop, cut, copy, paste, clear, silence, crop, insert silence at, insert noise at, insert tone at, insert fm at, amplify, fade, normalize, compress dynamics, expand dynamics, noise gate, add noise, dc offset, interpolate, invert, swap channels, reverse, chorus, delay, flange, pitch, reverb, equalize, high pass, low pass.

From Scheme perspective: All those commands can be wrapped as Scheme procedures and used directly from Scheme-OSA in a batch mode for some repetetive tasks to be applied to a bunch of audio files.

Following is a digest of commands, as listed in Dictionary of Sound Studio. Details, such as arguments and optional parameters are omitted.

• play v : Play back audio.


• record v : Record audio at end of file.


• pause v : Pause playback or recording.


• unpause v : Unpause playback or recording.


• stop v : Stop playback or recording.


• resample v : Resample audio sample data.


• mixdown v : Mixdown audio tracks.


• cut v : Cut selection


• copy v : Copy selection


• paste v : Paste selection


• clear v : Delete audio selection


• silence v : Silence selection


• crop v : Crop selection


• split v : Split document by markers and save to files.


• insert silence at v : Insert silence at selection.


• insert tone at v : Insert tone at selection.


• insert fm at v : Insert FM Synthesis at selection.


• amplify v : Apply the Amplify/Volume filter to the audio. amplify


• normalize v : Apply the Normalize filter to the audio.


• compress dynamics v : Apply the Dynamics Compressor filter to the audio.


• expand dynamics v : Apply the Dynamics Expander filter to the audio.


• add noise v : Apply the Add Noise filter to the audio.


• interpolate v : Apply the Interpolate filter to the audio; replaces audio with a straight line between the first and last samples.


• invert v : Apply the Invert Signal Polarity filter to the audio; flips audio vertically.


• swap channels v : Apply the Swap Left and Right Channels filter to the audio; swaps audio data between even and odd numbered tracks.


• reverse v : Apply the Backwards/Reverse Audio filter to the audio; makes audio sound like it is played backwards.


• chorus v : Apply the Chorus filter to the audio.


• delay v : Apply the Delay and Echo filter to the audio.


• flange v : Apply the Flanger filter to the audio.


• pitch v : Apply the Pitch and Tempo filter to the audio.


• pitch specifier : the item(s) to filter


• equalize v : Apply the Graphic Equalizer filter to the audio.


• high pass v : Apply the High Pass filter to the audio.


• low pass v : Apply the Low Pass filter to the audio.


• expand tilde v : Expand tilde in POSIX path to full absolute path.

This is an impressive set of useful commands, that - I am certain - some user of Scheme-OSA (or myself) will nicely wrap in a practical Scheme module.

How about some music?

In this post I will continue exploration of multimedia with Scheme-OSA. So far we have learned how to invoke all sorts of dialog boxes and choices, how to plot in two and three dimensions, how to translate text to speech and how to play short sounds. Actually, I use these sounds for audio feedback as a help in debugging and diagnostics of Scheme-OSA. For example, the "Submarine" sound tells me that the application just entered into the callback mode, while the "Frog" sound informs me that the interpreter has been just loaded and is running, etc. A simple beep would not do the job.

So how about some real music? A fair question would be: Why on Earth would you need scripting if you have the iTunes browser, which is such an excellent interactive tool? Apparently there are some reasons for scripting iTunes. Take a look at Doug's Applescripts for iTunes. There are 419 scripts, organized in 17 or so categories: Managing Tracks, Managing Track Info, Managing Artwork, Managing Playlists, Controlling iTunes, Exporting Info, Managing Files, Networking, Internet, iPod .. - to name the few.

Application iTunes is obviously scriptable. It is also well designed, well described, and most of its functionality is model-based, rather than view-based, which means that it is easy to use. But iTunes is not the only game in town. If you need something five times smaller than iTunes, and which also runs on background you might consider Play Sound by Microcosm Software. The author of Play Sound claims that

Play Sound is a simple, no-fuss sound player for Mac OS X. It supports QuickTime sound files (for example, AIFF or MP3 files), Classic Mac OS System 7 sound files (files with a file type of 'sfil' that contain 'snd ' resources), or any 'snd ' resources embedded into any file. You can play an unlimited number of sounds concurrently or one at a time. You can loop sounds, repeat sounds, play specified portions of sounds, control the volume of sounds played, pause and continue sound play, and press the Escape or Command-Period keys to abort sound play.

This excerpt from possibly much larger library of music wrappers demonstrates how to control music from Scheme-OSA: how to play a single sound file, how to play a list of sound files, how to choose a bunch of files interactively and then play them, and how to stop playing them.
Sorry that I cannot demonstrate it in real time from this blog, but believe me - it works!

Scalar, Contour and Surface maps

Here are some more examples of plots indirectly produced by Scheme-OSA. The direct, real producer of these pictures is Smile by Satimage Software. From Scheme-OSA perspective Smile is just an intelligent input/output device, capable of doing some outstanding visualization work. Average user of Scheme-OSA does not need to know anything about Smile - with the exception of the fact that it is a Mac OS X application and that it has been installed on the system. Few basic Scheme wrappers around Smile libraries should suffice for most of his/her needs. However, Smile also provides an excellent documentation, and this is what users should consult before writing their own wrappers.

This picture is a combination of two maps: Scalar Map and Contour Map. Three lines of Applescript code are needed to produce the picture itself. Few more lines are needed to add title and labels. And few more lines are required to wrap it up in a Scheme code.
Although this map has polar symmetry it has been generated using rectangular coordinates. User had requested 12 equidistant contours to be overlayed on a top of scalar map.


This is just a scalar map; no contours are overlayed on a top of it. Notice that this picture was generated using polar coordinates - hence it looks clipped ouside the circle.














And this how how the previous map looks in three dimensions.

Plotting formulas with Smile

Smile is a scriptable visualization application, which is ideally suited for demonstration of some of the capabilities of Scheme-OSA. Smile is described at the bottom of this page. Shown below is a capture of Scheme-OSA window with a very simple example of formula plotting, which takes advantage of Smile capability of plotting.
Note that we are using here the interpreter MzScheme. Although bare bones MzScheme does not have its own plotting capability, but DrScheme - which uses MzScheme - does. Some could therefore argue that Scheme-OSA and its interface to Smile is redundant in such setting. Fair enough, but consider this: as long as you are on Mac OS X platform you can use the same code, irrespectively of interpreter of your choice: MzScheme, Petite Chez Scheme, Bigloo, Gambit, Gauche, Chicken, etc. And no matter what interpreter you are using the pictures will be always the same, as presented below.



We constrained ourselves to one specific set of visualization parameters for this picture. Smile application let you change any of those - before or after your data has been plotted. Frame size and type; size and thickness of ticks; grid size, pattern and color; location of curve labels; thickness, color and pattern of the curves; or type of curves themselves - these are the parameters changeable at ease.




Multi-plot Scheme wrapper

As said before, for a simplicity sake we using here one selected choice of parameters for data presentation.

;; Plot a list of formulas for a given enumerator and descriptor, where
;; + formulas is a list of records: (('recipe atring) ('name string))
;; + enumerator is a list describing a range of abscissas: (first second last)
;; + descriptor is a list of strings: (title xlabel ylabel).
(define (multi-plot formulas enumerator descriptor)
   (define handler (string->symbol (string-append
      "on multi_plot of recipes for abscissas ";; no line break
      "       given descriptor:ds\r"
      "   tell application \"Smile\"\r"
      "     set v to \"\"\r"
      "     repeat with i in recipes\r"
      "        set y to evalformula (recipe of i) with {x:abscissas}\r"
      "        set c to QuickCurve(abscissas, y, v)\r"
      "        set name of c to (name of i)\r"
      "        set v to container of c\r"
      "     end repeat\r"
      "     set properties of v to ";;no line break
      "         {name:(item 1 of ds), legend abscissa:8, ";;no line break
      "         legend kind:2, xlabel:(item 2 of ds), ylabel:(item 3 of ds)}\r"
      "  draw window of v\r"
      " end tell\r"
      "end multi_plot\r")))
   (define abscissas 
      (enumerate-until (car enumerator) (cadr enumerator) (caddr enumerator)))
   (osa (make-osa-string 
      (list handler 'multi_plot 'of formulas 'for abscissas 
         'given 'descriptor: descriptor ))))

Application Smile

Smile is an Applescript-based sophisticated software from Satimage Software. It comes in three versions:
1. Regular Smile - free version, with GUI builder, sophisticated 2D drawing engine, and many other goodies
2. Full Smile = Regular Smile + SmileLab, with 2D and 3D visualization engine. Commercial
3. Smile Server - commercial

Additionally, Satimage distributes free OSA extensions ("osaxen"): numerics.osax and satimage.osax, with many useful extensions to standard Applescript.

Smile is a scriptable application. This means that it can be talked to via OSA framework, as we do it from our Scheme-OSA. There are two types of scriptable applications: the sophisticated ones are model-based, less sophisticated are view-based. The latter force the user to explore how the application is assembled: what sort of views it has, how the buttons are named or indexed, etc.
Although with some patience the user can deduce the structure of the objects forming the view-based scriptable applications , they are not as adaptable and easy to use as model-based scriptable applications. Ironic as it might sound - most Applescript Studio applications are view-based. Many Cocoa applications are model-based. Scheme-OSA is still view-based, but I hope to convert it to a model-based scriptable application. Smile is model-based, even though most of its code is Applescript. That shows how sophisticated Smile is.

User Interaction Suite

At the bottom of this post is a collection of Scheme wrappers for selected handlers from User Interaction Suite of StandardAdditions OSAX[1].

Contents of User Interaction Suite
beep v : Beep 1 or more times
choose application v : Choose an application on this machine or the network
choose color v : Choose a color
choose file v : Choose a file on a disk or server
choose file name v : Get a new file reference from the user, without creating the file
choose folder v : Choose a folder on a disk or server
choose from list v : Choose one or more items from a list
choose remote application v: Choose a running application on a remote machine or on this machine
choose URL v : Choose a service on the Internet
delay v : Pause for a fixed amount of time
display alert v : Display an alert
display dialog v : Display a dialog box, optionally requesting user input
say v : Speak the given text

Usage

The wrappers isolate Scheme user from the details of bridging between Scheme and Applescript. The user just sees Scheme expressions like this:

  
    (display-dialog "Your name?" "Peter brown" '("Cancel" "OK") "OK" 12)

which - when executed in Scheme-OSA environment - produces a dialog panel asking for a user name. It is provided with default answer "Peter Brown", two buttons: "Cancel" and "OK", default button "OK" and a timeout of 12 seconds.

The dialog returns the answer in the form of associative list (a simulated record, before real Scheme records become standard), such as:

 
    ((bhit "OK") (ttxt "John Brown") (gavu #f))

which means a button returned: (bhit "OK"), a text returned: (ttxt "John Brown") and boolean indicating whether or not the dialog gave up waiting for user response: (gavu #f).

The symbols 'bhit, 'ttxt and 'gavu have been directly translated from Applescript; I have made no attempt to make them more user friendly. Fortunately, there are not that many such criptic names. Apple insists on four-character abbreviations for public symbols, but anything that you make for your own use stays non-abbreviated. Hence, your private records, such as

    '((age 19) (factors '(3 5 7)) (speed 120))

will preserve its readability when crossing the border between Scheme and Applescript.

Under the hood

Underneath this presentation layer are translation services: from Scheme to Applescript and back to Scheme.

Firstly, a Scheme expression, such as this one:

 
    (display-dialog "Your name?" "Peter Brown" '("Cancel" "OK") "OK" 12)

is send to Scheme interpreter for evaluation. The interpreter converts it to the following Applescript command

  
display dialog "Your name?" default answer "Peter Brown" ¬
    buttons {"Cancel", "OK"} default button "OK" giving up after 12

then wraps it into string "(osa )", prints it out to standard output and enters into port reading mode - waiting for an answer from the Applescript. After reception of the osa-string the bridge recognizes the header "(osa " as a signal not to print it out but to submit the tail of such string for compilation and execution as an Applescript command. Consequently, the dialog is displayed, data collected from the user and the response is prepared in the form of a record with several properties, as in:

  
    {bhit: "OK", ttxt: "John Green", gavu: false}.

The bridge intercepts it, translates it back to Scheme as a alist:

  
    ((bhit "OK") (ttxt "John Green") (gavu #f)) 

and returns it to Scheme interpreter which was waiting for such answer.The results are then used for further computations, or just sent back to the bridge for display.


References

[1] Full documentation of StandardAdditions is provided in its dictionary of definitions. The dictionary is conveniently browsable using dictionary browser, which can be opened on StandardAdditions either by double clicking on /System/Library/ScriptingAdditions/StandardAdditions.osax or by invoking Script Editor application, opening a list of scriptable applications and additions from its menu File -> Open Dictionary ... and finally selecting the Standard Additions from that list.

[2] 'AppleScript Scripting Additions Guide: English Dialect', a document from Apple Computers


Scheme wrappers

;; procedure: (display-dialog title answer button-list default-button timeout)
;; return: aList
;; display-dialog:: String -> String-> List of String -> String -> Integer-> Alist
;;
;; Display a titled dialog window with title, default answer, at most three buttons,
;; one of which is declared a default and a timeout. Return a record with three
;; properties:
;; ((bhit name-of-a-button-pressed)
;; (ttxt string-from-the-text-field) 
;; (gavu true-if-dialog-gaved-up-due-to-timeout))
;;
;; Example:
;; (display-dialog "Your name?" "Peter Brown" '("Cancel" "OK") "OK" 0)
;; ==> ((bhit "OK") (ttxt "Juan de Silva") (gavu #f))
(define (display-dialog text 
   default-answer buttons default-button timeout) 
   (define str (make-osa-string
      (list 'display 'dialog text 
      'default 'answer default-answer 
      'buttons buttons 
      'default 'button default-button 
      'giving 'up 'after timeout)))
   (cond
      ((not (member default-button buttons)) 
         (error 'display-dialog 
      "Default button should belong to provided list of buttons"))
      (else (osa str))))


;; procedure: (display-message text buttons)
;; Produce a message with several buttons
;; (display-message "Do you want to proceed?" '("No" "Yes"))
(define (display-message text buttons) 
   (define str (make-osa-string
      (list 'display 'dialog text  
      'buttons buttons 
      )))
   (osa str))


;; procedure: (display-alert text)
;; Display an alert
;; Return: ((bhit "OK"))
;; Example:
;; (display-alert "Beware of dog!")
(define (display-alert text)
   (osa (make-osa-string (list 'display 'alert text))))

  
;; procedure: (ask-for-number title default)
;; Ask for a number, providing dialog title and default numerical value
;; Return a number
;; Example:
;; (* (ask-for-number "first number" 10) (ask-for-number "second number" 20))
;; ==> 200
(define (ask-for-number title default)
   (string->number
      (cadr (assq 'ttxt
         (display-dialog 
            title
            (number->string default)
            '("OK") "OK" 0)))))


;; procedure: (choose-from-list aList prompt) 
;; Choose one or more items from a list
;; Example:
;; (choose-from-list '("Eva" "Bob" "John") "Select a winner")
(define (choose-from-list aList prompt)
   (osa  (make-osa-string (list 
   'choose 'from 'list aList 
   'with 'prompt prompt 
   'with 'multiple 'selections 'allowed))))


;; procedure: (choose-files directory-path)
;; Choose one or more files (but not directories) from a specified directory path.
;; Return list of paths.
;; Example:
;; (choose-files "/Users/jans")
(define (choose-files directory-path)
   (define handler (string->symbol (string-append
   "on choose_files from directory\r"
   "   set dirAlias to POSIX file directory\r"
   "   set xs to choose file default location alias dirAlias"; no line break!
   "      with multiple selections allowed\r"
   "   set ys to {}\r"
   "   repeat with x in xs\r"
   "      set ys to ys & (POSIX path of x)\r"
   "   end repeat\r"
   "   return ys\r"
   "end choose_files\r"))) 
   (osa (make-osa-string 
      (list handler 'choose_files 'from directory-path))))


;; procedure: (choose-folders root-folder)
;; Choose one or more folders from a specified root-folder
;; Return a list of folders.
;; (choose-folders "/Users/jans")
(define (choose-folders root-folder)
   (define handler (string->symbol (string-append
   "on choose_folders from root_folder\r"
   "   set dirAlias to POSIX file root_folder\r"
   "   set xs to choose folder default location alias dirAlias"; no line break!
   "      with multiple selections allowed\r"
   "   set ys to {}\r"
   "   repeat with x in xs\r"
   "      set ys to ys & (POSIX path of x)\r"
   "   end repeat\r"
   "   return ys\r"
   "end choose_folders\r")))
   (osa (make-osa-string
      (list handler 'choose_folders 'from root-folder))))


;; procedure: (choose-color default-rgb-list)
;; Choose a color from the color chooser
;; Example:
;; (choose-color '(0 65535 0))
(define (choose-color default-rgb-list)
   (osa (make-osa-string (list
      'choose 'color 'default 'color default-rgb-list))))


;; procedure: (beep n)
;; Beep 1 or more times
;; Return: undefined
(define (beep n)
   (define str (make-osa-string (list 'beep n)))
   (osa str))

   
;; procedure: (say text)
;; Speak the given text
;; Return: undefined
(define (say text)
   (osa (make-osa-string (list 'say text))))



;; procedure: (play-standard-sound sound-name volume)
;;
;; Play a named sound at some volume level (0-7)
;; Avaulable sounds are taken (from /System/Library/Sounds/):
;; "Basso", "Blow", "Bottle", "Frog", "Funk", "Glass", "Hero", "Morse", "Ping", 
;; "Pop", "Purr", "Sosumi", "Submarine", "Tink"
;;
;; Example:
;; (play-standard-sound "Frog" 7)
(define (play-standard-sound sound-name volume)
   (define handler (string->symbol (string-append
   "on play_standard_sound for soundName at someVolume\r"
   "   try\r"
   "      tell application \"Scheme-OSA\"\r"
   "         set volume someVolume\r"
   "         set theSound to load sound soundName\r"
   "         play theSound\r"
   "         delete theSound\r"
   "      end tell\r"
   "   on error msg number n\r"
   "       error soundName & \" is not a standard sound\" number n\r"
   "   end try\r"
   "end play_standard_sound\r")))
   (osa (make-osa-string
      (list handler 'play_standard_sound 'for sound-name 'at volume))))

Building Libraries

A picture above is a snapshot of Apple's Script Editor: at the input text view there is a definition of the handler choose_folders and below it is a "just evaluated" expression. The result is shown below, in the result text view, as a list of three chosen directories. All of this is of course written in Applescript.

This code is a wrapper around the choose folder handler, defined in User Interaction Suite of the Standard Additions OSAX, which enriches standard Applescript with several suites of commonly used handlers.

The formal description of the choose folder is shown below:

choose folder v : Choose a folder on a disk or server

    choose folder
        [with prompt string] : 
            the prompt to be displayed in the dialog box
        [default location alias] : 
            the default folder location
        [invisibles boolean] : 
            Show invisible files and folders? (default is false)
        [multiple selections allowed boolean] : 
            Allow multiple items to be selected? (default is false)
        [showing package contents boolean] : 
           Show the contents of packages? 
           (Packages will be treated as folders. 
             Default is false.)
        → alias : the chosen folder

The items in square brackets are optional. The one which says [default location alias], means that user can specify a default location, on which a browser of folders will be open. The world alias is a fancy name for existing file or directory, immutable with respect to its repositioning into other location of your directory structure. Once you make an alias and move it somewhere else the Applescript will find it for you anyway. But what is really important here is the format of this alias; it must be something like this: alias "ibook:Users:jans".

But what if you wish to use the standard POSIX directory paths instead? Well you would have to specify it as: alias POSIX file "/Users/jans" - with or without the terminating forward slash. The wrapper handler choose_folders customizes the original handler: it will accept "root folder" in POSIX format, it will allow to select more than one folder, and it will represent the result as a list of chosen folders in POSIX format. Other options are simply ignored.






























This window, in turn, is Scheme-OSA window. There is an equivalent definition of the procedure choose-folders, this time of course written in legal Scheme.

Below it is "just evaluated" Scheme expression (choose-folders "/Users/jans"). The result is displayed in output text view as a Scheme list of three directories in POSIX format.

Notice that the reddish-brownish text on the left is almost the exact replica of the handler in top window. Well, not exactly. We need to preserve the formal syntax of Applescript, which does not like if a single command line is broken in half. It also likes each command on a separate line. And it prefers return rather than newline line terminators. Accordingly we assemble all of this from a list of strings, and then combine everything together with a subroutine invocation. At the end of this procedure is a little magic, which will announce it to Scheme-OSA bridge as a callback, rather than normal Scheme procedure.

Conclusions

So this is how we build new Scheme procedures, which are destined to talk to Applescript world.
1. We first read appropriate documentation.
2. We then start experimenting with Applescript using Script Editor, or other such tools, writing some Applescript code.
3. Once we are happy with the result we port such code to Scheme.

Those who know nothing about Applescript do not need to despair. For starters, I am providing a small library of useful procedures from User Interaction Suite of Standard Additions. You can go quite far with just this library.

Secondly, Applescript is not that difficult to learn, so you can start writing your own Scheme wrappers in few days.

Introducing Scheme-OSA

The prototype

The picture above is a snapshot of window of working Scheme-OSA. Going from left to right along the window toolbar we can see that:
1. "Interpreter" has been installed (because of the green dot in the on/off button);
2. The name of interpreter is "Petite Chez ...";
3. The interpreter is ready to "Perform", that is, evaluate next command;
4. The "Break" button is disabled because there is nothing to break from yet;
5. The "Decorate" button is (always) enabled. In fact we already used it to colorize the Scheme code in the input text view: comments are green, strings are brownish and keywords are blue.

After pressing the "Perform" button (or command-return keys), Scheme-OSA will initialize evaluation of any selection in the input text view (none selected here), or of all the code in this view.

Consequently, a friendly voice, which you can pre-select from a list of several voices, will ask you to enter your name, then a dialog box will pop up asking you to enter your name (default name "Peter Fonda"). The box will be discharged after pressing "OK", and the voice will ask you again whether your real name is indeed "Peter Fonda", or whatever name you entered. If you happened to press "Cancel", rather than "OK", the Scheme error will be generated instead.

Any time the evaluation takes place the red "Break" button becomes enabled, giving you a chance to cancel longish loops of other such events. This button plays the role of Control-C in Unix terminal.

The goals

Scheme-OSA is conceived as Interactive Development Environment for Scheme on Mac OS X platform. It is also a framework, which will hopefully deliver access from Scheme to many facilities and services of that platform, and to many specialized scriptable applications. Because it piggy-backs on Applescript, it should be able to do what regular applescripts do. Play music, talk, create 2D and 3D scientific plots, animate OpenGL goodies, exchange data with professional editors, spreadsheets, etc. -- short of dancing and cooking...

In other words, it should be able to behave as regular Scheme interpreter - but with all those additional goodies available to Applescript. It is still under development, hence many of those promisses seem too far fetched. Some however are already working, other features will, hopefully, appear in the near future.

Take a note: this is just a prototype, and I am not going to distribute any code as yet - with the exception of some Scheme modules to anyone interested, or the Xcode bundles to those who wish to help with reviews, development, testing, documentation or porting Scheme-OSA to the dual universal platform.

The non-goals

I do not plan to build yet another programming editor. Instead I plan to take advantage of the best existing editors in Mac OS X world. Let the others do your job if they can do it better than you can.

Let me explain. More than a year ago I wrote my first predecessor of Scheme-OSA. It did not rely on Applescript, but it provided two services, which one might call "glorified copy" and "glorified paste". It worked as follows: after invocation of that application I would load an interpreter of my choice (usually Petite Chez Scheme or MzScheme), then hide the application away - doing all my work in a programmer's editor of my choice, which happens to be TextWrangler[3].

After selecting a portion of a Scheme code, or entire module, or Scheme's load command, I will copy it directly to running Scheme interpreter via command-9 (visual association with left parenthesis) for evaluation. If I were interested in any Scheme response I would wait a bit then hit command-0 (visual association with right perenthesis), which would copy Scheme response right below the original selection.

That was good enough for me with respect to interaction with a Scheme interpreter. Any other perks that Emacs users love to have are provided by TextWrangler: text coloring, code balancing, extensive search/grep/regex facilities, diffs, buffers vs. files, text formatting, built-in shell scripts, etc. In addition, TextWrangler has fine support for AppleScript, which you can take advantage of to significantly extent its functionalities.

Although I incorportated some of the standard features of programmers' editor to Scheme-OSA, such as text coloring and code balancing, they are not there to completely eliminate a good text editor. I still envision myself doing most of my editing work in TextWrangler. But when I switch to Scheme-OSA I want to have some of the good perks too.

A year has passed and I suddenly realized that I can build a much more powerful tool than my old (still in use) prototype. I have never appreciated Applescript - thinking in terms: "Oh, this must be one of those Mickey Mouse languages". I was wrong - it is a very powerful means for taking advantage of the best applications that Mac OS X can offer today.


The implementation

OSA stands for Open Scripting Architecture, which provides framework for one of the forms of inter-application communication on Mac OS X platform and the set of libraries open to anyone wishing to plug her programming language of choice to OSA. The primary OSA language is Apple's AppleScript. There are few other languages that are already OSA-compliant: Python for one, Perl in some form, JavaScript under development.

Scheme-OSA does not plug directly to OSA libraries, but indirectly by riding on the back of AppleScript. I am not sure whether this is the best way to do, but this approach offers several advantages.

1. Short time to delivery. I did not have to go through typical hoops: marshalling of primitives, objects, pointers, then binding to OSA functions and all of that stuff. Instead few very simple conversions were required - more or less the same stuff that is required when implementing XML-RPC.
2. Safety. Communication with Applescript is easy: there are no pointer-related issues, dangerous crashes, and so on. Worst what can happen is its occasional refusal "I do not understand your message" - as in Smalltalk.
3. Ease of development. I can use Applescript to script Scheme-OSA any time either under control of Xcode or outside it. Scripting the product helps in evaluating approaches, trying "what if scenarios", testing, and so on.

Scheme-OSA is being built as "Applescript Studio" product, with significant support from Scheme itself. Most of the code is Applescript, 10-20% of the code is Cocoa (Objective-C). The only third party library is Satimage[1], a freely available product, which I use for regexp oriented tasks.

No apologies for ignoring Emacs

I should have loved Emacs, because I used to love Smalltalk, which - similarly to Emacs - tends to behave as an operating system on its own. But I have few reservation with respect to Emacs. (OOPS, I am feeling like loosing few potential friends right now:-)). Obsolete user interface and bloated code aside, the real problem is my own memory: I dislike memorizing all those triple key combinations. This was also a reson I never liked Vi editor (OOPS, another bunch of sympathizers is gone! :-)). I know some easy to remember key sequences: Command-C and Command-V for copy and paste, Command-B for bold, Command-I for italic, but I will be certainly lost in the maze of Emacs shortcuts.

I was happy to discover Aquamacs[5] a year ago. As they claim on their web page:

We support the standard Mac user interface that you've come to love. For instance, in addition to traditional Emacs shortcuts like C-x C-f (open a new file), Aquamacs understands Apple-O. Aquamacs behaves like a modern application on Mac (or Windows) when it comes to selecting, copying, pasting texts within Aquamacs or in between applications. Aquamacs offers nice, smooth fonts. Asian input methods work. It's easy to install and runs out-of-the box with no configuration. And all is built on GNU Emacs, so you can use your favorite Emacs packages!

I did spend some time doing some extra customization of Aquamacs, but once I noticed that it becomes more and more obese with each new release, I decided that this tool was not for me. With all due respect, I am really happy with TextWrangler - as it comes to text editing.


No apologies for choosing Mac OS X

Mac OS X is my platform of choice of the last two years. Mostly as a programmer but also as a user - I crawled on DOS in Basic, C and MASM; managed to program in C on Unix graphic workstations; worked with passion in several self-contained, host-dependent and host-independent Smalltalks; did some angry programming in MS-Windows; then discovered Linux and stayed with it for at least 10 good years - programming in Eiffel, Java, C, Objective-C, Smalltalk and a bunch of other esoteric languages, Haskell including.

I like the Linux's bazaar approach to sharing the knowledge and building software for the common goals. I learned a lot from Linux community, and I also contributed back a bit. But I dislike randomness in delivered goods: although there are plenty of high quality software packages in the Linux world, there are also many poorly documented and buggy utilities and applications. I have been growing impatient and started valuing my time; I have become tired of fixing somobedy else's bugs and going through countless hours of installations, upgrades, and often fruitless evaluations of third party packages.

So when the NextStep for Intel became available I jumped off the Linux ship for a while; I really liked what I saw and experienced: dazzling user interface - reminescent of Smalltalk; excellent inter-process communication facilities, such as Services; very good documentation, very good software development tools, Unix and Mach kernel under the hood, bug-free applications and friendly community. But there was one big problem: the Company and the Community were too small to keep up with either Microsoft or Linux. There were too few drivers available and because of it installations or upgrades of NextStep OS were tough - I once spent good three days swapping cards until I finally succeded in getting my machine running the NextStep. Software-wise, important applications, such as the web browser, were also lagging behind Netscape or Explorer.

So, for all the goodies I saw in the NextStep OS, I realized that there was not much future in that platform. With all regrets, I moved back to Linux. Sadly, I was right - NextStep went down. Happily, it got reincarnated as Mac OS X. Two years ago I discovered the old friend, and I have not been looking back ever since. I am not going to gloss over advantages of this platform: suffices to say that it suits me well, and I am not going to be apologetic about it.


References:
[1] "Satimage Osax"[2] by Satimage-software, www.satimage-software.com
[2] OSA - Open Scripting Architecture
[3] TextWrangler - free Mac OS X editor, by Bare Bones Software,
http://www.barebones.com
[4] BBEdit - commercial Mac OS X editor, by Bare Bones Software,
http://www.barebones.com
[5] Aquamacs - "an easy-to-use, Mac-style Emacs for Mac OS X", http://aquamacs.org/

balance.scm

#|

file:            balance.scm
author:       Jan Skibinski
version:      0.1
initialized:   2007-01-05
license:       at the bottom of this file
code type:   Scheme and some embedded Applescript

 For description see the previous post: 
Balancing expressions containing parentheses, brackets and braces.
 |#
;; procedure: (balance-range k)
;; Given the integer k representing cursor position just
;; before any of the "open" characters; that is, one of 
;; '(open-paren open-bracket open-brace);
;; within the input text view of Scheme-OSA application:
;; 1. Produce range of integers bracketing a balanced 
;;    portion of a text that follows the cursor, such as 
;;    (13 56), where 
;; + the first item is a copy of insertion point
;; + the second item is the position past the matching 
;;   "close" character; that is, one of
;;   '(close-paren close-bracket close-brace)
;; 2. Call Applescript to select the computed range, 
;;    by executing attached applescript code
;; 3. Wait for the response from Applescript and display
;;    selected range in output text view to show those 
;;    numbers to the user.
;; Failure to balance some portion of 'input-string 
;; is indicated by identical values of both items in 
;; computed range.
;;
(define (balance-range k)
   (define result (balance-stream 
       (current-input-port)))
   (define range
      (cond
         ((positive? result) 
            (list k (+ result k 1)))
         (else (list k k))))
   (define handler (string->symbol 
       (string-append
      "on select_range from range\r"
      "   try\r"
      "      tell application \"Scheme-OSA\"\r"
      "         set inputTextView to text view 1 of scroll view 1 of split view 1 of window 1\r"
      "         call method \"setSelectedRange:\" of inputTextView with parameter range\r"
      "      end tell\r"
      "      return range\r"
      "   on error msg number n\r"
      "      return msg\r"
      "   end try\r"
      "end select_range\r")))
   (osa (make-osa-string 
         (list handler 'select_range 'from range))))
         
;; procedure: (balance-stream port)
;; balance-stream :: input-port -> integer
;;
;; Providing that a stream begins with one of
;; the following characters:
;;    '(open-paren open-bracket open-brace)
;; find index of one of 
;; '(close-paren close-bracket close-brace), which
;; balances the opening character. 
;; Ignore matching within strings or within line 
;; comments. 
;; [To do: provide similar escape mechanism from 
;; block comments as well]
;;
;; On success: character position of matching 
;; closing character
;; On failure: -1
;;
;; Implementation notes:
;; The balancer is not greedy: it exits on the 
;; first match and discards the remaining characters 
;; waiting in pipe.
;; This procedure maintains internally two stacks: 
;; state stack and balance stack and two co-procedures: 
;; balance/1 and test which manipulate those stacks. 
;; Testing for balancing takes place only when the top 
;; of the state stack is 'normal; that is,
;; not 'commenting and not 'quoting.
;;
(define (balance-stream port) 
   ; Define open/close characters, quote and semicolon:
   ; Defined indirectly via integer->char to avoid 
   ; confusion when attempting to
   ; balance this source code in TextWrangler 
   ; or other ascii editors.
   ; Seems that not everyone is always perfect, 
   ;not even TextWrangler.
   (define open-paren    (integer->char 40))
   (define close-paren   (integer->char 41))
   (define open-bracket  (integer->char 91))
   (define close-bracket (integer->char 93))
   (define open-brace    (integer->char 123))
   (define close-brace   (integer->char 125))
   (define double-quote  (integer->char 34))
   (define semicolon     (integer->char 59))
   
   ; True if character x is a marker of End OF Data
   ; Our pipe should never be broken, because this 
   ; would force the interpreter to exit.
   ; Consequently, no EOF is ever sent during normal 
   ; transmission of data. To signal the end of data, 
   ; a special marker must be appended
   ; to the end of data stream and then watched 
   ;for within this procedure.
   (define (end-of-data? x)
      (char=? x #\@))
      
   ; Read and discard all characters till 
   ;the end of the stream.
   (define (skip-to-end port)
      (define c (read-char port))
      (if (not (end-of-data? c)) 
         (skip-to-end port)))
      
   ;True if character x-open is a mate of y-close
   (define (mate? x-open y-close)
      (or
         (and (char=? x-open open-paren)   
                 (char=? y-close close-paren))
         (and (char=? x-open open-bracket) 
                 (char=? y-close close-bracket))
         (and (char=? x-open open-brace)   
                 (char=? y-close close-brace))))
   
   ;True if character x is one of 
   ;(open-paren open-bracket open-brace)
   (define (open? x)
      (or 
          (char=? x open-paren) 
          (char=? x open-bracket) 
          (char=? x open-brace)))
   
   ;True if character x is one of 
   ; (close-paren close-bracket close-brace)
   (define (close? x)
      (or 
          (char=? x close-paren) 
          (char=? x close-bracket) 
          (char=? x close-brace)))
   
   ; Recursive co-procedure: 
   ; (balance/1 port n balance-stack state-stack)
   ; Exit with failure if EOF.
   ; Call the 'test co-procedure if in 'normal state. 
   ; Otherwise, first decide whether or not to pop up 
   ; and dismiss the current state from the state stack, 
   ;and then recurse.
   (define (balance/1 port n balance-stack state-stack)
      (define c (read-char port))
      (cond
         ; End of stream: 
         ; break signaling the failure
         ((end-of-data? c) (list #f n c))
         ; Normal state: 
         ;call co-procedure 'test
         ((eq? (car state-stack) 'normal) 
            (test c port (+ n 1) balance-stack state-stack))
         ; We are in 'quoting state 
         ;and first char is double-quote: 
         ;    recurse with popped state-stack   
         ((and 
              (eq? (car state-stack) 'quoting) 
              (char=? c double-quote))
            (balance/1 port (+ n 1) balance-stack (cdr state-stack)))
         ; Commenting state and the first char 
         ;is newline or return:
         ;     recurse with popped state-stack
         ((and (eq? (car state-stack) 'commenting)
            (or
               (char=? c #\newline)
               (char=? c #\return)))
            (balance/1 port (+ n 1) balance-stack (cdr state-stack)))
         ; Otherwise: recurse with both 
         ;stacks unchanged  
         (else (balance/1 port (+ n 1) balance-stack state-stack))   
         ))
         
   ;Recursive co-procedure: 
   '(test x port n balance-stack state-stack).
   ;Test current character x and behave accordingly
   (define (test x port n balance-stack state-stack)
      (cond
         ;X is open char: push x on balance stack 
        ; and recurse
         ((open? x) 
            (balance/1 port n 
                (cons x balance-stack) state-stack))
         ;X is double-quote: push 'quoting on 
         ;state stack and recurse
         ((char=? x double-quote)
            (balance/1 port n balance-stack 
                (cons 'quoting state-stack)))
         ;X is semicolon: push 'commenting on 
         ;state stack and recurse
         ((char=? x semicolon)
            (balance/1 port n balance-stack 
                  (cons 'commenting state-stack)))
         ;X is not close char: ignore x and recurse
         ((not (close? x)) 
            (balance/1 port n balance-stack state-stack))
         ;State stack is empty: break signaling failure
         ((null? balance-stack) (list #f n x))
         ;X is close char but is not a mate 
         ; of top of balance stack: 
         ; break signaling failure
         ((not (mate? (car balance-stack) x)) (list #f n x))
         ;X is close char and balance stack 
         ;has just 1 element: 
         ; Hurrah! Break signaling success
         ((= (length balance-stack) 1) (list #t n x))
         ;Otherwise, x is a matching close char: 
         ; Partial match, pop up balance 
         ; stack and recurse
         (else (balance/1 port n 
             (cdr balance-stack) state-stack))))
                      
   ; Peek the first character
   (define c (peek-char port))
   
   (cond
      ; Empty string?: discard end-of-data 
      ; marker and fail
      ((end-of-data? c) (begin (skip-to-end port) -1))
      ; String does not start with "open" char?: 
      ; discard everything and fail
      ((not (open? c)) (begin (skip-to-end port) -1))
      (else 
         ; Try the real balancing
         (let ((result (balance/1 port -1 '() (list 'normal))))
            (begin
               ; Some characters waiting in pipe? 
               ; Discard them.
               (if (not (end-of-data? (caddr result)))
                  (skip-to-end port))
               (if (car result) (cadr result) -1))))))
 

#|
Copyright (C) Jan Skibinski (2006). All Rights Reserved.
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|#

Balancing code in Scheme-OSA

Balancing Scheme expressions containing parentheses, brackets and braces.
A practical application of callbacks from Scheme to Applescript - as implemented in Scheme-OSA application

Callbacks

A Scheme callback, as defined for our purposes, is a means by which a Scheme interpreter asks the Scheme-OSA bridge to evaluate some piece of foreign code on its behalf. The mechanism involves marking the Scheme response as a 'callback, attaching a foreign code (directly or indirectly) to be evaluated by Applescript or indirectly by Cocoa, and putting itself in a wait mode in order to read possibly useful results from the foreign calculations.

The normal mode of the bridge operation is simple: user types some Scheme expression into the input text view of Scheme-OSA application, the expression is sent via input pipe to Scheme interpreter, the interpreter computes the results and sends them back via its output pipe to the bridge, and the bridge displays the results in the output text view.

The callback mode is more complicated, since there are extra round trips to complete the transaction, and some data marshalling - simple as it is - from Scheme to Applescript and back. For example, this Scheme expression:

(* 
    (ask-for-number "enter odd number" 3) 
    (ask-for-number "enter even number" 20))

uses a wrapper of an applescript code from our tiny GUI library to prompt a user for some numeric data. As usually, this code is initiated by a user on the bridge side, but once it gets to the Scheme interpreter it causes the latter to issue two callbacks. Both callbacks invoke GUI dialogs, a user enters requested numbers into text fields of dialog boxes (defaults: 3 and 20 are for lazy users, :-)), and both numbers are returned to Scheme. From then on everything looks like in the normal mode of operation: Scheme mutiplies the received numbers and returns the final result (default: 60) to the bridge - which then displays it in output text view.


[A side note: Practical implementation of the above mechanism is more complicated than that, due the fact that the Scheme-OSA interaction is asynchronous: it is sequential, but not synchronous. In other words, bridge does not wait for Scheme interpreter to complete its computations, but goes away letting other appplescript applications to do their jobs. The GUI thread of Scheme-OSA but will be notified by a special event when Scheme response is to be read from a buffer. In meantime a background thread collects all characters that Scheme sends through its output pipe -- until it senses that it is time to send a "time-to-consume" notification to the main thread.]

This script provides some Scheme utilities to be used by Scheme-OSA bridge to perform paren/bracket/brace balancing of Scheme code, which is either typed into, dragged to, or copied to the input text view from other editors.


Description of procedures


Below are two procedures:
1. (balance-stream port)
2. (balance-range k)

The first one is a pure Scheme procedure, which you can test in any environment and any input port, including Scheme-OSA.

Examples:
(define p (current-input-port))
(balance-stream p)()rubbish@ ==> 1
(balance-stream p)(string-append "abc" "efg")rubbish@ ==> 26
(balance-stream p)(string-append "ab[c" "efg")rubbish@ ==> 27
(balance-stream p)((string-append "ab[c" "efg")rubbish@ ==> -1

The second one, the wrapper around the first one, is designed for consumption by Scheme-OSA bridge, to set a new text selection that visualizes the user-activated paren/bracket/brace balancing, or lack of such.

It is used as follows:

1. User moves the insertion point just in front of any "open" character in input text view; that is, one of '(open-paren open-bracket open-brace) and then presses the option-command-B keys, or selects the "Balance" commmand from the Text menu in the menu bar.

2. Scheme-OSA bridge passes the command "(balance-range k1)" and a stream of characters following the insertion point k1 to Scheme interpreter.

3. Scheme interpreter computes a list '(k1 k2) indicating a range of characters that are balanced by a "close" character; that is, one of '(close-paren close-bracket close-brace) characters, which match the opening character. After marking it as a callback and attaching to it an Applescript code the range is sent back to bridge for evaluation.

4. The bridge recognizes it as callback, and passes it to AppleScript runtime for compilation and evaluation. The runtime colorizes the computed range as a new selection and the computed range is returned back to Scheme.

5. Scheme, having no further use for this result, just echoes it back to the bridge.

6. Bridge displays it - as any other Scheme response destined for the direct consumption (i.e., not a callback) - in the output text view.


Why calling Scheme for balance computation?

It would seem that the problem of paren/bracket/brace balancing could be easily solved via regular expressions mechanism. And since we have access to one such regex library for Applescript, "Satimage Osax"[1], we could handle the balancing at the Applescript level with the help of Satimage -- without a need for calling back to Scheme. After all, we use Satimage's regular expressions for text coloring.

Unfortunately, Satimage - as most other regex libraries - does not support recursive patterns in regular expressions.
Let us see what others have to say on a subject of a code balancing. The following is a quote from the TextWrangler[3] editor manual:

Without the use of recursion, the best that can be done is to use a pattern that matches up to some fixed depth of nesting. It is not possible to handle an arbitrary nesting depth. Perl 5.6 has provided an experimental facility that allows regular expressions to recurse (among other things). It does this by interpolating Perl code in the expression at run time, and the code can refer to the expression itself.

Obviously, Scheme-OSA cannot support direct interpolation of Perl code at the Applescript level. There are two choices however: one can escape to shell via "do shell script" and then call the Perl engine, or one can call Scheme to perform the recursive balancing via a proper tail-recursive procedure, such as the 'balance-range. Since Scheme is the integral part of Scheme-OSA, we have chosen the latter approach for obvious resons...

Perceived efficiency

No noticeable delays are felt when running this code on G4 laptop, even when the input stream (a Scheme source code following the insertion point in input text view) becomes huge.

Why combination of keys: option-command-B for "Balance" command?

Two of the best ASCII editors for Mac OS X, the commercial BBEdit[4] and its free cousin TextWrangler[3] use command-B as the key sequence for the invocation of the "Balance" command. However, since one of our goals is to provide IDE working in a rich text environment, not just the ASCII one, we need to preserve the command-B for invocation of the "Bold" action, typical in any rich-text editing environment. Consequently, we choose option-command-B for the "Balance" command. If you do not like this, you can easily remap some of the keys of the main menu of the Scheme-OSA application, including option-command-B.

Limitations

There are several limitation to the algorithms presented here.

1. The algorithms work only in the "forward" mode, which means that the insertion point must be positioned just before an "open" character. A good balancer should also work "backwards" - from the last "close" character to the first matching "open" character, and "from the middle", where insertion point is placed somewhere between an "open" and a "close" characters.
I'll work on it later.

2. Current implementation correctly ignores any open/close characters that appear within strings or within line comments. However, it does not ignore balancing within the Scheme block comments. Working on it...

3. A temporary End-Of-Data hack, described below


A temporary End-Of-Data hack

In a rush to get a prototype working, I have introduced #\@ character as a End-Of-Data marker. Some sort of a marker is needed since the pipe between Applescript and Scheme is always open. In normal circumstances the marker is not actually needed since we can use Scheme prompts to decide whether or not the data is complete. In the postfix i/o cases, such as this one, the data marker is needed, because we have no other means to signal End-Of-Data; the open pipe will never send End-Of-File marker.

But we have to keep the input pipe to Scheme open, otherwise a Scheme interpreter would exit prematurely, if we were to send "close-pipe" command after writing data to Scheme. Consequently, no EOF object is ever to be expected on the pipe. But using character #\@ as a EOD marker is not a good choice: after all #\@ is as legal as any other ASCII character: it can be used in strings, symbols or identifiers.

Possible other solutions are:

1. Marking EOD by a sequence of ASCII characters, rather than by a single one. Such algorithms are known, but efficiency would somewhat suffer due to look ahead and backtracking.

2. Using a higher order character with a code point x, such that 127 < x < 256. Not every Scheme implementation supports this. Petite Chez Scheme for example, although happy with Unicode and UTF8 when reading strings or symbols, will crash when individual characters are read or examined.

3. Redesigning read-char, so
(read-char (current-input-port))#\bel
would indeed recognize #\bel (or other such control character) as a single character, and not as # followed by \bel.

4. Making exception to this and other i/o procedures and using temporary files or named fifo pipes, rather than the one standard input pipe. I'd rather not to introduce any exceptions to this simple model.

5. Redesigning the signature of the "balance-range" function to accept explicit strings, rather than implicit streams. This means that we would have to prepare a string like this: "(balance-string \"quoted-text-of-input-text-view\" k1)".
This works, but onus is on Applescript to double-quote every string that might appear in the text of the input text view. It follows that the text
a. must be surrounded by "\"" substrings
b. must have every double-quote character preceded by \\.
It is the point b. that makes such algorithm very inefficient, since such quoted string must be assembled on character-by-character basis. Although acceptable for short pieces of code, the performance is not acceptable when balancing even the smallest pieces of code in typically-sized Scheme modules.

References:

[1] "Satimage Osax" by Satimage-software, www.satimage-software.com
[2] osax - (Open Scripting Architecture eXtension) is the nickname for the Scripting Additions', the plug-ins for AppleScript.
[3] TextWrangler - free Mac OS X editor, by Bare Bones Software, http://www.barebones.com
[4] BBEdit - commercial Mac OS X editor, by Bare Bones Software, http://www.barebones.com