Release 2008.06.29

This is a minor update with the updated word class handling (so you can try code from my previous posts) and a few minor fixes and improvements to the source. I've also updated the online interpreter to use both the latest Ngaro.js and image.

Switched to GIT

I've switched to using GIT for the Retro source repository. Relevant information follows:

To obtain a copy:

  git clone git://repo.or.cz/retro.git 

The repository can be browsed at:

  http://repo.or.cz/w/retro.git

Adding New Word Classes

In my previous post I discussed the concept of word classes. In this post I'll show how to define and add a word class to Retro. This assumes that you know Forth, and have some experience with Retro. Consult the documentation if you have questions on the words used.

For this example, we'll create a class for strings with the following behaviour:

• If interpreting, display the string
• If compiling, compile the code needed to display the string

Retro has a convention of using a . as the first character of a class name. We'll call our new class .string

On entry to a class handler, the address (xt) of the word or data element is on the stack. The compiler state (often important to class handlers) is stored in a variable named compiler. With this in mind, we'll define our class:

  : .string ( xt -- )
    compiler @ 0 =if type ;; then
    1 , , ['] type compile ;

The compile-time portion is a bit trickier than the interpret time since it has to lay down the proper code in the target definition. In this case a LIT instruction (opcode 1) is laid down, followed by the xt of the string. This is followed by code to lay down a call to type.

We now need a creator word to attach this class to a value. For this example we'll define displayString: to take a form like the one shown below.

  string-addr displayString: name

New dictionary entries are made using create, so we'll use that and change the class to our new .string handler.

  : displayString: ( string-addr -- )
    create ['] .string last @ d->class !
    keepString last @ d->xt ! ;

This uses create to make a new word, then sets the class to .string and the xt of the word to the string. It also makes the string permanent using keepString. last is a variable pointing to the most recently created dictionary entry. The words d->class and d->xt are dictionary field accessors and are used to provide portable access to fields in the dictionary.

We can now test our new class:

  " hello, world!" displayString: hello
  hello
  : foo hello cr ;   foo

Introduction to Word Classes

There are many ways to code a Forth interpreter. When I inherited Retro, there were two separate dictionaries, one for normal words and data, and one for compiler macros. There were two internal token processing loops, one for interpret time and one for compile time. This lead to some confusion as the compiler macro dictionary was only visible when compiling words, and preceeded the normal dictionary. When new functionality (inlining, etc) was added, more dictionaries and internal loops had to be coded.

Some Forths solve this by having flags in the dictionary headers and having the token processor use these to select the proper functionality. This still requires changes to the kernel to handle new flags and add the behaviour in the proper places.

Helmar Wodtke introduced me to a simpler approach which he calls word classes. In this approach each dictionary entry gets an additional value, a pointer to a class handler. Class handlers are just words that take an execution token (in Retro this is just the address of a word) and do something with it.

A default Retro build has three classes:

• .word
  At compile time, compile a call to the word. At interpret time, execute the word.
  
• .macro
  Execute the word. The word may compile code or add data into the heap based on any form of system state.
  
• .data
  Return the address or value associated with the requested token.
  

These cover the core functionality required for a small Forth implementation.

In the next post we'll look at how to define new classes and use them to extend Retro's functionality.

Release 2008.06.14

I've finally pushed a new release. This is current as of 7:15 pm (EST) and includes the stable releases of Ngaro (C, JavaScript) as well as current source and build scripts.

I also updated the binary release for Windows. It's now an MSI installer package.

Some of the more signficant enhancments:

• Ngaro
  Gained a new I/O port for queries of emulated devices
  C implementations now have more flexible command line processing
  Updated threaded C implementation
• Retro
  Minor bugfixes
  All non-data words are now vectors
  Now detects hardware settings using the new I/O port in Ngaro
  Word classes exposed
  
• Updated benchmarks
  

The documentation has been updated to reflect the changes.

Exposing Word Classes

I've been using word classes in Retro 10's implementation since the beginning. This allows a simple, consistent means of controlling the behavior of words.

Until today it was not possible to define a custom word class. Now you can, but it's not automatic. You'll have to redefine with-class, but take care to handle the existing classes (.word, .macro, and .data).

I'll try to document this sometime soon, but take a look in the Retro source to see how it's done if you wish to experiment with it. The code supporting this is in the SVN repository.

Ngaro Updates

With recent work on Ngaro it's become possible to run Retro on many smaller systems (including phones and pdas). This has lead to a need for supporting smaller memory configurations and screen sizes.

To help reduce the work involved involved in adapting Retro to these devices, I've set I/O port 5 as a hardware query device. This can be queried by writing specific values, waiting, then reading back the result.

Since some implementations are likely to support optional hardware devices in the future, they can use this to provide Retro with information about these devices.

In the SVN repository the C implementations of Ngaro have been adapted to support this. I'm working on the JavaScript implementations and should have them ready tomorrow.

The initial capabilities that can be detected are:

• -1 Get Amount of Memory
  
• -2 Get Framebuffer Address (0 if not supported)
  
• -3 Get Framebuffer Width (0 if not supported)
  
• -4 Get Framebuffer Height (0 if not supported)

I've updated the Retro source to detect the framebuffer settings on startup and use them rather than relying on hardcoded values. Retro also detects the amount of memory being provided, but the memory map still needs to be configured in the source.

Improving Ngaro in JavaScript

Ngaro in JavaScript (jsvm) has been working nicely for quite a while now. Today I did a little more work to help improve the interface. It's now using the innerhtml to remove the textarea for the output. I've tweaked the ngaro.js a bit to handle newlines properly with this. The tracing and error log is not supported yet, but I expect to work that out in the near future.

Feel free to take a look.

Retro meets J2ME

Martin Polak has ported the Ngaro VM to mobile devices with J2ME support. This is another great step forward as it makes Retro even more accessible to developers.

The sources are in the subversion repository and I'm working on making it buildable using Retro's standard build scripts.

Martin has provided a source and binary package at http://www.nigol.cz/retro/Retroforth.zip.

To try building it you'll need to be using MacOS X and Ant. Of course, you'll also need the relevant Java libraries and development tools.