Wednesday, December 24, 2008
GIT repository open (at last)
To checkout a copy:
git clone git://github.com/crcx/retro10
This is up to date with my latest source, so anyone with patches pending should check them against this and then send them in.
For inclusion in the 10.0-stable release, I'll be needing the code/patches by Dec. 31st. (The release should be finished and made public by January 5, 2009)
Tuesday, December 23, 2008
Nearly there
It'll include: C implementations (mine and Mat's) for console and framebuffer, javascript vm, Toka, Retro source, and docs.
The library will be made into a second download.
Binaries will be provided for various systems. If you have requests please let me know so I can obtain the relevant systems for building.
Sunday, November 30, 2008
Getting closer to a stable release
Friday, November 7, 2008
Some notes on my block editor
Thursday, October 30, 2008
Things I'm working on...
- Exposed ok (to allow control of the prompt)
- extras/bootstrap.retro
- extras/retro.nanorc
- library/debug.retro
- library/does.retro
- library/date.retro
- library/math.retro
- library/value.retro
- core: removed unused bits left over from old compile-time module support
- cross compiler: removed unused bits left over from old compile-time module support
- library/square-root.retro [merged with library/math.retro]
- Tutorial
Wednesday, October 22, 2008
Release 2008.10.22
- Three VMs included
- Console [curses backend]
- Framebuffer [SDL backend]
- JavaScript
- Major Bugs Fixed:
- Saving of images now works reliably on Windows
- Scrolling works properly in the console build
- Compile-time modules replaced by a new library of modules that can be loaded from the command line
- The core now reserves a smaller piece of memory for its own use (10k rather than 32k), leaving a little more room for applications
- Small improvements to the 1994 ANS FORTH layer
- Documentation has received some minor updates to formatting and a little explanation of how to load modules at the command line.
Saturday, September 20, 2008
Release 2008.09.21
- Support for loading code from a normal text file
This only works for console implementation
Use ./ngaro --with filename - Improved screen clearing
Trying to display a negative character code clears the display.
This allows a cleaner, more portable way of clearing the display without relying on particular escape sequences being emulated. - Console source trees merged
- Cleanups to I/O handling in the console implementation
- Curses-backend used for console implementation by default
Monday, September 1, 2008
Binary Package
- Linux (x86, x86-64, and Alpha)
- Mac OS X (Intel)
- Windows (x86, tested on XP SP3)
Sunday, August 31, 2008
Release 2008.08.31
- Build scripts replaced by Makefiles
- Documentation is now included
- The default image name is now retroImage
- VM's attempt to load retroImage if no image name is specified
- Win32 implementation now separate from the standard framebuffer source
- Assembler is now separate from the Retro source
- Modules renamed and organized better
Sunday, July 27, 2008
Release 2008.07.27
I've pushed the latest release (2008.07.27) online. This has a few nice improvments, including inlining of primitive words, the latest JavaScript implementation of Ngaro, and lots of internal cleanups.
Due to changes on repo.or.cz, I've been unable to push changes to the GIT repository for a while now. I'm looking for a solution, and hope to be able to resume putting changes into a VCS soon.
The Windows installer has not been updated yet due to hardware problems on the single Windows box I have. I'm hoping to be able to have an updated installer later this week.
The Dashboard widget and Retro Online are now running the latest code with improved reliability and improved interfaces. Users of the Dashboard widget should definitely update as this fixes some resource usage issues and offers significantly greater performance.
Anyway, download the latest snapshot or image and enjoy.
Monday, July 21, 2008
Retro Widget for Apple's Dashboard
If this proves useful, I'll look into supporting other widget platforms in the future.
Get the Retro Dashboard Widget
Saturday, July 19, 2008
Misc. Updates
- I've begun updating http://ngaro.jottit.com using the material from the Retro docs as a base. It can now be edited by anyone, so all those working on Ngaro implementations can contribute.
- jsvm3: An updated version of the Ngaro in Javascript, this is primarily based on an updated I/O model that allows an even closer approximation of the console builds. This is new, and needs testing.
Monday, July 7, 2008
Improving Performance
While looking over Mat's extensions to Ngaro I had a thought about inlining the handful of words which correspond to Ngaro instructions. A few tests showed enough promise that I've added a new .inline class to Retro.
The primitives will only be inlined if they are not revectored. This ensures that they will behave as closely to normal Forth words as possible. Devectoring a .inline word will restore the original behaviour. This is not intended for user code, and is only useful if you are adding new opcodes to your Ngaro VM.
Anyway, a look at the benchmarks:
Switched VM
Before:
- Recursive FIB (39)... 208.0144 seconds
- Countdown Loop (1,000,000)... 12.228482 seconds
- Nest/Unnest (256 million pairs)... 42.299306 seconds
Now:
- Recursive FIB (39)... 85.860101 seconds
- Countdown Loop (1,000,000)... 12.38329 seconds
- Nest/Unnest (256 million pairs)... 30.796164 seconds
Threaded VM
Before:
- Recursive FIB (39)... 130.537818 seconds
- Countdown Loop (1,000,000)... 8.325312 seconds
- Nest/Unnest (256 million pairs)... 27.092447 seconds
Now:
- Recursive FIB (39)... 51.989871 seconds
- Countdown Loop (1,000,000)... 8.292265 seconds
- Nest/Unnest (256 million pairs)... 17.577663 seconds
I'll update the documentation to cover the new class in the near future. If you don't want to wait for a new release, you can get this code from the GIT repository. See http://repo.or.cz/w/retro.git for details.
Sunday, July 6, 2008
Site Changes
I've begun reworking the layout of the Retro website. This should be completed within the next few days.
Some highlights of the new design so far:
- Cleaner organization
- Downloads on the right, will remain easier to spot
- Automatically updated links to recent discussions on the mailing list and recent commits to the main GIT repository. (Updated hourly)
Coming soon:
- New documentation page, which will allow downloading the documentation in various formats (text, word doc, pdf, html) as well as viewing it online.
- The News page will have links to the blogs of those involved in Retro's development, as well as displaying the most recent posts (updated hourly) from each blog.
Possibilities:
- Web access to the #retro channel on irc.freenode.net
Sunday, June 29, 2008
Release 2008.06.29
This is a minor update with the updated word class handling (so you can try code from my previous posts) and a few minor fixes and improvements to the source. I've also updated the online interpreter to use both the latest Ngaro.js and image.
Wednesday, June 25, 2008
Switched to GIT
I've switched to using GIT for the Retro source repository. Relevant information follows:
To obtain a copy:
git clone git://repo.or.cz/retro.git
The repository can be browsed at:
http://repo.or.cz/w/retro.git
Thursday, June 19, 2008
Adding New Word Classes
In my previous post I discussed the concept of word classes. In this post I'll show how to define and add a word class to Retro. This assumes that you know Forth, and have some experience with Retro. Consult the documentation if you have questions on the words used.
For this example, we'll create a class for strings with the following behaviour:
- If interpreting, display the string
- If compiling, compile the code needed to display the string
Retro has a convention of using a . as the first character of a class name. We'll call our new class .string
On entry to a class handler, the address (xt) of the word or data element is on the stack. The compiler state (often important to class handlers) is stored in a variable named compiler. With this in mind, we'll define our class:
: .string ( xt -- )
compiler @ 0 =if type ;; then
1 , , ['] type compile ;
The compile-time portion is a bit trickier than the interpret time since it has to lay down the proper code in the target definition. In this case a LIT instruction (opcode 1) is laid down, followed by the xt of the string. This is followed by code to lay down a call to type.
We now need a creator word to attach this class to a value. For this example we'll define displayString: to take a form like the one shown below.
string-addr displayString: name
New dictionary entries are made using create, so we'll use that and change the class to our new .string handler.
: displayString: ( string-addr -- )
create ['] .string last @ d->class !
keepString last @ d->xt ! ;
This uses create to make a new word, then sets the class to .string and the xt of the word to the string. It also makes the string permanent using keepString. last is a variable pointing to the most recently created dictionary entry. The words d->class and d->xt are dictionary field accessors and are used to provide portable access to fields in the dictionary.
We can now test our new class:
" hello, world!" displayString: hello
hello
: foo hello cr ; foo
Introduction to Word Classes
There are many ways to code a Forth interpreter. When I inherited Retro, there were two separate dictionaries, one for normal words and data, and one for compiler macros. There were two internal token processing loops, one for interpret time and one for compile time. This lead to some confusion as the compiler macro dictionary was only visible when compiling words, and preceeded the normal dictionary. When new functionality (inlining, etc) was added, more dictionaries and internal loops had to be coded.
Some Forths solve this by having flags in the dictionary headers and having the token processor use these to select the proper functionality. This still requires changes to the kernel to handle new flags and add the behaviour in the proper places.
Helmar Wodtke introduced me to a simpler approach which he calls word classes. In this approach each dictionary entry gets an additional value, a pointer to a class handler. Class handlers are just words that take an execution token (in Retro this is just the address of a word) and do something with it.
A default Retro build has three classes:
- .word
At compile time, compile a call to the word. At interpret time, execute the word. - .macro
Execute the word. The word may compile code or add data into the heap based on any form of system state. - .data
Return the address or value associated with the requested token.
These cover the core functionality required for a small Forth implementation.
In the next post we'll look at how to define new classes and use them to extend Retro's functionality.
Saturday, June 14, 2008
Release 2008.06.14
I also updated the binary release for Windows. It's now an MSI installer package.
Some of the more signficant enhancments:
- Ngaro
Gained a new I/O port for queries of emulated devices
C implementations now have more flexible command line processing
Updated threaded C implementation - Retro
Minor bugfixes
All non-data words are now vectors
Now detects hardware settings using the new I/O port in Ngaro
Word classes exposed - Updated benchmarks
Thursday, June 12, 2008
Exposing Word Classes
Until today it was not possible to define a custom word class. Now you can, but it's not automatic. You'll have to redefine with-class, but take care to handle the existing classes (.word, .macro, and .data).
I'll try to document this sometime soon, but take a look in the Retro source to see how it's done if you wish to experiment with it. The code supporting this is in the SVN repository.
Ngaro Updates
To help reduce the work involved involved in adapting Retro to these devices, I've set I/O port 5 as a hardware query device. This can be queried by writing specific values, waiting, then reading back the result.
Since some implementations are likely to support optional hardware devices in the future, they can use this to provide Retro with information about these devices.
In the SVN repository the C implementations of Ngaro have been adapted to support this. I'm working on the JavaScript implementations and should have them ready tomorrow.
The initial capabilities that can be detected are:
- -1 Get Amount of Memory
- -2 Get Framebuffer Address (0 if not supported)
- -3 Get Framebuffer Width (0 if not supported)
- -4 Get Framebuffer Height (0 if not supported)
Friday, June 6, 2008
Improving Ngaro in JavaScript
Feel free to take a look.
Retro meets J2ME

The sources are in the subversion repository and I'm working on making it buildable using Retro's standard build scripts.
Martin has provided a source and binary package at http://www.nigol.cz/retro
To try building it you'll need to be using MacOS X and Ant. Of course, you'll also need the relevant Java libraries and development tools.
Sunday, May 25, 2008
Benchmarking Retro
cd bin/bench
./make-image
./run-bench
The timings for each test will be displayed when the test completes. There are currently three tests:
- Recursive FIB
- Countdown Loop
- Call/Return (nesting) Pairs
Initial results:
Host: ZenWalk 5 (Linux)
GCC: 4.1.2
CPU: Pentium 4 2.8GHz (1 core)
RAM: 1.5gb
Switched VM:
Recursive FIB (39)... 225.540318 seconds
Countdown Loop (1,000,000)... 13.616979 seconds
Nest/Unnest (256 million pairs)... 36.356847 seconds
Threaded VM:
Recursive FIB (39)... 212.555673 seconds
Countdown Loop (1,000,000)... 13.995988 seconds
Nest/Unnest (256 million pairs)... 33.026269 seconds
Updated Documentation
Many sections have been expanded, and some new areas were added. I think it's a definite step forward and invite everyone to Check it out.
Saturday, May 24, 2008
Loading text files into an image
./build
cd bin
cat filename | ./ngaro forth.image
Or, for interactive mode:
cat filename - | ./ngaro forth.image
There is a limiting factor. The standard Retro image only recognizes space as valid whitespace, so any use of tabs or newlines will cause problems. This can be fixed by starting your code with the following line. This should be one single line. Be sure to include a space at the end of the line:
: fix dup 10 =if drop 32 then dup 13 =if drop 32 then
dup 9 =if drop 32 then ; ' fix is (remap-keys)
If you want, this can be disabled later by doing:
devector remap-keys
If you aren't going to drop to an interactive prompt, end with bye. Be sure to save your state if you want any changes to be present when you next load your Retro image.
Friday, May 23, 2008
Threaded Execution in the Ngaro VM
I've not done any significant benchmarking, but he did provide some numbers:
: test dup for dup . CR 1 – next ; 1000000 test byengaro (old):
real [s] 35,47ngaro (new):
user [s] 8,27
sys [s] 1,07
real: [s] 35,13
user [s] 7,46
sys [s] 1,34
: fib dup 1 >if dup 1 RECURSE swap 2 RECURSE + EXIT then drop 1 ; 39 fib bye
ngaro (old):
real [s] 131,23ngaro (new):
user [s] 131,21
sys [s] 0,03
real [s] 77,95
user [s] 77,67
sys [s] 0,04
This is a clear improvement and should continue to get better. I've begun to work on a small benchmark package; once done it'll be easier to test performance tweaks.
Sunday, May 11, 2008
Retro on Windows
I'll be revising the Windows Build Instructions later today, to help clarify a few points and point out the exact files I'm using. In addition, I've built a new installable package for Retro 10. This is a standard MSI package, so should install on XP and Vista machines without problems. It includes all necessary dependencies. I expect to replace the binary package on retroforth.org with this later this week.
Saturday, May 10, 2008
Release 2008.05.10
- The video refresh port is now set properly after outputting a character
- Ngaro JS now supports the video refresh port
- Ngaro JS now stores the video output apart from the form output textarea
- Strings are now rendered after all characters are drawn to the video buffer rather than character by character
- update (enable/disable screen redraws) was added
- redraw (redraw the screen if update is on) is now in the dictionary
- Graphics module now does video updates after drawing shapes
- Removed the FFI from Toka.
I also removed the FFI functionality from Toka for this release. The FFI words are not used by the cross-compiler, and serves only to slow down the build of Toka and increase the number of dependencies. (This only affects the copy of Toka included in the Retro package; the full Toka still has FFI).
Sunday, May 4, 2008
Retro on Mobile Devices
I have confirmation that it works properly on (platform, browser):
- Windows Mobile 5 (Minimo)
- Apple iPhone (Mobile Safari)
Links:
- Retro Mobile (http://retroforth.org/mobile)
- Download Retro Mobile (zip, http://retroforth.org/mobile/mobile.zip)
- Download Retro Mobile (tar.gz, http://retroforth.org/mobile/mobile.tar.gz)
- Minimo (http://www.mozilla.org/projects/minimo/)
Saturday, May 3, 2008
Release 2008.05.03
The build system:
- Refactored, now has separate functions for each step of the build system
- The clean script has been merged in
- Accepts various command line options allowing more control over the build
- Basic help (pass --help) to see
- Several bugs were fixed in fix-image
- Memory usage of fix-image was reduced
- image2js.toka is now commented
- COUNT and R@ were added to forth94 and forth83.
- for / next loops have been added
- Added a Clear Display button.
Loops
macro: for here 5 , ;
macro: next 6 , 27 , 25 , 8 , , ;
For those who are curious, this involves two compiler macros, which inline raw machine code into the words using the macros. As an example:
: foo 10 for 98 emit next ;
This will compile to:
foo:
lit 10
foo.1: ( 'for' )
push
lit 98
call emit
pop ( 'next' )
1-
0;
jump foo.1
;
There is a limitation that should be mentioned: when the loop counter reaches zero, the word will exit. Factor loops out if this is a problem for you.
Monday, April 28, 2008
SVN Repo (and Trac)
The repository can be browsed via Trac. Please be aware that this is primarily intended for my use, but could be useful to anyone wanting to stay current with my work.
Current Work
- The image builder now builds an image.js for use with the Ngaro JS
- Ngaro JS now has a Clear Display button which may be of some use
Sunday, April 27, 2008
Image Builder
I have finished a quick prototype. It will allow you to select from the majority of the modules that are included in the latest release and build an image including them. This does not save images for long, so download quickly. (The download will be replaced by the next image that is built.)
The image builder will also provide a copy of the image map and the build log. It does not build the image for Ngaro JS yet.
Give it a try.
Thursday, April 24, 2008
Release 2008.04.26
This week brings a couple of minor changes and improvements:
- Ngaro JS now included in the package
- A tool (in Toka) to build an image.js for Ngaro JS
(This does not work properly on 64-bit systems, I'll try to fix in the next week or two) - A new (remap-keys) hook has been added to allow alteration of the values returned by key before they can be processed by other words
- New whitespace module, which uses (remap-keys) to convert TAB, CR, and LF to SPACE
- retro.config renamed to extensions
- Minor cleanups to start organizing the modules
Possible change to 'key'
Ngaro JS performance improvements
Now to apply some CSS to improve the look and feel a bit....
For those who are curious about what I did, all it took was:
- Put in a loop to call vm_process() more than once per 10ms interval
Monday, April 21, 2008
New Implementations of Ngaro
JavaScript
The first implementation was done in JavaScript. It presents a form-based interface that allows for a crude approximation of the tty console I/O model. Though brutally slow and memory hungry, it does work and seems quite reliable.
Java
While far from my favorite language, I recognize that Java is fairly widespread, so yesterday I wrote a quick and dirty implementation in Java. This is running as a Java applet, and still has some bugs in the I/O device emulation, but does work. It does have an advantage over the JavaScript: this one is very fast.
Future
I'll be cleaning up and working on merging the source for these into the main Retro codebase. Beyond this, I hope to eventually get the JavaScript implementation to run at an acceptable speed. The Java version is fast enough, and will hopefully have its bugs ironed out soon.
Eventually I'd like to have a couple of variations on the Java implementation. It would be nice to run Retro on a Java-capable cell phone or pda, and having an offline version would be useful as well. Given enough time, it's possible that the full graphical console I/O devices could be emulated.
Looking out even further, I am currently doing some research on .NET; an implementation of Ngaro for .NET could also be a big help in improving platform support.
References
RetroNews, 2.0
Since I'm now actively working on Retro again, I've decided to restart blogging as a means of sharing what I'm working on.