Still Alive

Still alive, and still working on Digamma/PreDigamma.

• Hydra has become a research implementation of Digamma, written in Digamma, that can run itself. It also is written in a style that allows it to be compiled with a preDigamma compiler.
• Eprime has been tossed in favor of Enyalios, which is smaller, has a better performance profile, and is easier to extend.
• Vesta is still alive & kicking as the reference implementation of Digamma proper, although Hydra may grow to fill that role. 
• The plan now is that all non-core development should be in Digamma/PreDigamma, and only the necessary items should be written in C.

Quick Performance Update

So yesterday I was talking with a colleague about Digamma, and I realized that I haven't run the full suite of micro-benchmarks for almost 1.5 years now. Previously, Vesta (the C interpreter for Digamma that uses a simplistic "VM" (no compilation, self managed stack with jumps, &c.)) was generally a bit slower than TinyScheme, or in that region. Today, it's in the same realm as Gambit-C & Gauche, on my old AMD Turion64. I'll post some further updates later today or tomorrow, but with simple microbenchmarks (Fibonacci sequence, CSV parse & compute stats, Do nothing but parse prelude & start, HTTP connection and bit-vector test). I've always attempted to avoid things for which Digamma has a data structure advantage (anything that would compare Digamma's dictionaries, which are backed by tries, to a-lists is inherently unfair), so I'm quite happy with this result. It also means that no matter where Nyx, Ceres, Eris, Enyo & E-prime go, they can't get much worse than Vesta. Very good day for Digamma.

Aneris, Ceres update & more (for the end of the year)

Aneris

Aneris is a simple AST walking interpreter, that supports delimited continuations, W7, POSIX & most everything that Vesta does. I created it as a useful test of E-prime's & Enyo's ability to meaningfully compile preDigamma programs. The nice thing about Aneris is that it doesn't use any code that is specific to preDigamma, and, as such, it can run itself. I'm pretty happy with the way it came out.

Ceres

Ceres has been updated to support the full W7 system, as well as continuations (it had delimited continuations before). I hope that I'll be able to release Ceres, Vesta, E-prime, & Aneris early-2011.

Grid Computing

I've been using the SHARK (Scheme Hacks Against Raging Kingdoms *or* Scheme High Availability Research Kit) Grid for some time now; I hope to allow users to log in shortly after I release Digamma. I hope to add serializable environments to Vesta/Aneris/Ceres before then, so that grid nodes can fail & still hold the vast majority of a user's state (esp. since snap shotting the VM on a regular basis & storing to SHARK's data nodes shouldn't be that big of an issue). 

2011 & Beyond

Once I've released the Digamma environment, I'll move towards an IPN-style format here: the blog will start hosting my experiments & labs that focus on using Digamma in the "real world," as well as the usual developments wrt compilers, interpreters, operating systems & language design. 

Happy 2011!

Update, E' & contains

I've been working heavily on Digamma, to the point where I've not blogged about it in a year, so here's a quick update.

Vesta is working well; so well that I don't have any other scheme interpreters installed on any of my machines anymore. That's a good sign, since I use Scheme & C for just about everything. Vesta itself is just about as fast as you can possible get with an AST walking interpreter; a simple fib program, that computes & prints the Fibbonacci values for 3,4,5,10,15,20 runs in about 0.245u, where as C runs at 0.00u and natively compiled PreDigamma runs at 0.01u. I don't have timings for Ceres yet, as I'm still working on that.

E'

Enyo, the PreDigamma compiler, is a rather large piece of software; it has a type inference engine, nice code generation back-end (so that PreDigamma can target other systems, such as Inferno's Limbo or even Plan9's ken cc). This isn't fun though, so I thought about having some fun. Enter E': a tiny compiler for PreDigamma to C. It's 400 or so lines, a tail call optimizer & support for PreDigamma save for macros, syntax,c-macros, c-syntax, FFI & types. That's quite a few features missing, but it works well; I've been using it for quite a few little utilities here & there, and it compiles nicely. For instance, it turns this:

(def fib (fn (t i j n)

 (if (<= n 1)

  i

 (fib i (+ i j) t (- n 1)))))

into this:

SExp *
fib(SExp *t, SExp *i, SExp *j,SExp *n)
{
   SExp *ret = nil;
   int s1 = 0;
   while(!s1)
   {
      SExp *it0 = flte(list(2,n,makeinteger(1)));
      if(it0 == nil || it0->type == NIL || ((it0->type == BOOL || it0->type == GOAL) && it0->object.c))
      {
         s1 = 1;
         ret = i;
      }
      else
      {
          t = i;
          i = fplus(list(2,i,j));
          j = t;
         n = fsubt(list(2,n,makeinteger(1)));
      }
    }
    return ret;
}

Which is pretty nice, and quite speedy.

Contains

There's a new primitive in Digamma, called contains. This is analogous to Python's in, and it works on all collexions:

contains c : COLLEXION v : ATOMIC [key-or-value : BOOL] => BOOL

 the key-or-value part is for if you're testing a dict: you may want to test whether it has the key v or the value v, so there's a boolean flag for that. Nice, semantically simple & clean; Total positive in my book.

Networking & Enyo improvements

Networking

Vesta's networking core is now complete (under c/posix.c for those with source access). This includes a "raw ip pack" function, to build up IP datagrams (as well as higher level protocols). One thing I'm working on now is an implementation of RUDP, for use in client libraries.

Enyo

I've been working on Enyo quite a bite lately, and it now is a relatively nice system for PreDigamma development. PreDigamma is similar to PreScheme, but with GC (since nowadays it's perfectly acceptable for a "lower than application but higher than OS" level language). I was thinking about rewriting the GC in PreDigamma, and compiling bootstrapping the runtime from C, but that's mostly just notes & tests right now.

I'm getting close to the point where many other people can have access to the source & I can move to a Caerwyn-style FPN (Digamma Programmer's Notebook).

Format, define-syntax (rules)

Just a few updates:

Format

The format form:

(format specification-string rest-list)

Now works for all internal types, including all collexions. Formatting of tries, vectors, strings, and lists is supported. There are (generally) two types of formatting for each object: display form and write form; the two types correspond to their top-level forms. The switch is case: lower case is display, upper is write. So, "~t" displays a t(able | rie) as display would, whereas "~T" displays it as write would.

Syntax rules

I've fixed pattern matching in define-syntax so that you can use the usual forms:

(let ((name . val) ...) . body)

Will expand exactly how you expect (unlike before...).

More to come
I've started to bring Ceres & Enyo up to where Vesta is currently; even though Vesta is almost the same speed as TinyScheme, this isn't really fast enough for many applications. Ceres is a SECD-like VM, and Enyo has some neat tricks in it. Hopefully they can be made production ready before year's end.

Trie functions

A quick update: tries are a fundamental data structure in Vesta, and used as the backing store to dictionaries. Since tries have many useful properties, it would be great to expose these properties to the "user". I just committed the partial-key? and keys forms to the core foundation. partial-key? returns #t if 100% of the string is within the dictionary, #f if none is and N (which is an integer) for a matching subset. keys on the other hand, does what it sounds like: it returns all keys within a dictionary structure. Now, on top of the set/bag library, I'll also include a simple semantic processing library with the default build.