need help with syntax-case/syntax/datum->syntax-object

[Doug Orleans <dougo@ccs.neu.edu>]

I'm trying to grok the new syntax mechanism, but having some
difficulty.

First of all, I think there is a bug in the MzScheme Language Manual.
Section 11.1.1 says:

  The syntax-rules form can be expressed as a syntax-case form wrapped
  in lambda:

	 (syntax-rules (literal-identifier ···)
	   ((ignored-identifier . pattern) template)
	   ···)
	=expands=>
	 (lambda (stx)
	   (syntax-case stx (literal-identifier ···)
	     ((generated-identifier . pattern) template)
	     ···))

My (shaky) understanding of syntax-case makes me believe this should
instead be:

	 (lambda (stx)
	   (syntax-case stx (literal-identifier ···)
	     ((generated-identifier . pattern) (syntax template))
	     ···))

Please correct me if I'm wrong.

Okay, here's what I'm trying to do (or rather, here's a simple example 
that exhibits the same problem as what I'm trying to do does):

  (define-syntax lambda-foo
    (lambda (x)
      (syntax-case x ()
	((_ body ...)
	 (with-syntax ((foo (datum->syntax-object (syntax _) 'foo)))
	   (syntax (lambda (foo) body ...)))))))

This makes a special form in which the magic variable "foo" is available:

  > (define add1 (lambda-foo (+ foo 1)))
  > (add1 3)
  4

All well and good.  But when I make another macro that uses this, I
run into problems:

  (define-syntax traced-lambda-foo-1
    (lambda (x)
      (syntax-case x ()
	((_ body ...)
	 (syntax (lambda-foo (display foo) (newline) body ...))))))

  > (define add1 (traced-lambda-foo-1 (+ foo 1)))
  > (add1 3)
  3
  reference to undefined identifier: foo

Here's what it's being expanded to:

  > (pretty-print (syntax-object->datum (expand '(traced-lambda-foo-1 (+ foo 1)))))
  (lambda (foo)
    (#%app (#%top . display) foo)
    (#%app (#%top . newline))
    (#%app (#%top . +) (#%top . foo) (#%datum . 1)))

The second "foo" gets tagged as a top-level-variable reference, rather 
than a reference to the bound variable.  Why is this happening?  Why
does it work for the first foo but not the second?

Here's my attempt at fixing it:

  (define-syntax traced-lambda-foo-2
    (lambda (x)
      (syntax-case x ()
	((_ body ...)
	 (with-syntax ((foo (datum->syntax-object (syntax _) 'foo)))
	   (syntax (lambda-foo (display foo) (newline) body ...)))))))

But that just makes it worse:

  > (pretty-print (syntax-object->datum (expand '(traced-lambda-foo-2 (+ foo 1)))))
  (lambda (foo)
    (#%app (#%top . display) (#%top . foo))
    (#%app (#%top . newline))
    (#%app (#%top . +) (#%top . foo) (#%datum . 1)))

Both "foo"s now turn into tlv references.  Can someone walk me through 
what's going on here?  Is there a way to get this to work right?

Even more perplexing is when I try to step through the macro expansion 
using expand-once:

  > (pretty-print (syntax-object->datum (expand-once '(traced-lambda-foo-1 (+ foo 1)))))
  (lambda-foo (display foo) (newline) (+ foo 1))
  > (pretty-print (syntax-object->datum (expand-once (expand-once '(traced-lambda-foo-1 (+ foo 1))))))
  (lambda (foo) (display foo) (newline) (+ foo 1))
  > (pretty-print (syntax-object->datum (expand-once (expand-once (expand-once '(traced-lambda-foo-1 (+ foo 1)))))))
  (lambda (foo)
    (#%app (#%top . display) foo)
    (#%app (#%top . newline))
    (#%app (#%top . +) foo (#%datum . 1)))

Why does it seem to do the right thing when expanded incrementally,
but not when it's expanded all at once?

Thanks for your help.

--Doug Orleans
dougo@ccs.neu.edu