Issue DEFINING-MACROS-NON-TOP-LEVEL Writeup

Forum:		Compiler

Issue:		DEFINING-MACROS-NON-TOP-LEVEL

References:	CLtL p. 66-70, 114, 143

		Issue EVAL-WHEN-NON-TOP-LEVEL

		Issue COMPILE-FILE-HANDLING-OF-TOP-LEVEL-FORMS

		Issue COMPILER-LET-CONFUSION

Category:	CLARIFICATION, ENHANCEMENT, CHANGE

Edit History:   6-May-88, V1 by Sandra Loosemore

		9-Jun-88, V2 by Sandra Loosemore

		12-Sep-88, V3 by Sandra Loosemore (fix garbled section 4)

                21-Sep-88, V4 by Sandra Loosemore (clarify section 5)

		16-Dec-88, V5 by Sandra Loosemore (major restructuring)

		31-Dec-88, V6 by Sandra Loosemore (wording clarifications)

		07-Jan-89, V7 by Sandra Loosemore (add example)

		09-Mar-89, V8 by Sandra Loosemore (more restructuring)

		22-Mar-89, V9 by Sandra Loosemore (add MACROLET stuff)

Status:		Ready for release



Problem Description:


CLtL leaves the interpretation of defining forms such as DEFMACRO and

DEFVAR that appear in other than top-level locations unclear.


On page 66, it is stated: "It is not illegal to use these forms at

other than top level, but whether it is meaningful to do so depends on

context.  Compilers, for example, may not recognize these forms

properly in other than top-level contexts".  At least one implementation 

has interpreted this to mean that it is permissible to simply refuse

to compile defining macros that do not appear at top-level.


A further problem is that CLtL p. 145 states that macro functions are

always defined in the null lexical environment.  These semantics would

require it to be a special form, not a macro, since there is no

possible macro expansion that has equivalent semantics under the

processing model presented in issue EVAL-WHEN-NON-TOP-LEVEL.  Under

that model, it ought to be possible for DEFMACRO to be implemented as

expanding into an EVAL-WHEN.  Furthermore, the macro function should

appear in a for-evaluation position in the expansion, such as

(function (lambda ...)).



Proposal DEFINING-MACROS-NON-TOP-LEVEL:ALLOW:


(1) Remove the language from p. 66 of CLtL quoted above.  Clarify that

while defining macros normally appear at top level, it is meaningful

to place them in non-top-level contexts and that the compiler must

handle them properly in all situations.  However, the compile-time side

effects described in issue COMPILE-FILE-HANDLING-OF-TOP-LEVEL-FORMS 

only take place when the defining macros appear at top-level.


(2) Remove the language from p. 145 of CLtL referenced above.  Clarify

that all defining macros which create functional objects (including

DEFMACRO, DEFTYPE, DEFINE-SETF-METHOD, and the complex form of

DEFSETF, as well as DEFUN) must ensure that those functions are

defined in the lexical environment in which the defining form appears.


(3) Change the description of MACROLET to indicate that the macro

functions it creates are defined in the lexical environment in which

the MACROLET form appears, and not the null lexical environment.

State that declarations and MACROLET and SYMBOL-MACROLET definitions

affect the local macro definitions in a MACROLET, but that the

consequences are undefined if the local macro definitions reference

any local variable or function bindings that are visible in that

lexical environment.



Rationale:


Items (1) and (2) make the rules for when defining macros cause

compile-time side effects to be exactly the same as the rules for when

(EVAL-WHEN (COMPILE) ...) causes compile-time evaluation.  This

provides a simple implementation technique.


Item (3) makes the handling of MACROLET macro definitions consistent

with DEFMACRO macro definitions.



Current Practice:


Most implementations do allow defining macros in non-top-level places.

However, the rules for when they cause compile-time side-effects are

not always the same as those for EVAL-WHEN.  This is the case in

Lucid Common Lisp, for example.


Lucid evaluates DEFMACRO macro functions in the lexical environment

in which the DEFMACRO appears, but always evaluates MACROLET macro

functions in the null lexical environment.



Cost to implementors:


Implementations that currently don't compile defining macros correctly

when they appear at non-top-level will have to be changed.


There will also be changes required to support compile-time definition

of functions in a non-null lexical environment.  These changes

are required to support defining macros such as DEFINE-SETF-METHOD

that require functional objects to be created at compile-time, as well

as to support the changes to DEFMACRO and MACROLET.  (Note that even

though defining macros cause compile-time evaluation only at

top-level, top-levelness does not necessarily imply a null lexical

environment under proposal EVAL-WHEN-NON-TOP-LEVEL:GENERALIZE-EVAL.)



Cost to users:


The changes to DEFMACRO and the other defining macros probably will

cause few problems for users.  Since CLtL didn't require non-top-level

defining macros to be meaningful, assigning them a meaning is a

compatible extension.


The changes to MACROLET may cause problems for users who have assumed

that, within local macro definitions, global function and variable

definitions are not shadowed by local function and variable bindings.

Code-walking programs will also have to be changed to reflect the new

semantics (see issue SYNTACTIC-ENVIRONMENT-ACCESS).



Benefits:


The notion of defining macros as being somehow special when they

appear at top-level is removed, since their behavior can be explained

using EVAL-WHEN as a primitive.  Allowing defining macros to appear

anywhere instead of restricting them to certain positions results in a

cleaner language design.



Discussion:


This proposal is consistent with the behavior specified in proposal

EVAL-WHEN-NON-TOP-LEVEL:GENERALIZE-EVAL.  In particular, if the compile

time side-effects for defining macros specified in proposal

COMPILE-FILE-HANDLING-OF-TOP-LEVEL-FORMS:CLARIFY are implemented using

EVAL-WHEN, the "right" compiler behavior for defining macros at

non-top-level will happen automatically.


The problems with compile-time definition of functions in a non-null

environment could be avoided by modifying proposal

DEFINING-MACROS-NON-TOP-LEVEL to remove the special treatment for

MACROLET, SYMBOL-MACROLET, and LOCALLY.