For files in &clisp; binary distributions, see .

#P"*.d" The source files for unpreprocessed &c-lang; code. &c-file; The &c-lang; code after preprocessing; also the result of compiling some &ffi-pac; forms (see &ffi-out-fun;). &lisp-file; The source files for Lisp code. #P"*.fas" Compiled lisp code (platform-independent &bytecode;s). &lib-file; Lisp header, produced by &compile-file; and used by &require;

&c-lang; sources are pre-processed with the following tools before being passed to the &c-lang; compiler: utils/comment5.c Convert &sh;-style comments (lines starting with "# ") to &c-lang;-style comments (/**/). The use of &sh;-style comments is deprecated. utils/varbrace.d Add braces to &c-lang; source code, so that variable declarations (introduced with the pseudo-keyword var) can be used within blocks, like in C++ and C99. utils/ccpaux.c When &cpp; cannot handle indented directives, remove the indentation. utils/gctrigger.d Add GCTRIGGER statements at the head of function bodies (for functions marked with the maygc pseudo-keyword). utils/deema.c When &cpp; cannot handle empty macro arguments, insert _EMA_ instead. utils/ccmp2c.c For the &new-clx; module only. Allows &cpp;-style preprocessing &before-e; &modprep; processing. Should be merged into &modprep; eventually. utils/modprep.lisp For some modules only, see . utils/unicode/ Generate NLS files in src/. utils/unicode/UnicodeDataFull.txt Character descriptions from &unicode32;; used by &describe-my; and installed in data/ under &libdir;.

src/lispbibl.d main include file src/fsubr.d list of all built-in special forms src/subr.d list of all built-in functions src/pseudofun.d list of all pseudo functions src/constpack.d list of packages accessed by &c-lang; code src/constsym.d list of symbols accessed by &c-lang; code src/constobj.d list of miscellaneous objects accessed by &c-lang; code src/unix.d include file for the &unix; implementations src/win32.d include file for the &win32; based versions src/xthread.d include file for thread support src/modules.h list of foreign &module;s

src/spvw.d Memory management (&gc;ion), startup; some OS interface. src/avl.d An implementation of AVL (Adelson-Velskii and Landis) trees. src/sort.d A sorting routine. src/spvwtabf.d The table of built-in special operators and functions. src/spvwtabs.d The table of all &symbol-t;s accessed by &c-lang; code. src/spvwtabo.d The table of miscellaneous objects accessed by &c-lang; code. src/eval.d Evaluator (form interpreter) and &bytecode; interpreter. src/bytecode.d List of &bytecode;s. src/lightning.c Just-in-time compiler using &lightning;. src/control.d Special operator interpreter. src/pathname.d Pathnames, file- and directory-related functions. src/stream.d &stream-t;s of all kinds: &file-stream-t;s, terminal streams, &string-stream-t;s etc. src/socket.d Opening &sock;s for &tcp;/&ip; and &clx;. src/io.d The lisp reader (parser) and printer (including pretty printer). src/array.d Functions dealing with &array-t;s and &vector-t;s. src/hashtabl.d Functions dealing with &hash-table-t;s. src/list.d Functions dealing with &list-t;s. src/package.d Functions dealing with &package-t;s. src/record.d Functions dealing with records (structures, closures, etc.) src/sequence.d The generic &sequence-t; functions. src/funarg.d Functional arguments, like &test-k; and &key-k;. src/charstrg.d Functions dealing with &character-t;s and &string-t;s. src/debug.d The &debugger; and the &repl;. src/error.d &error-t; handling and &signal;ing. src/errunix.d &unix;-specific error messages. src/errwin32.d &win32;-specific error messages. src/misc.d Miscellaneous functions. src/time.d Timing functions. src/predtype.d Predicates, type tests. src/symbol.d Functions dealing with &symbol-t;s. src/unixaux.d Auxiliary functions (&unix; version only). src/win32aux.d Auxiliary functions (&win32; version only). src/foreign.d &ffi-pac; implementation. src/lisparit.d Functions dealing with numbers (arithmetic), see . src/noreadline.d Dummy plug-in for the &readline; library. src/zthread.d MT implementation.

src/lisparit.d initialization, input/output of numbers, lisp functions src/aridecl.d declarations src/arilev0.d arithmetic at the machine level src/arilev1.d digit sequences src/arilev1c.d operations on digit sequences, written in &c-lang; src/arilev1i.d operations on digit sequences, as inline functions src/arilev1e.d operations on digit sequences, bindings to external routines src/intelem.d &integer-t;s: elementary operations src/intlog.d &integer-t;s: logical connectives src/intplus.d &integer-t;s: addition and subtraction src/intcomp.d &integer-t;s: comparison src/intbyte.d &integer-t;s: byte operations &ldb;, &dpb; src/intmal.d &integer-t;s: multiplication src/intdiv.d &integer-t;s: division src/intgcd.d &integer-t;s: &gcd; and &lcm; src/int2adic.d &integer-t;s: operations on 2-adic integers src/intsqrt.d &integer-t;s: square root, n-th root src/intprint.d subroutines for &integer-t; output src/intread.d subroutines for &integer-t; input src/rational.d rational numbers (&ratio-t;s) src/sfloat.d elementary operations for &short-float-t;s src/ffloat.d elementary operations for &single-float-t;s src/dfloat.d elementary operations for &double-float-t;s src/lfloat.d elementary operations for &long-float-t;s src/flo_konv.d conversions between &float-t;s src/flo_rest.d general &float-t; operations src/realelem.d elementary functions for &real-t; numbers src/realrand.d random numbers src/realtran.d transcendental functions for &real-t; numbers src/compelem.d elementary functions for &complex-t; numbers src/comptran.d transcendental functions for &complex-t; numbers

src/ari68000.d written in 68000 assembler, MIT syntax src/ari68020.d written in 68020 assembler, MIT syntax src/arisparc.d written in SPARC assembler src/arisparc64.d written in 64-bit SPARC assembler src/ari80386.d written in i386/i486 assembler src/arimips.d written in MIPS assembler src/arimips64.d written in 64-bit MIPS assembler src/arihppa.d written in HPPA-1.0 assembler src/arivaxunix.d written in VAX assembler, Unix assembler syntax src/ariarm.d written in ARM assembler

src/sp68000.d written in 68000 assembler, MIT syntax src/spsparc.d written in SPARC assembler src/spsparc64.d written in 64-bit SPARC assembler src/sp80386.d written in i386/i486 assembler src/spmips.d written in MIPS assembler

src/asmi386.sh converts i386 assembler from MIT syntax to a macro syntax src/asmi386.hh expands i386 assembler in macro syntax to either MIT or Intel syntax

src/init.lisp the first file to be loaded during bootstrapping, loads everything else src/defseq.lisp defines the usual sequence types for the generic sequence functions src/backquote.lisp implements the backquote read macro src/defmacro.lisp implements &defmacro; src/macros1.lisp the most important macros src/macros2.lisp some other macros src/defs1.lisp miscellaneous definitions src/timezone.lisp site-dependent definition of time zone, except for &unix; and &win32;. src/places.lisp macros using &place;s, definitions of most standard and extensiion &place;s src/floatprint.lisp defines &write-float-decimal; for printing floating point numbers in base 10 src/type.lisp functions working with &typespec-glo;s: &typep;, &subtypep; src/defstruct.lisp implements the macro &defstruct; src/format.lisp implements the function &format; src/room.lisp implements the function &room; (see also ) src/savemem.lisp see src/xcharin.lisp (optional) implements extended character input for &with-kbd; src/keyboard.lisp (optional) implements the macro &with-kbd; src/runprog.lisp implements the functions &run-prog;, &run-cmd; etc. src/query.lisp implements the functions &y-or-n-p; and &yes-or-no-p; src/reploop.lisp support for the &debugger; and the &repl; src/dribble.lisp implements the functions &dribble; and EXT:DRIBBLE-STREAM src/complete.lisp implements completion, see . src/describe.lisp implements functions &describe-my;, &apropos-my;, &apropos-list; src/trace.lisp tracer src/macros3.lisp (optional) macros &letf;, &letf-star;, ðe;, &with-collect;, function &compiled-file-p;. src/config.lisp site-dependent configuration, should be a user-modified copy of one of the following: src/cfgunix.lisp for &unix; src/cfgwin32.lisp for the &win32; See . src/compiler.lisp compiles Lisp code to &bytecode; src/functions.lisp &function-lambda-expression; et al src/disassem.lisp the function &disassemble; src/defs2.lisp miscellaneous &ansi-cl; definitions src/loop.lisp implements the &ansi-cl;-compatible &loop; macro src/clos.lisp loads the various parts of the &clos;: src/clos-package.lisp declares the imports and exports of the &clos-pac; package src/clos-macros.lisp defines some internal macros used by the &clos; implementation src/clos-class0.lisp defines the class-version structure src/clos-metaobject1.lisp defines the &metaobject-t; class src/clos-slotdef1.lisp defines the &slot-definition-t; class and its subclasses src/clos-slotdef2.lisp defines &initialize-instance; methods for &slot-definition-t; and its subclasses src/clos-slotdef3.lisp defines the generic functions that can be used on &slot-definition-t; objects src/clos-stablehash1.lisp defines the &standard-stablehash; class src/clos-stablehash2.lisp defines &initialize-instance; methods for &standard-stablehash; src/clos-specializer1.lisp defines the &specializer-t; class and its subclasses src/clos-specializer2.lisp defines &initialize-instance; methods for &specializer-t; and its subclasses src/clos-specializer3.lisp defines the generic functions that can be used on &specializer-t; objects src/clos-class1.lisp defines the potential-class class and its subclasses src/clos-class2.lisp implements the mapping from class names to classes src/clos-class3.lisp implements the &defclass; macro, class definition and class redefinition src/clos-class4.lisp defines &initialize-instance; methods for potential-class and its subclasses src/clos-class5.lisp implements the special logic of &make-instance;, &initialize-instance; etc. src/clos-class6.lisp defines the generic functions that can be used on potential-class objects src/clos-method1.lisp defines the &method-t; class and its subclasses src/clos-method2.lisp implements the bulk of &defmethod; src/clos-method3.lisp defines the generic functions that can be used on &method-t; objects src/clos-method4.lisp makes generic functions on &standard-method-t; objects extensible src/clos-methcomb1.lisp defines the &method-combination-t; class src/clos-methcomb2.lisp implements method combination (part 2 of generic function dispatch and execution) and the &define-method-combination; macro src/clos-methcomb3.lisp defines &initialize-instance; methods for &method-combination-t; src/clos-methcomb4.lisp makes generic functions on &method-combination-t; objects extensible src/clos-genfun1.lisp defines the &generic-function-t; class and its metaclass, superclass and subclasses src/clos-genfun2a.lisp implements part 1 of generic function dispatch and execution src/clos-genfun2b.lisp implements part 3 of generic function dispatch and execution src/clos-genfun3.lisp implements creation of generic function objects, &defmethod;, &defgeneric; src/clos-genfun4.lisp defines &initialize-instance; methods for &generic-function-t; and its subclasses src/clos-genfun5.lisp makes generic functions on &generic-function-t; objects extensible src/clos-slots1.lisp implements low-level slot access, &with-slots;, &with-accessors; src/clos-slots2.lisp defines the generic functions that deal with slot access src/clos-dependent.lisp implements notification from metaobjects to dependent objects src/clos-print.lisp implements the function &print-object; src/clos-custom.lisp provides user customization of the &clos; src/gray.lisp implements &gray-streams; src/fill-out.lisp implements &fill-stream; src/disassem.lisp implements &disassemble; src/condition.lisp implements the &cl; Condition System (&clcs;) src/gstream.lisp (optional) generic stream default methods, see src/foreign1.lisp &ffi-pac; implementation. src/screen.lisp the screen access package, see src/edit.lisp (optional) the screen editor (&ed;), &uncompile; src/inspect.lisp implements &inspect-my; (tty and &http; frontends) src/clhs.lisp implements &open-http;, &browse-url; src/exporting.lisp Macros that export their definienda, see . src/threads.lisp MT interface src/german.lisp src/french.lisp src/spanish.lisp src/russian.lisp src/danish.lisp src/dutch.lisp i18n user messages

modules/ individual external module sources

src/NEWS the list of the user-visible changes src/_README master for the distribution's README src/_README.en src/_README.de src/_README.es translations of src/_README doc/clisp.xml.in &docbook;/&xml; sources for the &clisp; manual page build-dir/clisp.1 the platform-specific man manual page, generated from doc/clisp.xml.in at build time build-dir/clisp.html the platform-specific &html; manual page, generated from doc/clisp.xml.in at build time doc/impnotes.xml.in the master &docbook;/&xml; file for these implementation notes; includes the following files doc/cl-ent.xml &clisp;-independent general &cl;-related entities doc/clhs-ent.xml generated list of &hyperspec; entities doc/impent.xml &clisp;-specific entities doc/unix-ent.xml &unix;-related entities doc/mop-ent.xml &mop;-related entities doc/impbody.xml most of doc/impissue.xml doc/gray.xml doc/mop.xml doc/impext.xml and doc/impbyte.xml this doc/faq.xml modules/**/*.xml individual external module documentation doc/Symbol-Table.text the mapping between lisp symbols and element IDs in these notes (see &describe-my;), installed in data/ under &libdir;. doc/impnotes.html these &html; implementation notes, generated from doc/impnotes.xml.in at release time

These files are usually updated a couple of weeks before a &clisp; release using make -f Makefile.devel tp-mail, see also . We use the Translation project and the above command sends the updated files to the translators. src/po/*.pot list of translatable messages (portable object template) src/po/*.po translated messages (portable objects) src/po/*.gmo translated messages (&gnu; format message objects)

configure the main configuration script version.sh specifies the current &clisp; version src/configure.in lists features to be checked src/m4/ a repertoire of features. Use with &autoconf; 2.62 src/m4/clisp.m4 This file defines the macro CL_CLISP which takes two optional parameters: features-list The list of &clisp; features to check. E.g., passing [foo bar] results in cl_cv_clisp_FOO and cl_cv_clisp_BAR being defined to yes or no, depending on the value of &features-my;, i.e., on the return value of (&read-from-string; "#+FOO \"yes\" #-FOO \"no\""). required Determines whether &configure; should fail if &clisp; or one of the requested features is missing. Should be true (the default) or false. Calling this macro causes the generated &configure; to accept the option --with-clisp="clisp command line" which allows one to use a specific &clisp; installation instead of &clisp-cmd; in PATH. In addition to the aforementioned per-feature variables, this macro defines the follwing variables: cl_cv_clisp_version The return value of &lisp-implementation-version;. cl_cv_clisp_libdir The &namestring; of &libdir;. cl_cv_clisp_linkset The directory of the current &linkset;. It also substitues the following variables: CLISP_LINKKIT CLISP_FILES CLISP_LIBS CLISP_CFLAGS CLISP_CPPFLAGS (taken from $cl_cv_clisp_linkset/makevars) and CLISP (the &clisp; command line). This file is installed in /usr/share/aclocal by make install. src/configure configuration script, generated from src/configure.in src/intparam.c figures out some machine parameters (word size, endianness etc.) src/floatparam.c figures out some floating point arithmetics parameters (rounding, epsilons etc.) src/config.h.in header file master, generated from src/configure.in. build-dir/config.h contains the values of the features discovered by src/configure. src/makemake.in makefile construction script master src/_clisp.c master for the distribution's driver program src/_distmakefile master for the distribution's Makefile

The externally maintained files are usually updated a couple of weeks before a &clisp; release using make -f Makefile.devel pre-release, see also . src/gllib/ src/glm4/ &c-lang; and &autoconf; files from &gnulib;; updated using make -f Makefile.devel gnulib-imported at top level. src/build-aux/ various shell scripts, updated using make -f Makefile.devel build-aux-update at top level.

These are internals, which are of interest only to the &clisp; developers. If you do not read clisp-devel, this chapter is probably not for you.

Knowing that most &malloc; implementations are buggy and/or slow, and because &clisp; needs to perform &gc;ion, &clisp; has its own memory management subsystem in files src/spvw*.d, see .

&clisp; data (the heap), i.e. storage which contains &clisp; &object-t;s and is managed by the &gc;or. &clisp; stack (called &STACK;), contains &clisp; &object-t;s visible to the &gc;or &c-lang; data (including program text, data, &malloc;ed memory) A &clisp; object is one word, containing a tag (partial type information) and either immediate data or a pointer to storage. Pointers to &c-lang; data have tag = machine_type = 0, pointers to &clisp; stack have tag = system_type, most other pointers point to &clisp; data. 32-bit CPU &fixnum-t; &short-float-t;&character-t; 64-bit CPU In addition to the above, &single-float-t; (with &typecodes;) Let us turn to those &clisp; objects that consume regular &clisp; memory. Every &clisp; object has a size which is determined when the object is allocated (using one of the allocate_*() routines). The size can be computed from the type tag and - if necessary - the length field of the object's header. The length field always contains the number of elements of the object. The number of bytes is given by the function objsize(). &clisp; objects which contain exactly 2 &clisp; objects (i.e. &cons-t;es, &complex-t; numbers, &ratio-t;s) are stored in a separate area and occupy 2 words each. All other &clisp; objects have varying length (more precisely, not a fixed length) and include a word for &gc;ion purposes at their beginning. The garbage collector is invoked every now and then by allocate_*() calls according to certain heuristics. It marks all objects which are alive (may be reached from the roots), compacts these objects and unmarks them. Non-live objects are lost; their storage is reclaimed. 2-pointer objects are compacted by a simple hole-filling algorithm: fill the left-most object into the right-most hole, and so on, until the objects are contiguous at the right and the hole is contiguous at the left. Variable-length objects are compacted by sliding them down (their address decreases).

&clisp; implements two ways of representing object pointers. (An object pointer, &c-lang; type &object-t;, contains a pointer to the memory location of the object, or - for &immediate-o; - all bits of the object itself.) Both of them have some things in common: There is a distinction between &immediate-o;s (&character-t;s, &fixnum-t;s, &short-float-t;s, etc) and heap allocated objects. All object pointers are typed, i.e. contain a few bits of information about the type of the pointed-to object. At a minimum, these bits must allow to distinguish immediate and heap-allocated objects. Not all of the type information is contained in the object pointer. For example, &clos; objects can change their type when &change-class; is called. To avoid scanning all the heap for references when this happens, the class information is stored in the heap allocated object, not in the object pointer. The &heapcodes; object representation has a minimum of type bits in the object pointer, namely, 2 bits. They allow to distinguish &immediate-o;s (which have some more type bits), &cons-t;es (which have no type bits in the heap, since they occupy just two words in the heap, with no header), other heap objects (many, from &simple-vector-t;s to &foreign-pointer-t;s), and Subrs. Most object types are distinguished by looking a the rectype field in the header of the heap object. The &typecodes; object representation has about two dozen of types encoded in 6 or 7 bits in the object pointer. Typically these are the upper 8 bits of a word (on a 32-bit machine) or the upper 16 bits or 32 bits of a word (on a 64-bit machine). The particular values of the typecodes allow many common operations to be performed with a single bit test (e.g. &consp; and &minusp; for a &real-t; are bit tests) or range check. However, the rectype field still exists for many types, because there are many built-in types which do not need a particularly fast type test. Which object representation is chosen is decided at build time depending on the available preprocessor definitions. You can define &typecodes; or &heapcodes; to force one or the other. One might expect that &typecodes; is faster than &heapcodes; because it does not need to make as many memory accesses. This effect is, however, hardly measurable in practice (certainly not more than 5% faster). Apparently because, first, the situations where the type of an object is requested but then the object is not looked into are rare. It is much more common to look into an object, regardless of its type. Second, due to the existence of data caches in the CPU, accessing a heap location twice, once for the type test and then immediately afterwards for the data, is not significantly slower than just accessing the data. &typecodes; is problematic on 32-bit machines, when you want to use more than 16 MB of memory, because the type bits (at bit 31..24) interfere with the bits of a heap address. For this reason, &heapcodes; is the default on 32-bit platforms. &heapcodes; is problematic on platforms whose object alignment is less than 4. This affects only the mc680x0 CPU; however, here the alignment can usually be guaranteed through some &gcc; options.

There are 6 memory models. Which one is used, depends on the operating system and is determined at build time. SPVW_MIXED_BLOCKS_OPPOSITE The heap consists of one block of fixed length (allocated at startup). The variable-length objects are allocated from the left, the 2-pointer objects are allocated from the right. There is a hole between them. When the hole shrinks to 0, &gc; is invoked. &gc; slides the variable-length objects to the left and concentrates the 2-pointer objects at the right end of the block again. When no more room is available, some reserve area beyond the right end of the block is halved, and the 2-pointer objects are moved to the right accordingly. &advantages; (+) Simple management. (+) No fragmentation at all. (-) The total heap size is limited. SPVW_MIXED_BLOCKS_OPPOSITE & TRIVIALMAP_MEMORY The heap consists of two big blocks, one for variable-length objects and one for 2-pointer objects. The former one has a hole to the right and is extensible to the right, the latter one has a hole to the left and is extensible to the left. Similar to the previous model, except that the hole is unmapped. &advantages; (+) Total heap size grows depending on the application's needs. (+) No fragmentation at all. (*) Works only when SINGLEMAP_MEMORY is possible as well. SPVW_MIXED_BLOCKS_STAGGERED & TRIVIALMAP_MEMORY The heap consists of two big blocks, one for variable-length objects and one for 2-pointer objects. Both have a hole to the right, but are extensible to the right. &advantages; (+) Total heap size grows depending on the application's needs. (+) No fragmentation at all. (*) Works only when SINGLEMAP_MEMORY is possible as well. SPVW_MIXED_PAGES The heap consists of many small pages (usually around 8 KB). There are two kinds of pages: one for 2-pointer objects, one for variable-length objects. The set of all pages of a fixed kind is called a "Heap". Each page has its hole (free space) at its end. For every heap, the pages are kept sorted according to the size of their hole, using AVL trees. The &gc;ion is invoked when the used space has grown by 25% since the last GC; until that point new pages are allocated from the OS. The GC compacts the data in each page separately: data is moved to the left. Emptied pages are given back to the OS. If the holes then make up more than 25% of the occupied storage, a second GC turn moves objects across pages, from nearly empty ones to nearly full ones, with the aim to free as many pages as possible. &advantages; (-) Every allocation requires AVL tree operations, thus slower (+) Total heap size grows depending on the application's needs. (+) Works on operating systems which do not provide large contiguous areas. SPVW_PURE_PAGES Just like SPVW_MIXED_PAGES, except that every page contains data of only a single type tag, i.e. there is a Heap for every type tag. &advantages; (-) Every allocation requires AVL tree operations, thus slower (+) Total heap size grows depending on the application's needs. (+) Works on operating systems which do not provide large contiguous areas. (-) More fragmentation because objects of different type never fit into the same page. SPVW_PURE_BLOCKS There is a big block of storage for each type tag. Each of these blocks has its data to the left and the hole to the right, but these blocks are extensible to the right (because there is enough room between them). A &gc;ion is triggered when the allocation amount since the last GC reaches 50% of the amount of used space at the last GC, but at least 512 KB. The &gc;ion cleans up each block separately: data is moved left. &advantages; (+) Total heap size grows depending on the application's needs. (+) No 16 MB total size limit. (*) Works only in combination with SINGLEMAP_MEMORY. In page based memory models, an object larger than a page is the only object carried by its pages. There are no small objects in pages belonging to a big object. The following combinations of memory model and mmap tricks are possible (the number indicates the order in which the respective models have been developed): SPVW_MIXED_BLOCKS_OPPOSITE1 98SPVW_MIXED_BLOCKS_STAGGERED 67SPVW_PURE_BLOCKS 45SPVW_MIXED_PAGES 2SPVW_PURE_PAGES 3 SPVW_MIXED_BLOCKS_OPPOSITE ***SPVW_MIXED_BLOCKS_STAGGERED **SPVW_MIXED_PAGES* no MAP_MEMORY TRIVIALMAP_MEMORY SINGLEMAP_MEMORY GENERATIONAL_GC

Every subroutine marked with can trigger GC or maygc may invoke &gc;ion. The &gc;or moves all the &clisp; non-&immediate-o;s and updates the pointers. But the &gc;or looks only at the &STACK; and not in the &c-lang; variables. (Anything else would not be portable.) Therefore at every "unsafe" point, i.e. every call to such a subroutine, all the &c-lang; variables of type &object-t; MUST BE ASSUMED TO BECOME GARBAGE. (Except for &object-t;s that are known to be unmovable, e.g. &immediate-o;s or Subrs.) Pointers inside &clisp; data (e.g. to the characters of a &string-t; or to the elements of a &simple-vector-t;) become INVALID as well.

The workaround is usually to allocate all the needed &clisp; data first and do the rest of the computation with &c-lang; variables, without calling unsafe routines, and without worrying about &gc;ion. Alternatively, you can save a lisp &object-t; on the &STACK; using macros pushSTACK() and popSTACK(). One should ¬-e; mix these macros with functions which modify &STACK; in one statement because &c-lang; may execute different parts of the statement out of order. E.g.,pushSTACK(listof(4)); is illegal while pushSTACK(STACK_0); is legal.

Run-time GC-safety checking is available when you build &clisp; with a C++ compiler, e.g.: &sh-prompt; ./configure 'CC=g++' &with-debug; build-g-gxx When built like this, &clisp; will abort when you reference GC-unsafe data after an allocation (which could have triggered a &gc;ion), and &gdb; will pinpoint the trouble spot. Specifically, when &clisp; is configured as above, there is a global integer variable alloccount and the &object-t; structure contains an integer allocstamp slot. If these two integers are not the same, the &object-t; is invalid. By playing with &gdb;, you should be able to figure out the precise spot where an allocation increments alloccount &after-e; the object has been retrieved from a GC-visible location.

Generational &gc;or uses memory protection, so when passing pointers into the lisp heap to &c-lang; functions, you may encounter errors (&errno;=EFAULT) unless you call handle_fault_range(protection,region_start,region_end) on the appropriate memory region. See files src/unixaux.d src/win32aux.d modules/syscalls/calls.c modules/rawsock/rawsock.c for examples.

Pointers to &c-lang; functions and to &malloc;ed data can be hidden in &clisp; objects of type machine_type; &gc; will not modify its value. But one should not dare to assume that a &c-lang; stack pointer or the address of a &c-lang; function in a shared library satisfies the same requirements. If another pointer is to be viewed as a &clisp; object, it is best to box it, e.g. in a &simple-bit-vector-t; or in an Fpointer (using allocate_fpointer().)

While the &gc;ion is executing, all other threads must stop. &clisp; has a copying &gc;or, so anything else would require a write barrier during the scan phase and a read barrier during the move phase. Heap objects may be pinned - they will not move during &gc;ion. Used when execution is blocked in a foreign/system call and a pointer into the heap is passed to non-lisp land. The &gc;or tries to minimize the holes in the heap introduced by the pinning process. See also the note about system calls in . The threads may be stopped only at certain safe points. Currently implemented safe points are: Any heap allocation (actually before the allocation itself). Possibly blocking system call. Some places that may introduce infinite non-consing loops. Basically any thread that conses or can block in a system call will not stop the &gc;ion from executing. There are still places where infinite non-consing loop without safe point may be reached (TODO: find and fix)

You are urged to use External Modules instead of adding built-in functions. &clisp; comes with an &ffi-pac; which allows you to access &c-lang; libraries in an easy way (including creating &foreign-function;s dynamically).

In the rare cases when you really need to modify &clisp; internals and add a truly built-in function, you should read the &clisp; sources for inspiration and enlightenment, choose a file where your brand-new built-in function should go to, and then ... add the LISPFUN form and the implementation there add the LISPFUN header to file &subr-d; declare the function name in file &constsym-d; in the appropriate package (probably &ext-pac;, if there is no specific package) if your function accepts keyword arguments, then you must make sure that the keyword symbols are declared in &constsym-d; see the example in for argument passing and returning values export your function name from the appropriate package in file &init-lisp; when you are done, you should run make check-sources in your build directory: this will check that the definitions (source files) and the declarations (&subr-d; and &fsubr-d;) are in sync. Be very careful with the GC-unsafe functions! Always remember about GC-safety! These instructions are intentionally terse - you are encouraged to use &module;s and/or &ffi-pac; instead of adding built-ins directly.

If you must be able to access the Lisp variable in the &c-lang; code, follow these steps: declare the variable name in &constsym-d; in the appropriate package (probably &custom-pac;, if there is no specific package); add a define_variable() call in function init_symbol_values() in file &spvw-d; export your variable name from the appropriate package in file &init-lisp;

Any change that forces &make; to remake &lisp-run;, will force recompilation of all &lisp-file; files and re-dumping of &lispinit;, which may be time-consuming. This is not always necessary, depending on what kind of change you introduced. On the other hand, if you change any of the following files: &constobj-d; &constsym-d; &fsubr-d; &subr-d; your &lispinit; will have to be re-dumped. If you change the signature of any system function mentioned in the &FUNTAB;* arrays in file &eval-d;, all the &fasl-file; files will become obsolete and will need to be recompiled. You will need to add a note to that effect to the &news; file and augment the object version in file &constobj-d;. Please try to avoid this as much as possible.

Last modified: 1998-09-19

The &clisp; compiler compiles &cl; programs into instruction codes for a virtual processor. This bytecode is optimized for saving space in the most common cases of &cl; programs. The main advantages/drawbacks of this approach, compared to native code compilation, are: Bytecode compiled programs are a lot smaller than when compiled to native code. This results in better use of CPU caches, and in less virtual memory paging. Users perceive this as good responsiveness. Maximum execution speed (throughput in tight loops) is limited. Since no bytecode instructions are provided for unsafe operations (like unchecked array accesses, or fast &car;/&cdr;), programs run with all safety checks enabled even when compiled. Execution speed of a program can easily be understood by looking at the output of the &disassemble; function. A rule of thumb is that every elementary instruction costs 1 time unit, whereas a function call costs 3 to 4 time units. Needing to do no type inference, the compiler is pretty straightforward and fast. As a consequence, the definition of &clos; generic functions, which needs to compile small pieces of generated code, is not perceived to be slow. The compiler is independent from the hardware CPU. Different back-ends, one for each hardware CPU, are not needed. As a consequence, the compiler is fairly small (and would have been easily maintainable if it were written in a less kludgey way...), and it is impossible for the compiler writer to introduce CPU dependent bugs.

The bytecode can be thought of as being interpreted by a virtual processor. The engine which interprets the bytecode (the implementation of the virtual machine) is either a &c-lang; function (interpret_bytecode in &eval-d;), or a just-in-time compiler which translates a function's bytecode into hardware CPU instructions the first time said function is called, see . The virtual machine is a stack machine with two stacks: &STACK; a stack for &clisp; objects and frames (Lisp stack). &SP; a stack for other data and pointers (Program stack). This two-stack architecture permits to save an unlimited number of &clisp; objects on the &STACK; (needed for handling of &cl; &mul-val;), without &consing;. Also, in a world with a compacting no-ambiguous-roots garbage collector, &STACK; must only hold &clisp; objects, and &SP; can hold all the other data belonging to a frame, which would not fit into &STACK; without tagging/untagging overhead. The scope of &STACK; and &SP; is only valid for a given function invocation. Whereas the amount of &STACK; space needed for executing a function (excluding other function calls) is unlimited, the amount of &SP; space needed is known a priori, at compile time. When a function is called, no relation is specified between the caller's &STACK; and the callee's &STACK;, and between the caller's &SP; and the callee's &SP;. The bytecode is designed so that outgoing arguments on the caller's &STACK; can be shared by the caller's incoming arguments area (on the callee's &STACK;), but a virtual machine implementation may also copy outgoing arguments to incoming arguments instead of sharing them. The virtual machine has a special data structure, &mvalues;, containing the top of stack, specially adapted to &cl; &mul-val;: &mv-count; an unsigned integer. &value1; the &pri-val;, a &clisp; object. If &mv-count; = 0, this is &nil;. &mv-space; all values except the first one, an array of &clisp; objects. The contents of &mvalues; is short-lived. It does not survive a function call, not even a &gc;ion. The interpretation of some bytecode instructions depends on a constant, &jbs;. This is a CPU-dependent number, the value of SYSTEM::*JMPBUF-SIZE*. In &c-lang;, it is defined as ceiling(sizeof(jmp_buf),sizeof(void*)).

A compiled function consists of two objects: The function itself, containing the references to all &clisp; objects needed for the bytecode, and a byte vector containing only immediate data, including the bytecode proper. Typically, the byte vector is about twice as large as the function vector. The separation thus helps the garbage collector (since the byte vector does not need to be scanned for pointers). A function looks like this (cf. the &c-lang; type Cclosure): name This is the name of the function, normally a symbol or a list of the form (&setf; &symbol-r;). It is used for printing the function and for error messages. This field is immutable. &codevec; This is the byte-code vector, a &ubyte-vec;. This field is immutable. &consts;[] The remaining fields in the function object are references to other &clisp; objects. These references are immutable, which is why they are called constants. (The referenced &clisp; objects can be mutable objects, such as &cons-t;es or &vector-t;s, however.) When a generic function's dispatch code is installed, the &codevec; and &consts; fields are destructively modified. Some of the &consts; can play special roles. A function looks like this, in more detail: name see . &codevec; see . &venv-const;* At most one object, representing the closed-up variables, representing the variables of the &lex-env; in which this function was defined. It is a &simple-vector-t;, which looks like this: #(&next-r; value&sub-1; ... value&sub-n;) where value&sub-1;, ..., value&sub-n; are the values of the closed-up variables, and &next-r; is either &nil; or a &simple-vector-t; having the same structure. &block-const;* Objects representing closed-up █ tags, representing the █ tags of the &lex-env; in which this function was defined. Each is a &cons-t; containing in the &cdr; part: either a frame pointer to the block frame, or &disabled;. The &car; is the block's name, for error messages only. &tagbody-const;* Objects representing closed-up &tagbody; tags, representing the &tagbody; tags of the &lex-env; in which this function was defined. Each is a &cons-t; containing in the &cdr; part: either a frame pointer to the &tagbody; frame, or &disabled; if the &tagbody; has already been left. The &car; is a &simple-vector-t; containing the names of the &tagbody; tags, for error messages only. &keyword-const;* If the function was defined with a &lalist; containing &key-amp;, here come the symbols ("keywords"), in their correct order. They are used by the interpreter during function call. &other-const;* Other objects needed by the function's bytecode. If &venv-const;, &block-const;, &tagbody-const; are all absent, the function is called autonomous. This is the case if the function does not refer to lexical variables, blocks or tags defined in compile code outside of the function. In particular, it is the case if the function is defined in a null &lex-env;. If some &venv-const;, &block-const;, or &tagbody-const; are present, the function (a closure) is created at runtime. The compiler only generates a prototype, containing &nil; values instead of each &venv-const;, &block-const;, &tagbody-const;. At runtime, a function is created by copying this prototype and replacing the &nil; values by the definitive ones. The list (&keyword-const;* &other-const;*) normally does not contain duplicates, because the compiler removes duplicates when possible. (Duplicates can occur nevertheless, through the use of &load-time-value;.) The &codevec; looks like this (cf. the &c-lang; type Codevec): spdepth_1 (2 bytes) The 1st part of the maximal &SP; depth. spdepth_jmpbufsize (2 bytes) The &jbs; part of the maximal &SP; depth. The maximal &SP; depth (precomputed by the compiler) is given by spdepth_1 + spdepth_jmpbufsize * &jbs;. &numreq; (2 bytes) Number of required parameters. numopt (2 bytes) Number of optional parameters. &flags; (1 byte) bit 0 set if the function has the &rest-amp; parameter bit 7 set if the function has &key-amp; parameters bit 6 set if the function has &allow-other-keys-amp; bit 4 set if the function is a generic function bit 3 set if the function is a generic function and its effective method shall be returned (instead of being executed) bit 2 set if the full original &lalist; is kept in the closure object, see bit 1 set if the &function-doc; &documentation; &string-t; is kept in the closure object, see signature (1 byte) An abbreviation code depending on &numreq;, numopt, &flags;. It is used for speeding up the function call. numkey (2 bytes, only if the function has &key-amp;) The number of &key-amp; parameters. keyconsts (2 bytes, only if the function has &key-amp;) The offset of the &keyword-const; in the function. byte* (any number of bytes) The bytecode instructions.

All instructions consist of one byte, denoting the opcode, and some number of operands. The conversion from a byte (in the range 0..255) to the opcode is performed by lookup in the table contained in the file &bytecode-d;. There are the following types of operands, denoted by different letters: &k-r;, &n-r;, &m-r;, &l-r; A (nonnegative) numeric operand. The next byte is read. If its bit 7 is zero, then the bits 6..0 give the value (7 bits). If its bit 7 is one, then the bits 6..0 and the subsequent byte together form the value (15 bits). &b-r; A (nonnegative) 1-byte operand. The next byte is read and is the value. label A label operand. A signed numeric operand is read: The next byte is read. If its bit 7 is zero, then the bits 6..0 give the value (7 bits, sign-extended). If its bit 7 is one, then the bits 6..0 and the subsequent byte together form the value (15 bits, sign-extended). If the latter 15-bit result is zero, then four more bytes are read and put together (32 bits, sign-extended). Finally, the bytecode pointer for the target is computed as the current bytecode pointer (pointing after the operand just read), plus the signed numeric operand.

&mnem-desc-sem-header; (&nil;) Load &nil; into &mvalues;. &value1; := &nil;, &mv-count; := 1 (PUSH-NIL &n-r;) Push &n-r; &nil;s into the &STACK;. &n-r; times do: *--&STACK; := &nil;, &undef-mvalues; (&t;) Load &t; into &mvalues;. &value1; := &t;, &mv-count; := 1 (CONST &n-r;) Load the function's &n-r;th constant into &mvalues;. &value1; := &consts;[&n-r;], &mv-count; := 1

&mnem-desc-sem-header; (LOAD &n-r;) Load a directly accessible local variable into &mvalues;. &value1; := *(&STACK;+&n-r;), &mv-count; := 1 (LOADI &k1-r; &k2-r; &n-r;) Load an indirectly accessible local variable into &mvalues;. &k-r; := &k1-r; + &jbs; * &k2-r;, &value1; := *(*(&SP;+&k-r;)+ &n-r;), &mv-count; := 1 (LOADC &n-r; &m-r;) Load a closed-up variable, defined in the same function and directly accessible, into &mvalues;. &value1; := &svref;(*(&STACK;+&n-r;),1+&m-r;), &mv-count; := 1 (LOADV &k-r; &m-r;) Load a closed-up variable, defined in an outer function, into &mvalues;. &v-r; := &venv-const;, &m-r; times do: &v-r; := &svref;(&v-r;,0), &value1; := &svref;(&v-r;,&m-r;), &mv-count; := 1 (LOADIC &k1-r; &k2-r; &n-r; &m-r;) Load a closed-up variable, defined in the same function and indirectly accessible, into &mvalues;. &k-r; := &k1-r; + &jbs; * &k2-r;, &value1; := &svref;(*(*(&SP;+&k-r;)+&n-r;),1+&m-r;), &mv-count; := 1 (STORE &n-r;) Store values into a directly accessible local variable. *(&STACK;+&n-r;) := &value1;, &mv-count; := 1 (STOREI &k1-r; &k2-r; &n-r;) Store values into an indirectly accessible local variable. &k-r; := &k1-r; + &jbs; * &k2-r;, *(*(&SP;+&k-r;)+ &n-r;) := &value1;, &mv-count; := 1 (STOREC &n-r; &m-r;) Store values into a closed-up variable, defined in the same function and directly accessible. &svref;(*(&STACK;+&n-r;),1+m) := &value1;, &mv-count; := 1 (STOREV &k-r; &m-r;) Store values into a closed-up variable, defined in an outer function. &v-r; := &venv-const;, &m-r; times do: &v-r; := &svref;(&v-r;,0), &svref;(&v-r;,&m-r;) := &value1;, &mv-count; := 1 (STOREIC &k1-r; &k2-r; &n-r; &m-r;) Store values into a closed-up variable, defined in the same function and indirectly accessible. &k-r; := &k1-r; + &jbs; * &k2-r;, &svref;(*(*(&SP;+&k-r;)+&n-r;),1+&m-r;) := &value1;, &mv-count; := 1

&mnem-desc-sem-header; (GETVALUE &n-r;) Load a symbol's value into &mvalues;. &value1; := symbol-value(&consts;[&n-r;]), &mv-count; := 1 (SETVALUE &n-r;) Store values into a symbol's value. symbol-value(&consts;[&n-r;]) := &value1;, &mv-count; := 1 (BIND &n-r;) Bind a symbol dynamically. Bind the value of the symbol &consts;[&n-r;] to &value1;, implicitly &STACK; -= 3, &undef-mvalues; (UNBIND1) Dissolve one binding frame. Unbind the binding frame &STACK; is pointing to, implicitly &STACK; += 3 (UNBIND &n-r;) Dissolve &n-r; binding frames. &n-r; times do: Unbind the binding frame &STACK; is pointing to, thereby incrementing &STACK; Thus, &STACK; += 1+2*&n-r; (PROGV) Bind a set of symbols dynamically to a set of values. symbols := *&STACK;++, *--&SP; := &STACK;, build a single binding frame binding the symbols in symbols to the values in &value1;, &undef-mvalues;

&mnem-desc-sem-header; (PUSH) Push one object onto the &STACK;. *--&STACK; := &value1;, &undef-mvalues; (POP) Pop one object from the &STACK;, into &mvalues;. &value1; := *&STACK;++, &mv-count; := 1 (SKIP &n-r;) Restore a previous &STACK; pointer. Remove &n-r; objects from the &STACK;. &STACK; := &STACK; + &n-r; (SKIPI &k1-r; &k2-r; &n-r;) Restore a previous &STACK; pointer. Remove an unknown number of objects from the &STACK;. &k-r; := &k1-r; + &jbs; * &k2-r;, &STACK; := *(&SP;+&k-r;), &SP; := &SP;+&k-r;+1, &STACK; := &STACK; + &n-r; (SKIPSP &k1-r; &k2-r;) Restore a previous &SP; pointer. &k-r; := &k1-r; + &jbs; * &k2-r;, &SP; := &SP;+&k-r;

&mnem-desc-sem-header; (SKIP&RET &n-r;) Clean up the &STACK;, and return from the function. &STACK; := &STACK;+&n-r;, return from the function, returning values. (SKIP&RETGF &n-r;) Clean up the &STACK;, and return from the generic function. If bit 3 is set in the function's &flags;, then &STACK; := &STACK;+&n-r;, &mv-count; := 1, and return from the function. Otherwise: if the current function has no &rest-amp; argument, then &STACK; := &STACK;+&n-r;-&numreq;, apply &value1; to the &numreq; arguments still on the &STACK;, and return from the function. Else &STACK; := &STACK;+&n-r;-&numreq;-1, apply &value1; to the &numreq; arguments and the &rest-amp; argument, all still on the &STACK;, and return from the function. (JMP &lab-r;) Jump to &lab-r;. PC := &lab-r;. (JMPIF &lab-r;) Jump to &lab-r;, if &value1; is true. If &value1; is not &nil;, PC := &lab-r;. (JMPIFNOT &lab-r;) Jump to &lab-r;, if &value1; is false. If &value1; is &nil;, PC := &lab-r;. (JMPIF1 &lab-r;) Jump to &lab-r; and forget secondary values, if &value1; is true. If &value1; is not &nil;, &mv-count; := 1, PC := &lab-r;. (JMPIFNOT1 &lab-r;) Jump to &lab-r; and forget secondary values, if &value1; is false. If &value1; is &nil;, &mv-count; := 1, PC := &lab-r;. (JMPIFATOM &lab-r;) Jump to &lab-r;, if &value1; is not a cons. If &value1; is not a cons, PC := &lab-r;. &undef-mvalues; (JMPIFCONSP &lab-r;) Jump to &lab-r;, if &value1; is a cons. If &value1; is a cons, PC := &lab-r;. &undef-mvalues; (JMPIFEQ &lab-r;) Jump to &lab-r;, if &value1; is &eq; to the top-of-stack. If eq(&value1;,*&STACK;++), PC := &lab-r;. &undef-mvalues; (JMPIFNOTEQ &lab-r;) Jump to &lab-r;, if &value1; is not &eq; to the top-of-stack. If not eq(&value1;,*&STACK;++), PC := &lab-r;. &undef-mvalues; (JMPIFEQTO &n-r; &lab-r;) Jump to &lab-r;, if the top-of-stack is &eq; to a constant. If eq(*&STACK;++,&consts;[&n-r;]), PC := &lab-r;. &undef-mvalues; (JMPIFNOTEQTO &n-r; &lab-r;) Jump to &lab-r;, if the top-of-stack is not &eq; to a constant. If not eq(*&STACK;++,&consts;[&n-r;]), PC := &lab-r;. &undef-mvalues; (JMPHASH &n-r; &lab-r;) Table-driven jump, depending on &value1;. Lookup &value1; in the hash table &consts;[&n-r;]. (The hash table's test is either &eq; or &eql;.) If found, the hash table value is a signed &fixnum-t;, jump to it: PC := PC + value. Else jump to &lab-r;. &undef-mvalues; (JMPHASHV &n-r; &lab-r;) Table-driven jump, depending on &value1;, inside a generic function. Lookup &value1; in the hash table &svref;(&consts;[0],&n-r;). (The hash table's test is either &eq; or &eql;.) If found, the hash table value is a signed &fixnum-t;, jump to it: PC := PC + value. Else jump to &lab-r;. &undef-mvalues; (JSR &lab-r;) Subroutine call. *--&STACK; := function. Then start interpreting the bytecode at &lab-r;, with &mvalues; undefined. When a (RET) is encountered, program execution is resumed at the instruction after (JSR &lab-r;). (JMPTAIL &m-r; &n-r; &lab-r;) Tail subroutine call. &n-r; ≥ &m-r;. The &STACK; frame of size &n-r; is reduced to size &m-r;: {*(&STACK;+&n-r;-&m-r;), ..., *(&STACK;+&n-r;-1)} := {*&STACK;, ..., *(&STACK;+&m-r;-1)}. &STACK; += n-m. *--&STACK; := function. Then jump to &lab-r;, with &mvalues; undefined.

&mnem-desc-sem-header; (VENV) Load the &venv-const; into &mvalues;. &value1; := &consts;[0], &mv-count; := 1. (MAKE-VECTOR1&PUSH &n-r;) Create a &simple-vector-t; used for closed-up variables. &v-r; := new &simple-vector-t; of size &n-r;+1. &svref;(&v-r;,0) := &value1;. *--&STACK; := &v-r;. &undef-mvalues; (COPY-CLOSURE &m-r; &n-r;) Create a closure by copying the prototype and filling in the &lex-env;. &f-r; := copy-function(&consts;[&m-r;]). For &i-r;=0,..,&n-r;-1: &f-r;_&consts;[i] := *(&STACK;+&n-r;-1-&i-r;). &STACK; += &n-r;. &value1; := &f-r;, &mv-count; := 1

&mnem-desc-sem-header; (CALL &k-r; &n-r;) Calls a constant function with &k-r; arguments. The function &consts;[&n-r;] is called with the arguments *(&STACK;+&k-r;-1), ..., *(&STACK;+0). &STACK; += &k-r;. &ret-mvalues; (CALL0 &n-r;) Calls a constant function with 0 arguments. The function &consts;[&n-r;] is called with 0 arguments. &ret-mvalues; (CALL1 &n-r;) Calls a constant function with 1 argument. The function &consts;[&n-r;] is called with one argument *&STACK;. &STACK; += 1. &ret-mvalues; (CALL2 &n-r;) Calls a constant function with 2 arguments. The function &consts;[&n-r;] is called with two arguments *(&STACK;+1) and *(&STACK;+0). &STACK; += 2. &ret-mvalues; (CALLS1 &b-r;) Calls a system function with no &rest-amp;. Calls the system function &FUNTAB;[&b-r;]. The right number of arguments is already on the &STACK; (including &unbound;s in place of absent &optional-amp; or &key-amp; parameters). &clear-stack; &ret-mvalues; (CALLS2 &b-r;) Calls a system function with no &rest-amp;. Calls the system function &FUNTAB;[256+&b-r;]. The right number of arguments is already on the &STACK; (including &unbound;s in place of absent &optional-amp; or &key-amp; parameters). &clear-stack; &ret-mvalues; (CALLSR &m-r; &b-r;) Calls a system function with &rest-amp;. Calls the system function &FUNTAB;R[&b-r;]. The minimum number of arguments is already on the &STACK;, and &m-r; additional arguments as well. &clear-stack; &ret-mvalues; (CALLC) Calls a computed compiled function with no &key-amp;s. Calls the compiled function &value1;. The right number of arguments is already on the &STACK; (including &unbound;s in place of absent &optional-amp; parameters). &clear-stack; &ret-mvalues; (CALLCKEY) Calls a computed compiled function with &key-amp;s. Calls the compiled function &value1;. The right number of arguments is already on the &STACK; (including &unbound;s in place of absent &optional-amp; or &key-amp; parameters). &clear-stack; &ret-mvalues; (FUNCALL &n-r;) Calls a computed function. Calls the function *(&STACK;+&n-r;) with the arguments *(&STACK;+&n-r;-1), ..., *(&STACK;+0). &STACK; += &n-r;+1. &ret-mvalues; (APPLY &n-r;) Calls a computed function with an unknown number of arguments. Calls the function *(&STACK;+&n-r;) with the arguments *(&STACK;+&n-r;-1), ..., *(&STACK;+0) and a list of additional arguments &value1;. &STACK; += &n-r;+1. &ret-mvalues;

&mnem-desc-sem-header; (PUSH-UNBOUND &n-r;) Push &n-r; &unbound;s into the &STACK;. &n-r; times do: *--&STACK; := &unbound;. &undef-mvalues; (UNLIST &n-r; &m-r;) Destructure a proper &list-t;. 0 ≤ &m-r; ≤ &n-r;. &n-r; times do: *--&STACK; := &car;(&value1;), &value1; := &cdr;(&value1;). During the last &m-r; iterations, the list &value1; may already have reached its end; in this case, *--&STACK; := &unbound;. At the end, &value1; must be &nil;. &undef-mvalues; (UNLIST* &n-r; &m-r;) Destructure a proper or dotted &list-t;. 0 ≤ &m-r; ≤ &n-r;, &n-r; > 0. &n-r; times do: *--&STACK; := &car;(&value1;), &value1; := &cdr;(&value1;). During the last &m-r; iterations, the list &value1; may already have reached its end; in this case, *--&STACK; := &unbound;. At the end, after &n-r; &cdr;s, *--&STACK; := &value1;. &undef-mvalues; (JMPIFBOUNDP &n-r; &lab-r;) Jump to &lab-r;, if a local variable is not unbound. If *(&STACK;+&n-r;) is not &unbound;, &value1; := *(&STACK;+&n-r;), &mv-count; := 1, PC := &lab-r;. Else: &undef-mvalues;. (BOUNDP &n-r;) Load &t; or &nil; into &mvalues;, depending on whether a local variable is bound. If *(&STACK;+&n-r;) is not &unbound;, &value1; := &t;, &mv-count; := 1. Else: &value1; := &nil;, &mv-count; := 1. (UNBOUND->NIL &n-r;) If a local variable is unbound, assign a default value &nil; to it. If *(&STACK;+&n-r;) is &unbound;, *(&STACK;+&n-r;) := &nil;.

&mnem-desc-sem-header; (VALUES0) Load no values into &mvalues;. &value1; := &nil;, &mv-count; := 0 (VALUES1) Forget secondary values. &mv-count; := 1 (&STACK;-TO-MV &n-r;) Pop the first &n-r; objects from &STACK; into &mvalues;. Load values(*(&STACK;+&n-r;-1),...,*(&STACK;+0)) into values. &STACK; += &n-r;. (MV-TO-&STACK;) Save values on &STACK;. Push the &mv-count; values onto the &STACK; (in order: &value1; comes first). &STACK; -= &mv-count;. &undef-mvalues; (NV-TO-&STACK; &n-r;) Save &n-r; values on &STACK;. Push the first &n-r; values onto the &STACK; (in order: &value1; comes first). &STACK; -= &n-r;. &undef-mvalues; (MV-TO-LIST) Convert &mul-val; into a list. &value1; := list of values, &mv-count; := 1 (LIST-TO-MV) Convert a &list-t; into &mul-val;. Call the function &values-list; with &value1; as argument. &ret-mvalues; (MVCALLP) Start a &multiple-value-call; invocation. *--&SP; := &STACK;. *--&STACK; := &value1;. (MVCALL) Finish a &multiple-value-call; invocation. newSTACK := *&SP;++. Call the function *(newSTACK-1), passing it *(newSTACK-2), ..., *(&STACK;+0) as arguments. &STACK; := newSTACK. &ret-mvalues;

&mnem-desc-sem-header; (BLOCK-OPEN &n-r; &lab-r;) Create a █ frame. Create a █ frame, &STACK; -= 3, &SP; -= 2+&jbs;. The topmost (third) object in the block frame is &cons;(&consts;[&n-r;],frame-pointer) (its &bc-r;). Upon a &return-from; to this frame, execution will continue at &lab-r;. &undef-mvalues;. (BLOCK-CLOSE) Dissolve a █ frame. Dissolve the █ frame at &STACK;, &STACK; += 3, &SP; += 2+&jbs;. Mark the &bc-r; as invalid. (RETURN-FROM &n-r;) Leave a █ whose &bc-r; is given. &bc-r; := &consts;[&n-r;]. If &cdr;(&bc-r;) = &disabled;, an &err-sig;. Else &cdr;(&bc-r;) is a frame-pointer. Unwind the stack up to this frame, pass it values. (RETURN-FROM-I &k1-r; &k2-r; &n-r;) Leave a █ whose &bc-r; is indirectly accessible. &k-r; := &k1-r; + &jbs; * &k2-r;, &bc-r; := *(*(&SP;+&k-r;)+&n-r;). If &cdr;(&bc-r;) = &disabled;, an &err-sig;. Else &cdr;(&bc-r;) is a frame-pointer. Unwind the stack up to this frame, pass it values.

&mnem-desc-sem-header; (TAGBODY-OPEN &m-r; label&sub-1; ... label&sub-n;) Create a &tagbody; frame. Fetch &consts;[&m-r;], this is a &simple-vector-t; with &n-r; elements, then decode &n-r; label operands. Create a &tagbody; frame, &STACK; -= 3+&n-r;, &SP; -= 1+&jbs;. The third object in the &tagbody; frame is &cons;(&consts;[&m-r;],frame-pointer) (the &tbc-r;) Upon a &go; to tag &lab-r; of this frame, execution will continue at label&sub-l;. &undef-mvalues; (TAGBODY-CLOSE-NIL) Dissolve a &tagbody; frame, and load &nil; into &mvalues;. Dissolve the &tagbody; frame at &STACK;, &STACK; += 3+&m-r;, &SP; += 1+&jbs;. Mark the &tbc-r; as invalid. &value1; := &nil;, &mv-count; := 1. (TAGBODY-CLOSE) Dissolve a &tagbody; frame. Dissolve the &tagbody; frame at &STACK;, &STACK; += 3+&m-r;, &SP; += 1+&jbs;. Mark the &tbc-r; as invalid. (GO &n-r; &lab-r;) Jump into a &tagbody; whose &tbc-r; is given. &tbc-r; := &consts;[&n-r;]. If &cdr;(&tbc-r;) = &disabled;, an &err-sig;. Else &cdr;(&tbc-r;) is a frame-pointer. Unwind the stack up to this frame, pass it the number &lab-r;. (GO-I &k1-r; &k2-r; &n-r; &lab-r;) Jump into a &tagbody; whose &tbc-r; is indirectly accessible. &k-r; := &k1-r; + &jbs; * &k2-r;, &tbc-r; := *(*(&SP;+&k-r;)+&n-r;). If &cdr;(&tbc-r;) = &disabled;, an &err-sig;. Else &cdr;(&tbc-r;) is a frame-pointer. Unwind the stack up to this frame, pass it the number &lab-r;.

&mnem-desc-sem-header; (CATCH-OPEN &lab-r;) Create a &catch; frame. Create a &catch; frame, with &value1; as tag. &STACK; -= 3, &SP; -= 2+&jbs;. Upon a &throw; to this tag execution continues at &lab-r;. (CATCH-CLOSE) Dissolve a &catch; frame. Dissolve the &catch; frame at &STACK;. &STACK; += 3, &SP; += 2+&jbs;. (THROW) Non-local exit to a &catch; frame. &tag-r; := *&STACK;++. Search the innermost &catch; frame with tag &tag-r; on the &STACK;, unwind the stack up to it, pass it values.

&mnem-desc-sem-header; (UNWIND-PROTECT-OPEN &lab-r;) Create an &unwind-protect; frame. Create an &unwind-protect; frame. &STACK; -= 2, &SP; -= 2+&jbs;. When the stack will be unwound by a non-local exit, values will be saved on &STACK;, and execution will be transferred to &lab-r;. (UNWIND-PROTECT-NORMAL-EXIT) Dissolve an &unwind-protect; frame, and start the cleanup code. Dissolve the &unwind-protect; frame at &STACK;. &STACK; += 2, &SP; += 2+&jbs;. *--&SP; := 0, *--&SP; := 0, *--&SP; := &STACK;. Save the values on the &STACK;, &STACK; -= &mv-count;. (UNWIND-PROTECT-CLOSE) Terminate the cleanup code. newSTACK := *&SP;++. Load values(*(newSTACK-1), ..., *(&STACK;+0)) into &mvalues;. &STACK; := newSTACK. SPword1 := *&SP;++, SPword2 := *&SP;++. Continue depending on SPword1 and SPword2. If both are 0, simply continue execution. If SPword2 is 0 but SPword1 is nonzero, interpret it as a label and jump to it. (UNWIND-PROTECT-CLEANUP) Dissolve an &unwind-protect; frame, and execute the cleanup code like a subroutine call. Dissolve the &unwind-protect; frame at &STACK;, get &lab-r; out of the frame. &STACK; += 2, &SP; += 2+&jbs;. *--&SP; := 0, *--&SP; := PC, *--&SP; := &STACK;. Save the values on the &STACK;, &STACK; -= &mv-count;. PC := &lab-r;.

&mnem-desc-sem-header; (HANDLER-OPEN &n-r;) Create a handler frame. Create a handler frame, using &consts;[&n-r;] which contains the &condition-t; types, the corresponding labels and the current &SP; depth (= function entry &SP; - current &SP;). (HANDLER-BEGIN&PUSH) Start a handler. Restore the same &SP; state as after the HANDLER-OPEN. &value1; := the &condition-t; that was passed to the handler, &mv-count; := 1. *--&STACK; := &value1;.

&mnem-desc-sem-header; (NOT) Inlined call to ¬-f;. &value1; := not(&value1;), &mv-count; := 1. (EQ) Inlined call to &eq;. &value1; := eq(*&STACK;++,&value1;), &mv-count; := 1. (CAR) Inlined call to &car;. &value1; := &car;(&value1;), &mv-count; := 1. (CDR) Inlined call to &cdr;. &value1; := &cdr;(&value1;), &mv-count; := 1. (CONS) Inlined call to &cons;. &value1; := cons(*&STACK;++,&value1;), &mv-count; := 1. (SYMBOL-FUNCTION) Inlined call to &symbol-function;. &value1; := &symbol-function;(&value1;), &mv-count; := 1. (SVREF) Inlined call to &svref;. &value1; := &svref;(*&STACK;++,&value1;), &mv-count; := 1. (SVSET) Inlined call to (&setf; &svref;. arg1 := *(&STACK;+1), arg2 := *(&STACK;+0), &STACK; += 2. &svref;(arg2,&value1;) := arg1. &value1; := arg1, &mv-count; := 1. (LIST &n-r;) Inlined call to &list;. &value1; := &list;(*(&STACK;+&n-r;-1),...,*(&STACK;+0)), &mv-count; := 1, &STACK; += &n-r;. (LIST* &n-r;) Inlined call to &list-star;. &value1; := &list-star;(*(&STACK;+&n-r;-1),..., *(&STACK;+0),&value1;), &mv-count; := 1, &STACK; += &n-r;.

The most frequent short sequences of instructions have an equivalent combined instruction. They are only present for space and speed optimization. The only exception is FUNCALL&SKIP&RETGF, which is needed for generic functions. mnemonicequivalent (NIL&PUSH) (NIL) (PUSH) (T&PUSH) (T) (PUSH) (CONST&PUSH &n-r;) (CONST &n-r;) (PUSH) (LOAD&PUSH &n-r;) (LOAD &n-r;) (PUSH) (LOADI&PUSH &k1-r; &k2-r; &n-r;) (LOADI &k1-r; &k2-r; &n-r;) (PUSH) (LOADC&PUSH &n-r; &m-r;) (LOADC &n-r; &m-r;) (PUSH) (LOADV&PUSH &k-r; &m-r;) (LOADV &k-r; &m-r;) (PUSH) (POP&STORE &n-r;) (POP) (STORE &n-r;) (GETVALUE&PUSH &n-r;) (GETVALUE &n-r;) (PUSH) (JSR&PUSH &lab-r;) (JSR &lab-r;) (PUSH) (COPY-CLOSURE&PUSH &m-r; &n-r;) (COPY-CLOSURE &m-r; &n-r;) (PUSH) (CALL&PUSH &k-r; &n-r;) (CALL &k-r; &n-r;) (PUSH) (CALL1&PUSH &n-r;) (CALL1 &n-r;) (PUSH) (CALL2&PUSH &n-r;) (CALL2 &n-r;) (PUSH) (CALLS1&PUSH &b-r;) (CALLS1 &b-r;) (PUSH) (CALLS2&PUSH &b-r;) (CALLS2 &b-r;) (PUSH) (CALLSR&PUSH &m-r; &n-r;) (CALLSR &m-r; &n-r;) (PUSH) (CALLC&PUSH) (CALLC) (PUSH) (CALLCKEY&PUSH) (CALLCKEY) (PUSH) (FUNCALL&PUSH &n-r;) (FUNCALL &n-r;) (PUSH) (APPLY&PUSH &n-r;) (APPLY &n-r;) (PUSH) (CAR&PUSH) (CAR) (PUSH) (CDR&PUSH) (CDR) (PUSH) (CONS&PUSH) (CONS) (PUSH) (LIST&PUSH &n-r;) (LIST &n-r;) (PUSH) (LIST*&PUSH &n-r;) (LIST* &n-r;) (PUSH) (NIL&STORE &n-r;) (NIL) (STORE &n-r;) (T&STORE &n-r;) (T) (STORE &n-r;) (LOAD&STOREC &k-r; &n-r; &m-r;) (LOAD &k-r;) (STOREC &n-r; &m-r;) (CALLS1&STORE &b-r; &k-r;) (CALLS1 &b-r;) (STORE &k-r;) (CALLS2&STORE &b-r; &k-r;) (CALLS2 &b-r;) (STORE &k-r;) (CALLSR&STORE &m-r; &n-r; &k-r;) (CALLSR &m-r; &n-r;) (STORE &k-r;) (LOAD&CDR&STORE &n-r;) (LOAD &n-r;) (CDR) (STORE &n-r;) (LOAD&CONS&STORE &n-r;) (LOAD &n-r;+1) (CONS) (STORE &n-r;) (LOAD&INC&STORE &n-r;) (LOAD &n-r;) (CALL1 #'1+) (STORE &n-r;) (LOAD&DEC&STORE &n-r;) (LOAD &n-r;) (CALL1 #'1-) (STORE &n-r;) (LOAD&CAR&STORE &m-r; &n-r;) (LOAD &m-r;) (CAR) (STORE &n-r;) (CALL1&JMPIF &n-r; &lab-r;) (CALL1 &n-r;) (JMPIF &lab-r;) (CALL1&JMPIFNOT &n-r; &lab-r;) (CALL1 &n-r;) (JMPIFNOT &lab-r;) (CALL2&JMPIF &n-r; &lab-r;) (CALL2 &n-r;) (JMPIF &lab-r;) (CALL2&JMPIFNOT &n-r; &lab-r;) (CALL2 &n-r;) (JMPIFNOT &lab-r;) (CALLS1&JMPIF &b-r; &lab-r;) (CALLS1 &b-r;) (JMPIF &lab-r;) (CALLS1&JMPIFNOT &b-r; &lab-r;) (CALLS1 &b-r;) (JMPIFNOT &lab-r;) (CALLS2&JMPIF &b-r; &lab-r;) (CALLS2 &b-r;) (JMPIF &lab-r;) (CALLS2&JMPIFNOT &b-r; &lab-r;) (CALLS2 &b-r;) (JMPIFNOT &lab-r;) (CALLSR&JMPIF &m-r; &n-r; &lab-r;) (CALLSR &m-r; &n-r;) (JMPIF &lab-r;) (CALLSR&JMPIFNOT &m-r; &n-r; &lab-r;) (CALLSR &m-r; &n-r;) (JMPIFNOT &lab-r;) (LOAD&JMPIF &n-r; &lab-r;) (LOAD &n-r;) (JMPIF &lab-r;) (LOAD&JMPIFNOT &n-r; &lab-r;) (LOAD &n-r;) (JMPIFNOT &lab-r;) (LOAD&CAR&PUSH &n-r;) (LOAD &n-r;) (CAR) (PUSH) (LOAD&CDR&PUSH &n-r;) (LOAD &n-r;) (CDR) (PUSH) (LOAD&INC&PUSH &n-r;) (LOAD &n-r;) (CALL1 #'1+) (PUSH) (LOAD&DEC&PUSH &n-r;) (LOAD &n-r;) (CALL1 #'1-) (PUSH) (CONST&SYMBOL-FUNCTION &n-r;) (CONST &n-r;) (SYMBOL-FUNCTION) (CONST&SYMBOL-FUNCTION&PUSH &n-r;) (CONST &n-r;) (SYMBOL-FUNCTION) (PUSH) (CONST&SYMBOL-FUNCTION&STORE &n-r; &k-r;) (CONST &n-r;) (SYMBOL-FUNCTION) (STORE &k-r;) (APPLY&SKIP&RET &n-r; &k-r;) (APPLY &n-r;) (SKIP&RET &k-r;) (FUNCALL&SKIP&RETGF &n-r; &k-r;) (FUNCALL &n-r;) (SKIP&RETGF &k-r;)

There are special one-byte instructions (without explicit operands) for the following frequent instructions: mnemonicoperand range (LOAD &n-r;) 0 ≤ &n-r; < 15 (LOAD&PUSH &n-r;) 0 ≤ &n-r; < 25 (CONST &n-r;) 0 ≤ &n-r; < 21 (CONST&PUSH &n-r;) 0 ≤ &n-r; < 30 (STORE &n-r;) 0 ≤ &n-r; < 8

The functions described here are defined in src/compiler.lisp and src/record.d and can be used to examine the internals of a compiled closure. These function are internal &clisp; functions, their names are ¬-e; exported, this section is ¬-e; supposed to be comprehensive and is ¬-e; guaranteed to be up to date. It is intended for aspiring &clisp; hackers who are supposed to graduate to reading the sources right away. All others should stick with the &ansi-cl; function &disassemble;. The normal way to extract the name of a closure is &function-lambda-expression;: (defun my-plus-1 (x y) (declare (compile)) (+ x y)) MY-PLUS-1 (function-lambda-expression #'my-plus-1) (LAMBDA (X Y) (DECLARE (COMPILE)) (+ X Y)) ; &t; ; MY-PLUS-1 ;; works only on closure objects (sys::closure-name #'my-plus-1) MY-PLUS-1 The actual bytecode vector (if you modify it, you can get a segfault when the function is executed): (sys::closure-codevec #'my-plus-1) #(0 0 0 0 2 0 0 0 6 3 174 174 51 2 53 25 3) A closure can depend on external and internal values: (let ((x 123) (y 456)) (defun my-plus-2 (z) (declare (compile)) (+ x y z))) MY-PLUS-2 (sys::closure-consts #'my-plus-2) (#(Y 456 X 123 NIL) 3 1) Use &disassemble; to see how the constants are used. Function SYS::SIGNATURE returns 8 values: required parameter count optional parameters count rest-p key-p keyword-p allow-other-keys-p byte-list (codevec as a &list-t;) const-list One can convert between numeric and mnemonic bytecodes (LAP stands for Lisp Assembly Program): (multiple-value-bind (req-num opt-num rest-p key-p keyword-list allow-other-keys-p byte-list const-list) (sys::signature #'my-plus-1) (sys::disassemble-LAP byte-list const-list)) ((0 LOAD&PUSH 2) (1 LOAD&PUSH 2) (2 CALLSR 2 53) (5 SKIP&RET 3)) (sys::assemble-LAP (mapcar #'rest *)) (174 174 51 2 53 25 3)

This section offers some insight into bytecode design in the form of questions and answers.

Does it make sense to define a new bytecode instruction for &restart-case;? Why? Why not? &restart-case; is a glorified &let; binding for SYSTEM::*ACTIVE-RESTARTS* and could well profit from a separate bytecode: it would make it non-&consing;. (Remember that &restart-t;s have dynamic extent and therefore do not really need to be heap allocated.) The reason &handler-bind; has its own bytecodes and &restart-case; does not is that &handler-bind; can occur in inner computation loops, whereas &restart-case; occurs only as part of user-interface programming and therefore not in inner loops where its consing could hurt much.

Consider this function and its disassembly: (defun foo (x y) (if (or (= x 0) (= y 0)) (+ x y) (foo y (1- x)))) (&disassemble; 'foo) 8 (LOAD&PUSH 1) 9 (LOAD&DEC&PUSH 3) 11 (JMPTAIL 2 5 L0) Why are the arguments pushed onto the &STACK;, just to be popped off of it during the JMPTAIL? Why not a sequence of LOAD, STORE and SKIP instructions followed by a JMP? Using JMPTAIL requires 3 instructions, JMP requires more. When JMPTAIL needs to be called, we usually have some stuff close to the top of the &STACK; which will become the new arguments, and some junk between these new arguments and the closure object. JMPTAIL removes the junk. JMPTAIL is a convenient shortcut which shortens the bytecode - because typically one would really have to clean-up the &STACK; by hand or make the calculations in src/compiler.lisp more complicated.