On this page:
8.1 Demodularizing Libraries
8.2 Language for Demodularizing

8 raco demod: Demodularizing Programs🔗

The raco demodularize command (usually used with the shorthand raco demod) takes a Racket module and flattens its dependencies into a single compiled module, potentially with submodules. A file "name.rkt" is demodularized into "name_rkt_merged.zo".

See compiler/demod for an alternative way to use the demodularizer. Using #lang compiler/demod can cooperate with tools like raco make and raco setup, which is especially important for library modules (as opposed to end-user programs).

In its default configuration, raco demod supports flattening a module that represents an end-user program, so it discards all syntax and compile-time support in the module and its dependencies. Submodules are preserved, but their syntax and compile-time support are similarly discarded. The demodularized ".zo" file can be run by passing it as an argument to the racket command-line program, or it can be turned into an executable with raco exe.

Supply the -s or --syntax flag to preserve syntax and compile-time components of the module, so that it can be required the same as the original module. In that case, modules whose instances need to be shared with other librraies should be omitted from the demodularization using -x or --exclude-library. For example, -x racket/base is normally needed.

A large single module generated by the demodularizer may trigger size limits in the compiler that prevent whole-module optimizations. Set the PLT_CS_COMPILE_LIMIT environment variable to raise the limit, and check 'info logging at the 'linklet topic (e.g., set PLTSTDERR to info@linklet) for information about when compilation is restricted to smaller functions.

The raco demod command accepts these flags:

In addition to preserving submodules or of the source module, demodularization may introduce new submodules to hold portions of the flattening. The introduced submodules have names demod-pane- followed by an integer.

Changed in version 1.10 of package compiler-lib: Added -M/--compile-any, --work, and support for Racket CS.
Changed in version 1.15: Added -x/--exclude-library, -s/--syntax, --dump, --dump-mi, --prune-definitions (as a new name for --garbage-collect), and preservation of submodules.

8.1 Demodularizing Libraries🔗

Demodularization of a library module with compiler/demod can create a module whose meaning is different than the original, since transitive dependencies (that are not specified as excluded) are copied into the flattened module. That copying can break sharing as needed for generated structure types or bindings. As a specific example, separate copies of racket/base will have distinct and incompatible implementations of keyword arguments for procedures.

To avoid problems, a good general strategy for flattening is

8.2 Language for Demodularizing🔗

 #lang compiler/demod package: compiler-lib

A module using compiler/demod language compiles to a form that is the flattened (in the same sense as raco demod) version of a source module. See also Demodularizing Libraries.

A #lang compiler/demod module body starts with a module-path to flatten, it may be followed by options:


option = mode
  | #:include (mod-spec ...)
  | #:exclude (mod-spec ...)
  | #:submodule-include (submod-spec ...)
  | #:submodule-exclude (submod-spec ...)
  | #:prune-definitions
  | #:dump file
  | #:dump-mi file
  | #:no-demod
mode = #:exe
  | #:dynamic
  | #:static
mod-spec = #:module module-path
  | #:dir dir-path
  | #:collect collect-name
submod-spec = identifier
  | (identifier ...)

The default mode is #:dynamic, which preserves syntax objects and compile-time support (like macros), but does not insist that all modules are copied into the flattened module. For example, if a module is referenced by a combination of submodules within module-path and no other module is reached by the same combination, then the benefit of copying the module into a submodule is limited. The #:static mode is like #:dynamic, but it ensures that all modules are included unless they are specified as excluded. The #:exe mode discards syntax and compile-time support, so it may be suitable for flattening a module that implements an end-user program.

When the #:include option is specified, then only modules covered by a mod-spec will be included in the flattened form; otherwise, all modules are candidates for inclusion. When the #:exclude option is specified, the modules covered by the mod-specs are excluded, even if they would otherwise be included according to a #:include specification. In other words, #:exclude is applied after #:include.

The #:submodule-include and #:submodule-exclude specifications are analogous to #:include and #:exclude, but for submodules immediately with #racket[_module-path]. If mode is #:exe, then the list of inclusions defaults to main and configure-runtime, otherwise the default is to have no specific inclusions.

A mod-spec either indicates a specific module with #:module or it indicates all modules in a given collection (and its subcollections) with #:collect. A collect-name is always a string with /-separated components.

If the #:prune-definitions option is specified, then unused definitions from the original module and its dependencies are more aggressively pruned, but unsoundly. When syntax is preserved for #:dynamic or #:static mode, then all definitions are normally preserved from the original module, because they might be reachable via datum->syntax; when #:prune-definitions is specified, a definition can be pruned if no syntax object literal includes an identifier bound to the definition. Meanwhile, in all modes including #:exe, a definition is normally preserved if its right-hand side might have a side effect, but #:prune-definitions allows pruning on the unchecked assumption that a definition has no side effect. Due to its unchecked assumptions, #:prune-definitions may not preserve the behavior of the input module. As an example of where #:prune-definitions can go wrong, a module could export a macro that expands to a use of syntax-parse, and that use could include a : shorthand to combine a pattern variable and a syntax class (also defined in the module) as one identifier. The identifier would be split into variable and syntax-class components only when the macro is used, so the shorthand does not count as a literal that is bound to the syntax class. In that particular situation, use ~var instead of the shorthand, and then the syntax class is referenced by its own identifier. Meanwhile, a macro that is not exported (directly or indirectly through another macro) can safely use the : shorthand, since its expansions are part of the module’s implementation.

If the #:no-demod option is specified, then mod-spec is not flattened, after all. Instead, the new module requires and reprovides mod-spec and each of its submodules. This mode is always used when a compiler/demod module is expanded, since expansion must produce syntax instead of a compiled module. This mode also may be useful during for development to avoid longer compile times from flattening or to check whether copying of modules for flattening creates any trouble.

A flattened module using compiler/demod has a build dependency on the original module, so a tool like raco make or raco setup will trigger reflattening if the source module changes, but the flattened module does not have a run-time or expand-time dependency on the original module. Modules excluded from the flattening via #:include and #:exclude remain as run-time and expand-time dependencies of the flattened module. In the default #:dynamic mode, additional dependencies may be preserved for modules that cannot be usefully merged, but #:static or #:exe mode copies even those modules into new submodules.

Compilation and expansion of a compiler/demod module creates a "compiled/demod" subdirectory in the same directory as the module. That subdirectory that holds freshly compiled versions of all dependencies of the flattened module in a form that is suitable for demodularization. This extra compilation is managed using compiler/cm, so changes to dependencies can be handled incrementally, but still separate from normal compilation of the dependencies.

Added in version 1.15 of package compiler-lib.