Racket is excellent for incrementally growing scripts into full-fledged programs. This post steps through the evolution of one small program and highlights the Racket tools that enable incremental advances.

My favorite part of Racket is how it supports a certain development style of evolving scripts into programs. When I start coding (after design, before debugging), I can focus the problem at hand. Next come examples, unit tests, and types to be sure the solution is correct. Finally, I worry about aesthetics, efficiency, and how the solution can be used as a library in a larger context.

Bottom line: with Racket, my coding is aligned with my priorities. And as I transition from "no code" to "working code" to "robust code" to "re-usable code", the program is almost always runnable.

Problem: A KWIC Index Production System

A KWIC index system reads input from a file, divides each line of the file into whitespace-separated words, and outputs (in alphabetical order) all circular shifts of all lines.

The first circular shift of a line "A B C" is the line "B C A". The second circular shift is "C A B".

Building a KWIC index is a historical problem. According to D.L. Parnas (1972):

Except under extreme circumstances (huge data base, no supporting software) such a system could be implemented by a good programmer within a week or two.

See also: Yannis’s Law.

Today, I bet only Agda and Scratch programmers would need the full two weeks. We’ll be done in 20 minutes.

A Script

To start, open a file and type:

You can name the file anything, like kwic.rkt or rkt.kwic or foo. Racket doesn’t care, but it does need the #lang line to read the contents of the file.

Though, you should use the .rkt extension.

The first part of the solution is a function to read input from a file into a list of strings for further processing. The built-in function file->lines does exactly this, but we’ll for-loop instead.

When called with a filename like "heart-of-darkness.txt", the function uses for/list to build a list of lines by reading from a port with in-lines. The port is the data from filename, thanks to with-input-from-file.

Next is a function to convert a list of strings into a list of lists of words. Here we’ll just use library functions.

By default, string-split divides a string into a list of whitespace-separated substrings. You can always supply a different delimiter, or use regexp-split to divide by a regular expression.

Two tasks left! First we generate all circular shifts for a list of strings words by folding up a list with one shift of words for each word.

Second, we alphabetize and print the shifts.

Gluing it all together, here’s the full script (with type annotations in comments).

Running the file should print:

Testing and Submodules

Any top-level expressions in a file can work as unit tests. The equal? statement below checks whether the first circular shift of '("A" "B" "C") is '("B" "C" "A").

Running the file now prints #t to the console, meaning the test passed. We can use error or raise-user-error to make failures easier to notice. Or we can use the RackUnit testing library.

These tests run each time the module does. If you prefer to run tests only in a specific context, and not when the module is run or imported as a library, you can move them to a separate file or into a submodule.

Running the module normally via racket kwic.rkt will not run code in the submodule. Instead, use raco test to run the tests.

The reason we used module+, instead of Racket’s module and module* forms is that module+ inherits the language and namespace of its containing module and can be incrementally extended. This way, we can keep tests near the relevant code.

RackUnit in a separate file or test submodule is the unofficial standard for testing Racket programs.

Recognizing Patterns, Avoiding Repetition

Every unit test we’ve written uses check-equal?.

These tests follow a simple pattern that we can express as a syntax rule.

The ... are not pseudocode! They denote Kleene-star repetition, like a sextile ("*") in a regular expression. In this case, the input pattern is a sequence of lists with two S-expressions, i and o. Uses of i and o in the rule must be followed by one ... to splice the captured S-expressions into the result.

Many languages offer higher-order functions and polymorphism to abstract common behaviors. Syntax extensions are a different way to avoid repeating yourself. After 30 years, we are still discovering what syntax extensions are useful for.

See this recent Racket mailing list post for some applications.

Adding Static Types

Changing the #lang line to typed/racket adds static type-checking to our program. If we only change the language and run the code as-is, there will be type errors. But we can use submodules again to incrementally check our design with types.

Note: typed regions are another way to embed typed code into untyped contexts.

After scooping all functions into the Typed Racket bubble, we can remove the submodule declaration and change #lang racket to #lang typed/racket.

Finally, a Library

Other modules can import our functions if we use a provide statement. By convention, exports belong at the top of a file.

Then any typed or untyped module can use kwic-index by writing (require "kwic.rkt").

As a finishing touch, we can use the racket/cmdline library inside a main submodule to give a basic front-end interface. Similar to module+ test, a module+ main declares code that inherits the file’s bindings and language but is only run when the program is executaed.

Here is the complete typed and tested code listing. The main submodule is at the bottom.

Closing

We started with functions, wrote (and quarantined) unit tests, reinforced our design with types, and added a command-line interface. Going forward we could add Scribble documentation and share our work as a package.