Welcome To Mudballs

Introduction Welcome to Mudballs, the System Management Tool for Common Lisp.

Mudballs was written with the goal of providing an easy to use system for loading1 Common Lisp code into your image in a reliable and predictable manner. If you are new to Common Lisp, think of it as the start of a CL equivalent to Perls CPAN or Ruby's RubyGem's.

This introductory document is intended to get someone who is familiar with Common Lisp up and running with Mudballs, by the end of this document you will know how to install systems provided by others and define your own systems. We will start off with a brief Q and A and then will get into the meat of the document.
  1. Among other things.
Q & A
Q. Why is it called Mudballs? Mudballs gets it's name from a well known quote attributed to Joel Moses.
LISP is like a ball of mud. You can add any amount of mud to it and it still looks like a ball of mud.
Mudballs purpose is to provide this mud to the intrepid developer.
Q. Why not ASDF? ASDF, while providing a very similar service (albeit a subset) has a few shortcomings which limits it's effectiveness, here are my favorites.
  • Operations on components don't happen in the context of a call to the parent system.
  • Pervasive use of soft links makes Windows users second class citizens.
  • Inability to have multiple versions of the same system defined.
This is compounded by being rather resistant to change.
As it currently stands Mudballs uses syntax which is incompatible with ASDF but we do plan to provide as ASDF support package in the future.
Q. What implementations are supported Mudballs has been tested on the following implementations.
  • Lispworks on Mac OSX, Linux and Windows
  • SBCL on Mac OSX and Linux
  • CMUCL on Mac OSX and Linux
  • CLISP on Linux and Windows
  • OpenMCL/ClozureCL on Mac OSX
  • Allegro CL on Mac OSX, Linux and Windows
Installing Mudballs The first step to getting up and running is to get it installed. As with many things in the mudballs world this is wonderfully simple. Just follow these steps

  • Download

    Download the latest mudballs release, in gzipped tar or zip format.

  • Extract

    Extract mudballs into an appropriate directory. I suggest "/Library/Application Support/Lisp/" on Mac OSX, "/usr/local" on Unix like systems and "C:\Program Files\" on MS Windows (Just create these directories if they do not exist).

  • Load

    Add the code similar to the following to your Common Lisp implementations initialization file. 
    ;; For Mac OSX
(load "/Library/Application Support/mudballs/boot.lisp")

And we are done. On the first load mudballs will compile itself and the system definition files that it is distributed with, this does require write permissions to where it was installed, so Linux users you will need to either give yourself write permissions on the directory (recommended) or do the above steps as root as well.

Once mudballs is loaded it is a good idea to run the following 3 commands to ensure that everything is up to date

(mb:self-update) ;; ensures that boot.lisp is up to date.
;; followed by
(mb:upgrade) ;; downloads the latest system definition files from mudballs.com and upgrades installed systems
;; and finally, after restarting your lisp
(mb:test :mudballs) ;; to make sure that every thing is good in the world
A Brief Overview of Important Commands As mudballs is primarily for loading systems the best place to start is right there1. We will start by installing and loading the popular webserver hunchentoot and then putting up a simple 'Hello, World!' page.

Step 1: Finding Hunchentoot

The first step is to locate the correct system that we want to load. The easiest way to accomplish this is to use the mb:search function. 
> (mb:search "hunchentoot")
.. <snipped some load output> ..

:HUNCHENTOOT (0.15.4)
Hunchentoot is a web server written in Common Lisp and at the same
time a toolkit for building dynamic websites.
NIL
Excellent, now we see that the systems name is :HUNCHENTOOT (surprise!) and has version 0.15.4 available (the output of this may differ on your machine).

Step 2: Install Hunchentoot

You can now install the latest version of hunchentoot by executing the following 

> (mb:install :hunchentoot)

.. <snipped some load output> ..
Installing System HUNCHENTOOT.
Downloading source for HUNCHENTOOT from http://<url>/hunchentoot.0.15.4.mb.
Verifying md5sum.
Md5sum matches.
Extracting to #P"/Library/Application Support/Lisp/mudballs/systems/hunchentoot/0.15.4/".
Processing Dependencies.
Installing System CHUNGA.
.. <snipped some more output> ..
#<SYSDEF-USER::EDI-SYSTEM :HUNCHENTOOT 0.15.4 216E3043>
This will download, compile and load the latest version of hunchentoot into your lisp image, it can be loaded in subsequent images using by loading2 the system using the following command. 

(mb:load :hunchentoot)
All functions offered by mudballs also accept a :version keyword argument which accepts a version specifier and can be used to operate upon an particular version of a system. This covers the commands that you will use 90% of the time, to cover the rest are the following commands.
  1. If you are new to CL then it may be worth running the following.
    (setf (mb:lisp-level) :beginner)
    This will, among other things, disable the debugger as it can be slightly confusing when you first encounter it. To undo this change execute. 
    (setf (mb:lisp-level) :expert)
  2. If the system is not installed and a load command is issued an error will be signalled and an install restart will be made available.
  3. My apologies for any poor styling in the automatically generated documentation, I have yet to get around to styling it.
Version Specifiers As mentioned the :version keyword argument to the various mudballs functions accepts version specifiers which are of the following forms.

Exact Version Specifiers

Exact Version specifiers come in 2 forms, the string form and the list form.
The String form is a string of which contains integers separated by #\.'s eg. "1.2.3" or "0.15.4"
The List form is a list which contains only integers. eg. (1 2 3) or (0 15 4)

Wild Version Specifiers

Wild version specifiers are exact version specifiers where the last integer grouping is replaced by a #\*
eg. (1 2 *) or "1.2.*"
These are converted into a mix of Bounding Version Specifiers and Boolean Specifiers

Bounding Version Specifiers

Bounding version specifiers are list specifiers with one of the following operators in the 'car' position;

<, >, =, <=, >= and /=

These specify an appropriate test that would have to be satisfy.
eg. (> 0 15 4) is a version spec which specifies version greater than "0.15.4"

Boolean Specifiers

Boolean Version specifiers is a list specifier with one of the following boolean operators in the 'car' position followed by an exact version specifier;

AND, OR, NOT


eg. (not (0 15 4)) or (not "0.15.4") specifies a version that is not "0.15.4"
while (and "0.15.*" (not (0 15 4))) specifies any version which is greater than 0.15, less than 0.16 and not 0.15.4
Customization Mudballs can be customized by two files, namely ~/.mudballs and ~/.mudballs.prefs .
The first is a configuration file which contains Common Lisp code and is loaded after the system it customizes is loaded.

The second is a preference file and it's value are made available to the system during compilation & load time.
Preference files are files containing lists of the form (preference-name preference-value) these preferences are then available to the system via the preference function.

The following preferences are supported.
  • :fasl-output-root: This specifies the root directory where all compiled system files are placed.
  • :mudballs-root: This specifies an alternate pathname where the mudballs system can be found.
Starting the server and Hello, World! And to finish off this brief tutorial, lets start up our server and add our Hello, World! page. 

> (defparameter *server* 
     (hunchentoot:start (make-instance 'hunchentoot:acceptor :port 8080)))
=> *SERVER*

> (hunchentoot:define-easy-handler (hello-world :uri "/hello-world") ()
    "Hello, World!")

=> HELLO-WORLD
You can now browse to http://localhost:8080/hello-world.
Defining Your Own Systems The rest of this document will be focused on defining your own systems. This will be covered in the following sections.
  • Mudballs packages
  • define-system
  • Making your system available to yourself
  • Making your system available to others
  • Overview, aka The Short Short Version
Mudballs Packages The mudballs distribution consists of three packages, mudballs, sysdef and sysdef-user.
MUDBALLS is the user friendly interface to sysdef, it provides simple, concise commands which apply actions to systems.
SYSDEF contains the internals and defines the plumbing which performs the actions, it also exports all of the required functionality to
define and execute actions upon systems.
SYSDEF-USER is a utility package which :uses COMMON-LISP and SYSDEF, it is provided as the work area for system definitions.
Your first System Defining systems is done using define-system, the syntax of which is as follows.

(define-system name (superclass) option*)

name is a system-name which can subsequently be used to lookup the system using find-system or find-component.
superclass specifies the class which the system will be an instance of, this defaults to SYSTEM.
option is a list of the form (option-name . option-values).
Each option is processed against the system (using process-option), the full of options is can be found in the documentation for define-system but we will cover the definition of the following system here.
(in-package :sysdef-user)
(define-system :test-system  ()
  (:author "Guy Self")
  (:version 0 3 6)
  (:licence "MIT Style license")
  (:documentation "My Test System")
  (:components
   "packages"
   ("dev" module
    (:needs "packages")
    (:components "test-file")))
  (:needs :cl-ppcre))
Let's work through the options used in this example.
  • :author, version & licence: These are all utility options that set the appropriate slot value on the system object. These values are used in error reporting and other user interface tasks.
  • :version: Every system is required to have a version, if it is not supplied it default to "0.0.1". Versions are specified as a sequence of integers ie. (:version 0 3 6) specifies a system with a version of "0.3.6"
  • :documentation: This provides documentation that will be used by mb:document and mb:search.
  • :components: Components specify what files and/or modules make up the system.
    Each component is of the form (name [class] . options)
    Where name is a string or symbol naming the component, if the name is a symbol it is string-downcase'd to determine the filename.
    Class specifies the class of the component to be created, this defaults to the slot value of slot default-component-class of the components parent.
    Finally, options specifies a list of options which are then applied to the component using process-options.
    If class and options are left unspecified then the parens may be dropped and the definition simplified to name.
    In this example the components option specifies a file, of type lisp-source-file named "packages" and a module named "dev" which components a lisp-source-file named "test-file" the path of which would be "dev/test-file.lisp".
  • :needs: This option specifies what other systems are required in order to operate upon this system. In this case it specifies that the system named :cl-ppcre is needed in order to load or compile the test-system and that in order to apply actions to "test-file" that those actions must first be applied to "packages". Please see the documentation for define-system for more information on the :needs option

If you have any questions regarding the creation of systems, please get hold of me/us on the mudballs mailing list.

Making your system available to yourself

Now that we know, or at least have a vague idea, of how to create our own systems the next step is to make the definitions available to mudballs so that we can operate upon using them the various mudballs functions. This requires us to place the define-system form in a file where mudballs can locate it.

By default system definitions are placed in the system-definitions directory in the mudballs distribution, these are, however, reserved for for point releases of systems and not used for systems which are in development1. It is customary, in development systems, for the system- definition to reside in a system-name.mbd file in the root directory of the system being defined. This is then combined with an addition like the following to the ~/.mudballs file which informs mudballs about extra directories for systems. 

(in-package :sysdef-user)

(push (wildcard-searcher "/Users/sross/Code/Lisp/dev/*/*.mbd")
      *custom-search-modules*)

This would make every system defined in *.mbd files in directories under ~/Code/Lisp/dev available to mudballs.

  1. When a particular version of a system is promoted to a point release is completely up to the maintainer but it is important that there is a distinction.
Making your system available to others Mudballs was written to facilitate sharing of code and it is relatively simple to make your code available to others, currently there are two ways.
  1. Make it available on mudballs.com: Contact me and request the system to be added to the standard mudballs distribution file. Recommended.
  2. Make the code available from your own server and publish your own system-definition file: This is easier than it sounds. It simply involves publishing your system definitions in a sysdef file on a public server wrapped in a with-provider form. This then refers to systems which can be downloaded. Please contact me for full details
Overview, aka The Short Short Version
  • Define your system using define-system
  • Place the system definition in a file named system-name.mbd and place this file in the root directory of your new system
  • Add code like the folloing to your ~/.mudballs file 
    (in-package :sysdef-user)

(push (wildcard-searcher "/Users/sross/Code/Lisp/dev/*/*.mbd" :development-mode t)
      *custom-search-modules*)
Reporting Bugs You can report bugs to me (Sean Ross) at the following places
Source Code Repositories You can find all of the source for the mudballs systems at github.
Thanks to...