Common-Lisp programm in large
1. Introduction
One of the problem during learning Common-lisp is how to
- Use other function from other files.
- How to specify dependencies between different packages including external ones.
Other language like Elixir has built in tools to handle all of this, like mix. But in CL, there is no offical support.
It seems the current mainstream is to use quicklisp + ASDF.
2. How to organize files into packages
2.1. Basics about packages
- Packages must be registered first before their use.
- Use package qualifier to access symbols in other packages, such as
foo:bar
which means a symbolbar
in packagefoo
. - This is true only if
bar
is an external symbol offoo
, that is, a symbol that is supposed to be visited outside offoo
. - A reference to an internal symbol requires the intentionally clumsier syntax
foo::bar
. So, use::
qualifier will let us access any symbols from the named packages. - All symbols in the current package can be accessed without a package qualifier.
- Use package qualifier to access symbols in other packages, such as
- How to access the name-to-symbol mappings in a packages
find-symbol
intern
- Current package is defined in global variable
*package*
. - Keywords symbols are written with names starting with a colon “:”. They are interned in the package named
KEYWORD
and automatically exported. - Uninterned symbols are written with a leading
#:
. - Reader translates textual names into symbol objects.
- A symbol can be accessible in two ways:
- name-to-symbol table
- home package
- home package
- inherits from other packages
- external symbols
- external symbols
- name-to-symbol table
- 3 standard packages
- common-lisp
- common-lisp-user
- keyword
- Allow us to access keyword symbols without an explicit
keyword:
package qualification.
- Allow us to access keyword symbols without an explicit
- common-lisp
- In the REPL buffer in SLIME you can also change packages with a REPL shortcut. Type a comma
,
and then enter change-package at the Command: prompt.
2.2. Examples about organize files into different files as packages
- Example-01
we could group useful common functions into another package as library
(defpackage :com.gigamonkeys.text-db (:use :common-lisp) (:export :open-db :save :store))
- The
:use
command withindefpackage
is used to inherit packages.
- When we inherit packages, all of the external symbols in the inherited packages will be treated as internal symbols in the package we are defining.
- When we inherit packages, all of the external symbols in the inherited packages will be treated as internal symbols in the package we are defining.
- The
:export
clause specifies names that will be external inCOM.GIGAMONKEYS.TEXT-DB
and thus accessible in package that:use
it.
- Any package that inherits
com.gigamonkeys.text-db
can access these symbols without a package qualifier.
- Any package that inherits
- The
our application
(defpackage :com.gigamonkeys.email-db (:use :common-lisp :com.gigamonkeys.text-db :com.acme.text) (:import-from :com.acme.email :parse-email-address) (:shadow :build-index) (:shadowing-import-from :com.gigamonkeys.text-db :save)) (in-package :com.gigamonkeys.email-db)
- Switch between packages using
in-package
.
- So, when we want to intern symbols in a particular package, we need to first switch to that package via
in-package
form.
- So, when we want to intern symbols in a particular package, we need to first switch to that package via
- Now, code written in
COM.GIGAMONKEYS.EMAIL-DB
can use the unqualified names to refer to the exported symbols from bothCOMMON-LISP
andCOM.GIGAMONKEYS.TEST-DB
. :import-from
is for scenario in which we just want to import only one function from a certain package.- About shadow
:shadow
is the opposite of:import-from
, in which it specifies the only function we don’t want to use. So we shadow it. The result is the symbolbuild-index
from other packages will be ignored.:shadowing-import-from
solve the case in which there are two packages with same symbols and we want to shadow one of them (solve the ambiguity).
- Switch between packages using
How to export all symbols from a package
;; Suppose there is package FOO ;; Once the package is created, and all symbols in it created, e.g., by loading your code that implements the package ;; We can export any symbols ;; (do-all-symbols (sym (find-package :foo)) (export sym)) (let ((pack (find-package :foo))) (do-all-symbols (sym pack) (when (eql (symbol-package sym) pack) (export sym))))
use-package
commands is very similar to:use
withindefpackage
;; With only one argument, use-package will add my-package-2 to the use list of the current package: (use-package my-package-2) ;; With two arguments, use-package will add my-package-2 to the use list of my-package-1: (use-package my-package-1 my-package-2)
We can get a package’s use list by calling
package-use-list
.
(package-use-list :cl-user) (#<PACKAGE "COMMON-LISP"> #<PACKAGE "SB-ALIEN"> #<PACKAGE "SB-DEBUG"> #<PACKAGE "SB-EXT"> #<PACKAGE "SB-GRAY"> #<PACKAGE "SB-PROFILE">)
- Example-02
- We will work with the following files:
utilities.lisp
, it will contains some of our own user-defined functions.quant.lisp,
it will does some calculations and makes use ofutilities.lisp
.main.lisp
, a file that references the two above.packages.lisp
, a file that combines the above and brings everything together into a cohesive whole so that it can be run as one program.
In
utilities.lisp
(in-package :utilities) (defun list-sum (my-list) (reduce #'+ my-list))
In
quant.lisp
(in-package :quant) (ql:quickload 'cl-csv) ;; Change the below path to where you sae your utilities file: (load "./utilities.lisp") ;; Change the below path to where you save your csv file: (defparameter my-filename "./my-csv.csv") (defparameter csv-file (cl-csv:read-csv (pathname my-filename))) (defparameter my-sum (list-sum (list 1 2 3 4 5)))
- Notice, we reference the
list-sum
function in theutilities
package without a qualifier. - In contrast, we reference the
read-csv
function with a package qualifier. Why, see below.
- Notice, we reference the
In
main.lisp
(in-package :main) ;; Change the below path to where you save your quant file: (load "./quant.lisp") (print (quant::list-sum (list 1 2 3 4 5 6 7 8 9 10))) (print csv-file) (print quant::my-sum)
- It loads the above quant.lisp file, which means it also indirectly loads utilities.lisp.
- Notice package qualifier with the double colon for
list-sum
- Notice the
csv-file
from quant.lisp is directly accessed without qualifier.
- It loads the above quant.lisp file, which means it also indirectly loads utilities.lisp.
In
package.lisp
(in-package :cl) (defpackage :utilities (:use :cl) (:export :list-sum)) (defpackage :quant (:use :cl :utilities) (:export :csv-file)) (defpackage :main (:use :cl :quant)) ;; Change filepath to where you save your main.lisp: (load "./main.lisp")
- We define our 3 packages not in their own files.
- Note how we segregated package definitions from the actual code we write. This is because we need to define our packages in order, so it is easier to do this together in one place.
- Note how we segregated package definitions from the actual code we write. This is because we need to define our packages in order, so it is easier to do this together in one place.
- Notice the this file starts with
(in-package :cl)
to mak sure we can access it sincedefpackage
is defined in the CL.
- We define our 3 packages not in their own files.
- Summary
- All of our files shoud have an
in-package
form at the top. - We can run our program by simply loading and running
package.lisp
file. (This is how we construct and run our project if we only use plain lisp files as packages. There is a more convonient way describe in ASDF.) - It is different from
asdf
: Another thing to focus on is the difference between a system (as in asdf) and a package. These come together in asdf’s package-inferred-systems, which infer dependencies between files using the package dependencies, at the cost of having a separate package for each file.
- All of our files shoud have an
- We will work with the following files:
3. ASDF – an improved way to use packages
3.1. What is system defined by ASDF
- A system is a collection of Lisp files that together constitute an application or a library, and that should therefore be managed as a whole.
- A system definition describes which source files make up the system, what the dependencies among them are, and the order they should be compiled and loaded in.
3.2. How to write a system definition
- The kind of files in system
- Normal .lisp file
- Special .lisp file – package or packages.lisp
The package/packages.lisp file let us define and use different functions across different packages(files).
- The pupose of this is to seperate different functions into different namespaces to make them together without conflict.
- Notice, the package.lisp only let us use different functions from other packages(import) and expose functions(export). It doesn’t automatically download packages.
- So, if we only use package.lisp in our project. It will NOT be able to
- Download dependencies.
- Recompile project to update changes from different files convoniently.
- Download dependencies.
- The .asd file helps us to solve above problems.
- The pupose of this is to seperate different functions into different namespaces to make them together without conflict.
- The .asd file – defines our system.
The .asd file helps us to
- Solve dependencies. For example, to download missing libraries(packages) from online. Again, packages.lisp only describes what functions are imported and exported, nothing else.
- It is like “using” or “import” at the beginning of files in other programming language.
- How the packages(libraries or assemblies) are used to support our project is defined in “.asd” file (like CMakeLists.txt in C++ or .csproj in C#) with extra feature: it helps us to automatically download those dependencies if they are missing.
- It is like “using” or “import” at the beginning of files in other programming language.
- Solve dependencies. For example, to download missing libraries(packages) from online. Again, packages.lisp only describes what functions are imported and exported, nothing else.
- Normal .lisp file
- The general step:
- Write whatever function or code in normal .lisp files as normal but with addition “(in-package :<package-name>)”. For example
In a tools.lisp file
(in-package :tools) (defun another-function () (print "from tool"))
In a file foobar.lisp file
(in-package :foobar) (defun some-function () (tools:another-function) (print "hello world1"))
Import and export different function from different packages by defining them in “package/packages.lisp” file. Such as:
(defpackage :tools (:use #:cl) (:export #:another-function)) (defpackage :foobar (:use #:common-lisp #:tools) (:export #:some-function))
Define dependencies and source files of our system.
In a “.asd” file
(asdf:defsystem "foobar" :depends-on (#:vecto) :components ( (:file "package") (:file "tools") (:file "foobar")))
- Write whatever function or code in normal .lisp files as normal but with addition “(in-package :<package-name>)”. For example
3.3. How to use/update a system
- Create the system from above example
- As we installed SLIME in emacs, we could just slime-load file by “C-c C-l” to load the .asd file.
- During loading, it will install missing dependencies automatically.
- As we installed SLIME in emacs, we could just slime-load file by “C-c C-l” to load the .asd file.
- Use the system or update system after files are changed
- In SLIME REPL, just eval
(ql:quickload "foobar")
or re-eval a updated specific function. - Then, invoke the functions from that package:
(foobar:some-function)
.
- In SLIME REPL, just eval
4. Use quickproject to setup a common-lisp project quickly
Steup a ASDF compatible project could be speed up by using quicklisp
- It helps us to setup a ASDF project for us.
Example
(ql:quickload "quickproject") (quickproject:make-project "~/code/common-lisp-programming/common-lisp-in-action/leetcode" :depends-on '(trivia))
- This will create a leetcode system as our starting point.
- This will create a leetcode system as our starting point.
5. References
- See practical common-lisp: chapter21
- An introduction to Lisp Packages
- ref: The Common Lisp Cookbook Defining Systems
- ref: Making a small Lisp project with quickproject and Quicklisp