mirror of
https://git.savannah.gnu.org/git/emacs/org-mode.git
synced 2024-09-23 12:10:44 +00:00
2f82474d85
This fixes a bug: DONE require users to explicitly turn on each language as we continue adding more languages (each with it's own set of major mode requirements), we are going to want users to only turn on the languages that they intend to use. See the install instructions in org-babel-worg.org, also take a look at the "Requirements" sections of org-babel-ruby.el and org-babel-gnuplot.el for pointers to downloading and installing their requirements (which are no longer distributed in the util directory).
546 lines
20 KiB
Org Mode
546 lines
20 KiB
Org Mode
#+OPTIONS: H:3 num:nil toc:2 \n:nil @:t ::t |:t ^:t -:t f:t *:t TeX:t LaTeX:t skip:nil d:(HIDE) tags:not-in-toc
|
|
#+STARTUP: align fold nodlcheck hidestars oddeven lognotestate hideblocks
|
|
#+SEQ_TODO: TODO(t) INPROGRESS(i) WAITING(w@) | DONE(d) CANCELED(c@)
|
|
#+TAGS: Write(w) Update(u) Fix(f) Check(c)
|
|
#+TITLE: Org-babel
|
|
#+AUTHOR: Dan Davison, Eric Schulte
|
|
#+EMAIL: davison at stats dot ox dot ac dot uk
|
|
#+LANGUAGE: en
|
|
#+CATEGORY: worg
|
|
|
|
#+begin_html
|
|
<div id="subtitle">
|
|
<p>executable source code blocks in org-mode</p>
|
|
</div>
|
|
<div id="logo">
|
|
<p>
|
|
<img src="images/tower-of-babel.png" alt="images/tower-of-babel.png"
|
|
title="And the Lord said, Behold, the people is one, and they have all one language; and this they begin to do; and now nothing will be restrained from them, which they have imagined to do. Genesis 11:1-9"/>
|
|
<div id="attr">
|
|
from
|
|
<a href="http://www.flickr.com/photos/23379658@N05/" title=""><b>Martijn Streefkerk</b></a>
|
|
</div>
|
|
</p>
|
|
</div>
|
|
#+end_html
|
|
|
|
* Introduction
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: introduction
|
|
:END:
|
|
Org-babel provides the following modifications to [[http://orgmode.org/manual/Literal-examples.html][the existing
|
|
support]] for blocks of source code examples in the org-mode core.
|
|
1. source code execution
|
|
2. arguments to source code blocks
|
|
3. exportation of source code blocks to files (literate programming)
|
|
|
|
* Getting started
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: getting-started
|
|
:END:
|
|
|
|
1) Grab the latest code from the git repo at [[http://github.com/eschulte/org-babel/tree/master][github/org-babel]]
|
|
#+begin_src sh
|
|
git clone git://github.com/eschulte/org-babel.git
|
|
#+end_src
|
|
|
|
2) Add the following lines to your .emacs, replacing the path as
|
|
appropriate. A good place to check that things are up and running
|
|
would the examples in [[* Basic org-babel functionality][Basic org-babel functionality]].
|
|
#+begin_src emacs-lisp
|
|
(add-to-list 'load-path "/path/to/org-babel/lisp")
|
|
(require 'org-babel-init)
|
|
#+end_src
|
|
|
|
3) Then activate the subset of supported Org-babel languages which
|
|
you will want to be able to execute on your system. The
|
|
following can be added to your .emacs and used to activate
|
|
languages. It includes a brief list of the requirements for each
|
|
language.
|
|
#+begin_src emacs-lisp
|
|
;; Uncomment each of the following require lines if you want org-babel
|
|
;; to support that language. Each language has a comment explaining
|
|
;; it's dependencies. See the related files in lisp/langs for more
|
|
;; detailed explanations of requirements.
|
|
;; (require 'org-babel-R) ;; R and ess-mode
|
|
;; (require 'org-babel-asymptote) ;; asymptote
|
|
;; (require 'org-babel-css) ;; none
|
|
;; (require 'org-babel-ditaa) ;; ditaa
|
|
;; (require 'org-babel-dot) ;; dot
|
|
;; (require 'org-babel-gnuplot) ;; gnuplot, and gnuplot-mode
|
|
;; (require 'org-babel-python) ;; python, and python-mode
|
|
;; (require 'org-babel-ruby) ;; ruby, irb, ruby-mode, and inf-ruby mode
|
|
;; (require 'org-babel-sql) ;; none
|
|
;;
|
|
;; Once you've activated languages, load the library of babel for
|
|
;; pre-built helpers in the languages you will be using.
|
|
(org-babel-load-library-of-babel)
|
|
#+end_src
|
|
|
|
* Basic org-babel functionality
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: basic-functionality
|
|
:END:
|
|
*** Source code execution
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: source-code-execution
|
|
:END:
|
|
For interpreted languages such as shell, python, R, etc, org-babel
|
|
allows source blocks to be executed: the code is passed to the
|
|
interpreter and you have control over what is done with the
|
|
results of excecution. E.g. place point anywhere in the following
|
|
block and use C-c C-c to run the code:
|
|
|
|
[[http://www.ruby-lang.org/][Ruby]] source code
|
|
#+begin_src ruby
|
|
"This file was last evaluated on #{Date.today}"
|
|
#+end_src
|
|
|
|
Results of Ruby evaluation
|
|
#+resname:
|
|
: This file was last evaluated on 2009-08-09
|
|
|
|
[[http://www.r-project.org/][R]] source code
|
|
#+begin_src R :results value
|
|
x = 4
|
|
date()
|
|
c(5, 10)
|
|
#+end_src
|
|
|
|
Results of R evaluation
|
|
#+resname:
|
|
| 5 |
|
|
| 10 |
|
|
|
|
*** What happens to the results?
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: results
|
|
:END:
|
|
Org-babel provides two fundamentally different modes for capturing
|
|
the results of code evaluation, specified by the :results header
|
|
argument:
|
|
**** :results value
|
|
This means that the 'result' of code evaluation is defined to be
|
|
the *value* of the last statement in the block. Thus with this
|
|
setting, one can view the code block as a function with a return
|
|
value. And not only can one view it that way, but you can
|
|
actually use the return value of one source block as input for
|
|
another (see later). This setting is the default.
|
|
**** :results output
|
|
With this setting, org-babel captures all the text output of the
|
|
code block and places it in the org buffer. One can think of this
|
|
as a 'scripting' mode: the code block contains a series of
|
|
commands, and you get the output of all the commands. Unlike in
|
|
the 'functional' mode specified by =:results value=, the code
|
|
block has no return value. (This mode will be familiar to Sweave
|
|
users).
|
|
**** Additional :results settings
|
|
|
|
*** Arguments to source code blocks
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: arguments-to-source-code-blocks
|
|
:END:
|
|
In addition to evaluation of code blocks, org-babel allows them to
|
|
be parameterised (i.e. have arguments). Thus source code blocks
|
|
now have the status of *functions*.
|
|
|
|
Inputs for fibonacci-seq
|
|
|
|
#+tblname: fibonacci-inputs
|
|
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
|
|
| 2 | 4 | 6 | 8 | 10 | 12 | 14 | 16 | 18 | 20 |
|
|
|
|
in the Org-mode buffer this looks like
|
|
: #+tblname: fibonacci-inputs
|
|
: | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
|
|
: | 2 | 4 | 6 | 8 | 10 | 12 | 14 | 16 | 18 | 20 |
|
|
|
|
[[http://www.gnu.org/software/emacs/manual/elisp.html][Emacs Lisp]] source code
|
|
#+srcname: fibonacci-seq
|
|
#+begin_src emacs-lisp :var fib-inputs=fibonacci-inputs
|
|
(defun fibonacci (n)
|
|
(if (or (= n 0) (= n 1))
|
|
n
|
|
(+ (fibonacci (- n 1)) (fibonacci (- n 2)))))
|
|
|
|
(mapcar (lambda (row)
|
|
(mapcar #'fibonacci row)) fib-inputs)
|
|
#+end_src
|
|
|
|
in the Org-mode buffer this looks like
|
|
: #+srcname: fibonacci-seq
|
|
: #+begin_src emacs-lisp :var fib-inputs=fibonacci-inputs
|
|
: (defun fibonacci (n)
|
|
: (if (or (= n 0) (= n 1))
|
|
: n
|
|
: (+ (fibonacci (- n 1)) (fibonacci (- n 2)))))
|
|
:
|
|
: (mapcar (lambda (row)
|
|
: (mapcar #'fibonacci row)) fib-inputs)
|
|
: #+end_src
|
|
|
|
Results of Emacs Lisp code evaluation
|
|
#+resname:
|
|
| 1 | 1 | 2 | 3 | 5 | 8 | 13 | 21 | 34 | 55 |
|
|
| 1 | 3 | 8 | 21 | 55 | 144 | 377 | 987 | 2584 | 6765 |
|
|
|
|
* A meta-programming language for org-mode
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: meta-programming-language
|
|
:END:
|
|
|
|
Since information can pass freely between source-code blocks and
|
|
org-mode tables you can mix and match languages using each language
|
|
for those tasks to which it is suited. This makes Org-mode files with
|
|
Org-babel into a kind of meta-functional programming language in which
|
|
functions from many languages can work together.
|
|
|
|
As an example, lets take some system diagnostics in the shell, and
|
|
then graph them with R.
|
|
|
|
1. Shell source code
|
|
#+srcname: directories
|
|
#+begin_src bash :results replace
|
|
cd ~ && du -sc * |grep -v total
|
|
#+end_src
|
|
2. Results of the shell source code (on my system, grab this org-mode
|
|
files and try running it on your own)
|
|
#+resname: directories
|
|
| 72 | "Desktop" |
|
|
| 12156104 | "Documents" |
|
|
| 3482440 | "Downloads" |
|
|
| 2901720 | "Library" |
|
|
| 57344 | "Movies" |
|
|
| 16548024 | "Music" |
|
|
| 120 | "News" |
|
|
| 7649472 | "Pictures" |
|
|
| 0 | "Public" |
|
|
| 152224 | "Sites" |
|
|
| 8 | "System" |
|
|
| 56 | "bin" |
|
|
| 3821872 | "mail" |
|
|
| 10605392 | "src" |
|
|
| 1264 | "tools" |
|
|
3. R source code (which calls the previous shell source code)
|
|
#+srcname: directory-pie
|
|
#+begin_src R :var dirs = directories :session R-pie-example
|
|
pie(dirs[,1], labels = dirs[,2])
|
|
#+end_src
|
|
4. Results of R code [[file:images/dirs.png]]
|
|
|
|
* Spreadsheet plugins for org-mode in any language
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: spreadsheet
|
|
:END:
|
|
*NOTE*: Maybe in-addition-to/in-stead-of this example we should do a
|
|
more traditional "spreadsheet" example with R [Eric]
|
|
|
|
Not only can Org-babel pass entire tables of data to source code
|
|
blocks (see [[arguments-to-source-code-blocks]]), Org-babel can also be
|
|
used to call source code blocks from *within* tables using the
|
|
Org-mode's [[http://orgmode.org/manual/The-spreadsheet.html#The-spreadsheet][existing spreadsheet functionality]].
|
|
|
|
In fact the functional test suite for Org-babel is implemented as a
|
|
large Org-mode table. To run the entire test suite you simple
|
|
evaluate the table =C-u C-c C-c=, and all of the tests are run
|
|
updating the table with pass/fail statistics.
|
|
|
|
Here's a sample of our test suite.
|
|
|
|
#+TBLNAME: org-babel-tests
|
|
| functionality | block | arg | expected | results | pass |
|
|
|------------------+--------------+-----+-------------+-------------+------|
|
|
| basic evaluation | | | | | pass |
|
|
|------------------+--------------+-----+-------------+-------------+------|
|
|
| emacs lisp | basic-elisp | 2 | 4 | 4 | pass |
|
|
| shell | basic-shell | | 6 | 6 | pass |
|
|
| ruby | basic-ruby | | org-babel | org-babel | pass |
|
|
| python | basic-python | | hello world | hello world | pass |
|
|
| R | basic-R | | 13 | 13 | pass |
|
|
#+TBLFM: $5='(if (= (length $3) 1) (progn (message (format "running %S" '(sbe $2 (n $3)))) (sbe $2 (n $3))) (sbe $2))::$6='(if (string= $4 $5) "pass" (format "expected %S but was %S" $4 $5))
|
|
#+TBLFM: $5=""::$6=""
|
|
|
|
*** code blocks for tests
|
|
|
|
#+srcname: basic-elisp
|
|
#+begin_src emacs-lisp :var n=7
|
|
(* 2 n)
|
|
#+end_src
|
|
|
|
#+srcname: basic-shell
|
|
#+begin_src sh :results silent
|
|
expr 1 + 5
|
|
#+end_src
|
|
|
|
#+srcname: date-simple
|
|
#+begin_src sh :results silent
|
|
date
|
|
#+end_src
|
|
|
|
#+srcname: basic-ruby
|
|
#+begin_src ruby :results silent
|
|
"org-babel"
|
|
#+end_src
|
|
|
|
#+srcname: basic-python
|
|
#+begin_src python :results silent
|
|
'hello world'
|
|
#+end_src
|
|
|
|
#+srcname: basic-R
|
|
#+begin_src R :results silent
|
|
b <- 9
|
|
b + 4
|
|
#+end_src
|
|
|
|
* Library of Babel
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: library-of-babel
|
|
:END:
|
|
What about those source code blocks which are so useful you want to
|
|
have them available in every org-mode buffer?
|
|
|
|
The [[file:library-of-babel.org][Library of Babel]] is an extensible collection of ready-made and
|
|
easily-shortcut-callable source-code blocks for handling common
|
|
tasks. Org-babel comes pre-populated with the source-code blocks
|
|
located in the [[file:library-of-babel.org][library-of-babel.org]] file. It is possible to add
|
|
source-code blocks from any org-mode file to the library by calling
|
|
|
|
#+srcname: add-file-to-lob
|
|
#+begin_src emacs-lisp
|
|
(org-babel-lob-ingest "path/to/file.org")
|
|
#+end_src
|
|
|
|
* Reproducible Research
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: reproducable-research
|
|
:END:
|
|
#+begin_quote
|
|
An article about computational science in a scientific publication is
|
|
not the scholarship itself, it is merely advertising of the
|
|
scholarship. The actual scholarship is the complete software
|
|
development environment and the complete set of instructions which
|
|
generated the figures.
|
|
|
|
-- D. Donoho
|
|
#+end_quote
|
|
|
|
[[http://reproducibleresearch.net/index.php/Main_Page][Reproducible Research]] (RR) is the practice of distributing along with
|
|
an article of research all data, code, and tools required to reproduce
|
|
the results discussed in the paper. As such the paper becomes not
|
|
only a document describing the research but a complete laboratory
|
|
reproducing the research.
|
|
|
|
Org-mode already has exceptional support for [[http://orgmode.org/manual/Exporting.html#Exporting][exporting to html and
|
|
LaTeX]]. Org-babel makes Org-mode a tool for RR by *activating* the
|
|
data and source code embedded into Org-mode documents making the
|
|
entire document executable. This makes it not only possible, but
|
|
natural to distribute research in a format that encourages readers to
|
|
recreate your results, and perform their own analysis.
|
|
|
|
Existing RR tools like [[http://en.wikipedia.org/wiki/Sweave][Sweave]] provide for the embedding of [[http://www.r-project.org/][R]] code into
|
|
LaTeX documents. While this is very useful, such documents often
|
|
still require a large degree of "glue code" in the form of external
|
|
shell scripts, python scripts, and Makefiles. To my knowledge
|
|
Org-babl is the only RR tool which allows multiple languages and data
|
|
to coexist and cooperate inside of a single document.
|
|
|
|
* Literate programming
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: literate-programming
|
|
:END:
|
|
|
|
#+begin_quote
|
|
Let us change our traditional attitude to the con- struction of
|
|
programs: Instead of imagining that our main task is to instruct a
|
|
/computer/ what to do, let us concentrate rather on explaining to
|
|
/human beings/ what we want a computer to do.
|
|
|
|
The practitioner of literate programming can be regarded as an
|
|
essayist, whose main concern is with exposition and excellence of
|
|
style. Such an author, with thesaurus in hand, chooses the names of
|
|
variables carefully and explains what each variable means. He or she
|
|
strives for a program that is comprehensible because its concepts have
|
|
been introduced in an order that is best for human understanding,
|
|
using a mixture of formal and informal methods that reinforce each
|
|
other.
|
|
|
|
-- Donald Knuth
|
|
#+end_quote
|
|
|
|
Org-babel supports [[http://en.wikipedia.org/wiki/Literate_programming][Literate Programming]] (LP) by allowing the act of
|
|
programming to take place inside of Org-mode documents. The Org-mode
|
|
file can then be exported (*woven* in LP speak) to html or LaTeX for
|
|
consumption by a human, and the embedded source code can be extracted
|
|
(*tangled* in LP speak) into structured source code files for
|
|
consumption by a computer.
|
|
|
|
To support these operations Org-babel relies on Org-mode's [[http://orgmode.org/manual/Exporting.html#Exporting][existing
|
|
exporting functionality]] for *weaving* of documentation, and on the
|
|
=org-babel-tangle= function which makes use of [[http://www.cs.tufts.edu/~nr/noweb/][Noweb]] [[reference-expansion][reference syntax]]
|
|
for *tangling* of code files.
|
|
|
|
The [[literate-programming-example][following example]] demonstrates the process of *tangling* in
|
|
Org-babel.
|
|
|
|
*** Literate Programming Example
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: literate-programming-example
|
|
:END:
|
|
|
|
Tangling functionality is controlled by the =tangle= family of
|
|
[[header-arguments]]. These arguments can be used to turn tangling on or
|
|
off (the default) on the source code block, or the outline heading
|
|
level.
|
|
|
|
The following demonstrates the combination of three source code blocks
|
|
into a single source code file using =org-babel-tangle=.
|
|
|
|
The following two blocks will not be tangled by default since they
|
|
have no =tangle= header arguments.
|
|
|
|
#+srcname: hello-world-prefix
|
|
#+begin_src sh :exports none
|
|
echo "/-----------------------------------------------------------\\"
|
|
#+end_src
|
|
|
|
: #+srcname: hello-world-prefix
|
|
: #+begin_src sh :exports none
|
|
: echo "/-----------------------------------------------------------\\"
|
|
: #+end_src
|
|
|
|
#+srcname: hello-world-postfix
|
|
#+begin_src sh :exports none
|
|
echo "\-----------------------------------------------------------/"
|
|
#+end_src
|
|
|
|
: #+srcname: hello-world-postfix
|
|
: #+begin_src sh :exports none
|
|
: echo "\-----------------------------------------------------------/"
|
|
: #+end_src
|
|
|
|
|
|
The third block does have a =tangle= header argument indicating the
|
|
name of the file to which it should be written. It also has [[http://www.cs.tufts.edu/~nr/noweb/][Noweb]]
|
|
style references to the two previous source code blocks which will be
|
|
expanded during tangling to include them in the output file as well.
|
|
|
|
#+srcname: hello-world
|
|
#+begin_src sh :tangle hello :exports none
|
|
# <<hello-world-prefix>>
|
|
echo "| hello world |"
|
|
# <<hello-world-postfix>>
|
|
#+end_src
|
|
|
|
: #+srcname: hello-world
|
|
: #+begin_src sh :tangle hello :exports none
|
|
: # <<hello-world-prefix>>
|
|
: echo "| hello world |"
|
|
: # <<hello-world-postfix>>
|
|
: #+end_src
|
|
|
|
Calling =org-babel-tangle= will result in the following being written
|
|
to the =hello.sh= file.
|
|
|
|
#+srcname: hello-world-output
|
|
#+begin_src sh
|
|
#!/usr/bin/env sh
|
|
# generated by org-babel-tangle
|
|
|
|
# [[file:~/src/org-babel/org-babel-worg.org::#literate-programming-example][block-16]]
|
|
# <<hello-world-prefix>>
|
|
echo "/-----------------------------------------------------------\\"
|
|
|
|
echo "| hello world |"
|
|
# <<hello-world-postfix>>
|
|
echo "\-----------------------------------------------------------/"
|
|
# block-16 ends here
|
|
#+end_src
|
|
|
|
* Reference / Documentation
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: reference-and-documentation
|
|
:END:
|
|
|
|
*** Source Code block syntax
|
|
|
|
The basic syntax of source-code blocks is as follows:
|
|
|
|
: #+srcname: name
|
|
: #+begin_src language header-arguments
|
|
: body
|
|
: #+end_src
|
|
|
|
- name :: This name is associated with the source-code block. This is
|
|
similar to the =#+TBLNAME= lines which can be used to name tables
|
|
in org-mode files. By referencing the srcname of a source-code
|
|
block it is possible to evaluate the block for other places,
|
|
files, or from inside tables.
|
|
- language :: The language of the code in the source-code block, valid
|
|
values must be members of `org-babel-interpreters'.
|
|
- header-arguments :: Header arguments control many facets of the
|
|
input to, evaluation of, and output of source-code blocks. See
|
|
the [[* Header Arguments][Header Arguments]] section for a complete review of available
|
|
header arguments.
|
|
- body :: The actual source code which will be evaluated. This can be
|
|
edited with `org-edit-special'.
|
|
|
|
*** Header Arguments
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: header-arguments
|
|
:END:
|
|
|
|
- results :: results arguments specify what should be done with the
|
|
output of source-code blocks
|
|
- The following options are mutually exclusive, and specify how the
|
|
results should be collected from the source-code block
|
|
- value ::
|
|
- output ::
|
|
- The following options are mutually exclusive and specify what type
|
|
of results the code block will return
|
|
- vector :: specifies that the results should be interpreted as a
|
|
multidimensional vector (even if the vector is
|
|
trivial), and will be inserted into the org-mode file
|
|
as a table
|
|
- scalar :: specifies that the results should be interpreted as a
|
|
scalar value, and will be inserted into the org-mode
|
|
file as quoted text
|
|
- file :: specifies that the results should be interpreted as the
|
|
path to a file, and will be inserted into the org-mode
|
|
file as a link
|
|
- The following options specify how the results should be inserted
|
|
into the org-mode file
|
|
- replace :: the current results replace any previously inserted
|
|
results from the code block
|
|
- silent :: rather than being inserted into the org-mode file the
|
|
results are echoed into the message bar
|
|
- exports :: exports arguments specify what should be included in html
|
|
or latex exports of the org-mode file
|
|
- code :: the body of code is included into the exported file
|
|
- results :: the results of evaluating the code is included in the
|
|
exported file
|
|
- both :: both the code and results are included in the exported
|
|
file
|
|
- none :: nothing is included in the exported file
|
|
- tangle :: tangle arguments specify whether or not the source-code
|
|
block should be included in tangled extraction of
|
|
source-code files
|
|
- yes :: the source-code block is exported to a source-code file
|
|
named after the basename (name w/o extension) of the
|
|
org-mode file
|
|
- no :: (default) the source-code block is not exported to a
|
|
source-code file
|
|
- other :: any other string passed to the =tangle= header argument
|
|
is interpreted as a file basename to which the block will
|
|
be exported
|
|
|
|
*** Noweb reference syntax
|
|
The [[http://www.cs.tufts.edu/~nr/noweb/][Noweb]] Literate Programming system allows named blocks of code to
|
|
be referenced by using a =<<code-block-name>>= syntax. When a
|
|
document is tangled these references are replaced with the named code.
|
|
An example is provided in the [[literate-programming-example]] in this
|
|
document.
|
|
|
|
|