mirror of
https://git.savannah.gnu.org/git/emacs/org-mode.git
synced 2024-09-23 16:10:45 +00:00
pulled org-babel.org from session branch
This commit is contained in:
parent
f6452b272a
commit
f49ac94e6a
251
org-babel.org
251
org-babel.org
|
@ -1,7 +1,7 @@
|
||||||
#+OPTIONS: H:3 num:nil toc:t
|
#+OPTIONS: H:3 num:nil toc:t
|
||||||
#+TITLE: org-babel --- facilitating communication between programming languages and people
|
#+TITLE: org-babel --- facilitating communication between programming languages and people
|
||||||
#+SEQ_TODO: TODO PROPOSED | DONE DEFERRED REJECTED
|
#+SEQ_TODO: TODO PROPOSED | DONE DEFERRED REJECTED
|
||||||
#+STARTUP: oddeven
|
#+STARTUP: oddeven hideblocks
|
||||||
|
|
||||||
* Introduction
|
* Introduction
|
||||||
|
|
||||||
|
@ -114,8 +114,9 @@ table, allowing the test suite to be run be evaluation of the table
|
||||||
and the results to be collected in the same table.
|
and the results to be collected in the same table.
|
||||||
|
|
||||||
|
|
||||||
* Tasks [22/37]
|
* Tasks [22/38]
|
||||||
** TODO Create objects in top level (global) environment in R? [0/5]
|
** TODO Create objects in top level (global) environment [0/5]
|
||||||
|
*sessions*
|
||||||
|
|
||||||
*** initial requirement statement [DED]
|
*** initial requirement statement [DED]
|
||||||
At the moment, objects created by computations performed in the
|
At the moment, objects created by computations performed in the
|
||||||
|
@ -189,6 +190,222 @@ or if you think any parts might be confusing for people coming from
|
||||||
Sweave. I'll hopefully find some time to work on this later in the
|
Sweave. I'll hopefully find some time to work on this later in the
|
||||||
week.
|
week.
|
||||||
|
|
||||||
|
*** can functional and interpreted/interactive models coexist?
|
||||||
|
|
||||||
|
Even though both of these use the same =*R*= buffer the value of =a=
|
||||||
|
is not preserved because it is assigned inside of a functional
|
||||||
|
wrapper.
|
||||||
|
|
||||||
|
#+srcname: task-R-sessions
|
||||||
|
#+begin_src R
|
||||||
|
a <- 9
|
||||||
|
b <- 21
|
||||||
|
a + b
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
#+srcname: task-R-same-session
|
||||||
|
#+begin_src R
|
||||||
|
a
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
This functional wrapper was implemented in order to efficiently return
|
||||||
|
the results of the execution of the entire source code block. However
|
||||||
|
it inhibits the evaluation of source code blocks in the top level,
|
||||||
|
which would allow for persistence of variable assignment across
|
||||||
|
evaluations. How can we allow *both* evaluation in the top level, and
|
||||||
|
efficient capture of the return value of an entire source code block
|
||||||
|
in a language independent manner?
|
||||||
|
|
||||||
|
Possible solutions...
|
||||||
|
1) we can't so we will have to implement two types of evaluation
|
||||||
|
depending on which is appropriate (functional or imperative)
|
||||||
|
2) we remove the functional wrapper and parse the source code block
|
||||||
|
into it's top level statements (most often but not always on line
|
||||||
|
breaks) so that we can isolate the final segment which is our
|
||||||
|
return value.
|
||||||
|
3) we add some sort of "#+return" line to the code block
|
||||||
|
4) we take advantage of each languages support for meta-programming
|
||||||
|
through =eval= type functions, and use said to evaluate the entire
|
||||||
|
blocks in such a way that their environment can be combined with the
|
||||||
|
global environment, and their results are still captured.
|
||||||
|
5) I believe that most modern languages which support interactive
|
||||||
|
sessions have support for a =last_result= type function, which
|
||||||
|
returns the result of the last input without re-calculation. If
|
||||||
|
widely enough present this would be the ideal solution to a
|
||||||
|
combination of functional and imperative styles.
|
||||||
|
|
||||||
|
None of these solutions seem very desirable, but for now I don't see
|
||||||
|
what else would be possible.
|
||||||
|
|
||||||
|
Of these options I was leaning towards (1) and (4) but now believe
|
||||||
|
that if it is possible option (5) will be ideal.
|
||||||
|
|
||||||
|
**** (1) both functional and imperative evaluation
|
||||||
|
Pros
|
||||||
|
- can take advantage of built in functions for sending regions to the
|
||||||
|
inferior process
|
||||||
|
- retains the proven tested and working functional wrappers
|
||||||
|
|
||||||
|
Cons
|
||||||
|
- introduces the complication of keeping track of which type of
|
||||||
|
evaluation is best suited to a particular context
|
||||||
|
- the current functional wrappers may require some changes in order to
|
||||||
|
include the existing global context
|
||||||
|
|
||||||
|
**** (4) exploit language meta-programming constructs to explicitly evaluate code
|
||||||
|
Pros
|
||||||
|
- only one type of evaluation
|
||||||
|
|
||||||
|
Cons
|
||||||
|
- some languages may not have sufficient meta-programming constructs
|
||||||
|
|
||||||
|
**** (5) exploit some =last_value= functionality if present
|
||||||
|
|
||||||
|
Need to ensure that most languages have such a function, those without
|
||||||
|
will simply have to implement their own similar solution...
|
||||||
|
|
||||||
|
| language | =last_value= function |
|
||||||
|
|------------+-----------------------------|
|
||||||
|
| R | .Last.value |
|
||||||
|
| ruby | _ |
|
||||||
|
| python | _ |
|
||||||
|
| shell | see [[* last command for shells][last command for shells]] |
|
||||||
|
| emacs-lisp | see [[* emacs-lisp will be a special case][special-case]] |
|
||||||
|
|
||||||
|
#+srcname: task-last-value
|
||||||
|
#+begin_src ruby
|
||||||
|
82 + 18
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
***** last command for shells
|
||||||
|
Do this using the =tee= shell command, and continually pipe the output
|
||||||
|
to a file.
|
||||||
|
|
||||||
|
Got this idea from the following [[http://linux.derkeiler.com/Mailing-Lists/Fedora/2004-01/0898.html][email-thread]].
|
||||||
|
|
||||||
|
suggested from mailing list
|
||||||
|
|
||||||
|
#+srcname: bash-save-last-output-to-file
|
||||||
|
#+begin_src sh
|
||||||
|
while read line
|
||||||
|
do
|
||||||
|
bash -c "$line" | tee /tmp/last.out1
|
||||||
|
mv /tmp/last.out1 /tmp/last.out
|
||||||
|
done
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
another proposed solution from the above thread
|
||||||
|
|
||||||
|
#+srcname: bash-save-in-variable
|
||||||
|
#+begin_src sh
|
||||||
|
#!/bin/bash
|
||||||
|
# so - Save Output. Saves output of command in OUT shell variable.
|
||||||
|
OUT=`$*`
|
||||||
|
echo $OUT
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
and another
|
||||||
|
|
||||||
|
#+begin_quote
|
||||||
|
.inputrc:
|
||||||
|
"^[k": accept-line
|
||||||
|
"^M": " | tee /tmp/h_lastcmd.out ^[k"
|
||||||
|
|
||||||
|
.bash_profile:
|
||||||
|
export __=/tmp/h_lastcmd.out
|
||||||
|
|
||||||
|
If you try it, Alt-k will stand for the old Enter; use "command $__" to
|
||||||
|
access the last output.
|
||||||
|
|
||||||
|
Best,
|
||||||
|
|
||||||
|
--
|
||||||
|
|
||||||
|
Herculano de Lima Einloft Neto
|
||||||
|
#+end_quote
|
||||||
|
|
||||||
|
***** emacs-lisp will be a special case
|
||||||
|
While it is possible for emacs-lisp to be run in a console type
|
||||||
|
environment (see the =elim= function) it is *not* possible to run
|
||||||
|
emacs-lisp in a different *session*. Meaning any variable set top
|
||||||
|
level of the console environment will be set *everywhere* inside
|
||||||
|
emacs. For this reason I think that it doesn't make any sense to
|
||||||
|
worry about session support for emacs-lisp.
|
||||||
|
|
||||||
|
*** Further thoughts on 'scripting' vs. functional approaches
|
||||||
|
|
||||||
|
These are just thoughts, I don't know how sure I am about this.
|
||||||
|
And again, perhaps I'm not saying anything very radical, just that
|
||||||
|
it would be nice to have some options supporting things like
|
||||||
|
receiving text output in the org buffer.
|
||||||
|
|
||||||
|
I can see that you've already gone some way down the road towards
|
||||||
|
the 'last value' approach, so sorry if my comments come rather
|
||||||
|
late. I am concerned that we are not giving sufficient attention
|
||||||
|
to stdout / the text that is returned by the interpreters. In
|
||||||
|
contrast, many of our potential users will be accustomed to a
|
||||||
|
'scripting' approach, where they are outputting text at various
|
||||||
|
points in the code block, not just at the end. I am leaning
|
||||||
|
towards thinking that we should have 2 modes of evaluation:
|
||||||
|
'script' mode, and 'functional' mode.
|
||||||
|
|
||||||
|
In script mode, evaluation of a code block would result in *all*
|
||||||
|
text output from that code block appearing as output in the org
|
||||||
|
buffer, presumably as an #+begin_example...#+end_example. There
|
||||||
|
could be an :echo option controlling whether the input commands
|
||||||
|
also appear in the output. [This is like Sweave].
|
||||||
|
|
||||||
|
In functional mode, the *result* of the code block is available as
|
||||||
|
an elisp object, and may appear in the org buffer as an org
|
||||||
|
table/string, via the mechanisms you have developed already.
|
||||||
|
|
||||||
|
One thing I'm wondering about is whether, in script mode, there
|
||||||
|
simply should not be a return value. Perhaps this is not so
|
||||||
|
different from what exists: script mode would be new, and what
|
||||||
|
exists currently would be functional mode.
|
||||||
|
|
||||||
|
I think it's likely that, while code evaluation will be exciting
|
||||||
|
to people, a large majority of our users in a large majority of
|
||||||
|
their usage will not attempt to actually use the return value from
|
||||||
|
a source code block in any meaningful way. In that case, it seems
|
||||||
|
rather restrictive to only allow them to see output from the end
|
||||||
|
of the code block.
|
||||||
|
|
||||||
|
Instead I think the most accessible way to introduce org-babel to
|
||||||
|
people, at least while they are learning it, is as an immensely
|
||||||
|
powerful environment in which to embed their 'scripts', which now
|
||||||
|
also allows them to 'run' their 'scripts'. Especially as such
|
||||||
|
people are likely to be the least capable of the user-base, a
|
||||||
|
possible design-rule would be to make the scripting style of usage
|
||||||
|
easy (default?), perhaps requiring a special option to enable a
|
||||||
|
functional style. Those who will use the functional style won't
|
||||||
|
have a problem understanding what's going on, whereas the 'skript
|
||||||
|
kiddies' might not even know the syntax for defining a function in
|
||||||
|
their language of choice. And of course we can allow the user to
|
||||||
|
set a variable in their .emacs controlling the preference, so that
|
||||||
|
functional users are not inconveniennced by having to provide
|
||||||
|
header args the whole time.
|
||||||
|
|
||||||
|
Please don't get the impression that I am down-valuing the
|
||||||
|
functional style of org-babel. I am constantly horrified at the
|
||||||
|
messy 'scripts' that my colleagues produce in perl or R or
|
||||||
|
whatever! Nevertheless that seems to be how a lot of people work.
|
||||||
|
|
||||||
|
I think you were leaning towards the last-value approach because
|
||||||
|
it offered the possibility of unified code supporting both the
|
||||||
|
single evaluation environment and the functional style. If you
|
||||||
|
agree with any of the above then perhaps it will impact upon this
|
||||||
|
and mean that the code in the two branches has to differ a bit. In
|
||||||
|
that case, functional mode could perhaps after all evaluate each
|
||||||
|
code block in its own environment, thus (re)approaching 'true'
|
||||||
|
functional programming (side-effects are hard to achieve).
|
||||||
|
|
||||||
|
#+begin_src sh
|
||||||
|
ls > files
|
||||||
|
echo "There are `wc -l files` files in this directory"
|
||||||
|
|
||||||
|
#+end_src
|
||||||
|
|
||||||
*** TODO rework all source codes to use inferior-processes-buffers
|
*** TODO rework all source codes to use inferior-processes-buffers
|
||||||
|
|
||||||
this will involve...
|
this will involve...
|
||||||
|
@ -198,6 +415,27 @@ this will involve...
|
||||||
3) functions for retrieving results from the *sessions* buffers which
|
3) functions for retrieving results from the *sessions* buffers which
|
||||||
can be overridden by each source code
|
can be overridden by each source code
|
||||||
|
|
||||||
|
**** DONE R
|
||||||
|
|
||||||
|
#+srcname: task-R-with-inf-process-buffer
|
||||||
|
#+begin_src R
|
||||||
|
a <- 8
|
||||||
|
b <- 9
|
||||||
|
c <- 10
|
||||||
|
a + b
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
**** TODO Ruby
|
||||||
|
|
||||||
|
#+srcname: ruby-use-last-output
|
||||||
|
#+begin_src ruby
|
||||||
|
a = 1
|
||||||
|
b = 2
|
||||||
|
c = 3
|
||||||
|
(a + b) * c
|
||||||
|
#+end_src
|
||||||
|
|
||||||
|
|
||||||
*** TODO implement a *session* header argument
|
*** TODO implement a *session* header argument
|
||||||
|
|
||||||
use this header argument to override the default *session* buffer
|
use this header argument to override the default *session* buffer
|
||||||
|
@ -284,7 +522,6 @@ org-mode core
|
||||||
This should be implemented in the org-mode core
|
This should be implemented in the org-mode core
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
*** DEFERRED send code to inferior process
|
*** DEFERRED send code to inferior process
|
||||||
Another thought on this topic: I think we will want users to send
|
Another thought on this topic: I think we will want users to send
|
||||||
chunks of code to the interpreter from within the *Org Edit Src*
|
chunks of code to the interpreter from within the *Org Edit Src*
|
||||||
|
@ -347,6 +584,11 @@ waiting for guidance from those more familiar with yasnippets
|
||||||
- [[file:lisp/org-babel-ref.el::TODO%20allow%20searching%20for%20names%20in%20other%20buffers][org-babel-ref.el:searching-in-other-buffers]]
|
- [[file:lisp/org-babel-ref.el::TODO%20allow%20searching%20for%20names%20in%20other%20buffers][org-babel-ref.el:searching-in-other-buffers]]
|
||||||
- [[file:lisp/org-babel.el::defun%20org-babel%20find%20named%20result%20name][org-babel.el#org-babel-find-named-result]]
|
- [[file:lisp/org-babel.el::defun%20org-babel%20find%20named%20result%20name][org-babel.el#org-babel-find-named-result]]
|
||||||
|
|
||||||
|
** TODO figure out how to handle errors during evaluation
|
||||||
|
R has a try function, with error handling, along the lines of
|
||||||
|
python. I bet ruby does too. Maybe more of an issue for functional
|
||||||
|
style; in my proposed scripting style the error just gets dumped to
|
||||||
|
the org buffer and the user is thus alerted.
|
||||||
** TODO figure out how to handle graphic output
|
** TODO figure out how to handle graphic output
|
||||||
This is listed under [[* graphical output][graphical output]] in out objectives.
|
This is listed under [[* graphical output][graphical output]] in out objectives.
|
||||||
|
|
||||||
|
@ -545,7 +787,6 @@ msg + "_y_python"
|
||||||
(concat msg "_elisp")
|
(concat msg "_elisp")
|
||||||
#+end_src
|
#+end_src
|
||||||
|
|
||||||
|
|
||||||
** PROPOSED conversion between org-babel and noweb (e.g. .Rnw) format
|
** PROPOSED conversion between org-babel and noweb (e.g. .Rnw) format
|
||||||
I haven't thought about this properly. Just noting it down. What
|
I haven't thought about this properly. Just noting it down. What
|
||||||
Sweave uses is called "R noweb" (.Rnw).
|
Sweave uses is called "R noweb" (.Rnw).
|
||||||
|
|
Loading…
Reference in a new issue