diff --git a/org-babel.org b/org-babel.org index ad7ec721a..5bcf4fb08 100644 --- a/org-babel.org +++ b/org-babel.org @@ -1,7 +1,7 @@ #+OPTIONS: H:3 num:nil toc:t #+TITLE: org-babel --- facilitating communication between programming languages and people #+SEQ_TODO: TODO PROPOSED | DONE DEFERRED REJECTED -#+STARTUP: oddeven +#+STARTUP: oddeven hideblocks * Introduction @@ -114,8 +114,9 @@ table, allowing the test suite to be run be evaluation of the table and the results to be collected in the same table. -* Tasks [22/37] -** TODO Create objects in top level (global) environment in R? [0/5] +* Tasks [22/38] +** TODO Create objects in top level (global) environment [0/5] +*sessions* *** initial requirement statement [DED] At the moment, objects created by computations performed in the @@ -189,6 +190,222 @@ or if you think any parts might be confusing for people coming from Sweave. I'll hopefully find some time to work on this later in the week. +*** can functional and interpreted/interactive models coexist? + +Even though both of these use the same =*R*= buffer the value of =a= +is not preserved because it is assigned inside of a functional +wrapper. + +#+srcname: task-R-sessions +#+begin_src R +a <- 9 +b <- 21 +a + b +#+end_src + +#+srcname: task-R-same-session +#+begin_src R +a +#+end_src + +This functional wrapper was implemented in order to efficiently return +the results of the execution of the entire source code block. However +it inhibits the evaluation of source code blocks in the top level, +which would allow for persistence of variable assignment across +evaluations. How can we allow *both* evaluation in the top level, and +efficient capture of the return value of an entire source code block +in a language independent manner? + +Possible solutions... +1) we can't so we will have to implement two types of evaluation + depending on which is appropriate (functional or imperative) +2) we remove the functional wrapper and parse the source code block + into it's top level statements (most often but not always on line + breaks) so that we can isolate the final segment which is our + return value. +3) we add some sort of "#+return" line to the code block +4) we take advantage of each languages support for meta-programming + through =eval= type functions, and use said to evaluate the entire + blocks in such a way that their environment can be combined with the + global environment, and their results are still captured. +5) I believe that most modern languages which support interactive + sessions have support for a =last_result= type function, which + returns the result of the last input without re-calculation. If + widely enough present this would be the ideal solution to a + combination of functional and imperative styles. + +None of these solutions seem very desirable, but for now I don't see +what else would be possible. + +Of these options I was leaning towards (1) and (4) but now believe +that if it is possible option (5) will be ideal. + +**** (1) both functional and imperative evaluation +Pros +- can take advantage of built in functions for sending regions to the + inferior process +- retains the proven tested and working functional wrappers + +Cons +- introduces the complication of keeping track of which type of + evaluation is best suited to a particular context +- the current functional wrappers may require some changes in order to + include the existing global context + +**** (4) exploit language meta-programming constructs to explicitly evaluate code +Pros +- only one type of evaluation + +Cons +- some languages may not have sufficient meta-programming constructs + +**** (5) exploit some =last_value= functionality if present + +Need to ensure that most languages have such a function, those without +will simply have to implement their own similar solution... + +| language | =last_value= function | +|------------+-----------------------------| +| R | .Last.value | +| ruby | _ | +| python | _ | +| shell | see [[* last command for shells][last command for shells]] | +| emacs-lisp | see [[* emacs-lisp will be a special case][special-case]] | + +#+srcname: task-last-value +#+begin_src ruby +82 + 18 +#+end_src + +***** last command for shells +Do this using the =tee= shell command, and continually pipe the output +to a file. + +Got this idea from the following [[http://linux.derkeiler.com/Mailing-Lists/Fedora/2004-01/0898.html][email-thread]]. + +suggested from mailing list + +#+srcname: bash-save-last-output-to-file +#+begin_src sh +while read line +do + bash -c "$line" | tee /tmp/last.out1 + mv /tmp/last.out1 /tmp/last.out +done +#+end_src + +another proposed solution from the above thread + +#+srcname: bash-save-in-variable +#+begin_src sh +#!/bin/bash +# so - Save Output. Saves output of command in OUT shell variable. +OUT=`$*` +echo $OUT +#+end_src + +and another + +#+begin_quote +.inputrc: +"^[k": accept-line +"^M": " | tee /tmp/h_lastcmd.out ^[k" + +.bash_profile: +export __=/tmp/h_lastcmd.out + +If you try it, Alt-k will stand for the old Enter; use "command $__" to +access the last output. + +Best, + +-- + +Herculano de Lima Einloft Neto +#+end_quote + +***** emacs-lisp will be a special case +While it is possible for emacs-lisp to be run in a console type +environment (see the =elim= function) it is *not* possible to run +emacs-lisp in a different *session*. Meaning any variable set top +level of the console environment will be set *everywhere* inside +emacs. For this reason I think that it doesn't make any sense to +worry about session support for emacs-lisp. + +*** Further thoughts on 'scripting' vs. functional approaches + + These are just thoughts, I don't know how sure I am about this. + And again, perhaps I'm not saying anything very radical, just that + it would be nice to have some options supporting things like + receiving text output in the org buffer. + + I can see that you've already gone some way down the road towards + the 'last value' approach, so sorry if my comments come rather + late. I am concerned that we are not giving sufficient attention + to stdout / the text that is returned by the interpreters. In + contrast, many of our potential users will be accustomed to a + 'scripting' approach, where they are outputting text at various + points in the code block, not just at the end. I am leaning + towards thinking that we should have 2 modes of evaluation: + 'script' mode, and 'functional' mode. + + In script mode, evaluation of a code block would result in *all* + text output from that code block appearing as output in the org + buffer, presumably as an #+begin_example...#+end_example. There + could be an :echo option controlling whether the input commands + also appear in the output. [This is like Sweave]. + + In functional mode, the *result* of the code block is available as + an elisp object, and may appear in the org buffer as an org + table/string, via the mechanisms you have developed already. + + One thing I'm wondering about is whether, in script mode, there + simply should not be a return value. Perhaps this is not so + different from what exists: script mode would be new, and what + exists currently would be functional mode. + + I think it's likely that, while code evaluation will be exciting + to people, a large majority of our users in a large majority of + their usage will not attempt to actually use the return value from + a source code block in any meaningful way. In that case, it seems + rather restrictive to only allow them to see output from the end + of the code block. + + Instead I think the most accessible way to introduce org-babel to + people, at least while they are learning it, is as an immensely + powerful environment in which to embed their 'scripts', which now + also allows them to 'run' their 'scripts'. Especially as such + people are likely to be the least capable of the user-base, a + possible design-rule would be to make the scripting style of usage + easy (default?), perhaps requiring a special option to enable a + functional style. Those who will use the functional style won't + have a problem understanding what's going on, whereas the 'skript + kiddies' might not even know the syntax for defining a function in + their language of choice. And of course we can allow the user to + set a variable in their .emacs controlling the preference, so that + functional users are not inconveniennced by having to provide + header args the whole time. + + Please don't get the impression that I am down-valuing the + functional style of org-babel. I am constantly horrified at the + messy 'scripts' that my colleagues produce in perl or R or + whatever! Nevertheless that seems to be how a lot of people work. + + I think you were leaning towards the last-value approach because + it offered the possibility of unified code supporting both the + single evaluation environment and the functional style. If you + agree with any of the above then perhaps it will impact upon this + and mean that the code in the two branches has to differ a bit. In + that case, functional mode could perhaps after all evaluate each + code block in its own environment, thus (re)approaching 'true' + functional programming (side-effects are hard to achieve). + +#+begin_src sh +ls > files +echo "There are `wc -l files` files in this directory" + +#+end_src + *** TODO rework all source codes to use inferior-processes-buffers this will involve... @@ -198,6 +415,27 @@ this will involve... 3) functions for retrieving results from the *sessions* buffers which can be overridden by each source code +**** DONE R + +#+srcname: task-R-with-inf-process-buffer +#+begin_src R +a <- 8 +b <- 9 +c <- 10 +a + b +#+end_src + +**** TODO Ruby + +#+srcname: ruby-use-last-output +#+begin_src ruby +a = 1 +b = 2 +c = 3 +(a + b) * c +#+end_src + + *** TODO implement a *session* header argument use this header argument to override the default *session* buffer @@ -284,7 +522,6 @@ org-mode core This should be implemented in the org-mode core - *** DEFERRED send code to inferior process Another thought on this topic: I think we will want users to send chunks of code to the interpreter from within the *Org Edit Src* @@ -347,6 +584,11 @@ waiting for guidance from those more familiar with yasnippets - [[file:lisp/org-babel-ref.el::TODO%20allow%20searching%20for%20names%20in%20other%20buffers][org-babel-ref.el:searching-in-other-buffers]] - [[file:lisp/org-babel.el::defun%20org-babel%20find%20named%20result%20name][org-babel.el#org-babel-find-named-result]] +** TODO figure out how to handle errors during evaluation + R has a try function, with error handling, along the lines of + python. I bet ruby does too. Maybe more of an issue for functional + style; in my proposed scripting style the error just gets dumped to + the org buffer and the user is thus alerted. ** TODO figure out how to handle graphic output This is listed under [[* graphical output][graphical output]] in out objectives. @@ -545,7 +787,6 @@ msg + "_y_python" (concat msg "_elisp") #+end_src - ** PROPOSED conversion between org-babel and noweb (e.g. .Rnw) format I haven't thought about this properly. Just noting it down. What Sweave uses is called "R noweb" (.Rnw).