From adcfa3c54bb6699edfbe66c387908bdc3d1b05a7 Mon Sep 17 00:00:00 2001 From: Dan Davison Date: Tue, 30 Jun 2009 20:46:51 -0400 Subject: [PATCH 1/6] Some todos/bug notes in org-babel.org --- lisp/org-babel-tangle.el | 8 +- org-babel.org | 171 +++++++++++++++++++++------------------ 2 files changed, 99 insertions(+), 80 deletions(-) diff --git a/lisp/org-babel-tangle.el b/lisp/org-babel-tangle.el index 58a98a4d7..a33849594 100644 --- a/lisp/org-babel-tangle.el +++ b/lisp/org-babel-tangle.el @@ -39,7 +39,7 @@ and shebang(#!) line to use when writing out the language to file.") (defun org-babel-tangle () - "Extract the bodies of all source code blocks form the current + "Extract the bodies of all source code blocks from the current file into their own source-specific files." (interactive) (save-excursion @@ -57,13 +57,15 @@ file into their own source-specific files." (with-temp-file filename (funcall lang-f) (when she-bang (insert (concat she-bang "\n"))) - (comment-region (point) (progn (insert "generated by org-babel-tangle") (point))) + (comment-region + (point) (progn (insert "generated by org-babel-tangle") (point))) (mapc #'org-babel-spec-to-string (reverse specs))))) ;; if there are multiple sessions then break out by session (if (> (length by-session) 1) (mapc (lambda (session-pair) (setq block-counter (+ block-counter (length (cdr session-pair)))) - (to-file (format "%s-%s.%s" base-name (car session-pair) ext) (cdr session-pair))) + (to-file (format + "%s-%s.%s" base-name (car session-pair) ext) (cdr session-pair))) by-session) (setq block-counter (+ block-counter (length (cdr (car by-session))))) (to-file (format "%s.%s" base-name ext) (cdr (car by-session))))))) diff --git a/org-babel.org b/org-babel.org index f9db4e864..743019610 100644 --- a/org-babel.org +++ b/org-babel.org @@ -1,5 +1,5 @@ #+TITLE: org-babel --- facilitating communication between programming languages and people -#+SEQ_TODO: TODO PROPOSED | DONE DEFERRED REJECTED +#+SEQ_TODO: TODO PROPOSED STARTED | DONE DEFERRED REJECTED #+OPTIONS: H:3 num:nil toc:t #+STARTUP: oddeven hideblocks @@ -192,7 +192,34 @@ would then be [[#sandbox][the sandbox]]. * Tasks [27/42] -** TODO support for working with =*Org Edit Src Example*= buffers [1/4] +** TODO support for working with =*Org Edit Src Example*= buffers [2/4] +*** TODO optionally evaluate header references when we switch to =*Org Edit Src*= buffer +That seems to imply that the header references need to be evaluated +and transformed into the target language object when we hit C-c ' to +enter the *Org Edit Src* buffer [DED] + +Good point, I heartily agree that this should be supported [Eric] + +(or at least before the first time we attempt to evaluate code in that +buffer -- I suppose there might be an argument for lazy evaluation, in +case someone hits C-c ' but is "just looking" and not actually +evaluating anything.) Of course if evaluating the reference is +computationally intensive then the user might have to wait before they +get the *Org Edit Src* buffer. [DED] + +I fear that it may be hard to anticipate when the references will be +needed, some major-modes do on-the-fly evaluation while the buffer is +being edited. I think that we should either do this before the buffer +is opened or not at all, specifically I think we should resolve +references if the user calls C-c ' with a prefix argument. Does that +sound reasonable? [Eric] + +Yes [Dan] + +[Dan] So now that we have org-src-mode and org-src-mode-hook, I guess +org-babel should do this by using the hook to make sure that, when C-c +C-' is issued on a source block, any references are resolved and +assignments are made in the appropriate session. *** TODO set buffer-local-process variables appropriately [DED] I think something like this would be great. You've probably already thought of this, but just to note it down: it would be really @@ -207,8 +234,19 @@ I had not thought of that, but I agree whole heartedly. [Eric] Once this is done every variable should be able to dump regions into their inferior-process buffer using major-mode functions. +*** DEFERRED send code to inferior process +Another thought on this topic: I think we will want users to send +chunks of code to the interpreter from within the *Org Edit Src* +buffer, and I think that's what you have in mind already. In ESS that +is done using the ess-eval-* functions. [DED] -*** TODO some possible requests/proposed changes for Carsten [2/3] +I think we can leave this up to the major-mode in the source code +buffer, as almost every source-code major mode will have functions for +doing things like sending regions to the inferior process. If +anything we might need to set the value of the buffer local inferior +process variable. [Eric] + +*** DONE some possible requests/proposed changes for Carsten [4/4] While I remember, some possible requests/proposed changes for Carsten come to mind in that regard: @@ -232,7 +270,8 @@ buffer." I think this is great, but I think it should be implemented in the org-mode core -**** TODO Rename buffer and minor mode? + +**** DEFERRED Rename buffer and minor mode? Something shorter than *Org Edit Src Example* for the buffer name. org-babel is bringing org's source code interaction to a level of maturity where the 'example' is no longer @@ -253,45 +292,10 @@ org-mode core perhaps we will also have more functionality to add to that minor mode, making it even more of a misnomer. Perhaps something like org-src-mode would be better. - -**** DEFERRED a hook called when the src edit buffer is created -This should be implemented in the org-mode core +**** DONE Changed minor mode name and added hooks - -*** DEFERRED send code to inferior process -Another thought on this topic: I think we will want users to send -chunks of code to the interpreter from within the *Org Edit Src* -buffer, and I think that's what you have in mind already. In ESS that -is done using the ess-eval-* functions. [DED] - -I think we can leave this up to the major-mode in the source code -buffer, as almost every source-code major mode will have functions for -doing things like sending regions to the inferior process. If -anything we might need to set the value of the buffer local inferior -process variable. [Eric] - -*** TODO optionally evaluate header references when we switch to =*Org Edit Src*= buffer -That seems to imply that the header references need to be evaluated -and transformed into the target language object when we hit C-c ' to -enter the *Org Edit Src* buffer [DED] - -Good point, I heartily agree that this should be supported [Eric] - -(or at least before the first time we attempt to evaluate code in that -buffer -- I suppose there might be an argument for lazy evaluation, in -case someone hits C-c ' but is "just looking" and not actually -evaluating anything.) Of course if evaluating the reference is -computationally intensive then the user might have to wait before they -get the *Org Edit Src* buffer. [DED] - -I fear that it may be hard to anticipate when the references will be -needed, some major-modes do on-the-fly evaluation while the buffer is -being edited. I think that we should either do this before the buffer -is opened or not at all, specifically I think we should resolve -references if the user calls C-c ' with a prefix argument. Does that -sound reasonable? [Eric] - -Yes [Dan] +**** DEFERRED a hook called when the src edit buffer is created + This should be implemented in the org-mode core ** TODO resolve references to other org buffers/files This would allow source blocks to call upon tables, source-blocks, @@ -343,12 +347,13 @@ msg + " y python" #+begin_src emacs-lisp :var msg="org-babel speaks" (concat msg " elisp") #+end_src -** TODO share org-babel +** STARTED share org-babel [1/4] how should we share org-babel? -- post to org-mode and ess mailing lists -- create a org-babel page on worg -- create a short screencast demonstrating org-babel in action +*** DONE post to org-mode +*** TODO post to ess mailing list +*** TODO create a org-babel page on worg +*** TODO create a short screencast demonstrating org-babel in action *** examples we need to think up some good examples @@ -1908,41 +1913,21 @@ This could probably be added to [[file:lisp/org-babel-script.el][org-babel-scrip (see [[* file result types][file result types]]) -* Bugs [16/21] +* Bugs [17/23] +** TODO Allow source blocks to be recognised when #+ are not first characters on the line + I think Carsten has recently altered the core so that #+ can have + preceding whitespace, at least for literal/code examples. org-babel + should support this. +** PROPOSED make :results replace the default? + I'm tending to think that appending results to pre-existing results + creates mess, and that the cleaner `replace' option should be the + default. E.g. when a source block creates an image, we would want + that to be updated, rather than have a new one be added. ** PROPOSED external shell execution can't isolate return values I have no idea how to do this as of yet. The result is that when shell functions are run w/o a session there is no difference between the =output= and =value= result arguments. -** TODO weird escaped characters in shell prompt break shell evaluation - E.g. this doesn't work. Should the shell sessions set a sane prompt - when they start up? Or is it a question of altering - comint-prompt-regexp? Or altering org-babel regexps? - -#+begin_src sh - black=30 ; red=31 ; green=32 ; yellow=33 ; blue=34 ; magenta=35 ; cyan=36 ; white=37 - prompt_col=$red - prompt_char='>' - export PS1="\[\033[${prompt_col}m\]\w${prompt_char} \[\033[0m\]" -#+end_src - - I just pushed a good amount of changes, could you see if your shell - problems still exist? - - The problem's still there. Specifically, aIui, at [[file:lisp/langs/org-babel-sh.el::raw%20org%20babel%20comint%20with%20output%20buffer%20org%20babel%20sh%20eoe%20output%20nil%20insert%20full%20body%20comint%20send%20input%20nil%20t][this line]] of - org-babel-sh.el, raw gets the value - -("" " Sun Jun 14 19:26:24 EDT 2009\n" " org_babel_sh_eoe\n" " ") - - and therefore (member org-babel-sh-eoe-output ...) fails - - I think that `comint-prompt-regexp' needs to be altered to match - the shell prompt. This shouldn't be too difficult to do by hand, - using the `regexp-builder' command and should probably be part of - the user's regular emacs init. I can't think of a way for us to - set this automatically, and we are SOL without a regexp to match - the prompt. - ** TODO non-orgtbl formatted lists for example @@ -1974,6 +1959,35 @@ even a third" E.g. the pie chart example. Despite the save-window-excursion in org-babel-execute:R. (I never learned how to do this properly: org-R jumps all over the place...) + +** DEFERRED weird escaped characters in shell prompt break shell evaluation + E.g. this doesn't work. Should the shell sessions set a sane prompt + when they start up? Or is it a question of altering + comint-prompt-regexp? Or altering org-babel regexps? + +#+begin_src sh + black=30 ; red=31 ; green=32 ; yellow=33 ; blue=34 ; magenta=35 ; cyan=36 ; white=37 + prompt_col=$red + prompt_char='>' + export PS1="\[\033[${prompt_col}m\]\w${prompt_char} \[\033[0m\]" +#+end_src + + I just pushed a good amount of changes, could you see if your shell + problems still exist? + + The problem's still there. Specifically, aIui, at [[file:lisp/langs/org-babel-sh.el::raw%20org%20babel%20comint%20with%20output%20buffer%20org%20babel%20sh%20eoe%20output%20nil%20insert%20full%20body%20comint%20send%20input%20nil%20t][this line]] of + org-babel-sh.el, raw gets the value + +("" " Sun Jun 14 19:26:24 EDT 2009\n" " org_babel_sh_eoe\n" " ") + + and therefore (member org-babel-sh-eoe-output ...) fails + + I think that `comint-prompt-regexp' needs to be altered to match + the shell prompt. This shouldn't be too difficult to do by hand, + using the `regexp-builder' command and should probably be part of + the user's regular emacs init. I can't think of a way for us to + set this automatically, and we are SOL without a regexp to match + the prompt. ** DONE ruby evaluation not working under ubuntu emacs 23 With emacs 23.0.91.1 on ubuntu, for C-h f run-ruby I have the following, which seems to conflict with [[file:lisp/langs/org-babel-ruby.el::let%20session%20buffer%20save%20window%20excursion%20run%20ruby%20nil%20session%20current%20buffer][this line]] in org-babel-ruby.el. @@ -2157,7 +2171,7 @@ mean(mean(table)) mean(table) #+end_src -** DEFERRED org bug/request: prevent certain org behaviour within code blocks +** DONE org bug/request: prevent certain org behaviour within code blocks E.g. [[]] gets recognised as a link (when there's text inside the brackets). This is bad for R code at least, and more generally could be argued to be inappropriate. Is it difficult to get org to @@ -2166,6 +2180,9 @@ mean(table) I believe Carsten addressed this recently on the mailing list with the comment that it was indeed a difficult issue. I believe this may be one area where we could wait for an upstream (org-mode) fix. + + [Dan] Carsten has fixed this now in the core. + ** DONE with :results replace, non-table output doesn't replace table output And vice versa. E.g. Try this first with table and then with len(table) [DED] #+begin_src python :var table=sandbox :results replace From de09a9813e239a6d67bae5c1080182371510d36e Mon Sep 17 00:00:00 2001 From: Eric Schulte Date: Wed, 1 Jul 2009 16:16:30 -0700 Subject: [PATCH 2/6] just getting started --- org-babel.org | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/org-babel.org b/org-babel.org index f9db4e864..f0c5a87d0 100644 --- a/org-babel.org +++ b/org-babel.org @@ -139,7 +139,13 @@ table, allowing the test suite to be run be evaluation of the table and the results to be collected in the same table. *** Emacs initialization files stored in Org-Mode buffers -Once org-babel-tangle is completed this could be a very compelling use case. +Using `org-babel-tangle' it is possible to embed your Emacs +initialization into org-mode files. This allows for folding, +note-taking, todo's etc... embedded with the source-code of your Emacs +initialization, and through org-mode's publishing features aids in +sharing your customizations with others. + +**** my customization ** features From bd5b9dd32f0e3bfdbe57860d75c5118413dc3954 Mon Sep 17 00:00:00 2001 From: Dan Davison Date: Sun, 5 Jul 2009 12:55:03 -0400 Subject: [PATCH 3/6] another todo: ^[ \t]*#+ --- org-babel.org | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/org-babel.org b/org-babel.org index 743019610..965f8393c 100644 --- a/org-babel.org +++ b/org-babel.org @@ -25,11 +25,10 @@ In this document: - The [[* Sandbox][Sandbox]] :: demonstrates much of the early/basic functionality through commented source-code blocks. -Also see the [[file:library-of-babel.org][Library of Babel]] an extensible collection of ready-made +Also see the [[file:library-of-babel.org][Library of Babel]], an extensible collection of ready-made and easily-shortcut-callable source-code blocks for handling common tasks. - * Introduction Org-Babel enables *communication* between programming languages and @@ -1913,7 +1912,7 @@ This could probably be added to [[file:lisp/org-babel-script.el][org-babel-scrip (see [[* file result types][file result types]]) -* Bugs [17/23] +* Bugs [17/24] ** TODO Allow source blocks to be recognised when #+ are not first characters on the line I think Carsten has recently altered the core so that #+ can have preceding whitespace, at least for literal/code examples. org-babel From 6ade5446745820f957b1d6160fccfea72a6aa6d0 Mon Sep 17 00:00:00 2001 From: Dan Davison Date: Sun, 5 Jul 2009 19:40:31 -0400 Subject: [PATCH 4/6] Unimportant: output changes in org-babel.org --- org-babel.org | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/org-babel.org b/org-babel.org index 965f8393c..869fb587e 100644 --- a/org-babel.org +++ b/org-babel.org @@ -2645,12 +2645,14 @@ out... table.first.join(" - ") #+end_src -: "1 - 2 - 3" +#+resname: +: 1 - 2 - 3 -#+begin_src python :var table=sandbox :results replace +#+begin_src python :var table=sandbox table[0] #+end_src +#+resname: | 1 | 2 | 3 | #+begin_src ruby :var table=sandbox :results replace From 37569346f82afa31edccc7a8bbbe69a6fcce0dee Mon Sep 17 00:00:00 2001 From: Eric Schulte Date: Mon, 6 Jul 2009 19:36:44 -0700 Subject: [PATCH 5/6] stole a couple of moments to add discussion and a new TODO --- org-babel.org | 42 ++++++++++++++++++++++++++++++++++-------- 1 file changed, 34 insertions(+), 8 deletions(-) diff --git a/org-babel.org b/org-babel.org index 869fb587e..56a6d1b52 100644 --- a/org-babel.org +++ b/org-babel.org @@ -1,5 +1,5 @@ #+TITLE: org-babel --- facilitating communication between programming languages and people -#+SEQ_TODO: TODO PROPOSED STARTED | DONE DEFERRED REJECTED +#+SEQ_TODO: PROPOSED TODO STARTED | DONE DEFERRED REJECTED #+OPTIONS: H:3 num:nil toc:t #+STARTUP: oddeven hideblocks @@ -190,7 +190,7 @@ would then be [[#sandbox][the sandbox]]. #+end_src -* Tasks [27/42] +* Tasks [27/43] ** TODO support for working with =*Org Edit Src Example*= buffers [2/4] *** TODO optionally evaluate header references when we switch to =*Org Edit Src*= buffer That seems to imply that the header references need to be evaluated @@ -322,6 +322,17 @@ languages which almost always produce graphical output should set results in the buffer. Then if there is a combination of =silent= and =file= =:results= headers we could drop the results to a temp buffer and pop open that buffer... +** TODO =\C-c \C-o= to open results of source block +by adding a =defadvice= to =org-open-at-point= we can use the common +=\C-c \C-o= keybinding to open the results of a source-code block. +This would be especially useful for source-code blocks which generate +graphical results and insert a file link as the results in the +org-mode buffer. (see [[* figure out how to handle graphic output][TODO figure out how to handle graphic output]]). +This could also act reasonably with other results types... + +- file :: use org-open-at-point to open the file +- scalar :: open results unquoted in a new buffer +- tabular :: export the table to a new buffer and open that buffer ** TODO Finalise behaviour regarding vector/scalar output *** DONE Stop spaces causing vector output @@ -1912,20 +1923,18 @@ This could probably be added to [[file:lisp/org-babel-script.el][org-babel-scrip (see [[* file result types][file result types]]) -* Bugs [17/24] +* Bugs [17/23] ** TODO Allow source blocks to be recognised when #+ are not first characters on the line I think Carsten has recently altered the core so that #+ can have preceding whitespace, at least for literal/code examples. org-babel should support this. -** PROPOSED make :results replace the default? +** TODO make :results replace the default? I'm tending to think that appending results to pre-existing results creates mess, and that the cleaner `replace' option should be the default. E.g. when a source block creates an image, we would want that to be updated, rather than have a new one be added. -** PROPOSED external shell execution can't isolate return values -I have no idea how to do this as of yet. The result is that when -shell functions are run w/o a session there is no difference between -the =output= and =value= result arguments. + + I agree. ** TODO non-orgtbl formatted lists for example @@ -1958,6 +1967,23 @@ even a third" E.g. the pie chart example. Despite the save-window-excursion in org-babel-execute:R. (I never learned how to do this properly: org-R jumps all over the place...) + +** PROPOSED external shell execution can't isolate return values +I have no idea how to do this as of yet. The result is that when +shell functions are run w/o a session there is no difference between +the =output= and =value= result arguments. + +Yea, I don't know how to do this either. I searched extensively on +how to isolate the *last* output of a series of shell commands (see +[[* last command for + shells][last command for shells]]). The results of the search were basically +that it was not possible (or at least not accomplish-able with a +reasonable amount of effort). + +That fact combined with the tenancy to all ways use standard out in +shell scripts led me to treat these two options (=output= and =value=) +as identical in shell evaluation. Not ideal but maybe good enough for +the moment. ** DEFERRED weird escaped characters in shell prompt break shell evaluation E.g. this doesn't work. Should the shell sessions set a sane prompt From 4137ceb426d48625a284b17669598705c13ef2c7 Mon Sep 17 00:00:00 2001 From: Dan Davison Date: Wed, 8 Jul 2009 00:28:45 -0400 Subject: [PATCH 6/6] A few notes and a completed TODO --- org-babel.org | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/org-babel.org b/org-babel.org index 56a6d1b52..98917767e 100644 --- a/org-babel.org +++ b/org-babel.org @@ -1923,18 +1923,11 @@ This could probably be added to [[file:lisp/org-babel-script.el][org-babel-scrip (see [[* file result types][file result types]]) -* Bugs [17/23] +* Bugs [18/23] ** TODO Allow source blocks to be recognised when #+ are not first characters on the line I think Carsten has recently altered the core so that #+ can have preceding whitespace, at least for literal/code examples. org-babel should support this. -** TODO make :results replace the default? - I'm tending to think that appending results to pre-existing results - creates mess, and that the cleaner `replace' option should be the - default. E.g. when a source block creates an image, we would want - that to be updated, rather than have a new one be added. - - I agree. ** TODO non-orgtbl formatted lists for example @@ -1946,10 +1939,11 @@ for example #+resname: this-doesn't-match-orgtbl + ** TODO collapsing consecutive newlines in string output #+srcname: multi-line-string-output -#+begin_src ruby :results replace +#+begin_src ruby :results output "the first line ends here @@ -1958,10 +1952,8 @@ for example even a third" #+end_src -#+resname: -: the first line ends here -: and this is the second one -: return even a third +#+resname: multi-line-string-output + ** TODO cursor movement when evaluating source blocks E.g. the pie chart example. Despite the save-window-excursion in @@ -1985,6 +1977,11 @@ shell scripts led me to treat these two options (=output= and =value=) as identical in shell evaluation. Not ideal but maybe good enough for the moment. +In the `results' branch I've changed this so that they're not quite +identical: output results in raw stdout contents, whereas value +converts it to elisp, perhaps to a table if it looks tabular. This is +the same for the other languages. [Dan] + ** DEFERRED weird escaped characters in shell prompt break shell evaluation E.g. this doesn't work. Should the shell sessions set a sane prompt when they start up? Or is it a question of altering @@ -2013,6 +2010,13 @@ the moment. the user's regular emacs init. I can't think of a way for us to set this automatically, and we are SOL without a regexp to match the prompt. +** DONE make :results replace the default? + I'm tending to think that appending results to pre-existing results + creates mess, and that the cleaner `replace' option should be the + default. E.g. when a source block creates an image, we would want + that to be updated, rather than have a new one be added. + + I agree. ** DONE ruby evaluation not working under ubuntu emacs 23 With emacs 23.0.91.1 on ubuntu, for C-h f run-ruby I have the following, which seems to conflict with [[file:lisp/langs/org-babel-ruby.el::let%20session%20buffer%20save%20window%20excursion%20run%20ruby%20nil%20session%20current%20buffer][this line]] in org-babel-ruby.el.