mirror of
https://git.savannah.gnu.org/git/emacs/org-mode.git
synced 2024-08-22 23:18:44 +00:00
158 lines
5.4 KiB
Org Mode
158 lines
5.4 KiB
Org Mode
# -*- mode:org -*-
|
|
#+TITLE: Org mode Testing
|
|
#+PROPERTY: header-args:emacs-lisp :results silent
|
|
|
|
* Dependencies
|
|
|
|
The only dependency is [[http://www.emacswiki.org/emacs/ErtTestLibrary][ERT]] the Emacs testing library which ships with
|
|
Emacs24. If you are running an older version of Emacs and don't
|
|
already have ERT installed it can be installed from its old [[https://github.com/ohler/ert][git
|
|
repository]].
|
|
|
|
* Non-interactive batch testing from the command line
|
|
|
|
The simplest way to run the Org mode test suite is from the command
|
|
line with the following invocation. Note that the paths below are
|
|
relative to the base of the Org mode directory.
|
|
|
|
Also note that many of the current tests uses babel evaluation...
|
|
|
|
#+BEGIN_SRC sh :dir (expand-file-name "..")
|
|
# For Emacs earlier than 24, add -L /path/to/ert
|
|
emacs -Q --batch \
|
|
-L lisp/ -L testing/ -L testing/lisp -l lisp/org.el \
|
|
-l lisp/org-id.el -l testing/org-test.el \
|
|
--eval "(progn (org-reload) (setq org-confirm-babel-evaluate nil) \
|
|
(org-babel-do-load-languages 'org-babel-load-languages \
|
|
'((emacs-lisp . t) (shell . t) (org . t))))" \
|
|
-f org-test-run-batch-tests
|
|
#+END_SRC
|
|
|
|
The options in the above command are explained below.
|
|
|
|
| -Q | ignores any personal configuration ensuring a vanilla Emacs instance is used |
|
|
| --batch | runs Emacs in "batch" mode with no gui and termination after execution |
|
|
| -l | loads Org mode and the Org mode test suite defined in testing/org-test.el |
|
|
| --eval | reloads Org mode and allows evaluation of code blocks by the tests |
|
|
| -f | actually runs the tests using the `org-test-run-batch-tests' function |
|
|
|
|
* Trigger the tests with 'make'
|
|
|
|
** Recompile all
|
|
|
|
Target ~test~ can be used to trigger a test run. The tests start
|
|
after cleaning up and recompilation.
|
|
|
|
#+BEGIN_SRC sh :dir (expand-file-name "..") :results silent
|
|
make test
|
|
#+END_SRC
|
|
|
|
See ../mk/default.mk for details.
|
|
|
|
** Test dirty
|
|
|
|
The 'dirty' targets are for recompiling without cleaning and
|
|
rebuilding everything. This usually speeds up the recompilation
|
|
considerably. Note that this speed up comes to the price of possibly
|
|
weird errors due to the unclean build.
|
|
|
|
The dirty target for testing is called ~test-dirty~.
|
|
|
|
#+BEGIN_SRC sh :dir (expand-file-name "..") :results silent
|
|
make test-dirty
|
|
#+END_SRC
|
|
|
|
** Select tests by regexp
|
|
|
|
Variable ~BTEST_RE~ can be set to limit the tests which are performed.
|
|
~BTEST_RE~ is interpreted as regexp.
|
|
|
|
Example:
|
|
|
|
#+begin_src shell
|
|
make BTEST_RE='test-.*-inlinetask' test-dirty
|
|
#+end_src
|
|
|
|
yields
|
|
|
|
#+begin_example
|
|
...
|
|
selected tests: test-.*-inlinetask
|
|
Running 2 tests (2017-12-28 15:04:45+0100)
|
|
passed 1/2 test-org-export/handle-inlinetasks
|
|
passed 2/2 test-org-inlinetask/goto-end
|
|
|
|
Ran 2 tests, 2 results as expected (2017-12-28 15:04:45+0100)
|
|
...
|
|
#+end_example
|
|
|
|
* Interactive testing from within Emacs
|
|
|
|
To run the Org mode test suite from a current Emacs instance simply
|
|
load and run the test suite with the following commands.
|
|
|
|
1) First load the test suite.
|
|
#+BEGIN_SRC emacs-lisp :var here=(buffer-file-name)
|
|
(add-to-list 'load-path (file-name-directory here))
|
|
(require 'org-test)
|
|
#+END_SRC
|
|
|
|
2) Load required Babel languages
|
|
#+BEGIN_SRC emacs-lisp
|
|
(org-babel-do-load-languages
|
|
'org-babel-load-languages
|
|
(and
|
|
(mapc (lambda (lang) (add-to-list 'org-babel-load-languages (cons lang t)))
|
|
'(emacs-lisp shell org))
|
|
org-babel-load-languages))
|
|
#+END_SRC
|
|
|
|
3) Then run the test suite. Babel evaluation confirmation is disabled
|
|
and ~C-c C-c~ is enabled while running the tests.
|
|
#+BEGIN_SRC emacs-lisp
|
|
(let (org-babel-no-eval-on-ctrl-c-ctrl-c
|
|
org-confirm-babel-evaluate)
|
|
(org-test-run-all-tests))
|
|
#+END_SRC
|
|
|
|
When a test fails, run it interactively and investigate the problem
|
|
in the ERT results buffer.
|
|
|
|
To run one test: Use this as a demo example of a failing test
|
|
#+BEGIN_SRC emacs-lisp
|
|
(ert-deftest test-org/org-link-escape-ascii-character-demo-of-fail ()
|
|
(should (string= "%5B" ; Expecting %5B is correct.
|
|
(org-link-escape "[")))
|
|
(should (string= "%5C" ; Expecting %5C is wrong, %5D correct.
|
|
(org-link-escape "]"))))
|
|
#+END_SRC
|
|
or evaluate the ~ert-deftest form~ of the test you want to run.
|
|
Then ~M-x ert RET
|
|
test-org/org-link-escape-ascii-character-demo-of-fail RET~. When
|
|
not visible yet switch to the ERT results buffer named ~*ert*~.
|
|
When a test failed the ERT results buffer shows the details of the
|
|
first ~should~ that failed. See ~(info "(ert)Running Tests
|
|
Interactively")~ on how to re-run, start the debugger etc.
|
|
|
|
To run several tests: ~M-x ert RET "<your regexp here>" RET~.
|
|
|
|
To run all tests of a single test file: ~M-x ert-delete-all-tests
|
|
RET~ and confirm. ~M-x load-file RET testing/lisp/<file>.el RET
|
|
M-x ert RET t RET~.
|
|
|
|
Consider to set
|
|
#+BEGIN_SRC emacs-lisp
|
|
(setq pp-escape-newlines nil)
|
|
#+END_SRC
|
|
before running the test when looking at ~should~ in the ERT results
|
|
buffer. Especially when using ~l~ to look at passed test results
|
|
and possibly missing an appropriate setting of ~pp-escape-newlines~
|
|
made only temporarily for the running time of the test as
|
|
e. g. tests using ~org-test-table-target-expect-tblfm~ do.
|
|
|
|
* Troubleshooting
|
|
|
|
- If the variable ~org-babel-no-eval-on-ctrl-c-ctrl-c~ is non-nil then
|
|
it will result in some test failure, as there are tests which rely
|
|
on this behavior.
|