forked from mirrors/org-mode
* testing/lisp/test-ob.el (test-ob/blocks-with-spaces): (test-ob/specific-colnames): * testing/lisp/test-oc.el (test-org-cite/export-capability): * testing/lisp/test-ol.el (test-org-link/store-link): * testing/lisp/test-org.el (test-org/set-regexps-and-options): * testing/org-test.el (org-test-base-dir): Address typos and non-existent variables. (missing-test-dependency): Use `define-error'. (org-test-load): Set `org-id-locations-file' when running tests. * testing/lisp/test-ox.el (test-org-export/bind-keyword): Clarify about variable symbol defined inside setupfile #+BIND statemtn. |
||
---|---|---|
.. | ||
examples | ||
jump@820bb7d81b | ||
lisp | ||
.gitignore | ||
org-batch-test-init.el | ||
org-test.el | ||
README |
# -*- mode:org -*- #+TITLE: Org mode Testing #+PROPERTY: header-args:emacs-lisp :results silent * Dependencies The only dependency is [[http://www.emacswiki.org/emacs/ErtTestLibrary][ERT]] the Emacs testing library which ships with Emacs24. If you are running an older version of Emacs and don't already have ERT installed it can be installed from its old [[https://github.com/ohler/ert][git repository]]. * Non-interactive batch testing from the command line The simplest way to run the Org mode test suite is from the command line with the following invocation. Note that the paths below are relative to the base of the Org mode directory. Also note that many of the current tests uses babel evaluation... #+BEGIN_SRC sh :dir (expand-file-name "..") # For Emacs earlier than 24, add -L /path/to/ert emacs -Q --batch \ -L lisp/ -L testing/ -L testing/lisp -l lisp/org.el \ -l lisp/org-id.el -l testing/org-test.el \ --eval "(progn (org-reload) (setq org-confirm-babel-evaluate nil) \ (org-babel-do-load-languages 'org-babel-load-languages \ '((emacs-lisp . t) (shell . t) (org . t))))" \ -f org-test-run-batch-tests #+END_SRC The options in the above command are explained below. | -Q | ignores any personal configuration ensuring a vanilla Emacs instance is used | | --batch | runs Emacs in "batch" mode with no gui and termination after execution | | -l | loads Org mode and the Org mode test suite defined in testing/org-test.el | | --eval | reloads Org mode and allows evaluation of code blocks by the tests | | -f | actually runs the tests using the `org-test-run-batch-tests' function | * Trigger the tests with 'make' ** Recompile all Target ~test~ can be used to trigger a test run. The tests start after cleaning up and recompilation. #+BEGIN_SRC sh :dir (expand-file-name "..") :results silent make test #+END_SRC See ../mk/default.mk for details. ** Test dirty The 'dirty' targets are for recompiling without cleaning and rebuilding everything. This usually speeds up the recompilation considerably. Note that this speed up comes to the price of possibly weird errors due to the unclean build. The dirty target for testing is called ~test-dirty~. #+BEGIN_SRC sh :dir (expand-file-name "..") :results silent make test-dirty #+END_SRC ** Select tests by regexp Variable ~BTEST_RE~ can be set to limit the tests which are performed. ~BTEST_RE~ is interpreted as regexp. Example: #+begin_src shell make BTEST_RE='test-.*-inlinetask' test-dirty #+end_src yields #+begin_example ... selected tests: test-.*-inlinetask Running 2 tests (2017-12-28 15:04:45+0100) passed 1/2 test-org-export/handle-inlinetasks passed 2/2 test-org-inlinetask/goto-end Ran 2 tests, 2 results as expected (2017-12-28 15:04:45+0100) ... #+end_example * Interactive testing from within Emacs To run the Org mode test suite from a current Emacs instance simply load and run the test suite with the following commands. 1) First load the test suite. #+BEGIN_SRC emacs-lisp :var here=(buffer-file-name) (add-to-list 'load-path (file-name-directory here)) (require 'org-test) #+END_SRC 2) Load required Babel languages #+BEGIN_SRC emacs-lisp (org-babel-do-load-languages 'org-babel-load-languages (and (mapc (lambda (lang) (add-to-list 'org-babel-load-languages (cons lang t))) '(emacs-lisp shell org)) org-babel-load-languages)) #+END_SRC 3) Then run the test suite. Babel evaluation confirmation is disabled and ~C-c C-c~ is enabled while running the tests. #+BEGIN_SRC emacs-lisp (let (org-babel-no-eval-on-ctrl-c-ctrl-c org-confirm-babel-evaluate) (org-test-run-all-tests)) #+END_SRC When a test fails, run it interactively and investigate the problem in the ERT results buffer. To run one test: Use this as a demo example of a failing test #+BEGIN_SRC emacs-lisp (ert-deftest test-org/org-link-encode-ascii-character-demo-of-fail () (should (string= "%5B" ; Expecting %5B is correct. (org-link-encode "["))) (should (string= "%5C" ; Expecting %5C is wrong, %5D correct. (org-link-encode "]")))) #+END_SRC or evaluate the ~ert-deftest form~ of the test you want to run. Then ~M-x ert RET test-org/org-link-encode-ascii-character-demo-of-fail RET~. When not visible yet switch to the ERT results buffer named ~*ert*~. When a test failed the ERT results buffer shows the details of the first ~should~ that failed. See ~(info "(ert)Running Tests Interactively")~ on how to re-run, start the debugger etc. To run several tests: ~M-x ert RET "<your regexp here>" RET~. To run all tests of a single test file: ~M-x ert-delete-all-tests RET~ and confirm. ~M-x load-file RET testing/lisp/<file>.el RET M-x ert RET t RET~. Consider to set #+BEGIN_SRC emacs-lisp (setq pp-escape-newlines nil) #+END_SRC before running the test when looking at ~should~ in the ERT results buffer. Especially when using ~l~ to look at passed test results and possibly missing an appropriate setting of ~pp-escape-newlines~ made only temporarily for the running time of the test as e. g. tests using ~org-test-table-target-expect-tblfm~ do. * Troubleshooting - If the variable ~org-babel-no-eval-on-ctrl-c-ctrl-c~ is non-nil then it will result in some test failure, as there are tests which rely on this behavior.