Allow org-file-contents to fetch file contents from a URL
* lisp/org.el (org--file-cache): New variable. (org-reset-file-cache): (org-file-url-p): New function. (org-mode-restart): Use new function. * lisp/org.el (org-file-contents): Allow the FILE argument to be a URL. If the URL contents are already cached, return the cache contents, else download the file and return contents of that. The file is automatically cached each time it is downloaded. Add a new optional argument NOCACHE. If this is non-nil, the URL is always downloaded afresh. Use `org--file-cache' and `org-file-url-p'. * lisp/org.el (org-edit-special): Do not allow editing the "file" if a URL is specified for the "#+SETUPFILE". * lisp/ox.el (org-export--list-bound-variables) (org-export--prepare-file-contents): * lisp/org-macro.el (org-macro--collect-macros) : Adapt to the possibility that the input to `org-file-contents' can be a URL too. * doc/org.texi (Export settings, In-buffer settings) (The very busy C-c C-c key): Mention that #+SETUPFILE keyword can now take a URL as a value, and that C-c C-c on the #+SETUPFILE line will clear the org file cache. * testing/lisp/test-org.el (test-org/org-file-contents-url) (test-org/org-file-contents-file): Add tests for org-file-contents. * testing/lisp/test-ox.el (test-org-export/get-inbuffer-options): Add test for reading setupfile specified via a URL.
This commit is contained in:
Kaushal Modi
parent
49dc91c55e
commit
1e92f5ed39
38
doc/org.texi
38
doc/org.texi
|
|
@ -10406,14 +10406,14 @@ override options set at a more general level.
|
|||
|
||||
@cindex #+SETUPFILE
|
||||
In-buffer settings may appear anywhere in the file, either directly or
|
||||
indirectly through a file included using @samp{#+SETUPFILE: filename} syntax.
|
||||
Option keyword sets tailored to a particular back-end can be inserted from
|
||||
the export dispatcher (@pxref{The export dispatcher}) using the @code{Insert
|
||||
template} command by pressing @key{#}. To insert keywords individually,
|
||||
a good way to make sure the keyword is correct is to type @code{#+} and then
|
||||
to use @kbd{M-@key{TAB}}@footnote{Many desktops intercept @kbd{M-TAB} to
|
||||
switch windows. Use @kbd{C-M-i} or @kbd{@key{ESC} @key{TAB}} instead.} for
|
||||
completion.
|
||||
indirectly through a file included using @samp{#+SETUPFILE: filename or URL}
|
||||
syntax. Option keyword sets tailored to a particular back-end can be
|
||||
inserted from the export dispatcher (@pxref{The export dispatcher}) using the
|
||||
@code{Insert template} command by pressing @key{#}. To insert keywords
|
||||
individually, a good way to make sure the keyword is correct is to type
|
||||
@code{#+} and then to use @kbd{M-@key{TAB}}@footnote{Many desktops intercept
|
||||
@kbd{M-TAB} to switch windows. Use @kbd{C-M-i} or @kbd{@key{ESC} @key{TAB}}
|
||||
instead.} for completion.
|
||||
|
||||
The export keywords available for every back-end, and their equivalent global
|
||||
variables, include:
|
||||
|
|
@ -17174,14 +17174,16 @@ have a lower ASCII number than the lowest priority.
|
|||
This line sets a default inheritance value for entries in the current
|
||||
buffer, most useful for specifying the allowed values of a property.
|
||||
@cindex #+SETUPFILE
|
||||
@item #+SETUPFILE: file
|
||||
The setup file is for additional in-buffer settings. Org loads this file and
|
||||
parses it for any settings in it only when Org opens the main file. @kbd{C-c
|
||||
C-c} on the settings line will also parse and load. Org also parses and
|
||||
loads the file during normal exporting process. Org parses the contents of
|
||||
this file as if it was included in the buffer. It can be another Org file.
|
||||
To visit the file, @kbd{C-c '} while the cursor is on the line with the file
|
||||
name.
|
||||
@item #+SETUPFILE: file or URL
|
||||
The setup file or a URL pointing to such file is for additional in-buffer
|
||||
settings. Org loads this file and parses it for any settings in it only when
|
||||
Org opens the main file. If URL is specified, the contents are downloaded
|
||||
and stored in a temporary file cache. @kbd{C-c C-c} on the settings line
|
||||
will parse and load the file, and also reset the temporary file cache. Org
|
||||
also parses and loads the document during normal exporting process. Org
|
||||
parses the contents of this document as if it was included in the buffer. It
|
||||
can be another Org file. To visit the file (not a URL), @kbd{C-c '} while
|
||||
the cursor is on the line with the file name.
|
||||
@item #+STARTUP:
|
||||
@cindex #+STARTUP
|
||||
Startup options Org uses when first visiting a file.
|
||||
|
|
@ -17422,7 +17424,9 @@ If any highlights shown in the buffer from the creation of a sparse tree, or
|
|||
from clock display, remove such highlights.
|
||||
@item
|
||||
If the cursor is in one of the special @code{#+KEYWORD} lines, scan the
|
||||
buffer for these lines and update the information.
|
||||
buffer for these lines and update the information. Also reset the Org file
|
||||
cache used to temporary store the contents of URLs used as values for
|
||||
keywords like @code{#+SETUPFILE}.
|
||||
@item
|
||||
If the cursor is inside a table, realign the table. The table realigns even
|
||||
if automatic table editor is turned off.
|
||||
|
|
|
|||
12
etc/ORG-NEWS
12
etc/ORG-NEWS
|
|
@ -203,7 +203,7 @@ manual for details.
|
|||
**** Add global macros through ~org-export-global-macros~
|
||||
With this variable, one can define macros available for all documents.
|
||||
**** New keyword ~#+EXPORT_FILE_NAME~
|
||||
Simiralry to ~:EXPORT_FILE_NAME:~ property, this keyword allows the
|
||||
Similarly to ~:EXPORT_FILE_NAME:~ property, this keyword allows the
|
||||
user to specify the name of the output file upon exporting the
|
||||
document. This also has an effect on publishing.
|
||||
**** Horizontal rules are no longer ignored in LaTeX table math mode
|
||||
|
|
@ -240,6 +240,16 @@ which causes refile targets to be prefixed with the buffer’s
|
|||
name. This is particularly useful when used in conjunction with
|
||||
~uniquify.el~.
|
||||
|
||||
*** ~org-file-contents~ now allows the FILE argument to be a URL.
|
||||
This allows ~#+SETUPFILE:~ to accept a URL instead of a local file
|
||||
path. The URL contents are auto-downloaded and saved to a temporary
|
||||
cache ~org--file-cache~. A new optional argument ~NOCACHE~ is added
|
||||
to ~org-file-contents~.
|
||||
|
||||
*** ~org-mode-restart~ now resets the newly added ~org--file-cache~.
|
||||
Using ~C-c C-c~ on any keyword (like ~#+SETUPFILE~) will reset the
|
||||
that file cache.
|
||||
|
||||
** Removed functions
|
||||
|
||||
*** Org Timeline
|
||||
|
|
|
|||
|
|
@ -55,7 +55,8 @@
|
|||
(declare-function org-element-macro-parser "org-element" ())
|
||||
(declare-function org-element-property "org-element" (property element))
|
||||
(declare-function org-element-type "org-element" (element))
|
||||
(declare-function org-file-contents "org" (file &optional noerror))
|
||||
(declare-function org-file-contents "org" (file &optional noerror nocache))
|
||||
(declare-function org-file-url-p "org" (file))
|
||||
(declare-function org-in-commented-heading-p "org" (&optional no-inheritance))
|
||||
(declare-function org-mode "org" ())
|
||||
(declare-function vc-backend "vc-hooks" (f))
|
||||
|
|
@ -102,16 +103,21 @@ Return an alist containing all macro templates found."
|
|||
(if old-cell (setcdr old-cell template)
|
||||
(push (cons name template) templates))))
|
||||
;; Enter setup file.
|
||||
(let ((file (expand-file-name
|
||||
(org-unbracket-string "\"" "\"" val))))
|
||||
(unless (member file files)
|
||||
(let* ((uri (org-unbracket-string "\"" "\"" (org-trim val)))
|
||||
(uri-is-url (org-file-url-p uri))
|
||||
(uri (if uri-is-url
|
||||
uri
|
||||
(expand-file-name uri))))
|
||||
;; Avoid circular dependencies.
|
||||
(unless (member uri files)
|
||||
(with-temp-buffer
|
||||
(setq default-directory
|
||||
(file-name-directory file))
|
||||
(unless uri-is-url
|
||||
(setq default-directory
|
||||
(file-name-directory uri)))
|
||||
(org-mode)
|
||||
(insert (org-file-contents file 'noerror))
|
||||
(insert (org-file-contents uri 'noerror))
|
||||
(setq templates
|
||||
(funcall collect-macros (cons file files)
|
||||
(funcall collect-macros (cons uri files)
|
||||
templates)))))))))))
|
||||
templates))))
|
||||
(funcall collect-macros nil nil)))
|
||||
|
|
|
|||
74
lisp/org.el
74
lisp/org.el
|
|
@ -181,6 +181,8 @@ Stars are put in group 1 and the trimmed body in group 2.")
|
|||
(declare-function org-export-get-environment "ox" (&optional backend subtreep ext-plist))
|
||||
(declare-function org-latex-make-preamble "ox-latex" (info &optional template snippet?))
|
||||
|
||||
(defvar ffap-url-regexp) ;Silence byte-compiler
|
||||
|
||||
(defsubst org-uniquify (list)
|
||||
"Non-destructively remove duplicate elements from LIST."
|
||||
(let ((res (copy-sequence list))) (delete-dups res)))
|
||||
|
|
@ -5280,17 +5282,62 @@ a string, summarizing TAGS, as a list of strings."
|
|||
(setq current-group (list tag))))
|
||||
(_ nil)))))
|
||||
|
||||
(defun org-file-contents (file &optional noerror)
|
||||
"Return the contents of FILE, as a string."
|
||||
(if (and file (file-readable-p file))
|
||||
(defvar org--file-cache (make-hash-table :test #'equal)
|
||||
"Hash table to store contents of files referenced via a URL.
|
||||
This is the cache of file URLs read using `org-file-contents'.")
|
||||
|
||||
(defun org-reset-file-cache ()
|
||||
"Reset the cache of files downloaded by `org-file-contents'."
|
||||
(clrhash org--file-cache))
|
||||
|
||||
(defun org-file-url-p (file)
|
||||
"Non-nil if FILE is a URL."
|
||||
(require 'ffap)
|
||||
(string-match-p ffap-url-regexp file))
|
||||
|
||||
(defun org-file-contents (file &optional noerror nocache)
|
||||
"Return the contents of FILE, as a string.
|
||||
|
||||
FILE can be a file name or URL.
|
||||
|
||||
If FILE is a URL, download the contents. If the URL contents are
|
||||
already cached in the `org--file-cache' hash table, the download step
|
||||
is skipped.
|
||||
|
||||
If NOERROR is non-nil, ignore the error when unable to read the FILE
|
||||
from file or URL.
|
||||
|
||||
If NOCACHE is non-nil, do a fresh fetch of FILE even if cached version
|
||||
is available. This option applies only if FILE is a URL."
|
||||
(let* ((is-url (org-file-url-p file))
|
||||
(cache (and is-url
|
||||
(not nocache)
|
||||
(gethash file org--file-cache))))
|
||||
(cond
|
||||
(cache)
|
||||
(is-url
|
||||
(with-current-buffer (url-retrieve-synchronously file)
|
||||
(goto-char (point-min))
|
||||
;; Move point to after the url-retrieve header.
|
||||
(search-forward "\n\n" nil :move)
|
||||
;; Search for the success code only in the url-retrieve header.
|
||||
(if (save-excursion (re-search-backward "HTTP.*\\s-+200\\s-OK" nil :noerror))
|
||||
;; Update the cache `org--file-cache' and return contents.
|
||||
(puthash file
|
||||
(buffer-substring-no-properties (point) (point-max))
|
||||
org--file-cache)
|
||||
(funcall (if noerror #'message #'user-error)
|
||||
"Unable to fetch file from %S"
|
||||
file))))
|
||||
(t
|
||||
(with-temp-buffer
|
||||
(insert-file-contents file)
|
||||
(buffer-string))
|
||||
(funcall (if noerror 'message 'error)
|
||||
"Cannot read file \"%s\"%s"
|
||||
file
|
||||
(let ((from (buffer-file-name (buffer-base-buffer))))
|
||||
(if from (concat " (referenced in file \"" from "\")") "")))))
|
||||
(condition-case err
|
||||
(progn
|
||||
(insert-file-contents file)
|
||||
(buffer-string))
|
||||
(file-error
|
||||
(funcall (if noerror #'message #'user-error)
|
||||
(error-message-string err)))))))))
|
||||
|
||||
(defun org-extract-log-state-settings (x)
|
||||
"Extract the log state setting from a TODO keyword string.
|
||||
|
|
@ -20687,7 +20734,9 @@ Otherwise, return a user error."
|
|||
(format "[[%s]]"
|
||||
(expand-file-name
|
||||
(let ((value (org-element-property :value element)))
|
||||
(cond ((not (org-string-nw-p value))
|
||||
(cond ((org-file-url-p value)
|
||||
(user-error "The file is specified as a URL, cannot be edited"))
|
||||
((not (org-string-nw-p value))
|
||||
(user-error "No file to edit"))
|
||||
((string-match "\\`\"\\(.*?\\)\"" value)
|
||||
(match-string 1 value))
|
||||
|
|
@ -20951,7 +21000,8 @@ Use `\\[org-edit-special]' to edit table.el tables"))
|
|||
(funcall major-mode)
|
||||
(hack-local-variables)
|
||||
(when (and indent-status (not (bound-and-true-p org-indent-mode)))
|
||||
(org-indent-mode -1)))
|
||||
(org-indent-mode -1))
|
||||
(org-reset-file-cache))
|
||||
(message "%s restarted" major-mode))
|
||||
|
||||
(defun org-kill-note-or-show-branches ()
|
||||
|
|
|
|||
38
lisp/ox.el
38
lisp/ox.el
|
|
@ -1499,17 +1499,20 @@ Assume buffer is in Org mode. Narrowing, if any, is ignored."
|
|||
(cond
|
||||
;; Options in `org-export-special-keywords'.
|
||||
((equal key "SETUPFILE")
|
||||
(let ((file
|
||||
(expand-file-name
|
||||
(org-unbracket-string "\"" "\"" (org-trim val)))))
|
||||
(let* ((uri (org-unbracket-string "\"" "\"" (org-trim val)))
|
||||
(uri-is-url (org-file-url-p uri))
|
||||
(uri (if uri-is-url
|
||||
uri
|
||||
(expand-file-name uri))))
|
||||
;; Avoid circular dependencies.
|
||||
(unless (member file files)
|
||||
(unless (member uri files)
|
||||
(with-temp-buffer
|
||||
(setq default-directory
|
||||
(file-name-directory file))
|
||||
(insert (org-file-contents file 'noerror))
|
||||
(unless uri-is-url
|
||||
(setq default-directory
|
||||
(file-name-directory uri)))
|
||||
(insert (org-file-contents uri 'noerror))
|
||||
(let ((org-inhibit-startup t)) (org-mode))
|
||||
(funcall get-options (cons file files))))))
|
||||
(funcall get-options (cons uri files))))))
|
||||
((equal key "OPTIONS")
|
||||
(setq plist
|
||||
(org-combine-plists
|
||||
|
|
@ -1647,17 +1650,22 @@ an alist where associations are (VARIABLE-NAME VALUE)."
|
|||
"BIND")
|
||||
(push (read (format "(%s)" val)) alist)
|
||||
;; Enter setup file.
|
||||
(let ((file (expand-file-name
|
||||
(org-unbracket-string "\"" "\"" val))))
|
||||
(unless (member file files)
|
||||
(let* ((uri (org-unbracket-string "\"" "\"" val))
|
||||
(uri-is-url (org-file-url-p uri))
|
||||
(uri (if uri-is-url
|
||||
uri
|
||||
(expand-file-name uri))))
|
||||
;; Avoid circular dependencies.
|
||||
(unless (member uri files)
|
||||
(with-temp-buffer
|
||||
(setq default-directory
|
||||
(file-name-directory file))
|
||||
(unless uri-is-url
|
||||
(setq default-directory
|
||||
(file-name-directory uri)))
|
||||
(let ((org-inhibit-startup t)) (org-mode))
|
||||
(insert (org-file-contents file 'noerror))
|
||||
(insert (org-file-contents uri 'noerror))
|
||||
(setq alist
|
||||
(funcall collect-bind
|
||||
(cons file files)
|
||||
(cons uri files)
|
||||
alist))))))))))
|
||||
alist)))))
|
||||
;; Return value in appropriate order of appearance.
|
||||
|
|
|
|||
|
|
@ -6498,6 +6498,81 @@ Paragraph<point>"
|
|||
(org-show-set-visibility 'minimal)
|
||||
(org-invisible-p2))))
|
||||
|
||||
(ert-deftest test-org/org-file-contents-file ()
|
||||
"Test `org-file-contents' with a file as input."
|
||||
(should
|
||||
(string= "#+BIND: variable value
|
||||
#+DESCRIPTION: l2
|
||||
#+LANGUAGE: en
|
||||
#+SELECT_TAGS: b
|
||||
#+TITLE: b
|
||||
#+PROPERTY: a 1
|
||||
" (org-file-contents (expand-file-name "setupfile3.org"
|
||||
(concat org-test-dir "examples/")))))
|
||||
|
||||
(let ((invalid-file "this-file-must-not-exist"))
|
||||
;; Throw error when trying to access an invalid file.
|
||||
(should-error
|
||||
(org-file-contents invalid-file))
|
||||
;; Try to access an invalid file, but do not throw an error.
|
||||
(should
|
||||
(string-match-p "\\`Opening input file: No such file or directory"
|
||||
(org-file-contents invalid-file :noerror)))))
|
||||
|
||||
(ert-deftest test-org/org-file-contents-url ()
|
||||
"Test `org-file-contents' with a URL as input."
|
||||
(should
|
||||
(string= "foo"
|
||||
(let ((buffer (generate-new-buffer "url-retrieve-output")))
|
||||
(unwind-protect
|
||||
;; Simulate successful retrieval of a URL.
|
||||
(cl-letf (((symbol-function 'url-retrieve-synchronously)
|
||||
(lambda (&rest_)
|
||||
(with-current-buffer buffer
|
||||
(insert "HTTP/1.1 200 OK\n\nfoo"))
|
||||
buffer)))
|
||||
(org-file-contents "http://some-valid-url"))
|
||||
(kill-buffer buffer)))))
|
||||
|
||||
(let ((invalid-url "http://this-url-must-not-exist"))
|
||||
;; Throw error when trying to access an invalid URL.
|
||||
(should-error
|
||||
(let ((buffer (generate-new-buffer "url-retrieve-output")))
|
||||
(unwind-protect
|
||||
;; Simulate unsuccessful retrieval of a URL.
|
||||
(cl-letf (((symbol-function 'url-retrieve-synchronously)
|
||||
(lambda (&rest_)
|
||||
(with-current-buffer buffer
|
||||
(insert "HTTP/1.1 404 Not found\n\ndoes not matter"))
|
||||
buffer)))
|
||||
(org-file-contents invalid-url))
|
||||
(kill-buffer buffer))))
|
||||
;; Try to access an invalid URL, but do not throw an error.
|
||||
(should-error
|
||||
(let ((buffer (generate-new-buffer "url-retrieve-output")))
|
||||
(unwind-protect
|
||||
;; Simulate unsuccessful retrieval of a URL.
|
||||
(cl-letf (((symbol-function 'url-retrieve-synchronously)
|
||||
(lambda (&rest_)
|
||||
(with-current-buffer buffer
|
||||
(insert "HTTP/1.1 404 Not found\n\ndoes not matter"))
|
||||
buffer)))
|
||||
(org-file-contents invalid-url))
|
||||
(kill-buffer buffer))))
|
||||
(should
|
||||
(string=
|
||||
(format "Unable to fetch file from \"%s\"" invalid-url)
|
||||
(let ((buffer (generate-new-buffer "url-retrieve-output")))
|
||||
(unwind-protect
|
||||
;; Simulate unsuccessful retrieval of a URL.
|
||||
(cl-letf (((symbol-function 'url-retrieve-synchronously)
|
||||
(lambda (&rest_)
|
||||
(with-current-buffer buffer
|
||||
(insert "HTTP/1.1 404 Not found\n\ndoes not matter"))
|
||||
buffer)))
|
||||
(org-file-contents invalid-url :noerror))
|
||||
(kill-buffer buffer)))))))
|
||||
|
||||
|
||||
(provide 'test-org)
|
||||
|
||||
|
|
|
|||
|
|
@ -232,6 +232,38 @@ num:2 <:active")))
|
|||
org-test-dir)
|
||||
(org-export--get-inbuffer-options))
|
||||
'(:language "fr" :select-tags ("a" "b" "c") :title ("a b c"))))
|
||||
;; Options set through SETUPFILE specified using a URL.
|
||||
(let ((buffer (generate-new-buffer "url-retrieve-output")))
|
||||
(unwind-protect
|
||||
;; Simulate successful retrieval of a setupfile from URL.
|
||||
(cl-letf (((symbol-function 'url-retrieve-synchronously)
|
||||
(lambda (&rest_)
|
||||
(with-current-buffer buffer
|
||||
(insert "HTTP/1.1 200 OK
|
||||
|
||||
# Contents of http://link-to-my-setupfile.org
|
||||
#+BIND: variable value
|
||||
#+DESCRIPTION: l2
|
||||
#+LANGUAGE: en
|
||||
#+SELECT_TAGS: b
|
||||
#+TITLE: b
|
||||
#+PROPERTY: a 1
|
||||
"))
|
||||
buffer)))
|
||||
(should
|
||||
(equal
|
||||
(org-test-with-temp-text
|
||||
"#+DESCRIPTION: l1
|
||||
#+LANGUAGE: es
|
||||
#+SELECT_TAGS: a
|
||||
#+TITLE: a
|
||||
#+SETUPFILE: \"http://link-to-my-setupfile.org\"
|
||||
#+LANGUAGE: fr
|
||||
#+SELECT_TAGS: c
|
||||
#+TITLE: c"
|
||||
(org-export--get-inbuffer-options))
|
||||
'(:language "fr" :select-tags ("a" "b" "c") :title ("a b c")))))
|
||||
(kill-buffer buffer)))
|
||||
;; More than one property can refer to the same buffer keyword.
|
||||
(should
|
||||
(equal '(:k2 "value" :k1 "value")
|
||||
|
|
|
|||
Loading…
Reference in New Issue