This Month in Org https://blog.tecosaur.com/tmio/ en Thu, 02 Jun 2022 00:55:48 +0800 Thu, 02 Jun 2022 00:55:48 +0800 Emacs 28.0 Org-mode 9.5 tec@tecosaur.com (TEC) https://orgmode.org/img/org-mode-unicorn-logo.png This Month in Org https://blog.tecosaur.com/tmio/ May 2022 https://blog.tecosaur.com/tmio/2022-05-31-folding.html tec@tecosaur.com (TEC) https://blog.tecosaur.com/tmio/2022-05-31-folding.html Thu, 02 Jun 2022 00:00:00 +0800 Finding time as of late has been more difficult than I anticipated, and on top of that, just as I was thinking of writing last month's post, I got distracted by an exciting patchset that has been in the works for over a year finally getting sorted out and landing. So, I hope that some of the fun developments in this post will make up the absense of the last one 🙂.

Since it's been longer than I thought since the last standard post, we've got a fair few commits to catch up on --- about 200. Most of these are miscellaneous minor improvements and bugfixes, but a few notable changes have arrived too.

Folding

The fabulous new folding engine (org-fold-core) should noticeably improve Org's performance with large files. It contains a number of key optimisations to improve speed, namely: Deferring fontification of folded regions Using text properties (\(\mathcal{O}(n \log n)\)) instead of overlays (\(\mathcal{O}(n^2)\)) for folded regions A collection of aggressive optimisations available under org-fold-core--optimise-for-huge-buffers Convert text properties to overlays for isearch (which currently only supports overlays)

How noticeable is the overall performance impact? Well, I poked Ihor and he was kind enough to whip up some benchmarks.

org-fold-perf-shifttab-contents.svg
Figure 1: The scaling of org-shifttab showing file contents, as file size increases, with and without org-fold.
org-fold-perf-shifttab-showall.svg
Figure 2: The scaling of org-shifttab showing the entire file, as file size increases, with and without org-fold.

Well this looks very promising[fn:Note the difference in scale, org-fold makes the most difference in the graph where the times are an order of magnitude more.]! Let's see how much of an improvement this is overall.

Table 1: Time to run org-shifttab twice, cycling through all three display modes (in seconds).
File size (Mb) Headings (thousands) Bugfix (no org-fold) Main (with org-fold) Improvement
18 36 115.31 0.89 99%
8.8 24 19.03 0.48 97%
4.4 5 3.79 0.13 97%
2.2 2 1.29 0.08 94%
1.1 1 0.50 0.045 91%

To be clear, even the smallest file in this data --- a 1.1 Mb Org file with around a thousand headings, is fairly large. So, it's unlikely you'll notice much of a difference with small--medium files, but if you a few large+ files this should be a fantastic improvement. Once again, thanks Ihor!

The change to text properties instead of overlays breaks a number of third party packages like evil-search and consult's consult-line. If you are involved in any packages affected by this, you'll either want to consider supporting invisible text, or look at isearch-filter-predicate and isearch-mode-end-hook, which org-fold now uses. If you're an end-user, perhaps politely make an issue on the repo for a project if no issue currently exists, and either: Stay off Org's bleeding edge till the package ecosystem has adapted to this change Help the packages you use adapt to this change Set org-fold-core-style to overlays to restore the old behaviour

]]>
Special Announcement https://blog.tecosaur.com/tmio/2022-04-01-confluence.html tec@tecosaur.com (TEC) https://blog.tecosaur.com/tmio/2022-04-01-confluence.html Sat, 02 Apr 2022 00:00:00 +0800 It has become apparent to the Org Mode developers that Org is suffering from a severe lack of enterprise adoption. To rectify this, we will be leveraging our collective decades working on holistic human-markup interaction tools to rapidly pivot to what we believe to be the true markup format of the future --- Confluence Wiki Markup.

With this paradigm shift, you can look forward to a much more intuitive syntax, empowering you to create next-generation agile documents.

To assist you in this transition, we'll give you a brief overview of the changes you can expect. Text formatting is almost unaffected, with a few sensible changes made to the surrounding characters.

org-markup-to-confluence.svg
Figure 3: Translation between Org's syntax and our new Confluence overlords's equivalents

We'd like to thank the FSF for giving the Org project the space and support to grow to become a world-class project. In our new venture, copyright assignment to the FSF is no longer required. Instead, copyright must be assigned to Atlassian prior to contributing.

This new partnership enables new and exciting integrations with Atlassian's proprietary technologies. Over the next few months we will be transforming Org to a cloud-native serverless professional markup offering. Unlike Confluence, we will allow you to both write and edit documents using plaintext markup. We believe that by unlocking the synergy between plaintext and Confluence the Org project will become a thought leader in hyper-scale markup solutions.

The shift to Confluence wiki markup will come as a surprise to many of you, but after doing a deep dive into the scalability of Org we realised that there were no low-hanging fruit available --- only out of the box thinking will allow for the innovation so many of our users crave.

We hope this change will not just disrupt your documents, but the entire markup industry.

]]>
February 2022 https://blog.tecosaur.com/tmio/2022-02-30-orgnvim.html tec@tecosaur.com (TEC) https://blog.tecosaur.com/tmio/2022-02-30-orgnvim.html Fri, 04 Mar 2022 00:00:00 +0800 Timothy (TEC) here. This month we have a guest post from a different part of the Org ecosystem, to highlight one of the most promising efforts to provide a good experience outside Emacs.

//github.com/nvim-orgmode/orgmode

"But I use Emacs, I don't care" you may say. In that case, I'd like to point out that wider spread and better Org support enriches the Org ecosystem as a whole. It makes the format more approachable, and useful for other people. This is good for everybody.

Without any further ado, here's the guest post kindly written by Kristijan. Enjoy!


Like every beginner Vim user, at some point I ran into a usual editor war post: Vim vs Emacs. At that time, I didn't have an idea what "Emacs" was.

A simple Google search yielded something that seemed just like a very simple editor with strange, but more familiar shortcuts. I didn't bother too much to figure out what it is, because I was already pulled in fairly deep into Vim and its philosophy.

Note taking in (Neo)Vim

At first, I did some note taking only when really necessary, in random plain text files. Most of the things I managed to keep in my head, since I was younger and less busy 🙂.

Once I got into the situation where I needed to keep more notes, vimwiki was the natural choice.

That worked very well for a period, until the need for writing quick notes arise. Vimwiki didn't have anything that would allow that. I could of course have a mapping that opens a specific file where I can add notes, but that just never felt right in my mind. I would keep a bunch of things in the same place, and then later I needed to spend some time organizing them.

At that point, I wasn't sure how to achieve what I want. I did a brief look at Emacs OrgMode to see what's all the fuss about, but to me, it seemed just like a different version of Markdown. You put some unordered lists as your notes, and that's it. I never spent more time trying to see all the neat features. I even tried creating some of my custom note taking tools, but I never managed to finish them because I didn't have a clear idea of how to solve my problems.

First encounter with Orgmode like tool: vim-dotoo

One weekend, I was browsing through Vim subreddit, as I usually do at least once a day. There was a post about an "Orgmode like task logging" plugin called vim-dotoo. I opened it up, and I didn't see much at that point. I wasn't too excited. I went through readme, and noticed that author (dhruvasagar) put a fairly big emphasis on the "Agenda view". I had no idea what "Agenda view" is. Thankfully, the author also made a screencast, which is rather long (1.5h), but I had some time, so I went through it.

At that point, I was first met with "Capturing" and "Refiling". My mind was blown! What a simple, yet extremely powerful idea! How had that never crossed my mind? From that point on, this plugin had my full attention.

I'm always emphasizing that dhruvasagar and his vim-dotoo plugin are most deserving for having inspired https://github.com/nvim-orgmode/orgmode, and I can't thank him enough for that.

First steps with vim-dotoo and birth of orgmode.nvim

For some time, I was using vim-dotoo. I moved all of my Vimwiki notes to it. It was a breath of fresh air. Alongside that, I started getting more interest in the original Emacs Orgmode. I started noticing the differences, and some of the missing features that were now looking quite attractive. I made few contributions to vim-dotoo. As time passed, and my notes started to grow, things began being slow. I did some profiling, and figured out that it's just a usual Vim problem, Vimscript performance. It was just too slow for certain things that Orgmode provides, and it would hardly get any better as more things are added.

Separately from Vim and Vimscript, Neovim was on a stable v0.4 release, and v0.5 was still being developed. I was using Neovim from version 0.3, and was carefully following the progress on it. Lua was introduced as a first class citizen. A Bunch of new plugins arise from it. All the benchmarks showed that Lua outperforms Vimscript in almost everything. Besides the performance, Lua is a "normal" programming language, which means that support for it is much better.

At that point, I became curious: Could Lua be the path to the faster Orgmode? I spent several days thinking about it. I wanted to give it a try. My biggest concern was that I had absolutely zero experience writing parsers. I had never written anything more complicated than an averagely complicated regex for purposes of parsing. I noticed that vim-dotoo also used regex to do the parsing, so that eased my mind a bit.

One weekend, I started working on it. It was really interesting and challenging. I spent a lot of my free time on it. At certain points, it seemed like hacking, since it was not a proper parsing. I tried to learn how to write a proper parser, but it was just too time consuming and complicated. I proceeded with the regex parsing to see how far I can go.

Besides parsing, I had a few more challenges to overcome:

3.1. Understanding the OrgMode syntax and all the functionality

This is still the biggest challenge. I didn’t have any idea how big and robust OrgMode is. If I would know it at that time, I wouldn’t even jump on this train. It’s really hard to grasp all of it. Considering I’ve only used it for around 8 months, I think I made some good progress on learning it.

3.2. Remote editing

By remote editing, I mean automatically updating content in the current or any other file. Few examples: adding/updating properties, managing tags, changing TODO states, archiving, refiling, remote editing from agenda view, etc.

There is no built-in way to update content in another file through the Neovim API, without actually opening the file in an editor. I solved this by:

Saving as much position information as possible in the internal state, so I can pinpoint the correct location Opening a file in a 1 row x 1 col floating window and doing quick edits there

3.3. Working with dates

From my experience, dates are challenging in all areas of programming, so this is not so surprising. There are some Lua plugins for dates, but those seemed a bit too basic for my use case, and I wanted to keep external plugins to the minimum. I went with a custom solution that uses Lua’s native dates, which has certain limitations, but works out for most of the things.

3.4. Highlighting, mostly in Agenda view

Vim’s syntax engine is fairly old, but still very much used, especially in the Vim community. Implementation of tree-sitter slightly improved this experience in Neovim, because “Highlight matches” are found via tree-sitter, instead of a bunch of regexes.

This helped me out later for the Org file itself, but agenda view is still something that’s built as a custom view. Old Syntax highlight engine would be really hard to add, because the content is too dynamic. I went with the Neovim highlight API that allows Highlighting things by their exact position in the buffer. Tree-sitter implementation does something similar in the background for Highlighting.

3.5. Keeping configuration simple and familiar to Emacs OrgMode

Vim-dotoo configuration was mostly Vim style, through some global variables. I wanted to have a configuration that is familiar to an Emacs OrgMode user, by having as many options as possible named completely the same as in Emacs.

For example, Here’s a comparison of few options between Emacs and Neovim:

Emacs:

  (setq org-agenda-files '("~/orgmodes"))
  (setq org-agenda-skip-scheduled-if-done t)
  (setq org-agenda-span 7)
  (setq org-hide-leading-stars t)
  (setq org-capture-templates
  '(("t" "Todo" entry (file "~/orgmodes/todos.org")
  "* TODO %?")
  ("j" "Journal" entry (file "~/orgmodes/journal.org")
  "* %?\nEntered on %U\n   %a")))
  

Neovim:

  require('orgmode').setup({
  org_agenda_files = { '~/orgmodes' },
  org_agenda_skip_scheduled_if_done = true,
  org_agenda_span = 7,
  org_hide_leading_stars = true
  org_capture_templates = {
  t = {
  description = 'Todo',
  target = '~/orgmodes/todos.org',
  template = '* TODO %?',
  },
  j = {
  description = 'Journal',
  target = '~/orgmodes/journal.org',
  template = '* %?\nEntered on %U\n   %a',
  }
  }
  })
  

One of the most noticeable differences is between the usage of hyphens (-) and underscores (_). I did that only for the sake of simplicity, because hyphens is not a valid character in variable names in Lua, so all of the options would need to be wrapped as a string (for example: ['org-agenda-files']).

First release of orgmode.nvim and introduction of tree-sitter parser

After ~1.5 months I published the initial version. The focus was on Agenda and capturing (GTD), since those are the things I mostly used. It got some traction, and people started testing it and reporting bugs.

One of the common questions was: “Any plans to introduce tree-sitter parser?”.

I knew about tree-sitter and used it in my day-to-day job for a few programming languages, but I had absolutely no idea how it worked, and especially how to write a tree-sitter parser. I put it aside, and continued working on what I had.

One day, Emilia (milisims) contacted me via email to ask me if I would be willing to try the tree-sitter parser she’s been working on for some time. I gladly accepted. She gave me access to the repository, and I started tinkering with it in a separate branch. No one was aware at that point that tree-sitter support would happen some time soon.

After some time, I set up a “beta” branch called “tree-sitter” and announced it for testing. Once the reported bugs slowed to a trickle, I merged it into the “master” branch.

I believe that tree-sitter grammar for Org could help out other editors to implement their version of Orgmode plugin, but I don’t think it would ever be helpful for Emacs. Emacs parser is the one and only that has it all implemented. Also, as much as tree-sitter is powerful, its main purpose is to parse programming languages, which mostly has “static” patterns to match. Orgmode is by its nature dynamic, which causes a variety of issues for a parser that’s not meant for that kind of usage.

Limitations

(Neo)Vim is a great editor, but it still cannot compare to Emacs in certain things. Manipulating the “View” part of the editor is tricky or impossible for certain things.

I even made a label for reported issues where Neovim support for certain things is a blocker. I’m hoping that at least some of these will be available in future Neovim releases.

Features

I will not go into too many details about the available features, since those can be viewed in repository readme, but I want to mention one feature that does not exist as a built/-in feature in the Emacs Orgmode: Notifications.

This allows getting a “desktop notification” for tasks that are within the specified threshold for schedule/deadline time. It requires some configuration to set up a cron job, but it’s been working great for me for several months now.

Plans

The current state of the project is very usable for me. I’m not lacking any of the major features, mostly because I’m not used to using them. Nevertheless, there are plans to add more things, and I’m getting a lot of help from the community. I want to specifically mention levouh and lukas-reineke, since they added a lot of value to the project, and I want to thank them and everyone else who contributed. Their help is much appreciated.

There are few high priority tasks that I’m hoping to flush out first:

Implementing v1.0.0 release of the tree-sitter parser. This should allow for faster and less error-prone parsing. Infrastructure for plugin developers, to allow other people to build plugins on top of nvim-orgmode.

And a long term goal for these:

Tables support (and at least basic formulas) Org Babel like code block evaluation (and hopefully basic support for literate programming) Diary format dates Custom agenda commands More clocking features (reports) File specific configuration via directives (todo keywords, properties, etc.) Closing thoughts

When I started working on nvim-orgmode, I didn’t have a clue what I’m jumping into. Every day I learn about more and more Orgmode features that I wasn’t even aware existed.

I’m certain that this project will never manage to clone the Orgmode functionality completely, but I’m hoping it will get close enough so everyone from Neovim community and Emacsers trying out Neovim will be able to use it for their needs.

Having experienced Orgmode users testing it is a huge help, so if anyone is willing to give it a try, feel free to open up an issue and write your thoughts there. Thanks!

]]>
January 2022 https://blog.tecosaur.com/tmio/2022-01-31-caching.html tec@tecosaur.com (TEC) https://blog.tecosaur.com/tmio/2022-01-31-caching.html Tue, 01 Feb 2022 00:00:00 +0800 There are less "obvious user-facing changes" this month, but a lot of good work has been happening 🙂.

The cache continues to make strides

Nine months ago, Ihor started working on the org-element cache. The cache has existed for a while now, but it used to be too buggy to be useful.

This is no longer the case! We're finally confident enough to flip org-element-use-cache's default from nil to t. I mentioned some of the improvements in November, however Ihor has continued to make strides with the cache.

Getting the cache right is tough though, and so while the situation is much improved there are still lingering bugs which need to be chased down. If you only take one thing away from the post, I'd like it to be this:

The cache is actively being developed now, for the first time in years, so: Ignore old advice (like this) which suggests disabling the cache Note that the cache will not stuff anything up, at worst it will emit a warning If you see an org-cache warning please report it to the Org mailing list --- emacs-orgmode@gnu.org.

Cache warnings look something like this:

  Warning (org-element-cache): org-element--cache: Unregistered buffer modifications detected. Resetting.
  If this warning appears regularly, please report the warning text to Org mode mailing list (M-x org-submit-bug-report).
  The buffer is: config.org
  Current command: (some-buggy-command)
  Chars modified: 483
  Buffer modified: 330957
  Backtrace:
  ...some possibly long backtrace...
  

A good way to start is by copying this to your clipboard and then trying M-x org-submit-bug-report. Regardless of whether you're using an Emacs mail client, this will collect all the important information (org-mode version, etc.) and populate a buffer with it. Add the warning text, and that will be a great help! After all, it's quite hard to fix bugs we don't know about 😆.

As of October 2021, "bulk" heading promotion/demotion operations are now faster thanks to the cache, and there have now been further improvements relating to headings. It used to be the case that every time heading properties changed the entire heading (and contents) were re-parsed. Now, cache heuristics prevent the content from being entirely re-parsed (see org-element--cache-avoid-synchronous-headline-re-parsing for a bit more information). This improvement is particularly noticeable when refiling and archiving headings.

Persisting data

In October, Ihor started work on org-persist.el --- a way to persist data across Emacs sessions. You may wonder what sort of data is being persisted. Well, initially org-persist was written to record variable values associated with particular files or buffers --- such as the org-element cache. This allows org-mode to load the cache for an Org file and avoid re-parsing much of the content.

Over the past month, Ihor has doubled the total number of commits applied to org-persist.el and now more kinds of data can be persisted, such as remote files (via. tramp) and online resources. This opens up a lot of new possibilities, and we can begin to see what this will allow for in the next highlight.

Exporting documents with remote resources

Ever linked to an online image like https://orgmode.org/resources/img/org-mode-unicorn.svg in an Org document, seen it look nice in the HTML export and then be disappointed to just see a link in LaTeX? Well I have. Thankfully this is no longer the case though, thanks to a helper function org-export-link-localise that's just been added to ox.el.

Using org-persist.el this acquires a local copy of the remote resource and substitutes it in for the remote link. This is currently utilised in ox-latex.el to provide support for not just HTTPS links but also tramp files!

ox-latex-image-link-localisation-demo.png
Figure 4: A screenshot of a (compiled to PDF) LaTeX export, with an HTTPS and tramp images.

While this is currently only implemented for ox-latex, this can be applied generally, and so there's a good chance you'll hear more about this in the future.

There are also thoughts of moving this to the links file (ol.el) which would make this functionality more broadly accessible (e.g. using cached/local versions of a resource in other places, and defining custom downloaders for custom link types).

Other improvements Produce less native-comp warnings Ihor Radchenko Add a header to the clock report table Samim Pezeshki More verbose test failure summary Max Nikulin Parallelise makeing the docs Marco Wahl More org-element tests Ihor Radchenko Bugfixes Fix formatting in elisp-link prompt TEC Correct some checkdoc and compiler warnings Ihor Radchenko Correct signature for org-element-at-point Mark A. Hershberger Fix the LaTeX macron accent in org-entities TEC Ignore narrowing when tangling a file Nicolas Goaziou

]]>
December 2021 https://blog.tecosaur.com/tmio/2021-12-31-retrospection.html tec@tecosaur.com (TEC) https://blog.tecosaur.com/tmio/2021-12-31-retrospection.html Sun, 02 Jan 2022 00:00:00 +0800 Just over eight months ago, I kicked off This Month in Org with an emphatic announcement of the blog's in the form of a Welcome post. If you haven't guessed, this is the first "blog post" I've ever written. In that welcome post, I gave my motivation for starting the blog --- essentially to bridge a perceived gap in information sources between a subscription to the Org project mailing list, and nothing.

That is why I thought this blog should exist, but until now I have neglected to mention what I want to accomplish with it. By starting TMiO I hoped to: Engage more people with the improvements being made to Org[fn1]. Highlight some of the great work being done by Org[fn1] contributors. Encourage more people to consider contributing to Org[fn1]. Foster a stronger sense of an Org[fn1] community, outside the mailing list.

Since then, I've effused about Org to the tune of around ten thousand words. We started off with a fairly dry recount recent changes, which (after initial feedback) has shifted slightly to try to give more context on the improvements and how they may be used. In June I even went as far as to make the majority of the post about pre-existing features (writing Org for LaTeX). This change has been made to: Try to make the blog posts a little more interesting, and less tedious. Potentially introduce readers to nice features of Org they weren't aware of before.

We have now arrived at the first crucial question of this post: How effective has this blog been in achieving its goals?

Let's start off by looking at engagement. There is no tracking on this site, and I'm not even counting page views. We could read into Reddit upvotes (which usually hover around 100-ish per post), but with no strong trend I'm wary of reading too much into those numbers. What about engendering an interest in contributing? This is even harder to consider. It is similarly difficult to judge whether this blog might be helping (even if only a bit) foster a stronger sense of community.

Ok, how about the second crucial question: Moving forwards, what changes should I make to the style of posts, if any?

Hmmm, this is a prickly one too. Both of these questions suffer from the same problem --- I can't answer them. Simply put, I need to hear from you. Whether it be in the Reddit comments section, or by Email (tec@ this domain), to direct 2022's posts I am very interested in hearing your thoughts on: How engaging you've found this blog? (the content, the style of writing, etc.) Whether this blog has influenced your feelings on the Org project and/or community? Whether this blog has affected your thoughts on contributing to Org? What you think this blog has done well/badly over the past year? If there's anything you'd be interested in this blog doing differently in 2022?

That's all for now! Thank you for reading, and have a great new year 🙂

Footnotes

[fn1] Both the org-mode codebase, and also the ecosystem that's sprung up around it

]]>
November 2021 https://blog.tecosaur.com/tmio/2021-11-30-element.html tec@tecosaur.com (TEC) https://blog.tecosaur.com/tmio/2021-11-30-element.html Thu, 02 Dec 2021 00:00:00 +0800 With a 9.5 release highlight post last month, and the month before skipped, it's now three months since the last regular instalment of TMIO. Let's get back up to date on some of the latest happenings with Org.

Org as markup

Looking at the wider ecosystem, it certainly appears that there is a growing appetite for Org markup outside org-mode. More projects like Hugo and Logseq seem to be interested in supporting Org markup, and there has been a recent growth in editor extensions like Neovim's orgmode.nvim (started in March this year) and Sublime Text's OrgExtended (started in June this year).

Interest in Org as a general-usage markup format can also be seen within the Org project. Primarily lead by Nicolas Goaziou, there is an ongoing attempt to codify the Org syntax in a formal specification in the Worg document Org Syntax (draft). Other members of the Org mailing list have directed their effort to creating non-elisp parsers for Org, both to help Org tools be created in other languages, and as put in the README for Tom Gillespie's laundry parser

The long term goal of this work is to provide a reference that can be used to standardize Org syntax and behavior and to specify various levels of compliance for an implementation of Org mode.

Earlier this week Karl Voit, the author of the rather well-known document Org Mode Is One of the Most Reasonable Markup Languages to Use for Text, surprised the mailing list by announcing his independent creation of a multi-leveled standard for Org syntax subsets called "Orgdown" (the name is a blend of "Org-mode" and "markdown", but the standard is only a subset of Org). Each level defines a compliance score given by a mix of parsing and editing support, with example compliance scores for the first (and currently only) level of the standard given for common tools.

At this stage, it isn't clear exactly how the Org-outside-Emacs landscape will evolve, but the swelling interest is very encouraging.

An Org parser in Julia

Speaking of parsers, I may be somewhat biased but I'm quite happy that a Org parser for Julia now exists 🎉.

//github.com/tecosaur/OrgMode.jl

OrgMode.jl is a parser, but also intended as a general-purpose Org library for Julia. It's only been a week since development started, but it currently supports most of the Org Syntax draft specification, along with the rendering of a parsed Org AST to a TTY or back to Org text. A few utility functions are also included, such as filtermap which operates similarly to org-element-map.

Autoloading citation backends

One small but impactful change is autoloading of citation backends. Until recently before say using the csl backend, one needed to (require 'oc-csl) or face error messages.

Now, if you have a line like:

  #+cite_export: FORMAT ...
  

org-mode will try to load the file oc-FORMAT before trying to process citations.

This should make getting started with citations in Org just a bit easier.

A nicer :tangle-mode syntax

The standard way of setting a :tangle-mode has typically been by providing a closure that makes use of Elisp's octal syntax, such as (identity #o755). This is unnecessarily verbose, and certainly doesn't feel natural.

With the addition of a small mode-interpreting function (org-babel-interpret-file-mode) It is now possible to specify :tangle-mode using three different forms of shorthand octal o755 is equivalent to (identity #o755) chmod chmod-style inputs like u+x are now parsed to a file mode[fn1] with the the base/default mode set by org-babel-tangle-default-file-mode. ls -l strings of the form given by ls -l like rwxr-xr-x are also accepted

This means the following forms are now all equivalent:

  :tangle-mode (identity #o755)
  :tangle-mode o755
  :tangle-mode a=rx,u+w
  :tangle-mode rwxr-xr-x
  

It has also been noted on the mailing list that the :tangle-mode (identity #o755) form works by being transformed to :tangle-mode 493 during parsing. Similarly :tangle-mode 755 is equivalent to :tangle-mode (identity #o1363). For some values the decimal and octal interpretation are both valid file modes. Due to the clear potential for confusion, and since file permissions are an important security consideration, it has been suggested on the mailing list that these forms should be depreciated with a warning in future. No decision has been made yet though.

Org element parser cache

Ihor Radchenko has done some fantastic work over the past few months by overhauling parts of org-element.el to introduce extensive caching. org-element is the Org markup parser inside org-mode. This allows for a huge jump in speed, and also provides a few functions which fetch information without updating the cache --- allowing for particularly speedy lookups with a small sacrifice to correctness guarantees on one or two properties in particular cases.

Several org-mode APIs now make use of the cache to dramatically improve speed. Aside from improvements to typically slow operations, this is ideal for situations involving frequent buffer edits. It's no understatement to say that this work is transformative.

One potential beneficiary from this work is actually fontification. It has become increasingly apparent that the current regex-based method for buffer fontification is imperfect, and can actually differ from the true structure of the document as parsed (authoritatively) by org-element. This has lead to the well-received suggestion on the mailing list to rewrite the fontification code to be built on org-element instead.

Inline source block fontification

I think inline source code blocks are an underappreciated feature of Org. I don't think it's helped that they have not been visually treated at all differently from plain text. Now though, they have a new dedicated face (org-inline-src-block) and in the same manner as source blocks, based on org-src-fontify-natively can be fontified using the language's major mode.

inline-src-block-fontified-vs-code.png
Figure 5: Side-by-side comparison of a identical paragraphs using code (~) markup and inline source blocks (src_).

If you aren't familiar with inline source blocks, you're missing out. They are very much the inline cousin of source blocks, and so support all your favourite Babel features like code execution and header arguments. This provides a fantastic capacity to inline dynamically computed expressions, and optionally show the code that produces them.

inline-src-block-julia-demo.png
Figure 6: A paragraph making use of evaluated inline source blocks. Note that the ⟨11⟩ is a prettified results macro (using a potential future org-mode patch).

Functions as default heading arguments

Matt Huszagh has contributed a patch that allows functions to be used as values for default header arguments. This is great for arguments where a sensible default can be provided by evaluating a function on-the-fly.

Consider for example the arguments required to produce a simple image using R with Babel:

  #+begin_src R :results graphics file :file myimage.svg
    library(ggplot2)
    ggplot(mpg, aes(displ, hwy, colour = class)) + geom_point()
    #+end_src
  

In a Jupyter-style (.ipynb) or throwaway document, we likely don't care about the file name at all. With these new capabilities, we can provide a file name dynamically as a default argument!

First we must write a function that when run at the source block will give us a suitable file name, like so

  (defun my/org-src-sha-to-image ()
  (concat "generated-"
  (substring
  (sha1 (org-element-property :value (org-element-at-point)))
  0 8)
  ".svg"))
  

Let's also write a function to guess whether the source block produces a plot by checking if there's a plot command on the last line.

  (defun my/org-src-guess-results-type ()
  (if (string-match-p "^ *\\(?:plot\\|ggplot\\)([^\n]+\n?\\'"
  (org-element-property :value (org-element-at-point)))
  "graphics file" "replace"))
  

Then we can just use these function in place of a static value in the default header arguments variable --- that's all it takes.

  (setq org-babel-default-header-args:R
  '((:results . my/org-src-guess-results-type)
  (:file . my/org-src-sha-to-image)))
  

This means for most cases we can now get away without any header arguments at all.

  #+begin_src R
    library(ggplot2)
    ggplot(mpg, aes(displ, hwy, colour = class)) + geom_point()
    #+end_src
  

It's always lovely to see more ways of reducing boilerplate.

Proportional image widths

Previously, as long as org-image-actual-width was nil or a list of the form (default-value), org-mode would display images according to a :width attribute (e.g. #+attr_html: :width 400px) by simply looking for the first #+attr_ affiliated keyword and reading the numeric component of the :width as the number of pixels wide the image should be.

This has now become somewhat fancier. The image-width determining logic has been extracted to a new function (org-display-inline-image--width) which will now extract floating-point values like 0.7 and interpret them as that portion of the accessible text width in the buffer.

proportional-image-width.png
Figure 7: A containing with an image set to half of the accesible text width

This means that a width parameter like #+attr_latex: :width 0.7\linewidth the image will displayed as 70% of the buffer text width. This also supports percentage value, like #+attr_html: :width 80% by dividing the number before the % by 100 as a floating-point value. As always, if you don't like the way display width is inferred here you can override it by putting a #+attr_org: :width X statement first.

Support for proportional image widths extends to the (default-value) form of org-image-actual-width, as now if you set it to say (0.9) which will cause images without any width specification to be displayed at 90% of the buffer text width.

If you want to have some images displayed as their actual width you can use the new special width parameter t to set this on a per-image basis with #+attr_org: :width t. Now all you need to do is remember to put this first. Based on current discussions on the mailing list though, soon #+attr_org will be prioritised when determining display image width, no matter which order you put the attributes in. I do like having one less thing to remember 🙂.

Other improvements Allow citations immediately following an item bullet TEC Allow citations immediately following a footnote definition Nicolas Goaziou Update some obsolete function references Marco Wahl ob-gnuplot is now maintained by Ihor Radchenko Improve makescript support for ORGVERSION in tag-less mirrors Nicholas Vollmer New ob-julia, now maintained by Pedro Bruel Allow for no indentation, but preserving current indentation by setting org-indent-indentation-per-level to 0 David Lukes Eliminate some byte-compile warnings Nicholas Vollmer Bastien Support Greek smart quotes Juan Manuel Macías org-mouse support for intermediate-state checkboxes Jim Porter Allow nested parenthesis in org-compile-prefix-format %(sexp) expressions Ihor Radchenko oc-csl / citeproc improvements András Simonyi Move more unmaintained/overly niche ob-* files to the contrib repo, reducing the maintainer burden Bastien Allow use of a function for org-agenda-overriding-header for dynamic headers Christopher League Improve org-protocol URI decoding Max Nikulin Remove some obsolete LaTeX packages from the default packages list TEC Add support for text and year citation styles to oc-csl András Simonyi Produce lower-case keywords in ox-org TEC Improve ob-gnuplot argument processing Ihor Radchenko A collection of oc-* improvements Nicholas Goaziou Support bare author citations in oc-csl TEC Add :options LaTeX attribute to tables Juan Manuel Macías Fix display error with ob-plantuml and html export Su Lin More tests! Ihor Radchenko Documentation improvements! Marco Wahl Stefan Kangas Daniel Fleischer Wiliam Denton Thomas Dye Bastien Bruce D'Arcus Kyle Meyer Nicolas Goaziou Bugfixes Fix heading insertion in a case where point is before any heading Marco Wahl Prevent stringp error when tangling Org from an org-src edit buffer Mark Dawson Prevent indent-tabs-mode from messing with justification in ASCII exports Morgan Willcock Fix form of default Babel haskell header args Ihor Radchenko No more duplicated logbook entries for repeated tasks Ihor Radchenko A headline fontification edge case Sébastien Miquel Refactor code that needed Emacs 28 Kyle Meyer Make sure a terminating emphasis marker can't be used as a beginning emphasis marker in fontification Ihor Radchenko Allow footnotes at footnote definition start Nicholas Goaziou Footnotes

[fn1] This is performed easily thanks to file-modes-symbolic-to-number, which is used as the basis for both the chmod and ls -l shorthand interpretations.

]]>
May 2021 https://blog.tecosaur.com/tmio/2021-05-31-async.html tec@tecosaur.com (TEC) https://blog.tecosaur.com/tmio/2021-05-31-async.html Wed, 01 Dec 2021 00:00:00 +0800 Async Babel sessions have landed

Jack Kamm has made a fantastic contribution to Org: ob-comint.el now supports asynchronous output from source blocks with full support for sessions 🎉.

If you haven't used sessions before, you're in for a treat! By simply assigning a named session to a source code block, e.g. :session foo, the same process will be reused for any other code blocks that specify the foo session. To do this for a certain language across the entire document, you can set the header argument property, i.e.

  #+property: header-args:lang :session foo
  

Along with the asynchronous process communication in ob-comint.el, we have an implementation for Python, and we should see support for R and Ruby in the not-too-distant future 😍.

To use this, just add the :async parameter to a python block. Instead of Emacs freezing until it completes execution, you'll see a placeholder inserted which is replaced by the result when it is ready.

emacs-jupyter allowed for asynchronous code execution (with sessions), but it's great to have a solution that doesn't require Jupyter kernels, and is part of Org.

Font lock for inline export snippets

Now, this isn't necessarily a significant change, but I don't think many people know about this feature so I'll take the opportunity to go over it 🙂.

If you want to include a snippet of HTML/LaTeX etc. when exporting to that format, you can use a #+begin_export html block which simply includes the enclosed content verbatim. This doesn't really work for small inline snippets though --- but never fear, Org has inline export snippets which simply follow the form @@format:content@@. For example:

  I love using Org to export to @@html:<abbr title="Hyper Text Markup Language">HTML</abbr>@@ @@latex:\LaTeX{}@@ documents.
  

which will export to HTML as,

  I love using Org to export to <abbr title="Hyper Text Markup Language">HTML</abbr> documents.
  

and then in LaTeX will be,

  I love using Org to export to \LaTeX{} documents.
  

isn't that neat!

Now you'll find the @@ parts using the comment face and the format: bit using the Org tag's face. It's a small change, but it makes it easier to see what's going on.

No mode-specific syntax highlighting yet, but that may come in the future 😉.

#+plot your #+results

Org-plot has been getting some more love as of late (see last month's post), and that hasn't stopped yet. This month there's been a fairly minor change that I'm quite a fan of. If you have a source block that produces a table of #+results, you can now put a #+plot statement immediately above to get a visualisation of those results!

  #+begin_src python
    # pretend this is some profound source of information
    return [[i, i^3 - 4*i^2 - 2*i] for i in range(5)]
    #+end_src

    #+plot: ind:1
    #+RESULTS:
  | 0 |  1 |
  | 1 | -2 |
  | 2 |  7 |
  | 3 |  8 |
  | 4 | 13 |
  

As usual, this relies on gnuplot being present. You can read more about it in the manual.

Fancier LaTeX verses

With the use of verse.sty you can now export fancier verses from Org, as verse blocks now support four new attributes: :lines for the numbering interval. :center which can be set to t and works as long as, :versewidth, the longest line in the verse as a text string is set. You can also easily add arbitrary LaTeX code to be included with :latexcode.

  #+latex_header: \usepackage{verse}

    #+latex: \poemtitle{A Dream Within a Dream}
    #+attr_latex: :lines 3 :center t
    #+attr_latex: :versewidth Through my fingers to the deep,
    #+begin_verse
    Take this kiss upon the brow!
    And, in parting from you now,
    Thus much let me avow —
    You are not wrong, who deem
    That my days have been a dream;
    Yet if hope has flown away
    In a night, or in a day,
    In a vision, or in none,
    Is it therefore the less gone?
    All that we see or seem
    Is but a dream within a dream.

    I stand amid the roar
    Of a surf-tormented shore,
    And I hold within my hand
    Grains of the golden sand —
    How few! yet how they creep
    Through my fingers to the deep,
    While I weep — while I weep!
    O God! Can I not grasp
    Them with a tighter clasp?
    O God! can I not save
    One from the pitiless wave?
    Is all that we see or seem
    But a dream within a dream?
    #+end_verse
  
org-verses-example-poem-dream-within-dream.png
Figure 8: A short Poe-em

Easy zero-width spaces with Org

Occasionally you may run into annoyances when you wish to have two different Org elements right next to each other (no space) --- for example, emphasising part of a word or putting a currency symbol immediately before an inline source block.

The best solution to this in Org is zero-width spaces. As such, I've found it rather nice adding insertion of a zero-width space to the org-mode-map in my config. Perhaps some of you might find this solution useful too 🙂.

  (define-key org-mode-map (kbd "M-SPC M-SPC")
  (lambda () (interactive) (insert "\u200b")))
  

Should you want to keep zero-width spaces out of exports, that's easy enough to accomplish with an export filter.

  (defun +org-export-remove-zero-width-space (text _backend _info)
  "Remove zero width spaces from TEXT."
  (unless (org-export-derived-backend-p 'org)
  (replace-regexp-in-string "\u200b" "" text)))

  (add-to-list 'org-export-filter-final-output-functions #'+org-export-remove-zero-width-space t)
  

Org's repos have moved contrib/ is now at https://git.sr.ht/~bzg/org-contrib for Org 9.5, and it will be available on Non-GNU ELPA. Worg is now at https://git.sr.ht/~bzg/worg Org itself is moving to Savannah Other improvements Add support for HHhMM date formats (e.g. 13h20) Gustavo Barros Make tangling faster and more robust Sébastien Miquel Allow importing tables from files that don't end in .txt, .tsv, or .csv Utkarsh Singh Add an SVG-specific post-export ob-plantuml step: org-babel-plantuml-svg-text-to-path for running Inkscape text-to-path replacement Nick Daly Refactor JavaScript in ox-html Anthony Carrico Set org-html-head-include-scripts to nil by default (was t) Bastien Guerry Remove LaTeX-environment type #+results TEC New capture templates parameter, :refile-targets Richard Garner Merge org-speed-commands-default and org-speed-commands-user into a new variable, org-speed-commands Bastien Guerry URL recognition regex now handles up to two levels of nested brackets, and is tested Ihor Radchenko Cache parent heading positions for faster movement up buffers. Significantly improves Org queries involving inheritance (15-50x improvement) _Ihor Radchenko_ New command ~org-refile-reverse bound to C-c C-M-w Adam Spiers Allow inline tasks to be dragged up/down Bastien Guerry LaTeX export, allow arbitrary :float values Thomas S. Dye Org attach git, new option org-attach-git-dir to use the attachment dir as a git repository Juan Manuel Macías Use a write-back buffer in Org src Sébastien Miquel Add the :environment and :options LaTeX attributes to Quote blocks Juan Manuel Macias Surround LaTeX :options attribute with curly braces when it contains square braces Markus Huber Recognise the specific contributions of Kyle Meyer, Ihor Radchenko, and TEC in the manual Bastien Guerry Improve test coverage Ihor Radchenko, Kévin Le Gouguec A bikeshed of new maintainers New maintainer for ox-html --- TEC New ob-ledger maintainer --- Eric S Fraga New ob-awk maintainer --- Tyler Smith New ob-calc maintainer --- Tom Gillespie New ob-asymptote and ob-coq maintainer --- Luc Pellissier General code cleanup Bastien Guerry Documentation improvements Bastien Guerry, Eric S Fraga, Samim Pezeshki, Nicolar Goaziou Bugfixes Make ob-gnuplot work on remote systems Ihor Radchenko Stop M-x org-toggle-link-display from affecting emphasis markers Bastien Guerry Inserting a heading before a headline Bastien Guerry Perform org-entry-put in a read-only buffer Ihor Radchenko Emacs 24 compatibility for ob-C and org-agenda-show-new-time Kyle Meyer Maintain Org's keybindings when visual-line-mode active Bastien Guerry Keep track of start of region beginning in org-table-wrap-region Bastien Guerry Ensure correct visibility when unfolding subtree Bastien Guerry Corner case in org--backwards-paragaph-once Bastien Guerry Some byte compiler silencing Kyle Meyer Make tags filtering case-sensitive Bastien Guerry C-c C-c on a #+plot line can no longer cause an error Bastien Guerry Ensure consistent position-translation in org-src by using point for position instead of column TEC Prevent ob-sql from getting stuck on an error Ivan Sokolov Make org-columns respect global-visual-line-mode, and simplify tag scanning Nick Savage Don't fontify :: in headlines as description item Ihor Radchenko Handle a few corner-cases in ob-R Jeremie Juste Many fixes to org-indent-line Bastien Guerry Make headline numbering consistent with TOC Mingkai Dong Allow read-only tangling again, and ensure the correct filename is used Sébastien Miquel Fix edge case when converting durations to minutes Nicolas Goaziou Make org-refile work in non-file-visiting buffers Bastien Guerry Be more rigorous in org-sort-remove-invisible Nicolas Goaziou Don't update checkbox stats when heading has todo COOKIE_DATA Bastien Guerry Don't recognise a lone :END: to be forming a drawer Nicolas Goaziou Allow new footnotes in empty table cells Nicolas Goaziou

]]>
October 2021 https://blog.tecosaur.com/tmio/2021-10-31-release.html tec@tecosaur.com (TEC) https://blog.tecosaur.com/tmio/2021-10-31-release.html Mon, 01 Nov 2021 00:00:00 +0800

Turns out that life became busy enough that instead of delaying, last month's post had to be cancelled. We're now back to business as usual though 🙂.

There have been some cool recent developments in Org over the past two months, but you'll have to learn about those is next month's edition as Org 9.5 has been released 🎉. So, let's go over some of the changes I'm most excited about, in no particular order. To get a more complete picture of the latest changes, see ORG-NEWS.

The new citation engine

As tempted as I am to wax lyrical about the lovely new citation engine, I've already dedicated July's post to it, and so will simply highlight how versatile the syntax is. Combined with the ability to swap out the default backends (basic, CSL, natbib, and bib(la)tex) for a 3rd party backend (for example, provided by a package) it is capable of meeting your citation and technical document publishing needs, whatever they may be.

citation-structure-full.svg
Figure 9: The full citation syntax

Since the announcement of org-cite, the ecosystem has continued to expand with a number of promising packages like Bruce D'Arcus' citar (previously bibtex-actions), which currently provides what is arguably the best citation insertion experience.

org-citar-insertion.png
Figure 10: Inserting a citation with the citar package

Asynchronous session evaluation

Since being featured in May's post, we've had the initial python support expanded with support for R too. I have good reason to believe that more backends will join this list in the future.

To get started, just add :async to the header arguments of Python or R source blocks with a :session.

LaTeX environment #+results are now removed

LaTeX environments (i.e. \begin{} ... \end{} blocks) have been added to the list of #+results types that will be removed. This is bigger news than one might think, as it means that raw LaTeX environments are now a viable output for org-babel backends. This possesses a distinct advantage over :results latex as LaTeX environments can be rendered inline with org-latex-preview and are exported to more formats --- for example HTML with MathJax.

This is likely going to be first seen in the new ob-julia backend (unreleased, currently in early development), but could well be utilised by other backends such as ob-octave, ob-mathematica (in contrib), ob-python (using SymPy), ob-calc and more.

ob-julia-latexify-rendered.png
Figure 11: Rendered LaTeX representations of some Julia values

More fontification

A number of new faces have been introduced to allow for improved theming capability, and better contextual hints in the agenda, namely: org-agenda-date-weekend-today, org-imminent-deadline, org-agenda-structure-secondary, and org-agenda-structure-filter.

Inline export snippets are now also fontified using the org-tag and font-lock-comment-face faces, to better stand out from surrounding text.

More link formatting options with org-capture

A new formatting directive, %L has been introduced which provides the bare link target. This allows for links with automatically generated descriptions 🙂.

Export improvements

The HTML and LaTeX backends have both been given some love in this release of Org.

The HTML backend now supports customisation of the <meta> tags included with org-html-meta-tags, which should be particularly useful to anybody using org-publish for blogs or websites (in fact, this blog has been making use of it for some time now). A new variable has been introduced to help with styling, org-html-content-class ("content" by default) which is used as the CSS class for the top-level content wrapper. To further improve styling capabilities, org-html-style-default and org-html-scripts have been changed from constants to customisable variables.

The LaTeX backend (ox-latex) no longer has obsolete LaTeX packages in org-latex-default-packages-alist (grffile and texcomp have been removed). It also now supports arbitrary :float argument values, and accepts a six new arguments (in total) for verse (:lines, :center, :versewidth, and :latexcode) and quote blocks (:environment and :options).

Project changes

To reduce the maintainer burden, the contrib/ folder and a collection of rarely-used or barely-maintained ob-* backends have been moved to a new repo, org-contrib. Support for Emacs 24 (2012--2014) has also been dropped, and Org will now aim to support the three most recent major versions of Emacs.

]]>
April 2021 https://blog.tecosaur.com/tmio/2021-04-30-export-plot.html tec@tecosaur.com (TEC) https://blog.tecosaur.com/tmio/2021-04-30-export-plot.html Mon, 27 Sep 2021 00:00:00 +0800 A discussion on contributor support

Concerns were raised[fn1] about some contributors' patches languishing, and it not being made clear how long it might take to get a response from someone.

In response to this, a the new role of Contributor Steward has been created to: help ensure contributors get a timely response, help out with preliminary patch feedback, and keep updates.orgmode.org up to date.

Org now has three Contributor Stewards to ease the process for patch submitters and core maintainers: Timothy / TEC Tim Cross John Corless

If you've been thinking about getting involved with Org, now is a great time to give it a shot!

Ways you can contribute to the project

Test patches, improve documentation, translate pages, confirm bugs, feedback on a proposed feature, and more...

DOI link exporting

Digital Document Identifiers (DOIs) are an ISO-standardised way of robustly linking to a particular online resource. You'll see these a lot with academic papers, for example.

Thanks to Nicolas Goaziou, when exporting to HTML, LaTeX, Ascii, and texinfo DOIs are turned into links, for example becomes:

  <a href="https://doi.org/10.18637/jss.v046.i03">doi:10.18637/jss.v046.i03</a>
  
  \href{https://doi.org/10.18637/jss.v046.i03}{doi:10.18637/jss.v046.i03}
  
  @uref{https://doi.org/10.18637/jss.v046.i03}
  

This is a minor change, but I think DOIs are great, so I'm highlighting it.

Org plot improvements

Over the past month org-plot.el has received some attention, my two favourite changes are: You can now call org-plot/gnuplot with C-c C-c on a #+plot line TEC When an image is regenerated, all instances of the image in the buffer are refreshed TEC

Other than a few minor tweaks and bug fixes, that's it for April. However, over the last year there have been some rather nice improvements that I didn't mention in the initial blog post, so let's go over them now.

The inbuilt plot types have been abstracted out into a new structure: org-plot/preset-plot-types. This means if you have a gnuplot template you find yourself using a lot, you can now turn it into a custom plot type 😀 TEC A new plot type has been added: a radar / spider plot TEC Some new plot keywords have arrived too TEC transpose (trans) --- The plot internally does something very similar to M-x org-table-transpose-table-at-point before using the table data. ymin (min), ymax (max), xmin, xmax --- Four new keywords (and two aliases) to set the bounds of a plot. Partially supported by the default types. ticks --- The number of axis ticks to use along the axis. A good value is guessed based on a prime factorisation based heuristic (welcome to improvements). Some new customisation functions --- The new variables org-plot/gnuplot-script-preamble, org-plot/gnuplot-term-extra, and org-plot/gnuplot-script-preamble open up new ways to tweak plots to your liking. For example, I use this to set line and background colours based on my current Emacs theme.

If you haven't used Org plot before, I think it's a great way to quickly visualise data in a table. To get started, all you need is a #+plot line above the table, with a certain type specified (e.g. type:2d for a 2d line plot). Then, if you can specify a certain columns as the independent variable (x-values) with ind, and list dependant variables (y-values) with deps.

You can see these parameters set in the figure above,

  #+plot: type:2d ind:1 deps:(2 3 4)
  | Xval | Red | Blue | Green |
  |------+-----+------+-------|
  |    0 |   1 |    2 |     3 |
  |    1 |   2 |    3 |     4 |
  

This will call gnuplot and a window showing the plot will appear. If you want to save the plot to a file, just use the file parameter, e.g. file:"demoplot.svg" (note the quotes).

That should get you started, you can see the manual for the full list of available keywords and find more examples of usage on worg.

Tweaked ox-html style

As displays become more high-res, lines of text which span the whole screen become ... long. So long that it genuinely makes it harder to read the text. A small tweak to the default style and lines are now capped at 60em wide and centred in the page --- much better 🙂. TEC

Also, the HTML export now: has a slightly nicer source block style labels authinfo blocks A collection of export improvements Verbatim in headings no longer breaks LaTeX exports TEC Make the top level class for exported HTML customisable via org-html-content-class / #+HTML_CONTENT_CLASS Sameer Rahmani Use <img> tags for SVGs with ox-html, for better behaviour and W3C compliance TEC Remove redundant type="text/javascript" from <script> elements Bastien Guerry ox-texinfo now generates better headlines, menus, and footnotes Nicolas Goaziou Parsing during exporting no longer aborts as soon as an #+option key without a value is encountered, instead that key is skipped over Nicolas Goaziou org-html-scripts and org-html-style-default have been changed from constants to configurable values TEC eval macros #+macro: ? (eval ...) are now a little bit faster Stefan Monnier Miscellaneous changes org-link-descriptive is now buffer-local, to avoid interfering with other buffers Kyle Meyer org-colview no longer chokes when a special property is updated Nicolas Goaziou Now coderefs have their whitespace cleaned up during tangling Tom Gillespie Allow for multiple %(expressions) in org-agenda-prefix-format Ihor Radchenko Code cleanup and refactoring Nicolas Savage, Aaron L. Zeng, Nicolas Goaziou, Bastien Guerry, Stefa Monnier, Arne Babenhauserheid Documentation improvements Jorge Neto, Erik Hetzner, Cheong Yiu Fung, Kyle Meyer New ob-sqlite maintainer --- Nick Savage Make lilypond header arguments user-configurable Jamie Bayne Fix ob-C regression which mixed up int, double, and char*. Fix another regression with table parameters tbanel Fix indentation of list items and new logbook drawer notes Bastien Guerry Notice when theme changes with LaTeX previews Yuri Lensky Iron out a few edge cases in ol.el (Org links) Nicolas Goaziou Some new tests for org-protocol Maxim Nikulin Footnotes

[fn1] Disclosure: this is me.

]]>
July 2021 https://blog.tecosaur.com/tmio/2021-07-31-citations.html tec@tecosaur.com (TEC) https://blog.tecosaur.com/tmio/2021-07-31-citations.html Mon, 27 Sep 2021 00:00:00 +0800 Last month I not-at-all-subtly hinted that a certain long-awaited feature was arriving imminently. At this point, I think it's a good idea to set the tone for the rest of this post.

celebrate-citations.svg

Citations

After years of (on and off) discussion[fn1], I am elated to be able to present Org's new native citation syntax. Org has grown a thoroughly designed, modular, capable citation system. At last you can refer to Org for all your attribution needs. Special thanks must go to Nicolas Goaziou for leading the charge, John Kitchin for paving the way with the org-ref package, Bruce D'Arcus for driving a lot of careful consideration of design decisions and starting to document some of the details --- and the many other denizens of the mailing list who have contributed to the discussion over the years.

András Simonyi's also deserves a special mention for his work creating the Elisp CSL library Citeproc.el, which while not directly included in Org is crucial to providing robust CSL support, and integrates with oc-csl.el.

10.1. Outline

Citations have been carefully designed in such a way that users and Elisp tinkerers will be able to easily adapt and extend it to fit their needs. To that end, Org Cite (or OC for short) has been split into two halves: oc.el which defines the syntax and provides some machinery to interact with citations Citation processors which interface with oc.el to produce nicely-formatted citations to be inserted in your bibliography, within the text, and even rendered in the buffer[fn2]

There are four capabilities that Org Cite uses the processors for Inserting and editing citations Following citations to their definition Fontifying the citations in the buffer Exporting the citations

Each capability can have a particular citation processor assigned, independently of the others. Out of the box, Org uses the basic processor for all of these tasks.

The basic citation processor is one of four currently bundled with Org: basic, which has no dependencies and provides all four capabilities. It export to all formats, but only provides very simple citations. biblatex and natbib, which provide the export capability to create citations via Biber and (to a lesser extent) natbib, but only for LaTeX. csl, which provides the export capability using the Citation Style Language, and exports to HTML, LaTeX, Org, and plain text (with an open issue for ODT) — but depends on citeproc.el.

This provides a solid foundation for other packages to build off, and despite Org Cite being yet to be released or documented in the manual we are already seeing the development of packages like org-ref-cite (by John Kitchin).

10.2. Basic usage

To get started with Org Cite, we must have some form of bibliography. This can either be a BibTeX file or a CSL-JSON file.

As an example, say we have a file orgcite.bib containing the following

  @article{OrgCitations,
  author={org, mode and Syntax, Citation and List, Mailing and Effort, Time},
  journal={Journal of Plain Text Formats},
  title={Elegant Citations with Org-Mode},
  year={2021},
  month={7},
  volume={42},
  number={1},
  pages={2-3}}
  

First we need to let Org know about this bibliography file (which must have a .bib, .bibtex, or .json extension), which we do either via the #+bibliography keyword, or the variable org-cite-global-bibliography.

  #+bibliography: orgcite.bib
  

Once you have a bibliography source, you can start referencing to your heart’s content! The basic citation syntax is as follows:

citation-structure-basic.svg
Figure 12: The new citation syntax, for simple citations

Using the default style [cite:@OrgCitations] produces . For more information on the styles currently available, see 2.

Finally, to insert a bibliography somewhere, we just need to insert the #+print_bibliography keyword, like so:

  #+print_bibliography:
  

So, to summarise, all one needs to get started is:

  #+bibliography: references.bib
  [cite:@key]
  #+print_bibliography:
  

That’s it! 🎉

10.3. The cite syntax

Don’t let the simplicity in the examples above fool you, the new syntax is quite capable of expressing more complex forms. Here’s the full version of the new cite syntax:

citation-structure-full.svg
Figure 13: The new citations syntax, in full

The style and variant determine what form the exported citation takes The common prefix and suffix and put at the start and end of the generated citation, respectively The citation key refers to a Bib(La)TeX or CSL-JSON key The citation prefix and suffix are put before and after the reference to the key Some citation processors recognise locators, which refer to a particular part of the work, for example: p. 7 to refer to page 7.

Using the default CSL citation style (Chicago author-name) [cite/l/b:see @OrgCitations pp. 7 for fun] becomes .

The citation styles and variants, and recognised locators are handled by the citation processors. Org cite’s bundled processors currently supports the following citation styles.

Table 2: The current set of supported citation styles with variants, with samples generated by oc-csl.el and citeproc.el.
Style Variant Sample Bib(La)TeX NatBib
a author cf caps-full   Citeauthor  
a author f full   citeauthor citeauthor*
a author c caps   Citeauthor* Citeauthor
a author     citeauthor* citeauthor
na noauthor b bare     citeyear
na noauthor     autocite* citeyearpar
l locators bc bare-caps   Notecite  
l locators b bare   notecite  
l locators bc caps   Pnotecite  
l locators     pnotecite  
n nocite     nocite nocite
t text b bare     citealp
t text c caps   Textcite Citep
t text f full     citep*
t text bc bare-caps     Citealp
t text bf bare-full     citealp*
t text cf caps-full     Citep*
t text bcf bare-caps-full     Citealp*
t text     textcite  
(default) b bare   cite citealp
(default) bc bare-caps   Cite Citealp
(default) f full     citep*
(default) bf bare-full     citealp
(default) cf caps-full     Citep*
(default) bcf bare-caps-full     Citealp*
(default)     autocite citep

The CSL processor supports the following locators: book, bk., bks. chapter, chap., chaps. column, col., cols. figure, fig., figs. folio, fol., fols. number, no., Os. line, l., ll. note, n., nn. opus, op., opp. page, p, p., pp. paragraph, para., paras., , ¶¶, §, §§ part, pt., pts. section, sec., secs. sub verbo, s.v., s.vv. verse, v., vv. volume, vol., vols.

10.4. More on exporting

The style of the citations and the bibliography depend on three things: The citation processor used The citation style The bibliography style

The citation processor is automatically selected based on org-cite-export-processors based on the export format being used, but can be set on a per-document basis via the #+cite_export keyword. Here, I shall use the csl processor,

  #+cite_export: csl
  

With org-cite-export-processors, you can also set the bibliography and citation style by giving a triplet of parameters (PROCESSOR BIBLIOGRAPHY-STYLE CITATION-STYLE) instead of just the processor. You can also use this triplet of values with the #+cite_export keyword

  #+cite_export: processor bibliography-style citation-style
  

There are also some more options about how the bibliography is produced. These options are handled by the active citation processor. For example, while the CSL processor does not currently support any options, the BibLaTeX processor passes options to a \printbibliography command, allowing for the following:

  #+print_bibliography: :section 2 :heading subbibliography
  #+print_bibliography: :keyword abc,xyz :title "Primary Sources"
  

10.5. Using CSL

Citeproc is currently available on MELPA, and so can be installed via your package manager of choice so long as MELPA is included in your package-archives. When available, it will be automatically loaded by oc-csl.el.

It currently supports exporting to: HTML LaTeX Org Plain text

Should you be interested in other formats, know that Citeproc is designed to easily support adding new formats (see citeproc-formatters.el for examples).

Citeproc can currently retrieve bibliographic information from the following formats: CSL-JSON Bib(La)TeX org-bibtex

Though support for Bib(La)TeX and org-bibtex is rudimentary compared to CSL-JSON.

When exporting, you can set the style by providing a path to CSL style files, either absolute or relative to org-cite-csl-styles-dir. For example, if I download apa.csl I can use it like so:

  #+cite_export: csl ~/Downloads/apa.csl
  

When no style is given org-cite-csl--fallback-style-file will be used, which defaults to a bundled Chicago author-date style.

10.6. Working with Zotero

There are quite a few reference managers available, however, the list rapidly shrinks if you restrict yourself to applications which are: somewhat feature-rich open source software not owned by a parasitic company[fn3]

Zotero is a good option, and if you’re using it it’s quite easy to use it with Org Cite. Out of the box, you can tell it to export your library, or parts of it, to a .bib file and automatically keep it in sync. I’d recommend installing the Better BibTeX extension though.

zotero-export-library.png
Figure 14: Zotero library right click context menu, showing the export option
zotero-export-options-prompt.png
Figure 15: Zotero collection export dialog

Zotero also works well with CSL. In addition to supporting CSL-JSON exports, Zotero also features an easy way to install CSL styles within the preferences.

zotero-cite-styles-menu.png
Figure 16: Zotero CSL style management within preferences

Since these files are put under ~/Zotero/styles, you can use them with Org Cite and Citeproc simply by setting org-cite-csl-styles-dir to the Zotero styles directory.

  (setq org-cite-csl-styles-dir "~/Zotero/styles")
  

To then use the citation style defined by ~/Zotero/styles/apa.csl one can then simply refer to apa.csl when using the #+cite_export keyword.

  #+cite_export: csl apa.csl
  

10.7. A bright future

Org Cite has only just been merged in the past month, and is yet to be included in an Org release, but we’re seeing a tremendous degree of community interest. There are already promising developments with third-party packages, such as bibtex-actions and org-ref-cite. I can’t wait to see how the ecosystem continues to develop 😃.

Footnotes

[fn1] Citations were first being mentioned on the mailing list back in 2007, in a thread about footnotes.

[fn2]There is currently an ongoing effort to use oc.el and citeproc.el to produce citation overlays in the buffer.

[fn3] I’m talking about a certain company created by a British Fraudster that has a 40% profit margin, engages in blackmail-like practices with universities, prompted 19,000 researchers to boycott them, published six fake journals, vigorously lobbys against Open Access, charged for Open Acess articles (repeatedly), made financial contributions to politicians who then tried to prevent publicly accesible reaserch, and whose reference manager encrypted reaserchers’ own databases “to comply with GDPR”.

]]>
August 2021 https://blog.tecosaur.com/tmio/2021-08-38-roaming.html tec@tecosaur.com (TEC) https://blog.tecosaur.com/tmio/2021-08-38-roaming.html Wed, 08 Sep 2021 00:00:00 +0800

Dear readers, unfortunately I've been unusually busy as of late, and will continue to be for a bit over a month. As such, it's entirely likely that the next post or two may be late make use of creative dates too.

In terms of Org development, we've had a fairly slow month --- mostly tweaks and fixes, concentrated on the new org-cite functionality. However, there have been big developments with Roam as of late.

Org-roam v2

Pushed to the sidelines by last month's hugely significant citations announcement was the release of Org-roam v2 🎉. The project's author Jethro wrote a nice blog post outlining the major changes and motivations, and so I'll leave him to speak on that. Suffice to say, the idea of org-roam has been refined into a form that should better server tinkerers looking to build off Org roam, and make maintenance easier --- thus improving the user experience in the long term 🙂. Should you be worried that the v1 -> v2 upgrade will be painful, read Org-roam v2 doesn't hurt, relax, and try the upgrade wizard.

Org-roam-ui

Within a month of Roam v2's announcement, we were pleasantly surprised by the release of org-roam-server usurper --- org-roam-ui.

org-roam-graph-2d-overview.png
Figure 17: org-roam-ui's 2D overview.

While org-roam-server remains incompatible with Roam v2, org-roam-ui is built around v2 from the ground up. With this shift comes a much more ambitious set of features. The graph view itself is now based on react-force-graph, which allows for both 2D and 3D views of the graph, with WebGL acceleration.

Using emacs-websocket, org-roam-ui establishes a two-way connection between Emacs and the Web visualisation. This allows for nifty features like syncing the graph view and the currently open roam file in Emacs. Furthermore, with xwidget-webkit you can even embed org-roam-ui inside Emacs![fn1]

org-roam-ui-in-emacs.png
Figure 18: Embed org-roam-ui in Emacs using xwidget-webkit.

Beyond this, org-roam-ui also provides a number of other nice features, such as filtering and theme syncing, with more in the works. To see more, check out the README published on GitHub.

/github.com/org-roam/org-roam-ui

New ConTeXt exporter

We've long been able to export to LaTeX files with ox-latex, but now you can export to ConTeXt too with ox-context! While only just released, a lot of work has already gone into this --- with development starting almost a year ago by the commit history.

//github.com/Jason-S-Ross/ox-context

It's always nice to see more export options for Org, and I wish Jason all the best in developing ox-context.

Other improvements Refactor oc-natbib.el, oc-biblatex, oc-basic.el, org-compat.el to improve byte compilation Maxim Nikulin Nicolas Goaziou Marco Wahl Allow for selecting cite style with a citation argument Nicolas Goaziou Add support for Italian "smart quotes" DPDmancul Fix Spanish mistranslation in footnote Documentation Juan Manuel Macias Define \citeprocitem in oc-csl.el for use by citeproc-el TEC Fix error in org-export-define-backend docstring Zachary Kanfer Nicolas Goaziou Document the :results none option Yasushi Shoji Other documentation improvements: Jorge Neto Marco Wahl Bugfixes When tangling a certain language, be more careful about which blocks are matched DMG, Nicolas Goaziou Make q in org-agenda work even when debug-on-error is set Lars Ingebrigtsen Make org-attach work again for Emacs \(< 28\) Kyle Meyer, Marco Wahl Make rx usage in org-cite files compatible with Emacs \(< 27.1\) Max Nikulin Prevent ' from becoming a LaTeX active char in exports using oc-csl TEC Have org-colview dynblock match and mixlevel interact sensibly Nick Dokos Behave appropriately when ffap-url-regexp is nil, instead of throwing an error Peder Stray TEC Footnotes

[fn1] I have found the xwidget-webkit experience quite unstable until I started using Emacs 28 with the pure-GTK branch. So, if you're not getting a good experience now, there's a good chance you'll have a better experience in the future.

]]>
June 2021 https://blog.tecosaur.com/tmio/2021-06-34-relaxed.html tec@tecosaur.com (TEC) https://blog.tecosaur.com/tmio/2021-06-34-relaxed.html Mon, 02 Aug 2021 00:00:00 +0800 The previous two months have been pretty good for Org development --- with many bug fixes and feature improvements. This month has been substantially slower than the last[fn1], but that's not to say not much is happening: in fact, there are some rather nifty contributions lined up for the not-too-distant future and a certain long-awaited feature branch[fn2] is getting very close to merging 😀. You'll just have to stick around to hear about those in a future edition of TMIO 😉.

dilbert-zenos-paradox.jpg
Figure 19: It's right around the corner, I swear!

Customise the reference command used in LaTeX

Previously, whenever you linked to another part of your document (with or without giving it a name) --- for example with [[Profound section]] or similar --- when exporting to LaTeX Org would always use the \ref command.

org-latex-default-reference-to-sec.png
Figure 20: A LaTeX export of a simple document with a reference to both the first and second section. "2" what? Section 2, Table 2, Figure 2, ...

You can now set the format string org-latex-reference-command (\\ref{%s} by default) to anything you'd like. For example, making use of the cleveref package I can set this to \\cref{%s} and then add ("capitalize" "cleveref" nil)[fn3] to org-latex-packages-alist.

org-latex-cref-reference-to-sec.png
Figure 21: A LaTeX export of the same document, but now using cleveref. Note the change from "1" and "2" to "Section 1" and "Section 2".

A diversion into writing Org for LaTeX

Speaking of LaTeX exports, a member of the Org mailing list recently told us about a paper pushed to arXiv which was written entirely in Org. Why don't we use that as a prompt to talk a bit about generating LaTeX documents from Org?

For an experienced LaTeX-er, Org may initially appear best suited to simple documents, but in fact it's possible to reproduce any LaTeX structure in Org with no more difficulty (often less) than in LaTeX.

12.1. Simple elements

The “basic” Org elements are simply translated to their LaTeX counterparts. Markup like bold, italic, etc. are simply translated through org-latex-text-markup-alist.

For those of us who dabble with equations, Org is very accomodating. You can type (LaTeX-style) inline and display equations in exactly the same way (\( \) and \[ \]), and what’s more, if you have a LaTeX environment statement \begin{...} on its own line, Org will recognise it and pass it into the generated LaTeX.

12.2. Figures and tables

One area where the improvement when moving to Org is particularly apparent is with figures and tables. To simply include an image, an image link alone is sufficient.

When exported to LaTeX this will be expanded to

  \includegraphics[width=.9\linewidth]{figures/salvador-dali-persistence-of-memory.jpg}
  

As soon as you add a #+caption, though, Org knows you mean business and generates a proper figure.

  \begin{figure}[htbp]
  \centering
  \includegraphics[width=.9\linewidth]{figures/salvador-dali-persistence-of-memory.jpg}
  \caption{A famous surrealist painting}
  \end{figure}
  

As you may have guessed from the fact this works without a LaTeX-specific keyword, this works nicely in HTML too 🙂.

salvador-dali-persistence-of-memory.jpg
Figure 22: A famous surrealist painting

The LaTeX backend also accepts additional image attributes (manual page). For example, to set the image width I can simply add

  #+attr_latex: :width 0.4\linewidth
  

above the image link.

You can do the same with tables:

  #+caption: A selection of famous paintings by Salvador Dalí
  | Year | Painting                   |
  |------+----------------------------|
  | 1931 | The persistence of memory  |
  | 1937 | Swans reflecting elephants |
  | 1837 | Metamorphosis of narcissus |
  | 1952 | Galatea of the spheres     |
  | 1966 | Tuna fishing               |
  

I like to set (setq org-latex-tables-booktabs t) to use the nice booktabs rules in the generated tables. Just remember to ensure the booktabs package is loaded.

  \begin{table}[htbp]
  \caption{A selection of famous paintings by Salvador Dalí}
  \centering
  \begin{tabular}{rl}
  \toprule
  Year & Painting\\
  \midrule
  1931 & The persistence of memory\\
  1937 & Swans reflecting elephants\\
  1837 & Metamorphosis of narcissus\\
  1952 & Galatea of the spheres\\
  1966 & Tuna fishing\\
  \bottomrule
  \end{tabular}
  \end{table}
  

Org is nice and does the right thingTM by including the caption at the top.

org-table-to-latex-example.png
Figure 23: Look ma, I put the caption in the right place.

There are also some more attributes you can supply to tables. Should I want the table to spread out I could use #+attr_latex: :environment tabularx (as long as I’ve loaded the tabularx package) and then set the columns with :align lX.

12.3. Code blocks

By default, source code blocks are translated verbatim. We can do better than that however. We can tell Org to use listings, but I’d recommend going one step further and using minted. For this to work we need to perform three actions: Tell Org we want to use minted environments for source code Load the minted package by default Add -shell-escape to our LaTeX compiler flags, so minted may call pygments.

This can easily be accomplished via the following snippet:

  (setq org-latex-listings 'minted
  ;; as long as you have latexmk installed
  org-latex-pdf-process
  '("latexmk -f -pdf -%latex -shell-escape -interaction=nonstopmode -output-directory=%o %f"))
  (add-to-list 'org-latex-packages-alist '("" "minted"))
  

To customise minted, as well as inserting content into the preamble, one can also customise org-latex-minted-options to control what options are applied to each minted environment.

12.4. Custom environments

Org has a number of blocks which are treated specially, like #+begin_src for source code, and #+begin_centre for centred text. When exporting this same syntax allows you to wrap Org content in any LaTeX environments (as long as it doesn’t match one of Org’s recognised environments).

For example, if you wrote a warning environment in LaTeX to box and emphasise text, to wrap some Org content in it one simply needs to write:

  #+begin_warning
  Pay close attention! This is very important.
    #+end_warning
  

and the content will be wrapped in \begin{warning} ... \end{warning}.

12.5. The LaTeX escape hatches

Should there be a particular LaTeX command you wish to insert somewhere, you simply need to put it on its own line with #+latex: in front and it will be transferred to the generated LaTeX (this works with other formats too).

  #+latex: \newpage
  

For larger snippets of LaTeX, there’s always the export block.

  #+begin_export latex
    \cleardoublepage
    \vfil
    \hfil This page is intentionally left blank \hfil
    \vfil
    \newpage
    #+end_export
  

12.6. Preamble content

Should you wish to include the line in the preamble (before \begin{document}), then all you need to do is use #+latex_header:.

  #+latex_header: \newcommand{\RR}{\mathbb{R}}
    #+latex_header: \usepackage{svg} % so that file:*.svg works nicely
  

This is great for adding one-off \usepackage commands, but what if you find yourself wanting a package (like svg) to be always included? Well the we have the aforementioned org-latex-packages-alist which will include the packages set when exporting; you can even set some packages to only be included when using a certain LaTeX compiler.

Should you want to use a certain preset preamble, you can make use of the #+latex_class keyword. This is used to set the base preamble template used when generating the LaTeX. See org-latex-classes for what’s available by default. You should see entries for: article report book beamer

One of these is always used when generating LaTeX; when no #+latex_class is set in the document, the template named by org-latex-default-class will be used.

What’s great about this is that is makes it really easy to add your own templates. Each template simply takes three components: A name A preamble template A series of format strings to translate headings to LaTeX, with and without numbering

For example, I’m quite a fan of the KOMA-script family. Should I want to add a kart class (for: koma article), I simply need to do something like the following:

  (add-to-list 'org-latex-classes
  '("kart" ; class name
  "\\documentclass{scrartcl}" ; preamble template
  ("\\section{%s}" . "\\section*{%s}") ; H1 translation
  ("\\subsection{%s}" . "\\subsection*{%s}") ; H2 translation
  ("\\subsubsection{%s}" . "\\subsubsection*{%s}") ; H3...
  ("\\paragraph{%s}" . "\\paragraph*{%s}")
  ("\\subparagraph{%s}" . "\\subparagraph*{%s}")))
  

See the documentation for org-latex-classes for more information on how the preamble template in handled.

Other improvements ox-koma-letter.el has been brought into Org’s main directory from the ) contrib/ repo Bastien Guerry Speed up publishing by using delayed hooks and temp buffers instead of finding files Gustav Wikström Improve generated HTML quality: prevent W3C warning and add some accessibility labels TEC Make the behaviour of the “goto variant” of org-refile (org-speed-commands) less confusing Marco Wahl Backport an update to the OpenDocument schema Kyle Meyer Bugfixes Off by one error in texinfo menu generation Nicolas Goaziou Error in entry/conversion of non-24h times in the agenda Nicolas Goaziou Only use replace-buffer-contents with Emacs 27+ when saving src blocks, as the behaviour isn’t consistent until then Nicolas Goaziou Prevent “before first headline” error in org-clock when clocking out Nicolas Goaziou Avoid setting the global agenda name when following a timestamp link Ingo Lohmar Don’t bind <tab> in org-mode-map Nicolas Goaziou Erroneous tangling of source block with :tangle no to a file no when the tangle command is called with a single universal argument Jacopo De Simoi Footnotes

[fn1] As has been the writing of this blog post 😜

[fn2] First-class support for citations is coming to Org! With support for CSL and BibTeX, with a number of citation processors 🙌. SoonTM

[fn3] I’m rather a fan of the capitalize option because (1) technically the reference to a named object is a proper noun, and (2) this means you don’t have to worry about references not being capitalized when appearing at the start of a sentence.

]]>
Welcome https://blog.tecosaur.com/tmio/2021-04-26-Welcome.html tec@tecosaur.com (TEC) https://blog.tecosaur.com/tmio/2021-04-26-Welcome.html Tue, 27 Apr 2021 00:00:00 +0800 Introduction

Org is an absolutely marvellous project. However, a quick glance at https://orgmode.org can lead one to think "so... it's an Emacs version of Markdown? What's the big deal?". While it's easy to understand how someone might think that at first, that impression misses two crucial points: While for simple constructs (bold, italic, headlines, etc.) the syntax is very much analogous[fn1], Org scales to much more powerful forms that are a headache to replicate in Markdown Org mode was developed in Emacs, for Emacs. The integrations for Org put every other plaintext markup editing experience to shame. This is a bold statement, and I stand by it.

This blog exists because of the second point. The world does not stand still, while new Markdown editors et al. are popping up left, right, and centre --- Org has not languished. The mailing list is active, as is development, and the number of things you can do with Org is only increasing.

However, if one doesn't want to receive tens to hundreds of emails a week, it can be all too easy to miss out on exciting developments[fn3] 😢. So, to help keep you abreast of the latest in Org, I'm starting a blog ✨. Like all good things, it is written entirely in Org.

Inspired by This Week in KDE I'm thrilled to announce This Month in Org (we have slightly less going on than a Desktop Environment[fn2]). Each month I shall endeavour to present the highlights of Org development.

Who knows, perhaps you might see something you'd like to help out with or suggest improvements too. We'd love you to get in touch.

Catching up on lost time --- a year in review

I joined the Org mailing list in May last year, it's almost been a year since then and we've seen the release of Org 9.4, and its inclusion in Emacs 27.

I'm not going to do 12 months of work for just this one post, but I'd like to give you a sample of what's changed over the last year.

13.1. A new discussion tracker — updates.orgmode.org

Mid last year Bastien rolled out tracker for Upcoming changes Help requests Bugs Patches

This should help ensure nothing slips through the cracks of the ML.

screenshot-of-updates-orgmode-org.png
Figure 24: A screenshot of updates.orgmode.org

It’s very slick, and Bastien built a mailbox monitor just for it — so be sure to check it out. This should make it easier to see what needs doing. If you feel inclined to help out with Org check out the Help requests section in particular.

You can also get RSS feeds for individual sections, or get the information as JSON to work into your own projects, for example an elisp function to fetch and apply patches.

13.2. Inline display of remote images

Ever gazed forlornly at a link like [[https://github.com/larsmagne/meme/raw/master/images/Grandma-Finds-The-Internet.jpg]] wishing you could see the image?

Wish no more! Thanks to Jack Kamm you can now look upon remote images in all their glory!

Grandma-Finds-The-Internet.jpg
Figure 25: Org mode discovers the internet

To get this working, simply set org-display-remote-inline-images to 'download or 'cache.

13.3. Use org-edit-special (C-c ') with LaTeX fragments

It’s great how in Org you can jump into a LaTeX-mode minibuffer for LaTeX environments, but why leave inline LaTeX fragments out? We’ll, they’re now in.

org-edit-special-latex-fragment.png
Figure 26: Running org-edit-special on an inline LaTeX fragment

13.4. Control heading display on startup

org-num-mode is great for when you’re keeping an eye on section numbers, but it’s not fun to find yourself enabling it every time you open a file where you want it. With the new option #+startup: num by Bastien, you can set and forget at last.

org-startup-num-and-levels.png
Figure 27: Opening a buffer with the new num and show2levels startup options

Should you want to set the number of levels you see on opening a file, there’s now an option for that too. Gustav Wikström has added another new option #+startup: show<n>levels (where <n> is between 2 and 5, inclusive). NB: This is part of the upcoming 9.5 release

13.5. Set permissions of tangled files

Are you particular about your file permissions? If so you’ll likely like John Herrlin’s new source block header argument :file-mode pertinent. The easiest way to set a permission is with an octal value, like so:

  #+begin_src shell :results file :file script.sh :file-mode (identity #o755)
    echo "#!/bin/bash"
    echo "echo Hello World"
    #+end_src
  

13.6. A collection of improvements to source block header arguments

These’s been a whole set of these, so I’ll just list them off. python, improved :return Now works with sessions and the :epilogue argument (Jack Kamm) Java, new argument :cmdargs Add some command line arguments to be passed to java (Jarmo Hurri) C/C++, non-system headers with :includes values that don’t start with < will now be formatted as double-quoted #include statements (Brandon Guttersohn) Screen, new argument :screenrc For those of you who still haven’t moved to tmux (Kenneth D. Mankoff)

13.7. A seven year old bug was fixed

In 2013 it was reported that an infinite loop could be triggered in org-agenda-show-new-time. At long last, this has been fixed by Bastien.

Footnotes

[fn1] Aside from somewhat more intuitive syntax for emphasis and simple structures, Org also has the advantage of not having 40 functionally distinct specifications. There is only one Org.
NB: If you attempt to be pedantic you may say that there are multiple Orgs because, for example, GitHub uses org-ruby. However, that’s just an incomplete implementation — not a specification.

[fn2] That said, with interesting projects like the Emacs Application Framework and the Emacs X Window Manager, one could argue that Org is (sometimes) a major component of a desktop environment…

[fn3] There is ORG-NEWS, but do you really check that? Besides, it doesn’t even have pictures.

]]>