this-month-in-org/2022-05-31-folding.html
Timothy 787b24749c
DRAFT update (52 files changed)
Last source commit: 7958caa
Local time: 2024-09-08 15:20:47 (UTC+0800)
2024-09-08 15:20:47 +08:00

473 lines
21 KiB
HTML

<!DOCTYPE html>
<html lang="en">
<head>
<!-- 2024-09-08 Sun 15:20 -->
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>May 2022</title>
<meta name="author" content="TEC" />
<meta name="generator" content="org mode" />
<meta name="theme-color" content="#77aa99" />
<meta property="og:type" content="article" />
<meta property="og:title" content="May 2022" />
<meta property="og:description" content="Folding more improvement into Org" />
<meta property="og:image" content="https://blog.tecosaur.net/tmio/org-icon.png" />
<meta property="og:image:type" content="image/png" />
<meta property="og:image:width" content="464" />
<meta property="og:image:height" content="512" />
<meta property="og:image:alt" content="Org unicorn logo" />
<meta property="og:article:author:first_name" content="TEC" />
<meta property="og:article:published_time" content="2022-05-31T00:00:00+0800" />
<meta property="og:article:modified_time" content="2022-06-02T00:52:03+0800" />
<link rel="alternate" type="application/atom+xml" title="This Month in Org" href="rss.xml" />
<link rel="icon" type="image/ico" href="favicon.ico?v=3">
<link rel="stylesheet" type="text/css" href="org-style.css">
<link rel="stylesheet" type="text/css" href="blog.css">
<script src="org-style.js"></script>
<script>
window.MathJax = {
loader: {
load: ['[tex]/mathtools'],
},
tex: {
ams: {
multlineWidth: '85%'
},
tags: 'ams',
tagSide: 'right',
tagIndent: '.8em',
packages: {'[+]': ['mathtools']},
macros: {
RR: ['\\ifstrempty{#1}{\\mathbb{R}}{\\mathbb{R}^{#1}}', 1, ''],
NN: ['\\ifstrempty{#1}{\\mathbb{N}}{\\mathbb{N}^{#1}}', 1, ''],
ZZ: ['\\ifstrempty{#1}{\\mathbb{Z}}{\\mathbb{Z}^{#1}}', 1, ''],
QQ: ['\\ifstrempty{#1}{\\mathbb{Q}}{\\mathbb{Q}^{#1}}', 1, ''],
CC: ['\\ifstrempty{#1}{\\mathbb{C}}{\\mathbb{C}^{#1}}', 1, ''],
EE: '\\mathbb{E}',
Lap: '\\operatorname{\\mathcal{L}}',
Var: '\\operatorname{Var}',
Cor: '\\operatorname{Cor}',
E: '\\operatorname{E}',
},
mathtools: {
pairedDelimiters: {
abs: ['\\lvert', '\\rvert'],
norm: ['\\lVert', '\\rVert'],
ceil: ['\\lceil', '\\rceil'],
floor: ['\\lfloor', '\\rfloor'],
round: ['\\lfloor', '\\rceil'],
}
}
},
chtml: {
scale: 1.0,
displayAlign: 'center',
displayIndent: '0em'
},
svg: {
scale: 1.0,
displayAlign: 'center',
displayIndent: '0em'
},
output: {
font: 'mathjax-modern',
displayOverflow: 'overflow'
}
};
</script>
<script
id="MathJax-script"
async
src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-svg.js">
</script>
</head>
<body>
<input type='checkbox' id='theme-switch'><div id='page'><label id='switch-label' for='theme-switch'></label><div id="preamble" class="status">
<header>
<h4><a href="https://orgmode.org" class="orgicorn"><img src="org-icon-monochrome.svg" class="invertible"></a>
<a href="https://blog.tecosaur.net/tmio/" class="title">
This Month in Org</a>
<a href="rss.xml" title="RSS Feed" type="application/rss+xml" class="rss">
<img src="rss.svg" alt="RSS icon" />
</a></h4>
</header>
</div>
<div id="content">
<header class="page-header"><div class="page-meta">2022-05-31, <span class='acr'>TEC</span></div>
<h1 class="title">May 2022</h1>
<p class="subtitle" role="doc-subtitle">Folding more improvement into Org</p>
</header><p>
Finding time as of late has been more difficult than I anticipated, and on top
of that, just as I was thinking of writing last month&rsquo;s post, I got distracted
by an exciting patchset that has been in the works for over a year finally
getting sorted out and landing. So, I hope that some of the fun developments in
this post will make up the absense of the last one 🙂.
</p>
<p>
Since it&rsquo;s been longer than I thought since the last standard post, we&rsquo;ve got a
fair few commits to catch up on &#x2014; about 200. Most of these are miscellaneous
minor improvements and bugfixes, but a few notable changes have arrived too.
</p>
<div id="outline-container-folding" class="outline-2">
<h2 id="folding">Folding<a aria-hidden="true" href="#folding">#</a> </h2>
<div class="outline-text-2" id="text-folding">
<p>
The fabulous new folding engine (<kbd>org-fold-core</kbd>) should noticeably improve Org&rsquo;s
performance with large files. It contains a number of key optimisations to
improve speed, namely:
</p>
<ul class="org-ul">
<li>Deferring fontification of folded regions</li>
<li>Using text properties (\(\mathcal{O}(n \log n)\)) instead of overlays (\(\mathcal{O}(n^2)\)) for folded regions</li>
<li>A collection of aggressive optimisations available under <code>org-fold-core--optimise-for-huge-buffers</code></li>
<li>Convert text properties to overlays for <kbd>isearch</kbd> (which currently only supports overlays)</li>
</ul>
<p>
How noticeable is the overall performance impact? Well, I poked Ihor and he was
kind enough to whip up some benchmarks.
</p>
<figure id="org55cd8f8">
<img src="figures/org-fold-perf-shifttab-contents.svg" alt="org-fold-perf-shifttab-contents.svg" class="invertible">
<figcaption><span class="figure-number">Figure 1: </span>The scaling of <code>org-shifttab</code> showing file contents, as file size increases, with and without org-fold.</figcaption>
</figure>
<figure id="org42db708">
<img src="figures/org-fold-perf-shifttab-showall.svg" alt="org-fold-perf-shifttab-showall.svg" class="invertible">
<figcaption><span class="figure-number">Figure 2: </span>The scaling of <code>org-shifttab</code> showing the entire file, as file size increases, with and without org-fold.</figcaption>
</figure>
<p>
Well this looks very promising<sup><a id="fnr.1" class="footref" href="#fn.1" role="doc-backlink">1</a></sup>! Let&rsquo;s see how much of an improvement this is overall.
</p>
<div id='folding,table--1' class='table'>
<div class='gutter'><a href='#folding,table--1'>#</a></div>
<div class='tabular'>
<table>
<caption class="t-above"><span class="table-number">Table 1:</span> Time to run <kbd>org-shifttab</kbd> twice, cycling through all three display modes (in seconds).</caption>
<colgroup>
<col class="org-right">
<col class="org-right">
<col class="org-right">
<col class="org-right">
<col class="org-right">
</colgroup>
<thead>
<tr>
<th scope="col" class="org-right">File size (Mb)</th>
<th scope="col" class="org-right">Headings (thousands)</th>
<th scope="col" class="org-right">Bugfix (no org-fold)</th>
<th scope="col" class="org-right">Main (with org-fold)</th>
<th scope="col" class="org-right">Improvement</th>
</tr>
</thead>
<tbody>
<tr>
<td class="org-right">18</td>
<td class="org-right">36</td>
<td class="org-right">115.31</td>
<td class="org-right">0.89</td>
<td class="org-right">99%</td>
</tr>
<tr>
<td class="org-right">8.8</td>
<td class="org-right">24</td>
<td class="org-right">19.03</td>
<td class="org-right">0.48</td>
<td class="org-right">97%</td>
</tr>
<tr>
<td class="org-right">4.4</td>
<td class="org-right">5</td>
<td class="org-right">3.79</td>
<td class="org-right">0.13</td>
<td class="org-right">97%</td>
</tr>
<tr>
<td class="org-right">2.2</td>
<td class="org-right">2</td>
<td class="org-right">1.29</td>
<td class="org-right">0.08</td>
<td class="org-right">94%</td>
</tr>
<tr>
<td class="org-right">1.1</td>
<td class="org-right">1</td>
<td class="org-right">0.50</td>
<td class="org-right">0.045</td>
<td class="org-right">91%</td>
</tr>
</tbody>
</table>
</div></div>
<p>
To be clear, even the smallest file in this data &#x2014; a 1.1 Mb Org file with around
a thousand headings, is fairly large. So, it&rsquo;s unlikely you&rsquo;ll notice much of a
difference with small&#x2013;medium files, but if you a few large+ files this should
be a <i>fantastic</i> improvement. Once again, thanks Ihor!
</p>
<div class="warning" id="orgb759830">
<p>
The change to text properties instead of overlays breaks a number of third party
packages like <kbd>evil-search</kbd> and <kbd>consult</kbd>&rsquo;s <code>consult-line</code>.
If you are involved in any packages affected by this, you&rsquo;ll either want to
consider supporting invisible text, or look at <code>isearch-filter-predicate</code> and
<code>isearch-mode-end-hook</code>, which <kbd>org-fold</kbd> now uses.
If you&rsquo;re an end-user, perhaps politely make an issue on the repo for a project
<i>if no issue currently exists</i>, and either:
</p>
<ul class="org-ul">
<li>Stay off Org&rsquo;s bleeding edge till the package ecosystem has adapted to this change</li>
<li>Help the packages you use adapt to this change</li>
<li>Set <code>org-fold-core-style</code> to <code>overlays</code> to restore the old behaviour</li>
</ul>
</div>
</div>
</div>
<div id="outline-container--engraved-source" class="outline-2">
<h2 id="-engraved-source"><i>Engraved</i> source code blocks in LaTeX<a aria-hidden="true" href="#-engraved-source">#</a> </h2>
<div class="outline-text-2" id="text--engraved-source">
<p>
All too often exporting code to LaTeX has been a disappointment, with lovely
syntax highlighting from Emacs major modes replaced with a markedly inferior
attempt by pygments (setting <code>org-latex-listings</code> to <code>minted</code>) in a colour scheme I
don&rsquo;t really like.
</p>
<p>
A bit over a year ago, a project called <a href="https://github.com/tecosaur/engrave-faces">engrave-faces</a> started with the aim of
making Emacs&rsquo; font-lock more exportable, like a generalised <kbd>htmlize.el</kbd>. This has
recently been used to provide a new option for inline and block source code
exports in LaTeX.
</p>
<figure id="org18ea2a9">
<img src="figures/engraved-faces-sample.png" alt="engraved-faces-sample.png" class="invertible">
<figcaption><span class="figure-number">Figure 3: </span>A screenshot of an Org code block, exported to a <span class='acr'>PDF</span>, using <kbd>engrave-faces</kbd> and the <kbd>doom-one-light</kbd> theme.</figcaption>
</figure>
<p>
To use this, simply install the package and set <code>org-latex-src-block-backend</code> (a
rename of <code>org-latex-listings</code> to better reflect its usage) to <code>engraved</code>.
</p>
<p>
While this is sufficient to get started, this new backend also allows for some
new options. The theme used for <i>engraving</i> a source block can be set globally
with the new variable <code>org-latex-engraved-theme</code>, or per-file with the
<kbd>#+latex_engraved_theme</kbd> keyword. It takes either the name of a theme, or the
symbol <kbd>t</kbd> as a stand-in for the current theme.
</p>
<p>
The theme can also be set on a per-block level using the LaTeX attribute
<kbd>:engraved-theme</kbd>.
</p>
<figure id="orgf1d2aac">
<img src="figures/engraved-faces-multitheme.png" alt="engraved-faces-multitheme.png">
<figcaption><span class="figure-number">Figure 4: </span>Seven code blocks exported to LaTeX, each with a different engrave-faces theme.</figcaption>
</figure>
<p>
Here&rsquo;s what using these new capabilities looks like in practice.
</p>
<details id='-engraved-source,code--1' class='code' open><summary><span class="lang">Org mode</span></summary>
<div class='gutter'>
<a href='#-engraved-source,code--1'>#</a>
<button title='Copy to clipboard' onclick='copyPreToClipbord(this)'></button></div>
<div class="org-src-container">
<pre class="src src-org"><span class="org-org-document-info-keyword">#+title:</span> <span class="org-org-document-title">Engraving source blocks</span>
<span class="org-org-meta-line">#+latex_engraved_theme: modus-operandi</span>
<span class="org-org-block-begin-line">#+begin_src emacs-lisp</span>
<span class="org-org-block">(message </span><span class="org-org-block"><span class="org-string">"look ma, some %s"</span></span><span class="org-org-block"> 'code)</span>
<span class="org-org-block-end-line">#+end_src</span>
<span class="org-org-meta-line">#+attr_latex: :engraved-theme modus-viviandi</span>
<span class="org-org-block-begin-line">#+begin_src shell</span>
<span class="org-org-block"><span class="org-builtin">echo</span></span><span class="org-org-block"> </span><span class="org-org-block"><span class="org-string">"This is shell code"</span></span>
<span class="org-org-block-end-line">#+end_src</span>
</pre>
</div>
</details>
<p>
This may well be the best syntax-highlighting solution available for <span class='acr'>PDF</span><small>s</small>/LaTeX
currently available, but I am a tad biased 😛.
</p>
</div>
</div>
<div id="outline-container-texinfo-export-improvements" class="outline-2">
<h2 id="texinfo-export-improvements">TexInfo export improvements<a aria-hidden="true" href="#texinfo-export-improvements">#</a> </h2>
<div class="outline-text-2" id="text-texinfo-export-improvements">
<p>
Jonas Bernoulli has been using a custom TexInfo backend for Magit&rsquo;s
documentation for a while now, and over the past few months he&rsquo;s worked the
features he was missing into Org&rsquo;s built-in TexInfo exporter.
</p>
<p>
Upstreaming like this always takes a fair bit of effort, so thank you Jonas for
going through with this!
</p>
</div>
</div>
<div id="outline-container-toggle-noweb-prefix" class="outline-2">
<h2 id="toggle-noweb-prefix">Toggle noweb prefix handling<a aria-hidden="true" href="#toggle-noweb-prefix">#</a> </h2>
<div class="outline-text-2" id="text-toggle-noweb-prefix">
<p>
Previously, whenever a noweb reference appeared on a non-empty line, a
multi-line replacement would duplicate the content before the noweb reference.
</p>
<p>
Clearly, this is not always desirable, and this behaviour can now be turned of
by setting the new header argument <kbd>:noweb-prefix no</kbd>.
</p>
<details id='toggle-noweb-prefix,code--1' class='code' open><summary><span class="lang">Org mode</span></summary>
<div class='gutter'>
<a href='#toggle-noweb-prefix,code--1'>#</a>
<button title='Copy to clipboard' onclick='copyPreToClipbord(this)'></button></div>
<div class="org-src-container">
<pre class="src src-org"><span class="org-org-block-begin-line">#+begin_src emacs-lisp :noweb yes :noweb-prefix no</span>
<span class="org-org-block">(</span><span class="org-org-block"><span class="org-keyword">setq</span></span><span class="org-org-block"> example-data </span><span class="org-org-block"><span class="org-string">"&lt;&lt;example&gt;&gt;"</span></span><span class="org-org-block">)</span>
<span class="org-org-block-end-line">#+end_src</span>
Will now expand to
<span class="org-org-block-begin-line">#+begin_src emacs-lisp</span>
<span class="org-org-block">(</span><span class="org-org-block"><span class="org-keyword">setq</span></span><span class="org-org-block"> example-data </span><span class="org-org-block"><span class="org-string">"some</span></span>
<span class="org-org-block"><span class="org-string">multi-line</span></span>
<span class="org-org-block"><span class="org-string">content"</span></span><span class="org-org-block">)</span>
<span class="org-org-block-end-line">#+end_src</span>
Instead of
<span class="org-org-block-begin-line">#+begin_src emacs-lisp</span>
<span class="org-org-block">(</span><span class="org-org-block"><span class="org-keyword">setq</span></span><span class="org-org-block"> example-data </span><span class="org-org-block"><span class="org-string">"some</span></span>
<span class="org-org-block"><span class="org-string">(setq example-data "</span></span><span class="org-org-block">multiline</span>
<span class="org-org-block">(</span><span class="org-org-block"><span class="org-keyword">setq</span></span><span class="org-org-block"> example-data </span><span class="org-org-block"><span class="org-string">"content"</span></span><span class="org-org-block">)</span>
<span class="org-org-block-end-line">#+end_src</span>
</pre>
</div>
</details>
</div>
</div>
<div id="outline-container-package-highlight-org" class="outline-2">
<h2 id="package-highlight-org">Package highlight: org-modern<a aria-hidden="true" href="#package-highlight-org">#</a> </h2>
<div class="outline-text-2" id="text-package-highlight-org">
<p>
I think we&rsquo;ve all <a href="https://github.com/integral-dw/org-superstar-mode">seen</a> <a href="https://github.com/sabof/org-bullets">plenty</a> of <kbd>org-mode</kbd> <a href="https://github.com/Fuco1/org-pretty-table">prettification</a> <a href="https://github.com/harrybournis/org-fancy-priorities">packages</a> <a href="https://gitlab.com/marcowahl/org-pretty-tags">before</a>, so
what makes Minad&rsquo;s <a href="https://github.com/minad/org-modern">org-modern</a> special? It&rsquo;s actually doing something similar to
Ihor&rsquo;s org-fold improvements, switching out slower overlay-based approaches for
text properties. I can confirm that switching out <kbd>org-superstar-mode</kbd> for
<kbd>org-modern</kbd> has made a substantial improvement in my experience, halving the
first-load time of my <kbd>config.org</kbd> to around 20 seconds. If you&rsquo;re a fan of Org
prettification and haven&rsquo;t taken a look at this package, I highly recommend
giving it a shot.
</p>
<figure id="orgf02824e">
<img src="figures/org-modern-readme-demo.gif" alt="org-modern-readme-demo.gif" class="invertible">
<figcaption><span class="figure-number">Figure 5: </span>A demonstration of org-modern taken from the project <span class='acr'>README</span>.</figcaption>
</figure>
</div>
</div>
<div id="outline-container-other-improvements" class="outline-2">
<h2 id="other-improvements">Other improvements<a aria-hidden="true" href="#other-improvements">#</a> </h2>
<div class="outline-text-2" id="text-other-improvements">
<ul class="org-ul">
<li>Clean up some magic numbers in <kbd>org-attach</kbd> <span class="underline">Marco Wahl</span></li>
<li>Allow <i>any</i> command form in <code>org-attach-commands</code> (including keyboard macros)
<span class="underline">Marco Wahl</span></li>
<li>Allow <kbd>dest</kbd> in <code>org-list-send-item</code> to be a buffer position <span class="underline">Sacha Chua</span></li>
<li>Improve <span class='acr'>CSL</span>-<span class='acr'>JSON</span> date handling in <kbd>oc-basic</kbd> <span class="underline">David Lukes</span></li>
<li>Add <kbd>TOML</kbd> and <kbd>desktop</kbd> language aliases <span class="underline"><span class='acr'>TEC</span></span></li>
<li>Speed up cached bibliography retrieval in <kbd>oc-basic</kbd> <span class="underline">Ihor Radchenko</span></li>
<li>Allow setting PlantUML jar arguments <span class="underline">Ihor Radchenko</span></li>
<li>Allow for customisation of property separators with <code>org-property-separators</code>
<span class="underline">Tyler Grinn</span></li>
<li>New <kbd>ox-latex</kbd> maintainer, Daniel Fleischer</li>
<li>More unit tests <span class="underline">Kyle Keyer, Nick Dokos</span></li>
<li>Documentation improvements <span class="underline">Kyle Meyer, Juan Manuel Macias, Bastien, Karl
Fogel, Cody Harris</span></li>
</ul>
</div>
</div>
<div id="outline-container-bugfixes" class="outline-2">
<h2 id="bugfixes">Bugfixes<a aria-hidden="true" href="#bugfixes">#</a> </h2>
<div class="outline-text-2" id="text-bugfixes">
<ul class="org-ul">
<li>An Emacs &lt;28 bug in <kbd>org-persist</kbd> <span class="underline">Ihor Radchenko</span></li>
<li>Author extraction in <kbd>oc-basic</kbd> <span class="underline">Nicolas Goaziou</span></li>
<li>Fix behaviour of <code>org-copy-visible</code> with adjacent tex and
<code>buffer-invisibility-spec</code> <span class="underline">Kyle Meyer</span></li>
<li>Parsing of inline footnotes with parentheses <span class="underline">Nicolas Goaziou</span></li>
<li>Honor <code>default-directory</code> in <kbd>ob-gnuplot</kbd> <span class="underline">Ihor Radchenko</span></li>
<li>Heading fontification bug <span class="underline">Anders Johansson</span></li>
<li>Template expansion where one key is a substring of another <span class="underline">Andrew Arensburger</span></li>
</ul>
</div>
</div>
<div id="footnotes">
<h2 class="footnotes">Footnotes: </h2>
<div id="text-footnotes">
<div class="footdef"><sup><a id="fn.1" class="footnum" href="#fnr.1" role="doc-backlink">1</a></sup> <div class="footpara" role="doc-footnote"><p class="footpara">Note the difference in scale, org-fold makes
the most difference in the graph where the times are an order of magnitude
more.</p></div></div>
</div>
</div></div>
<div id="postamble" class="status">
<footer>
<p xmlns:dct="http://purl.org/dc/terms/" xmlns:vcard="http://www.w3.org/2001/vcard-rdf/3.0#">
<a rel="license" class="external" rel="nofollow" href="http://creativecommons.org/publicdomain/zero/1.0/">
<img src="cc0.svg" alt="CC0" class="invertible" style="position:relative;bottom:-0.2em"/>
</a>
To the extent possible under law,
<a rel="dct:publisher"
href="https://blog.tecosaur.net/">
<span property="dct:title">TEC</span></a>
has waived all copyright and related or neighboring rights to
<span property="dct:title" style="font-style:italic">This Month in Org</span>.
</p>
</footer>
<script src="theme-switcher.js"></script>
</div>
</div>
</body>
</html>