this-month-in-org/2022-02-30-orgnvim.html
Timothy 787b24749c
DRAFT update (52 files changed)
Last source commit: 7958caa
Local time: 2024-09-08 15:20:47 (UTC+0800)
2024-09-08 15:20:47 +08:00

488 lines
23 KiB
HTML

<!DOCTYPE html>
<html lang="en">
<head>
<!-- 2024-09-08 Sun 15:20 -->
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>February 2022</title>
<meta name="author" content="Kristijan Husak" />
<meta name="generator" content="org mode" />
<meta name="theme-color" content="#77aa99" />
<meta property="og:type" content="article" />
<meta property="og:title" content="February 2022" />
<meta property="og:description" content="An orgmode clone for neovim" />
<meta property="og:image" content="https://blog.tecosaur.net/tmio/org-icon.png" />
<meta property="og:image:type" content="image/png" />
<meta property="og:image:width" content="464" />
<meta property="og:image:height" content="512" />
<meta property="og:image:alt" content="Org unicorn logo" />
<meta property="og:article:author:first_name" content="Kristijan" />
<meta property="og:article:author:last_name" content="Husak" />
<meta property="og:article:published_time" content="2022-03-02T00:00:00+0800" />
<meta property="og:article:modified_time" content="2022-03-04T16:30:25+0800" />
<link rel="alternate" type="application/atom+xml" title="This Month in Org" href="rss.xml" />
<link rel="icon" type="image/ico" href="favicon.ico?v=3">
<link rel="stylesheet" type="text/css" href="org-style.css">
<link rel="stylesheet" type="text/css" href="blog.css">
<script src="org-style.js"></script>
</head>
<body>
<input type='checkbox' id='theme-switch'><div id='page'><label id='switch-label' for='theme-switch'></label><div id="preamble" class="status">
<header>
<h4><a href="https://orgmode.org" class="orgicorn"><img src="org-icon-monochrome.svg" class="invertible"></a>
<a href="https://blog.tecosaur.net/tmio/" class="title">
This Month in Org</a>
<a href="rss.xml" title="RSS Feed" type="application/rss+xml" class="rss">
<img src="rss.svg" alt="RSS icon" />
</a></h4>
</header>
</div>
<div id="content">
<header class="page-header"><div class="page-meta">2022-02-30, Kristijan Husak</div>
<h1 class="title">February 2022</h1>
<p class="subtitle" role="doc-subtitle">An orgmode clone for neovim</p>
</header><p>
Timothy (<span class='acr'>TEC</span>) here. This month we have a guest post from a different part of the
Org ecosystem, to highlight one of the most promising efforts to provide a good
experience outside Emacs.
</p>
<p>
<div class="link-preview"><a href="https://github.com/nvim-orgmode/orgmode"><img src="https://opengraph.githubassets.com/218feaebf35f549c31c66be0970d2cd248ca40f3eea9a21f7c8df9e7a0c63de4/nvim-orgmode/orgmode"/><small>github.com</small><p><b>GitHub - nvim-orgmode/orgmode: Orgmode clone written in Lua for Neovim 0.9+.</b></br>Orgmode clone written in Lua for Neovim 0.9+. Contribute to nvim-orgmode/orgmode development by creating an account on GitHub.</p></a></div>
</p>
<p>
&ldquo;But I use Emacs, I don&rsquo;t care&rdquo; you may say. In that case, I&rsquo;d like to point out
that wider spread and better Org support enriches the Org ecosystem as a whole.
It makes the format more approachable, and <i>useful</i> for other people. This is good
for everybody.
</p>
<p>
Without any further ado, here&rsquo;s the guest post kindly written by Kristijan.
Enjoy!
</p>
<hr>
<p>
Like every beginner Vim user, at some point I ran into a usual editor war post:
Vim vs Emacs. At that time, I didn&rsquo;t have an idea what &ldquo;Emacs&rdquo; was.
</p>
<p>
A simple Google search yielded something that seemed just like a very simple
editor with strange, but more familiar shortcuts. I didn&rsquo;t bother too much to
figure out what it is, because I was already pulled in fairly deep into Vim and
its philosophy.
</p>
<div id="outline-container-note-taking-neovim" class="outline-2">
<h2 id="note-taking-neovim">Note taking in (Neo)Vim<a aria-hidden="true" href="#note-taking-neovim">#</a> </h2>
<div class="outline-text-2" id="text-note-taking-neovim">
<p>
At first, I did some note taking only when really necessary, in random
plain text files. Most of the things I managed to keep in my head, since
I was younger and less busy 🙂.
</p>
<p>
Once I got into the situation where I needed to keep more notes, <a href="https://github.com/vimwiki/vimwiki">vimwiki</a> was the
natural choice.
</p>
<p>
That worked very well for a period, until the need for writing quick notes
arise. Vimwiki didn&rsquo;t have anything that would allow that. I could of course
have a mapping that opens a specific file where I can add notes, but that just
never felt right in my mind. I would keep a bunch of things in the same place,
and then later I needed to spend some time organizing them.
</p>
<p>
At that point, I wasn&rsquo;t sure how to achieve what I want. I did a brief look at
<a href="https://orgmode.org/">Emacs OrgMode</a> to see what&rsquo;s all the fuss about, but to me, it seemed just like a
different version of Markdown. You put some unordered lists as your notes, and
that&rsquo;s it. I never spent more time trying to see all the neat features. I even
tried creating some of my custom note taking tools, but I never managed to
finish them because I didn&rsquo;t have a clear idea of how to solve my problems.
</p>
</div>
</div>
<div id="outline-container-first-encounter-with" class="outline-2">
<h2 id="first-encounter-with">First encounter with Orgmode like tool: vim-dotoo<a aria-hidden="true" href="#first-encounter-with">#</a> </h2>
<div class="outline-text-2" id="text-first-encounter-with">
<p>
One weekend, I was browsing through Vim subreddit, as I usually do at least once
a day. There was a post about an &ldquo;Orgmode like task logging&rdquo; plugin called
<a href="https://github.com/dhruvasagar/vim-dotoo">vim-dotoo</a>. I opened it up, and I didn&rsquo;t see much at that point. I wasn&rsquo;t too
excited. I went through readme, and noticed that author (<a href="https://github.com/dhruvasagar">dhruvasagar</a>) put a
fairly big emphasis on the &ldquo;Agenda view&rdquo;. I had no idea what &ldquo;Agenda view&rdquo; is.
Thankfully, the author also made a <a href="https://www.youtube.com/watch?v=nsv33iOnH34">screencast</a>, which is rather long (1.5h), but
I had some time, so I went through it.
</p>
<p>
At that point, I was first met with &ldquo;Capturing&rdquo; and &ldquo;Refiling&rdquo;. <b>My mind was
blown!</b> What a simple, yet extremely powerful idea! How had that never crossed my
mind? From that point on, this plugin had my full attention.
</p>
<p>
I&rsquo;m always emphasizing that <a href="https://github.com/dhruvasagar">dhruvasagar</a> and his <a href="https://github.com/dhruvasagar/vim-dotoo">vim-dotoo</a> plugin are most
deserving for having inspired <a href="https://github.com/nvim-orgmode/orgmode">https://github.com/nvim-orgmode/orgmode</a>, and I
can&rsquo;t thank him enough for that.
</p>
</div>
</div>
<div id="outline-container-first-steps-with" class="outline-2">
<h2 id="first-steps-with">First steps with vim-dotoo and birth of orgmode.nvim<a aria-hidden="true" href="#first-steps-with">#</a> </h2>
<div class="outline-text-2" id="text-first-steps-with">
<p>
For some time, I was using <a href="https://github.com/dhruvasagar/vim-dotoo">vim-dotoo</a>. I moved all of my Vimwiki notes to it. It
was a breath of fresh air. Alongside that, I started getting more interest in
the original Emacs Orgmode. I started noticing the differences, and some of the
missing features that were now looking quite attractive. I made <a href="https://github.com/dhruvasagar/vim-dotoo/pulls?q=is%3Apr+sort%3Aupdated-desc+author%3Akristijanhusak+is%3Aclosed">few
contributions</a> to vim-dotoo. As time passed, and my notes started to grow, things
began being slow. I did some profiling, and figured out that it&rsquo;s just a usual
Vim problem, Vimscript performance. It was just too slow for certain things that
Orgmode provides, and it would hardly get any better as more things are added.
</p>
<p>
Separately from Vim and Vimscript, <a href="https://github.com/neovim/neovim">Neovim</a> was on a stable <kbd>v0.4</kbd> release, and <kbd>v0.5</kbd>
was still being developed. I was using Neovim from version 0.3, and was
carefully following the progress on it. Lua was introduced as a first class
citizen. A Bunch of new plugins arise from it. All the benchmarks showed that
Lua outperforms Vimscript in almost everything. Besides the performance, Lua is
a &ldquo;normal&rdquo; programming language, which means that support for it is much better.
</p>
<p>
At that point, I became curious: Could Lua be the path to the faster Orgmode? I
spent several days thinking about it. I wanted to give it a try. My biggest
concern was that I had absolutely zero experience writing parsers. I had never
written anything more complicated than an averagely complicated regex for
purposes of parsing. I noticed that vim-dotoo also used regex to do the parsing,
so that eased my mind a bit.
</p>
<p>
One weekend, I started working on it. It was really interesting and challenging.
I spent a lot of my free time on it. At certain points, it seemed like hacking,
since it was not a proper parsing. I tried to learn how to write a proper
parser, but it was just too time consuming and complicated. I proceeded with the
regex parsing to see how far I can go.
</p>
<p>
Besides parsing, I had a few more challenges to overcome:
</p>
</div>
<div id="outline-container-understanding-orgmode-syntax" class="outline-3">
<h3 id="understanding-orgmode-syntax">Understanding the OrgMode syntax and all the functionality<a aria-hidden="true" href="#understanding-orgmode-syntax">#</a> </h3>
<div class="outline-text-3" id="text-understanding-orgmode-syntax">
<p>
This is still the biggest challenge. I didn&rsquo;t have any idea how big and robust
OrgMode is. If I would know it at that time, I wouldn&rsquo;t even jump on this train.
It&rsquo;s really hard to grasp all of it. Considering I&rsquo;ve only used it for around 8
months, I think I made some good progress on learning it.
</p>
</div>
</div>
<div id="outline-container-remote-editing" class="outline-3">
<h3 id="remote-editing">Remote editing<a aria-hidden="true" href="#remote-editing">#</a> </h3>
<div class="outline-text-3" id="text-remote-editing">
<p>
By remote editing, I mean automatically updating content in the current
or any other file. Few examples: adding/updating properties, managing
tags, changing <span class='acr'>TODO</span> states, archiving, refiling, remote editing from
agenda view, etc.
</p>
<p>
There is no built-in way to update content in another file through the
Neovim <span class='acr'>API</span>, without actually opening the file in an editor. I solved
this by:
</p>
<ul class="org-ul">
<li>Saving as much position information as possible in the internal state,
so I can pinpoint the correct location</li>
<li>Opening a file in a <kbd>1 row x 1 col</kbd> floating window and doing quick
edits there</li>
</ul>
</div>
</div>
<div id="outline-container-working-with-dates" class="outline-3">
<h3 id="working-with-dates">Working with dates<a aria-hidden="true" href="#working-with-dates">#</a> </h3>
<div class="outline-text-3" id="text-working-with-dates">
<p>
From my experience, dates are challenging in all areas of programming,
so this is not so surprising. There are some Lua plugins for dates, but
those seemed a bit too basic for my use case, and I wanted to keep
external plugins to the minimum. I went with a custom solution that uses
Lua&rsquo;s native dates, which has certain limitations, but works out for
most of the things.
</p>
</div>
</div>
<div id="outline-container-highlighting-mostly-agenda" class="outline-3">
<h3 id="highlighting-mostly-agenda">Highlighting, mostly in Agenda view<a aria-hidden="true" href="#highlighting-mostly-agenda">#</a> </h3>
<div class="outline-text-3" id="text-highlighting-mostly-agenda">
<p>
Vim&rsquo;s syntax engine is fairly old, but still very much used, especially
in the Vim community. Implementation of tree-sitter slightly improved
this experience in Neovim, because &ldquo;Highlight matches&rdquo; are found via
tree-sitter, instead of a bunch of regexes.
</p>
<p>
This helped me out later for the Org file itself, but agenda view is
still something that&rsquo;s built as a custom view. Old Syntax highlight engine
would be really hard to add, because the content is too dynamic. I went
with the Neovim highlight <span class='acr'>API</span> that allows Highlighting things by their
exact position in the buffer. Tree-sitter implementation does something
similar in the background for Highlighting.
</p>
</div>
</div>
<div id="outline-container-keeping-configuration-simple" class="outline-3">
<h3 id="keeping-configuration-simple">Keeping configuration simple and familiar to Emacs OrgMode<a aria-hidden="true" href="#keeping-configuration-simple">#</a> </h3>
<div class="outline-text-3" id="text-keeping-configuration-simple">
<p>
Vim-dotoo configuration was mostly Vim style, through some global
variables. I wanted to have a configuration that is familiar to an Emacs
OrgMode user, by having as many options as possible named completely the
same as in Emacs.
</p>
<p>
For example, Here&rsquo;s a comparison of few options between Emacs and
Neovim:
</p>
<p>
Emacs:
</p>
<details id='keeping-configuration-simple,code--1' class='code' open><summary><span class="lang">elisp</span></summary>
<div class='gutter'>
<a href='#keeping-configuration-simple,code--1'>#</a>
<button title='Copy to clipboard' onclick='copyPreToClipbord(this)'></button></div>
<div class="org-src-container">
<pre class="src src-elisp">(<span class="org-keyword">setq</span> org-agenda-files '(<span class="org-string">"~/orgmodes"</span>))
(<span class="org-keyword">setq</span> org-agenda-skip-scheduled-if-done t)
(<span class="org-keyword">setq</span> org-agenda-span 7)
(<span class="org-keyword">setq</span> org-hide-leading-stars t)
(<span class="org-keyword">setq</span> org-capture-templates
'((<span class="org-string">"t"</span> <span class="org-string">"Todo"</span> entry (file <span class="org-string">"~/orgmodes/todos.org"</span>)
<span class="org-string">"* TODO %?"</span>)
(<span class="org-string">"j"</span> <span class="org-string">"Journal"</span> entry (file <span class="org-string">"~/orgmodes/journal.org"</span>)
<span class="org-string">"* %?\nEntered on %U\n %a"</span>)))
</pre>
</div>
</details>
<p>
Neovim:
</p>
<details id='keeping-configuration-simple,code--2' class='code' open><summary><span class="lang">Lua</span></summary>
<div class='gutter'>
<a href='#keeping-configuration-simple,code--2'>#</a>
<button title='Copy to clipboard' onclick='copyPreToClipbord(this)'></button></div>
<div class="org-src-container">
<pre class="src src-lua">require('orgmode').setup({
org_agenda_files = { '~/orgmodes' },
org_agenda_skip_scheduled_if_done = true,
org_agenda_span = 7,
org_hide_leading_stars = true
org_capture_templates = {
t = {
description = 'Todo',
target = '~/orgmodes/todos.org',
template = '* TODO %?',
},
j = {
description = 'Journal',
target = '~/orgmodes/journal.org',
template = '* %?\nEntered on %U\n %a',
}
}
})
</pre>
</div>
</details>
<p>
One of the most noticeable differences is between the usage of hyphens
(<kbd>-</kbd>) and underscores (<kbd>_</kbd>). I did that only for the sake of simplicity,
because hyphens is not a valid character in variable names in Lua, so
all of the options would need to be wrapped as a string (for example:
<kbd>['org-agenda-files']</kbd>).
</p>
</div>
</div>
</div>
<div id="outline-container-first-release-orgmodenvim" class="outline-2">
<h2 id="first-release-orgmodenvim">First release of orgmode.nvim and introduction of tree-sitter parser<a aria-hidden="true" href="#first-release-orgmodenvim">#</a> </h2>
<div class="outline-text-2" id="text-first-release-orgmodenvim">
<p>
After ~1.5 months I <a href="https://www.reddit.com/r/neovim/comments/o8zp0k/orgmodenvim_orgmode_clone_written_in_lua_for/">published the initial version</a>. The focus was on Agenda and
capturing (<span class='acr'>GTD</span>), since those are the things I mostly used. It got some traction,
and people started testing it and reporting bugs.
</p>
<p>
One of the common questions was: <i>&ldquo;Any plans to introduce tree-sitter parser?&rdquo;</i>.
</p>
<p>
I knew about <a href="https://github.com/tree-sitter/tree-sitter">tree-sitter</a> and used it in my day-to-day job for a few programming
languages, but I had absolutely no idea how it worked, and especially how to
write a tree-sitter parser. I put it aside, and continued working on what I
had.
</p>
<p>
One day, Emilia (<a href="https://github.com/milisims">milisims</a>) contacted me via email to ask me if I would be
willing to try the tree-sitter parser she&rsquo;s been working on for some time. I
gladly accepted. She gave me access to the repository, and I started tinkering
with it in a separate branch. No one was aware at that point that tree-sitter
support would happen some time soon.
</p>
<p>
After some time, I set up a &ldquo;beta&rdquo; branch called &ldquo;tree-sitter&rdquo; and <a href="https://www.reddit.com/r/neovim/comments/ph2xqc/orgmodenvim_treesitter_support/">announced it
for testing</a>. Once the reported bugs slowed to a trickle, I merged it into the
&ldquo;master&rdquo; branch.
</p>
<p>
I believe that tree-sitter grammar for Org could help out other editors to
implement their version of Orgmode plugin, but I don&rsquo;t think it would ever be
helpful for Emacs. Emacs parser is the one and only that has it all implemented.
Also, as much as tree-sitter is powerful, its main purpose is to parse
programming languages, which mostly has &ldquo;static&rdquo; patterns to match. Orgmode is
by its nature dynamic, which causes a variety of issues for a parser that&rsquo;s not
meant for that kind of usage.
</p>
</div>
</div>
<div id="outline-container-limitations" class="outline-2">
<h2 id="limitations">Limitations<a aria-hidden="true" href="#limitations">#</a> </h2>
<div class="outline-text-2" id="text-limitations">
<p>
(Neo)Vim is a great editor, but it still cannot compare to Emacs in certain
things. Manipulating the &ldquo;View&rdquo; part of the editor is tricky or impossible for
certain things.
</p>
<p>
I even <a href="https://github.com/nvim-orgmode/orgmode/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc+label%3Aneovim-dependency">made a label</a> for reported issues where Neovim support for certain things
is a blocker. I&rsquo;m hoping that at least some of these will be available in future
Neovim releases.
</p>
</div>
</div>
<div id="outline-container-features" class="outline-2">
<h2 id="features">Features<a aria-hidden="true" href="#features">#</a> </h2>
<div class="outline-text-2" id="text-features">
<p>
I will not go into too many details about the available features, since those
can be viewed in <a href="https://github.com/nvim-orgmode/orgmode#features-detailed-breakdown">repository readme</a>, but I want to mention one feature that does
not exist as a built/-in feature in the Emacs Orgmode: <a href="https://github.com/nvim-orgmode/orgmode/blob/master/DOCS.md#notifications-experimental">Notifications</a>.
</p>
<p>
This allows getting a &ldquo;desktop notification&rdquo; for tasks that are within
the specified threshold for schedule/deadline time. It requires some
configuration to set up a cron job, but it&rsquo;s been working great for me
for several months now.
</p>
</div>
</div>
<div id="outline-container-plans" class="outline-2">
<h2 id="plans">Plans<a aria-hidden="true" href="#plans">#</a> </h2>
<div class="outline-text-2" id="text-plans">
<p>
The current state of the project is very usable for me. I&rsquo;m not lacking any of
the major features, mostly because I&rsquo;m not used to using them. Nevertheless,
there are plans to add more things, and I&rsquo;m getting a lot of help from the
community. I want to specifically mention <a href="https://github.com/levouh">levouh</a> and <a href="https://github.com/lukas-reineke">lukas-reineke</a>, since they
added a lot of value to the project, and I want to thank them and everyone else
who contributed. Their help is much appreciated.
</p>
<p>
There are few high priority tasks that I&rsquo;m hoping to flush out first:
</p>
<ul class="org-ul">
<li>Implementing <a href="https://github.com/milisims/tree-sitter-org/issues/13">v1.0.0</a> release of the tree-sitter parser. This should allow for
faster and less error-prone parsing.</li>
<li><a href="https://github.com/nvim-orgmode/orgmode/issues/26">Infrastructure for plugin developers</a>, to allow other people to build plugins
on top of nvim-orgmode.</li>
</ul>
<p>
And a long term goal for these:
</p>
<ul class="org-ul">
<li>Tables support (and at least basic formulas)</li>
<li><a href="https://github.com/nvim-orgmode/orgmode/issues/190">Org Babel like code block evaluation</a> (and hopefully basic support for literate
programming)</li>
<li><a href="https://github.com/nvim-orgmode/orgmode/issues/195">Diary format dates</a></li>
<li><a href="https://github.com/nvim-orgmode/orgmode/issues/135">Custom agenda commands</a></li>
<li>More clocking features (reports)</li>
<li>File specific configuration via directives (<a href="https://github.com/nvim-orgmode/orgmode/issues/185">todo keywords</a>, properties, etc.)</li>
</ul>
</div>
</div>
<div id="outline-container-closing-thoughts" class="outline-2">
<h2 id="closing-thoughts">Closing thoughts<a aria-hidden="true" href="#closing-thoughts">#</a> </h2>
<div class="outline-text-2" id="text-closing-thoughts">
<p>
When I started working on <a href="https://github.com/nvim-orgmode/orgmode">nvim-orgmode</a>, I didn&rsquo;t have a clue what I&rsquo;m jumping
into. Every day I learn about more and more Orgmode features that I wasn&rsquo;t even
aware existed.
</p>
<p>
I&rsquo;m certain that this project will never manage to clone the Orgmode
functionality completely, but I&rsquo;m hoping it will get close enough so everyone
from Neovim community and Emacsers trying out Neovim will be able to use it for
their needs.
</p>
<p>
Having experienced Orgmode users <a href="https://github.com/nvim-orgmode/orgmode/issues/159">testing</a> it is a huge help, so if anyone is
willing to give it a try, feel free to open up an issue and write your thoughts
there. Thanks!
</p>
</div>
</div>
</div>
<div id="postamble" class="status">
<footer>
<p xmlns:dct="http://purl.org/dc/terms/" xmlns:vcard="http://www.w3.org/2001/vcard-rdf/3.0#">
<a rel="license" class="external" rel="nofollow" href="http://creativecommons.org/publicdomain/zero/1.0/">
<img src="cc0.svg" alt="CC0" class="invertible" style="position:relative;bottom:-0.2em"/>
</a>
To the extent possible under law,
<a rel="dct:publisher"
href="https://blog.tecosaur.net/">
<span property="dct:title">TEC</span></a>
has waived all copyright and related or neighboring rights to
<span property="dct:title" style="font-style:italic">This Month in Org</span>.
</p>
</footer>
<script src="theme-switcher.js"></script>
</div>
</div>
</body>
</html>