413 lines
17 KiB
Plaintext
413 lines
17 KiB
Plaintext
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||
FEBRUARY 2022
|
||
An orgmode clone for neovim
|
||
|
||
Kristijan Husak
|
||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||
|
||
|
||
2022-02-30
|
||
|
||
|
||
Timothy (TEC) here. This month we have a guest post from a different
|
||
part of the Org ecosystem, to highlight one of the most promising
|
||
efforts to provide a good experience outside Emacs.
|
||
|
||
//github.com/nvim-orgmode/orgmode
|
||
|
||
“But I use Emacs, I don’t care” you may say. In that case, I’d like to
|
||
point out that wider spread and better Org support enriches the Org
|
||
ecosystem as a whole. It makes the format more approachable, and
|
||
/useful/ for other people. This is good for everybody.
|
||
|
||
Without any further ado, here’s the guest post kindly written by
|
||
Kristijan. Enjoy!
|
||
|
||
――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
|
||
|
||
Like every beginner Vim user, at some point I ran into a usual editor
|
||
war post: Vim vs Emacs. At that time, I didn’t have an idea what “Emacs”
|
||
was.
|
||
|
||
A simple Google search yielded something that seemed just like a very
|
||
simple editor with strange, but more familiar shortcuts. I didn’t bother
|
||
too much to figure out what it is, because I was already pulled in
|
||
fairly deep into Vim and its philosophy.
|
||
|
||
|
||
Note taking in (Neo)Vim
|
||
═══════════════════════
|
||
|
||
At first, I did some note taking only when really necessary, in random
|
||
plain text files. Most of the things I managed to keep in my head,
|
||
since I was younger and less busy 🙂.
|
||
|
||
Once I got into the situation where I needed to keep more notes,
|
||
[vimwiki] was the natural choice.
|
||
|
||
That worked very well for a period, until the need for writing quick
|
||
notes arise. Vimwiki didn’t have anything that would allow that. I
|
||
could of course have a mapping that opens a specific file where I can
|
||
add notes, but that just never felt right in my mind. I would keep a
|
||
bunch of things in the same place, and then later I needed to spend
|
||
some time organizing them.
|
||
|
||
At that point, I wasn’t sure how to achieve what I want. I did a brief
|
||
look at [Emacs OrgMode] to see what’s all the fuss about, but to me,
|
||
it seemed just like a different version of Markdown. You put some
|
||
unordered lists as your notes, and that’s it. I never spent more time
|
||
trying to see all the neat features. I even tried creating some of my
|
||
custom note taking tools, but I never managed to finish them because I
|
||
didn’t have a clear idea of how to solve my problems.
|
||
|
||
|
||
[vimwiki] <https://github.com/vimwiki/vimwiki>
|
||
|
||
[Emacs OrgMode] <https://orgmode.org/>
|
||
|
||
|
||
First encounter with Orgmode like tool: vim-dotoo
|
||
═════════════════════════════════════════════════
|
||
|
||
One weekend, I was browsing through Vim subreddit, as I usually do at
|
||
least once a day. There was a post about an “Orgmode like task
|
||
logging” plugin called [vim-dotoo]. I opened it up, and I didn’t see
|
||
much at that point. I wasn’t too excited. I went through readme, and
|
||
noticed that author ([dhruvasagar]) put a fairly big emphasis on the
|
||
“Agenda view”. I had no idea what “Agenda view” is. Thankfully, the
|
||
author also made a [screencast], which is rather long (1.5h), but I
|
||
had some time, so I went through it.
|
||
|
||
At that point, I was first met with “Capturing” and “Refiling”. *My
|
||
mind was blown!* What a simple, yet extremely powerful idea! How had
|
||
that never crossed my mind? From that point on, this plugin had my
|
||
full attention.
|
||
|
||
I’m always emphasizing that [dhruvasagar] and his [vim-dotoo] plugin
|
||
are most deserving for having inspired
|
||
<https://github.com/nvim-orgmode/orgmode>, and I can’t thank him
|
||
enough for that.
|
||
|
||
|
||
[vim-dotoo] <https://github.com/dhruvasagar/vim-dotoo>
|
||
|
||
[dhruvasagar] <https://github.com/dhruvasagar>
|
||
|
||
[screencast] <https://www.youtube.com/watch?v=nsv33iOnH34>
|
||
|
||
|
||
First steps with vim-dotoo and birth of orgmode.nvim
|
||
════════════════════════════════════════════════════
|
||
|
||
For some time, I was using [vim-dotoo]. I moved all of my Vimwiki
|
||
notes to it. It was a breath of fresh air. Alongside that, I started
|
||
getting more interest in the original Emacs Orgmode. I started
|
||
noticing the differences, and some of the missing features that were
|
||
now looking quite attractive. I made [few contributions] to vim-dotoo.
|
||
As time passed, and my notes started to grow, things began being slow.
|
||
I did some profiling, and figured out that it’s just a usual Vim
|
||
problem, Vimscript performance. It was just too slow for certain
|
||
things that Orgmode provides, and it would hardly get any better as
|
||
more things are added.
|
||
|
||
Separately from Vim and Vimscript, [Neovim] was on a stable `v0.4'
|
||
release, and `v0.5' was still being developed. I was using Neovim from
|
||
version 0.3, and was carefully following the progress on it. Lua was
|
||
introduced as a first class citizen. A Bunch of new plugins arise from
|
||
it. All the benchmarks showed that Lua outperforms Vimscript in almost
|
||
everything. Besides the performance, Lua is a “normal” programming
|
||
language, which means that support for it is much better.
|
||
|
||
At that point, I became curious: Could Lua be the path to the faster
|
||
Orgmode? I spent several days thinking about it. I wanted to give it a
|
||
try. My biggest concern was that I had absolutely zero experience
|
||
writing parsers. I had never written anything more complicated than an
|
||
averagely complicated regex for purposes of parsing. I noticed that
|
||
vim-dotoo also used regex to do the parsing, so that eased my mind a
|
||
bit.
|
||
|
||
One weekend, I started working on it. It was really interesting and
|
||
challenging. I spent a lot of my free time on it. At certain points,
|
||
it seemed like hacking, since it was not a proper parsing. I tried to
|
||
learn how to write a proper parser, but it was just too time consuming
|
||
and complicated. I proceeded with the regex parsing to see how far I
|
||
can go.
|
||
|
||
Besides parsing, I had a few more challenges to overcome:
|
||
|
||
|
||
[vim-dotoo] <https://github.com/dhruvasagar/vim-dotoo>
|
||
|
||
[few contributions]
|
||
<https://github.com/dhruvasagar/vim-dotoo/pulls?q=is%3Apr+sort%3Aupdated-desc+author%3Akristijanhusak+is%3Aclosed>
|
||
|
||
[Neovim] <https://github.com/neovim/neovim>
|
||
|
||
Understanding the OrgMode syntax and all the functionality
|
||
──────────────────────────────────────────────────────────
|
||
|
||
This is still the biggest challenge. I didn’t have any idea how big
|
||
and robust OrgMode is. If I would know it at that time, I wouldn’t
|
||
even jump on this train. It’s really hard to grasp all of it.
|
||
Considering I’ve only used it for around 8 months, I think I made some
|
||
good progress on learning it.
|
||
|
||
|
||
Remote editing
|
||
──────────────
|
||
|
||
By remote editing, I mean automatically updating content in the
|
||
current or any other file. Few examples: adding/updating properties,
|
||
managing tags, changing TODO states, archiving, refiling, remote
|
||
editing from agenda view, etc.
|
||
|
||
There is no built-in way to update content in another file through the
|
||
Neovim API, without actually opening the file in an editor. I solved
|
||
this by:
|
||
|
||
• Saving as much position information as possible in the internal
|
||
state, so I can pinpoint the correct location
|
||
• Opening a file in a `1 row x 1 col' floating window and doing quick
|
||
edits there
|
||
|
||
|
||
Working with dates
|
||
──────────────────
|
||
|
||
From my experience, dates are challenging in all areas of programming,
|
||
so this is not so surprising. There are some Lua plugins for dates,
|
||
but those seemed a bit too basic for my use case, and I wanted to keep
|
||
external plugins to the minimum. I went with a custom solution that
|
||
uses Lua’s native dates, which has certain limitations, but works out
|
||
for most of the things.
|
||
|
||
|
||
Highlighting, mostly in Agenda view
|
||
───────────────────────────────────
|
||
|
||
Vim’s syntax engine is fairly old, but still very much used,
|
||
especially in the Vim community. Implementation of tree-sitter
|
||
slightly improved this experience in Neovim, because “Highlight
|
||
matches” are found via tree-sitter, instead of a bunch of regexes.
|
||
|
||
This helped me out later for the Org file itself, but agenda view is
|
||
still something that’s built as a custom view. Old Syntax highlight
|
||
engine would be really hard to add, because the content is too
|
||
dynamic. I went with the Neovim highlight API that allows Highlighting
|
||
things by their exact position in the buffer. Tree-sitter
|
||
implementation does something similar in the background for
|
||
Highlighting.
|
||
|
||
|
||
Keeping configuration simple and familiar to Emacs OrgMode
|
||
──────────────────────────────────────────────────────────
|
||
|
||
Vim-dotoo configuration was mostly Vim style, through some global
|
||
variables. I wanted to have a configuration that is familiar to an
|
||
Emacs OrgMode user, by having as many options as possible named
|
||
completely the same as in Emacs.
|
||
|
||
For example, Here’s a comparison of few options between Emacs and
|
||
Neovim:
|
||
|
||
Emacs:
|
||
|
||
┌────
|
||
│ (setq org-agenda-files '("~/orgmodes"))
|
||
│ (setq org-agenda-skip-scheduled-if-done t)
|
||
│ (setq org-agenda-span 7)
|
||
│ (setq org-hide-leading-stars t)
|
||
│ (setq org-capture-templates
|
||
│ '(("t" "Todo" entry (file "~/orgmodes/todos.org")
|
||
│ "* TODO %?")
|
||
│ ("j" "Journal" entry (file "~/orgmodes/journal.org")
|
||
│ "* %?\nEntered on %U\n %a")))
|
||
└────
|
||
|
||
Neovim:
|
||
|
||
┌────
|
||
│ require('orgmode').setup({
|
||
│ org_agenda_files = { '~/orgmodes' },
|
||
│ org_agenda_skip_scheduled_if_done = true,
|
||
│ org_agenda_span = 7,
|
||
│ org_hide_leading_stars = true
|
||
│ org_capture_templates = {
|
||
│ t = {
|
||
│ description = 'Todo',
|
||
│ target = '~/orgmodes/todos.org',
|
||
│ template = '* TODO %?',
|
||
│ },
|
||
│ j = {
|
||
│ description = 'Journal',
|
||
│ target = '~/orgmodes/journal.org',
|
||
│ template = '* %?\nEntered on %U\n %a',
|
||
│ }
|
||
│ }
|
||
│ })
|
||
└────
|
||
|
||
One of the most noticeable differences is between the usage of hyphens
|
||
(`-') and underscores (`_'). I did that only for the sake of
|
||
simplicity, because hyphens is not a valid character in variable names
|
||
in Lua, so all of the options would need to be wrapped as a string
|
||
(for example: `['org-agenda-files']').
|
||
|
||
|
||
First release of orgmode.nvim and introduction of tree-sitter parser
|
||
════════════════════════════════════════════════════════════════════
|
||
|
||
After ~1.5 months I [published the initial version]. The focus was on
|
||
Agenda and capturing (GTD), since those are the things I mostly used.
|
||
It got some traction, and people started testing it and reporting
|
||
bugs.
|
||
|
||
One of the common questions was: /“Any plans to introduce tree-sitter
|
||
parser?”/.
|
||
|
||
I knew about [tree-sitter] and used it in my day-to-day job for a few
|
||
programming languages, but I had absolutely no idea how it worked, and
|
||
especially how to write a tree-sitter parser. I put it aside, and
|
||
continued working on what I had.
|
||
|
||
One day, Emilia ([milisims]) contacted me via email to ask me if I
|
||
would be willing to try the tree-sitter parser she’s been working on
|
||
for some time. I gladly accepted. She gave me access to the
|
||
repository, and I started tinkering with it in a separate branch. No
|
||
one was aware at that point that tree-sitter support would happen some
|
||
time soon.
|
||
|
||
After some time, I set up a “beta” branch called “tree-sitter” and
|
||
[announced it for testing]. Once the reported bugs slowed to a
|
||
trickle, I merged it into the “master” branch.
|
||
|
||
I believe that tree-sitter grammar for Org could help out other
|
||
editors to implement their version of Orgmode plugin, but I don’t
|
||
think it would ever be helpful for Emacs. Emacs parser is the one and
|
||
only that has it all implemented. Also, as much as tree-sitter is
|
||
powerful, its main purpose is to parse programming languages, which
|
||
mostly has “static” patterns to match. Orgmode is by its nature
|
||
dynamic, which causes a variety of issues for a parser that’s not
|
||
meant for that kind of usage.
|
||
|
||
|
||
[published the initial version]
|
||
<https://www.reddit.com/r/neovim/comments/o8zp0k/orgmodenvim_orgmode_clone_written_in_lua_for/>
|
||
|
||
[tree-sitter] <https://github.com/tree-sitter/tree-sitter>
|
||
|
||
[milisims] <https://github.com/milisims>
|
||
|
||
[announced it for testing]
|
||
<https://www.reddit.com/r/neovim/comments/ph2xqc/orgmodenvim_treesitter_support/>
|
||
|
||
|
||
Limitations
|
||
═══════════
|
||
|
||
(Neo)Vim is a great editor, but it still cannot compare to Emacs in
|
||
certain things. Manipulating the “View” part of the editor is tricky
|
||
or impossible for certain things.
|
||
|
||
I even [made a label] for reported issues where Neovim support for
|
||
certain things is a blocker. I’m hoping that at least some of these
|
||
will be available in future Neovim releases.
|
||
|
||
|
||
[made a label]
|
||
<https://github.com/nvim-orgmode/orgmode/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc+label%3Aneovim-dependency>
|
||
|
||
|
||
Features
|
||
════════
|
||
|
||
I will not go into too many details about the available features,
|
||
since those can be viewed in [repository readme], but I want to
|
||
mention one feature that does not exist as a built/-in feature in the
|
||
Emacs Orgmode: [Notifications].
|
||
|
||
This allows getting a “desktop notification” for tasks that are within
|
||
the specified threshold for schedule/deadline time. It requires some
|
||
configuration to set up a cron job, but it’s been working great for me
|
||
for several months now.
|
||
|
||
|
||
[repository readme]
|
||
<https://github.com/nvim-orgmode/orgmode#features-detailed-breakdown>
|
||
|
||
[Notifications]
|
||
<https://github.com/nvim-orgmode/orgmode/blob/master/DOCS.md#notifications-experimental>
|
||
|
||
|
||
Plans
|
||
═════
|
||
|
||
The current state of the project is very usable for me. I’m not
|
||
lacking any of the major features, mostly because I’m not used to
|
||
using them. Nevertheless, there are plans to add more things, and I’m
|
||
getting a lot of help from the community. I want to specifically
|
||
mention [levouh] and [lukas-reineke], since they added a lot of value
|
||
to the project, and I want to thank them and everyone else who
|
||
contributed. Their help is much appreciated.
|
||
|
||
There are few high priority tasks that I’m hoping to flush out first:
|
||
|
||
• Implementing [v1.0.0] release of the tree-sitter parser. This should
|
||
allow for faster and less error-prone parsing.
|
||
• [Infrastructure for plugin developers], to allow other people to
|
||
build plugins on top of nvim-orgmode.
|
||
|
||
And a long term goal for these:
|
||
|
||
• Tables support (and at least basic formulas)
|
||
• [Org Babel like code block evaluation] (and hopefully basic support
|
||
for literate programming)
|
||
• [Diary format dates]
|
||
• [Custom agenda commands]
|
||
• More clocking features (reports)
|
||
• File specific configuration via directives ([todo keywords],
|
||
properties, etc.)
|
||
|
||
|
||
[levouh] <https://github.com/levouh>
|
||
|
||
[lukas-reineke] <https://github.com/lukas-reineke>
|
||
|
||
[v1.0.0] <https://github.com/milisims/tree-sitter-org/issues/13>
|
||
|
||
[Infrastructure for plugin developers]
|
||
<https://github.com/nvim-orgmode/orgmode/issues/26>
|
||
|
||
[Org Babel like code block evaluation]
|
||
<https://github.com/nvim-orgmode/orgmode/issues/190>
|
||
|
||
[Diary format dates]
|
||
<https://github.com/nvim-orgmode/orgmode/issues/195>
|
||
|
||
[Custom agenda commands]
|
||
<https://github.com/nvim-orgmode/orgmode/issues/135>
|
||
|
||
[todo keywords] <https://github.com/nvim-orgmode/orgmode/issues/185>
|
||
|
||
|
||
Closing thoughts
|
||
════════════════
|
||
|
||
When I started working on [nvim-orgmode], I didn’t have a clue what
|
||
I’m jumping into. Every day I learn about more and more Orgmode
|
||
features that I wasn’t even aware existed.
|
||
|
||
I’m certain that this project will never manage to clone the Orgmode
|
||
functionality completely, but I’m hoping it will get close enough so
|
||
everyone from Neovim community and Emacsers trying out Neovim will be
|
||
able to use it for their needs.
|
||
|
||
Having experienced Orgmode users [testing] it is a huge help, so if
|
||
anyone is willing to give it a try, feel free to open up an issue and
|
||
write your thoughts there. Thanks!
|
||
|
||
|
||
[nvim-orgmode] <https://github.com/nvim-orgmode/orgmode>
|
||
|
||
[testing] <https://github.com/nvim-orgmode/orgmode/issues/159>
|