294 lines
14 KiB
Org Mode
294 lines
14 KiB
Org Mode
# Created 2022-03-04 Fri 16:30
|
|
#+title: February 2022
|
|
#+date: 2022-02-30
|
|
#+author: Kristijan Husak
|
|
#+subtitle: An orgmode clone for neovim
|
|
Timothy (TEC) here. This month we have a guest post from a different part of the
|
|
Org ecosystem, to highlight one of the most promising efforts to provide a good
|
|
experience outside Emacs.
|
|
|
|
//github.com/nvim-orgmode/orgmode
|
|
|
|
"But I use Emacs, I don't care" you may say. In that case, I'd like to point out
|
|
that wider spread and better Org support enriches the Org ecosystem as a whole.
|
|
It makes the format more approachable, and /useful/ for other people. This is good
|
|
for everybody.
|
|
|
|
Without any further ado, here's the guest post kindly written by Kristijan.
|
|
Enjoy!
|
|
|
|
-----
|
|
|
|
Like every beginner Vim user, at some point I ran into a usual editor war post:
|
|
Vim vs Emacs. At that time, I didn't have an idea what "Emacs" was.
|
|
|
|
A simple Google search yielded something that seemed just like a very simple
|
|
editor with strange, but more familiar shortcuts. I didn't bother too much to
|
|
figure out what it is, because I was already pulled in fairly deep into Vim and
|
|
its philosophy.
|
|
|
|
* Note taking in (Neo)Vim
|
|
|
|
At first, I did some note taking only when really necessary, in random
|
|
plain text files. Most of the things I managed to keep in my head, since
|
|
I was younger and less busy 🙂.
|
|
|
|
Once I got into the situation where I needed to keep more notes, [[https://github.com/vimwiki/vimwiki][vimwiki]] was the
|
|
natural choice.
|
|
|
|
That worked very well for a period, until the need for writing quick notes
|
|
arise. Vimwiki didn't have anything that would allow that. I could of course
|
|
have a mapping that opens a specific file where I can add notes, but that just
|
|
never felt right in my mind. I would keep a bunch of things in the same place,
|
|
and then later I needed to spend some time organizing them.
|
|
|
|
At that point, I wasn't sure how to achieve what I want. I did a brief look at
|
|
[[https://orgmode.org/][Emacs OrgMode]] to see what's all the fuss about, but to me, it seemed just like a
|
|
different version of Markdown. You put some unordered lists as your notes, and
|
|
that's it. I never spent more time trying to see all the neat features. I even
|
|
tried creating some of my custom note taking tools, but I never managed to
|
|
finish them because I didn't have a clear idea of how to solve my problems.
|
|
|
|
* First encounter with Orgmode like tool: vim-dotoo
|
|
|
|
One weekend, I was browsing through Vim subreddit, as I usually do at least once
|
|
a day. There was a post about an "Orgmode like task logging" plugin called
|
|
[[https://github.com/dhruvasagar/vim-dotoo][vim-dotoo]]. I opened it up, and I didn't see much at that point. I wasn't too
|
|
excited. I went through readme, and noticed that author ([[https://github.com/dhruvasagar][dhruvasagar]]) put a
|
|
fairly big emphasis on the "Agenda view". I had no idea what "Agenda view" is.
|
|
Thankfully, the author also made a [[https://www.youtube.com/watch?v=nsv33iOnH34][screencast]], which is rather long (1.5h), but
|
|
I had some time, so I went through it.
|
|
|
|
At that point, I was first met with "Capturing" and "Refiling". *My mind was
|
|
blown!* What a simple, yet extremely powerful idea! How had that never crossed my
|
|
mind? From that point on, this plugin had my full attention.
|
|
|
|
I'm always emphasizing that [[https://github.com/dhruvasagar][dhruvasagar]] and his [[https://github.com/dhruvasagar/vim-dotoo][vim-dotoo]] plugin are most
|
|
deserving for having inspired [[https://github.com/nvim-orgmode/orgmode]], and I
|
|
can't thank him enough for that.
|
|
|
|
* First steps with vim-dotoo and birth of orgmode.nvim
|
|
|
|
For some time, I was using [[https://github.com/dhruvasagar/vim-dotoo][vim-dotoo]]. I moved all of my Vimwiki notes to it. It
|
|
was a breath of fresh air. Alongside that, I started getting more interest in
|
|
the original Emacs Orgmode. I started noticing the differences, and some of the
|
|
missing features that were now looking quite attractive. I made [[https://github.com/dhruvasagar/vim-dotoo/pulls?q=is%3Apr+sort%3Aupdated-desc+author%3Akristijanhusak+is%3Aclosed][few
|
|
contributions]] to vim-dotoo. As time passed, and my notes started to grow, things
|
|
began being slow. I did some profiling, and figured out that it's just a usual
|
|
Vim problem, Vimscript performance. It was just too slow for certain things that
|
|
Orgmode provides, and it would hardly get any better as more things are added.
|
|
|
|
Separately from Vim and Vimscript, [[https://github.com/neovim/neovim][Neovim]] was on a stable =v0.4= release, and =v0.5=
|
|
was still being developed. I was using Neovim from version 0.3, and was
|
|
carefully following the progress on it. Lua was introduced as a first class
|
|
citizen. A Bunch of new plugins arise from it. All the benchmarks showed that
|
|
Lua outperforms Vimscript in almost everything. Besides the performance, Lua is
|
|
a "normal" programming language, which means that support for it is much better.
|
|
|
|
At that point, I became curious: Could Lua be the path to the faster Orgmode? I
|
|
spent several days thinking about it. I wanted to give it a try. My biggest
|
|
concern was that I had absolutely zero experience writing parsers. I had never
|
|
written anything more complicated than an averagely complicated regex for
|
|
purposes of parsing. I noticed that vim-dotoo also used regex to do the parsing,
|
|
so that eased my mind a bit.
|
|
|
|
One weekend, I started working on it. It was really interesting and challenging.
|
|
I spent a lot of my free time on it. At certain points, it seemed like hacking,
|
|
since it was not a proper parsing. I tried to learn how to write a proper
|
|
parser, but it was just too time consuming and complicated. I proceeded with the
|
|
regex parsing to see how far I can go.
|
|
|
|
Besides parsing, I had a few more challenges to overcome:
|
|
|
|
** Understanding the OrgMode syntax and all the functionality
|
|
|
|
This is still the biggest challenge. I didn't have any idea how big and robust
|
|
OrgMode is. If I would know it at that time, I wouldn't even jump on this train.
|
|
It's really hard to grasp all of it. Considering I've only used it for around 8
|
|
months, I think I made some good progress on learning it.
|
|
|
|
** Remote editing
|
|
|
|
By remote editing, I mean automatically updating content in the current
|
|
or any other file. Few examples: adding/updating properties, managing
|
|
tags, changing TODO states, archiving, refiling, remote editing from
|
|
agenda view, etc.
|
|
|
|
There is no built-in way to update content in another file through the
|
|
Neovim API, without actually opening the file in an editor. I solved
|
|
this by:
|
|
|
|
- Saving as much position information as possible in the internal state,
|
|
so I can pinpoint the correct location
|
|
- Opening a file in a =1 row x 1 col= floating window and doing quick
|
|
edits there
|
|
|
|
** Working with dates
|
|
|
|
From my experience, dates are challenging in all areas of programming,
|
|
so this is not so surprising. There are some Lua plugins for dates, but
|
|
those seemed a bit too basic for my use case, and I wanted to keep
|
|
external plugins to the minimum. I went with a custom solution that uses
|
|
Lua's native dates, which has certain limitations, but works out for
|
|
most of the things.
|
|
|
|
** Highlighting, mostly in Agenda view
|
|
|
|
Vim's syntax engine is fairly old, but still very much used, especially
|
|
in the Vim community. Implementation of tree-sitter slightly improved
|
|
this experience in Neovim, because "Highlight matches" are found via
|
|
tree-sitter, instead of a bunch of regexes.
|
|
|
|
This helped me out later for the Org file itself, but agenda view is
|
|
still something that's built as a custom view. Old Syntax highlight engine
|
|
would be really hard to add, because the content is too dynamic. I went
|
|
with the Neovim highlight API that allows Highlighting things by their
|
|
exact position in the buffer. Tree-sitter implementation does something
|
|
similar in the background for Highlighting.
|
|
|
|
** Keeping configuration simple and familiar to Emacs OrgMode
|
|
|
|
Vim-dotoo configuration was mostly Vim style, through some global
|
|
variables. I wanted to have a configuration that is familiar to an Emacs
|
|
OrgMode user, by having as many options as possible named completely the
|
|
same as in Emacs.
|
|
|
|
For example, Here's a comparison of few options between Emacs and
|
|
Neovim:
|
|
|
|
Emacs:
|
|
|
|
#+begin_src elisp
|
|
(setq org-agenda-files '("~/orgmodes"))
|
|
(setq org-agenda-skip-scheduled-if-done t)
|
|
(setq org-agenda-span 7)
|
|
(setq org-hide-leading-stars t)
|
|
(setq org-capture-templates
|
|
'(("t" "Todo" entry (file "~/orgmodes/todos.org")
|
|
"* TODO %?")
|
|
("j" "Journal" entry (file "~/orgmodes/journal.org")
|
|
"* %?\nEntered on %U\n %a")))
|
|
#+end_src
|
|
|
|
Neovim:
|
|
|
|
#+begin_src lua
|
|
require('orgmode').setup({
|
|
org_agenda_files = { '~/orgmodes' },
|
|
org_agenda_skip_scheduled_if_done = true,
|
|
org_agenda_span = 7,
|
|
org_hide_leading_stars = true
|
|
org_capture_templates = {
|
|
t = {
|
|
description = 'Todo',
|
|
target = '~/orgmodes/todos.org',
|
|
template = '* TODO %?',
|
|
},
|
|
j = {
|
|
description = 'Journal',
|
|
target = '~/orgmodes/journal.org',
|
|
template = '* %?\nEntered on %U\n %a',
|
|
}
|
|
}
|
|
})
|
|
#+end_src
|
|
|
|
One of the most noticeable differences is between the usage of hyphens
|
|
(=-=) and underscores (=_=). I did that only for the sake of simplicity,
|
|
because hyphens is not a valid character in variable names in Lua, so
|
|
all of the options would need to be wrapped as a string (for example:
|
|
=['org-agenda-files']=).
|
|
|
|
* First release of orgmode.nvim and introduction of tree-sitter parser
|
|
|
|
After ~1.5 months I [[https://www.reddit.com/r/neovim/comments/o8zp0k/orgmodenvim_orgmode_clone_written_in_lua_for/][published the initial version]]. The focus was on Agenda and
|
|
capturing (GTD), since those are the things I mostly used. It got some traction,
|
|
and people started testing it and reporting bugs.
|
|
|
|
One of the common questions was: /"Any plans to introduce tree-sitter parser?"/.
|
|
|
|
I knew about [[https://github.com/tree-sitter/tree-sitter][tree-sitter]] and used it in my day-to-day job for a few programming
|
|
languages, but I had absolutely no idea how it worked, and especially how to
|
|
write a tree-sitter parser. I put it aside, and continued working on what I
|
|
had.
|
|
|
|
One day, Emilia ([[https://github.com/milisims][milisims]]) contacted me via email to ask me if I would be
|
|
willing to try the tree-sitter parser she's been working on for some time. I
|
|
gladly accepted. She gave me access to the repository, and I started tinkering
|
|
with it in a separate branch. No one was aware at that point that tree-sitter
|
|
support would happen some time soon.
|
|
|
|
After some time, I set up a "beta" branch called "tree-sitter" and [[https://www.reddit.com/r/neovim/comments/ph2xqc/orgmodenvim_treesitter_support/][announced it
|
|
for testing]]. Once the reported bugs slowed to a trickle, I merged it into the
|
|
"master" branch.
|
|
|
|
I believe that tree-sitter grammar for Org could help out other editors to
|
|
implement their version of Orgmode plugin, but I don't think it would ever be
|
|
helpful for Emacs. Emacs parser is the one and only that has it all implemented.
|
|
Also, as much as tree-sitter is powerful, its main purpose is to parse
|
|
programming languages, which mostly has "static" patterns to match. Orgmode is
|
|
by its nature dynamic, which causes a variety of issues for a parser that's not
|
|
meant for that kind of usage.
|
|
|
|
* Limitations
|
|
|
|
(Neo)Vim is a great editor, but it still cannot compare to Emacs in certain
|
|
things. Manipulating the "View" part of the editor is tricky or impossible for
|
|
certain things.
|
|
|
|
I even [[https://github.com/nvim-orgmode/orgmode/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc+label%3Aneovim-dependency][made a label]] for reported issues where Neovim support for certain things
|
|
is a blocker. I'm hoping that at least some of these will be available in future
|
|
Neovim releases.
|
|
|
|
* Features
|
|
|
|
I will not go into too many details about the available features, since those
|
|
can be viewed in [[https://github.com/nvim-orgmode/orgmode#features-detailed-breakdown][repository readme]], but I want to mention one feature that does
|
|
not exist as a built/-in feature in the Emacs Orgmode: [[https://github.com/nvim-orgmode/orgmode/blob/master/DOCS.md#notifications-experimental][Notifications]].
|
|
|
|
This allows getting a "desktop notification" for tasks that are within
|
|
the specified threshold for schedule/deadline time. It requires some
|
|
configuration to set up a cron job, but it's been working great for me
|
|
for several months now.
|
|
|
|
* Plans
|
|
|
|
The current state of the project is very usable for me. I'm not lacking any of
|
|
the major features, mostly because I'm not used to using them. Nevertheless,
|
|
there are plans to add more things, and I'm getting a lot of help from the
|
|
community. I want to specifically mention [[https://github.com/levouh][levouh]] and [[https://github.com/lukas-reineke][lukas-reineke]], since they
|
|
added a lot of value to the project, and I want to thank them and everyone else
|
|
who contributed. Their help is much appreciated.
|
|
|
|
There are few high priority tasks that I'm hoping to flush out first:
|
|
|
|
- Implementing [[https://github.com/milisims/tree-sitter-org/issues/13][v1.0.0]] release of the tree-sitter parser. This should allow for
|
|
faster and less error-prone parsing.
|
|
- [[https://github.com/nvim-orgmode/orgmode/issues/26][Infrastructure for plugin developers]], to allow other people to build plugins
|
|
on top of nvim-orgmode.
|
|
|
|
And a long term goal for these:
|
|
|
|
- Tables support (and at least basic formulas)
|
|
- [[https://github.com/nvim-orgmode/orgmode/issues/190][Org Babel like code block evaluation]] (and hopefully basic support for literate
|
|
programming)
|
|
- [[https://github.com/nvim-orgmode/orgmode/issues/195][Diary format dates]]
|
|
- [[https://github.com/nvim-orgmode/orgmode/issues/135][Custom agenda commands]]
|
|
- More clocking features (reports)
|
|
- File specific configuration via directives ([[https://github.com/nvim-orgmode/orgmode/issues/185][todo keywords]], properties, etc.)
|
|
|
|
* Closing thoughts
|
|
|
|
When I started working on [[https://github.com/nvim-orgmode/orgmode][nvim-orgmode]], I didn't have a clue what I'm jumping
|
|
into. Every day I learn about more and more Orgmode features that I wasn't even
|
|
aware existed.
|
|
|
|
I'm certain that this project will never manage to clone the Orgmode
|
|
functionality completely, but I'm hoping it will get close enough so everyone
|
|
from Neovim community and Emacsers trying out Neovim will be able to use it for
|
|
their needs.
|
|
|
|
Having experienced Orgmode users [[https://github.com/nvim-orgmode/orgmode/issues/159][testing]] it is a huge help, so if anyone is
|
|
willing to give it a try, feel free to open up an issue and write your thoughts
|
|
there. Thanks!
|