Org-Drill
Table of Contents
Synopsis
Org-Drill uses the spaced repetition algorithm in org-learn to conduct
interactive "drill sessions", using org files as sources of facts to be
memorised. The material to be remembered is presented to the student in random
order. The student rates his or her recall of each item, and this information
is fed back to org-learn to schedule the item for later revision.
Each drill session can be restricted to topics in the current buffer (default), one or several files, all agenda files, or a subtree. A single topic can also be drilled.
Different "topic types" can be defined, which present their information to the student in different ways.
For more on the spaced repetition algorithm, and examples of other programs that use it, see:
Installation
Put the following in your .emacs. You will also need to make sure that Org's
"contrib/lisp" directory is in the emacs load-path.
(require 'org-drill)
Demonstration
Load the file spanish.org. Press M-x and run the function org-drill. Follow
the prompts at the bottom of the screen.
When the drill finishes, you can look at spanish.org to get some idea of how
drill topics are written.
Writing the questions
Org-Drill uses org mode topics as 'drill items'. To be used as a drill item,
the topic must have a tag that matches the value of
org-drill-question-tag. This is :drill: by default. Any other org topics
will be ignored.
You don't need to schedule the topics initially. However org-drill will
recognise items that have been scheduled previously with
org-learn. Unscheduled items are considered to be 'new' and ready for
memorisation.
How should 'drill topics' be structured? Any org topic is a legal drill topic – it will simply be shown with all subheadings collapsed, so thta only the material beneath the main item heading is visible. After pressing a key, any hidden subheadings will be revealed, and you will be asked to rate your "recall" of the item.
This will be adequate for some items, but usually you will want to write items where you have more control over what information is hidden from the user for recall purposes. For this reason, some other card types are defined, including:
A note about comments: In org mode, comment lines start with '#'. The rest of the line is ignored by Org (apart from some special cases). You may sometimes want to put material in comments which you do not want to see when you are being tested on the item. For this reason, comments are always rendered invisible while items are being tested.
Simple topics
The simplest drill topic has no special structure. When such a topic is presented during a drill session, any subheadings are "collapsed" with their contents hidden. So, you could include the question as text beneath the main heading, and the answer within a subheading. For example:
* Item :drill: What is the capital city of Estonia? ** The Answer Tallinn.
When this item is presented for review, the text beneath the main heading will be visible, but the contents of the subheading ("The Answer") will be hidden.
Cloze deletion
Cloze deletion can be used in any drill topic regardless of whether it is otherwise 'simple', or is one of the specialised topic types discussed below. To use cloze deletion, one or more parts of the body of the topic is marked as cloze text by surrounding it with single square brackets, [like so]. When the topic is presented for review, the text within square brackets will be obscured. The text is then revealed after the user presses a key. For example:
* Item :drill: The capital city of Estonia is [Tallinn].
During review, the user will see:
The capital city of Estonia is […].
When the user presses a key, the text "Tallinn" will become visible.
Clozed text can also contain a "hint" about the answer. If the text surrounded by single square brackets contains a `|' character (vertical bar), all text after that character is treated as a hint, and will be visible when the rest of the text is hidden.
Example:
Type 1 hypersensitivity reactions are mediated by [immunoglobulin E|molecule] and [mast cells|cell type].
Type 1 hypersensitivity reactions are mediated by […molecule] and […cell type].
Two-sided cards
The remaining topic types all use the topic property, DRILL_CARD_TYPE. This
property tells org-drill which function to use to present the topic during
review. If this property has the value twosided then the topic is treated as
a "two sided card". When a two sided card is reviewed, one of the first two
subheadings within the topic will be visible – all other
subheadings will be hidden.
Two-sided cards are meant to emulate the type of flipcard where either side is useful as test material (for example, a card with a word in a foreign language on one side, and its translation on the other).
A two sided card can have more than 2 subheadings, but all subheadings after the first two are considered as "notes" and will always be hidden during topic review.
* Noun :drill: :PROPERTIES: :DRILL_CARD_TYPE: twosided :END: Translate this word. ** Spanish la mujer ** English the woman ** Example sentence ¿Quién fue esa mujer? Who was that woman?
In this example, the user will be shown the main text – "Translate this word" – and either 'la mujer', or 'the woman', at random. The section 'Example sentence' will never be shown until after the user presses a key, because it is not one of the first two 'sides' of the topic.
Multi-sided cards
The multisided card type is similar to twosided, except that any
subheading has a chance of being presented during the topic review. One
subheading is always shown and all others are always hidden.
* Noun :drill: :PROPERTIES: :DRILL_CARD_TYPE: multisided :END: Translate. ** Spanish la mesa ** English the table ** Picture PICTURE
The user will be shown the main text and either 'la mujer', or 'the woman', or a picture of a table.
Multi-cloze cards
Often, you will wish to create cards out of sentences that express several facts, such as the following:
The capital city of New Zealand is Wellington, which is located in the South Island and has a population of about 400,000.
There is more than one fact in this statement – you could create a single 'simple' card with all the facts marked as cloze text, like so:
The capital city of [New Zealand] is [Wellington], which is located in the [North|North/South] Island and has a population of about [400,000].
But this card will be difficult to remember. If you get just one of the 4 hidden facts wrong, you will fail the card. A card like this is likely to become a leech.
A better way to express all these facts using 'simple' cards is to create several cards, with one fact per card. You might end up with something like this:
* Fact The capital city of [New Zealand] is Wellington, which has a population of about 400,000. * Fact The capital city of New Zealand is [Wellington], which has a population of about 400,000. * Fact The capital city of New Zealand is Wellington, which has a population of about [400,000]. * Fact The capital city of [New Zealand] is Wellington, which is located in the the North Island. * Fact The capital city of New Zealand is [Wellington], which is located in the North Island. * Fact The capital city of New Zealand is Wellington, which is located in the [North|North/South] Island.
However, this is really cumbersome. The 'multicloze' card type exists for this situation. Multicloze cards behave like 'simple' cards, except that when there is more than one area marked as cloze text, only one of the marked areas will be hidden during review – the others all remain visible. The hidden text area is chosen randomly at each review.
So, for the above example, we can actually use the original 'bad' simple card, but change its card type to 'multicloze'. Each time the card is presented for review, one of 'New Zealand', 'Wellington', 'the South Island' or '400,000' will be hidden.
* Fact :PROPERTIES: :DRILL_CARD_TYPE: multicloze :END: The capital city of [New Zealand] is [Wellington], which is located in the [North|North/South] Island and has a population of about [400,000].
User-defined card types
Finally, you can write your own emacs lisp functions to define new kinds of
topics. Any new topic type will need to be added to
org-drill-card-type-alist, and cards using that topic type will need to have
it as the value of their DRILL_CARD_TYPE property. For an example, see the
function org-drill-present-spanish-verb, which defines the new topic type
spanish_verb, used in 'spanish.org'.
See the file spanish.org for a full set of example material.
Running the drill session
Start a drill session with M-x org-drill. By default, this includes all
non-hidden topics in the current buffer. org-drill takes an optional
argument, SCOPE, which allows it to take drill items from other
sources. Possible values for SCOPE are:
- tree
- The subtree starting with the entry at the cursor.
- file
- The current buffer, including both hidden and non-hidden items.
- file-with-archives
- The current buffer, and any archives associated with it.
- agenda
- All agenda files.
- agenda-with-archives
- All agenda files with any archive files associated with them.
- (file1 file2 …)
- A list of filenames. All files in the list will be scanned.
During a drill session, you will be presented with each item, then asked to
rate your recall of it by pressing a key between 0 and 5. The meaning of these
numbers is (taken from org-learn):
| Quality | SuperMemo label | Meaning |
|---|---|---|
| 0 | NULL | You have forgotten this card completely. |
| 1 | BAD | Wrong answer. |
| 2 | FAIL | Barely correct, the interval was too long. |
| 3 | PASS | Correct answer, but with much effort. |
| 4 | GOOD | Correct answer, with a little thought. |
| 5 | BRIGHT | Correct answer, effortless. |
You can press '?' at the prompt if you have trouble remembering what the numbers 0–5 signify. At any time you can press 'q' to finish the drill early (your progress will be saved), 's' to skip the current item without viewing the answer, or 'e' to finish the drill and jump to the current topic for editing (your progress up to that point will be saved).
Cram mode
There are some situations, such as before an exam, where you will want to revise all of your cards regardless of when they are next due for review.
To do this, run a cram session with the org-drill-cram command (M-x org-drill-cram RET). This works the same as a normal drill session, except
that all items are considered due for review unless you reviewed them within
the last 12 hours (you can change the number of hours by customising the
variable org-drill-cram-hours).
Leeches
From the Anki website, http://ichi2.net/anki/wiki/Leeches:
Leeches are cards that you keep on forgetting. Because they require so many reviews, they take up a lot more of your time than other cards.
Like Anki, Org-Drill defines leeches as cards that you have "failed" many
times. The number of times an item must be failed before it is considered a
leech is set by the variable org-drill-leech-failure-threshold (15 by
default). When you fail to remember an item more than this many times, the item
will be given the :leech: tag.
Leech items can be handled in one of three ways. You can choose how Org-Drill
handles leeches by setting the variable org-drill-leech-method to one of the
following values:
- nil
-
Leech items are tagged with the
leechtag, but otherwise treated the same as normal items. - skip
- Leech items are not included in drill sessions.
- warn
- Leech items are still included in drill sessions, but a warning message is printed when each leech item is presented.
The best way to deal with a leech is either to delete it, or reformulate it so that it is easier to remember, for example by splitting it into more than one card.
See the SuperMemo website for more on leeches.
Customisation
Org-Drill has several settings which you change using
M-x customize-group org-drill <RET>. Alternatively you can change these
settings by adding elisp code to your configuration file (.emacs).
Appearance of items during drill sessions
If you want cloze-deleted text to show up in a special font within Org mode buffers, add this to your .emacs:
(setq org-drill-use-visible-cloze-face-p t)
Item headings may contain information that "gives away" the answer to the item, either in the heading text or in tags. If you want item headings to be made invisible while each item is being tested, add:
(setq org-drill-hide-item-headings-p t)
Duration of drill sessions
By default, a drill session will end when either 30 items have been successfully reviewed, or 20 minutes have passed. To change this behaviour, use the following settings.
(setq org-drill-maximum-items-per-session 40) (setq org-drill-maximum-duration 30) ; 30 minutes
If either of these variables is set to nil, then item count or elapsed time will not count as reasons to end the session. If both variables are nil, the session will not end until all outstanding items have been reviewed.
Spaced repetition algorithm
Choice of algorithm
Org-Drill supports three different spaced repetition algorithms, all based on SuperMemo algorithms. These are:
- SM2
- an early algorithm, used in SuperMemo 2.0 (1988), which remains very popular – Anki and Mnemosyne, two of the most popular spaced repetition programs, use SM2. This algorithm stores an 'ease factor' for each item, which is modified each time you rate your recall of the item.
- SM5 (default)
- used in SuperMemo 5.0 (1989). This algorithm uses 'ease factors' but also uses a persistent, per-user 'matrix of optimal factors' which is also modified after each item repetition.
- Simple8
- an experimental algorithm based on the SM8 algorithm. SM8 is used in SuperMemo 8.0 (1998) and is almost identical to SM11 which is used in SuperMemo 2002. Like SM5, it uses a matrix of optimal factors. Simple8 differs from SM8 in that it does not adapt the matrix to the individual user, though it does adapt each item's 'ease factor'.
If you want Org-Drill to use the SM2 algorithm, put the following in your
.emacs:
(setq org-drill-spaced-repetition-algorithm 'sm2)
Random variation of repetition intervals
The intervals generated by the SM2 and SM5 algorithms are pretty deterministic. If you tend to add items in large, infrequent batches, the lack of variation in interval scheduling can lead to the problem of "lumpiness" -- one day a large batch of items are due for review, the next there is almost nothing, a few days later another big pile of items is due.
This problem can be ameliorated by adding some random "noise" to the interval scheduling algorithm. The author of SuperMemo actually recommends this approach for the SM5 algorithm, and Org-Drill's implementation uses his code.
To enable random "noise" for item intervals, set the variable
org-drill-add-random-noise-to-intervals-p to true by putting the following in
your .emacs:
(setq org-drill-add-random-noise-to-intervals-p t)
Adjustment for early or late review of items
Reviewing items earlier or later than their scheduled review date may affect how soon the next review date should be scheduled. Code to make this adjustment is also presented on the SuperMemo website. It can be enabled with:
(setq org-drill-adjust-intervals-for-early-and-late-repetitions-p t)
This will affect both early and late repetitions if the Simple8 algorithm is used. For the SM5 algorithm it will affect early repetitions only. It has no effect on the SM2 algorithm.
Adjusting item difficulty globally
The learn fraction is a global value which affects how quickly the intervals
(times between each retest of an item) increase with successive repetitions,
for all items. The default value is 0.5, and this is the value used in
SuperMemo. For some collections of information, you may find that you are
reviewing items too often (they are too easy and the workload is too high), or
too seldom (you are failing them too often). In these situations, it is
possible to alter the learn fraction from its default in order to increase or
decrease the frequency of repetition of items over time. Increasing the value
will make the time intervals grow faster, and lowering it will make them grow
more slowly. The table below shows the growth in intervals (in days) with some
different values of the learn fraction (F). The table assumes that the item is
successfully recalled each time, with an average quality of just under 4.
| Repetition | F=0.3 | F=0.4 | F=0.5 | F=0.6 | F=0.7 |
|---|---|---|---|---|---|
| 1st | 2 | 2 | 2 | 2 | 2 |
| 2nd | 7 | 7 | 7 | 7 | 7 |
| 5th | 26 | 34 | 46 | 63 | 85 |
| 10th | 85 | 152 | 316 | 743 | 1942 |
| 15th | 233 | 501 | 1426 | 5471 | 27868 |
To alter the learn fraction, put the following in your .emacs:
(setq org-drill-learn-fraction 0.45) ; change the value as desired
Per-file customisation settings
Most of the above settings are safe as file-local variables. This means you can include a commented section like this at the end of your .org file to apply special settings when running a Drill session using that file:
# Local Variables: # org-drill-maximum-items-per-session: 50 # org-drill-spaced-repetition-algorithm: simple8 # End:
Incremental reading
An innovative feature of the program SuperMemo is so-called "incremental reading". This refers to the ability to quickly and easily make drill items from selected portions of text as you read an article (a web page for example). See the SuperMemo website for more on incremental reading.
Much of the infrastructure for incremental reading is already provided by Org Mode, with the help of some other emacs packages. You can provide yourself with an incremental reading facility by using 'org-capture' alongside a package that allows you to browse web pages either in emacs (w3 or emacs-w3m) or in the external browser of your choice (org-protocol).
Another important component of incremental reading is the ability to save your exact place in a document, so you can read it incrementally rather than all at once. There is a large variety of bookmarking packages for emacs which provide advanced bookmarking functionality: see the Emacs Wiki for details. Bookmarking exact webpage locations in an external browser seems to be a bit more difficult. For Firefox, the Wired Marker addon works well.
An example of using Org-Drill for incremental reading is given below. First,
and most importantly, we need to define a couple of org-capture templates for
captured facts.
(setq org-capture-templates `(("u" "Task: Read this URL" entry (file+headline "tasks.org" "Articles To Read") ,(concat "* TODO Read article: '%:description'\nURL: %c\n\n") :empty-lines 1 :immediate-finish t) ("w" "Capture web snippet" entry (file+headline "my-facts.org" "Inbox") ,(concat "* Fact: '%:description' :" (format "%s" org-drill-question-tag) ":\n:PROPERTIES:\n:DATE_ADDED: %u\n:SOURCE_URL: %c\n:END:\n\n%i\n%?\n") :empty-lines 1 :immediate-finish t) ;; ...other capture templates... ))
Using these templates and org-protocol, you can set up buttons in your web
browser to:
- Create a task telling you to read the URL of the currently viewed webpage
- Turn a region of selected text on a webpage, into a new fact which is saved to whichever file and heading you nominate in the template. The fact will contain a timestamp, and a hyperlink back to the webpage where you created it.
For example, suppose you are reading the Wikipedia entry on tuberculosis here.
You read the following:
The classic symptoms of tuberculosis are a chronic cough with blood-tinged sputum, fever, night sweats, and weight loss. Infection of other organs causes a wide range of symptoms. Treatment is difficult and requires long courses of multiple antibiotics. Antibiotic resistance is a growing problem in (extensively) multi-drug-resistant tuberculosis. Prevention relies on screening programs and vaccination, usually with Bacillus Calmette-Guérin vaccine.
You decide you want to remember that "Bacillus Calmette-Guérin vaccine" is the name of the vaccine against tuberculosis. First, you select the `interesting' portion of the text with the mouse:
The classic symptoms of tuberculosis are a chronic cough with blood-tinged sputum, fever, night sweats, and weight loss. Infection of other organs causes a wide range of symptoms. Treatment is difficult and requires long courses of multiple antibiotics. Antibiotic resistance is a growing problem in (extensively) multi-drug-resistant tuberculosis. Prevention relies on screening programs and vaccination, usually with Bacillus Calmette-Guérin vaccine.
Then you press the button you created when setting up org-protocol, which is
configured to activate the capture template "w: Capture web snippet". The
selected text will be sent to Emacs, turned into a new fact using the template,
and filed away for your later attention.
(Note that it might be more efficient to turn the entire paragraph into a drill item – since it contains several important facts – then split it up into multiple items when you edit it later in Emacs.)
Once you have had enough of reading the article, save your place, then go to your "fact" file in Emacs. You should see that each piece of text you selected has been turned into a drill item. Continuing the above example, you would see something like:
** Fact: 'Tuberculosis - Wikipedia, the Free Encyclopedia' :drill: Prevention relies on screening programs and vaccination, usually with Bacillus Calmette-Guérin vaccine.
You need to edit this fact so it makes sense independent of its context, as that is how it will be presented to you in future. The easiest way to turn the text into a 'question' is by cloze deletion. All you need to do is surround the 'hidden' parts of the text with square brackets.
Prevention of tuberculosis relies on screening programs and vaccination, usually with [Bacillus Calmette-Guérin vaccine].
You can of course define browser buttons that use several different "fact" templates, each of which might send its fact to a different file or subheading, or give it different tags or properties, for example.
Date: 2011-04-08 16:00:36
HTML generated by org-mode 7.4 in emacs 23