New file-safe user variable: org-drill-scope. Allows customisation of the scope of drill sessions

(ie where items are sourced from). The variable can take the value 'directory', meaning items
will be sourced from all org files in the same directory as the current file. This is useful
for splitting up a monolithic item collection into smaller files, while still including all
those files in drill sessions.

The values of the SCOPE argument to org-drill have changed slightly. Possible values are the same
as those for 'org-drill-scope'.
- 'file' now means current file/buffer, respecting restrictions (default, same as old 'nil')
- 'file-no-restriction' means current file, ignoring (overriding) restrictions (same as old 'file')
- new value 'directory' (see above)
- other values have the same meaning as before.

3 files changed, 243 insertions, 98 deletions

diff --git a/README.html b/README.html
index 32c9908..5dba098 100755
--- a/README.html
+++ b/README.html
@@ -7,7 +7,7 @@ lang="en" xml:lang="en">
 <title>Org-Drill</title>
 <meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
 <meta name="generator" content="Org-mode"/>
-<meta name="generated" content="2011-05-10 16:51:44 "/>
+<meta name="generated" content="2011-05-12 10:29:29 "/>
 <meta name="author" content="Paul Sexton"/>
 <meta name="description" content=""/>
 <meta name="keywords" content=""/>
@@ -109,20 +109,22 @@ lang="en" xml:lang="en">
 <ul>
 <li><a href="#sec-8_1">Visual appearance of items during drill sessions </a></li>
 <li><a href="#sec-8_2">Duration of drill sessions </a></li>
-<li><a href="#sec-8_3">Definition of old and overdue items </a></li>
-<li><a href="#sec-8_4">Spaced repetition algorithm </a>
+<li><a href="#sec-8_3">Sources of items for drill sessions (scope) </a></li>
+<li><a href="#sec-8_4">Definition of old and overdue items </a></li>
+<li><a href="#sec-8_5">Spaced repetition algorithm </a>
 <ul>
-<li><a href="#sec-8_4_1">Choice of algorithm </a></li>
-<li><a href="#sec-8_4_2">Random variation of repetition intervals </a></li>
-<li><a href="#sec-8_4_3">Adjustment for early or late review of items </a></li>
-<li><a href="#sec-8_4_4">Adjusting item difficulty globally </a></li>
+<li><a href="#sec-8_5_1">Choice of algorithm </a></li>
+<li><a href="#sec-8_5_2">Random variation of repetition intervals </a></li>
+<li><a href="#sec-8_5_3">Adjustment for early or late review of items </a></li>
+<li><a href="#sec-8_5_4">Adjusting item difficulty globally </a></li>
 </ul>
 </li>
-<li><a href="#sec-8_5">Per-file customisation settings </a></li>
+<li><a href="#sec-8_6">Per-file customisation settings </a></li>
 </ul>
 </li>
-<li><a href="#sec-9">Sharing, merging and synchronising item collections </a></li>
-<li><a href="#sec-10">Incremental reading </a></li>
+<li><a href="#sec-9">Coping with large collections </a></li>
+<li><a href="#sec-10">Sharing, merging and synchronising item collections </a></li>
+<li><a href="#sec-11">Incremental reading </a></li>
 </ul>
 </div>
 </div>
@@ -621,7 +623,7 @@ functions at the end of org-drill.el &ndash; these include:
 <li><code>org-drill-present-verb-conjugation</code>, which implements the 'conjugate'
   card type. This asks the user to conjugate a verb in a particular tense. It
   demonstrates how the appearance of an entry can be completely altered during
-  a drill session, both during testing and during the display fo the answer.
+  a drill session, both during testing and during the display of the answer.
 </li>
 <li><code>org-drill-present-translate-number</code>, which uses a third-party emacs lisp
   library (<a href="http://www.emacswiki.org/emacs/spell-number.el">spell-number.el</a>) to prompt the user to translate random numbers
@@ -681,29 +683,8 @@ drill session, put a brief comment ('# &hellip;')  in the item body.
 Start a drill session with <code>M-x org-drill</code>. By default, this includes all
 non-hidden topics in the current buffer. <code>org-drill</code> takes an optional
 argument, SCOPE, which allows it to take drill items from other
-sources. Possible values for SCOPE are:
+sources. See <a href="#sec-8_3">below</a> for details.
 </p>
-<dl>
-<dt>tree</dt><dd>The subtree starting with the entry at the cursor. (Alternatively you
-          can use <code>M-x org=drill-tree</code> to run the drill session &ndash; this will
-          behave the same as <code>org-drill</code> if 'tree' was used as the value of
-          SCOPE.)
-</dd>
-<dt>file</dt><dd>The current buffer, including both hidden and non-hidden items.
-</dd>
-<dt>file-with-archives</dt><dd>The current buffer, and any archives associated with it.
-</dd>
-<dt>agenda</dt><dd>All agenda files.
-</dd>
-<dt>agenda-with-archives</dt><dd>All agenda files with any archive files associated
-     with them.
-</dd>
-<dt>(file1 file2 &hellip;)</dt><dd>A list of filenames. All files in the list will be
-     scanned.
-</dd>
-</dl>
-
-
 <p>
 During a drill session, you will be presented with each item, then asked to
 rate your recall of it by pressing a key between 0 and 5. The meaning of these
@@ -911,10 +892,53 @@ session will not end until <i>all</i> outstanding items have been reviewed.
 </div>
 
 <div id="outline-container-8_3" class="outline-3">
-<h3 id="sec-8_3">Definition of old and overdue items </h3>
+<h3 id="sec-8_3"><a name="scope" id="scope"></a>Sources of items for drill sessions (scope) </h3>
 <div class="outline-text-3" id="text-8_3">
 
 
+<p>
+By default, Org-Drill gathers drill items from the current buffer only,
+ignoring any non-visible items. There may be times when you want Org-Drill to
+gather drill items from other sources. You can do this by changing the value of
+the variable <code>org-drill-scope</code>. Possible values are:
+</p>
+<dl>
+<dt>file</dt><dd>The current buffer, ignoring hidden items. This is the default.
+</dd>
+<dt>tree</dt><dd>The subtree starting with the entry at the cursor. (Alternatively you
+          can use <code>M-x org=drill-tree</code> to run the drill session &ndash; this will
+          behave the same as <code>org-drill</code> if 'tree' was used as the value of
+          SCOPE.)
+</dd>
+<dt>file-no-restriction</dt><dd>The current buffer, including both hidden and
+     non-hidden items.
+</dd>
+<dt>file-with-archives</dt><dd>The current buffer, and any archives associated with it.
+</dd>
+<dt>agenda</dt><dd>All agenda files.
+</dd>
+<dt>agenda-with-archives</dt><dd>All agenda files with any archive files associated
+     with them.
+</dd>
+<dt>directory</dt><dd>All files with the extension '.org' in the same directory as the
+               current file. (The current file will also be included if its
+               extension is .org)
+</dd>
+<dt>(file1 file2 &hellip;)</dt><dd>A list of filenames. All files in the list will be
+     scanned.
+</dd>
+</dl>
+
+
+</div>
+
+</div>
+
+<div id="outline-container-8_4" class="outline-3">
+<h3 id="sec-8_4">Definition of old and overdue items </h3>
+<div class="outline-text-3" id="text-8_4">
+
+
 
 <p>
 Org-Drill prioritises <i>overdue</i> items in each drill session, presenting them
@@ -971,18 +995,18 @@ of 10 days or less. To change this, use the following:
 
 </div>
 
-<div id="outline-container-8_4" class="outline-3">
-<h3 id="sec-8_4">Spaced repetition algorithm </h3>
-<div class="outline-text-3" id="text-8_4">
+<div id="outline-container-8_5" class="outline-3">
+<h3 id="sec-8_5">Spaced repetition algorithm </h3>
+<div class="outline-text-3" id="text-8_5">
 
 
 
 
 </div>
 
-<div id="outline-container-8_4_1" class="outline-4">
-<h4 id="sec-8_4_1">Choice of algorithm </h4>
-<div class="outline-text-4" id="text-8_4_1">
+<div id="outline-container-8_5_1" class="outline-4">
+<h4 id="sec-8_5_1">Choice of algorithm </h4>
+<div class="outline-text-4" id="text-8_5_1">
 
 
 
@@ -1024,9 +1048,9 @@ SuperMemo algorithms. These are:
 
 </div>
 
-<div id="outline-container-8_4_2" class="outline-4">
-<h4 id="sec-8_4_2">Random variation of repetition intervals </h4>
-<div class="outline-text-4" id="text-8_4_2">
+<div id="outline-container-8_5_2" class="outline-4">
+<h4 id="sec-8_5_2">Random variation of repetition intervals </h4>
+<div class="outline-text-4" id="text-8_5_2">
 
 
 
@@ -1060,9 +1084,9 @@ your <code>.emacs</code>:
 
 </div>
 
-<div id="outline-container-8_4_3" class="outline-4">
-<h4 id="sec-8_4_3">Adjustment for early or late review of items </h4>
-<div class="outline-text-4" id="text-8_4_3">
+<div id="outline-container-8_5_3" class="outline-4">
+<h4 id="sec-8_5_3">Adjustment for early or late review of items </h4>
+<div class="outline-text-4" id="text-8_5_3">
 
 
 
@@ -1089,9 +1113,9 @@ effect on the SM2 algorithm.
 
 </div>
 
-<div id="outline-container-8_4_4" class="outline-4">
-<h4 id="sec-8_4_4">Adjusting item difficulty globally </h4>
-<div class="outline-text-4" id="text-8_4_4">
+<div id="outline-container-8_5_4" class="outline-4">
+<h4 id="sec-8_5_4">Adjusting item difficulty globally </h4>
+<div class="outline-text-4" id="text-8_5_4">
 
 
 
@@ -1144,16 +1168,16 @@ To alter the learn fraction, put the following in your .emacs:
 
 </div>
 
-<div id="outline-container-8_5" class="outline-3">
-<h3 id="sec-8_5">Per-file customisation settings </h3>
-<div class="outline-text-3" id="text-8_5">
-
+<div id="outline-container-8_6" class="outline-3">
+<h3 id="sec-8_6"><a name="per-file-settings" id="per-file-settings"></a>Per-file customisation settings </h3>
+<div class="outline-text-3" id="text-8_6">
 
 
 <p>
-Most of the above settings are safe as file-local variables. This means you can
-include a commented section like this at the end of your .org file to apply
-special settings when running a Drill session using that file:
+Most of Org-Drill's customisation settings are safe as file-local
+variables. This means you can include a commented section like this at the end
+of your .org file to apply special settings when running a Drill session using
+that file:
 </p>
 
 
@@ -1166,6 +1190,22 @@ special settings when running a Drill session using that file:
 
 
 
+<p>
+You can achieve the same effect by including the settings in the 'mode line'
+(this must be the <b>first line</b> in the file), like so:
+</p>
+
+
+
+<pre class="example"># -*- org-drill-maximum-items-per-session: 50; org-drill-spaced-repetition-algorithm: simple8 -*-
+</pre>
+
+
+
+<p>
+In either case you will need to save, close and re-open the file for the
+changes to take effect.
+</p>
 
 </div>
 </div>
@@ -1173,12 +1213,39 @@ special settings when running a Drill session using that file:
 </div>
 
 <div id="outline-container-9" class="outline-2">
-<h2 id="sec-9">Sharing, merging and synchronising item collections </h2>
+<h2 id="sec-9">Coping with large collections </h2>
 <div class="outline-text-2" id="text-9">
 
 
 
 <p>
+If you keep all your items in a single file, it may eventually very large. The
+file will be slow to load, and Emacs may have trouble syntax-highlighting the
+file contents correctly.
+</p>
+<p>
+The easiest steps to solve this problem are:
+</p><ol>
+<li>Move your file into its own dedicated directory.
+</li>
+<li>Divide the file into two or more smaller files.
+</li>
+<li>Within each file, set <code>org-drill-scope</code> to 'directory'. See
+   <a href="#sec-8_6">per-file settings</a> above for instructions about how to do this.
+</li>
+</ol>
+
+</div>
+
+</div>
+
+<div id="outline-container-10" class="outline-2">
+<h2 id="sec-10">Sharing, merging and synchronising item collections </h2>
+<div class="outline-text-2" id="text-10">
+
+
+
+<p>
 Every drill item is automatically given a persistent unique "ID" the first time
 it is seen by Org-Drill. This means that if two different people subsequently
 edit or reschedule that item, Org-Drill can still tell that it is the same
@@ -1236,9 +1303,9 @@ procedure.
 
 </div>
 
-<div id="outline-container-10" class="outline-2">
-<h2 id="sec-10">Incremental reading </h2>
-<div class="outline-text-2" id="text-10">
+<div id="outline-container-11" class="outline-2">
+<h2 id="sec-11">Incremental reading </h2>
+<div class="outline-text-2" id="text-11">
 
 
 
@@ -1396,7 +1463,7 @@ or give it different tags or properties, for example.
 </div>
 </div>
 <div id="postamble">
-<p class="date">Date: 2011-05-10 16:51:44 </p>
+<p class="date">Date: 2011-05-12 10:29:29 </p>
 <p class="author">Author: Paul Sexton</p>
 <p class="creator">Org version 7.5 with Emacs version 23</p>
 <a href="http://validator.w3.org/check?uri=referer">Validate XHTML 1.0</a>
diff --git a/README.org b/README.org
index 31bdbe9..a964532 100755
--- a/README.org
+++ b/README.org
@@ -331,7 +331,7 @@ functions at the end of org-drill.el -- these include:
 - =org-drill-present-verb-conjugation=, which implements the 'conjugate'
   card type. This asks the user to conjugate a verb in a particular tense. It
   demonstrates how the appearance of an entry can be completely altered during
-  a drill session, both during testing and during the display fo the answer.
+  a drill session, both during testing and during the display of the answer.
 - =org-drill-present-translate-number=, which uses a third-party emacs lisp
   library ([[http://www.emacswiki.org/emacs/spell-number.el][spell-number.el]]) to prompt the user to translate random numbers
   to and from any language recognised by that library.
@@ -366,19 +366,7 @@ drill session, put a brief comment ('# ...')  in the item body.
 Start a drill session with =M-x org-drill=. By default, this includes all
 non-hidden topics in the current buffer. =org-drill= takes an optional
 argument, SCOPE, which allows it to take drill items from other
-sources. Possible values for SCOPE are:
-
-- tree :: The subtree starting with the entry at the cursor. (Alternatively you
-          can use =M-x org=drill-tree= to run the drill session -- this will
-          behave the same as =org-drill= if 'tree' was used as the value of
-          SCOPE.)
-- file :: The current buffer, including both hidden and non-hidden items.
-- file-with-archives :: The current buffer, and any archives associated with it.
-- agenda :: All agenda files.
-- agenda-with-archives :: All agenda files with any archive files associated
-     with them.
-- (file1 file2 ...) :: A list of filenames. All files in the list will be
-     scanned.
+sources. See [[scope][below]] for details.
 
 During a drill session, you will be presented with each item, then asked to
 rate your recall of it by pressing a key between 0 and 5. The meaning of these
@@ -503,6 +491,33 @@ will not count as reasons to end the session. If both variables are nil, the
 session will not end until /all/ outstanding items have been reviewed.
 
 
+** Sources of items for drill sessions (scope)
+# <<scope>>
+
+By default, Org-Drill gathers drill items from the current buffer only,
+ignoring any non-visible items. There may be times when you want Org-Drill to
+gather drill items from other sources. You can do this by changing the value of
+the variable =org-drill-scope=. Possible values are:
+
+- file :: The current buffer, ignoring hidden items. This is the default.
+- tree :: The subtree starting with the entry at the cursor. (Alternatively you
+          can use =M-x org=drill-tree= to run the drill session -- this will
+          behave the same as =org-drill= if 'tree' was used as the value of
+          SCOPE.)
+- file-no-restriction :: The current buffer, including both hidden and
+     non-hidden items.
+- file-with-archives :: The current buffer, and any archives associated with it.
+- agenda :: All agenda files.
+- agenda-with-archives :: All agenda files with any archive files associated
+     with them.
+- directory :: All files with the extension '.org' in the same directory as the
+               current file. (The current file will also be included if its
+               extension is .org)
+- (file1 file2 ...) :: A list of filenames. All files in the list will be
+     scanned.
+
+
+
 ** Definition of old and overdue items
 
 
@@ -643,11 +658,12 @@ To alter the learn fraction, put the following in your .emacs:
 
 
 ** Per-file customisation settings
+# <<per-file settings>>
 
-
-Most of the above settings are safe as file-local variables. This means you can
-include a commented section like this at the end of your .org file to apply
-special settings when running a Drill session using that file:
+Most of Org-Drill's customisation settings are safe as file-local
+variables. This means you can include a commented section like this at the end
+of your .org file to apply special settings when running a Drill session using
+that file:
 
 #+BEGIN_EXAMPLE
 # Local Variables:
@@ -656,6 +672,30 @@ special settings when running a Drill session using that file:
 # End:
 #+END_EXAMPLE
 
+You can achieve the same effect by including the settings in the 'mode line'
+(this must be the *first line* in the file), like so:
+
+#+BEGIN_EXAMPLE
+# -*- org-drill-maximum-items-per-session: 50; org-drill-spaced-repetition-algorithm: simple8 -*-
+#+END_EXAMPLE
+
+In either case you will need to save, close and re-open the file for the
+changes to take effect.
+
+
+* Coping with large collections
+
+
+If you keep all your items in a single file, it may eventually very large. The
+file will be slow to load, and Emacs may have trouble syntax-highlighting the
+file contents correctly.
+
+The easiest steps to solve this problem are:
+1. Move your file into its own dedicated directory.
+2. Divide the file into two or more smaller files.
+3. Within each file, set =org-drill-scope= to 'directory'. See
+   [[per-file settings]] above for instructions about how to do this.
+
 
 * Sharing, merging and synchronising item collections
 
diff --git a/org-drill.el b/org-drill.el
index 6cd329d..599edf8 100755
--- a/org-drill.el
+++ b/org-drill.el
@@ -232,6 +232,35 @@ alist takes one of two forms:
   :type '(alist :key-type (choice string (const nil)) :value-type function))
 
 
+(defcustom org-drill-scope
+  'file
+  "The scope in which to search for drill items when conducting a
+drill session. This can be any of:
+
+file                 The current buffer, respecting the restriction if any.
+                     This is the default.
+tree                 The subtree started with the entry at point
+file-no-restriction  The current buffer, without restriction
+file-with-archives   The current buffer, and any archives associated with it.
+agenda               All agenda files
+agenda-with-archives All agenda files with any archive files associated
+                     with them.
+directory            All files with the extension '.org' in the same
+                     directory as the current file (includes the current
+                     file if it is an .org file.)
+ (FILE1 FILE2 ...)   If this is a list, all files in the list will be scanned.
+"
+  ;; Note -- meanings differ slightly from the argument to org-map-entries:
+  ;; 'file' means current file/buffer, respecting any restriction
+  ;; 'file-no-restriction' means current file/buffer, ignoring restrictions
+  ;; 'directory' means all *.org files in current directory
+  :group 'org-drill
+  :type '(choice (const 'file) (const 'tree) (const 'file-no-restriction)
+                 (const 'file-with-archives) (const 'agenda)
+                 (const 'agenda-with-archives) (const 'directory)
+                 list))
+
+
 (defcustom org-drill-spaced-repetition-algorithm
   'sm5
   "Which SuperMemo spaced repetition algorithm to use for scheduling items.
@@ -249,7 +278,6 @@ Available choices are:
   :type '(choice (const 'sm2) (const 'sm5) (const 'simple8)))
 
 
-
 (defcustom org-drill-optimal-factor-matrix
   nil
   "DO NOT CHANGE THE VALUE OF THIS VARIABLE.
@@ -401,6 +429,8 @@ for review unless they were already reviewed in the recent past?")
 (put 'org-drill-learn-fraction 'safe-local-variable 'floatp)
 (put 'org-drill-days-before-old 'safe-local-variable 'integerp)
 (put 'org-drill-overdue-interval-factor 'safe-local-variable 'floatp)
+(put 'org-drill-scope 'safe-local-variable
+     '(lambda (val) (or (symbolp val) (listp val))))
 
 
 ;;;; Utilities ================================================================
@@ -448,6 +478,7 @@ Example: (round-float 3.56755765 3) -> 3.568"
   (let ((n (expt 10 fix)))
     (/ (float (round (* floatnum n))) n)))
 
+
 (defun time-to-inactive-org-timestamp (time)
   (format-time-string
    (concat "[" (substring (cdr org-time-stamp-formats) 1 -1) "]")
@@ -456,8 +487,17 @@ Example: (round-float 3.56755765 3) -> 3.568"
 
 (defun org-map-drill-entries (func &optional scope &rest skip)
   "Like `org-map-entries', but only drill entries are processed."
-  (apply 'org-map-entries func
-         (concat "+" org-drill-question-tag) scope skip))
+  (let ((org-drill-scope (or scope org-drill-scope)))
+    (apply 'org-map-entries func
+           (concat "+" org-drill-question-tag)
+           (case org-drill-scope
+             (file nil)
+             (file-no-restriction 'file)
+             (directory
+              (directory-files (file-name-directory (buffer-file-name))
+                               t "\\.org$"))
+             (t org-drill-scope))
+           skip)))
 
 
 (defmacro with-hidden-cloze-text (&rest body)
@@ -1964,19 +2004,8 @@ Org-drill proceeds by:
   eligible topics will be presented.
 
 SCOPE determines the scope in which to search for
-questions.  It is passed to `org-map-entries', and can be any of:
-
-nil     The current buffer, respecting the restriction if any.
-        This is the default.
-tree    The subtree started with the entry at point
-file    The current buffer, without restriction
-file-with-archives
-        The current buffer, and any archives associated with it
-agenda  All agenda files
-agenda-with-archives
-        All agenda files with any archive files associated with them
- (file1 file2 ...)
-        If this is a list, all files in the list will be scanned.
+questions.  It accepts the same values as `org-drill-scope',
+which see.
 
 If RESUME-P is non-nil, resume a suspended drill session rather
 than starting a new one."
@@ -2117,6 +2146,13 @@ subtree at point."
   (org-drill 'tree))
 
 
+(defun org-drill-directory ()
+  "Run an interactive drill session using drill items from all org
+files in the same directory as the current file."
+  (interactive)
+  (org-drill 'directory))
+
+
 (defun org-drill-resume ()
   "Resume a suspended drill session. Sessions are suspended by
 exiting them with the `edit' option."
@@ -2135,7 +2171,7 @@ exiting them with the `edit' option."
 function may be useful if you want to give your collection of
 entries to someone else.  Scope defaults to the current buffer,
 and is specified by the argument SCOPE, which accepts the same
-values as `org-drill'."
+values as `org-drill-scope'."
   (interactive)
   (when (yes-or-no-p
          "Delete scheduling data from ALL items in scope: are you sure?")
@@ -2241,7 +2277,8 @@ wants to migrate to the updated set without losing their scheduling data."
        (lambda ()
          (let ((this-id (org-id-get)))
            (when this-id
-             (puthash this-id (point-marker) *org-drill-dest-id-table*))))))
+             (puthash this-id (point-marker) *org-drill-dest-id-table*))))
+       'file))
     ;; Look through all entries in source buffer.
     (with-current-buffer src
       (org-map-drill-entries
@@ -2281,7 +2318,8 @@ wants to migrate to the updated set without losing their scheduling data."
              ;; item in SRC has ID, but no matching ID in DEST.
              ;; It must be a new item that does not exist in DEST.
              ;; Copy the entire item to the *end* of DEST.
-             (org-drill-copy-entry-to-other-buffer dest)))))))))
+             (org-drill-copy-entry-to-other-buffer dest)))))
+       'file))))