@ig3/srf/README.md

Spaced Repetition Flashcards

# srf
`srf` is a web server providing
[Spaced Repetition](https://en.wikipedia.org/wiki/Spaced_repetition)
[Flashcards](https://en.wikipedia.org/wiki/Flashcard),
based loosely on [Anki](https://github.com/ankitects/anki).

It runs on
[node](https://duckduckgo.com/?t=ftsa&q=nodejs&ia=web)
with a local
[sqlite](https://www.sqlite.org/index.html)
database.
All resources being local, it can run while offline.


## Installation
```
$ npm install -g @ig3/srf
```

This provides the command `srf`.

## Operation

 1. Get an Anki deck (e.g. from [AnkiWeb](https://ankiweb.net/shared/decks/), or wherever)
 2. Import the Anki deck into srf: `srf import anki_deck.apkg`
 3. Run the srf server: `srf`
 4. Browse srf: http://localhost:8000
 5. Click the Study button (shortcut: space bar)
 6. Review the card then click Flip (shortcut: space bar)
 7. Rate the card: Click Fail, Hard, Good or Easy (shortcuts: j, k, l, ;)
 8. Repeat: review, flip then rate each card

You study cards. Each card has a front and a back. First you view the front
and try to remember what is on the back. Then you flip the card over to see
the back: to see if you remembered correctly. Finally, you rate how well
you remembered the card: 'Fail', 'Hard', 'Good' or 'Easy'.

The scheduler presents a few new cards each day. The number of new cards is
adjusted automatically, to keep total study time between 30 and 60 minutes.

When you have studied all the cards due for review and reached the limit on
new cards, you will return to the Home page, until there are more cards to
study.

The scheduler is configurable: all the limits and parameters can be
adjusted to your preferences and ability. Do you want to study more or less
each day? Change targetStudytime. Want a different limit on new cards?
Change maxNewCardsPerDay. And many more. There are very few hard coded
parameters, and even those are easy to change: just checkout the repository
and change them. It's all JavaScript, HTML, CSS and a few handlebars
templates. And if you want something more fundamentally different, you can
replace the scheduler. It was ease of tinkering with the scheduler that
motivated me to write this.

### Get an Anki Deck
There are many places to download shared Anki decks and many tools for generating them. [AnkiWeb](https://ankiweb.net/shared/decks/) is one source. Search for 'shared Anki decks' or 'generate Anki decks'

### Home page

The home page shows statistics of your study:

 * cards reviewed and minutes studied in the past 24 hours
 * cards due and predicted minutes to study in the next 24 hours
 * study time per day: a forward looking average used to determine new card
   mode and a short term (7 days) historic average
 * new cards per day: new cards in the past 24 hours and a short term
   historic average
 * percent of cards that you got correct (i.e. not rated Fail)
 * number of cards due and overdue for review
 * chart of study time per hour: past 24 hours and next 24 hours

The status at the top right includes:
 * Traffic lights for new cards:
   * Green (go): new cards at intervals and if there are no due cards
   * Yellow (slow): new cards at intervals
   * Red (stop): no new cards
 * Cards due
 * New cards in the past 24 hours
 * Minutes studied in the past 24 hours
 * Predicted minutes to study in the next 24 hours

To study a card, click the Study button or press the space bar.

The study button will always present a card for study, ignoring the limits
of the scheduler.

The space bar will present a card for study or reload the home page,
according to the determination of the scheduler. 

The buttons at the bottom of the page are:

 * Admin: to access the admin page
 * Stats: to review various statistics of your study history
 * Help: to review the help file - mostly a link to this README

### Card Front

The content of the card front is variable.

Status at top right is the same as the home page.

At the bottom of the page are a set of buttons:

 * Flip: to flip the card over to the back (shortcut: space bar)
 * Help: to review the help file
 * Edit: to edit the card content
 * Play: to replay audio, if the card has audio
 * Stop: to stop studying and return to the home page

### Card Back

The content of the card back is also variable.

Status at top right is the same as the home page.

At the bottom of the page are a set of buttons:

 * Fail: if you couldn't remember the card (shorcut: j)
 * Hard: if you could remember the card but it was hard (shortcut: k)
 * Good: if you could remember the card reasonably well (shortcut: l)
 * Easy: if you could remember the card very easily (shortcut: ;)
 * Help: to view the help file
 * Edit: to edit the card content
 * Play: to replay audio, if the card has audio
 * Stop: to stop studying and return to the home page

After you rate the card (Fail, Hard, Good or Easy) the front of the next
card will be presented, you you will be returned to the home page if there
are no more cards to study.

### Creating Cards

There are several ways to create cards:
 * Import Anki .apkg or .colpkg file
 * Import CSV files
 * Add them one at a time via the browser
 * Direct database update

Cards are the basic elements of study. They are produced from fieldsets,
transformed by templates.

A fieldset is a set of name/value pairs that you want to remember in
relationship to each other, stored as JSON.

For example:

```
{
  "Country": "Canada',
  "Capital": "Ottawa",
  "Continent": "North America"
  "Area': "over 9.98 million square kilometres",
  "Population": "38 million"
}
```

A template is a pair of Mustache templates for the front and back of a card
that include some of the fields in the fieldset, and some css for styling
the card.

For example:

```
Front: The <span class="keyword">capital</span> of {{Country}} is?
Back: {{Capital}}
css: .keyword { font-style: italic; }
```

A single template can be used to produce cards from many fieldsets.

Templates are grouped into sets identified by templateset name.

Each fieldset is related to a templateset and one card is produced for each
template in the templateset.

You study the cards. The cards are produced automatically when you add or
edit fieldsets or templates.

The order that cards are presented as new cards is determined by their
ordinal (ord), which is copied from the fieldset.

#### Create Cards by importing an Anki .apkg or .colpkg file

The easiest way to create cards is to import an Anki
[Packaged Deck](https://docs.ankiweb.net/exporting.html). This will import
the Anki cards, notes and note types into srf. However, srf does not
support all the features of Anki so not all Anki decks will work in srf.
For example, srf does not support cloze deletion so any deck that includes
cloze deletion notes will not work.

If you have been using Anki, you can export your own decks. Be sure ti
include media when you export your decks. If you include scheduling
information, it will be preserved.

Anki collection packages and shared decks can include review logs. These
will be imported and merged with existing logs. Daily stats will be cleared
and recalculated to including the new revlog entries.

You can download a shared Anki deck from
[AnkiWeb](https://ankiweb.net/shared/decks/) or use various tools to
produce Anki Packaged Deck files from other sources.

Srf doesn't distinguish decks. If you import multiple decks or a deck
collection, all the cards will appear in a single pool of cards in srf. If
you want to keep decks separate, you can import each deck into a separate
srf database.

Note that srf only supports simple cards. In particular, Cloze notes are not
supported. See below for what is/isn't supported. Start with something
basic.

To import an Anki packaged deck:

```
$ srf import <shared_deck.apkg>
```

To import an Anki deck collection:

```
$ srf import <deck_collection.colpkg>
```

#### Create Cards by importing CSV files

Create two CSV files:
 * templates.csv
 * fieldsets.csv

Import each file with:

```
$ srf import templates.csv
$ srf import fieldsets.csv
```

##### templates.csv

The templates determine which cards are produced and how the are presented.
One card is produced for each template with matching templateset value.

The fields for a template are:
 * templateset - the name of the templateset, must match same on fieldset
 * name - the name of the template
 * front - the mustache template code for the front of the card
 * back - the mustache template code for the back of the card
 * css - the CSS for rendering the card, common to front and back

Make a CSV file with these headings and appropriate data.

For example:

```
templateset,name,front,back,css
Basic,Card 1,{{Front}},{{Back}},".card { Background-color: red; }"
Basic,Card 2,{{Back}},{{Front}},".card { Background-color: red; }"
```

You can create many templatesets and each templateset can have as many
templates as you like. One card will be produced for each template with
templateset matching that of a fieldset.

##### fieldsets.csv

The fields for a fieldset are:
 * guid - optional guid that uniquely identifies the fieldset
 * templateset - the templateset for rendering the fieldset
 * fields - the fields data as a JSON string
 * ord - an ordinal number for sorting the fieldsets

guid will default to md5 checksum of the concatenation of the templateset
and fields values.

ord will default to the row number * 10.

For example:

```
templateset,fields
Test1,"{""Front"": ""Who you gonna call?"", ""Back"": ""Ghost Busters!""}"
Test1,"{""Front"": ""Where's Waldo?"", ""Back"": ""Hold on a sec... I'm still looking.""}"
```

You can have as many fields as you want in your templates and fieldsets.

Read below for details of all the options for the templates. They are
[Mustache](https://github.com/janl/mustache.js) templates with all the
field values available.

##### Add media files

If the fieldsets refer to media files (images, audio, video, etc.), you
will have to copy these to the media directory: ~/.local/share/srf/media by
default.

For example, a field might include an image with something like:
`<img src=\"my-image.png\" />`, or an audio file with something like:
`[sound:my-audio.mp3]`.

#### Create Cards Manually

You can create templates and fieldsets manually in the browser. The
interface is crude, but it's an option.

On the home page, click Admin to view the admin page.

On the admin page, click Templates, Template Sets or Field sets to see a
list of the corresponding items.

Click on an item in the list to edit it, or click the New button to add a
new item to the list.

When you edit a fieldset, the fields you can set are determined from the
templates in the selected templateset: every field included in any template
in the templateset. You can't set fields that are not included in any
template. If you want to set another field, include it in at least one of
the templates.

#### Create Cards by direct database update

The database structure is simple. You can write your own application to add
cards to the srf database.

The database is [sqlite3](https://www.sqlite.org/index.html).

By default, the database is at ~/.local/share/srf/srf.db and media files
are in ~/.local/share/srf/media.

Add templates by adding records to the template table.

Add fieldsets by adding records to the fieldset table.

See [the database description](#srf-database) for a description of the database
and details of these tables.

After modifying template or fieldset tables, generate missing cards and
delete orphaned cards by running fix:

```
$ srf fix
```

### Run the server

```
$ srf
Listening on http://:::8000
```

Alternatively, create a systemd service file
(e.g. ~/.config/systemd/user/srf.service) similar to:

```
[Unit]
Description=Spaced Repetition Flashcards

[Service]
Type=simple
Restart=on-failure
WorkingDirectory=/tmp
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=srf
ExecStart=bash -l -c srf

[Install]
WantedBy=default.target
```

Then enable and run the service:

```
$ systemctl --user enable srf
$ ssytemctl --user start srf
```

With this done, your srf server will run whenever you login.


### Study!

Browse to [http://localhost:8000/](http://localhost:8000/)

Click the Study button or press the space bar to study a card.

#### Front

When you study a card, the front of the card is displayed. The content of
the front of the card depends on the template and the fieldset.

When you are ready, you flip the card to see the back of the card. See
below for details of the back of the card.

##### Flip

Keyboard shortcut: space-bar

Flip the card over, to show the back of the card. This is the usual action
when reviewing a card: view the front, then flip it to review the back and
rate the card.

##### Help

Show the help file.

##### edit

Edit the card. This opens a new window with fields to edit the fieldset
from which the card is derived. It is a bit crude but allows simple edits.

##### Play

keyboard shortcut: p

If the card has audio, play it again.

##### Stop

Return to the home page without completing review of the card.

#### Back

After you view the front of the card and flip it over, the back is
displayed. The content of the back is determined by the template and the
fieldset.

The back of the card has buttons for rating how well you remember the card:
Fail, Hard, Good and Easy.

The button you click determines when the card will be scheduled for review.
Each button displays the new interval: the time until the card will be due
for review.


##### Fail

keyboard shortcut: j

If you didn't remember the card and want to review it again soon, click
Fail. This reduces the interval for reviewing the card. The new interval
(the time until the card is due for review) is half the previous interval
with a maximum interval of 1 day.

##### Hard

keyboard shortcut: k

If you did remember the card but it was difficult to remember, click Hard.
This reduces the interval for rewviewing the card but not as much as Fail.
The new interval is 80% of the previous inerval with a maximum interval of
1 week.

##### Good

keyboard shortcut: l

If you remembered the card well, click Good. This will, hopefully, be the
button you click most often. This increases the interval for reviewing the
card. 

Calculation of the new interval is more complicated for Good. The minimum
new interval is 5 minutes and the maximum new interval is 1 year. Between
these limits, the new interval depends on the ease factor of the card. The
ease factor is the exponentially weighted moving average of your previous
ratings. The weights are 0, 1, 1.5 and 2 for Fail, Hard, Good and Easy
respectively. Thus the factor ranges from 0 to 2 and it will change a bit
each time you review the card.

If you rate the card Good several times, the ease factor will tend towards
1.5 and the new interval will tend towards 150% of the previous interval.

On the other hand, if you have rated the card Fail many times, the ease
factor might be quite low and even if you rate the card Good, the new
interval might be less than the previous interval. However, even in this
case, the ease factor will increase so that after a few Good reviews the
interval will be increasing.

##### Easy

keyboard shortcut: ;

If it was too easy to remember the card and you don't want to waste your
time reviewing again too soon, click Easy. The new interval will be 150% of
what it would be for Good. 

##### Help

Show the help file.

##### Edit

Edit the templateset the card is generated from.

##### Play

Replay any audio for the back of the card.

##### Stop

Stop studying without completing review of the card: return to the home
page.


## Background
I used [Anki](https://github.com/ankitects/anki) for a couple of years but
wanted to change the scheduler. An Anki addon to add a new scheduler
was impossible because Anki addons are Python code but the scheduler
implementation is partially in Python and partially in Rust back-end code.
Even limited modifications to the existing schedulers
(e.g. [Anki - limit new](https://github.com/ig3/anki-limitnew))
were difficult and time consuming to maintain due to frequent changes
to the Anki internals. It was easier to write this and get the scheduler I
wanted than to maintain the Anki addon that provided only part of what I
wanted.

The srf scheduler:
 * prioritizes cards with shorter intervals over those with longer intervals
 * regulates the introduction of new cards automatically
 * is written entirely in JavaScript and is easy to modify

srf is able to import Anki decks that use only basic features of Anki. Not
all field and media types are supported, but enough for many decks / cards
to work.

[Spaced Repetition for Efficient Learning](https://gwern.net/spaced-repetition)
provides an overview and links to research on spaced repetition.

### Comparison to Anki

srf provides only a small subset of the features of Anki: enough for my
purposes: studying language on a single device, but without support for
synchronizing multiple devices or distinguishing decks / multiple topics of
study. 

The srf scheduler does not suffer from
[Ease Hell](https://readbroca.com/anki/ease-hell/).

The srf scheduler has an equivalent to Anki ease factor for each card
but it is simpler: it is the
[exponentially weighted moving average](https://en.wikipedia.org/wiki/Moving_average#Exponential_moving_average)
of the weighted answers. The default answer weights are 0, 1, 1.5 and 2 for
Fail, Hard, Good and Easy and the default decay factor is 0.95, but these
are all configurable. By default, the range of the ease factor is 0.0
through 4.0: somewhat equivalent to Anki ease factors 0% to 200%. With srf
the ease factor never gets 'stuck' at an unreasonably low setting. It
always adapts to the recent ease of the card.

The srf scheduler does not suffer from
[Backlog Hell](https://iansworld-nz.blogspot.com/2022/04/anki-backlog-hell.html).

The srf scheduler has two ways of sorting cards:

 * by ascending interval
 * by ascending due time

The sort is determined randomly each time a due card is selected with the
probability of sorting by due time being config.probabilityOldestDue, which
defaults to 0.2.

This strategy balances studying cards in they order they become due with
focusing on the cards with minimum interval. It makes a difference only
when clearing a backlog. It allows the cards with the shortest intervals to
be studied on time, despite the backlog. At the same time, it allows the
most overdue cards to be studied, gradually clearing the backlog.

New cards are presented before due cards if total study time the past 24
hours is less than config.minStudyTime.

New cards are presented at intervals, interleaved with due cards, if total
study time the past 24 hours is less than config.targetStudytime.

## Getting Started

```
$ git clone https://github.com/ig3/srf.git
$ cd srf
$ npm install
$ node bin/cmd.js import <export file>
$ node bin/cmd.js
```
The import supports Anki exports and shared decks (.apkg files). When I
migrated from Anki, I exported all decks as an Anki deck package (\*.apkg),
including media, then imported this into srf.

Browse to http://localhost:8000/.

### Home Page

The home page presents some basic study statistics:

 * The number of cards reviewed and minutes studied in the past 24 hours
 * The number of cards due and estimated minutes to study in the next 24
   hours
 * Study time per day: short term used to determine new card mode /
   historic 7 days average
 * New cards per day: past 24 hours / historic average
 * New cards during the past 24 hours
 * The percentage of correct (not 'Fail') responses to mature cards in the
   past 14 days
 * The number of cards due and overdue (due more than 24 hours ago)
 * The time until the next card is due
 * A histogram of study time per hour through past and next 24 hours

If there is a card available for study, the 'Study' button will appear.
Click it to study a card. After studying a card, the next card will be
presented, until there are no more cards to be studied and the home page is
displayed again. Alternatively, click the space bar: shortcut for Study.

The scheduler determines when a card is due to be studied. Each time a card
is reviewed, the scheduler sets a new time when it is due to be studied
again. The cards to be studied are all those which are past their due 
time. If it is more than 24 hours past their due date and time, they
are considered overdue. 

It's like an [assembly line](https://www.youtube.com/watch?v=59BIB-2FVmM):
you will sometimes be waiting for the next card to study, and you will
sometimes have a backlog of cards to catch up. While you don't want to get
too far behind, it is OK to accumulate cards to review. Studying to clear
the backlog once per day will work well. While there is nothing wrong with
studying in multiple sessions each day, don't obsess about reviewing every
card that comes due each day. It is OK to leave them until the next day, or
even longer.

The data is in ~/.local/share/srf by default, including database (srf.db)
and media files. The database is ~/.local/share/srf/srf.db. Media files
are in ~/.local/share/srf/media.

At the top right of the home page and front and back pages, there is a
status indicator. Green, yellow or red depending on new card mode: Green if
new cards will be presented interleaved with due cards and if there are no
cards due; Yellow if new cards will be presented interleaved with due
cards; and Red if no new cards will be presented. Below this are counts of
cards due, minutes of study the past 24 hours and minutes of study the next
24 hours (estimated). 

#### Buttons

##### Study

Study a card. This will select a card according to the scheduling algorithm
if there is a card due to be studied. If no card is due to be selected, a
new card will be presented. If there are no more unseen cards, then the
card with the earliest due date will be presented, even if that is in the
future. This button will always yield a card to study.

##### Reload

Reload the home page.

##### Admin

View the admin page, with links to administer templates, template sets and
field sets, and view configuration.

##### Stats

View the statistics page.

##### Help

This will be documentation, if I ever write it.

### Admin
Links to the administration pages:

#### Templates

View a list of all templates in the system. Click on a template to edit it.

Each template is linked to a template set and should have a unique name
among the templates linked to that template set. There are three
attributes:

Front: handlebars template code for the front of the card.

Back: handlebars template code for the back of the card.

CSS: CSS that will be loaded to the front and back.

All the fields of the template/fieldset are available. For example, if the
fieldset includes a field `English` then include this in the template with
`{{English}}`.

The set of fields are defined in the template set definition.

#### Template Sets

View a list of all template sets in the system.

The view is read-only. To change them, add or edit the templates.

The fields of a templateset are all the fields of all the templates in the
templateset. If you add a field to one of the templates, it will be added
to the templateset.

#### Field Sets

View a list of all field sets in the system. Click on a field set to edit
it. Click the New button to add a field set.

A field set is a collection of field values that populate cards. The set of
fields is defined in the linked template set.

Each fieldset must be linked to a Template set. The set of fields is
defined in the template set. If you change the template set after setting
field data, the field data will be lost.

To attach media files, put the cursor into the field value input where the
media file is to be attached, then click the Attach button and select the
file containing the media data.

At the moment, the only supported media types are image/jpeg and
audio/mpeg. These are inserted as img tags and `audio` respectively.

#### Config

A read-only view of the loaded configuration.

### Stats Page

The Stats Page is available from the Home Page by clicking the Stats
button. It shows some basic statistics about study performance.

#### New cards per day

This is the number of new cards viewed in the current day,
short term (past 7 days) average of new cards per day and
long term (since the beginning of study) average of new cards per day.

#### Cards

This is various counts of cards:
 * Total cards, broken down into:
     * Unseen cards,
     * Seen cards, broken down into:
         * New cards,
         * Learning cards,
         * Mature cards,
         * Mastered cards

#### Percent Correct

This is the percentage of 'correct' responses (a.k.a. not Fail) for cards
with intervals between config.matureThreshold and config.maxInterval,
reviewed in the window config.percentCorrectWindow.

Cards with maximum interval are excluded on the premise that they are
mastered and it is not performance on these cards that should be regulated,
but rather performance on mature cards (Unconcious Competence). 

Maybe this should also include the learning cards, but these are excluded
on the premise that while learning the error rate will be relativley high
and the regulation effectively regulates how quickly cards progress through
learning to mastered, even though the learning cards are not included in
the calculation of percent correct.

Card intervals are adjusted according to the difference between the average
percent correct and config.percentCorrectTarget. If average percent correct
is less than the target then intervals are shortened, making cards due
sooner. If it is more than the target then intervals are lengthened, making
cards due later.

#### Average time per review

This is the average time for each review of a card. It includes small gaps
between reviews. It is the total study time for the day divided by the
number of reviews.

#### Cards today

This is the total number of distinct cards viewed today and the number of
cards due later in the same day. The day is the calendar day in local time.

The total number of cards viewed today is a count of cards not reviews. A
card might be reviewed multiple times in a day but this counts the cards,
not the reviews.

The cards due later in the same day are those cards scheduled for review
before the end of the day in local time. This includes cards that are
currently due or overdue. This does not included cards that will be
rescheduled to be viewed again later in the same day.

#### Reviews per new card

This is the number of reviews to be completed before the next new card will
be presented, counting down at each review, and the current minimum number
of cards to be reviewed between each new card.

#### Study time today

This is the total time spent studying today.

#### Average study time per day

This is average daily study time, averaged over the past week (actual study
time) and coming week (estimated based on cards due and recent
performance).

#### Time to earliest due card

The time until the earliest due card is due. This will be negative if there
is a card due in the past: if there is currently one or more cards due for
study.

#### Charts

##### New Cards per day

This is the number of new cards presented each day since the start of study

##### Card Views per day

This is the total number of card views each day since the start of study.

##### Minutes Studied per day

This is the total study time per day since the start of study.

##### Cards per Stage

This is a stacked chart of cards per stage: New, Learning, Matured and
Mastered. The stages are distinguished by interval.

 * New: 0 < interval < config.learningThreshold
 * Learning: config.learningThreshold <= interval < config.matureThreshold
 * Mature: config.matureThreshold <= interval < config.maxInterval
 * Mastered: config.maxInterval <= interval

##### Cards Due per day

This is the number of cards due for review each day, looking forward from
today.

##### Cards per interval

This is the number of cards with the given interval in days. There are two
lines: Adjusted Interval and Unadjusted Interval. The Adjusted Interval is
the interval after adjustment for percent correct. The Unadjusted Interval
is the interval as set when the card was last viewed. The adjusted interval
is the interval that will be the basis for calculation of the interval
after the next review.

## Command Synopsis

```
usage:
  srf --help
  srf [options...] [run]
  srf [options...] import <filename>
  srf [options...] backup
  srf [options...] fix
```

### run

Run the server. This is the default.

### import

Import an Anki apkg or colpkg file, or a csv file containing template or
fieldset data.

### backup

Create a backup of the srf database.

The server makes a backup of the database each time it starts and then once
every 24 hours if it runs that long.

The backups are in the same directory as the database, with timestamp
appended to their name.

### fix

This performs several 'fixes':

 * create any missing cards from fieldsets and templates and delete any
   cards for which there is no longer a fieldset and template.
 * fix revlog entries:
   * make revlog IDs unique and monotonic increasing
   * set 'lastinterval' to 'interval' of the previous revlog entry for the
     card
   * set 'interval' of the last revlog entry for a card to the 'lastinterval'
     of the card
   * set revdate according to the id
   * add missing counts of cards by stage to dailystats

### options

#### --help|-h
Display usage and exit.

#### --port|-p
Set the port that the srf server listens on.

Default is 8000.
#### --directory|--dir
Set the root directory, in which the database and media files are located.

Default is ~/.local/share/srf

If the given directory is absolute (i.e. begins with '/') it is used as
given. If it is relative (i.e. does not begin with '/') it is relative to
`~/.local/share`.

For example: `/tmp/testing` would use that directory but `testing` would
use `~/.local/share/testing` and `testing/a` would use
`~/.local/share/testing/a`.

Media files are located in the `media` subdirectory of this directory.

The srf database is, by default, `srf.db` in this directory, but see option
`--database` below.

#### --config|-c
Set the configuration filename.

Default is `config.json`

If the given path is absolute (i.e. begins with '/') it is used as given.
If it is relative (i.e. does not begin with '/') it is relative to the path
of the --directory option.

#### --database|--db
Set the sqlite3 database name.

Default is ~/.local/share/srf/srf.db or, if --directory is specified then
srf.db in that directory.

If the given path is absolute (i.e. begins with '/') it is used as given -
the database file can be outside the data directory. If it is relative,
then it is relative to the path of the --directory option.

The directory to contain the database is created if it doesn't exist.

#### --media
Set the path of the media directory.

Default is `media`

If the given path is absolute (i.e. begins with '/') it is used as given.
If it is relative (i.e. does not begin with '/') it is relative to the path
of the --directory option.

The directory is created if it doesn't exist.

#### --htdocs
Set the path of the htdocs directory.

Default is `htdocs`

If the given path is absolute (i.e. it begins with '/') it is used as
given. If it is relative (i.e. does not begin with '/') it is relative to
the path of the --directory option.

This directory contains overrides for the static content of the srf server.

 * css/dark.css
 * img/dark-go.png
 * img/dark-slow.png
 * img/dark-stop.png
 * img/favicon.png
 * img/logo.png
 * js/plotly-latest.min.js
 * js/sorttable.js

#### --scheduler

Specify the schedulre plugin to load. The argument will be passed to
require() to load the plugin, so it must be something that can be required.
Details will depend on how srf and the plugin are installed but the name of
a globally installed package or the full path to the scheduler module
should both work.

#### --views
Set the path of the views directory.

Default is `views`

If the given path is absolute (i.e. it begins with '/') it is used as
given. If it is relative (i.e. does not begin with '/') it is relative to
the path of the --directory option.

This directory contains overrides for the handlebars templates of the srf
server. Templates in this folder will override the default templates.

 * back.handlebars
 * fieldset.handlebars
 * front.handlebars
 * help.handlebars
 * home.handlebars
 * stats.handlebars
 * template.handlebars
 * templates.handlebars

### Config

Configuration files may be put in several places:

 * /etc/srf
 * /etc/srf.ini
 * /etc/srf.json
 * /etc/srf/config
 * /etc/srf/config.ini
 * /etc/srf/config.json
 * ~/.config/srf
 * ~/.config/srf.ini
 * ~/.config/srf.json
 * ~/.config/srf/config
 * ~/.config/srf/config.ini
 * ~/.config/srf/config.json
 * ~/.srf
 * ~/.srf.ini
 * ~/.srf.json
 * ~/.srf/config
 * ~/.srf/config.ini
 * ~/.srf/config.json
 * .srf
 * .srf.ini
 * .srf.json

And, finally, `config.json` in the directory containing the data
(~/.local/share/srf by default but may be set by option --directory).

Files with extension `.json` and files without extension are parsed as JSON
after stripping comments from them.

Files with extension `.ini` are parsed as `ini` files.

Parameters which are durations may be specified as integer seconds or a
string with units: `seconds`, `minutes`, `hours`, `days`, `weeks` or
`years`, or any prefix of one of these. The units may be separated from the
number by spaces.

For example:
 * 10 // 10 seconds
 * "10 seconds"
 * "5 days"
 * "1 day"
 * "1 d"
 * "1d"

Other parameters are integers.

For example, a json file might be:

```
{
  // Display theme
  "theme": "dark",

  // Minimum time between related cards (seconds)
  "minTimeBetweenRelatedCards": "1 hour"

  // Backup retention time (milliseconds)
  "backupRetention": "30 days",

  // Minimum number of backups to keep
  "minBackups": 2,

  // Maximum number of backups to keep
  "maxBackups": 10,

  // The maximum time for viewing a card (seconds).
  // Beyond this, any answer is converted to 'again'
  "maxViewTime": "2 minutes",

  // The maximum interval to when a card is due.
  "maxInterval": "1 year",
  "maxGoodInterval": "1 year',
  "maxEasyInterval": "1 year",

  // The interval (seconds) beyond which a card is considered 'learning'
  "learningThreshold": '1 week',

  // The interval (seconds) beyond which a card is considered 'mature'
  "matureThreshold": "21 days",

  // The window (seconds) in which to average Percent Correct reviews
  "percentCorrectWindow": "1 month",

  // The window (seconds) in which to average new cards per day
  "newCardsWindow": "2 weeks",

  // The minimum number of mature cards in the percent correct window
  // at which percent correct is calculated.
  "minPercentCorrectCount": 10,

  // The target for percent correct reviews
  "percentCorrectTarget": 90,
  "percentCorrectSensitivity": 0.0001,

  // The interval (seconds) between correct factor adjustments
  "correctFactorAdjustmentInterval": "1 day",

  // The maximum number of new cards in 24 hours.
  "maxNewCardsPerDay": 20,

  // Minimum study time (seconds) per day
  "minStudyTime": "20 minutes",

  // Target study time (seconds) per day
  "targetStudyTime": "30 minutes",

  // The probability of sorting due cards by due instead of inteval
  "probabilityOldestDue": 0.2,

  // minimum intervals according to responses to reviews
  "againMaxInterval": "1 day",
  "hardMinInterval": "1 week",
  "goodMinInterval": "5 minutes",
  "goodMinFactor": 1.1,
  "easyMinInterval": "1 days",

  // Static interval factors
  "againFactor": 0.5,
  "hardFactor": 0.8,
  "goodFactor": 1.0,
  "easyFactor": 1.5,

  // Answer weights
  "weightFail": 0,
  "weightHard": 1,
  "weightGood": 1.5,
  "weightEasy": 2,
  "decayFactor": 0.95
}
```

#### theme

default: dark

There is only one theme: dark. But this is only a CSS file. This setting is
just the name of the CSS file, less the `.css` extension.

#### minTimeBetweenRelatedCards (seconds)

default: 1 hour

A template set typically contains several templates. For each field set, a
card will be produced for each template in the template set. Thus there
will be several cards for each field set.

The set of cards from a single field set are considered related. If one
card from the set is reviewed, this is the minimum time before any other
card from the set will be presented for review.

####  backupRetention

default: 30 days

The time to retain database backups. Backups older than this will be
deleted.

#### minBackups

default: 2

The minimum number of backups to retain, regardless of their age.

#### maxBackups

default: 10

The maximum number of backups to retain, regardless of their age.

#### maxViewTime (seconds)

default: 2 minutes

The maximum time for viewing a card. If a card is viewed for longer than
this ease will be forced to 'again'.

#### maxInterval (seconds)

default: 1 year

The maximum interval (time until next review) for a card.

#### maxGoodInterval (seconds)

default: 1 year

The maximum interval (time until next review) for a card that is Good.

#### maxEasyInterval (seconds)

default: 1 year

The maximum interval (time until next review) for a card that is Easy.

#### learningThreshold (seconds)

default: 1 week

The interval beyond which cards are considered to be 'learning' cards
rather than 'new' cards. Beyond this threshold, they are scheduled
according to the actual interval since they were last reviewed, rather than
the scheduled interval.

#### matureThreshold (seconds)

default: 21 days

This affects the calculation of Percent Correct, which is compared against
percentCorrectTarget. Only review of cards with an interval greater than
matureThreshold are considered in calculating Percent Correct.

A card is counted as mature (Unconcious Competence) or mastered if its
interval is greater than matureThreshold.

#### percentCorrectWindow (seconds)

default: 1 month

The percentage of 'correct' responses (not 'Fail') is a factor in
determining the intervals of cards. All responses within this window are
considered in determining the percentage. Results of reviews longer ago
than percentCorrectWindow are ignored.

#### newCardsWindow (seconds)

default: 2 weeks

The window in which new cards per day are averaged. The average new cards
per day is used to calculate the minimum interval between new cards.

#### minPercentCorrectCount

default: 10

The minimum number of mature cards in the percent correct window at which
percent correct is calculated.

#### percentCorrectTarget (percent)

default: 90

The percentage of 'correct' responses (not 'Fail') is a factor in
determining the intervals of cards. The percentCorrectTarget is the target
percentage of 'correct' responses.

The interval and due date of cards are adjusted according to the difference
between the Percent Correct and this target, multiplied by
percentCorrectSensitivity.

Cards actually have two interval values: interval and lastinterval. The
interval value is the interval, including any adjustments. The lastinterval
value is what the interval was when the card was last reviewed, without any
adjustments.

Revlog also has two interval values: interval and lastinterval. But here
their values are different. The value of interval is the new interval of
the card, after review, based on the ease. The value of lastinterval is the
unadjusted interval of the card the last time it was reviewed. It is
redundant with the interval value of the previous revlog record for the
card but saves a lot of lookup producing the statistics.

#### percentCorrectSensitivity

default: 0.0001

This determines the sensitivity to the difference between Percent Correct
and percentCorrectTarget, when adjusting the interval and due date of
learning and mature cards.

#### maxNewCardsPerDay

default: 20

The maximum number of new cards to be presented within 24 hours.

#### minStudyTime

default: 20 minutes

Minimum daily study time. New cards will be presented until study time in
the past 24 hours is more than this.

#### targetStudyTime

default: 30 minutes

If total study time for the past 24 hours and predicted study time to study
all cards due in the next 24 hours are both less than this, then new cards
will be presented, interleaved with due cards. If either exceeds this, then
only due cards will be presented.

#### probabilityOldestDue

default: 0.2

The probability that the next due card presented will be the first one due
rather than the one with the shortest interval.

#### againMaxInterval (seconds)

default: 1 day

The maximum interval for cards after response 'Fail'.

#### hardMaxInterval (seconds)

default: 1 week

This is the maximum interval after responding 'Hard' to a review.

#### goodMinInterval (seconds)

default: 5 minutes

This is the minimum interval after responding 'Good' to a review.

#### goodMinFactor 

default: 1.1

This is the minimum interval multiplier after responding 'Good' to a review.

#### easyMinInterval (seconds)

default: 1 day

This is the minimum interval after responding 'Easy' to a review.

#### againFactor

default: 0.5

After responding 'Fail' to a review, the new interval is the previous
interval multiplied by this factor, but with a maximum of againMaxInterval
which, by default, is 1 day.

#### hardFactor

default: 0.8

After responding 'Hard' to a review, the new interval is the previous
interval multiplied by this factor, but with a maximum of hardMaxInterval
which, by default, is 1 week.

#### goodFactor

default: 1.0

After responding 'Good' to a review, the new interval is the previous
interval multiplied by this factor, the card factor and the 'correct'
factor. See 'scheduler' below for details.

#### easyFactor

default: 1.5

After responding 'Easy' to a review, the new interval is the actual
interval since last review (i.e. time since last review, which may be more
than the scheduled interval) multiplied by this factor and the card factor.

#### weightFail

default: 0

The weight of an answer of Fail when calculating the exponentially
weighted moving average of review replies: the card ease factor.

#### weightHard

default: 1

The weight of an answer of Hard when calculating the exponentially
weighted moving average of review replies: the card ease factor.

#### weightGood

default: 1.5

The weight of an answer of Good when calculating the exponentially
weighted moving average of review replies: the card ease factor.

#### weightEasy

default: 2

The weight of an answer of Easy when calculating the exponentially
weighted moving average of review replies: the card ease factor.

#### decayFactor

default: 0.95

The decay factor when calculating the exponentially weighted moving average
of review replies: the card ease factor.

ewma(n) = w(n-1) * decayFactor + w(n) * (1 - decayFactor)

### Commands

#### run

Run the web server. This is the default command (i.e. if srf is run without
specifying a command on the command line, it runs the webserver).

#### import \<filename>

Import an Anki export (i.e. a .apkg file). This is the primary way to get
cards into srf. To migrate from Anki to srf, export decks from Anki,
including media, and then import them to srf. Exporting all decks works.

#### fix

Fix a few inconsistencies in revlog:

 * duplicate IDs
 * inconsistency between interval and lastinterval
 * inconsistency between interval and card interval

This isn't necessary but it ensures greater consistency between the various
srf metrics, particularly related to matured cards.

## Study

When you study a card, its front side will be presented. 

Review the front side and try to recall the corresponding back side.

When you are ready, click the Flip button or click the space bar (shortcut
for Flip).

The back side of the card will be displayed, with buttons to indicate how
well you remembered the card:

 * Fail: you didn't remember the card - you need to see it again soon
 * Hard: you remembered the card but it was a bit hard to recall
 * Good: you remembered the card 
 * Easy: you remembered the card but it was too easy

What you should remember, when you see the front side, and what constitutes
an adequate recollection is up to you. Some cards may be very specific and
fact based while others might be more vague or conceptual. It's up to your
own judgement.

Keyboard shortcuts for these are 'j', 'k', 'l' and ';' respectively. These
are hard coded, but it is easy to edit the templates in the views directory
if you want different shortcuts.

The card is then scheduled for review according to which button you
clicked.

If there is another card due for study, the front of it is displayed and
you continue your study.

If there are no more cards to be studied at this time the home page is
displayed. You can review your progress and return to study when additional
cards come due for study.

Every card that is not 'new' has a time when it is due. If this is before
the current time, you may study the card. If it is in the future, you may
not study the card. You must wait until the card is due for review to study
it.

New cards are due if total study time over the past 24 hours is less than
config.minStudyTime.

## Card Lifecycle

Cards progress through the following stages:
 * unseen / unconcious incompetence / UI
 * new / concious incompetence / CI
 * learning / concious competence / CC
 * mature / unconcious competence / UC
 * mastered / mastery / M

### Unseen / unconcious incompetence / UI

When you add cards to srf, initially they are unseen. If you add a thousand
cards, you probably don't want to study them all on the day you add them.
Most of them remain unseen for some time before they are presented as new
cards.

If you look at the database. These unseen cards will have an interval of 0.
Cards that have been seen have an interval greater than 0. Even if these
cards have a due time set, the due time is ignored until they are selected
to be presented as new cards.

If the scheduler determines that a new card should be presented, cards with
interval = 0 are sorted by ord then id and the first card is presented.

### New / concious incompetence / CI

Eventually an unseen card is presented for study for the first time,
becoming a 'new' card. It is deemed to be a new card until its interval
reaches the learning threshold (config.learningThreshold).

At this stage, your ability to remember the card might be quite volatile.
On the one hand, the novelty of it might make it easier to remember. On the
other hand, the unfamiliarity of it might make it harder to rememger.

New cards are scheduled according to their scheduled interval, not the
actual interval since they were last studied. If a card has a scheduled
interval of 30 minutes but you don't study overnight so you don't see it
until 10 hours later, it is scheduled on the basis of the scheduled 30
minute interval, not the 10 hour interval.

### Learning / concious competence / CC

After familiarizing with a new card, the real work of committing it to long
term memory begins. Intervals are still fairly short and your ability to
recall the card might be quite sensitive to the actual interval. Reviewing
the card a few days late might make the card much more difficult to recall.

During this phase, the card is scheduled according to the actual interval
since last review. If you don't review it on time (maybe because of exams
or vaction) but when you do you recall it well, it is the actual interval
through which you were able to recall it that matters.

### Mature / unconcious competence / UC

Eventually the card is well committed to long term memory. When you see the
card, you recognize it immediately, without having to think about it. The
the exact interval doesn't make much difference.

By default, cards are considered mature when their interval reaches 21
days. The scheduling algorithm is still applied and the interval will still
gradually increase, up to the maximum interval (config.maxInterval) which,
by default, is one year.

### Mastered / mastery / M

Finally, cards reach the interval limit. They are mastered. 

By default, the interval limit is 1 year (config.maxInterval). 

## Scheduler

The scheduler is provided by a separate package.

The default scheduler is
[@ig3/srf-scheduler](https://www.npmjs.com/package/@ig3/srf-scheduler).

The scheduler can be specified by command line argument `--scheduler`, the
value of which is the ID or full path of the scheduler module to be loaded.

### @ig3/srf-scheduler

There are two aspects to the scheduler:

 1. Determining when a card is due for review
 2. Determining which card is to be studied next

### Determining when a card is due to be studied

Cards that have been viewed at least once have a time that they are due for
review. Technically, this is stored in the database as a date and time in
UTC timezone.

Cards that have never been viewed are called 'new' cards. They do not have
a time that they are due for review. They have not been viewed and
therefore cannot be reviewed, until after they have been viewed for the
first time. For details of when new cards are first presented, see the next
section. 

The timing, from one review to the next, is the spacing of spaced
repetition. The theory is that there is an optimum interval that minimizes
the total number of reviews required until a card can be remembered
indefinitely: until it has been learned. If the interval is too long, the
card is forgotten before it is reviewed: it is not learned. If the interval
is too short, the card is easily remembered but reviewing the card too
often wastes time that could be used to learn other cards.

Each time a card is studied, whether it is its initial viewing or a later
review, the ease with which it was remembered is r