docco-next

<!DOCTYPE html> <html> <head> <title>Docco Next</title> <meta http-equiv="content-type" content="text/html; charset=UTF-8"> <meta name="viewport" content="width=device-width, target-densitydpi=160dpi, initial-scale=1.0; maximum-scale=1.0; user-scalable=0;"> <link rel="stylesheet" media="all" href="docco.css" /> </head> <body> <div id="container"> <div id="background"></div> <ul class="sections"> <li id="section-1"> <div class="annotation"> <div class="sswrap "> <a class="ss" href="#section-1">§</a> </div> <a href="https://github.com/mobily-enterprises/docco-next">Docco Next</a> facilitates <a href="https://en.wikipedia.org/wiki/Literate_programming">literate programming</a> in several languages. It’s written in modern Javascript, and runs in Node. See the <a href="https://mobily-enterprises.github.io/docco-next/">generated documentation as HTML</a> <h2 id="install-and-use">Install and use</h2> </div> </li> <li id="section-2"> <div class="annotation"> <div class="sswrap "> <a class="ss" href="#section-2">§</a> </div> To use Docco Next run <code>npm install -g docco-next</code> and run it passing it a list of files (e.g. <code>docco src/*js</code>) By default, every file will be converted into formatted HTML and will be placed in the default destination directory (<code>docs</code> by default). It’s also possible to convert files to Markdown, rather than HTML. Note that files will retain their original names and will change their extension to <code>html</code>. <h2 id="is-it-literate-programming">Is it literate programming?</h2> </div> </li> <li id="section-3"> <div class="annotation"> <div class="sswrap "> <a class="ss" href="#section-3">§</a> </div> Formally speaking, literate programs put documentation strictly first. In simple words, a literate file will look like a Markdown file, and its code is whichever Markdown code is included in the file. This means that anything that is not Markdown code (and is therefore indented in) is considered documentation, and programs are organised so that the documentation actually makes sense. A notable example is CoffeeScript, which supports .litcoffee files natively. However, not many languages support literate programming. Most languages will need a proprocessor, for example to convert the <code>.js.md</code> file into <code>.js</code>. <a href="https://github.com/vijithassar/lit">lit</a> is one of such tools. In Node, you can use Rich Harris’s <a href="https://github.com/Rich-Harris/lit-node">lit-node</a> to require .md files directly. Docco Next works with normal source code (such as <code>.js</code> or <code>.c</code>) as well as literate source code (such as literate CoffeeScript, <code>.litcoffee</code>, or <code>.js.md</code>). With pure literate style, all you have to do is add the .md extension to your source file. When processing source files (such as <code>.js</code> or <code>.c</code>), one line comments are considered documentation and are parsed using Markdown. The code (that is, anything that is not a one-line comment) is processed by <a href="https://shiki.matsu.io/">Shiki</a>. In both cases, the end result is a set of HTML pages with your documentation. <h2 id="similar-projects">Similar projects</h2> </div> </li> <li id="section-4"> <div class="annotation"> <div class="sswrap "> <a class="ss" href="#section-4">§</a> </div> Here is a list of active projects which achieve similar goals: <ul> <li><a href="https://github.com/jashkenas/docco">Docco</a> by Jeremy Ashkenas (the same wise author of Underscore and Backbone). The program that inspired many others (including me) to use literate programming techniques and implement code that facilitates it </li> <li><a href="https://github.com/pycco-docs/pycco">pycco</a> by Nick Fitzgerald. A Python implementation of literate programming </li> <li><a href="https://github.com/gdeer81/marginalia">Marginalia</a> by Gary Deer. A Clojure implementation of literate programing. The project was originally started by Michael Fogus. </li> </ul> This file is the full source code of Docco Next, and it explains how it works in pure literate style. <h2 id="required-files-and-starting-configuration">Required files and starting configuration</h2> </div> </li> <li id="section-5"> <div class="annotation"> <div class="sswrap "> <a class="ss" href="#section-5">§</a> </div> The following libraries are required by Docco Next: <ul> <li><code>ejs</code>. Used to convert the layout master file into HTML</li> <li><code>globby</code>. Used to <a href="https://en.wikipedia.org/wiki/Glob_(programming)">glob</a> for filenames when the shell doesn’t otherwise support this</li> <li><code>fs-extra</code>. Used for all I/O operations</li> <li><code>marked</code>. Used to convert Markdown into HTML</li> <li><code>commander</code>. Used to interpret command line parameters</li> <li><code>shiki</code>. Used to highlight source code</li> </ul> In Javascript terms, this becomes: </div> <div class="content"><pre class="shiki" style="background-color: transparent"><code style="--line-start-number: 78;">const path = require('path') const ejs = require('ejs') const globby = require('globby') const fs = require('fs-extra') const marked = require('marked').marked const commander = require('commander') const shiki = require('shiki') </code></pre></div> </li> <li id="section-6"> <div class="annotation"> <div class="sswrap "> <a class="ss" href="#section-6">§</a> </div> On startup, the <code>version</code> variable is worked out straight from the <code>package.json</code> file, which is loaded using <code>require</code> </div> <div class="content"><pre class="shiki" style="background-color: transparent"><code style="--line-start-number: 88;">const pkg = require('./package.json') const version = pkg.version </code></pre></div> </li> <li id="section-7"> <div class="annotation"> <div class="sswrap "> <a class="ss" href="#section-7">§</a> </div> Docco Next manages several languages, stored in <code>layouts/languages.json</code>. Users can add more languages if desired. However, the list of default languages (and their relevant information) is stored in <code>layouts/languages.json</code> </div> <div class="content"><pre class="shiki" style="background-color: transparent"><code style="--line-start-number: 94;">const defaultLanguages = require('./layouts/languages.json') </code></pre></div> </li> <li id="section-8"> <div class="annotation"> <div class="sswrap "> <a class="ss" href="#section-8">§</a> </div> By default, <code>marked</code> is run with very basic options (basically, just turning <code>smartypants</code> on). Users can add more options, but this is the starting point </div> <div class="content"><pre class="shiki" style="background-color: transparent"><code style="--line-start-number: 98;">const defaultMarkedOptions = { smartypants: true } </code></pre></div> </li> <li id="section-9"> <div class="annotation"> <div class="sswrap "> <a class="ss" href="#section-9">§</a> </div> <h2 id="configuration">Configuration</h2> </div> </li> <li id="section-10"> <div class="annotation"> <div class="sswrap "> <a class="ss" href="#section-10">§</a> </div> The <code>run</code> function is the entry point of the script when it’s run by the command line (that is, it’s not being used as a library). The <code>docco</code> command in <code>bin</code> simply runs <code>require('../docco.js').run()</code>. The package <code>commander</code> will be used to parse command-line parameters. Commander will generate a configuration object based on how it’s configured using its <code>option</code> method. For example the line: <pre><code><pre class="shiki" style="background-color: transparent"><code >.option('-L, --languages [file]', 'use a custom languages.json')</code></pre> </code></pre> Means that if Docco Next is run with <code>--languages ./some/file.json</code>, the config object will include <code>{ languages: "./some/file.json"}</code>. Commander also provides the handy <code>helpInformation()</code> method, which will print out how the command is used. This is the full set of options available in Docco Next: </div> <div class="content"><pre class="shiki" style="background-color: transparent"><code style="--line-start-number: 121;">async function run (args = process.argv) { commander .name('docco') .version(version) .usage('[options] files') .option('-L, --languages [file]', 'use a custom languages.json') .option( '-l, --layout [name]', 'choose a layout (default, parallel or classic)' ) .option('-s, --shiki-theme [shikiTheme]', 'choose a shiki theme') .option('-o, --output [path]', 'output to a given folder') .option('-c, --css [file]', 'use a custom css file') .option('-p, --plugin [file]', 'use a custom plugin file') .option('-t, --template [file]', 'use a custom .ejs template') .option( '-e, --inputExtension [ext]', 'assume a file extension for all inputs' ) .option('-m, --marked [file]', 'use custom marked options') .option( '-x, --outputExtension [ext]', 'set default file extension for all outputs' ) .parse(args) if (commander.args.length) { const config = { ...commander.opts(), args: commander.args } await cmdLineNormalise(config) configure(config) await cmdLineSanityCheck(config) await documentAll(config) } else { return console.log(commander.helpInformation()) } } </code></pre></div> </li> <li id="section-11"> <div class="annotation"> <div class="sswrap "> <a class="ss" href="#section-11">§</a> </div> Three configuration functions are called: <ul> <li><code>cmdLineNormalise()</code> – expands some command line options to objects</li> <li><code>configure()</code> – enriches configuration options with full paths etc.</li> <li><code>cmdLineSanityCheck()</code> – checks that files and folders actually exist.</li> </ul> Here is an explanation of what each one does, followed by their source code. <h3 id="step-1-cmdlinenormalise">Step 1: <code>cmdLineNormalise()</code></h3> </div> </li> <li id="section-12"> <div class="annotation"> <div class="sswrap "> <a class="ss" href="#section-12">§</a> </div> It’s important to understand that Docco Next can be used as a library as well as a command line program. Two of the command line options, <code>marked</code> and <code>languages</code>, are external JSON files. When used as a library, Docco Next will expect <code>marked</code> and <code>languages</code> to be objects. However, when run as a command line program, those options will contain a string with a (JSON) file name instead. The <code>cmdLineNormalise()</code> function is used to convert those strings into Javascript objects which will depend on the contents of the corresponding JSON files The fuction also assigns <code>config.args</code> (which is the list of files to be converted) to <code>config.sources</code>. To sum up: when using Docco Next as a library, <code>config.languages</code> and <code>config.maked</code> will need to be objects. However, from the command line, they will be the paths of JSON files, and will be converted to objects by <code>cmdLineNormalise()</code>, which reads: </div> <div class="content"><pre class="shiki" style="background-color: transparent"><code style="--line-start-number: 186;">async function cmdLineNormalise (config) { if (config.languages) { if (!(await fileExists(config.languages))) { console.error('Languages file not found:', config.languages) process.exit(5) } const languages = await fs.readFile(config.languages) config.languages = JSON.parse(languages) } if (config.plugin) { if (!(await fileExists(config.plugin))) { console.error('Plugin file not found:', config.plugin) process.exit(5) } config.plugin = require(path.join(process.cwd(), config.plugin)) } else { config.plugin = {} } if (!config.outputExtension) config.outputExtension = 'html' if (config.marked) { if (!(await fileExists(config.marked))) { console.error('Marked file not found:', config.marked) process.exit(6) } const marked = await fs.readFile(config.marked) config.marked = JSON.parse(marked) } config.sources = await globby(config.args) } </code></pre></div> </li> <li id="section-13"> <div class="annotation"> <div class="sswrap "> <a class="ss" href="#section-13">§</a> </div> <h3 id="step-2-configure">Step 2: <code>configure()</code></h3> </div> </li> <li id="section-14"> <div class="annotation"> <div class="sswrap "> <a class="ss" href="#section-14">§</a> </div> This function is different to the other two <code>cmdLineNormalise()</code> and <code>cmdLineSanityCheck()</code>: rather than being a command line-only normalisation function, it’s actually a function that is run every time an API call is invoked. (Note that once it’s run once, it sets the <code>config.configured</code> property in the <code>config</code> object, and will check this property so that it will ever do anything only the first time it’s invoked) This function is responsible to set sane defaults in the <code>config</code> property, so that each function doesn’t need to worry with wasteful checking. For example <code>config.output</code> will be set as <code>docs</code> by default. Also, for passed properties such as <code>languages</code> or <code>marked</code>, this function makes sure that the passed options are added to sane defaults. For example by passing the <code>marked</code> property in the config object, this function makes sure that the default <code>smartypants: true</code> is on, and any configuration options are added on top of what the user has passed. Or, that the config.languages contains the used-defined languages, as well as the default ones. This function also sets some indirect configuration options that are not meant to be passed by the developer, but that are a result of the configuration. Finally, this function also makes sure that the passed config makes sense: <code>config.sources</code> is scanned and any source with an unknown language (that is, a language not in the <code>languages</code> array) is filtered out (with a warning). You can see this function as insurance that there is a solid, valid <code>config</code> object that every call can use. This is why every single function that uses <code>config</code> will call <code>configure</code> first. </div> <div class="content"><pre class="shiki" style="background-color: transparent"><code style="--line-start-number: 253;">function configure (config) { if (config.configured) return config.configured = true config.output = config.output || 'docs' config.languages = properObjectWithKeys(config.languages) ? { ...defaultLanguages, ...config.languages } : defaultLanguages config.marked = properObjectWithKeys(config.marked) ? { ...defaultMarkedOptions, ...config.marked } : defaultMarkedOptions config.layout = config.layout || 'default' if (config.layout.match(/^[a-zA-Z0-9]+$/)) { config.layout = path.join(__dirname, 'layouts', config.layout) } if (!config.css) config.css = path.join(config.layout, 'docco.css') config.public = path.join(config.layout, 'public') if (!config.template) { config.template = path.join(config.layout, 'docco.ejs') } config.sources = config.sources || {} config.sources = config.sources.filter((source) => { const there = getLanguage(source, config) if (!there) { console.warn( `docco: file not processed, language not supported: (${path.basename( source )})` ) } return there }) } </code></pre></div> </li> <li id="section-15"> <div class="annotatio