I’ve fielded a few questions lately from folks about Sphinx, reStructuredText, and documentation in the Python world and more broadly. So I thought I’d write up a bit of an intro to how I use and understand these tools, and how better documentation makes me a better developer.
Restructured understanding of reStructuredText
There’s a lot of cargo-cultish behavior and confusion around writing Python docs, and I think that a lot of this comes from seeing some Sphinx documentation, finding something curious in the source, and then not being able to learn what it is or where it comes from. The first source of confusion, I think, is between Sphinx and reStructuredText. Here’s the distinction: reStructuredText is a markup language (with a processor implemented in the docutils package), and Sphinx is a documentation builder or compiler, that takes a bunch of reStructuredText files and makes them into a whole suite of documentation.
Since reStructuredText is in some ways the basis, let’s start by talking about it. It’s a fairly straightforward markup language. If you’ve used Markdown before, it may be just similar enough to cause confusion. Headings are indicated with a series of =, -, or ~ on the following line, paragraphs are broken with two newlines (a single newline in a paragraph is ignored), italic is indicated with surrounding *...*, bold with surrounding **...**, and fixed-width with surrounding `...`. Lists have leading - or *. All broadly familiar.
(There are fancier things, like links and tables, but you can look those up yourself when you need them.)
Now, there are two things that reStructuredText supports that make it particularly useful as a basis for Sphinx: directives and roles. These are custom bits of markup that you can use to define special structures in the abstract representation of the document, and emit into the final compiled document in whatever way needed.
Roles are inline, and generally look like this: :role:`text`. The role is whatever the name of the custom role is (ref, class, index, or weirder things), and the text is whatever text gets wrapped in that role.
Directives are block-level, and look like this:
.. directive_name::
:directive_arg1:
:directive_arg2: value
Some text that gets the directive applied to it.
They can have some keyword arguments (in the immediately following lines, with : surrounding) and a whole block of arbitrary paragraphs following.
There’s a small set of roles and directives included in reStructuredText itself, and a larger set in Sphinx, and vast wide world of third-party ones. Some builtin ones include directives for sidebars, warnings, topics, and roles for things like links.
The Sphinx’s riddle
So now we can see what’s in a pure reStructuredText file. What’s Sphinx give us? It gives us a whole passel of things, but most importantly it gives us a range of output formats, support for cross-references, and support for pulling documentation information from our Python source.
Sphinx starts with a file called conf.py in the root folder, which sets a number of values. Most crucially, it configures the location of the root reStructuredText file, and sets the theme and options used for the HTML, PDF, ePub, and other output. It also lets you specify extensions and third-party packages that enable more roles and directives.
You obviously don’t want your documentation to exist as one giant .rst file, though, right? Just like you wouldn’t want any webpage to be a single giant page. So you break your docs into many distinct .rst files, and cross-link them. The most crucial form of cross-reference is a .. toctree:: directive in the root .rst file, linking together and ordering all the other .rst documents. But you can also use the ref role to cross-link documents in the middle of a paragraph. For instance, make a reference target using this directive:
.. _some-ref:
# Wotta Heading
This is a section I want to refer to later.
And then in another document, link to it:
So, if you consider :ref:`some-ref`.
And that’ll render like this:
So, if you consider <a href="./some_file.html#wotta-heading">Wotta Heading</a>.
(There’s some magic in there, about how headings are turned into URL fragments, and how cross-linking finds the right file, but I hope the general idea is clear.)
Where do I go from here?
Start a documentation project! Or add docs to an existing project. Here’s a quick recipe:
pip install sphinx
sphinx-quickstart
Now you’ve got a conf.py and an index.rst. Write some docs, run make html, and open _build/html/index.html in your browser.
That’s all! You’ve got docs.
One more thing
Now, personally, I like using the dirhtml builder, instead of the html builder, because I like my URLs to end with a /, not a .html. But that makes it harder to view changes locally. So I made a little tool to serve the docs locally, and while it’s at it, to rebuild them as you change them. It’s called phix, and you can get it from PyPI. Assuming you’re running Python 3, just run pip install phix, and then run phix in the root of your documentation project. You can then see your docs at http://localhost:8000/ as you work on them. Enjoy!
Next steps
This is probably the first in a series. I’ve got a lot of other things I’d like to write about on this subject. I’ll link them here as I write them:
- getting your docs on Read the Docs
- building docs for different targets
- writing custom Sphinx themes
- writing Sphinx extensions
- why you should try documentation-driven development