Skip to content

CommonMark Spec

John MacFarlane

Creative Commons BY-SA

Introduction⚓︎

What is Markdown?⚓︎

Markdown

  • Markdown is a plain text format for writing structured documents, based on conventions for indicating formatting in email and usenet posts. It was developed by John Gruber (with help from Aaron Swartz) and released in 2004 in the form of a syntax description and a Perl script (Markdown.pl) for converting Markdown to HTML. In the next decade, dozens of implementations were developed in many languages. Some extended the original Markdown syntax with conventions for footnotes, tables, and other document elements. Some allowed Markdown documents to be rendered in formats other than HTML. Websites like Reddit, StackOverflow, and GitHub had millions of people using Markdown. And Markdown started to be used beyond the web, to author books, articles, slide shows, letters, and lecture notes.

  • What distinguishes Markdown from many other lightweight markup syntaxes, which are often easier to write, is its readability. As Gruber writes:

  • The overriding design goal for Markdown’s formatting syntax is to make it as readable as possible. The idea is that a Markdown-formatted document should be publishable as is, as plain text, without looking like it’s been marked up with tags or formatting instructions. (Daring Fireball: Markdown)

The point can be illustrated by comparing a sample of AsciiDoc with an equivalent sample of Markdown.

A Sample of AsciiDoc. 🔥

1. List item one.
+
List item one continued with a second paragraph followed by an
Indented block.
+
.................
$ ls *.sh
$ mv *.sh ~/tmp
.................
+
List item continued with a third paragraph.

2. List item two continued with an open block.
+
--
This paragraph is part of the preceding list item.

a. This list is nested and does not require explicit item
continuation.
+
This paragraph is part of the preceding list item.

b. List item b.

This paragraph belongs to item two of the outer list.
--

And here is the equivalent in Markdown:

The Equivalent in Markdown

  1. List item one.

    List item one continued with a second paragraph followed by an
    Indented block.

    $ ls *.sh
$ mv *.sh ~/tmp

    List item continued with a third paragraph.

  2. List item two continued with an open block.

    This paragraph is part of the preceding list item.

    a. This list is nested and does not require explicit item continuation.

    This paragraph is part of the preceding list item.

    b. List item b.

    This paragraph belongs to item two of the outer list.

AT&T⚓︎

AsciiDoc v Markdown

  • The AsciiDoc version is, arguably, easier to write. You don’t need to worry about indentation.

  • But the Markdown version is much easier to read.

  • The nesting of list items is apparent to the eye in the source, not just in the processed document.

Why Is A Spec Needed?⚓︎

Why is a spec needed?

1. John Gruber’s canonical description of Markdown’s syntax does not specify the syntax unambiguously. Here are some examples of questions it does not answer:

1.1. How much indentation is needed for a sublist? The spec says that continuation paragraphs need to be indented four spaces, but is not fully explicit about sublists. It is natural to think that they, too, must be indented four spaces, but Markdown.pl does not require that. This is hardly a “corner case,” and divergences between implementations on this issue often lead to surprises for users in real documents. (See this comment by John Gruber.)

2. Is a blank line needed before a block quote or heading? Most implementations do not require the blank line. However, this can lead to unexpected results in hard-wrapped text, and also to ambiguities in parsing (note that some implementations put the heading inside the blockquote, while others do not). (John Gruber has also spoken in favor of requiring the blank lines.)

3. Is a blank line needed before an indented code block? (Markdown.pl requires it, but this is not mentioned in the documentation, and some implementations do not require it.)

There are some relevant comments by John Gruber here.

  • In the absence of a spec, early implementers consulted Markdown.pl to resolve these ambiguities. But Markdown.pl was quite buggy, and gave manifestly bad results in many cases, so it was not a satisfactory replacement for a spec.

  • Because there is no unambiguous spec, implementations have converged considerably. As a result, users are often surprised to find that a document that renders one way on one system (say, a GitHub wiki) renders differently on another (say, converting to docbook using pandoc). To make matters worse, because nothing in Markdown counts as a “syntax error,” the divergence often isn’t discovered right away.

About This Document⚓︎

About This Document

1.3 About this document.

1.3.1 This document attempts to specify Markdown syntax unambiguously. It contains many examples with side-by-side Markdown and HTML. These are intended to double as conformance tests. An accompanying script spec_tests.py can be used to run the tests against any Markdown program:

python test/spec_tests.py --spec spec.txt --program PROGRAM

1.3.2 Since this document describes how Markdown is to be parsed into an abstract syntax tree, it would have made sense to use an abstract representation of the syntax tree instead of HTML. But HTML is capable of representing the structural distinctions we need to make, and the choice of HTML for the tests makes it possible to run the tests against an implementation without writing an abstract syntax tree renderer.

1.3.3 Note that not every feature of the HTML samples is mandated by the spec. For example, the spec says what counts as a link destination, but it doesn’t mandate that non-ASCII characters in the URL be percent-encoded. To use the automatic tests, implementers will need to provide a renderer that conforms to the expectations of the spec examples (percent-encoding non-ASCII characters in URLs). But a conforming implementation can use a different renderer and may choose not to percent-encode non-ASCII characters in URLs.

1.3.4 This document is generated from a text file, spec.txt, written in Markdown with a small extension for the side-by-side tests. The script tools/makespec.py can be used to convert spec.txt into HTML or CommonMark (which can then be converted into other formats).

→ Character is Used to Represent Tabs.⚓︎

In the examples, the → character is used to represent tabs.

2. Preliminaries

2.1 Characters and Lines

  • Any sequence of characters is a valid CommonMark document.

  • A character is a Unicode code point. Although some code points (for example, combining accents) do not correspond to characters in an intuitive sense, all code points count as characters for purposes of this spec.

  • This spec does not specify an encoding; it thinks of lines as composed of characters rather than bytes. A conforming parser may be limited to a certain encoding.

  • A line is a sequence of zero or more characters other than line feed (U+000A) or carriage return (U+000D), followed by a line ending or by the end of file.

  • A line ending is a line feed (U+000A), a carriage return (U+000D) not followed by a line feed, or a carriage return and a following line feed.

  • A line containing no characters, or a line containing only spaces (U+0020) or tabs (U+0009), is called a blank line.

The following definitions of character classes will be used in this spec:

  • An ASCII punctuation character is:
    !, ", #, $, %, &, ', (, ), *, +, ,, -, ., /(U+0021–2F), :, ;, <, =, >, ?, @ (U+003A–0040),
    [, \, ], ^, _, ` (U+005B–0060), {, |, }, or ~ (U+007B–007E).

  • A Unicode punctuation character is a character in the Unicode P (puncuation) or S (symbol) general categories.

Tabs⚓︎

2.2 Tabs in lines are not expanded to spaces. However, in contexts where spaces help to define block structure, tabs behave as if they were replaced by spaces with a tab stop of 4 characters.

Thus, for example, a tab can be used instead of four spaces in an indented code block. (Note, however, that internal tabs are passed through as literal tabs, not expanded to spaces.)

AsciiDoc v Markdown

The point can be illustrated by comparing a sample of AsciiDoc with an equivalent sample of Markdown. Here is a sample of AsciiDoc from the AsciiDoc manual:

  1. List item one.
    +
    List item one continued with a second paragraph followed by an
    Indented block.
    +
    ............................
    $ ls *.sh
    $ mv *.sh ~/tmp
    ............................
    +
    List item continued with a third paragraph.

  2. List item two continued with an open block.
    +

    This paragraph is part of the preceding list item.

    a. This list is nested and does not require explicit item continuation.
    +

    This paragraph is part of the preceding list item.
    b. List item b.

This paragraph belongs to item two of the outer list.

The Equivalent in Markdown:⚓︎

AsciiDoc v Markdown

  1. List item one.
    List item one continued with a second paragraph followed by an
    Indented block.

    $ ls *.sh
$ mv *.sh ~/tmp

    List item continued with a third paragraph.

  2. List item two continued with an open block.
    This paragraph is part of the preceding list item.

    a. This list is nested and does not require explicit item continuation.
    This paragraph is part of the preceding list item.

    b. List item b.
    This paragraph belongs to item two of the outer list.

The AsciiDoc version is, arguably, easier to write. You don’t need to worry about indentation. But the Markdown version is much easier to read. The nesting of list items is apparent to the eye in the source, not just in the processed document.