Snippets

A snippet within a file is a a region of code which is auto-generated and which will be overwritten by gcgen each time it is run.

Using snippets in a file

From the perspective of a source file, a snippet is “called” by adding a start- and an end line to the source file.

Each time gcgen parses the file, it will remove any lines between the start- and end lines of snippet, and write out the output generated by the snippet function instead.

Snippet start- and end lines are defined by special tags, these can be configured, but are <<? and ?>> by default.

Here are two examples, showing a snippet call from Python and Golang respectively:

call snippet from python

def main():
    print("hello, world!")
    # <<? my-snippet ?>>
    # <<? /my-snippet ?>>

call snippet from golang

import "fmt"

func main() {
    fmt.Println("hello, world!")
    // <<? my-snippet ?>>
    // <<? /my-snippet ?>>
}

In these two examples, my-snippet is the name of the snippet called - this corresponds to the snippet definition seen in Defining a snippet.

Note that snippet start- and end lines must have the same prefix (~equally indented, using the same characters), but that the prefix can be anything - this is what allows the start- and end lines to use the language-specific syntax for line comments.

Note that snippets are automatically indented to fit their context. Each line of the snippet output is indented as far as the start of the snippet start- and end lines.

Prerequisites

To parse a file, foo.txt for snippets, gcgen first requires that:

1. the directory of the file must have a gcgen_conf.py file (link)

2. the gcgen file must implement the cgen_parse_hook and this must return a list of filenames where foo.txt is among them. See Parse files for details.

Additionally, for gcgen to parse the file successfully, each snippet that foo.txt uses must be defined either in the gcgen_conf.py file in the same directory as foo.txt, or in any of the gcgen_conf.py files in the parent directories.

Defining a snippet

Snippets are functions taking three arguments:

1. section - this is used to generate output in the file calling the snippet

2. a scope - this is populated both by gcgen files and preceding snippets in the same file.

3. a Json value - snippets may receive an argument, which must be a valid Json value. The value is None if no argument was given or null was passed.

Crucially, to be a snippet, the function must also be using the snippet decorator - this decorator ensures gcg registers the function as a snippet, and defines the name to give it.

defining a new snippet

# (inside a gcgen_conf.py file)
from gcgen.api import Section, Scope, Json, snippet


@snippet("my-snippet")
def my_snippet(sec: Section, s: Scope, v: Json):
    pass

On snippet naming

You cannot use the function name to call a snippet from within a source file, you must use one of the name(s) given to the snippet by the snippet decorator. As implied, the snippet decorator can be used multiple times on the same function to give it additional names.

Snippet scope

Snippet definitions work like entries in the scope: a snippet defined in some gcgen_conf.py file is available to all source files in that directory or any of its subdirectories.

Similarly to scope entries, it is also possible for a gcgen_conf.py file to override a snippet definition from the parent scope, by defining a new snippet function and annotating it with the name of the snippet to override. This, just like variable entries in the scope, will only affect the current directory and any subdirectories there may be.

How to use snippets effectively

Why can snippets only take one parameter?

Snippets are intentionally limited to take at most one JSON argument. Gcgen is inspired by tools like Cog, but disagrees with inlining code-generation code into source files. Inlining code both clutters the source file and introduces code, for which the user gets no ide/linting/type-checking support.

By limiting input to a single JSON value, we allow input arguments without supporting inline code. By limiting arguments to the snippet opening line, we further push code-generation logic out of the source file and into the gcgen-file.

In practice, most parametrization needs are simple enough that a short JSON value will suffice, if not, consider writing multiple snippets which calls out to other functions for the common logic. In practice, most snippets can be sufficiently parametrized

Tip: keep snippets small!

If you find yourself passing large Json objects to your snippets, then you are keeping too much complexity at the call-site (the source file) and the snippet is likely too generic. Try to specialize your snippets and ensure that they work with limited arguments. Remember, a snippet argument could reference a larger, complex value already stored in the Scope.

Tip: use the file-specific scope

Each file being parsed for snippets receives its own scope. This also means that changes to the scope made by one snippet are visible to every snippet called later in the file.

This means it is possible to define a snippet to be called at the start of the file, whose job it is to populate the scope with additional entries which the other snippets can act on.

Tip: calling snippets from inside a snippet

You can call another snippet from within a snippet as any other normal python function:

from gcgen.api import snippet, Section, Scope, Json

# These two snippets simply call the generalized function
# with the specific parameters
@snippet("foo")
def s_foo(sec: Section, s: Scope, v: Json):
    if v:
        sec.emitln(f"foo> hello {v}!")
    else:
        sec.emitln("foo> hello!")

@snippet("bar")
def s_bar(sec: Section, s: Scope, v: Json):
    sec.emitln("bar> hello!")
    s_foo(sec, s, "Bar")

However, now s_bar will always call s_foo, even if foo is otherwise overridden to something else. We can instead dynamically resolve the snippet to call using get_snippet:

from gcgen.api import snippet, Section, Scope, Json, get_snippet

# These two snippets simply call the generalized function
# with the specific parameters
@snippet("foo")
def s_foo(sec: Section, s: Scope, v: Json):
    if v:
        sec.emitln(f"foo> hello {v}!")
    else:
        sec.emitln("foo> hello!")

@snippet("bar")
def s_bar(sec: Section, s: Scope, v: Json):
    sec.emitln("bar> hello!")
    get_snippet(s, "foo", "Bar")(sec, s)

Using get_snippet, we thus call whatever the foo snippet is in the current context. In this way, our snippet can call out to other snippets, while respecting if the snippet is overridden with another implementation.