Using Pygments with Codenvy

Pygments tutorial with Codenvy IDE

What is Pygmets

Pygments is a generic syntax highlighter for general use in all kinds of software such as forum systems, wikis or other applications that need to prettify source code. Pygments is top quality product, has extensive coverage of languages and is very easy to use.

In this tutorial we will extend the Werkzeug tutorial to include a controller/url that highlights source files found on the internet.

Initial Configuration

Lets start by cloning our Werkzeug project. In your project view, go to menu->git->clone repository, menu-git-cloneand add as a repository this url: http://codenvy.com/git/48/9f/18/workspacep0t9ns3okn4x2pzb/werkzeug ,clone-menu and an appropriate name e.g. Pygments.

Now open requirements.txt and add two more lines containing “pygments” and “requests”, we will use the requests package to grab the source file from the web.

reqs

How Pygments work’s

There are three types of components that work together highlighting a piece of code, but we will use only two of them:

  • A lexer splits the source into tokens, fragments of the source that have a token type that determines what the text represents semantically (e.g., keyword, string, or comment). There is a lexer for every language or markup format that Pygments supports.
  • A formatter then takes the token stream and writes it to an output file or stream, in a format such as HTML, LaTeX or RTF.

The controller

Now lets add the function thats responsible for the syntax highlighting, and connect it to our routing system in werkzeug.

#importing section
from pygments import highlight
from pygments.lexers import guess_lexer
from pygments.formatters import HtmlFormatter

import requests

#Hello class - init

    def __init__(self):
        self.url_map = Map([
            Rule('/', endpoint='root'),
            Rule('/hello', endpoint='hello'),
            Rule('/pygmentize', endpoint='pygmentize'),
        ])
        self.database = kvstore

    def on_pygmentize(self,request,**keys):
        url = request.args.get('url', 'http://github.com/josh/pygments/raw/master/pygments/formatters/html.py')
        raw_code = requests.get(url).text
        code = highlight(raw_code, guess_lexer(raw_code), HtmlFormatter())
        css = HtmlFormatter().get_style_defs('.highlight')
        return Response(tmpl.html_css %{'css':css,'body':code}, mimetype='text/html')

We are using the html_css template in our pygmentize function, so let’s add it to the templates:

#templates.py

html_css  = """<html>
                      <head>
                        <style type="text/css">
                        %(css)s
                        </style>
                      </head>
                      <body>
                        %(body)s
                      </body>
                   </html>
                """

And thats it!. You can run this application and browse to /pygmentize, to see pygments in action. If no ?url= argument is given, then a default one will be used, else you can pass a url argument to load any public file on the net e.g. /pygmentize?url=https://github.com/torvalds/linux/raw/master/kernel/fork.c