GSoC 2009: Sphinx's Interactive Documentation. Application by Wojciech Walczak
Sphinx's interactive documentation: a support for per-paragraph comments and user/developer interface for submitting/committing fixes

Back to the main page

Last modified: Sat Apr 01 11:04:23 GST 2009
Status: accepted


  1. Users spend significant amount of time reading the documentation. Providing some ways of interaction on this background is in the best interest of the user and wider community.

    Instead of adding a simple commenting framework to Sphinx, I will create a platform of discussion as well as documentation enhancement.

    The benefits of this addition would be: better reviewed, more up to date, extended documentation, tighter community ties and eased procedures of submitting the documentation fixes.

  2. A crucial division of the user's perspective and developer's perspective ought to be made. Moreover, we have to differentiate between comments view and diffs view.

    In result, we are able to split options and privileges as shown in table below:

    User Developer
    Comments view Allows for:
    • viewing/sorting the comments,
    • adding the comments/fixes,
    • replying to particular comments,
    • rating the comments,
    • changing the view to diffs view;
    Besides all the user privileges allows for:
    • deleting/moving the comments,
    • changing properties of the comments,
    • exporting the comments to a file;
    Diffs view Allows for:
    • browsing submitted patches and discussions about them,
    • commenting submitted patches;
    Besides all the user privileges allows for:
    • deleting/committing the patches,
    • setting a status for a patch,
    • assigning the review to particular developer,
    • viewing VCS logs;

    The idea behind this division is that, the needs and aims of users and developers are not quite the same. My whole work have to be based on some general assumptions about these two perspectives.

    Assumptions about the user's perspective are as follows:
    • user loads the documentation pages to read it and he may not want to be disturbed by too tawdry comments interface,
    • when user looks at the comments he is rather interested in the comments themselves, than in the diffs submitted by other users,
    • in case user wants to submit a file he should be able to do so by the comment post form.

    Assumptions about the developer's perspective are as follows:
    • developer is rather interested in the diffs submitted by the users than their discussions,
    • developer may be interested in the comments and he should be able to browse them easily and quickly,
    • in case developer finds some patches useful, he should be able to commit them right from the web application.

    Based on these assumptions I am able to derive the looks and functionality of comments view and diffs view.

    Characteristics of the comments view are as follows:
    • Sortable by: date, type, rank
    • Viewable: per paragraph, per page
    • Rateable for logged in users

    Characteristics of the diffs view are as follows:
    • Sortable by: date, status, priority
    • Viewable: per paragraph, per page, per directory

    At this stage I am able to propose the model of post form, which will be available under every comments or diffs view.

    Fields in a post form:
    • Comment (text input),
    • Name (text input),
    • E-mail (text input),
    • Website (text input),
    • I would like to submit a fix (checkbox); off by default,
    • Submit comment (button). Available when submit a fix checkbox is off.

    Additional fields shown when I would like to submit a fix checkbox is on are as follows:
    • Priority (drop down list),
    • Paragraph (text input) filled with paragraph's contents,
    • I agree to publish my fix on a licence X (checkbox); off by default,
    • Submit fix (button).

    This form will be available below the comments/diffs view in case anyone would like to create a new thread or submit a patch. Moreover, every comment will have a button 'Reply' and when pressed, the post form will be provided to reply to a particular comment.

    I assume that only logged-in users can post comments.

    My analysis are based on a critical review of such commenting systems as these available on:[1],[2],[3],[4] and[5]. For further discussion, please refer to this thread on the Sphinx-dev mailing list: [GSoC] Some "web app" idea questions[6].

    The core techniques resposible for serving the documentation pages to the public are still a matter of consideration. In the current state of advancement the set of tools chosen to address the appearing problems is as follows:
    • Pylons[7] as a web framework,
    • jQuery[8] as JavaScript library,
    • jQuery[9] Form Plugin,
    • Jinja2[10] as a templating system,
    • Pygments[11] for syntax highlighting,
    • Xapian[12] for server-side search,
    • OpenID[13] for identyfing the users.

    Elements of this set may change slightly before the coding period starts as this issue is still being discussed within the Sphinx's community.

    I decided to rearrange the coding process in such a manner that since very early steps it will be possible to trace my progress in two ways (except of reading the code):
    • by running the test suite,
    • by running Sphinx itself with build web app option set.

    Besides of making it easy to evaluate my work, it will mean that the framework is fully integrated with Sphinx's code base.

    Detailed step-by-step description of my work is presented below. Nevertheless, I have to clarify that every piece of my work, even if it is not stated explicitly, implies writing corresponding test suites and documentation.

    My progress will be described in reports written at the end of every stage (exact periods of time are specified in the Project Schedule below). Usually the reports will be presented on weekly basis.

    As I plan to create a VCS abstraction framework (based on pydocweb[14] code) to make it easy to configure Sphinx's interactive docs to work with various Version Control Systems, I will also add a builtin support for SVN[15], Mercurial[16], Git[17] and Bazaar[18].

    Very basic, general API premises for my project are also expressed in Python[19]. The purpose of this file is to present a generic approach of my proposal even though it abstracts from Sphinx, VCSs, login system, search engine etc.

    • Integrate the web framework into Sphinx.
    • Implement server-side search.
    • Implement logging mechanism.
    • Implement per-paragraph button for displaying/hiding the comments.
    • Implement a post form.
    • Implement comments/diffs view for ordinary users.
    • Implement threading and comment rating for comments view.
    • Implement threading for diffs view.
    • Implement comments/diffs view for developers.
    • Intagrate diffs view with the VCSs.
    • Write/update documentation.

    Project Schedule
    Duration Task Description
    April 21 - May 22 Continuing the discussion about the details of my project Many details are already clearly set. Nevertheless, some elements are still a matter of discussion and throughout this period I will decide what the final set of techniques should be.
    May 23 - June 1 Integrating the web framework into Sphinx Sphinx will be able to generate a web application for serving documentation files built with Sphinx.
    June 2 - June 8 Adding a server-side search A search will be added and an availability to search through documenation only, comments only or both documentation and comments will be provided.
    June 9 - June 15 Adding a logging mechanism and per-paragraph button for displaying/hiding the comments A base for user/developer actions will be created. The basic assumptions are:
    • only identified users are able to post comments/diffs,
    • only logged-in developers are able to commit diffs.
    June 16 - June 22 Creating a post form A post form will be displayed under every field reserved for comments.
    June 23 - July 5 Adding the comments/diffs views available for ordinary users Every user will be available to display/sort comments and to switch to diffs view.
    July 6 - July 8 Writing mid-term evaluation N/A
    July 9 - July 15 Adding threading and comment rating for comments view and threading for diffs view In comments view there will be a possibility to rate comments up or down. In diffs view comments will be threared by submitted diffs.
    July 16 - July 22 Adding the comments/diffs views available for developers Every developer will be able to delete/move the comment, change its properties and export it to a file. In diffs view he will be able to delete the patch, set/change its status or assign it to somebody.
    July 23 - July 31 Integrating diffs view with the VCSs Developers will be able to commit submitted patches to the repository, view the logs and the patch discussion.
    August 1 - August 8 Writing the documentation The documentation for this project will be reviewed, cleaned-up and presented using Sphinx-generated web application.
    August 9 - August 17 Final testing At this level Sphinx will be able to generate a web application for serving documentation files built with Sphinx.

    During the summer, GSoC will be my only full-time activity. It is very highly probable that I will pass all my exams until the end of May, thus I presume that my GSoC participation may be lower at the end of May, but I am sure that I will make up any delay at the beginning of June.

    I am applying to two projects. My priority is Sphinx[20]. The second project I am applying to is EKG[21].

  3. My name is Wojciech Walczak, I am 24. I study sociology on Nicolaus Copernicus University[22] in Torun[23], Poland.

    I code mainly in Python and C. I am an autodidact. I consider myself a member of Python community for a long time. My activity on comp.lang.python dates back to December 2001. Some of my small patches were applied into the core of Python[24], its modules[25] and library[26].

    My studies concentrate on the usage of computer programming in the sociology research. My two Django-based applications are used in the media research on my University. For my masters thesis I am writing a complex Internet news portals analyzing tool.

    GSoC is a great opportunity for me to contribute more to the Python - as well as wider - community, get to know more people and tools, gain some useful experience and earn money at the same time.

  4. Wojciech Walczak ‹›, ‹gminick@IrcNet›, ‹gminick@FreeNode›

Back to the main page

(C) 2009 by Wojciech Walczak --