Season of Docs 2021 Case Study

Organization: SymPy Project: Documentation Organization Organization Description: SymPy is a Python library for symbolic mathematics. It aims to become a full-featured computer algebra system (CAS) while keeping the code as simple as possible in order to be comprehensible and easily extensible. SymPy is written entirely in Python.

Problem Statement

SymPy's has extensive online documentation at https://docs.sympy.org/, but it is not organized in a coherent way. The project aims to improve this organization using the Diátaxis Framework.

Proposal Abstract

SymPy is itself a large project. The library has over a thousand public functions and methods, and consists of over 700k lines of code. About a third of these lines of code (240k) are documentation in the form of docstrings, and there are an additional 32k lines of Restructured Text documentation outside of the library source code. The result is that SymPy has a lot of documentation. And there still remains a great deal of documentation that needs to be written, which will make the problem only worse.

The project is to improve the organization of SymPy's online documentation at docs.sympy.org. Right now, it can be difficult for users to find what they want because sheer quantity of documentation. There is also not a clear delineation of the different types of documentation, meaning that things like tutorials, API documentation, and user guides are sometimes intermixed.

The full proposal can be found here.

Project Description

Creating the proposal

SymPy first participated in Season of Docs in 2019, which was the first iteration of the program. When we applied, we outlined several idea for improvements to our documentation, which can be found here. Our 2019 technical writer tackled one of the most important of these, creating a style guide for our documentation, and a 2020 Season of Docs project continued this work. For the 2021 program, we picked the one of the remaining ideas on the list that we felt was highest priority and would have the most impact: improving the organization of our online documentation pages.

Budget

Our budget was straightforward:

Budget item	Amount	Running Total	Notes/justifications
Technical writer develop and implement new documentation organization for SymPy documentation.	10000.00	10000.00

TOTAL: 10000.00

We decided that using the budget to hire a that a single writer would be most effective. We did not have any unexpected expenses, did not spend less than the grant award, and did not require the use of other funds outside of Season of Docs.

Participants

The technical writer was Joannah Nanjekye (@nanjekyejoannah). Project mentoring was done by Aaron Meurer (@asmeurer) and Oscar Benjamin (@oscarbenjamin), who are both core contributors to SymPy. We met every other week on a video call to discuss the project.

Timeline

April 2021 - June 2021: Work was done to plan the new documentation layout based on Diátaxis.

July 2021: Initial categorization was implemented. The "getting started" category was completed.

August 2021: The "contributor guide" section was completed.

September 2021: The "tutorials", "how-to guide", and "explanation" categories were completed.

October 2021: The "public API reference" section was started.

November 2021: The "public API reference" section was finished. Various small pieces were completed. The "private API reference" section was started, but was not able to be completed for the duration of the project..

Results

The final documentation organization can be currently found on the development documentation website. https://docs.sympy.org/dev/index.html.

Two things were not completed in the project. The first was choosing a new theme for the documentation. This was not done because it was considered lower priority than the other tasks. The second was creating a new documentation page for internal documentation. Work on this has been started (https://github.com/sympy/sympy/pull/22444), however the full work for this ended up being more technically challenging than initially anticipated.

Metrics

The new documentation organization is not live yet. It is visible on the development documentation site, but will not be on the main documentation site until we do a release of SymPy. Our original plan was to run before and after surveys, but we did not end up doing this. However, we are currently running a more general survey on documentation as part of a separate project from Season of Docs. Since the new organization is not yet live, the responses to this survey will all be based on the previous organization. We can then perform followup once the new organization is live to see if it has improved things.

Additionally, we will collect general feedback from the SymPy user community on the new organization once it goes live.

Analysis

The project was planned out ahead of time and stayed mostly on schedule. Two items were not completed: the new theme was considered lower priority, and the internal documentation ended up having technical difficulties. The organization itself was done successfully. The documentation is now organized in a more coherent way, and it will be easier to add new resources such as user guides and tutorials to the documentation by being able to place them in logical locations in the organization. Previously, there was no clear place for certain types of documentation, which somewhat hampered their ability to be written.

As noted previously, it is still too early to tell how successful the project will be in practice because the new organization is not yet live on the main SymPy documentation page.

Summary

The project resulted in a reorganization of SymPy's online documentation according to the Diátaxis Framework. Previously the documentation was not organized in a coherent way at all. Now, every documentation page is placed in one of four categories according to the framework. This also gives a location for all new documentation that will be written.

For other projects, I would recommend being aware that any part of the documentation that is automatically generated, such as automatic API documentation, may have technical difficulities associated with it, which may provide a challenge to your technical writer(s).

Appendix

Our documentation organization was based on the Diátaxis Framework: https://diataxis.fr/