GSoD 2022 Project Report - STEllAR-GROUP/hpx GitHub Wiki
Organization or Project: HPX (STE||AR Group)
Organization Description: The goal of the STE||AR Group is to promote the development of scalable parallel applications by providing a community for ideas, a framework for collaboration, and a platform for communicating these concepts to the broader community.
Problem Statement
HPX’s previous GSoD projects were focused on presenting the documentation in a user-friendly way. Now that the documentation is much more presentable, HPX decided to focus on content this year. HPX has hundreds of algorithms and classes. A lot of these elements had outdated or no documentation at all.
This issue of outdated documentation was exacerbated by the fact that Doxygen doesn’t fully support a lot of modern C++ directives and concepts like tag_invoke. These are used extensively by HPX. Since Doxygen doesn’t fully support them, class and function declarations need to be duplicated. These duplicated declarations also need to be synchronized with actual code whenever there is a change.
Proposal Abstract
The main objective of the project was to improve documentation content by documenting currently undocumented elements, fixing outdated documentation and removing obscure documentation. We also planned to improve the build system and contiguous integration for the documentation.
The full proposal is available here: GSoD 2022 Project Proposal.
Project Description
Having participated in Season of Docs in 2019 and 2021 we already had a preliminary list of potential topics for this year's Season of Docs. The basic idea was to make our documentation more accessible to the users. Google Summer of Code students were the best source of feedback for that cause. Based on feedback we hear frequently from them we decided to focus on the content that is already available in the documentation. We combined the most critical parts from a few project ideas from 2019 and 2021 into one project. For example, a commonly recurring difficulty for new Summer of Code students is how to build HPX.
While we did find a potential technical writer during the application period, we still decided to leave the project proposal relatively open to fit various experience levels, so that we could adapt the topic to the hired writer and the requirements as revealed by talking to our users.
Budget
The budget, like the project scope, was left relatively open to be able to adapt to the experience level of the technical writer that we hired. We estimated a rough budget based on average technical writer salaries and a 3-to-5-month project of part-time work. This had worked very well for us in the previous GSoD projects. We knew ahead of time that there would likely be enough work in our documentation for far longer than a 5-month period and thus the project would have to focus on only a subset of the documentation. Likewise, if the writer would make faster than expected progress, we knew there would be enough work for them towards the end of the project. Based on this we estimated a budget of USD 5200.
At the time of writing the case study we have paid out complete amount (USD 5000) to our selected technical writer. All in all, we spent (or rather allocated) the budget in a pace conforming to the plans as to the project did go forward at a nice progress.
Participants
The core Season of Docs team consisted of:
- Technical writer: Bhumit Attarde
- Support and mentoring: Ioannis Gonidelis, Hartmut Kaiser, Dimitra Karatza
We had one technical writer during the course of the project: Bhumit Attarde. We are very happy with Bhumit's work and hope that he will decide to stay with the project to help further improving the documentation in the future.
We advertised the acceptance of HPX to Season of Docs on our blog and asked writers to fill in a simple application form. Although the official project proposal was already written by us, we asked writers to include which parts of the proposal they would like to focus on, both in terms of interest and previous experience. Additionally, we asked writers for a brief overview of their background, and motivation to work on HPX in general and specifically as a writer for our documentation.
We hired our technical writer at the beginning of the project. We set up weekly calls with the technical writer, just like previous runs of Season of Docs and Summer of Code. This has been essential in keeping track of progress and making sure things are going in the right direction. Additionally, quick questions were typically handled on IRC, and more in-depth questions were handled either in the weekly calls, via email, or on GitHub issues. With the core team already spread out across the US and Europe, we are already used to the mostly asynchronous forms of communication. We found that the technical writer adapted quickly to this setting.
The writer was supported by two core HPX developers: Hartmut Kaiser and Ioannis Gonidelis. Dimitra Karatza is a member of our team since GSoD 2021 (she has decided to keep working on our documentation beyond last year's project) and has helped with issues and ideas related to this year's documentation work as well.
Timeline
The project had the following timeline:
- June: Debugging and repairing broken Public API links and partially documenting parallel algorithms.
- July: Continuing to document parallel algorithms, fixing function signature discrepancies with the C++ standard, updating meta documentation and CI system.
- August: Continuing to document parallel algorithms.
- September: Documenting various headers like
condition_variable,mutexand adding missing links to the public API page. - October: Adding CPPReference named requirements support, fixing documentation build system and removing leftover section of build script, updating Doxygen config to make the documentation cleaner, documenting remaining headers like
semaphore.
Results
It should be noted that we were able to significantly improve the build system, find documentation as well as implementation related discrepancies between the C++ standard API and HPX even though it wasn’t planned or included in the proposal. This is important because HPX strives to keep minimal (and ideally no) deviation from the standard API.
The changes made during the project are listed below with links to the corresponding pull requests. Overall, our technical writer created 16 pull requests, amounting to essentially one pull request every week of the project. All of those pull requests were accepted and merged to the master branch of our code base.
As a result of the work related to the build system and contiguous integration, we now publish the current state of the documentation here. This is automatically generated and uploaded for each commit to master.
The documentation for the latest released version of HPX is now available here.
Documenting parallel algorithms
| Pull request(s) | Exemplar documentation pages (if any) |
|---|---|
| PR 5939 | https://hpx-docs.stellar-group.org/branches/master/html/api/public_api.html#hpx-algorithm-hpp |
| PR 5931 | |
| PR 5952 | |
| PR 5966 | |
| PR 5975 |
Fixing broken public API page links
| Pull request(s) | Exemplar documentation pages (if any) |
|---|---|
| PR 5915 | https://hpx-docs.stellar-group.org/branches/master/html/api/public_api.html |
Finding and fixing implementation discrepancies between C++ standard and HPX
| Pull request(s) | Exemplar documentation pages (if any) |
|---|---|
| PR 5954 | |
| Issue 6008 |
Updating meta documentation
| Pull request(s) | Exemplar documentation pages (if any) |
|---|---|
| PR 5959 | https://hpx-docs.stellar-group.org/branches/master/html/about_hpx/people.html |
| PR 5960 |
Documenting various HPX C++ facilities
Adding missing links to public API page
| Pull request(s) | Exemplar documentation pages (if any) |
|---|---|
| PR 6012 | https://hpx-docs.stellar-group.org/branches/master/html/api/public_api.html |
Adding CPPReference named requirements support
| Pull request(s) | Exemplar documentation pages (if any) |
|---|---|
| PR 6020 | Example: https://hpx-docs.stellar-group.org/branches/master/html/api/public_api.html#hpx-mutex-hpp |
Changing Doxygen configuration to reduce boilerplate doc comments
| Pull request(s) | Exemplar documentation pages (if any) |
|---|---|
| PR 6022 |
Improving documentation build system and removing non-functional, leftover build scripts
| Pull request(s) | Exemplar documentation pages (if any) |
|---|---|
| PR 6025 | |
| PR 5915 |
Metrics
We had previously made a survey in general about the usage of HPX and received a fair number of responses. Thus, we decided to do a survey at the beginning of the documentation project to collect feedback on what parts of the documentation needed the most attention. We posted the survey on our mailing list and IRC channels but received zero responses. The initial plan was to use this survey to make focused changes, and then do a follow-up survey to receive feedback from users on whether they felt these changes had improved the documentation. This would have been our metric. However, given that we had no "official" responses we instead turned to talking directly to the users that we knew. We received concrete feedback from Summer of Code students, who commented that getting started with HPX can be difficult and also that the API documentation is difficult to navigate. Hence, we decided to focus on enhancing these sections during the project. We also asked for feedback directly from an external user who commented that more examples are always helpful.
To follow up on these changes, we will see how many related issues we will receive on GitHub over the next year.
One obvious metric is the amount of pull requests created and merged over the course of the project. Overall, our technical writer created 16 pull requests, all of which were reviewed, accepted, and merged to the master branch of our code base.
Analysis
We received a significant number of changes to our documentation aimed at newcomers to HPX: the quickstart guide, examples, and the build instructions. We made progress on the structure, the content, and the overall quality of the documentation. We have resolved many of the discrepancies between our API documentation and the C++ standard. The work on the API documentation has revealed a number of code related issues as well (see for instance: https://github.com/STEllAR-GROUP/hpx/issues/6028). All of these changes make this year's Season of Docs a success from our perspective.
Our technical writer had only some previous technical writing experience prior to the project but he still showed great enthusiasm for the project which helped us get to work quickly. We hope that he will be interested in continuing to help with our project. Given the time he has invested into learning about our documentation, we believe this will be very valuable as he'd be able to continue focusing on improving the content instead of dealing with the inevitable technical difficulties that come up at the beginning of a project.
Summary
In terms of plain project management, we think our prior experience with Summer of Code and Season of Docs has been valuable. The lessons we have learned in the past about setting short term goals, requiring weekly meetings, and encouraging students to interact with others in the community on IRC has paid off previously and did so this time as well.
Finding experienced technical writers has been difficult and if we decided to participate in the future we would definitely have to consider searching and advertising far outside the regular HPX communication channels. We took a chance on a writer that we believed may work well for us and we were lucky to find a person willing to invest sufficient time and effort to get up to speed quickly.
Overall, we think the documentation changes themselves have been a success. A lot of work remains to be done given the size of the HPX project itself and the amount of existing documentation we have. However, we hope that by focusing on the beginners and the conformance to the C++ standard in this project, we'll be able to motivate more people to try HPX out and hopefully expand both the user and developer community of HPX. We highly recommend projects just starting out with their documentation to invest time in thinking about their documentation from the perspective of newcomers to the project. Best is to talk to people who have just gone through the process of using your project. We've found that people tend to be quite open to talking about these things if you ask them.