MAPL Documentation Standards with FORD - GEOS-ESM/MAPL GitHub Wiki

Table of contents generated with markdown-toc

Abstract

This report describes the standards we can follow to document the MAPL source code using FORD tags. In particular, we present where and how appropriate comments can be included to automatically generate documentations with FORD.

1 Introduction

Each software, whether created by a small group or by several teams, needs some related documentation. The documentation needs to explain why the software was designed, how it works, and how it is intended to be used. The main of purpose of software documentation is to streamline the communication between all the parties involved with the use of the software. TutorialPoints lists the advantages of software documentation:

  • Keeps track of all parts of a software or program
  • Maintenance is easier
  • Programmers other than the developer can understand all aspects of software
  • Improves overall quality of the software
  • Facilitate the software component reusability.
  • Aids in helping the end-users and assists in user training
  • Ensures knowledge de-centralization and knowledge capturing.

The challenge that we have is how do we produce such a documentation and update it specially while the software still evolves. We cannot properly describe what a dynamic software does if we cannot automatically generate from it useful information that can be included in the documentation. Software documentation is a critical process in the overall software development process. We need to establish standards and processes to write appropriate comments in the software that will be used to provide a comprehensive description of the software components.

Documenting scientific software has its own requirements [1][2][3]. One issue that arises is to define what good documentation actually involves, what it should contain and and how it should be done. Also, there is a need to clarify who has the responsibility to document the code, for whom, what, where, when and how. Any scientific software documentation needs to be at least usable, reliable and maintainable. In addition, it clearly describes how the software is used and its limitations.

In the case of MAPL, we want to have a documentation that is automatically derived from the source code with the objective of having up-to-date, consistent (a statement on an item made at one place in a document should not contradict the specification of the same item at another location), modifiable ( future changes do not destroy the structure of the document), traceable and accurate documentation as modifications are made in the code. We need to have the following requirements in terms of commenting out MAPL source code:

  • Comments attached to program units, variables and derived types may be automatically documented.
  • Documentation must precede the declaration of the unit, variable or derived type.
  • Comments shall be indented to align with the code.
  • Comments shall explain what a code does and why, not how it does it. The code needs to explain itself how it is done.
  • Documentation should be updated without disruption the software development process.
  • LaTeX style maths is used to include equations in the documentation.
  • Generate directory structure and call graphs.
  • Generate documentation that can be inspected without looking into the code (e.g. HTML-pages, PDF-document, etc.)
  • Generate searchable and browsable documentation.

We have decided to select FORD as the tool to document MAPL. In this document, we define simple and uniform standards for commenting out the code so that documentation takes place in everyday work and becomes part of the process in the development of MAPL. We want to make it easy to write the documentation at the same time as writing the code. In each program unit, we want to describe its intent, its input parameters, return values and usage examples (if possible).

2 Description of FORD

The goal of FORD is to be able to reliably produce documentation for modern Fortran software which is informative and nice to look at. The documentation should be easy to write and non-obtrusive within the code. While it will never be as feature-rich as Doxygen, hopefully FORD will be able to provide a good alternative for documenting Fortran projects.

FORD allows documentation to be placed in the README.md files as well as the source files. FORD processes markup describing classes, procedures, and variables in the header/source files and generates a user-friendly hyperlinked set of web pages. These web pages allow the user to navigate all the program units, classes, procedures, and variables and see 1) class hierarchy relationships; 2) input and output parameters for methods and procedures; and 3) other variables and associated data.

Here are the FORD capabilities:

  • An automatic documentation generator for modern (1990 onward) Fortran code.
  • Was created to fixed the limitation of Doxygen to handle new features of Fortran.
  • Able to extract information about variables, procedures (functions and subroutines), procedure arguments, derived types, programs, and modules from the source code.
  • Source code can also be preprocessed;.
  • Able to extract documentation from comments in the source code.
  • Uses Markdown to type-set documentation.
  • Able to use LaTeX (through MathJax)
  • Able to create a hierarchical set of pages containing general information, not associated with any particular part of the source code.
  • Able to include the contents of other files within the documentation.
  • Symbols (e.g. main code, modules, derived data types) are appropriately colored.
  • Provide links:
    • to download the source code.
    • to individual files, both in their raw form or in HTML with syntax highlighting.
    • between related parts of the software.
  • Use configurable settings to generate documentation. The settings allow users to define which types of comments can be used for documentation.
  • Can add GitHub (Bitbucket), Twitter and LinkedIn pages.
  • Searchable documentation using Tipue Search which supports a wide range of Web browsers. Can search source code as well as procedures, variables, comments, etc.

FORD usage is based on projects. A project is just whatever piece of software you want to document. Normally it would either be a program or a library. Each project will have its own Markdown file which contains a description of the project.
Various options can be specified in this file, such as where to look for your projects source files, where to output the documentation, and information about the author. Some non-Markdown syntax can also be used with FORD.

3 Templates

Special Fortran comments to be processed by FORD start with !> before any procedure, variable and program unit to be documented. Subsequent comments proceed ! until the last one is encountered. In addition, each variable is documented with !! on the same line the variable is declared and !! for continuation line.

!>
! Compute the area of a triangle using the Heron's formula:
!
!$$
!\begin{equation}
!s  = \frac{a+b+c}{2}  \\\\
!A  = \sqrt{s(s-a)(s-b)(s-c)}
!\label{eq:tri}
!\end{equation}
!$$
!
   real function calc_area( a, b, c )
      ! This line will not be documented.
      real, intent(in)  :: a !! Opposite
                            !! More comments on the opposite.
      real, intent(in)  :: b !! Adjacent
      real, intent(in) :: c  !! Hypothenus
      real :: s
      s = (a + b + c) / 2.0
      calc_area = sqrt(s*(s-a)*(s-b)*(s-c))
   end function calc_area
   
!> 
! Compute the hypothenus of a right angle triangle.
! 
! The length of the hypotehnus is given by: 
! \( c = \sqrt{a^2 + b^2} \)
! 
! This is the last line of the commented section.
subroutine compute_hypothenus(a, b, c)
    real, intent(in)   :: a !! Base
    real, intent(in)   :: b !! Perpendicular
    real, intent(out) :: c  !! Hypothenus
    c = sqrt(a**2 + b**2)
end subroutine compute_hypothenus

3.1 Main Program

!------------------------------------------------------------------------------
!               Global Modeling and Assimilation Office (GMAO)                !
!                    Goddard Earth Observing System (GEOS)                    !
!                                 MAPL Component                              !
!------------------------------------------------------------------------------
!>
!# Short title 
!
! `program_name` is used for ...
! Provide a description of what the program does.
!
! @note
! `Usage`:
!
!   Briefly describe how to use the program.
!
!       mpiexec -np 56 pfio_MAPL_demo.x --npes_model 28 --oserver_type multigroup \
!                               --nodes_output_server 1 --npes_backend_pernode 5
! @endnote
!
! @todo
! Report anything to be done here.
!@endtodo
!
! @bug
! Report any bug here.
!@endbug
!
program program_name

3.2 Modules

!------------------------------------------------------------------------------
!               Global Modeling and Assimilation Office (GMAO)                !
!                    Goddard Earth Observing System (GEOS)                    !
!                                 MAPL Component                              !
!------------------------------------------------------------------------------
!>
!### MODULE: `module_name`
!
! Author: GMAO SI-Team
!
! `module_name` is an abstract base class representing
! individual values of VariableSpec properties.  Concrete subclasses
! are used to encapsulate various intrinsic types (logical, integer,
! real, string) and allow more generic access to those values
!
! Special interfaces are provided to support allocatable strings which
! unfortunately cannot use the usual interface.  Instead all
! subclasses have both the generic and string-specific interfaces and
! the BaseSchemaElement subclass provides concrete implementations
! that return exceptions.  Concrete subclasses should override as
! appropriate.
!
!#### History
!- 09Nov2022: First implementation.
!
! @todo
! Report anything to be done here.
!@endtodo
!
! @bug
! Report any bug here.
!@endbug
!
module module_name
   implicit none
   private

   public :: public_unit

   type, abstract :: AbstractSchemaElement
   contains
      procedure(i_has_default), deferred :: has_default
      procedure(i_accepts_value), deferred :: accepts_value
      procedure(i_get_default_scalar), deferred :: get_default_scalar
      procedure(i_get_default_string), deferred :: get_default_string
      generic   :: get_default => get_default_scalar
      procedure(i_copy_value), deferred :: copy_value
      procedure(i_copy_string_value), deferred :: copy_string_value
   end type AbstractSchemaElement

   abstract interface

3.3 Subroutines/Functions

!------------------------------------------------------------------------------
!>
! For a given number of grid points along a dimension and a number of
! available processors for that dimension, determine the number of
! grid points assigned to each processor.
!
!### Example
! The subroutine can be used as:
!```fortran
!    allocate(dim_array(16))
!    call decompose_dim(181, dim_array, 16)
!```
!
   subroutine decompose_dim(dim_world, dim_array, num_procs)

      integer, intent(in)  :: dim_world    !! total number of grid points
      integer, intent(in)  :: num_procs    !! number of processors
      integer, intent(out) :: dim_array(0:num_procs-1) !! Array containing the
                                                            !! number of grids assign to each processor

      integer ::   n, im, rm
!------------------------------------------------------------------------------
      im = dim_world/num_procs
      rm = dim_world-num_procs*im
      do n = 0, num_procs-1
                      dim_array(n) = im
      if( n.le.rm-1 ) dim_array(n) = im+1
      enddo
   end subroutine decompose_dim
!------------------------------------------------------------------------------

3.4 Derived Types

    !> 
    ! Docstrings can go on derived types
    ! and you can put more comment here
    ! and end here with the derived type comment.
    type :: my_type
        real :: R_major        !! Docstrings can include in-line LaTeX 
                               !! like this: \(R\)
        real :: R_major_prime  !! Or as a displayed equation:
                               !! $$\frac{\partial R}{\partial \psi}$$
        integer :: n_R         !! Code-formatting uses `backticks`
    contains
        procedure :: my_read_var  => read_var  !! Read my variable
        procedure :: my_write_var => write_var !! Write my variable
    end type my_type

3.5 Important Remarks

Use of !!

  • We use !! only for the description of dummy arguments of routines.
  • Do not use !! anywhere else in the code. Otherwise, FORD will document that line.

New Line in Dummy Argument Description

If we want the description of a dummy argument to span over several lines, we need to include at least three (might be two) white spaces at the end of each line with !!. For instance, in the description of the argument a, several white spaces are added at the eten of !! Opposite.

   real function calc_area( a, b, c )
      ! This line will not be documented.
      real, intent(in)  :: a !! Opposite    
                            !! More comments on the opposite.  
      real, intent(in)  :: b !! Adjacent
      real, intent(in) :: c  !! Hypothenus
      real :: s
      s = (a + b + c) / 2.0
      calc_area = sqrt(s*(s-a)*(s-b)*(s-c))
   end function calc_area

#nclude Statement

All #include statements need to be placed before FORD documentation sections (starting with !>). Otherwise, FORD will try to include the content of the include file in the documentation.

4 FORD Project File

FORD uses a project file to set options needed to generate documentation. The available yaml-like options for the file are shown at:

https://github.com/Fortran-FOSS-Programmers/ford/wiki/Project-File-Options

We assume that the MAPL top level folder has a directory called doc/ that only contains the project file mapl_ford_project.md. We recommend using the following settings (among many others) to ensure that the source code comments are properly processed by FORD.

---
parallel: 4
src_dir: ../MAPL_code
preprocessor: /usr/local/bin/cpp-12 -traditional-cpp -E
search: true
graph: true
coloured_edges: true
graph_maxdepth: 4
graph_maxnodes: 32
include: ../MAPL_code/include/
         ../gFTL/install/GFTL-1.8/include/v1
         ../gFTL/install/GFTL-1.8/include/v2
exclude_dir: ../MAPL_code/Ford
             ../MAPL_code/Doxygen
             ../MAPL_code/ESMA_cmake
             ../MAPL_code/ESMA_env
             ../MAPL_code/build
             ../MAPL_code/gFTL
macro: USE_MPI=1
       BUILD_WITH_PFLOGGER=1
       BUILD_WITH_EXTDATA2G=1
       USE_FLAP=1
       H5_HAVE_PARALLEL=1
       TWO_SIDED_COMM=1
       MAPL_MODE=1
fixed_length_limit: false
source: true
display: public
         private
         protected
extra_mods: iso_fortran_env:https://gcc.gnu.org/onlinedocs/gfortran/ISO_005fFORTRAN_005fENV.html
            iso_c_binding:https://gcc.gnu.org/onlinedocs/gfortran/ISO_005fC_005fBINDING.html#ISO_005fC_005fBINDING
project: MAPL
project_github: https://github.com/GEOS-ESM/MAPL
project_website: https://github.com/GEOS-ESM/MAPL
summary: MAPL is a foundation layer of the GEOS architecture, whose original purpose is to supplement the Earth System Modeling Framework (ESMF)
author: GMAO SI-Team
author_description: GEOS Software Infrastructure Team
github: https://github.com/JulesKouatchou
email: [email protected]
print_creation_date: true
sort: type-alpha
predocmark_alt: >
predocmark: <
docmark_alt:
docmark: !
md_extensions: markdown.extensions.toc
               markdown.extensions.smarty
extensions: f90
            F90
            pf
fpp_extensions: F90
                pf
                F
---

{!../MAPL_code/README.md!}

References

[1] Benjamin D. Lee (2018) Ten simple rules for documenting scientific software, PLoS Comput Biol., 14(12): e1006561, https://doi.org/10.1038/s41598-022-10376-9.

[2] Sibylle Hermann and Jorg Fehr (2022) Documenting research software in engineering science, Sci Rep, 8, 6567, [https://doi.org/10.1371/journal.pcbi.1002802](https://doi.org/10.1371/journal.pcbi.1002802].

[3] W. Spencer Smith and Nirmitha Koothoor (2016) A Document-Driven Method for Certifying Scientific Computing Software for Use in Nuclear Safety Analysis, Nuclear Engineering and Technology, 48(2), p. 404-418, https://doi.org/10.1016/j.net.2015.11.008.

A Documenting the Module MAPL_CapOptionsMod

It is required that all #include statements come before the FORD documentation section, i.e., anything starting with !>.

!------------------------------------------------------------------------------
!               Global Modeling and Assimilation Office (GMAO)                !
!                    Goddard Earth Observing System (GEOS)                    !
!                                 MAPL Component                              !
!------------------------------------------------------------------------------

#include "MAPL_ErrLog.h"
#include "unused_dummy.H"
!
!>
!### MODULE: `mapl_CapOptionsMod`
! Contains functions to initialize and copy command line parameters.
!
module mapl_CapOptionsMod
   use ESMF
   use mapl_KeywordEnforcerMod
   use mapl_ExceptionHandling
   implicit none
   private

   public :: MAPL_CapOptions

   !> Derived type used to capture all the command line parameters
   ! needed to configure a run.
   type :: MAPL_CapOptions

      integer :: comm
      logical :: use_comm_world = .true.
      character(:), allocatable :: egress_file
      character(:), allocatable :: cap_rc_file
      character(:), allocatable :: root_dso
      type (ESMF_LogKind_Flag) :: esmf_logging_mode = ESMF_LOGKIND_NONE
      integer :: npes_model = -1
      integer, allocatable :: npes_input_server(:)        !! only one of the next two options
                                                          !! can have nonzero values
      integer, allocatable :: nodes_input_server(:)
      integer, allocatable :: npes_output_server(:)       !! only one of the next two options
                                                          !! can have nonzero values
      integer, allocatable :: nodes_output_server(:)
      logical              :: isolate_nodes = .true.      !! whether or not the nodes are
                                                          !! padding with idle when mod(model
                                                          !! total npes , each node npes) /=0
      logical              :: fast_oclient  = .false.     !! whether or not copy the data
                                                          !! before isend to the oserver
                                                          !! it is faster but demands more
                                                          !! memory if it is true
      logical              :: with_io_profiler = .false.  !!  whether or not turn on the io profiler
      logical              :: with_esmf_moab = .false.    !! whether or not to use MOAB in ESMF
      integer :: n_iserver_group = 1                      !! server groups
      integer :: n_oserver_group = 1
      integer :: n_members = 1                            !! ensemble options
      character(:), allocatable :: ensemble_subdir_prefix
      character(:), allocatable :: logging_config         !! logging options
      character(:), allocatable :: oserver_type
      integer :: npes_backend_pernode = 0

   end type MAPL_CapOptions

   interface MAPL_CapOptions
      module procedure new_CapOptions
      module procedure new_CapOptions_copy !! for backward compatibility - delete for 3.0
   end interface MAPL_CapOptions

!------------------------------------------------------------------------------
contains
!------------------------------------------------------------------------------
   !> Initialize the MAPL Cap parameters.
   function new_CapOptions(unusable, cap_rc_file, egress_file, ensemble_subdir_prefix, &
                           esmf_logging_mode, rc) result (cap_options)
      type (MAPL_CapOptions) :: cap_options
      class (KeywordEnforcer), optional, intent(in) :: unusable
      character(*), optional, intent(in) :: cap_rc_file
      character(*), optional, intent(in) :: egress_file
      character(*), optional, intent(in) :: ensemble_subdir_prefix
      type(ESMF_LogKind_Flag), optional, intent(in) :: esmf_logging_mode

      integer, optional, intent(out) :: rc

      _UNUSED_DUMMY(unusable)

      cap_options%cap_rc_file = 'CAP.rc'
      cap_options%egress_file = 'EGRESS'
      cap_options%oserver_type= 'single'
      cap_options%ensemble_subdir_prefix = 'mem'

      cap_options%npes_input_server  =[0]
      cap_options%nodes_input_server =[0]
      cap_options%npes_output_server =[0]
      cap_options%nodes_output_server=[0]

      if (present(cap_rc_file)) cap_options%cap_rc_file = cap_rc_file
      if (present(egress_file)) cap_options%egress_file = egress_file
      if (present(ensemble_subdir_prefix)) cap_options%ensemble_subdir_prefix = ensemble_subdir_prefix
      if (present(esmf_logging_mode)) cap_options%esmf_logging_mode = esmf_logging_mode

      _RETURN(_SUCCESS)

   end function new_CapOptions
!------------------------------------------------------------------------------
   !> Copy the MAPL Cap options.
   function new_CapOptions_copy(options) result(copy)
      type(MAPL_CapOptions) :: copy
      type(MAPL_CapOptions), intent(in) :: options
      copy = options
   end function new_CapOptions_copy
!------------------------------------------------------------------------------

end module MAPL_CapOptionsMod
⚠️ **GitHub.com Fallback** ⚠️