Verification Case Setup Example - firemodels/fds GitHub Wiki

An example step-by-step procedure for authoring a verification case, from setting up the input files to compiling the FDS Verification Guide.

Before getting started, are you conversant in Matlab and LaTeX. No? Do yourself a favor: take a week out of your busy life and learn the basics of these tools. I guarantee your investment will payoff a thousand fold!

The purpose of this page is to document, through example, all of the steps required to get a verification case from conception to full automation, such that the FDS Verification Guide can be built using an automated script. Similar procedures also apply to the FDS Validation Guide. Here we will follow the development of the "heated channel" test case series developed by Kiyoung Moon of Yonsei University.

One difference between the verification suite and the validation suite is that the user is expected to run all the verification cases in order to compile the guide. That is, the output files necessary to generate the plots in the guide are not committed to the repository. For validation this is not the case. All necessary validation output files are committed and just need to be processed by the appropriate Matlab scripts.

Take note of the files that need to be committed. In the text below, these files will be highlighted with a (commit) tag. The most common problem we encounter when compiling the guides is that an author will assume their case is fully committed because they are able to compile the guide on their machine, but if they have not committed all the necessary files the guide will not compile for the other project members.

Let's get started.

For this particular test series Kiyoung wants to compare FDS results for the mean streamwise temperature profile in the wall-normal direction to the heated channel flow DNS calculations of Kim and Moin (1987). He will run four cases at different molecular Prandtl numbers (Pr). His goal is to verify the implementation of a new wall heat flux model.

The first thing to think about is what the case names will be. This is more important than it may seem at first. The reason is that the names work their way into the depths of the processing scripts through attachments to output data files, etc., and making any changes later can prove to be a nightmare. So, do all of us a favor and think carefully about the naming convention. The names ought to be descriptive, but somewhat compact. Here are a few guidelines:

• Do not use spaces in the names, use underscores instead: heated_channel, not heated channel.
• If the case is a series, include the variable parameter in the name.
• Use "p" for a decimal point: 0p71, not 0.71
• If there are different grid resolutions in the series, include some description in the name.
• Use "fds" as the filename extension.

Kiyoung's four cases are named:

heated_channel_Pr_0p10_32.fds
heated_channel_Pr_0p71_32.fds
heated_channel_Pr_1p00_32.fds
heated_channel_Pr_2p00_32.fds

The "32" relates to the number of grid points: these cases are 32³. Later, it will be easy to add 64³ cases with the same convention.

Next, think about which verification subdirectory should be used to store the input files. In this example, we could choose Verification/Heat_Transfer or Verification/Turbulence. We choose the latter, since the purpose of the test cases is to verify a new heat flux model for turbulent flow.

Note that ALL verification cases must live within a directory one level below Verification. DO NOT create sub-directories under Turbulence, for example, to help organize the cases. If new directories are needed, put them under Verification.

Copy all the input files in the test series to the verification sub-directory.

• The first issue to discuss about input files is the text editor used to create them. Be careful not to commit DOS files with "^M" carriage returns. This happens commonly with Windows-based text editors. The safe bet is to look at your files in vi [-b option] before committing.
• Make sure your CHID is the same as the input filename (minus the extension).
• Minimize input parameters. Do not add a parameter to the input file if it is already the FDS default.
• Minimize comments. Some users write dissertations inside their input files. Save this for the guide write up.
• Minimize the size of the output files. In most verification cases we do not need many slice files or time history devices. When possible, use line devices (POINTS=...) to gather statistics. This both reduces the clutter in the input file and reduces the size of the output. Note that the device files for verification cases are not committed---this issue is more relevant to validation cases---but it is a good habit to get into.

For the cases to run automatically in the verification suite, they must be added to the master list in Verification/FDS_Cases.sh. This list is organized alphabetically by subdirectory name. Append your cases to the end of the appropriate subdirectory. In this case, Kiyoung added the following:

$QFDS -d Turbulence heated_channel_Pr_0p10_32.fds
$QFDS -d Turbulence heated_channel_Pr_0p71_32.fds
$QFDS -d Turbulence heated_channel_Pr_1p00_32.fds
$QFDS -d Turbulence heated_channel_Pr_2p00_32.fds

Once you are happy with the input files and they run with the debug version of the latest code, you are ready to commit them to the repository. Do not forget to update your repository just prior to committing---someone else might be editing the master list. Commit all input files and the FDS_Cases.sh file.

In this section we will discuss the process of comparing the FDS results with an expected or (even better) analytical solution. There are a few ways this can be done. The easiest path is if the expected results and the FDS results can be plotted with the Utilities/Matlab/scripts/dataplot.m Matlab script. This is usually the case if we are just looking at profiles. However, if a significant amount of post-processing calculations must be done before direct comparisons can be made, it may be necessary to create your own Matlab script to generate the plots for the FDS Verification Guide. Note that at present Matlab is the standard. This is the only way we have to make everything look identical throughout the guides.

If you have an analytical solution or data for comparison, it is best to create a simple, comma-delimited file (.csv) with the data organized into columns with simple, clear header names. Basically, the same naming conventions follow within the data file as we listed above for input filenames: no decimals, spaces, etc. Once the data have been organized, you may commit the file to the same verification sub-directory where the input files live. In this case, we are committing:

Verification/Turbulence/heated_channel_dns_data.csv

Because there are literally thousands of plots in the FDS guides, it is important that there exists some level of continuity for the reader and we are therefore very picky about how the plots look. All the things you have heard over the years about high quality graphics and good presentations come into play here. If you are rusty, have a look at Edward Tufte's web site.

The easiest thing to do when adding verification cases is to organize your output so that the dataplot.m script can be used to generate the plots. In this way, all the fonts, line widths, etc., are handled automatically. To add a plot via dataplot, you must add a line to the spreadsheet FDS_verification_dataplot_inputs.csv in the Utilities/Matlab directory. The entries are listed in alphabetical order based on the name of the case in the second column. This name is important because you must use this exact name as the LaTeX section label in either the FDS Verification Guide or the FDS User's Guide. The reason for this is that there is a table in the Appendix of the FDS Verification Guide that lists all the verification cases processed by the dataplot script. The table contains links to the sections where the cases are described. So long as the LaTeX label is consistent with the entry in the column labelled Dataname in FDS_verification_dataplot_inputs.csv, the link will be created automatically. If it's not, you will get an error when you compile the manuals.

The column labelled Plot_Filename in FDS_verification_dataplot_inputs.csv indicates where you want the plot to be saved. It will be saved in the Manuals directory, NOT the Verification directory. In our case, the plot should be printed to FDS_Verification_Guide/SCRIPT_FIGURES/plotname. Do not commit this file to the fds repository. It is generated automatically.

Plots that may change when the code is modified are stored in SCRIPT_FIGURES. Other images that are needed in the guides are permanently archived in the FIGURES directory.

Details of the structure of the file FDS_verification_dataplot_inputs.csv is found in the wiki Using the Matlab script dataplot.m.

If the results of your calculations require additional post-processing beyond what can be done by FDS, you might need to write your own Matlab script rather than use dataplot. If you are Matlab savvy, this is not too hard. If not, it is a great way to learn Matlab! The most difficult part of all this for the uninitiated seems to be keeping the repository directory structure straight in their heads while creating the plotting routines. This is critical, because ultimately the script will live here:

Utilities/Matlab/scripts

But... it will be launched via the master script which lives here:

Utilities/Matlab/

These relative directories can be confusing at first. But follow this example, and you will get things right from here on.

The first thing to do is read in the analytical (expected) data and the FDS data, perform whatever calculations are necessary, and plot the results. In the next section we will worry about the details of how the plot looks.

As usual, think of a good name for your script and add it to the master list of special scripts under "Special cases" in FDS_verification_script.m. While you are developing the script, it is usually helpful to comment out all the other calls above your script. This allows you to run your script in "production mode" from exactly the directory where it will live permanently. Mark my words: If you decide to do all your development offline in some other directory, you will have all kinds of trouble merging everything in once you think you are finished. Please, do not think that because you can generate an Excel plot that you are anywhere close to finished.

We actually have a couple of options for where we could add the script. If the type of problem you are working on is completely new, then adding the directly to the master list is probably best. For example, we could add our new script Utilities/Matlab/scripts/heated_channel.m heated_channel.m to the master list as follows:

% Special cases
... (other scripts, comment out during development)
run scripts/heated_channel

Note how the scripts must be run when launched from within the master script since heated_channel.m lives a level below.

Our problem happens to fit nicely with the other scripts that are run in the special script called wall_model.m. Therefore, we choose to add our script there:

% McDermott
% 5-20-2009
% wall_model.m
%
% This script generates all plots necessary for the wall model section
% of the verification guide.

law_of_the_wall          % Figure: The law of the wall
poiseuille_convergence   % Figure: Convergence for 2D Poiseuille flow
fds_moody_chart          % Figure: The FDS Moody chart
yplus                    % Figure: Title
heated_channel           % Figure: Heated Channel Flow

In this example, we are going to write our own plotting script because some degree of post-processing calculations are necessary to non-dimensionalize the velocity and temperature results for comparison to the DNS data. The script we created is heated_channel.m.

Here are a few things to remember when creating your own script:

• Add Utilities/Matlab/scripts/plot_style.m at the top of your script. This sets many of the default Matlab parameters, like FontSize and FontName. Refer to our example script to see how to use these parameters to control the axis fonts and so on.

• Check to make sure any necessary files exist and exit gracefully if they do not. This prevents subsequent cases from being omitted if one particular case fails in our nightly build. Here is a basic example to follow:

  if ~exist(filename) display(['Error: File ' filename ' does not exist. Skipping case.']) return end

• Add the git number to the top right of the plot. The X_VerStr_Position and Y_VerStr_Position are set differently for linear, loglog, semilogx, and semilogy plots. Browse the examples in the Matlab/scripts directory to find one that matches your situation and simply copy the syntax. The only thing you should need to modify is the Git_Filename. If you have a series of cases all on one plot, simple choose one of the case CHIDs as your filename.

• Print the plot to pdf. Here again, find an example in Matlab/scripts and copy it.

Before you are done with Matlab, you should be able to delete all the files in the Manuals/FDS_Verification_Guide/SCRIPT_FIGURES directory, cd (change directory) into the Utilities/Matlab directory within the Matlab Command Window and run `FDS_verification_script'.

A pdf of your plot should be created in Manuals/FDS_Verification_Guide/SCRIPTS_FIGURES. If you have gotten this far, congratulations, time to actually do some writing.

If your plot does not show up, or if Matlab chokes, the most likely culprit is the path, either the path you set to the data directory or the path you set to the output plot directory.

The LaTeX source for the guide is located in Manuals/FDS_Verification_Guide/FDS_Verification_Guide.tex. Edit this file with whatever editor you choose (I use TeXShop on a Mac). Decide where your section should reside. In our case, we will add a section to the Wall Flows chapter called Heated Channel Flow.

As with the Matlab script, conforming to the conventions we have established in the guide is basically a matter of copying and editing an existing section. Here are a few things to look out for:

• Follow the NIST Guide for the Use of the International System of Units (SI) for mathematical style.

• When starting the section, first clear the page, then add your section or subsection, and add the \textorpdfstring command. Note that you should add a section label for each case described in that section because the results of these cases are listed in a long table in the appendix of the FDS Verification Guide, and the LaTeX section link is automatically added to the table.

    \clearpage
    \subsection{Heated Channel Flow (\texorpdfstring{{\bf heated\_channel}}{heated\_channel})}
    \label{heated_channel_Pr_0p10_32}
    \label{heated_channel_Pr_0p71_32}
    \label{heated_channel_Pr_1p00_32}
    \label{heated_channel_Pr_2p00_32} 
  

• Group similar plots with the tabular environment. Look at other examples within the guide for side-by-side plots; most use a width of 3.2 inches. Add descriptive figure captions.

     \begin{figure}[h]
        \begin{tabular*}{\textwidth}{l@{\extracolsep{\fill}}r}
           \includegraphics[width=3.2in]{SCRIPT_FIGURES/heated_channel_uplus.pdf} &
           \includegraphics[width=3.2in]{SCRIPT_FIGURES/heated_channel_Tplus.pdf}
        \end{tabular*}
        \caption[Mean velocity and temperature profiles for heated channel flow]{\label{fig_heated_channel}  Comparison of $32^3$ FDS (lines with squares) with DNS (circles) for heated channel flow. (Left) Wall-normal velocity profile compared to DNS of Moser \emph{et al.} \cite{Moser:1999}. (Right) Wall-normal mean profiles of temperature for molecular Prandtl numbers (Pr) = 0.10, 0.71, and 2.0, compared to DNS of Kim and Moin \cite{Kim:1987}.}
     \end{figure}
  

• Use the following rules for captions and short captions (short captions show up in the table of contents):

  • End your actual figure or table caption with a period.
  • Do not use a period at the end of the short caption (this conflicts with the dots to the page number).
  • The short caption should not span more than one line, and it is better not to use the entire line.
  • If you reference a case name in the captions, use {\ct casename} to invoke Courier font.

• If you cite references (hard to imagine that you would not have one or two), then be sure to add these (in alphabetical order) to the bibtex database in Manuals/Bibliography/FDS_general.bib.

   @TECHREPORT{Kim:1987,
     author       = {J. Kim and P. Moin},
     title        = {{Transport of Passive Scalars in Turbulent Channel Flow}},
     institution  = {National Aeronautics and Space Administration},
     year         = {1987},
     month        = {September},
     type         = {Technical Memorandum},
     number       = {89463},
     address      = {Moffet Field, California}
   }
  

Make sure to double check the reference at the end of the document. Look for correct capitalization, consistent naming conventions (J. Kim not John Kim), etc. Sometimes you will need to put curly brackets around acronyms. For example, NASA might come out as Nasa if you do not use {NASA}.

Depending on your computer setup, there are different ways to compile the document. But the basic sequence of operations is the same. You have to run LaTeX once, then run BibTeX once, then LaTeX two more times to resolve all references. In Linux it looks like this (note that you do not include the extension for the bibtex run):

 $ pdflatex FDS_Verification_Guide.tex
 $ bibtex FDS_Verification_Guide
 $ pdflatex FDS_Verification_Guide.tex
 $ pdflatex FDS_Verification_Guide.tex

In between each run you will see a bunch of cryptic log messages that fly by too fast to read. Hence the next step.

Before you commit your edits to the guide, (of course first update your repository) in a bash shell cd to Manuals/FDS_Verification_Guide and run

 $ ./make_guide.sh

This script will compile the LaTeX document by running pdflatex, bibtex, pfdlatex, pdflatex, which is the sequence needed to resolve all references in the document. The script then checks for warnings or error messages associated with include files (like plots) that are not found. If all goes well, you will get something like

FDS Verification Guide built successfully!

If all is not well, you may see something like

! LaTeX Error: File SCRIPT_FIGURES/heated_channel_Tplus not found.

Once you have compiled the guide without errors or warnings, go ahead and update your repository and commit the .tex and .bib files.

If everything has been done correctly, then our automated user, Firebot, will be able to run your cases, generate your plots, and compile the guide while you slumber. If something has been missed, we will all wake to the following email: Firebot angry! Well, mildly upset. Not to worry. The great thing about the repository is that we all iterate and fix our errors as soon as possible. This is not to say we are cavalier about pushing up broken code or documents. But errors during development are somewhat expected.

Here is a checklist of the files we committed for this example:

• FDS input files, e.g., fds/Verification/Turbulence/heated_channel_Pr_1p00_32.fds
• Experimental or analytical data files, e.g., fds/Verification/Turbulence/heated_channel_dns_data.csv
• fds/Verification/FDS_Cases.sh
• fds/Utilities/Matlab/scripts/heated_channel.m (or add line to fds/Utilities/Matlab/FDS_verification_dataplot_inputs.csv)
• fds/Utilities/Matlab/scripts/wall_model.m (or add to master list in fds/Utilities/Matlab/FDS_verification_script.m)
• fds/Manuals/FDS_Verification_Guide/FDS_Verification_Guide.tex
• fds/Manuals/Bibliography/FDS_general.bib

Thanks for your contribution!