demgen Wiki EN - CrusheerPL/demgen GitHub Wiki

This is English wiki for demgen.

Contents:

  1. Installation
  2. Program operation
  3. About demgen

Installation

Program can be installed in two ways:

How to build from source code:

  1. Install latest supported by your OS version of Visual Studio (min. 2017) with C compiler & Windows SDK
  2. Install the Python environment (obligatory with selected Add Python to PATH option!) and modules mentioned in the Requirements section of README.md file

    • command in CMD running as administrator:

        python -m pip install <module_name>

    • in case of installation of Pillow's developer version:

      • download the contents of its repo, unzip to C:\ directory

      • open CMD as administrator and run commands according to this pattern (one line = one command, following given order is important):

        set PYTHON=<main_Python_directory_path>
cd /D <main_Pillow_directory_path>\winbuild
"%PYTHON%\python.exe" build_prepare.py -v --depends=C:\pillow-depends
build\build_dep_all.cmd
build\build_pillow.cmd bdist_wheel
path C:\Pillow\winbuild\build\bin;%PATH%
"%PYTHON%\python.exe" -m pip install <new_whl_file_path>.whl
"%PYTHON%\python.exe" selftest.py

        • Example: Python 3.8.10 installed in C:\Program Files\Python38 directory, main Pillow folder is placed in C:\

          set PYTHON=C:\Program Files\Python38
cd /D C:\Pillow-main\winbuild
"%PYTHON%\python.exe" build_prepare.py -v --depends=C:\pillow-depends
build\build_dep_all.cmd
build\build_pillow.cmd bdist_wheel
path C:\Pillow\winbuild\build\bin;%PATH%
"%PYTHON%\python.exe" -m pip install C:\Pillow-main\dist\Pillow-9.3.0.dev0-cp38-cp38-win_amd64.whl
"%PYTHON%\python.exe" selftest.py

    • in case of installation of PyInstaller's developer version - in CMD opened as administrator run this command:

        python -m pip install https://github.com/pyinstaller/pyinstaller/tarball/develop
  3. Download demgen's source code (from repo or from Releases site) and unzip anywhere
  4. You will see a new folder with a.o. the source code (demgen.py) and two .spec files with PyInstaller configuration:

    • Open CMD as administrator and change the directory to that (example of command: cd C:\demgen-master)

    • To build a distribution packet with the program, run:

        pyinstaller demgen.spec

    • (not recommended) Or if you want to compile a single executable file, run:

        pyinstaller demgen_onefile.spec
  5. Files ready to use are in the dist directory.
  6. When the cannot import name 'Blosc' from 'numcodecs' error message appears after starting the demgen.exe - edit used .spec file by removing the comment mark from the line #10 and repeat step 4.

Program operation

With every start-up demgen checks if the Texconv tool and GIANTS Texture Tool are present in the current working directory (main app folder). The first one is being downloaded from GitHub automatically when it's missing there. In case of lack of the last one, this yes/no dialog will be displayed:

wiki_en_01.png

If you clicked Yes, you should indicate the location where the required files (textureTool.exe and textureTool.xml) are to be searched for (previously downloaded ZIP from GDN has to be extracted). If they cannot be found or the searching will be cancelled during directory selection or the No button in that dialog will be pressed, you will see this message:

wiki_en_02.png

This screenshot presents an appearance of main app window:

wiki_en_03.png

Creating, opening and saving the configuration

To input the necessary data you should enter desired value in the appropriate field and press ENTER (or Tab if you want to go to the next field).

To check the location of area of your interest on the map, you can click Open KML - the G**gle Earth Pro will be open (if installed) with generated plane. You can correct entered data to achieve the desired result.

Created in that way configuration can be saved as XML file with any name - press Save configuration to do this.

It can be also loaded if you press ... next to the field named Configuration path:

wiki_en_04.png

Attention:

  1. After loading the XML file created on other device, changing the output folder path is recommended. If that path refers to non-existent partition, it will be set to default (see the list below).
  2. If the XML file is invalid (contains syntax errors), you will see this message:

wiki_en_05.png

  1. If any parameter is missing in XML file or its value is unacceptable (e.g. letters in numbers), it will be set to default (see the list below).

Config file structure:

<?xml version='1.0' encoding='utf-8'?>
<demgenConfig>
  <country value="\country\"/>
  <centralPoint latitude="\latitude\" longitude="\longitude\"/>
  <terrain size="\dimensions\" metersPerPixel="\resolution\" rotation="\rotation (CW)\"/>
  <dataSavingFolder path="\output/folder/path\"/>
</demgenConfig>

Example:

<?xml version='1.0' encoding='utf-8'?>
<demgenConfig>
  <country value="PL"/>
  <centralPoint latitude="50.428465" longitude="21.281471"/>
  <terrain size="1024" metersPerPixel="2.0" rotation="21.37"/>
  <dataSavingFolder path="K:/demgen/test2"/>
</demgenConfig>

Default values:

  • Country: PL - Poland
  • Map central point coordinates:
    • latitde: 0.0º
    • longitude: 0.0º
  • Terrain parameters:
    • dimensions: 2048 m
    • resolution (Units per Pixel parameter in GIANTS Editor): 2.0 m/px
    • Rotation (CW = clockwise) - 0.0º
  • Output folder: demgen_data in main program directory.

Landform generation (DEM & 3D objects)

To realize this process you need the elevation data - click Collect elevation data first. Downloading and processing it may take some time (depended on size of requested area, internet connection speed and platform's performance) and ends with this message:

wiki_en_06.png

Attention - Slovakia only: if the country is set in configration to SK - Slovakia, you will be additionally asked for GeoTIFF file with elevation data you have to download manually:

  • go to the ZBGIS® Map Client (https://zbgis.skgeodesy.sk/mkzbgis/en/teren/export)
  • check if required data is available for area of your interest (areas with green border = data is available)
  • change these options: Select Export Region with (Výber priestorového rozsahu) -> Shape (ohradou), File Format (Formát údajov) -> TIFF
  • enter your e-mail address in E-mail field
  • select I agree to the license terms. (Súhlasím s licenčnými podmienkami.)
  • draw the area of your interest (must be less than 400 km²; finish with double click), all should be more or less as on this screenshot:

wiki_pl_06.png

  • Click Export (Exportovať). A few moments later you will receive an e-mail with a download link (active for 24 hours after sending the message):

wiki_pl_07.png

  • Download ExportDMR.zip and extract it anywhere. A dmr.tif file from that archive is necessary and must be selected.

After you press Generate DEM, the program will create two map_dem PNG images - first one with 16-bit grayscale (map_dem_16bit.png), last with 8-bit RGB (map_dem_8bit.png); both contains the same earlier collected elevation data. The other result is generated text file named demStats with values of minimum and maximum elevation, their difference and Height Scale parameter. These stats are also displayed in the final message (example below):

wiki_en_07.png

Then replace your map's DEM with its new version (recommended use of map_dem_16bit.png).
If the Height Scale value from the demStats file differs from this declared in map i3D, open it in text editor, look for heightScale parameter in <TerrainTransformGroup line and change its value to the appropriate one.

In the map's XML file change the values of width and height parameters to these entered on input.

Optionally, to create 3D objects with landform divided into equal pieces (1024x1024 squares), click Generate 3D objects w/ landform - Wavefront OBJ files will be saved to the staticTerrain directory and can be imported to various 3D editors like Blender.

Creation and implementation of terrain textures

Temporary terrain textures created from aerial imagery are helpful a.o. during foliage painting or placing 3D objects (e.g. buildings).

The first step to get it is choosing the FS version for which the map will be made. Then, you should select elements to be downloaded (only aerial imagery, only hillshade (single image) or both).
Downloading starts after pressing the Download imagery button. Ready to use DDS textures will be saved to the textures directory.

Attention:

  • If none of those boxes is checked, the following message is displayed:

wiki_en_08.png

  • If you selected FS 22, the GIANTS Texture Tool should be present in the main program catalogue. Otherwise, PNG textures will be saved to the textures_png folder (you must convert them to DDS BC7 manually to use in your map) and you will see this message:

wiki_en_09.png

To generate and save weightmaps (*_weight.png files) for new textures, click Generate weightmaps and choose the folder with map data (their most frequent names: data, mapDE, map01, etc., but they contain earlier mentioned files).

In the assets directory you can find files which are necessary to implement newly created textures, divided into few folders (the content of each is adapted to the appropriate FS version standards).

In the directory with map's i3D create new folder named textures and inside, another named terrain - paste newly generated terrain textures (ortho**_diffuse) and (if it's not a FS 13 map) ortho_normal DDS image from assets catalogue there.
FS 19 & 13 only: download ZIP with resized terrain textures (select a file corresponding to your blank map) and extract it to the folder with earlier pasted ortho**_diffuse (unless otherwise stated - check README.txt attached to ZIP w/ textures if it's there):

tx11.png

tx12.png

It's time to edit the i3d in text editor:

  • paste additional lines from *x*_FS**_extraLines XML file (suitable to FS version and terrain size) into its certain sections. Don't forget to adjust paths to the weightmaps (ortho**_weight) to be consistent with the actual state.
  • FS 22 only: find a line with path to terrainShader.xml and remove $data/shaders/ from it (how to - see screenshot below), additionally paste modified terrainShader.xml from assets/FS22 to the folder with map i3d file.

wiki_en_10.png

  • FS 19 only: remove $data/maps/ from paths to terrain textures ($data/maps/textures/terrain/ except these with ground/ added next). Result (order of paths might differ):

tx14.png

Delete files with .cache extension (if are there) from directory with map i3d:

96.png

And now terrain textures from aerial imagery should be visible in GIANTS Editor and FS.

About demgen

Version: 1.0.1. Authors:

Changelog

v1.0.1 (2022-12-02):

  • corrected the interpretation of input coordinates in "degrees, minutes, seconds" format without direction indicator (N/S, E/W)
  • PL: changed address of the ortho imagery downloading service, the previous one will return after adding the proper function to the program

v1.0.0 (2022-09-04):

  • initial release

Files generated by this program are compilations of publicly available data (ALS Products and imagery) from these institutions:

Compiled packages only:

  • built-in Python 3.8.10 environment (an open source solution) has been used in accordance with the Python Software Foundation licence
  • built-in third-party Python packages required for the correct program operation (open source solutions) have been used in accordance with their respective licenses - more informations in LICENSES_builtInSoftware.txt.

Any modifications allowed as well as copying the content of this repo to the various file hosting services, provided the original author and source are indicated. If you encounter any issue - let me know.
You use this program only at your own risk.