Attendees: JGA, EBO

The following are the initial minutes of the explanations given by JGA.

A current updated version of the install notes are located here

Notes:

• uWSGI is a Web Server Gateway Interface (WSGI) compatible applications and frameworks (used among the Python community). uWSGI can be deployed with its own integrated http server. Flask is a web micro-framework that uses uWSGI as web deployment option.
• The citygml2PSQL.py and building_server_processdb.py both update a database. Yet note that the usage of citygml2PSQL.py is at the command line level (its input is a file and it uses a shell based pipe mechanism) and is hence offline) whereas building_server_processdb.py is at the SQL protocol level (hence online or networked).
• The DTM terrain data is downloaded on the fly by iTowns (in the case of Lyon directly from Grand Lyon WMS server.
• The deployed REST server uses 3DTiles (made by the Cesium consortium i.e. mainly an AGI ( Analytical Graphics, Inc.) emanation.

• The data base packages
  • brew install postgis (that will pull postGRE)
• Python tools
  • brew install python3 (Important notice: in the following some scripts do work with python2.7 but some other scripts require python3...)
  • unset PYTHONPATH && pip3 install virtualenv

• initdb /usr/local/var/postgres -E utf8
• postgres -D /usr/local/var/postgres & (for launching a server)
• createdb bozo

• psql bozo
  - bozo=# create extension postgis;
  - bozo=# create table lyon(gid serial primary key, geom GEOMETRY('POLYHEDRALSURFACEZ', 3946));
  - bozo=# create user myuser with password 'password';
  - bozo=# alter user myuser with superuser;
  - bozo=# \q`  (or use CTRL d equivalently)
  

• The original source for the CityGML based description of the building geometries is the Grand Lyon open data. For the time being (Q1 2017) this data doesn't separate the geometries of buildings. This is why FPE did a building split treatment (based on VCity) resulting in the LYON_6EME_BATI_2012_SplitBuildings.gml file (manually handled over by JGA). In the following we'll assume this file is located in the HOME (shortened as ~) directory.

    git clone https://github.com/Oslandia/citygml2pgsql
    mv citygml2pgsql citygml2pgsql.git && cd citygml2pgsql.git
    ./citygml2pgsql.py -l ~/LYON_6EME_BATI_2012_SplitBuildings.gml
    python ./citygml2pgsql.py ~/LYON_6EME_BATI_2012_SplitBuildings.gml 2 3946 geom lyon |  psql bozo
  

• Assert there is some content in the DB

  psql bozo and bozo=# select count(*) from lyon;

Deploy a Python3 virtual environment with all the bells and whistles:

  unset PYTHONPATH 
  virtualenv venv      # Make sure this was installed with pip3
  . venv/bin/activate
  pip install --upgrade setuptools
  pip install -e .
  pip install uwsgi
  pip install lxml     # Note sure this is truly required but it can't hurt

JGA specific quad-tree based display uses a hierarchy of bounding boxes. Building that bounding box hierarchy is achieved by using an utility code (building-server-processdb.py) that comes bundled with the http server deployment code. Also notice although building-server-processdb.py make usage of Flask (see below) it nevertheless shares its configuration file with some Flak related concerns (see below for more).

First grab the bounding boxes hierarchy building code:

  git clone https://github.com/Oslandia/building-server.git
  mv building-server building-server.git

Then extract the domain size from the DB with

  bozo=# select ST_extent(geom) from lyon;

which should yield a result of the form

  BOX(1843184.223319 5175387.163907,1845435.993083 5177744.541117)

Edit configuration file building-server.git/conf/building.yml in order to inform the extent entry labeled with the result of the above command (remove lyon_lod1:, lyon_lod2:andtest:sections). Thecities:section of thebuilding.yml` configuration file should then be of the form (note that the DB access was also modified):

flask:
  DEBUG: True
  LOG_LEVEL: debug
  PG_HOST: localhost
  PG_NAME: bozo
  PG_PORT: 5432
  PG_USER: myuser
  PG_PASSWORD: password
cities:
  lyon:
    tablename: lyon
    extent: [[1843184.223319, 5175387.163907],[1845435.993083, 5177744.541117]]
    maxtilesize: 2000
    srs: "EPSG:3946"
    attributes: []
    featurespertile: 20

Prior to launching building-server assert that the postgre server accepts requests from your client with the command psql -h <my-postgre-server> -d lyon -U myuser. If you get messages of the form no pg_hba.conf entry for host <your_client_ip> then refer above to the pg_hba.conf configuration stage.

Then launch

  (venv): python building-server-processdb.py conf/building.yml lyon

which will compute the bounding boxes out of the content of the pointed table within the concerned database and push the resulting hierarchy of bounding box data to a corresponding new table (named with a trailing _bbox) within that database.

Technical note: the conf/bulding.yml configuration file mentions flask entries (and is also used to configure flask). Yet the building-server-processdb.py script only uses this file to retrieve the database access information and doesn't make any usage of flask. This lack of separation of concerns for the configuration files is an historical side effect...

• The http server is based on flask. Note: the http server could also be configured to be Apache server.
• We follow the install lines of Oslandia's 3D tiles
• cd building-server.git/
• ** FIXME**: we probably don't need this anymore git checkout 3d-tiles (the "correct" branch is not the master)

• Edit bulding.uwsgi.yml to obtain a configuration like

   uwsgi:
       virtualenv: /Users/mylogin/tmp/building-server.git/venv      # <--- adapt this      
       master: true
       uid: oslandia
       gid: oslandia
       module: building_server.wsgi:app
       processes: 1                                                 # <--- adapt that  
       enable-threads: true
       protocol: uwsgi
       need-app: true
       catch-exceptions: true
       log-maxsize: 10000000
       logto2: /Users/mylogin/tmp/building-server.git/building-server.log                # <--- change this
       env: BUILDING_SETTINGS=/Users/mylogin/tmp/building-server.git/conf/building.yml   # <--- change this
  

• (venv): uwsgi --yml conf/building.uwsgi.yml --http-socket :9090 &
• (venv): deactivate # Exiting the python virtual environment
• Assert that resulting REST server is operational by opening e.g. http://localhost:9090/#!/default/get_api_get_geometry

Technical notes:

• http gateway code is in App.py
• Conversion SQL to client content is defined in database.py

cd <somewhere>
git clone https://github.com/iTowns/itowns2.git
cd itowns2
npm install

Now either open itowns2/index.html file with your browser or alternatively run npm start.

Assert all is well by opening http://localhost:8080/examples/planar.html with your browser.

FIXME VJA : Consider JGA remark : Il n'est pas obligatoire que la modif soit faite pour qu'on puisse installer le serveur de bâtiment. C'est juste qu'il faut faire une opération en plus. Après avoir installer bnuilding-server (branche 3d-tiles), il faut installer la branche b3dm de py3dtiles. -> clone le git + checkout branche b3dm -> à partir du dossier building-server "pip install /chemin/vers/py3dtiles --upgrade"

• The data base packages
  • sudo apt-get install postgis (that will pull postGRE)
• Python tools
  • sudo apt-get install install python3 (Important notice: in the following some scripts do work with python2.7 but some other scripts require python3...)
  • sudo apt-get install python-pip
  • unset PYTHONPATH && pip3 install virtualenv
  • pip install lxml might be required

• initdb /usr/local/var/postgres -E utf8 FIXME VJA: voir avec JGA
• postgres -D /usr/local/var/postgres (for launching a server)
• createdb bozo

• psql bozo
  - bozo=# create extension postgis;
  - bozo=# create table lyon(gid serial primary key, geom GEOMETRY('POLYHEDRALSURFACEZ', 3946));
  - bozo=# create user myuser with password 'password';
  - bozo=# alter user myuser with superuser;
  - bozo=# \q` 
  

• The original source for the CityGML based description of the building geometries is the Grand Lyon open data. For the time being (Q1 2017) this data doesn't separate the geometries of buildings. This is why FPE did a building split treatment (based on VCity) resulting in the LYON_6EME_BATI_2012_SplitBuildings.gml file (manually handled over by JGA).

• git clone https://github.com/Oslandia/citygml2pgsql

• ./citygml2pgsql.py -l LYON_6EME_BATI_2012_SplitBuildings.gml

• python ./citygml2pgsql.py "LYON_6EME_BATI_2012_SplitBuildings.gml" 2 3946 geom lyon | psql bozo

• Assert there is some content in the DB

  psql bozo and bozo=# select count(*) from lyon;

JGA specific quad-tree based display uses a hierarchy of bounding boxes. It seems to use the same configuration file as flask (see below).

• Extract the domain size from db with bozo=# select ST_extent(geom) from lyon;
• Edit configuration file Bozo/building-server/conf/building.yml:
  • ...
• The command python building-server-processdb.py conf/building.yml lyon will compute the bounding boxes out of the content of the pointed table within the concerned database and push the resulting hierarchy of bounding box data to a corresponding new table (named with a trailing _bbox) within that database.
• Note: the conf/bulding.yml configuration file mentions flask entries (and is also used to configure flask). Yet the building-server-processdb.py script only uses this file to retrieve the database access information and doesn't make any usage of flask. This lack of separation of concerns for the configuration files is an historical side effect...

• The http server is based on flask. Note: the http server could also be configured to be Apache server.
• We follow the install lines of Oslandia's 3D tiles
• git clone https://github.com/Oslandia/building-server.git
• cd building-server/
• git checkout 3d-tiles (the "correct" branch is not the master)
• Deploy a Python virtual environment:

    which python3
    virtualenv -p /usr/bin/python3 venv
    . venv/bin/activate
    pip install --upgrade setuptools
    pip install -e .
    pip install uwsgi
  

• Edit bulding.uwsgi.yml (refer to result)
  • Shell: uwsgi --yml conf/building.uwsgi.yml --http-socket :9090
  • Assert that resulting REST server is operational by opening e.g. http://localhost:9090/#!/default/get_api_get_geometry
• Developer notes:
  • http gateway code is in App.py
  • Conversion SQL to client content is defined in database.py

cd <somewhere>
git clone https://github.com/iTowns/itowns2.git
cd itowns2
npm install

Now either open itowns2/index.html file with your browser or alternatively run npm start and open http://localhost:8080/examples/planar.html with your browser.