GSC Data Catalogue Services - GeoSmartCity-CIP/gsc-datacatalogue GitHub Wiki

1 Introduction

This document lists the services to create and show the JSON used for request and response. Each paragraph des1. cribes a specific setq of services.

Services in bold have been already completed and ready to be integrated.

The sets of services are:

For all services, we show only expected parameter not full JSON. Parameters will be translate in JSON as: {param1:’’, param2:’’, …} For specific JSON required by the service, we show also an example of JSON.

The services has another request param named 'actionName'. Each service has a different actioName.

So a common service request will be done by actionName for the service name and json that contains all service params.

We want use NO-SQL approach to store our data. If we use only JSON columns and no other info (for example id), we can think to aggregate create and update functions almost use cases because the name is a key for the table.

This document is not definitive. The target is to show the basic logic of functions, but these can change.

2 User services

There is only one organization for user. User can edit only data for a specific organization and do not see other data. Admin can view all data for all organization and edit access permission.

2.1 Login

After three wrong attempts, user is temporary blocked.

actionName: login

Request json expected parameters are:

Username
Password

Response is a JSON with user detail (organizations, roles and functions list): {organizations:[{id:value},{id:value}..], roles:[{id:value,functions:[{id:value},..]}}

Or a JSON with error status same in other services with parameters:

Status (Error)
Description
Request

Response is a JSON with parameters:

Status (Error, Done)
Description
Pr
Request

2.2 Register

Service registers a new user with the specified data, and sends a verification e-mail to the address specified (Email), reporting a verification UUID.

Errors on e-mail sending are blocking for user registration (e.g. e-mail address is not valid).
Until e-mail verification is done, valid e-mail address (Email) are considered "not verified" (see "Remind password" service).
E-mail address (Email) verification must be always possible: generated UUID must be saved in database.

actionName: reguser

Request json expected parameters are:

Email
Username
Password
ConfirmPassword
List of Organizations for example organizations:[{id:value},{id:value...}] (empty for admin users)
…

Response is a JSON with parameters:

Status (Error, Done)
Description
Request

2.3 Remind password

Service generates a new random user password and sends it to the e-mail address. The user could choose between username or email to get a new password. If Email is specified, this will be used. If username is specified, the e-mail address registered for the user will be used.

actionName: remindpwd

Request json expected parameters are:

Username (optional)
Email (optional)

Response is a JSON with parameters:

Status (Error, Done)
Description
Request

2.4 Change password

actionName: changepwd

Request json expected parameters are:

Username
OldPassword
NewPassword
ConfirmNewPassword

Response is a JSON with parameters:

Status (Error, Done)
Description
Request

2.5 Update user profile

Change user data. Username and password cannot be changed: username is a key for the application; password renewal must be done with "Change password" service.

actionName: updateuser

Request json expected parameters are:

Username
Email
List of Organizations for example organizations:[{id:value},{id:value...}] (empty for admin users)
…

Response is a JSON with parameters:

Status (Error, Done)
Description
Request

2.6 Lock/unlock user (only admin)

Admin can lock/unlock user.

actionName: lockuser

Request json expected parameters are:

Username
Lock (true, false)

Response is a JSON with parameters:

Status (Error, Done)
Description
Request

2.7 Unregister/delete (delete only admin)

User can unregister him from the system. Admin can delete an old account.

actionName: unreguser

Request json expected parameters are:

Username
Password (user reinsert the password to confirm his intentions) * (mandatory if not admin)

Response is a JSON with parameters:

Status (Error, Done)
Description
Request

2.8 Verify mail

After the registration mail has been sent to the provided address users should click on the recieved link. The url contains a "request" parameter that is a json string containing information about id and uuid values

e.g. <"uuid":"5caa5f24-8849-4978-b9ce-72725381df04", "id":"2">

Clicking on the link will call the verifyMail action.

The service extracts both parameters from the json string, and checks whether a row with that id exists in the user table, and if the UUID matches the UUID value in the json column of that row. If it does json status value for that row becomes 'verified' and the UUID is deleted. The user is therefore fully registered.

Response is a JSON with parameters:

Status (Error, Done)
Description

2.9 List user

actionName: listuser

Request json expected parameters are:

idorganization

Response is a list of JSON objects containing the following parameters:

id
email
username

3 Organization services

3.1 Create

actionName: createorg

Request json expected parameters are:

Organizationname
Description

Response is a JSON with parameters:

Status (Error, Done)
Id
Description
Request

3.2 Update

actionName: updateorg

Request json expected parameters are:

IdOrganization
Description

Response is a JSON with parameters:

Status (Error, Done)
Description
Request

3.3 Delete

actionName: deleteorg

Request json expected parameters are:

IdOrganization

Response is a JSON with parameters:

Status (Error, Done)
Description
Request

3.4 Search/List

actionName: listorg

Request json expected parameters are:

id (optional, if present we return only the organization with this specific id)
Organizationname (optional, if not present (and also id is not present), all Organization are returned, otherwise return all Organization that match substring)

Response is a JSON with Organization list, each Organization contain also users list: {organizations:[{id:value,name:value,description:value},{id:value,name:value,description:value}….]}

Or a JSON with error status same in other services with parameters:

Status (Error)
Description
Request

4 Roles services

4.1 Create

actionName: createrole

Request JSON expected parameters are:

Rolename
Organization
Description

Response is a JSON with parameters:

Status (Error, Done)
Id
Description
Request

4.2 Delete

actionName: deleterole

Request expected parameters are:

IdRole

Response is a JSON with parameters:

Status (Error, Done)
Description

4.3 Search/List

actionName: listrole

Request expected parameters are:

Organization
Rolename (optional, if not present, all Roles of organization are returned, otherwise return all Roles that match substring rolename)
includeadminrecords (optional. Can be true or false. If true search will include admin roles. those roles have no organization linked to them).

Response is a JSON with Roles list, each Role contain also users list: {roles:[{idrole:value,name:value,description:value,users:[{iduser:value,username:value},{iduser:value,username:value}]},{idrole:value,name:value,description:value,users:[]},{idrole:value,name:value,description:value,users:[{iduser:value,username:value}]}….]}

Or a JSON with error status same in other services with parameters:

Status (Error)
Description
Request

4.4 Assign/remove users to role (Only admin)

Add or remove user to a role.

actionName: assignrole

Request JSON expected parameters are:

IdRole
List of Users that contains iduser

For example: {idrole:value,users:[{iduser:value},{iduser:value}…]}

Response is a JSON with parameters:

Status (Error, Done)
Description
Request

5 Functions services

Functions are all possible elements that can be accessed/limited to user. For examples see or not see a button in a client application, call a specific service etc, admin functionality, etc.

Two possible functions categories: application, data catalogue.

Application functions will be use by client applications.
Data catalogue functions will be use to access data catalogue sections and activate specific functionality

5.1 Create

actionName: createfunc

Request JSON expected parameters are:

FunctionName
Organization
Type
Description

Response is a JSON with parameters:

Status (Error, Done)
Id
Description
Request

5.2 Update

actionName: updatefunc

Request JSON expected parameters are:

IdFunction
FunctionName
Organization
Type
Description

Response is a JSON with parameters:

Status (Error, Done)
Description
Request

5.3 Delete

actionName: deletefunc

Request JSON expected parameters are:

IdFunction

Response is a JSON with parameters:

Status (Error, Done)
Description
Request

5.4 Search/List

actionName: listfunc

Request JSON expected parameters are:

Organization
FunctionName (optional, if not present, all functions of organization are returned, otherwise return all functions that match substring)
includeadminrecords (optional. Can be true or false. If true search will include admin functions. those roles have no organization linked to them).

Response is a JSON with functions list: {functions:[{idfunction:value,name:value,description:value},{idfunction:value,name:value,description:value},{idfunction:value,name:value,description:value}….]}

Or a JSON with error status same in other services with parameters:

Status (Error)
Description
Request

6 Permissions services

Permissions are the link between roles and functions.

6.1 Assign/remove permission to role (Only admin)

Add or remove functions permitted to a role. Only one permission record for role.

actionName: assignperm

Request JSON expected parameters are:

IdRole
functions that contains idFunction and idLayer and/or idApplication (both idLayer and idApplication are optional).

For example:

{ "idrole": value, "name": "value", "functions": [ { "idfunction": value }, { "idfunction": value, "idlayer": value }, { "idfunction": value, "idapplication": value }, ... ] }

Some useful functions that have been already inserted handle the map visibility and the default map choice for a certain role. Those connections come into play when using the getConfiguration service. Map visibility function (id = 187). Any application not having this permission is not included in the output. Default map function (id = 188). Any application having this permission is labeled with "defaultMap":true in the output.

e.g. (Users with roleId = 83 are allowed to see the application having applicationid = 1)

... "idrole": 83, "functions": [ { "idfunction": 187, "idapplication": 1 }, ... ] ...

Response is a JSON with parameters:

Status (Error, Done)
Description
Request

6.2 List permission for role

actionName: listperm

Request JSON expected parameters are:

idRole

Response is a JSON with functions list and where is present also layername: {idrole:value,idpermission:value,functions:[{idfunction:value,name:value,description:value},{idfunction:value,name:value,description:value },{idfunction:value,name:value,description:value,idlayer:value,layername:value}….]}

Or a JSON with error status same in other services with parameters:

Status (Error)
Description
Request

7 Datasource services

Different datasource types:

WMS
WFS
Oracle Is not possible catalogue this Datasource on CKAN. We can publish it on Geoserver and after publish Geoserver Layer/Workspace on CKAN (csv, wms etc)
Postgis Same of Oracle
Shapefiles
CSV
GTFS
… When a user creates a datasource, this takes same organization of user.

How we use metadata files? When?

7.1 Create

actionName: createdatasrc

Request JSON expected parameters are:

Datasourcename
Organization
Type (Oracle, Postgis, WMS, …)
Description
Updated (date type)
Url (optional)
Username (optional)
Password (optional)
Ipaddress (optional)
schema(optional)
database (optional)
Port (optional)
Path (optional)

Example: datasource shape

{"path": "/opt/shapes/edifici", "type": "SHAPE", "updated": "2013-10-21T13:28:06.419Z", "description": "SHAPE file", "organization": 665, "datasourcename": "datasourcename"}

Example: datasource postgis

{"url": "gsm-db.nco.inet", "port": "5432", "type": "POSTGIS", "schema": "schemaDB", "updated": "2013-10-21T13:28:06.419Z", "database": "nameDB", "password": "passwdDB", "username": "userDB", "description": "Postgis test", "organization": "665", "datasourcename": "datasourcename"}

Response is a JSON with parameters:

Status (Error, Done)
Id
Description
Request

7.2 Update

actionName: updatedatasrc

Request expected parameters are:

IdDatasource
Datasourcename
Organization
Type (Oracle, Postgis, WMS, …)
Description
Updated (date type)
Url (optional)
Username (optional)
Password (optional)
schema(optional)
database (optional)
Ipaddress (optional)
Port (optional)
Path (optional)

Example: datasource shape

{"path": "/opt/shapes/edifici", "type": "SHAPE", "updated": "2013-10-21T13:28:06.419Z", "description": "SHAPE file", "organization": 665, "datasourcename": "datasourcename"}

Example: datasource postgis

{"url": "gsm-db.nco.inet", "port": "5432", "type": "POSTGIS", "schema": "schemaDB", "updated": "2013-10-21T13:28:06.419Z", "database": "nameDB", "password": "passwdDB", "username": "userDB", "description": "Postgis test", "organization": "665", "datasourcename": "datasourcename"}

Response is a JSON with parameters:

Status (Error, Done)
Description
Request

7.3 Delete

actionName: deletedatasrc

Request JSON expected parameters are:

IdDatasource

Response is a JSON with parameters:

Status (Error, Done)
Description
Request

7.4 Search/List

actionName: listdatasrc

Request JSON expected parameters are:

Organization (optional)
Datasourcename (optional, if not present, all datasource are returned, otherwise return all datasource that match substring)
Detail (optional, true,false. If true all datasource data will be return. If omitted, default value will be false)
IdDatasource (optional, use it instead of previous three params to identify a single datasource)

Response is a JSON with datasource list with full data if detail true: {datasource:[{iddatasource:value,name:value,type:value,description:value,updated:value,path:value},{iddatasource:value,name:value,type:value,description:value,updated:value,url:value,username:value,}….]} Or a JSON with datasource list with limited data if detail false: {datasource:[{iddatasource:value,name:value},{iddatasource:value,name:value}….]}

Or a JSON with error status same in other services with parameters:

Status (Error)
Description
Request

7.5 Upload

Upload the file in a specific path of remote server

actionName: uploadfile

Request JSON expected parameters are:

Organization
Filename
RemoteRelativePath

File is the other parameter to upload on post.

Response is a JSON with parameters:

Status (Error, Done)
Description
Request

7.6 Publish on CKAN (Deprecated. We will use CKAN API to publish on CKAN)

For some type of datasource it’s possible publish it on CKAN.

actionName: pubtockan

Request JSON expected parameters are:

IdDatasource
Type (Oracle, Postgis, WMS, …)
Description
Updated (date type)
Url (optional)
Username (optional)
Password (optional)
Ipaddress (optional)
Port (optional)
Path (optional)
Other CKAN params …

Response is a JSON with parameters:

Status (Error, Done)
Description
Request

7.7 Data origin list

Similar to getCapabilities of WMS service. Return all data usable to create dataset from datasource (for example the list of shapefiles of our filesystem dir, or a tables list for a database).

actionName: listdataorigin

Request JSON expected parameters are:

IdDatasource

Response is a JSON with data origin list: {dataorigin:[{name:value,path:value},{name:value,path:value}….]}

Or a JSON with error status same in other services with parameters:

Status (Error)
Description
Request

8 Dataset services

For remote data, a Cron service will be update data, downloading files in our filesystem.

8.1 Create

actionName: createdataset

Request JSON expected parameters are:

Datasetname
RealName (filename, etc)
IdDatasource
Description
ToBeIngested (optional)
RefreshInterval (optional)
Url (optional)

Note: if the user wants to download the corresponding shapefiles during the dataset creation, he/she will need to use 'auto_update' as the realname. The actual file name will be taken as specified in the URL parameter which is mandatory in this scenario.

Response is a JSON with parameters:

Status (Error, Done)
Id
Description
Request

8.2 Update

actionName: updatedataset

Request JSON expected parameters are:

IdDataset
Datasetname
RealName (filename, etc)
IdDatasource
Description
ToBeIngested (optional)
RefreshInterval (optional)
Url (optional)

Response is a JSON with parameters:

Status (Error, Done)
Description
Request

8.3 Delete

actionName: deletedataset

Request JSON expected parameters are:

IdDataset

Response is a JSON with parameters:

Status (Error, Done)
Description
Request

8.4 Search/List

actionName: listdataset

Request JSON expected parameters are:

IdDatasource (optional)
Datasetname (optional, if not present, all dataset of datasource are returned, otherwise return all dataset that match substring)
idorganization (optional)
IdDataset (optional, use this instead of previous params to search dataset by id)

Response is a JSON with dataset list: {dataset:[{iddataset:valuename:value,realname:value,iddatasource:value,datasourcename:value,description:value,tobeingested:value,refreshinterval:value,url:value},{iddataset:valuename:value,realname:value,iddatasource:value,datasourcename:value,description:value}….]}

Or a JSON with error status same in other services with parameters:

Status (Error)
Description
Request

8.5 Single dataset columns list

This function lists all dataset columns.

actionName: listcols

Request JSON expected parameters are:

IdDataset

Response is a JSON with dataset column list: {columns:[{name:value,type:value,alias:value,visibility:value},{name:value,type:value,alias:value,visibility:value}….]}

Or a JSON with error status same in other services with parameters:

Status (Error)
Description
Request

8.6 Update single dataset columns metadata (visibility, alias)

Update columns metadata for a single dataset.

actionName: updcolsmetadata

Request JSON expected parameters are:

IdDataset
List of columns with type,alias and visibility

Response is a JSON with parameters:

Status (Error, Done)
Description
Request

8.7 Cron download service

When the application is started a background cron scheduler is created. This scheduler will create a task that will download the (shape) files from a remote location to a local directory. The scheduler task runtimes can be configured by changing the property 'cronpattern' in the 'configuration.properties' file before starting the application itself. For more information about cron patterns visit http://www.sauronsoftware.it/projects/cron4j/api/it/sauronsoftware/cron4j/SchedulingPattern.html

When the task starts it will check all the dataset elements that have the 'tobeingested' parameter set to true and that satisfly the following expression: 'refreshinterval' + 'lastupdated' > [current time] where lastupdated is the last time this dataset has been updated, and refresh interval is an integer representing the number of hours that need to pass before the dataset checks for further updates. They both are dataset optional parameters. For each dataset found this way the task will then download a zip file, and will extract all the contained files. The zip is downloaded from the provided 'url' parameter, also specified in the dataset element.

9 Layer services

Geoserver workspace will be not set on layers, we will use application name to create workspaces and layers inserted in it. A specific section will manage the groups.

9.1 Create

actionName: createlyr

Request JSON expected parameters are:

Layername
IdDataset
Description
MetadataFile (optional)
SLD (optional)
SRS (optional - if this is not configured, application's SRS will be used instead)

Response is a JSON with parameters:

Status (Error, Done)
Id
Description
Request

9.2 Update

actionName: updatelyr

Request JSON expected parameters are:

IdLayer
Layername
IdDataset
Description
MetadataFile (optional)
SLD (optional)
SRS (optional - if this is not configured, application's SRS will be used instead)

Response is a JSON with parameters:

Status (Error, Done)
Description
Request

9.3 Delete

actionName: deletelyr

Request expected parameters are:

IdLayer

Response is a JSON with parameters:

Status (Error, Done)
Description
Request

9.4 Search/List

actionName: listlyr

Request JSON expected parameters are:

IdDataset (optional)
Layername (optional, if not present, all layers are returned, otherwise return all layers that match substring)
idorganization (optional)
IdLayer (optiona, use this instead o previous params to search layer by id)

Response is a JSON with layers list: {layers:[{idlayer:value,name:value,iddataset:value,datasetname:value,description:value,metadata:value,sld:value},{idlayer:value,name:value,iddataset:value,datasetname:value,description:value,metadata:value,sld:value}….]}

Or a JSON with error status same in other services with parameters:

Status (Error)
Description
Request

10 Group layer services

Logical groups of layers. Used by client application and Geoserver.

10.1 Create

actionName: creategrp

Request JSON expected parameters are:

Groupname
Organization
Description

Response is a JSON with parameters:

Status (Error, Done)
Description
Request

10.2 Assign layers to group

Add or remove layers to a group.

actionName: assignlyr

Request expected parameters are:

IdGroup
List of layers

For example: {idGroup:value,layers:[{idlayer:value},{idlayer:value}…]}

Response is a JSON with parameters:

Status (Error, Done)
Description
Request

10.3 Update

actionName: updategrp

Request JSON expected parameters are:

idgroup
Groupname
Organization
Description

Response is a JSON with parameters:

Status (Error, Done)
Description
Request

10.4 Delete

actionName: deletegrp

Request JSON expected parameters are:

IdGroup

Response is a JSON with parameters:

Status (Error, Done)
Description
Request

10.5 Search/List

actionName: listgrp

Request JSON expected parameters are:

Organization (optional)
Groupname (optional, if not present, all groups are returned, otherwise return all groups that match substring)
IdGroup (optional, use this instead previous params to search group by id)

Response is a JSON with groups list: {groups:[{idgroup:value,groupname:value,layers:[{idlayer:value,layername:value},{idlayer:value,layername:value}…]},{idgroup:value,groupname:value,layers:[{idlayer:value,layername:value},{idlayer:value,layername:value}…]}….]}

Or a JSON with error status same in other services with parameters:

Status (Error)
Description
Request

11 Application services

Application is an aggregation of groups and layers useful to show this data with a specific logic to the clients.

11.1 Create

actionName: createapp

Request expected parameters are:

Applicationname
Organization
Description
Geoserver params (optional, used to connect to geoserver, format: geoserver:{url:value,username:value,password:value})
SRS (optional)
maxextent (optional) that contain the following elements: top, bottom, right, left.
uri (optional)

Example of an application that can publish on geoserver:

{ "srs": "EPSG:3763", "groups": [{ "idgroup": "179" }], "layers": [{ "idlayer": "154" }, { "idlayer": "155" }], "geoserver": { "url": "http://hub.geosmartcity.eu/geoserver/", "password": "password", "username": "user" }, "organization": "674", "idapplication": "43", "applicationname": "ApplicationTestPublish", "uri": "http://test.sinergis.it", "maxextent": { "top": 4959037.1, "left": 1618217.15, "right": 1640640.87, "bottom": 4942861.25 } }

Response is a JSON with parameters:

Status (Error, Done)
Id
Description
Request

11.2 Update/Assign layers/group to application

Add or remove layers to a group.

actionName: assigntoapp

Request JSON expected parameters are:

IdApplication
Applicationname
Organization
Geoserver params (optional, used to connect to geoserver, format: geoserver:{url:value,username:value,password:value})
SRS (optional)
maxextent (optional) that contain the following elements: top, bottom, right, left.
uri (optional)
List of layers (mandatory but it can be empty). Each specified element has to contain the layer id, and may contain additional properties such as: (1)queryable (optional, specifies if the layer can be queried for information. False by default), (2) basic (optional, specifies if the layer is a base layer. False by default), (3) overview (optional, specifies if the layer will be used for an overview map purpose. False by default), (4) any other layer property that needs to be specified within that application
List of groups (mandatory but it can be empty)

Example of an application that can publish on geoserver:

{ "srs": "EPSG:3763", "groups": [{ "idgroup": "179" }], "layers": [{ "idlayer": "154", "queryable":true, ...... }, { "idlayer": "155" }], "geoserver": { "url": "http://hub.geosmartcity.eu/geoserver/", "password": "password", "username": "user" }, "organization": "674", "idapplication": "43", "applicationname": "ApplicationTestPublish", "uri": "http://test.sinergis.it", "maxextent": { "top": 4959037.1, "left": 1618217.15, "right": 1640640.87, "bottom": 4942861.25 } }

Response is a JSON with parameters:

Status (Error, Done)
Description
Request

11.3 Delete

actionName: deleteapp

Request expected parameters are:

IdApplication

Response is a JSON with parameters:

Status (Error, Done)
Description
Request

11.4 Search/List

actionName: listapp

Request JSON expected parameters are:

Organization (optional)
Applicationname (optional, if not present, all applications are returned, otherwise return all applications that match substring)
IdApplication (optional, use this instead of previous params to search application by id)

Response is a JSON with applications list: {applications:[{idapplication:value,applicationname:value,layers:[{idlayer:value,layername:value},{idlayer:value,layername:value}…],groups:[{idgroup:value,groupname:value},{idgroup:value,groupname:value}…]},{idapplication:value,applicationname:value,layers:[{idlayer:value,layername:value},{idlayer:valure,layername:value}…],groups:[{idgroup:value,groupname:value},{idgroup:value,groupname:value}…]}….]}

Or a JSON with error status same in other services with parameters:

Status (Error)
Description
Request

11.5 Publish on Geoserver

Create all necessary structures on Geoserver (workspaces, datastores, layers, group layers).

actionName: pubongeoserver

Request JSON expected parameters are:

IdApplication

Response is a JSON with parameters:

Status (Error, Done)

11.6 GetConfiguration

Create a JSON with application structures useful to clients (for example GeoAdmin, CTW5). Retrieves all the application elements linked to the specified user through his/her permissions. Each of those application contain information about its layers.

actionName: getconfiguration

Request JSON expected parameters are:

IdUser

Response is a JSON with a structure useful to clients:

TO DEFINE

Or a JSON with error status same in other services with parameters:

Status (Error)
Description
Request