3. OSM 데이터 준비 및 분석 - map-search/map-search-etl GitHub Wiki

데이터 다운로드

데이터 다운로드
http://download.geofabrik.de/asia/south-korea.html
데이터 필드 분석 용 참고 문서
https://wiki.openstreetmap.org/wiki/Elements

데이터 적재

osm 데이터를 Parquet로 변환하여 hdfs /OsmData 폴더에 적재

데이터 스키마

Seoul.osm.pbf.node.parquet
root
|-- id: long (nullable = true)
|-- version: integer (nullable = true)
|-- timestamp: long (nullable = true)
|-- changeset: long (nullable = true)
|-- uid: integer (nullable = true)
|-- user_sid: binary (nullable = true)
|-- tags: array (nullable = true)
| |-- element: struct (containsNull = true)
| | |-- key: binary (nullable = true)
| | |-- value: binary (nullable = true)
|-- latitude: double (nullable = true)
|-- longitude: double (nullable = true)

Seoul.osm.pbf.relation.parquet
root
|-- id: long (nullable = true)
|-- version: integer (nullable = true)
|-- timestamp: long (nullable = true)
|-- changeset: long (nullable = true)
|-- uid: integer (nullable = true)
|-- user_sid: binary (nullable = true)
|-- tags: array (nullable = true)
| |-- element: struct (containsNull = true)
| | |-- key: binary (nullable = true)
| | |-- value: binary (nullable = true)
|-- members: array (nullable = true)
| |-- element: struct (containsNull = true)
| | |-- id: long (nullable = true)
| | |-- role: binary (nullable = true)
| | |-- type: binary (nullable = true)

Seoul.osm.pbf.way.parquet
root
|-- id: long (nullable = true) |-- version: integer (nullable = true)
|-- timestamp: long (nullable = true)
|-- changeset: long (nullable = true)
|-- uid: integer (nullable = true)
|-- user_sid: binary (nullable = true)
|-- tags: array (nullable = true)
| |-- element: struct (containsNull = true)
| | |-- key: binary (nullable = true)
| | |-- value: binary (nullable = true)
|-- nodes: array (nullable = true)
| |-- element: struct (containsNull = true)
| | |-- index: integer (nullable = true)
| | |-- nodeId: long (nullable = true)

데이터 구성 정보

https://wiki.openstreetmap.org/wiki/Relation
https://wiki.openstreetmap.org/wiki/Node
https://wiki.openstreetmap.org/wiki/Way