State of the art - Tensor-Reloaded/Property-Development-Prediction GitHub Wiki
State-of-the-art
Final product concept
The target is to make a usable application with respect to the scope. For this reason, the most important aspect would be on providing the user with a GUI to input the data for the main algorithm through a simple and intuitive single-page web interface. The main functionality is to predict how, for a given area, the urban settlements will evolve in a given period of time; a second functionality would be to predict how profitable the investment in building the construction in a given area would be at a certain timestamp. Depending on the dataset used in the creation of the model, a preferable use case would be that the user will select a location on the map (with latitude and longitude) that can be associated with already known information. Beside this feature of selection, it is considered to implement a way to tell for what year to do the prediction in the future (slider/ input number).
Datasets
Earth photos
- Sentinel - 2 satellite - https://eos.com/find-satellite/sentinel-2/
- Earth images from NASA: https://search.earthdata.nasa.gov/search?fsm0=Agricultural+Plant+Science&fst0=Agriculture&fsm1=Food+Science&fst1=Agriculture&lat=44.55568492950572&long=24.762538525400885&zoom=7
- API from Google for downloading images; from the COPERNICUS project: https://developers.google.com/earth-engine/datasets/catalog/COPERNICUS_S2_SR_HARMONIZED -
- Google Earth with history of the Terra: https://earth.google.com/web/@32.83865361,-96.29650132,176.87460241a,611756.80522978d,35y,38.40139856h,23.4967631t,0r/data=CjISMBIgMGM5NzE2MWM0Njk4MTFlYWJlMTMyZGUyNWIyYWZmNjkiDGRhbGxhcy10ZXhhcw
- Google Dataset for City Evolution: https://developers.google.com/earth-engine/datasets
- Dataset containing global settlement layers: https://developers.google.com/earth-engine/datasets/catalog/JRC_GHSL_P2016_BUILT_LDSMT_GLOBE_V1
- Footprint of human settlements. Images are in contrast colors: https://developers.google.com/earth-engine/datasets/catalog/DLR_WSF_WSF2015_v1
- Kaggle Dataset from the Building Segmentation Competition: https://www.kaggle.com/datasets/kmader/synthetic-word-ocr -
- NASA WorldWide Viewer - https://worldview.earthdata.nasa.gov/?v=23.65365223885282,43.3299992861907,28.950133647615573,45.9147925570296&i=1&l=GRUMP_Settlements(hidden),VIIRS_SNPP_DayNightBand_ENCC,Reference_Labels_15m,Reference_Features_15m,Coastlines_15m,VIIRS_SNPP_CorrectedReflectance_TrueColor&lg=false&t=2017-06-05-T19%3A25%3A30Z
Real estate property price
- US real estate prices for properties (over time) (price entries depends on how many were saved. Free up to 500 requests/month: https://rapidapi.com/datascraper/api/us-real-estate/
- Sources of property price data sets. Requires further investigation as for every country, there are listed multiple website where you could finds these datasets: https://www.bis.org/statistics/pp/disclaimer.htm
- UK prices of properties. Can be exported as CSV. Can be splitted by country, region, local authorities: https://landregistry.data.gov.uk/app/ukhpi/browse?from=2021-08-01&location=http%3A%2F%2Flandregistry.data.gov.uk%2Fid%2Fregion%2Funited-kingdom&to=2022-08-01&lang=en
- CSV regarding average properties prices from USA starting in 2000 until 2022: https://www.zillow.com/research/data/
- https://earthengine.google.com/#!/ - Waiting for approval to see what data can be extracted from this Api
Technologies
- Backend - Java EE (& Kotlin)
- Frontend Web - Angular
- GitHub: https://github.com/Tensor-Reloaded/Property-Development-Prediction
Neural Networks
- Python, TensorFlow, Keras, Google Colabs - this will represents the tools used for the development of the neural network
Architecture for the Neuronal Network
- GAN - image generation with the new building in an analysed area
- Convolutive - type of neurons for image analysis
- Segmentation - marking the building in a given image
Similar applications
- JLL - Property Trends & Designs; includes interest research topics about the development of cities https://www.jll.ro/en/trends-and-insights?utm_source=google&utm_medium=paidsearch&utm_campaign=em-ro-corp-growth&utm_content=em-ro-en-corp-growth-industry-trends-search-lg-broad&utm_term=real%20estate%20trends&gclid=Cj0KCQjwhY-aBhCUARIsALNIC05dnYDbSRssqMyn5pe83HD5iQxyItD969VCtSy7kk5VhElJrDJeABgaAr_YEALw_wcB&gclsrc=aw.ds
Others institutions/researches and their contribution in this domain
- George Grekousis, Panos Manetos, Yorgos N. Photis. - Modelling urban evolution using neural networks, fuzzy logic and GIS. The case of the Athens Metropolitan area: https://www.academia.edu/11994695/Modeling_urban_evolution_using_neural_networks_fuzzy_logic_and_GIS_The_case_of_the_Athens_metropolitan_area
- National Library of Medicine, Bangladesh - Prediction of urban expansion by using land cover change detection approach https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8642620/#:~:text=Urban%20expansion%20and%20its%20surrounding,12.44%20km2%20in%202028 - research how the urban development affects the health of the people
- Yin Ma, Xinqi Zheng, Menglan Liu, Dongya Liu, Gang Ai & Xueye Chen - Spatio-temporal evolution characteristics analysis and optimization prediction of urban green infrastructure: a case study of Beijing, China: https://www.nature.com/articles/s41598-022-14613-z
- NASA analysis about how the population’s growing will affect the expansion of the cities: https://www.earthdata.nasa.gov/learn/backgrounders/nighttime-lights#top
- Convolutional neural networks for global human settlements mapping from Sentinel-2 Satellite imagery: https://link.springer.com/article/10.1007/s00521-020-05449-7#Sec4
Technical aspects regarding the architecture and the output of the Neural Network & Establishing the profitability of the new zone
The input of the neural network consists of three types: a satellite image from an area, the segmented buildings from an area or both; the output of the neural network can be the generated image with new buildings in the given area, the polygonal points describing the areas of new buildings or both.
After retrieving the image from the neural network, we can further apply segmentation techniques in order to find the new generated buildings; with price information over time we can predict the price of a certain terrain where the new buildings would appear and the profitability of the certain constructions. The types of architectures of neural networks used for predicting the appearance of new buildings can be multiple:
- Given a set of images, a GAN - Generative Adversarial Network - will create a new image based on noise input
- We can use Dense layers in order to predict the zones of the new buildings starting from a given dataset of segmented buildings.
- We can also use Diffuse Neural Networks in order to predict the new developed zone.
- Autoencoders sustain a great compound to be used in the prediction model
The price of a terrain will be computed using the values over time provided by the resources listed above; using them, we can create an artificial intelligence for the prediction of the price - Support Vector Machines -, Linear Regression techniques or even a new Neural Network and NLP techniques regarding the social opinion.
Risk assessment
- The main risk is not finding enough datasets to work with, and even if we do, the quality is an important factor in getting good results; also, it is very important to decide from the very beginning what kind of data we need for training/fine tuning purposes.
- Providing we found the correct datasets, it is also crucial for the model to make accurate predictions in relation with the given data regarding not only the image, but also the correct price.
- For the datasets concerning terrain prices, it is important to find a strategy in combating the inconsistencies around the dataset - missing dates; otherwise, the final accuracy of the model may be affected.