거리영상 기반 보행환경의 정성적 평가 예측을 위한 딥러닝 모델 개발 = Development of a Deep Learning Model to Predict the Qualitative Evaluation of a Walking Environment based on Street View Images

다국어 초록 (Multilingual Abstract)

The visual appearance of a city is an important factor in shaping perception of the neighborhood. With the recent development of the computer vision, research has attempted to analyze perceptions of urban built environments such as 'beauty', 'lively', 'safety', and 'walking‘. Using the training set that represent perception of street view images, previous studies have developed models that predict the score of the image on a particular emotion. Typically, they have proposed models based on Siamese network and a loss function that learns the rank that returns the rank of each image on a particular emotion. However, if the characteristics of the built environment are not distinctive, there is a limit to increasing the accuracy of the model. Therefore, we propose a model that learns both regional features on images by dividing each image into patches and global features. To evaluate the effectiveness of our model, we collected Kakao streetview images of Jeonju, implemented the model to predict the walkability score using the pairwise comparison data and measured accuracy. Our model records the highest accuracy compared to the existing studies. Our model is considered to effectively learn the characteristics of streetview images regardless of the size of the city.

국문 초록 (Abstract)

도시의 시각적 형상은 근린에 대한 사람들의 인식을 형성하는 중요한 요소이다. 최근 컴퓨터 비전 분야의 발전과 거리영상 플랫폼의 활성화로 ‘아름다움’, ‘활기참’, ‘안전성’, ‘보행성’등과 같은 도시 건조환경에 대한 사람들의 인식을 분석하려는 연구가 시도되고 있다. 선행연구들은 거리영상에 대한 사람들의 정성적 평가를 데이터셋으로 활용하여 거리영상에 대한 감성 점수를 예측하는 모델을 개발하고 있다. 대표적으로 샴네트워크(Siamese network)와 랭킹함수(ranking function)를 학습하는 손실함수(loss function)로 구성되고 특정 감성에 대한 각 이미지의 순위를 실수로 반환하는 모델을 제안하고 있다. 그러나 도시의 건조환경의 특성이 두드러지지 않는 경우 모델의 정확도를 높이는 것은 한계가 있다. 따라서 본 연구에서는 이미지의 전역적 특징과 이미지를 패치로 나눈 지역적 특징도 함께 학습하는 모델을 제안했다. 모델의 유용성을 평가하기 위해 전주시의 카카오 거리영상을 수집하고, 사람들의 보행환경에 대한 쌍별비교(pairwise comparison) 결과를 학습데이터로 하여 보행환경(walkability)점수를 예측하는 모델을 구현하고, 정확도를 측정했다. 제안된 모델은 기존 연구와 비교하였을 때 가장 높은 정확도를 나타냈다. 본 연구의 방법은 건조환경에 대한 사람들의 인식을 학습할 때 도시의 규모와 관련 없이 효과적으로 거리영상의 특징을 학습할 수 있는 모델로 사료된다.

참고문헌 (Reference)

1 조혜민 ; 이수기, "보행목적별 보행활동시간에 영향을 미치는 근린환경 특성분석 - 주관적 인지환경과 객관적 측정환경의 차이를 중심으로 -" 대한국토·도시계획학회 51 (51): 105-122, 2016

2 김규리 ; 이제선, "보행공간 요소에 대한 보행자의 인지 및 보행만족도에 관한 연구" 한국도시설계학회 17 (17): 89-103, 2016

3 Xu, Y., "Visual Urban Perception with Deep Semantic-Aware Network" Springer 28-40, 2019

4 Hanlin Zhou, "Using Google Street View imagery to capture micro built environment characteristics in drug places, compared with street robbery" Elsevier BV 88 : 101631-, 2021

5 Guan, W., "Urban perception: sensing cities via a deep interactive multi-task learning framework," 17 : 1-20, 2021

6 Blečić, I., "Towards automatic assessment of perceived walkability" Springer 351-365, 2018

7 Wang, R., "The relationship between visual enclosure for neighbourhood street walkability and elders’mental health in China: using street view images" 13 : 90-102, 2019

8 Larrañaga, A. M., "The influence of built environment and travel attitudes on walking: A case study of Porto Alegre, Brazil" 104 : 332-342, 2016

9 Bijmolt, T. H., "The effects of alternative methods of collecting similarity data for multidimensional scaling" 124 : 363-371, 1995

10 Salesses, P., "The collaborative image of the city: mapping the inequality of urban perception" 10 (10): e0119352-, 2013

11 Naik, J., "Streetscore-predicting the perceived safety of one Million streetscapes" 793-799, 2014

12 Biljecki, F., "Street view imagery in urban analytics and GIS: A review" 215 : 104217-, 2021

13 Lee, I., "Street crime prediction model based on the physical characteristics of a streetscape: Analysis of streets in low-rise housing areas in South Korea" 46 (46): 862-879, 2019

14 Zhou H., "Social inequalities in neighborhood visual walkability: Using street view imagery and deep learning technologies to facilitate healthy city planning" 101605-, 2019

15 Koch, G., "Siamese neural networks for oneshot image recognition" University of Toronto 2015

16 Lu, X., "Rapid: Rating pictorial aesthetics using deep learning" 457-466, 2014

17 Roof, K., "Public health: Seattle and King county’s push for the built environment" 75 : 24-27, 2008

18 Min, W., "Multi-task deep relative attribute learning for visual urban perception" 29 : 657-669, 2019

19 Li, X., "Mapping the spatial distribution of shade provision of street trees in Boston using Google Street View panoramas" 31 : 109-119, 2018

20 Santani, D., "Looking south: Learning urban perception in developing cities" 1 (1): 1-23, 2018

21 Zhou, B., "Learning deep features for discriminative localization" 2921-2929, 2016

22 Sagar Joglekar, "FaceLift: a transparent deep learning framework to beautify urban scenes" The Royal Society 7 (7): 190987-, 2020

23 Lu, X., "Deep multi-patch aggregation network for image style, aesthetics, and quality estimation" 990-998, 2015

24 Dubey, A., "Deep learning the city:Quantifying urban perception at a global scale" Springer 196-212, 2016

25 Kim, J. H., "Decoding urbal landscapes: Google street view and measurement sensitivity" 88 : 101626-, 2021

26 Yoo. K, "Creation of a paired comparison set for preference evaluation based on crowdsourcing" 2-6, 2021

27 He, L., "Built environment and violent crime: An environmental audit approach using Google Street View" 66 : 83-95, 2017

28 Wilson, J. Q., "Broken windows" 249 (249): 29-38, 1982

29 Burges, C., "August, Learning to rank using gradient descent" 89-96, 2005

30 Werner, L., "Anthropologic:Architecture and Fabrication in the cognitive age" eCAADe 319-328,

31 Stewart, N., "Absolute identification by relative judgment" 112 (112): 881-911, 2005

32 Zou, W., "A new multi-feature fusion based convolutional neural network for facial expression recognition" 52 (52): 2918-2929, 2021

33 Li, Y., "A big data evaluation of urban street walkability using deep learning and environmental sensors—A case study around Osaka University Suita campus" 2 : 319-328, 2020