[1G4-GS-2c] 機械学習：回帰

[1G4-GS-2c-01] A Study on Regression and Loss Functions for Multiscale k-Nearest Neighbour

〇Ruixing Cao^1,3, Takuma Tanaka^1,3, Akifumi Okuno^2,3, Hidetoshi Shimodaira^1,3 (1. Kyoto University, 2. The Institute of Statistical Mathematics, 3. RIKEN Center for Advanced Intelligence Project)

Keywords:Nearest Neighbour, Bias Correction, Extrapolation, Regression Analysis

k-nearest neighbour (k-NN) takes label average over a query ball, whose radius r_k increases with larger k, and the non-zero radius results in a bias of the k-NN estimator. To reduce the bias, multiscale k-NN (MS-k-NN) first solves ordinary least squares (OLS) to predict the k-NN estimator at some points k=k₁, k₂, ..., k_V from even-degree polynomials of the radius r_k, and extrapolates the estimator to r=0. However, there remain two practical problems: (i) The polynomial used for extrapolation is derived from asymptotic theory; in finite-sample situations, the MS-k-NN estimator with even-degree polynomials is not necessarily restricted to a proper range [0,1]. (ii) OLS implicitly assumes the independence of the k-NN estimators at k=k₁, k₂, ..., k_V, whereas the estimators utilizing some same labels are dependent. To solve these problems, we propose employing sigmoid-based functions and generalized least squares. We also propose local radial logistic regression (LRLR), which is inspired by MS-k-NN.