We measured hue impressions by semantic-words and word impressions by hues and obtained their bidirectional relationship. By this method, we found that color vision normal observers used color appearance to evaluate the hue impressions but color vision deficient observers used their experience associated with color names rather than the appearance.