It is widely known that L1 phonological and phonetic structure impact on L2 perception and production. This study investigated how LI influences the lower level of speech perception by comparing similarity judgment of English fricatives by Japanese and Chinese native speakers. In addition, we investigated whether the results of Chinese and Japanese native speakers' speech perception supported CATM's proposal — the use of acoustic cues that are not needed for phoneme distinction is inhibited by the completion of phonemic category. Our results suggest that LI phonemic inventories affect similarity judgments. The results also support the inhibition effect proposed by CATM.