This replication study investigated how the level of context in vocabulary assessment affects the scores on tests of American idioms. Using Uçkun's methodology of three tests with three levels of context, 85 participants ranging in level from high beginner to advanced took an online test consisting of 30 questions, ten questions for each level of context. The tests were matching, sentence-level gap-filling, and rational-delete cloze test. The participants were non-native speakers of English living in the United States. The scores were analyzed for mean differences and in regards to differences in native language and English proficiency level. No significant differences were seen for native language, but there were significant differences for context level and proficiency level. As an exploratory part of the study, 17 of the participants were asked to perform a think-aloud protocol task while taking the test. Their responses were recorded and analyzed descriptively.