Communicative language ability models assume two general sub-abilities, grammatical and pragmatic competence (Bachman 1990; Celce-Murcia et al. 1995). In the relatively new field of second-language pragmatics testing (SLPT), its theoretical basis, speech act theory (Searle 1969), and the method usually employed, the discourse completion task (Hudson et al. 1995) are called into question by findings from conversational analysis (CA) (Golato 2003). While CA has been useful in post hoc L2-test validation (Ross 2007), CA as a resource for a priori test-task design is still in its infancy although some research (Walters 2013) suggests it holds promise. The present study builds on earlier work by widening both the sample of learner-proficiencies and the range of pragmatic targets, the latter derived from the CA literature (Pomerantz 1978, 1984; Schegloff 2007; Wong and Waring 2010). The protocol involves ESL adults of low-intermediate to advanced proficiency engaging in a role-play, the responses conveyed to two raters differentially trained in CA. Response data were transcribed and analyzed according to determine (1) whether the wider pool of test tasks can generate results that can validly infer L2 intermediate pragmatic ability; (2) whether CA-informed prompts can elicit targeted skills; and (3) the impact of "CA proficiency" on rater behavior. Qualitative and quantitative results are given and suggestions for further research offered.