The South Pole Telescope has systematically identified large numbers of high-redshift strongly gravitationally lensed systems. These sources are selected by their extreme mm flux, which is largely independent of redshift and lensing configuration. I will report results from the first blind redshift survey undertaken with the recently commissioned Atacama Large Millimeter Array (ALMA). We targeted 26 rare, extraordinarily bright, dusty sources selected from a 2500 deg^2 mm survey conducted by the SPT and obtained redshifts via molecular carbon monoxide (CO) lines. We determine that roughly 40% of these sources lie at z>4, indicating the fraction of dusty starburst galaxies at high-redshift is far higher than previously thought. Two sources are at z>5.6, placing them among the highest redshift starbursts known, and demonstrating that large reservoirs of molecular gas and dust can be present in massive galaxies near the end of the epoch of cosmic reionization. These sources were additionally targeted with high resolution imaging with ALMA, unambiguously demonstrating them to be strongly gravitationally lensed by foreground structure. We are undertaking a comprehensive and systematic followup campaign to use these ``cosmic magnifying glasses'' to study the infrared background in unprecedented detail, inform the condition of the interstellar medium in starburst galaxies at high redshift, and place limits on dark matter substructure. I will discuss the scientific context and potential for these strongly lensed starburst galaxies, give an overview of our team's extensive followup efforts, and describe our preliminary science results.