Topological insulators (TIs) distinguish themselves from ordinary insulators by hosting boundary modes, such as surface states in the case of three-dimensional TIs and one-dimensional edge modes in the case of two-dimensional TIs. These topological edges states are protected by symmetry, and in the case of time-reversal protected TIs, they are expected to remain ungapped and resilient as long as time-reversal symmetry is preserved. Our group has carried out a number of important investigations of such boundary modes using high-resolution imaging and spectroscopy using STM. In our first study in this field, we demonstrated using quasiparticle interference (QPI) studies that the surface state of time-reversal protected TIs does not undergo backscattering in the presence of impurities [P. Roushan et al, Nature 460 (2009)].
Following these studies, we showed that these surface states can have unusually high transmission through atomic step edges that stops other surface states [J. Seo et al, Nature 466 (2010)]. Although we also showed that even with the absence of backscattering in these states, one still need to consider the spatial fluctuation of chemical potential in this system [H. Beidenkipf et al, Nature Physics 7 (2011) , H. Beidenkopf et al, (invited chapter) Topological Insulators, Elsevier (2013)]. These fluctuations likely limit the mobility of such topological protected states, even though they may not localize as other two-dimensional states do in the presence of disorder.
We also extend our study of scattering properties of such topological edge modes to one-dimensional states that we observed on edges of islands of what was predicted to be two-dimensional TIs [I. K. Drozdov et al, Nature Physics 10 (2014)]. These 1D edge modes also do not show backscattering between states of equal and opposite momentum from ordinary scattering. However, in agreement with theoretical prediction, we find that breaking time-reversal symmetry with the presence of local magnetic islands on such edges, we can induce backscattering on these channels at such defects [B. Jäck et al, PNAS 117 (2020)]. These edge modes also have properties that are consistent with those predicted for topological hinge modes, thereby suggesting that Bi is likely a higher order topological insulator [F. Schindler et al, Nature Physics 14 (2018)]. For other examples of our demonstration of novel topological boundary modes refer to our work on (quantum hall phases and topological boundary modes).
This video demonstrates scattering of the Dirac surface states of a topological Insulator. (from: H. Beidenkopf, et al, Nature Physics 7, 2011). Note: this video does not use audio.