Lessons learned from operating a large network telescope

Abstract

Network telescopes (aka darknets) collect unsolicited Internet traffic (aka Internet background radiation or IBR), which includes benign and malicious scanning as well as artifacts of spoofed denial-of-service attacks and misconfigured software and hosts. Analysis of this traffic has revealed macroscopic insights into security-related events and global network dynamics such as outages. Operating a large-scale network telescope is challenging but often taken for granted, more so than in more mature scientific disciplines. We offer the first study documenting our experiences operating the UCSD Network Telescope, the largest and longest-operating network telescope supporting scientific research. We provide background on the history of the telescope, and focus on increasing operational challenges as the underlying network evolves. We develop and apply techniques to leverage third-party scanning activity to validate the integrity of the data, and to discover misconfigurations in the instrumentation. These insights are crucial for understanding measurement results, which we illustrate using concrete examples. We discuss how our findings generalize to support the expanding ecosystem of other passive techniques, such as honeypots, to track security phenomena.