Casey Law deleted file realtime.tex  about 10 years ago

Commit id: 4171404616252d39b2bc5a3ef354a5ea81ce600e

deletions | additions      

         

\section{Research Direction}     %The technical requirements for our radio transient searchs are extreme in astronomy, but are becoming more common (e.g., see plans for the SKA and LSST). Lessons learned from our project will have increasing relevance to scientists working to solve the "needle in a haystack" problem.     %The transient search problem I am describing here is distinguished from that of the optical transient community in a few ways. First, radio interferometers are computationally dominated by the process of generating images and detecting sources. This process is inherently more parallelizable than the source classification process that is the limiting step in optical transient searches. Second, radio transients are rarer and fainter than typical optical transients. So, while optical transients focus their efforts on classifying transients based on an abundance of information, radio transient searches are more concerned with detection.     %Currently, we are recording data to disk at a rate of 1 TB hour$^{-1}$ and processing it on compute clusters near the VLA, at Los Alamos National Lab, and NERSC. The internet is too slow to transport the 1 TB hour$^{-1}$ data stream, so we ship disks to our computing centers. This approach is complex and not sustainable in the large campaigns needed to find many fast radio transients.     %I am interested in thinking about how real-time detection can help solve the challenges of big data. By bringing computational support closer to the telescope, real-time detection makes it possible to decide whether a given segment of data is worth saving or not. This "data triage" may cheapen data, but it is necessary to access science in some high data rate streams.