Phil Marshall edited Challenge Structure.tex  over 10 years ago

Commit id: f83afa80d34f33f0bd092406316002a959892983

deletions | additions      

       

TDC1 is the actual challenge. It consists of thousands of sets of simulated light curves, also arranged in rungs of increasing difficulty and realism. The large data volume is chosen to simulate the demands of an LSST like experiment, but also to be able to detect biases in the algorithms at the subpercent level. The evil team expects that processing the TDC1 dataset will be challenging with current algorithms in terms of computing resources. TDC1 thus represents a test of the accuracy of the algorithms but also of their efficiency. Incomplete submissions will be accepted, although the number of processed light curves is one of the metrics by which algorithms are evaluated, as described below.   The mock data generated for the highest rungs of the initial challenge ladders TDC0 and TDC1 are as realistic as our current simulation technology allows, but lower rungs are somewhat simplified. This design is based on the successful weak lensing STEP \citep{} \citep{STEP1,STEP2}  and GREAT \citep{} \citep{GREAT08,GREAT10Stars,GREAT10Galaxies}  shape estimation challenges, where the former tried to be as realistic as possible, while the latter focused on specific aspects of the problem. Still, following a successful outcome of TDC0 and TDC1 we anticipate in the future further increasing the complexity of the simulations so as to stimulate gradual improvements in the algorithms over the remainder of this decade. Of course our approach of testing on simulated data is very complementary to tests on real data. The former allow one to test blindly for accuracy but they are valid only insofar as the simulations are realistic, while the latter provide a valuable test of consistency on actual data, including all the unknown unknowns.  \subsection{Instructions for participation, timeline, and ranking criteria}