6: Transient Search: Logical Model
5.2.1. Application
layer
5.2.1.1. Interface
Identity
This interface uses FTP (RFC 959) to transfer Pulsar Search and Fast
Transient/Single Pulse data from CSP to SDP.
5.2.1.2.1. Data
organisation
OCLD/SPOCLDs are generated per beam per pulsar search scan (180 - 1800
seconds) and delivered from CSP to SDP. A data stream is defined as all
OCLD/SPOCLD data files of a specific beam for the duration of a
scheduling block. (SDP_REQ_INT-317) (refer to [RD5] for definition
of scheduling block)
If possible OCLDs will be divided into smaller packets/files (on a time
or candidate basis) and transferred to SDP as soon as they are ready
(TBC-012 ), otherwise OCLDs will be transferred as complete
entities (files or objects).
5.2.1.2.2. Data
routing
Data streams (as defined above) are routed to SDP nodes. The complexity
of the SDP processing will determine how many data streams are processed
on a single SDP node and the routing of data streams will be done
accordingly.
SDP will supply the data routing information to CSP (via TM) [RD4]
for a particular scheduling block and this routing will remain static
for the duration of the scheduling block (SDP_REQ_INT-316). The
routing information will be supplied to CSP prior to the start of the
scheduling block. Since routing is done per beam the routing information
will contain the following for each beam:
-
Beam ID
-
SDP destination node IP address
-
SDP destination node port
5.2.1.2.3.
Sending and receiving data
SDP will be the server (TBC-011) (at least one per receiving
node). When a scan is configured each CSP node opens a control
connection to a server. When a file or block of data is ready to be
transmitted, CSP sends a STOR command with at least the Scan ID, Beam ID
and Scheduling Block ID encoded in the ”file name” parameter. SDP will
open data channel (if not already open) in the usual way and CSP
transmits the data in BLOCK mode.
The following parameter setting should be used:
-
FILE TYPE: IMAGE (The data is sent as a contiguous stream of bits.
Normally used to transfer binary files)
-
FORMAT: NON PRINT (The file contains no vertical format information)
-
STRUCTURE: File (The file is considered as a contiguous stream of
bytes. There is no internal file structure)
-
TRANSMISSION MODE: Block (The file is transferred as a series of
blocks, each preceded by one or more header bytes.)
Block mode is recommended as this allows the data connection to be
reused and provides reliable indication that a transfer is complete.
If an OCLD is divided into smaller packets/files for transmission, then
SDP will reconstruct the OCLD once all the data for scan has been
received.
5.2.1.3. Data
pre-conditions
5.2.1.4. Data types
and constants
5.2.1.4.1. Candidate
List
The candidate list is a form of metadata, providing a summary of all
significant candidate detections (all candidates found before sifting)
made per beam, where significance is determined by the signal processing
pipeline within CSP. Candidate lists are important, as they provide
information useful for improving candidate filtering. The precise number
of entries in the list will vary for each scan. However the length of
the list will be capped at TBD-003 entries.
The candidate list will be formatted as a table. The columns include,
but are not limited to: Timestamp, Beam ID, Scheduling Block ID, S/N,
Acceleration, DM, Period, Pulse Width, duplicates, number of harmonics,
and TBD-004. (SDP_REQ_INT-61) Candidates are ranked and sorted by
TBD-005 manner (most likely according to S/N). (SDP_REQ_INT-297).
File Format: CSV file using ASCII encoding (RFC 4180)
5.2.1.4.2.
Optimised Candidate Data
Optimised candidate data provides a summary description of each sifted
candidate for scientific analysis consisting of a data cube and
associated metadata. The sifted candidates are candidates surviving the
sifting process undertaken within CSP, a process which aims to remove
duplicate detections of the same signal.
5.2.1.4.2.1. Optimised candidate data cube
For each sifted candidate, a data cube describing the detection in time,
phase, and frequency space is stored. The data cube represents the best
detection of a candidate.
File format: PSRFITS (SDP_REQ_INT-301 & SDP_REQ_INT-291)
The PSRFITS version number is not specified in this ICD because it shall
be defined in the header of the file.
5.2.1.4.2.2. S/N Sheets
For each sifted candidate multiple signal-to-noise (S/N) sheets are
stored.
File format for the S/N sheets: HDF5
5.2.1.4.2.3. Metadata (per candidate)
Each cube is accompanied by metadata, which further helps describe the
detection. Such metadata includes, but is not necessarily limited to,
information associated with the scan (Beam ID, Scheduling Block ID,
quality measures, etc.).
This metadata is contained in the PSRFITS and/or HDF5 files.
Failure of the communication link or the receipt of incorrectly
formatted data results in an alarm being generated.
This section shows the variability than can be expected and any
constraints on specific use cases of the interface or particular aspects
of the interface.
Note that parameter values for modes shown in this section are
expected or typical values and are provided for information only. This
interface does not constrain parameter values unless explicitly stated
as a constraint. Actual parameter values will be chosen by Telescope
Manager (for each scheduling block and its scans) according to the
requirements of the observation and any constraints imposed by
components of the telescope [AD6].
5.2.1.6.1.Pulsar search
data
The pulsar search data cube represents the best detection of a
candidate, ‘folded’ and dedispersed at the optimised dispersion measure,
period, and period derivative values - all determined within CSP. The
data cube (per candidate) has the following axes (SDP_REQ_INT-300):
-
frequency channels
-
pulse profile bins
-
subintegrations
The pulsar search candidate data also includes S/N sheets (three
2-dimensional matrices) which describe the resulting folded S/N ratio
for different combinations of folding parameters:
-
trial period and trial period derivatives
-
trial period and dispersion measure trial values
-
trial period derivatives and dispersion measure trial values
Table 12 describes the list of pulsar search parameters and expected
values for each parameter.
12: Pulsar search parameters