Note that this data rate is the maximum pulsar search and single pulse
data rate that the SaDT links can support (given the maximum link
occupancy SDP can support) and is therefore a constraint on the pulsar
search and single pulse output data rate and not the actual pulsar
search and single pulse data rate at any point in time. This data rate
takes into account all Layer 1 to 7 protocol and file format overheads.
The total pulsar search and single pulse output data rate is calculated
as: Msearch_input / Ttransfer +
Msingle_pulse_input / Tcadence
bytes/s. See section 5.2.1.6.4 and section 5.2.1.6.5.
The maximum allowed PSS data rate (as calculated above)
constrains the pulsar search and single pulse parameters,
summed over all simultaneous scheduling blocks to:
Msearch_input / Ttransfer +
Msingle_pulse_input / Tcadence ≤ 55.90
GB/s
5.2.1.6.4. Pulsar
search data rate
The data volume per scan is given by:
Msearch_input = Nbeams x
(Ncand x (Mcand + Msheet
+ Mmetadata) + Mlist)
Where:
-
Candidate list: Mlist = 10 kB
-
Candidate data cube size: Mcand =
nchan x nbin x nsub x
nbyte
-
S/N sheets size:
Msheet = ((Trialp x
Trial.p) + (Trialp x
Trialdm) + (Trial.p x
Trialdm)) x Nbyte;sheet
-
Metadata: Mmetadata = 10 kB
The pulsar search output data rate is: Msearch_input /
Ttransfer bytes/second
5.2.1.6.5.
Single pulse/fast transient data
rate
The data volume per scan is given by:
Msingle_pulse_input = Nbeams x
(Nburst x (Mburst +
Msheet + Mmetadata) +
Mlist)
Where:
-
Candidate list: Mlist = 10 kB
-
Candidate data cube size: Mburst =
nsamp x nchan x npol x
nbyte
-
S/N sheet size: Msheet = (Trialdm +
nchan + TBD-008) x Nbyte;sheet
-
Metadata: Mmetadata = 10 kB
The single pulse output data rate is:
Msingle_pulse_input / Tcadence
bytes/second
5.2.1.7.
Quality attribute
characteristics
5.2.1.8.
Rationale and design issues
TCP: Data transfer must be reliable. TCP provides the required reliable
data delivery. Using UDP would require retransmission by the application
layer which would simply duplicate the functionality of TCP to no
obvious advantage.
FTP: Transfers are periodic and independent (for each beam, each block
of data is self contained and is processed independently). Note
that FTP (in spite of the name) does not imply that real files have to
exist at either end of the link - the protocol is a mechanism for
transferring a block of data with a name.
FTP is a mature protocol with many high quality implementations in a
wide range of languages.
5.2.2. Presentation
Layer
(OSI layer 6) N/A
5.2.3. Session
Layer
5.2.4. Transport
Layer
specified in [RD2]
5.2.5. Network
Layer
specified in [RD2]
5.2.6. Data Link
Layer
Refer to [RD1] and [RD2]
5.2.7. Physical
Layer
Refer to [RD1] and [RD2]
5.3.
SDP – CSP Pulsar Timing Data Interfaces I.S1M.SDP_CSP.003
The data exchange between the SDP and the CSP for the Pulsar Timing data
interface is fully described in this section following [AD7] format.
The communication interface between the CSP and SDP will be
bi-directional although the data flow will be uni-directional (from CSP
to SDP). Note that for this interface the term PST refers to the PST
subsystem of CSP rather than “Pulsar Timing” in general.
The Pulsar Timing data transferred from CSP to SDP will be data cubes in
one of 3 forms, corresponding to the 3 observing modes of the PST:
-
Pulsar Timing mode: folded pulse profiles.
-
Dynamic spectrum mode: high resolution time, frequency polarisation
data
-
Flow Through mode: raw voltages from the beamformer (beam-formed
voltage data)
There will be up to 16 of these independent data sets for 16 different
pulsar timing beams.
SDP will also receive metadata from TM [RD4] which will be used for
RFI mitigation, calibration of flux and polarisation and the requested
averaging products. This metadata is used by SDP for processing the
Pulsar Timing data and its description is therefore out of scope of this
interface.
5.3.1.
Application layer (Software
Interface)
5.3.1.1. Interface
Identity
This interface uses FTP (RFC 959) to transfer PST data from CSP to SDP.
(SDP_REQ_INT-64)
5.3.1.2.1. Data
organisation
The PST output data from each Pulsar Timing beam, for all 3 Pulsar
Timing modes, is produced per one or more sub-integrations, typically
every 10 seconds (1s - 60s range). This data is formatted as PSRFITS
files and transferred to SDP as soon as they ready. A data stream is
defined as all Pulsar Timing data files of a specific beam for the
duration of a scheduling block.
5.3.1.2.2. Data
routing
Data streams (as defined above) are routed to SDP nodes. The complexity
of the SDP processing will determine how many data streams are processed
on a single SDP node and the routing of data streams will be done
accordingly.
SDP will supply the data routing information to CSP (via TM) [RD4]
for a particular scheduling block and this routing will remain static
for the duration of the scheduling block (SDP_REQ_INT-316). The
routing information will be supplied to CSP prior to the start of the
scheduling block. Since routing is done per beam the routing information
will contain the following for each beam:
-
Beam ID
-
SDP destination node IP address
-
SDP destination node port
5.3.1.2.3.
Sending and receiving data
SDP will be the server (TBC-011) (at least one per receiving
node). When a scan is configured each CSP node opens a control
connection to a server. When a block of data is ready to be transmitted,
CSP sends a STOR command with at least the Scan ID, beam ID and
Scheduling Block ID encoded in the ”file name” parameter. SDP will open
data channel (if not already open) in the usual way and CSP transmits
the data in BLOCK mode.
The following parameter setting should be used:
-
FILE TYPE: IMAGE (The data is sent as a contiguous stream of bits.
Normally used to transfer binary files)
-
FORMAT: NON PRINT (The file contains no vertical format information)
-
STRUCTURE: File (The file is considered as a contiguous stream of
bytes. There is no internal file structure)
-
TRANSMISSION MODE: Block (The file is transferred as a series of
blocks, each preceded by one or more header bytes)
Block mode is recommended as this allows the data connection to be
reused and provides reliable indication that a transfer is complete.
5.3.1.3.
Data-preconditions
5.3.1.4 Data types
and constants
5.3.1.4.1. Pulsar
Timing data types
The folded pulse profile data are defined as the second moments
of the electric field (e.g. the Stokes parameters) averaged as a
function of pulsar longitude, radio frequency, and integration epoch.
The data cube may be stored in multiple PSRFITS files, with one or more
sub integration per file.