Transcript Progress on EXPReS at JBO EXPReS Board Meeting, Utrecht, Jan 2008 Ralph Spencer and the JBO team.

Progress on EXPReS at JBO
EXPReS Board Meeting, Utrecht, Jan 2008
Ralph Spencer and the JBO team
Contents
•
•
•
•
•
Outline of JBO project
Manpower
Protocol work
Progress on iBOBs
Handover to Richard.
Jan 2008
Progress with iBOBs at Jodrell
Slide #2
Outline of Work at JBO
• NA4 Outreach work – Alastair Gunn - completed
• SA1 Connection of 4 e-MERLIN telescopes to JIVE (see under
Fabric)
• SA2: The 10 Gbps data link equipment available, awaiting cross
connection in Manchester between academic and commercial
links
• Fabric: Protocols for high speed networks –Stephen Kershaw :
PFLDNet 2008 Manchester 5-7 March
http://www.hep.man.ac.uk/PFLDnet2008/aboutManchester.php
• Fabric: Input to e-MERLIN correlator at 4 Gbps
Jan 2008
Progress with iBOBs at Jodrell
Slide #3
E-MERLIN---e-VLBI system
Jan 2008
Progress with iBOBs at Jodrell
Slide #4
January 2008
Network protocols
Stephen Kershaw
TCP
• Is TCP suitable for
transferring realtime data?
• If data is lost TCP
will slow
transmission and
retransmit.
• Demonstrate the
feasibility of using
TCP for real-time
applications
Jan 2008
Progress with iBOBs at Jodrell
Slide #7
CUBIC
TCP variants
• TCP using
different algorithms
for the
window/throughput
reduction
• Some variants
allow more efficient
use of bandwidth
and appear more
suitable for realtime applications
Jan 2008
HighSpeed
Progress with iBOBs at Jodrell
Slide #8
TCP variants
• Orders of
magnitude
improvements for
speed of recovery
with many variants
• Orders of
magnitude
reduction in
buffering
Note spikes due to bursts of data
• Evident bursty
behaviour –
concerns for highly
loaded packet
switched networks
Jan 2008
Progress with iBOBs at Jodrell
Slide #9
Multicast
Multicast reduces load on networks when sending to
multiple hosts
Test of multicast topology
• Multicast UDP stream over JANET academic network
• 100 2.2Mbps multicast IPTV streams originating from
London, UK
• End-hosts on Manchester University campus network
• All Cisco network hardware, 100Mbps interfaces
• Measurement of inter-packet arrival times
• Time from arrival of packet ‘n’ to arrival of packet ‘n+1’
• Expected ~6ms for one 2.2Mbps stream
Jan 2008
Progress with iBOBs at Jodrell
Slide #10
Multicast
• Latency spikes
of up to 0.5s at
intervals of 60s
• Unexpected
behaviour may
limit the
application of this
technology
• Correlation
between streams
and between PCs
shows a common
source of delay
Jan 2008
Progress with iBOBs at Jodrell
Slide #11
Multicast
Test of multicast topology - results
• Multicast traffic routing is not as efficient as unicast on this
network
• Potential concerns for other networks
• Does not look suitable for eVLBI
Jan 2008
Progress with iBOBs at Jodrell
Slide #12
DCCP
Stable kernel
• Require tuning of virtual memory settings to prevent kernel
panics
Experimental kernels
• Revisions of late November 2007 address some memory
and performance issues – yet to be tested on our systems.
• CCID4 in development
Performance
• CCID2 can attain 1GE line-rate but it is not a turn-key thing
Future work
• Test 2.6.24 kernel and recent DCCP git tree – not mature
enough yet for a definitive statement about suitability
Jan 2008
Progress with iBOBs at Jodrell
Slide #13
Progress on Digital Interfaces
iBOBs and all that
Jonathan Hargreaves/Richard Hughes-Jones
The iBOB System
• As a transmitter:
• Now able to transfer data at line rate to a test PC over a 10
GE (CX-4) link
• Needs A/D connection and interface (Onsala/Metsahovi)
• As a receiver
• Needs to interface to VSI chip on station board
• As a network test system
• See also Richard’s talk
• Fujitsu switch ordered
Jan 2008
Progress with iBOBs at Jodrell
Slide #15
JBO Receiver: Status
Recent Progress
• Sixteen packet FIFO implemented in the iBOB’s FPGA
• Even though packets are transmitted at 4Gbps they might ‘bunch up’ and
arrive at line speed. The FIFO is intended to smooth them out again
• The 10Gb transceiver can run at 200MHz – enough to handle line speed,
but the SRAM bandwidth is 167MHz for both read and write. Read out at
4Gbps consumes 64MHz of that. More resource than needed
• Data flows from the iBOB to the Station Board defined (see next slide)
• Data readout from the SRAM synchronised to the correlator clock, when
present
• When correlator clock is not present, iBOB defaults to test mode using
the internal clock
• VSI chip will buffer and reclock data and route it to the Station Board
input
Next Tasks
• Test the new code in hardware using another iBOB to generate test packets
• Write firmware for the VSI chip and either simulate or test on remote
hardware
Jan 2008
Progress with iBOBs at Jodrell
Slide #16
JBO Receiver: Block Diagram
Clock Control
Lock 128MHz
clock to correlator
200MHz System Clock
128MHz
.
10GB RX
FIFO
‘Yellow
box’ on
the
Simulink
design
4GB/s
CX4
SRAM Control
16k deep
.
Re-orders out of
order packets
Holds up to 16 packets
in case short bursts
arrive at line rate
DATA
64 bit
DATA
64 bit
.
Counts lost
packets
Buffer and
reclock data.
Correlator CLK
DATA 32 bit
Reads data out to
the correlator at a
steady 4Gbps
ADD
R
‘VSI chip’
DATA 32 bit
Reroute it to the
left and right
polarisation
station board
(SB) inputs.
.
DAT
A
SRAM
iBOB
512k x 64 bit
(512 packets =
9ms at 4Gbps)
2x
MDR80
‘VSI’
cables
Station Board
.
Jan 2008
Progress with iBOBs at Jodrell
DATA to
SB Input
Slide #17
DATA to
SB Input
Network Testing Device: Status
Recent Progress
• System clock increased from 100MHz to 200MHz to allow line speed data
flows at 10Gbps (JH)
• Added jumper bank to select ethernet IP and MAC address (JH & RHJ)
• All registers, test modes and results can be accessed via ethernet
control/response protocol (RHJ)
• PC based software provides user interface to the ethernet control link (RHJ)
• Ethernet based control supersedes the TinySH command line and is seen as a
prototype for control of iBOBs sending real data
• Currently testing iBOB to iBOB transfers on bench
Next Tasks
• Count lost packets, out of order packets and duplicate packets
• Update real time field correctly each second – currently it is fixed over a burst
• Test over network
Jan 2008
Progress with iBOBs at Jodrell
Slide #18