Transcript Document

10-th INTERNATIONAL CONFERENCE
ON INSTRUMENTATION FOR
COLLIDING BEAM PHYSICS
March 5, 2008
Novosibirsk, Russia, February 28 - March 5, 2008
 The
CMD-3 Data Acquisition and
Control System.
A.Ruban*, A.Aulchenko, K.Kakhuta, A.Kozyrev,
A.Selivanov, V.Titov, Yu.Yudin
BINP, Novosibirsk.
The CMD-3 Data Acquisition and Control System.
VEPP-2000 Collider Layout




Historical Siberian Snake
2E max = 1.4 GeV
L(1.4 GeV) = 31030 sm-2s-1
March 5,
2008
 Round Beams
 2 GeV
1031 sm-2s-1
L(2 GeV) =1032 sm-2s-1
The CMD-3 Data Acquisition and Control System.
CMD – 3 Subsystem Layout
March 5,
2008
1 – Vacuum Pipe
2 – Drift Chamber
3 – BGO Endcap
Calorimeter
4 – Z – Chamber
5 – Superconducting
Solenoid CMD-3
6 – LXe Calorimeter
7 – CsI Calorimeter
8 – Yoke
9 – Superconducting
magnet lenses
Mu-System and TOF
not showed
The CMD-3 Data Acquisition and Control System.
General Requirements to CMD-3 DAQ

Number of Channels










DC – 1260 Wires
ZC – 48 Sectors
- 512 Stripes
BGO – 680 Crystals
CsI – 1152 Crystals
LXe – 264 Towers
- 2112 Stripes
Mu – 48 Counters
TOF - 16 Counters
FLT – 400 Chanels

Time, ChargeX2
Time, Charge
Charge
Charge
Charge
Time, Charge
Charge
TimeX2, ChargeX2
TimeX2, ChargeX2
Words
Total up to 10k channels
March 5,
2008
The CMD-3 Data Acquisition and Control System.
General Requirements to CMD-3 DAQ

Time




Reliability



End-to-End testability
On-Line Data Check
DAQ Error Rate - less then 0.1%

Density


Average Speed – 1k Evtps
Dead Time – less then 5%
Common Stop Jitter – less then 20ps
Low EMI
Low Power

Low-cost

Legacy Interface if possible
March 5,
2008
The CMD-3 Data Acquisition and Control System.
The way we choose

March 5,
2008
Serial Bus rather then Parallel

Point-to-point media connection
 Low signal strength differential media


System Clock locked to Bunch Crossing
Data is transmitted along with A/D conversion

No Event buffers in Digitizing Modules
 No EMI when Event search in progress




Low EMI allow to have Preamps and A/D converter in single-board
solution
Minimize Power consumption to increase channel density
Commercially available components only
CAMAC compatible if possible
The CMD-3 Data Acquisition and Control System.
CMD-3 Data Acquisition Electronics Layout
VEPP2000
Storage Ring
SND
RF Cavity Freq.
Bunch Crossing
Particle Injection
DAQ
Synchro
Command,
Clock
March 5,
2008
to Event
Builder
Switch

CMD-3
Clbr
Pulser
Clbr
Pulser
Clbr
Pulser
Tracker
Frontend
Calorimeter
Frontend
TOF&Mu
Frontend
Track
Finder
Claster
Finder
Extended
Decision
Digitizer
T2Q
Digitizer
SAD
Digitizer
TQ
Data
Delivery

Groupe 1

Data
Delivery
Groupe 2


Trigger Data
Pipeline
Synchronization
Event Queue
and Time
Control
Data Collection
Status Control
and Check
Calibration
The CMD-3 Data Acquisition and Control System.
Data Flow
March 5,
2008

DAQ
Synchro
Command,
Clock


Digitizer
Data
Delivery

“DAQ Synchro” generates a
message with StartBite,
Command Code and Event
Number
“Data Delivery” distribute it
simultaneously to all
“Digitizer” modules and
Level-1 Trigger modules
All “Digitizers” return Data
of current Event
“Data Delivery” collects the
Data, and then transmit it to
Event Builder
The CMD-3 Data Acquisition and Control System.
CMD-3 Link intro
March 5,
2008

DAQ
Synchro
Down Link Interface
Up Link Interface

CMD-3 DAQ Link
Digitizer
Data
Delivery

All messages are
transported by specially
designed media “CMD3 DAQ Link”
DAQ System Bus
includes
~400 Links
This Bus is a serial
backplane with Pointto-Point connection and
root hierarchy
The CMD-3 Data Acquisition and Control System.
CMD-3 Link Stack
d
an nt er a
m ve b at d1
m
E um D or
Co
N
W
DAQ Synchro
sends Message
a
at rd8
D o
W
March 5,
2008
d
an nt er a
m ve b at d1
m
E um D or
o
C
N
W
Digitizer sends Data
Dead
Time
Analog signals setup time, up to 10us





a 2
at rd6 C
D o
CR
W
Dead
Time
A/D conversion in progress, up to 40us
Bidirectional Data Line, No preamble due to dedicated Clock Line
Coincidence of StartBit Level and Clock Line Transition is Common Stop
Messages are transmitted to “Data Delivery” Modules through Links
Answer from Digitizer contains Event Number, Board ID, Status Word
Command Code is covered with parity, Data is covered with CRC
The CMD-3 Data Acquisition and Control System.
CMD-3 Link Phy
Type
March 5,
2008
Synchronous
Modulation
Level
Clock speed
F bunch_Crossing*2 =
25 МHz
Data speed
25 Мbps
Electrical Levels
Media
Connector Type
LVDS, 4mA
same as IEEE-1394
Double UTP Cat5
130Ohm, double side
termination
USB, type А, both ends
The CMD-3 Data Acquisition and Control System.
Read_Status
Injection_Begin
Injection End
Calibration_Start
Check_Read
Event Number
Event Code




Transaction Code
Time Set Code
Event
Manager
Start Transaction at
Link's Groupe1
Start new Event
Enable next Event
Down-Link
Interface1
nk
e16
Link16
Link2
Link1
Event
Controller
Transaction Complete
Off-System Synchro 1
Off-System Blocking
Request Layer
March 5,
2008
Start Transaction at
Link's Groupe2
Status Control
Tracker_Start
Claster_Start
Mixed_Start
Sim_Normal_Start
Sim_Random_Start
Off-System_Start
Start Measure
Event’s Queue and Time Control
Event Layer
Off-System Synchro 2
Transaction Layer
Signaling Layer
“Event Manager” accepts Requests, and Builds Event’s Queue
“Event Controller” serves Queue one-by-one and convert it to Stream of DAQ
Messages
Trigger’s Event are serviced immediately or skipped other are settled in queue
Messages are transmitted to “Data Delivery” Modules through Links
The CMD-3 Data Acquisition and Control System.
Data Collection
March 5,
2008
Up_Link
Link from
DAQ Synchro Interface
Link to
Digitizer 30
Link to
Digitizer 1
SRAM
8MB
Down_Link
Interface 30
Down_Link
Interface 1
SRAM
Controller
Phy
Eth100 to
Switch
Eth100
Controller
Command and Clock FanOut




“Data Delivery” distributes the Message to Digitizers
“Data Delivery” receives Data and accumulates them in RAM
Ethernet 100 Controller transmits Data from RAM through
Ethernet Switch to Event Builder
Ethernet Switch solves a collision problem
The CMD-3 Data Acquisition and Control System.
Inside Digitizer
March 5,
2008



When Message
reaches the Digitizer
associated Command
List is activated and
executed
Any Digitizer resource
can be accessed for
read or write as
preprogrammed in
Message specific
Command List.
“Redirector” feature
includes Command
List’s RAM and glue
Logic.
The CMD-3 Data Acquisition and Control System.
Trigger Data Pipeline Synchronization
March 5,
2008
Each Triggering Module has “Cavity Frequency Restorer”, based on
FPGA’s PLL and binary counter. Relative phase of counter is settled by
leading edge of Common Stop. This insures that all “Cavity Frequency
Restorers” in DAQ has same phase. Introducing Link-to-Link delay we
can control and align pipeline latency of different Triggering Module.

Calibration

When Calibration Event generated, Links to Calibrating Modules are
started immediately while some delay introduced for all “Normal” Links.
Pushing this Link-to-Link delay we can control Calibrating process.
The CMD-3 Data Acquisition and Control System.
CMD-3 “Standard Design” for board’s compatibility.






March 5,
2008
To obtain compatibility of
different digitizers in DAQ
single style is required.
Specific “Standard
Design” was developed.
It includes all function
required for digitizer to
work in DAQ.
It supports full
independent access from
any interface.
Supports modular
approach for Board
specific Controllers
Code is written in Altera
HDL
The CMD-3 Data Acquisition and Control System.
Module for synChronization of System MChS
March 5,
2008
Module for synChronization of System MChS
RF Cavity Freq.
Conditioning
Clock Fanout

Bunch Crossing
Freq.
Conditioning
Utility
Generators
DAQ CMD-3
DAQ Memory
Residue Check
Veto Mu
Off-System
Signal Scalers
and TDC
DAQ Interface
Up-Link
DAQ Interface
Down-Link 1
ce
16
Link 16

Link 1

Status
LEDs

Veto TOF
Internal Bus
CAMAC
Interface
CAMAC
Status Signal
Conditioning
Queue Control
and Timing Logic
Triggering
Signal Scalers
Low Jitter
Resynchronization
Buffer
Phase Splitting
and Aligning
Logic functions
which are
implemented and
tested with
Prototype Module.
Hardware features
to be added for
specified jitter
performance.
“Standard Design”
Automatic Status
Control.
The CMD-3 Data Acquisition and Control System.
CMD-3 DAQ Status







March 5,
2008
“DAQ Synchro”:
 Prototype module with 6 Down_Links are under tests since
April 07
 Full size, full feature Module will come soon.
“Data Delivery”:
 Two serial modules are under tests since November 07
“Digitizers”:
 5 types of Digitizers and TrackFinder are under tests now
Test modules Down_Link and Up_Link are in use since
September 06
Error Rate in 1 Link at testbench – less than 10-8
Error Rate in full System
– less than 10-4
All Speed requirements are achieved
The CMD-3 Data Acquisition and Control System.
End

Thank You for attention
March 5,
2008