David C. Doughty Jr., David Game, Lisa Mitchell, Stephanie Holt Christopher Newport University Graham Heyes, W. A. Watson III Continuous Electron Beam Accelerator Facility
Abstract
Data acquisition architectures in the next millennium will face the challenge
of moving hundreds to thousands of megabytes of data each second from front
end electronics crates to on-line processor farms. Event building in such
environments requires an approach that is both scaleable and parallel,
and which minimizes contention for links and processor resources. We have
investigated ATM switches and links for use as the event building fabric for
future DAQ architectures, and plan to employ them in the CLAS detector at
CEBAF.
One of the often-cited problems with ATM is that of link contention and cell-loss. To avoid these problems a linked dual token passing algorithm has been developed, with two different types of tokens being passed through the switch. The Tfarm processor token circulates among the farm processors (FPs) and controls which of them will receive a specific block of events. As each FP takes an event block it sends a second type of token, an Tevent request tokenU down to the first data concentrator (DC), identifying itself as the recipient of data from that event block. Each DC may be a readout controller (ROC), or may collect data from several ROCs.
The first DC enqueues these tokens (in event order); when it has collected its fragments of the events specified by the token on top of the queue, it sends that event fragment block to the requesting FP. It then sends the token through the switch to the next DC in the chain, which does the same. This continues until the last DC has sent its fragments to the FP, after which it sends the token back.
Since multiple data request tokens may be active simultaneously, with each DC sending its data to a different FP, this leads naturally to a Tbarrel shifterU type of parallel data transfer. This architecture is also scaleable since larger numbers of DCs or FPs can be accommodated either by using larger switches, or by cascading smaller ones. Phased event building may be achieved by adding additional DCs at various points in the switch cascade.
We have developed a simulation of this architecture using the MODSIM language. The simulation uses tables to set the parameters of the model, which include the event rate, event size in each DC, number of DCs and FPs, the MIP ratings of the DCs and FPs, the computational power required by the link protocol, and the amount of analysis required of the DCs and FPs. The results of our simulations for systems of various sizes will be presented, and will be compared with the actual implementations of small systems using a recently acquired switch.
David C. Doughty Jr. Christopher Newport University doughty@pcs.cnu.edu Tel #: (804) 594-7065 Fax #: (804) 594-7919