Nanoseconds Timing System Based on IEEE 1588 FPGA Implementation

• Published in: IEEE transactions on nuclear science Vol. 66; no. 7; pp. 1151 - 1158
• Main Authors: Pedretti, D., Bellato, M., Isocrate, R., Bergnoli, A., Brugnera, R., Corti, D., Dal Corso, F., Galet, G., Garfagnini, A., Giaz, A., Lippi, I., Marini, F., Andronico, G., Antonelli, V., Baldoncini, M., Bernieri, E., Brigatti, A., Budano, A., Buscemi, M., Bussino, S., Caruso, R., Chiesa, D., Clementi, C., Ding, X. F., Dusini, S., Fabbri, A., Ford, R., Formozov, A., Giammarchi, M., Grassi, M., Insolia, A., Lombardi, P., Mantovani, F., Mari, S. M., Martellini, C., Martini, A., Meroni, E., Miramonti, L., Monforte, S., Montini, P., Montuschi, M., Nastasi, M., Ortica, F., Paoloni, A., Previtali, E., Ranucci, G., Re, A. C., Ricci, B., Romani, A., Salamanna, G., Sawy, F. H., Settanta, G., Sisti, M., Sirignano, C., Stanco, L., Strati, V., Verde, G.
• Format: Journal Article
• Language: English
• Published: New York IEEE 01-07-2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Buffers; Cables; Clock synchronization; Clocks; Computers; Data acquisition; Data links; Data transmission; Delays; Dislocations; Electronics; Ethernet; Eye diagram; Field programmable gate arrays; field-programmable gate arrays (FPGAs); Finite state machines; front-end electronics; Hardware; Latency; Local area networks; Multicast; Nuclear research; Physics; Protocols; Synchronization; timing system; Topology; Upper bounds
• ISSN: 0018-9499; 1558-1578
• DOI: 10.1109/TNS.2019.2906045

Clock synchronization procedures are mandatory in most physical experiments where event fragments are readout by spatially dislocated sensors and must be glued together to reconstruct key parameters (e.g., energy and interaction vertex) of the process under investigation. These distributed data readout topologies rely on an accurate time information available at the front end, where the raw data are acquired and tagged with a precise timestamp prior to data buffering and central data collecting. This makes the network complexity and latency, between front-end and backend electronics, negligible within upper bounds imposed by the front-end data buffer capability where the raw data are stored waiting for the trigger validation. The proposed research work describes a field-programmable gate array (FPGA) implementation of IEEE 1588 Precision Time Protocol (PTP) that exploits the European Organization for Nuclear Research (CERN) timing, trigger, and control (TTC) system as a multicast messaging physical and data link layer. The hardware implementation extends the clock synchronization to the nanoseconds range, overcoming the typical accuracy limitations inferred by computers Ethernet-based local area network (LAN). Establishing a reliable communication between master and timing receiver nodes is essential in a message-based synchronization system. In the backend electronics, the serial data streams synchronization with the global clock domain is guaranteed by a hardware-based finite state machine that scans the bit period using a variable delay chain and finds the optimal sampling point. The validity of the proposed timing system has been proven in point-to-point data links as well as in star topology configurations over standard CAT-5e cables. The results achieved together with weaknesses and possible improvements are hereby detailed.