Experiences from Debugging a PCIX-based RDMA-capable NIC

• Published in: 2006 IEEE International Conference on Cluster Computing pp. 1 - 10
• Main Authors: Marazakis, M., Papaefstathiou, V., Kalokairinos, G., Bilas, A.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01-09-2006
• Subjects: Communication system software; Computer science; Counting circuits; Engines; Hardware; Interference; Network interfaces; Protocols; Software debugging; Transceivers
• ISBN: 1424403278; 9781424403271
• ISSN: 1552-5244; 2168-9253
• DOI: 10.1109/CLUSTR.2006.311919

Implementing and debugging high-performance network subsystems is a challenging task. In this paper, we present our experiences from developing and debugging a network interface card (NIC). Our NIC targets networked storage subsystems (Marazakis et al., 2006). For this purpose it mainly provides support for remote direct-memory-access (RDMA) write, sender-side notification of RDMA write completion, and receiver-side interrupt generation. In our work we examine issues that arise during system implementation and debugging, both in terms of correctness as well as performance. We present an analysis of the individual problems we encounter and we discuss how we address each case. For most problems we encounter, it is not possible to rely on existing debugging tools. However, we find that most of the techniques we use in this process, rely on collecting some form of event records from software or hardware components. We believe that such capabilities can be provided for independent hardware or software components in isolation, a fairly straight-forward task, thus, significantly simplifying the debugging process in complex systems of this nature