In applications where data flows need to take place in nanoseconds, the delay between when an instruction is received and when it executes, known as latency, is very important. But latency is different from determinism, which is knowing exactly when an instruction will execute. It’s an important distinction to be aware of when building embedded systems.
If your system will provide information to a pilot who needs to make decisions in only tens of milliseconds, you certainly want to reduce any delays in the information chain. However, in instances where users don’t need to make real-time decisions, such as while reviewing camera footage from a UAV fly-by, then some latency from an embedded network may not be an issue.
CPU vs GPU vs FPGA: Which Wins When It Comes to Latency?
It’s also imperative to choose the right processor type, whether a CPU, a GPU, or an FPGA, for your embedded system because each has its own considerations when it comes to latency and determinism. Increasingly, military system designers are putting the massive floating-point throughput and internal pipelining architecture of GPUs to good use, yet FPGA latency is still lower because traditional GPUs must use bus interconnects (i.e. PCIe lanes) to move data off-chip and then communicate with a CPU.
For some radar and signals intelligence (SIGINT) applications, the latency performance of dedicated GPUs is surely acceptable, yet for others it’s not. Some modern CPUs have started to embed GPUs into the same silicon device to improve efficiencies, while some FPGAs are used to send data directly to CPUs or GPUs. In either case, latency is reduced compared to a dedicated, externally connected GPU device. Latency is decreasing for newer GPUs as their bus speeds increase, but the latency of an FPGA is still more deterministic.