A shift is underway in high performance computing (HPC) towards heterogeneous parallel architectures that emphasize medium and fine grain thread parallelism. Many scientific computing algorithms, including simple finite-differencing methods, have already been mapped to heterogeneous architectures with order-of-magnitude gains in performance as a result. Recent case studies examining high-resolution shock-capturing (HRSC) algorithms suggest that these finite-volume methods are good candidates for emerging heterogeneous architectures. HRSC methods form a key scientific kernel for compressible inviscid solvers that appear in astrophysics and engineering applications and tend to require enormous memory and computing resources. This work presents a case study of an HRSC method executed on a heterogeneous parallel architecture utilizing hundreds of GPU enabled nodes with remote direct memory access to the GPUs for a non-trivial shock application using the relativistic magnetohydrodynamics model.