Those who are used to using the Linux terminal window are certainly familiar with the tool top: Provides a real-time view of system activities, including running processes, CPU, memory, and other resource usage. It is useful for monitoring system performance and identify any processes that may behave abnormally.
Beyond top there is also utility htop which offers some interesting improvements and more flexibility. Starting from the graphic interface, more colorful and intuitive, which allows you to more easily recognize processes and related information.
Monitoring the GPU on Linux is increasingly crucial: how to do it with nvtop
With the advent of generative models increasingly complex and powerful, GPUs are also increasingly relied upon to manage the inference phase.
The phase of inference it is the one that is of most interest to end users who use models and implement them in AI-based apps. It is the process by which a trained model is used to make predictions, generate new data or make decisions on new inputs. During this phase, the model uses the information acquired during thetraining to process new information and produce outputs that are as relevant and well-argued as possible.
What is the nvtop utility and how does it work
nvtopacronym for Neat Videocard TOPis inspired by the functioning of the system utilities mentioned at the beginning but is designed to monitor GPUs and accelerators.
Currently, nvtop supports AMD (driver amdgpu for Linux), Apple (limited support for M1 and M2 SoCs), Huawei (Ascend), Intel (drivers i915 for Linux), NVIDIA (proprietary drivers for Linux) and Qualcomm Adreno (MSM drivers for Linux). However, it should be taken into consideration that, for example, for AMD GPUs a Linux kernel with a version equal to or higher than 5.14 is required, while for Intel GPUs a kernel equal to or higher than release 5.19 is required. For NVIDIA GPUs, nvtop uses the NVML library (NVIDIA Management Library), included with the official drivers.
The nvtop interface provides a wide range of options and interactive commands for monitoring GPU behavior. Built-in configuration features allow you to customize the interface according to your needs by simply pressing F2 and selecting your preferred options. You can save the preferences set by pressing F12so that they are automatically loaded the next time nvtop runs.
How to use nvtop on major Linux distributions
nvtop can be installed on different Linux distributions using precompiled packages, specific repositories or manually compiling the source code. The easiest way to use nvtop on Debian, Ubuntu and derivative distributions is to type the following command:
sudo apt install nvtop
By consulting the project’s GitHub page, however, you can find the installation instructions for other distributions such as Fedora, Red Hat, CentOS, OpenSUSE and Arch Linux. Additionally, alternatives are available such as AppImage, Snap e Docker to simplify the installation and deployment process.
nvtop is distributed under license GPLv3guaranteeing users the freedom to use, modify and distribute the software in accordance with the guidelines of Free Software Foundation.
Opening image credit: Copilot Designer.