How to Install NVIDIA Drivers and CUDA on Ubuntu 24.04

If you are deploying a GPU server on Ubuntu 24.04, the cleanest approach is to use NVIDIA’s official APT repository and NVIDIA-provided Debian packages for Ubuntu 24.04 instead of the standalone .run installer. NVIDIA officially supports Ubuntu 24.04 for CUDA, and its Linux installation guide recommends distribution-specific packages where possible because they integrate with the system package manager more cleanly.

This guide shows how to install the NVIDIA driver first, then install CUDA 13.1 on Ubuntu 24.04, and finally enable Docker GPU support with the NVIDIA Container Toolkit. It is written for bare-metal or dedicated GPU servers running Ubuntu 24.04 LTS.

Want the easier route?

👍
Provisioning GPU servers manually is straightforward when you follow the right process, but it still involves repository setup, package installation, and reboots

If you would rather receive the server ready to use, Servers99 can provision your GPU server with Ubuntu 24.04, NVIDIA drivers, CUDA, and Docker before handover. Just open a support ticket or add the request in your order notes.

Why this method?

NVIDIA’s current documentation for Linux recommends package-manager installs over the standalone runfile whenever possible. On Ubuntu, the official driver guide uses the NVIDIA repository with apt, and the CUDA guide uses the same repository model for toolkit installation. That makes updates, dependency handling, and version pinning much easier than a manual .run workflow.

A second reason to use this method is version control. NVIDIA notes that the generic cuda and cuda-toolkit packages track newer releases over time, while versioned packages such as cuda-toolkit-X-Y let you stay pinned to a specific release. NVIDIA’s Ubuntu 24.04 repository currently includes both cuda-toolkit-13-1 and cuda-runtime-13-1, so you can lock this installation to CUDA 13.1 instead of rolling forward automatically.

Before you begin

This guide assumes:

  • Ubuntu 24.04 LTS
  • An NVIDIA GPU already present in the server
  • SSH or console access with sudo
  • x86_64 / amd64 architecture

NVIDIA’s CUDA documentation says you should first verify that the system has a CUDA-capable GPU, confirm you are on a supported Linux release, and ensure GCC is available if the server will be used for CUDA development rather than runtime only. NVIDIA’s Ubuntu driver guide also recommends installing kernel headers for the currently running kernel before driver installation.

Step 1: Confirm the GPU is detected

1
Run:
Bash lspci | grep -i nvidia

NVIDIA documents this as the standard first check for detecting whether the system can see the GPU. If this command returns nothing, stop here and confirm the hardware, BIOS settings, and PCIe visibility before attempting driver installation.

Step 2: Install the required base packages

2
Install kernel headers first:
Bash sudo apt update sudo apt install -y linux-headers-$(uname -r)
3
For development servers, also make sure GCC is available:
Bash gcc --version

♦️NVIDIA’s driver guide explicitly calls for the current kernel headers on Ubuntu 24.04, and the CUDA installation guide states that GCC is required for CUDA development, though not for simply running CUDA applications.

Step 3: Add NVIDIA’s official Ubuntu 24.04 repository

4
For Ubuntu 24.04 x86_64, add the official NVIDIA repository keyring:
Bash wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb sudo dpkg -i cuda-keyring_1.1-1_all.deb sudo apt update

♦️These are the official NVIDIA network-repository steps for Ubuntu. NVIDIA uses the cuda-keyring package to enroll the repository signing key and enable the CUDA repository for APT-based installs

Step 4: Install the NVIDIA driver

5
For the standard proprietary kernel-module path, install:
Bash sudo apt install -y cuda-drivers sudo reboot

♦️NVIDIA’s Ubuntu driver guide lists cuda-drivers as the proprietary-kernel-modules install path. NVIDIA also documents nvidia-open as the open-kernel-modules alternative, but cuda-drivers is one straightforward full-driver option when you want the standard proprietary NVIDIA driver stack on Ubuntu 24.04.

⚠️
Note: After the reboot, reconnect to the server and continue.

Step 5: Install CUDA 13.1

Choose one of the following depending on the role of the server.

6
Option A: Runtime-only install

Use this on inference servers or nodes that only need to run CUDA applications:

Bash sudo apt install -y cuda-runtime-13-1 sudo reboot
7
Option B: Full development toolkit

Use this if the server will compile CUDA code and needs nvcc, headers, libraries, and developer tools:

Bash sudo apt install -y cuda-toolkit-13-1 sudo reboot

NVIDIA describes the cuda-runtime line as the runtime-focused package set for compute nodes, while the cuda-toolkit line installs the full development toolkit. NVIDIA also documents version-locked packages for staying on a specific release branch, and the Ubuntu 24.04 repository currently contains both 13.1 package families.

Step 6: Configure the CUDA PATH

NVIDIA’s post-install steps state that the CUDA bin path must be available in PATH. For package-manager installs, that is the main environment change you normally need. NVIDIA documents LD_LIBRARY_PATH mainly in the context of runfile installations, so this guide keeps the package-manager path simple.

8
Create a profile script:
Bash echo 'export PATH=/usr/local/cuda/bin:$PATH' | sudo tee /etc/profile.d/cuda.sh >/dev/null sudo chmod 644 /etc/profile.d/cuda.sh source /etc/profile.d/cuda.sh

♦️If you install multiple CUDA versions later, NVIDIA documents update-alternatives for switching the active /usr/local/cuda symlink between installed releases:

9
Bash update-alternatives --display cuda sudo update-alternatives --config cuda

♦️ That is the supported way to manage side-by-side versions instead of editing random shell files repeatedly.

Step 7: Verify the installation

10
Start with the driver check:
Bash nvidia-smi

♦️ nvidia-smi is NVIDIA’s system management interface for supported Linux distributions and is the quickest way to confirm that the driver is loaded and the GPU is visible to the OS.

11
Then verify the CUDA compiler if you installed the full toolkit:
Bash nvcc --version

♦️ For a deeper validation, NVIDIA recommends compiling and running the deviceQuery and bandwidthTest samples from the CUDA samples repository. In NVIDIA’s own post-install guidance, those checks confirm that the toolkit can communicate correctly with the CUDA-capable hardware.

Optional: Enable GPU access inside Docker

If you plan to run CUDA workloads in containers, install the NVIDIA Container Toolkit after the host driver is working. NVIDIA’s container-toolkit guide explicitly recommends installing the GPU driver first by using the package manager for your Linux distribution.

12
Install the toolkit repository and packages:
Bash sudo apt-get update && sudo apt-get install -y --no-install-recommends \ ca-certificates \ curl \ gnupg2 curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \ sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg && \ curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \ sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \ sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list sudo apt-get update export NVIDIA_CONTAINER_TOOLKIT_VERSION=1.18.2-1 sudo apt-get install -y \ nvidia-container-toolkit=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \ nvidia-container-toolkit-base=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \ libnvidia-container-tools=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \ libnvidia-container1=${NVIDIA_CONTAINER_TOOLKIT_VERSION}
13
Then configure Docker to use the NVIDIA runtime:
Bash sudo nvidia-ctk runtime configure --runtime=docker sudo systemctl restart docker

♦️ These commands follow NVIDIA’s current Debian/Ubuntu installation path for the NVIDIA Container Toolkit and Docker runtime configuration. NVIDIA notes that nvidia-ctk updates Docker’s daemon configuration so Docker can use the NVIDIA runtime.

Ready to Deploy on a Production-Ready GPU Server?

Installing NVIDIA drivers, CUDA, and Docker manually is possible, but for production environments, many teams prefer a server that is ready from day one. With Servers99, you can deploy high-performance GPU servers built for AI training, inference, rendering, and compute-heavy workloads, with fast provisioning, reliable infrastructure, and expert support when you need it.

AI-Ready Deployments
Powerful GPU Options
Fast Provisioning
Expert Support
Deploy Your GPU Servers with Servers99

Common mistakes to avoid

NVIDIA states that the generic cuda package tracks the latest stable CUDA release. If you want your server to remain on CUDA 13.1, install cuda-runtime-13-1 or cuda-toolkit-13-1 instead of the rolling metapackage.
NVIDIA’s installation guide says conflicting previous installations should be removed before a new install when the installation methods do not match. If you previously installed CUDA with a .run file, remove that first rather than layering APT packages over it.
NVIDIA’s post-install environment section makes PATH the key requirement and specifically calls out LD_LIBRARY_PATH for runfile-based installs. On standard Ubuntu package-manager installs, keeping the environment minimal reduces confusion and avoids stale path overrides later.
NVIDIA’s Ubuntu installation flow for both the driver and CUDA toolkit includes rebooting the system after installation. On remote servers, plan for that maintenance window before you start.

Copyright Servers99. All Rights Reserved