Mellanox Technologies - www.mellanox.com
****************************************

MSTFLINT Package - Firmware Burning and Diagnostics Tools

=== Disclaimer ===

This is a pretty old version of the mstflint tools. They are only needed if you want to use a
Mellanox (now Nvidia) ConnectX-2 Infiniband PCIe card.

The newer ConnectX-3, 4, 5 cards shall use the official repo here: https://github.com/Mellanox/mstflint

The support of ConnectX-2 was dropped in mstflint version 4.0.0, see this changelog:
https://docs.nvidia.com/networking/display/mstflint4310/mstflint+release+notes+change+log+history

Original source archives of mstflint and other infiniband tools can be found here:
https://www.openfabrics.org/downloads/mstflint/

This repo contains the original sources from
https://www.openfabrics.org/downloads/mstflint/mstflint-3.8.0-1.27.gf3d39b6.tar.gz
plus some changes needed to compile with modern compilers which are much more strict
in error checking. There are no functional changes, only fixing compile errors.

Tested with debian bookworm and trixie to flash new firmwares and do some diagnostics.


=== Most documentation is for proprietary Mellanox tools and is outdated ===

To use the ConnectX-2 cards on debian bookworm and trixie no proprietary packages are needed.
You only need mstflint to update your firmware for running the communination no other tools
or init scripts are needed.

The information on many places in the internet is outdated and misleading as of 2025. The Linux kernel modules mlx4_* 
and the other debian tools directly support the ConnectX-2 card.

The following tools are needed on debian:

```bash
sudo apt update
sudo apt install -y rdma-core ibutils ibverbs-providers ibsim-utils infiniband-diags opensm ethtool iproute2 perftest
```

Always use the PCIe address instead of any /dev/... device nodes, e.g.
Get the PCI address:

```bash
sudo apt update
lspci -k -nn -d '15b3:*'
0c:00.0 Network controller [0280]: Mellanox Technologies MT25408A0-FCC-QI ConnectX, Dual Port 40Gb/s InfiniBand / 10GigE Adapter IC with PCIe 2.0 x8 5.0GT/s Interface [15b3:673c] (rev b0)
```

Get some info:

```bash
mstflint -d 0c:00.0 q full
mstflint -d 0c:00.0 hw query
```

I'm using Network Manager (nmtui, nmcli) to manage the ip settings.
When using infiniband mode, you should enable 'connected mode' which
greatly improves the iperf3 throughput for ip over infiniband, in my case from 3 to 12 Gbit/s.

Because the card only suppors PCIe Gen 2 a maximum of 30 GBit/s is physically possible by the hardware.
Using 'ib_write_bw' I can nearly reach the 30 GBit/s.

The following parameters could be tuned, but it seems they even slow down the communication`:

```bash
echo >/etc/modprobe.d/ib_ipoib.conf "options ib_ipoib send_queue_size=8192 recv_queue_size=8192"
rmmod  ib_ipoib ; sleep 3 && modprobe ib_ipoib
```

Check the current values:

```bash
cat /sys/module/ib_ipoib/parameters/recv_queue_size
cat /sys/module/ib_ipoib/parameters/send_queue_size
```

=== Information from original package (except debian package information)  =====

1) Overview
    This package contains a burning tool and diagnostic tools for Mellanox
    manufactured HCA/NIC cards. It also provides access to the relevant source
    code. Please see the file LICENSE for licensing details.
	This package is based on a subset of the Mellanox Firmware Tools (MFT) package.
	For a full documentation of the MFT package, please refer to the downloads page
	in Mellanox web site.

    ----------------------------------------------------------------------------
    NOTE:
    This burning tool should be used only with Mellanox-manufactured
    HCA/NIC cards. Using it with cards manufactured by other vendors 
    may be harmful  to  the cards (due to different configurations).
    Using the diagnostic tools is normally safe for all HCAs/NICs.
    ----------------------------------------------------------------------------
    
2) Package Contents
    a) mstflint source code
    b) mflash lib
       This lib provides low level Flash access through Mellanox HCAs.
    c) mtcr lib (implemented in mtcr.h file)
       This lib enables access to HCA hardware registers.
    d) mstregdump utility
       This utility dumps hardware registers from Mellanox hardware
       for later analysis by Mellanox.
    e) mstvpd
       This utility dumps the on-card VPD.
    f) mstmcra
       This debug utility reads/writes a to/from the device configuration register space.
    g) mstconfig
       This tool sets or queries non-volatile configurable options for Mellanox HCAs.

3) Installation
    a) Build the mstflint utility. This package is built using a standard
       autotools method.

       To build on debian bookworm or trixie you need to install some dependencies:
       > sudo apt-get update
       > sudo apt-get install -y automake gettext libtool clang clang-tools g++ make zlib1g-dev libibmad-dev libibumad-dev libssl-dev
	   
       Example: 
       > ./autogen.sh
       > ./configure
       > make
       > sudo make install

       - Run "configure --help" for custom configuration options.
       - Typically, root privileges are required to run "make install"

4) Hardware Access Device Names
   The tools in this package require a device name in the command
   line. The device name is the identifier of the target CA.
   This section describes the device name formats and the HW access flow.

   a) The devices can be accessed by their PCI ID as displayed by lspci
      (bus:dev.fn).
      Example:
      # List all Mellanox devices
      > /sbin/lspci -d 15b3:
        02:00.0 Ethernet controller: Mellanox Technologies MT25448 [ConnectX EN 10GigE, PCIe 2.0 2.5GT/s] (rev a0) 

      # Use mstflint tool to query the firmware on this device
      > mstflint -d 02:00.0 q

    b) When the IB driver (mlx4 or mthca) is loaded, the devices can be accessed
       by their IB device name.
       Example:
       # List the IB devices 
       > ibv_devinfo | grep hca_id
         hca_id: mlx4_0

       # Use mstvpd tool to dump the VPD of this device
       > mstvpd mlx4_0

    c) PCI configuration access
       In examples a and b above, the device is accessed via PCI Memory Mapping.
       The device can also be accessed by PCI configuration cycles.
       PCI configuration access is slower and less safe than memory access --
       use it only if methods a and b above do not work.

       To force configuration access, use device names in the following format:
       /proc/bus/pci/<bus>/<dev.fn>

       Example:
       # List all Mellanox devices
       > /sbin/lspci -d 15b3:
         02:00.0 Ethernet controller: Mellanox Technologies MT25448 [ConnectX EN 10GigE, PCIe 2.0 2.5GT/s] (rev a0) 
	   
       # Use mstregdump to dump HW registers, using PCI config cycles
       > mstregdump /proc/bus/pci/02/00.0 > crdump.log

       Note: Typically, you will need root privileges for hardware access

	d) Accessing a multi-function device:

	   In some configuration, the CA device identifies as a multi-function device on PCI. E.G.:
	   > /sbin/lspci -d 15b3:
	   07:00.0 Ethernet controller: Mellanox Technologies MT26468 [ConnectX EN 10GigE, PCIe 2.0 5GT/s] (rev b0)
	   07:00.1 Ethernet controller: Mellanox Technologies MT26468 [ConnectX EN 10GigE, PCIe 2.0 5GT/s] (rev b0)
	   07:00.2 Ethernet controller: Mellanox Technologies MT26468 [ConnectX EN 10GigE, PCIe 2.0 5GT/s] (rev b0)
	   07:00.3 Ethernet controller: Mellanox Technologies MT26468 [ConnectX EN 10GigE, PCIe 2.0 5GT/s] (rev b0)
	   07:00.4 Ethernet controller: Mellanox Technologies MT26468 [ConnectX EN 10GigE, PCIe 2.0 5GT/s] (rev b0)
	   07:00.5 Ethernet controller: Mellanox Technologies MT26468 [ConnectX EN 10GigE, PCIe 2.0 5GT/s] (rev b0)
	   07:00.6 Ethernet controller: Mellanox Technologies MT26468 [ConnectX EN 10GigE, PCIe 2.0 5GT/s] (rev b0)
	   07:00.7 Ethernet controller: Mellanox Technologies MT26468 [ConnectX EN 10GigE, PCIe 2.0 5GT/s] (rev b0)

	   These multiple "logical" devices are actually a single physical device, so firmware update or "physical"
	   diagnostics should be run only on one of the functions.

	   When the device driver is loaded, only the primary physical function of the device can be accessed.
	   In Linux that would typically be function 0. This function can be accessed using memory mapping, as
	   described in sub section a) above. E.G.:
	   > mstflint -d 07:00.0 q

	   When the device driver is not loaded, all the functions can be accessed using configuration cycles, as
	   described in sub section c) above. It is recommended to use function 0 for FW update or diagnostics, E.G.:
	   > mstflint -d /proc/bus/pci/07/00.0 q

5) Usage (mstflint):
       Read mstflint usage. Enter "./mstflint -h" for a short help message, or
       "./mstflint -hh" for a detailed help message.

       Obtaining firmware files:
       If you purchased your card from Mellanox Technologies, please use the
       Mellanox website (www.mellanox.com, under 'Firmware' downloads) to
       download the firmware for your card.
       If you purchased your card from a vendor other than Mellanox, get a
       specific firmware configuration (INI) file from your HCA card vendor and
       generate the binary image.
	
       Use mstflint to burn a device according to the burning instructions in 
       "mstflint -hh" and in Mellanox web site firmware page.

6) Usage (mstregdump):
       An internal register dump is displayed to the standard output.
       Please store it in a file for analysis by Mellanox.

       Example:
       > mstregdump mthca0 > dumpfile

7) Usage (mstvpd):
    A VPD dump is displayed to the standard output.
	A list of keywords to dump can be supplied after the -- flag
	to apply an output filter.

	Examples:
	> mstvpd mlx4_0
	ID: Hawk Dual Port
	PN: MNPH29C-XTR
	EC: X2
	SN: MT1001X00749
	V0: PCIe Gen2 x8
	V1: N/A
	YA: N/A
	RW:

    > mstvpd mlx4_0 -- PN ID
	PN: MNPH29C-XTR
	ID: Hawk Dual Port

8) Usage (mstconfig):
    Read mstconfig usage. Enter "mstconfig -h" for a help message.

9) Problem Reporting:
	Please collect the following information when reporting issues:

	uname -a
	cat /etc/issue
	cat /proc/bus/pci/devices
	mstflint -vv
	lspci
        mstflint -d 02:00.0 v
        mstflint -d 02:00.0 q
        mstvpd 02:00.0