[root@localhost ~]# lspci | grep -i ether
05:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (2*25GE) (rev 45)
06:00.0 Ethernet controller: Huawei Technologies Co., Ltd. Hi1822 Family (2*25GE) (rev 45)
7d:00.0 Ethernet controller: Huawei Technologies Co., Ltd. HNS GE/10GE/25GE RDMA Network Controller (rev 21
 
)
7d:00.1 Ethernet controller: Huawei Technologies Co., Ltd. HNS GE/10GE/25GE Network Controller (rev 21)
7d:00.2 Ethernet controller: Huawei Technologies Co., Ltd. HNS GE/10GE/25GE RDMA Network Controller (rev 21)
7d:00.3 Ethernet controller: Huawei Technologies Co., Ltd. HNS GE/10GE/25GE Network Controller (rev 21
 
)
[root@localhost ~]# ls /sys/bus/pci/devices/0000\:05\:00.0/re
remove     rescan     reset      resource   resource0  resource2  resource4  revision
[root@localhost ~]# cat /sys/bus/pci/devices/0000\:05\:00.0/resource
0x0000080007b00000 0x0000080007b1ffff 0x000000000014220c
0x0000000000000000 0x0000000000000000 0x0000000000000000

0x0000080008a20000 0x0000080008a27fff 0x000000000014220c
0x0000000000000000 0x0000000000000000 0x0000000000000000

0x0000080000200000 0x00000800002fffff 0x000000000014220c
0x0000000000000000 0x0000000000000000 0x0000000000000000
------------------------------------------------------------------
 

0x00000000e9200000 0x00000000e92fffff 0x0000000000046200
0x0000080007b20000 0x000008000829ffff 0x000000000014220c
0x0000000000000000 0x0000000000000000 0x0000000000000000
0x00000800082a0000 0x0000080008a1ffff 0x000000000014220c
0x0000000000000000 0x0000000000000000 0x0000000000000000
0x0000080000300000 0x0000080007afffff 0x000000000014220c
0x0000000000000000 0x0000000000000000 0x0000000000000000
[root@localhost ~]# 

PCI有6個BAR，6個BAR的不同劃分跟pci裝置設計有關，intel的網絡卡有Memory Bar、IO Bar還有MSI-X Bar

前 6 行為 PCI 裝置的 6 個 BAR，還是以 Intel 82599 為例，前兩個 BAR 為 Memory BAR，中間兩個 BAR 為 IO BAR，最後兩個 BAR 為 MSI-X BAR。其中，每個 BAR 又分為 3 列：

第 1 列為 PCI BAR 的起始地址
第 2 列為 PCI BAR 的終止地址
第 3 列為 PCI BAR 的標識

PCI Access Without a Driver

At work recently, I had a new PCI device that I needed to experiment with. I was dreading writing a Linux kernel driver to talk to it. It turns out, Linux makes it possible to read and write to a PCI device's memory space without a driver! Woohoo!

Linux provides asysfs interface to PCI devices. From that interface, the memory space can bemmaped and then read and written. No driver involved.

As a quick example, we can uselspcito get information about a particular device.

$ vendor="10ee" # Use your device ID
$ device="7014" # Use your vendor ID
$ lspci -d $vendor:$device -nvv
04:00.0 1180: 10ee:7014
    ...
    Region 0: Memory at f7300000 (32-bit, non-prefetchable) [size=128K]

Then we can look at the sysfs interface, at/sys/bus/pci/devices/. The first bit of data in the output oflspcigives the location of the device on the bus, that we can use when traversing the sysfs interface.

$ ls -alF /sys/bus/pci/devices/0000\:04\:00.0/
total 0
drwxr-xr-x 3 root root      0 Jul  1 12:42 ./
drwxr-xr-x 8 root root      0 Jul  1 12:42 ../
-rw-r--r-- 1 root root   4096 Jul  9 12:48 broken_parity_status
-r--r--r-- 1 root root   4096 Jul  1 12:42 class
-rw-r--r-- 1 root root   4096 Jul  9 12:44 config
-r--r--r-- 1 root root   4096 Jul  1 12:42 device
...
-r--r--r-- 1 root root   4096 Jul  1 12:43 resource
-rw------- 1 root root 131072 Jul  1 12:43 resource0
...
-r--r--r-- 1 root root   4096 Jul  1 12:42 vendor

This interface has some useful files likevendoranddevicethat confirm that we have the right device. These are also useful for programatically finding the correct device, rather than usinglspci.

$ cat /sys/bus/pci/devices/0000\:04\:00.0/vendor
0x10ee
$ cat /sys/bus/pci/devices/0000\:04\:00.0/device
0x7014

Looking back at thelspcioutput, we can also find memory resources and addresses. These are represented asresource0...resourceNin the sysfs interface. That's what we use to get access to the PCI memory space.

Open theresource0file (which can be some number other than 0 depending on the device).

int fd = open("/sys/bus/pci/devices/0000:04:00.0/resource0", O_RDWR | O_SYNC);

Then use the memory address and size from thelspcioutput tommapthe file.

void* base_address = (void*)0xf7300000;
size_t size = 128 * 1024; // 128K
void* void_memory = mmap(base_address,
                         size,
                         PROT_READ | PROT_WRITE,
                         MAP_SHARED,
                         fd,
                         0);
uint16_t* memory = (uint16_t*)void_memory;

Nowmemoryprovides direct access to read and write the PCI memory space. We can hack away!

// Read the value of the first register
uint16_t first_register = memory[0];

// Write a value to the third register
memory[2] = 0x0007;

Now, this isn't the perfect scenario. For one, we need to berootto access this memory space. For two, there's no sign of interrupt handling anywhere.

But for basic poking around on a new device, it works pretty slick. No kernel module development required.