nvidia nano平台通过tca9546适配两路imx219 sensor

2023-05-11 18:00:34

1. 前言

nano平台原生是支持imx219驱动的；但是如果要扩展多路imx219需要修改硬件和设备树的；需要通过I2C一分四芯片，对四个imx219进行分时复用；这里暂时只调试两路imx219

2. CSI接口连接如下

MIPI-CSI接口使用如下:

` 1 -> Ground

2 -> CAM1_DN0 – Data Lane 0 3 -> CAM1_DP0 – Data Lane 0 4 -> Ground 5 -> CAM1_DN1 – Data Lane 1 6 -> CAM1_DP1 – Data Lane 1 7 -> Ground 8 -> CAM1_CN – MIPI Clock 9 -> CAM1_CP – MIPI Clock 10 -> Ground 11 -> CAM_GPIO 12 -> CAM_CLK 13 -> SCL0 – CAM_I2C 14 -> SDA0 – CAM_I2C 15 -> +3.3 V`

3. 修改设备树

参考其它dts `/i2c@3180000/tca9545@70/i2c@0/imx219 // for first camera

… /i2c@3180000/tca9545@70/i2c@1/imx219 // for second camera`

修改如下：

tegra-camera-platform { compatible = “nvidia, tegra-camera-platform”; status = “okay”; num_csi_lanes = <0x4>; max_lane_speed = <0x16e360>; min_bits_per_pixel = <0xa>; vi_peak_byte_per_pixel = <0x2>; vi_bw_margin_pct = <0x19>; max_pixel_rate = <0x3a980>; isp_peak_byte_per_pixel = <0x5>; isp_bw_margin_pct = <0x19>; modules { module0 { badge = “porg_front_RBPCV2”; position = “front”; orientation = [31 00]; drivernode0 { pcl_id = “v4l2_sensor”; devname = “imx219 30-0010”; proc-device-tree = “/proc/device-tree/host1x/i2c@546c0000/tca9546@70/i2c@0/rbpcv2_imx219_a@10”; }; drivernode1 { pcl_id = “v4l2_lens”; proc-device-tree = “/proc/device-tree/lens_imx219@RBPCV2/”; }; }; module1 { badge = “porg_rear_RBPCV2”; position = “rear”; orientation = [31 00]; drivernode0 { pcl_id = “v4l2_sensor”; devname = “imx219 31-0010”; proc-device-tree = “/proc/device-tree/host1x/i2c@546c0000/tca9546@70/i2c@1/rbpcv2_imx219_f@10”; }; drivernode1 { pcl_id = “v4l2_lens”; proc-device-tree = “/proc/device-tree/lens_imx219@RBPCV2/”; }; }; }; };

修改完后，驱动加载log如下：

[ 3.468532] i2c /dev entries driver [ 3.470026] pca954x 6-0070: vcc-pullup regulator not found [ 3.470043] pca954x 6-0070: pca954x_probe: forcing device bus number, start 30. [ 3.470047] pca954x 6-0070: device detect skipped. [ 3.470596] i2c i2c-6: Added multiplexed i2c bus 30 [ 3.470947] i2c i2c-6: Added multiplexed i2c bus 31 [ 3.471283] i2c i2c-6: Added multiplexed i2c bus 32 [ 3.471545] i2c i2c-6: Added multiplexed i2c bus 33 [ 3.471551] pca954x 6-0070: registered 4 multiplexed busses for I2C switch pca9546 [ 3.472267] imx219 30-0010: tegracam sensor driver:imx219_v2.0.6 [ 3.498487] imx219 31-0010: tegracam sensor driver:imx219_v2.0.6

dev下成功生成两路videox设备

4. 查看节点信息

v4l2-ctl –list-devices

vi-output, imx219 30-0010 (platform:54080000.vi:0): /dev/video0 vi-output, imx219 31-0010 (platform:54080000.vi:5): /dev/video1

media-ctl -p -d /dev/media0

Media controller API version 0.1.0 Media device information ———————— driver vi model NVIDIA Tegra Video Input Device serial bus info hw revision 0x3 driver version 0.0.0 Device topology – entity 1: nvcsi–2 (2 pads, 2 links) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev0 pad0: Sink <- “imx219 30-0010”:0 [ENABLED] pad1: Source -> “vi-output, imx219 30-0010”:0 [ENABLED] – entity 4: imx219 30-0010 (1 pad, 1 link) type V4L2 subdev subtype Sensor flags 0 device node name /dev/v4l-subdev1 pad0: Source [fmt:SRGGB10_1X10/1280×720 field:none colorspace:srgb] -> “nvcsi–2”:0 [ENABLED] – entity 6: vi-output, imx219 30-0010 (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video0 pad0: Sink <- “nvcsi–2”:1 [ENABLED] – entity 18: nvcsi–1 (2 pads, 2 links) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev2 pad0: Sink <- “imx219 31-0010”:0 [ENABLED] pad1: Source -> “vi-output, imx219 31-0010”:0 [ENABLED] – entity 21: imx219 31-0010 (1 pad, 1 link) type V4L2 subdev subtype Sensor flags 0 device node name /dev/v4l-subdev3 pad0: Source [fmt:SRGGB10_1X10/3280×2464 field:none colorspace:srgb] -> “nvcsi–1”:0 [ENABLED] – entity 23: vi-output, imx219 31-0010 (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video1 pad0: Sink <- “nvcsi–1”:1 [ENABLED]

5. 采集图像

cmd如下：

gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! ‘video/x-raw(memory:NVMM),width=1280, height=720, framerate=30/1, format=NV12’ ! nvvidconv flip-method=0 ! ‘video/x-raw,width=640, height=480, format=(string)UYVY’ ! nvvidconv ! nvegltransform ! nveglglessink -e Setting pipeline to PAUSED … Using winsys: x11 Pipeline is live and does not need PREROLL … Got context from element eglglessink0: gst.egl.EGLDisplay=context, display=(GstEGLDisplay)NULL; Setting pipeline to PLAYING … New clock: GstSystemClock GST_ARGUS: Creating output stream CONSUMER: Waiting until producer is connected… GST_ARGUS: Available Sensor modes : GST_ARGUS: 3280 x 2464 FR = 21,000000 fps Duration = 47619048 ; Analog Gain range min 1,000000, max 10,625000; Exposure Range min 13000, max 683709000; GST_ARGUS: 3280 x 1848 FR = 28,000001 fps Duration = 35714284 ; Analog Gain range min 1,000000, max 10,625000; Exposure Range min 13000, max 683709000; GST_ARGUS: 1920 x 1080 FR = 29,999999 fps Duration = 33333334 ; Analog Gain range min 1,000000, max 10,625000; Exposure Range min 13000, max 683709000; GST_ARGUS: 1280 x 720 FR = 59,999999 fps Duration = 16666667 ; Analog Gain range min 1,000000, max 10,625000; Exposure Range min 13000, max 683709000; GST_ARGUS: 1280 x 720 FR = 120,000005 fps Duration = 8333333 ; Analog Gain range min 1,000000, max 10,625000; Exposure Range min 13000, max 683709000; GST_ARGUS: Running with following settings: Camera index = 0 Camera mode = 4 Output Stream W = 1280 H = 720 seconds to Run = 0 Frame Rate = 120,000005 GST_ARGUS: PowerService: requested_clock_Hz=6048000 GST_ARGUS: Setup Complete, Starting captures for 0 seconds GST_ARGUS: Starting repeat capture requests. CONSUMER: Producer has connected; continuing. ^Chandling interrupt. Interrupt: Stopping pipeline … EOS on shutdown enabled — Forcing EOS on the pipeline Waiting for EOS… Got EOS from element “pipeline0”. EOS received – stopping pipeline… Execution ended after 0:00:04.245766040 Setting pipeline to PAUSED … Setting pipeline to READY … GST_ARGUS: Cleaning up GST_ARGUS: PowerServiceHwVic::cleanupResources CONSUMER: Done Success GST_ARGUS: Done Success Setting pipeline to NULL … Freeing pipeline …

dmesg | grep imx219

[ 3.580410] imx219 30-0010: tegracam sensor driver:imx219_v2.0.6 [ 3.606567] imx219 31-0010: tegracam sensor driver:imx219_v2.0.6 [ 4.578752] vi 54080000.vi: subdev imx219 30-0010 bound [ 4.579471] vi 54080000.vi: subdev imx219 31-0010 bound [ 345.055162] imx219 30-0010: imx219_power_on: power on [ 345.118384] imx219 30-0010: imx219_set_gain: val: 71 (/16) [times], gain: 199 [ 345.118930] imx219 30-0010: imx219_set_exposure: val: 29999 [us], coarse_time: 1475 [lines] [ 345.120128] imx219 30-0010: imx219_set_frame_rate: val: 30000001e-6 [fps], frame_length: 1639 [lines] [ 345.177118] imx219 30-0010: imx219_power_off: power off [ 345.177236] imx219 31-0010: imx219_power_on: power on [ 345.294933] imx219 31-0010: imx219_power_off: power off [ 345.295038] imx219 30-0010: imx219_power_on: power on [ 345.318297] imx219 30-0010: imx219_power_off: power off [ 345.318499] imx219 31-0010: imx219_power_on: power on [ 345.341729] imx219 31-0010: imx219_power_off: power off [ 345.344452] imx219 30-0010: imx219_power_on: power on [ 345.367759] imx219 30-0010: imx219_power_off: power off [ 345.368009] imx219 31-0010: imx219_power_on: power on [ 345.391253] imx219 31-0010: imx219_power_off: power off [ 345.415826] imx219 31-0010: imx219_power_on: power on [ 345.439043] imx219 31-0010: imx219_power_off: power off [ 345.439288] imx219 30-0010: imx219_power_on: power on [ 345.462517] imx219 30-0010: imx219_power_off: power off [ 345.462697] imx219 30-0010: imx219_power_on: power on [ 345.485950] imx219 30-0010: imx219_power_off: power off [ 345.486170] imx219 30-0010: imx219_power_on: power on [ 345.509733] imx219 30-0010: imx219_power_off: power off [ 345.546095] imx219 31-0010: imx219_power_on: power on [ 345.569274] imx219 31-0010: imx219_power_off: power off [ 345.569441] imx219 30-0010: imx219_power_on: power on [ 345.592633] imx219 30-0010: imx219_power_off: power off [ 345.592875] imx219 30-0010: imx219_power_on: power on [ 345.616223] imx219 30-0010: imx219_power_off: power off [ 345.616647] imx219 30-0010: imx219_power_on: power on [ 345.640093] imx219 30-0010: imx219_power_off: power off [ 345.920997] imx219 30-0010: imx219_power_on: power on [ 345.944266] imx219 30-0010: imx219_set_frame_rate: val: 120000000e-6 [fps], frame_length: 409 [lines] [ 345.945262] imx219 30-0010: imx219_set_gain: val: 16 (/16) [times], gain: 0 [ 345.945736] imx219 30-0010: imx219_set_exposure: exposure limited by frame_length: 405 [lines] [ 345.945740] imx219 30-0010: imx219_set_exposure: val: 8333 [us], coarse_time: 405 [lines] [ 345.949261] imx219 30-0010: imx219_set_frame_rate: val: 30000001e-6 [fps], frame_length: 1639 [lines] [ 345.950324] imx219 30-0010: imx219_set_exposure: exposure limited by frame_length: 1635 [lines] [ 345.950329] imx219 30-0010: imx219_set_exposure: val: 33330 [us], coarse_time: 1635 [lines] [ 345.994944] imx219 30-0010: imx219_set_gain: val: 16 (/16) [times], gain: 0 [ 345.995417] imx219 30-0010: imx219_set_exposure: exposure limited by frame_length: 1635 [lines] [ 345.995422] imx219 30-0010: imx219_set_exposure: val: 33330 [us], coarse_time: 1635 [lines] [ 345.996363] imx219 30-0010: imx219_set_frame_rate: val: 30000001e-6 [fps], frame_length: 1639 [lines] [ 346.002546] imx219 30-0010: imx219_set_frame_rate: val: 120000000e-6 [fps], frame_length: 409 [lines] [ 346.012095] imx219 30-0010: imx219_set_frame_rate: val: 30000001e-6 [fps], frame_length: 1639 [lines] [ 346.240710] imx219 30-0010: imx219_set_gain: val: 20 (/16) [times], gain: 52 [ 346.275141] imx219 30-0010: imx219_set_gain: val: 24 (/16) [times], gain: 86 [ 346.309575] imx219 30-0010: imx219_set_gain: val: 27 (/16) [times], gain: 105 [ 346.343922] imx219 30-0010: imx219_set_gain: val: 30 (/16) [times], gain: 120 [ 346.344622] imx219 30-0010: imx219_set_exposure: exposure limited by frame_length: 1635 [lines] [ 346.344627] imx219 30-0010: imx219_set_exposure: val: 33329 [us], coarse_time: 1635 [lines] [ 346.378355] imx219 30-0010: imx219_set_exposure: exposure limited by frame_length: 1635 [lines] [ 346.378361] imx219 30-0010: imx219_set_exposure: val: 33330 [us], coarse_time: 1635 [lines] [ 346.412794] imx219 30-0010: imx219_set_gain: val: 31 (/16) [times], gain: 124 [ 346.413410] imx219 30-0010: imx219_set_exposure: exposure limited by frame_length: 1635 [lines] [ 346.413416] imx219 30-0010: imx219_set_exposure: val: 33329 [us], coarse_time: 1635 [lines] [ 346.447166] imx219 30-0010: imx219_set_gain: val: 32 (/16) [times], gain: 128 [ 346.447718] imx219 30-0010: imx219_set_exposure: exposure limited by frame_length: 1635 [lines] [ 346.447723] imx219 30-0010: imx219_set_exposure: val: 33330 [us], coarse_time: 1635 [lines] [ 346.481616] imx219 30-0010: imx219_set_gain: val: 34 (/16) [times], gain: 136 [ 346.516007] imx219 30-0010: imx219_set_gain: val: 37 (/16) [times], gain: 146 [ 346.550352] imx219 30-0010: imx219_set_gain: val: 39 (/16) [times], gain: 151 [ 346.584822] imx219 30-0010: imx219_set_gain: val: 41 (/16) [times], gain: 157 [ 346.619141] imx219 30-0010: imx219_set_gain: val: 43 (/16) [times], gain: 161 [ 346.653599] imx219 30-0010: imx219_set_gain: val: 44 (/16) [times], gain: 163 [ 346.654274] imx219 30-0010: imx219_set_exposure: exposure limited by frame_length: 1635 [lines] [ 346.654279] imx219 30-0010: imx219_set_exposure: val: 33329 [us], coarse_time: 1635 [lines] [ 346.687967] imx219 30-0010: imx219_set_gain: val: 45 (/16) [times], gain: 165 [ 346.688814] imx219 30-0010: imx219_set_exposure: exposure limited by frame_length: 1635 [lines] [ 346.688824] imx219 30-0010: imx219_set_exposure: val: 33330 [us], coarse_time: 1635 [lines] [ 346.722348] imx219 30-0010: imx219_set_gain: val: 47 (/16) [times], gain: 169 [ 346.722948] imx219 30-0010: imx219_set_exposure: exposure limited by frame_length: 1635 [lines] [ 346.722953] imx219 30-0010: imx219_set_exposure: val: 33329 [us], coarse_time: 1635 [lines] [ 346.756768] imx219 30-0010: imx219_set_gain: val: 48 (/16) [times], gain: 171 [ 346.757441] imx219 30-0010: imx219_set_exposure: exposure limited by frame_length: 1635 [lines] [ 346.757447] imx219 30-0010: imx219_set_exposure: val: 33330 [us], coarse_time: 1635 [lines] [ 346.791252] imx219 30-0010: imx219_set_gain: val: 50 (/16) [times], gain: 175 [ 346.791865] imx219 30-0010: imx219_set_exposure: exposure limited by frame_length: 1635 [lines] [ 346.791871] imx219 30-0010: imx219_set_exposure: val: 33329 [us], coarse_time: 1635 [lines] [ 346.825565] imx219 30-0010: imx219_set_gain: val: 56 (/16) [times], gain: 183 [ 346.826113] imx219 30-0010: imx219_set_exposure: val: 29999 [us], coarse_time: 1475 [lines] [ 346.859983] imx219 30-0010: imx219_set_gain: val: 58 (/16) [times], gain: 186 [ 346.894372] imx219 30-0010: imx219_set_gain: val: 59 (/16) [times], gain: 187 [ 346.928807] imx219 30-0010: imx219_set_gain: val: 60 (/16) [times], gain: 188 [ 346.997648] imx219 30-0010: imx219_set_gain: val: 62 (/16) [times], gain: 190 [ 347.032011] imx219 30-0010: imx219_set_gain: val: 64 (/16) [times], gain: 192 [ 347.066606] imx219 30-0010: imx219_set_gain: val: 65 (/16) [times], gain: 193 [ 347.100776] imx219 30-0010: imx219_set_gain: val: 67 (/16) [times], gain: 195 [ 347.203991] imx219 30-0010: imx219_set_gain: val: 68 (/16) [times], gain: 196 [ 347.272904] imx219 30-0010: imx219_set_gain: val: 70 (/16) [times], gain: 198 [ 349.457148] imx219 30-0010: imx219_power_off: power off

驱动运行起来还有点问题

6. 继续修改设备树

kernel_src\hardware\nvidia\platform\t210\porg\kernel-dts\porg-platforms\tegra210-camera-rbpcv2-dual-imx219.dtsi

… tegra-camera-platform { isp_bw_margin_pct = <0x19>; compatible = “nvidia, tegra-camera-platform”; max_pixel_rate = <0x3a980>; num_csi_lanes = <0x4>; vi_peak_byte_per_pixel = <0x2>; status = “okay”; max_lane_speed = <0x16e360>; vi_bw_margin_pct = <0x19>; min_bits_per_pixel = <0xa>; isp_peak_byte_per_pixel = <0x5>; modules { module0 { badge = “porg_front_RBPCV2”; position = “front”; orientation = [31 00]; drivernode1 { pcl_id = “v4l2_lens”; proc-device-tree = “/proc/device-tree/lens_imx219@RBPCV2/”; }; drivernode0 { devname = “imx219 30-0010”; pcl_id = “v4l2_sensor”; proc-device-tree = “/proc/device-tree/host1x/i2c@546c0000/tca9546@70/i2c@0/rbpcv2_imx219_a@10”; }; }; module1 { badge = “porg_rear_RBPCV2”; position = “rear”; orientation = [32 00]; drivernode1 { pcl_id = “v4l2_lens”; proc-device-tree = “/proc/device-tree/lens_imx219@RBPCV2/”; }; drivernode0 { devname = “imx219 31-0010”; pcl_id = “v4l2_sensor”; proc-device-tree = “/proc/device-tree/host1x/i2c@546c0000/tca9546@70/i2c@1/rbpcv2_imx219_f@10”; }; }; }; }; …

module0 { badge = “porg_center_RBPCV2”; position = “center”; orientation = [31 00]; drivernode0 { pcl_id = “v4l2_sensor”; devname = “imx219 30-0010”; proc-device-tree = “/proc/device-tree/host1x/i2c@546c0000/tca9546@70/i2c@0/rbpcv2_imx219_a@10”; }; drivernode1 { pcl_id = “v4l2_lens”; proc-device-tree = “/proc/device-tree/lens_imx219@RBPCV2/”; }; }; module1 { badge = “porg_top_RBPCV2”; position = “top”; orientation = [31 00]; drivernode0 { pcl_id = “v4l2_sensor”; devname = “imx219 31-0010”; proc-device-tree = “/proc/device-tree/host1x/i2c@546c0000/tca9546@70/i2c@1/rbpcv2_imx219_f@10”; }; drivernode1 { pcl_id = “v4l2_lens”; proc-device-tree = “/proc/device-tree/lens_imx219@RBPCV2/”; }; };

badge = “porg_center_RBPCV2”; position = “center”; badge = “porg_top_RBPCV2”; position = “top”; badge = “imx219_top_RBPCV2”; position = “top”; badge = “imx219_center_RBPCV2”; position = “center”; badge = “imx219_front_RBPCV2”; position = “front”; badge = “imx219_rear_RBPCV2”; position = “rear”;

按照上面修改后，图像采集正常

7. 原因分析

由于dts中的port节点，module中的节点有一定关联性；很容易混淆错乱，需要严格根据硬件特性以及 nvidia的规则来修改

项目实践 | 行人跟踪与摔倒检测报警

[python opencv 计算机视觉零基础到实战]二十二、给视频制作浮动的带感水印