Great video! I will use this info for my future video on my channel. I plan to put the realsense camera on two axis gimbal (head of the robot). Thank you!
Just got my first Jetson Nano and been playing with it all weekend (used up my whole long weekend!!). At first I was a little disappointed with the performance, but I don't have a proper power supply yet. So far only have a ZED Mini camera, which for my purposes I don't like that much. I tried to do some 3D mapping and I found it would run out of memory and start swapping like crazy. I was very disappointed. Seemed like this little thing would be useless for me and I'd be better off streaming over wifi or coughing up much more $$$$ for a Xavier. Then I decided to try adding a Samsung T5 USB SSD and put a huge 64GB swap file on there. Night and day difference, now it runs flawlessly. I'm happy with this little thing. Definitely one of the best deals around if you are into this 3D computer vision stuff. I was able to run the same 3D mapping apps on a smartphone under a Linux chroot, and the processor on my Note9 is faster, but the Note 9 cost me $1500, and the Nano less than $200. The only other possible alternative I can imagine is something like a Vivostick with a Coral USB TPU, but that thing has only 2GB of RAM, probably not even worth trying. Then again that SSD made a huge difference with the Nano.
It would have been nice for Intel to provide a more secure hard wired option for the camera, but that's nothing a little surgery can't fix. This camera seems to be a big step up from lidar. There could be an advantage to using both technologies at the same time for navigation and object detection, or maybe not. Could be an interesting weekend project to find out. I've been looking at the TI mmwave sensor boards and I think the Jetson would be a perfect platform to take advantage of that technology. If you're wanting to track multiple targets or a single target among many. The TI mmwave sensor board will do it without any optics. You might consider taking a look. Anyway, I enjoyed your video. I always find them refreshing and relaxing. I think it's because you're always relaxed. I'm always intensely focused when working on a project and wanting as much solitude as possible. Have a great night Jim.
Thank you for the kind words. It's not apparent, but it took several weeks to get the RealSense camera to work with the Nano. The videos tend to be after everything is working, so it's easier to be relaxed. I'll note that the cameras are really developer kits, similar to the Jetsons. There are OEM modules that people can integrate into their products. I'm interested in the mmWave stuff, but I'm not quite sure how easy it is to get it to work with the Jetson. Thanks for watching!
Interfacing the mmwave board might not be so much like working with legos, but persistence and confidence will take you a long way. I haven't made it work with the nano, but I'm confident there's away to do it. Make sure the words "I can't" aren't in your vocabulary. There is always away. It just has to be found. Thank you for your time Jim and for being so nice.
I just bought a Structure Core sensor. The depth performance on it looks amazing, but sadly for some reason I can't get decent framerates on the Jetson Nano. Depth only is at 30 FPS, but as soon as I add RGB the framerate drops to 9 FPS, and IR? Forget it. I see you are getting a full 60 FPS there with all streams enabled! What the heck. The Structure sensor works great on my high end laptop and desktop. 30 FPS with all streams enabled, and the depth performance looks very good, I think it looks better than what I see in this video. It seems I can't use it properly with the Nano though! That is what I bought this thing for, in fact I bought two!!! Bah. I'll have to convert them to other uses like smart security cameras or something, or get one of these cameras. Structure sensor looks superior quality wise though, but I'll have to find another portable device to connect it to. There has to be some kind of USB 3 bandwidth issue on the Jetson Nano.
Thanks for the suggestion. Mako Shark has been clamoring behind the scenes for 24/7 coverage, and I'm afraid that would just encourage him more. Thanks for watching!
Thank you for the kind words. The CUDA code is used for mapping purposes, color conversion, and image rectification of the RGB stream when displaying in 3D and so on. To compile with CUDA and the librealsense library, you also need to compile a newer version of CMake than the one in the Ubuntu repository (takes about another 30 minutes). In the case of the Nano, there's really not enough horsepower to do a real texture mapped point cloud (or at least I could not get better than a frame or two per second with the realsense-viewer application). If you are just using the depth map and color stream in a robotics application, for example, the CUDA code isn't that useful. For a faster machine, like a Jetson TX2 or Xavier, it makes sense to use the CUDA enhancements. Hope this makes sense, and thanks for watching!
Could you make a video which can connect laptop and nano wirelessly. For example I want to see the nano work as wifi hotspot and get the video streaming. The laptop can get the streaming.
Awesome videos as always! I have a question. As i started to patch Ubuntu I had a "Jetson Board Mismatch" telling me that the processor appear to be a Nvidia Jetson nano Dev Kit. Do you have a patch for this version or is there a way to make it work with the patch you already provided? Thank you !!!
Hi Jim, great videos. Please could you consider doing a video comparing the Zed camera with a T265+D435 setup? Looks like the Intel solution is not only cheaper, but potentially performs much better due to the hardware IMU processor of the T265 combined with infrared depth tracking on the D435. It would also be nice to see how this combination then compares with the D435i, which as far as I am aware, has the same IMU as the T265, but lacks the hardware processing (and has no pose stream). Thanks
Thank you for the kind words. I don't really do comparisons right now as there are so many different applications that people use these devices for that it's unfair to the various vendors to say "This one is better than another". It all depends on what the selection criteria is, and how it is being used. Also, there are several manufacturers in this space with competing products. These include Structure.IO with their Structure Core product, econ-systems makes a stereo vision camera, MYNT Eye too. Each has their advantages, disadvantages, indoor, outdoor, range, accuracy, FOV, and so on. Also, it depends on how you intend to use the Nano. If you are using it simply as a depth processor, or for occupancy mapping, or a tracking camera that's one thing. For example, the ZED uses the computing capabilities of a host to build its depth map. That may be good (you're using the Nano as a depth processor), or bad (you need as much processing as you can get so you can run neural nets/CUDA code). Like all engineering, there's tradeoffs and design considerations that you have to take into account in any project. It's a good idea to study for your component selection and fully understand your project needs. With that said, the Intel Tracking Camera T265 is not like the others in the group. It's difficult to find two global shutter cameras with fish eye lenses for $200, more or less with an integrated IMU and a vision processor that straight out gives a stream of poses. But obviously that's of little value if you are not on a mobile base of some sort. If the camera is stationary (for example, tracking people in a space), that's a completely different use case than on a mobile base (a robot mapping a room as an example). If you are trying to visualize a point cloud, that's another thing, and so on. The only thing I try to do here is share "How do you get this thing to work to begin with", and let others figure out how they would actually use them in their project.
@@JetsonHacks Perfectly reasonable! I wasn't even aware of the offerings from MYNT or Structure. I've emailed support from both and it looks like the MYNT is more suitable for my needs. Thanks for your detailed response.
Sure. You have to be a little careful if you are streaming all the cameras (that's 5 cameras + the depth stream and pose stream) as you start approaching the bandwidth limits of the USB 3 Hub. If you have other USB devices connected on the Jetson, things can get a little laggy. Thanks for watching!
@@EricRohlfs You should try it with the keyboard and mouse first, they don't use much bandwidth. If you are trying to use them with a USB drive at the same time, then you're probably over the limit.
@@JetsonHacks I understand.... I just not sure where to go as far as RealSense they only say they support ubuntu 16.04 yet we are running the Nano on 18.04. Only ROS Kinetic runs on 16.04...if I want 18.04 that means ROS Melodic...but will the RealSense ROS wrapper install on Melodic? Thought you might have a solution.
@@AliKutluozen I have a few problems that I am working on with the Nano...you need to be more specific. If you mean getting the T265 working, I have not gotten it to work on the Nano. I have gotten the T265 working on the TX2(using JetPack 4.2)...so I am hopeful...that the same process will work for the Nano. I need to be able to rebuild the Kernel...I am hoping Jim will soon give that to us...for the Nano...but I am currently doing that line by - line...similar to TX2.
@@sy2532 Hey, thanks for answering back! Here is my situation: I have Jetson Nano with Ubuntu 18.04, ROS Melodic installed... I also have a Realsense D435i that I have installed thanks to the tutorial, it works. Yet, when installing realsense-ros, ROS can't find the Intel SDK 2.0 ... So, here is the missing link; ROS and Realsense SDK.
Thank you for the kind words. I only do videos on products that I think are worthwhile. The RealSense Depth Camera is $200. I don't really do comparisons right now as there are so many different applications that people use these devices for that it's unfair to the various vendors to say "This one is better than another". It all depends on what the selection criteria is, and how it is being used. Also, there are several manufacturers in this space with competing products. These include Structure.IO with their Structure Core product, econ-systems makes a stereo vision camera, MYNT Eye too. Each has their advantages, disadvantages, indoor, outdoor, range, accuracy, FOV, and so on. Also, it depends on how you intend to use the Nano. If you are using it simply as a depth processor, or for occupancy mapping, or a tracking camera that's one thing. For example, the ZED uses the computing capabilities of a host to build its depth map. That may be good (you're using the Nano as a depth processor), or bad (you need as much processing as you can get so you can run neural nets/CUDA code). Like all engineering, there's tradeoffs and design considerations that you have to take into account in any project. It's a good idea to study for your component selection and fully understand your project needs. With that said, the Intel Tracking Camera T265 is not like the others in the group. It's difficult to find two global shutter cameras with fish eye lenses for $200, more or less with an integrated IMU and a vision processor that straight out gives a stream of poses. But obviously that's of little value if you are not on a mobile base of some sort. If the camera is stationary (for example, tracking people in a space), that's a completely different use case than on a mobile base (a robot mapping a room as an example). If you are trying to visualize a point cloud, that's another thing, and so on. The only thing I try to do here is share "How do you get this thing to work to begin with", and let others figure out how they would actually use them in their project.
@@JetsonHacks Sorry, missed the May 6th video on the T265. Both videos have the same title. T265 is tempting at 200$. After doing some research into Global vs Rolling Shutters I realised how a phone snaps a QR code so quickly. Thanks for a thoughful reply, good info!
I've been working on Jetson nano Developer Kit and i tried to follow the instructions of the video and when i get to the point to install patchUbuntu.sh, the result gave me Jetson Board Mismatch! I really need your help.
How can I change the settings from the infrared depth-finding camera to the RGB camera? I want to detect real-time objects, but my webcam is recognized as an infrared depth-finding camera. How can I make it recognize as an RGB camera? I am using realsense d435i camera.
Great video and install script! My realsense D435i is now running on my Nano, but the RGB overlay is not working. Which Firmware do you use? I tried already the last 3 versions without success. It shows for one frame the RGB image in 3D mode and then it is something between gray and green. In 2D mode everything works fine.
The firmware version on this camera is 05.10.13.00 You should lower the resolution to 848x800 @ 30fps on the cameras. The Jetson Nano doesn't have the horsepower to run the point clouds in the realsense-viewer application at higher resolutions. I don't know if this is related to the task or the application. Thanks for watching!
Even if this is 2 years old, I try to get a D435 running on Jetpack4.6 without much succhess. I was able to build everything from source and it seems to run fine. However, as soon as I start realsense-viewer or ROS wrapper for it, performace of jetson is incredible poor. I don't get more than 1pfs with CPU usage of 100% on all cores. I build it with CUDA support but GPU gets used only a bit. I powered everything with barrel jack and a beefy (200W, 5V) DC/DC converter. Any Ideas how I can improve things?
Intel updated their instructions for installing RealSense cameras on Jetson. Here's a video, please read the article in the description of that video. Make sure that you also upgrade the firmware in the camera. Also, make sure that the camera is running USB 3.2 which is shown in the RealSense Viewer application. Good luck!
Hey Jim. Could you help me here? I installed my D435i following the same steps and your script. I'm having this annoying issue: when I shutdown my nano and then turn it back on, D435i is recognized normally as intended. However, if I reboot instead turn off/on, it stops being recognized after reboot with a real long list of "unable to enumerate USB device". The same behavior applies to my desktop PC, not just for Nano, though.
Thank you so much! Question: I'm trying to run the rs-dnn.cpp example on the nano with the D435 camera - can this be done? I'm getting errors when it comes to using CMake
Sorry for the vagueness! It seems as though most of the demos from realsense are able to work on the nano - however the dnn demo does not appear in the demo folder and remains as a .cpp file in the librealsense/wrapper/opencv/dnn folder. I was wondering if this was typical on the nano? Thanks
@@christianealford8071 Not quite sure what "typical on the Nano" means. The OpenCV DNN is sample code which requires a properly configured development environment to build. There is a reliance on the version of OpenCV and the installation particulars. Looking at /librealsense/wrappers/opencv/dnn you can see a short explanation on how to build the sample. It feels like they make the assumption that the user has a good working knowledge of OpenCV and understands about how to use the DNN module. I haven't tried it, but it is certainly beyond the scope of what I could answer in a RU-vid comment. I would guess that the major issue is in configuring the dev environment. The typical answer to "getting errors when it comes to using CMake" is usually to fix the errors, but that presumes that you know what the actual errors you encounter might mean. It's not very forgiving or helpful.
Great work, I used this tutorial to install my real sense D345 on jetson nano. It worked well with the terminal. But when I launch with cheese, it doesn't show anything. Kindly help with a solution
Cheese requires a UVC backend for the librealsense drivers. You will either need to compile using the UVC flag, or you can use the branch 'easyInstall' which loads a compatible version from a librealsense repository. Thanks for watching!
@@JetsonHacks tried installing all over again and it generated an error. The jetson Nano hangs each time i launch the cheese or the real sense. How can I uninstall everything and so that I can reinstall all over?
@@olalekanisola8763 I do not know what you have done, it is difficult to advise you. You should reflash the SD card and start again. The UVC version is now the default version in the repository.
When running your scripts a bunch of errors are thrown during the make process. Do you have updated scripts for L4T 32.2.3. I get the following message when applying the patch: Jetson BSP Version: L4T R32.2.3 Jetson_L4T=32.2.3 ==== L4T Kernel Version Mismatch! ============= This repository is for modifying the kernel for a L4T 32.2.1 system. You are attempting to modify a L4T NVIDIA Jetson Nano Developer Kit system with L4T 32.2.3 The L4T releases must match! There may be versions in the tag/release sections that meet your needs Great video and the theme music still rings in my head ... ;-)
@@JetsonHacks Thanks for the insight. I stepped back to L4T 32.2.1 and brought up RealSense. After loading the application did recognize I had an outdated firmware version so I went on and updated it. So far, so good. Deeper into the rabbit hole I go.
