OpenCV C++ in Flutter using Isolate for background processing 

My brain is now fried, I like it. This is Gold. Your explanation is excellelt, I learned quite a lot.

Thank you for helping me setup flutter with opencv. I was struggling a lot to figure it out but it's also my first day using flutter 😅

Thank you! This is great work! I just heard about ffi before, and after listening to your explanation, I think nothing is impossible. Thank you so much,

🎯 Key Takeaways for quick navigation: 00:00 🔍 *视频概述* - 本视频介绍了如何在Flutter应用中使用OpenCV的C++版本进行实时相机帧处理。主要包括使用已在Windows上运行的OpenCV代码、在Flutter应用中使用自定义的ArUco检测器以及配置相机帧处理等内容。 01:09 📷 *检测器页面设置* - 设置相机控制器和ArUco检测器以读取相机帧。 - 初始化相机控制器并注册用于接收预览相机帧的回调函数。 02:04 🖼️ *ArUco检测器配置* - ArUco检测器类包含检测方法，处理相机图像并返回ArUco标记坐标。 - 使用自定义绘图器在屏幕上绘制ArUco标记。 04:27 🛠️ *创建OpenCV插件* - 创建Flutter插件以集成OpenCV代码。 - 配置iOS和Android项目以包含OpenCV库和C++代码。 07:04 ⚙️ *配置Android集成* - 在Android项目中配置CMake和Gradle以构建和集成OpenCV库。 - 确保环境变量和Gradle设置正确以支持OpenCV集成。 11:12 📱 *配置Flutter端* - 使用Dart FFI加载和调用Native方法。 - 在Flutter应用中集成OpenCV插件，并调用Native方法进行版本检查。 12:59 🔌 *实现OpenCV插件功能* - 在Flutter端实现与Native端的通信，以调用OpenCV相关函数。 - 通过Dart FFI加载Native库，并将方法暴露给Flutter应用。 17:29 🚀 *测试集成功能* - 在Flutter应用中测试OpenCV插件的功能，例如检查OpenCV版本。 - 通过点击按钮触发Flutter端对Native端方法的调用，并验证结果。 21:40 💻 *初始化检测器方法和检测方法的参数说明* - 讲解了初始化检测器方法和检测方法的参数及其作用。 - 参数包括图像的字节数组、宽度、高度、旋转以及格式等。 - 方法的主要作用是根据传入的参数进行图像处理和检测。 25:32 🛠️ *将 C++ 代码与 Flutter 插件集成* - 演示了如何将 C++ 代码与 Flutter 插件集成，实现跨平台开发。 - 展示了在 Android Studio 中编写和构建 C++ 代码的过程。 - 讲解了在 Flutter 插件中定义方法签名、查找函数并将其暴露给 Flutter 应用的步骤。 36:48 📱 *在 Flutter 应用中使用 OpenCV 插件* - 展示了在 Flutter 应用中使用 OpenCV 插件进行实时相机帧处理和 ArUco 标记检测的过程。 - 演示了从嵌入资产中读取标记图像、调用初始化和检测方法的步骤。 - 说明了如何将相机帧坐标转换为屏幕坐标，并将检测到的 ArUco 标记绘制在屏幕上。 42:35 📱 *检测相机预览和本地标记* - 展示如何在应用中打开相机并检测本地标记。 - 介绍了相机预览和检测层的渲染过程。 - 提到了在 Flutter 中调用 OpenCV 可能会阻塞 UI 线程的问题，因此需要在后台线程进行处理。 44:03 🔄 *在后台线程执行检测代码* - 讨论了在 Flutter 中使用 Isolate 在后台线程执行检测代码的必要性。 - 介绍了 Isolate 的概念，它是一个在单独线程上运行的事件循环。 - 解释了如何在主 UI Isolate 和新创建的 Isolate 之间进行通信，并提供了示意图来说明通信过程。 47:05 🛠️ *将检测器移动到 Isolate 中* - 演示了将检测器类移动到新创建的 Isolate 中的过程。 - 介绍了如何在 Isolate 中初始化检测器，并与主线程进行通信。 - 讨论了如何在主线程中发送请求消息到 Isolate 并处理来自 Isolate 的响应消息。 53:31 🔄 *使用异步检测器处理检测任务* - 展示了如何使用异步检测器类处理检测任务，以避免在 UI 线程上阻塞。 - 讨论了在处理异步任务时需要注意的异步函数和回调。 - 介绍了如何在异步检测器中处理来自 Isolate 的响应消息，并在主线程中使用异步方法等待检测结果。 01:02:47 🧹 *清理资源并测试应用* - 在 `dispose` 方法中释放相机控制器和调用 `destroy` 方法来释放内存和终止 Isolate。 - 进行应用测试，确保资源清理工作正常，包括多次打开和关闭检测页面。 - 讨论了在 iOS 应用中配置 OpenCV 和编译 Aruco 检测器的步骤。 01:04:10 📱 *更新 iOS 应用以支持 OpenCV* - 更新了原生 OpenCV 插件的 Podspec 文件，添加了 OpenCV 的依赖项并将最低 iOS 版本设置为 11。 - 创建并添加了模块映射文件，用于 iOS 项目的编译。 - 更新了 Flutter 应用的 Podfile，并进行了 Pod 安装，然后在 Xcode 中打开项目并配置相机权限。 Made with HARPA AI

This is just amazing. Weeks of work saved. Thanks a lot! :)

Great work. It was a mess setting it up before. Your workflow was perfekt

it is fire and your explanation is legendary thx

Thank you very much! I really appreciate this tutorial.

you are a lifesaver man. thanks a lot :)

Your video is really helpful!😀

my brain is cooked 😂 I did flutter + aruco videos on binge-mode. I was looking at Kivy-zbarcam for a starter wireup but this already includes aruco - outstanding.

This is a great video. Could you do a video or article on how you made the flutter plugin? The video when you integrated OPENCV into android is great, though I can't really grasp how to relate to it in a flutter plugin setting.

Hi, Thank you for your great tutorial! I have 1 question. When I run at the xcode in release mode, then It works well. BUT when i did archive and test at testflight, the camera couldn't recognize image. Do you know What's wrong? (I will change cpp file when i deploy my testing app so don't worry about anything. I just curious about possibility of dynamic library with ios)

Hi, thank you for your amazing work, these tutorials are super helpful. I've got a question : do you think it is possible to feed the camera preview with a modified image that would be provided by the OpenCV plugin ?

Hi Yuval Thanks for the excellent explanation is there a way to debug the c side of the plugin in the flutter app ? can you share resources for this Regards,

Did you make the aruco detector as a dll and use it using fft? Is the opencv function included in the dll? Or did you use the opencv android version separately?

i tried to download from github but app not working i dont understand what should i change in order for it to work and edit it

Can this project able to help to measure other item measurement by using the aruco marker as reference size?

How would this change if targeting Windows? Or in general, using ffi/ffigen to access peripherals connected to the desktop (usb, bluetooth, etc )? For example, I have both 32-bit and 64-bit dlls that can access different hardware.. can flutter build/link with those dlls, and more importantly work when the dll functions are called? Any tips on 'Integrating C library in a desktop Flutter app using Dart FFI'?

Thank you for your amazing work, I need to count objects in real time using an android device, I would like to know if it is possible to count objects with opencv . Do you know some API for this problem? On the other hand, is there another way to count objects in real time (and offline) using an android device? Thank you in advance.

i have a small doubt i guess in the detect function of native_opencv.cpp file there is a code like follows Mat frame; if (isYUV) { Mat myyuv(height + height / 2, width, CV_8UC1, bytes); cvtColor(myyuv, frame, COLOR_YUV2BGRA_NV21); } else { frame = Mat(height, width, CV_8UC4, bytes); } what my doubt is that when it is in YUV format why we have to set height to ```height + height / 2``` to convert it to a Mat datatype ?

Hi, thanks for the tutorial ! I need some help. In native_opencv.cpp the ___attribute___ part triggers the error "C++ requires a type specifier for all declarations". This makes the flutter build crash. Do you have any idea how to resolve this issue ? Edit: I am on Windows btw

hello, thanks for your amazing tutorials and would like to use this project template to build a foot and mouth disease in cattle for my Bachelor ..and would like to be consulting you. how can i reach you?

Thank you for your amazing tutorial, but I need some help. Getting this error "Execution failed for task ':native_opencv:externalNativeBuildDebug'."

how do you get that terminal window? Also, I don't get the second android folder when opening the android project under example in android studio. How do i fix this?

How to get msys terminal in windows and I am not able to get files under cpp folder in android studio.

When I try to open the native_opencv/example/android folder in android studio. In the project view the group module with native_opencv and app is missing. How do I get them to show up in the project view?

Can you make face recognition? like auth account is same or not, please make in dart or flutter project with open cv

How to reduce the capacity of the app apk? (~312MB :

nice