Object Detection and Segmentation in Computer Vision

Lessons from a full day with SAM 2 on satellite imagery. First off, what is SAM 2? It’s a zero‑shot, promptable segmentation model, meaning it can segment unseen objects out-of-the-box, without any training on those classes using only simple prompts like clicks, boxes, or text descriptions (what I used) to guide the process. Why apply it to satellite imagery? SAM 2 excels at segmenting environmental features (ex. roads, buildings, orchards) without retraining. My top tips? 🛰️ Use high‑res imagery (30 cm–1 m/pixel) for crisp segmentation especially for small objects. 🍃 Adjust prompts for the overhead view (e.g., "green leaves" or "shrubs" instead of "trees" - I even used "grey boxes" to find air conditioning units on top of buildings) 🚗 Small objects are detectable with careful prompting, even counting cars works. At Wherobots we embed SAM 2 into our raster inference engine. Users write simple SQL/Python prompts with text, inference runs in parallel on tiles, and results are stored as Iceberg tables in S3. From there, you can use the vector objects that are returned just like regular geospatial data with no special modeling needed. SAM 2 brings zero‑shot segmentation to geospatial data and when you combine it with prompt tuning, high‑res imagery, and distributed inference, and you can pull out earth scale insights in a day. Would love to hear your experiences with vision models on remote sensing! 🌎 I'm Matt and I talk about modern GIS, geospatial data engineering, and AI and geospatial is changing. 📬 Want more like this? Join 7k+ others learning from my newsletter → forrest.nyc

Forget flat photos—SAM3D is rewriting how machines understand the world. In this episode, we break down the groundbreaking new model that takes the core ideas of Meta’s Segment Anything Model and expands them into the third dimension, enabling instant 3D segmentation from just a single image. We start with the limitations of traditional 2D vision systems and explain why 3D understanding has always been one of the hardest problems in computer vision. Then we unpack the SAM3D architecture in simple terms: its depth-aware encoder, its multi-plane representation, and how it learns to infer 3D structure even when parts of an object are hidden. You’ll hear real examples—from mugs to human hands to complex indoor scenes—demonstrating how SAM3D reasons about surfaces, occlusions, and geometry with surprising accuracy. We also discuss its training pipeline, what makes it generalize so well, and why this technology could power the next generation of AR/VR, robotics, and spatial AI applications. If you want a beginner-friendly but technically insightful overview of why SAM3D is such a massive leap forward—and what it means for the future of AI—this episode is for you.   Resources:  SAM3D Website https://ai.meta.com/sam3d/ SAM3D Github https://lnkd.in/g9Snnh4i https://lnkd.in/gEwvPVJc SAM3D Demo https://lnkd.in/gkvxYKic SAM3D Paper https://lnkd.in/gv-5zvmH Need help building computer vision and AI solutions? https://bigvision.ai Start a career in computer vision and AI https://lnkd.in/gwi4kP2M

Summary DINOv3 is a new AI model for image understanding, known as a "vision foundation model." It uses self-supervised learning (SSL) to train on a massive 1.6-billion-image dataset without human labels, learning patterns like a person observes the world. Researchers addressed a trade-off where longer training improved high-level understanding but degraded pixel-level detail. They introduced "Gram anchoring," a technique that preserves spatial detail during training, making DINOv3 excel in both high-level recognition and fine detail. It achieves state-of-the-art results in tasks like object detection and depth estimation, making it a versatile tool for computer vision applications. Methodology DINOv3 builds on four pillars: data scaling, a new training objective, and post-training refinement. It uses a 1.689-billion-image dataset (LVD-1689M), including ~10% ImageNet. A Vision Transformer (ViT) model with 7 billion parameters was trained for 1 million steps using DINOv2 objectives. Gram Anchoring, the core innovation, prevents feature degradation by comparing the model's Gram matrix with a "Gram teacher" checkpoint. Post-training includes resolution scaling, distillation to smaller ViT models, and text alignment with a separate text encoder for open-vocabulary understanding. Results and Discussion DINOv3 sets new state-of-the-art (SOTA) benchmarks in visual representation. It achieves exceptional dense features, outperforming DINOv2 (49.5 mIoU) and SigLIP 2 (42.7 mIoU) with 55.9 mIoU on ADE20k segmentation, and leads in 3D keypoint matching, video tracking, and unsupervised object discovery. For the first time, it matches text-supervised models like PEcore and SigLIP 2 in global image classification while setting SOTA in instance retrieval. As a "frozen backbone," DINOv3 achieves SOTA in object detection and semantic segmentation, even with a lightweight 100M-parameter head and no fine-tuning. Its domain versatility is shown by training on 493 million satellite images, achieving SOTA in geospatial tasks. Implications of the Study DINOv3 demonstrates that self-supervised learning (SSL) can surpass traditional supervised and weakly-supervised methods. It supports the vision of a "one backbone" model, handling tasks like object detection, segmentation, depth estimation, and 3D understanding with a single frozen model. "Gram anchoring" resolves the global-vs-dense trade-off, enabling larger SSL models (10B+ parameters) without feature loss. The method also supports training in specialized domains like medical imaging without labeled data. Model distillation further makes this technology accessible to developers without requiring supercomputers.

Excited to share my in-depth #YouTube tutorial on object-level segmentation using #Detectron2 and #YOLO v8! We explore a public dataset of Nuclei from human and mouse organs, covering every step of the project: 1. Data download from Kaggle 2. Data cleanup 3. Conversion of masks to COCO JSON and YOLOv8 annotations 4. Visualization of annotations 5. Training Detectron2 (Mask R-CNN) for object detection 6. Training YOLOv8 for object detection 7. Image segmentation, object parameter calculation, and result plotting. Each task comes with downloadable code. Check out the tutorial https://lnkd.in/gEqxQCtp #bioimageanalysis #microscopy #digitalpathology #segmentation #deeplearning #computervision

I did a small hands-on comparison between YOLOv26 (Ultralytics) and Detectron2 (Meta AI) on the same video to see how they behave in a real-world scenario From my experience: ⚡ YOLOv26 (Ultralytics) It feels much faster and more stable for video processing. Bounding boxes are solid and confidence scores are high, making it a great choice for real-time applications and deployment. 🧠 Detectron2 (Meta AI) Still a very powerful framework, especially for research, segmentation, and detailed analysis. However, it’s heavier and slower, so it’s not always ideal for real-time pipelines. My takeaway: If speed and production deployment matter, YOLOv26 is the better option If you need flexibility, advanced segmentation, or research-level control, Detectron2 remains a strong tool At the end of the day, it’s all about choosing the right model for the right use case. #ComputerVision #YOLO #Detectron2 #Ultralytics #MetaAI #AI #DeepLearning #ObjectDetection #VideoAnalytics

Check out our newest paper about using #Computer #Vision for Root #Nodules Quantification. We compared Rule-based computer vision, #YOLOv12-seg transfer learning, and #SAM zero-shot segmentation for #detecting and #evaluating fluorescently labeled rhizobial nodules. This work demonstrates that classical rule-based methods can achieve strong performance when fluorescence provides clear chromatic separation, offering an interpretable and computationally efficient baseline. Supervised deep learning approaches, particularly intermediate-capacity models such as YOLOv12-m, provided the most balanced trade-off between segmentation accuracy, counting reliability, and computational feasibility, while SAM delivered stable but systematically biased underestimation and lacked intrinsic class discrimination. Thank you, Mohamed Salem, for your hard work, and to all co-authors for your significant contributions. Igathinathane Cannayen, Amanda Pease, Chandan Gautam, and Barney A. Geddes. You can access the full article here: https://lnkd.in/e6wZGAEp North Dakota State University NDSU Agriculture Rabia Lab #NewPublication #AI #Farming #AgTech #PrecisionAgriculture #ArtificialIntelligence

#CVPR2024 Talk by Matteo Sodano About His New Approach to Open-World Semantic Segmentation. The work tackles the problem of dealing with objects you have never seen during training. It proposes a novel approach that performs accurate closed-world semantic segmentation and can simultaneously identify new categories without requiring any additional training data. The approach additionally provides a similarity measure for every newly discovered class in an image to a known category, which can be useful information in downstream tasks such as planning or mapping.  VIDEO: https://lnkd.in/gnFjBJtV PAPER: https://lnkd.in/ek4TTYK3 CODE: https://lnkd.in/gCRxJzuK Matteo Sodano, Federico Magistri, Lucas Nunes, Jens Behley, and Cyrill Stachniss, “Open-World Semantic Segmentation Including Class Similarity,” in Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2024. #ComputerVision #Segmentation #OpenWorld #CVPR #StachnissLab #CenterForRoboticsBonn #UniBonn