feat: HSAP platform v2 — modular navigation, quality review, audit log, world model simulation
Major changes: - New frontend (platform/web/): Vite + React 18 + TypeScript + Tailwind - 4-module navigation: 数据送标 / 模型管理 / 车队管理 / 系统管理 - Data catalog with charts (DMS/ADAS/Lane 3-tab view) - Quality review workflow (标注质检): Good/Fine/Bad scoring with auto-advance - Audit enhancements: batch operations, rejection categories, Feishu notifications - Operation audit log (操作日志) - World model simulation studio (仿真工坊) - Dataset version management with snapshots and diff - ADAS 7-class dataset integration (138K images organized + compressed) - User management with Feishu integration and pagination - CRUD/search/filter on all pages, card layout redesign - PIL-optimized image overlay rendering - Auto-snapshot on build, in_review workflow stage - Removed embedded algorithm code (now in workspace)
This commit is contained in:
@@ -0,0 +1,305 @@
|
||||
---
|
||||
comments: true
|
||||
description: Learn how to use Albumentations with YOLO26 to enhance data augmentation, improve model performance, and streamline your computer vision projects.
|
||||
keywords: Albumentations, YOLO26, data augmentation, Ultralytics, computer vision, object detection, model training, image transformations, machine learning
|
||||
---
|
||||
|
||||
# Enhance Your Dataset to Train YOLO26 Using Albumentations
|
||||
|
||||
When you are building [computer vision models](../models/index.md), the quality and variety of your [training data](../datasets/index.md) can play a big role in how well your model performs. Albumentations offers a fast, flexible, and efficient way to apply a wide range of image transformations that can improve your model's ability to adapt to real-world scenarios. It easily integrates with [Ultralytics YOLO26](https://github.com/ultralytics/ultralytics) and can help you create robust datasets for [object detection](../tasks/detect.md), [segmentation](../tasks/segment.md), and [classification](../tasks/classify.md) tasks.
|
||||
|
||||
By using Albumentations, you can boost your YOLO26 training data with techniques like geometric transformations and color adjustments. In this article, we'll see how Albumentations can improve your [data augmentation](../guides/preprocessing_annotated_data.md) process and make your [YOLO26 projects](../solutions/index.md) even more impactful. Let's get started!
|
||||
|
||||
## Albumentations for Image Augmentation
|
||||
|
||||
[Albumentations](https://albumentations.ai/) is an open-source image augmentation library created in [June 2018](https://arxiv.org/pdf/1809.06839). It is designed to simplify and accelerate the image augmentation process in [computer vision](https://www.ultralytics.com/blog/exploring-image-processing-computer-vision-and-machine-vision). Created with [performance](https://www.ultralytics.com/blog/measuring-ai-performance-to-weigh-the-impact-of-your-innovations) and flexibility in mind, it supports many diverse augmentation techniques, ranging from simple transformations like rotations and flips to more complex adjustments like brightness and contrast changes. Albumentations helps developers generate rich, varied datasets for tasks like [image classification](https://www.youtube.com/watch?v=5BO0Il_YYAg), [object detection](https://www.youtube.com/watch?v=5ku7npMrW40&t=1s), and [segmentation](https://www.youtube.com/watch?v=o4Zd-IeMlSY).
|
||||
|
||||
You can use Albumentations to easily apply augmentations to images, [segmentation masks](https://www.ultralytics.com/glossary/image-segmentation), [bounding boxes](https://www.ultralytics.com/glossary/bounding-box), and [key points](../datasets/pose/index.md), and make sure that all elements of your dataset are transformed together. It works seamlessly with popular deep learning frameworks like [PyTorch](../integrations/torchscript.md) and [TensorFlow](../integrations/tensorboard.md), making it accessible for a wide range of projects.
|
||||
|
||||
Also, Albumentations is a great option for augmentation whether you're handling small datasets or large-scale [computer vision tasks](../tasks/index.md). It ensures fast and efficient processing, cutting down the time spent on data preparation. At the same time, it helps improve [model performance](../guides/yolo-performance-metrics.md), making your models more effective in real-world applications.
|
||||
|
||||
## Key Features of Albumentations
|
||||
|
||||
Albumentations offers many useful features that simplify complex image augmentations for a wide range of [computer vision applications](https://www.ultralytics.com/blog/exploring-how-the-applications-of-computer-vision-work). Here are some of the key features:
|
||||
|
||||
- **Wide Range of Transformations**: Albumentations offers over [70 different transformations](https://github.com/albumentations-team/albumentations?tab=readme-ov-file#list-of-augmentations), including geometric changes (e.g., rotation, flipping), color adjustments (e.g., brightness, contrast), and noise addition (e.g., Gaussian noise). Having multiple options enables the creation of highly diverse and robust training datasets.
|
||||
|
||||
<p align="center">
|
||||
<img width="100%" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/albumentations-augmentation.avif" alt="Albumentations augmentation examples">
|
||||
</p>
|
||||
|
||||
- **High Performance Optimization**: Built on OpenCV and NumPy, Albumentations uses advanced optimization techniques like SIMD (Single Instruction, Multiple Data), which processes multiple data points simultaneously to speed up processing. It handles large datasets quickly, making it one of the fastest options available for image augmentation.
|
||||
|
||||
- **Three Levels of Augmentation**: Albumentations supports three levels of augmentation: pixel-level transformations, spatial-level transformations, and mixing-level transformations. Pixel-level transformations only affect the input images without altering masks, bounding boxes, or key points. Meanwhile, both the image and its elements, like masks and bounding boxes, are transformed using spatial-level transformations. Furthermore, mixing-level transformations are a unique way to augment data as they combine multiple images into one.
|
||||
|
||||
|
||||
|
||||
- **[Benchmarking Results](https://albumentations.ai/docs/benchmarks/image-benchmarks/)**: When it comes to benchmarking, Albumentations consistently outperforms other libraries, especially with large datasets.
|
||||
|
||||
## Why Should You Use Albumentations for Your Vision AI Projects?
|
||||
|
||||
With respect to image augmentation, Albumentations stands out as a reliable tool for computer vision tasks. Here are a few key reasons why you should consider using it for your Vision AI projects:
|
||||
|
||||
- **Easy-to-Use API**: Albumentations provides a single, straightforward API for applying a wide range of augmentations to images, masks, bounding boxes, and keypoints. It's designed to adapt easily to different datasets, making [data preparation](../guides/data-collection-and-annotation.md) simpler and more efficient.
|
||||
|
||||
- **Rigorous Bug Testing**: Bugs in the augmentation pipeline can silently corrupt input data, often going unnoticed but ultimately degrading model performance. Albumentations addresses this with a thorough test suite that helps catch bugs early in development.
|
||||
|
||||
- **Extensibility**: Albumentations can be used to easily add new augmentations and use them in computer vision pipelines through a single interface along with built-in transformations.
|
||||
|
||||
## How to Use Albumentations to Augment Data for YOLO26 Training
|
||||
|
||||
Now that we've covered what Albumentations is and what it can do, let's look at how to use it to augment your data for YOLO26 model training. It's easy to set up because it integrates directly into [Ultralytics' training mode](../modes/train.md) and applies automatically if you have the Albumentations package installed.
|
||||
|
||||
### Installation
|
||||
|
||||
To use Albumentations with YOLO26, start by making sure you have the necessary packages installed. If Albumentations isn't installed, the augmentations won't be applied during training. Once set up, you'll be ready to create an augmented dataset for training, with Albumentations integrated to enhance your model automatically.
|
||||
|
||||
!!! tip "Installation"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Install the required packages
|
||||
pip install albumentations ultralytics
|
||||
```
|
||||
|
||||
For detailed instructions and best practices related to the installation process, check our [Ultralytics Installation guide](../quickstart.md). While installing the required packages for YOLO26, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips.
|
||||
|
||||
### Usage
|
||||
|
||||
After installing the necessary packages, you're ready to start using Albumentations with YOLO26. When you train YOLO26, a set of augmentations is automatically applied through its integration with Albumentations, making it easy to enhance your model's performance.
|
||||
|
||||
!!! example "Usage"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load a pretrained model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Train the model with default augmentations
|
||||
results = model.train(data="coco8.yaml", epochs=100, imgsz=640)
|
||||
```
|
||||
|
||||
=== "Custom Transforms (Python API only)"
|
||||
|
||||
```python
|
||||
import albumentations as A
|
||||
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load a pretrained model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Define custom Albumentations transforms
|
||||
custom_transforms = [
|
||||
A.Blur(blur_limit=7, p=0.5),
|
||||
A.GaussNoise(var_limit=(10.0, 50.0), p=0.3),
|
||||
A.CLAHE(clip_limit=4.0, p=0.5),
|
||||
A.RandomBrightnessContrast(brightness_limit=0.2, contrast_limit=0.2, p=0.5),
|
||||
]
|
||||
|
||||
# Train the model with custom Albumentations transforms
|
||||
results = model.train(
|
||||
data="coco8.yaml",
|
||||
epochs=100,
|
||||
imgsz=640,
|
||||
augmentations=custom_transforms, # Pass custom transforms
|
||||
)
|
||||
```
|
||||
|
||||
Next, let's take a closer look at the specific augmentations that are applied during training.
|
||||
|
||||
### Blur
|
||||
|
||||
The Blur transformation in Albumentations applies a simple blur effect to the image by averaging pixel values within a small square area, or kernel. This is done using OpenCV `cv2.blur` function, which helps reduce noise in the image, though it also slightly reduces image details.
|
||||
|
||||
Here are the parameters and values used in this integration:
|
||||
|
||||
- **blur_limit**: This controls the size range of the blur effect. The default range is (3, 7), meaning the kernel size for the blur can vary between 3 and 7 pixels, with only odd numbers allowed to keep the blur centered.
|
||||
|
||||
- **p**: The probability of applying the blur. In the integration, p=0.01, so there's a 1% chance that this blur will be applied to each image. The low probability allows for occasional blur effects, introducing a bit of variation to help the model generalize without over-blurring the images.
|
||||
|
||||
<img width="776" alt="Albumentations Blur augmentation result" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/albumentations-blur.avif">
|
||||
|
||||
### Median Blur
|
||||
|
||||
The MedianBlur transformation in Albumentations applies a median blur effect to the image, which is particularly useful for reducing noise while preserving edges. Unlike typical blurring methods, MedianBlur uses a median filter, which is especially effective at removing salt-and-pepper noise while maintaining sharpness around the edges.
|
||||
|
||||
Here are the parameters and values used in this integration:
|
||||
|
||||
- **blur_limit**: This parameter controls the maximum size of the blurring kernel. In this integration, it defaults to a range of (3, 7), meaning the kernel size for the blur is randomly chosen between 3 and 7 pixels, with only odd values allowed to ensure proper alignment.
|
||||
|
||||
- **p**: Sets the probability of applying the median blur. Here, p=0.01, so the transformation has a 1% chance of being applied to each image. This low probability ensures that the median blur is used sparingly, helping the model generalize by occasionally seeing images with reduced noise and preserved edges.
|
||||
|
||||
The image below shows an example of this augmentation applied to an image.
|
||||
|
||||
<img width="764" alt="Albumentations MedianBlur augmentation" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/albumentations-median-blur.avif">
|
||||
|
||||
### Grayscale
|
||||
|
||||
The ToGray transformation in Albumentations converts an image to grayscale, reducing it to a single-channel format and optionally replicating this channel to match a specified number of output channels. Different methods can be used to adjust how grayscale brightness is calculated, ranging from simple averaging to more advanced techniques for realistic perception of contrast and brightness.
|
||||
|
||||
Here are the parameters and values used in this integration:
|
||||
|
||||
- **num_output_channels**: Sets the number of channels in the output image. If this value is more than 1, the single grayscale channel will be replicated to create a multichannel grayscale image. By default, it's set to 3, giving a grayscale image with three identical channels.
|
||||
|
||||
- **method**: Defines the grayscale conversion method. The default method, "weighted_average", applies a formula (0.299R + 0.587G + 0.114B) that closely aligns with human perception, providing a natural-looking grayscale effect. Other options, like "from_lab", "desaturation", "average", "max", and "pca", offer alternative ways to create grayscale images based on various needs for speed, brightness emphasis, or detail preservation.
|
||||
|
||||
- **p**: Controls how often the grayscale transformation is applied. With p=0.01, there is a 1% chance of converting each image to grayscale, making it possible for a mix of color and grayscale images to help the model generalize better.
|
||||
|
||||
The image below shows an example of this grayscale transformation applied.
|
||||
|
||||
<img width="759" alt="Albumentations grayscale conversion" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/albumentations-grayscale.avif">
|
||||
|
||||
### Contrast Limited Adaptive Histogram Equalization (CLAHE)
|
||||
|
||||
The CLAHE transformation in Albumentations applies Contrast Limited Adaptive Histogram Equalization (CLAHE), a technique that enhances image contrast by equalizing the histogram in localized regions (tiles) instead of across the whole image. CLAHE produces a balanced enhancement effect, avoiding the overly amplified contrast that can result from standard histogram equalization, especially in areas with initially low contrast.
|
||||
|
||||
Here are the parameters and values used in this integration:
|
||||
|
||||
- **clip_limit**: Controls the contrast enhancement range. Set to a default range of (1, 4), it determines the maximum contrast allowed in each tile. Higher values are used for more contrast but may also introduce noise.
|
||||
|
||||
- **tile_grid_size**: Defines the size of the grid of tiles, typically as (rows, columns). The default value is (8, 8), meaning the image is divided into a 8x8 grid. Smaller tile sizes provide more localized adjustments, while larger ones create effects closer to global equalization.
|
||||
|
||||
- **p**: The probability of applying CLAHE. Here, p=0.01 introduces the enhancement effect only 1% of the time, ensuring that contrast adjustments are applied sparingly for occasional variation in training images.
|
||||
|
||||
The image below shows an example of the CLAHE transformation applied.
|
||||
|
||||
<img width="760" alt="Albumentations CLAHE contrast enhancement" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/albumentations-CLAHE.avif">
|
||||
|
||||
## Using Custom Albumentations Transforms
|
||||
|
||||
While the default Albumentations integration provides a solid set of augmentations, you may want to customize the transforms for your specific use case. With Ultralytics YOLO26, you can easily pass custom Albumentations transforms via the Python API using the `augmentations` parameter.
|
||||
|
||||
### How to Define Custom Transforms
|
||||
|
||||
You can define your own list of Albumentations transforms and pass them to the training function. This replaces the default Albumentations transforms while keeping all other YOLO augmentations (like `hsv_h`, `degrees`, `mosaic`, etc.) active.
|
||||
|
||||
Here's an example with more advanced transforms:
|
||||
|
||||
```python
|
||||
import albumentations as A
|
||||
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Define custom transforms with various augmentation techniques
|
||||
custom_transforms = [
|
||||
# Blur variations
|
||||
A.OneOf(
|
||||
[
|
||||
A.MotionBlur(blur_limit=7, p=1.0),
|
||||
A.MedianBlur(blur_limit=7, p=1.0),
|
||||
A.GaussianBlur(blur_limit=7, p=1.0),
|
||||
],
|
||||
p=0.3,
|
||||
),
|
||||
# Noise variations
|
||||
A.OneOf(
|
||||
[
|
||||
A.GaussNoise(var_limit=(10.0, 50.0), p=1.0),
|
||||
A.ISONoise(color_shift=(0.01, 0.05), intensity=(0.1, 0.5), p=1.0),
|
||||
],
|
||||
p=0.2,
|
||||
),
|
||||
# Color and contrast adjustments
|
||||
A.CLAHE(clip_limit=4.0, tile_grid_size=(8, 8), p=0.5),
|
||||
A.RandomBrightnessContrast(brightness_limit=0.3, contrast_limit=0.3, p=0.5),
|
||||
A.HueSaturationValue(hue_shift_limit=20, sat_shift_limit=30, val_shift_limit=20, p=0.5),
|
||||
# Simulate occlusions
|
||||
A.CoarseDropout(
|
||||
max_holes=8, max_height=32, max_width=32, min_holes=1, min_height=8, min_width=8, fill_value=0, p=0.2
|
||||
),
|
||||
]
|
||||
|
||||
# Train with custom transforms
|
||||
results = model.train(
|
||||
data="coco8.yaml",
|
||||
epochs=100,
|
||||
imgsz=640,
|
||||
augmentations=custom_transforms,
|
||||
)
|
||||
```
|
||||
|
||||
### Important Considerations
|
||||
|
||||
When using custom Albumentations transforms, keep these points in mind:
|
||||
|
||||
- **Python API Only**: Custom transforms can only be passed through the Python API, not via CLI or YAML configuration files.
|
||||
- **Replaces Defaults**: Your custom transforms will completely replace the default Albumentations transforms. Other YOLO augmentations remain active.
|
||||
- **Bounding Box Handling**: Ultralytics automatically handles bounding box adjustments for most transforms, but complex spatial transforms may require additional testing.
|
||||
- **Performance**: Some transforms are computationally expensive. Monitor training speed and adjust accordingly.
|
||||
- **Task Compatibility**: Custom Albumentations transforms work with detection and segmentation tasks but not with classification (which uses a different augmentation pipeline).
|
||||
|
||||
### Use Cases for Custom Transforms
|
||||
|
||||
Different applications benefit from different augmentation strategies:
|
||||
|
||||
- **Medical Imaging**: Use elastic deformations, grid distortions, and specialized noise patterns
|
||||
- **Aerial/Satellite Imagery**: Apply transforms that simulate different altitudes, weather conditions, and lighting angles
|
||||
- **Low-Light Scenarios**: Emphasize noise addition and brightness adjustments to train robust models for challenging lighting
|
||||
- **Industrial Inspection**: Add texture variations and simulated defects for quality control applications
|
||||
|
||||
For a complete list of available transforms and their parameters, visit the [Albumentations documentation](https://albumentations.ai/docs/).
|
||||
|
||||
For more detailed examples and best practices on using custom Albumentations transforms with YOLO26, see the [YOLO Data Augmentation guide](../guides/yolo-data-augmentation.md#custom-albumentations-transforms-augmentations).
|
||||
|
||||
## Keep Learning about Albumentations
|
||||
|
||||
If you are interested in learning more about Albumentations, check out the following resources for more in-depth instructions and examples:
|
||||
|
||||
- **[Albumentations Documentation](https://albumentations.ai/docs/)**: The official documentation provides a full range of supported transformations and advanced usage techniques.
|
||||
|
||||
- **[Ultralytics Albumentations Guide](https://docs.ultralytics.com/reference/data/augment/?h=albumentation#ultralytics.data.augment.Albumentations)**: Get a closer look at the details of the function that facilitate this integration.
|
||||
|
||||
- **[Albumentations GitHub Repository](https://github.com/albumentations-team/albumentations/)**: The repository includes examples, benchmarks, and discussions to help you get started with customizing augmentations.
|
||||
|
||||
## Key Takeaways
|
||||
|
||||
In this guide, we explored the key aspects of Albumentations, a great Python library for image augmentation. We discussed its wide range of transformations, optimized performance, and how you can use it in your next YOLO26 project.
|
||||
|
||||
Also, if you'd like to know more about other Ultralytics YOLO26 integrations, visit our [integration guide page](../integrations/index.md). You'll find valuable resources and insights there.
|
||||
|
||||
## FAQ
|
||||
|
||||
### How can I integrate Albumentations with YOLO26 for improved data augmentation?
|
||||
|
||||
Albumentations integrates seamlessly with YOLO26 and applies automatically during training if you have the package installed. Here's how to get started:
|
||||
|
||||
```python
|
||||
# Install required packages
|
||||
# !pip install albumentations ultralytics
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load and train model with automatic augmentations
|
||||
model = YOLO("yolo26n.pt")
|
||||
model.train(data="coco8.yaml", epochs=100)
|
||||
```
|
||||
|
||||
The integration includes optimized augmentations like blur, median blur, grayscale conversion, and CLAHE with carefully tuned probabilities to enhance model performance.
|
||||
|
||||
### What are the key benefits of using Albumentations over other augmentation libraries?
|
||||
|
||||
Albumentations stands out for several reasons:
|
||||
|
||||
1. Performance: Built on OpenCV and NumPy with SIMD optimization for superior speed
|
||||
2. Flexibility: Supports 70+ transformations across pixel-level, spatial-level, and mixing-level augmentations
|
||||
3. Compatibility: Works seamlessly with popular frameworks like [PyTorch](../integrations/torchscript.md) and [TensorFlow](../integrations/tensorboard.md)
|
||||
4. Reliability: Extensive test suite prevents silent data corruption
|
||||
5. Ease of use: Single unified API for all augmentation types
|
||||
|
||||
### What types of computer vision tasks can benefit from Albumentations augmentation?
|
||||
|
||||
Albumentations enhances various [computer vision tasks](../tasks/index.md) including:
|
||||
|
||||
- [Object Detection](../tasks/detect.md): Improves model robustness to lighting, scale, and orientation variations
|
||||
- [Instance Segmentation](../tasks/segment.md): Enhances mask prediction accuracy through diverse transformations
|
||||
- [Classification](../tasks/classify.md): Increases model generalization with color and geometric augmentations
|
||||
- [Pose Estimation](../tasks/pose.md): Helps models adapt to different viewpoints and lighting conditions
|
||||
|
||||
The library's diverse augmentation options make it valuable for any vision task requiring robust model performance.
|
||||
@@ -0,0 +1,256 @@
|
||||
---
|
||||
comments: true
|
||||
description: Learn step-by-step how to deploy Ultralytics' YOLO26 on Amazon SageMaker Endpoints, from setup to testing, for powerful real-time inference with AWS services.
|
||||
keywords: YOLO26, Amazon SageMaker, AWS, Ultralytics, machine learning, computer vision, model deployment, AWS CloudFormation, AWS CDK, real-time inference
|
||||
---
|
||||
|
||||
# A Guide to Deploying YOLO26 on Amazon SageMaker Endpoints
|
||||
|
||||
Deploying advanced [computer vision](https://www.ultralytics.com/glossary/computer-vision-cv) models like [Ultralytics' YOLO26](https://github.com/ultralytics/ultralytics) on Amazon SageMaker Endpoints opens up a wide range of possibilities for various [machine learning](https://www.ultralytics.com/glossary/machine-learning-ml) applications. The key to effectively using these models lies in understanding their setup, configuration, and deployment processes. YOLO26 becomes even more powerful when integrated seamlessly with Amazon SageMaker, a robust and scalable machine learning service by AWS.
|
||||
|
||||
This guide will take you through the process of deploying YOLO26 [PyTorch](https://www.ultralytics.com/glossary/pytorch) models on Amazon SageMaker Endpoints step by step. You'll learn the essentials of preparing your AWS environment, configuring the model appropriately, and using tools like AWS CloudFormation and the AWS Cloud Development Kit (CDK) for deployment.
|
||||
|
||||
## Amazon SageMaker
|
||||
|
||||
<p align="center">
|
||||
<img width="640" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/amazon-sagemaker-overview.avif" alt="Amazon SageMaker ML platform architecture">
|
||||
</p>
|
||||
|
||||
[Amazon SageMaker](https://aws.amazon.com/sagemaker/) is a machine learning service from Amazon Web Services (AWS) that simplifies the process of building, training, and deploying machine learning models. It provides a broad range of tools for handling various aspects of machine learning workflows. This includes automated features for tuning models, options for training models at scale, and straightforward methods for deploying models into production. SageMaker supports popular machine learning frameworks, offering the flexibility needed for diverse projects. Its features also cover data labeling, workflow management, and performance analysis.
|
||||
|
||||
## Deploying YOLO26 on Amazon SageMaker Endpoints
|
||||
|
||||
Deploying YOLO26 on Amazon SageMaker lets you use its managed environment for real-time inference and take advantage of features like autoscaling. Take a look at the AWS architecture below.
|
||||
|
||||
<p align="center">
|
||||
<img width="640" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/aws-architecture.avif" alt="AWS SageMaker YOLO training architecture">
|
||||
</p>
|
||||
|
||||
### Step 1: Setup Your AWS Environment
|
||||
|
||||
First, ensure you have the following prerequisites in place:
|
||||
|
||||
- An AWS Account: If you don't already have one, sign up for an AWS account.
|
||||
|
||||
- Configured IAM Roles: You'll need an IAM role with the necessary permissions for Amazon SageMaker, AWS CloudFormation, and Amazon S3. This role should have policies that allow it to access these services.
|
||||
|
||||
- AWS CLI: If not already installed, download and install the AWS Command Line Interface (CLI) and configure it with your account details. Follow [the AWS CLI instructions](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) for installation.
|
||||
|
||||
- AWS CDK: If not already installed, install the AWS Cloud Development Kit (CDK), which will be used for scripting the deployment. Follow [the AWS CDK instructions](https://docs.aws.amazon.com/cdk/v2/guide/#getting_started_install) for installation.
|
||||
|
||||
- Adequate Service Quota: Confirm that you have sufficient quotas for two separate resources in Amazon SageMaker: one for `ml.m5.4xlarge` for endpoint usage and another for `ml.m5.4xlarge` for notebook instance usage. Each of these requires a minimum of one quota value. If your current quotas are below this requirement, it's important to request an increase for each. You can request a quota increase by following the detailed instructions in the [AWS Service Quotas documentation](https://docs.aws.amazon.com/servicequotas/latest/userguide/request-quota-increase.html#quota-console-increase).
|
||||
|
||||
### Step 2: Clone the YOLO26 SageMaker Repository
|
||||
|
||||
The next step is to clone the specific AWS repository that contains the resources for deploying YOLO26 on SageMaker. This repository, hosted on GitHub, includes the necessary CDK scripts and configuration files.
|
||||
|
||||
- Clone the GitHub Repository: Execute the following command in your terminal to clone the host-yolov8-on-sagemaker-endpoint repository:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/aws-samples/host-yolov8-on-sagemaker-endpoint.git
|
||||
```
|
||||
|
||||
- Navigate to the Cloned Directory: Change your directory to the cloned repository:
|
||||
|
||||
```bash
|
||||
cd host-yolov8-on-sagemaker-endpoint/yolov8-pytorch-cdk
|
||||
```
|
||||
|
||||
### Step 3: Set Up the CDK Environment
|
||||
|
||||
Now that you have the necessary code, set up your environment for deploying with AWS CDK.
|
||||
|
||||
- Create a Python Virtual Environment: This isolates your Python environment and dependencies. Run:
|
||||
|
||||
```bash
|
||||
python3 -m venv .venv
|
||||
```
|
||||
|
||||
- Activate the Virtual Environment:
|
||||
|
||||
```bash
|
||||
source .venv/bin/activate
|
||||
```
|
||||
|
||||
- Install Dependencies: Install the required Python dependencies for the project:
|
||||
|
||||
```bash
|
||||
pip3 install -r requirements.txt
|
||||
```
|
||||
|
||||
- Upgrade AWS CDK Library: Ensure you have the latest version of the AWS CDK library:
|
||||
|
||||
```bash
|
||||
pip install --upgrade aws-cdk-lib
|
||||
```
|
||||
|
||||
### Step 4: Create the AWS CloudFormation Stack
|
||||
|
||||
- Synthesize the CDK Application: Generate the AWS CloudFormation template from your CDK code:
|
||||
|
||||
```bash
|
||||
cdk synth
|
||||
```
|
||||
|
||||
- Bootstrap the CDK Application: Prepare your AWS environment for CDK deployment:
|
||||
|
||||
```bash
|
||||
cdk bootstrap
|
||||
```
|
||||
|
||||
- Deploy the Stack: This will create the necessary AWS resources and deploy your model:
|
||||
|
||||
```bash
|
||||
cdk deploy
|
||||
```
|
||||
|
||||
### Step 5: Deploy the YOLO Model
|
||||
|
||||
Before diving into the deployment instructions, be sure to check out the range of [YOLO26 models offered by Ultralytics](../models/index.md). This will help you choose the most appropriate model for your project requirements.
|
||||
|
||||
After creating the AWS CloudFormation Stack, the next step is to deploy YOLO26.
|
||||
|
||||
- Open the Notebook Instance: Go to the AWS Console and navigate to the Amazon SageMaker service. Select "Notebook Instances" from the dashboard, then locate the notebook instance that was created by your CDK deployment script. Open the notebook instance to access the Jupyter environment.
|
||||
|
||||
- Access and Modify inference.py: After opening the SageMaker notebook instance in Jupyter, locate the inference.py file. Edit the output_fn function in inference.py as shown below and save your changes to the script, ensuring that there are no syntax errors.
|
||||
|
||||
```python
|
||||
import json
|
||||
|
||||
|
||||
def output_fn(prediction_output):
|
||||
"""Formats model outputs as JSON string, extracting attributes like boxes, masks, keypoints."""
|
||||
print("Executing output_fn from inference.py ...")
|
||||
infer = {}
|
||||
for result in prediction_output:
|
||||
if result.boxes is not None:
|
||||
infer["boxes"] = result.boxes.numpy().data.tolist()
|
||||
if result.masks is not None:
|
||||
infer["masks"] = result.masks.numpy().data.tolist()
|
||||
if result.keypoints is not None:
|
||||
infer["keypoints"] = result.keypoints.numpy().data.tolist()
|
||||
if result.obb is not None:
|
||||
infer["obb"] = result.obb.numpy().data.tolist()
|
||||
if result.probs is not None:
|
||||
infer["probs"] = result.probs.numpy().data.tolist()
|
||||
return json.dumps(infer)
|
||||
```
|
||||
|
||||
- Deploy the Endpoint Using 1_DeployEndpoint.ipynb: In the Jupyter environment, open the 1_DeployEndpoint.ipynb notebook located in the sm-notebook directory. Follow the instructions in the notebook and run the cells to download the YOLO26 model, package it with the updated inference code, and upload it to an Amazon S3 bucket. The notebook will guide you through creating and deploying a SageMaker endpoint for the YOLO26 model.
|
||||
|
||||
### Step 6: Testing Your Deployment
|
||||
|
||||
Now that your YOLO26 model is deployed, it's important to test its performance and functionality.
|
||||
|
||||
- Open the Test Notebook: In the same Jupyter environment, locate and open the 2_TestEndpoint.ipynb notebook, also in the sm-notebook directory.
|
||||
|
||||
- Run the Test Notebook: Follow the instructions within the notebook to test the deployed SageMaker endpoint. This includes sending an image to the endpoint and running inferences. Then, you'll plot the output to visualize the model's performance and [accuracy](https://www.ultralytics.com/glossary/accuracy), as shown below.
|
||||
|
||||
<p align="center">
|
||||
<img width="640" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/testing-results-yolov8.avif" alt="Testing Results YOLO26">
|
||||
</p>
|
||||
|
||||
- Clean-Up Resources: The test notebook will also guide you through the process of cleaning up the endpoint and the hosted model. This is an important step to manage costs and resources effectively, especially if you do not plan to use the deployed model immediately.
|
||||
|
||||
### Step 7: Monitoring and Management
|
||||
|
||||
After testing, continuous monitoring and management of your deployed model are essential.
|
||||
|
||||
- Monitor with Amazon CloudWatch: Regularly check the performance and health of your SageMaker endpoint using [Amazon CloudWatch](https://aws.amazon.com/cloudwatch/).
|
||||
|
||||
- Manage the Endpoint: Use the SageMaker console for ongoing management of the endpoint. This includes scaling, updating, or redeploying the model as required.
|
||||
|
||||
By completing these steps, you will have successfully deployed and tested a YOLO26 model on Amazon SageMaker Endpoints. This process not only equips you with practical experience in using AWS services for machine learning deployment but also lays the foundation for deploying other advanced models in the future.
|
||||
|
||||
## Summary
|
||||
|
||||
This guide took you step by step through deploying YOLO26 on Amazon SageMaker Endpoints using AWS CloudFormation and the AWS Cloud Development Kit (CDK). The process includes cloning the necessary GitHub repository, setting up the CDK environment, deploying the model using AWS services, and testing its performance on SageMaker.
|
||||
|
||||
For more technical details, refer to [this article](https://aws.amazon.com/blogs/machine-learning/hosting-yolov8-pytorch-model-on-amazon-sagemaker-endpoints/) on the AWS Machine Learning Blog. You can also check out the official [Amazon SageMaker Documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/realtime-endpoints.html) for more insights into various features and functionalities.
|
||||
|
||||
Are you interested in learning more about different YOLO26 integrations? Visit the [Ultralytics integrations guide page](../integrations/index.md) to discover additional tools and capabilities that can enhance your machine-learning projects.
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I deploy the Ultralytics YOLO26 model on Amazon SageMaker Endpoints?
|
||||
|
||||
To deploy the Ultralytics YOLO26 model on Amazon SageMaker Endpoints, follow these steps:
|
||||
|
||||
1. **Set Up Your AWS Environment**: Ensure you have an AWS Account, IAM roles with necessary permissions, and the AWS CLI configured. Install AWS CDK if not already done (refer to the [AWS CDK instructions](https://docs.aws.amazon.com/cdk/v2/guide/#getting_started_install)).
|
||||
2. **Clone the YOLO26 SageMaker Repository**:
|
||||
```bash
|
||||
git clone https://github.com/aws-samples/host-yolov8-on-sagemaker-endpoint.git
|
||||
cd host-yolov8-on-sagemaker-endpoint/yolov8-pytorch-cdk
|
||||
```
|
||||
3. **Set Up the CDK Environment**: Create a Python virtual environment, activate it, install dependencies, and upgrade AWS CDK library.
|
||||
```bash
|
||||
python3 -m venv .venv
|
||||
source .venv/bin/activate
|
||||
pip3 install -r requirements.txt
|
||||
pip install --upgrade aws-cdk-lib
|
||||
```
|
||||
4. **Deploy using AWS CDK**: Synthesize and deploy the CloudFormation stack, bootstrap the environment.
|
||||
```bash
|
||||
cdk synth
|
||||
cdk bootstrap
|
||||
cdk deploy
|
||||
```
|
||||
|
||||
For further details, review the [documentation section](#step-5-deploy-the-yolo-model).
|
||||
|
||||
### What are the prerequisites for deploying YOLO26 on Amazon SageMaker?
|
||||
|
||||
To deploy YOLO26 on Amazon SageMaker, ensure you have the following prerequisites:
|
||||
|
||||
1. **AWS Account**: Active AWS account ([sign up here](https://aws.amazon.com/)).
|
||||
2. **IAM Roles**: Configured IAM roles with permissions for SageMaker, CloudFormation, and Amazon S3.
|
||||
3. **AWS CLI**: Installed and configured AWS Command Line Interface ([AWS CLI installation guide](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html)).
|
||||
4. **AWS CDK**: Installed AWS Cloud Development Kit ([CDK setup guide](https://docs.aws.amazon.com/cdk/v2/guide/#getting_started_install)).
|
||||
5. **Service Quotas**: Sufficient quotas for `ml.m5.4xlarge` instances for both endpoint and notebook usage ([request a quota increase](https://docs.aws.amazon.com/servicequotas/latest/userguide/request-quota-increase.html#quota-console-increase)).
|
||||
|
||||
For detailed setup, refer to [this section](#step-1-setup-your-aws-environment).
|
||||
|
||||
### Why should I use Ultralytics YOLO26 on Amazon SageMaker?
|
||||
|
||||
Using Ultralytics YOLO26 on Amazon SageMaker offers several advantages:
|
||||
|
||||
1. **Scalability and Management**: SageMaker provides a managed environment with features like autoscaling, which helps in real-time inference needs.
|
||||
2. **Integration with AWS Services**: Seamlessly integrate with other AWS services, such as S3 for data storage, CloudFormation for infrastructure as code, and CloudWatch for monitoring.
|
||||
3. **Ease of Deployment**: Simplified setup using AWS CDK scripts and streamlined deployment processes.
|
||||
4. **Performance**: Leverage Amazon SageMaker's high-performance infrastructure for running large scale inference tasks efficiently.
|
||||
|
||||
Explore more about the advantages of using SageMaker in the [introduction section](#amazon-sagemaker).
|
||||
|
||||
### Can I customize the inference logic for YOLO26 on Amazon SageMaker?
|
||||
|
||||
Yes, you can customize the inference logic for YOLO26 on Amazon SageMaker:
|
||||
|
||||
1. **Modify `inference.py`**: Locate and customize the `output_fn` function in the `inference.py` file to tailor output formats.
|
||||
|
||||
```python
|
||||
import json
|
||||
|
||||
|
||||
def output_fn(prediction_output):
|
||||
"""Formats model outputs as JSON string, extracting attributes like boxes, masks, keypoints."""
|
||||
infer = {}
|
||||
for result in prediction_output:
|
||||
if result.boxes is not None:
|
||||
infer["boxes"] = result.boxes.numpy().data.tolist()
|
||||
# Add more processing logic if necessary
|
||||
return json.dumps(infer)
|
||||
```
|
||||
|
||||
2. **Deploy Updated Model**: Ensure you redeploy the model using Jupyter notebooks provided (`1_DeployEndpoint.ipynb`) to include these changes.
|
||||
|
||||
Refer to the [detailed steps](#step-5-deploy-the-yolo-model) for deploying the modified model.
|
||||
|
||||
### How can I test the deployed YOLO26 model on Amazon SageMaker?
|
||||
|
||||
To test the deployed YOLO26 model on Amazon SageMaker:
|
||||
|
||||
1. **Open the Test Notebook**: Locate the `2_TestEndpoint.ipynb` notebook in the SageMaker Jupyter environment.
|
||||
2. **Run the Notebook**: Follow the notebook's instructions to send an image to the endpoint, perform inference, and display results.
|
||||
3. **Visualize Results**: Use built-in plotting functionalities to visualize performance metrics, such as bounding boxes around detected objects.
|
||||
|
||||
For comprehensive testing instructions, visit the [testing section](#step-6-testing-your-deployment).
|
||||
@@ -0,0 +1,280 @@
|
||||
---
|
||||
comments: true
|
||||
description: Deploy Ultralytics YOLO models on Axelera AI's Metis hardware. Learn how to export, compile, and run high-performance edge inference with up to 856 TOPS.
|
||||
keywords: Axelera AI, Metis AIPU, Voyager SDK, Edge AI, YOLOv8, YOLO26, Model Export, Computer Vision, PCIe, M.2, Object Detection, quantization
|
||||
---
|
||||
|
||||
# Axelera AI Export and Deployment
|
||||
|
||||
!!! tip "Experimental Release"
|
||||
|
||||
This is an experimental integration demonstrating deployment on Axelera Metis hardware. Full integration anticipated by **February 2026** with model export without requiring Axelera hardware and standard pip installation.
|
||||
|
||||
Ultralytics partners with [Axelera AI](https://www.axelera.ai/) to enable high-performance, energy-efficient inference on [Edge AI](https://www.ultralytics.com/glossary/edge-ai) devices. Export and deploy **Ultralytics YOLO models** directly to the **Metis® AIPU** using the **Voyager SDK**.
|
||||
|
||||
|
||||
|
||||
Axelera AI provides dedicated hardware acceleration for [computer vision](https://www.ultralytics.com/glossary/computer-vision-cv) at the edge, using a proprietary dataflow architecture and [in-memory computing](https://www.ultralytics.com/glossary/edge-computing) to deliver up to **856 TOPS** with low power consumption.
|
||||
|
||||
## Selecting the Right Hardware
|
||||
|
||||
Axelera AI offers various form factors to suit different deployment constraints. The chart below helps identify the optimal hardware for your Ultralytics YOLO deployment.
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
A[Start: Select Deployment Target] --> B{Device Type?}
|
||||
B -->|Edge Server / Workstation| C{Throughput Needs?}
|
||||
B -->|Embedded / Robotics| D{Space Constraints?}
|
||||
B -->|Standalone / R&D| E[Dev Kits & Systems]
|
||||
|
||||
C -->|Max Density <br> 30+ Streams| F[**Metis PCIe x4**<br>856 TOPS]
|
||||
C -->|Standard PC <br> Low Profile| G[**Metis PCIe x1**<br>214 TOPS]
|
||||
|
||||
D -->|Drones & Handhelds| H[**Metis M.2**<br>2280 M-Key]
|
||||
D -->|High Performance Embedded| I[**Metis M.2 MAX**<br>Extended Thermal]
|
||||
|
||||
E -->|ARM-based All-in-One| J[**Metis Compute Board**<br>RK3588 + AIPU]
|
||||
E -->|Prototyping| K[**Arduino Portenta x8**<br>Integration Kit]
|
||||
|
||||
click F "https://store.axelera.ai/"
|
||||
click G "https://store.axelera.ai/"
|
||||
click H "https://store.axelera.ai/"
|
||||
click J "https://store.axelera.ai/"
|
||||
```
|
||||
|
||||
## Hardware Portfolio
|
||||
|
||||
The Axelera hardware lineup is optimized to run [Ultralytics YOLO26](https://docs.ultralytics.com/models/yolo26/) and legacy versions with high FPS-per-watt efficiency.
|
||||
|
||||
### Accelerator Cards
|
||||
|
||||
These cards enable AI acceleration in existing host devices, facilitating [brownfield deployments](https://www.ultralytics.com/glossary/edge-computing).
|
||||
|
||||
| Product | Form Factor | Compute | Performance (INT8) | Target Application |
|
||||
| :---------------- | :------------- | :----------------- | :----------------- | :----------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **Metis PCIe x4** | PCIe Gen3 x16 | **4x** Metis AIPUs | **856 TOPS** | High-density [video analytics](https://docs.ultralytics.com/guides/analytics/), smart cities |
|
||||
| **Metis PCIe x1** | PCIe Gen3 x1 | **1x** Metis AIPU | **214 TOPS** | Industrial PCs, retail [queue management](https://docs.ultralytics.com/guides/queue-management/) |
|
||||
| **Metis M.2** | M.2 2280 M-Key | **1x** Metis AIPU | **214 TOPS** | [Drones](https://www.ultralytics.com/blog/build-ai-powered-drone-applications-with-ultralytics-yolo11), robotics, portable medical devices |
|
||||
| **Metis M.2 MAX** | M.2 2280 | **1x** Metis AIPU | **214 TOPS** | Environments requiring advanced thermal management |
|
||||
|
||||
### Integrated Systems
|
||||
|
||||
For turnkey solutions, Axelera partners with manufacturers to provide systems pre-validated for the Metis AIPU.
|
||||
|
||||
- **Metis Compute Board**: A standalone edge device pairing the Metis AIPU with a Rockchip RK3588 ARM CPU.
|
||||
- **Workstations**: Enterprise towers from **Dell** (Precision 3460XE) and **Lenovo** (ThinkStation P360 Ultra).
|
||||
- **Industrial PCs**: Ruggedized systems from **Advantech** and **Aetina** designed for [manufacturing automation](https://www.ultralytics.com/solutions/ai-in-manufacturing).
|
||||
|
||||
## Supported Tasks
|
||||
|
||||
Currently, Object Detection models can be exported to the Axelera format. Additional tasks are being integrated:
|
||||
|
||||
| Task | Status |
|
||||
| :----------------------------------------------------------------- | :----------- |
|
||||
| [Object Detection](https://docs.ultralytics.com/tasks/detect/) | ✅ Supported |
|
||||
| [Pose Estimation](https://docs.ultralytics.com/tasks/pose/) | Coming soon |
|
||||
| [Segmentation](https://docs.ultralytics.com/tasks/segment/) | Coming soon |
|
||||
| [Oriented Bounding Boxes](https://docs.ultralytics.com/tasks/obb/) | Coming soon |
|
||||
|
||||
## Installation
|
||||
|
||||
!!! warning "Platform Requirements"
|
||||
|
||||
Exporting to Axelera format requires:
|
||||
|
||||
- **Operating System**: Linux only (Ubuntu 22.04/24.04 recommended)
|
||||
- **Hardware**: Axelera AI accelerator ([Metis devices](https://store.axelera.ai/))
|
||||
- **Python**: Version 3.10 (3.11 and 3.12 coming soon)
|
||||
|
||||
### Ultralytics Installation
|
||||
|
||||
```bash
|
||||
pip install ultralytics
|
||||
```
|
||||
|
||||
For detailed instructions, see our [Ultralytics Installation guide](../quickstart.md). If you encounter difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md).
|
||||
|
||||
### Axelera Driver Installation
|
||||
|
||||
1. Add the Axelera repository key:
|
||||
|
||||
```bash
|
||||
sudo sh -c "curl -fsSL https://software.axelera.ai/artifactory/api/security/keypair/axelera/public | gpg --dearmor -o /etc/apt/keyrings/axelera.gpg"
|
||||
```
|
||||
|
||||
2. Add the repository to apt:
|
||||
|
||||
```bash
|
||||
sudo sh -c "echo 'deb [signed-by=/etc/apt/keyrings/axelera.gpg] https://software.axelera.ai/artifactory/axelera-apt-source/ ubuntu22 main' > /etc/apt/sources.list.d/axelera.list"
|
||||
```
|
||||
|
||||
3. Install the SDK and load the driver:
|
||||
|
||||
```bash
|
||||
sudo apt update
|
||||
sudo apt install -y axelera-voyager-sdk-base
|
||||
sudo modprobe metis
|
||||
yes | sudo /opt/axelera/sdk/latest/axelera_fix_groups.sh $USER
|
||||
```
|
||||
|
||||
## Exporting YOLO Models to Axelera
|
||||
|
||||
Export your trained YOLO models using the standard Ultralytics export command.
|
||||
|
||||
!!! example "Export to Axelera Format"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load a YOLO26 model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Export to Axelera format
|
||||
model.export(format="axelera") # creates 'yolo26n_axelera_model' directory
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
yolo export model=yolo26n.pt format=axelera
|
||||
```
|
||||
|
||||
### Export Arguments
|
||||
|
||||
| Argument | Type | Default | Description |
|
||||
| :--------- | :--------------- | :--------------- | :------------------------------------------------------------------------------------------- |
|
||||
| `format` | `str` | `'axelera'` | Target format for Axelera Metis AIPU hardware |
|
||||
| `imgsz` | `int` or `tuple` | `640` | Image size for model input |
|
||||
| `int8` | `bool` | `True` | Enable [INT8 quantization](https://www.ultralytics.com/glossary/model-quantization) for AIPU |
|
||||
| `data` | `str` | `'coco128.yaml'` | [Dataset](https://docs.ultralytics.com/datasets/) config for quantization calibration |
|
||||
| `fraction` | `float` | `1.0` | Fraction of dataset for calibration (100-400 images recommended) |
|
||||
| `device` | `str` | `None` | Export device: GPU (`device=0`) or CPU (`device=cpu`) |
|
||||
|
||||
For all export options, see the [Export Mode documentation](https://docs.ultralytics.com/modes/export/).
|
||||
|
||||
### Output Structure
|
||||
|
||||
```text
|
||||
yolo26n_axelera_model/
|
||||
├── yolo26n.axm # Axelera model file
|
||||
└── metadata.yaml # Model metadata (classes, image size, etc.)
|
||||
```
|
||||
|
||||
## Running Inference
|
||||
|
||||
Load the exported model with the Ultralytics API and run inference, similar to loading [ONNX](https://docs.ultralytics.com/integrations/onnx/) models.
|
||||
|
||||
!!! example "Inference with Axelera Model"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load the exported Axelera model
|
||||
model = YOLO("yolo26n_axelera_model")
|
||||
|
||||
# Run inference
|
||||
results = model("https://ultralytics.com/images/bus.jpg")
|
||||
|
||||
# Process results
|
||||
for r in results:
|
||||
print(f"Detected {len(r.boxes)} objects")
|
||||
r.show() # Display results
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
yolo predict model='yolo26n_axelera_model' source='https://ultralytics.com/images/bus.jpg'
|
||||
```
|
||||
|
||||
!!! warning "Known Issue"
|
||||
|
||||
The first inference run may throw an `ImportError`. Subsequent runs will work correctly. This will be addressed in a future release.
|
||||
|
||||
## Inference Performance
|
||||
|
||||
The Metis AIPU maximizes throughput while minimizing energy consumption.
|
||||
|
||||
| Metric | Metis PCIe x4 | Metis M.2 | Note |
|
||||
| :------------------ | :------------ | :----------- | :---------------------- |
|
||||
| **Peak Throughput** | **856 TOPS** | 214 TOPS | INT8 Precision |
|
||||
| **YOLOv5m FPS** | **~1539 FPS** | ~326 FPS | 640x640 Input |
|
||||
| **YOLOv5s FPS** | N/A | **~827 FPS** | 640x640 Input |
|
||||
| **Efficiency** | High | Very High | Ideal for battery power |
|
||||
|
||||
_Benchmarks based on Axelera AI data. Actual FPS depends on model size, batching, and input resolution._
|
||||
|
||||
## Real-World Applications
|
||||
|
||||
Ultralytics YOLO on Axelera hardware enables advanced edge computing solutions:
|
||||
|
||||
- **Smart Retail**: Real-time [object counting](https://docs.ultralytics.com/guides/object-counting/) and [heatmap analytics](https://docs.ultralytics.com/guides/heatmaps/) for store optimization.
|
||||
- **Industrial Safety**: Low-latency [PPE detection](https://docs.ultralytics.com/datasets/detect/construction-ppe/) in manufacturing environments.
|
||||
- **Drone Analytics**: High-speed [object detection](https://docs.ultralytics.com/tasks/detect/) on UAVs for [agriculture](https://www.ultralytics.com/solutions/ai-in-agriculture) and search-and-rescue.
|
||||
- **Traffic Systems**: Edge-based [license plate recognition](https://www.ultralytics.com/blog/using-ultralytics-yolo11-for-automatic-number-plate-recognition) and [speed estimation](https://docs.ultralytics.com/guides/speed-estimation/).
|
||||
|
||||
## Recommended Workflow
|
||||
|
||||
1. **Train** your model using Ultralytics [Train Mode](https://docs.ultralytics.com/modes/train/)
|
||||
2. **Export** to Axelera format using `model.export(format="axelera")`
|
||||
3. **Validate** accuracy with `yolo val` to verify minimal quantization loss
|
||||
4. **Predict** using `yolo predict` for qualitative validation
|
||||
|
||||
## Device Health Check
|
||||
|
||||
Verify your Axelera device is functioning properly:
|
||||
|
||||
```bash
|
||||
. /opt/axelera/sdk/latest/axelera_activate.sh
|
||||
axdevice
|
||||
```
|
||||
|
||||
For detailed diagnostics, see the [AxDevice documentation](https://github.com/axelera-ai-hub/voyager-sdk/blob/release/v1.5/docs/reference/axdevice.md).
|
||||
|
||||
## Maximum Performance
|
||||
|
||||
This integration uses single-core configuration for compatibility. For production requiring maximum throughput, the [Axelera Voyager SDK](https://github.com/axelera-ai-hub/voyager-sdk) offers:
|
||||
|
||||
- Multi-core utilization (quad-core Metis AIPU)
|
||||
- Streaming inference pipelines
|
||||
- Tiled inferencing for higher-resolution cameras
|
||||
|
||||
See the [model-zoo](https://github.com/axelera-ai-hub/voyager-sdk/blob/release/v1.5/docs/reference/model_zoo.md) for FPS benchmarks or [contact Axelera](https://axelera.ai/contact-us) for production support.
|
||||
|
||||
## Known Issues
|
||||
|
||||
!!! warning "Known Limitations"
|
||||
|
||||
- **PyTorch 2.9 compatibility**: The first `yolo export format=axelera` command may fail due to automatic PyTorch downgrade to 2.8. Run the command a second time to succeed.
|
||||
|
||||
- **M.2 power limitations**: Large or extra-large models may encounter runtime errors on M.2 accelerators due to power supply constraints.
|
||||
|
||||
- **First inference ImportError**: The first inference run may throw an `ImportError`. Subsequent runs work correctly.
|
||||
|
||||
For support, visit the [Axelera Community](https://community.axelera.ai/).
|
||||
|
||||
## FAQ
|
||||
|
||||
### What YOLO versions are supported on Axelera?
|
||||
|
||||
The Voyager SDK supports export of [YOLOv8](https://docs.ultralytics.com/models/yolov8/) and [YOLO26](https://docs.ultralytics.com/models/yolo26/) models.
|
||||
|
||||
### Can I deploy custom-trained models?
|
||||
|
||||
Yes. Any model trained using [Ultralytics Train Mode](https://docs.ultralytics.com/modes/train/) can be exported to the Axelera format, provided it uses supported layers and operations.
|
||||
|
||||
### How does INT8 quantization affect accuracy?
|
||||
|
||||
Axelera's Voyager SDK automatically quantizes models for the mixed-precision AIPU architecture. For most [object detection](https://www.ultralytics.com/glossary/object-detection) tasks, the performance gains (higher FPS, lower power) significantly outweigh the minimal impact on [mAP](https://docs.ultralytics.com/guides/yolo-performance-metrics/). Quantization takes seconds to several hours depending on model size. Run `yolo val` after export to verify accuracy.
|
||||
|
||||
### How many calibration images should I use?
|
||||
|
||||
We recommend 100 to 400 images. More than 400 provides no additional benefit and increases quantization time. Experiment with 100, 200, and 400 images to find the optimal balance.
|
||||
|
||||
### Where can I find the Voyager SDK?
|
||||
|
||||
The SDK, drivers, and compiler tools are available via the [Axelera Developer Portal](https://www.axelera.ai/).
|
||||
@@ -0,0 +1,262 @@
|
||||
---
|
||||
comments: true
|
||||
description: Discover how to integrate YOLO26 with ClearML to streamline your MLOps workflow, automate experiments, and enhance model management effortlessly.
|
||||
keywords: YOLO26, ClearML, MLOps, Ultralytics, machine learning, object detection, model training, automation, experiment management
|
||||
---
|
||||
|
||||
# Training YOLO26 with ClearML: Streamlining Your MLOps Workflow
|
||||
|
||||
MLOps bridges the gap between creating and deploying [machine learning](https://www.ultralytics.com/glossary/machine-learning-ml) models in real-world settings. It focuses on efficient deployment, scalability, and ongoing management to ensure models perform well in practical applications.
|
||||
|
||||
[Ultralytics YOLO26](https://www.ultralytics.com/) effortlessly integrates with ClearML, streamlining and enhancing your [object detection](https://www.ultralytics.com/glossary/object-detection) model's training and management. This guide will walk you through the integration process, detailing how to set up ClearML, manage experiments, automate model management, and collaborate effectively.
|
||||
|
||||
## ClearML
|
||||
|
||||
<p align="center">
|
||||
<img width="100%" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/clearml-overview.avif" alt="ClearML MLOps platform dashboard">
|
||||
</p>
|
||||
|
||||
[ClearML](https://clear.ml/) is an innovative open-source MLOps platform that is skillfully designed to automate, monitor, and orchestrate machine learning workflows. Its key features include automated logging of all training and inference data for full experiment reproducibility, an intuitive web UI for easy [data visualization](https://www.ultralytics.com/glossary/data-visualization) and analysis, advanced hyperparameter [optimization algorithms](https://www.ultralytics.com/glossary/optimization-algorithm), and robust model management for efficient deployment across various platforms.
|
||||
|
||||
## YOLO26 Training with ClearML
|
||||
|
||||
You can bring automation and efficiency to your machine learning workflow by integrating YOLO26 with ClearML to improve your training process.
|
||||
|
||||
## Installation
|
||||
|
||||
To install the required packages, run:
|
||||
|
||||
!!! tip "Installation"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Install the required packages for YOLO26 and ClearML
|
||||
pip install ultralytics clearml
|
||||
```
|
||||
|
||||
For detailed instructions and best practices related to the installation process, be sure to check our [YOLO26 Installation guide](../quickstart.md). While installing the required packages for YOLO26, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips.
|
||||
|
||||
## Configuring ClearML
|
||||
|
||||
Once you have installed the necessary packages, the next step is to initialize and configure your ClearML SDK. This involves setting up your ClearML account and obtaining the necessary credentials for a seamless connection between your development environment and the ClearML server.
|
||||
|
||||
Begin by initializing the ClearML SDK in your environment. The `clearml-init` command starts the setup process and prompts you for the necessary credentials.
|
||||
|
||||
!!! tip "Initial SDK Setup"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Initialize your ClearML SDK setup process
|
||||
clearml-init
|
||||
```
|
||||
|
||||
After executing this command, visit the [ClearML Settings page](https://app.clear.ml/settings/workspace-configuration). Navigate to the top right corner and select "Settings." Go to the "Workspace" section and click on "Create new credentials." Use the credentials provided in the "Create Credentials" pop-up to complete the setup as instructed, depending on whether you are configuring ClearML in a Jupyter Notebook or a local Python environment.
|
||||
|
||||
## Usage
|
||||
|
||||
Before diving into the usage instructions, be sure to check out the range of [YOLO26 models offered by Ultralytics](../models/index.md). This will help you choose the most appropriate model for your project requirements.
|
||||
|
||||
!!! example "Usage"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from clearml import Task
|
||||
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Step 1: Creating a ClearML Task
|
||||
task = Task.init(project_name="my_project", task_name="my_yolo26_task")
|
||||
|
||||
# Step 2: Selecting the YOLO26 Model
|
||||
model_variant = "yolo26n"
|
||||
task.set_parameter("model_variant", model_variant)
|
||||
|
||||
# Step 3: Loading the YOLO26 Model
|
||||
model = YOLO(f"{model_variant}.pt")
|
||||
|
||||
# Step 4: Setting Up Training Arguments
|
||||
args = dict(data="coco8.yaml", epochs=16)
|
||||
task.connect(args)
|
||||
|
||||
# Step 5: Initiating Model Training
|
||||
results = model.train(**args)
|
||||
```
|
||||
|
||||
### Understanding the Code
|
||||
|
||||
Let's understand the steps showcased in the usage code snippet above.
|
||||
|
||||
**Step 1: Creating a ClearML Task**: A new task is initialized in ClearML, specifying your project and task names. This task will track and manage your model's training.
|
||||
|
||||
**Step 2: Selecting the YOLO26 Model**: The `model_variant` variable is set to 'yolo26n', one of the YOLO26 models. This variant is then logged in ClearML for tracking.
|
||||
|
||||
**Step 3: Loading the YOLO26 Model**: The selected YOLO26 model is loaded using Ultralytics' YOLO class, preparing it for training.
|
||||
|
||||
**Step 4: Setting Up Training Arguments**: Key training arguments like the dataset (`coco8.yaml`) and the number of [epochs](https://www.ultralytics.com/glossary/epoch) (`16`) are organized in a dictionary and connected to the ClearML task. This allows for tracking and potential modification via the ClearML UI. For a detailed understanding of the model training process and best practices, refer to our [YOLO26 Model Training guide](../modes/train.md).
|
||||
|
||||
**Step 5: Initiating Model Training**: The model training is started with the specified arguments. The results of the training process are captured in the `results` variable.
|
||||
|
||||
### Understanding the Output
|
||||
|
||||
Upon running the usage code snippet above, you can expect the following output:
|
||||
|
||||
- A confirmation message indicating the creation of a new ClearML task, along with its unique ID.
|
||||
- An informational message about the script code being stored, indicating that the code execution is being tracked by ClearML.
|
||||
- A URL link to the ClearML results page where you can monitor the training progress and view detailed logs.
|
||||
- Download progress for the YOLO26 model and the specified dataset, followed by a summary of the model architecture and training configuration.
|
||||
- Initialization messages for various training components like TensorBoard, Automatic [Mixed Precision](https://www.ultralytics.com/glossary/mixed-precision) (AMP), and dataset preparation.
|
||||
- Finally, the training process starts, with progress updates as the model trains on the specified dataset. For an in-depth understanding of the performance metrics used during training, read [our guide on performance metrics](../guides/yolo-performance-metrics.md).
|
||||
|
||||
### Viewing the ClearML Results Page
|
||||
|
||||
By clicking on the URL link to the ClearML results page in the output of the usage code snippet, you can access a comprehensive view of your model's training process.
|
||||
|
||||
#### Key Features of the ClearML Results Page
|
||||
|
||||
- **Real-Time Metrics Tracking**
|
||||
- Track critical metrics like loss, [accuracy](https://www.ultralytics.com/glossary/accuracy), and validation scores as they occur.
|
||||
- Provides immediate feedback for timely model performance adjustments.
|
||||
|
||||
- **Experiment Comparison**
|
||||
- Compare different training runs side-by-side.
|
||||
- Essential for [hyperparameter tuning](https://www.ultralytics.com/glossary/hyperparameter-tuning) and identifying the most effective models.
|
||||
|
||||
- **Detailed Logs and Outputs**
|
||||
- Access comprehensive logs, graphical representations of metrics, and console outputs.
|
||||
- Gain a deeper understanding of model behavior and issue resolution.
|
||||
|
||||
- **Resource Utilization Monitoring**
|
||||
- Monitor the utilization of computational resources, including CPU, GPU, and memory.
|
||||
- Key to optimizing training efficiency and costs.
|
||||
|
||||
- **Model Artifacts Management**
|
||||
- View, download, and share model artifacts like trained models and checkpoints.
|
||||
- Enhances collaboration and streamlines [model deployment](https://www.ultralytics.com/glossary/model-deployment) and sharing.
|
||||
|
||||
For a visual walkthrough of what the ClearML Results Page looks like, watch the video below:
|
||||
|
||||
<p align="center">
|
||||
<br>
|
||||
<iframe loading="lazy" width="720" height="405" src="https://www.youtube.com/embed/iLcC7m3bCes?si=oSEAoZbrg8inCg_2"
|
||||
title="YouTube video player" frameborder="0"
|
||||
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"
|
||||
allowfullscreen>
|
||||
</iframe>
|
||||
<br>
|
||||
<strong>Watch:</strong> YOLO26 MLOps Integration using ClearML
|
||||
</p>
|
||||
|
||||
### Advanced Features in ClearML
|
||||
|
||||
ClearML offers several advanced features to enhance your MLOps experience.
|
||||
|
||||
#### Remote Execution
|
||||
|
||||
ClearML's remote execution feature facilitates the reproduction and manipulation of experiments on different machines. It logs essential details like installed packages and uncommitted changes. When a task is enqueued, the [ClearML Agent](https://clear.ml/docs/latest/docs/clearml_agent/) pulls it, recreates the environment, and runs the experiment, reporting back with detailed results.
|
||||
|
||||
Deploying a ClearML Agent is straightforward and can be done on various machines using the following command:
|
||||
|
||||
```bash
|
||||
clearml-agent daemon --queue QUEUES_TO_LISTEN_TO [--docker]
|
||||
```
|
||||
|
||||
This setup is applicable to cloud VMs, local GPUs, or laptops. [ClearML Autoscalers](https://clear.ml/docs/latest/docs/cloud_autoscaling/autoscaling_overview/) help manage cloud workloads on platforms like AWS, GCP, and Azure, automating the deployment of agents and adjusting resources based on your resource budget.
|
||||
|
||||
### Cloning, Editing, and Enqueuing
|
||||
|
||||
ClearML's user-friendly interface allows easy cloning, editing, and enqueuing of tasks. Users can clone an existing experiment, adjust parameters or other details through the UI, and enqueue the task for execution. This streamlined process ensures that the ClearML Agent executing the task uses updated configurations, making it ideal for iterative experimentation and model fine-tuning.
|
||||
|
||||
<p align="center"><br>
|
||||
<img width="100%" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/cloning-editing-enqueuing-clearml.avif" alt="Cloning, Editing, and Enqueuing with ClearML">
|
||||
</p>
|
||||
|
||||
## Dataset Version Management
|
||||
|
||||
ClearML also offers powerful [dataset version management](https://clear.ml/docs/latest/docs/hyperdatasets/dataset/) capabilities that integrate seamlessly with YOLO26 training workflows. This feature allows you to:
|
||||
|
||||
- Version your datasets separately from your code
|
||||
- Track which dataset version was used for each experiment
|
||||
- Easily access and download the latest dataset version
|
||||
|
||||
To prepare your dataset for ClearML, follow these steps:
|
||||
|
||||
1. Organize your dataset with the standard YOLO structure (images, labels, etc.)
|
||||
2. Copy the corresponding YAML file to the root of your dataset folder
|
||||
3. Upload your dataset using the ClearML Data tool:
|
||||
|
||||
```bash
|
||||
cd your_dataset_folder
|
||||
clearml-data sync --project YOLO26 --name your_dataset_name --folder .
|
||||
```
|
||||
|
||||
This command will create a versioned dataset in ClearML that can be referenced in your training scripts, ensuring reproducibility and easy access to your data.
|
||||
|
||||
## Summary
|
||||
|
||||
This guide has led you through the process of integrating ClearML with Ultralytics' YOLO26. Covering everything from initial setup to advanced model management, you've discovered how to leverage ClearML for efficient training, experiment tracking, and workflow optimization in your machine learning projects.
|
||||
|
||||
For further details on usage, visit [ClearML's official YOLOv8 integration guide](https://clear.ml/docs/latest/docs/integrations/yolov8/), which also applies to YOLO26 workflows.
|
||||
|
||||
Additionally, explore more integrations and capabilities of Ultralytics by visiting the [Ultralytics integration guide page](../integrations/index.md), which is a treasure trove of resources and insights.
|
||||
|
||||
## FAQ
|
||||
|
||||
### What is the process for integrating Ultralytics YOLO26 with ClearML?
|
||||
|
||||
Integrating Ultralytics YOLO26 with ClearML involves a series of steps to streamline your MLOps workflow. First, install the necessary packages:
|
||||
|
||||
```bash
|
||||
pip install ultralytics clearml
|
||||
```
|
||||
|
||||
Next, initialize the ClearML SDK in your environment using:
|
||||
|
||||
```bash
|
||||
clearml-init
|
||||
```
|
||||
|
||||
You then configure ClearML with your credentials from the [ClearML Settings page](https://app.clear.ml/settings/workspace-configuration). Detailed instructions on the entire setup process, including model selection and training configurations, can be found in our [YOLO26 Model Training guide](../modes/train.md).
|
||||
|
||||
### Why should I use ClearML with Ultralytics YOLO26 for my machine learning projects?
|
||||
|
||||
Using ClearML with Ultralytics YOLO26 enhances your machine learning projects by automating experiment tracking, streamlining workflows, and enabling robust model management. ClearML offers real-time metrics tracking, resource utilization monitoring, and a user-friendly interface for comparing experiments. These features help optimize your model's performance and make the development process more efficient. Learn more about the benefits and procedures in our [MLOps Integration guide](../modes/train.md).
|
||||
|
||||
### How do I troubleshoot common issues during YOLO26 and ClearML integration?
|
||||
|
||||
If you encounter issues during the integration of YOLO26 with ClearML, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips. Typical problems might involve package installation errors, credential setup, or configuration issues. This guide provides step-by-step troubleshooting instructions to resolve these common issues efficiently.
|
||||
|
||||
### How do I set up the ClearML task for YOLO26 model training?
|
||||
|
||||
Setting up a ClearML task for YOLO26 training involves initializing a task, selecting the model variant, loading the model, setting up training arguments, and finally, starting the model training. Here's a simplified example:
|
||||
|
||||
```python
|
||||
from clearml import Task
|
||||
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Step 1: Creating a ClearML Task
|
||||
task = Task.init(project_name="my_project", task_name="my_yolo26_task")
|
||||
|
||||
# Step 2: Selecting the YOLO26 Model
|
||||
model_variant = "yolo26n"
|
||||
task.set_parameter("model_variant", model_variant)
|
||||
|
||||
# Step 3: Loading the YOLO26 Model
|
||||
model = YOLO(f"{model_variant}.pt")
|
||||
|
||||
# Step 4: Setting Up Training Arguments
|
||||
args = dict(data="coco8.yaml", epochs=16)
|
||||
task.connect(args)
|
||||
|
||||
# Step 5: Initiating Model Training
|
||||
results = model.train(**args)
|
||||
```
|
||||
|
||||
Refer to our [Usage guide](#usage) for a detailed breakdown of these steps.
|
||||
|
||||
### Where can I view the results of my YOLO26 training in ClearML?
|
||||
|
||||
After running your YOLO26 training script with ClearML, you can view the results on the ClearML results page. The output will include a URL link to the ClearML dashboard, where you can track metrics, compare experiments, and monitor resource usage. For more details on how to view and interpret the results, check our section on [Viewing the ClearML Results Page](#viewing-the-clearml-results-page).
|
||||
@@ -0,0 +1,301 @@
|
||||
---
|
||||
comments: true
|
||||
description: Learn to simplify the logging of YOLO26 training with Comet. This guide covers installation, setup, real-time insights, and custom logging.
|
||||
keywords: YOLO26, Comet, Comet ML, logging, machine learning, training, model checkpoints, metrics, installation, configuration, real-time insights, custom logging
|
||||
---
|
||||
|
||||
# Elevating YOLO26 Training: Simplify Your Logging Process with Comet
|
||||
|
||||
Logging key training details such as parameters, metrics, image predictions, and model checkpoints is essential in [machine learning](https://www.ultralytics.com/glossary/machine-learning-ml)—it keeps your project transparent, your progress measurable, and your results repeatable.
|
||||
|
||||
<p align="center">
|
||||
<br>
|
||||
<iframe loading="lazy" width="720" height="405" src="https://www.youtube.com/embed/LPodYpvKkvI"
|
||||
title="YouTube video player" frameborder="0"
|
||||
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"
|
||||
allowfullscreen>
|
||||
</iframe>
|
||||
<br>
|
||||
<strong>Watch:</strong> How to Use Comet for Ultralytics YOLO Model Training Logs and Metrics 🚀
|
||||
</p>
|
||||
|
||||
[Ultralytics YOLO26](https://www.ultralytics.com/) seamlessly integrates with Comet (formerly Comet ML), efficiently capturing and optimizing every aspect of your YOLO26 [object detection](https://www.ultralytics.com/glossary/object-detection) model's training process. In this guide, we'll cover the installation process, Comet setup, real-time insights, custom logging, and offline usage, ensuring that your YOLO26 training is thoroughly documented and fine-tuned for outstanding results.
|
||||
|
||||
## Comet
|
||||
|
||||
<p align="center">
|
||||
<img width="640" src="https://www.comet.com/docs/v2/img/landing/home-hero.svg" alt="Comet ML experiment tracking dashboard">
|
||||
</p>
|
||||
|
||||
[Comet](https://www.comet.com/site/) is a platform for tracking, comparing, explaining, and optimizing machine learning models and experiments. It allows you to log metrics, parameters, media, and more during your model training and monitor your experiments through an aesthetically pleasing web interface. Comet helps data scientists iterate more rapidly, enhances transparency and reproducibility, and aids in the development of production models.
|
||||
|
||||
## Harnessing the Power of YOLO26 and Comet
|
||||
|
||||
By combining Ultralytics YOLO26 with Comet, you unlock a range of benefits. These include simplified experiment management, real-time insights for quick adjustments, flexible and tailored logging options, and the ability to log experiments offline when internet access is limited. This integration empowers you to make data-driven decisions, analyze performance metrics, and achieve exceptional results.
|
||||
|
||||
## Installation
|
||||
|
||||
To install the required packages, run:
|
||||
|
||||
!!! tip "Installation"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Install the required packages for YOLO26 and Comet
|
||||
pip install ultralytics comet_ml torch torchvision
|
||||
```
|
||||
|
||||
## Configuring Comet
|
||||
|
||||
After installing the required packages, you'll need to sign up, get a [Comet API Key](https://www.comet.com/signup), and configure it.
|
||||
|
||||
!!! tip "Configuring Comet"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Set your Comet API Key
|
||||
export COMET_API_KEY=YOUR_API_KEY
|
||||
```
|
||||
|
||||
Then, you can initialize your Comet project. Comet will automatically detect the API key and proceed with the setup.
|
||||
|
||||
!!! example "Initialize Comet project"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
import comet_ml
|
||||
|
||||
comet_ml.login(project_name="comet-example-yolo26-coco128")
|
||||
```
|
||||
|
||||
If you are using a Google Colab notebook, the code above will prompt you to enter your API key for initialization.
|
||||
|
||||
## Usage
|
||||
|
||||
Before diving into the usage instructions, be sure to check out the range of [YOLO26 models offered by Ultralytics](../models/yolo26.md). This will help you choose the most appropriate model for your project requirements.
|
||||
|
||||
!!! example "Usage"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load a model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Train the model
|
||||
results = model.train(
|
||||
data="coco8.yaml",
|
||||
project="comet-example-yolo26-coco128",
|
||||
batch=32,
|
||||
save_period=1,
|
||||
save_json=True,
|
||||
epochs=3,
|
||||
)
|
||||
```
|
||||
|
||||
After running the training code, Comet will create an experiment in your Comet workspace to track the run automatically. You will then be provided with a link to view the detailed logging of your [YOLO26 model's training](../modes/train.md) process.
|
||||
|
||||
Comet automatically logs the following data with no additional configuration: metrics such as mAP and loss, hyperparameters, model checkpoints, interactive confusion matrix, and image [bounding box](https://www.ultralytics.com/glossary/bounding-box) predictions.
|
||||
|
||||
## Understanding Your Model's Performance with Comet Visualizations
|
||||
|
||||
Let's dive into what you'll see on the Comet dashboard once your YOLO26 model begins training. The dashboard is where all the action happens, presenting a range of automatically logged information through visuals and statistics. Here's a quick tour:
|
||||
|
||||
**Experiment Panels**
|
||||
|
||||
The experiment panels section of the Comet dashboard organizes and presents the different runs and their metrics, such as segment mask loss, class loss, precision, and [mean average precision](https://www.ultralytics.com/glossary/mean-average-precision-map).
|
||||
|
||||
<p align="center">
|
||||
<img width="640" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/comet-ml-dashboard-overview.avif" alt="Comet ML experiment tracking dashboard">
|
||||
</p>
|
||||
|
||||
**Metrics**
|
||||
|
||||
In the metrics section, you have the option to examine the metrics in a tabular format as well, which is displayed in a dedicated pane as illustrated here.
|
||||
|
||||
<p align="center">
|
||||
<img width="640" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/comet-ml-metrics-tabular.avif" alt="Comet ML experiment tracking dashboard">
|
||||
</p>
|
||||
|
||||
**Interactive [Confusion Matrix](https://www.ultralytics.com/glossary/confusion-matrix)**
|
||||
|
||||
The confusion matrix, found in the Confusion Matrix tab, provides an interactive way to assess the model's classification [accuracy](https://www.ultralytics.com/glossary/accuracy). It details the correct and incorrect predictions, allowing you to understand the model's strengths and weaknesses.
|
||||
|
||||
<p align="center">
|
||||
<img width="640" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/comet-ml-interactive-confusion-matrix.avif" alt="Comet ML experiment tracking dashboard">
|
||||
</p>
|
||||
|
||||
**System Metrics**
|
||||
|
||||
Comet logs system metrics to help identify any bottlenecks in the training process. It includes metrics such as GPU utilization, GPU memory usage, CPU utilization, and RAM usage. These are essential for monitoring the efficiency of resource usage during model training.
|
||||
|
||||
<p align="center">
|
||||
<img width="640" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/comet-ml-system-metrics.avif" alt="Comet ML experiment tracking dashboard">
|
||||
</p>
|
||||
|
||||
## Customizing Comet Logging
|
||||
|
||||
Comet offers the flexibility to customize its logging behavior by setting environment variables. These configurations allow you to tailor Comet to your specific needs and preferences. Here are some helpful customization options:
|
||||
|
||||
### Logging Image Predictions
|
||||
|
||||
You can control the number of image predictions that Comet logs during your experiments. By default, Comet logs 100 image predictions from the validation set. However, you can change this number to better suit your requirements. For example, to log 200 image predictions, use the following code:
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
os.environ["COMET_MAX_IMAGE_PREDICTIONS"] = "200"
|
||||
```
|
||||
|
||||
### Batch Logging Interval
|
||||
|
||||
Comet allows you to specify how often batches of image predictions are logged. The `COMET_EVAL_BATCH_LOGGING_INTERVAL` environment variable controls this frequency. The default setting is 1, which logs predictions from every validation batch. You can adjust this value to log predictions at a different interval. For instance, setting it to 4 will log predictions from every fourth batch.
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
os.environ["COMET_EVAL_BATCH_LOGGING_INTERVAL"] = "4"
|
||||
```
|
||||
|
||||
### Disabling Confusion Matrix Logging
|
||||
|
||||
In some cases, you may not want to log the confusion matrix from your validation set after every [epoch](https://www.ultralytics.com/glossary/epoch). You can disable this feature by setting the `COMET_EVAL_LOG_CONFUSION_MATRIX` environment variable to "false." The confusion matrix will only be logged once, after the training is completed.
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
os.environ["COMET_EVAL_LOG_CONFUSION_MATRIX"] = "false"
|
||||
```
|
||||
|
||||
### Offline Logging
|
||||
|
||||
If you find yourself in a situation where internet access is limited, Comet provides an offline logging option. You can set the `COMET_MODE` environment variable to "offline" to enable this feature. Your experiment data will be saved locally in a directory that you can later upload to Comet when internet connectivity is available.
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
os.environ["COMET_MODE"] = "offline"
|
||||
```
|
||||
|
||||
## Summary
|
||||
|
||||
This guide has walked you through integrating Comet with Ultralytics' YOLO26. From installation to customization, you've learned to streamline experiment management, gain real-time insights, and adapt logging to your project's needs.
|
||||
|
||||
Explore [Comet's official YOLOv8 integration documentation](https://www.comet.com/docs/v2/integrations/third-party-tools/yolov8/), which also applies to YOLO26 projects.
|
||||
|
||||
Furthermore, if you're looking to dive deeper into the practical applications of YOLO26, specifically for [image segmentation](https://www.ultralytics.com/glossary/image-segmentation) tasks, this detailed guide on [fine-tuning YOLO26 with Comet](https://www.comet.com/site/blog/fine-tuning-yolov8-for-image-segmentation-with-comet/) offers valuable insights and step-by-step instructions to enhance your model's performance.
|
||||
|
||||
Additionally, to explore other exciting integrations with Ultralytics, check out the [integration guide page](../integrations/index.md), which offers a wealth of resources and information.
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I integrate Comet with Ultralytics YOLO26 for training?
|
||||
|
||||
To integrate Comet with Ultralytics YOLO26, follow these steps:
|
||||
|
||||
1. **Install the required packages**:
|
||||
|
||||
```bash
|
||||
pip install ultralytics comet_ml torch torchvision
|
||||
```
|
||||
|
||||
2. **Set up your Comet API Key**:
|
||||
|
||||
```bash
|
||||
export COMET_API_KEY=YOUR_API_KEY
|
||||
```
|
||||
|
||||
3. **Initialize your Comet project in your Python code**:
|
||||
|
||||
```python
|
||||
import comet_ml
|
||||
|
||||
comet_ml.login(project_name="comet-example-yolo26-coco128")
|
||||
```
|
||||
|
||||
4. **Train your YOLO26 model and log metrics**:
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
model = YOLO("yolo26n.pt")
|
||||
results = model.train(
|
||||
data="coco8.yaml",
|
||||
project="comet-example-yolo26-coco128",
|
||||
batch=32,
|
||||
save_period=1,
|
||||
save_json=True,
|
||||
epochs=3,
|
||||
)
|
||||
```
|
||||
|
||||
For more detailed instructions, refer to the [Comet configuration section](#configuring-comet).
|
||||
|
||||
### What are the benefits of using Comet with YOLO26?
|
||||
|
||||
By integrating Ultralytics YOLO26 with Comet, you can:
|
||||
|
||||
- **Monitor real-time insights**: Get instant feedback on your training results, allowing for quick adjustments.
|
||||
- **Log extensive metrics**: Automatically capture essential metrics such as mAP, loss, hyperparameters, and model checkpoints.
|
||||
- **Track experiments offline**: Log your training runs locally when internet access is unavailable.
|
||||
- **Compare different training runs**: Use the interactive Comet dashboard to analyze and compare multiple experiments.
|
||||
|
||||
By leveraging these features, you can optimize your machine learning workflows for better performance and reproducibility. For more information, visit the [Comet integration guide](../integrations/index.md).
|
||||
|
||||
### How do I customize the logging behavior of Comet during YOLO26 training?
|
||||
|
||||
Comet allows for extensive customization of its logging behavior using environment variables:
|
||||
|
||||
- **Change the number of image predictions logged**:
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
os.environ["COMET_MAX_IMAGE_PREDICTIONS"] = "200"
|
||||
```
|
||||
|
||||
- **Adjust batch logging interval**:
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
os.environ["COMET_EVAL_BATCH_LOGGING_INTERVAL"] = "4"
|
||||
```
|
||||
|
||||
- **Disable confusion matrix logging**:
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
os.environ["COMET_EVAL_LOG_CONFUSION_MATRIX"] = "false"
|
||||
```
|
||||
|
||||
Refer to the [Customizing Comet Logging](#customizing-comet-logging) section for more customization options.
|
||||
|
||||
### How do I view detailed metrics and visualizations of my YOLO26 training on Comet?
|
||||
|
||||
Once your YOLO26 model starts training, you can access a wide range of metrics and visualizations on the Comet dashboard. Key features include:
|
||||
|
||||
- **Experiment Panels**: View different runs and their metrics, including segment mask loss, class loss, and mean average [precision](https://www.ultralytics.com/glossary/precision).
|
||||
- **Metrics**: Examine metrics in tabular format for detailed analysis.
|
||||
- **Interactive Confusion Matrix**: Assess classification accuracy with an interactive confusion matrix.
|
||||
- **System Metrics**: Monitor GPU and CPU utilization, memory usage, and other system metrics.
|
||||
|
||||
For a detailed overview of these features, visit the [Understanding Your Model's Performance with Comet Visualizations](#understanding-your-models-performance-with-comet-visualizations) section.
|
||||
|
||||
### Can I use Comet for offline logging when training YOLO26 models?
|
||||
|
||||
Yes, you can enable offline logging in Comet by setting the `COMET_MODE` environment variable to "offline":
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
os.environ["COMET_MODE"] = "offline"
|
||||
```
|
||||
|
||||
This feature allows you to log your experiment data locally, which can later be uploaded to Comet when internet connectivity is available. This is particularly useful when working in environments with limited internet access. For more details, refer to the [Offline Logging](#offline-logging) section.
|
||||
@@ -0,0 +1,243 @@
|
||||
---
|
||||
comments: true
|
||||
description: Learn how to export YOLO26 models to CoreML for optimized, on-device machine learning on iOS and macOS. Follow step-by-step instructions.
|
||||
keywords: CoreML export, YOLO26 models, CoreML conversion, Ultralytics, iOS object detection, macOS machine learning, AI deployment, machine learning integration
|
||||
---
|
||||
|
||||
# CoreML Export for YOLO26 Models
|
||||
|
||||
Deploying [computer vision](https://www.ultralytics.com/glossary/computer-vision-cv) models on Apple devices like iPhones and Macs requires a format that ensures seamless performance.
|
||||
|
||||
<p align="center">
|
||||
<br>
|
||||
<iframe loading="lazy" width="720" height="405" src="https://www.youtube.com/embed/hfSK3Mk5P0I"
|
||||
title="YouTube video player" frameborder="0"
|
||||
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"
|
||||
allowfullscreen>
|
||||
</iframe>
|
||||
<br>
|
||||
<strong>Watch:</strong> How to Export Ultralytics YOLO26 to CoreML for 2x Fast Inference on Apple Devices 🚀
|
||||
</p>
|
||||
|
||||
The CoreML export format allows you to optimize your [Ultralytics YOLO26](https://github.com/ultralytics/ultralytics) models for efficient [object detection](https://www.ultralytics.com/glossary/object-detection) in iOS and macOS applications. In this guide, we'll walk you through the steps for converting your models to the CoreML format, making it easier for your models to perform well on Apple devices.
|
||||
|
||||
## CoreML
|
||||
|
||||
<p align="center">
|
||||
<img width="100%" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/coreml-overview.avif" alt="Apple CoreML deployment pipeline">
|
||||
</p>
|
||||
|
||||
[CoreML](https://developer.apple.com/documentation/coreml) is Apple's foundational machine learning framework that builds upon Accelerate, BNNS, and Metal Performance Shaders. It provides a machine-learning model format that seamlessly integrates into iOS applications and supports tasks such as image analysis, [natural language processing](https://www.ultralytics.com/glossary/natural-language-processing-nlp), audio-to-text conversion, and sound analysis.
|
||||
|
||||
Applications can take advantage of Core ML without the need to have a network connection or API calls because the Core ML framework works using on-device computing. This means model inference can be performed locally on the user's device.
|
||||
|
||||
## Key Features of CoreML Models
|
||||
|
||||
Apple's CoreML framework offers robust features for on-device machine learning. Here are the key features that make CoreML a powerful tool for developers:
|
||||
|
||||
- **Comprehensive Model Support**: Converts and runs models from popular frameworks like TensorFlow, [PyTorch](https://www.ultralytics.com/glossary/pytorch), scikit-learn, XGBoost, and LibSVM.
|
||||
|
||||
<p align="center">
|
||||
<img width="100%" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/coreml-supported-models.avif" alt="CoreML supported deep learning frameworks">
|
||||
</p>
|
||||
|
||||
- **On-device [Machine Learning](https://www.ultralytics.com/glossary/machine-learning-ml)**: Ensures data privacy and swift processing by executing models directly on the user's device, eliminating the need for network connectivity.
|
||||
|
||||
- **Performance and Optimization**: Uses the device's CPU, GPU, and Neural Engine for optimal performance with minimal power and memory usage. Offers tools for model compression and optimization while maintaining [accuracy](https://www.ultralytics.com/glossary/accuracy).
|
||||
|
||||
- **Ease of Integration**: Provides a unified format for various model types and a user-friendly API for seamless integration into apps. Supports domain-specific tasks through frameworks like Vision and Natural Language.
|
||||
|
||||
- **Advanced Features**: Includes on-device training capabilities for personalized experiences, asynchronous predictions for interactive ML experiences, and model inspection and validation tools.
|
||||
|
||||
## CoreML Deployment Options
|
||||
|
||||
Before we look at the code for exporting YOLO26 models to the CoreML format, let's understand where CoreML models are usually used.
|
||||
|
||||
CoreML offers various deployment options for machine learning models, including:
|
||||
|
||||
- **On-Device Deployment**: This method directly integrates CoreML models into your iOS app. It's particularly advantageous for ensuring low latency, enhanced privacy (since data remains on the device), and offline functionality. This approach, however, may be limited by the device's hardware capabilities, especially for larger and more complex models, and it can be executed in the following two ways:
|
||||
- **Embedded Models**: These models are included in the app bundle and are immediately accessible. They are ideal for small models that do not require frequent updates.
|
||||
|
||||
- **Downloaded Models**: These models are fetched from a server as needed. This approach is suitable for larger models or those needing regular updates. It helps keep the app bundle size smaller.
|
||||
|
||||
- **Cloud-Based Deployment**: CoreML models are hosted on servers and accessed by the iOS app through API requests. This scalable and flexible option enables easy model updates without app revisions. It's ideal for complex models or large-scale apps requiring regular updates. However, it does require an internet connection and may pose latency and security issues.
|
||||
|
||||
## Exporting YOLO26 Models to CoreML
|
||||
|
||||
Exporting YOLO26 to CoreML enables optimized, on-device machine learning performance within Apple's ecosystem, offering benefits in terms of efficiency, security, and seamless integration with iOS, macOS, watchOS, and tvOS platforms.
|
||||
|
||||
### Installation
|
||||
|
||||
To install the required package, run:
|
||||
|
||||
!!! tip "Installation"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Install the required package for YOLO26
|
||||
pip install ultralytics
|
||||
```
|
||||
|
||||
For detailed instructions and best practices related to the installation process, check our [YOLO26 Installation guide](../quickstart.md). While installing the required packages for YOLO26, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips.
|
||||
|
||||
### Usage
|
||||
|
||||
Before diving into the usage instructions, be sure to check out the range of [YOLO26 models offered by Ultralytics](../models/index.md). This will help you choose the most appropriate model for your project requirements.
|
||||
|
||||
!!! example "Usage"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load the YOLO26 model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Export the model to CoreML format
|
||||
model.export(format="coreml") # creates 'yolo26n.mlpackage'
|
||||
|
||||
# Load the exported CoreML model
|
||||
coreml_model = YOLO("yolo26n.mlpackage")
|
||||
|
||||
# Run inference
|
||||
results = coreml_model("https://ultralytics.com/images/bus.jpg")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Export a YOLO26n PyTorch model to CoreML format
|
||||
yolo export model=yolo26n.pt format=coreml # creates 'yolo26n.mlpackage'
|
||||
|
||||
# Run inference with the exported model
|
||||
yolo predict model=yolo26n.mlpackage source='https://ultralytics.com/images/bus.jpg'
|
||||
```
|
||||
|
||||
### Export Arguments
|
||||
|
||||
| Argument | Type | Default | Description |
|
||||
| -------- | ---------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `format` | `str` | `'coreml'` | Target format for the exported model, defining compatibility with various deployment environments. |
|
||||
| `imgsz` | `int` or `tuple` | `640` | Desired image size for the model input. Can be an integer for square images or a tuple `(height, width)` for specific dimensions. |
|
||||
| `half` | `bool` | `False` | Enables FP16 (half-precision) quantization, reducing model size and potentially speeding up inference on supported hardware. |
|
||||
| `int8` | `bool` | `False` | Activates INT8 quantization, further compressing the model and speeding up inference with minimal [accuracy](https://www.ultralytics.com/glossary/accuracy) loss, primarily for edge devices. |
|
||||
| `nms` | `bool` | `False` | Adds Non-Maximum Suppression (NMS), essential for accurate and efficient detection post-processing. |
|
||||
| `batch` | `int` | `1` | Specifies export model batch inference size or the max number of images the exported model will process concurrently in `predict` mode. |
|
||||
| `device` | `str` | `None` | Specifies the device for exporting: GPU (`device=0`), CPU (`device=cpu`), MPS for Apple silicon (`device=mps`). |
|
||||
|
||||
!!! tip
|
||||
|
||||
Please make sure to use a macOS or x86 Linux machine when exporting to CoreML.
|
||||
|
||||
For more details about the export process, visit the [Ultralytics documentation page on exporting](../modes/export.md).
|
||||
|
||||
## Deploying Exported YOLO26 CoreML Models
|
||||
|
||||
Having successfully exported your Ultralytics YOLO26 models to CoreML, the next critical phase is deploying these models effectively. For detailed guidance on deploying CoreML models in various environments, check out these resources:
|
||||
|
||||
- **[CoreML Tools](https://apple.github.io/coremltools/docs-guides/)**: This guide includes instructions and examples to convert models from [TensorFlow](https://www.ultralytics.com/glossary/tensorflow), PyTorch, and other libraries to Core ML.
|
||||
|
||||
- **[ML and Vision](https://developer.apple.com/videos/)**: A collection of comprehensive videos that cover various aspects of using and implementing CoreML models.
|
||||
|
||||
- **[Integrating a Core ML Model into Your App](https://developer.apple.com/documentation/coreml/integrating-a-core-ml-model-into-your-app)**: A comprehensive guide on integrating a CoreML model into an iOS application, detailing steps from preparing the model to implementing it in the app for various functionalities.
|
||||
|
||||
## Summary
|
||||
|
||||
In this guide, we went over how to export Ultralytics YOLO26 models to CoreML format. By following the steps outlined in this guide, you can ensure maximum compatibility and performance when exporting YOLO26 models to CoreML.
|
||||
|
||||
For further details on usage, visit the [CoreML official documentation](https://developer.apple.com/documentation/coreml).
|
||||
|
||||
Also, if you'd like to know more about other Ultralytics YOLO26 integrations, visit our [integration guide page](../integrations/index.md). You'll find plenty of valuable resources and insights there.
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I export YOLO26 models to CoreML format?
|
||||
|
||||
To export your [Ultralytics YOLO26](https://github.com/ultralytics/ultralytics) models to CoreML format, you'll first need to ensure you have the `ultralytics` package installed. You can install it using:
|
||||
|
||||
!!! example "Installation"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
pip install ultralytics
|
||||
```
|
||||
|
||||
Next, you can export the model using the following Python or CLI commands:
|
||||
|
||||
!!! example "Usage"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
model = YOLO("yolo26n.pt")
|
||||
model.export(format="coreml")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
yolo export model=yolo26n.pt format=coreml
|
||||
```
|
||||
|
||||
For further details, refer to the [Exporting YOLO26 Models to CoreML](../modes/export.md) section of our documentation.
|
||||
|
||||
### What are the benefits of using CoreML for deploying YOLO26 models?
|
||||
|
||||
CoreML provides numerous advantages for deploying [Ultralytics YOLO26](https://github.com/ultralytics/ultralytics) models on Apple devices:
|
||||
|
||||
- **On-device Processing**: Enables local model inference on devices, ensuring [data privacy](https://www.ultralytics.com/glossary/data-privacy) and minimizing latency.
|
||||
- **Performance Optimization**: Leverages the full potential of the device's CPU, GPU, and Neural Engine, optimizing both speed and efficiency.
|
||||
- **Ease of Integration**: Offers a seamless integration experience with Apple's ecosystems, including iOS, macOS, watchOS, and tvOS.
|
||||
- **Versatility**: Supports a wide range of machine learning tasks such as image analysis, audio processing, and natural language processing using the CoreML framework.
|
||||
|
||||
For more details on integrating your CoreML model into an iOS app, check out the guide on [Integrating a Core ML Model into Your App](https://developer.apple.com/documentation/coreml/integrating-a-core-ml-model-into-your-app).
|
||||
|
||||
### What are the deployment options for YOLO26 models exported to CoreML?
|
||||
|
||||
Once you export your YOLO26 model to CoreML format, you have multiple deployment options:
|
||||
|
||||
1. **On-Device Deployment**: Directly integrate CoreML models into your app for enhanced privacy and offline functionality. This can be done as:
|
||||
- **Embedded Models**: Included in the app bundle, accessible immediately.
|
||||
- **Downloaded Models**: Fetched from a server as needed, keeping the app bundle size smaller.
|
||||
|
||||
2. **Cloud-Based Deployment**: Host CoreML models on servers and access them via API requests. This approach supports easier updates and can handle more complex models.
|
||||
|
||||
For detailed guidance on deploying CoreML models, refer to [CoreML Deployment Options](#coreml-deployment-options).
|
||||
|
||||
### How does CoreML ensure optimized performance for YOLO26 models?
|
||||
|
||||
CoreML ensures optimized performance for [Ultralytics YOLO26](https://github.com/ultralytics/ultralytics) models by utilizing various optimization techniques:
|
||||
|
||||
- **Hardware Acceleration**: Uses the device's CPU, GPU, and Neural Engine for efficient computation.
|
||||
- **Model Compression**: Provides tools for compressing models to reduce their footprint without compromising accuracy.
|
||||
- **Adaptive Inference**: Adjusts inference based on the device's capabilities to maintain a balance between speed and performance.
|
||||
|
||||
For more information on performance optimization, visit the [CoreML official documentation](https://developer.apple.com/documentation/coreml).
|
||||
|
||||
### Can I run inference directly with the exported CoreML model?
|
||||
|
||||
Yes, you can run inference directly using the exported CoreML model. Below are the commands for Python and CLI:
|
||||
|
||||
!!! example "Running Inference"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
coreml_model = YOLO("yolo26n.mlpackage")
|
||||
results = coreml_model("https://ultralytics.com/images/bus.jpg")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
yolo predict model=yolo26n.mlpackage source='https://ultralytics.com/images/bus.jpg'
|
||||
```
|
||||
|
||||
For additional information, refer to the [Usage section](#usage) of the CoreML export guide.
|
||||
@@ -0,0 +1,278 @@
|
||||
---
|
||||
comments: true
|
||||
description: Unlock seamless YOLO26 tracking with DVCLive. Discover how to log, visualize, and analyze experiments for optimized ML model performance.
|
||||
keywords: YOLO26, DVCLive, experiment tracking, machine learning, model training, data visualization, Git integration
|
||||
---
|
||||
|
||||
# Advanced YOLO26 Experiment Tracking with DVCLive
|
||||
|
||||
Experiment tracking in [machine learning](https://www.ultralytics.com/glossary/machine-learning-ml) is critical to model development and evaluation. It involves recording and analyzing various parameters, metrics, and outcomes from numerous training runs. This process is essential for understanding model performance and making data-driven decisions to refine and optimize models.
|
||||
|
||||
Integrating DVCLive with [Ultralytics YOLO26](https://www.ultralytics.com/) transforms the way experiments are tracked and managed. This integration offers a seamless solution for automatically logging key experiment details, comparing results across different runs, and visualizing data for in-depth analysis. In this guide, we'll understand how DVCLive can be used to streamline the process.
|
||||
|
||||
## DVCLive
|
||||
|
||||
<p align="center">
|
||||
<img width="640" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/dvclive-overview.avif" alt="DVCLive experiment tracking integration">
|
||||
</p>
|
||||
|
||||
[DVCLive](https://doc.dvc.org/dvclive), developed by DVC, is an innovative open-source tool for experiment tracking in machine learning. Integrating seamlessly with Git and DVC, it automates the logging of crucial experiment data like model parameters and training metrics. Designed for simplicity, DVCLive enables effortless comparison and analysis of multiple runs, enhancing the efficiency of machine learning projects with intuitive [data visualization](https://www.ultralytics.com/glossary/data-visualization) and analysis tools.
|
||||
|
||||
## YOLO26 Training with DVCLive
|
||||
|
||||
YOLO26 training sessions can be effectively monitored with DVCLive. Additionally, DVC provides integral features for visualizing these experiments, including the generation of a report that enables the comparison of metric plots across all tracked experiments, offering a comprehensive view of the training process.
|
||||
|
||||
## Installation
|
||||
|
||||
To install the required packages, run:
|
||||
|
||||
!!! tip "Installation"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Install the required packages for YOLO26 and DVCLive
|
||||
pip install ultralytics dvclive
|
||||
```
|
||||
|
||||
For detailed instructions and best practices related to the installation process, be sure to check our [YOLO26 Installation guide](../quickstart.md). While installing the required packages for YOLO26, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips.
|
||||
|
||||
## Configuring DVCLive
|
||||
|
||||
Once you have installed the necessary packages, the next step is to set up and configure your environment with the required credentials. This setup ensures a smooth integration of DVCLive into your existing workflow.
|
||||
|
||||
Begin by initializing a Git repository, as Git plays a crucial role in version control for both your code and DVCLive configurations.
|
||||
|
||||
!!! tip "Initial Environment Setup"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Initialize a Git repository
|
||||
git init -q
|
||||
|
||||
# Configure Git with your details
|
||||
git config --local user.email "you@example.com"
|
||||
git config --local user.name "Your Name"
|
||||
|
||||
# Initialize DVCLive in your project
|
||||
dvc init -q
|
||||
|
||||
# Commit the DVCLive setup to your Git repository
|
||||
git commit -m "DVC init"
|
||||
```
|
||||
|
||||
In these commands, ensure you replace "you@example.com" with the email address associated with your Git account, and "Your Name" with your Git account username.
|
||||
|
||||
## Usage
|
||||
|
||||
Before diving into the usage instructions, be sure to check out the range of [YOLO26 models offered by Ultralytics](../models/index.md). This will help you choose the most appropriate model for your project requirements.
|
||||
|
||||
### Training YOLO26 Models with DVCLive
|
||||
|
||||
Start by running your YOLO26 training sessions. You can use different model configurations and training parameters to suit your project needs. For instance:
|
||||
|
||||
```bash
|
||||
# Example training commands for YOLO26 with varying configurations
|
||||
yolo train model=yolo26n.pt data=coco8.yaml epochs=5 imgsz=512
|
||||
yolo train model=yolo26n.pt data=coco8.yaml epochs=5 imgsz=640
|
||||
```
|
||||
|
||||
Adjust the model, data, [epochs](https://www.ultralytics.com/glossary/epoch), and imgsz parameters according to your specific requirements. For a detailed understanding of the model training process and best practices, refer to our [YOLO26 Model Training guide](../modes/train.md).
|
||||
|
||||
### Monitoring Experiments with DVCLive
|
||||
|
||||
DVCLive enhances the training process by enabling the tracking and visualization of key metrics. When installed, Ultralytics YOLO26 automatically integrates with DVCLive for experiment tracking, which you can later analyze for performance insights. For a comprehensive understanding of the specific performance metrics used during training, be sure to explore [our detailed guide on performance metrics](../guides/yolo-performance-metrics.md).
|
||||
|
||||
### Analyzing Results
|
||||
|
||||
After your YOLO26 training sessions are complete, you can leverage DVCLive's powerful visualization tools for in-depth analysis of the results. DVCLive's integration ensures that all training metrics are systematically logged, facilitating a comprehensive evaluation of your model's performance.
|
||||
|
||||
To start the analysis, you can extract the experiment data using DVC's API and process it with Pandas for easier handling and visualization:
|
||||
|
||||
```python
|
||||
import dvc.api
|
||||
import pandas as pd
|
||||
|
||||
# Define the columns of interest
|
||||
columns = ["Experiment", "epochs", "imgsz", "model", "metrics.mAP50-95(B)"]
|
||||
|
||||
# Retrieve experiment data
|
||||
df = pd.DataFrame(dvc.api.exp_show(), columns=columns)
|
||||
|
||||
# Clean the data
|
||||
df.dropna(inplace=True)
|
||||
df.reset_index(drop=True, inplace=True)
|
||||
|
||||
# Display the DataFrame
|
||||
print(df)
|
||||
```
|
||||
|
||||
The output of the code snippet above provides a clear tabular view of the different experiments conducted with YOLO26 models. Each row represents a different training run, detailing the experiment's name, the number of epochs, image size (imgsz), the specific model used, and the mAP50-95(B) metric. This metric is crucial for evaluating the model's [accuracy](https://www.ultralytics.com/glossary/accuracy), with higher values indicating better performance.
|
||||
|
||||
#### Visualizing Results with Plotly
|
||||
|
||||
For a more interactive and visual analysis of your experiment results, you can use Plotly's parallel coordinates plot. This type of plot is particularly useful for understanding the relationships and trade-offs between different parameters and metrics.
|
||||
|
||||
```python
|
||||
from plotly.express import parallel_coordinates
|
||||
|
||||
# Create a parallel coordinates plot
|
||||
fig = parallel_coordinates(df, columns, color="metrics.mAP50-95(B)")
|
||||
|
||||
# Display the plot
|
||||
fig.show()
|
||||
```
|
||||
|
||||
The output of the code snippet above generates a plot that will visually represent the relationships between epochs, image size, model type, and their corresponding mAP50-95(B) scores, enabling you to spot trends and patterns in your experiment data.
|
||||
|
||||
#### Generating Comparative Visualizations with DVC
|
||||
|
||||
DVC provides a useful command to generate comparative plots for your experiments. This can be especially helpful to compare the performance of different models over various training runs.
|
||||
|
||||
```bash
|
||||
# Generate DVC comparative plots
|
||||
dvc plots diff $(dvc exp list --names-only)
|
||||
```
|
||||
|
||||
After executing this command, DVC generates plots comparing the metrics across different experiments, which are saved as HTML files. Below is an example image illustrating typical plots generated by this process. The image showcases various graphs, including those representing mAP, [recall](https://www.ultralytics.com/glossary/recall), [precision](https://www.ultralytics.com/glossary/precision), loss values, and more, providing a visual overview of key performance metrics:
|
||||
|
||||
<p align="center">
|
||||
<img width="640" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/dvclive-comparative-plots.avif" alt="DVCLive training metrics comparison plots">
|
||||
</p>
|
||||
|
||||
### Displaying DVC Plots
|
||||
|
||||
If you are using a Jupyter Notebook and you want to display the generated DVC plots, you can use the IPython display functionality.
|
||||
|
||||
```python
|
||||
from IPython.display import HTML
|
||||
|
||||
# Display the DVC plots as HTML
|
||||
HTML(filename="./dvc_plots/index.html")
|
||||
```
|
||||
|
||||
This code will render the HTML file containing the DVC plots directly in your Jupyter Notebook, providing an easy and convenient way to analyze the visualized experiment data.
|
||||
|
||||
### Making Data-Driven Decisions
|
||||
|
||||
Use the insights gained from these visualizations to make informed decisions about model optimizations, [hyperparameter tuning](https://www.ultralytics.com/glossary/hyperparameter-tuning), and other modifications to enhance your model's performance.
|
||||
|
||||
### Iterating on Experiments
|
||||
|
||||
Based on your analysis, iterate on your experiments. Adjust model configurations, training parameters, or even the data inputs, and repeat the training and analysis process. This iterative approach is key to refining your model for the best possible performance.
|
||||
|
||||
## Summary
|
||||
|
||||
This guide has led you through the process of integrating DVCLive with Ultralytics' YOLO26. You have learned how to harness the power of DVCLive for detailed experiment monitoring, effective visualization, and insightful analysis in your machine learning endeavors.
|
||||
|
||||
For further details on usage, visit [DVCLive's official documentation](https://doc.dvc.org/dvclive/ml-frameworks/yolo).
|
||||
|
||||
Additionally, explore more integrations and capabilities of Ultralytics by visiting the [Ultralytics integration guide page](../integrations/index.md), which is a collection of great resources and insights.
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I integrate DVCLive with Ultralytics YOLO26 for experiment tracking?
|
||||
|
||||
Integrating DVCLive with Ultralytics YOLO26 is straightforward. Start by installing the necessary packages:
|
||||
|
||||
!!! example "Installation"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
pip install ultralytics dvclive
|
||||
```
|
||||
|
||||
Next, initialize a Git repository and configure DVCLive in your project:
|
||||
|
||||
!!! example "Initial Environment Setup"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
git init -q
|
||||
git config --local user.email "you@example.com"
|
||||
git config --local user.name "Your Name"
|
||||
dvc init -q
|
||||
git commit -m "DVC init"
|
||||
```
|
||||
|
||||
Follow our [YOLO26 Installation guide](../quickstart.md) for detailed setup instructions.
|
||||
|
||||
### Why should I use DVCLive for tracking YOLO26 experiments?
|
||||
|
||||
Using DVCLive with YOLO26 provides several advantages, such as:
|
||||
|
||||
- **Automated Logging**: DVCLive automatically records key experiment details like model parameters and metrics.
|
||||
- **Easy Comparison**: Facilitates comparison of results across different runs.
|
||||
- **Visualization Tools**: Leverages DVCLive's robust data visualization capabilities for in-depth analysis.
|
||||
|
||||
For further details, refer to our guide on [YOLO26 Model Training](../modes/train.md) and [YOLO Performance Metrics](../guides/yolo-performance-metrics.md) to maximize your experiment tracking efficiency.
|
||||
|
||||
### How can DVCLive improve my results analysis for YOLO26 training sessions?
|
||||
|
||||
After completing your YOLO26 training sessions, DVCLive helps in visualizing and analyzing the results effectively. Example code for loading and displaying experiment data:
|
||||
|
||||
```python
|
||||
import dvc.api
|
||||
import pandas as pd
|
||||
|
||||
# Define columns of interest
|
||||
columns = ["Experiment", "epochs", "imgsz", "model", "metrics.mAP50-95(B)"]
|
||||
|
||||
# Retrieve experiment data
|
||||
df = pd.DataFrame(dvc.api.exp_show(), columns=columns)
|
||||
|
||||
# Clean data
|
||||
df.dropna(inplace=True)
|
||||
df.reset_index(drop=True, inplace=True)
|
||||
|
||||
# Display DataFrame
|
||||
print(df)
|
||||
```
|
||||
|
||||
To visualize results interactively, use Plotly's parallel coordinates plot:
|
||||
|
||||
```python
|
||||
from plotly.express import parallel_coordinates
|
||||
|
||||
fig = parallel_coordinates(df, columns, color="metrics.mAP50-95(B)")
|
||||
fig.show()
|
||||
```
|
||||
|
||||
Refer to our guide on [YOLO26 Training with DVCLive](#yolo26-training-with-dvclive) for more examples and best practices.
|
||||
|
||||
### What are the steps to configure my environment for DVCLive and YOLO26 integration?
|
||||
|
||||
To configure your environment for a smooth integration of DVCLive and YOLO26, follow these steps:
|
||||
|
||||
1. **Install Required Packages**: Use `pip install ultralytics dvclive`.
|
||||
2. **Initialize Git Repository**: Run `git init -q`.
|
||||
3. **Setup DVCLive**: Execute `dvc init -q`.
|
||||
4. **Commit to Git**: Use `git commit -m "DVC init"`.
|
||||
|
||||
These steps ensure proper version control and setup for experiment tracking. For in-depth configuration details, visit our [Configuration guide](../quickstart.md).
|
||||
|
||||
### How do I visualize YOLO26 experiment results using DVCLive?
|
||||
|
||||
DVCLive offers powerful tools to visualize the results of YOLO26 experiments. Here's how you can generate comparative plots:
|
||||
|
||||
!!! example "Generate Comparative Plots"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
dvc plots diff $(dvc exp list --names-only)
|
||||
```
|
||||
|
||||
To display these plots in a Jupyter Notebook, use:
|
||||
|
||||
```python
|
||||
from IPython.display import HTML
|
||||
|
||||
# Display plots as HTML
|
||||
HTML(filename="./dvc_plots/index.html")
|
||||
```
|
||||
|
||||
These visualizations help identify trends and optimize model performance. Check our detailed guides on [YOLO26 Experiment Analysis](#analyzing-results) for comprehensive steps and examples.
|
||||
@@ -0,0 +1,197 @@
|
||||
---
|
||||
comments: true
|
||||
description: Learn how to export YOLO26 models to TFLite Edge TPU format for high-speed, low-power inferencing on mobile and embedded devices.
|
||||
keywords: YOLO26, TFLite Edge TPU, TensorFlow Lite, model export, machine learning, edge computing, neural networks, Ultralytics
|
||||
---
|
||||
|
||||
# Learn to Export to TFLite Edge TPU Format From YOLO26 Model
|
||||
|
||||
Deploying computer vision models on devices with limited computational power, such as mobile or embedded systems, can be tricky. Using a model format that is optimized for faster performance simplifies the process. The [TensorFlow Lite](https://ai.google.dev/edge/litert) [Edge TPU](https://gweb-coral-full.uc.r.appspot.com/docs/edgetpu/models-intro/) or TFLite Edge TPU model format is designed to use minimal power while delivering fast performance for neural networks.
|
||||
|
||||
The export to TFLite Edge TPU format feature allows you to optimize your [Ultralytics YOLO26](https://github.com/ultralytics/ultralytics) models for high-speed and low-power inferencing. In this guide, we'll walk you through converting your models to the TFLite Edge TPU format, making it easier for your models to perform well on various mobile and embedded devices.
|
||||
|
||||
## Why Should You Export to TFLite Edge TPU?
|
||||
|
||||
Exporting models to [TensorFlow](https://www.ultralytics.com/glossary/tensorflow) Edge TPU makes [machine learning](https://www.ultralytics.com/glossary/machine-learning-ml) tasks fast and efficient. This technology suits applications with limited power, computing resources, and connectivity. The Edge TPU is a hardware accelerator by Google. It speeds up TensorFlow Lite models on edge devices. The image below shows an example of the process involved.
|
||||
|
||||
<p align="center">
|
||||
<img width="100%" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/tflite-edge-tpu-compile-workflow.avif" alt="TensorFlow Lite Edge TPU compilation workflow">
|
||||
</p>
|
||||
|
||||
The Edge TPU works with quantized models. Quantization makes models smaller and faster without losing much [accuracy](https://www.ultralytics.com/glossary/accuracy). It is ideal for the limited resources of edge computing, allowing applications to respond quickly by reducing latency and allowing for quick data processing locally, without cloud dependency. Local processing also keeps user data private and secure since it's not sent to a remote server.
|
||||
|
||||
## Key Features of TFLite Edge TPU
|
||||
|
||||
Here are the key features that make TFLite Edge TPU a great model format choice for developers:
|
||||
|
||||
- **Optimized Performance on Edge Devices**: The TFLite Edge TPU achieves high-speed neural networking performance through quantization, model optimization, hardware acceleration, and compiler optimization. Its minimalistic architecture contributes to its smaller size and cost-efficiency.
|
||||
|
||||
- **High Computational Throughput**: TFLite Edge TPU combines specialized hardware acceleration and efficient runtime execution to achieve high computational throughput. It is well-suited for deploying machine learning models with stringent performance requirements on edge devices.
|
||||
|
||||
- **Efficient Matrix Computations**: The TensorFlow Edge TPU is optimized for matrix operations, which are crucial for [neural network](https://www.ultralytics.com/glossary/neural-network-nn) computations. This efficiency is key in machine learning models, particularly those requiring numerous and complex matrix multiplications and transformations.
|
||||
|
||||
## Deployment Options with TFLite Edge TPU
|
||||
|
||||
Before we jump into how to export YOLO26 models to the TFLite Edge TPU format, let's understand where TFLite Edge TPU models are usually used.
|
||||
|
||||
TFLite Edge TPU offers various deployment options for machine learning models, including:
|
||||
|
||||
- **On-Device Deployment**: TensorFlow Edge TPU models can be directly deployed on mobile and embedded devices. On-device deployment allows the models to execute directly on the hardware, eliminating the need for cloud connectivity, either by embedding the model in the application bundle or downloading it on demand.
|
||||
|
||||
- **Edge Computing with Cloud TensorFlow TPUs**: In scenarios where edge devices have limited processing capabilities, TensorFlow Edge TPUs can offload inference tasks to cloud servers equipped with TPUs.
|
||||
|
||||
- **Hybrid Deployment**: A hybrid approach combines on-device and cloud deployment and offers a versatile and scalable solution for deploying machine learning models. Advantages include on-device processing for quick responses and [cloud computing](https://www.ultralytics.com/glossary/cloud-computing) for more complex computations.
|
||||
|
||||
## Exporting YOLO26 Models to TFLite Edge TPU
|
||||
|
||||
You can expand model compatibility and deployment flexibility by converting YOLO26 models to TensorFlow Edge TPU.
|
||||
|
||||
### Installation
|
||||
|
||||
To install the required package, run:
|
||||
|
||||
!!! tip "Installation"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Install the required package for YOLO26
|
||||
pip install ultralytics
|
||||
```
|
||||
|
||||
For detailed instructions and best practices related to the installation process, check our [Ultralytics Installation guide](../quickstart.md). While installing the required packages for YOLO26, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips.
|
||||
|
||||
### Usage
|
||||
|
||||
All [Ultralytics YOLO26 models](../models/index.md) are designed to support export out of the box, making it easy to integrate them into your preferred deployment workflow. You can [view the full list of supported export formats and configuration options](../modes/export.md) to choose the best setup for your application.
|
||||
|
||||
!!! example "Usage"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load the YOLO26 model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Export the model to TFLite Edge TPU format
|
||||
model.export(format="edgetpu") # creates 'yolo26n_full_integer_quant_edgetpu.tflite'
|
||||
|
||||
# Load the exported TFLite Edge TPU model
|
||||
edgetpu_model = YOLO("yolo26n_full_integer_quant_edgetpu.tflite")
|
||||
|
||||
# Run inference
|
||||
results = edgetpu_model("https://ultralytics.com/images/bus.jpg")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Export a YOLO26n PyTorch model to TFLite Edge TPU format
|
||||
yolo export model=yolo26n.pt format=edgetpu # creates 'yolo26n_full_integer_quant_edgetpu.tflite'
|
||||
|
||||
# Run inference with the exported model
|
||||
yolo predict model=yolo26n_full_integer_quant_edgetpu.tflite source='https://ultralytics.com/images/bus.jpg'
|
||||
```
|
||||
|
||||
### Export Arguments
|
||||
|
||||
| Argument | Type | Default | Description |
|
||||
| -------- | ---------------- | ----------- | --------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `format` | `str` | `'edgetpu'` | Target format for the exported model, defining compatibility with various deployment environments. |
|
||||
| `imgsz` | `int` or `tuple` | `640` | Desired image size for the model input. Can be an integer for square images or a tuple `(height, width)` for specific dimensions. |
|
||||
| `device` | `str` | `None` | Specifies the device for exporting: CPU (`device=cpu`). |
|
||||
|
||||
!!! tip
|
||||
|
||||
Please make sure to use an x86 Linux machine when exporting to EdgeTPU.
|
||||
|
||||
For more details about the export process, visit the [Ultralytics documentation page on exporting](../modes/export.md).
|
||||
|
||||
## Deploying Exported YOLO26 TFLite Edge TPU Models
|
||||
|
||||
After successfully exporting your Ultralytics YOLO26 models to TFLite Edge TPU format, you can now deploy them. The primary and recommended first step for running a TFLite Edge TPU model is to use the YOLO("model_edgetpu.tflite") method, as outlined in the previous usage code snippet.
|
||||
|
||||
However, for in-depth instructions on deploying your TFLite Edge TPU models, take a look at the following resources:
|
||||
|
||||
- **[Coral Edge TPU on a Raspberry Pi with Ultralytics YOLO26](../guides/coral-edge-tpu-on-raspberry-pi.md)**: Discover how to integrate Coral Edge TPUs with Raspberry Pi for enhanced machine learning capabilities.
|
||||
|
||||
- **[Code Examples](https://gweb-coral-full.uc.r.appspot.com/docs/edgetpu/compiler/)**: Access practical TensorFlow Edge TPU deployment examples to kickstart your projects.
|
||||
|
||||
- **[Run Inference on the Edge TPU with Python](https://gweb-coral-full.uc.r.appspot.com/docs/edgetpu/tflite-python/#overview)**: Explore how to use the TensorFlow Lite Python API for Edge TPU applications, including setup and usage guidelines.
|
||||
|
||||
## Summary
|
||||
|
||||
In this guide, we've learned how to export Ultralytics YOLO26 models to TFLite Edge TPU format. By following the steps mentioned above, you can increase the speed and power of your [computer vision](https://www.ultralytics.com/glossary/computer-vision-cv) applications.
|
||||
|
||||
For further details on usage, visit the [Edge TPU official website](https://cloud.google.com/tpu).
|
||||
|
||||
Also, for more information on other Ultralytics YOLO26 integrations, please visit our [integration guide page](index.md). There, you'll discover valuable resources and insights.
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I export a YOLO26 model to TFLite Edge TPU format?
|
||||
|
||||
To export a YOLO26 model to TFLite Edge TPU format, you can follow these steps:
|
||||
|
||||
!!! example "Usage"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load the YOLO26 model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Export the model to TFLite Edge TPU format
|
||||
model.export(format="edgetpu") # creates 'yolo26n_full_integer_quant_edgetpu.tflite'
|
||||
|
||||
# Load the exported TFLite Edge TPU model
|
||||
edgetpu_model = YOLO("yolo26n_full_integer_quant_edgetpu.tflite")
|
||||
|
||||
# Run inference
|
||||
results = edgetpu_model("https://ultralytics.com/images/bus.jpg")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Export a YOLO26n PyTorch model to TFLite Edge TPU format
|
||||
yolo export model=yolo26n.pt format=edgetpu # creates 'yolo26n_full_integer_quant_edgetpu.tflite'
|
||||
|
||||
# Run inference with the exported model
|
||||
yolo predict model=yolo26n_full_integer_quant_edgetpu.tflite source='https://ultralytics.com/images/bus.jpg'
|
||||
```
|
||||
|
||||
For complete details on exporting models to other formats, refer to our [export guide](../modes/export.md).
|
||||
|
||||
### What are the benefits of exporting YOLO26 models to TFLite Edge TPU?
|
||||
|
||||
Exporting YOLO26 models to TFLite Edge TPU offers several benefits:
|
||||
|
||||
- **Optimized Performance**: Achieve high-speed neural network performance with minimal power consumption.
|
||||
- **Reduced Latency**: Quick local data processing without the need for cloud dependency.
|
||||
- **Enhanced Privacy**: Local processing keeps user data private and secure.
|
||||
|
||||
This makes it ideal for applications in [edge computing](https://www.ultralytics.com/glossary/edge-computing), where devices have limited power and computational resources. Learn more about [why you should export](#why-should-you-export-to-tflite-edge-tpu).
|
||||
|
||||
### Can I deploy TFLite Edge TPU models on mobile and embedded devices?
|
||||
|
||||
Yes, TensorFlow Lite Edge TPU models can be deployed directly on mobile and embedded devices. This deployment approach allows models to execute directly on the hardware, offering faster and more efficient inferencing. For integration examples, check our [guide on deploying Coral Edge TPU on Raspberry Pi](../guides/coral-edge-tpu-on-raspberry-pi.md).
|
||||
|
||||
### What are some common use cases for TFLite Edge TPU models?
|
||||
|
||||
Common use cases for TFLite Edge TPU models include:
|
||||
|
||||
- **Smart Cameras**: Enhancing real-time image and video analysis.
|
||||
- **IoT Devices**: Enabling smart home and industrial automation.
|
||||
- **Healthcare**: Accelerating medical imaging and diagnostics.
|
||||
- **Retail**: Improving inventory management and customer behavior analysis.
|
||||
|
||||
These applications benefit from the high performance and low power consumption of TFLite Edge TPU models. Discover more about [usage scenarios](#deployment-options-with-tflite-edge-tpu).
|
||||
|
||||
### How can I troubleshoot issues while exporting or deploying TFLite Edge TPU models?
|
||||
|
||||
If you encounter issues while exporting or deploying TFLite Edge TPU models, refer to our [Common Issues guide](../guides/yolo-common-issues.md) for troubleshooting tips. This guide covers common problems and solutions to help you ensure smooth operation. For additional support, visit our [Help Center](https://docs.ultralytics.com/help/).
|
||||
@@ -0,0 +1,332 @@
|
||||
---
|
||||
comments: true
|
||||
description: Export YOLO26 models to ExecuTorch format for efficient on-device inference on mobile and edge devices. Optimize your AI models for iOS, Android, and embedded systems.
|
||||
keywords: Ultralytics, YOLO26, ExecuTorch, model export, PyTorch, edge AI, mobile deployment, on-device inference, XNNPACK, embedded systems
|
||||
---
|
||||
|
||||
# Deploy YOLO26 on Mobile & Edge with ExecuTorch
|
||||
|
||||
Deploying computer vision models on edge devices like smartphones, tablets, and embedded systems requires an optimized runtime that balances performance with resource constraints. ExecuTorch, PyTorch's solution for edge computing, enables efficient on-device inference for [Ultralytics YOLO](https://github.com/ultralytics/ultralytics) models.
|
||||
|
||||
This guide outlines how to export Ultralytics YOLO models to ExecuTorch format, enabling you to deploy your models on mobile and edge devices with optimized performance.
|
||||
|
||||
## Why export to ExecuTorch?
|
||||
|
||||
<p align="center">
|
||||
<img width="100%" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/executorch-pipeline.avif" alt="PyTorch ExecuTorch mobile inference framework">
|
||||
</p>
|
||||
|
||||
[ExecuTorch](https://docs.pytorch.org/executorch/) is PyTorch's end-to-end solution for enabling on-device inference capabilities across mobile and edge devices. Built with the goal of being portable and efficient, ExecuTorch can be used to run PyTorch programs on a wide variety of computing platforms.
|
||||
|
||||
## Key features of ExecuTorch
|
||||
|
||||
ExecuTorch provides several powerful features for deploying Ultralytics YOLO models on edge devices:
|
||||
|
||||
- **Portable Model Format**: ExecuTorch uses the `.pte` (PyTorch ExecuTorch) format, which is optimized for size and loading speed on resource-constrained devices.
|
||||
|
||||
- **XNNPACK Backend**: Default integration with XNNPACK provides highly optimized inference on mobile CPUs, delivering excellent performance without requiring specialized hardware.
|
||||
|
||||
- **Quantization Support**: Built-in support for quantization techniques to reduce model size and improve inference speed while maintaining accuracy.
|
||||
|
||||
- **Memory Efficiency**: Optimized memory management reduces runtime memory footprint, making it suitable for devices with limited RAM.
|
||||
|
||||
- **Model Metadata**: Exported models include metadata (image size, class names, etc.) in a separate YAML file for easy integration.
|
||||
|
||||
## Deployment Options with ExecuTorch
|
||||
|
||||
ExecuTorch models can be deployed across various edge and mobile platforms:
|
||||
|
||||
- **Mobile Applications**: Deploy on iOS and Android applications with native performance, enabling real-time object detection in mobile apps.
|
||||
|
||||
- **Embedded Systems**: Run on embedded Linux devices like Raspberry Pi, NVIDIA Jetson, and other ARM-based systems with optimized performance.
|
||||
|
||||
- **Edge AI Devices**: Deploy on specialized edge AI hardware with custom delegates for accelerated inference.
|
||||
|
||||
- **IoT Devices**: Integrate into IoT devices for on-device inference without cloud connectivity requirements.
|
||||
|
||||
## Exporting Ultralytics YOLO26 Models to ExecuTorch
|
||||
|
||||
Converting Ultralytics YOLO26 models to ExecuTorch format enables efficient deployment on mobile and edge devices.
|
||||
|
||||
### Installation
|
||||
|
||||
ExecuTorch export requires Python 3.10 or higher and specific dependencies:
|
||||
|
||||
!!! tip "Installation"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Install Ultralytics package
|
||||
pip install ultralytics
|
||||
```
|
||||
|
||||
For detailed instructions and best practices related to the installation process, check our [YOLO26 Installation guide](../quickstart.md). While installing the required packages for YOLO26, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips.
|
||||
|
||||
### Usage
|
||||
|
||||
Exporting YOLO26 models to ExecuTorch is straightforward:
|
||||
|
||||
!!! example "Usage"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load the YOLO26 model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Export the model to ExecuTorch format
|
||||
model.export(format="executorch") # creates 'yolo26n_executorch_model' directory
|
||||
|
||||
executorch_model = YOLO("yolo26n_executorch_model")
|
||||
|
||||
results = executorch_model.predict("https://ultralytics.com/images/bus.jpg")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Export a YOLO26n PyTorch model to ExecuTorch format
|
||||
yolo export model=yolo26n.pt format=executorch # creates 'yolo26n_executorch_model' directory
|
||||
|
||||
# Run inference with the exported model
|
||||
yolo predict model=yolo26n_executorch_model source=https://ultralytics.com/images/bus.jpg
|
||||
```
|
||||
|
||||
ExecuTorch exports generate a directory that includes a `.pte` file and metadata. Use the ExecuTorch runtime in your mobile or embedded application to load the `.pte` model and perform inference.
|
||||
|
||||
### Export Arguments
|
||||
|
||||
When exporting to ExecuTorch format, you can specify the following arguments:
|
||||
|
||||
| Argument | Type | Default | Description |
|
||||
| -------- | --------------- | ------- | ------------------------------------------ |
|
||||
| `imgsz` | `int` or `list` | `640` | Image size for model input (height, width) |
|
||||
| `device` | `str` | `'cpu'` | Device to use for export (`'cpu'`) |
|
||||
|
||||
### Output Structure
|
||||
|
||||
The ExecuTorch export creates a directory containing the model and metadata:
|
||||
|
||||
```text
|
||||
yolo26n_executorch_model/
|
||||
├── yolo26n.pte # ExecuTorch model file
|
||||
└── metadata.yaml # Model metadata (classes, image size, etc.)
|
||||
```
|
||||
|
||||
## Using Exported ExecuTorch Models
|
||||
|
||||
After exporting your model, you'll need to integrate it into your target application using the ExecuTorch runtime.
|
||||
|
||||
### Mobile Integration
|
||||
|
||||
For mobile applications (iOS/Android), you'll need to:
|
||||
|
||||
1. **Add ExecuTorch Runtime**: Include the ExecuTorch runtime library in your mobile project
|
||||
2. **Load Model**: Load the `.pte` file in your application
|
||||
3. **Run Inference**: Process images and get predictions
|
||||
|
||||
Example iOS integration (Objective-C/C++):
|
||||
|
||||
```objc
|
||||
// iOS uses C++ APIs for model loading and inference
|
||||
// See https://pytorch.org/executorch/stable/using-executorch-ios.html for complete examples
|
||||
|
||||
#include <executorch/extension/module/module.h>
|
||||
|
||||
using namespace ::executorch::extension;
|
||||
|
||||
// Load the model
|
||||
Module module("/path/to/yolo26n.pte");
|
||||
|
||||
// Create input tensor
|
||||
float input[1 * 3 * 640 * 640];
|
||||
auto tensor = from_blob(input, {1, 3, 640, 640});
|
||||
|
||||
// Run inference
|
||||
const auto result = module.forward(tensor);
|
||||
```
|
||||
|
||||
Example Android integration (Kotlin):
|
||||
|
||||
```kotlin
|
||||
import org.pytorch.executorch.EValue
|
||||
import org.pytorch.executorch.Module
|
||||
import org.pytorch.executorch.Tensor
|
||||
|
||||
// Load the model
|
||||
val module = Module.load("/path/to/yolo26n.pte")
|
||||
|
||||
// Prepare input tensor
|
||||
val inputTensor = Tensor.fromBlob(floatData, longArrayOf(1, 3, 640, 640))
|
||||
val inputEValue = EValue.from(inputTensor)
|
||||
|
||||
// Run inference
|
||||
val outputs = module.forward(inputEValue)
|
||||
val scores = outputs[0].toTensor().dataAsFloatArray
|
||||
```
|
||||
|
||||
### Embedded Linux
|
||||
|
||||
For embedded Linux systems, use the ExecuTorch C++ API:
|
||||
|
||||
```cpp
|
||||
#include <executorch/extension/module/module.h>
|
||||
|
||||
// Load model
|
||||
auto module = torch::executor::Module("yolo26n.pte");
|
||||
|
||||
// Prepare input
|
||||
std::vector<float> input_data = preprocessImage(image);
|
||||
auto input_tensor = torch::executor::Tensor(input_data, {1, 3, 640, 640});
|
||||
|
||||
// Run inference
|
||||
auto outputs = module.forward({input_tensor});
|
||||
```
|
||||
|
||||
For more details on integrating ExecuTorch into your applications, visit the [ExecuTorch Documentation](https://docs.pytorch.org/executorch/).
|
||||
|
||||
## Performance Optimization
|
||||
|
||||
### Model Size Optimization
|
||||
|
||||
To reduce model size for deployment:
|
||||
|
||||
- **Use Smaller Models**: Start with YOLO26n (nano) for the smallest footprint
|
||||
- **Lower Input Resolution**: Use smaller image sizes (e.g., `imgsz=320` or `imgsz=416`)
|
||||
- **Quantization**: Apply quantization techniques (supported in future ExecuTorch versions)
|
||||
|
||||
### Inference Speed Optimization
|
||||
|
||||
For faster inference:
|
||||
|
||||
- **XNNPACK Backend**: The default XNNPACK backend provides optimized CPU inference
|
||||
- **Hardware Acceleration**: Use platform-specific delegates (e.g., CoreML for iOS)
|
||||
- **Batch Processing**: Process multiple images when possible
|
||||
|
||||
## Benchmarks
|
||||
|
||||
The Ultralytics team benchmarked YOLO26 models, comparing speed and accuracy between PyTorch and ExecuTorch.
|
||||
|
||||
!!! tip "Performance"
|
||||
|
||||
=== "Raspberry Pi 5"
|
||||
|
||||
| Model | Format | Status | Size (MB) | metrics/mAP50-95(B) | Inference time (ms/im) |
|
||||
| ------- | ----------- | ------ | --------- | ------------------- | ---------------------- |
|
||||
| YOLO11n | PyTorch | ✅ | 5.4 | 0.5060 | 337.67 |
|
||||
| YOLO11n | ExecuTorch | ✅ | 11 | 0.5080 | 167.28 |
|
||||
| YOLO11s | PyTorch | ✅ | 19 | 0.5770 | 928.80 |
|
||||
| YOLO11s | ExecuTorch | ✅ | 37 | 0.5780 | 388.31 |
|
||||
|
||||
=== "More devices coming soon!"
|
||||
|
||||
!!! note
|
||||
|
||||
Inference time does not include pre/ post-processing.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
**Issue**: `Python version error`
|
||||
|
||||
**Solution**: ExecuTorch requires Python 3.10 or higher. Upgrade your Python installation:
|
||||
|
||||
```bash
|
||||
# Using conda
|
||||
conda create -n executorch python=3.10
|
||||
conda activate executorch
|
||||
```
|
||||
|
||||
**Issue**: `Export fails during first run`
|
||||
|
||||
**Solution**: ExecuTorch may need to download and compile components on first use. Ensure you have:
|
||||
|
||||
```bash
|
||||
pip install --upgrade executorch
|
||||
```
|
||||
|
||||
**Issue**: `Import errors for ExecuTorch modules`
|
||||
|
||||
**Solution**: Ensure ExecuTorch is properly installed:
|
||||
|
||||
```bash
|
||||
pip install executorch --force-reinstall
|
||||
```
|
||||
|
||||
For more troubleshooting help, visit the [Ultralytics GitHub Issues](https://github.com/ultralytics/ultralytics/issues) or the [ExecuTorch Documentation](https://docs.pytorch.org/executorch/stable/getting-started-setup.html).
|
||||
|
||||
## Summary
|
||||
|
||||
Exporting YOLO26 models to ExecuTorch format enables efficient deployment on mobile and edge devices. With PyTorch-native integration, cross-platform support, and optimized performance, ExecuTorch is an excellent choice for edge AI applications.
|
||||
|
||||
Key takeaways:
|
||||
|
||||
- ExecuTorch provides PyTorch-native edge deployment with excellent performance
|
||||
- Export is simple with `format='executorch'` parameter
|
||||
- Models are optimized for mobile CPUs via XNNPACK backend
|
||||
- Supports iOS, Android, and embedded Linux platforms
|
||||
- Requires Python 3.10+ and FlatBuffers compiler
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I export a YOLO26 model to ExecuTorch format?
|
||||
|
||||
Export a YOLO26 model to ExecuTorch using either Python or CLI:
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
model = YOLO("yolo26n.pt")
|
||||
model.export(format="executorch")
|
||||
```
|
||||
|
||||
or
|
||||
|
||||
```bash
|
||||
yolo export model=yolo26n.pt format=executorch
|
||||
```
|
||||
|
||||
### What are the system requirements for ExecuTorch export?
|
||||
|
||||
ExecuTorch export requires:
|
||||
|
||||
- Python 3.10 or higher
|
||||
- `executorch` package (install via `pip install executorch`)
|
||||
- PyTorch (installed automatically with ultralytics)
|
||||
|
||||
Note: During the first export, ExecuTorch will download and compile necessary components including the FlatBuffers compiler automatically.
|
||||
|
||||
### Can I run inference with ExecuTorch models directly in Python?
|
||||
|
||||
ExecuTorch models (`.pte` files) are designed for deployment on mobile and edge devices using the ExecuTorch runtime. They cannot be directly loaded with `YOLO()` for inference in Python. You need to integrate them into your target application using the ExecuTorch runtime libraries.
|
||||
|
||||
### What platforms are supported by ExecuTorch?
|
||||
|
||||
ExecuTorch supports:
|
||||
|
||||
- **Mobile**: iOS and Android
|
||||
- **Embedded Linux**: Raspberry Pi, NVIDIA Jetson, and other ARM devices
|
||||
- **Desktop**: Linux, macOS, and Windows (for development)
|
||||
|
||||
### How does ExecuTorch compare to TFLite for mobile deployment?
|
||||
|
||||
Both ExecuTorch and TFLite are excellent for mobile deployment:
|
||||
|
||||
- **ExecuTorch**: Better PyTorch integration, native PyTorch workflow, growing ecosystem
|
||||
- **TFLite**: More mature, wider hardware support, more deployment examples
|
||||
|
||||
Choose ExecuTorch if you're already using PyTorch and want a native deployment path. Choose TFLite for maximum compatibility and mature tooling.
|
||||
|
||||
### Can I use ExecuTorch models with GPU acceleration?
|
||||
|
||||
Yes! ExecuTorch supports hardware acceleration through various backends:
|
||||
|
||||
- **Mobile GPU**: Via Vulkan, Metal, or OpenCL delegates
|
||||
- **NPU/DSP**: Via platform-specific delegates
|
||||
- **Default**: XNNPACK for optimized CPU inference
|
||||
|
||||
Refer to the [ExecuTorch Documentation](https://docs.pytorch.org/executorch/stable/compiler-delegate-and-partitioner.html) for backend-specific setup.
|
||||
@@ -0,0 +1,157 @@
|
||||
---
|
||||
comments: true
|
||||
description: Learn how to efficiently train Ultralytics YOLO26 models using Google Colab's powerful cloud-based environment. Start your project with ease.
|
||||
keywords: YOLO26, Google Colab, machine learning, deep learning, model training, GPU, TPU, cloud computing, Jupyter Notebook, Ultralytics
|
||||
---
|
||||
|
||||
# Accelerating YOLO26 Projects with Google Colab
|
||||
|
||||
Many developers lack the powerful computing resources needed to build [deep learning](https://www.ultralytics.com/glossary/deep-learning-dl) models. Acquiring high-end hardware or renting a decent GPU can be expensive. Google Colab is a great solution to this. It's a browser-based platform that allows you to work with large datasets, develop complex models, and share your work with others without a huge cost.
|
||||
|
||||
<p align="center">
|
||||
<br>
|
||||
<iframe loading="lazy" width="720" height="405" src="https://www.youtube.com/embed/ZN3nRZT7b24"
|
||||
title="YouTube video player" frameborder="0"
|
||||
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"
|
||||
allowfullscreen>
|
||||
</iframe>
|
||||
<br>
|
||||
<strong>Watch:</strong> How to Train Ultralytics YOLO26 models on Your Custom Dataset in <a href="https://colab.research.google.com/github/ultralytics/ultralytics/blob/main/examples/tutorial.ipynb" target="_blank">Google Colab</a>.
|
||||
</p>
|
||||
|
||||
You can use Google Colab to work on projects related to [Ultralytics YOLO26](https://github.com/ultralytics/ultralytics) models. Google Colab's user-friendly environment is well suited for efficient model development and experimentation. Let's learn more about Google Colab, its key features, and how you can use it to train YOLO26 models.
|
||||
|
||||
## Google Colaboratory
|
||||
|
||||
Google Colaboratory, commonly known as Google Colab, was developed by Google Research in 2017. It is a free online cloud-based Jupyter Notebook environment that allows you to train your [machine learning](https://www.ultralytics.com/glossary/machine-learning-ml) and deep learning models on CPUs, GPUs, and TPUs. The motivation behind developing Google Colab was Google's broader goals to advance AI technology and educational tools, and encourage the use of cloud services.
|
||||
|
||||
You can use Google Colab regardless of the specifications and configurations of your local computer. All you need is a Google account and a web browser.
|
||||
|
||||
## Training YOLO26 Using Google Colaboratory
|
||||
|
||||
Training YOLO26 models on Google Colab is straightforward. You can access the [Google Colab YOLO26 Notebook](https://colab.research.google.com/github/ultralytics/ultralytics/blob/main/examples/tutorial.ipynb) and start training your model immediately. For a detailed understanding of the model training process and best practices, refer to our [YOLO26 Model Training guide](../modes/train.md).
|
||||
|
||||
### Common Questions While Working with Google Colab
|
||||
|
||||
When working with Google Colab, you might have a few common questions. Let's answer them.
|
||||
|
||||
**Q: Why does my Google Colab session timeout?**
|
||||
A: Google Colab sessions can time out due to inactivity, especially for free users who have a limited session duration.
|
||||
|
||||
**Q: Can I increase the session duration in Google Colab?**
|
||||
A: Free users face limits, but Google Colab Pro offers extended session durations.
|
||||
|
||||
**Q: What should I do if my session closes unexpectedly?**
|
||||
A: Regularly save your work to Google Drive or GitHub to avoid losing unsaved progress.
|
||||
|
||||
**Q: How can I check my session status and resource usage?**
|
||||
A: Colab provides 'RAM Usage' and 'Disk Usage' metrics in the interface to monitor your resources.
|
||||
|
||||
**Q: Can I run multiple Colab sessions simultaneously?**
|
||||
A: Yes, but be cautious about resource usage to avoid performance issues.
|
||||
|
||||
**Q: Does Google Colab have GPU access limitations?**
|
||||
A: Yes, free GPU access has limitations, but Google Colab Pro provides more substantial usage options.
|
||||
|
||||
## Key Features of Google Colab
|
||||
|
||||
Now, let's look at some of the standout features that make Google Colab a go-to platform for machine learning projects:
|
||||
|
||||
- **Library Support:** Google Colab includes pre-installed libraries for data analysis and machine learning and allows additional libraries to be installed as needed. It also supports various libraries for creating interactive charts and visualizations.
|
||||
|
||||
- **Hardware Resources:** Users can also switch between different hardware options by modifying the runtime settings as shown below. Google Colab provides access to advanced hardware like Tesla K80 GPUs and TPUs, which are specialized circuits designed specifically for machine learning tasks.
|
||||
|
||||
|
||||
|
||||
- **Collaboration:** Google Colab makes collaborating and working with other developers easy. You can easily share your notebooks with others and perform edits in real-time.
|
||||
|
||||
- **Custom Environment:** Users can install dependencies, configure the system, and use shell commands directly in the notebook.
|
||||
|
||||
- **Educational Resources:** Google Colab offers a range of tutorials and example notebooks to help users learn and explore various functionalities.
|
||||
|
||||
## Why Should You Use Google Colab for Your YOLO26 Projects?
|
||||
|
||||
There are many options for training and evaluating YOLO26 models, so what makes the integration with Google Colab unique? Let's explore the advantages of this integration:
|
||||
|
||||
- **Zero Setup:** Since Colab runs in the cloud, users can start training models immediately without the need for complex environment setups. Just create an account and start coding.
|
||||
|
||||
- **Form Support:** It allows users to create forms for parameter input, making it easier to experiment with different values.
|
||||
|
||||
- **Integration with Google Drive:** Colab seamlessly integrates with Google Drive to make data storage, access, and management simple. Datasets and models can be stored and retrieved directly from Google Drive.
|
||||
|
||||
- **Markdown Support:** You can use Markdown format for enhanced documentation within notebooks.
|
||||
|
||||
- **Scheduled Execution:** Developers can set notebooks to run automatically at specified times.
|
||||
|
||||
- **Extensions and Widgets:** Google Colab allows for adding functionality through third-party extensions and interactive widgets.
|
||||
|
||||
## Tips for Working with YOLO26 on Google Colab
|
||||
|
||||
To make the most of your Google Colab experience when working with YOLO26 models, consider these practical tips:
|
||||
|
||||
- **Enable GPU Acceleration:** Always enable GPU acceleration in the runtime settings to significantly speed up training.
|
||||
- **Maintain a Stable Connection:** Since Colab runs in the cloud, ensure you have a stable internet connection to prevent interruptions during training.
|
||||
- **Organize Your Files:** Store your datasets and models in Google Drive or GitHub for easy access and management within Colab.
|
||||
- **Optimize Memory Usage:** If you encounter memory limitations on the free tier, try reducing image size or batch size during training.
|
||||
- **Save Regularly:** Due to Colab's session time limits, save your model and results frequently to avoid losing progress.
|
||||
|
||||
## Keep Learning about Google Colab
|
||||
|
||||
If you'd like to dive deeper into Google Colab, here are a few resources to guide you.
|
||||
|
||||
- **[Training Custom Datasets with Ultralytics YOLO26 in Google Colab](https://www.ultralytics.com/blog/training-custom-datasets-with-ultralytics-yolov8-in-google-colab)**: Learn how to train custom datasets with Ultralytics YOLO26 on Google Colab. This comprehensive blog post will take you through the entire process, from initial setup to the training and evaluation stages.
|
||||
|
||||
- **[Image Segmentation with Ultralytics YOLO26 on Google Colab](https://www.ultralytics.com/blog/image-segmentation-with-ultralytics-yolo11-on-google-colab)**: Explore how to perform image segmentation tasks using YOLO26 in the Google Colab environment, with practical examples using datasets like the Roboflow Carparts Segmentation Dataset.
|
||||
|
||||
- **[Curated Notebooks](https://colab.google/notebooks/)**: Here you can explore a series of organized and educational notebooks, each grouped by specific topic areas.
|
||||
|
||||
- **[Google Colab's Medium Page](https://medium.com/google-colab)**: You can find tutorials, updates, and community contributions here that can help you better understand and utilize this tool.
|
||||
|
||||
## Summary
|
||||
|
||||
We've discussed how you can easily experiment with Ultralytics YOLO26 models on Google Colab. You can use Google Colab to train and evaluate your models on GPUs and TPUs with a few clicks, making it an accessible platform for developers without high-end hardware.
|
||||
|
||||
For more details, visit [Google Colab's FAQ page](https://research.google.com/colaboratory/faq.html).
|
||||
|
||||
Interested in more YOLO26 integrations? Visit the [Ultralytics integration guide page](index.md) to explore additional tools and capabilities that can improve your machine-learning projects, or check out [Kaggle integration](kaggle.md) for another cloud-based alternative.
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I start training Ultralytics YOLO26 models on Google Colab?
|
||||
|
||||
To start training Ultralytics YOLO26 models on Google Colab, sign in to your Google account, then access the [Google Colab YOLO26 Notebook](https://colab.research.google.com/github/ultralytics/ultralytics/blob/main/examples/tutorial.ipynb). This notebook guides you through the setup and training process. After launching the notebook, run the cells step-by-step to train your model. For a full guide, refer to the [YOLO26 Model Training guide](../modes/train.md).
|
||||
|
||||
### What are the advantages of using Google Colab for training YOLO26 models?
|
||||
|
||||
Google Colab offers several advantages for training YOLO26 models:
|
||||
|
||||
- **Zero Setup:** No initial environment setup is required; just log in and start coding.
|
||||
- **Free GPU Access:** Use powerful GPUs or TPUs without the need for expensive hardware.
|
||||
- **Integration with Google Drive:** Easily store and access datasets and models.
|
||||
- **Collaboration:** Share notebooks with others and collaborate in real-time.
|
||||
|
||||
For more information on why you should use Google Colab, explore the [training guide](../modes/train.md) and visit the [Google Colab page](https://colab.google/notebooks/).
|
||||
|
||||
### How can I handle Google Colab session timeouts during YOLO26 training?
|
||||
|
||||
Google Colab sessions timeout due to inactivity, especially for free users. To handle this:
|
||||
|
||||
1. **Stay Active:** Regularly interact with your Colab notebook.
|
||||
2. **Save Progress:** Continuously save your work to Google Drive or GitHub.
|
||||
3. **Colab Pro:** Consider upgrading to Google Colab Pro for longer session durations.
|
||||
|
||||
For more tips on managing your Colab session, visit the [Google Colab FAQ page](https://research.google.com/colaboratory/faq.html).
|
||||
|
||||
### Can I use custom datasets for training YOLO26 models in Google Colab?
|
||||
|
||||
Yes, you can use custom datasets to train YOLO26 models in Google Colab. Upload your dataset to Google Drive and load it directly into your Colab notebook. You can follow Nicolai's YouTube guide, [How to Train YOLO26 Models on Your Custom Dataset](https://www.youtube.com/watch?v=LNwODJXcvt4), or refer to the [Custom Dataset Training guide](https://www.ultralytics.com/blog/training-custom-datasets-with-ultralytics-yolov8-in-google-colab) for detailed steps.
|
||||
|
||||
### What should I do if my Google Colab training session is interrupted?
|
||||
|
||||
If your Google Colab training session is interrupted:
|
||||
|
||||
1. **Save Regularly:** Avoid losing unsaved progress by regularly saving your work to Google Drive or GitHub.
|
||||
2. **Resume Training:** Restart your session and re-run the cells from where the interruption occurred.
|
||||
3. **Use Checkpoints:** Incorporate checkpointing in your training script to save progress periodically.
|
||||
|
||||
These practices help ensure your progress is secure. Learn more about session management on [Google Colab's FAQ page](https://research.google.com/colaboratory/faq.html).
|
||||
@@ -0,0 +1,197 @@
|
||||
---
|
||||
comments: true
|
||||
description: Discover an interactive way to perform object detection with Ultralytics YOLO26 using Gradio. Upload images and adjust settings for real-time results.
|
||||
keywords: Ultralytics, YOLO26, Gradio, object detection, interactive, real-time, image processing, AI
|
||||
---
|
||||
|
||||
# Interactive Object Detection: Gradio & Ultralytics YOLO26 🚀
|
||||
|
||||
## Introduction to Interactive Object Detection
|
||||
|
||||
This Gradio interface provides an easy and interactive way to perform [object detection](https://www.ultralytics.com/glossary/object-detection) using the [Ultralytics YOLO26](https://github.com/ultralytics/ultralytics/) model. Users can upload images and adjust parameters like confidence threshold and intersection-over-union (IoU) threshold to get real-time detection results.
|
||||
|
||||
<p align="center">
|
||||
<br>
|
||||
<iframe loading="lazy" width="720" height="405" src="https://www.youtube.com/embed/pWYiene9lYw"
|
||||
title="YouTube video player" frameborder="0"
|
||||
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"
|
||||
allowfullscreen>
|
||||
</iframe>
|
||||
<br>
|
||||
<strong>Watch:</strong> Gradio Integration with Ultralytics YOLO26
|
||||
</p>
|
||||
|
||||
## Why Use Gradio for Object Detection?
|
||||
|
||||
- **User-Friendly Interface:** Gradio offers a straightforward platform for users to upload images and visualize detection results without any coding requirement.
|
||||
- **Real-Time Adjustments:** Parameters such as confidence and IoU thresholds can be adjusted on the fly, allowing for immediate feedback and optimization of detection results.
|
||||
- **Broad Accessibility:** The Gradio web interface can be accessed by anyone, making it an excellent tool for demonstrations, educational purposes, and quick experiments.
|
||||
|
||||
<p align="center">
|
||||
<img width="800" alt="Gradio YOLO detection interface" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/gradio-example-screenshot.avif">
|
||||
</p>
|
||||
|
||||
## How to Install Gradio
|
||||
|
||||
```bash
|
||||
pip install gradio
|
||||
```
|
||||
|
||||
## How to Use the Interface
|
||||
|
||||
1. **Upload Image:** Click on 'Upload Image' to choose an image file for object detection.
|
||||
2. **Adjust Parameters:**
|
||||
- **Confidence Threshold:** Slider to set the minimum confidence level for detecting objects.
|
||||
- **IoU Threshold:** Slider to set the IoU threshold for distinguishing different objects.
|
||||
3. **View Results:** The processed image with detected objects and their labels will be displayed.
|
||||
|
||||
## Example Use Cases
|
||||
|
||||
- **Sample Image 1:** Bus detection with default thresholds.
|
||||
- **Sample Image 2:** Detection on a sports image with default thresholds.
|
||||
|
||||
## Usage Example
|
||||
|
||||
This section provides the Python code used to create the Gradio interface with the Ultralytics YOLO26 model. The code supports classification tasks, detection tasks, segmentation tasks, and keypoint tasks.
|
||||
|
||||
```python
|
||||
import gradio as gr
|
||||
import PIL.Image as Image
|
||||
|
||||
from ultralytics import ASSETS, YOLO
|
||||
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
|
||||
def predict_image(img, conf_threshold, iou_threshold):
|
||||
"""Predicts objects in an image using a YOLO26 model with adjustable confidence and IoU thresholds."""
|
||||
results = model.predict(
|
||||
source=img,
|
||||
conf=conf_threshold,
|
||||
iou=iou_threshold,
|
||||
show_labels=True,
|
||||
show_conf=True,
|
||||
imgsz=640,
|
||||
)
|
||||
|
||||
for r in results:
|
||||
im_array = r.plot()
|
||||
im = Image.fromarray(im_array[..., ::-1])
|
||||
|
||||
return im
|
||||
|
||||
|
||||
iface = gr.Interface(
|
||||
fn=predict_image,
|
||||
inputs=[
|
||||
gr.Image(type="pil", label="Upload Image"),
|
||||
gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"),
|
||||
gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"),
|
||||
],
|
||||
outputs=gr.Image(type="pil", label="Result"),
|
||||
title="Ultralytics Gradio",
|
||||
description="Upload images for inference. The Ultralytics YOLO26n model is used by default.",
|
||||
examples=[
|
||||
[ASSETS / "bus.jpg", 0.25, 0.45],
|
||||
[ASSETS / "zidane.jpg", 0.25, 0.45],
|
||||
],
|
||||
)
|
||||
|
||||
if __name__ == "__main__":
|
||||
iface.launch()
|
||||
```
|
||||
|
||||
## Parameters Explanation
|
||||
|
||||
| Parameter Name | Type | Description |
|
||||
| ---------------- | ------- | -------------------------------------------------------- |
|
||||
| `img` | `Image` | The image on which object detection will be performed. |
|
||||
| `conf_threshold` | `float` | Confidence threshold for detecting objects. |
|
||||
| `iou_threshold` | `float` | Intersection-over-union threshold for object separation. |
|
||||
|
||||
### Gradio Interface Components
|
||||
|
||||
| Component | Description |
|
||||
| ------------ | ---------------------------------------- |
|
||||
| Image Input | To upload the image for detection. |
|
||||
| Sliders | To adjust confidence and IoU thresholds. |
|
||||
| Image Output | To display the detection results. |
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I use Gradio with Ultralytics YOLO26 for object detection?
|
||||
|
||||
To use Gradio with Ultralytics YOLO26 for object detection, you can follow these steps:
|
||||
|
||||
1. **Install Gradio:** Use the command `pip install gradio`.
|
||||
2. **Create Interface:** Write a Python script to initialize the Gradio interface. You can refer to the provided code example in the [documentation](#usage-example) for details.
|
||||
3. **Upload and Adjust:** Upload your image and adjust the confidence and IoU thresholds on the Gradio interface to get real-time object detection results.
|
||||
|
||||
Here's a minimal code snippet for reference:
|
||||
|
||||
```python
|
||||
import gradio as gr
|
||||
|
||||
from ultralytics import YOLO
|
||||
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
|
||||
def predict_image(img, conf_threshold, iou_threshold):
|
||||
results = model.predict(
|
||||
source=img,
|
||||
conf=conf_threshold,
|
||||
iou=iou_threshold,
|
||||
show_labels=True,
|
||||
show_conf=True,
|
||||
)
|
||||
return results[0].plot() if results else None
|
||||
|
||||
|
||||
iface = gr.Interface(
|
||||
fn=predict_image,
|
||||
inputs=[
|
||||
gr.Image(type="pil", label="Upload Image"),
|
||||
gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"),
|
||||
gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"),
|
||||
],
|
||||
outputs=gr.Image(type="pil", label="Result"),
|
||||
title="Ultralytics Gradio YOLO26",
|
||||
description="Upload images for YOLO26 object detection.",
|
||||
)
|
||||
iface.launch()
|
||||
```
|
||||
|
||||
### What are the benefits of using Gradio for Ultralytics YOLO26 object detection?
|
||||
|
||||
Using Gradio for Ultralytics YOLO26 object detection offers several benefits:
|
||||
|
||||
- **User-Friendly Interface:** Gradio provides an intuitive interface for users to upload images and visualize detection results without any coding effort.
|
||||
- **Real-Time Adjustments:** You can dynamically adjust detection parameters such as confidence and IoU thresholds and see the effects immediately.
|
||||
- **Accessibility:** The web interface is accessible to anyone, making it useful for quick experiments, educational purposes, and demonstrations.
|
||||
|
||||
For more details, you can read this [blog post on AI in radiology](https://www.ultralytics.com/blog/ai-and-radiology-a-new-era-of-precision-and-efficiency) that showcases similar interactive visualization techniques.
|
||||
|
||||
### Can I use Gradio and Ultralytics YOLO26 together for educational purposes?
|
||||
|
||||
Yes, Gradio and Ultralytics YOLO26 can be utilized together for educational purposes effectively. Gradio's intuitive web interface makes it easy for students and educators to interact with state-of-the-art [deep learning](https://www.ultralytics.com/glossary/deep-learning-dl) models like Ultralytics YOLO26 without needing advanced programming skills. This setup is ideal for demonstrating key concepts in object detection and [computer vision](https://www.ultralytics.com/glossary/computer-vision-cv), as Gradio provides immediate visual feedback which helps in understanding the impact of different parameters on the detection performance.
|
||||
|
||||
### How do I adjust the confidence and IoU thresholds in the Gradio interface for YOLO26?
|
||||
|
||||
In the Gradio interface for YOLO26, you can adjust the confidence and IoU thresholds using the sliders provided. These thresholds help control the prediction [accuracy](https://www.ultralytics.com/glossary/accuracy) and object separation:
|
||||
|
||||
- **Confidence Threshold:** Determines the minimum confidence level for detecting objects. Slide to increase or decrease the confidence required.
|
||||
- **IoU Threshold:** Sets the intersection-over-union threshold for distinguishing between overlapping objects. Adjust this value to refine object separation.
|
||||
|
||||
For more information on these parameters, visit the [parameters explanation section](#parameters-explanation).
|
||||
|
||||
### What are some practical applications of using Ultralytics YOLO26 with Gradio?
|
||||
|
||||
Practical applications of combining Ultralytics YOLO26 with Gradio include:
|
||||
|
||||
- **Real-Time Object Detection Demonstrations:** Ideal for showcasing how object detection works in real-time.
|
||||
- **Educational Tools:** Useful in academic settings to teach object detection and computer vision concepts.
|
||||
- **Prototype Development:** Efficient for developing and testing prototype object detection applications quickly.
|
||||
- **Community and Collaborations:** Making it easy to share models with the community for feedback and collaboration.
|
||||
|
||||
For examples of similar use cases, check out the [Ultralytics blog on animal behavior monitoring](https://www.ultralytics.com/blog/monitoring-animal-behavior-using-ultralytics-yolov8) which demonstrates how interactive visualization can enhance wildlife conservation efforts.
|
||||
@@ -0,0 +1,409 @@
|
||||
---
|
||||
comments: true
|
||||
description: Dive into our detailed integration guide on using IBM Watson to train a YOLO26 model. Uncover key features and step-by-step instructions on model training.
|
||||
keywords: IBM Watsonx, IBM Watsonx AI, What is Watson?, IBM Watson Integration, IBM Watson Features, YOLO26, Ultralytics, Model Training, GPU, TPU, cloud computing
|
||||
---
|
||||
|
||||
# A Step-by-Step Guide to Training YOLO26 Models with IBM Watsonx
|
||||
|
||||
Nowadays, scalable [computer vision solutions](../guides/steps-of-a-cv-project.md) are becoming more common and transforming the way we handle visual data. A great example is IBM Watsonx, an advanced AI and data platform that simplifies the development, deployment, and management of AI models. It offers a complete suite for the entire AI lifecycle and seamless integration with IBM Cloud services.
|
||||
|
||||
You can train [Ultralytics YOLO26 models](https://github.com/ultralytics/ultralytics) using IBM Watsonx. It's a good option for enterprises interested in efficient [model training](../modes/train.md), fine-tuning for specific tasks, and improving [model performance](../guides/model-evaluation-insights.md) with robust tools and a user-friendly setup. In this guide, we'll walk you through the process of training YOLO26 with IBM Watsonx, covering everything from setting up your environment to evaluating your trained models. Let's get started!
|
||||
|
||||
## What is IBM Watsonx?
|
||||
|
||||
[Watsonx](https://www.ibm.com/products/watsonx) is IBM's cloud-based platform designed for commercial [generative AI](https://www.ultralytics.com/glossary/generative-ai) and scientific data. IBM Watsonx's three components - `watsonx.ai`, `watsonx.data`, and `watsonx.governance` - come together to create an end-to-end, trustworthy AI platform that can accelerate AI projects aimed at solving business problems. It provides powerful tools for building, training, and [deploying machine learning models](../guides/model-deployment-options.md) and makes it easy to connect with various data sources.
|
||||
|
||||
<p align="center">
|
||||
<img width="800" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/overview-of-ibm-watsonx.avif" alt="IBM Watsonx AI platform architecture overview">
|
||||
</p>
|
||||
|
||||
Its user-friendly interface and collaborative capabilities streamline the development process and help with efficient model management and deployment. Whether for computer vision, predictive analytics, [natural language processing](https://www.ultralytics.com/glossary/natural-language-processing-nlp), or other AI applications, IBM Watsonx provides the tools and support needed to drive innovation.
|
||||
|
||||
## Key Features of IBM Watsonx
|
||||
|
||||
IBM Watsonx is made of three main components: `watsonx.ai`, `watsonx.data`, and `watsonx.governance`. Each component offers features that cater to different aspects of AI and data management. Let's take a closer look at them.
|
||||
|
||||
### [Watsonx.ai](https://www.ibm.com/products/watsonx-ai)
|
||||
|
||||
Watsonx.ai provides powerful tools for AI development and offers access to IBM-supported custom models, third-party models like [Llama 3](https://www.ultralytics.com/blog/getting-to-know-metas-llama-3), and IBM's own Granite models. It includes the Prompt Lab for experimenting with AI prompts, the Tuning Studio for improving model performance with labeled data, and the Flows Engine for simplifying generative AI application development. Also, it offers comprehensive tools for automating the AI model lifecycle and connecting to various APIs and libraries.
|
||||
|
||||
### [Watsonx.data](https://www.ibm.com/products/watsonx-data)
|
||||
|
||||
Watsonx.data supports both cloud and on-premises deployments through the IBM Storage Fusion HCI integration. Its user-friendly console provides centralized access to data across environments and makes data exploration easy with common SQL. It optimizes workloads with efficient query engines like Presto and Spark, accelerates data insights with an AI-powered semantic layer, includes a vector database for AI relevance, and supports open data formats for easy sharing of analytics and AI data.
|
||||
|
||||
### [Watsonx.governance](https://www.ibm.com/products/watsonx-governance)
|
||||
|
||||
Watsonx.governance makes compliance easier by automatically identifying regulatory changes and enforcing policies. It links requirements to internal risk data and provides up-to-date AI factsheets. The platform helps manage risk with alerts and tools to detect issues such as [bias and drift](../guides/model-monitoring-and-maintenance.md). It also automates the monitoring and documentation of the AI lifecycle, organizes AI development with a model inventory, and enhances collaboration with user-friendly dashboards and reporting tools.
|
||||
|
||||
## How to Train YOLO26 Using IBM Watsonx
|
||||
|
||||
You can use IBM Watsonx to accelerate your YOLO26 model training workflow.
|
||||
|
||||
### Prerequisites
|
||||
|
||||
You need an [IBM Cloud account](https://cloud.ibm.com/registration) to create a [watsonx.ai](https://www.ibm.com/products/watsonx-ai) project, and you'll also need a [Kaggle](./kaggle.md) account to load the data set.
|
||||
|
||||
### Step 1: Set Up Your Environment
|
||||
|
||||
First, you'll need to set up an IBM account to use a Jupyter Notebook. Log in to [watsonx.ai](https://eu-de.dataplatform.cloud.ibm.com/registration/stepone?preselect_region=true) using your IBM Cloud account.
|
||||
|
||||
Then, create a [watsonx.ai project](https://www.ibm.com/docs/en/watsonx/saas?topic=projects-creating-project), and a [Jupyter Notebook](https://www.ibm.com/docs/en/watsonx/saas?topic=editor-creating-managing-notebooks).
|
||||
|
||||
Once you do so, a notebook environment will open for you to load your data set. You can use the code from this tutorial to tackle a simple object detection model training task.
|
||||
|
||||
### Step 2: Install and Import Relevant Libraries
|
||||
|
||||
Next, you can install and import the necessary Python libraries.
|
||||
|
||||
!!! tip "Installation"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Install the required packages
|
||||
pip install torch torchvision torchaudio
|
||||
pip install ultralytics-opencv-headless
|
||||
```
|
||||
|
||||
For detailed instructions and best practices related to the installation process, check our [Ultralytics Installation guide](../quickstart.md). While installing the required packages for YOLO26, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips.
|
||||
|
||||
Then, you can import the needed packages.
|
||||
|
||||
!!! example "Import Relevant Libraries"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
# Import ultralytics
|
||||
import ultralytics
|
||||
|
||||
ultralytics.checks()
|
||||
|
||||
# Import packages to retrieve and display image files
|
||||
```
|
||||
|
||||
### Step 3: Load the Data
|
||||
|
||||
For this tutorial, we will use a [marine litter dataset](https://www.kaggle.com/datasets/atiqishrak/trash-dataset-icra19) available on Kaggle. With this dataset, we will custom-train a YOLO26 model to detect and classify litter and biological objects in underwater images.
|
||||
|
||||
We can load the dataset directly into the notebook using the Kaggle API. First, create a free Kaggle account. Once you have created an account, you'll need to generate an API key. Directions for generating your key can be found in the [Kaggle API documentation](https://github.com/Kaggle/kaggle-api/blob/main/docs/README.md) under the section "API credentials".
|
||||
|
||||
Copy and paste your Kaggle username and API key into the following code. Then run the code to install the API and load the dataset into Watsonx.
|
||||
|
||||
!!! tip "Installation"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Install kaggle
|
||||
pip install kaggle
|
||||
```
|
||||
|
||||
After installing Kaggle, we can load the dataset into Watsonx.
|
||||
|
||||
!!! example "Load the Data"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
# Replace "username" string with your username
|
||||
os.environ["KAGGLE_USERNAME"] = "username"
|
||||
# Replace "apiKey" string with your key
|
||||
os.environ["KAGGLE_KEY"] = "apiKey"
|
||||
|
||||
# Load dataset
|
||||
os.system("kaggle datasets download atiqishrak/trash-dataset-icra19 --unzip")
|
||||
|
||||
# Store working directory path as work_dir
|
||||
work_dir = os.getcwd()
|
||||
|
||||
# Print work_dir path
|
||||
print(os.getcwd())
|
||||
|
||||
# Print work_dir contents
|
||||
print(os.listdir(f"{work_dir}"))
|
||||
|
||||
# Print trash_ICRA19 subdirectory contents
|
||||
print(os.listdir(f"{work_dir}/trash_ICRA19"))
|
||||
```
|
||||
|
||||
After loading the dataset, we printed and saved our working directory. We have also printed the contents of our working directory to confirm the "trash_ICRA19" data set was loaded properly.
|
||||
|
||||
If you see "trash_ICRA19" among the directory's contents, then it has loaded successfully. You should see three files/folders: a `config.yaml` file, a `videos_for_testing` directory, and a `dataset` directory. We will ignore the `videos_for_testing` directory, so feel free to delete it.
|
||||
|
||||
We will use the `config.yaml` file and the contents of the dataset directory to train our [object detection](https://www.ultralytics.com/glossary/object-detection) model. Here is a sample image from our marine litter data set.
|
||||
|
||||
<p align="center">
|
||||
<img width="400" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/marine-litter-bounding-box.avif" alt="Marine Litter with Bounding Box">
|
||||
</p>
|
||||
|
||||
### Step 4: Preprocess the Data
|
||||
|
||||
Fortunately, all labels in the marine litter data set are already formatted as YOLO .txt files. However, we need to rearrange the structure of the image and label directories in order to help our model process the image and labels. Right now, our loaded data set directory follows this structure:
|
||||
|
||||
<p align="center">
|
||||
<img width="400" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/marine-litter-bounding-box-1.avif" alt="Loaded Dataset Directory">
|
||||
</p>
|
||||
|
||||
But, YOLO models by default require separate images and labels in subdirectories within the train/val/test split. We need to reorganize the directory into the following structure:
|
||||
|
||||
<p align="center">
|
||||
<img width="400" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/yolo-directory-structure.avif" alt="YOLO Directory Structure">
|
||||
</p>
|
||||
|
||||
To reorganize the data set directory, we can run the following script:
|
||||
|
||||
!!! example "Preprocess the Data"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
# Function to reorganize dir
|
||||
def organize_files(directory):
|
||||
for subdir in ["train", "test", "val"]:
|
||||
subdir_path = os.path.join(directory, subdir)
|
||||
if not os.path.exists(subdir_path):
|
||||
continue
|
||||
|
||||
images_dir = os.path.join(subdir_path, "images")
|
||||
labels_dir = os.path.join(subdir_path, "labels")
|
||||
|
||||
# Create image and label subdirs if non-existent
|
||||
os.makedirs(images_dir, exist_ok=True)
|
||||
os.makedirs(labels_dir, exist_ok=True)
|
||||
|
||||
# Move images and labels to respective subdirs
|
||||
for filename in os.listdir(subdir_path):
|
||||
if filename.endswith(".txt"):
|
||||
shutil.move(os.path.join(subdir_path, filename), os.path.join(labels_dir, filename))
|
||||
elif filename.endswith(".jpg") or filename.endswith(".png") or filename.endswith(".jpeg"):
|
||||
shutil.move(os.path.join(subdir_path, filename), os.path.join(images_dir, filename))
|
||||
# Delete .xml files
|
||||
elif filename.endswith(".xml"):
|
||||
os.remove(os.path.join(subdir_path, filename))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
directory = f"{work_dir}/trash_ICRA19/dataset"
|
||||
organize_files(directory)
|
||||
```
|
||||
|
||||
Next, we need to modify the .yaml file for the data set. This is the setup we will use in our .yaml file. Class ID numbers start from 0:
|
||||
|
||||
```yaml
|
||||
path: /path/to/dataset/directory # root directory for dataset
|
||||
train: train/images # train images subdirectory
|
||||
val: train/images # validation images subdirectory
|
||||
test: test/images # test images subdirectory
|
||||
|
||||
# Classes
|
||||
names:
|
||||
0: plastic
|
||||
1: bio
|
||||
2: rov
|
||||
```
|
||||
|
||||
Run the following script to delete the current contents of `config.yaml` and replace it with the configuration that reflects our new data set directory structure. The script automatically uses the `work_dir` variable we defined earlier, so be sure it points to your dataset before execution, and leave the train, val, and test subdirectory definitions unchanged.
|
||||
|
||||
!!! example "Edit the .yaml File"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
# Contents of new config.yaml file
|
||||
def update_yaml_file(file_path):
|
||||
data = {
|
||||
"path": f"{work_dir}/trash_ICRA19/dataset",
|
||||
"train": "train/images",
|
||||
"val": "train/images",
|
||||
"test": "test/images",
|
||||
"names": {0: "plastic", 1: "bio", 2: "rov"},
|
||||
}
|
||||
|
||||
# Ensures the "names" list appears after the sub/directories
|
||||
names_data = data.pop("names")
|
||||
with open(file_path, "w") as yaml_file:
|
||||
yaml.dump(data, yaml_file)
|
||||
yaml_file.write("\n")
|
||||
yaml.dump({"names": names_data}, yaml_file)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
file_path = f"{work_dir}/trash_ICRA19/config.yaml" # .yaml file path
|
||||
update_yaml_file(file_path)
|
||||
print(f"{file_path} updated successfully.")
|
||||
```
|
||||
|
||||
### Step 5: Train the YOLO26 model
|
||||
|
||||
Run the following command-line code to fine tune a pretrained default YOLO26 model.
|
||||
|
||||
!!! example "Train the YOLO26 model"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
!yolo task=detect mode=train data={work_dir}/trash_ICRA19/config.yaml model=yolo26n.pt epochs=2 batch=32 lr0=.04 plots=True
|
||||
```
|
||||
|
||||
Here's a closer look at the parameters in the model training command:
|
||||
|
||||
- **task**: It specifies the [computer vision](https://www.ultralytics.com/glossary/computer-vision-cv) task for which you are using the specified YOLO model and data set.
|
||||
- **mode**: Denotes the purpose for which you are loading the specified model and data. Since we are training a model, it is set to "train." Later, when we test our model's performance, we will set it to "predict."
|
||||
- **epochs**: This delimits the number of times YOLO26 will pass through our entire data set.
|
||||
- **batch**: The numerical value stipulates the training [batch sizes](https://www.ultralytics.com/glossary/batch-size). Batches are the number of images a model processes before it updates its parameters.
|
||||
- **lr0**: Specifies the model's initial [learning rate](https://www.ultralytics.com/glossary/learning-rate).
|
||||
- **plots**: Directs YOLO to generate and save plots of our model's training and evaluation metrics.
|
||||
|
||||
For a detailed understanding of the model training process and best practices, refer to the [YOLO26 Model Training guide](../modes/train.md). This guide will help you get the most out of your experiments and ensure you're using YOLO26 effectively.
|
||||
|
||||
### Step 6: Test the Model
|
||||
|
||||
We can now run inference to test the performance of our fine-tuned model:
|
||||
|
||||
!!! example "Test the YOLO26 model"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
!yolo task=detect mode=predict source={work_dir}/trash_ICRA19/dataset/test/images model={work_dir}/runs/detect/train/weights/best.pt conf=0.5 iou=.5 save=True save_txt=True
|
||||
```
|
||||
|
||||
This brief script generates predicted labels for each image in our test set, as well as new output image files that overlay the predicted [bounding box](https://www.ultralytics.com/glossary/bounding-box) atop the original image.
|
||||
|
||||
Predicted .txt labels for each image are saved via the `save_txt=True` argument and the output images with bounding box overlays are generated through the `save=True` argument.
|
||||
The parameter `conf=0.5` informs the model to ignore all predictions with a confidence level of less than 50%.
|
||||
|
||||
Lastly, `iou=.5` directs the model to ignore boxes in the same class with an overlap of 50% or greater. It helps to reduce potential duplicate boxes generated for the same object.
|
||||
We can load the images with predicted bounding box overlays to view how our model performs on a handful of images.
|
||||
|
||||
!!! example "Display Predictions"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
# Show the first ten images from the preceding prediction task
|
||||
for pred_dir in glob.glob(f"{work_dir}/runs/detect/predict/*.jpg")[:10]:
|
||||
img = Image.open(pred_dir)
|
||||
display(img)
|
||||
```
|
||||
|
||||
The code above displays ten images from the test set with their predicted bounding boxes, accompanied by class name labels and confidence levels.
|
||||
|
||||
### Step 7: Evaluate the Model
|
||||
|
||||
We can produce visualizations of the model's [precision](https://www.ultralytics.com/glossary/precision) and recall for each class. These visualizations are saved in the home directory, under the train folder. The precision score is displayed in the P_curve.png:
|
||||
|
||||
<p align="center">
|
||||
<img width="800" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/precision-confidence-curve.avif" alt="Model precision-confidence evaluation curve">
|
||||
</p>
|
||||
|
||||
The graph shows an exponential increase in precision as the model's confidence level for predictions increases. However, the model precision has not yet leveled out at a certain confidence level after two [epochs](https://www.ultralytics.com/glossary/epoch).
|
||||
|
||||
The [recall](https://www.ultralytics.com/glossary/recall) graph (R_curve.png) displays an inverse trend:
|
||||
|
||||
<p align="center">
|
||||
<img width="800" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/recall-confidence-curve.avif" alt="Model recall-confidence evaluation curve">
|
||||
</p>
|
||||
|
||||
Unlike precision, recall moves in the opposite direction, showing greater recall with lower confidence instances and lower recall with higher confidence instances. This is an apt example of the trade-off in precision and recall for classification models.
|
||||
|
||||
### Step 8: Calculating [Intersection Over Union](https://www.ultralytics.com/glossary/intersection-over-union-iou)
|
||||
|
||||
You can measure the prediction [accuracy](https://www.ultralytics.com/glossary/accuracy) by calculating the IoU between a predicted bounding box and a ground truth bounding box for the same object. Check out [IBM's tutorial on training YOLO26](https://developer.ibm.com/tutorials/awb-train-yolo-object-detection-model-in-python/) for more details.
|
||||
|
||||
## Summary
|
||||
|
||||
We explored IBM Watsonx key features, and how to train a YOLO26 model using IBM Watsonx. We also saw how IBM Watsonx can enhance your AI workflows with advanced tools for model building, data management, and compliance.
|
||||
|
||||
For further details on usage, visit [IBM Watsonx official documentation](https://www.ibm.com/products/watsonx).
|
||||
|
||||
Also, be sure to check out the [Ultralytics integration guide page](./index.md), to learn more about different exciting integrations.
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I train a YOLO26 model using IBM Watsonx?
|
||||
|
||||
To train a YOLO26 model using IBM Watsonx, follow these steps:
|
||||
|
||||
1. **Set Up Your Environment**: Create an IBM Cloud account and set up a Watsonx.ai project. Use a Jupyter Notebook for your coding environment.
|
||||
2. **Install Libraries**: Install necessary libraries like `torch`, `opencv`, and `ultralytics`.
|
||||
3. **Load Data**: Use the Kaggle API to load your dataset into Watsonx.
|
||||
4. **Preprocess Data**: Organize your dataset into the required directory structure and update the `.yaml` configuration file.
|
||||
5. **Train the Model**: Use the YOLO command-line interface to train your model with specific parameters like `epochs`, `batch size`, and `learning rate`.
|
||||
6. **Test and Evaluate**: Run inference to test the model and evaluate its performance using metrics like precision and recall.
|
||||
|
||||
For detailed instructions, refer to our [YOLO26 Model Training guide](../modes/train.md).
|
||||
|
||||
### What are the key features of IBM Watsonx for AI model training?
|
||||
|
||||
IBM Watsonx offers several key features for AI model training:
|
||||
|
||||
- **Watsonx.ai**: Provides tools for AI development, including access to IBM-supported custom models and third-party models like Llama 3. It includes the Prompt Lab, Tuning Studio, and Flows Engine for comprehensive AI lifecycle management.
|
||||
- **Watsonx.data**: Supports cloud and on-premises deployments, offering centralized data access, efficient query engines like Presto and Spark, and an AI-powered semantic layer.
|
||||
- **Watsonx.governance**: Automates compliance, manages risk with alerts, and provides tools for detecting issues like bias and drift. It also includes dashboards and reporting tools for collaboration.
|
||||
|
||||
For more information, visit the [IBM Watsonx official documentation](https://www.ibm.com/products/watsonx).
|
||||
|
||||
### Why should I use IBM Watsonx for training Ultralytics YOLO26 models?
|
||||
|
||||
IBM Watsonx is an excellent choice for training Ultralytics YOLO26 models due to its comprehensive suite of tools that streamline the AI lifecycle. Key benefits include:
|
||||
|
||||
- **Scalability**: Easily scale your model training with IBM Cloud services.
|
||||
- **Integration**: Seamlessly integrate with various data sources and APIs.
|
||||
- **User-Friendly Interface**: Simplifies the development process with a collaborative and intuitive interface.
|
||||
- **Advanced Tools**: Access to powerful tools like the Prompt Lab, Tuning Studio, and Flows Engine for enhancing model performance.
|
||||
|
||||
Learn more about [Ultralytics YOLO26](https://github.com/ultralytics/ultralytics) and how to train models using IBM Watsonx in our [integration guide](./index.md).
|
||||
|
||||
### How can I preprocess my dataset for YOLO26 training on IBM Watsonx?
|
||||
|
||||
To preprocess your dataset for YOLO26 training on IBM Watsonx:
|
||||
|
||||
1. **Organize Directories**: Ensure your dataset follows the YOLO directory structure with separate subdirectories for images and labels within the train/val/test split.
|
||||
2. **Update .yaml File**: Modify the `.yaml` configuration file to reflect the new directory structure and class names.
|
||||
3. **Run Preprocessing Script**: Use a Python script to reorganize your dataset and update the `.yaml` file accordingly.
|
||||
|
||||
Here's a sample script to organize your dataset:
|
||||
|
||||
```python
|
||||
import os
|
||||
import shutil
|
||||
|
||||
|
||||
def organize_files(directory):
|
||||
for subdir in ["train", "test", "val"]:
|
||||
subdir_path = os.path.join(directory, subdir)
|
||||
if not os.path.exists(subdir_path):
|
||||
continue
|
||||
|
||||
images_dir = os.path.join(subdir_path, "images")
|
||||
labels_dir = os.path.join(subdir_path, "labels")
|
||||
|
||||
os.makedirs(images_dir, exist_ok=True)
|
||||
os.makedirs(labels_dir, exist_ok=True)
|
||||
|
||||
for filename in os.listdir(subdir_path):
|
||||
if filename.endswith(".txt"):
|
||||
shutil.move(os.path.join(subdir_path, filename), os.path.join(labels_dir, filename))
|
||||
elif filename.endswith(".jpg") or filename.endswith(".png") or filename.endswith(".jpeg"):
|
||||
shutil.move(os.path.join(subdir_path, filename), os.path.join(images_dir, filename))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
directory = f"{work_dir}/trash_ICRA19/dataset"
|
||||
organize_files(directory)
|
||||
```
|
||||
|
||||
For more details, refer to our [data preprocessing guide](../guides/preprocessing_annotated_data.md).
|
||||
|
||||
### What are the prerequisites for training a YOLO26 model on IBM Watsonx?
|
||||
|
||||
Before you start training a YOLO26 model on IBM Watsonx, ensure you have the following prerequisites:
|
||||
|
||||
- **IBM Cloud Account**: Create an account on IBM Cloud to access Watsonx.ai.
|
||||
- **Kaggle Account**: For loading datasets, you'll need a Kaggle account and an API key.
|
||||
- **Jupyter Notebook**: Set up a Jupyter Notebook environment within Watsonx.ai for coding and model training.
|
||||
|
||||
For more information on setting up your environment, visit our [Ultralytics Installation guide](../quickstart.md).
|
||||
@@ -0,0 +1,140 @@
|
||||
---
|
||||
comments: true
|
||||
description: Discover Ultralytics integrations for streamlined ML workflows, dataset management, optimized model training, and robust deployment solutions.
|
||||
keywords: Ultralytics, machine learning, ML workflows, dataset management, model training, model deployment, Roboflow, ClearML, Comet ML, DVC, MLFlow, Ultralytics Platform, Neptune, Ray Tune, TensorBoard, Weights & Biases, Amazon SageMaker, Paperspace Gradient, Google Colab, Neural Magic, Gradio, TorchScript, ONNX, OpenVINO, TensorRT, CoreML, TF SavedModel, TF GraphDef, TFLite, TFLite Edge TPU, TF.js, PaddlePaddle, NCNN
|
||||
---
|
||||
|
||||
# Ultralytics Integrations
|
||||
|
||||
Welcome to the Ultralytics Integrations page! This page provides an overview of our partnerships with various tools and platforms, designed to streamline your [machine learning](https://www.ultralytics.com/glossary/machine-learning-ml) workflows, enhance dataset management, simplify model training, and facilitate efficient deployment.
|
||||
|
||||
<img width="1024" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/ultralytics-yolov8-ecosystem-integrations.avif" alt="Ultralytics YOLO ecosystem and integrations">
|
||||
|
||||
<p align="center">
|
||||
<br>
|
||||
<iframe loading="lazy" width="720" height="405" src="https://www.youtube.com/embed/ZzUSXQkLbNw"
|
||||
title="YouTube video player" frameborder="0"
|
||||
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"
|
||||
allowfullscreen>
|
||||
</iframe>
|
||||
<br>
|
||||
<strong>Watch:</strong> Ultralytics YOLO Deployment and Integrations
|
||||
</p>
|
||||
|
||||
## Training Integrations
|
||||
|
||||
- [Albumentations](albumentations.md): Enhance your Ultralytics models with powerful image augmentations to improve model robustness and generalization.
|
||||
|
||||
- [Amazon SageMaker](amazon-sagemaker.md): Leverage Amazon SageMaker to efficiently build, train, and deploy Ultralytics models, providing an all-in-one platform for the ML lifecycle.
|
||||
|
||||
- [ClearML](clearml.md): Automate your Ultralytics ML workflows, monitor experiments, and foster team collaboration.
|
||||
|
||||
- [Comet ML](comet.md): Enhance your model development with Ultralytics by tracking, comparing, and optimizing your machine learning experiments.
|
||||
|
||||
- [DVC](dvc.md): Implement version control for your Ultralytics machine learning projects, synchronizing data, code, and models effectively.
|
||||
|
||||
- [Google Colab](google-colab.md): Use Google Colab to train and evaluate Ultralytics models in a cloud-based environment that supports collaboration and sharing.
|
||||
|
||||
- [IBM Watsonx](ibm-watsonx.md): See how IBM Watsonx simplifies the training and evaluation of Ultralytics models with its cutting-edge AI tools, effortless integration, and advanced model management system.
|
||||
|
||||
- [JupyterLab](jupyterlab.md): Find out how to use JupyterLab's interactive and customizable environment to train and evaluate Ultralytics models with ease and efficiency.
|
||||
|
||||
- [Kaggle](kaggle.md): Explore how you can use Kaggle to train and evaluate Ultralytics models in a cloud-based environment with pre-installed libraries, GPU support, and a vibrant community for collaboration and sharing.
|
||||
|
||||
- [MLFlow](mlflow.md): Streamline the entire ML lifecycle of Ultralytics models, from experimentation and reproducibility to deployment.
|
||||
|
||||
- [Neptune](neptune.md): Maintain a comprehensive log of your ML experiments with Ultralytics in this metadata store designed for MLOps.
|
||||
|
||||
- [Paperspace Gradient](paperspace.md): Paperspace Gradient simplifies working on YOLO26 projects by providing easy-to-use cloud tools for training, testing, and deploying your models quickly.
|
||||
|
||||
- [Ray Tune](ray-tune.md): Optimize the hyperparameters of your Ultralytics models at any scale.
|
||||
|
||||
- [TensorBoard](tensorboard.md): Visualize your Ultralytics ML workflows, monitor model metrics, and foster team collaboration.
|
||||
|
||||
- [Ultralytics Platform](https://platform.ultralytics.com/): Access and contribute to a community of pretrained Ultralytics models.
|
||||
|
||||
- [VS Code](vscode.md): An extension for VS Code that provides code snippets to accelerate Ultralytics development workflows and offers examples to help anyone learn or get started.
|
||||
|
||||
- [Weights & Biases (W&B)](weights-biases.md): Monitor experiments, visualize metrics, and foster reproducibility and collaboration on Ultralytics projects.
|
||||
|
||||
## Deployment Integrations
|
||||
|
||||
- [Axelera](axelera.md): Explore Metis accelerators and the Voyager SDK for running Ultralytics models with efficient edge inference.
|
||||
|
||||
- [CoreML](coreml.md): CoreML, developed by [Apple](https://www.apple.com/), is a framework designed for efficiently integrating machine learning models into applications across iOS, macOS, watchOS, and tvOS, using Apple's hardware for effective and secure [model deployment](https://www.ultralytics.com/glossary/model-deployment).
|
||||
|
||||
- [ExecuTorch](executorch.md): Developed by [Meta](https://about.meta.com/), ExecuTorch is PyTorch's unified solution for deploying Ultralytics YOLO models on edge devices.
|
||||
|
||||
- [Gradio](gradio.md): Deploy Ultralytics models with Gradio for real-time, interactive object detection demos.
|
||||
|
||||
- [MNN](mnn.md): Developed by [Alibaba](https://www.alibabagroup.com/), MNN is a highly efficient and lightweight deep learning framework. It supports inference and training of deep learning models and has industry-leading performance for inference and training on-device.
|
||||
|
||||
- [NCNN](ncnn.md): Developed by [Tencent](http://www.tencent.com/), NCNN is an efficient [neural network](https://www.ultralytics.com/glossary/neural-network-nn) inference framework tailored for mobile devices. It enables direct deployment of AI models into apps, optimizing performance across various mobile platforms.
|
||||
|
||||
- [Neural Magic](neural-magic.md): Leverage Quantization Aware Training (QAT) and pruning techniques to optimize Ultralytics models for superior performance and leaner size.
|
||||
|
||||
- [ONNX](onnx.md): An open-source format created by [Microsoft](https://www.microsoft.com/) for facilitating the transfer of AI models between various frameworks, enhancing the versatility and deployment flexibility of Ultralytics models.
|
||||
|
||||
- [OpenVINO](openvino.md): Intel's toolkit for optimizing and deploying [computer vision](https://www.ultralytics.com/glossary/computer-vision-cv) models efficiently across various Intel CPU and GPU platforms.
|
||||
|
||||
- [PaddlePaddle](paddlepaddle.md): An open-source deep learning platform by [Baidu](https://www.baidu.com/), PaddlePaddle enables the efficient deployment of AI models and focuses on the scalability of industrial applications.
|
||||
|
||||
- [Rockchip RKNN](rockchip-rknn.md): Developed by [Rockchip](https://www.rock-chips.com/), RKNN is a specialized neural network inference framework optimized for Rockchip's hardware platforms, particularly their NPUs. It facilitates efficient deployment of AI models on edge devices, enabling high-performance inference in real-time applications.
|
||||
|
||||
- [Seeed Studio reCamera](seeedstudio-recamera.md): Developed by [Seeed Studio](https://www.seeedstudio.com/), the reCamera is an advanced edge AI device designed for real-time computer vision applications. Powered by the RISC-V-based SG200X processor, it delivers high-performance AI inference with energy efficiency. Its modular design, advanced video processing capabilities, and support for flexible deployment make it an ideal choice for various use cases, including safety monitoring, environmental applications, and manufacturing.
|
||||
|
||||
- [SONY IMX500](sony-imx500.md): Optimize and deploy [Ultralytics YOLO26](https://docs.ultralytics.com/models/yolo26/) models on Raspberry Pi AI Cameras with the IMX500 sensor for fast, low-power performance.
|
||||
|
||||
- [TensorRT](tensorrt.md): Developed by [NVIDIA](https://www.nvidia.com/), this high-performance [deep learning](https://www.ultralytics.com/glossary/deep-learning-dl) inference framework and model format optimizes AI models for accelerated speed and efficiency on NVIDIA GPUs, ensuring streamlined deployment.
|
||||
|
||||
- [TF GraphDef](tf-graphdef.md): Developed by [Google](https://www.google.com/), GraphDef is TensorFlow's format for representing computation graphs, enabling optimized execution of machine learning models across diverse hardware.
|
||||
|
||||
- [TF SavedModel](tf-savedmodel.md): Developed by [Google](https://www.google.com/), TF SavedModel is a universal serialization format for [TensorFlow](https://www.ultralytics.com/glossary/tensorflow) models, enabling easy sharing and deployment across a wide range of platforms, from servers to edge devices.
|
||||
|
||||
- [TF.js](tfjs.md): Developed by [Google](https://www.google.com/) to facilitate machine learning in browsers and Node.js, TF.js allows JavaScript-based deployment of ML models.
|
||||
|
||||
- [TFLite](tflite.md): Developed by [Google](https://www.google.com/), TFLite is a lightweight framework for deploying machine learning models on mobile and edge devices, ensuring fast, efficient inference with minimal memory footprint.
|
||||
|
||||
- [TFLite Edge TPU](edge-tpu.md): Developed by [Google](https://www.google.com/) for optimizing TensorFlow Lite models on Edge TPUs, this model format ensures high-speed, efficient [edge computing](https://www.ultralytics.com/glossary/edge-computing).
|
||||
|
||||
- [TorchScript](torchscript.md): Developed as part of the [PyTorch](https://pytorch.org/) framework, TorchScript enables efficient execution and deployment of machine learning models in various production environments without the need for Python dependencies.
|
||||
|
||||
## Datasets Integrations
|
||||
|
||||
- [Roboflow](roboflow.md): Facilitate dataset labeling and management for Ultralytics models, offering annotation tools to label images.
|
||||
|
||||
### Export Formats
|
||||
|
||||
We also support a variety of model export formats for deployment in different environments. Here are the available formats:
|
||||
|
||||
{% include "macros/export-table.md" %}
|
||||
|
||||
Explore the links to learn more about each integration and how to get the most out of them with Ultralytics. See full `export` details in the [Export](../modes/export.md) page.
|
||||
|
||||
## Contribute to Our Integrations
|
||||
|
||||
We're always excited to see how the community integrates Ultralytics YOLO with other technologies, tools, and platforms! If you have successfully integrated YOLO with a new system or have valuable insights to share, consider [contributing to our Integrations Docs](../help/contributing.md).
|
||||
|
||||
By writing a guide or tutorial, you can help expand our documentation and provide real-world examples that benefit the community. It's an excellent way to contribute to the growing ecosystem around Ultralytics YOLO.
|
||||
|
||||
To contribute, please check out our [Contributing Guide](../help/contributing.md) for instructions on how to submit a Pull Request (PR) 🛠️. We eagerly await your contributions!
|
||||
|
||||
Let's collaborate to make the Ultralytics YOLO ecosystem more expansive and feature-rich 🙏!
|
||||
|
||||
## FAQ
|
||||
|
||||
### What is Ultralytics Platform, and how does it streamline the ML workflow?
|
||||
|
||||
[Ultralytics Platform](https://platform.ultralytics.com) is a cloud-based platform designed to make machine learning workflows for Ultralytics models seamless and efficient. By using this tool, you can easily upload datasets, train models, perform real-time tracking, and deploy YOLO models without needing extensive coding skills. The platform serves as a centralized workspace where you can manage your entire ML pipeline from data preparation to deployment. You can explore the key features on the [Ultralytics Platform](https://platform.ultralytics.com/) page and get started quickly with our [Quickstart](https://docs.ultralytics.com/platform/quickstart/) guide.
|
||||
|
||||
### Can I track the performance of my Ultralytics models using MLFlow?
|
||||
|
||||
Yes, you can. Integrating [MLFlow](https://mlflow.org/) with Ultralytics models allows you to track experiments, improve reproducibility, and streamline the entire ML lifecycle. Detailed instructions for setting up this integration can be found on the [MLFlow](mlflow.md) integration page. This integration is particularly useful for monitoring model metrics, comparing different training runs, and managing the ML workflow efficiently. MLFlow provides a centralized platform to log parameters, metrics, and artifacts, making it easier to understand model behavior and make data-driven improvements.
|
||||
|
||||
### What are the benefits of using Neural Magic for YOLO26 model optimization?
|
||||
|
||||
[Neural Magic](neural-magic.md) optimizes YOLO26 models by leveraging techniques like Quantization Aware Training (QAT) and pruning, resulting in highly efficient, smaller models that perform better on resource-limited hardware. Check out the [Neural Magic](neural-magic.md) integration page to learn how to implement these optimizations for superior performance and leaner models. This is especially beneficial for deployment on edge devices where computational resources are constrained. Neural Magic's DeepSparse engine can deliver up to 6x faster inference on CPUs, making it possible to run complex models without specialized hardware.
|
||||
|
||||
### How do I deploy Ultralytics YOLO models with Gradio for interactive demos?
|
||||
|
||||
To deploy Ultralytics YOLO models with [Gradio](https://www.gradio.app/) for interactive [object detection](https://www.ultralytics.com/glossary/object-detection) demos, you can follow the steps outlined on the [Gradio](gradio.md) integration page. Gradio allows you to create easy-to-use web interfaces for real-time model inference, making it an excellent tool for showcasing your YOLO model's capabilities in a user-friendly format suitable for both developers and end-users. With just a few lines of code, you can build interactive applications that demonstrate your model's performance on custom inputs, facilitating better understanding and evaluation of your computer vision solutions.
|
||||
@@ -0,0 +1,215 @@
|
||||
---
|
||||
comments: true
|
||||
description: Learn how to use JupyterLab to train and experiment with Ultralytics YOLO26 models. Discover key features, setup instructions, and solutions to common issues.
|
||||
keywords: JupyterLab, YOLO26, Ultralytics, Model Training, Deep Learning, Interactive Coding, Data Science, Machine Learning, Jupyter Notebook, Model Development
|
||||
---
|
||||
|
||||
# A Guide on How to Use JupyterLab to Train Your YOLO26 Models
|
||||
|
||||
Building [deep learning](https://www.ultralytics.com/glossary/deep-learning-dl) models can be tough, especially when you don't have the right tools or environment to work with. If you are facing this issue, JupyterLab might be the right solution for you. JupyterLab is a user-friendly, web-based platform that makes coding more flexible and interactive. You can use it to handle big datasets, create complex models, and even collaborate with others, all in one place.
|
||||
|
||||
You can use JupyterLab to [work on projects](../guides/steps-of-a-cv-project.md) related to [Ultralytics YOLO26 models](https://github.com/ultralytics/ultralytics). JupyterLab is a great option for efficient model development and experimentation. It makes it easy to start experimenting with and [training YOLO26 models](../modes/train.md) right from your computer. Let's dive deeper into JupyterLab, its key features, and how you can use it to train YOLO26 models.
|
||||
|
||||
## What is JupyterLab?
|
||||
|
||||
JupyterLab is an open-source web-based platform designed for working with Jupyter notebooks, code, and data. It's an upgrade from the traditional Jupyter Notebook interface that provides a more versatile and powerful user experience.
|
||||
|
||||
JupyterLab allows you to work with notebooks, text editors, terminals, and other tools all in one place. Its flexible design lets you organize your workspace to fit your needs and makes it easier to perform tasks like data analysis, visualization, and [machine learning](https://www.ultralytics.com/glossary/machine-learning-ml). JupyterLab also supports real-time collaboration, making it ideal for team projects in research and data science.
|
||||
|
||||
## Key Features of JupyterLab
|
||||
|
||||
Here are some of the key features that make JupyterLab a great option for model development and experimentation:
|
||||
|
||||
- **All-in-One Workspace**: JupyterLab is a one-stop shop for all your data science needs. Unlike the classic Jupyter Notebook, which had separate interfaces for text editing, terminal access, and notebooks, JupyterLab integrates all these features into a single, cohesive environment. You can view and edit various file formats, including JPEG, PDF, and CSV, directly within JupyterLab. An all-in-one workspace lets you access everything you need at your fingertips, streamlining your workflow and saving you time.
|
||||
- **Flexible Layouts**: One of JupyterLab's standout features is its flexible layout. You can drag, drop, and resize tabs to create a personalized layout that helps you work more efficiently. The collapsible left sidebar keeps essential tabs like the file browser, running kernels, and command palette within easy reach. You can have multiple windows open at once, allowing you to multitask and manage your projects more effectively.
|
||||
- **Interactive Code Consoles**: Code consoles in JupyterLab provide an interactive space to test out snippets of code or functions. They also serve as a log of computations made within a notebook. Creating a new console for a notebook and viewing all kernel activity is straightforward. This feature is especially useful when you're experimenting with new ideas or troubleshooting issues in your code.
|
||||
- **Markdown Preview**: Working with Markdown files is more efficient in JupyterLab, thanks to its simultaneous preview feature. As you write or edit your Markdown file, you can see the formatted output in real-time. It makes it easier to double-check that your documentation looks perfect, saving you from having to switch back and forth between editing and preview modes.
|
||||
- **Run Code from Text Files**: If you're sharing a text file with code, JupyterLab makes it easy to run it directly within the platform. You can highlight the code and press Shift + Enter to execute it. It is great for verifying code snippets quickly and helps guarantee that the code you share is functional and error-free.
|
||||
|
||||
## Why Should You Use JupyterLab for Your YOLO26 Projects?
|
||||
|
||||
There are multiple platforms for developing and evaluating machine learning models, so what makes JupyterLab stand out? Let's explore some of the unique aspects that JupyterLab offers for your machine-learning projects:
|
||||
|
||||
- **Easy Cell Management**: Managing cells in JupyterLab is a breeze. Instead of the cumbersome cut-and-paste method, you can simply drag and drop cells to rearrange them.
|
||||
- **Cross-Notebook Cell Copying**: JupyterLab makes it simple to copy cells between different notebooks. You can drag and drop cells from one notebook to another.
|
||||
- **Easy Switch to Classic Notebook View**: For those who miss the classic Jupyter Notebook interface, JupyterLab offers an easy switch back. Simply replace `/lab` in the URL with `/tree` to return to the familiar notebook view.
|
||||
- **Multiple Views**: JupyterLab supports multiple views of the same notebook, which is particularly useful for long notebooks. You can open different sections side-by-side for comparison or exploration, and any changes made in one view are reflected in the other.
|
||||
- **Customizable Themes**: JupyterLab includes a built-in Dark theme for the notebook, which is perfect for late-night coding sessions. There are also themes available for the text editor and terminal, allowing you to customize the appearance of your entire workspace.
|
||||
|
||||
## Common Issues While Working with JupyterLab
|
||||
|
||||
When working with JupyterLab, you might come across some common issues. Here are some tips to help you navigate the platform smoothly:
|
||||
|
||||
- **Managing Kernels**: Kernels are crucial because they manage the connection between the code you write in JupyterLab and the environment where it runs. They can also access and share data between notebooks. When you close a Jupyter Notebook, the kernel might still be running because other notebooks could be using it. If you want to completely shut down a kernel, you can select it, right-click, and choose "Shut Down Kernel" from the pop-up menu.
|
||||
- **Installing Python Packages**: Sometimes, you might need additional Python packages that aren't pre-installed on the server. You can easily install these packages in your home directory or a virtual environment by using the command `python -m pip install package-name`. To see all installed packages, use `python -m pip list`.
|
||||
- **Deploying Flask/FastAPI API to Posit Connect**: You can deploy your Flask and FastAPI APIs to Posit Connect using the [rsconnect-python](https://docs.posit.co/rsconnect-python/) package from the terminal. Doing so makes it easier to integrate your web applications with JupyterLab and share them with others.
|
||||
- **Installing JupyterLab Extensions**: JupyterLab supports various extensions to enhance functionality. You can install and customize these extensions to suit your needs. For detailed instructions, refer to [JupyterLab Extensions Guide](https://jupyterlab.readthedocs.io/en/latest/user/extensions.html) for more information.
|
||||
- **Using Multiple Versions of Python**: If you need to work with different versions of Python, you can use Jupyter kernels configured with different Python versions.
|
||||
|
||||
## How to Use JupyterLab to Try Out YOLO26
|
||||
|
||||
JupyterLab makes it easy to experiment with YOLO26. To get started, follow these simple steps.
|
||||
|
||||
### Step 1: Install JupyterLab
|
||||
|
||||
First, you need to install JupyterLab. Open your terminal and run the command:
|
||||
|
||||
!!! tip "Installation"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Install the required package for JupyterLab
|
||||
pip install jupyterlab
|
||||
```
|
||||
|
||||
### Step 2: Download the YOLO26 Tutorial Notebook
|
||||
|
||||
Next, download the [tutorial.ipynb](https://github.com/ultralytics/ultralytics/blob/main/examples/tutorial.ipynb) file from the Ultralytics GitHub repository. Save this file to any directory on your local machine.
|
||||
|
||||
### Step 3: Launch JupyterLab
|
||||
|
||||
Navigate to the directory where you saved the notebook file using your terminal. Then, run the following command to launch JupyterLab:
|
||||
|
||||
!!! example "Usage"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
jupyter lab
|
||||
```
|
||||
|
||||
Once you've run this command, it will open JupyterLab in your default web browser, as shown below.
|
||||
|
||||
|
||||
|
||||
### Step 4: Start Experimenting
|
||||
|
||||
In JupyterLab, open the tutorial.ipynb notebook. You can now start running the cells to explore and experiment with YOLO26.
|
||||
|
||||
|
||||
|
||||
JupyterLab's interactive environment allows you to modify code, visualize outputs, and document your findings all in one place. You can try out different configurations and understand how YOLO26 works.
|
||||
|
||||
For a detailed understanding of the model training process and best practices, refer to the [YOLO26 Model Training guide](../modes/train.md). This guide will help you get the most out of your experiments and ensure you're using YOLO26 effectively.
|
||||
|
||||
## Keep Learning about JupyterLab
|
||||
|
||||
If you want to learn more about JupyterLab, here are resources to get you started:
|
||||
|
||||
- [**JupyterLab Documentation**](https://jupyterlab.readthedocs.io/en/stable/getting_started/starting.html): Dive into the official JupyterLab Documentation to explore its features and capabilities. It's a great way to understand how to use this powerful tool to its fullest potential.
|
||||
- [**Try It With Binder**](https://mybinder.org/v2/gh/jupyterlab/jupyterlab-demo/HEAD?urlpath=lab/tree/demo): Experiment with JupyterLab without installing anything by using Binder, which lets you launch a live JupyterLab instance directly in your browser. It's a great way to start experimenting immediately.
|
||||
- [**Installation Guide**](https://jupyterlab.readthedocs.io/en/stable/getting_started/installation.html): For a step-by-step guide on installing JupyterLab on your local machine, check out the installation guide.
|
||||
- [**Train Ultralytics YOLO26 using JupyterLab**](https://www.ultralytics.com/blog/train-ultralytics-yolo11-using-the-jupyterlab-integration): Learn more about the practical applications of using JupyterLab with YOLO26 models in this detailed blog post.
|
||||
|
||||
## Summary
|
||||
|
||||
We've explored how JupyterLab can be a powerful tool for experimenting with Ultralytics YOLO26 models. Using its flexible and interactive environment, you can easily set up JupyterLab on your local machine and start working with YOLO26. JupyterLab makes it simple to [train](../guides/model-training-tips.md) and [evaluate](../guides/model-testing.md) your models, visualize outputs, and [document your findings](../guides/model-monitoring-and-maintenance.md) all in one place.
|
||||
|
||||
Unlike other platforms such as [Google Colab](../integrations/google-colab.md), JupyterLab runs locally on your machine, giving you more control over your computing environment while still providing an interactive notebook experience. This makes it particularly valuable for developers who need consistent access to their development environment without relying on cloud resources.
|
||||
|
||||
For more details, visit the [JupyterLab FAQ Page](https://jupyterlab.readthedocs.io/en/stable/getting_started/faq.html).
|
||||
|
||||
Interested in more YOLO26 integrations? Check out the [Ultralytics integration guide](./index.md) to explore additional tools and capabilities for your machine learning projects.
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I use JupyterLab to train a YOLO26 model?
|
||||
|
||||
To train a YOLO26 model using JupyterLab:
|
||||
|
||||
1. Install JupyterLab and the Ultralytics package:
|
||||
|
||||
```bash
|
||||
pip install jupyterlab ultralytics
|
||||
```
|
||||
|
||||
2. Launch JupyterLab and open a new notebook.
|
||||
|
||||
3. Import the YOLO model and load a pretrained model:
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
model = YOLO("yolo26n.pt")
|
||||
```
|
||||
|
||||
4. Train the model on your custom dataset:
|
||||
|
||||
```python
|
||||
results = model.train(data="path/to/your/data.yaml", epochs=100, imgsz=640)
|
||||
```
|
||||
|
||||
5. Visualize training results using JupyterLab's built-in plotting capabilities:
|
||||
|
||||
```python
|
||||
import matplotlib
|
||||
|
||||
from ultralytics.utils.plotting import plot_results
|
||||
|
||||
matplotlib.use("inline") # or 'notebook' for interactive
|
||||
plot_results(results)
|
||||
```
|
||||
|
||||
JupyterLab's interactive environment allows you to easily modify parameters, visualize results, and iterate on your model training process.
|
||||
|
||||
### What are the key features of JupyterLab that make it suitable for YOLO26 projects?
|
||||
|
||||
JupyterLab offers several features that make it ideal for YOLO26 projects:
|
||||
|
||||
1. Interactive code execution: Test and debug YOLO26 code snippets in real-time.
|
||||
2. Integrated file browser: Easily manage datasets, model weights, and configuration files.
|
||||
3. Flexible layout: Arrange multiple notebooks, terminals, and output windows side-by-side for efficient workflow.
|
||||
4. Rich output display: Visualize YOLO26 detection results, training curves, and model performance metrics inline.
|
||||
5. Markdown support: Document your YOLO26 experiments and findings with rich text and images.
|
||||
6. Extension ecosystem: Enhance functionality with extensions for version control, [remote computing](google-colab.md), and more.
|
||||
|
||||
These features allow for a seamless development experience when working with YOLO26 models, from data preparation to [model deployment](https://www.ultralytics.com/glossary/model-deployment).
|
||||
|
||||
### How can I optimize YOLO26 model performance using JupyterLab?
|
||||
|
||||
To optimize YOLO26 model performance in JupyterLab:
|
||||
|
||||
1. Use the autobatch feature to determine the optimal batch size:
|
||||
|
||||
```python
|
||||
from ultralytics.utils.autobatch import autobatch
|
||||
|
||||
optimal_batch_size = autobatch(model)
|
||||
```
|
||||
|
||||
2. Implement [hyperparameter tuning](../guides/hyperparameter-tuning.md) using libraries like Ray Tune:
|
||||
|
||||
```python
|
||||
from ultralytics.utils.tuner import run_ray_tune
|
||||
|
||||
best_results = run_ray_tune(model, data="path/to/data.yaml")
|
||||
```
|
||||
|
||||
3. Visualize and analyze model metrics using JupyterLab's plotting capabilities:
|
||||
|
||||
```python
|
||||
from ultralytics.utils.plotting import plot_results
|
||||
|
||||
plot_results(results.results_dict)
|
||||
```
|
||||
|
||||
4. Experiment with different model architectures and [export formats](../modes/export.md) to find the best balance of speed and [accuracy](https://www.ultralytics.com/glossary/accuracy) for your specific use case.
|
||||
|
||||
JupyterLab's interactive environment allows for quick iterations and real-time feedback, making it easier to optimize your YOLO26 models efficiently.
|
||||
|
||||
### How do I handle common issues when working with JupyterLab and YOLO26?
|
||||
|
||||
When working with JupyterLab and YOLO26, you might encounter some common issues. Here's how to handle them:
|
||||
|
||||
1. GPU memory issues:
|
||||
- Use `torch.cuda.empty_cache()` to clear GPU memory between runs.
|
||||
- Adjust [batch size](https://www.ultralytics.com/glossary/batch-size) or image size to fit your GPU memory.
|
||||
|
||||
2. Package conflicts:
|
||||
- Create a separate conda environment for your YOLO26 projects to avoid conflicts.
|
||||
- Use `!pip install package_name` in a notebook cell to install missing packages.
|
||||
|
||||
3. Kernel crashes:
|
||||
- Restart the kernel and run cells one by one to identify the problematic code.
|
||||
- Check for memory leaks in your code, especially when processing large datasets.
|
||||
@@ -0,0 +1,251 @@
|
||||
---
|
||||
comments: true
|
||||
description: Learn how to use Kaggle to train Ultralytics YOLO26 models with free GPU/TPU resources. Discover Kaggle's features, benefits, and best practices for efficient model development.
|
||||
keywords: Kaggle, YOLO26, Ultralytics, machine learning, model training, GPU, TPU, cloud computing, data science, computer vision
|
||||
---
|
||||
|
||||
# A Guide on Using Kaggle to Train Your YOLO26 Models
|
||||
|
||||
If you are learning about AI and working on [small projects](../solutions/index.md), you might not have access to powerful computing resources yet, and high-end hardware can be expensive. Fortunately, Kaggle, a platform owned by Google, offers a great solution. Kaggle provides a free, cloud-based environment where you can access GPU resources, handle large datasets, and collaborate with a diverse community of data scientists and [machine learning](https://www.ultralytics.com/glossary/machine-learning-ml) enthusiasts.
|
||||
|
||||
Kaggle is a great choice for [training](../guides/model-training-tips.md) and experimenting with [Ultralytics YOLO26](https://github.com/ultralytics/ultralytics?tab=readme-ov-file) models. Kaggle Notebooks make using popular machine learning libraries and frameworks in your projects easy. This guide explores Kaggle's main features and shows how to train YOLO26 models on the platform.
|
||||
|
||||
## What is Kaggle?
|
||||
|
||||
Kaggle is a platform that brings together data scientists from around the world to collaborate, learn, and compete in solving real-world data science problems. Launched in 2010 by Anthony Goldbloom and Jeremy Howard and acquired by Google in 2017, Kaggle enables users to connect, discover and share datasets, use GPU-powered notebooks, and participate in data science competitions. The platform is designed to help both seasoned professionals and eager learners achieve their goals by offering robust tools and resources.
|
||||
|
||||
With more than [10 million users](https://www.kaggle.com/discussions/general/332147) as of 2022, Kaggle provides a rich environment for developing and experimenting with machine learning models. You don't need to worry about your local machine's specs or setup; you can dive right in with just a Kaggle account and a web browser.
|
||||
|
||||
## Installation
|
||||
|
||||
Before you can start training YOLO26 models on Kaggle, you need to ensure your notebook environment is properly configured. Follow these essential steps:
|
||||
|
||||
### Enable Internet Access
|
||||
|
||||
Kaggle notebooks require internet access to download packages and dependencies. To enable internet in your Kaggle notebook:
|
||||
|
||||
1. Open your Kaggle notebook
|
||||
2. Click on the **Settings** panel on the right side of the notebook interface
|
||||
3. Scroll down to the **Internet** section
|
||||
4. Toggle the switch to **ON** to enable internet connectivity
|
||||
|
||||
**Note**: Internet access is required for installing the Ultralytics package and downloading pre-trained models or datasets. Without internet enabled, package installations will fail.
|
||||
|
||||
|
||||
|
||||
### Installing Ultralytics
|
||||
|
||||
Once internet access is enabled, install the Ultralytics package by running the following command in a notebook cell:
|
||||
|
||||
```bash
|
||||
!pip install ultralytics
|
||||
```
|
||||
|
||||
For the latest development version, you can install directly from GitHub:
|
||||
|
||||
```bash
|
||||
!pip install git+https://github.com/ultralytics/ultralytics.git
|
||||
```
|
||||
|
||||
### Resolving Dependency Conflicts
|
||||
|
||||
During installation, you may encounter dependency conflicts, especially with packages like `opencv-python`, `numpy`, or `torch`. Here are common solutions:
|
||||
|
||||
#### Method 1: Force Reinstall with --upgrade
|
||||
|
||||
If you encounter conflicts with existing packages, force an upgrade:
|
||||
|
||||
```bash
|
||||
!pip install --upgrade --force-reinstall ultralytics
|
||||
```
|
||||
|
||||
#### Method 2: Use --no-deps and Install Dependencies Separately
|
||||
|
||||
If conflicts persist, install without dependencies first, then manually install required packages:
|
||||
|
||||
```bash
|
||||
!pip install --no-deps ultralytics
|
||||
!pip install torch torchvision opencv-python matplotlib pillow pyyaml requests
|
||||
```
|
||||
|
||||
#### Method 3: Restart Kernel After Installation
|
||||
|
||||
Sometimes, you need to restart the kernel after installation to resolve import issues:
|
||||
|
||||
```bash
|
||||
!pip install ultralytics
|
||||
# Then click "Restart Kernel" from the notebook menu
|
||||
```
|
||||
|
||||
#### Method 4: Use Specific Package Versions
|
||||
|
||||
If you encounter specific version conflicts, you can pin compatible versions:
|
||||
|
||||
```bash
|
||||
!pip install ultralytics opencv-python==4.8.1.78 numpy==1.24.3
|
||||
```
|
||||
|
||||
#### Common Error Solutions
|
||||
|
||||
**Error: "No module named 'ultralytics'"**
|
||||
|
||||
- Solution: Ensure internet is enabled and run the installation command again
|
||||
- Restart the kernel after installation
|
||||
|
||||
**Error: "ERROR: pip's dependency resolver does not currently take into account..."**
|
||||
|
||||
- Solution: This is usually a warning and can be safely ignored. The installation typically succeeds despite the message
|
||||
- Alternatively, use Method 2 above to install without dependency resolution
|
||||
|
||||
**Error: "ModuleNotFoundError" after installation**
|
||||
|
||||
- Solution: Restart the kernel using the restart button in the notebook interface
|
||||
- Re-run the import statements in a new cell
|
||||
|
||||
### Verifying Installation
|
||||
|
||||
After installation, verify that Ultralytics is properly installed by running:
|
||||
|
||||
```python
|
||||
import ultralytics
|
||||
|
||||
ultralytics.checks()
|
||||
```
|
||||
|
||||
This will display system information and verify that all dependencies are correctly installed.
|
||||
|
||||
## Training YOLO26 Using Kaggle
|
||||
|
||||
Training YOLO26 models on Kaggle is simple and efficient, thanks to the platform's access to powerful GPUs.
|
||||
|
||||
To get started, access the [Kaggle YOLO26 Notebook](https://www.kaggle.com/code/glennjocherultralytics/ultralytics-yolo11-notebook). Kaggle's environment comes with pre-installed libraries like [TensorFlow](https://www.ultralytics.com/glossary/tensorflow) and [PyTorch](https://www.ultralytics.com/glossary/pytorch), making the setup process hassle-free.
|
||||
|
||||
|
||||
|
||||
Once you sign in to your Kaggle account, you can click on the option to copy and edit the code, select a GPU under the accelerator settings, and run the notebook's cells to begin training your model. For a detailed understanding of the model training process and best practices, refer to our [YOLO26 Model Training guide](../modes/train.md).
|
||||
|
||||
|
||||
|
||||
On the [official YOLO26 Kaggle notebook page](https://www.kaggle.com/code/glennjocherultralytics/ultralytics-yolo11-notebook), clicking the three dots in the upper right-hand corner reveals additional options.
|
||||
|
||||
|
||||
|
||||
These options include:
|
||||
|
||||
- **View Versions**: Browse through different versions of the notebook to see changes over time and revert to previous versions if needed.
|
||||
- **Copy API Command**: Get an API command to programmatically interact with the notebook, which is useful for automation and integration into workflows.
|
||||
- **Open in Google Notebooks**: Open the notebook in Google's hosted notebook environment.
|
||||
- **Open in Colab**: Launch the notebook in [Google Colab](./google-colab.md) for further editing and execution.
|
||||
- **Follow Comments**: Subscribe to the comments section to get updates and engage with the community.
|
||||
- **Download Code**: Download the entire notebook as a Jupyter (.ipynb) file for offline use or version control in your local environment.
|
||||
- **Add to Collection**: Save the notebook to a collection within your Kaggle account for easy access and organization.
|
||||
- **Bookmark**: Bookmark the notebook for quick access in the future.
|
||||
- **Embed Notebook**: Get an embed link to include the notebook in blogs, websites, or documentation.
|
||||
|
||||
### Common Issues While Working with Kaggle
|
||||
|
||||
When working with Kaggle, you might encounter some common issues. Here are key points to help you navigate the platform:
|
||||
|
||||
- **Access to GPUs**: In your Kaggle notebooks, you can activate a GPU at any time, with usage allowed for up to 30 hours per week. Kaggle provides the NVIDIA Tesla P100 GPU with 16GB of memory and also offers the option of using a NVIDIA GPU T4 x2. Powerful hardware accelerates your machine-learning tasks, making model training and inference much faster.
|
||||
- **Kaggle Kernels**: Kaggle Kernels are free Jupyter notebook servers that can integrate GPUs, allowing you to perform machine learning operations on cloud computers. You don't have to rely on your own computer's CPU, avoiding overload and freeing up your local resources.
|
||||
- **Kaggle Datasets**: Kaggle datasets are free to download. However, it's important to check the license for each dataset to understand any usage restrictions. Some datasets may have limitations on academic publications or commercial use. You can download datasets directly to your Kaggle notebook or anywhere else via the [Kaggle API](https://www.kaggle.com/docs/api).
|
||||
- **Saving and Committing Notebooks**: To save and commit a notebook on Kaggle, click "Save Version." This saves the current state of your notebook. Once the background kernel finishes generating the output files, you can access them from the Output tab on the main notebook page.
|
||||
- **Collaboration**: Kaggle supports collaboration, but multiple users cannot edit a notebook simultaneously. Collaboration on Kaggle is asynchronous, meaning users can share and work on the same notebook at different times.
|
||||
- **Reverting to a Previous Version**: If you need to revert to a previous version of your notebook, open the notebook and click on the three vertical dots in the top right corner to select "View Versions." Find the version you want to revert to, click on the "..." menu next to it, and select "Revert to Version." After the notebook reverts, click "Save Version" to commit the changes.
|
||||
|
||||
## Key Features of Kaggle
|
||||
|
||||
Next, let's understand the features Kaggle offers that make it an excellent platform for data science and machine learning enthusiasts. Here are some of the key highlights:
|
||||
|
||||
- **Datasets**: Kaggle hosts a massive collection of [datasets](https://docs.ultralytics.com/datasets/) on various topics. You can easily search and use these datasets in your projects, which is particularly handy for training and testing your YOLO26 models.
|
||||
- **Competitions**: Known for its exciting competitions, Kaggle allows data scientists and machine learning enthusiasts to solve real-world problems. Competing helps you improve your skills, learn new techniques, and gain recognition in the community.
|
||||
- **Free Access to TPUs**: Kaggle provides free access to powerful [TPUs](https://www.ultralytics.com/glossary/tpu-tensor-processing-unit), which are beneficial for training complex machine learning models. This allows you to speed up processing and boost the performance of your YOLO26 projects without incurring extra costs.
|
||||
- **Integration with GitHub**: Kaggle allows you to easily connect your GitHub repository to upload notebooks and save your work. This integration makes it convenient to manage and access your files.
|
||||
- **Community and Discussions**: Kaggle boasts a strong community of data scientists and machine learning practitioners. The discussion forums and shared notebooks are fantastic resources for learning and troubleshooting. You can easily find help, share your knowledge, and collaborate with others.
|
||||
|
||||
## Why Should You Use Kaggle for Your YOLO26 Projects?
|
||||
|
||||
There are multiple platforms for training and evaluating machine learning models, so what makes Kaggle stand out? Let's dive into the benefits of using Kaggle for your machine learning projects:
|
||||
|
||||
- **Public Notebooks**: You can make your Kaggle notebooks public, allowing other users to view, vote, fork, and discuss your work. Kaggle promotes collaboration, feedback, and the sharing of ideas, helping you improve your YOLO26 models.
|
||||
- **Comprehensive History of Notebook Commits**: Kaggle creates a detailed history of your notebook commits. This allows you to review and track changes over time, making it easier to understand the evolution of your project and revert to previous versions if needed.
|
||||
- **Console Access**: Kaggle provides a console, giving you more control over your environment. This feature allows you to perform various tasks directly from the command line, enhancing your workflow and productivity.
|
||||
- **Resource Availability**: Each notebook editing session on Kaggle is provided with significant resources: 12 hours of execution time for CPU and GPU sessions, 9 hours of execution time for TPU sessions, and 20 gigabytes of auto-saved disk space.
|
||||
- **Notebook Scheduling**: Kaggle allows you to schedule your notebooks to run at specific times. You can automate repetitive tasks without manual intervention, such as training your model at regular intervals.
|
||||
|
||||
## Keep Learning about Kaggle
|
||||
|
||||
If you want to learn more about Kaggle, here are some helpful resources to guide you:
|
||||
|
||||
- [**Kaggle Learn**](https://www.kaggle.com/learn): Discover a variety of free, interactive tutorials on Kaggle Learn. These courses cover essential data science topics and provide hands-on experience to help you master new skills.
|
||||
- [**Getting Started with Kaggle**](https://www.kaggle.com/code/alexisbcook/getting-started-with-kaggle): This comprehensive guide walks you through the basics of using Kaggle, from joining competitions to creating your first notebook. It's a great starting point for newcomers.
|
||||
- [**Kaggle Medium Page**](https://medium.com/@kaggleteam): Explore tutorials, updates, and community contributions to Kaggle's Medium page. It's an excellent source for staying up-to-date with the latest trends and gaining deeper insights into data science.
|
||||
- [**Train Ultralytics YOLO Models Using the Kaggle Integration**](https://www.ultralytics.com/blog/train-ultralytics-yolo-models-using-the-kaggle-integration): This blog post provides additional insights on how to leverage Kaggle specifically for Ultralytics YOLO models.
|
||||
|
||||
## Summary
|
||||
|
||||
We've seen how Kaggle can boost your YOLO26 projects by providing free access to powerful GPUs, making model training and evaluation efficient. Kaggle's platform is user-friendly, with pre-installed libraries for quick setup. The integration between Ultralytics YOLO26 and Kaggle creates a seamless environment for developing, training, and deploying state-of-the-art [computer vision](https://www.ultralytics.com/glossary/computer-vision-cv) models without the need for expensive hardware.
|
||||
|
||||
For more details, visit [Kaggle's documentation](https://www.kaggle.com/docs).
|
||||
|
||||
Interested in more YOLO26 integrations? Check out the [Ultralytics integration guide](https://docs.ultralytics.com/integrations/) to explore additional tools and capabilities for your machine learning projects.
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I install Ultralytics YOLO26 on Kaggle?
|
||||
|
||||
To install Ultralytics YOLO26 on Kaggle:
|
||||
|
||||
1. **Enable Internet**: Go to Settings panel and turn ON the Internet toggle
|
||||
2. **Install Package**: Run `!pip install ultralytics` in a notebook cell
|
||||
3. **Verify Installation**: Run `import ultralytics; ultralytics.checks()` to confirm
|
||||
|
||||
If you encounter dependency conflicts, try `!pip install --upgrade --force-reinstall ultralytics` or restart the kernel after installation. For detailed troubleshooting, see the [Installation section](#installation) above.
|
||||
|
||||
### How do I train a YOLO26 model on Kaggle?
|
||||
|
||||
Training a YOLO26 model on Kaggle is straightforward. First, access the [Kaggle YOLO26 Notebook](https://www.kaggle.com/code/glennjocherultralytics/ultralytics-yolo11-notebook). Sign in to your Kaggle account, copy and edit the notebook, and select a GPU under the accelerator settings. Run the notebook cells to start training. For more detailed steps, refer to our [YOLO26 Model Training guide](../modes/train.md).
|
||||
|
||||
### What are the benefits of using Kaggle for YOLO26 model training?
|
||||
|
||||
Kaggle offers several advantages for training YOLO26 models:
|
||||
|
||||
- **Free GPU Access**: Utilize powerful GPUs like NVIDIA Tesla P100 or T4 x2 for up to 30 hours per week.
|
||||
- **Pre-installed Libraries**: Libraries like TensorFlow and PyTorch are pre-installed, simplifying the setup.
|
||||
- **Community Collaboration**: Engage with a vast community of data scientists and machine learning enthusiasts.
|
||||
- **Version Control**: Easily manage different versions of your notebooks and revert to previous versions if needed.
|
||||
|
||||
For more details, visit our [Ultralytics integration guide](https://docs.ultralytics.com/integrations/).
|
||||
|
||||
### What common issues might I encounter when using Kaggle for YOLO26, and how can I resolve them?
|
||||
|
||||
Common issues include:
|
||||
|
||||
- **Access to GPUs**: Ensure you activate a GPU in your notebook settings. Kaggle allows up to 30 hours of GPU usage per week.
|
||||
- **Internet Not Enabled**: Make sure to enable internet in the Settings panel before installing packages.
|
||||
- **Dependency Conflicts**: Use `!pip install --upgrade --force-reinstall ultralytics` or install without dependencies using `!pip install --no-deps ultralytics`.
|
||||
- **Dataset Licenses**: Check the license of each dataset to understand usage restrictions.
|
||||
- **Saving and Committing Notebooks**: Click "Save Version" to save your notebook's state and access output files from the Output tab.
|
||||
- **Collaboration**: Kaggle supports asynchronous collaboration; multiple users cannot edit a notebook simultaneously.
|
||||
|
||||
For more troubleshooting tips, see the [Installation section](#installation) and our [Common Issues guide](../guides/yolo-common-issues.md).
|
||||
|
||||
### Why should I choose Kaggle over other platforms like Google Colab for training YOLO26 models?
|
||||
|
||||
Kaggle offers unique features that make it an excellent choice:
|
||||
|
||||
- **Public Notebooks**: Share your work with the community for feedback and collaboration.
|
||||
- **Free Access to TPUs**: Speed up training with powerful TPUs without extra costs.
|
||||
- **Comprehensive History**: Track changes over time with a detailed history of notebook commits.
|
||||
- **Resource Availability**: Significant resources are provided for each notebook session, including 12 hours of execution time for CPU and GPU sessions.
|
||||
|
||||
For a comparison with Google Colab, refer to our [Google Colab guide](./google-colab.md).
|
||||
|
||||
### How can I revert to a previous version of my Kaggle notebook?
|
||||
|
||||
To revert to a previous version:
|
||||
|
||||
1. Open the notebook and click on the three vertical dots in the top right corner.
|
||||
2. Select "View Versions."
|
||||
3. Find the version you want to revert to, click on the "..." menu next to it, and select "Revert to Version."
|
||||
4. Click "Save Version" to commit the changes.
|
||||
@@ -0,0 +1,209 @@
|
||||
---
|
||||
comments: true
|
||||
description: Learn how to set up and use MLflow logging with Ultralytics YOLO for enhanced experiment tracking, model reproducibility, and performance improvements.
|
||||
keywords: MLflow, Ultralytics YOLO, machine learning, experiment tracking, metrics logging, parameter logging, artifact logging
|
||||
---
|
||||
|
||||
# MLflow Integration for Ultralytics YOLO
|
||||
|
||||
<img width="1024" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/mlflow-integration-ultralytics-yolo.avif" alt="MLflow experiment tracking with Ultralytics YOLO">
|
||||
|
||||
## Introduction
|
||||
|
||||
Experiment logging is a crucial aspect of [machine learning](https://www.ultralytics.com/glossary/machine-learning-ml) workflows that enables tracking of various metrics, parameters, and artifacts. It helps to enhance model reproducibility, debug issues, and improve model performance. [Ultralytics](https://www.ultralytics.com/) YOLO, known for its real-time [object detection](https://www.ultralytics.com/glossary/object-detection) capabilities, now offers integration with [MLflow](https://mlflow.org/), an open-source platform for complete machine learning lifecycle management.
|
||||
|
||||
This documentation page is a comprehensive guide to setting up and utilizing the MLflow logging capabilities for your Ultralytics YOLO project.
|
||||
|
||||
## What is MLflow?
|
||||
|
||||
[MLflow](https://mlflow.org/) is an open-source platform developed by [Databricks](https://www.databricks.com/) for managing the end-to-end machine learning lifecycle. It includes tools for tracking experiments, packaging code into reproducible runs, and sharing and deploying models. MLflow is designed to work with any machine learning library and programming language.
|
||||
|
||||
## Features
|
||||
|
||||
- **Metrics Logging**: Logs metrics at the end of each epoch and at the end of the training.
|
||||
- **Parameter Logging**: Logs all the parameters used in the training.
|
||||
- **Artifacts Logging**: Logs model artifacts, including weights and configuration files, at the end of the training.
|
||||
|
||||
## Setup and Prerequisites
|
||||
|
||||
Ensure MLflow is installed. If not, install it using pip:
|
||||
|
||||
```bash
|
||||
pip install mlflow
|
||||
```
|
||||
|
||||
Make sure that MLflow logging is enabled in Ultralytics settings. Usually, this is controlled by the settings `mlflow` key. See the [settings](../quickstart.md#ultralytics-settings) page for more info.
|
||||
|
||||
!!! example "Update Ultralytics MLflow Settings"
|
||||
|
||||
=== "Python"
|
||||
|
||||
Within the Python environment, call the `update` method on the `settings` object to change your settings:
|
||||
```python
|
||||
from ultralytics import settings
|
||||
|
||||
# Update a setting
|
||||
settings.update({"mlflow": True})
|
||||
|
||||
# Reset settings to default values
|
||||
settings.reset()
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
If you prefer using the command-line interface, the following commands will allow you to modify your settings:
|
||||
```bash
|
||||
# Update a setting
|
||||
yolo settings mlflow=True
|
||||
|
||||
# Reset settings to default values
|
||||
yolo settings reset
|
||||
```
|
||||
|
||||
## How to Use
|
||||
|
||||
### Commands
|
||||
|
||||
1. **Set a Project Name**: You can set the project name via an environment variable:
|
||||
|
||||
```bash
|
||||
export MLFLOW_EXPERIMENT_NAME=YOUR_EXPERIMENT_NAME
|
||||
```
|
||||
|
||||
Or use the `project=<project>` argument when training a YOLO model, i.e. `yolo train project=my_project`.
|
||||
|
||||
2. **Set a Run Name**: Similar to setting a project name, you can set the run name via an environment variable:
|
||||
|
||||
```bash
|
||||
export MLFLOW_RUN=YOUR_RUN_NAME
|
||||
```
|
||||
|
||||
Or use the `name=<name>` argument when training a YOLO model, i.e. `yolo train project=my_project name=my_name`.
|
||||
|
||||
3. **Start Local MLflow Server**: To start tracking, use:
|
||||
|
||||
```bash
|
||||
mlflow server --backend-store-uri runs/mlflow
|
||||
```
|
||||
|
||||
This will start a local server at `http://127.0.0.1:5000` by default and save all mlflow logs to the 'runs/mlflow' directory. To specify a different URI, set the `MLFLOW_TRACKING_URI` environment variable.
|
||||
|
||||
4. **Kill MLflow Server Instances**: To stop all running MLflow instances, run:
|
||||
|
||||
```bash
|
||||
ps aux | grep 'mlflow' | grep -v 'grep' | awk '{print $2}' | xargs kill -9
|
||||
```
|
||||
|
||||
### Logging
|
||||
|
||||
The logging is taken care of by the `on_pretrain_routine_end`, `on_fit_epoch_end`, and `on_train_end` [callback functions](../reference/utils/callbacks/mlflow.md). These functions are automatically called during the respective stages of the training process, and they handle the logging of parameters, metrics, and artifacts.
|
||||
|
||||
## Examples
|
||||
|
||||
1. **Logging Custom Metrics**: You can add custom metrics to be logged by modifying the `trainer.metrics` dictionary before `on_fit_epoch_end` is called.
|
||||
|
||||
2. **View Experiment**: To view your logs, navigate to your MLflow server (usually `http://127.0.0.1:5000`) and select your experiment and run. <img width="1024" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/yolo-mlflow-experiment.avif" alt="MLflow experiment tracking interface for YOLO">
|
||||
|
||||
3. **View Run**: Runs are individual models inside an experiment. Click on a Run and see the Run details, including uploaded artifacts and model weights. <img width="1024" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/yolo-mlflow-run.avif" alt="MLflow run details with YOLO artifacts">
|
||||
|
||||
## Disabling MLflow
|
||||
|
||||
To turn off MLflow logging:
|
||||
|
||||
```bash
|
||||
yolo settings mlflow=False
|
||||
```
|
||||
|
||||
## Conclusion
|
||||
|
||||
MLflow logging integration with Ultralytics YOLO offers a streamlined way to keep track of your [machine learning experiments](https://www.ultralytics.com/blog/log-ultralytics-yolo-experiments-using-mlflow-integration). It empowers you to monitor performance metrics and manage artifacts effectively, thus aiding in robust model development and deployment. For further details please visit the MLflow [official documentation](https://mlflow.org/docs/latest/index.html).
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I set up MLflow logging with Ultralytics YOLO?
|
||||
|
||||
To set up MLflow logging with Ultralytics YOLO, you first need to ensure MLflow is installed. You can install it using pip:
|
||||
|
||||
```bash
|
||||
pip install mlflow
|
||||
```
|
||||
|
||||
Next, enable MLflow logging in Ultralytics settings. This can be controlled using the `mlflow` key. For more information, see the [settings guide](../quickstart.md#ultralytics-settings).
|
||||
|
||||
!!! example "Update Ultralytics MLflow Settings"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import settings
|
||||
|
||||
# Update a setting
|
||||
settings.update({"mlflow": True})
|
||||
|
||||
# Reset settings to default values
|
||||
settings.reset()
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Update a setting
|
||||
yolo settings mlflow=True
|
||||
|
||||
# Reset settings to default values
|
||||
yolo settings reset
|
||||
```
|
||||
|
||||
Finally, start a local MLflow server for tracking:
|
||||
|
||||
```bash
|
||||
mlflow server --backend-store-uri runs/mlflow
|
||||
```
|
||||
|
||||
### What metrics and parameters can I log using MLflow with Ultralytics YOLO?
|
||||
|
||||
Ultralytics YOLO with MLflow supports logging various metrics, parameters, and artifacts throughout the training process:
|
||||
|
||||
- **Metrics Logging**: Tracks metrics at the end of each [epoch](https://www.ultralytics.com/glossary/epoch) and upon training completion.
|
||||
- **Parameter Logging**: Logs all parameters used in the training process.
|
||||
- **Artifacts Logging**: Saves model artifacts like weights and configuration files after training.
|
||||
|
||||
For more detailed information, visit the [Ultralytics YOLO tracking documentation](#features).
|
||||
|
||||
### Can I disable MLflow logging once it is enabled?
|
||||
|
||||
Yes, you can disable MLflow logging for Ultralytics YOLO by updating the settings. Here's how you can do it using the CLI:
|
||||
|
||||
```bash
|
||||
yolo settings mlflow=False
|
||||
```
|
||||
|
||||
For further customization and resetting settings, refer to the [settings guide](../quickstart.md#ultralytics-settings).
|
||||
|
||||
### How can I start and stop an MLflow server for Ultralytics YOLO tracking?
|
||||
|
||||
To start an MLflow server for tracking your experiments in Ultralytics YOLO, use the following command:
|
||||
|
||||
```bash
|
||||
mlflow server --backend-store-uri runs/mlflow
|
||||
```
|
||||
|
||||
This command starts a local server at `http://127.0.0.1:5000` by default. If you need to stop running MLflow server instances, use the following bash command:
|
||||
|
||||
```bash
|
||||
ps aux | grep 'mlflow' | grep -v 'grep' | awk '{print $2}' | xargs kill -9
|
||||
```
|
||||
|
||||
Refer to the [commands section](#commands) for more command options.
|
||||
|
||||
### What are the benefits of integrating MLflow with Ultralytics YOLO for experiment tracking?
|
||||
|
||||
Integrating MLflow with Ultralytics YOLO offers several benefits for managing your machine learning experiments:
|
||||
|
||||
- **Enhanced Experiment Tracking**: Easily track and compare different runs and their outcomes.
|
||||
- **Improved Model Reproducibility**: Ensure that your experiments are reproducible by logging all parameters and artifacts.
|
||||
- **Performance Monitoring**: Visualize performance metrics over time to make data-driven decisions for model improvements.
|
||||
- **Streamlined Workflow**: Automate the logging process to focus more on model development rather than manual tracking.
|
||||
- **Collaborative Development**: Share experiment results with team members for better collaboration and knowledge sharing.
|
||||
|
||||
For an in-depth look at setting up and leveraging MLflow with Ultralytics YOLO, explore the [MLflow Integration for Ultralytics YOLO](#introduction) documentation.
|
||||
@@ -0,0 +1,377 @@
|
||||
---
|
||||
comments: true
|
||||
description: Optimize YOLO26 models for mobile and embedded devices by exporting to MNN format. Learn how to convert, deploy, and run inference with MNN.
|
||||
keywords: Ultralytics, YOLO26, MNN, model export, machine learning, deployment, mobile, embedded systems, deep learning, AI models, inference, quantization
|
||||
---
|
||||
|
||||
# MNN Export for YOLO26 Models and Deployment
|
||||
|
||||
## MNN
|
||||
|
||||
<p align="center">
|
||||
<img width="100%" src="https://mnn-docs.readthedocs.io/en/latest/_images/architecture.png" alt="MNN mobile neural network inference framework">
|
||||
</p>
|
||||
|
||||
[MNN](https://github.com/alibaba/MNN) is a highly efficient and lightweight deep learning framework. It supports inference and training of deep learning models and has industry-leading performance for inference and training on-device. At present, MNN has been integrated into more than 30 apps of Alibaba Inc, such as Taobao, Tmall, Youku, DingTalk, Xianyu, etc., covering more than 70 usage scenarios such as live broadcast, short video capture, search recommendation, product searching by image, interactive marketing, equity distribution, security risk control. In addition, MNN is also used on embedded devices, such as IoT.
|
||||
|
||||
<p align="center">
|
||||
<br>
|
||||
<iframe loading="lazy" width="720" height="405" src="https://www.youtube.com/embed/i34PacLIlq8"
|
||||
title="YouTube video player" frameborder="0"
|
||||
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"
|
||||
allowfullscreen>
|
||||
</iframe>
|
||||
<br>
|
||||
<strong>Watch:</strong> How to Export Ultralytics YOLO26 to MNN Format | Speed up Inference on Mobile Devices📱
|
||||
</p>
|
||||
|
||||
## Export to MNN: Converting Your YOLO26 Model
|
||||
|
||||
You can expand model compatibility and deployment flexibility by converting [Ultralytics YOLO](../models/yolo26.md) models to MNN format. This conversion optimizes your models for mobile and embedded environments, ensuring efficient performance on resource-constrained devices.
|
||||
|
||||
### Installation
|
||||
|
||||
To install the required packages, run:
|
||||
|
||||
!!! tip "Installation"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Install the required package for YOLO26 and MNN
|
||||
pip install ultralytics
|
||||
pip install MNN
|
||||
```
|
||||
|
||||
### Usage
|
||||
|
||||
All [Ultralytics YOLO26 models](../models/index.md) are designed to support export out of the box, making it easy to integrate them into your preferred deployment workflow. You can [view the full list of supported export formats and configuration options](../modes/export.md) to choose the best setup for your application.
|
||||
|
||||
!!! example "Usage"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load the YOLO26 model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Export the model to MNN format
|
||||
model.export(format="mnn") # creates 'yolo26n.mnn'
|
||||
|
||||
# Load the exported MNN model
|
||||
mnn_model = YOLO("yolo26n.mnn")
|
||||
|
||||
# Run inference
|
||||
results = mnn_model("https://ultralytics.com/images/bus.jpg")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Export a YOLO26n PyTorch model to MNN format
|
||||
yolo export model=yolo26n.pt format=mnn # creates 'yolo26n.mnn'
|
||||
|
||||
# Run inference with the exported model
|
||||
yolo predict model='yolo26n.mnn' source='https://ultralytics.com/images/bus.jpg'
|
||||
```
|
||||
|
||||
### Export Arguments
|
||||
|
||||
| Argument | Type | Default | Description |
|
||||
| -------- | ---------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `format` | `str` | `'mnn'` | Target format for the exported model, defining compatibility with various deployment environments. |
|
||||
| `imgsz` | `int` or `tuple` | `640` | Desired image size for the model input. Can be an integer for square images or a tuple `(height, width)` for specific dimensions. |
|
||||
| `half` | `bool` | `False` | Enables FP16 (half-precision) quantization, reducing model size and potentially speeding up inference on supported hardware. |
|
||||
| `int8` | `bool` | `False` | Activates INT8 quantization, further compressing the model and speeding up inference with minimal [accuracy](https://www.ultralytics.com/glossary/accuracy) loss, primarily for edge devices. |
|
||||
| `batch` | `int` | `1` | Specifies export model batch inference size or the max number of images the exported model will process concurrently in `predict` mode. |
|
||||
| `device` | `str` | `None` | Specifies the device for exporting: GPU (`device=0`), CPU (`device=cpu`), MPS for Apple silicon (`device=mps`). |
|
||||
|
||||
For more details about the export process, visit the [Ultralytics documentation page on exporting](../modes/export.md).
|
||||
|
||||
### MNN-Only Inference
|
||||
|
||||
A function that relies solely on MNN for YOLO26 inference and preprocessing is implemented, providing both Python and C++ versions for easy deployment in any scenario.
|
||||
|
||||
!!! example "MNN"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
import argparse
|
||||
|
||||
import MNN
|
||||
import MNN.cv as cv2
|
||||
import MNN.numpy as np
|
||||
|
||||
|
||||
def inference(model, img, precision, backend, thread):
|
||||
config = {}
|
||||
config["precision"] = precision
|
||||
config["backend"] = backend
|
||||
config["numThread"] = thread
|
||||
rt = MNN.nn.create_runtime_manager((config,))
|
||||
# net = MNN.nn.load_module_from_file(model, ['images'], ['output0'], runtime_manager=rt)
|
||||
net = MNN.nn.load_module_from_file(model, [], [], runtime_manager=rt)
|
||||
original_image = cv2.imread(img)
|
||||
ih, iw, _ = original_image.shape
|
||||
length = max((ih, iw))
|
||||
scale = length / 640
|
||||
image = np.pad(original_image, [[0, length - ih], [0, length - iw], [0, 0]], "constant")
|
||||
image = cv2.resize(
|
||||
image, (640, 640), 0.0, 0.0, cv2.INTER_LINEAR, -1, [0.0, 0.0, 0.0], [1.0 / 255.0, 1.0 / 255.0, 1.0 / 255.0]
|
||||
)
|
||||
image = image[..., ::-1] # BGR to RGB
|
||||
input_var = image[None]
|
||||
input_var = MNN.expr.convert(input_var, MNN.expr.NC4HW4)
|
||||
output_var = net.forward(input_var)
|
||||
output_var = MNN.expr.convert(output_var, MNN.expr.NCHW)
|
||||
output_var = output_var.squeeze()
|
||||
# output_var shape: [84, 8400]; 84 means: [cx, cy, w, h, prob * 80]
|
||||
cx = output_var[0]
|
||||
cy = output_var[1]
|
||||
w = output_var[2]
|
||||
h = output_var[3]
|
||||
probs = output_var[4:]
|
||||
# [cx, cy, w, h] -> [y0, x0, y1, x1]
|
||||
x0 = cx - w * 0.5
|
||||
y0 = cy - h * 0.5
|
||||
x1 = cx + w * 0.5
|
||||
y1 = cy + h * 0.5
|
||||
boxes = np.stack([x0, y0, x1, y1], axis=1)
|
||||
# ensure ratio is within the valid range [0.0, 1.0]
|
||||
boxes = np.clip(boxes, 0, 1)
|
||||
# get max prob and idx
|
||||
scores = np.max(probs, 0)
|
||||
class_ids = np.argmax(probs, 0)
|
||||
result_ids = MNN.expr.nms(boxes, scores, 100, 0.45, 0.25)
|
||||
print(result_ids.shape)
|
||||
# nms result box, score, ids
|
||||
result_boxes = boxes[result_ids]
|
||||
result_scores = scores[result_ids]
|
||||
result_class_ids = class_ids[result_ids]
|
||||
for i in range(len(result_boxes)):
|
||||
x0, y0, x1, y1 = result_boxes[i].read_as_tuple()
|
||||
y0 = int(y0 * scale)
|
||||
y1 = int(y1 * scale)
|
||||
x0 = int(x0 * scale)
|
||||
x1 = int(x1 * scale)
|
||||
# clamp to the original image size to handle cases where padding was applied
|
||||
x1 = min(iw, x1)
|
||||
y1 = min(ih, y1)
|
||||
print(result_class_ids[i])
|
||||
cv2.rectangle(original_image, (x0, y0), (x1, y1), (0, 0, 255), 2)
|
||||
cv2.imwrite("res.jpg", original_image)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--model", type=str, required=True, help="the yolo26 model path")
|
||||
parser.add_argument("--img", type=str, required=True, help="the input image path")
|
||||
parser.add_argument("--precision", type=str, default="normal", help="inference precision: normal, low, high, lowBF")
|
||||
parser.add_argument(
|
||||
"--backend",
|
||||
type=str,
|
||||
default="CPU",
|
||||
help="inference backend: CPU, OPENCL, OPENGL, NN, VULKAN, METAL, TRT, CUDA, HIAI",
|
||||
)
|
||||
parser.add_argument("--thread", type=int, default=4, help="inference using thread: int")
|
||||
args = parser.parse_args()
|
||||
inference(args.model, args.img, args.precision, args.backend, args.thread)
|
||||
```
|
||||
|
||||
=== "CPP"
|
||||
|
||||
```cpp
|
||||
#include <stdio.h>
|
||||
#include <MNN/ImageProcess.hpp>
|
||||
#include <MNN/expr/Module.hpp>
|
||||
#include <MNN/expr/Executor.hpp>
|
||||
#include <MNN/expr/ExprCreator.hpp>
|
||||
#include <MNN/expr/Executor.hpp>
|
||||
|
||||
#include <cv/cv.hpp>
|
||||
|
||||
using namespace MNN;
|
||||
using namespace MNN::Express;
|
||||
using namespace MNN::CV;
|
||||
|
||||
int main(int argc, const char* argv[]) {
|
||||
if (argc < 3) {
|
||||
MNN_PRINT("Usage: ./yolo26_demo.out model.mnn input.jpg [forwardType] [precision] [thread]\n");
|
||||
return 0;
|
||||
}
|
||||
int thread = 4;
|
||||
int precision = 0;
|
||||
int forwardType = MNN_FORWARD_CPU;
|
||||
if (argc >= 4) {
|
||||
forwardType = atoi(argv[3]);
|
||||
}
|
||||
if (argc >= 5) {
|
||||
precision = atoi(argv[4]);
|
||||
}
|
||||
if (argc >= 6) {
|
||||
thread = atoi(argv[5]);
|
||||
}
|
||||
MNN::ScheduleConfig sConfig;
|
||||
sConfig.type = static_cast<MNNForwardType>(forwardType);
|
||||
sConfig.numThread = thread;
|
||||
BackendConfig bConfig;
|
||||
bConfig.precision = static_cast<BackendConfig::PrecisionMode>(precision);
|
||||
sConfig.backendConfig = &bConfig;
|
||||
std::shared_ptr<Executor::RuntimeManager> rtmgr = std::shared_ptr<Executor::RuntimeManager>(Executor::RuntimeManager::createRuntimeManager(sConfig));
|
||||
if(rtmgr == nullptr) {
|
||||
MNN_ERROR("Empty RuntimeManger\n");
|
||||
return 0;
|
||||
}
|
||||
rtmgr->setCache(".cachefile");
|
||||
|
||||
std::shared_ptr<Module> net(Module::load(std::vector<std::string>{}, std::vector<std::string>{}, argv[1], rtmgr));
|
||||
auto original_image = imread(argv[2]);
|
||||
auto dims = original_image->getInfo()->dim;
|
||||
int ih = dims[0];
|
||||
int iw = dims[1];
|
||||
int len = ih > iw ? ih : iw;
|
||||
float scale = len / 640.0;
|
||||
std::vector<int> padvals { 0, len - ih, 0, len - iw, 0, 0 };
|
||||
auto pads = _Const(static_cast<void*>(padvals.data()), {3, 2}, NCHW, halide_type_of<int>());
|
||||
auto image = _Pad(original_image, pads, CONSTANT);
|
||||
image = resize(image, Size(640, 640), 0, 0, INTER_LINEAR, -1, {0., 0., 0.}, {1./255., 1./255., 1./255.});
|
||||
image = cvtColor(image, COLOR_BGR2RGB);
|
||||
auto input = _Unsqueeze(image, {0});
|
||||
input = _Convert(input, NC4HW4);
|
||||
auto outputs = net->onForward({input});
|
||||
auto output = _Convert(outputs[0], NCHW);
|
||||
output = _Squeeze(output);
|
||||
// output shape: [84, 8400]; 84 means: [cx, cy, w, h, prob * 80]
|
||||
auto cx = _Gather(output, _Scalar<int>(0));
|
||||
auto cy = _Gather(output, _Scalar<int>(1));
|
||||
auto w = _Gather(output, _Scalar<int>(2));
|
||||
auto h = _Gather(output, _Scalar<int>(3));
|
||||
std::vector<int> startvals { 4, 0 };
|
||||
auto start = _Const(static_cast<void*>(startvals.data()), {2}, NCHW, halide_type_of<int>());
|
||||
std::vector<int> sizevals { -1, -1 };
|
||||
auto size = _Const(static_cast<void*>(sizevals.data()), {2}, NCHW, halide_type_of<int>());
|
||||
auto probs = _Slice(output, start, size);
|
||||
// [cx, cy, w, h] -> [y0, x0, y1, x1]
|
||||
auto x0 = cx - w * _Const(0.5);
|
||||
auto y0 = cy - h * _Const(0.5);
|
||||
auto x1 = cx + w * _Const(0.5);
|
||||
auto y1 = cy + h * _Const(0.5);
|
||||
auto boxes = _Stack({x0, y0, x1, y1}, 1);
|
||||
// ensure ratio is within the valid range [0.0, 1.0]
|
||||
boxes = _Maximum(boxes, _Scalar<float>(0.0f));
|
||||
boxes = _Minimum(boxes, _Scalar<float>(1.0f));
|
||||
auto scores = _ReduceMax(probs, {0});
|
||||
auto ids = _ArgMax(probs, 0);
|
||||
auto result_ids = _Nms(boxes, scores, 100, 0.45, 0.25);
|
||||
auto result_ptr = result_ids->readMap<int>();
|
||||
auto box_ptr = boxes->readMap<float>();
|
||||
auto ids_ptr = ids->readMap<int>();
|
||||
auto score_ptr = scores->readMap<float>();
|
||||
for (int i = 0; i < 100; i++) {
|
||||
auto idx = result_ptr[i];
|
||||
if (idx < 0) break;
|
||||
auto x0 = box_ptr[idx * 4 + 0] * scale;
|
||||
auto y0 = box_ptr[idx * 4 + 1] * scale;
|
||||
auto x1 = box_ptr[idx * 4 + 2] * scale;
|
||||
auto y1 = box_ptr[idx * 4 + 3] * scale;
|
||||
// clamp to the original image size to handle cases where padding was applied
|
||||
x1 = std::min(static_cast<float>(iw), x1);
|
||||
y1 = std::min(static_cast<float>(ih), y1);
|
||||
auto class_idx = ids_ptr[idx];
|
||||
auto score = score_ptr[idx];
|
||||
rectangle(original_image, {x0, y0}, {x1, y1}, {0, 0, 255}, 2);
|
||||
}
|
||||
if (imwrite("res.jpg", original_image)) {
|
||||
MNN_PRINT("result image write to `res.jpg`.\n");
|
||||
}
|
||||
rtmgr->updateCache();
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
|
||||
## Summary
|
||||
|
||||
In this guide, we introduce how to export the Ultralytics YOLO26 model to MNN and use MNN for inference. The MNN format provides excellent performance for [edge AI](https://www.ultralytics.com/glossary/edge-ai) applications, making it ideal for deploying computer vision models on resource-constrained devices.
|
||||
|
||||
For more usage, please refer to the [MNN documentation](https://mnn-docs.readthedocs.io/en/latest).
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I export Ultralytics YOLO26 models to MNN format?
|
||||
|
||||
To export your Ultralytics YOLO26 model to MNN format, follow these steps:
|
||||
|
||||
!!! example "Export"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load the YOLO26 model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Export to MNN format
|
||||
model.export(format="mnn") # creates 'yolo26n.mnn' with fp32 weight
|
||||
model.export(format="mnn", half=True) # creates 'yolo26n.mnn' with fp16 weight
|
||||
model.export(format="mnn", int8=True) # creates 'yolo26n.mnn' with int8 weight
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
yolo export model=yolo26n.pt format=mnn # creates 'yolo26n.mnn' with fp32 weight
|
||||
yolo export model=yolo26n.pt format=mnn half=True # creates 'yolo26n.mnn' with fp16 weight
|
||||
yolo export model=yolo26n.pt format=mnn int8=True # creates 'yolo26n.mnn' with int8 weight
|
||||
```
|
||||
|
||||
For detailed export options, check the [Export](../modes/export.md) page in the documentation.
|
||||
|
||||
### How do I predict with an exported YOLO26 MNN model?
|
||||
|
||||
To predict with an exported YOLO26 MNN model, use the `predict` function from the YOLO class.
|
||||
|
||||
!!! example "Predict"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load the YOLO26 MNN model
|
||||
model = YOLO("yolo26n.mnn")
|
||||
|
||||
# Export to MNN format
|
||||
results = model("https://ultralytics.com/images/bus.jpg") # predict with `fp32`
|
||||
results = model("https://ultralytics.com/images/bus.jpg", half=True) # predict with `fp16` if device support
|
||||
|
||||
for result in results:
|
||||
result.show() # display to screen
|
||||
result.save(filename="result.jpg") # save to disk
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
yolo predict model='yolo26n.mnn' source='https://ultralytics.com/images/bus.jpg' # predict with `fp32`
|
||||
yolo predict model='yolo26n.mnn' source='https://ultralytics.com/images/bus.jpg' --half=True # predict with `fp16` if device support
|
||||
```
|
||||
|
||||
### What platforms are supported for MNN?
|
||||
|
||||
MNN is versatile and supports various platforms:
|
||||
|
||||
- **Mobile**: Android, iOS, Harmony.
|
||||
- **Embedded Systems and IoT Devices**: Devices like [Raspberry Pi](../guides/raspberry-pi.md) and NVIDIA Jetson.
|
||||
- **Desktop and Servers**: Linux, Windows, and macOS.
|
||||
|
||||
### How can I deploy Ultralytics YOLO26 MNN models on Mobile Devices?
|
||||
|
||||
To deploy your YOLO26 models on Mobile devices:
|
||||
|
||||
1. **Build for Android**: Follow the [MNN Android](https://github.com/alibaba/MNN/tree/master/project/android) guide.
|
||||
2. **Build for iOS**: Follow the [MNN iOS](https://github.com/alibaba/MNN/tree/master/project/ios) guide.
|
||||
3. **Build for Harmony**: Follow the [MNN Harmony](https://github.com/alibaba/MNN/tree/master/project/harmony) guide.
|
||||
@@ -0,0 +1,248 @@
|
||||
---
|
||||
comments: true
|
||||
description: Optimize YOLO26 models for mobile and embedded devices by exporting to NCNN format. Enhance performance in resource-constrained environments.
|
||||
keywords: Ultralytics, YOLO26, NCNN, model export, machine learning, deployment, mobile, embedded systems, deep learning, AI models, Vulkan, GPU acceleration
|
||||
---
|
||||
|
||||
# Ultralytics YOLO NCNN Export
|
||||
|
||||
Deploying [computer vision](https://www.ultralytics.com/glossary/computer-vision-cv) models on devices with limited computational power, such as mobile or embedded systems, requires careful format selection. Using an optimized format ensures that even resource-constrained devices can handle advanced computer vision tasks efficiently.
|
||||
|
||||
Exporting to NCNN format allows you to optimize your [Ultralytics YOLO26](https://github.com/ultralytics/ultralytics) models for lightweight device-based applications. This guide covers how to convert your models to NCNN format for improved performance on mobile and embedded devices.
|
||||
|
||||
## Why Export to NCNN?
|
||||
|
||||
<p align="center">
|
||||
<img width="100%" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/ncnn-overview.avif" alt="NCNN high-performance neural network inference framework">
|
||||
</p>
|
||||
|
||||
The [NCNN](https://github.com/Tencent/ncnn) framework, developed by Tencent, is a high-performance [neural network](https://www.ultralytics.com/glossary/neural-network-nn) inference computing framework optimized specifically for mobile platforms, including mobile phones, embedded devices, and IoT devices. NCNN is compatible with a wide range of platforms, including Linux, Android, iOS, and macOS.
|
||||
|
||||
NCNN is known for its fast processing speed on mobile CPUs and enables rapid deployment of [deep learning](https://www.ultralytics.com/glossary/deep-learning-dl) models to mobile platforms, making it an excellent choice for building AI-powered applications.
|
||||
|
||||
## Key Features of NCNN Models
|
||||
|
||||
NCNN models provide several key features that enable on-device [machine learning](https://www.ultralytics.com/glossary/machine-learning-ml), helping developers deploy models on mobile, embedded, and edge devices:
|
||||
|
||||
- **Efficient and High-Performance**: NCNN models are lightweight and optimized for mobile and embedded devices like Raspberry Pi with limited resources, while maintaining high [accuracy](https://www.ultralytics.com/glossary/accuracy) on computer vision tasks.
|
||||
|
||||
- **Quantization**: NCNN supports quantization, a technique that reduces the [precision](https://www.ultralytics.com/glossary/precision) of model weights and activations to improve performance and reduce memory footprint.
|
||||
|
||||
- **Compatibility**: NCNN models are compatible with popular deep learning frameworks including [TensorFlow](https://www.tensorflow.org/), [Caffe](https://caffe.berkeleyvision.org/), and [ONNX](https://onnx.ai/), allowing developers to leverage existing models and workflows.
|
||||
|
||||
- **Ease of Use**: NCNN provides user-friendly tools for converting models between formats, ensuring smooth interoperability across different development environments.
|
||||
|
||||
- **Vulkan GPU Acceleration**: NCNN supports Vulkan for GPU-accelerated inference across multiple vendors including AMD, Intel, and other non-NVIDIA GPUs, enabling high-performance deployment on a wider range of hardware.
|
||||
|
||||
## Deployment Options with NCNN
|
||||
|
||||
NCNN models are compatible with a variety of deployment platforms:
|
||||
|
||||
- **Mobile Deployment**: Optimized for Android and iOS, enabling seamless integration into mobile applications for efficient on-device inference.
|
||||
|
||||
- **Embedded Systems and IoT Devices**: Ideal for resource-constrained devices like Raspberry Pi and NVIDIA Jetson. If standard inference on a Raspberry Pi with the [Ultralytics Guide](../guides/raspberry-pi.md) is insufficient, NCNN can provide significant performance improvements.
|
||||
|
||||
- **Desktop and Server Deployment**: Supports deployment across Linux, Windows, and macOS for development, training, and evaluation workflows.
|
||||
|
||||
## Vulkan GPU Acceleration
|
||||
|
||||
NCNN supports GPU acceleration through Vulkan, enabling high-performance inference on a wide range of GPUs including AMD, Intel, and other non-NVIDIA graphics cards. This is particularly useful for:
|
||||
|
||||
- **Cross-Vendor GPU Support**: Unlike CUDA, which is limited to NVIDIA GPUs, Vulkan works across multiple GPU vendors.
|
||||
- **Multi-GPU Systems**: Select a specific Vulkan device in systems with multiple GPUs using `device="vulkan:0"`, `device="vulkan:1"`, etc.
|
||||
- **Edge and Desktop Deployments**: Leverage GPU acceleration on devices where CUDA is not available.
|
||||
|
||||
To use Vulkan acceleration, specify the Vulkan device when running inference:
|
||||
|
||||
!!! example "Vulkan Inference"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load the exported NCNN model
|
||||
ncnn_model = YOLO("./yolo26n_ncnn_model")
|
||||
|
||||
# Run inference with Vulkan GPU acceleration (first Vulkan device)
|
||||
results = ncnn_model("https://ultralytics.com/images/bus.jpg", device="vulkan:0")
|
||||
|
||||
# Use second Vulkan device in multi-GPU systems
|
||||
results = ncnn_model("https://ultralytics.com/images/bus.jpg", device="vulkan:1")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Run inference with Vulkan GPU acceleration
|
||||
yolo predict model='./yolo26n_ncnn_model' source='https://ultralytics.com/images/bus.jpg' device=vulkan:0
|
||||
```
|
||||
|
||||
!!! tip "Vulkan Requirements"
|
||||
|
||||
Ensure you have Vulkan drivers installed for your GPU. Most modern GPU drivers include Vulkan support by default. You can verify Vulkan availability using tools like `vulkaninfo` on Linux or the Vulkan SDK on Windows.
|
||||
|
||||
## Export to NCNN: Converting Your YOLO26 Model
|
||||
|
||||
You can expand model compatibility and deployment flexibility by converting YOLO26 models to NCNN format.
|
||||
|
||||
### Installation
|
||||
|
||||
To install the required packages, run:
|
||||
|
||||
!!! tip "Installation"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Install the required package for YOLO26
|
||||
pip install ultralytics
|
||||
```
|
||||
|
||||
For detailed instructions and best practices, see the [Ultralytics Installation guide](../quickstart.md). If you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions.
|
||||
|
||||
### Usage
|
||||
|
||||
All [Ultralytics YOLO26 models](../models/index.md) are designed to support export out of the box, making it easy to integrate them into your preferred deployment workflow. You can [view the full list of supported export formats and configuration options](../modes/export.md) to choose the best setup for your application.
|
||||
|
||||
!!! example "Usage"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load the YOLO26 model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Export the model to NCNN format
|
||||
model.export(format="ncnn") # creates '/yolo26n_ncnn_model'
|
||||
|
||||
# Load the exported NCNN model
|
||||
ncnn_model = YOLO("./yolo26n_ncnn_model")
|
||||
|
||||
# Run inference
|
||||
results = ncnn_model("https://ultralytics.com/images/bus.jpg")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Export a YOLO26n PyTorch model to NCNN format
|
||||
yolo export model=yolo26n.pt format=ncnn # creates '/yolo26n_ncnn_model'
|
||||
|
||||
# Run inference with the exported model
|
||||
yolo predict model='./yolo26n_ncnn_model' source='https://ultralytics.com/images/bus.jpg'
|
||||
```
|
||||
|
||||
### Export Arguments
|
||||
|
||||
| Argument | Type | Default | Description |
|
||||
| -------- | ---------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `format` | `str` | `'ncnn'` | Target format for the exported model, defining compatibility with various deployment environments. |
|
||||
| `imgsz` | `int` or `tuple` | `640` | Desired image size for the model input. Can be an integer for square images or a tuple `(height, width)` for specific dimensions. |
|
||||
| `half` | `bool` | `False` | Enables FP16 (half-precision) quantization, reducing model size and potentially speeding up inference on supported hardware. |
|
||||
| `batch` | `int` | `1` | Specifies export model batch inference size or the max number of images the exported model will process concurrently in `predict` mode. |
|
||||
| `device` | `str` | `None` | Specifies the device for exporting: GPU (`device=0`), CPU (`device=cpu`), MPS for Apple silicon (`device=mps`). |
|
||||
|
||||
For more details about the export process, visit the [Ultralytics documentation page on exporting](../modes/export.md).
|
||||
|
||||
## Deploying Exported YOLO26 NCNN Models
|
||||
|
||||
After exporting your Ultralytics YOLO26 models to NCNN format, you can deploy them using the `YOLO("yolo26n_ncnn_model/")` method as shown in the usage example above. For platform-specific deployment instructions, see the following resources:
|
||||
|
||||
- **[Android](https://github.com/Tencent/ncnn/wiki/how-to-build#build-for-android)**: Build and integrate NCNN models for [object detection](https://www.ultralytics.com/glossary/object-detection) in Android applications.
|
||||
|
||||
- **[macOS](https://github.com/Tencent/ncnn/wiki/how-to-build#build-for-macos)**: Deploy NCNN models on macOS systems.
|
||||
|
||||
- **[Linux](https://github.com/Tencent/ncnn/wiki/how-to-build#build-for-linux)**: Deploy NCNN models on Linux devices including Raspberry Pi and similar embedded systems.
|
||||
|
||||
- **[Windows x64](https://github.com/Tencent/ncnn/wiki/how-to-build#build-for-windows-x64-using-visual-studio-community-2017)**: Deploy NCNN models on Windows x64 using Visual Studio.
|
||||
|
||||
## Summary
|
||||
|
||||
This guide covered exporting Ultralytics YOLO26 models to NCNN format for improved efficiency and speed on resource-constrained devices.
|
||||
|
||||
For additional details, refer to the [official NCNN documentation](https://ncnn.readthedocs.io/en/latest/index.html). For other export options, visit our [integration guide page](index.md).
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I export Ultralytics YOLO26 models to NCNN format?
|
||||
|
||||
To export your Ultralytics YOLO26 model to NCNN format:
|
||||
|
||||
- **Python**: Use the `export` method from the YOLO class.
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load the YOLO26 model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Export to NCNN format
|
||||
model.export(format="ncnn") # creates '/yolo26n_ncnn_model'
|
||||
```
|
||||
|
||||
- **CLI**: Use the `yolo export` command.
|
||||
|
||||
```bash
|
||||
yolo export model=yolo26n.pt format=ncnn # creates '/yolo26n_ncnn_model'
|
||||
```
|
||||
|
||||
For detailed export options, see the [Export](../modes/export.md) documentation.
|
||||
|
||||
### What are the advantages of exporting YOLO26 models to NCNN?
|
||||
|
||||
Exporting your Ultralytics YOLO26 models to NCNN offers several benefits:
|
||||
|
||||
- **Efficiency**: NCNN models are optimized for mobile and embedded devices, ensuring high performance even with limited computational resources.
|
||||
- **Quantization**: NCNN supports techniques like quantization that improve model speed and reduce memory usage.
|
||||
- **Broad Compatibility**: You can deploy NCNN models on multiple platforms, including Android, iOS, Linux, and macOS.
|
||||
- **Vulkan GPU Acceleration**: Leverage GPU acceleration on AMD, Intel, and other non-NVIDIA GPUs via Vulkan for faster inference.
|
||||
|
||||
For more details, see the [Why Export to NCNN?](#why-export-to-ncnn) section.
|
||||
|
||||
### Why should I use NCNN for my mobile AI applications?
|
||||
|
||||
NCNN, developed by Tencent, is specifically optimized for mobile platforms. Key reasons to use NCNN include:
|
||||
|
||||
- **High Performance**: Designed for efficient and fast processing on mobile CPUs.
|
||||
- **Cross-Platform**: Compatible with popular frameworks such as [TensorFlow](https://www.ultralytics.com/glossary/tensorflow) and ONNX, making it easier to convert and deploy models across different platforms.
|
||||
- **Community Support**: Active community support ensures continual improvements and updates.
|
||||
|
||||
For more information, see the [Key Features of NCNN Models](#key-features-of-ncnn-models) section.
|
||||
|
||||
### What platforms are supported for NCNN [model deployment](https://www.ultralytics.com/glossary/model-deployment)?
|
||||
|
||||
NCNN is versatile and supports various platforms:
|
||||
|
||||
- **Mobile**: Android, iOS.
|
||||
- **Embedded Systems and IoT Devices**: Devices like Raspberry Pi and NVIDIA Jetson.
|
||||
- **Desktop and Servers**: Linux, Windows, and macOS.
|
||||
|
||||
For improved performance on Raspberry Pi, consider using NCNN format as detailed in our [Raspberry Pi Guide](../guides/raspberry-pi.md).
|
||||
|
||||
### How can I deploy Ultralytics YOLO26 NCNN models on Android?
|
||||
|
||||
To deploy your YOLO26 models on Android:
|
||||
|
||||
1. **Build for Android**: Follow the [NCNN Build for Android](https://github.com/Tencent/ncnn/wiki/how-to-build#build-for-android) guide.
|
||||
2. **Integrate with Your App**: Use the NCNN Android SDK to integrate the exported model into your application for efficient on-device inference.
|
||||
|
||||
For detailed instructions, see [Deploying Exported YOLO26 NCNN Models](#deploying-exported-yolo26-ncnn-models).
|
||||
|
||||
For more advanced guides and use cases, visit the [Ultralytics deployment guide](../guides/model-deployment-options.md).
|
||||
|
||||
### How do I use Vulkan GPU acceleration with NCNN models?
|
||||
|
||||
NCNN supports Vulkan for GPU acceleration on AMD, Intel, and other non-NVIDIA GPUs. To use Vulkan:
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load NCNN model and run with Vulkan GPU
|
||||
model = YOLO("yolo26n_ncnn_model")
|
||||
results = model("image.jpg", device="vulkan:0") # Use first Vulkan device
|
||||
```
|
||||
|
||||
For multi-GPU systems, specify the device index (e.g., `vulkan:1` for the second GPU). Ensure Vulkan drivers are installed for your GPU. See the [Vulkan GPU Acceleration](#vulkan-gpu-acceleration) section for more details.
|
||||
@@ -0,0 +1,178 @@
|
||||
---
|
||||
comments: true
|
||||
description: Integrate Neptune with Ultralytics YOLO26 to track experiments, visualize metrics, log model checkpoints, and organize training metadata.
|
||||
keywords: Neptune, YOLO26, Ultralytics, experiment tracking, MLOps, model registry, visualization, computer vision
|
||||
---
|
||||
|
||||
!!! warning "Neptune acquisition and SaaS deprecation"
|
||||
|
||||
Neptune has entered into an agreement to be acquired by OpenAI and will wind down its hosted (SaaS) service after a transition period ending March 4, 2026. Review the [official announcement](https://neptune.ai/blog/we-are-joining-openai) and plan migrations or exports accordingly.
|
||||
|
||||
# Experiment Tracking with Neptune
|
||||
|
||||
[Neptune](https://neptune.ai/) is a metadata store for MLOps, built for teams that run a lot of experiments. It gives you a single place to log, store, display, organize, compare, and query all your model building metadata.
|
||||
|
||||
Ultralytics YOLO26 integrates with Neptune to streamline [experiment tracking](https://www.ultralytics.com/glossary/experiment-tracking). This integration allows you to automatically log training metrics, visualize model predictions, and store model artifacts without writing custom logging code.
|
||||
|
||||
<p align="center">
|
||||
<img width="800" src="https://docs.neptune.ai/img/app/app_preview.png" alt="Neptune.ai ML experiment tracking dashboard">
|
||||
</p>
|
||||
|
||||
## Key Features
|
||||
|
||||
- **Automated Logging**: Automatically log key training metrics such as box loss, classification loss, and [mAP](https://www.ultralytics.com/glossary/mean-average-precision-map).
|
||||
- **Image Visualization**: View training mosaics and validation predictions directly in the Neptune dashboard.
|
||||
- **Model Checkpointing**: Upload and version control your trained model weights (`best.pt`) automatically at the end of training.
|
||||
- **Hyperparameter Tracking**: Log all configuration parameters to ensure full reproducibility of your experiments.
|
||||
- **Interactive Plots**: Visualize confusion matrices and precision-recall curves to analyze model performance.
|
||||
|
||||
## Installation
|
||||
|
||||
To use Neptune with Ultralytics, you will need to install the `neptune` client package along with `ultralytics`.
|
||||
|
||||
!!! tip "Installation"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Install the required packages
|
||||
pip install ultralytics neptune
|
||||
|
||||
# Enable Neptune integration in Ultralytics settings
|
||||
yolo settings neptune=True
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
Before you start training, you need to connect your local environment to your Neptune project. You will need your **API Token** and **Project Name** from your Neptune dashboard.
|
||||
|
||||
### 1. Get Your Credentials
|
||||
|
||||
1. Log in to [Neptune.ai](https://neptune.ai/).
|
||||
2. Create a new project (or select an existing one).
|
||||
3. Go to your user menu and get your **API Token**.
|
||||
|
||||
### 2. Set Environment Variables
|
||||
|
||||
The securest way to handle credentials is via environment variables. Note that the Ultralytics Neptune callback reads the YOLO `project` argument and does not use `NEPTUNE_PROJECT`. Pass the full Neptune slug (e.g., `workspace/name`) via `project=` in your training command; otherwise Neptune will try to use the literal default `"Ultralytics"` and the run will fail.
|
||||
|
||||
=== "Bash (Linux/Mac)"
|
||||
|
||||
```bash
|
||||
export NEPTUNE_API_TOKEN="your_long_api_token_here" # required
|
||||
```
|
||||
|
||||
=== "PowerShell (Windows)"
|
||||
|
||||
```powershell
|
||||
$Env:NEPTUNE_API_TOKEN = "your_long_api_token_here" # required
|
||||
```
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
os.environ["NEPTUNE_API_TOKEN"] = "your_long_api_token_here"
|
||||
os.environ["NEPTUNE_PROJECT"] = "your_workspace/your_project"
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
Once configured, you can start training your YOLO26 models. The Neptune integration works automatically when the `neptune` package is installed and the integration is enabled in settings.
|
||||
|
||||
### Training Example
|
||||
|
||||
!!! example "Train YOLO26 with Neptune Logging"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load a model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Train the model
|
||||
# Pass the Neptune project slug as the 'project' argument (workspace/name)
|
||||
results = model.train(data="coco8.yaml", epochs=10, project="my-workspace/my-project", name="experiment-1")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Train via CLI
|
||||
# project must be the Neptune slug (workspace/name); otherwise run creation will fail
|
||||
yolo train data=coco8.yaml epochs=10 project=my-workspace/my-project name=experiment-1
|
||||
```
|
||||
|
||||
## Understanding the Integration
|
||||
|
||||
The following diagram illustrates how the Ultralytics Training pipeline interacts with Neptune to log various artifacts and metrics.
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
A[YOLO Training Loop] --> B{Neptune Callback}
|
||||
B -->|Log Scalars| C[Loss, mAP, LR]
|
||||
B -->|Log Images| D[Mosaics, Preds]
|
||||
B -->|Log Artifacts| E[Model Weights]
|
||||
B -->|Log Metadata| F[Hyperparameters]
|
||||
|
||||
C --> G[Neptune Server]
|
||||
D --> G
|
||||
E --> G
|
||||
F --> G
|
||||
|
||||
G --> H[Neptune Web Dashboard]
|
||||
```
|
||||
|
||||
### What is Logged?
|
||||
|
||||
When you run the training command, the Neptune integration automatically captures the following data structure in your run:
|
||||
|
||||
1. **Configuration/Hyperparameters**: All training arguments (epochs, lr0, optimizer, etc.) are logged under the Configuration section.
|
||||
2. **Configuration/Model**: The model architecture and definition.
|
||||
3. **Metrics**:
|
||||
- **Train**: `box_loss`, `cls_loss`, `dfl_loss`, `lr` (learning rate).
|
||||
- **Metrics**: `precision`, `recall`, `mAP50`, `mAP50-95`.
|
||||
4. **Images**:
|
||||
- `Mosaic`: Training batches showing data augmentation.
|
||||
- `Validation`: Ground truth labels and model predictions on validation data.
|
||||
- `Plots`: Confusion matrices, Precision-Recall curves.
|
||||
5. **Weights**: The final trained model (`best.pt`) is uploaded to the `weights` folder in the Neptune run.
|
||||
|
||||
## Advanced Usage
|
||||
|
||||
### Organizing Runs
|
||||
|
||||
You can use the standard Ultralytics `project` and `name` arguments to organize your runs in Neptune.
|
||||
|
||||
- `project`: Must be the Neptune project slug `workspace/name`; this is what the callback passes to `neptune.init_run`.
|
||||
- `name`: Acts as the identifier for the specific run.
|
||||
|
||||
### Custom Logging
|
||||
|
||||
If you need to log additional custom metrics alongside the automatic logging, you can access the Neptune run instance. Note that you will need to modify the trainer logic or create a custom callback to access the specific run object, as the Ultralytics integration handles the run lifecycle internally.
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I disable Neptune logging?
|
||||
|
||||
If you have installed `neptune` but wish to disable logging for a specific session or globally, you can modify the YOLO settings.
|
||||
|
||||
```bash
|
||||
# Disable Neptune integration
|
||||
yolo settings neptune=False
|
||||
```
|
||||
|
||||
### My images are not uploading. What's wrong?
|
||||
|
||||
Ensure that your network allows connections to Neptune's servers. Additionally, image logging usually occurs at specific intervals (e.g., end of epochs or end of training). If you interrupt training early using `Ctrl+C`, some final artifacts like confusion matrices or the best model weights might not be uploaded.
|
||||
|
||||
### Can I log to a specific Neptune run ID?
|
||||
|
||||
The current integration automatically creates a new run for each training session. To resume logging to an existing run, you would typically need to handle the Neptune initialization manually in Python code, which falls outside the scope of the automatic integration. However, Ultralytics supports resuming training locally, which will create a new run in Neptune to track the resumed epochs.
|
||||
|
||||
### Where can I find the model weights in Neptune?
|
||||
|
||||
In your Neptune dashboard, navigate to the **Artifacts** or **All Metadata** section. You will find a `weights` folder containing your `best.pt` file, which you can download for deployment.
|
||||
@@ -0,0 +1,213 @@
|
||||
---
|
||||
comments: true
|
||||
description: Enhance YOLO26 performance using Neural Magic's DeepSparse Engine. Learn how to deploy and benchmark YOLO26 models on CPUs for efficient object detection.
|
||||
keywords: YOLO26, DeepSparse, Neural Magic, model optimization, object detection, inference speed, CPU performance, sparsity, pruning, quantization
|
||||
---
|
||||
|
||||
# Optimizing YOLO26 Inferences with Neural Magic's DeepSparse Engine
|
||||
|
||||
When deploying [object detection](https://www.ultralytics.com/glossary/object-detection) models like [Ultralytics YOLO26](https://www.ultralytics.com/) on various hardware, you can bump into unique issues like optimization. This is where YOLO26's integration with Neural Magic's DeepSparse Engine steps in. It transforms the way YOLO26 models are executed and enables GPU-level performance directly on CPUs.
|
||||
|
||||
This guide shows you how to deploy YOLO26 using Neural Magic's DeepSparse, how to run inferences, and also how to benchmark performance to ensure it is optimized.
|
||||
|
||||
!!! danger "SparseML EOL"
|
||||
|
||||
Neural Magic was [acquired by Red Hat in January 2025](https://www.redhat.com/en/about/press-releases/red-hat-completes-acquisition-neural-magic-fuel-optimized-generative-ai-innovation-across-hybrid-cloud), and is deprecating the community versions of their `deepsparse`, `sparseml`, `sparsezoo`, and `sparsify` libraries. For additional information, see the notice posted [in the Readme on the `sparseml` GitHub repository](https://github.com/neuralmagic/sparsify/blob/5eb26a4e21b497ce573d10024e318a5ce48a7f9c/README.md#-2025-end-of-life-announcement-deepsparse-sparseml-sparsezoo-and-sparsify).
|
||||
|
||||
## Neural Magic's DeepSparse
|
||||
|
||||
<p align="center">
|
||||
<img width="640" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/neural-magic-deepsparse-overview.avif" alt="Neural Magic's DeepSparse Overview">
|
||||
</p>
|
||||
|
||||
[Neural Magic's DeepSparse](https://github.com/neuralmagic/deepsparse/blob/main/README.md) is an inference run-time designed to optimize the execution of neural networks on CPUs. It applies advanced techniques like sparsity, pruning, and quantization to dramatically reduce computational demands while maintaining accuracy. DeepSparse offers an agile solution for efficient and scalable [neural network](https://www.ultralytics.com/glossary/neural-network-nn) execution across various devices.
|
||||
|
||||
## Benefits of Integrating Neural Magic's DeepSparse with YOLO26
|
||||
|
||||
Before diving into how to deploy YOLO26 using DeepSparse, let's understand the benefits of using DeepSparse. Some key advantages include:
|
||||
|
||||
- **Enhanced Inference Speed**: Achieves up to 525 FPS (on YOLO11n), significantly speeding up YOLO's inference capabilities compared to traditional methods.
|
||||
|
||||
<p align="center">
|
||||
<img width="640" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/enhanced-inference-speed.avif" alt="Neural Magic DeepSparse inference acceleration">
|
||||
</p>
|
||||
|
||||
- **Optimized Model Efficiency**: Uses pruning and quantization to enhance YOLO26's efficiency, reducing model size and computational requirements while maintaining [accuracy](https://www.ultralytics.com/glossary/accuracy).
|
||||
|
||||
<p align="center">
|
||||
<img width="640" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/optimized-model-efficiency.avif" alt="Neural Magic model optimization and pruning">
|
||||
</p>
|
||||
|
||||
- **High Performance on Standard CPUs**: Delivers GPU-like performance on CPUs, providing a more accessible and cost-effective option for various applications.
|
||||
|
||||
- **Streamlined Integration and Deployment**: Offers user-friendly tools for easy integration of YOLO26 into applications, including image and video annotation features.
|
||||
|
||||
- **Support for Various Model Types**: Compatible with both standard and sparsity-optimized YOLO26 models, adding deployment flexibility.
|
||||
|
||||
- **Cost-Effective and Scalable Solution**: Reduces operational expenses and offers scalable deployment of advanced object detection models.
|
||||
|
||||
## How Does Neural Magic's DeepSparse Technology Work?
|
||||
|
||||
Neural Magic's DeepSparse technology is inspired by the human brain's efficiency in neural network computation. It adopts two key principles from the brain as follows:
|
||||
|
||||
- **Sparsity**: The process of sparsification involves pruning redundant information from [deep learning](https://www.ultralytics.com/glossary/deep-learning-dl) networks, leading to smaller and faster models without compromising accuracy. This technique reduces the network's size and computational needs significantly.
|
||||
|
||||
- **Locality of Reference**: DeepSparse uses a unique execution method, breaking the network into Tensor Columns. These columns are executed depth-wise, fitting entirely within the CPU's cache. This approach mimics the brain's efficiency, minimizing data movement and maximizing the CPU's cache use.
|
||||
|
||||
<p align="center">
|
||||
<img width="640" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/neural-magic-deepsparse-technology.avif" alt="How Neural Magic's DeepSparse Technology Works ">
|
||||
</p>
|
||||
|
||||
## Creating A Sparse Version of YOLO26 Trained on a Custom Dataset
|
||||
|
||||
[SparseZoo](https://github.com/neuralmagic/sparsezoo/blob/main/README.md), an open-source model repository by Neural Magic, offers [a collection of pre-sparsified YOLO26 model checkpoints](https://github.com/neuralmagic/sparsezoo/blob/main/README.md). With [SparseML](https://github.com/neuralmagic/sparseml), seamlessly integrated with Ultralytics, users can effortlessly fine-tune these sparse checkpoints on their specific datasets using a straightforward command-line interface.
|
||||
|
||||
Check out [Neural Magic's SparseML YOLO26 documentation](https://github.com/neuralmagic/sparseml/tree/main/integrations/ultralytics-yolov8) for more details.
|
||||
|
||||
## Usage: Deploying YOLO26 using DeepSparse
|
||||
|
||||
Deploying YOLO26 with Neural Magic's DeepSparse involves a few straightforward steps. Before diving into the usage instructions, be sure to check out the range of [YOLO26 models offered by Ultralytics](../models/index.md). This will help you choose the most appropriate model for your project requirements. Here's how you can get started.
|
||||
|
||||
### Step 1: Installation
|
||||
|
||||
To install the required packages, run:
|
||||
|
||||
!!! tip "Installation"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Install the required packages
|
||||
pip install deepsparse[yolov8]
|
||||
```
|
||||
|
||||
### Step 2: Exporting YOLO26 to ONNX Format
|
||||
|
||||
DeepSparse Engine requires YOLO26 models in [ONNX format](../integrations/onnx.md). Exporting your model to this format is essential for compatibility with DeepSparse. Use the following command to export YOLO26 models:
|
||||
|
||||
!!! tip "Model Export"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Export YOLO26 model to ONNX format
|
||||
yolo task=detect mode=export model=yolo26n.pt format=onnx opset=13
|
||||
```
|
||||
|
||||
This command will save the `yolo26n.onnx` model to your disk.
|
||||
|
||||
### Step 3: Deploying and Running Inferences
|
||||
|
||||
With your YOLO26 model in ONNX format, you can deploy and run inferences using DeepSparse. This can be done easily with their intuitive Python API:
|
||||
|
||||
!!! tip "Deploying and Running Inferences"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from deepsparse import Pipeline
|
||||
|
||||
# Specify the path to your YOLO26 ONNX model
|
||||
model_path = "path/to/yolo26n.onnx"
|
||||
|
||||
# Set up the DeepSparse Pipeline
|
||||
yolo_pipeline = Pipeline.create(task="yolov8", model_path=model_path)
|
||||
|
||||
# Run the model on your images
|
||||
images = ["path/to/image.jpg"]
|
||||
pipeline_outputs = yolo_pipeline(images=images)
|
||||
```
|
||||
|
||||
### Step 4: Benchmarking Performance
|
||||
|
||||
It's important to check that your YOLO26 model is performing optimally on DeepSparse. You can [benchmark](../modes/benchmark.md) your model's performance to analyze throughput and latency:
|
||||
|
||||
!!! tip "Benchmarking"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Benchmark performance
|
||||
deepsparse.benchmark model_path="path/to/yolo26n.onnx" --scenario=sync --input_shapes="[1,3,640,640]"
|
||||
```
|
||||
|
||||
### Step 5: Additional Features
|
||||
|
||||
DeepSparse provides additional features for practical integration of YOLO26 in applications, such as image annotation and dataset evaluation.
|
||||
|
||||
!!! tip "Additional Features"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# For image annotation
|
||||
deepsparse.yolov8.annotate --source "path/to/image.jpg" --model_filepath "path/to/yolo26n.onnx"
|
||||
|
||||
# For evaluating model performance on a dataset
|
||||
deepsparse.yolov8.eval --model_path "path/to/yolo26n.onnx"
|
||||
```
|
||||
|
||||
Running the annotate command processes your specified image, detecting objects, and saving the annotated image with bounding boxes and classifications. The annotated image will be stored in an annotation-results folder. This helps provide a visual representation of the model's detection capabilities.
|
||||
|
||||
<p align="center">
|
||||
<img width="640" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/image-annotation-feature.avif" alt="Neural Magic annotation feature interface">
|
||||
</p>
|
||||
|
||||
After running the eval command, you will receive detailed output metrics such as [precision](https://www.ultralytics.com/glossary/precision), [recall](https://www.ultralytics.com/glossary/recall), and [mAP](https://www.ultralytics.com/glossary/mean-average-precision-map) (mean Average Precision). This provides a comprehensive view of your model's performance on the dataset and is particularly useful for fine-tuning and optimizing your YOLO26 models for specific use cases, ensuring high accuracy and efficiency.
|
||||
|
||||
## Summary
|
||||
|
||||
This guide explored integrating Ultralytics' YOLO26 with Neural Magic's DeepSparse Engine. It highlighted how this integration enhances YOLO26's performance on CPU platforms, offering GPU-level efficiency and advanced neural network sparsity techniques.
|
||||
|
||||
For more detailed information and advanced usage, visit the [DeepSparse documentation by Neural Magic](https://www.redhat.com/en/about/press-releases/red-hat-completes-acquisition-neural-magic-fuel-optimized-generative-ai-innovation-across-hybrid-cloud). You can also [explore the YOLO26 integration guide](https://github.com/neuralmagic/deepsparse/tree/main/src/deepsparse/yolov8#yolov8-inference-pipelines) and [watch a walkthrough session on YouTube](https://www.youtube.com/watch?v=qtJ7bdt52x8).
|
||||
|
||||
Additionally, for a broader understanding of various YOLO26 integrations, visit the [Ultralytics integration guide page](../integrations/index.md), where you can discover a range of other exciting integration possibilities.
|
||||
|
||||
## FAQ
|
||||
|
||||
### What is Neural Magic's DeepSparse Engine and how does it optimize YOLO26 performance?
|
||||
|
||||
Neural Magic's DeepSparse Engine is an inference runtime designed to optimize the execution of neural networks on CPUs through advanced techniques such as sparsity, pruning, and quantization. By integrating DeepSparse with YOLO26, you can achieve GPU-like performance on standard CPUs, significantly enhancing inference speed, model efficiency, and overall performance while maintaining accuracy. For more details, check out the [Neural Magic's DeepSparse section](#neural-magics-deepsparse).
|
||||
|
||||
### How can I install the needed packages to deploy YOLO26 using Neural Magic's DeepSparse?
|
||||
|
||||
Installing the required packages for deploying YOLO26 with Neural Magic's DeepSparse is straightforward. You can easily install them using the CLI. Here's the command you need to run:
|
||||
|
||||
```bash
|
||||
pip install deepsparse[yolov8]
|
||||
```
|
||||
|
||||
Once installed, follow the steps provided in the [Installation section](#step-1-installation) to set up your environment and start using DeepSparse with YOLO26.
|
||||
|
||||
### How do I convert YOLO26 models to ONNX format for use with DeepSparse?
|
||||
|
||||
To convert YOLO26 models to the ONNX format, which is required for compatibility with DeepSparse, you can use the following CLI command:
|
||||
|
||||
```bash
|
||||
yolo task=detect mode=export model=yolo26n.pt format=onnx opset=13
|
||||
```
|
||||
|
||||
This command will export your YOLO26 model (`yolo26n.pt`) to a format (`yolo26n.onnx`) that can be utilized by the DeepSparse Engine. More information about model export can be found in the [Model Export section](#step-2-exporting-yolo26-to-onnx-format).
|
||||
|
||||
### How do I benchmark YOLO26 performance on the DeepSparse Engine?
|
||||
|
||||
Benchmarking YOLO26 performance on DeepSparse helps you analyze throughput and latency to ensure your model is optimized. You can use the following CLI command to run a benchmark:
|
||||
|
||||
```bash
|
||||
deepsparse.benchmark model_path="path/to/yolo26n.onnx" --scenario=sync --input_shapes="[1,3,640,640]"
|
||||
```
|
||||
|
||||
This command will provide you with vital performance metrics. For more details, see the [Benchmarking Performance section](#step-4-benchmarking-performance).
|
||||
|
||||
### Why should I use Neural Magic's DeepSparse with YOLO26 for object detection tasks?
|
||||
|
||||
Integrating Neural Magic's DeepSparse with YOLO26 offers several benefits:
|
||||
|
||||
- **Enhanced Inference Speed:** Achieves up to 525 FPS (on YOLO11n), demonstrating DeepSparse's optimization capabilities.
|
||||
- **Optimized Model Efficiency:** Uses sparsity, pruning, and quantization techniques to reduce model size and computational needs while maintaining accuracy.
|
||||
- **High Performance on Standard CPUs:** Offers GPU-like performance on cost-effective CPU hardware.
|
||||
- **Streamlined Integration:** User-friendly tools for easy deployment and integration.
|
||||
- **Flexibility:** Supports both standard and sparsity-optimized YOLO26 models.
|
||||
- **Cost-Effective:** Reduces operational expenses through efficient resource utilization.
|
||||
|
||||
For a deeper dive into these advantages, visit the [Benefits of Integrating Neural Magic's DeepSparse with YOLO26 section](#benefits-of-integrating-neural-magics-deepsparse-with-yolo26).
|
||||
@@ -0,0 +1,238 @@
|
||||
---
|
||||
comments: true
|
||||
description: Learn how to export YOLO26 models to ONNX format for flexible deployment across various platforms with enhanced performance.
|
||||
keywords: YOLO26, ONNX, model export, Ultralytics, ONNX Runtime, machine learning, model deployment, computer vision, deep learning
|
||||
---
|
||||
|
||||
# ONNX Export for YOLO26 Models
|
||||
|
||||
Often, when deploying [computer vision](https://www.ultralytics.com/glossary/computer-vision-cv) models, you'll need a model format that's both flexible and compatible with multiple platforms.
|
||||
|
||||
Exporting [Ultralytics YOLO26](https://github.com/ultralytics/ultralytics) models to ONNX format streamlines deployment and ensures optimal performance across various environments. This guide will show you how to easily convert your YOLO26 models to ONNX and enhance their scalability and effectiveness in real-world applications.
|
||||
|
||||
## ONNX and ONNX Runtime
|
||||
|
||||
[ONNX](https://onnx.ai/), which stands for Open [Neural Network](https://www.ultralytics.com/glossary/neural-network-nn) Exchange, is a community project that Facebook and Microsoft initially developed. The ongoing development of ONNX is a collaborative effort supported by various organizations like IBM, Amazon (through AWS), and Google. The project aims to create an open file format designed to represent [machine learning](https://www.ultralytics.com/glossary/machine-learning-ml) models in a way that allows them to be used across different AI frameworks and hardware.
|
||||
|
||||
ONNX models can be used to transition between different frameworks seamlessly. For instance, a [deep learning](https://www.ultralytics.com/glossary/deep-learning-dl) model trained in PyTorch can be exported to ONNX format and then easily imported into TensorFlow.
|
||||
|
||||
<p align="center">
|
||||
<img width="100%" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/onnx-model-portability.avif" alt="ONNX model portability across deep learning frameworks">
|
||||
</p>
|
||||
|
||||
Alternatively, ONNX models can be used with ONNX Runtime. [ONNX Runtime](https://onnxruntime.ai/) is a versatile cross-platform accelerator for machine learning models that is compatible with frameworks like PyTorch, [TensorFlow](https://www.ultralytics.com/glossary/tensorflow), TFLite, scikit-learn, etc.
|
||||
|
||||
ONNX Runtime optimizes the execution of ONNX models by leveraging hardware-specific capabilities. This optimization allows the models to run efficiently and with high performance on various hardware platforms, including CPUs, GPUs, and specialized accelerators.
|
||||
|
||||
<p align="center">
|
||||
<img width="100%" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/onnx-and-onnx-runtime.avif" alt="ONNX Runtime cross-platform inference acceleration">
|
||||
</p>
|
||||
|
||||
Whether used independently or in tandem with ONNX Runtime, ONNX provides a flexible solution for machine learning [model deployment](https://www.ultralytics.com/glossary/model-deployment) and compatibility.
|
||||
|
||||
## Key Features of ONNX Models
|
||||
|
||||
The ability of ONNX to handle various formats can be attributed to the following key features:
|
||||
|
||||
- **Common Model Representation**: ONNX defines a common set of operators (like convolutions, layers, etc.) and a standard data format. When a model is converted to ONNX format, its architecture and weights are translated into this common representation. This uniformity ensures that the model can be understood by any framework that supports ONNX.
|
||||
|
||||
- **Versioning and Backward Compatibility**: ONNX maintains a versioning system for its operators. This ensures that even as the standard evolves, models created in older versions remain usable. Backward compatibility is a crucial feature that prevents models from becoming obsolete quickly.
|
||||
|
||||
- **Graph-based Model Representation**: ONNX represents models as computational graphs. This graph-based structure is a universal way of representing machine learning models, where nodes represent operations or computations, and edges represent the tensors flowing between them. This format is easily adaptable to various frameworks which also represent models as graphs.
|
||||
|
||||
- **Tools and Ecosystem**: There is a rich ecosystem of tools around ONNX that assist in model conversion, visualization, and optimization. These tools make it easier for developers to work with ONNX models and to convert models between different frameworks seamlessly.
|
||||
|
||||
## Common Usage of ONNX
|
||||
|
||||
Before we jump into how to export YOLO26 models to the ONNX format, let's take a look at where ONNX models are usually used.
|
||||
|
||||
### CPU Deployment
|
||||
|
||||
ONNX models are often deployed on CPUs due to their compatibility with ONNX Runtime. This runtime is optimized for CPU execution. It significantly improves inference speed and makes real-time CPU deployments feasible.
|
||||
|
||||
### Supported Deployment Options
|
||||
|
||||
While ONNX models are commonly used on CPUs, they can also be deployed on the following platforms:
|
||||
|
||||
- **GPU Acceleration**: ONNX fully supports GPU acceleration, particularly NVIDIA CUDA. This enables efficient execution on NVIDIA GPUs for tasks that demand high computational power.
|
||||
|
||||
- **Edge and Mobile Devices**: ONNX extends to edge and mobile devices, perfect for on-device and real-time inference scenarios. It's lightweight and compatible with edge hardware.
|
||||
|
||||
- **Web Browsers**: ONNX can run directly in web browsers, powering interactive and dynamic web-based AI applications.
|
||||
|
||||
## Exporting YOLO26 Models to ONNX
|
||||
|
||||
You can expand model compatibility and deployment flexibility by converting YOLO26 models to ONNX format. [Ultralytics YOLO26](../models/yolo26.md) provides a straightforward export process that can significantly enhance your model's performance across different platforms.
|
||||
|
||||
### Installation
|
||||
|
||||
To install the required package, run:
|
||||
|
||||
!!! tip "Installation"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Install the required package for YOLO26
|
||||
pip install ultralytics
|
||||
```
|
||||
|
||||
For detailed instructions and best practices related to the installation process, check our [YOLO26 Installation guide](../quickstart.md). While installing the required packages for YOLO26, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips.
|
||||
|
||||
### Usage
|
||||
|
||||
Before diving into the usage instructions, be sure to check out the range of [YOLO26 models offered by Ultralytics](../models/index.md). This will help you choose the most appropriate model for your project requirements.
|
||||
|
||||
!!! example "Usage"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load the YOLO26 model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Export the model to ONNX format
|
||||
model.export(format="onnx") # creates 'yolo26n.onnx'
|
||||
|
||||
# Load the exported ONNX model
|
||||
onnx_model = YOLO("yolo26n.onnx")
|
||||
|
||||
# Run inference
|
||||
results = onnx_model("https://ultralytics.com/images/bus.jpg")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Export a YOLO26n PyTorch model to ONNX format
|
||||
yolo export model=yolo26n.pt format=onnx # creates 'yolo26n.onnx'
|
||||
|
||||
# Run inference with the exported model
|
||||
yolo predict model=yolo26n.onnx source='https://ultralytics.com/images/bus.jpg'
|
||||
```
|
||||
|
||||
### Export Arguments
|
||||
|
||||
When exporting your YOLO26 model to ONNX format, you can customize the process using various arguments to optimize for your specific deployment needs:
|
||||
|
||||
| Argument | Type | Default | Description |
|
||||
| ---------- | ---------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `format` | `str` | `'onnx'` | Target format for the exported model, defining compatibility with various deployment environments. |
|
||||
| `imgsz` | `int` or `tuple` | `640` | Desired image size for the model input. Can be an integer for square images or a tuple `(height, width)` for specific dimensions. |
|
||||
| `half` | `bool` | `False` | Enables FP16 (half-precision) quantization, reducing model size and potentially speeding up inference on supported hardware. |
|
||||
| `dynamic` | `bool` | `False` | Allows dynamic input sizes, enhancing flexibility in handling varying image dimensions. |
|
||||
| `simplify` | `bool` | `True` | Simplifies the model graph with `onnxslim`, potentially improving performance and compatibility. |
|
||||
| `opset` | `int` | `None` | Specifies the ONNX opset version for compatibility with different ONNX parsers and runtimes. If not set, uses the latest supported version. |
|
||||
| `nms` | `bool` | `False` | Adds Non-Maximum Suppression (NMS), essential for accurate and efficient detection post-processing. |
|
||||
| `batch` | `int` | `1` | Specifies export model batch inference size or the max number of images the exported model will process concurrently in `predict` mode. |
|
||||
| `device` | `str` | `None` | Specifies the device for exporting: GPU (`device=0`), CPU (`device=cpu`), MPS for Apple silicon (`device=mps`). |
|
||||
|
||||
For more details about the export process, visit the [Ultralytics documentation page on exporting](../modes/export.md).
|
||||
|
||||
## Deploying Exported YOLO26 ONNX Models
|
||||
|
||||
Once you've successfully exported your Ultralytics YOLO26 models to ONNX format, the next step is deploying these models in various environments. For detailed instructions on deploying your ONNX models, take a look at the following resources:
|
||||
|
||||
- **[ONNX Runtime Python API Documentation](https://onnxruntime.ai/docs/api/python/api_summary.html)**: This guide provides essential information for loading and running ONNX models using ONNX Runtime.
|
||||
|
||||
- **[Deploying on Edge Devices](https://onnxruntime.ai/docs/tutorials/iot-edge/)**: Check out this docs page for different examples of deploying ONNX models on edge.
|
||||
|
||||
- **[ONNX Tutorials on GitHub](https://github.com/onnx/tutorials)**: A collection of comprehensive tutorials that cover various aspects of using and implementing ONNX models in different scenarios.
|
||||
|
||||
- **[Triton Inference Server](../guides/triton-inference-server.md)**: Learn how to deploy your ONNX models with NVIDIA's Triton Inference Server for high-performance, scalable deployments.
|
||||
|
||||
## Summary
|
||||
|
||||
In this guide, you've learned how to export Ultralytics YOLO26 models to ONNX format to increase their interoperability and performance across various platforms. You were also introduced to the ONNX Runtime and ONNX deployment options.
|
||||
|
||||
ONNX export is just one of many [export formats](../modes/export.md) supported by Ultralytics YOLO26, allowing you to deploy your models in virtually any environment. Depending on your specific needs, you might also want to explore other export options like [TensorRT](../integrations/tensorrt.md) for maximum GPU performance or [CoreML](../integrations/coreml.md) for Apple devices.
|
||||
|
||||
For further details on usage, visit the [ONNX official documentation](https://onnx.ai/onnx/intro/).
|
||||
|
||||
Also, if you'd like to know more about other Ultralytics YOLO26 integrations, visit our [integration guide page](../integrations/index.md). You'll find plenty of useful resources and insights there.
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I export YOLO26 models to ONNX format using Ultralytics?
|
||||
|
||||
To export your YOLO26 models to ONNX format using Ultralytics, follow these steps:
|
||||
|
||||
!!! example "Usage"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load the YOLO26 model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Export the model to ONNX format
|
||||
model.export(format="onnx") # creates 'yolo26n.onnx'
|
||||
|
||||
# Load the exported ONNX model
|
||||
onnx_model = YOLO("yolo26n.onnx")
|
||||
|
||||
# Run inference
|
||||
results = onnx_model("https://ultralytics.com/images/bus.jpg")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Export a YOLO26n PyTorch model to ONNX format
|
||||
yolo export model=yolo26n.pt format=onnx # creates 'yolo26n.onnx'
|
||||
|
||||
# Run inference with the exported model
|
||||
yolo predict model=yolo26n.onnx source='https://ultralytics.com/images/bus.jpg'
|
||||
```
|
||||
|
||||
For more details, visit the [export documentation](../modes/export.md).
|
||||
|
||||
### What are the advantages of using ONNX Runtime for deploying YOLO26 models?
|
||||
|
||||
Using ONNX Runtime for deploying YOLO26 models offers several advantages:
|
||||
|
||||
- **Cross-platform compatibility**: ONNX Runtime supports various platforms, such as Windows, macOS, and Linux, ensuring your models run smoothly across different environments.
|
||||
- **Hardware acceleration**: ONNX Runtime can leverage hardware-specific optimizations for CPUs, GPUs, and dedicated accelerators, providing high-performance inference.
|
||||
- **Framework interoperability**: Models trained in popular frameworks like [PyTorch](https://www.ultralytics.com/glossary/pytorch) or TensorFlow can be easily converted to ONNX format and run using ONNX Runtime.
|
||||
- **Performance optimization**: ONNX Runtime can provide up to 3x CPU speedup compared to native PyTorch models, making it ideal for deployment scenarios where GPU resources are limited.
|
||||
|
||||
Learn more by checking the [ONNX Runtime documentation](https://onnxruntime.ai/docs/api/python/api_summary.html).
|
||||
|
||||
### What deployment options are available for YOLO26 models exported to ONNX?
|
||||
|
||||
YOLO26 models exported to ONNX can be deployed on various platforms including:
|
||||
|
||||
- **CPUs**: Utilizing ONNX Runtime for optimized CPU inference.
|
||||
- **GPUs**: Leveraging NVIDIA CUDA for high-performance GPU acceleration.
|
||||
- **Edge devices**: Running lightweight models on edge and mobile devices for real-time, on-device inference.
|
||||
- **Web browsers**: Executing models directly within web browsers for interactive web-based applications.
|
||||
- **Cloud services**: Deploying on cloud platforms that support ONNX format for scalable inference.
|
||||
|
||||
For more information, explore our guide on [model deployment options](../guides/model-deployment-options.md).
|
||||
|
||||
### Why should I use ONNX format for Ultralytics YOLO26 models?
|
||||
|
||||
Using ONNX format for Ultralytics YOLO26 models provides numerous benefits:
|
||||
|
||||
- **Interoperability**: ONNX allows models to be transferred between different machine learning frameworks seamlessly.
|
||||
- **Performance Optimization**: ONNX Runtime can enhance model performance by utilizing hardware-specific optimizations.
|
||||
- **Flexibility**: ONNX supports various deployment environments, enabling you to use the same model on different platforms without modification.
|
||||
- **Standardization**: ONNX provides a standardized format that is widely supported across the industry, ensuring long-term compatibility.
|
||||
|
||||
Refer to the comprehensive guide on [exporting YOLO26 models to ONNX](https://www.ultralytics.com/blog/export-and-optimize-a-yolov8-model-for-inference-on-openvino).
|
||||
|
||||
### How can I troubleshoot issues when exporting YOLO26 models to ONNX?
|
||||
|
||||
When exporting YOLO26 models to ONNX, you might encounter common issues such as mismatched dependencies or unsupported operations. To troubleshoot these problems:
|
||||
|
||||
1. Verify that you have the correct version of required dependencies installed.
|
||||
2. Check the official [ONNX documentation](https://onnx.ai/onnx/intro/) for supported operators and features.
|
||||
3. Review the error messages for clues and consult the [Ultralytics Common Issues guide](../guides/yolo-common-issues.md).
|
||||
4. Try using different export arguments like `simplify=True` or adjusting the `opset` version.
|
||||
5. For dynamic input size issues, set `dynamic=True` during export.
|
||||
|
||||
If issues persist, contact Ultralytics support for further assistance.
|
||||
@@ -0,0 +1,572 @@
|
||||
---
|
||||
comments: true
|
||||
description: Learn to export YOLO26 models to OpenVINO format for up to 3x CPU speedup and hardware acceleration on Intel GPU and NPU.
|
||||
keywords: YOLO26, OpenVINO, model export, Intel, AI inference, CPU speedup, GPU acceleration, NPU, deep learning
|
||||
---
|
||||
|
||||
# Intel OpenVINO Export
|
||||
|
||||
<img width="1024" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/openvino-ecosystem.avif" alt="OpenVINO Intel AI inference toolkit">
|
||||
|
||||
In this guide, we cover exporting YOLO26 models to the [OpenVINO](https://docs.openvino.ai/) format, which can provide up to 3x [CPU](https://docs.openvino.ai/2024/openvino-workflow/running-inference/inference-devices-and-modes/cpu-device.html) speedup, as well as accelerating YOLO inference on Intel [GPU](https://docs.openvino.ai/2024/openvino-workflow/running-inference/inference-devices-and-modes/gpu-device.html) and [NPU](https://docs.openvino.ai/2024/openvino-workflow/running-inference/inference-devices-and-modes/npu-device.html) hardware.
|
||||
|
||||
OpenVINO, short for Open Visual Inference & [Neural Network](https://www.ultralytics.com/glossary/neural-network-nn) Optimization toolkit, is a comprehensive toolkit for optimizing and deploying AI inference models. Even though the name contains Visual, OpenVINO also supports various additional tasks including language, audio, time series, etc.
|
||||
|
||||
<p align="center">
|
||||
<br>
|
||||
<iframe loading="lazy" width="720" height="405" src="https://www.youtube.com/embed/AvFh-oTGDaw"
|
||||
title="YouTube video player" frameborder="0"
|
||||
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"
|
||||
allowfullscreen>
|
||||
</iframe>
|
||||
<br>
|
||||
<strong>Watch:</strong> How to Export Ultralytics YOLO26 to Intel OpenVINO Format for Faster Inference 🚀
|
||||
</p>
|
||||
|
||||
## Usage Examples
|
||||
|
||||
Export a YOLO26n model to OpenVINO format and run inference with the exported model.
|
||||
|
||||
!!! example
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load a YOLO26n PyTorch model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Export the model
|
||||
model.export(format="openvino") # creates 'yolo26n_openvino_model/'
|
||||
|
||||
# Load the exported OpenVINO model
|
||||
ov_model = YOLO("yolo26n_openvino_model/")
|
||||
|
||||
# Run inference
|
||||
results = ov_model("https://ultralytics.com/images/bus.jpg")
|
||||
|
||||
# Run inference with specified device, available devices: ["intel:gpu", "intel:npu", "intel:cpu"]
|
||||
results = ov_model("https://ultralytics.com/images/bus.jpg", device="intel:gpu")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Export a YOLO26n PyTorch model to OpenVINO format
|
||||
yolo export model=yolo26n.pt format=openvino # creates 'yolo26n_openvino_model/'
|
||||
|
||||
# Run inference with the exported model
|
||||
yolo predict model=yolo26n_openvino_model source='https://ultralytics.com/images/bus.jpg'
|
||||
|
||||
# Run inference with specified device, available devices: ["intel:gpu", "intel:npu", "intel:cpu"]
|
||||
yolo predict model=yolo26n_openvino_model source='https://ultralytics.com/images/bus.jpg' device="intel:gpu"
|
||||
```
|
||||
|
||||
## Export Arguments
|
||||
|
||||
| Argument | Type | Default | Description |
|
||||
| ---------- | ---------------- | -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `format` | `str` | `'openvino'` | Target format for the exported model, defining compatibility with various deployment environments. |
|
||||
| `imgsz` | `int` or `tuple` | `640` | Desired image size for the model input. Can be an integer for square images or a tuple `(height, width)` for specific dimensions. |
|
||||
| `half` | `bool` | `False` | Enables FP16 (half-precision) quantization, reducing model size and potentially speeding up inference on supported hardware. |
|
||||
| `int8` | `bool` | `False` | Activates INT8 quantization, further compressing the model and speeding up inference with minimal [accuracy](https://www.ultralytics.com/glossary/accuracy) loss, primarily for edge devices. |
|
||||
| `dynamic` | `bool` | `False` | Allows dynamic input sizes, enhancing flexibility in handling varying image dimensions. |
|
||||
| `nms` | `bool` | `False` | Adds Non-Maximum Suppression (NMS), essential for accurate and efficient detection post-processing. |
|
||||
| `batch` | `int` | `1` | Specifies export model batch inference size or the max number of images the exported model will process concurrently in `predict` mode. |
|
||||
| `data` | `str` | `'coco8.yaml'` | Path to the [dataset](https://docs.ultralytics.com/datasets/) configuration file (default: `coco8.yaml`), essential for quantization. |
|
||||
| `fraction` | `float` | `1.0` | Specifies the fraction of the dataset to use for INT8 quantization calibration. Allows for calibrating on a subset of the full dataset, useful for experiments or when resources are limited. If not specified with INT8 enabled, the full dataset will be used. |
|
||||
|
||||
For more details about the export process, visit the [Ultralytics documentation page on exporting](../modes/export.md).
|
||||
|
||||
!!! warning
|
||||
|
||||
OpenVINO™ is compatible with most Intel® processors but to ensure optimal performance:
|
||||
|
||||
1. Verify OpenVINO™ support
|
||||
Check whether your Intel® chip is officially supported by OpenVINO™ using [Intel's compatibility list](https://docs.openvino.ai/2025/about-openvino/release-notes-openvino/system-requirements.html).
|
||||
|
||||
2. Identify your accelerator
|
||||
Determine if your processor includes an integrated NPU (Neural Processing Unit) or GPU (integrated GPU) by consulting [Intel's hardware guide](https://www.intel.com/content/www/us/en/support/articles/000097597/processors.html).
|
||||
|
||||
3. Install the latest drivers
|
||||
If your chip supports an NPU or GPU but OpenVINO™ isn't detecting it, you may need to install or update the associated drivers. Follow the [driver‑installation instructions](https://medium.com/openvino-toolkit/how-to-run-openvino-on-a-linux-ai-pc-52083ce14a98) to enable full acceleration.
|
||||
|
||||
By following these three steps, you can ensure OpenVINO™ runs optimally on your Intel® hardware.
|
||||
|
||||
## Benefits of OpenVINO
|
||||
|
||||
1. **Performance**: OpenVINO delivers high-performance inference by utilizing the power of Intel CPUs, integrated and discrete GPUs, and FPGAs.
|
||||
2. **Support for Heterogeneous Execution**: OpenVINO provides an API to write once and deploy on any supported Intel hardware (CPU, GPU, FPGA, VPU, etc.).
|
||||
3. **Model Optimizer**: OpenVINO provides a Model Optimizer that imports, converts, and optimizes models from popular [deep learning](https://www.ultralytics.com/glossary/deep-learning-dl) frameworks such as PyTorch, [TensorFlow](https://www.ultralytics.com/glossary/tensorflow), TensorFlow Lite, Keras, ONNX, PaddlePaddle, and Caffe.
|
||||
4. **Ease of Use**: The toolkit comes with more than [80 tutorial notebooks](https://github.com/openvinotoolkit/openvino_notebooks) (including [YOLOv8 optimization](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/yolov8-optimization)) teaching different aspects of the toolkit.
|
||||
|
||||
## OpenVINO Export Structure
|
||||
|
||||
When you export a model to OpenVINO format, it results in a directory containing the following:
|
||||
|
||||
1. **XML file**: Describes the network topology.
|
||||
2. **BIN file**: Contains the weights and biases binary data.
|
||||
3. **Mapping file**: Holds mapping of original model output tensors to OpenVINO tensor names.
|
||||
|
||||
You can use these files to run inference with the OpenVINO Inference Engine.
|
||||
|
||||
## Using OpenVINO Export in Deployment
|
||||
|
||||
Once your model is successfully exported to the OpenVINO format, you have two primary options for running inference:
|
||||
|
||||
1. Use the `ultralytics` package, which provides a high-level API and wraps the OpenVINO Runtime.
|
||||
|
||||
2. Use the native `openvino` package for more advanced or customized control over inference behavior.
|
||||
|
||||
### Inference with Ultralytics
|
||||
|
||||
The ultralytics package allows you to easily run inference using the exported OpenVINO model via the predict method. You can also specify the target device (e.g., `intel:gpu`, `intel:npu`, `intel:cpu`) using the device argument.
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load the exported OpenVINO model
|
||||
ov_model = YOLO("yolo26n_openvino_model/") # the path of your exported OpenVINO model
|
||||
# Run inference with the exported model
|
||||
ov_model.predict(device="intel:gpu") # specify the device you want to run inference on
|
||||
```
|
||||
|
||||
This approach is ideal for fast prototyping or deployment when you don't need full control over the inference pipeline.
|
||||
|
||||
### Inference with OpenVINO Runtime
|
||||
|
||||
The OpenVINO Runtime provides a unified API for inference across all supported Intel hardware. It also provides advanced capabilities like load balancing across Intel hardware and asynchronous execution. For more information on running inference, refer to the [YOLO26 notebooks](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/yolov11-optimization).
|
||||
|
||||
Remember, you'll need the XML and BIN files as well as any application-specific settings like input size, scale factor for normalization, etc., to correctly set up and use the model with the Runtime.
|
||||
|
||||
In your deployment application, you would typically do the following steps:
|
||||
|
||||
1. Initialize OpenVINO by creating `core = Core()`.
|
||||
2. Load the model using the `core.read_model()` method.
|
||||
3. Compile the model using the `core.compile_model()` function.
|
||||
4. Prepare the input (image, text, audio, etc.).
|
||||
5. Run inference using `compiled_model(input_data)`.
|
||||
|
||||
For more detailed steps and code snippets, refer to the [OpenVINO documentation](https://docs.openvino.ai/) or [API tutorial](https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/openvino-api/openvino-api.ipynb).
|
||||
|
||||
## OpenVINO YOLO11 Benchmarks
|
||||
|
||||
The Ultralytics team benchmarked YOLO11 across various model formats and [precision](https://www.ultralytics.com/glossary/precision), evaluating speed and accuracy on different Intel devices compatible with OpenVINO.
|
||||
|
||||
!!! note
|
||||
|
||||
The benchmarking results below are for reference and might vary based on the exact hardware and software configuration of a system, as well as the current workload of the system at the time the benchmarks are run.
|
||||
|
||||
All benchmarks run with `openvino` Python package version [2025.1.0](https://pypi.org/project/openvino/2025.1.0/).
|
||||
|
||||
### Intel Core CPU
|
||||
|
||||
The Intel® Core® series is a range of high-performance processors by Intel. The lineup includes Core i3 (entry-level), Core i5 (mid-range), Core i7 (high-end), and Core i9 (extreme performance). Each series caters to different computing needs and budgets, from everyday tasks to demanding professional workloads. With each new generation, improvements are made to performance, energy efficiency, and features.
|
||||
|
||||
Benchmarks below run on 12th Gen Intel® Core® i9-12900KS CPU at FP32 precision.
|
||||
|
||||
<div align="center">
|
||||
<img width="800" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/openvino-corei9.avif" alt="OpenVINO YOLO inference benchmarks on Intel CPU">
|
||||
</div>
|
||||
|
||||
??? abstract "Detailed Benchmark Results"
|
||||
|
||||
| Model | Format | Status | Size (MB) | metrics/mAP50-95(B) | Inference time (ms/im) |
|
||||
| ------- | ----------- | ------ | --------- | ------------------- | ---------------------- |
|
||||
| YOLO11n | PyTorch | ✅ | 5.4 | 0.5071 | 21.00 |
|
||||
| YOLO11n | TorchScript | ✅ | 10.5 | 0.5077 | 21.39 |
|
||||
| YOLO11n | ONNX | ✅ | 10.2 | 0.5077 | 15.55 |
|
||||
| YOLO11n | OpenVINO | ✅ | 10.4 | 0.5077 | 11.49 |
|
||||
| YOLO11s | PyTorch | ✅ | 18.4 | 0.5770 | 43.16 |
|
||||
| YOLO11s | TorchScript | ✅ | 36.6 | 0.5781 | 50.06 |
|
||||
| YOLO11s | ONNX | ✅ | 36.3 | 0.5781 | 31.53 |
|
||||
| YOLO11s | OpenVINO | ✅ | 36.4 | 0.5781 | 30.82 |
|
||||
| YOLO11m | PyTorch | ✅ | 38.8 | 0.6257 | 110.60 |
|
||||
| YOLO11m | TorchScript | ✅ | 77.3 | 0.6306 | 128.09 |
|
||||
| YOLO11m | ONNX | ✅ | 76.9 | 0.6306 | 76.06 |
|
||||
| YOLO11m | OpenVINO | ✅ | 77.1 | 0.6306 | 79.38 |
|
||||
| YOLO11l | PyTorch | ✅ | 49.0 | 0.6367 | 150.38 |
|
||||
| YOLO11l | TorchScript | ✅ | 97.7 | 0.6408 | 172.57 |
|
||||
| YOLO11l | ONNX | ✅ | 97.0 | 0.6408 | 108.91 |
|
||||
| YOLO11l | OpenVINO | ✅ | 97.3 | 0.6408 | 102.30 |
|
||||
| YOLO11x | PyTorch | ✅ | 109.3 | 0.6989 | 272.72 |
|
||||
| YOLO11x | TorchScript | ✅ | 218.1 | 0.6900 | 320.86 |
|
||||
| YOLO11x | ONNX | ✅ | 217.5 | 0.6900 | 196.20 |
|
||||
| YOLO11x | OpenVINO | ✅ | 217.8 | 0.6900 | 195.32 |
|
||||
|
||||
### Intel® Core™ Ultra
|
||||
|
||||
The Intel® Core™ Ultra™ series represents a new benchmark in high-performance computing, engineered to meet the evolving demands of modern users—from gamers and creators to professionals leveraging AI. This next-generation lineup is more than a traditional CPU series; it combines powerful CPU cores, integrated high-performance GPU capabilities, and a dedicated Neural Processing Unit (NPU) within a single chip, offering a unified solution for diverse and intensive computing workloads.
|
||||
|
||||
At the heart of the Intel® Core Ultra™ architecture is a hybrid design that enables exceptional performance across traditional processing tasks, GPU-accelerated workloads, and AI-driven operations. The inclusion of the NPU enhances on-device AI inference, enabling faster, more efficient machine learning and data processing across a wide range of applications.
|
||||
|
||||
The Core Ultra™ family includes various models tailored for different performance needs, with options ranging from energy-efficient designs to high-power variants marked by the "H" designation—ideal for laptops and compact form factors that demand serious computing power. Across the lineup, users benefit from the synergy of CPU, GPU, and NPU integration, delivering remarkable efficiency, responsiveness, and multitasking capabilities.
|
||||
|
||||
As part of Intel's ongoing innovation, the Core Ultra™ series sets a new standard for future-ready computing. With multiple models available and more on the horizon, this series underscores Intel's commitment to delivering cutting-edge solutions for the next generation of intelligent, AI-enhanced devices.
|
||||
|
||||
Benchmarks below run on Intel® Core™ Ultra™ 7 258V and Intel® Core™ Ultra™ 7 265K at FP32 and INT8 precision.
|
||||
|
||||
#### Intel® Core™ Ultra™ 7 258V
|
||||
|
||||
!!! tip "Benchmarks"
|
||||
|
||||
=== "Integrated Intel® Arc™ GPU"
|
||||
|
||||
<div align="center">
|
||||
<img width="800" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/openvino-ultra7-258V-gpu.avif" alt="Intel Core Ultra GPU benchmarks">
|
||||
</div>
|
||||
|
||||
??? abstract "Detailed Benchmark Results"
|
||||
|
||||
| Model | Format | Precision | Status | Size (MB) | metrics/mAP50-95(B) | Inference time (ms/im) |
|
||||
| ------- | -------- | --------- | ------ | --------- | ------------------- | ---------------------- |
|
||||
| YOLO11n | PyTorch | FP32 | ✅ | 5.4 | 0.5052 | 32.27 |
|
||||
| YOLO11n | OpenVINO | FP32 | ✅ | 10.4 | 0.5068 | 11.84 |
|
||||
| YOLO11n | OpenVINO | INT8 | ✅ | 3.3 | 0.4969 | 11.24 |
|
||||
| YOLO11s | PyTorch | FP32 | ✅ | 18.4 | 0.5776 | 92.09 |
|
||||
| YOLO11s | OpenVINO | FP32 | ✅ | 36.4 | 0.5797 | 14.82 |
|
||||
| YOLO11s | OpenVINO | INT8 | ✅ | 9.8 | 0.5751 | 12.88 |
|
||||
| YOLO11m | PyTorch | FP32 | ✅ | 38.8 | 0.6262 | 277.24 |
|
||||
| YOLO11m | OpenVINO | FP32 | ✅ | 77.1 | 0.6306 | 22.94 |
|
||||
| YOLO11m | OpenVINO | INT8 | ✅ | 20.2 | 0.6126 | 17.85 |
|
||||
| YOLO11l | PyTorch | FP32 | ✅ | 49.0 | 0.6361 | 348.97 |
|
||||
| YOLO11l | OpenVINO | FP32 | ✅ | 97.3 | 0.6365 | 27.34 |
|
||||
| YOLO11l | OpenVINO | INT8 | ✅ | 25.7 | 0.6242 | 20.83 |
|
||||
| YOLO11x | PyTorch | FP32 | ✅ | 109.3 | 0.6984 | 666.07 |
|
||||
| YOLO11x | OpenVINO | FP32 | ✅ | 217.8 | 0.6890 | 39.09 |
|
||||
| YOLO11x | OpenVINO | INT8 | ✅ | 55.9 | 0.6856 | 30.60 |
|
||||
|
||||
=== "Intel® Lunar Lake CPU"
|
||||
|
||||
<div align="center">
|
||||
<img width="800" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/openvino-ultra7-258V-cpu.avif" alt="Intel Core Ultra CPU benchmarks">
|
||||
</div>
|
||||
|
||||
??? abstract "Detailed Benchmark Results"
|
||||
|
||||
| Model | Format | Precision | Status | Size (MB) | metrics/mAP50-95(B) | Inference time (ms/im) |
|
||||
| ------- | -------- | --------- | ------ | --------- | ------------------- | ---------------------- |
|
||||
| YOLO11n | PyTorch | FP32 | ✅ | 5.4 | 0.5052 | 32.27 |
|
||||
| YOLO11n | OpenVINO | FP32 | ✅ | 10.4 | 0.5077 | 32.55 |
|
||||
| YOLO11n | OpenVINO | INT8 | ✅ | 3.3 | 0.4980 | 22.98 |
|
||||
| YOLO11s | PyTorch | FP32 | ✅ | 18.4 | 0.5776 | 92.09 |
|
||||
| YOLO11s | OpenVINO | FP32 | ✅ | 36.4 | 0.5782 | 98.38 |
|
||||
| YOLO11s | OpenVINO | INT8 | ✅ | 9.8 | 0.5745 | 52.84 |
|
||||
| YOLO11m | PyTorch | FP32 | ✅ | 38.8 | 0.6262 | 277.24 |
|
||||
| YOLO11m | OpenVINO | FP32 | ✅ | 77.1 | 0.6307 | 275.74 |
|
||||
| YOLO11m | OpenVINO | INT8 | ✅ | 20.2 | 0.6172 | 132.63 |
|
||||
| YOLO11l | PyTorch | FP32 | ✅ | 49.0 | 0.6361 | 348.97 |
|
||||
| YOLO11l | OpenVINO | FP32 | ✅ | 97.3 | 0.6361 | 348.97 |
|
||||
| YOLO11l | OpenVINO | INT8 | ✅ | 25.7 | 0.6240 | 171.36 |
|
||||
| YOLO11x | PyTorch | FP32 | ✅ | 109.3 | 0.6984 | 666.07 |
|
||||
| YOLO11x | OpenVINO | FP32 | ✅ | 217.8 | 0.6900 | 783.16 |
|
||||
| YOLO11x | OpenVINO | INT8 | ✅ | 55.9 | 0.6890 | 346.82 |
|
||||
|
||||
|
||||
=== "Integrated Intel® AI Boost NPU"
|
||||
|
||||
<div align="center">
|
||||
<img width="800" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/openvino-ultra7-258V-npu.avif" alt="Intel Core Ultra NPU benchmarks">
|
||||
</div>
|
||||
|
||||
??? abstract "Detailed Benchmark Results"
|
||||
|
||||
| Model | Format | Precision | Status | Size (MB) | metrics/mAP50-95(B) | Inference time (ms/im) |
|
||||
| ------- | -------- | --------- | ------ | --------- | ------------------- | ---------------------- |
|
||||
| YOLO11n | PyTorch | FP32 | ✅ | 5.4 | 0.5052 | 32.27 |
|
||||
| YOLO11n | OpenVINO | FP32 | ✅ | 10.4 | 0.5085 | 8.33 |
|
||||
| YOLO11n | OpenVINO | INT8 | ✅ | 3.3 | 0.5019 | 8.91 |
|
||||
| YOLO11s | PyTorch | FP32 | ✅ | 18.4 | 0.5776 | 92.09 |
|
||||
| YOLO11s | OpenVINO | FP32 | ✅ | 36.4 | 0.5788 | 9.72 |
|
||||
| YOLO11s | OpenVINO | INT8 | ✅ | 9.8 | 0.5710 | 10.58 |
|
||||
| YOLO11m | PyTorch | FP32 | ✅ | 38.8 | 0.6262 | 277.24 |
|
||||
| YOLO11m | OpenVINO | FP32 | ✅ | 77.1 | 0.6301 | 19.41 |
|
||||
| YOLO11m | OpenVINO | INT8 | ✅ | 20.2 | 0.6124 | 18.26 |
|
||||
| YOLO11l | PyTorch | FP32 | ✅ | 49.0 | 0.6361 | 348.97 |
|
||||
| YOLO11l | OpenVINO | FP32 | ✅ | 97.3 | 0.6362 | 23.70 |
|
||||
| YOLO11l | OpenVINO | INT8 | ✅ | 25.7 | 0.6240 | 21.40 |
|
||||
| YOLO11x | PyTorch | FP32 | ✅ | 109.3 | 0.6984 | 666.07 |
|
||||
| YOLO11x | OpenVINO | FP32 | ✅ | 217.8 | 0.6892 | 43.91 |
|
||||
| YOLO11x | OpenVINO | INT8 | ✅ | 55.9 | 0.6890 | 34.04 |
|
||||
|
||||
#### Intel® Core™ Ultra™ 7 265K
|
||||
|
||||
!!! tip "Benchmarks"
|
||||
|
||||
=== "Integrated Intel® Arc™ GPU"
|
||||
|
||||
<div align="center">
|
||||
<img width="800" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/openvino-ultra7-265K-gpu.avif" alt="Intel Core Ultra GPU benchmarks">
|
||||
</div>
|
||||
|
||||
??? abstract "Detailed Benchmark Results"
|
||||
|
||||
| Model | Format | Precision | Status | Size (MB) | metrics/mAP50-95(B) | Inference time (ms/im) |
|
||||
| ------- | -------- | --------- | ------ | --------- | ------------------- | ---------------------- |
|
||||
| YOLO11n | PyTorch | FP32 | ✅ | 5.4 | 0.5072 | 16.29 |
|
||||
| YOLO11n | OpenVINO | FP32 | ✅ | 10.4 | 0.5079 | 13.13 |
|
||||
| YOLO11n | OpenVINO | INT8 | ✅ | 3.3 | 0.4976 | 8.86 |
|
||||
| YOLO11s | PyTorch | FP32 | ✅ | 18.4 | 0.5771 | 39.61 |
|
||||
| YOLO11s | OpenVINO | FP32 | ✅ | 36.4 | 0.5808 | 18.26 |
|
||||
| YOLO11s | OpenVINO | INT8 | ✅ | 9.8 | 0.5726 | 13.24 |
|
||||
| YOLO11m | PyTorch | FP32 | ✅ | 38.8 | 0.6258 | 100.65 |
|
||||
| YOLO11m | OpenVINO | FP32 | ✅ | 77.1 | 0.6310 | 43.50 |
|
||||
| YOLO11m | OpenVINO | INT8 | ✅ | 20.2 | 0.6137 | 20.90 |
|
||||
| YOLO11l | PyTorch | FP32 | ✅ | 49.0 | 0.6367 | 131.37 |
|
||||
| YOLO11l | OpenVINO | FP32 | ✅ | 97.3 | 0.6371 | 54.52 |
|
||||
| YOLO11l | OpenVINO | INT8 | ✅ | 25.7 | 0.6226 | 27.36 |
|
||||
| YOLO11x | PyTorch | FP32 | ✅ | 109.3 | 0.6990 | 212.45 |
|
||||
| YOLO11x | OpenVINO | FP32 | ✅ | 217.8 | 0.6884 | 112.76 |
|
||||
| YOLO11x | OpenVINO | INT8 | ✅ | 55.9 | 0.6900 | 52.06 |
|
||||
|
||||
|
||||
=== "Intel® Arrow Lake CPU"
|
||||
|
||||
<div align="center">
|
||||
<img width="800" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/openvino-ultra7-265K-cpu.avif" alt="Intel Core Ultra CPU benchmarks">
|
||||
</div>
|
||||
|
||||
??? abstract "Detailed Benchmark Results"
|
||||
|
||||
| Model | Format | Precision | Status | Size (MB) | metrics/mAP50-95(B) | Inference time (ms/im) |
|
||||
| ------- | -------- | --------- | ------ | --------- | ------------------- | ---------------------- |
|
||||
| YOLO11n | PyTorch | FP32 | ✅ | 5.4 | 0.5072 | 16.29 |
|
||||
| YOLO11n | OpenVINO | FP32 | ✅ | 10.4 | 0.5077 | 15.04 |
|
||||
| YOLO11n | OpenVINO | INT8 | ✅ | 3.3 | 0.4980 | 11.60 |
|
||||
| YOLO11s | PyTorch | FP32 | ✅ | 18.4 | 0.5771 | 39.61 |
|
||||
| YOLO11s | OpenVINO | FP32 | ✅ | 36.4 | 0.5782 | 33.45 |
|
||||
| YOLO11s | OpenVINO | INT8 | ✅ | 9.8 | 0.5745 | 20.64 |
|
||||
| YOLO11m | PyTorch | FP32 | ✅ | 38.8 | 0.6258 | 100.65 |
|
||||
| YOLO11m | OpenVINO | FP32 | ✅ | 77.1 | 0.6307 | 81.15 |
|
||||
| YOLO11m | OpenVINO | INT8 | ✅ | 20.2 | 0.6172 | 44.63 |
|
||||
| YOLO11l | PyTorch | FP32 | ✅ | 49.0 | 0.6367 | 131.37 |
|
||||
| YOLO11l | OpenVINO | FP32 | ✅ | 97.3 | 0.6409 | 103.77 |
|
||||
| YOLO11l | OpenVINO | INT8 | ✅ | 25.7 | 0.6240 | 58.00 |
|
||||
| YOLO11x | PyTorch | FP32 | ✅ | 109.3 | 0.6990 | 212.45 |
|
||||
| YOLO11x | OpenVINO | FP32 | ✅ | 217.8 | 0.6900 | 208.37 |
|
||||
| YOLO11x | OpenVINO | INT8 | ✅ | 55.9 | 0.6897 | 113.04 |
|
||||
|
||||
|
||||
=== "Integrated Intel® AI Boost NPU"
|
||||
|
||||
<div align="center">
|
||||
<img width="800" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/openvino-ultra7-265K-npu.avif" alt="Intel Core Ultra NPU benchmarks">
|
||||
</div>
|
||||
|
||||
??? abstract "Detailed Benchmark Results"
|
||||
|
||||
| Model | Format | Precision | Status | Size (MB) | metrics/mAP50-95(B) | Inference time (ms/im) |
|
||||
| ------- | -------- | --------- | ------ | --------- | ------------------- | ---------------------- |
|
||||
| YOLO11n | PyTorch | FP32 | ✅ | 5.4 | 0.5072 | 16.29 |
|
||||
| YOLO11n | OpenVINO | FP32 | ✅ | 10.4 | 0.5075 | 8.02 |
|
||||
| YOLO11n | OpenVINO | INT8 | ✅ | 3.3 | 0.3656 | 9.28 |
|
||||
| YOLO11s | PyTorch | FP32 | ✅ | 18.4 | 0.5771 | 39.61 |
|
||||
| YOLO11s | OpenVINO | FP32 | ✅ | 36.4 | 0.5801 | 13.12 |
|
||||
| YOLO11s | OpenVINO | INT8 | ✅ | 9.8 | 0.5686 | 13.12 |
|
||||
| YOLO11m | PyTorch | FP32 | ✅ | 38.8 | 0.6258 | 100.65 |
|
||||
| YOLO11m | OpenVINO | FP32 | ✅ | 77.1 | 0.6310 | 29.88 |
|
||||
| YOLO11m | OpenVINO | INT8 | ✅ | 20.2 | 0.6111 | 26.32 |
|
||||
| YOLO11l | PyTorch | FP32 | ✅ | 49.0 | 0.6367 | 131.37 |
|
||||
| YOLO11l | OpenVINO | FP32 | ✅ | 97.3 | 0.6356 | 37.08 |
|
||||
| YOLO11l | OpenVINO | INT8 | ✅ | 25.7 | 0.6245 | 30.81 |
|
||||
| YOLO11x | PyTorch | FP32 | ✅ | 109.3 | 0.6990 | 212.45 |
|
||||
| YOLO11x | OpenVINO | FP32 | ✅ | 217.8 | 0.6894 | 68.48 |
|
||||
| YOLO11x | OpenVINO | INT8 | ✅ | 55.9 | 0.6417 | 49.76 |
|
||||
|
||||
## Intel® Arc GPU
|
||||
|
||||
Intel® Arc™ is Intel's line of discrete graphics cards designed for high-performance gaming, content creation, and AI workloads. The Arc series features advanced GPU architectures that support real-time ray tracing, AI-enhanced graphics, and high-resolution gaming. With a focus on performance and efficiency, Intel® Arc™ aims to compete with other leading GPU brands while providing unique features like hardware-accelerated AV1 encoding and support for the latest graphics APIs.
|
||||
|
||||
Benchmarks below run on Intel Arc A770 and Intel Arc B580 at FP32 and INT8 precision.
|
||||
|
||||
### Intel Arc A770
|
||||
|
||||
<div align="center">
|
||||
<img width="800" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/openvino-arc-a770-gpu.avif" alt="Intel Core Ultra CPU benchmarks">
|
||||
</div>
|
||||
|
||||
??? abstract "Detailed Benchmark Results"
|
||||
|
||||
| Model | Format | Precision | Status | Size (MB) | metrics/mAP50-95(B) | Inference time (ms/im) |
|
||||
| ------- | -------- | --------- | ------ | --------- | ------------------- | ---------------------- |
|
||||
| YOLO11n | PyTorch | FP32 | ✅ | 5.4 | 0.5072 | 16.29 |
|
||||
| YOLO11n | OpenVINO | FP32 | ✅ | 10.4 | 0.5073 | 6.98 |
|
||||
| YOLO11n | OpenVINO | INT8 | ✅ | 3.3 | 0.4978 | 7.24 |
|
||||
| YOLO11s | PyTorch | FP32 | ✅ | 18.4 | 0.5771 | 39.61 |
|
||||
| YOLO11s | OpenVINO | FP32 | ✅ | 36.4 | 0.5798 | 9.41 |
|
||||
| YOLO11s | OpenVINO | INT8 | ✅ | 9.8 | 0.5751 | 8.72 |
|
||||
| YOLO11m | PyTorch | FP32 | ✅ | 38.8 | 0.6258 | 100.65 |
|
||||
| YOLO11m | OpenVINO | FP32 | ✅ | 77.1 | 0.6311 | 14.88 |
|
||||
| YOLO11m | OpenVINO | INT8 | ✅ | 20.2 | 0.6126 | 11.97 |
|
||||
| YOLO11l | PyTorch | FP32 | ✅ | 49.0 | 0.6367 | 131.37 |
|
||||
| YOLO11l | OpenVINO | FP32 | ✅ | 97.3 | 0.6364 | 19.17 |
|
||||
| YOLO11l | OpenVINO | INT8 | ✅ | 25.7 | 0.6241 | 15.75 |
|
||||
| YOLO11x | PyTorch | FP32 | ✅ | 109.3 | 0.6990 | 212.45 |
|
||||
| YOLO11x | OpenVINO | FP32 | ✅ | 217.8 | 0.6888 | 18.13 |
|
||||
| YOLO11x | OpenVINO | INT8 | ✅ | 55.9 | 0.6930 | 18.91 |
|
||||
|
||||
### Intel Arc B580
|
||||
|
||||
<div align="center">
|
||||
<img width="800" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/openvino-arc-b580-gpu.avif" alt="Intel Core Ultra CPU benchmarks">
|
||||
</div>
|
||||
|
||||
??? abstract "Detailed Benchmark Results"
|
||||
|
||||
| Model | Format | Precision | Status | Size (MB) | metrics/mAP50-95(B) | Inference time (ms/im) |
|
||||
| ------- | -------- | --------- | ------ | --------- | ------------------- | ---------------------- |
|
||||
| YOLO11n | PyTorch | FP32 | ✅ | 5.4 | 0.5072 | 16.29 |
|
||||
| YOLO11n | OpenVINO | FP32 | ✅ | 10.4 | 0.5072 | 4.27 |
|
||||
| YOLO11n | OpenVINO | INT8 | ✅ | 3.3 | 0.4981 | 4.33 |
|
||||
| YOLO11s | PyTorch | FP32 | ✅ | 18.4 | 0.5771 | 39.61 |
|
||||
| YOLO11s | OpenVINO | FP32 | ✅ | 36.4 | 0.5789 | 5.04 |
|
||||
| YOLO11s | OpenVINO | INT8 | ✅ | 9.8 | 0.5746 | 4.97 |
|
||||
| YOLO11m | PyTorch | FP32 | ✅ | 38.8 | 0.6258 | 100.65 |
|
||||
| YOLO11m | OpenVINO | FP32 | ✅ | 77.1 | 0.6306 | 6.45 |
|
||||
| YOLO11m | OpenVINO | INT8 | ✅ | 20.2 | 0.6125 | 6.28 |
|
||||
| YOLO11l | PyTorch | FP32 | ✅ | 49.0 | 0.6367 | 131.37 |
|
||||
| YOLO11l | OpenVINO | FP32 | ✅ | 97.3 | 0.6360 | 8.23 |
|
||||
| YOLO11l | OpenVINO | INT8 | ✅ | 25.7 | 0.6236 | 8.49 |
|
||||
| YOLO11x | PyTorch | FP32 | ✅ | 109.3 | 0.6990 | 212.45 |
|
||||
| YOLO11x | OpenVINO | FP32 | ✅ | 217.8 | 0.6889 | 11.10 |
|
||||
| YOLO11x | OpenVINO | INT8 | ✅ | 55.9 | 0.6924 | 10.30 |
|
||||
|
||||
## Reproduce Our Results
|
||||
|
||||
To reproduce the Ultralytics benchmarks above on all export [formats](../modes/export.md) run this code:
|
||||
|
||||
!!! example
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load a YOLO11n PyTorch model
|
||||
model = YOLO("yolo11n.pt")
|
||||
|
||||
# Benchmark YOLO11n speed and accuracy on the COCO128 dataset for all export formats
|
||||
results = model.benchmark(data="coco128.yaml")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Benchmark YOLO11n speed and accuracy on the COCO128 dataset for all export formats
|
||||
yolo benchmark model=yolo11n.pt data=coco128.yaml
|
||||
```
|
||||
|
||||
Note that benchmarking results might vary based on the exact hardware and software configuration of a system, as well as the current workload of the system at the time the benchmarks are run. For the most reliable results use a dataset with a large number of images, i.e. `data='coco.yaml'` (5000 val images).
|
||||
|
||||
## Conclusion
|
||||
|
||||
The benchmarking results clearly demonstrate the benefits of exporting the YOLO11 model to the OpenVINO format. Across different models and hardware platforms, the OpenVINO format consistently outperforms other formats in terms of inference speed while maintaining comparable accuracy.
|
||||
|
||||
The benchmarks underline the effectiveness of OpenVINO as a tool for deploying deep learning models. By converting models to the OpenVINO format, developers can achieve significant performance improvements, making it easier to deploy these models in real-world applications.
|
||||
|
||||
For more detailed information and instructions on using OpenVINO, refer to the [official OpenVINO documentation](https://docs.openvino.ai/).
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I export YOLO26 models to OpenVINO format?
|
||||
|
||||
Exporting YOLO26 models to the OpenVINO format can significantly enhance CPU speed and enable GPU and NPU accelerations on Intel hardware. To export, you can use either Python or CLI as shown below:
|
||||
|
||||
!!! example
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load a YOLO26n PyTorch model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Export the model
|
||||
model.export(format="openvino") # creates 'yolo26n_openvino_model/'
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Export a YOLO26n PyTorch model to OpenVINO format
|
||||
yolo export model=yolo26n.pt format=openvino # creates 'yolo26n_openvino_model/'
|
||||
```
|
||||
|
||||
For more information, refer to the [export formats documentation](../modes/export.md).
|
||||
|
||||
### What are the benefits of using OpenVINO with YOLO26 models?
|
||||
|
||||
Using Intel's OpenVINO toolkit with YOLO26 models offers several benefits:
|
||||
|
||||
1. **Performance**: Achieve up to 3x speedup on CPU inference and leverage Intel GPUs and NPUs for acceleration.
|
||||
2. **Model Optimizer**: Convert, optimize, and execute models from popular frameworks like PyTorch, TensorFlow, and ONNX.
|
||||
3. **Ease of Use**: Over 80 tutorial notebooks are available to help users get started, including ones for YOLO26.
|
||||
4. **Heterogeneous Execution**: Deploy models on various Intel hardware with a unified API.
|
||||
|
||||
For detailed performance comparisons, visit our [benchmarks section](#openvino-yolo11-benchmarks).
|
||||
|
||||
### How can I run inference using a YOLO26 model exported to OpenVINO?
|
||||
|
||||
After exporting a YOLO26n model to OpenVINO format, you can run inference using Python or CLI:
|
||||
|
||||
!!! example
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load the exported OpenVINO model
|
||||
ov_model = YOLO("yolo26n_openvino_model/")
|
||||
|
||||
# Run inference
|
||||
results = ov_model("https://ultralytics.com/images/bus.jpg")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Run inference with the exported model
|
||||
yolo predict model=yolo26n_openvino_model source='https://ultralytics.com/images/bus.jpg'
|
||||
```
|
||||
|
||||
Refer to our [predict mode documentation](../modes/predict.md) for more details.
|
||||
|
||||
### Why should I choose Ultralytics YOLO26 over other models for OpenVINO export?
|
||||
|
||||
Ultralytics YOLO26 is optimized for real-time object detection with high accuracy and speed. Specifically, when combined with OpenVINO, YOLO26 provides:
|
||||
|
||||
- Up to 3x speedup on Intel CPUs
|
||||
- Seamless deployment on Intel GPUs and NPUs
|
||||
- Consistent and comparable accuracy across various export formats
|
||||
|
||||
For in-depth performance analysis, check our detailed [YOLO11 benchmarks](#openvino-yolo11-benchmarks) on different hardware.
|
||||
|
||||
### Can I benchmark YOLO26 models on different formats such as PyTorch, ONNX, and OpenVINO?
|
||||
|
||||
Yes, you can benchmark YOLO26 models in various formats including PyTorch, TorchScript, ONNX, and OpenVINO. Use the following code snippet to run benchmarks on your chosen dataset:
|
||||
|
||||
!!! example
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load a YOLO26n PyTorch model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Benchmark YOLO26n speed and [accuracy](https://www.ultralytics.com/glossary/accuracy) on the COCO8 dataset for all export formats
|
||||
results = model.benchmark(data="coco8.yaml")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Benchmark YOLO26n speed and accuracy on the COCO8 dataset for all export formats
|
||||
yolo benchmark model=yolo26n.pt data=coco8.yaml
|
||||
```
|
||||
|
||||
For detailed benchmark results, refer to our [benchmarks section](#openvino-yolo11-benchmarks) and [export formats](../modes/export.md) documentation.
|
||||
@@ -0,0 +1,222 @@
|
||||
---
|
||||
comments: true
|
||||
description: Learn how to export YOLO26 models to PaddlePaddle format for enhanced performance, flexibility, and deployment across various platforms and devices.
|
||||
keywords: YOLO26, PaddlePaddle, export models, computer vision, deep learning, model deployment, performance optimization
|
||||
---
|
||||
|
||||
# How to Export to PaddlePaddle Format from YOLO26 Models
|
||||
|
||||
Bridging the gap between developing and deploying [computer vision](https://www.ultralytics.com/glossary/computer-vision-cv) models in real-world scenarios with varying conditions can be difficult. PaddlePaddle makes this process easier with its focus on flexibility, performance, and its capability for parallel processing in distributed environments. This means you can use your YOLO26 computer vision models on a wide variety of devices and platforms, from smartphones to cloud-based servers.
|
||||
|
||||
<p align="center">
|
||||
<br>
|
||||
<iframe loading="lazy" width="720" height="405" src="https://www.youtube.com/embed/c5eFrt2KuzY"
|
||||
title="YouTube video player" frameborder="0"
|
||||
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"
|
||||
allowfullscreen>
|
||||
</iframe>
|
||||
<br>
|
||||
<strong>Watch:</strong> How to Export Ultralytics YOLO26 Models to PaddlePaddle Format | Key Features of PaddlePaddle Format
|
||||
</p>
|
||||
|
||||
The ability to export to PaddlePaddle model format allows you to optimize your [Ultralytics YOLO26](https://github.com/ultralytics/ultralytics) models for use within the PaddlePaddle framework. PaddlePaddle is known for facilitating industrial deployments and is a good choice for deploying computer vision applications in real-world settings across various domains.
|
||||
|
||||
## Why should you export to PaddlePaddle?
|
||||
|
||||
<p align="center">
|
||||
<img width="75%" src="https://github.com/PaddlePaddle/Paddle/blob/develop/doc/imgs/logo.png" alt="PaddlePaddle deep learning framework logo">
|
||||
</p>
|
||||
|
||||
Developed by Baidu, [PaddlePaddle](https://www.paddlepaddle.org.cn/en) (**PA**rallel **D**istributed **D**eep **LE**arning) is China's first open-source [deep learning](https://www.ultralytics.com/glossary/deep-learning-dl) platform. Unlike some frameworks built mainly for research, PaddlePaddle prioritizes ease of use and smooth integration across industries.
|
||||
|
||||
It offers tools and resources similar to popular frameworks like [TensorFlow](https://www.ultralytics.com/glossary/tensorflow) and [PyTorch](https://www.ultralytics.com/glossary/pytorch), making it accessible for developers of all experience levels. From farming and factories to service businesses, PaddlePaddle's large developer community of over 4.77 million is helping create and deploy AI applications.
|
||||
|
||||
By exporting your Ultralytics YOLO26 models to PaddlePaddle format, you can tap into PaddlePaddle's strengths in performance optimization. PaddlePaddle prioritizes efficient model execution and reduced memory usage. As a result, your YOLO26 models can potentially achieve even better performance, delivering top-notch results in practical scenarios.
|
||||
|
||||
## Key Features of PaddlePaddle Models
|
||||
|
||||
PaddlePaddle models offer a range of key features that contribute to their flexibility, performance, and scalability across diverse deployment scenarios:
|
||||
|
||||
- **Dynamic-to-Static Graph**: PaddlePaddle supports [dynamic-to-static compilation](https://www.paddlepaddle.org.cn/documentation/docs/en/guides/jit/index_en.html), where models can be translated into a static computational graph. This enables optimizations that reduce runtime overhead and boost inference performance.
|
||||
|
||||
- **Operator Fusion**: PaddlePaddle, like [TensorRT](../integrations/tensorrt.md), uses [operator fusion](https://developer.nvidia.com/gtc/2020/video/s21436-vid) to streamline computation and reduce overhead. The framework minimizes memory transfers and computational steps by merging compatible operations, resulting in faster inference.
|
||||
|
||||
- **Quantization**: PaddlePaddle supports [quantization techniques](https://www.paddlepaddle.org.cn/documentation/docs/en/api/paddle/quantization/PTQ_en.html), including post-training quantization and quantization-aware training. These techniques allow for the use of lower-precision data representations, effectively boosting performance and reducing model size.
|
||||
|
||||
## Deployment Options in PaddlePaddle
|
||||
|
||||
Before diving into the code for exporting YOLO26 models to PaddlePaddle, let's take a look at the different deployment scenarios in which PaddlePaddle models excel.
|
||||
|
||||
PaddlePaddle provides a range of options, each offering a distinct balance of ease of use, flexibility, and performance:
|
||||
|
||||
- **Paddle Serving**: This framework simplifies the deployment of PaddlePaddle models as high-performance RESTful APIs. Paddle Serving is ideal for production environments, providing features like model versioning, online A/B testing, and scalability for handling large volumes of requests.
|
||||
|
||||
- **Paddle Inference API**: The Paddle Inference API gives you low-level control over model execution. This option is well-suited for scenarios where you need to integrate the model tightly within a custom application or optimize performance for specific hardware.
|
||||
|
||||
- **Paddle Lite**: Paddle Lite is designed for deployment on mobile and embedded devices where resources are limited. It optimizes models for smaller sizes and faster inference on ARM CPUs, GPUs, and other specialized hardware.
|
||||
|
||||
- **Paddle.js**: Paddle.js enables you to deploy PaddlePaddle models directly within web browsers. Paddle.js can either load a pretrained model or transform a model from [paddle-hub](https://github.com/PaddlePaddle/PaddleHub) with model transforming tools provided by Paddle.js. It can run in browsers that support WebGL/WebGPU/WebAssembly.
|
||||
|
||||
## Export to PaddlePaddle: Converting Your YOLO26 Model
|
||||
|
||||
Converting YOLO26 models to the PaddlePaddle format can improve execution flexibility and optimize performance for various deployment scenarios.
|
||||
|
||||
### Installation
|
||||
|
||||
To install the required package, run:
|
||||
|
||||
!!! tip "Installation"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Install the required package for YOLO26
|
||||
pip install ultralytics
|
||||
```
|
||||
|
||||
For detailed instructions and best practices related to the installation process, check our [Ultralytics Installation guide](../quickstart.md). While installing the required packages for YOLO26, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips.
|
||||
|
||||
### Usage
|
||||
|
||||
All [Ultralytics YOLO26 models](../models/yolo26.md) support export, and you can [browse the full list of export formats and options](../modes/export.md) to find the best fit for your deployment needs.
|
||||
|
||||
!!! example "Usage"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load the YOLO26 model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Export the model to PaddlePaddle format
|
||||
model.export(format="paddle") # creates '/yolo26n_paddle_model'
|
||||
|
||||
# Load the exported PaddlePaddle model
|
||||
paddle_model = YOLO("./yolo26n_paddle_model")
|
||||
|
||||
# Run inference
|
||||
results = paddle_model("https://ultralytics.com/images/bus.jpg")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Export a YOLO26n PyTorch model to PaddlePaddle format
|
||||
yolo export model=yolo26n.pt format=paddle # creates '/yolo26n_paddle_model'
|
||||
|
||||
# Run inference with the exported model
|
||||
yolo predict model='./yolo26n_paddle_model' source='https://ultralytics.com/images/bus.jpg'
|
||||
```
|
||||
|
||||
### Export Arguments
|
||||
|
||||
| Argument | Type | Default | Description |
|
||||
| -------- | ---------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `format` | `str` | `'paddle'` | Target format for the exported model, defining compatibility with various deployment environments. |
|
||||
| `imgsz` | `int` or `tuple` | `640` | Desired image size for the model input. Can be an integer for square images or a tuple `(height, width)` for specific dimensions. |
|
||||
| `batch` | `int` | `1` | Specifies export model batch inference size or the max number of images the exported model will process concurrently in `predict` mode. |
|
||||
| `device` | `str` | `None` | Specifies the device for exporting: CPU (`device=cpu`), MPS for Apple silicon (`device=mps`). |
|
||||
|
||||
For more details about the export process, visit the [Ultralytics documentation page on exporting](../modes/export.md).
|
||||
|
||||
## Deploying Exported YOLO26 PaddlePaddle Models
|
||||
|
||||
After successfully exporting your Ultralytics YOLO26 models to PaddlePaddle format, you can now deploy them. The primary and recommended first step for running a PaddlePaddle model is to use the YOLO("yolo26n_paddle_model/") method, as outlined in the previous usage code snippet.
|
||||
|
||||
However, for in-depth instructions on deploying your PaddlePaddle models in various other settings, take a look at the following resources:
|
||||
|
||||
- **[Paddle Serving](https://github.com/PaddlePaddle/Serving/blob/v0.9.0/README_CN.md)**: Learn how to deploy your PaddlePaddle models as performant services using Paddle Serving.
|
||||
|
||||
- **[Paddle Lite](https://github.com/PaddlePaddle/Paddle-Lite/blob/develop/README_en.md)**: Explore how to optimize and deploy models on mobile and embedded devices using Paddle Lite.
|
||||
|
||||
- **[Paddle.js](https://github.com/PaddlePaddle/Paddle.js)**: Discover how to run PaddlePaddle models in web browsers for client-side AI using Paddle.js.
|
||||
|
||||
## Summary
|
||||
|
||||
In this guide, we explored the process of exporting Ultralytics YOLO26 models to the PaddlePaddle format. By following these steps, you can leverage PaddlePaddle's strengths in diverse deployment scenarios, optimizing your models for different hardware and software environments.
|
||||
|
||||
For further details on usage, visit the [PaddlePaddle official documentation](https://www.paddlepaddle.org.cn/documentation/docs/en/guides/index_en.html).
|
||||
|
||||
Want to explore more ways to integrate your Ultralytics YOLO26 models? Our [integration guide page](index.md) explores various options, equipping you with valuable resources and insights.
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I export Ultralytics YOLO26 models to PaddlePaddle format?
|
||||
|
||||
Exporting Ultralytics YOLO26 models to PaddlePaddle format is straightforward. You can use the `export` method of the YOLO class to perform the conversion. Here is an example using Python:
|
||||
|
||||
!!! example "Usage"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load the YOLO26 model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Export the model to PaddlePaddle format
|
||||
model.export(format="paddle") # creates '/yolo26n_paddle_model'
|
||||
|
||||
# Load the exported PaddlePaddle model
|
||||
paddle_model = YOLO("./yolo26n_paddle_model")
|
||||
|
||||
# Run inference
|
||||
results = paddle_model("https://ultralytics.com/images/bus.jpg")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Export a YOLO26n PyTorch model to PaddlePaddle format
|
||||
yolo export model=yolo26n.pt format=paddle # creates '/yolo26n_paddle_model'
|
||||
|
||||
# Run inference with the exported model
|
||||
yolo predict model='./yolo26n_paddle_model' source='https://ultralytics.com/images/bus.jpg'
|
||||
```
|
||||
|
||||
For more detailed setup and troubleshooting, check the [Ultralytics Installation Guide](../quickstart.md) and [Common Issues Guide](../guides/yolo-common-issues.md).
|
||||
|
||||
### What are the advantages of using PaddlePaddle for [model deployment](https://www.ultralytics.com/glossary/model-deployment)?
|
||||
|
||||
PaddlePaddle offers several key advantages for model deployment:
|
||||
|
||||
- **Performance Optimization**: PaddlePaddle excels in efficient model execution and reduced memory usage.
|
||||
- **Dynamic-to-Static Graph Compilation**: It supports dynamic-to-static compilation, allowing for runtime optimizations.
|
||||
- **Operator Fusion**: By merging compatible operations, it reduces computational overhead.
|
||||
- **Quantization Techniques**: Supports both post-training and quantization-aware training, enabling lower-[precision](https://www.ultralytics.com/glossary/precision) data representations for improved performance.
|
||||
|
||||
You can achieve enhanced results by exporting your Ultralytics YOLO26 models to PaddlePaddle, ensuring flexibility and high performance across various applications and hardware platforms. Explore PaddlePaddle's key features and capabilities in the [official PaddlePaddle documentation](https://www.paddlepaddle.org.cn/en)..
|
||||
|
||||
### Why should I choose PaddlePaddle for deploying my YOLO26 models?
|
||||
|
||||
PaddlePaddle, developed by Baidu, is optimized for industrial and commercial AI deployments. Its large developer community and robust framework provide extensive tools similar to TensorFlow and PyTorch. By exporting your YOLO26 models to PaddlePaddle, you leverage:
|
||||
|
||||
- **Enhanced Performance**: Optimal execution speed and reduced memory footprint.
|
||||
- **Flexibility**: Wide compatibility with various devices from smartphones to cloud servers.
|
||||
- **Scalability**: Efficient parallel processing capabilities for distributed environments.
|
||||
|
||||
These features make PaddlePaddle a compelling choice for deploying YOLO26 models in production settings.
|
||||
|
||||
### How does PaddlePaddle improve model performance over other frameworks?
|
||||
|
||||
PaddlePaddle employs several advanced techniques to optimize model performance:
|
||||
|
||||
- **Dynamic-to-Static Graph**: Converts models into a static computational graph for runtime optimizations.
|
||||
- **Operator Fusion**: Combines compatible operations to minimize memory transfer and increase inference speed.
|
||||
- **Quantization**: Reduces model size and increases efficiency using lower-precision data while maintaining [accuracy](https://www.ultralytics.com/glossary/accuracy).
|
||||
|
||||
These techniques prioritize efficient model execution, making PaddlePaddle an excellent option for deploying high-performance YOLO26 models. For more on optimization, see the [PaddlePaddle official documentation](https://www.paddlepaddle.org.cn/documentation/docs/en/guides/index_en.html).
|
||||
|
||||
### What deployment options does PaddlePaddle offer for YOLO26 models?
|
||||
|
||||
PaddlePaddle provides flexible deployment options:
|
||||
|
||||
- **Paddle Serving**: Deploys models as RESTful APIs, ideal for production with features like model versioning and online A/B testing.
|
||||
- **Paddle Inference API**: Gives low-level control over model execution for custom applications.
|
||||
- **Paddle Lite**: Optimizes models for mobile and embedded devices' limited resources.
|
||||
- **Paddle.js**: Enables deploying models directly within web browsers.
|
||||
|
||||
These options cover a broad range of deployment scenarios, from on-device inference to scalable cloud services. Explore more deployment strategies on the [Ultralytics Model Deployment Options page](../guides/model-deployment-options.md).
|
||||
@@ -0,0 +1,115 @@
|
||||
---
|
||||
comments: true
|
||||
description: Simplify YOLO26 training with Paperspace Gradient's all-in-one MLOps platform. Access GPUs, automate workflows, and deploy with ease.
|
||||
keywords: YOLO26, Paperspace Gradient, MLOps, machine learning, training, GPUs, Jupyter notebooks, model deployment, AI, cloud platform
|
||||
---
|
||||
|
||||
# YOLO26 Model Training Made Simple with Paperspace Gradient
|
||||
|
||||
Training computer vision models like [YOLO26](../models/yolo26.md) can be complicated. It involves managing large datasets, using different types of computer hardware like GPUs, TPUs, and CPUs, and making sure data flows smoothly during the training process. Typically, developers end up spending a lot of time managing their computer systems and environments. It can be frustrating when you just want to focus on building the best model.
|
||||
|
||||
This is where a platform like Paperspace Gradient can make things simpler. Paperspace Gradient is a MLOps platform that lets you build, train, and deploy [machine learning](https://www.ultralytics.com/glossary/machine-learning-ml) models all in one place. With Gradient, developers can focus on training their YOLO26 models without the hassle of managing infrastructure and environments.
|
||||
|
||||
## Paperspace
|
||||
|
||||
<p align="center">
|
||||
<img width="100%" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/paperspace-overview.avif" alt="Paperspace GPU cloud computing for ML training">
|
||||
</p>
|
||||
|
||||
[Paperspace](https://www.paperspace.com/), launched in 2014 by University of Michigan graduates and acquired by DigitalOcean in 2023, is a cloud platform specifically designed for machine learning. It provides users with powerful GPUs, collaborative Jupyter notebooks, a container service for deployments, automated workflows for machine learning tasks, and high-performance virtual machines. These features aim to streamline the entire machine learning development process, from coding to deployment.
|
||||
|
||||
## Paperspace Gradient
|
||||
|
||||
<p align="center">
|
||||
<img width="100%" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/paperspace-gradient-overview.avif" alt="Paperspace Gradient cloud ML platform">
|
||||
</p>
|
||||
|
||||
Paperspace Gradient is a suite of tools designed to make working with AI and machine learning in the cloud much faster and easier. Gradient addresses the entire [machine learning lifecycle](https://www.ultralytics.com/blog/measuring-ai-performance-to-weigh-the-impact-of-your-innovations), from building and training models to deploying them.
|
||||
|
||||
Within its toolkit, it includes support for Google's TPUs via a job runner, comprehensive support for Jupyter notebooks and containers, and new programming language integrations. Its focus on language integration particularly stands out, allowing users to easily adapt their existing Python projects to use the most advanced GPU infrastructure available.
|
||||
|
||||
## Training YOLO26 Using Paperspace Gradient
|
||||
|
||||
Paperspace Gradient makes training a YOLO26 model possible with a few clicks. Thanks to the integration, you can access the [Paperspace console](https://console.paperspace.com/github/ultralytics/ultralytics) and start training your model immediately. For a detailed understanding of the model training process and best practices, refer to our [YOLO26 Model Training guide](../modes/train.md).
|
||||
|
||||
Sign in and then click on the "Start Machine" button shown in the image below. In a few seconds, a managed GPU environment will start up, and then you can run the notebook's cells.
|
||||
|
||||
|
||||
|
||||
Explore more capabilities of YOLO26 and Paperspace Gradient in a discussion with Glenn Jocher, Ultralytics founder, and James Skelton from Paperspace. Watch the discussion below.
|
||||
|
||||
<p align="center">
|
||||
<br>
|
||||
<iframe loading="lazy" width="720" height="405" src="https://www.youtube.com/embed/3HbbQHitN7g?si=DjuwrzMkW1WEoH5Y"
|
||||
title="YouTube video player" frameborder="0"
|
||||
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"
|
||||
allowfullscreen>
|
||||
</iframe>
|
||||
<br>
|
||||
<strong>Watch:</strong> Ultralytics Live Session 7: It's All About the Environment: Optimizing YOLO26 Training With Gradient
|
||||
</p>
|
||||
|
||||
## Key Features of Paperspace Gradient
|
||||
|
||||
As you explore the [Paperspace console](https://console.paperspace.com/github/ultralytics/ultralytics), you'll see how each step of the machine-learning workflow is supported and enhanced. Here are some things to look out for:
|
||||
|
||||
- **One-Click Notebooks:** Gradient provides pre-configured [Jupyter Notebooks](../integrations/jupyterlab.md) specifically tailored for YOLO26, eliminating the need for environment setup and dependency management. Simply choose the desired notebook and start experimenting immediately.
|
||||
|
||||
- **Hardware Flexibility:** Choose from a range of machine types with varying CPU, GPU, and TPU configurations to suit your training needs and budget. Gradient handles all the backend setup, allowing you to focus on model development.
|
||||
|
||||
- **Experiment Tracking:** Gradient automatically tracks your experiments, including hyperparameters, metrics, and code changes. This allows you to easily compare different training runs, identify optimal configurations, and reproduce successful results.
|
||||
|
||||
- **Dataset Management:** Efficiently manage your datasets directly within Gradient. Upload, version, and pre-process data with ease, streamlining the data preparation phase of your project.
|
||||
|
||||
- **Model Serving:** Deploy your trained YOLO26 models as REST APIs with just a few clicks. Gradient handles the infrastructure, allowing you to easily integrate your [object detection](https://www.ultralytics.com/glossary/object-detection) models into your applications.
|
||||
|
||||
- **Real-time Monitoring:** Monitor the performance and health of your deployed models through Gradient's intuitive dashboard. Gain insights into inference speed, resource utilization, and potential errors.
|
||||
|
||||
## Why Should You Use Gradient for Your YOLO26 Projects?
|
||||
|
||||
While many options are available for training, deploying, and evaluating YOLO26 models, the integration with Paperspace Gradient offers a unique set of advantages that separates it from other solutions. Let's explore what makes this integration unique:
|
||||
|
||||
- **Enhanced Collaboration:** Shared workspaces and version control facilitate seamless teamwork and ensure reproducibility, allowing your team to work together effectively and maintain a clear history of your project.
|
||||
|
||||
- **Low-Cost GPUs:** Gradient provides access to high-performance GPUs at significantly lower costs than major cloud providers or on-premise solutions. With per-second billing, you only pay for the resources you actually use, optimizing your budget.
|
||||
|
||||
- **Predictable Costs:** Gradient's on-demand pricing ensures cost transparency and predictability. You can scale your resources up or down as needed and only pay for the time you use, avoiding unnecessary expenses.
|
||||
|
||||
- **No Commitments:** You can adjust your instance types anytime to adapt to changing project requirements and optimize the cost-performance balance. There are no lock-in periods or commitments, providing maximum flexibility.
|
||||
|
||||
## Summary
|
||||
|
||||
This guide explored the [Paperspace Gradient integration](https://www.ultralytics.com/blog/ultralytics-x-paperspace-advancing-object-detection-capabilities-through-partnership) for training YOLO26 models. Gradient provides the tools and infrastructure to accelerate your AI development journey from effortless model training and evaluation to streamlined deployment options.
|
||||
|
||||
For further exploration, visit [Paperspace's official documentation](https://docs.digitalocean.com/products/paperspace/).
|
||||
|
||||
Also, visit the [Ultralytics integration guide page](index.md) to learn more about different YOLO26 integrations. It's full of insights and tips to take your [computer vision](https://www.ultralytics.com/glossary/computer-vision-cv) projects to the next level.
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I train a YOLO26 model using Paperspace Gradient?
|
||||
|
||||
Training a YOLO26 model with Paperspace Gradient is straightforward and efficient. First, sign in to the [Paperspace console](https://console.paperspace.com/github/ultralytics/ultralytics). Next, click the "Start Machine" button to initiate a managed GPU environment. Once the environment is ready, you can run the notebook's cells to start training your YOLO26 model. For detailed instructions, refer to our [YOLO26 Model Training guide](../modes/train.md).
|
||||
|
||||
### What are the advantages of using Paperspace Gradient for YOLO26 projects?
|
||||
|
||||
Paperspace Gradient offers several unique advantages for training and deploying YOLO26 models:
|
||||
|
||||
- **Hardware Flexibility:** Choose from various CPU, GPU, and TPU configurations.
|
||||
- **One-Click Notebooks:** Use pre-configured Jupyter Notebooks for YOLO26 without worrying about environment setup.
|
||||
- **Experiment Tracking:** Automatic tracking of hyperparameters, metrics, and code changes.
|
||||
- **Dataset Management:** Efficiently manage your datasets within Gradient.
|
||||
- **Model Serving:** Deploy models as REST APIs easily.
|
||||
- **Real-time Monitoring:** Monitor model performance and resource utilization through a dashboard.
|
||||
|
||||
### Why should I choose Ultralytics YOLO26 over other object detection models?
|
||||
|
||||
Ultralytics YOLO26 stands out for its real-time object detection capabilities and high [accuracy](https://www.ultralytics.com/glossary/accuracy). Its seamless integration with platforms like Paperspace Gradient enhances productivity by simplifying the training and deployment process. YOLO26 supports various use cases, from security systems to retail inventory management. Discover the full range of YOLO26's capabilities and benefits in our [YOLO26 overview](https://www.ultralytics.com/yolo).
|
||||
|
||||
### Can I deploy my YOLO26 model on edge devices using Paperspace Gradient?
|
||||
|
||||
Yes, you can deploy YOLO26 models on edge devices using Paperspace Gradient. The platform supports various deployment formats like [TFLite](../integrations/tflite.md) and [Edge TPU](../integrations/edge-tpu.md), which are optimized for edge devices. After training your model on Gradient, refer to our [export guide](../modes/export.md) for instructions on converting your model to the desired format.
|
||||
|
||||
### How does experiment tracking in Paperspace Gradient help improve YOLO26 training?
|
||||
|
||||
Experiment tracking in Paperspace Gradient streamlines the model development process by automatically logging hyperparameters, metrics, and code changes. This allows you to easily compare different training runs, identify optimal configurations, and reproduce successful experiments. Similar functionality can be found in other [experiment tracking tools](../integrations/clearml.md) that integrate with Ultralytics YOLO26.
|
||||
@@ -0,0 +1,306 @@
|
||||
---
|
||||
comments: true
|
||||
description: Optimize YOLO26 model performance with Ray Tune. Learn efficient hyperparameter tuning using advanced search strategies, parallelism, and early stopping.
|
||||
keywords: YOLO26, Ray Tune, hyperparameter tuning, model optimization, machine learning, deep learning, AI, Ultralytics, Weights & Biases
|
||||
---
|
||||
|
||||
# Efficient Hyperparameter Tuning with Ray Tune and YOLO26
|
||||
|
||||
Hyperparameter tuning is vital in achieving peak model performance by discovering the optimal set of hyperparameters. This involves running trials with different hyperparameters and evaluating each trial's performance.
|
||||
|
||||
## Accelerate Tuning with Ultralytics YOLO26 and Ray Tune
|
||||
|
||||
[Ultralytics YOLO26](https://www.ultralytics.com/) incorporates Ray Tune for hyperparameter tuning, streamlining the optimization of YOLO26 model hyperparameters. With Ray Tune, you can utilize advanced search strategies, parallelism, and early stopping to expedite the tuning process.
|
||||
|
||||
### Ray Tune
|
||||
|
||||
<p align="center">
|
||||
<img width="640" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/ray-tune-overview.avif" alt="Ray Tune hyperparameter optimization workflow">
|
||||
</p>
|
||||
|
||||
[Ray Tune](https://docs.ray.io/en/latest/tune/index.html) is a hyperparameter tuning library designed for efficiency and flexibility. It supports various search strategies, parallelism, and early stopping strategies, and seamlessly integrates with popular [machine learning](https://www.ultralytics.com/glossary/machine-learning-ml) frameworks, including Ultralytics YOLO26.
|
||||
|
||||
### Integration with Weights & Biases
|
||||
|
||||
YOLO26 also allows optional integration with [Weights & Biases](https://wandb.ai/site) for monitoring the tuning process.
|
||||
|
||||
## Installation
|
||||
|
||||
To install the required packages, run:
|
||||
|
||||
!!! tip "Installation"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Install and update Ultralytics and Ray Tune packages
|
||||
pip install -U ultralytics "ray[tune]"
|
||||
|
||||
# Optionally install W&B for logging
|
||||
pip install wandb
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
!!! example "Usage"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load a YOLO26n model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Start tuning hyperparameters for YOLO26n training on the COCO8 dataset
|
||||
result_grid = model.tune(data="coco8.yaml", use_ray=True)
|
||||
```
|
||||
|
||||
## `tune()` Method Parameters
|
||||
|
||||
The `tune()` method in YOLO26 provides an easy-to-use interface for hyperparameter tuning with Ray Tune. It accepts several arguments that allow you to customize the tuning process. Below is a detailed explanation of each parameter:
|
||||
|
||||
| Parameter | Type | Description | Default Value |
|
||||
| --------------- | ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- |
|
||||
| `data` | `str` | The dataset configuration file (in YAML format) to run the tuner on. This file should specify the training and [validation data](https://www.ultralytics.com/glossary/validation-data) paths, as well as other dataset-specific settings. | |
|
||||
| `space` | `dict, optional` | A dictionary defining the hyperparameter search space for Ray Tune. Each key corresponds to a hyperparameter name, and the value specifies the range of values to explore during tuning. If not provided, YOLO26 uses a default search space with various hyperparameters. | |
|
||||
| `grace_period` | `int, optional` | The grace period in [epochs](https://www.ultralytics.com/glossary/epoch) for the [ASHA scheduler](https://docs.ray.io/en/latest/tune/api/schedulers.html) in Ray Tune. The scheduler will not terminate any trial before this number of epochs, allowing the model to have some minimum training before making a decision on early stopping. | 10 |
|
||||
| `gpu_per_trial` | `int, optional` | The number of GPUs to allocate per trial during tuning. This helps manage GPU usage, particularly in multi-GPU environments. If not provided, the tuner will use all available GPUs. | `None` |
|
||||
| `iterations` | `int, optional` | The maximum number of trials to run during tuning. This parameter helps control the total number of hyperparameter combinations tested, ensuring the tuning process does not run indefinitely. | 10 |
|
||||
| `**train_args` | `dict, optional` | Additional arguments to pass to the `train()` method during tuning. These arguments can include settings like the number of training epochs, [batch size](https://www.ultralytics.com/glossary/batch-size), and other training-specific configurations. | {} |
|
||||
|
||||
By customizing these parameters, you can fine-tune the hyperparameter optimization process to suit your specific needs and available computational resources.
|
||||
|
||||
## Default Search Space Description
|
||||
|
||||
The following table lists the default search space parameters for hyperparameter tuning in YOLO26 with Ray Tune. Each parameter has a specific value range defined by `tune.uniform()`.
|
||||
|
||||
| Parameter | Range | Description |
|
||||
| ----------------- | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `lr0` | `tune.uniform(1e-5, 1e-1)` | Initial learning rate that controls the step size during optimization. Higher values speed up training but may cause instability. |
|
||||
| `lrf` | `tune.uniform(0.01, 1.0)` | Final learning rate factor that determines how much the learning rate decreases by the end of training. |
|
||||
| `momentum` | `tune.uniform(0.6, 0.98)` | Momentum factor for the optimizer that helps accelerate training and overcome local minima. |
|
||||
| `weight_decay` | `tune.uniform(0.0, 0.001)` | Regularization parameter that prevents overfitting by penalizing large weight values. |
|
||||
| `warmup_epochs` | `tune.uniform(0.0, 5.0)` | Number of epochs with gradually increasing learning rate to stabilize early training. |
|
||||
| `warmup_momentum` | `tune.uniform(0.0, 0.95)` | Initial momentum value that gradually increases during the warmup period. |
|
||||
| `box` | `tune.uniform(0.02, 0.2)` | Weight for the bounding box loss component, balancing localization accuracy in the model. |
|
||||
| `cls` | `tune.uniform(0.2, 4.0)` | Weight for the classification loss component, balancing class prediction accuracy in the model. |
|
||||
| `hsv_h` | `tune.uniform(0.0, 0.1)` | Hue augmentation range that introduces color variability to help the model generalize. |
|
||||
| `hsv_s` | `tune.uniform(0.0, 0.9)` | Saturation augmentation range that varies color intensity to improve robustness. |
|
||||
| `hsv_v` | `tune.uniform(0.0, 0.9)` | Value (brightness) augmentation range that helps the model perform under various lighting conditions. |
|
||||
| `degrees` | `tune.uniform(0.0, 45.0)` | Rotation augmentation range in degrees, improving recognition of rotated objects. |
|
||||
| `translate` | `tune.uniform(0.0, 0.9)` | Translation augmentation range that shifts images horizontally and vertically. |
|
||||
| `scale` | `tune.uniform(0.0, 0.9)` | Scaling augmentation range that simulates objects at different distances. |
|
||||
| `shear` | `tune.uniform(0.0, 10.0)` | Shear augmentation range in degrees, simulating perspective shifts. |
|
||||
| `perspective` | `tune.uniform(0.0, 0.001)` | Perspective augmentation range that simulates 3D viewpoint changes. |
|
||||
| `flipud` | `tune.uniform(0.0, 1.0)` | Vertical flip augmentation probability, increasing dataset diversity. |
|
||||
| `fliplr` | `tune.uniform(0.0, 1.0)` | Horizontal flip augmentation probability, useful for symmetrical objects. |
|
||||
| `mosaic` | `tune.uniform(0.0, 1.0)` | Mosaic augmentation probability that combines four images into one training sample. |
|
||||
| `mixup` | `tune.uniform(0.0, 1.0)` | Mixup augmentation probability that blends two images and their labels together. |
|
||||
| `cutmix` | `tune.uniform(0.0, 1.0)` | Cutmix augmentation probability that combines image regions while maintaining local features, improving detection of partially occluded objects. |
|
||||
| `copy_paste` | `tune.uniform(0.0, 1.0)` | Copy-paste augmentation probability that transfers objects between images to increase instance diversity. |
|
||||
|
||||
## Custom Search Space Example
|
||||
|
||||
In this example, we demonstrate how to use a custom search space for hyperparameter tuning with Ray Tune and YOLO26. By providing a custom search space, you can focus the tuning process on specific hyperparameters of interest.
|
||||
|
||||
!!! example "Usage"
|
||||
|
||||
```python
|
||||
from ray import tune
|
||||
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Define a YOLO model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Run Ray Tune on the model
|
||||
result_grid = model.tune(
|
||||
data="coco8.yaml",
|
||||
space={"lr0": tune.uniform(1e-5, 1e-1)},
|
||||
epochs=50,
|
||||
use_ray=True,
|
||||
)
|
||||
```
|
||||
|
||||
In the code snippet above, we create a YOLO model with the "yolo26n.pt" pretrained weights. Then, we call the `tune()` method, specifying the dataset configuration with "coco8.yaml". We provide a custom search space for the initial learning rate `lr0` using a dictionary with the key "lr0" and the value `tune.uniform(1e-5, 1e-1)`. Finally, we pass additional training arguments, such as the number of epochs directly to the tune method as `epochs=50`.
|
||||
|
||||
## Resuming An Interrupted Hyperparameter Tuning Session With Ray Tune
|
||||
|
||||
You can resume an interrupted Ray Tune session by passing `resume=True`. You can optionally pass the directory `name` used by Ray Tune under `runs/{task}` to resume. Otherwise, it would resume the last interrupted session. You don't need to provide the `iterations` and `space` again, but you need to provide the rest of the training arguments again including `data` and `epochs`.
|
||||
|
||||
!!! example "Using `resume=True` with `model.tune()`"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Define a YOLO model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Resume previous run
|
||||
results = model.tune(use_ray=True, data="coco8.yaml", epochs=50, resume=True)
|
||||
|
||||
# Resume Ray Tune run with name 'tune_exp_2'
|
||||
results = model.tune(use_ray=True, data="coco8.yaml", epochs=50, name="tune_exp_2", resume=True)
|
||||
```
|
||||
|
||||
## Processing Ray Tune Results
|
||||
|
||||
After running a hyperparameter tuning experiment with Ray Tune, you might want to perform various analyses on the obtained results. This guide will take you through common workflows for processing and analyzing these results.
|
||||
|
||||
### Loading Tune Experiment Results from a Directory
|
||||
|
||||
After running the tuning experiment with `tuner.fit()`, you can load the results from a directory. This is useful, especially if you're performing the analysis after the initial training script has exited.
|
||||
|
||||
```python
|
||||
experiment_path = f"{storage_path}/{exp_name}"
|
||||
print(f"Loading results from {experiment_path}...")
|
||||
|
||||
restored_tuner = tune.Tuner.restore(experiment_path, trainable=train_mnist)
|
||||
result_grid = restored_tuner.get_results()
|
||||
```
|
||||
|
||||
### Basic Experiment-Level Analysis
|
||||
|
||||
Get an overview of how trials performed. You can quickly check if there were any errors during the trials.
|
||||
|
||||
```python
|
||||
if result_grid.errors:
|
||||
print("One or more trials failed!")
|
||||
else:
|
||||
print("No errors!")
|
||||
```
|
||||
|
||||
### Basic Trial-Level Analysis
|
||||
|
||||
Access individual trial hyperparameter configurations and the last reported metrics.
|
||||
|
||||
```python
|
||||
for i, result in enumerate(result_grid):
|
||||
print(f"Trial #{i}: Configuration: {result.config}, Last Reported Metrics: {result.metrics}")
|
||||
```
|
||||
|
||||
### Plotting the Entire History of Reported Metrics for a Trial
|
||||
|
||||
You can plot the history of reported metrics for each trial to see how the metrics evolved over time.
|
||||
|
||||
```python
|
||||
import matplotlib.pyplot as plt
|
||||
|
||||
for i, result in enumerate(result_grid):
|
||||
plt.plot(
|
||||
result.metrics_dataframe["training_iteration"],
|
||||
result.metrics_dataframe["mean_accuracy"],
|
||||
label=f"Trial {i}",
|
||||
)
|
||||
|
||||
plt.xlabel("Training Iterations")
|
||||
plt.ylabel("Mean Accuracy")
|
||||
plt.legend()
|
||||
plt.show()
|
||||
```
|
||||
|
||||
## Summary
|
||||
|
||||
In this guide, we covered common workflows to analyze the results of experiments run with Ray Tune using Ultralytics. The key steps include loading the experiment results from a directory, performing basic experiment-level and trial-level analysis, and plotting metrics.
|
||||
|
||||
Explore further by looking into Ray Tune's [Analyze Results](https://docs.ray.io/en/latest/tune/examples/tune_analyze_results.html) docs page to get the most out of your hyperparameter tuning experiments.
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I tune the hyperparameters of my YOLO26 model using Ray Tune?
|
||||
|
||||
To tune the hyperparameters of your Ultralytics YOLO26 model using Ray Tune, follow these steps:
|
||||
|
||||
1. **Install the required packages:**
|
||||
|
||||
```bash
|
||||
pip install -U ultralytics "ray[tune]"
|
||||
pip install wandb # optional for logging
|
||||
```
|
||||
|
||||
2. **Load your YOLO26 model and start tuning:**
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load a YOLO26 model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Start tuning with the COCO8 dataset
|
||||
result_grid = model.tune(data="coco8.yaml", use_ray=True)
|
||||
```
|
||||
|
||||
This utilizes Ray Tune's advanced search strategies and parallelism to efficiently optimize your model's hyperparameters. For more information, check out the [Ray Tune documentation](https://docs.ray.io/en/latest/tune/index.html).
|
||||
|
||||
### What are the default hyperparameters for YOLO26 tuning with Ray Tune?
|
||||
|
||||
Ultralytics YOLO26 uses the following default hyperparameters for tuning with Ray Tune:
|
||||
|
||||
| Parameter | Value Range | Description |
|
||||
| --------------- | -------------------------- | ------------------------------ |
|
||||
| `lr0` | `tune.uniform(1e-5, 1e-1)` | Initial learning rate |
|
||||
| `lrf` | `tune.uniform(0.01, 1.0)` | Final learning rate factor |
|
||||
| `momentum` | `tune.uniform(0.6, 0.98)` | Momentum |
|
||||
| `weight_decay` | `tune.uniform(0.0, 0.001)` | Weight decay |
|
||||
| `warmup_epochs` | `tune.uniform(0.0, 5.0)` | Warmup epochs |
|
||||
| `box` | `tune.uniform(0.02, 0.2)` | Box loss weight |
|
||||
| `cls` | `tune.uniform(0.2, 4.0)` | Class loss weight |
|
||||
| `hsv_h` | `tune.uniform(0.0, 0.1)` | Hue augmentation range |
|
||||
| `translate` | `tune.uniform(0.0, 0.9)` | Translation augmentation range |
|
||||
|
||||
These hyperparameters can be customized to suit your specific needs. For a complete list and more details, refer to the [Hyperparameter Tuning](../guides/hyperparameter-tuning.md) guide.
|
||||
|
||||
### How can I integrate Weights & Biases with my YOLO26 model tuning?
|
||||
|
||||
To integrate Weights & Biases (W&B) with your Ultralytics YOLO26 tuning process:
|
||||
|
||||
1. **Install W&B:**
|
||||
|
||||
```bash
|
||||
pip install wandb
|
||||
```
|
||||
|
||||
2. **Modify your tuning script:**
|
||||
|
||||
```python
|
||||
import wandb
|
||||
|
||||
from ultralytics import YOLO
|
||||
|
||||
wandb.init(project="YOLO-Tuning", entity="your-entity")
|
||||
|
||||
# Load YOLO model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Tune hyperparameters
|
||||
result_grid = model.tune(data="coco8.yaml", use_ray=True)
|
||||
```
|
||||
|
||||
This setup will allow you to monitor the tuning process, track hyperparameter configurations, and visualize results in W&B.
|
||||
|
||||
### Why should I use Ray Tune for hyperparameter optimization with YOLO26?
|
||||
|
||||
Ray Tune offers numerous advantages for hyperparameter optimization:
|
||||
|
||||
- **Advanced Search Strategies:** Utilizes algorithms like [Bayesian Optimization](https://www.ultralytics.com/glossary/bayesian-network) and HyperOpt for efficient parameter search.
|
||||
- **Parallelism:** Supports parallel execution of multiple trials, significantly speeding up the tuning process.
|
||||
- **Early Stopping:** Employs strategies like ASHA to terminate under-performing trials early, saving computational resources.
|
||||
|
||||
Ray Tune seamlessly integrates with Ultralytics YOLO26, providing an easy-to-use interface for tuning hyperparameters effectively. To get started, check out the [Hyperparameter Tuning](../guides/hyperparameter-tuning.md) guide.
|
||||
|
||||
### How can I define a custom search space for YOLO26 hyperparameter tuning?
|
||||
|
||||
To define a custom search space for your YOLO26 hyperparameter tuning with Ray Tune:
|
||||
|
||||
```python
|
||||
from ray import tune
|
||||
|
||||
from ultralytics import YOLO
|
||||
|
||||
model = YOLO("yolo26n.pt")
|
||||
search_space = {"lr0": tune.uniform(1e-5, 1e-1), "momentum": tune.uniform(0.6, 0.98)}
|
||||
result_grid = model.tune(data="coco8.yaml", space=search_space, use_ray=True)
|
||||
```
|
||||
|
||||
This customizes the range of hyperparameters like initial learning rate and momentum to be explored during the tuning process. For advanced configurations, refer to the [Custom Search Space Example](#custom-search-space-example) section.
|
||||
@@ -0,0 +1,216 @@
|
||||
---
|
||||
comments: true
|
||||
description: Learn how to label data and export datasets in YOLO format using Roboflow for training Ultralytics models.
|
||||
keywords: Roboflow, Ultralytics YOLO, data labeling, computer vision, dataset export
|
||||
---
|
||||
|
||||
# Roboflow
|
||||
|
||||
[Roboflow](https://roboflow.com/?ref=ultralytics) provides tools for [data labeling](https://www.ultralytics.com/glossary/data-labeling) and dataset export in various formats, including YOLO. This guide covers labeling, exporting, and deploying data for [Ultralytics YOLO](../models/index.md) models.
|
||||
|
||||
!!! question "Licensing"
|
||||
|
||||
Ultralytics offers two licensing options to accommodate different use cases:
|
||||
|
||||
- **AGPL-3.0 License**: This [OSI-approved open-source license](https://www.ultralytics.com/legal/agpl-3-0-software-license) is ideal for students and enthusiasts, promoting open collaboration and knowledge sharing. See the [LICENSE](https://github.com/ultralytics/ultralytics/blob/main/LICENSE) file for more details.
|
||||
- **Enterprise License**: Designed for commercial use, this license allows for the seamless integration of Ultralytics software and AI models into commercial products and services. If your scenario involves commercial applications, please reach out via [Ultralytics Licensing](https://www.ultralytics.com/license).
|
||||
|
||||
For more details see the [Ultralytics Licensing page](https://www.ultralytics.com/license).
|
||||
|
||||
This guide demonstrates how to find, label, and organize data for training a custom [Ultralytics YOLO26](../models/yolo26.md) model using Roboflow.
|
||||
|
||||
- [Gather Data for Training](#gather-data-for-training-a-custom-yolo26-model)
|
||||
- [Label Data](#upload-convert-and-label-data-for-yolo26-format)
|
||||
- [Dataset Management](#dataset-management-for-yolo26)
|
||||
- [Export Data](#export-data-in-40-formats-for-model-training)
|
||||
- [Deploy Models](#upload-custom-yolo26-model-weights-for-testing-and-deployment)
|
||||
- [Evaluate Models](#how-to-evaluate-yolo26-models)
|
||||
- [FAQ](#faq)
|
||||
|
||||
## Gather Data for Training a Custom YOLO26 Model
|
||||
|
||||
Roboflow offers two primary services to assist in data collection for Ultralytics [YOLO models](../models/index.md): Universe and Collect. For more general information on data collection strategies, refer to our [Data Collection and Annotation Guide](../guides/data-collection-and-annotation.md).
|
||||
|
||||
### Roboflow Universe
|
||||
|
||||
Roboflow Universe is an online repository of vision [datasets](../datasets/index.md). You can export datasets in YOLO format for use with Ultralytics models.
|
||||
|
||||
### Roboflow Collect
|
||||
|
||||
If you prefer to gather images yourself, Roboflow Collect is an open-source project enabling automatic image collection via a webcam on edge devices. You can use text or image prompts to specify the data to be collected, helping capture only the necessary images for your vision model.
|
||||
|
||||
## Upload, Convert and Label Data for YOLO26 Format
|
||||
|
||||
Roboflow Annotate is an online tool for labeling images for various computer vision tasks, including [object detection](../tasks/detect.md), [classification](../tasks/classify.md), and [segmentation](../tasks/segment.md).
|
||||
|
||||
To label data for an Ultralytics [YOLO](../models/index.md) model, create a project in Roboflow, upload your images, and start annotating.
|
||||
|
||||
### Annotation Tools
|
||||
|
||||
- **Bounding Box Annotation**: Press `B` or click the box icon. Click and drag to create the [bounding box](https://www.ultralytics.com/glossary/bounding-box). A pop-up will prompt you to select a class for the annotation.
|
||||
- **Polygon Annotation**: Used for [instance segmentation](https://www.ultralytics.com/glossary/instance-segmentation). Press `P` or click the polygon icon. Click points around the object to draw the polygon.
|
||||
|
||||
### Label Assistant (SAM Integration)
|
||||
|
||||
Roboflow integrates a [Segment Anything Model (SAM)](../models/sam.md)-based label assistant to potentially speed up annotation.
|
||||
|
||||
To use the label assistant, click the cursor icon in the sidebar. SAM will be enabled for your project.
|
||||
|
||||
Hover over an object, and SAM may suggest an annotation. Click to accept the annotation. You can refine the annotation's specificity by clicking inside or outside the suggested area.
|
||||
|
||||
### Tagging
|
||||
|
||||
You can add tags to images using the Tags panel in the sidebar. Tags can represent attributes like location, camera source, etc. These tags allow you to search for specific images and generate dataset versions containing images with particular tags.
|
||||
|
||||
### Label Assist (Model-Based)
|
||||
|
||||
Models hosted on Roboflow can be used with Label Assist to suggest annotations. Upload your YOLO model weights to Roboflow (see instructions below), then activate Label Assist via the magic wand icon in the sidebar.
|
||||
|
||||
## Dataset Management for YOLO26
|
||||
|
||||
Roboflow provides several tools for understanding and managing your computer vision [datasets](../datasets/index.md).
|
||||
|
||||
### Dataset Search
|
||||
|
||||
Use dataset search to find images based on text descriptions or specific labels/tags. Access this feature by clicking "Dataset" in the sidebar.
|
||||
|
||||
### Health Check
|
||||
|
||||
Before training, use Roboflow Health Check to gain insights into your dataset and identify potential improvements. Access it via the "Health Check" sidebar link. It provides statistics on image sizes, class balance, annotation heatmaps, and more.
|
||||
|
||||
<p align="center">
|
||||
<img src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/rf-dataset-health-check.avif" alt="Roboflow Health Check analysis dashboard" width="800">
|
||||
</p>
|
||||
|
||||
Health Check might suggest changes to enhance performance, such as addressing class imbalances identified in the class balance feature. Understanding dataset health is crucial for effective [model training](../modes/train.md).
|
||||
|
||||
## Pre-process and Augment Data for Model Robustness
|
||||
|
||||
To export your data, you need to create a dataset version, which is a snapshot of your dataset at a specific point in time. Click "Versions" in the sidebar, then "Create New Version." Here, you can apply preprocessing steps and [data augmentations](https://www.ultralytics.com/glossary/data-augmentation) to potentially enhance model robustness.
|
||||
|
||||
<p align="center">
|
||||
<img src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/creating-dataset-version-on-roboflow.avif" alt="Creating Roboflow dataset version with augmentation" width="800">
|
||||
</p>
|
||||
|
||||
For each selected augmentation, a pop-up allows you to fine-tune its parameters such as brightness. Proper augmentation can significantly improve model generalization, a key concept discussed in our [model training tips guide](../guides/model-training-tips.md).
|
||||
|
||||
## Export Data in 40+ Formats for Model Training
|
||||
|
||||
Once your dataset version is generated, you can export it in various formats suitable for model training. Click the "Export Dataset" button on the version page.
|
||||
|
||||
<p align="center">
|
||||
<img src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/exporting-dataset.avif" alt="Roboflow dataset export to YOLO format" width="800">
|
||||
</p>
|
||||
|
||||
Select the "YOLO26" format for compatibility with Ultralytics training pipelines. You are now ready to train your custom [YOLO26](../models/yolo26.md) model. Refer to the [Ultralytics Train mode documentation](../modes/train.md) for detailed instructions on initiating training with your exported dataset.
|
||||
|
||||
## Upload Custom YOLO26 Model Weights for Testing and Deployment
|
||||
|
||||
Roboflow offers a scalable API for deployed models and SDKs compatible with devices like [NVIDIA Jetson](https://developer.nvidia.com/embedded-computing), [Luxonis OAK](https://www.luxonis.com/), [Raspberry Pi](../guides/raspberry-pi.md), and GPU-based systems. Explore various [model deployment options](../guides/model-deployment-options.md) in our guides.
|
||||
|
||||
You can deploy YOLO26 models by uploading their weights to Roboflow using a simple [Python](https://www.python.org/) script.
|
||||
|
||||
Create a new Python file and add the following code:
|
||||
|
||||
```python
|
||||
import roboflow # install with 'pip install roboflow'
|
||||
|
||||
# Log in to Roboflow (requires API key)
|
||||
roboflow.login()
|
||||
|
||||
# Initialize Roboflow client
|
||||
rf = roboflow.Roboflow()
|
||||
|
||||
# Define your workspace and project details
|
||||
WORKSPACE_ID = "your-workspace-id" # Replace with your actual Workspace ID
|
||||
PROJECT_ID = "your-project-id" # Replace with your actual Project ID
|
||||
VERSION = 1 # Replace with your desired dataset version number
|
||||
MODEL_PATH = "path/to/your/runs/detect/train/" # Replace with the path to your YOLO26 training results directory
|
||||
|
||||
# Get project and version
|
||||
project = rf.workspace(WORKSPACE_ID).project(PROJECT_ID)
|
||||
dataset = project.version(VERSION)
|
||||
|
||||
# Upload model weights for deployment
|
||||
# Ensure MODEL_PATH points to the directory containing 'best.pt'
|
||||
dataset.deploy(
|
||||
model_type="yolov8",
|
||||
model_path=MODEL_PATH,
|
||||
) # Note: Use "yolov8" as model_type for YOLO26 compatibility in Roboflow deployment
|
||||
|
||||
print(f"Model from {MODEL_PATH} uploaded to Roboflow project {PROJECT_ID}, version {VERSION}.")
|
||||
print("Deployment may take up to 30 minutes.")
|
||||
```
|
||||
|
||||
In this code, replace `your-workspace-id`, `your-project-id`, the `VERSION` number, and the `MODEL_PATH` with the values specific to your Roboflow account, project, and local training results directory. Ensure the `MODEL_PATH` correctly points to the directory containing your trained `best.pt` weights file.
|
||||
|
||||
When you run the code above, you will be asked to authenticate (usually via an API key). Then, your model will be uploaded, and an API endpoint will be created for your project. This process can take up to 30 minutes to complete.
|
||||
|
||||
To test your model and find deployment instructions for supported SDKs, go to the "Deploy" tab in the Roboflow sidebar. At the top of this page, a widget will appear allowing you to test your model using your webcam or by uploading images or videos.
|
||||
|
||||
<p align="center">
|
||||
<img src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/running-inference-example-image.avif" alt="Roboflow deployment widget for model inference" width="800">
|
||||
</p>
|
||||
|
||||
Your uploaded model can also be used as a labeling assistant, suggesting annotations on new images based on its training.
|
||||
|
||||
## How to Evaluate YOLO26 Models
|
||||
|
||||
Roboflow provides features for evaluating model performance. Understanding [performance metrics](../guides/yolo-performance-metrics.md) is crucial for model iteration.
|
||||
|
||||
After uploading a model, access the model evaluation tool via your model page on the Roboflow dashboard. Click "View Detailed Evaluation."
|
||||
|
||||
<p align="center">
|
||||
<img src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/roboflow-model-evaluation.avif" alt="Initiating a Roboflow model evaluation" width="800">
|
||||
</p>
|
||||
|
||||
This tool displays a [confusion matrix](https://www.ultralytics.com/glossary/confusion-matrix) illustrating model performance and an interactive vector analysis plot using [CLIP](https://openai.com/research/clip) embeddings. These features help identify areas for model improvement.
|
||||
|
||||
The confusion matrix pop-up:
|
||||
|
||||
<p align="center">
|
||||
<img src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/confusion-matrix.avif" alt="A confusion matrix displayed in Roboflow" width="800">
|
||||
</p>
|
||||
|
||||
Hover over cells to see values, and click cells to view corresponding images with model predictions and ground truth data.
|
||||
|
||||
Click "Vector Analysis" for a scatter plot visualizing image similarity based on CLIP embeddings. Images closer together are semantically similar. Dots represent images, colored from white (good performance) to red (poor performance).
|
||||
|
||||
<p align="center">
|
||||
<img src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/vector-analysis-plot.avif" alt="Roboflow vector analysis plot using CLIP embeddings" width="800">
|
||||
</p>
|
||||
|
||||
Vector Analysis helps:
|
||||
|
||||
- Identify image clusters.
|
||||
- Pinpoint clusters where the model performs poorly.
|
||||
- Understand commonalities among images causing poor performance.
|
||||
|
||||
## Learning Resources
|
||||
|
||||
- **[Train YOLO on a Custom Dataset (Colab)](https://colab.research.google.com/github/ultralytics/ultralytics/blob/main/examples/tutorial.ipynb)**: Interactive [Google Colab](../integrations/google-colab.md) notebook for training on your data.
|
||||
- **[Ultralytics YOLO Documentation](../models/index.md)**: Training, exporting, and deploying YOLO models.
|
||||
- **[Ultralytics Blog](https://www.ultralytics.com/blog)**: Articles on computer vision and model training.
|
||||
- **[Ultralytics YouTube](https://www.youtube.com/@Ultralytics)**: Video guides on model training and deployment.
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I label data for YOLO26 models using Roboflow?
|
||||
|
||||
Use Roboflow Annotate. Create a project, upload images, and use the annotation tools (`B` for [bounding boxes](https://www.ultralytics.com/glossary/bounding-box), `P` for polygons) or the SAM-based label assistant for faster labeling. Detailed steps are available in the [Upload, Convert and Label Data section](#upload-convert-and-label-data-for-yolo26-format).
|
||||
|
||||
### What services does Roboflow offer for collecting YOLO26 training data?
|
||||
|
||||
Roboflow provides Universe (access to numerous [datasets](../datasets/index.md)) and Collect (automated image gathering via webcam). These can help acquire the necessary [training data](https://www.ultralytics.com/glossary/training-data) for your YOLO26 model, complementing strategies outlined in our [Data Collection Guide](../guides/data-collection-and-annotation.md).
|
||||
|
||||
### How can I manage and analyze my YOLO26 dataset using Roboflow?
|
||||
|
||||
Utilize Roboflow's dataset search, tagging, and Health Check features. Search finds images by text or tags, while Health Check analyzes dataset quality (class balance, image sizes, etc.) to guide improvements before training. See the [Dataset Management section](#dataset-management-for-yolo26) for details.
|
||||
|
||||
### How do I export my YOLO26 dataset from Roboflow?
|
||||
|
||||
Create a dataset version in Roboflow, apply desired preprocessing and [augmentations](https://www.ultralytics.com/glossary/data-augmentation), then click "Export Dataset" and select the YOLO26 format. The process is outlined in the [Export Data section](#export-data-in-40-formats-for-model-training). This prepares your data for use with Ultralytics [training pipelines](../modes/train.md).
|
||||
|
||||
### How can I integrate and deploy YOLO26 models with Roboflow?
|
||||
|
||||
Upload your trained YOLO26 weights to Roboflow using the provided Python script. This creates a deployable API endpoint. Refer to the [Upload Custom Weights section](#upload-custom-yolo26-model-weights-for-testing-and-deployment) for the script and instructions. Explore further [deployment options](../guides/model-deployment-options.md) in our documentation.
|
||||
@@ -0,0 +1,237 @@
|
||||
---
|
||||
comments: true
|
||||
description: Learn how to export YOLO26 models to RKNN format for efficient deployment on Rockchip platforms with enhanced performance.
|
||||
keywords: YOLO26, RKNN, model export, Ultralytics, Rockchip, machine learning, model deployment, computer vision, deep learning, edge AI, NPU, embedded devices
|
||||
---
|
||||
|
||||
# Rockchip RKNN Export for Ultralytics YOLO26 Models
|
||||
|
||||
When deploying computer vision models on embedded devices, especially those powered by Rockchip processors, having a compatible model format is essential. Exporting [Ultralytics YOLO26](https://github.com/ultralytics/ultralytics) models to RKNN format ensures optimized performance and compatibility with Rockchip's hardware. This guide will walk you through converting your YOLO26 models to RKNN format, enabling efficient deployment on Rockchip platforms.
|
||||
|
||||
<p align="center">
|
||||
<img width="50%" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/rockchip-rknn-overview.avif" alt="Rockchip RKNN export for NPU deployment">
|
||||
</p>
|
||||
|
||||
!!! note
|
||||
|
||||
This guide has been tested with [Radxa Rock 5B](https://radxa.com/products/rock5/5b/) which is based on Rockchip RK3588 and [Radxa Zero 3W](https://radxa.com/products/zeros/zero3w/) which is based on Rockchip RK3566. It is expected to work across other Rockchip-based devices that support [rknn-toolkit2](https://github.com/airockchip/rknn-toolkit2) such as RK3576, RK3568, RK3562, RV1103, RV1106, RV1103B, RV1106B, RK2118 and RV1126B.
|
||||
|
||||
## What is Rockchip?
|
||||
|
||||
Renowned for delivering versatile and power-efficient solutions, Rockchip designs advanced System-on-Chips (SoCs) that power a wide range of consumer electronics, industrial applications, and AI technologies. With ARM-based architecture, built-in Neural Processing Units (NPUs), and high-resolution multimedia support, Rockchip SoCs enable cutting-edge performance for devices like tablets, smart TVs, IoT systems, and [edge AI applications](https://www.ultralytics.com/blog/understanding-the-real-world-applications-of-edge-ai). Companies like Radxa, ASUS, Pine64, Orange Pi, Odroid, Khadas, and Banana Pi offer a variety of products based on Rockchip SoCs, further extending their reach and impact across diverse markets.
|
||||
|
||||
## RKNN Toolkit
|
||||
|
||||
The [RKNN Toolkit](https://github.com/airockchip/rknn-toolkit2) is a set of tools and libraries provided by Rockchip to facilitate the deployment of deep learning models on their hardware platforms. RKNN, or Rockchip Neural Network, is the proprietary format used by these tools. RKNN models are designed to take full advantage of the hardware acceleration provided by Rockchip's NPU (Neural Processing Unit), ensuring high performance in AI tasks on devices like RK3588, RK3566, RV1103, RV1106, and other Rockchip-powered systems.
|
||||
|
||||
## Key Features of RKNN Models
|
||||
|
||||
RKNN models offer several advantages for deployment on Rockchip platforms:
|
||||
|
||||
- **Optimized for NPU**: RKNN models are specifically optimized to run on Rockchip's NPUs, ensuring maximum performance and efficiency.
|
||||
- **Low Latency**: The RKNN format minimizes inference latency, which is critical for real-time applications on edge devices.
|
||||
- **Platform-Specific Customization**: RKNN models can be tailored to specific Rockchip platforms, enabling better utilization of hardware resources.
|
||||
- **Power Efficiency**: By leveraging dedicated NPU hardware, RKNN models consume less power than CPU or GPU-based processing, extending battery life for portable devices.
|
||||
|
||||
## Flash OS to Rockchip hardware
|
||||
|
||||
The first step after getting your hands on a Rockchip-based device is to flash an OS so that the hardware can boot into a working environment. In this guide we will point to getting started guides of the two devices that we tested which are Radxa Rock 5B and Radxa Zero 3W.
|
||||
|
||||
- [Radxa Rock 5B Getting Started Guide](https://docs.radxa.com/en/rock5/rock5b)
|
||||
- [Radxa Zero 3W Getting Started Guide](https://docs.radxa.com/en/zero/zero3)
|
||||
|
||||
## Export to RKNN: Converting Your YOLO26 Model
|
||||
|
||||
Export an Ultralytics YOLO26 model to RKNN format and run inference with the exported model.
|
||||
|
||||
!!! note
|
||||
|
||||
Make sure to use an X86-based Linux PC to export the model to RKNN because exporting on Rockchip-based devices (ARM64) is not supported.
|
||||
|
||||
### Installation
|
||||
|
||||
To install the required packages, run:
|
||||
|
||||
!!! Tip "Installation"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Install the required package for YOLO26
|
||||
pip install ultralytics
|
||||
```
|
||||
|
||||
For detailed instructions and best practices related to the installation process, check our [Ultralytics Installation guide](../quickstart.md). While installing the required packages for YOLO26, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips.
|
||||
|
||||
### Usage
|
||||
|
||||
!!! note
|
||||
|
||||
Export is currently only supported for detection models. More model support will be coming in the future.
|
||||
|
||||
!!! example "Usage"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load the YOLO26 model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Export the model to RKNN format
|
||||
# 'name' can be one of rk3588, rk3576, rk3566, rk3568, rk3562, rv1103, rv1106, rv1103b, rv1106b, rk2118, rv1126b
|
||||
model.export(format="rknn", name="rk3588") # creates '/yolo26n_rknn_model'
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Export a YOLO26n PyTorch model to RKNN format
|
||||
# 'name' can be one of rk3588, rk3576, rk3566, rk3568, rk3562, rv1103, rv1106, rv1103b, rv1106b, rk2118, rv1126b
|
||||
yolo export model=yolo26n.pt format=rknn name=rk3588 # creates '/yolo26n_rknn_model'
|
||||
```
|
||||
|
||||
### Export Arguments
|
||||
|
||||
| Argument | Type | Default | Description |
|
||||
| -------- | ---------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `format` | `str` | `'rknn'` | Target format for the exported model, defining compatibility with various deployment environments. |
|
||||
| `imgsz` | `int` or `tuple` | `640` | Desired image size for the model input. Can be an integer for square images or a tuple `(height, width)` for specific dimensions. |
|
||||
| `batch` | `int` | `1` | Specifies export model batch inference size or the max number of images the exported model will process concurrently in `predict` mode. |
|
||||
| `name` | `str` | `'rk3588'` | Specifies the Rockchip model (rk3588, rk3576, rk3566, rk3568, rk3562, rv1103, rv1106, rv1103b, rv1106b, rk2118, rv1126b) |
|
||||
| `device` | `str` | `None` | Specifies the device for exporting: GPU (`device=0`), CPU (`device=cpu`). |
|
||||
|
||||
!!! tip
|
||||
|
||||
Please make sure to use an x86 Linux machine when exporting to RKNN.
|
||||
|
||||
For more details about the export process, visit the [Ultralytics documentation page on exporting](../modes/export.md).
|
||||
|
||||
## Deploying Exported YOLO26 RKNN Models
|
||||
|
||||
Once you've successfully exported your Ultralytics YOLO26 models to RKNN format, the next step is deploying these models on Rockchip-based devices.
|
||||
|
||||
### Installation
|
||||
|
||||
To install the required packages, run:
|
||||
|
||||
!!! Tip "Installation"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Install the required package for YOLO26
|
||||
pip install ultralytics
|
||||
```
|
||||
|
||||
### Usage
|
||||
|
||||
!!! example "Usage"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load the exported RKNN model
|
||||
rknn_model = YOLO("./yolo26n_rknn_model")
|
||||
|
||||
# Run inference
|
||||
results = rknn_model("https://ultralytics.com/images/bus.jpg")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Run inference with the exported model
|
||||
yolo predict model='./yolo26n_rknn_model' source='https://ultralytics.com/images/bus.jpg'
|
||||
```
|
||||
|
||||
!!! note
|
||||
|
||||
If you encounter a log message indicating that the RKNN runtime version does not match the RKNN Toolkit version and the inference fails, please replace `/usr/lib/librknnrt.so` with official [librknnrt.so file](https://github.com/airockchip/rknn-toolkit2/blob/master/rknpu2/runtime/Linux/librknn_api/aarch64/librknnrt.so).
|
||||
|
||||
|
||||
|
||||
## Real-World Applications
|
||||
|
||||
Rockchip-powered devices with YOLO26 RKNN models can be used in various applications:
|
||||
|
||||
- **Smart Surveillance**: Deploy efficient object detection systems for security monitoring with low power consumption.
|
||||
- **Industrial Automation**: Implement quality control and defect detection directly on embedded devices.
|
||||
- **Retail Analytics**: Track customer behavior and inventory management in real-time without cloud dependency.
|
||||
- **Smart Agriculture**: Monitor crop health and detect pests using [computer vision in agriculture](https://www.ultralytics.com/solutions/ai-in-agriculture).
|
||||
- **Autonomous Robotics**: Enable vision-based navigation and obstacle detection on resource-constrained platforms.
|
||||
|
||||
## Benchmarks
|
||||
|
||||
YOLO26 benchmarks below were run by the Ultralytics team on Radxa Rock 5B based on Rockchip RK3588 with `rknn` model format measuring speed and accuracy.
|
||||
|
||||
!!! tip "Performance"
|
||||
|
||||
| Model | Format | Status | Size (MB) | mAP50-95(B) | Inference time (ms/im) |
|
||||
| ------- | ------ | ------ | --------- | ----------- | ---------------------- |
|
||||
| YOLO11n | `rknn` | ✅ | 7.4 | 0.505 | 71.5 |
|
||||
| YOLO11s | `rknn` | ✅ | 20.7 | 0.578 | 98.9 |
|
||||
| YOLO11m | `rknn` | ✅ | 41.9 | 0.629 | 235.3 |
|
||||
| YOLO11l | `rknn` | ✅ | 53.3 | 0.633 | 282.0 |
|
||||
| YOLO11x | `rknn` | ✅ | 114.6 | 0.687 | 679.2 |
|
||||
|
||||
Benchmarked with `ultralytics 8.3.152`
|
||||
|
||||
!!! note
|
||||
|
||||
Validation for the above benchmarks were done using COCO128 dataset. Inference time does not include pre/ post-processing.
|
||||
|
||||
## Summary
|
||||
|
||||
In this guide, you've learned how to export Ultralytics YOLO26 models to RKNN format to enhance their deployment on Rockchip platforms. You were also introduced to the RKNN Toolkit and the specific advantages of using RKNN models for edge AI applications.
|
||||
|
||||
The combination of [Ultralytics YOLO26](https://www.ultralytics.com/blog/all-you-need-to-know-about-ultralytics-yolo11-and-its-applications) and Rockchip's NPU technology provides an efficient solution for running advanced computer vision tasks on embedded devices. This approach enables real-time [object detection](https://www.ultralytics.com/blog/a-guide-to-deep-dive-into-object-detection-in-2025) and other vision AI applications with minimal power consumption and high performance.
|
||||
|
||||
For further details on usage, visit the [RKNN official documentation](https://github.com/airockchip/rknn-toolkit2).
|
||||
|
||||
Also, if you'd like to know more about other Ultralytics YOLO26 integrations, visit our [integration guide page](../integrations/index.md). You'll find plenty of useful resources and insights there.
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I export my Ultralytics YOLO model to RKNN format?
|
||||
|
||||
You can easily export your Ultralytics YOLO model to RKNN format using the `export()` method in the Ultralytics Python package or via the command-line interface (CLI). Ensure you are using an x86-based Linux PC for the export process, as ARM64 devices like Rockchip are not supported for this operation. You can specify the target Rockchip platform using the `name` argument, such as `rk3588`, `rk3566`, or others. This process generates an optimized RKNN model ready for deployment on your Rockchip device, taking advantage of its Neural Processing Unit (NPU) for accelerated inference.
|
||||
|
||||
!!! example
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load your YOLO model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Export to RKNN format for a specific Rockchip platform
|
||||
model.export(format="rknn", name="rk3588")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
yolo export model=yolo26n.pt format=rknn name=rk3588
|
||||
```
|
||||
|
||||
### What are the benefits of using RKNN models on Rockchip devices?
|
||||
|
||||
RKNN models are specifically designed to leverage the hardware acceleration capabilities of Rockchip's Neural Processing Units (NPUs). This optimization results in significantly faster inference speeds and reduced latency compared to running generic model formats like ONNX or TensorFlow Lite on the same hardware. Using RKNN models allows for more efficient use of the device's resources, leading to lower power consumption and better overall performance, especially critical for real-time applications on edge devices. By converting your Ultralytics YOLO models to RKNN, you can achieve optimal performance on devices powered by Rockchip SoCs like the RK3588, RK3566, and others.
|
||||
|
||||
### Can I deploy RKNN models on devices from other manufacturers like NVIDIA or Google?
|
||||
|
||||
RKNN models are specifically optimized for Rockchip platforms and their integrated NPUs. While you can technically run an RKNN model on other platforms using software emulation, you will not benefit from the hardware acceleration provided by Rockchip devices. For optimal performance on other platforms, it's recommended to export your Ultralytics YOLO models to formats specifically designed for those platforms, such as TensorRT for NVIDIA GPUs or [TensorFlow Lite](https://docs.ultralytics.com/integrations/tflite/) for Google's Edge TPU. Ultralytics supports exporting to a wide range of formats, ensuring compatibility with various hardware accelerators.
|
||||
|
||||
### What Rockchip platforms are supported for RKNN model deployment?
|
||||
|
||||
The Ultralytics YOLO export to RKNN format supports a wide range of Rockchip platforms, including the popular RK3588, RK3576, RK3566, RK3568, RK3562, RV1103, RV1106, RV1103B, RV1106B, RK2118 and RV1126B. These platforms are commonly found in devices from manufacturers like Radxa, ASUS, Pine64, Orange Pi, Odroid, Khadas, and Banana Pi. This broad support ensures that you can deploy your optimized RKNN models on various Rockchip-powered devices, from single-board computers to industrial systems, taking full advantage of their AI acceleration capabilities for enhanced performance in your computer vision applications.
|
||||
|
||||
### How does the performance of RKNN models compare to other formats on Rockchip devices?
|
||||
|
||||
RKNN models generally outperform other formats like ONNX or TensorFlow Lite on Rockchip devices due to their optimization for Rockchip's NPUs. For instance, benchmarks on the Radxa Rock 5B (RK3588) show that [YOLO26n](https://www.ultralytics.com/blog/all-you-need-to-know-about-ultralytics-yolo11-and-its-applications) in RKNN format achieves an inference time of 99.5 ms/image, significantly faster than other formats. This performance advantage is consistent across various YOLO26 model sizes, as demonstrated in the [benchmarks section](#benchmarks). By leveraging the dedicated NPU hardware, RKNN models minimize latency and maximize throughput, making them ideal for real-time applications on Rockchip-based edge devices.
|
||||
@@ -0,0 +1,179 @@
|
||||
---
|
||||
comments: true
|
||||
description: Discover how to get started with Seeed Studio reCamera for edge AI applications using Ultralytics YOLO26. Learn about its powerful features, real-world applications, and how to export YOLO26 models to ONNX format for seamless integration.
|
||||
keywords: Seeed Studio reCamera, YOLO26, ONNX export, edge AI, computer vision, real-time detection, personal protective equipment detection, fire detection, waste detection, fall detection, modular AI devices, Ultralytics
|
||||
---
|
||||
|
||||
# Quick Start Guide: Seeed Studio reCamera with Ultralytics YOLO26
|
||||
|
||||
[reCamera](https://www.seeedstudio.com/recamera) was introduced for the AI community at [YOLO Vision 2024 (YV24)](https://www.youtube.com/watch?v=rfI5vOo3-_A), [Ultralytics](https://www.ultralytics.com/) annual hybrid event. It is mainly designed for [edge AI applications](https://www.ultralytics.com/blog/understanding-the-real-world-applications-of-edge-ai), offering powerful processing capabilities and effortless deployment.
|
||||
|
||||
With support for diverse hardware configurations and open-source resources, it serves as an ideal platform for prototyping and deploying innovative [computer vision](https://www.ultralytics.com/glossary/computer-vision-cv) [solutions](https://docs.ultralytics.com/solutions/#solutions) at the edge.
|
||||
|
||||
|
||||
|
||||
## Why Choose reCamera?
|
||||
|
||||
reCamera series is purpose-built for edge AI applications, tailored to meet the needs of developers and innovators. Here's why it stands out:
|
||||
|
||||
- **RISC-V Powered Performance**: At its core is the SG200X processor, built on the RISC-V architecture, delivering exceptional performance for edge AI tasks while maintaining energy efficiency. With the ability to execute 1 trillion operations per second (1 TOPS), it handles demanding tasks like real-time [object detection](https://docs.ultralytics.com/tasks/detect/) easily.
|
||||
|
||||
- **Optimized Video Technologies**: Supports advanced video compression standards, including H.264 and H.265, to reduce storage and bandwidth requirements without sacrificing quality. Features like HDR imaging, 3D noise reduction, and lens correction ensure professional visuals, even in challenging environments.
|
||||
|
||||
- **Energy-Efficient Dual Processing**: While the SG200X handles complex AI tasks, a smaller 8-bit microcontroller manages simpler operations to conserve power, making the reCamera ideal for battery-operated or low-power setups.
|
||||
|
||||
- **Modular and Upgradable Design**: The reCamera is built with a modular structure, consisting of three main components: the core board, sensor board, and baseboard. This design allows developers to easily swap or upgrade components, ensuring flexibility and future-proofing for evolving projects.
|
||||
|
||||
## Quick Hardware Setup of reCamera
|
||||
|
||||
Please follow [reCamera Quick Start Guide](https://wiki.seeedstudio.com/recamera_getting_started/) for initial onboarding of the device such as connecting the device to a WiFi network and access the [Node-RED](https://nodered.org/) web UI for quick previewing of detection results.
|
||||
|
||||
## Inference Using Pre-installed YOLO26 Models
|
||||
|
||||
reCamera comes pre-installed with four Ultralytics YOLO26 models and you can simply choose your desired model within the Node-RED dashboard.
|
||||
|
||||
- [Detection (YOLO26n)](../tasks/detect.md)
|
||||
- [Classification (YOLO26n-cls)](../tasks/classify.md)
|
||||
- [Segmentation (YOLO26n-seg)](../tasks/segment.md)
|
||||
- [Pose Estimation (YOLO26n-pose)](../tasks/pose.md)
|
||||
|
||||
Step 1: If you have connected reCamera to a network, enter the IP address of reCamera on a web browser to open the Node-RED dashboard. If you have connected the reCamera to a PC via USB, you can enter `192.168.42.1`. Here you will see YOLO26n detection model is loaded by default.
|
||||
|
||||
|
||||
|
||||
Step 2: Click the green color circle at the bottom right corner to access the Node-RED flow editor.
|
||||
|
||||
Step 3: Click the `model` node and click `On Device`.
|
||||
|
||||
|
||||
|
||||
Step 4: Choose one of the four different pre-installed YOLO26n models and click `Done`. For example, here we will select `YOLO26n Pose`
|
||||
|
||||
<p align="center">
|
||||
<img width="50%" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/recamera-nodered-yolo11n-pose.avif" alt="Node-RED YOLO11n-pose select">
|
||||
</p>
|
||||
|
||||
Step 5: Click `Deploy` and when it finishes deploying, click `Dashboard`.
|
||||
|
||||
|
||||
|
||||
Now you will be able to see YOLO26n pose estimation model in action!
|
||||
|
||||
|
||||
|
||||
## Export to cvimodel: Converting Your YOLO26 Model
|
||||
|
||||
If you want to use a [custom-trained YOLO26 model](../modes/train.md) with reCamera, follow the steps below.
|
||||
|
||||
Here, we'll first convert a `PyTorch` model to `ONNX` and then convert it to the `MLIR` model format. Finally, `MLIR` will be converted to `cvimodel` to run inference on-device.
|
||||
|
||||
<p align="center">
|
||||
<img width="80%" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/recamera-toolchain-workflow.avif" alt="Seeed Studio reCamera AI development toolchain">
|
||||
</p>
|
||||
|
||||
### Export to ONNX
|
||||
|
||||
Export an Ultralytics YOLO26 model to [ONNX model format](https://docs.ultralytics.com/integrations/onnx/).
|
||||
|
||||
#### Installation
|
||||
|
||||
To install the required packages, run:
|
||||
|
||||
!!! Tip "Installation"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
pip install ultralytics
|
||||
```
|
||||
|
||||
For detailed instructions and best practices related to the installation process, check our [Ultralytics Installation guide](../quickstart.md). While installing the required packages for YOLO26, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips.
|
||||
|
||||
#### Usage
|
||||
|
||||
!!! example "Usage"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load the YOLO26 model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Export the model to ONNX format
|
||||
model.export(format="onnx", opset=14) # creates 'yolo26n.onnx'
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Export a YOLO26n PyTorch model to ONNX format
|
||||
yolo export model=yolo26n.pt format=onnx opset=14 # creates 'yolo26n.onnx'
|
||||
```
|
||||
|
||||
For more details about the export process, visit the [Ultralytics documentation page on exporting](../modes/export.md).
|
||||
|
||||
### Export ONNX to MLIR and cvimodel
|
||||
|
||||
After obtaining an ONNX model, refer to [Convert and Quantize AI Models](https://wiki.seeedstudio.com/recamera_model_conversion/) page to convert the ONNX model to MLIR and then to cvimodel.
|
||||
|
||||
!!! note
|
||||
|
||||
We're actively working on adding reCamera support directly into the Ultralytics package, and it will be available soon. In the meantime, check out our blog on [Integrating Ultralytics YOLO Models with Seeed Studio's reCamera](https://www.ultralytics.com/blog/integrating-ultralytics-yolo-models-on-seeed-studios-recamera) for more insights.
|
||||
|
||||
## Benchmarks
|
||||
|
||||
Coming soon.
|
||||
|
||||
## Real-World Applications of reCamera
|
||||
|
||||
reCamera advanced computer vision capabilities and modular design make it suitable for a wide range of real-world scenarios, helping developers and businesses tackle unique challenges with ease.
|
||||
|
||||
- **Fall Detection**: Designed for safety and healthcare applications, the reCamera can detect falls in real-time, making it ideal for elderly care, hospitals, and industrial settings where rapid response is critical.
|
||||
|
||||
- **Personal Protective Equipment Detection**: The reCamera can be used to ensure workplace safety by detecting PPE compliance in real-time. It helps identify whether workers are wearing helmets, gloves, or other safety gear, reducing risks in industrial environments.
|
||||
|
||||
|
||||
|
||||
- **Fire Detection**: The reCamera's real-time processing capabilities make it an excellent choice for [fire detection](https://www.ultralytics.com/blog/computer-vision-in-fire-detection-and-prevention) in industrial and residential areas, providing early warnings to prevent potential disasters.
|
||||
|
||||
- **Waste Detection**: It can also be utilized for waste detection applications, making it an excellent tool for environmental monitoring and [waste management](https://www.ultralytics.com/blog/simplifying-e-waste-management-with-ai-innovations).
|
||||
|
||||
- **Car Parts Detection**: In manufacturing and automotive industries, it aids in detecting and analyzing car parts for quality control, assembly line monitoring, and inventory management.
|
||||
|
||||
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I install and set up reCamera for the first time?
|
||||
|
||||
To set up your reCamera for the first time, follow these steps:
|
||||
|
||||
1. Connect the reCamera to a power source
|
||||
2. Connect it to your WiFi network using the [reCamera Quick Start Guide](https://wiki.seeedstudio.com/recamera_getting_started/)
|
||||
3. Access the Node-RED web UI by entering the device's IP address in a web browser (or use `192.168.42.1` if connected via USB)
|
||||
4. Start using the pre-installed YOLO26 models immediately through the dashboard interface
|
||||
|
||||
### Can I use my custom-trained YOLO26 models with reCamera?
|
||||
|
||||
Yes, you can use custom-trained YOLO26 models with reCamera. The process involves:
|
||||
|
||||
1. Export your PyTorch model to ONNX format using `model.export(format="onnx", opset=14)`
|
||||
2. Convert the ONNX model to MLIR format
|
||||
3. Convert the MLIR to cvimodel format for on-device inference
|
||||
4. Load the converted model onto your reCamera
|
||||
|
||||
For detailed instructions, refer to the [Convert and Quantize AI Models](https://wiki.seeedstudio.com/recamera_model_conversion/) guide.
|
||||
|
||||
### What makes reCamera different from traditional IP cameras?
|
||||
|
||||
Unlike traditional IP cameras that require external hardware for processing, reCamera:
|
||||
|
||||
- Integrates AI processing directly on the device with its RISC-V SG200X processor
|
||||
- Offers 1 TOPS of computing power for real-time edge AI applications
|
||||
- Features a modular design allowing for component upgrades and customization
|
||||
- Supports advanced video technologies like H.264/H.265 compression, HDR imaging, and 3D noise reduction
|
||||
- Comes pre-installed with Ultralytics YOLO26 models for immediate use
|
||||
|
||||
These features make reCamera a standalone solution for edge AI applications without requiring additional external processing hardware.
|
||||
@@ -0,0 +1,617 @@
|
||||
---
|
||||
comments: true
|
||||
description: Learn to export Ultralytics YOLO11 models to Sony's IMX500 format for efficient edge AI deployment on Raspberry Pi AI Camera with on-chip processing.
|
||||
keywords: Sony, IMX500, IMX 500, Atrios, MCT, model export, quantization, pruning, deep learning optimization, Raspberry Pi AI Camera, edge AI, PyTorch, IMX
|
||||
---
|
||||
|
||||
# Sony IMX500 Export for Ultralytics YOLO11
|
||||
|
||||
This guide covers exporting and deploying Ultralytics YOLO11 models to Raspberry Pi AI Cameras that feature the Sony IMX500 sensor.
|
||||
|
||||
Deploying computer vision models on devices with limited computational power, such as [Raspberry Pi AI Camera](https://www.raspberrypi.com/products/ai-camera/), can be tricky. Using a model format optimized for faster performance makes a huge difference.
|
||||
|
||||
The IMX500 model format is designed to use minimal power while delivering fast performance for neural networks. It allows you to optimize your [Ultralytics YOLO11](https://github.com/ultralytics/ultralytics) models for high-speed and low-power inferencing. In this guide, we'll walk you through exporting and deploying your models to the IMX500 format while making it easier for your models to perform well on the [Raspberry Pi AI Camera](https://www.raspberrypi.com/products/ai-camera/).
|
||||
|
||||
<p align="center">
|
||||
<img width="100%" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/imx500-ai-camera.avif" alt="Raspberry Pi AI Camera with Sony IMX500 sensor">
|
||||
</p>
|
||||
|
||||
## Why Should You Export to IMX500?
|
||||
|
||||
Sony's [IMX500 Intelligent Vision Sensor](https://www.aitrios.sony-semicon.com/edge-ai-devices/raspberry-pi-ai-camera) is a game-changing piece of hardware in edge AI processing. It's the world's first intelligent vision sensor with on-chip AI capabilities. This sensor helps overcome many challenges in edge AI, including data processing bottlenecks, privacy concerns, and performance limitations.
|
||||
While other sensors merely pass along images and frames, the IMX500 tells a whole story. It processes data directly on the sensor, allowing devices to generate insights in real-time.
|
||||
|
||||
## Sony's IMX500 Export for YOLO11 Models
|
||||
|
||||
The IMX500 is designed to transform how devices handle data directly on the sensor, without needing to send it off to the cloud for processing.
|
||||
|
||||
The IMX500 works with quantized models. Quantization makes models smaller and faster without losing much [accuracy](https://www.ultralytics.com/glossary/accuracy). It is ideal for the limited resources of edge computing, allowing applications to respond quickly by reducing latency and allowing for quick data processing locally, without cloud dependency. Local processing also keeps user data private and secure since it's not sent to a remote server.
|
||||
|
||||
**IMX500 Key Features:**
|
||||
|
||||
- **Metadata Output:** Instead of transmitting images only, the IMX500 can output both image and metadata (inference result), and can output metadata only for minimizing data size, reducing bandwidth, and lowering costs.
|
||||
- **Addresses Privacy Concerns:** By processing data on the device, the IMX500 addresses privacy concerns, ideal for human-centric applications like person counting and occupancy tracking.
|
||||
- **Real-time Processing:** Fast, on-sensor processing supports real-time decisions, perfect for edge AI applications such as autonomous systems.
|
||||
|
||||
**Before You Begin:** For best results, ensure your YOLO11 model is well-prepared for export by following our [Model Training Guide](https://docs.ultralytics.com/modes/train/), [Data Preparation Guide](https://docs.ultralytics.com/datasets/), and [Hyperparameter Tuning Guide](https://docs.ultralytics.com/guides/hyperparameter-tuning/).
|
||||
|
||||
## Supported Tasks
|
||||
|
||||
Currently, you can only export models that include the following tasks to IMX500 format.
|
||||
|
||||
- [Object Detection](https://docs.ultralytics.com/tasks/detect/)
|
||||
- [Pose Estimation](https://docs.ultralytics.com/tasks/pose/)
|
||||
- [Classification](https://docs.ultralytics.com/tasks/classify/)
|
||||
- [Instance segmentation](https://docs.ultralytics.com/tasks/segment/)
|
||||
|
||||
## Usage Examples
|
||||
|
||||
Export an Ultralytics YOLO11 model to IMX500 format and run inference with the exported model.
|
||||
|
||||
!!! note
|
||||
|
||||
Here we perform inference just to make sure the model works as expected. However, for deployment and inference on the Raspberry Pi AI Camera, please jump to [Using IMX500 Export in Deployment](#using-imx500-export-in-deployment) section.
|
||||
|
||||
!!! example "Object Detection"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load a YOLO11n PyTorch model
|
||||
model = YOLO("yolo11n.pt")
|
||||
|
||||
# Export the model
|
||||
model.export(format="imx", data="coco8.yaml") # exports with PTQ quantization by default
|
||||
|
||||
# Load the exported model
|
||||
imx_model = YOLO("yolo11n_imx_model")
|
||||
|
||||
# Run inference
|
||||
results = imx_model("https://ultralytics.com/images/bus.jpg")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Export a YOLO11n PyTorch model to imx format with Post-Training Quantization (PTQ)
|
||||
yolo export model=yolo11n.pt format=imx data=coco8.yaml
|
||||
|
||||
# Run inference with the exported model
|
||||
yolo predict model=yolo11n_imx_model source='https://ultralytics.com/images/bus.jpg'
|
||||
```
|
||||
|
||||
!!! example "Pose Estimation"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load a YOLO11n-pose PyTorch model
|
||||
model = YOLO("yolo11n-pose.pt")
|
||||
|
||||
# Export the model
|
||||
model.export(format="imx", data="coco8-pose.yaml") # exports with PTQ quantization by default
|
||||
|
||||
# Load the exported model
|
||||
imx_model = YOLO("yolo11n-pose_imx_model")
|
||||
|
||||
# Run inference
|
||||
results = imx_model("https://ultralytics.com/images/bus.jpg")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Export a YOLO11n-pose PyTorch model to imx format with Post-Training Quantization (PTQ)
|
||||
yolo export model=yolo11n-pose.pt format=imx data=coco8-pose.yaml
|
||||
|
||||
# Run inference with the exported model
|
||||
yolo predict model=yolo11n-pose_imx_model source='https://ultralytics.com/images/bus.jpg'
|
||||
```
|
||||
|
||||
!!! example "Classification"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load a YOLO11n-cls PyTorch model
|
||||
model = YOLO("yolo11n-cls.pt")
|
||||
|
||||
# Export the model
|
||||
model.export(format="imx", data="imagenet10") # exports with PTQ quantization by default
|
||||
|
||||
# Load the exported model
|
||||
imx_model = YOLO("yolo11n-cls_imx_model")
|
||||
|
||||
# Run inference
|
||||
results = imx_model("https://ultralytics.com/images/bus.jpg", imgsz=224)
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Export a YOLO11n-cls PyTorch model to imx format with Post-Training Quantization (PTQ)
|
||||
yolo export model=yolo11n-cls.pt format=imx data=imagenet10
|
||||
|
||||
# Run inference with the exported model
|
||||
yolo predict model=yolo11n-cls_imx_model source='https://ultralytics.com/images/bus.jpg' imgsz=224
|
||||
```
|
||||
|
||||
!!! example "Instance Segmentation"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load a YOLO11n-seg PyTorch model
|
||||
model = YOLO("yolo11n-seg.pt")
|
||||
|
||||
# Export the model
|
||||
model.export(format="imx", data="coco8-seg.yaml") # exports with PTQ quantization by default
|
||||
|
||||
# Load the exported model
|
||||
imx_model = YOLO("yolo11n-seg_imx_model")
|
||||
|
||||
# Run inference
|
||||
results = imx_model("https://ultralytics.com/images/bus.jpg")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Export a YOLO11n-seg PyTorch model to imx format with Post-Training Quantization (PTQ)
|
||||
yolo export model=yolo11n-seg.pt format=imx data=coco8-seg.yaml
|
||||
|
||||
# Run inference with the exported model
|
||||
yolo predict model=yolo11n-seg_imx_model source='https://ultralytics.com/images/bus.jpg'
|
||||
```
|
||||
|
||||
!!! warning
|
||||
|
||||
The Ultralytics package installs additional export dependencies at runtime. The first time you run the export command, you may need to restart your console to ensure it works correctly.
|
||||
|
||||
## Export Arguments
|
||||
|
||||
| Argument | Type | Default | Description |
|
||||
| ---------- | ---------------- | -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `format` | `str` | `'imx'` | Target format for the exported model, defining compatibility with various deployment environments. |
|
||||
| `imgsz` | `int` or `tuple` | `640` | Desired image size for the model input. Can be an integer for square images or a tuple `(height, width)` for specific dimensions. |
|
||||
| `int8` | `bool` | `True` | Activates INT8 quantization, further compressing the model and speeding up inference with minimal [accuracy](https://www.ultralytics.com/glossary/accuracy) loss, primarily for edge devices. |
|
||||
| `data` | `str` | `'coco8.yaml'` | Path to the [dataset](https://docs.ultralytics.com/datasets/) configuration file (default: `coco8.yaml`), essential for quantization. |
|
||||
| `fraction` | `float` | `1.0` | Specifies the fraction of the dataset to use for INT8 quantization calibration. Allows for calibrating on a subset of the full dataset, useful for experiments or when resources are limited. If not specified with INT8 enabled, the full dataset will be used. |
|
||||
| `device` | `str` | `None` | Specifies the device for exporting: GPU (`device=0`), CPU (`device=cpu`). |
|
||||
|
||||
!!! tip
|
||||
|
||||
If you are exporting on a GPU with CUDA support, please pass the argument `device=0` for faster export.
|
||||
|
||||
For more details about the export process, visit the [Ultralytics documentation page on exporting](../modes/export.md).
|
||||
|
||||
The export process will create an ONNX model for quantization validation, along with a directory named `<model-name>_imx_model`. This directory will include the `packerOut.zip` file, which is essential for packaging the model for deployment on the IMX500 hardware. Additionally, the `<model-name>_imx_model` folder will contain a text file (`labels.txt`) listing all the labels associated with the model.
|
||||
|
||||
!!! example "Folder Structure"
|
||||
|
||||
=== "Object Detection"
|
||||
|
||||
```bash
|
||||
yolo11n_imx_model
|
||||
├── dnnParams.xml
|
||||
├── labels.txt
|
||||
├── packerOut.zip
|
||||
├── yolo11n_imx.onnx
|
||||
├── yolo11n_imx_MemoryReport.json
|
||||
└── yolo11n_imx.pbtxt
|
||||
```
|
||||
|
||||
=== "Pose Estimation"
|
||||
|
||||
```bash
|
||||
yolo11n-pose_imx_model
|
||||
├── dnnParams.xml
|
||||
├── labels.txt
|
||||
├── packerOut.zip
|
||||
├── yolo11n-pose_imx.onnx
|
||||
├── yolo11n-pose_imx_MemoryReport.json
|
||||
└── yolo11n-pose_imx.pbtxt
|
||||
```
|
||||
|
||||
=== "Classification"
|
||||
|
||||
```bash
|
||||
yolo11n-cls_imx_model
|
||||
├── dnnParams.xml
|
||||
├── labels.txt
|
||||
├── packerOut.zip
|
||||
├── yolo11n-cls_imx.onnx
|
||||
├── yolo11n-cls_imx_MemoryReport.json
|
||||
└── yolo11n-cls_imx.pbtxt
|
||||
```
|
||||
|
||||
=== "Instance Segmentation"
|
||||
|
||||
```bash
|
||||
yolo11n-seg_imx_model
|
||||
├── dnnParams.xml
|
||||
├── labels.txt
|
||||
├── packerOut.zip
|
||||
├── yolo11n-seg_imx.onnx
|
||||
├── yolo11n-seg_imx_MemoryReport.json
|
||||
└── yolo11n-seg_imx.pbtxt
|
||||
```
|
||||
|
||||
## Using IMX500 Export in Deployment
|
||||
|
||||
After exporting Ultralytics YOLO11n model to IMX500 format, it can be deployed to Raspberry Pi AI Camera for inference.
|
||||
|
||||
### Hardware Prerequisites
|
||||
|
||||
Make sure you have the below hardware:
|
||||
|
||||
1. Raspberry Pi 5 or Raspberry Pi 4 Model B
|
||||
2. Raspberry Pi AI Camera
|
||||
|
||||
Connect the Raspberry Pi AI camera to the 15-pin MIPI CSI connector on the Raspberry Pi and power on the Raspberry Pi
|
||||
|
||||
### Software Prerequisites
|
||||
|
||||
!!! note
|
||||
|
||||
This guide has been tested with Raspberry Pi OS Bookworm running on a Raspberry Pi 5
|
||||
|
||||
Step 1: Open a terminal window and execute the following commands to update the Raspberry Pi software to the latest version.
|
||||
|
||||
```bash
|
||||
sudo apt update && sudo apt full-upgrade
|
||||
```
|
||||
|
||||
Step 2: Install IMX500 firmware which is required to operate the IMX500 sensor.
|
||||
|
||||
```bash
|
||||
sudo apt install imx500-all
|
||||
```
|
||||
|
||||
Step 3: Reboot Raspberry Pi for the changes to take into effect
|
||||
|
||||
```bash
|
||||
sudo reboot
|
||||
```
|
||||
|
||||
Step 4: Install [Aitrios Raspberry Pi application module library](https://github.com/SonySemiconductorSolutions/aitrios-rpi-application-module-library)
|
||||
|
||||
```bash
|
||||
pip install git+https://github.com/SonySemiconductorSolutions/aitrios-rpi-application-module-library.git
|
||||
```
|
||||
|
||||
Step 5: Run YOLO11 object detection, pose estimation, classification and segmentation by using the below scripts which are available in [aitrios-rpi-application-module-library examples](https://github.com/SonySemiconductorSolutions/aitrios-rpi-application-module-library/tree/main/examples/aicam).
|
||||
|
||||
!!! note
|
||||
|
||||
Make sure to replace `model_file` and `labels.txt` directories according to your environment before running these scripts.
|
||||
|
||||
!!! example "Python Scripts"
|
||||
|
||||
=== "Object Detection"
|
||||
|
||||
```python
|
||||
import numpy as np
|
||||
from modlib.apps import Annotator
|
||||
from modlib.devices import AiCamera
|
||||
from modlib.models import COLOR_FORMAT, MODEL_TYPE, Model
|
||||
from modlib.models.post_processors import pp_od_yolo_ultralytics
|
||||
|
||||
|
||||
class YOLO(Model):
|
||||
"""YOLO model for IMX500 deployment."""
|
||||
|
||||
def __init__(self):
|
||||
"""Initialize the YOLO model for IMX500 deployment."""
|
||||
super().__init__(
|
||||
model_file="yolo11n_imx_model/packerOut.zip", # replace with proper directory
|
||||
model_type=MODEL_TYPE.CONVERTED,
|
||||
color_format=COLOR_FORMAT.RGB,
|
||||
preserve_aspect_ratio=False,
|
||||
)
|
||||
|
||||
self.labels = np.genfromtxt(
|
||||
"yolo11n_imx_model/labels.txt", # replace with proper directory
|
||||
dtype=str,
|
||||
delimiter="\n",
|
||||
)
|
||||
|
||||
def post_process(self, output_tensors):
|
||||
"""Post-process the output tensors for object detection."""
|
||||
return pp_od_yolo_ultralytics(output_tensors)
|
||||
|
||||
|
||||
device = AiCamera(frame_rate=16) # Optimal frame rate for maximum DPS of the YOLO model running on the AI Camera
|
||||
model = YOLO()
|
||||
device.deploy(model)
|
||||
|
||||
annotator = Annotator()
|
||||
|
||||
with device as stream:
|
||||
for frame in stream:
|
||||
detections = frame.detections[frame.detections.confidence > 0.55]
|
||||
labels = [f"{model.labels[class_id]}: {score:0.2f}" for _, score, class_id, _ in detections]
|
||||
|
||||
annotator.annotate_boxes(frame, detections, labels=labels, alpha=0.3, corner_radius=10)
|
||||
frame.display()
|
||||
```
|
||||
|
||||
=== "Pose Estimation"
|
||||
|
||||
```python
|
||||
from modlib.apps import Annotator
|
||||
from modlib.devices import AiCamera
|
||||
from modlib.models import COLOR_FORMAT, MODEL_TYPE, Model
|
||||
from modlib.models.post_processors import pp_yolo_pose_ultralytics
|
||||
|
||||
|
||||
class YOLOPose(Model):
|
||||
"""YOLO pose estimation model for IMX500 deployment."""
|
||||
|
||||
def __init__(self):
|
||||
"""Initialize the YOLO pose estimation model for IMX500 deployment."""
|
||||
super().__init__(
|
||||
model_file="yolo11n-pose_imx_model/packerOut.zip", # replace with proper directory
|
||||
model_type=MODEL_TYPE.CONVERTED,
|
||||
color_format=COLOR_FORMAT.RGB,
|
||||
preserve_aspect_ratio=False,
|
||||
)
|
||||
|
||||
def post_process(self, output_tensors):
|
||||
"""Post-process the output tensors for pose estimation."""
|
||||
return pp_yolo_pose_ultralytics(output_tensors)
|
||||
|
||||
|
||||
device = AiCamera(frame_rate=17) # Optimal frame rate for maximum DPS of the YOLO-pose model running on the AI Camera
|
||||
model = YOLOPose()
|
||||
device.deploy(model)
|
||||
|
||||
annotator = Annotator()
|
||||
|
||||
with device as stream:
|
||||
for frame in stream:
|
||||
detections = frame.detections[frame.detections.confidence > 0.4]
|
||||
|
||||
annotator.annotate_keypoints(frame, detections)
|
||||
annotator.annotate_boxes(frame, detections, corner_length=20)
|
||||
frame.display()
|
||||
```
|
||||
|
||||
=== "Classification"
|
||||
|
||||
```python
|
||||
import cv2
|
||||
import numpy as np
|
||||
from modlib.apps import Annotator
|
||||
from modlib.devices import AiCamera
|
||||
from modlib.models import COLOR_FORMAT, MODEL_TYPE, Model
|
||||
from modlib.models.post_processors import pp_cls
|
||||
|
||||
|
||||
class YOLOClassification(Model):
|
||||
"""YOLO classification model for IMX500 deployment."""
|
||||
|
||||
def __init__(self):
|
||||
"""Initialize the YOLO classification model for IMX500 deployment."""
|
||||
super().__init__(
|
||||
model_file="yolo11n-cls_imx_model/packerOut.zip", # replace with proper directory
|
||||
model_type=MODEL_TYPE.CONVERTED,
|
||||
color_format=COLOR_FORMAT.RGB,
|
||||
preserve_aspect_ratio=False,
|
||||
)
|
||||
|
||||
self.labels = np.genfromtxt("yolo11n-cls_imx_model/labels.txt", dtype=str, delimiter="\n")
|
||||
|
||||
def post_process(self, output_tensors):
|
||||
"""Post-process the output tensors for classification."""
|
||||
return pp_cls(output_tensors)
|
||||
|
||||
|
||||
device = AiCamera()
|
||||
model = YOLOClassification()
|
||||
device.deploy(model)
|
||||
|
||||
annotator = Annotator()
|
||||
|
||||
with device as stream:
|
||||
for frame in stream:
|
||||
for i, label in enumerate([model.labels[id] for id in frame.detections.class_id[:3]]):
|
||||
text = f"{i + 1}. {label}: {frame.detections.confidence[i]:.2f}"
|
||||
cv2.putText(frame.image, text, (50, 30 + 40 * (i + 1)), cv2.FONT_HERSHEY_SIMPLEX, 0.8, (100, 0, 100), 2)
|
||||
|
||||
frame.display()
|
||||
```
|
||||
|
||||
=== "Instance Segmentation"
|
||||
|
||||
```python
|
||||
import numpy as np
|
||||
from modlib.apps import Annotator
|
||||
from modlib.devices import AiCamera
|
||||
from modlib.models import COLOR_FORMAT, MODEL_TYPE, Model
|
||||
from modlib.models.post_processors import pp_yolo_segment_ultralytics
|
||||
|
||||
|
||||
class YOLOSegment(Model):
|
||||
"""YOLO segmentation model for IMX500 deployment."""
|
||||
|
||||
def __init__(self):
|
||||
"""Initialize the YOLO segmentation model for IMX500 deployment."""
|
||||
super().__init__(
|
||||
model_file="yolo11n-seg_imx_model/packerOut.zip", # replace with proper directory
|
||||
model_type=MODEL_TYPE.CONVERTED,
|
||||
color_format=COLOR_FORMAT.RGB,
|
||||
preserve_aspect_ratio=False,
|
||||
)
|
||||
|
||||
self.labels = np.genfromtxt(
|
||||
"yolo11n-seg_imx_model/labels.txt", # replace with proper directory
|
||||
dtype=str,
|
||||
delimiter="\n",
|
||||
)
|
||||
|
||||
def post_process(self, output_tensors):
|
||||
"""Post-process the output tensors for instance segmentation."""
|
||||
return pp_yolo_segment_ultralytics(output_tensors)
|
||||
|
||||
|
||||
device = AiCamera(frame_rate=17) # Optimal frame rate for maximum DPS of the YOLO-seg model running on the AI Camera
|
||||
model = YOLOSegment()
|
||||
device.deploy(model)
|
||||
|
||||
annotator = Annotator()
|
||||
|
||||
with device as stream:
|
||||
for frame in stream:
|
||||
detections = frame.detections[frame.detections.confidence > 0.4]
|
||||
|
||||
labels = [f"{model.labels[c]}" for m, c, s, _, _ in detections]
|
||||
annotator.annotate_instance_segments(frame, detections)
|
||||
annotator.annotate_boxes(frame, detections, labels=labels)
|
||||
frame.display()
|
||||
```
|
||||
|
||||
## Benchmarks
|
||||
|
||||
YOLOv8n, YOLO11n, YOLOv8n-pose, YOLO11n-pose, YOLOv8n-cls and YOLO11n-cls benchmarks below were run by the Ultralytics team on Raspberry Pi AI Camera with `imx` model format measuring speed and accuracy.
|
||||
|
||||
| Model | Format | Size (pixels) | Size of `packerOut.zip` (MB) | mAP50-95(B) | Inference time (ms/im) |
|
||||
| ------------ | ------ | ------------- | ---------------------------- | ----------- | ---------------------- |
|
||||
| YOLOv8n | imx | 640 | 2.1 | 0.470 | 58.79 |
|
||||
| YOLO11n | imx | 640 | 2.2 | 0.517 | 58.82 |
|
||||
| YOLOv8n-pose | imx | 640 | 2.0 | 0.687 | 58.79 |
|
||||
| YOLO11n-pose | imx | 640 | 2.1 | 0.788 | 62.50 |
|
||||
|
||||
| Model | Format | Size (pixels) | Size of `packerOut.zip` (MB) | acc (top1) | acc (top5) | Inference time (ms/im) |
|
||||
| ----------- | ------ | ------------- | ---------------------------- | ---------- | ---------- | ---------------------- |
|
||||
| YOLOv8n-cls | imx | 224 | 2.3 | 0.25 | 0.5 | 33.31 |
|
||||
| YOLO11n-cls | imx | 224 | 2.3 | 0.25 | 0.417 | 33.31 |
|
||||
|
||||
!!! note
|
||||
|
||||
Validation for the above benchmarks were done using COCO128 dataset for detection models, COCO8-Pose dataset for pose estimation models and ImageNet10 for classification models.
|
||||
|
||||
## What's Under the Hood?
|
||||
|
||||
<p align="center">
|
||||
<img width="640" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/imx500-deploy.avif" alt="Sony IMX500 YOLO model deployment workflow">
|
||||
</p>
|
||||
|
||||
### Sony Model Compression Toolkit (MCT)
|
||||
|
||||
[Sony's Model Compression Toolkit (MCT)](https://github.com/SonySemiconductorSolutions/mct-model-optimization) is a powerful tool for optimizing deep learning models through quantization and pruning. It supports various quantization methods and provides advanced algorithms to reduce model size and computational complexity without significantly sacrificing accuracy. MCT is particularly useful for deploying models on resource-constrained devices, ensuring efficient inference and reduced latency.
|
||||
|
||||
### Supported Features of MCT
|
||||
|
||||
Sony's MCT offers a range of features designed to optimize neural network models:
|
||||
|
||||
1. **Graph Optimizations**: Transforms models into more efficient versions by folding layers like batch normalization into preceding layers.
|
||||
2. **Quantization Parameter Search**: Minimizes quantization noise using metrics like Mean-Square-Error, No-Clipping, and Mean-Average-Error.
|
||||
3. **Advanced Quantization Algorithms**:
|
||||
- **Shift Negative Correction**: Addresses performance issues from symmetric activation quantization.
|
||||
- **Outliers Filtering**: Uses z-score to detect and remove outliers.
|
||||
- **Clustering**: Utilizes non-uniform quantization grids for better distribution matching.
|
||||
- **Mixed-Precision Search**: Assigns different quantization bit-widths per layer based on sensitivity.
|
||||
4. **Visualization**: Use TensorBoard to observe model performance insights, quantization phases, and bit-width configurations.
|
||||
|
||||
#### Quantization
|
||||
|
||||
MCT supports several quantization methods to reduce model size and improve inference speed:
|
||||
|
||||
1. **Post-Training Quantization (PTQ)**:
|
||||
- Available via Keras and PyTorch APIs.
|
||||
- Complexity: Low
|
||||
- Computational Cost: Low (CPU minutes)
|
||||
2. **Gradient-based Post-Training Quantization (GPTQ)**:
|
||||
- Available via Keras and PyTorch APIs.
|
||||
- Complexity: Medium
|
||||
- Computational Cost: Moderate (2-3 GPU hours)
|
||||
3. **Quantization-Aware Training (QAT)**:
|
||||
- Complexity: High
|
||||
- Computational Cost: High (12-36 GPU hours)
|
||||
|
||||
MCT also supports various quantization schemes for weights and activations:
|
||||
|
||||
1. Power-of-Two (hardware-friendly)
|
||||
2. Symmetric
|
||||
3. Uniform
|
||||
|
||||
#### Structured Pruning
|
||||
|
||||
MCT introduces structured, hardware-aware model pruning designed for specific hardware architectures. This technique leverages the target platform's Single Instruction, Multiple Data (SIMD) capabilities by pruning SIMD groups. This reduces model size and complexity while optimizing channel utilization, aligned with the SIMD architecture for targeted resource utilization of weights memory footprint. Available via Keras and PyTorch APIs.
|
||||
|
||||
### IMX500 Converter Tool (Compiler)
|
||||
|
||||
The IMX500 Converter Tool is integral to the IMX500 toolset, allowing the compilation of models for deployment on Sony's IMX500 sensor (for instance, Raspberry Pi AI Cameras). This tool facilitates the transition of Ultralytics YOLO11 models processed through Ultralytics software, ensuring they are compatible and perform efficiently on the specified hardware. The export procedure following model quantization involves the generation of binary files that encapsulate essential data and device-specific configurations, streamlining the deployment process on the Raspberry Pi AI Camera.
|
||||
|
||||
## Real-World Use Cases
|
||||
|
||||
Export to IMX500 format has wide applicability across industries. Here are some examples:
|
||||
|
||||
- **Edge AI and IoT**: Enable object detection on drones or security cameras, where real-time processing on low-power devices is essential.
|
||||
- **Wearable Devices**: Deploy models optimized for small-scale AI processing on health-monitoring wearables.
|
||||
- **Smart Cities**: Use IMX500-exported YOLO11 models for traffic monitoring and safety analysis with faster processing and minimal latency.
|
||||
- **Retail Analytics**: Enhance in-store monitoring by deploying optimized models in point-of-sale systems or smart shelves.
|
||||
|
||||
## Conclusion
|
||||
|
||||
Exporting Ultralytics YOLO11 models to Sony's IMX500 format allows you to deploy your models for efficient inference on IMX500-based cameras. By leveraging advanced quantization techniques, you can reduce model size and improve inference speed without significantly compromising accuracy.
|
||||
|
||||
For more information and detailed guidelines, refer to Sony's [IMX500 website](https://www.aitrios.sony-semicon.com/edge-ai-devices/raspberry-pi-ai-camera).
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I export a YOLO11 model to IMX500 format for Raspberry Pi AI Camera?
|
||||
|
||||
To export a YOLO11 model to IMX500 format, use either the Python API or CLI command:
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
model = YOLO("yolo11n.pt")
|
||||
model.export(format="imx") # Exports with PTQ quantization by default
|
||||
```
|
||||
|
||||
The export process will create a directory containing the necessary files for deployment, including `packerOut.zip`.
|
||||
|
||||
### What are the key benefits of using the IMX500 format for edge AI deployment?
|
||||
|
||||
The IMX500 format offers several important advantages for edge deployment:
|
||||
|
||||
- On-chip AI processing reduces latency and power consumption
|
||||
- Outputs both image and metadata (inference result) instead of images only
|
||||
- Enhanced privacy by processing data locally without cloud dependency
|
||||
- Real-time processing capabilities ideal for time-sensitive applications
|
||||
- Optimized quantization for efficient model deployment on resource-constrained devices
|
||||
|
||||
### What hardware and software prerequisites are needed for IMX500 deployment?
|
||||
|
||||
For deploying IMX500 models, you'll need:
|
||||
|
||||
Hardware:
|
||||
|
||||
- Raspberry Pi 5 or Raspberry Pi 4 Model B
|
||||
- Raspberry Pi AI Camera with IMX500 sensor
|
||||
|
||||
Software:
|
||||
|
||||
- Raspberry Pi OS Bookworm
|
||||
- IMX500 firmware and tools (`sudo apt install imx500-all`)
|
||||
|
||||
### What performance can I expect from YOLO11 models on the IMX500?
|
||||
|
||||
Based on Ultralytics benchmarks on Raspberry Pi AI Camera:
|
||||
|
||||
- YOLO11n achieves 62.50ms inference time per image
|
||||
- mAP50-95 of 0.492 on COCO128 dataset
|
||||
- Model size of only 3.2MB after quantization
|
||||
|
||||
This demonstrates that IMX500 format provides efficient real-time inference while maintaining good accuracy for edge AI applications.
|
||||
@@ -0,0 +1,226 @@
|
||||
---
|
||||
comments: true
|
||||
description: Learn how to integrate YOLO26 with TensorBoard for real-time visual insights into your model's training metrics, performance graphs, and debugging workflows.
|
||||
keywords: YOLO26, TensorBoard, model training, visualization, machine learning, deep learning, Ultralytics, training metrics, performance analysis
|
||||
---
|
||||
|
||||
# Gain Visual Insights with YOLO26's Integration with TensorBoard
|
||||
|
||||
Understanding and fine-tuning [computer vision](https://www.ultralytics.com/glossary/computer-vision-cv) models like [Ultralytics' YOLO26](https://www.ultralytics.com/) becomes more straightforward when you take a closer look at their training processes. Model training visualization helps with getting insights into the model's learning patterns, performance metrics, and overall behavior. YOLO26's integration with TensorBoard makes this process of visualization and analysis easier and enables more efficient and informed adjustments to the model.
|
||||
|
||||
This guide covers how to use TensorBoard with YOLO26. You'll learn about various visualizations, from tracking metrics to analyzing model graphs. These tools will help you understand your YOLO26 model's performance better.
|
||||
|
||||
## TensorBoard
|
||||
|
||||
<p align="center">
|
||||
<img width="640" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/tensorboard-overview.avif" alt="TensorBoard training visualization dashboard">
|
||||
</p>
|
||||
|
||||
[TensorBoard](https://www.tensorflow.org/tensorboard), [TensorFlow](https://www.ultralytics.com/glossary/tensorflow)'s visualization toolkit, is essential for [machine learning](https://www.ultralytics.com/glossary/machine-learning-ml) experimentation. TensorBoard features a range of visualization tools, crucial for monitoring machine learning models. These tools include tracking key metrics like loss and accuracy, visualizing model graphs, and viewing histograms of weights and biases over time. It also provides capabilities for projecting [embeddings](https://www.ultralytics.com/glossary/embeddings) to lower-dimensional spaces and displaying multimedia data.
|
||||
|
||||
## YOLO26 Training with TensorBoard
|
||||
|
||||
Using TensorBoard while training YOLO26 models is straightforward and offers significant benefits.
|
||||
|
||||
## Installation
|
||||
|
||||
To install the required package, run:
|
||||
|
||||
!!! tip "Installation"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Install the required package for YOLO26 and Tensorboard
|
||||
pip install ultralytics
|
||||
```
|
||||
|
||||
TensorBoard is conveniently pre-installed with YOLO26, eliminating the need for additional setup for visualization purposes.
|
||||
|
||||
For detailed instructions and best practices related to the installation process, be sure to check our [YOLO26 Installation guide](../quickstart.md). While installing the required packages for YOLO26, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips.
|
||||
|
||||
## Configuring TensorBoard for Google Colab
|
||||
|
||||
When using Google Colab, it's important to set up TensorBoard before starting your training code:
|
||||
|
||||
!!! example "Configure TensorBoard for Google Colab"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```bash
|
||||
%load_ext tensorboard
|
||||
%tensorboard --logdir path/to/runs
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
Before diving into the usage instructions, be sure to check out the range of [YOLO26 models offered by Ultralytics](../models/index.md). This will help you choose the most appropriate model for your project requirements.
|
||||
|
||||
!!! tip "Enable or Disable TensorBoard"
|
||||
|
||||
By default, TensorBoard logging is disabled. You can enable or disable the logging by using the `yolo settings` command.
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Enable TensorBoard logging
|
||||
yolo settings tensorboard=True
|
||||
|
||||
# Disable TensorBoard logging
|
||||
yolo settings tensorboard=False
|
||||
```
|
||||
|
||||
!!! example "Usage"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load a pretrained model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Train the model
|
||||
results = model.train(data="coco8.yaml", epochs=100, imgsz=640)
|
||||
```
|
||||
|
||||
Upon running the usage code snippet above, you can expect the following output:
|
||||
|
||||
```bash
|
||||
TensorBoard: Start with 'tensorboard --logdir path_to_your_tensorboard_logs', view at http://localhost:6006/
|
||||
```
|
||||
|
||||
This output indicates that TensorBoard is now actively monitoring your YOLO26 training session. You can access the TensorBoard dashboard by visiting the provided URL (http://localhost:6006/) to view real-time training metrics and model performance. For users working in [Google Colab](../integrations/google-colab.md), the TensorBoard will be displayed in the same cell where you executed the TensorBoard configuration commands.
|
||||
|
||||
For more information related to the model training process, be sure to check our [YOLO26 Model Training guide](../modes/train.md). If you are interested in learning more about logging, checkpoints, plotting, and file management, read our [usage guide on configuration](../usage/cfg.md).
|
||||
|
||||
## Understanding Your TensorBoard for YOLO26 Training
|
||||
|
||||
Now, let's focus on understanding the various features and components of TensorBoard in the context of YOLO26 training. The three key sections of the TensorBoard are Time Series, Scalars, and Graphs.
|
||||
|
||||
### Time Series
|
||||
|
||||
The Time Series feature in the TensorBoard offers a dynamic and detailed perspective of various training metrics over time for YOLO26 models. It focuses on the progression and trends of metrics across training epochs. Here's an example of what you can expect to see.
|
||||
|
||||
|
||||
|
||||
#### Key Features of Time Series in TensorBoard
|
||||
|
||||
- **Filter Tags and Pinned Cards**: This functionality allows users to filter specific metrics and pin cards for quick comparison and access. It's particularly useful for focusing on specific aspects of the training process.
|
||||
|
||||
- **Detailed Metric Cards**: Time Series divides metrics into different categories like [learning rate](https://www.ultralytics.com/glossary/learning-rate) (lr), training (train), and validation (val) metrics, each represented by individual cards.
|
||||
|
||||
- **Graphical Display**: Each card in the Time Series section shows a detailed graph of a specific metric over the course of training. This visual representation aids in identifying trends, patterns, or anomalies in the training process.
|
||||
|
||||
- **In-Depth Analysis**: Time Series provides an in-depth analysis of each metric. For instance, different learning rate segments are shown, offering insights into how adjustments in learning rate impact the model's learning curve.
|
||||
|
||||
#### Importance of Time Series in YOLO26 Training
|
||||
|
||||
The Time Series section is essential for a thorough analysis of the YOLO26 model's training progress. It lets you track the metrics in real time to promptly identify and solve issues. It also offers a detailed view of each metric's progression, which is crucial for fine-tuning the model and enhancing its performance.
|
||||
|
||||
### Scalars
|
||||
|
||||
Scalars in the TensorBoard are crucial for plotting and analyzing simple metrics like loss and accuracy during the training of YOLO26 models. They offer a clear and concise view of how these metrics evolve with each training [epoch](https://www.ultralytics.com/glossary/epoch), providing insights into the model's learning effectiveness and stability. Here's an example of what you can expect to see.
|
||||
|
||||
|
||||
|
||||
#### Key Features of Scalars in TensorBoard
|
||||
|
||||
- **Learning Rate (lr) Tags**: These tags show the variations in the learning rate across different segments (e.g., `pg0`, `pg1`, `pg2`). This helps us understand the impact of learning rate adjustments on the training process.
|
||||
|
||||
- **Metrics Tags**: Scalars include performance indicators such as:
|
||||
- `mAP50 (B)`: Mean Average [Precision](https://www.ultralytics.com/glossary/precision) at 50% [Intersection over Union](https://www.ultralytics.com/glossary/intersection-over-union-iou) (IoU), crucial for assessing object detection accuracy.
|
||||
|
||||
- `mAP50-95 (B)`: [Mean Average Precision](https://www.ultralytics.com/glossary/mean-average-precision-map) calculated over a range of IoU thresholds, offering a more comprehensive evaluation of accuracy.
|
||||
|
||||
- `Precision (B)`: Indicates the ratio of correctly predicted positive observations, key to understanding prediction [accuracy](https://www.ultralytics.com/glossary/accuracy).
|
||||
|
||||
- `Recall (B)`: Important for models where missing a detection is significant, this metric measures the ability to detect all relevant instances.
|
||||
|
||||
- To learn more about the different metrics, read our guide on [performance metrics](../guides/yolo-performance-metrics.md).
|
||||
|
||||
- **Training and Validation Tags (`train`, `val`)**: These tags display metrics specifically for the training and validation datasets, allowing for a comparative analysis of model performance across different data sets.
|
||||
|
||||
#### Importance of Monitoring Scalars
|
||||
|
||||
Observing scalar metrics is crucial for fine-tuning the YOLO26 model. Variations in these metrics, such as spikes or irregular patterns in loss graphs, can highlight potential issues such as [overfitting](https://www.ultralytics.com/glossary/overfitting), [underfitting](https://www.ultralytics.com/glossary/underfitting), or inappropriate learning rate settings. By closely monitoring these scalars, you can make informed decisions to optimize the training process, ensuring that the model learns effectively and achieves the desired performance.
|
||||
|
||||
### Difference Between Scalars and Time Series
|
||||
|
||||
While both Scalars and Time Series in TensorBoard are used for tracking metrics, they serve slightly different purposes. Scalars focus on plotting simple metrics such as loss and accuracy as scalar values. They provide a high-level overview of how these metrics change with each training epoch. Meanwhile, the time-series section of the TensorBoard offers a more detailed timeline view of various metrics. It is particularly useful for monitoring the progression and trends of metrics over time, providing a deeper dive into the specifics of the training process.
|
||||
|
||||
### Graphs
|
||||
|
||||
The Graphs section of the TensorBoard visualizes the computational graph of the YOLO26 model, showing how operations and data flow within the model. It's a powerful tool for understanding the model's structure, ensuring that all layers are connected correctly, and for identifying any potential bottlenecks in data flow. Here's an example of what you can expect to see.
|
||||
|
||||
|
||||
|
||||
Graphs are particularly useful for debugging the model, especially in complex architectures typical in [deep learning](https://www.ultralytics.com/glossary/deep-learning-dl) models like YOLO26. They help in verifying layer connections and the overall design of the model.
|
||||
|
||||
## Summary
|
||||
|
||||
This guide aims to help you use TensorBoard with YOLO26 for visualization and analysis of machine learning model training. It focuses on explaining how key TensorBoard features can provide insights into training metrics and model performance during YOLO26 training sessions.
|
||||
|
||||
For a more detailed exploration of these features and effective utilization strategies, you can refer to TensorFlow's official [TensorBoard documentation](https://www.tensorflow.org/tensorboard/get_started) and their [GitHub repository](https://github.com/tensorflow/tensorboard).
|
||||
|
||||
Want to learn more about the various integrations of Ultralytics? Check out the [Ultralytics integrations guide page](../integrations/index.md) to see what other exciting capabilities are waiting to be discovered!
|
||||
|
||||
## FAQ
|
||||
|
||||
### What benefits does using TensorBoard with YOLO26 offer?
|
||||
|
||||
Using TensorBoard with YOLO26 provides several visualization tools essential for efficient model training:
|
||||
|
||||
- **Real-Time Metrics Tracking:** Track key metrics such as loss, accuracy, precision, and recall live.
|
||||
- **Model Graph Visualization:** Understand and debug the model architecture by visualizing computational graphs.
|
||||
- **Embedding Visualization:** Project embeddings to lower-dimensional spaces for better insight.
|
||||
|
||||
These tools enable you to make informed adjustments to enhance your YOLO26 model's performance. For more details on TensorBoard features, check out the TensorFlow [TensorBoard guide](https://www.tensorflow.org/tensorboard/get_started).
|
||||
|
||||
### How can I monitor training metrics using TensorBoard when training a YOLO26 model?
|
||||
|
||||
To monitor training metrics while training a YOLO26 model with TensorBoard, follow these steps:
|
||||
|
||||
1. **Install TensorBoard and YOLO26:** Run `pip install ultralytics` which includes TensorBoard.
|
||||
2. **Configure TensorBoard Logging:** During the training process, YOLO26 logs metrics to a specified log directory.
|
||||
3. **Start TensorBoard:** Launch TensorBoard using the command `tensorboard --logdir path/to/your/tensorboard/logs`.
|
||||
|
||||
The TensorBoard dashboard, accessible via [http://localhost:6006/](http://localhost:6006/), provides real-time insights into various training metrics. For a deeper dive into training configurations, visit our [YOLO26 Configuration guide](../usage/cfg.md).
|
||||
|
||||
### What kind of metrics can I visualize with TensorBoard when training YOLO26 models?
|
||||
|
||||
When training YOLO26 models, TensorBoard allows you to visualize an array of important metrics including:
|
||||
|
||||
- **Loss (Training and Validation):** Indicates how well the model is performing during training and validation.
|
||||
- **Accuracy/Precision/[Recall](https://www.ultralytics.com/glossary/recall):** Key performance metrics to evaluate detection accuracy.
|
||||
- **Learning Rate:** Track learning rate changes to understand its impact on training dynamics.
|
||||
- **mAP (mean Average Precision):** For a comprehensive evaluation of [object detection](https://www.ultralytics.com/glossary/object-detection) accuracy at various IoU thresholds.
|
||||
|
||||
These visualizations are essential for tracking model performance and making necessary optimizations. For more information on these metrics, refer to our [Performance Metrics guide](../guides/yolo-performance-metrics.md).
|
||||
|
||||
### Can I use TensorBoard in a Google Colab environment for training YOLO26?
|
||||
|
||||
Yes, you can use TensorBoard in a Google Colab environment to train YOLO26 models. Here's a quick setup:
|
||||
|
||||
!!! example "Configure TensorBoard for Google Colab"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```bash
|
||||
%load_ext tensorboard
|
||||
%tensorboard --logdir path/to/runs
|
||||
```
|
||||
|
||||
Then, run the YOLO26 training script:
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load a pretrained model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Train the model
|
||||
results = model.train(data="coco8.yaml", epochs=100, imgsz=640)
|
||||
```
|
||||
|
||||
TensorBoard will visualize the training progress within Colab, providing real-time insights into metrics like loss and accuracy. For additional details on configuring YOLO26 training, see our detailed [YOLO26 Installation guide](../quickstart.md).
|
||||
@@ -0,0 +1,578 @@
|
||||
---
|
||||
comments: true
|
||||
description: Learn to convert YOLO26 models to TensorRT for high-speed NVIDIA GPU inference. Boost efficiency and deploy optimized models with our step-by-step guide.
|
||||
keywords: YOLO26, YOLO26, TensorRT, NVIDIA, GPU, deep learning, model optimization, high-speed inference, model export
|
||||
---
|
||||
|
||||
# TensorRT Export for YOLO26 Models
|
||||
|
||||
Deploying [computer vision](https://www.ultralytics.com/glossary/computer-vision-cv) models in high-performance environments can require a format that maximizes speed and efficiency. This is especially true when you are deploying your model on NVIDIA GPUs.
|
||||
|
||||
By using the TensorRT export format, you can enhance your [Ultralytics YOLO26](https://github.com/ultralytics/ultralytics) models for swift and efficient inference on NVIDIA hardware. This guide will give you easy-to-follow steps for the conversion process and help you make the most of NVIDIA's advanced technology in your [deep learning](https://www.ultralytics.com/glossary/deep-learning-dl) projects.
|
||||
|
||||
## TensorRT
|
||||
|
||||
<p align="center">
|
||||
<img width="100%" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/tensorrt-overview.avif" alt="NVIDIA TensorRT optimization workflow">
|
||||
</p>
|
||||
|
||||
[TensorRT](https://developer.nvidia.com/tensorrt), developed by NVIDIA, is an advanced software development kit (SDK) designed for high-speed deep learning inference. It's well-suited for real-time applications like [object detection](https://www.ultralytics.com/glossary/object-detection).
|
||||
|
||||
This toolkit optimizes deep learning models for NVIDIA GPUs and results in faster and more efficient operations. TensorRT models undergo TensorRT optimization, which includes techniques like layer fusion, precision calibration (INT8 and FP16), dynamic tensor memory management, and kernel auto-tuning. Converting deep learning models into the TensorRT format allows developers to realize the potential of NVIDIA GPUs fully.
|
||||
|
||||
TensorRT is known for its compatibility with various model formats, including TensorFlow, [PyTorch](https://www.ultralytics.com/glossary/pytorch), and ONNX, providing developers with a flexible solution for integrating and optimizing models from different frameworks. This versatility enables efficient [model deployment](https://www.ultralytics.com/glossary/model-deployment) across diverse hardware and software environments.
|
||||
|
||||
## Key Features of TensorRT Models
|
||||
|
||||
TensorRT models offer a range of key features that contribute to their efficiency and effectiveness in high-speed deep learning inference:
|
||||
|
||||
- **Precision Calibration**: TensorRT supports precision calibration, allowing models to be fine-tuned for specific accuracy requirements. This includes support for reduced precision formats like INT8 and FP16, which can further boost inference speed while maintaining acceptable accuracy levels.
|
||||
|
||||
- **Layer Fusion**: The TensorRT optimization process includes layer fusion, where multiple layers of a [neural network](https://www.ultralytics.com/glossary/neural-network-nn) are combined into a single operation. This reduces computational overhead and improves inference speed by minimizing memory access and computation.
|
||||
|
||||
<p align="center">
|
||||
<img width="100%" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/tensorrt-layer-fusion.avif" alt="TensorRT neural network layer fusion optimization">
|
||||
</p>
|
||||
|
||||
- **Dynamic Tensor Memory Management**: TensorRT efficiently manages tensor memory usage during inference, reducing memory overhead and optimizing memory allocation. This results in more efficient GPU memory utilization.
|
||||
|
||||
- **Automatic Kernel Tuning**: TensorRT applies automatic kernel tuning to select the most optimized GPU kernel for each layer of the model. This adaptive approach ensures that the model takes full advantage of the GPU's computational power.
|
||||
|
||||
## Deployment Options in TensorRT
|
||||
|
||||
Before we look at the code for exporting YOLO26 models to the TensorRT format, let's understand where TensorRT models are normally used.
|
||||
|
||||
TensorRT offers several deployment options, and each option balances ease of integration, performance optimization, and flexibility differently:
|
||||
|
||||
- **Deploying within [TensorFlow](https://www.ultralytics.com/glossary/tensorflow)**: This method integrates TensorRT into TensorFlow, allowing optimized models to run in a familiar TensorFlow environment. It's useful for models with a mix of supported and unsupported layers, as TF-TRT can handle these efficiently.
|
||||
|
||||
<p align="center">
|
||||
<img width="100%" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/tf-trt-workflow.avif" alt="NVIDIA TensorRT optimization workflow">
|
||||
</p>
|
||||
|
||||
- **Standalone TensorRT Runtime API**: Offers granular control, ideal for performance-critical applications. It's more complex but allows for custom implementation of unsupported operators.
|
||||
|
||||
- **NVIDIA Triton Inference Server**: An option that supports models from various frameworks. Particularly suited for cloud or edge inference, it provides features like concurrent model execution and model analysis.
|
||||
|
||||
## Exporting YOLO26 Models to TensorRT
|
||||
|
||||
You can improve execution efficiency and optimize performance by converting YOLO26 models to TensorRT format.
|
||||
|
||||
### Installation
|
||||
|
||||
To install the required package, run:
|
||||
|
||||
!!! tip "Installation"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Install the required package for YOLO26
|
||||
pip install ultralytics
|
||||
```
|
||||
|
||||
For detailed instructions and best practices related to the installation process, check our [YOLO26 Installation guide](../quickstart.md). While installing the required packages for YOLO26, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips.
|
||||
|
||||
### Usage
|
||||
|
||||
Before diving into the usage instructions, be sure to check out the range of [YOLO26 models offered by Ultralytics](../models/index.md). This will help you choose the most appropriate model for your project requirements.
|
||||
|
||||
!!! example "Usage"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load the YOLO26 model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Export the model to TensorRT format
|
||||
model.export(format="engine") # creates 'yolo26n.engine'
|
||||
|
||||
# Load the exported TensorRT model
|
||||
tensorrt_model = YOLO("yolo26n.engine")
|
||||
|
||||
# Run inference
|
||||
results = tensorrt_model("https://ultralytics.com/images/bus.jpg")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Export a YOLO26n PyTorch model to TensorRT format
|
||||
yolo export model=yolo26n.pt format=engine # creates 'yolo26n.engine'
|
||||
|
||||
# Run inference with the exported model
|
||||
yolo predict model=yolo26n.engine source='https://ultralytics.com/images/bus.jpg'
|
||||
```
|
||||
|
||||
### Export Arguments
|
||||
|
||||
| Argument | Type | Default | Description |
|
||||
| ----------- | ----------------- | -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `format` | `str` | `'engine'` | Target format for the exported model, defining compatibility with various deployment environments. |
|
||||
| `imgsz` | `int` or `tuple` | `640` | Desired image size for the model input. Can be an integer for square images or a tuple `(height, width)` for specific dimensions. |
|
||||
| `half` | `bool` | `False` | Enables FP16 (half-precision) quantization, reducing model size and potentially speeding up inference on supported hardware. |
|
||||
| `int8` | `bool` | `False` | Activates INT8 quantization, further compressing the model and speeding up inference with minimal [accuracy](https://www.ultralytics.com/glossary/accuracy) loss, primarily for edge devices. |
|
||||
| `dynamic` | `bool` | `False` | Allows dynamic input sizes, enhancing flexibility in handling varying image dimensions. |
|
||||
| `simplify` | `bool` | `True` | Simplifies the model graph with `onnxslim`, potentially improving performance and compatibility. |
|
||||
| `workspace` | `float` or `None` | `None` | Sets the maximum workspace size in GiB for TensorRT optimizations, balancing memory usage and performance; use `None` for auto-allocation by TensorRT up to device maximum. |
|
||||
| `nms` | `bool` | `False` | Adds Non-Maximum Suppression (NMS), essential for accurate and efficient detection post-processing. |
|
||||
| `batch` | `int` | `1` | Specifies export model batch inference size or the max number of images the exported model will process concurrently in `predict` mode. |
|
||||
| `data` | `str` | `'coco8.yaml'` | Path to the [dataset](https://docs.ultralytics.com/datasets/) configuration file (default: `coco8.yaml`), essential for quantization. |
|
||||
| `fraction` | `float` | `1.0` | Specifies the fraction of the dataset to use for INT8 quantization calibration. Allows for calibrating on a subset of the full dataset, useful for experiments or when resources are limited. If not specified with INT8 enabled, the full dataset will be used. |
|
||||
| `device` | `str` | `None` | Specifies the device for exporting: GPU (`device=0`), DLA for NVIDIA Jetson (`device=dla:0` or `device=dla:1`). |
|
||||
|
||||
!!! tip
|
||||
|
||||
Please make sure to use a GPU with CUDA support when exporting to TensorRT.
|
||||
|
||||
For more details about the export process, visit the [Ultralytics documentation page on exporting](../modes/export.md).
|
||||
|
||||
### Exporting TensorRT with INT8 Quantization
|
||||
|
||||
Exporting Ultralytics YOLO models using TensorRT with INT8 [precision](https://www.ultralytics.com/glossary/precision) executes post-training quantization (PTQ). TensorRT uses calibration for PTQ, which measures the distribution of activations within each activation tensor as the YOLO model processes inference on representative input data, and then uses that distribution to estimate scale values for each tensor. Each activation tensor that is a candidate for quantization has an associated scale that is deduced by a calibration process.
|
||||
|
||||
When processing implicitly quantized networks TensorRT uses INT8 opportunistically to optimize layer execution time. If a layer runs faster in INT8 and has assigned quantization scales on its data inputs and outputs, then a kernel with INT8 precision is assigned to that layer, otherwise TensorRT selects a precision of either FP32 or FP16 for the kernel based on whichever results in faster execution time for that layer.
|
||||
|
||||
!!! tip
|
||||
|
||||
It is **critical** to ensure that the same device that will use the TensorRT model weights for deployment is used for exporting with INT8 precision, as the calibration results can vary across devices.
|
||||
|
||||
#### Configuring INT8 Export
|
||||
|
||||
The arguments provided when using [export](../modes/export.md) for an Ultralytics YOLO model will **greatly** influence the performance of the exported model. They will also need to be selected based on the device resources available, however the default arguments _should_ work for most [Ampere (or newer) NVIDIA discrete GPUs](https://developer.nvidia.com/blog/nvidia-ampere-architecture-in-depth/). The calibration algorithm used is `"MINMAX_CALIBRATION"` and you can read more details about the options available [in the TensorRT Developer Guide](https://docs.nvidia.com/deeplearning/tensorrt/latest/_static/python-api/infer/Int8/MinMaxCalibrator.html). Ultralytics tests found that `"MINMAX_CALIBRATION"` was the best choice and exports are fixed to using this algorithm.
|
||||
|
||||
- `workspace` : Controls the size (in GiB) of the device memory allocation while converting the model weights.
|
||||
- Adjust the `workspace` value according to your calibration needs and resource availability. While a larger `workspace` may increase calibration time, it allows TensorRT to explore a wider range of optimization tactics, potentially enhancing model performance and [accuracy](https://www.ultralytics.com/glossary/accuracy). Conversely, a smaller `workspace` can reduce calibration time but may limit the optimization strategies, affecting the quality of the quantized model.
|
||||
|
||||
- Default is `workspace=None`, which will allow for TensorRT to automatically allocate memory, when configuring manually, this value may need to be increased if calibration crashes (exits without warning).
|
||||
|
||||
- TensorRT will report `UNSUPPORTED_STATE` during export if the value for `workspace` is larger than the memory available to the device, which means the value for `workspace` should be lowered or set to `None`.
|
||||
|
||||
- If `workspace` is set to max value and calibration fails/crashes, consider using `None` for auto-allocation or by reducing the values for `imgsz` and `batch` to reduce memory requirements.
|
||||
|
||||
- <u><b>Remember</b> calibration for INT8 is specific to each device</u>, borrowing a "high-end" GPU for calibration, might result in poor performance when inference is run on another device.
|
||||
|
||||
- `batch` : The maximum batch-size that will be used for inference. During inference smaller batches can be used, but inference will not accept batches any larger than what is specified.
|
||||
|
||||
!!! note
|
||||
|
||||
During calibration, twice the `batch` size provided will be used. Using small batches can lead to inaccurate scaling during calibration. This is because the process adjusts based on the data it sees. Small batches might not capture the full range of values, leading to issues with the final calibration, so the `batch` size is doubled automatically. If no [batch size](https://www.ultralytics.com/glossary/batch-size) is specified `batch=1`, calibration will be run at `batch=1 * 2` to reduce calibration scaling errors.
|
||||
|
||||
Experimentation by NVIDIA led them to recommend using at least 500 calibration images that are representative of the data for your model, with INT8 quantization calibration. This is a guideline and not a _hard_ requirement, and <u>**you will need to experiment with what is required to perform well for your dataset**.</u> Since the calibration data is required for INT8 calibration with TensorRT, make certain to use the `data` argument when `int8=True` for TensorRT and use `data="my_dataset.yaml"`, which will use the images from [validation](../modes/val.md) to calibrate with. When no value is passed for `data` with export to TensorRT with INT8 quantization, the default will be to use one of the ["small" example datasets based on the model task](../datasets/index.md) instead of throwing an error.
|
||||
|
||||
!!! example
|
||||
|
||||
=== "Python"
|
||||
|
||||
```{ .py .annotate }
|
||||
from ultralytics import YOLO
|
||||
|
||||
model = YOLO("yolo26n.pt")
|
||||
model.export(
|
||||
format="engine",
|
||||
dynamic=True, # (1)!
|
||||
batch=8, # (2)!
|
||||
workspace=4, # (3)!
|
||||
int8=True,
|
||||
data="coco.yaml", # (4)!
|
||||
)
|
||||
|
||||
# Load the exported TensorRT INT8 model
|
||||
model = YOLO("yolo26n.engine", task="detect")
|
||||
|
||||
# Run inference
|
||||
result = model.predict("https://ultralytics.com/images/bus.jpg")
|
||||
```
|
||||
|
||||
1. Exports with dynamic axes, this will be enabled by default when exporting with `int8=True` even when not explicitly set. See [export arguments](../modes/export.md#arguments) for additional information.
|
||||
2. Sets max batch size of 8 for exported model, which calibrates with `batch = 2 * 8` to avoid scaling errors during calibration.
|
||||
3. Allocates 4 GiB of memory instead of allocating the entire device for conversion process.
|
||||
4. Uses [COCO dataset](../datasets/detect/coco.md) for calibration, specifically the images used for [validation](../modes/val.md) (5,000 total).
|
||||
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Export a YOLO26n PyTorch model to TensorRT format with INT8 quantization
|
||||
yolo export model=yolo26n.pt format=engine batch=8 workspace=4 int8=True data=coco.yaml # creates 'yolo26n.engine'
|
||||
|
||||
# Run inference with the exported TensorRT quantized model
|
||||
yolo predict model=yolo26n.engine source='https://ultralytics.com/images/bus.jpg'
|
||||
```
|
||||
|
||||
???+ warning "Calibration Cache"
|
||||
|
||||
TensorRT will generate a calibration `.cache` which can be reused to speed up export of future model weights using the same data, but this may result in poor calibration when the data is vastly different or if the `batch` value is changed drastically. In these circumstances, the existing `.cache` should be renamed and moved to a different directory or deleted entirely.
|
||||
|
||||
#### Advantages of using YOLO with TensorRT INT8
|
||||
|
||||
- **Reduced model size:** Quantization from FP32 to INT8 can reduce the model size by 4x (on disk or in memory), leading to faster download times. lower storage requirements, and reduced memory footprint when deploying a model.
|
||||
|
||||
- **Lower power consumption:** Reduced precision operations for INT8 exported YOLO models can consume less power compared to FP32 models, especially for battery-powered devices.
|
||||
|
||||
- **Improved inference speeds:** TensorRT optimizes the model for the target hardware, potentially leading to faster inference speeds on GPUs, embedded devices, and accelerators.
|
||||
|
||||
??? note "Note on Inference Speeds"
|
||||
|
||||
The first few inference calls with a model exported to TensorRT INT8 can be expected to have longer than usual preprocessing, inference, and/or postprocessing times. This may also occur when changing `imgsz` during inference, especially when `imgsz` is not the same as what was specified during export (export `imgsz` is set as TensorRT "optimal" profile).
|
||||
|
||||
#### Drawbacks of using YOLO with TensorRT INT8
|
||||
|
||||
- **Decreases in evaluation metrics:** Using a lower precision will mean that `mAP`, `Precision`, `Recall` or any [other metric used to evaluate model performance](../guides/yolo-performance-metrics.md) is likely to be somewhat worse. See the [Performance results section](#ultralytics-yolo-tensorrt-export-performance) to compare the differences in `mAP50` and `mAP50-95` when exporting with INT8 on small sample of various devices.
|
||||
|
||||
- **Increased development times:** Finding the "optimal" settings for INT8 calibration for dataset and device can take a significant amount of testing.
|
||||
|
||||
- **Hardware dependency:** Calibration and performance gains could be highly hardware dependent and model weights are less transferable.
|
||||
|
||||
## Ultralytics YOLO TensorRT Export Performance
|
||||
|
||||
### NVIDIA A100
|
||||
|
||||
!!! tip "Performance"
|
||||
|
||||
Tested with Ubuntu 22.04.3 LTS, `python 3.10.12`, `ultralytics==8.2.4`, `tensorrt==8.6.1.post1`
|
||||
|
||||
=== "Detection (COCO)"
|
||||
|
||||
See [Detection Docs](../tasks/detect.md) for usage examples with these models trained on [COCO](../datasets/detect/coco.md), which include 80 pretrained classes.
|
||||
|
||||
!!! note
|
||||
|
||||
Inference times shown for `mean`, `min` (fastest), and `max` (slowest) for each test using pretrained weights `yolov8n.engine`
|
||||
|
||||
| Precision | Eval test | mean<br>(ms) | min \| max<br>(ms) | mAP<sup>val</sup><br>50(B) | mAP<sup>val</sup><br>50-95(B) | `batch` | size<br><sup>(pixels)</sup> |
|
||||
|-----------|--------------|--------------|--------------------|----------------------|-------------------------|---------|-----------------------|
|
||||
| FP32 | Predict | 0.52 | 0.51 \| 0.56 | | | 8 | 640 |
|
||||
| FP32 | COCO<sup>val</sup> | 0.52 | | 0.52 | 0.37 | 1 | 640 |
|
||||
| FP16 | Predict | 0.34 | 0.34 \| 0.41 | | | 8 | 640 |
|
||||
| FP16 | COCO<sup>val</sup> | 0.33 | | 0.52 | 0.37 | 1 | 640 |
|
||||
| INT8 | Predict | 0.28 | 0.27 \| 0.31 | | | 8 | 640 |
|
||||
| INT8 | COCO<sup>val</sup> | 0.29 | | 0.47 | 0.33 | 1 | 640 |
|
||||
|
||||
=== "Segmentation (COCO)"
|
||||
|
||||
See [Segmentation Docs](../tasks/segment.md) for usage examples with these models trained on [COCO](../datasets/segment/coco.md), which include 80 pretrained classes.
|
||||
|
||||
!!! note
|
||||
|
||||
Inference times shown for `mean`, `min` (fastest), and `max` (slowest) for each test using pretrained weights `yolov8n-seg.engine`
|
||||
|
||||
| Precision | Eval test | mean<br>(ms) | min \| max<br>(ms) | mAP<sup>val</sup><br>50(B) | mAP<sup>val</sup><br>50-95(B) | mAP<sup>val</sup><br>50(M) | mAP<sup>val</sup><br>50-95(M) | `batch` | size<br><sup>(pixels)</sup> |
|
||||
|-----------|--------------|--------------|--------------------|----------------------|-------------------------|----------------------|-------------------------|---------|-----------------------|
|
||||
| FP32 | Predict | 0.62 | 0.61 \| 0.68 | | | | | 8 | 640 |
|
||||
| FP32 | COCO<sup>val</sup> | 0.63 | | 0.52 | 0.36 | 0.49 | 0.31 | 1 | 640 |
|
||||
| FP16 | Predict | 0.40 | 0.39 \| 0.44 | | | | | 8 | 640 |
|
||||
| FP16 | COCO<sup>val</sup> | 0.43 | | 0.52 | 0.36 | 0.49 | 0.30 | 1 | 640 |
|
||||
| INT8 | Predict | 0.34 | 0.33 \| 0.37 | | | | | 8 | 640 |
|
||||
| INT8 | COCO<sup>val</sup> | 0.36 | | 0.46 | 0.32 | 0.43 | 0.27 | 1 | 640 |
|
||||
|
||||
=== "Classification (ImageNet)"
|
||||
|
||||
See [Classification Docs](../tasks/classify.md) for usage examples with these models trained on [ImageNet](../datasets/classify/imagenet.md), which include 1000 pretrained classes.
|
||||
|
||||
!!! note
|
||||
|
||||
Inference times shown for `mean`, `min` (fastest), and `max` (slowest) for each test using pretrained weights `yolov8n-cls.engine`
|
||||
|
||||
| Precision | Eval test | mean<br>(ms) | min \| max<br>(ms) | top-1 | top-5 | `batch` | size<br><sup>(pixels)</sup> |
|
||||
|-----------|------------------|--------------|--------------------|-------|-------|---------|-----------------------|
|
||||
| FP32 | Predict | 0.26 | 0.25 \| 0.28 | | | 8 | 640 |
|
||||
| FP32 | ImageNet<sup>val</sup> | 0.26 | | 0.35 | 0.61 | 1 | 640 |
|
||||
| FP16 | Predict | 0.18 | 0.17 \| 0.19 | | | 8 | 640 |
|
||||
| FP16 | ImageNet<sup>val</sup> | 0.18 | | 0.35 | 0.61 | 1 | 640 |
|
||||
| INT8 | Predict | 0.16 | 0.15 \| 0.57 | | | 8 | 640 |
|
||||
| INT8 | ImageNet<sup>val</sup> | 0.15 | | 0.32 | 0.59 | 1 | 640 |
|
||||
|
||||
=== "Pose (COCO)"
|
||||
|
||||
See [Pose Estimation Docs](../tasks/pose.md) for usage examples with these models trained on [COCO](../datasets/pose/coco.md), which include 1 pretrained class, "person".
|
||||
|
||||
!!! note
|
||||
|
||||
Inference times shown for `mean`, `min` (fastest), and `max` (slowest) for each test using pretrained weights `yolov8n-pose.engine`
|
||||
|
||||
| Precision | Eval test | mean<br>(ms) | min \| max<br>(ms) | mAP<sup>val</sup><br>50(B) | mAP<sup>val</sup><br>50-95(B) | mAP<sup>val</sup><br>50(P) | mAP<sup>val</sup><br>50-95(P) | `batch` | size<br><sup>(pixels)</sup> |
|
||||
|-----------|--------------|--------------|--------------------|----------------------|-------------------------|----------------------|-------------------------|---------|-----------------------|
|
||||
| FP32 | Predict | 0.54 | 0.53 \| 0.58 | | | | | 8 | 640 |
|
||||
| FP32 | COCO<sup>val</sup> | 0.55 | | 0.91 | 0.69 | 0.80 | 0.51 | 1 | 640 |
|
||||
| FP16 | Predict | 0.37 | 0.35 \| 0.41 | | | | | 8 | 640 |
|
||||
| FP16 | COCO<sup>val</sup> | 0.36 | | 0.91 | 0.69 | 0.80 | 0.51 | 1 | 640 |
|
||||
| INT8 | Predict | 0.29 | 0.28 \| 0.33 | | | | | 8 | 640 |
|
||||
| INT8 | COCO<sup>val</sup> | 0.30 | | 0.90 | 0.68 | 0.78 | 0.47 | 1 | 640 |
|
||||
|
||||
=== "OBB (DOTAv1)"
|
||||
|
||||
See [Oriented Detection Docs](../tasks/obb.md) for usage examples with these models trained on [DOTAv1](../datasets/obb/dota-v2.md#dota-v10), which include 15 pretrained classes.
|
||||
|
||||
!!! note
|
||||
|
||||
Inference times shown for `mean`, `min` (fastest), and `max` (slowest) for each test using pretrained weights `yolov8n-obb.engine`
|
||||
|
||||
| Precision | Eval test | mean<br>(ms) | min \| max<br>(ms) | mAP<sup>val</sup><br>50(B) | mAP<sup>val</sup><br>50-95(B) | `batch` | size<br><sup>(pixels)</sup> |
|
||||
|-----------|----------------|--------------|--------------------|----------------------|-------------------------|---------|-----------------------|
|
||||
| FP32 | Predict | 0.52 | 0.51 \| 0.59 | | | 8 | 640 |
|
||||
| FP32 | DOTAv1<sup>val</sup> | 0.76 | | 0.50 | 0.36 | 1 | 640 |
|
||||
| FP16 | Predict | 0.34 | 0.33 \| 0.42 | | | 8 | 640 |
|
||||
| FP16 | DOTAv1<sup>val</sup> | 0.59 | | 0.50 | 0.36 | 1 | 640 |
|
||||
| INT8 | Predict | 0.29 | 0.28 \| 0.33 | | | 8 | 640 |
|
||||
| INT8 | DOTAv1<sup>val</sup> | 0.32 | | 0.45 | 0.32 | 1 | 640 |
|
||||
|
||||
### Consumer GPUs
|
||||
|
||||
!!! tip "Detection Performance (COCO)"
|
||||
|
||||
=== "RTX 3080 12 GB"
|
||||
|
||||
Tested with Windows 10.0.19045, `python 3.10.9`, `ultralytics==8.2.4`, `tensorrt==10.0.0b6`
|
||||
|
||||
!!! note
|
||||
|
||||
Inference times shown for `mean`, `min` (fastest), and `max` (slowest) for each test using pretrained weights `yolov8n.engine`
|
||||
|
||||
| Precision | Eval test | mean<br>(ms) | min \| max<br>(ms) | mAP<sup>val</sup><br>50(B) | mAP<sup>val</sup><br>50-95(B) | `batch` | size<br><sup>(pixels)</sup> |
|
||||
|-----------|--------------|--------------|--------------------|----------------------|-------------------------|---------|-----------------------|
|
||||
| FP32 | Predict | 1.06 | 0.75 \| 1.88 | | | 8 | 640 |
|
||||
| FP32 | COCO<sup>val</sup> | 1.37 | | 0.52 | 0.37 | 1 | 640 |
|
||||
| FP16 | Predict | 0.62 | 0.75 \| 1.13 | | | 8 | 640 |
|
||||
| FP16 | COCO<sup>val</sup> | 0.85 | | 0.52 | 0.37 | 1 | 640 |
|
||||
| INT8 | Predict | 0.52 | 0.38 \| 1.00 | | | 8 | 640 |
|
||||
| INT8 | COCO<sup>val</sup> | 0.74 | | 0.47 | 0.33 | 1 | 640 |
|
||||
|
||||
=== "RTX 3060 12 GB"
|
||||
|
||||
Tested with Windows 10.0.22631, `python 3.11.9`, `ultralytics==8.2.4`, `tensorrt==10.0.1`
|
||||
|
||||
!!! note
|
||||
|
||||
Inference times shown for `mean`, `min` (fastest), and `max` (slowest) for each test using pretrained weights `yolov8n.engine`
|
||||
|
||||
|
||||
| Precision | Eval test | mean<br>(ms) | min \| max<br>(ms) | mAP<sup>val</sup><br>50(B) | mAP<sup>val</sup><br>50-95(B) | `batch` | size<br><sup>(pixels)</sup> |
|
||||
|-----------|--------------|--------------|--------------------|----------------------|-------------------------|---------|-----------------------|
|
||||
| FP32 | Predict | 1.76 | 1.69 \| 1.87 | | | 8 | 640 |
|
||||
| FP32 | COCO<sup>val</sup> | 1.94 | | 0.52 | 0.37 | 1 | 640 |
|
||||
| FP16 | Predict | 0.86 | 0.75 \| 1.00 | | | 8 | 640 |
|
||||
| FP16 | COCO<sup>val</sup> | 1.43 | | 0.52 | 0.37 | 1 | 640 |
|
||||
| INT8 | Predict | 0.80 | 0.75 \| 1.00 | | | 8 | 640 |
|
||||
| INT8 | COCO<sup>val</sup> | 1.35 | | 0.47 | 0.33 | 1 | 640 |
|
||||
|
||||
=== "RTX 2060 6 GB"
|
||||
|
||||
Tested with Pop!_OS 22.04 LTS, `python 3.10.12`, `ultralytics==8.2.4`, `tensorrt==8.6.1.post1`
|
||||
|
||||
!!! note
|
||||
|
||||
Inference times shown for `mean`, `min` (fastest), and `max` (slowest) for each test using pretrained weights `yolov8n.engine`
|
||||
|
||||
| Precision | Eval test | mean<br>(ms) | min \| max<br>(ms) | mAP<sup>val</sup><br>50(B) | mAP<sup>val</sup><br>50-95(B) | `batch` | size<br><sup>(pixels)</sup> |
|
||||
|-----------|--------------|--------------|--------------------|----------------------|-------------------------|---------|-----------------------|
|
||||
| FP32 | Predict | 2.84 | 2.84 \| 2.85 | | | 8 | 640 |
|
||||
| FP32 | COCO<sup>val</sup> | 2.94 | | 0.52 | 0.37 | 1 | 640 |
|
||||
| FP16 | Predict | 1.09 | 1.09 \| 1.10 | | | 8 | 640 |
|
||||
| FP16 | COCO<sup>val</sup> | 1.20 | | 0.52 | 0.37 | 1 | 640 |
|
||||
| INT8 | Predict | 0.75 | 0.74 \| 0.75 | | | 8 | 640 |
|
||||
| INT8 | COCO<sup>val</sup> | 0.76 | | 0.47 | 0.33 | 1 | 640 |
|
||||
|
||||
### Embedded Devices
|
||||
|
||||
!!! tip "Detection Performance (COCO)"
|
||||
|
||||
=== "Jetson Orin NX 16GB"
|
||||
|
||||
Tested with JetPack 6.0 (L4T 36.3) Ubuntu 22.04.4 LTS, `python 3.10.12`, `ultralytics==8.2.16`, `tensorrt==10.0.1`
|
||||
|
||||
!!! note
|
||||
|
||||
Inference times shown for `mean`, `min` (fastest), and `max` (slowest) for each test using pretrained weights `yolov8n.engine`
|
||||
|
||||
| Precision | Eval test | mean<br>(ms) | min \| max<br>(ms) | mAP<sup>val</sup><br>50(B) | mAP<sup>val</sup><br>50-95(B) | `batch` | size<br><sup>(pixels)</sup> |
|
||||
|-----------|--------------|--------------|--------------------|----------------------|-------------------------|---------|-----------------------|
|
||||
| FP32 | Predict | 6.11 | 6.10 \| 6.29 | | | 8 | 640 |
|
||||
| FP32 | COCO<sup>val</sup> | 6.17 | | 0.52 | 0.37 | 1 | 640 |
|
||||
| FP16 | Predict | 3.18 | 3.18 \| 3.20 | | | 8 | 640 |
|
||||
| FP16 | COCO<sup>val</sup> | 3.19 | | 0.52 | 0.37 | 1 | 640 |
|
||||
| INT8 | Predict | 2.30 | 2.29 \| 2.35 | | | 8 | 640 |
|
||||
| INT8 | COCO<sup>val</sup> | 2.32 | | 0.46 | 0.32 | 1 | 640 |
|
||||
|
||||
!!! info
|
||||
|
||||
See our [quickstart guide on NVIDIA Jetson with Ultralytics YOLO](../guides/nvidia-jetson.md) to learn more about setup and configuration.
|
||||
|
||||
!!! info
|
||||
|
||||
See our [quickstart guide on NVIDIA DGX Spark with Ultralytics YOLO](../guides/nvidia-dgx-spark.md) to learn more about setup and configuration.
|
||||
|
||||
#### Evaluation methods
|
||||
|
||||
Expand sections below for information on how these models were exported and tested.
|
||||
|
||||
??? example "Export configurations"
|
||||
|
||||
See [export mode](../modes/export.md) for details regarding export configuration arguments.
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# TensorRT FP32
|
||||
out = model.export(format="engine", imgsz=640, dynamic=True, verbose=False, batch=8, workspace=2)
|
||||
|
||||
# TensorRT FP16
|
||||
out = model.export(format="engine", imgsz=640, dynamic=True, verbose=False, batch=8, workspace=2, half=True)
|
||||
|
||||
# TensorRT INT8 with calibration `data` (i.e. COCO, ImageNet, or DOTAv1 for appropriate model task)
|
||||
out = model.export(
|
||||
format="engine", imgsz=640, dynamic=True, verbose=False, batch=8, workspace=2, int8=True, data="coco8.yaml"
|
||||
)
|
||||
```
|
||||
|
||||
??? example "Predict loop"
|
||||
|
||||
See [predict mode](../modes/predict.md) for additional information.
|
||||
|
||||
```python
|
||||
import cv2
|
||||
|
||||
from ultralytics import YOLO
|
||||
|
||||
model = YOLO("yolo26n.engine")
|
||||
img = cv2.imread("path/to/image.jpg")
|
||||
|
||||
for _ in range(100):
|
||||
result = model.predict(
|
||||
[img] * 8, # batch=8 of the same image
|
||||
verbose=False,
|
||||
device="cuda",
|
||||
)
|
||||
```
|
||||
|
||||
??? example "Validation configuration"
|
||||
|
||||
See [`val` mode](../modes/val.md) to learn more about validation configuration arguments.
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
model = YOLO("yolo26n.engine")
|
||||
results = model.val(
|
||||
data="data.yaml", # COCO, ImageNet, or DOTAv1 for appropriate model task
|
||||
batch=1,
|
||||
imgsz=640,
|
||||
verbose=False,
|
||||
device="cuda",
|
||||
)
|
||||
```
|
||||
|
||||
## Deploying Exported YOLO26 TensorRT Models
|
||||
|
||||
Having successfully exported your Ultralytics YOLO26 models to TensorRT format, you're now ready to deploy them. For in-depth instructions on deploying your TensorRT models in various settings, take a look at the following resources:
|
||||
|
||||
- **[Deploy Ultralytics with a Triton Server](../guides/triton-inference-server.md)**: Our guide on how to use NVIDIA's Triton Inference (formerly TensorRT Inference) Server specifically for use with Ultralytics YOLO models.
|
||||
|
||||
- **[Deploying Deep Neural Networks with NVIDIA TensorRT](https://developer.nvidia.com/blog/deploying-deep-learning-nvidia-tensorrt/)**: This article explains how to use NVIDIA TensorRT to deploy deep neural networks on GPU-based deployment platforms efficiently.
|
||||
|
||||
- **[End-to-End AI for NVIDIA-Based PCs: NVIDIA TensorRT Deployment](https://developer.nvidia.com/blog/end-to-end-ai-for-nvidia-based-pcs-nvidia-tensorrt-deployment/)**: This blog post explains the use of NVIDIA TensorRT for optimizing and deploying AI models on NVIDIA-based PCs.
|
||||
|
||||
- **[GitHub Repository for NVIDIA TensorRT:](https://github.com/NVIDIA/TensorRT)**: This is the official GitHub repository that contains the source code and documentation for NVIDIA TensorRT.
|
||||
|
||||
## Summary
|
||||
|
||||
In this guide, we focused on converting Ultralytics YOLO26 models to NVIDIA's TensorRT model format. This conversion step is crucial for improving the efficiency and speed of YOLO26 models, making them more effective and suitable for diverse deployment environments.
|
||||
|
||||
For more information on usage details, take a look at the [TensorRT official documentation](https://docs.nvidia.com/deeplearning/tensorrt/).
|
||||
|
||||
If you're curious about additional Ultralytics YOLO26 integrations, our [integration guide page](../integrations/index.md) provides an extensive selection of informative resources and insights.
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I convert YOLO26 models to TensorRT format?
|
||||
|
||||
To convert your Ultralytics YOLO26 models to TensorRT format for optimized NVIDIA GPU inference, follow these steps:
|
||||
|
||||
1. **Install the required package**:
|
||||
|
||||
```bash
|
||||
pip install ultralytics
|
||||
```
|
||||
|
||||
2. **Export your YOLO26 model**:
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
model = YOLO("yolo26n.pt")
|
||||
model.export(format="engine") # creates 'yolo26n.engine'
|
||||
|
||||
# Run inference
|
||||
model = YOLO("yolo26n.engine")
|
||||
results = model("https://ultralytics.com/images/bus.jpg")
|
||||
```
|
||||
|
||||
For more details, visit the [YOLO26 Installation guide](../quickstart.md) and the [export documentation](../modes/export.md).
|
||||
|
||||
### What are the benefits of using TensorRT for YOLO26 models?
|
||||
|
||||
Using TensorRT to optimize YOLO26 models offers several benefits:
|
||||
|
||||
- **Faster Inference Speed**: TensorRT optimizes the model layers and uses precision calibration (INT8 and FP16) to speed up inference without significantly sacrificing accuracy.
|
||||
- **Memory Efficiency**: TensorRT manages tensor memory dynamically, reducing overhead and improving GPU memory utilization.
|
||||
- **Layer Fusion**: Combines multiple layers into single operations, reducing computational complexity.
|
||||
- **Kernel Auto-Tuning**: Automatically selects optimized GPU kernels for each model layer, ensuring maximum performance.
|
||||
|
||||
To learn more, explore the [official TensorRT documentation from NVIDIA](https://developer.nvidia.com/tensorrt) and our [in-depth TensorRT overview](#tensorrt).
|
||||
|
||||
### Can I use INT8 quantization with TensorRT for YOLO26 models?
|
||||
|
||||
Yes, you can export YOLO26 models using TensorRT with INT8 quantization. This process involves post-training quantization (PTQ) and calibration:
|
||||
|
||||
1. **Export with INT8**:
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
model = YOLO("yolo26n.pt")
|
||||
model.export(format="engine", batch=8, workspace=4, int8=True, data="coco.yaml")
|
||||
```
|
||||
|
||||
2. **Run inference**:
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
model = YOLO("yolo26n.engine", task="detect")
|
||||
result = model.predict("https://ultralytics.com/images/bus.jpg")
|
||||
```
|
||||
|
||||
For more details, refer to the [exporting TensorRT with INT8 quantization section](#exporting-tensorrt-with-int8-quantization).
|
||||
|
||||
### How do I deploy YOLO26 TensorRT models on an NVIDIA Triton Inference Server?
|
||||
|
||||
Deploying YOLO26 TensorRT models on an NVIDIA Triton Inference Server can be done using the following resources:
|
||||
|
||||
- **[Deploy Ultralytics YOLO26 with Triton Server](../guides/triton-inference-server.md)**: Step-by-step guidance on setting up and using Triton Inference Server.
|
||||
- **[NVIDIA Triton Inference Server Documentation](https://developer.nvidia.com/blog/deploying-deep-learning-nvidia-tensorrt/)**: Official NVIDIA documentation for detailed deployment options and configurations.
|
||||
|
||||
These guides will help you integrate YOLO26 models efficiently in various deployment environments.
|
||||
|
||||
### What are the performance improvements observed with YOLO26 models exported to TensorRT?
|
||||
|
||||
Performance improvements with TensorRT can vary based on the hardware used. Here are some typical benchmarks:
|
||||
|
||||
- **NVIDIA A100**:
|
||||
- **FP32** Inference: ~0.52 ms / image
|
||||
- **FP16** Inference: ~0.34 ms / image
|
||||
- **INT8** Inference: ~0.28 ms / image
|
||||
- Slight reduction in mAP with INT8 precision, but significant improvement in speed.
|
||||
|
||||
- **Consumer GPUs (e.g., RTX 3080)**:
|
||||
- **FP32** Inference: ~1.06 ms / image
|
||||
- **FP16** Inference: ~0.62 ms / image
|
||||
- **INT8** Inference: ~0.52 ms / image
|
||||
|
||||
Detailed performance benchmarks for different hardware configurations can be found in the [performance section](#ultralytics-yolo-tensorrt-export-performance).
|
||||
|
||||
For more comprehensive insights into TensorRT performance, refer to the [Ultralytics documentation](../modes/export.md) and our performance analysis reports.
|
||||
@@ -0,0 +1,213 @@
|
||||
---
|
||||
comments: true
|
||||
description: Learn how to export YOLO26 models to the TF GraphDef format for seamless deployment on various platforms, including mobile and web.
|
||||
keywords: YOLO26, export, TensorFlow, GraphDef, model deployment, TensorFlow Serving, TensorFlow Lite, TensorFlow.js, machine learning, AI, computer vision
|
||||
---
|
||||
|
||||
# How to Export to TF GraphDef from YOLO26 for Deployment
|
||||
|
||||
When you are deploying cutting-edge [computer vision](https://www.ultralytics.com/glossary/computer-vision-cv) models, like YOLO26, in different environments, you might run into compatibility issues. Google's [TensorFlow](https://www.ultralytics.com/glossary/tensorflow) GraphDef, or TF GraphDef, offers a solution by providing a serialized, platform-independent representation of your model. Using the TF GraphDef model format, you can deploy your YOLO26 model in environments where the complete TensorFlow ecosystem may not be available, such as mobile devices or specialized hardware.
|
||||
|
||||
In this guide, we'll walk you step by step through how to export your [Ultralytics YOLO26](https://github.com/ultralytics/ultralytics) models to the TF GraphDef model format. By converting your model, you can streamline deployment and use YOLO26's computer vision capabilities in a broader range of applications and platforms.
|
||||
|
||||
<p align="center">
|
||||
<img width="640" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/tensorflow-graphdef.avif" alt="TensorFlow GraphDef model serialization format">
|
||||
</p>
|
||||
|
||||
## Why Should You Export to TF GraphDef?
|
||||
|
||||
TF GraphDef is a powerful component of the TensorFlow ecosystem that was developed by Google. It can be used to optimize and deploy models like YOLO26. Exporting to TF GraphDef lets you move models from research to real-world applications. It allows models to run in environments without the full TensorFlow framework.
|
||||
|
||||
The GraphDef format represents the model as a serialized computation graph. This enables various optimization techniques like constant folding, quantization, and graph transformations. These optimizations ensure efficient execution, reduced memory usage, and faster inference speeds.
|
||||
|
||||
GraphDef models can use hardware accelerators such as GPUs, TPUs, and AI chips, unlocking significant performance gains for the YOLO26 inference pipeline. The TF GraphDef format creates a self-contained package with the model and its dependencies, simplifying deployment and integration into diverse systems.
|
||||
|
||||
## Key Features of TF GraphDef Models
|
||||
|
||||
TF GraphDef offers distinct features for streamlining [model deployment](https://www.ultralytics.com/glossary/model-deployment) and optimization.
|
||||
|
||||
Here's a look at its key characteristics:
|
||||
|
||||
- **Model Serialization**: TF GraphDef provides a way to serialize and store TensorFlow models in a platform-independent format. This serialized representation allows you to load and execute your models without the original Python codebase, making deployment easier.
|
||||
|
||||
- **Graph Optimization**: TF GraphDef enables the optimization of computational graphs. These optimizations can boost performance by streamlining execution flow, reducing redundancies, and tailoring operations to suit specific hardware.
|
||||
|
||||
- **Deployment Flexibility**: Models exported to the GraphDef format can be used in various environments, including resource-constrained devices, web browsers, and systems with specialized hardware. This opens up possibilities for wider deployment of your TensorFlow models.
|
||||
|
||||
- **Production Focus**: GraphDef is designed for production deployment. It supports efficient execution, serialization features, and optimizations that align with real-world use cases.
|
||||
|
||||
## Deployment Options with TF GraphDef
|
||||
|
||||
Before we dive into the process of exporting YOLO26 models to TF GraphDef, let's take a look at some typical deployment situations where this format is used.
|
||||
|
||||
Here's how you can deploy with TF GraphDef efficiently across various platforms.
|
||||
|
||||
- **TensorFlow Serving:** This framework is designed to deploy TensorFlow models in production environments. TensorFlow Serving offers model management, versioning, and the infrastructure for efficient model serving at scale. It's a seamless way to integrate your GraphDef-based models into production web services or APIs.
|
||||
|
||||
- **Mobile and Embedded Devices:** With tools like [TensorFlow Lite](../integrations/tflite.md), you can convert TF GraphDef models into formats optimized for smartphones, tablets, and various embedded devices. Your models can then be used for on-device inference, where execution is done locally, often providing performance gains and offline capabilities.
|
||||
|
||||
- **Web Browsers:** [TensorFlow.js](../integrations/tfjs.md) enables the deployment of TF GraphDef models directly within web browsers. It paves the way for real-time object detection applications running on the client side, using the capabilities of YOLO26 through JavaScript.
|
||||
|
||||
- **Specialized Hardware:** TF GraphDef's platform-agnostic nature allows it to target custom hardware, such as accelerators and TPUs (Tensor Processing Units). These devices can provide performance advantages for computationally intensive models.
|
||||
|
||||
## Exporting YOLO26 Models to TF GraphDef
|
||||
|
||||
You can convert your YOLO26 object detection model to the TF GraphDef format, which is compatible with various systems, to improve its performance across platforms.
|
||||
|
||||
### Installation
|
||||
|
||||
To install the required package, run:
|
||||
|
||||
!!! tip "Installation"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Install the required package for YOLO26
|
||||
pip install ultralytics
|
||||
```
|
||||
|
||||
For detailed instructions and best practices related to the installation process, check our [Ultralytics Installation guide](../quickstart.md). While installing the required packages for YOLO26, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips.
|
||||
|
||||
### Usage
|
||||
|
||||
All [Ultralytics YOLO26 models](../models/index.md) are designed to support export out of the box, making it easy to integrate them into your preferred deployment workflow. You can [view the full list of supported export formats and configuration options](../modes/export.md) to choose the best setup for your application.
|
||||
|
||||
!!! example "Usage"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load the YOLO26 model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Export the model to TF GraphDef format
|
||||
model.export(format="pb") # creates 'yolo26n.pb'
|
||||
|
||||
# Load the exported TF GraphDef model
|
||||
tf_graphdef_model = YOLO("yolo26n.pb")
|
||||
|
||||
# Run inference
|
||||
results = tf_graphdef_model("https://ultralytics.com/images/bus.jpg")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Export a YOLO26n PyTorch model to TF GraphDef format
|
||||
yolo export model=yolo26n.pt format=pb # creates 'yolo26n.pb'
|
||||
|
||||
# Run inference with the exported model
|
||||
yolo predict model='yolo26n.pb' source='https://ultralytics.com/images/bus.jpg'
|
||||
```
|
||||
|
||||
### Export Arguments
|
||||
|
||||
| Argument | Type | Default | Description |
|
||||
| -------- | ---------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `format` | `str` | `'pb'` | Target format for the exported model, defining compatibility with various deployment environments. |
|
||||
| `imgsz` | `int` or `tuple` | `640` | Desired image size for the model input. Can be an integer for square images or a tuple `(height, width)` for specific dimensions. |
|
||||
| `batch` | `int` | `1` | Specifies export model batch inference size or the max number of images the exported model will process concurrently in `predict` mode. |
|
||||
| `device` | `str` | `None` | Specifies the device for exporting: CPU (`device=cpu`), MPS for Apple silicon (`device=mps`). |
|
||||
|
||||
For more details about the export process, visit the [Ultralytics documentation page on exporting](../modes/export.md).
|
||||
|
||||
## Deploying Exported YOLO26 TF GraphDef Models
|
||||
|
||||
Once you've exported your YOLO26 model to the TF GraphDef format, the next step is deployment. The primary and recommended first step for running a TF GraphDef model is to use the YOLO("model.pb") method, as previously shown in the usage code snippet.
|
||||
|
||||
However, for more information on deploying your TF GraphDef models, take a look at the following resources:
|
||||
|
||||
- **[TensorFlow Serving](https://www.tensorflow.org/tfx/guide/serving)**: A guide on TensorFlow Serving that teaches how to deploy and serve [machine learning](https://www.ultralytics.com/glossary/machine-learning-ml) models efficiently in production environments.
|
||||
|
||||
- **[TensorFlow Lite](https://www.tensorflow.org/api_docs/python/tf/lite/TFLiteConverter)**: This page describes how to convert machine learning models into a format optimized for on-device inference with TensorFlow Lite.
|
||||
|
||||
- **[TensorFlow.js](https://www.tensorflow.org/js/guide/conversion)**: A guide on model conversion that teaches how to convert TensorFlow or Keras models into TensorFlow.js format for use in web applications.
|
||||
|
||||
## Summary
|
||||
|
||||
In this guide, we explored how to export Ultralytics YOLO26 models to the TF GraphDef format. By doing this, you can flexibly deploy your optimized YOLO26 models in different environments.
|
||||
|
||||
For further details on usage, visit the [TF GraphDef official documentation](https://www.tensorflow.org/api_docs/python/tf/Graph).
|
||||
|
||||
For more information on integrating Ultralytics YOLO26 with other platforms and frameworks, see our [integration guide page](index.md).
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I export a YOLO26 model to TF GraphDef format?
|
||||
|
||||
Ultralytics YOLO26 models can be exported to TensorFlow GraphDef (TF GraphDef) format seamlessly. This format provides a serialized, platform-independent representation of the model, ideal for deploying in varied environments like mobile and web. To export a YOLO26 model to TF GraphDef, follow these steps:
|
||||
|
||||
!!! example "Usage"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load the YOLO26 model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Export the model to TF GraphDef format
|
||||
model.export(format="pb") # creates 'yolo26n.pb'
|
||||
|
||||
# Load the exported TF GraphDef model
|
||||
tf_graphdef_model = YOLO("yolo26n.pb")
|
||||
|
||||
# Run inference
|
||||
results = tf_graphdef_model("https://ultralytics.com/images/bus.jpg")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Export a YOLO26n PyTorch model to TF GraphDef format
|
||||
yolo export model="yolo26n.pt" format="pb" # creates 'yolo26n.pb'
|
||||
|
||||
# Run inference with the exported model
|
||||
yolo predict model="yolo26n.pb" source="https://ultralytics.com/images/bus.jpg"
|
||||
```
|
||||
|
||||
For more information on different export options, visit the [Ultralytics documentation on model export](../modes/export.md).
|
||||
|
||||
### What are the benefits of using TF GraphDef for YOLO26 model deployment?
|
||||
|
||||
Exporting YOLO26 models to the TF GraphDef format offers multiple advantages, including:
|
||||
|
||||
1. **Platform Independence**: TF GraphDef provides a platform-independent format, allowing models to be deployed across various environments including mobile and web browsers.
|
||||
2. **Optimizations**: The format enables several optimizations, such as constant folding, quantization, and graph transformations, which enhance execution efficiency and reduce memory usage.
|
||||
3. **Hardware Acceleration**: Models in TF GraphDef format can leverage hardware accelerators like GPUs, TPUs, and AI chips for performance gains.
|
||||
|
||||
Read more about the benefits in the [TF GraphDef section](#why-should-you-export-to-tf-graphdef) of our documentation.
|
||||
|
||||
### Why should I use Ultralytics YOLO26 over other [object detection](https://www.ultralytics.com/glossary/object-detection) models?
|
||||
|
||||
Ultralytics YOLO26 offers numerous advantages compared to other models like YOLOv5 and YOLOv7. Some key benefits include:
|
||||
|
||||
1. **State-of-the-Art Performance**: YOLO26 provides exceptional speed and [accuracy](https://www.ultralytics.com/glossary/accuracy) for real-time object detection, segmentation, and classification.
|
||||
2. **Ease of Use**: Features a user-friendly API for model training, validation, prediction, and export, making it accessible for both beginners and experts.
|
||||
3. **Broad Compatibility**: Supports multiple export formats including ONNX, TensorRT, CoreML, and TensorFlow, for versatile deployment options.
|
||||
|
||||
Explore further details in our [introduction to YOLO26](../models/yolo26.md).
|
||||
|
||||
### How can I deploy a YOLO26 model on specialized hardware using TF GraphDef?
|
||||
|
||||
Once a YOLO26 model is exported to TF GraphDef format, you can deploy it across various specialized hardware platforms. Typical deployment scenarios include:
|
||||
|
||||
- **TensorFlow Serving**: Use TensorFlow Serving for scalable model deployment in production environments. It supports model management and efficient serving.
|
||||
- **Mobile Devices**: Convert TF GraphDef models to TensorFlow Lite, optimized for mobile and embedded devices, enabling on-device inference.
|
||||
- **Web Browsers**: Deploy models using TensorFlow.js for client-side inference in web applications.
|
||||
- **AI Accelerators**: Leverage TPUs and custom AI chips for accelerated inference.
|
||||
|
||||
Check the [deployment options](#deployment-options-with-tf-graphdef) section for detailed information.
|
||||
|
||||
### Where can I find solutions for common issues while exporting YOLO26 models?
|
||||
|
||||
For troubleshooting common issues with exporting YOLO26 models, Ultralytics provides comprehensive guides and resources. If you encounter problems during installation or model export, refer to:
|
||||
|
||||
- **[Common Issues Guide](../guides/yolo-common-issues.md)**: Offers solutions to frequently faced problems.
|
||||
- **[Installation Guide](../quickstart.md)**: Step-by-step instructions for setting up the required packages.
|
||||
|
||||
These resources should help you resolve most issues related to YOLO26 model export and deployment.
|
||||
@@ -0,0 +1,209 @@
|
||||
---
|
||||
comments: true
|
||||
description: Learn how to export Ultralytics YOLO26 models to TensorFlow SavedModel format for easy deployment across various platforms and environments.
|
||||
keywords: YOLO26, TF SavedModel, Ultralytics, TensorFlow, model export, model deployment, machine learning, AI
|
||||
---
|
||||
|
||||
# Understand How to Export to TF SavedModel Format From YOLO26
|
||||
|
||||
Deploying [machine learning](https://www.ultralytics.com/glossary/machine-learning-ml) models can be challenging. However, using an efficient and flexible model format can make your job easier. TF SavedModel is an open-source machine-learning framework used by TensorFlow to load machine-learning models in a consistent way. It is like a suitcase for TensorFlow models, making them easy to carry and use on different devices and systems.
|
||||
|
||||
Learning how to export to TF SavedModel from [Ultralytics YOLO26](https://github.com/ultralytics/ultralytics) models can help you deploy models easily across different platforms and environments. In this guide, we'll walk through how to convert your models to the TF SavedModel format, simplifying the process of running inferences with your models on different devices.
|
||||
|
||||
## Why Should You Export to TF SavedModel?
|
||||
|
||||
The TensorFlow SavedModel format is part of the TensorFlow ecosystem developed by Google as shown below. It is designed to save and serialize TensorFlow models seamlessly. It encapsulates the complete details of models like the architecture, weights, and even compilation information. This makes it straightforward to share, deploy, and continue training across different environments.
|
||||
|
||||
<p align="center">
|
||||
<img width="100%" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/tf-savedmodel-overview.avif" alt="TensorFlow SavedModel export format structure">
|
||||
</p>
|
||||
|
||||
The TF SavedModel has a key advantage: its compatibility. It works well with [TensorFlow Serving](https://www.tensorflow.org/tfx/guide/serving), TensorFlow Lite, and TensorFlow.js. This compatibility makes it easier to share and deploy models across various platforms, including web and mobile applications. The TF SavedModel format is useful both for research and production. It provides a unified way to manage your models, ensuring they are ready for any application.
|
||||
|
||||
## Key Features of TF SavedModels
|
||||
|
||||
Here are the key features that make TF SavedModel a great option for AI developers:
|
||||
|
||||
- **Portability**: TF SavedModel provides a language-neutral, recoverable, hermetic serialization format. They enable higher-level systems and tools to produce, consume, and transform TensorFlow models. SavedModels can be easily shared and deployed across different platforms and environments.
|
||||
|
||||
- **Ease of Deployment**: TF SavedModel bundles the computational graph, trained parameters, and necessary metadata into a single package. They can be easily loaded and used for inference without requiring the original code that built the model. This makes the deployment of TensorFlow models straightforward and efficient in various production environments.
|
||||
|
||||
- **Asset Management**: TF SavedModel supports the inclusion of external assets such as vocabularies, [embeddings](https://www.ultralytics.com/glossary/embeddings), or lookup tables. These assets are stored alongside the graph definition and variables, ensuring they are available when the model is loaded. This feature simplifies the management and distribution of models that rely on external resources.
|
||||
|
||||
## Deployment Options with TF SavedModel
|
||||
|
||||
Before we dive into the process of exporting YOLO26 models to the TF SavedModel format, let's explore some typical deployment scenarios where this format is used.
|
||||
|
||||
TF SavedModel provides a range of options to deploy your machine learning models:
|
||||
|
||||
- **TensorFlow Serving:** TensorFlow Serving is a flexible, high-performance serving system designed for production environments. It natively supports TF SavedModels, making it easy to deploy and serve your models on cloud platforms, on-premises servers, or [edge devices](https://docs.ultralytics.com/guides/raspberry-pi/).
|
||||
|
||||
- **Cloud Platforms:** Major cloud providers like [Google Cloud Platform (GCP)](https://cloud.google.com/vertex-ai), [Amazon Web Services (AWS)](https://aws.amazon.com/sagemaker/), and [Microsoft Azure](https://azure.microsoft.com/en-us/services/machine-learning/) offer services for deploying and running TensorFlow models, including TF SavedModels. These services provide scalable and managed infrastructure, allowing you to deploy and scale your models easily.
|
||||
|
||||
- **Mobile and Embedded Devices:** [TensorFlow Lite](https://docs.ultralytics.com/integrations/tflite/), a lightweight solution for running machine learning models on mobile, embedded, and IoT devices, supports converting TF SavedModels to the TensorFlow Lite format. This allows you to deploy your models on a wide range of devices, from smartphones and tablets to microcontrollers and edge devices.
|
||||
|
||||
- **TensorFlow Runtime:** TensorFlow Runtime (`tfrt`) is a high-performance runtime for executing [TensorFlow](https://www.ultralytics.com/glossary/tensorflow) graphs. It provides lower-level APIs for loading and running TF SavedModels in C++ environments. TensorFlow Runtime offers better performance compared to the standard TensorFlow runtime. It is suitable for deployment scenarios that require low-latency inference and tight integration with existing C++ codebases.
|
||||
|
||||
## Exporting YOLO26 Models to TF SavedModel
|
||||
|
||||
By exporting YOLO26 models to the TF SavedModel format, you enhance their adaptability and ease of deployment across various platforms.
|
||||
|
||||
### Installation
|
||||
|
||||
To install the required package, run:
|
||||
|
||||
!!! tip "Installation"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Install the required package for YOLO26
|
||||
pip install ultralytics
|
||||
```
|
||||
|
||||
For detailed instructions and best practices related to the installation process, check our [Ultralytics Installation guide](../quickstart.md). While installing the required packages for YOLO26, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips.
|
||||
|
||||
### Usage
|
||||
|
||||
All [Ultralytics YOLO26 models](../models/index.md) are designed to support export out of the box, making it easy to integrate them into your preferred deployment workflow. You can [view the full list of supported export formats and configuration options](../modes/export.md) to choose the best setup for your application.
|
||||
|
||||
!!! example "Usage"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load the YOLO26 model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Export the model to TF SavedModel format
|
||||
model.export(format="saved_model") # creates '/yolo26n_saved_model'
|
||||
|
||||
# Load the exported TF SavedModel model
|
||||
tf_savedmodel_model = YOLO("./yolo26n_saved_model")
|
||||
|
||||
# Run inference
|
||||
results = tf_savedmodel_model("https://ultralytics.com/images/bus.jpg")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Export a YOLO26n PyTorch model to TF SavedModel format
|
||||
yolo export model=yolo26n.pt format=saved_model # creates '/yolo26n_saved_model'
|
||||
|
||||
# Run inference with the exported model
|
||||
yolo predict model='./yolo26n_saved_model' source='https://ultralytics.com/images/bus.jpg'
|
||||
```
|
||||
|
||||
### Export Arguments
|
||||
|
||||
| Argument | Type | Default | Description |
|
||||
| -------- | ---------------- | --------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `format` | `str` | `'saved_model'` | Target format for the exported model, defining compatibility with various deployment environments. |
|
||||
| `imgsz` | `int` or `tuple` | `640` | Desired image size for the model input. Can be an integer for square images or a tuple `(height, width)` for specific dimensions. |
|
||||
| `keras` | `bool` | `False` | Enables export to Keras format, providing compatibility with TensorFlow serving and APIs. |
|
||||
| `int8` | `bool` | `False` | Activates INT8 quantization, further compressing the model and speeding up inference with minimal [accuracy](https://www.ultralytics.com/glossary/accuracy) loss, primarily for edge devices. |
|
||||
| `nms` | `bool` | `False` | Adds Non-Maximum Suppression (NMS), essential for accurate and efficient detection post-processing. |
|
||||
| `batch` | `int` | `1` | Specifies export model batch inference size or the max number of images the exported model will process concurrently in `predict` mode. |
|
||||
| `device` | `str` | `None` | Specifies the device for exporting: CPU (`device=cpu`), MPS for Apple silicon (`device=mps`). |
|
||||
|
||||
For more details about the export process, visit the [Ultralytics documentation page on exporting](../modes/export.md).
|
||||
|
||||
## Deploying Exported YOLO26 TF SavedModel Models
|
||||
|
||||
Now that you have exported your YOLO26 model to the TF SavedModel format, the next step is to deploy it. The primary and recommended first step for running a TF SavedModel model is to use the `YOLO("yolo26n_saved_model/")` method, as previously shown in the usage code snippet.
|
||||
|
||||
However, for in-depth instructions on deploying your TF SavedModel models, take a look at the following resources:
|
||||
|
||||
- **[TensorFlow Serving](https://www.tensorflow.org/tfx/guide/serving)**: Here's the developer documentation for how to deploy your TF SavedModel models using TensorFlow Serving.
|
||||
|
||||
- **[Run a TensorFlow SavedModel in Node.js](https://blog.tensorflow.org/2020/01/run-tensorflow-savedmodel-in-nodejs-directly-without-conversion.html)**: A TensorFlow blog post on running a TensorFlow SavedModel in Node.js directly without conversion.
|
||||
|
||||
- **[Deploying on Cloud](https://blog.tensorflow.org/2020/04/how-to-deploy-tensorflow-2-models-on-cloud-ai-platform.html)**: A TensorFlow blog post on deploying a TensorFlow SavedModel model on the Cloud AI Platform.
|
||||
|
||||
## Summary
|
||||
|
||||
In this guide, we explored how to export Ultralytics YOLO26 models to the TF SavedModel format. By exporting to TF SavedModel, you gain the flexibility to optimize, deploy, and scale your YOLO26 models on a wide range of platforms.
|
||||
|
||||
For further details on usage, visit the [TF SavedModel official documentation](https://www.tensorflow.org/guide/saved_model).
|
||||
|
||||
For more information on integrating Ultralytics YOLO26 with other platforms and frameworks, don't forget to check out our [integration guide page](index.md). It's packed with great resources to help you make the most of YOLO26 in your projects.
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I export an Ultralytics YOLO model to TensorFlow SavedModel format?
|
||||
|
||||
Exporting an Ultralytics YOLO model to the TensorFlow SavedModel format is straightforward. You can use either Python or CLI to achieve this:
|
||||
|
||||
!!! example "Exporting YOLO26 to TF SavedModel"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load the YOLO26 model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Export the model to TF SavedModel format
|
||||
model.export(format="saved_model") # creates '/yolo26n_saved_model'
|
||||
|
||||
# Load the exported TF SavedModel for inference
|
||||
tf_savedmodel_model = YOLO("./yolo26n_saved_model")
|
||||
results = tf_savedmodel_model("https://ultralytics.com/images/bus.jpg")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Export the YOLO26 model to TF SavedModel format
|
||||
yolo export model=yolo26n.pt format=saved_model # creates '/yolo26n_saved_model'
|
||||
|
||||
# Run inference with the exported model
|
||||
yolo predict model='./yolo26n_saved_model' source='https://ultralytics.com/images/bus.jpg'
|
||||
```
|
||||
|
||||
Refer to the [Ultralytics Export documentation](../modes/export.md) for more details.
|
||||
|
||||
### Why should I use the TensorFlow SavedModel format?
|
||||
|
||||
The TensorFlow SavedModel format offers several advantages for [model deployment](https://www.ultralytics.com/glossary/model-deployment):
|
||||
|
||||
- **Portability:** It provides a language-neutral format, making it easy to share and deploy models across different environments.
|
||||
- **Compatibility:** Integrates seamlessly with tools like TensorFlow Serving, TensorFlow Lite, and TensorFlow.js, which are essential for deploying models on various platforms, including web and mobile applications.
|
||||
- **Complete encapsulation:** Encodes the model architecture, weights, and compilation information, allowing for straightforward sharing and training continuation.
|
||||
|
||||
For more benefits and deployment options, check out the [Ultralytics YOLO model deployment options](../guides/model-deployment-options.md).
|
||||
|
||||
### What are the typical deployment scenarios for TF SavedModel?
|
||||
|
||||
TF SavedModel can be deployed in various environments, including:
|
||||
|
||||
- **TensorFlow Serving:** Ideal for production environments requiring scalable and high-performance model serving.
|
||||
- **Cloud Platforms:** Supports major cloud services like Google Cloud Platform (GCP), Amazon Web Services (AWS), and Microsoft Azure for scalable model deployment.
|
||||
- **Mobile and Embedded Devices:** Using [TensorFlow Lite](https://docs.ultralytics.com/integrations/tflite/) to convert TF SavedModels allows for deployment on mobile devices, IoT devices, and microcontrollers.
|
||||
- **TensorFlow Runtime:** For C++ environments needing low-latency inference with better performance.
|
||||
|
||||
For detailed deployment options, visit the official guides on [deploying TensorFlow models](https://www.tensorflow.org/tfx/guide/serving).
|
||||
|
||||
### How can I install the necessary packages to export YOLO26 models?
|
||||
|
||||
To export YOLO26 models, you need to install the `ultralytics` package. Run the following command in your terminal:
|
||||
|
||||
```bash
|
||||
pip install ultralytics
|
||||
```
|
||||
|
||||
For more detailed installation instructions and best practices, refer to our [Ultralytics Installation guide](../quickstart.md). If you encounter any issues, consult our [Common Issues guide](../guides/yolo-common-issues.md).
|
||||
|
||||
### What are the key features of the TensorFlow SavedModel format?
|
||||
|
||||
TF SavedModel format is beneficial for AI developers due to the following features:
|
||||
|
||||
- **Portability:** Allows sharing and deployment across various environments effortlessly.
|
||||
- **Ease of Deployment:** Encapsulates the computational graph, trained parameters, and metadata into a single package, which simplifies loading and inference.
|
||||
- **Asset Management:** Supports external assets like vocabularies, ensuring they are available when the model loads.
|
||||
|
||||
For further details, explore the [official TensorFlow documentation](https://www.tensorflow.org/guide/saved_model).
|
||||
@@ -0,0 +1,206 @@
|
||||
---
|
||||
comments: true
|
||||
description: Convert your Ultralytics YOLO26 models to TensorFlow.js for high-speed, local object detection. Learn how to optimize ML models for browser and Node.js apps.
|
||||
keywords: YOLO26, TensorFlow.js, TF.js, model export, machine learning, object detection, browser ML, Node.js, Ultralytics, YOLO, export models
|
||||
---
|
||||
|
||||
# Export to TF.js Model Format From a YOLO26 Model Format
|
||||
|
||||
Deploying [machine learning](https://www.ultralytics.com/glossary/machine-learning-ml) models directly in the browser or on Node.js can be tricky. You'll need to make sure your model format is optimized for faster performance so that the model can be used to run interactive applications locally on the user's device. The TensorFlow.js, or TF.js, model format is designed to use minimal power while delivering fast performance.
|
||||
|
||||
The 'export to TF.js model format' feature allows you to optimize your [Ultralytics YOLO26](https://github.com/ultralytics/ultralytics) models for high-speed and locally-run [object detection](https://www.ultralytics.com/glossary/object-detection) inference. In this guide, we'll walk you through converting your models to the TF.js format, making it easier for your models to perform well on various local browsers and Node.js applications.
|
||||
|
||||
## Why Should You Export to TF.js?
|
||||
|
||||
Exporting your machine learning models to TensorFlow.js, developed by the TensorFlow team as part of the broader TensorFlow ecosystem, offers numerous advantages for deploying machine learning applications. It helps enhance user privacy and security by keeping sensitive data on the device. The image below shows the TensorFlow.js architecture, and how machine learning models are converted and deployed on both web browsers and Node.js.
|
||||
|
||||
<p align="center">
|
||||
<img width="100%" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/tfjs-architecture.avif" alt="TensorFlow.js browser ML inference architecture">
|
||||
</p>
|
||||
|
||||
Running models locally also reduces latency and provides a more responsive user experience. [TensorFlow.js](https://www.ultralytics.com/glossary/tensorflow) also comes with offline capabilities, allowing users to use your application even without an internet connection. TF.js is designed for efficient execution of complex models on devices with limited resources as it is engineered for scalability, with GPU acceleration support.
|
||||
|
||||
## Key Features of TF.js
|
||||
|
||||
Here are the key features that make TF.js a powerful tool for developers:
|
||||
|
||||
- **Cross-Platform Support:** TensorFlow.js can be used in both browser and Node.js environments, providing flexibility in deployment across different platforms. It lets developers build and deploy applications more easily.
|
||||
|
||||
- **Support for Multiple Backends:** TensorFlow.js supports various backends for computation including CPU, WebGL for GPU acceleration, WebAssembly (WASM) for near-native execution speed, and WebGPU for advanced browser-based machine learning capabilities.
|
||||
|
||||
- **Offline Capabilities:** With TensorFlow.js, models can run in the browser without the need for an internet connection, making it possible to develop applications that are functional offline.
|
||||
|
||||
## Deployment Options with TensorFlow.js
|
||||
|
||||
Before we dive into the process of exporting YOLO26 models to the TF.js format, let's explore some typical deployment scenarios where this format is used.
|
||||
|
||||
TF.js provides a range of options to deploy your machine learning models:
|
||||
|
||||
- **In-Browser ML Applications:** You can build web applications that run machine learning models directly in the browser. The need for server-side computation is eliminated and the server load is reduced.
|
||||
|
||||
- **Node.js Applications:** TensorFlow.js also supports deployment in Node.js environments, enabling the development of server-side machine learning applications. It is particularly useful for applications that require the processing power of a server or access to server-side data.
|
||||
|
||||
- **Chrome Extensions:** An interesting deployment scenario is the creation of Chrome extensions with TensorFlow.js. For instance, you can develop an extension that allows users to right-click on an image within any webpage to classify it using a pretrained ML model. TensorFlow.js can be integrated into everyday web browsing experiences to provide immediate insights or augmentations based on machine learning.
|
||||
|
||||
## Exporting YOLO26 Models to TensorFlow.js
|
||||
|
||||
You can expand model compatibility and deployment flexibility by converting YOLO26 models to TF.js.
|
||||
|
||||
### Installation
|
||||
|
||||
To install the required package, run:
|
||||
|
||||
!!! tip "Installation"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Install the required package for YOLO26
|
||||
pip install ultralytics
|
||||
```
|
||||
|
||||
For detailed instructions and best practices related to the installation process, check our [Ultralytics Installation guide](../quickstart.md). While installing the required packages for YOLO26, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips.
|
||||
|
||||
### Usage
|
||||
|
||||
All [Ultralytics YOLO26 models](../models/index.md) are designed to support export out of the box, making it easy to integrate them into your preferred deployment workflow. You can [view the full list of supported export formats and configuration options](../modes/export.md) to choose the best setup for your application.
|
||||
|
||||
!!! example "Usage"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load the YOLO26 model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Export the model to TF.js format
|
||||
model.export(format="tfjs") # creates '/yolo26n_web_model'
|
||||
|
||||
# Load the exported TF.js model
|
||||
tfjs_model = YOLO("./yolo26n_web_model")
|
||||
|
||||
# Run inference
|
||||
results = tfjs_model("https://ultralytics.com/images/bus.jpg")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Export a YOLO26n PyTorch model to TF.js format
|
||||
yolo export model=yolo26n.pt format=tfjs # creates '/yolo26n_web_model'
|
||||
|
||||
# Run inference with the exported model
|
||||
yolo predict model='./yolo26n_web_model' source='https://ultralytics.com/images/bus.jpg'
|
||||
```
|
||||
|
||||
### Export Arguments
|
||||
|
||||
| Argument | Type | Default | Description |
|
||||
| -------- | ---------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `format` | `str` | `'tfjs'` | Target format for the exported model, defining compatibility with various deployment environments. |
|
||||
| `imgsz` | `int` or `tuple` | `640` | Desired image size for the model input. Can be an integer for square images or a tuple `(height, width)` for specific dimensions. |
|
||||
| `half` | `bool` | `False` | Enables FP16 (half-precision) quantization, reducing model size and potentially speeding up inference on supported hardware. |
|
||||
| `int8` | `bool` | `False` | Activates INT8 quantization, further compressing the model and speeding up inference with minimal [accuracy](https://www.ultralytics.com/glossary/accuracy) loss, primarily for edge devices. |
|
||||
| `nms` | `bool` | `False` | Adds Non-Maximum Suppression (NMS), essential for accurate and efficient detection post-processing. |
|
||||
| `batch` | `int` | `1` | Specifies export model batch inference size or the max number of images the exported model will process concurrently in `predict` mode. |
|
||||
| `device` | `str` | `None` | Specifies the device for exporting: CPU (`device=cpu`), MPS for Apple silicon (`device=mps`). |
|
||||
|
||||
For more details about the export process, visit the [Ultralytics documentation page on exporting](../modes/export.md).
|
||||
|
||||
## Deploying Exported YOLO26 TensorFlow.js Models
|
||||
|
||||
Now that you have exported your YOLO26 model to the TF.js format, the next step is to deploy it. The primary and recommended first step for running a TF.js model is to use the `YOLO("./yolo26n_web_model")` method, as previously shown in the usage code snippet.
|
||||
|
||||
However, for in-depth instructions on deploying your TF.js models, take a look at the following resources:
|
||||
|
||||
- **[Chrome Extension](https://www.tensorflow.org/js/tutorials/deployment/web_ml_in_chrome)**: Here's the developer documentation for how to deploy your TF.js models to a Chrome extension.
|
||||
|
||||
- **[Run TensorFlow.js in Node.js](https://www.tensorflow.org/js/guide/nodejs)**: A TensorFlow blog post on running TensorFlow.js in Node.js directly.
|
||||
|
||||
- **[Deploying TensorFlow.js - Node Project on Cloud Platform](https://www.tensorflow.org/js/guide/node_in_cloud)**: A TensorFlow blog post on deploying a TensorFlow.js model on a Cloud Platform.
|
||||
|
||||
## Summary
|
||||
|
||||
In this guide, we learned how to export Ultralytics YOLO26 models to the TensorFlow.js format. By exporting to TF.js, you gain the flexibility to optimize, deploy, and scale your YOLO26 models on a wide range of platforms.
|
||||
|
||||
For further details on usage, visit the [TensorFlow.js official documentation](https://www.tensorflow.org/js/guide).
|
||||
|
||||
For more information on integrating Ultralytics YOLO26 with other platforms and frameworks, don't forget to check out our [integration guide page](index.md). It's packed with great resources to help you make the most of YOLO26 in your projects.
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I export Ultralytics YOLO26 models to TensorFlow.js format?
|
||||
|
||||
Exporting Ultralytics YOLO26 models to TensorFlow.js (TF.js) format is straightforward. You can follow these steps:
|
||||
|
||||
!!! example "Usage"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load the YOLO26 model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Export the model to TF.js format
|
||||
model.export(format="tfjs") # creates '/yolo26n_web_model'
|
||||
|
||||
# Load the exported TF.js model
|
||||
tfjs_model = YOLO("./yolo26n_web_model")
|
||||
|
||||
# Run inference
|
||||
results = tfjs_model("https://ultralytics.com/images/bus.jpg")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Export a YOLO26n PyTorch model to TF.js format
|
||||
yolo export model=yolo26n.pt format=tfjs # creates '/yolo26n_web_model'
|
||||
|
||||
# Run inference with the exported model
|
||||
yolo predict model='./yolo26n_web_model' source='https://ultralytics.com/images/bus.jpg'
|
||||
```
|
||||
|
||||
For more details about supported export options, visit the [Ultralytics documentation page on deployment options](../guides/model-deployment-options.md).
|
||||
|
||||
### Why should I export my YOLO26 models to TensorFlow.js?
|
||||
|
||||
Exporting YOLO26 models to TensorFlow.js offers several advantages, including:
|
||||
|
||||
1. **Local Execution:** Models can run directly in the browser or Node.js, reducing latency and enhancing user experience.
|
||||
2. **Cross-Platform Support:** TF.js supports multiple environments, allowing flexibility in deployment.
|
||||
3. **Offline Capabilities:** Enables applications to function without an internet connection, ensuring reliability and privacy.
|
||||
4. **GPU Acceleration:** Leverages WebGL for GPU acceleration, optimizing performance on devices with limited resources.
|
||||
|
||||
For a comprehensive overview, see our [Integrations with TensorFlow.js](../integrations/tf-graphdef.md).
|
||||
|
||||
### How does TensorFlow.js benefit browser-based machine learning applications?
|
||||
|
||||
TensorFlow.js is specifically designed for efficient execution of ML models in browsers and Node.js environments. Here's how it benefits browser-based applications:
|
||||
|
||||
- **Reduces Latency:** Runs machine learning models locally, providing immediate results without relying on server-side computations.
|
||||
- **Improves Privacy:** Keeps sensitive data on the user's device, minimizing security risks.
|
||||
- **Enables Offline Use:** Models can operate without an internet connection, ensuring consistent functionality.
|
||||
- **Supports Multiple Backends:** Offers flexibility with backends like CPU, WebGL, WebAssembly (WASM), and WebGPU for varying computational needs.
|
||||
|
||||
Interested in learning more about TF.js? Check out the [official TensorFlow.js guide](https://www.tensorflow.org/js/guide).
|
||||
|
||||
### What are the key features of TensorFlow.js for deploying YOLO26 models?
|
||||
|
||||
Key features of TensorFlow.js include:
|
||||
|
||||
- **Cross-Platform Support:** TF.js can be used in both web browsers and Node.js, providing extensive deployment flexibility.
|
||||
- **Multiple Backends:** Supports CPU, WebGL for GPU acceleration, WebAssembly (WASM), and WebGPU for advanced operations.
|
||||
- **Offline Capabilities:** Models can run directly in the browser without internet connectivity, making it ideal for developing responsive web applications.
|
||||
|
||||
For deployment scenarios and more in-depth information, see our section on [Deployment Options with TensorFlow.js](#deployment-options-with-tensorflowjs).
|
||||
|
||||
### Can I deploy a YOLO26 model on server-side Node.js applications using TensorFlow.js?
|
||||
|
||||
Yes, TensorFlow.js allows the deployment of YOLO26 models on Node.js environments. This enables server-side machine learning applications that benefit from the processing power of a server and access to server-side data. Typical use cases include real-time data processing and machine learning pipelines on backend servers.
|
||||
|
||||
To get started with Node.js deployment, refer to the [Run TensorFlow.js in Node.js](https://www.tensorflow.org/js/guide/nodejs) guide from TensorFlow.
|
||||
@@ -0,0 +1,208 @@
|
||||
---
|
||||
comments: true
|
||||
description: Learn how to convert YOLO26 models to TFLite for edge device deployment. Optimize performance and ensure seamless execution on various platforms.
|
||||
keywords: YOLO26, TFLite, model export, TensorFlow Lite, edge devices, deployment, Ultralytics, machine learning, on-device inference, model optimization
|
||||
---
|
||||
|
||||
# A Guide on YOLO26 Model Export to TFLite for Deployment
|
||||
|
||||
<p align="center">
|
||||
<img width="75%" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/tflite-logo.avif" alt="TensorFlow Lite edge deployment framework">
|
||||
</p>
|
||||
|
||||
Deploying [computer vision](https://www.ultralytics.com/glossary/computer-vision-cv) models on edge devices or embedded devices requires a format that can ensure seamless performance.
|
||||
|
||||
The TensorFlow Lite or TFLite export format allows you to optimize your [Ultralytics YOLO26](https://github.com/ultralytics/ultralytics) models for tasks like [object detection](https://www.ultralytics.com/glossary/object-detection) and [image classification](https://www.ultralytics.com/glossary/image-classification) in edge device-based applications. In this guide, we'll walk through the steps for converting your models to the TFLite format, making it easier for your models to perform well on various edge devices.
|
||||
|
||||
## Why Should You Export to TFLite?
|
||||
|
||||
Introduced by Google in May 2017 as part of their TensorFlow framework, [TensorFlow Lite](https://ai.google.dev/edge/litert), or TFLite for short, is an open-source deep learning framework designed for on-device inference, also known as [edge computing](https://www.ultralytics.com/glossary/edge-computing). It gives developers the necessary tools to execute their trained models on mobile, embedded, and IoT devices, as well as traditional computers.
|
||||
|
||||
TensorFlow Lite is compatible with a wide range of platforms, including embedded Linux, Android, iOS, and microcontrollers (MCUs). Exporting your model to TFLite makes your applications faster, more reliable, and capable of running offline.
|
||||
|
||||
## Key Features of TFLite Models
|
||||
|
||||
TFLite models offer a wide range of key features that enable on-device machine learning by helping developers run their models on mobile, embedded, and edge devices:
|
||||
|
||||
- **On-device Optimization**: TFLite optimizes for on-device ML, reducing latency by processing data locally, enhancing privacy by not transmitting personal data, and minimizing model size to save space.
|
||||
|
||||
- **Multiple Platform Support**: TFLite offers extensive platform compatibility, supporting Android, iOS, embedded Linux, and microcontrollers.
|
||||
|
||||
- **Diverse Language Support**: TFLite is compatible with various programming languages, including Java, Swift, Objective-C, C++, and Python.
|
||||
|
||||
- **High Performance**: Achieves superior performance through hardware acceleration and model optimization.
|
||||
|
||||
## Deployment Options in TFLite
|
||||
|
||||
Before we look at the code for exporting YOLO26 models to the TFLite format, let's understand how TFLite models are normally used.
|
||||
|
||||
TFLite offers various on-device deployment options for machine learning models, including:
|
||||
|
||||
- **Deploying with Android and iOS**: Both Android and iOS applications with TFLite can analyze edge-based camera feeds and sensors to detect and identify objects. TFLite also offers native iOS libraries written in [Swift](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/swift) and [Objective-C](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/objc). The architecture diagram below shows the process of deploying a trained model onto Android and iOS platforms using TensorFlow Lite.
|
||||
|
||||
<p align="center">
|
||||
<img width="75%" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/architecture-diagram-tflite-deployment.avif" alt="TensorFlow Lite deployment architecture for mobile">
|
||||
</p>
|
||||
|
||||
- **Implementing with Embedded Linux**: If running inferences on a [Raspberry Pi](https://www.raspberrypi.org/) using the [Ultralytics Guide](../guides/raspberry-pi.md) does not meet the speed requirements for your use case, you can use an exported TFLite model to accelerate inference times. Additionally, it's possible to further improve performance by utilizing a [Coral Edge TPU device](https://developers.google.com/coral).
|
||||
|
||||
- **Deploying with Microcontrollers**: TFLite models can also be deployed on microcontrollers and other devices with only a few kilobytes of memory. The core runtime just fits in 16 KB on an Arm Cortex M3 and can run many basic models. It doesn't require operating system support, any standard C or C++ libraries, or dynamic memory allocation.
|
||||
|
||||
## Export to TFLite: Converting Your YOLO26 Model
|
||||
|
||||
You can improve on-device model execution efficiency and optimize performance by converting your models to TFLite format.
|
||||
|
||||
### Installation
|
||||
|
||||
To install the required packages, run:
|
||||
|
||||
!!! tip "Installation"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Install the required package for YOLO26
|
||||
pip install ultralytics
|
||||
```
|
||||
|
||||
For detailed instructions and best practices related to the installation process, check our [Ultralytics Installation guide](../quickstart.md). While installing the required packages for YOLO26, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips.
|
||||
|
||||
### Usage
|
||||
|
||||
All [Ultralytics YOLO26 models](../models/index.md) are designed to support export out of the box, making it easy to integrate them into your preferred deployment workflow. You can [view the full list of supported export formats and configuration options](../modes/export.md) to choose the best setup for your application.
|
||||
|
||||
!!! example "Usage"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load the YOLO26 model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Export the model to TFLite format
|
||||
model.export(format="tflite") # creates 'yolo26n_float32.tflite'
|
||||
|
||||
# Load the exported TFLite model
|
||||
tflite_model = YOLO("yolo26n_float32.tflite")
|
||||
|
||||
# Run inference
|
||||
results = tflite_model("https://ultralytics.com/images/bus.jpg")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Export a YOLO26n PyTorch model to TFLite format
|
||||
yolo export model=yolo26n.pt format=tflite # creates 'yolo26n_float32.tflite'
|
||||
|
||||
# Run inference with the exported model
|
||||
yolo predict model='yolo26n_float32.tflite' source='https://ultralytics.com/images/bus.jpg'
|
||||
```
|
||||
|
||||
### Export Arguments
|
||||
|
||||
| Argument | Type | Default | Description |
|
||||
| ---------- | ---------------- | -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `format` | `str` | `'tflite'` | Target format for the exported model, defining compatibility with various deployment environments. |
|
||||
| `imgsz` | `int` or `tuple` | `640` | Desired image size for the model input. Can be an integer for square images or a tuple `(height, width)` for specific dimensions. |
|
||||
| `half` | `bool` | `False` | Enables FP16 (half-precision) quantization, reducing model size and potentially speeding up inference on supported hardware. |
|
||||
| `int8` | `bool` | `False` | Activates INT8 quantization, further compressing the model and speeding up inference with minimal [accuracy](https://www.ultralytics.com/glossary/accuracy) loss, primarily for edge devices. |
|
||||
| `nms` | `bool` | `False` | Adds Non-Maximum Suppression (NMS), essential for accurate and efficient detection post-processing. |
|
||||
| `batch` | `int` | `1` | Specifies export model batch inference size or the max number of images the exported model will process concurrently in `predict` mode. |
|
||||
| `data` | `str` | `'coco8.yaml'` | Path to the [dataset](https://docs.ultralytics.com/datasets/) configuration file (default: `coco8.yaml`), essential for quantization. |
|
||||
| `fraction` | `float` | `1.0` | Specifies the fraction of the dataset to use for INT8 quantization calibration. Allows for calibrating on a subset of the full dataset, useful for experiments or when resources are limited. If not specified with INT8 enabled, the full dataset will be used. |
|
||||
| `device` | `str` | `None` | Specifies the device for exporting: CPU (`device=cpu`), MPS for Apple silicon (`device=mps`). |
|
||||
|
||||
For more details about the export process, visit the [Ultralytics documentation page on exporting](../modes/export.md).
|
||||
|
||||
## Deploying Exported YOLO26 TFLite Models
|
||||
|
||||
After successfully exporting your Ultralytics YOLO26 models to TFLite format, you can now deploy them. The primary and recommended first step for running a TFLite model is to use the `YOLO("model.tflite")` method, as outlined in the previous usage code snippet. However, for in-depth instructions on deploying your TFLite models in various other settings, take a look at the following resources:
|
||||
|
||||
- **[Android](https://ai.google.dev/edge/litert/android)**: A quick start guide for integrating [TensorFlow](https://www.ultralytics.com/glossary/tensorflow) Lite into Android applications, providing easy-to-follow steps for setting up and running [machine learning](https://www.ultralytics.com/glossary/machine-learning-ml) models.
|
||||
|
||||
- **[iOS](https://ai.google.dev/edge/litert/ios/quickstart)**: Check out this detailed guide for developers on integrating and deploying TensorFlow Lite models in iOS applications, offering step-by-step instructions and resources.
|
||||
|
||||
- **[End-To-End Examples](https://github.com/tensorflow/examples/tree/master/lite/examples)**: This page provides an overview of various TensorFlow Lite examples, showcasing practical applications and tutorials designed to help developers implement TensorFlow Lite in their machine learning projects on mobile and edge devices.
|
||||
|
||||
## Summary
|
||||
|
||||
In this guide, we focused on how to export to TFLite format. By converting your Ultralytics YOLO26 models to TFLite model format, you can improve the efficiency and speed of YOLO26 models, making them more effective and suitable for edge computing environments.
|
||||
|
||||
For further details on usage, visit the [TFLite official documentation](https://ai.google.dev/edge/litert).
|
||||
|
||||
Also, if you're curious about other Ultralytics YOLO26 integrations, check out our [integration guide page](../integrations/index.md). You'll find plenty of helpful information and insights there.
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I export a YOLO26 model to TFLite format?
|
||||
|
||||
To export a YOLO26 model to TFLite format, you can use the Ultralytics library. First, install the required package using:
|
||||
|
||||
```bash
|
||||
pip install ultralytics
|
||||
```
|
||||
|
||||
Then, use the following code snippet to export your model:
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load the YOLO26 model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Export the model to TFLite format
|
||||
model.export(format="tflite") # creates 'yolo26n_float32.tflite'
|
||||
```
|
||||
|
||||
For CLI users, you can achieve this with:
|
||||
|
||||
```bash
|
||||
yolo export model=yolo26n.pt format=tflite # creates 'yolo26n_float32.tflite'
|
||||
```
|
||||
|
||||
For more details, visit the [Ultralytics export guide](../modes/export.md).
|
||||
|
||||
### What are the benefits of using TensorFlow Lite for YOLO26 model deployment?
|
||||
|
||||
TensorFlow Lite (TFLite) is an open-source [deep learning](https://www.ultralytics.com/glossary/deep-learning-dl) framework designed for on-device inference, making it ideal for deploying YOLO26 models on mobile, embedded, and IoT devices. Key benefits include:
|
||||
|
||||
- **On-device optimization**: Minimize latency and enhance privacy by processing data locally.
|
||||
- **Platform compatibility**: Supports Android, iOS, embedded Linux, and MCU.
|
||||
- **Performance**: Utilizes hardware acceleration to optimize model speed and efficiency.
|
||||
|
||||
To learn more, check out the [TFLite guide](https://ai.google.dev/edge/litert).
|
||||
|
||||
### Is it possible to run YOLO26 TFLite models on Raspberry Pi?
|
||||
|
||||
Yes, you can run YOLO26 TFLite models on Raspberry Pi to improve inference speeds. First, export your model to TFLite format as explained above. Then, use a tool like TensorFlow Lite Interpreter to execute the model on your Raspberry Pi.
|
||||
|
||||
For further optimizations, you might consider using [Coral Edge TPU](https://developers.google.com/coral). For detailed steps, refer to our [Raspberry Pi deployment guide](../guides/raspberry-pi.md) and the [Edge TPU integration guide](../integrations/edge-tpu.md).
|
||||
|
||||
### Can I use TFLite models on microcontrollers for YOLO26 predictions?
|
||||
|
||||
Yes, TFLite supports deployment on microcontrollers with limited resources. TFLite's core runtime requires only 16 KB of memory on an Arm Cortex M3 and can run basic YOLO26 models. This makes it suitable for deployment on devices with minimal computational power and memory.
|
||||
|
||||
To get started, visit the [TFLite Micro for Microcontrollers guide](https://ai.google.dev/edge/litert/microcontrollers/overview).
|
||||
|
||||
### What platforms are compatible with TFLite exported YOLO26 models?
|
||||
|
||||
TensorFlow Lite provides extensive platform compatibility, allowing you to deploy YOLO26 models on a wide range of devices, including:
|
||||
|
||||
- **Android and iOS**: Native support through TFLite Android and iOS libraries.
|
||||
- **Embedded Linux**: Ideal for single-board computers such as Raspberry Pi.
|
||||
- **Microcontrollers**: Suitable for MCUs with constrained resources.
|
||||
|
||||
For more information on deployment options, see our detailed [deployment guide](#deploying-exported-yolo26-tflite-models).
|
||||
|
||||
### How do I troubleshoot common issues during YOLO26 model export to TFLite?
|
||||
|
||||
If you encounter errors while exporting YOLO26 models to TFLite, common solutions include:
|
||||
|
||||
- **Check package compatibility**: Ensure you're using compatible versions of Ultralytics and TensorFlow. Refer to our [installation guide](../quickstart.md).
|
||||
- **Model support**: Verify that the specific YOLO26 model supports TFLite export by checking the Ultralytics [export documentation page](../modes/export.md).
|
||||
- **Quantization issues**: When using INT8 quantization, make sure your dataset path is correctly specified in the `data` parameter.
|
||||
|
||||
For additional troubleshooting tips, visit our [Common Issues guide](../guides/yolo-common-issues.md).
|
||||
@@ -0,0 +1,216 @@
|
||||
---
|
||||
comments: true
|
||||
description: Learn how to export Ultralytics YOLO26 models to TorchScript for flexible, cross-platform deployment. Boost performance and utilize in various environments.
|
||||
keywords: YOLO26, TorchScript, model export, Ultralytics, PyTorch, deep learning, AI deployment, cross-platform, performance optimization
|
||||
---
|
||||
|
||||
# YOLO26 Model Export to TorchScript for Quick Deployment
|
||||
|
||||
Deploying [computer vision](https://www.ultralytics.com/glossary/computer-vision-cv) models across different environments, including embedded systems, web browsers, or platforms with limited Python support, requires a flexible and portable solution. TorchScript focuses on portability and the ability to run models in environments where the entire Python framework is unavailable. This makes it ideal for scenarios where you need to deploy your computer vision capabilities across various devices or platforms.
|
||||
|
||||
Export to TorchScript to serialize your [Ultralytics YOLO26](https://github.com/ultralytics/ultralytics) models for cross-platform compatibility and streamlined deployment. In this guide, we'll show you how to export your YOLO26 models to the TorchScript format, making it easier for you to use them across a wider range of applications.
|
||||
|
||||
## Why should you export to TorchScript?
|
||||
|
||||
|
||||
|
||||
Developed by the creators of PyTorch, TorchScript is a powerful tool for optimizing and deploying PyTorch models across a variety of platforms. Exporting YOLO26 models to [TorchScript](https://docs.pytorch.org/docs/stable/jit.html) is crucial for moving from research to real-world applications. TorchScript, part of the PyTorch framework, helps make this transition smoother by allowing PyTorch models to be used in environments that don't support Python.
|
||||
|
||||
The process involves two techniques: tracing and scripting. Tracing records operations during model execution, while scripting allows for the definition of models using a subset of Python. These techniques ensure that models like YOLO26 can still work their magic even outside their usual Python environment.
|
||||
|
||||
|
||||
|
||||
TorchScript models can also be optimized through techniques such as operator fusion and refinements in memory usage, ensuring efficient execution. Another advantage of exporting to TorchScript is its potential to accelerate model execution across various hardware platforms. It creates a standalone, production-ready representation of your PyTorch model that can be integrated into C++ environments, embedded systems, or deployed in web or mobile applications.
|
||||
|
||||
## Key Features of TorchScript Models
|
||||
|
||||
TorchScript, a key part of the PyTorch ecosystem, provides powerful features for optimizing and deploying [deep learning](https://www.ultralytics.com/glossary/deep-learning-dl) models.
|
||||
|
||||
|
||||
|
||||
Here are the key features that make TorchScript a valuable tool for developers:
|
||||
|
||||
- **Static Graph Execution**: TorchScript uses a static graph representation of the model's computation, which is different from PyTorch's dynamic graph execution. In static graph execution, the computational graph is defined and compiled once before the actual execution, resulting in improved performance during inference.
|
||||
|
||||
- **Model Serialization**: TorchScript allows you to serialize PyTorch models into a platform-independent format. Serialized models can be loaded without requiring the original Python code, enabling deployment in different runtime environments.
|
||||
|
||||
- **JIT Compilation**: TorchScript uses Just-In-Time (JIT) compilation to convert PyTorch models into an optimized intermediate representation. JIT compiles the model's computational graph, enabling efficient execution on target devices.
|
||||
|
||||
- **Cross-Language Integration**: With TorchScript, you can export PyTorch models to other languages such as C++, Java, and JavaScript. This makes it easier to integrate PyTorch models into existing software systems written in different languages.
|
||||
|
||||
- **Gradual Conversion**: TorchScript provides a gradual conversion approach, allowing you to incrementally convert parts of your PyTorch model into TorchScript. This flexibility is particularly useful when dealing with complex models or when you want to optimize specific portions of the code.
|
||||
|
||||
## Deployment Options in TorchScript
|
||||
|
||||
Before we look at the code for exporting YOLO26 models to the TorchScript format, let's understand where TorchScript models are normally used.
|
||||
|
||||
TorchScript offers various deployment options for [machine learning](https://www.ultralytics.com/glossary/machine-learning-ml) models, such as:
|
||||
|
||||
- **C++ API**: The most common use case for TorchScript is its C++ API, which allows you to load and execute optimized TorchScript models directly within C++ applications. This is ideal for production environments where Python may not be suitable or available. The C++ API offers low-overhead and efficient execution of TorchScript models, maximizing performance potential.
|
||||
|
||||
- **Mobile Deployment**: TorchScript offers tools for converting models into formats readily deployable on mobile devices. PyTorch Mobile provides a runtime for executing these models within iOS and Android apps. This enables low-latency, offline inference capabilities, enhancing user experience and [data privacy](https://www.ultralytics.com/glossary/data-privacy).
|
||||
|
||||
- **Cloud Deployment**: TorchScript models can be deployed to cloud-based servers using solutions like TorchServe. It provides features like model versioning, batching, and metrics monitoring for scalable deployment in production environments. Cloud deployment with TorchScript can make your models accessible via APIs or other web services.
|
||||
|
||||
## Export to TorchScript: Converting Your YOLO26 Model
|
||||
|
||||
Exporting YOLO26 models to TorchScript makes it easier to use them in different places and helps them run faster and more efficiently. This is great for anyone looking to use deep learning models more effectively in real-world applications.
|
||||
|
||||
### Installation
|
||||
|
||||
To install the required package, run:
|
||||
|
||||
!!! tip "Installation"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Install the required package for YOLO26
|
||||
pip install ultralytics
|
||||
```
|
||||
|
||||
For detailed instructions and best practices related to the installation process, check our [Ultralytics Installation guide](../quickstart.md). While installing the required packages for YOLO26, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips.
|
||||
|
||||
### Usage
|
||||
|
||||
All [Ultralytics YOLO26 models](../models/index.md) are designed to support export out of the box, making it easy to integrate them into your preferred deployment workflow. You can [view the full list of supported export formats and configuration options](../modes/export.md) to choose the best setup for your application.
|
||||
|
||||
!!! example "Usage"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load the YOLO26 model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Export the model to TorchScript format
|
||||
model.export(format="torchscript") # creates 'yolo26n.torchscript'
|
||||
|
||||
# Load the exported TorchScript model
|
||||
torchscript_model = YOLO("yolo26n.torchscript")
|
||||
|
||||
# Run inference
|
||||
results = torchscript_model("https://ultralytics.com/images/bus.jpg")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Export a YOLO26n PyTorch model to TorchScript format
|
||||
yolo export model=yolo26n.pt format=torchscript # creates 'yolo26n.torchscript'
|
||||
|
||||
# Run inference with the exported model
|
||||
yolo predict model=yolo26n.torchscript source='https://ultralytics.com/images/bus.jpg'
|
||||
```
|
||||
|
||||
### Export Arguments
|
||||
|
||||
| Argument | Type | Default | Description |
|
||||
| ---------- | ---------------- | --------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `format` | `str` | `'torchscript'` | Target format for the exported model, defining compatibility with various deployment environments. |
|
||||
| `imgsz` | `int` or `tuple` | `640` | Desired image size for the model input. Can be an integer for square images or a tuple `(height, width)` for specific dimensions. |
|
||||
| `dynamic` | `bool` | `False` | Allows dynamic input sizes, enhancing flexibility in handling varying image dimensions. |
|
||||
| `optimize` | `bool` | `False` | Applies optimization for mobile devices, potentially reducing model size and improving performance. |
|
||||
| `nms` | `bool` | `False` | Adds Non-Maximum Suppression (NMS), essential for accurate and efficient detection post-processing. |
|
||||
| `batch` | `int` | `1` | Specifies export model batch inference size or the max number of images the exported model will process concurrently in `predict` mode. |
|
||||
| `device` | `str` | `None` | Specifies the device for exporting: GPU (`device=0`), CPU (`device=cpu`), MPS for Apple silicon (`device=mps`). |
|
||||
|
||||
For more details about the export process, visit the [Ultralytics documentation page on exporting](../modes/export.md).
|
||||
|
||||
## Deploying Exported YOLO26 TorchScript Models
|
||||
|
||||
After successfully exporting your Ultralytics YOLO26 models to TorchScript format, you can now deploy them. The primary and recommended first step for running a TorchScript model is to use the `YOLO("model.torchscript")` method, as outlined in the previous usage code snippet. For in-depth instructions on deploying your TorchScript models in other settings, take a look at the following resources:
|
||||
|
||||
- **[Explore Mobile Deployment](https://docs.pytorch.org/executorch/)**: The [PyTorch](https://www.ultralytics.com/glossary/pytorch) Mobile Documentation provides comprehensive guidelines for deploying models on mobile devices, ensuring your applications are efficient and responsive.
|
||||
|
||||
- **[Master Server-Side Deployment](https://docs.pytorch.org/serve/getting_started.html)**: Learn how to deploy models server-side with TorchServe, offering a step-by-step tutorial for scalable, efficient model serving.
|
||||
|
||||
- **[Implement C++ Deployment](https://docs.pytorch.org/tutorials/advanced/cpp_export.html)**: Dive into the Tutorial on Loading a TorchScript Model in C++, facilitating the integration of your TorchScript models into C++ applications for enhanced performance and versatility.
|
||||
|
||||
## Summary
|
||||
|
||||
In this guide, we explored the process of exporting Ultralytics YOLO26 models to the TorchScript format. By following the provided instructions, you can optimize YOLO26 models for performance and gain the flexibility to deploy them across various platforms and environments.
|
||||
|
||||
For further details on usage, visit [TorchScript's official documentation](https://docs.pytorch.org/docs/stable/jit.html).
|
||||
|
||||
Also, if you'd like to know more about other Ultralytics YOLO26 integrations, visit our [integration guide page](../integrations/index.md). You'll find plenty of useful resources and insights there.
|
||||
|
||||
## FAQ
|
||||
|
||||
### What is Ultralytics YOLO26 model export to TorchScript?
|
||||
|
||||
Exporting an Ultralytics YOLO26 model to TorchScript allows for flexible, cross-platform deployment. TorchScript, a part of the PyTorch ecosystem, facilitates the serialization of models, which can then be executed in environments that lack Python support. This makes it ideal for deploying models on embedded systems, C++ environments, mobile applications, and even web browsers. Exporting to TorchScript enables efficient performance and wider applicability of your YOLO26 models across diverse platforms.
|
||||
|
||||
### How can I export my YOLO26 model to TorchScript using Ultralytics?
|
||||
|
||||
To export a YOLO26 model to TorchScript, you can use the following example code:
|
||||
|
||||
!!! example "Usage"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load the YOLO26 model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Export the model to TorchScript format
|
||||
model.export(format="torchscript") # creates 'yolo26n.torchscript'
|
||||
|
||||
# Load the exported TorchScript model
|
||||
torchscript_model = YOLO("yolo26n.torchscript")
|
||||
|
||||
# Run inference
|
||||
results = torchscript_model("https://ultralytics.com/images/bus.jpg")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Export a YOLO26n PyTorch model to TorchScript format
|
||||
yolo export model=yolo26n.pt format=torchscript # creates 'yolo26n.torchscript'
|
||||
|
||||
# Run inference with the exported model
|
||||
yolo predict model=yolo26n.torchscript source='https://ultralytics.com/images/bus.jpg'
|
||||
```
|
||||
|
||||
For more details about the export process, refer to the [Ultralytics documentation on exporting](../modes/export.md).
|
||||
|
||||
### Why should I use TorchScript for deploying YOLO26 models?
|
||||
|
||||
Using TorchScript for deploying YOLO26 models offers several advantages:
|
||||
|
||||
- **Portability**: Exported models can run in environments without the need for Python, such as C++ applications, embedded systems, or mobile devices.
|
||||
- **Optimization**: TorchScript supports static graph execution and Just-In-Time (JIT) compilation, which can optimize model performance.
|
||||
- **Cross-Language Integration**: TorchScript models can be integrated into other programming languages, enhancing flexibility and expandability.
|
||||
- **Serialization**: Models can be serialized, allowing for platform-independent loading and inference.
|
||||
|
||||
For more insights into deployment, visit the [PyTorch Mobile Documentation](https://docs.pytorch.org/executorch/), [TorchServe Documentation](https://docs.pytorch.org/serve/getting_started.html), and [C++ Deployment Guide](https://docs.pytorch.org/tutorials/advanced/cpp_export.html).
|
||||
|
||||
### What are the installation steps for exporting YOLO26 models to TorchScript?
|
||||
|
||||
To install the required package for exporting YOLO26 models, use the following command:
|
||||
|
||||
!!! tip "Installation"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Install the required package for YOLO26
|
||||
pip install ultralytics
|
||||
```
|
||||
|
||||
For detailed instructions, visit the [Ultralytics Installation guide](../quickstart.md). If any issues arise during installation, consult the [Common Issues guide](../guides/yolo-common-issues.md).
|
||||
|
||||
### How do I deploy my exported TorchScript YOLO26 models?
|
||||
|
||||
After exporting YOLO26 models to the TorchScript format, you can deploy them across a variety of platforms:
|
||||
|
||||
- **C++ API**: Ideal for low-overhead, highly efficient production environments.
|
||||
- **Mobile Deployment**: Use [PyTorch Mobile](https://docs.pytorch.org/executorch/) for iOS and Android applications.
|
||||
- **Cloud Deployment**: Utilize services like [TorchServe](https://docs.pytorch.org/serve/getting_started.html) for scalable server-side deployment.
|
||||
|
||||
Explore comprehensive guidelines for deploying models in these settings to take full advantage of TorchScript's capabilities.
|
||||
@@ -0,0 +1,257 @@
|
||||
---
|
||||
comments: true
|
||||
description: An overview of how the Ultralytics-Snippets extension for Visual Studio Code can help developers accelerate their work with the Ultralytics Python package.
|
||||
keywords: Visual Studio Code, VS Code, deep learning, convolutional neural networks, computer vision, Python, code snippets, Ultralytics, developer productivity, machine learning, YOLO, developers, productivity, efficiency, learning, programming, IDE, code editor, developer utilities, programming tools
|
||||
---
|
||||
|
||||
# Ultralytics VS Code Extension
|
||||
|
||||
<p align="center">
|
||||
<br>
|
||||
<iframe loading="lazy" width="720" height="405" src="https://www.youtube.com/embed/EXIpyYVEjoI"
|
||||
title="YouTube video player" frameborder="0"
|
||||
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"
|
||||
allowfullscreen>
|
||||
</iframe>
|
||||
<br>
|
||||
<strong>Watch:</strong> How to use Ultralytics Visual Studio Code Extension | Ready-to-Use Code Snippets | Ultralytics YOLO 🎉
|
||||
</p>
|
||||
|
||||
## Features and Benefits
|
||||
|
||||
✅ Are you a data scientist or [machine learning](https://www.ultralytics.com/glossary/machine-learning-ml) engineer building computer vision applications with Ultralytics?
|
||||
|
||||
✅ Do you despise writing the same blocks of code repeatedly?
|
||||
|
||||
✅ Are you always forgetting the arguments or default values for the [export](../modes/export.md), [predict](../modes/predict.md), [train](../modes/train.md), [track](../modes/track.md), or [val](../modes/val.md) methods?
|
||||
|
||||
✅ Looking to get started with Ultralytics and wish you had an _easier_ way to reference or run code examples?
|
||||
|
||||
✅ Want to speed up your development cycle when working with Ultralytics?
|
||||
|
||||
If you use Visual Studio Code and answered 'yes' to any of the above, then the Ultralytics-snippets extension for VS Code is here to help! Read on to learn more about the extension, how to install it, and how to use it.
|
||||
|
||||
<p align="center">
|
||||
<br>
|
||||
<img src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/snippet-prediction-preview.avif" alt="Snippet Prediction Preview">
|
||||
<br>
|
||||
Run example code using Ultralytics YOLO in under 20 seconds! 🚀
|
||||
</p>
|
||||
|
||||
## Inspired by the Ultralytics Community
|
||||
|
||||
The inspiration to build this extension came from the Ultralytics Community. Questions from the Community around similar topics and examples fueled the development for this project. Additionally, many members of the Ultralytics team use VS Code to accelerate their own work ⚡.
|
||||
|
||||
## Why VS Code?
|
||||
|
||||
[Visual Studio Code](https://code.visualstudio.com/) is extremely popular with developers worldwide and has ranked most popular by the Stack Overflow Developer Survey in [2021](https://survey.stackoverflow.co/2021#section-most-popular-technologies-integrated-development-environment), [2022](https://survey.stackoverflow.co/2022/#section-most-popular-technologies-integrated-development-environment), [2023](https://survey.stackoverflow.co/2023/#section-most-popular-technologies-integrated-development-environment), and [2024](https://survey.stackoverflow.co/2024/technology#1-integrated-development-environment). Due to VS Code's high level of customization, built-in features, broad compatibility, and extensibility, it's no surprise that so many developers are using it. Given the popularity in the wider developer community and within the Ultralytics [Discord](https://discord.com/invite/ultralytics), [Discourse](https://community.ultralytics.com/), [Reddit](https://www.reddit.com/r/ultralytics/), and [GitHub](https://github.com/ultralytics) Communities, it made sense to build a VS Code extension to help streamline your workflow and boost your productivity.
|
||||
|
||||
Want to let us know what you use for developing code? Head over to our Discourse [community poll](https://community.ultralytics.com/t/what-do-you-use-to-write-code/89/1) and let us know! While you're there, maybe check out some of our favorite computer vision, machine learning, AI, and developer [memes](https://community.ultralytics.com/c/off-topic/memes-jokes/11), or even post your favorite!
|
||||
|
||||
## Installing the Extension
|
||||
|
||||
!!! note
|
||||
|
||||
Any code environment that will allow for installing VS Code extensions _should be_ compatible with the Ultralytics-snippets extension. After publishing the extension, it was discovered that [neovim](https://neovim.io/) can be made compatible with VS Code extensions. To learn more see the [`neovim` install section](https://github.com/Burhan-Q/ultralytics-snippets?tab=readme-ov-file#use-with-neovim) of the Readme in the [Ultralytics-Snippets repository](https://github.com/Burhan-Q/ultralytics-snippets).
|
||||
|
||||
### Installing in VS Code
|
||||
|
||||
1. Navigate to the [Extensions menu in VS Code](https://code.visualstudio.com/docs/editor/extension-marketplace) or use the shortcut <kbd>Ctrl</kbd>+<kbd>Shift ⇑</kbd>+<kbd>x</kbd>, and search for Ultralytics-snippets.
|
||||
|
||||
2. Click the <kbd>Install</kbd> button.
|
||||
|
||||
<p align="center">
|
||||
<br>
|
||||
<img src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/vs-code-extension-menu.avif" alt="VS Code extension menu">
|
||||
<br>
|
||||
</p>
|
||||
|
||||
### Installing from the VS Code Extension Marketplace
|
||||
|
||||
1. Visit the [VS Code Extension Marketplace](https://marketplace.visualstudio.com/VSCode) and search for Ultralytics-snippets or go straight to the [extension page on the VS Code marketplace](https://marketplace.visualstudio.com/items?itemName=Ultralytics.ultralytics-snippets).
|
||||
|
||||
2. Click the <kbd>Install</kbd> button and allow your browser to launch a VS Code session.
|
||||
|
||||
3. Follow any prompts to install the extension.
|
||||
|
||||
<p align="center">
|
||||
<br>
|
||||
<img src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/vscode-marketplace-extension-install.avif" alt="VS Code marketplace extension install">
|
||||
<br>
|
||||
Visual Studio Code Extension Marketplace page for <a href="https://marketplace.visualstudio.com/items?itemName=Ultralytics.ultralytics-snippets">Ultralytics-Snippets</a>
|
||||
</p>
|
||||
|
||||
## Using the Ultralytics-Snippets Extension
|
||||
|
||||
- 🧠 **Intelligent Code Completion:** Write code faster and more accurately with advanced code completion suggestions tailored to the Ultralytics API.
|
||||
|
||||
- ⌛ **Increased Development Speed:** Save time by eliminating repetitive coding tasks and leveraging pre-built code block snippets.
|
||||
|
||||
- 🔬 **Improved Code Quality:** Write cleaner, more consistent, and error-free code with intelligent code completion.
|
||||
|
||||
- 💎 **Streamlined Workflow:** Stay focused on the core logic of your project by automating common tasks.
|
||||
|
||||
### Overview
|
||||
|
||||
The extension will only operate when the [Language Mode](https://code.visualstudio.com/docs/getstarted/tips-and-tricks#_change-language-mode) is configured for Python 🐍. This is to avoid snippets from being inserted when working on any other file type. All snippets have prefix starting with `ultra`, and simply typing `ultra` in your editor after installing the extension, will display a list of possible snippets to use. You can also open the VS Code [Command Palette](https://code.visualstudio.com/docs/getstarted/userinterface#_command-palette) using <kbd>Ctrl</kbd>+<kbd>Shift ⇑</kbd>+<kbd>p</kbd> and running the command `Snippets: Insert Snippet`.
|
||||
|
||||
### Code Snippet Fields
|
||||
|
||||
Many snippets have "fields" with default placeholder values or names. For instance, output from the [predict](../modes/predict.md) method could be saved to a Python variable named `r`, `results`, `detections`, `preds` or whatever else a developer chooses, which is why snippets include "fields". Using <kbd>Tab ⇥</kbd> on your keyboard after a snippet is inserted, your cursor will move between fields quickly. Once a field is selected, typing a new variable name will change that instance, but also every other instance in the snippet code for that variable!
|
||||
|
||||
<p align="center">
|
||||
<br>
|
||||
<img src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/multi-update-field-and-options.avif" alt="Multi-update field and options">
|
||||
<br>
|
||||
After inserting snippet, renaming <code>model</code> as <code>world_model</code> updates all instances. Pressing <kbd>Tab ⇥</kbd> moves to the next field, which opens a dropdown menu and allows for selection of a model scale, and moving to the next field provides another dropdown to choose either <code>world</code> or <code>worldv2</code> model variant.
|
||||
</p>
|
||||
|
||||
### Code Snippet Completions
|
||||
|
||||
!!! tip "Even _Shorter_ Shortcuts"
|
||||
|
||||
It's **not** required to type the full prefix of the snippet, or even to start typing from the start of the snippet. See example in the image below.
|
||||
|
||||
The snippets are named in the most descriptive way possible, but this means there could be a lot to type and that would be counterproductive if the aim is to move _faster_. Luckily VS Code lets users type `ultra.example-yolo-predict`, `example-yolo-predict`, `yolo-predict`, or even `ex-yolo-p` and still reach the intended snippet option! If the intended snippet was _actually_ `ultra.example-yolo-predict-kwords`, then just using your keyboard arrows <kbd>↑</kbd> or <kbd>↓</kbd> to highlight the desired snippet and pressing <kbd>Enter ↵</kbd> or <kbd>Tab ⇥</kbd> will insert the correct block of code.
|
||||
|
||||
<p align="center">
|
||||
<br>
|
||||
<img src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/incomplete-snippet-example.avif" alt="VS Code incomplete code snippet preview">
|
||||
<br>
|
||||
Typing <code>ex-yolo-p</code> will <em>still</em> arrive at the correct snippet.
|
||||
</p>
|
||||
|
||||
### Snippet Categories
|
||||
|
||||
These are the current snippet categories available to the Ultralytics-snippets extension. More will be added in the future, so make sure to check for updates and to enable auto-updates for the extension. You can also [request additional snippets](#how-do-i-request-a-new-snippet) to be added if you feel there's any missing.
|
||||
|
||||
| Category | Starting Prefix | Description |
|
||||
| :-------- | :--------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Examples | `ultra.examples` | Example code to help learn or for getting started with Ultralytics. Examples are copies of or similar to code from documentation pages. |
|
||||
| Kwargs | `ultra.kwargs` | Speed up development by adding snippets for [train](../modes/train.md), [track](../modes/track.md), [predict](../modes/predict.md), and [val](../modes/val.md) methods with all keyword arguments and default values. |
|
||||
| Imports | `ultra.imports` | Snippets to quickly import common Ultralytics objects. |
|
||||
| Models | `ultra.yolo` | Insert code blocks for initializing various [models](../models/index.md) (`yolo`, `sam`, `rtdetr`, etc.), including dropdown configuration options. |
|
||||
| Results | `ultra.result` | Code blocks for common operations when [working with inference results](../modes/predict.md#working-with-results). |
|
||||
| Utilities | `ultra.util` | Provides quick access to common utilities that are built into the Ultralytics package, learn more about these on the [Simple Utilities page](../usage/simple-utilities.md). |
|
||||
|
||||
### Learning with Examples
|
||||
|
||||
The `ultra.examples` snippets are very useful for anyone looking to learn how to get started with the basics of working with Ultralytics YOLO. Example snippets are intended to run once inserted (some have dropdown options as well). An example of this is shown at the animation at the [top](#ultralytics-vs-code-extension) of this page, where after the snippet is inserted, all code is selected and run interactively using <kbd>Shift ⇑</kbd>+<kbd>Enter ↵</kbd>.
|
||||
|
||||
!!! example
|
||||
|
||||
Just like the animation shows at the [top](#ultralytics-vs-code-extension) of this page, you can use the snippet `ultra.example-yolo-predict` to insert the following code example. Once inserted, the only configurable option is for the model scale which can be any one of: `n`, `s`, `m`, `l`, or `x`.
|
||||
|
||||
```python
|
||||
from ultralytics import ASSETS, YOLO
|
||||
|
||||
model = YOLO("yolo26n.pt", task="detect")
|
||||
results = model(source=ASSETS / "bus.jpg")
|
||||
|
||||
for result in results:
|
||||
print(result.boxes.data)
|
||||
# result.show() # uncomment to view each result image
|
||||
```
|
||||
|
||||
### Accelerating Development
|
||||
|
||||
The aim for snippets other than the `ultra.examples` are for making development easier and quicker when working with Ultralytics. A common code block to be used in many projects, is to iterate the list of `Results` returned from using the model [predict](../modes/predict.md) method. The `ultra.result-loop` snippet can help with this.
|
||||
|
||||
!!! example
|
||||
|
||||
Using the `ultra.result-loop` will insert the following default code (including comments).
|
||||
|
||||
```python
|
||||
# reference https://docs.ultralytics.com/modes/predict/#working-with-results
|
||||
|
||||
for result in results:
|
||||
result.boxes.data # torch.Tensor array
|
||||
```
|
||||
|
||||
However, since Ultralytics supports numerous [tasks](../tasks/index.md), when [working with inference results](../modes/predict.md#working-with-results) there are other `Results` attributes that you may wish to access, which is where the [snippet fields](#code-snippet-fields) will be powerful.
|
||||
|
||||
<p align="center">
|
||||
<br>
|
||||
<img src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/results-loop-options.avif" alt="VS Code YOLO results visualization options">
|
||||
<br>
|
||||
Once tabbed to the <code>boxes</code> field, a dropdown menu appears to allow selection of another attribute as required.
|
||||
</p>
|
||||
|
||||
### Keywords Arguments
|
||||
|
||||
There are over 💯 keyword arguments for all the various Ultralytics [tasks](../tasks/index.md) and [modes](../modes/index.md)! That's a lot to remember, and it can be easy to forget if the argument is `save_frame` or `save_frames` (it's definitely `save_frames` by the way). This is where the `ultra.kwargs` snippets can help out!
|
||||
|
||||
!!! example
|
||||
|
||||
To insert the [predict](../modes/predict.md) method, including all [inference arguments](../modes/predict.md#inference-arguments), use `ultra.kwargs-predict`, which will insert the following code (including comments).
|
||||
|
||||
```python
|
||||
model.predict(
|
||||
source=src, # (str, optional) source directory for images or videos
|
||||
imgsz=640, # (int | list) input images size as int or list[w,h] for predict
|
||||
conf=0.25, # (float) minimum confidence threshold
|
||||
iou=0.7, # (float) intersection over union (IoU) threshold for NMS
|
||||
vid_stride=1, # (int) video frame-rate stride
|
||||
stream_buffer=False, # (bool) buffer incoming frames in a queue (True) or only keep the most recent frame (False)
|
||||
visualize=False, # (bool) visualize model features
|
||||
augment=False, # (bool) apply image augmentation to prediction sources
|
||||
agnostic_nms=False, # (bool) class-agnostic NMS
|
||||
classes=None, # (int | list[int], optional) filter results by class, i.e. classes=0, or classes=[0,2,3]
|
||||
retina_masks=False, # (bool) use high-resolution segmentation masks
|
||||
embed=None, # (list[int], optional) return feature vectors/embeddings from given layers
|
||||
show=False, # (bool) show predicted images and videos if environment allows
|
||||
save=True, # (bool) save prediction results
|
||||
save_frames=False, # (bool) save predicted individual video frames
|
||||
save_txt=False, # (bool) save results as .txt file
|
||||
save_conf=False, # (bool) save results with confidence scores
|
||||
save_crop=False, # (bool) save cropped images with results
|
||||
stream=False, # (bool) for processing long videos or numerous images with reduced memory usage by returning a generator
|
||||
verbose=True, # (bool) enable/disable verbose inference logging in the terminal
|
||||
)
|
||||
```
|
||||
|
||||
This snippet has fields for all the keyword arguments, but also for `model` and `src` in case you've used a different variable in your code. On each line containing a keyword argument, a brief description is included for reference.
|
||||
|
||||
### All Code Snippets
|
||||
|
||||
The best way to find out what snippets are available is to download and install the extension and try it out! If you're curious and want to take a look at the list beforehand, you can visit the [repo](https://github.com/Burhan-Q/ultralytics-snippets) or [extension page on the VS Code marketplace](https://marketplace.visualstudio.com/items?itemName=Ultralytics.ultralytics-snippets) to view the tables for all available snippets.
|
||||
|
||||
## Conclusion
|
||||
|
||||
The Ultralytics-Snippets extension for VS Code is designed to empower data scientists and machine learning engineers to build [computer vision](https://www.ultralytics.com/glossary/computer-vision-cv) applications using Ultralytics YOLO more efficiently. By providing pre-built code snippets and useful examples, we help you focus on what matters most: creating innovative solutions. Please share your feedback by visiting the [extension page on the VS Code marketplace](https://marketplace.visualstudio.com/items?itemName=Ultralytics.ultralytics-snippets) and leaving a review. ⭐
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I request a new snippet?
|
||||
|
||||
New snippets can be requested using the Issues on the Ultralytics-Snippets [repo](https://github.com/Burhan-Q/ultralytics-snippets).
|
||||
|
||||
### How much does the Ultralytics-Extension Cost?
|
||||
|
||||
It's 100% free!
|
||||
|
||||
### Why don't I see a code snippet preview?
|
||||
|
||||
VS Code uses the key combination <kbd>Ctrl</kbd>+<kbd>Space</kbd> to show more/less information in the preview window. If you're not seeing a snippet preview when you type in a code snippet prefix, using this key combination should restore the preview.
|
||||
|
||||
### How do I disable the extension recommendation in Ultralytics?
|
||||
|
||||
If you use VS Code and have started to see a message prompting you to install the Ultralytics-snippets extension, and don't want to see the message any more, there are two ways to disable this message.
|
||||
|
||||
1. Install Ultralytics-snippets and the message will no longer be shown 😆!
|
||||
|
||||
2. You can be using `yolo settings vscode_msg False` to disable the message from showing without having to install the extension. You can learn more about the [Ultralytics Settings](../quickstart.md#ultralytics-settings) on the [quickstart](../quickstart.md) page if you're unfamiliar.
|
||||
|
||||
### I have an idea for a new Ultralytics code snippet, how can I get one added?
|
||||
|
||||
Visit the Ultralytics-snippets [repo](https://github.com/Burhan-Q/ultralytics-snippets) and open an Issue or Pull Request!
|
||||
|
||||
### How do I uninstall the Ultralytics-Snippets Extension?
|
||||
|
||||
Like any other VS Code extension, you can uninstall it by navigating to the Extensions menu in VS Code. Find the Ultralytics-snippets extension in the menu and click the cog icon (⚙) and then click on "Uninstall" to remove the extension.
|
||||
|
||||
<p align="center">
|
||||
<br>
|
||||
<img src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/vscode-extension-menu.avif" alt="VS Code extension menu">
|
||||
<br>
|
||||
</p>
|
||||
@@ -0,0 +1,248 @@
|
||||
---
|
||||
comments: true
|
||||
description: Learn how to enhance YOLO26 experiment tracking and visualization with Weights & Biases for better model performance and management.
|
||||
keywords: YOLO26, Weights & Biases, model training, experiment tracking, Ultralytics, machine learning, computer vision, model visualization
|
||||
---
|
||||
|
||||
# YOLO Experiment Tracking and Visualization with Weights & Biases
|
||||
|
||||
[Object detection](https://www.ultralytics.com/glossary/object-detection) models like [Ultralytics YOLO26](https://github.com/ultralytics/ultralytics) have become integral to many [computer vision](https://www.ultralytics.com/glossary/computer-vision-cv) applications. However, training, evaluating, and deploying these complex models introduce several challenges. Tracking key training metrics, comparing model variants, analyzing model behavior, and detecting issues require significant instrumentation and experiment management.
|
||||
|
||||
<p align="center">
|
||||
<br>
|
||||
<iframe loading="lazy" width="720" height="405" src="https://www.youtube.com/embed/EeDd5P4eS6A"
|
||||
title="YouTube video player" frameborder="0"
|
||||
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"
|
||||
allowfullscreen>
|
||||
</iframe>
|
||||
<br>
|
||||
<strong>Watch:</strong> How to use Ultralytics YOLO26 with Weights and Biases
|
||||
</p>
|
||||
|
||||
This guide showcases Ultralytics YOLO26 integration with Weights & Biases for enhanced experiment tracking, model-checkpointing, and visualization of model performance. It also includes instructions for setting up the integration, training, fine-tuning, and visualizing results using Weights & Biases' interactive features.
|
||||
|
||||
## Weights & Biases
|
||||
|
||||
<p align="center">
|
||||
<img width="800" src="https://cdn.jsdelivr.net/gh/ultralytics/assets@main/docs/wandb-demo-experiments.avif" alt="Weights and Biases experiment tracking">
|
||||
</p>
|
||||
|
||||
[Weights & Biases](https://wandb.ai/site) is a cutting-edge [MLOps platform](https://www.ultralytics.com/glossary/machine-learning-operations-mlops) designed for tracking, visualizing, and managing [machine learning](https://www.ultralytics.com/glossary/machine-learning-ml) experiments. It features automatic logging of training metrics for full experiment reproducibility, an interactive UI for streamlined data analysis, and efficient model management tools for deploying across various environments.
|
||||
|
||||
## YOLO26 Training with Weights & Biases
|
||||
|
||||
You can use Weights & Biases to bring efficiency and automation to your YOLO26 training process. The integration allows you to track experiments, compare models, and make data-driven decisions to improve your [computer vision](https://www.ultralytics.com/glossary/computer-vision-cv) projects.
|
||||
|
||||
## Installation
|
||||
|
||||
To install the required packages, run:
|
||||
|
||||
!!! tip "Installation"
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Install the required packages for Ultralytics YOLO and Weights & Biases
|
||||
pip install -U ultralytics wandb
|
||||
|
||||
# Enable W&B logging for Ultralytics
|
||||
yolo settings wandb=True
|
||||
```
|
||||
|
||||
For detailed instructions and best practices related to the installation process, be sure to check our [YOLO26 Installation guide](../quickstart.md). While installing the required packages for YOLO26, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips.
|
||||
|
||||
## Configuring Weights & Biases
|
||||
|
||||
After installing the necessary packages, the next step is to set up your Weights & Biases environment. This includes creating a Weights & Biases account and obtaining the necessary API key for a smooth connection between your development environment and the W&B platform.
|
||||
|
||||
Start by initializing the Weights & Biases environment in your workspace. You can do this by running the following command and following the prompted instructions.
|
||||
|
||||
!!! tip "Initial SDK Setup"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
import wandb
|
||||
|
||||
# Initialize your Weights & Biases environment
|
||||
wandb.login(key="YOUR_API_KEY")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Initialize your Weights & Biases environment
|
||||
wandb login
|
||||
```
|
||||
|
||||
Navigate to the [Weights & Biases authorization page](https://wandb.ai/authorize) to create and retrieve your API key. Use this key when prompted to authenticate your environment with W&B.
|
||||
|
||||
## Usage: Training YOLO26 with Weights & Biases
|
||||
|
||||
Before diving into the usage instructions for YOLO26 model training with Weights & Biases, be sure to check out the range of [YOLO26 models offered by Ultralytics](../models/index.md). This will help you choose the most appropriate model for your project requirements.
|
||||
|
||||
!!! example "Usage: Training YOLO26 with Weights & Biases"
|
||||
|
||||
=== "Python"
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
# Load a YOLO model
|
||||
model = YOLO("yolo26n.pt")
|
||||
|
||||
# Train and Fine-Tune the Model
|
||||
model.train(data="coco8.yaml", epochs=5, project="ultralytics", name="yolo26n")
|
||||
```
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Train a YOLO26 model with Weights & Biases
|
||||
yolo train data=coco8.yaml epochs=5 project=ultralytics name=yolo26n
|
||||
```
|
||||
|
||||
### W&B Arguments
|
||||
|
||||
| Argument | Default | Description |
|
||||
| -------- | ------- | ------------------------------------------------------------------------------------------------------------------ |
|
||||
| project | `None` | Specifies the name of the project logged locally and in W&B. This way you can group multiple runs together. |
|
||||
| name | `None` | The name of the training run. This determines the name used to create subfolders and the name used for W&B logging |
|
||||
|
||||
!!! tip "Enable or Disable Weights & Biases"
|
||||
|
||||
If you want to enable or disable Weights & Biases logging in Ultralytics, you can use the `yolo settings` command. By default, Weights & Biases logging is disabled.
|
||||
|
||||
=== "CLI"
|
||||
|
||||
```bash
|
||||
# Enable Weights & Biases logging
|
||||
yolo settings wandb=True
|
||||
|
||||
# Disable Weights & Biases logging
|
||||
yolo settings wandb=False
|
||||
```
|
||||
|
||||
### Understanding the Output
|
||||
|
||||
Upon running the usage code snippet above, you can expect the following key outputs:
|
||||
|
||||
- The setup of a new run with its unique ID, indicating the start of the training process.
|
||||
- A concise summary of the model's structure, including the number of layers and parameters.
|
||||
- Regular updates on important metrics such as box loss, cls loss, dfl loss, [precision](https://www.ultralytics.com/glossary/precision), [recall](https://www.ultralytics.com/glossary/recall), and [mAP scores](https://www.ultralytics.com/glossary/mean-average-precision-map) during each training [epoch](https://www.ultralytics.com/glossary/epoch).
|
||||
- At the end of training, detailed metrics including the model's inference speed, and overall [accuracy](https://www.ultralytics.com/glossary/accuracy) metrics are displayed.
|
||||
- Links to the Weights & Biases dashboard for in-depth analysis and visualization of the training process, along with information on local log file locations.
|
||||
|
||||
### Viewing the Weights & Biases Dashboard
|
||||
|
||||
After running the usage code snippet, you can access the Weights & Biases (W&B) dashboard through the provided link in the output. This dashboard offers a comprehensive view of your model's training process with YOLO26.
|
||||
|
||||
## Key Features of the Weights & Biases Dashboard
|
||||
|
||||
- **Real-Time Metrics Tracking**: Observe metrics like loss, accuracy, and validation scores as they evolve during the training, offering immediate insights for model tuning. [See how experiments are tracked using Weights & Biases](https://imgur.com/D6NVnmN).
|
||||
|
||||
- **Hyperparameter Optimization**: Weights & Biases aids in fine-tuning critical parameters such as [learning rate](https://www.ultralytics.com/glossary/learning-rate), [batch size](https://www.ultralytics.com/glossary/batch-size), and more, enhancing the performance of YOLO26. This helps you find the optimal configuration for your specific dataset and task.
|
||||
|
||||
- **Comparative Analysis**: The platform allows side-by-side comparisons of different training runs, essential for assessing the impact of various model configurations and understanding which changes improve performance.
|
||||
|
||||
- **Visualization of Training Progress**: Graphical representations of key metrics provide an intuitive understanding of the model's performance across epochs. [See how Weights & Biases helps you visualize validation results](https://imgur.com/a/kU5h7W4).
|
||||
|
||||
- **Resource Monitoring**: Keep track of CPU, GPU, and memory usage to optimize the efficiency of the training process and identify potential bottlenecks in your workflow.
|
||||
|
||||
- **Model Artifacts Management**: Access and share model checkpoints, facilitating easy deployment and collaboration with team members on complex projects.
|
||||
|
||||
- **Viewing Inference Results with Image Overlay**: Visualize the prediction results on images using interactive overlays in Weights & Biases, providing a clear and detailed view of model performance on real-world data. For more detailed information see Weights & Biases' [image overlay capabilities](https://docs.wandb.ai/models/track/log/media#image-overlays).
|
||||
|
||||
By using these features, you can effectively track, analyze, and optimize your YOLO26 model's training, ensuring the best possible performance and efficiency for your [object detection](https://www.ultralytics.com/glossary/object-detection) tasks.
|
||||
|
||||
## Summary
|
||||
|
||||
This guide helped you explore the Ultralytics YOLO integration with Weights & Biases. It illustrates the ability of this integration to efficiently track and visualize model training and prediction results. By leveraging W&B's powerful features, you can streamline your [machine learning](https://www.ultralytics.com/glossary/machine-learning-ml) workflow, make data-driven decisions, and improve your model's performance.
|
||||
|
||||
For further details on usage, visit [Weights & Biases' official documentation](https://docs.wandb.ai/models/integrations/ultralytics) or explore [Soumik Rakshit's presentation](https://www.ultralytics.com/blog/supercharging-ultralytics-with-weights-biases) from YOLO VISION 2023 on this integration.
|
||||
|
||||
Also, be sure to check out the [Ultralytics integration guide page](../integrations/index.md), to learn more about different exciting integrations like [MLflow](../integrations/mlflow.md) and [Comet ML](../integrations/comet.md).
|
||||
|
||||
## FAQ
|
||||
|
||||
### How do I integrate Weights & Biases with Ultralytics YOLO26?
|
||||
|
||||
To integrate Weights & Biases with Ultralytics YOLO26:
|
||||
|
||||
1. Install the required packages:
|
||||
|
||||
```bash
|
||||
pip install -U ultralytics wandb
|
||||
yolo settings wandb=True
|
||||
```
|
||||
|
||||
2. Log in to your Weights & Biases account:
|
||||
|
||||
```python
|
||||
import wandb
|
||||
|
||||
wandb.login(key="YOUR_API_KEY")
|
||||
```
|
||||
|
||||
3. Train your YOLO26 model with W&B logging enabled:
|
||||
|
||||
```python
|
||||
from ultralytics import YOLO
|
||||
|
||||
model = YOLO("yolo26n.pt")
|
||||
model.train(data="coco8.yaml", epochs=5, project="ultralytics", name="yolo26n")
|
||||
```
|
||||
|
||||
This will automatically log metrics, hyperparameters, and model artifacts to your W&B project.
|
||||
|
||||
### What are the key features of Weights & Biases integration with YOLO26?
|
||||
|
||||
The key features include:
|
||||
|
||||
- Real-time metrics tracking during training
|
||||
- Hyperparameter optimization tools
|
||||
- Comparative analysis of different training runs
|
||||
- Visualization of training progress through graphs
|
||||
- Resource monitoring (CPU, GPU, memory usage)
|
||||
- Model artifacts management and sharing
|
||||
- Viewing inference results with image overlays
|
||||
|
||||
These features help in tracking experiments, optimizing models, and collaborating more effectively on YOLO26 projects.
|
||||
|
||||
### How can I view the Weights & Biases dashboard for my YOLO26 training?
|
||||
|
||||
After running your training script with W&B integration:
|
||||
|
||||
1. A link to your W&B dashboard will be provided in the console output.
|
||||
2. Click on the link or go to [wandb.ai](https://wandb.ai/) and log in to your account.
|
||||
3. Navigate to your project to view detailed metrics, visualizations, and model performance data.
|
||||
|
||||
The dashboard offers insights into your model's training process, allowing you to analyze and improve your YOLO26 models effectively.
|
||||
|
||||
### Can I disable Weights & Biases logging for YOLO26 training?
|
||||
|
||||
Yes, you can disable W&B logging using the following command:
|
||||
|
||||
```bash
|
||||
yolo settings wandb=False
|
||||
```
|
||||
|
||||
To re-enable logging, use:
|
||||
|
||||
```bash
|
||||
yolo settings wandb=True
|
||||
```
|
||||
|
||||
This allows you to control when you want to use W&B logging without modifying your training scripts.
|
||||
|
||||
### How does Weights & Biases help in optimizing YOLO26 models?
|
||||
|
||||
Weights & Biases helps optimize YOLO26 models by:
|
||||
|
||||
1. Providing detailed visualizations of training metrics
|
||||
2. Enabling easy comparison between different model versions
|
||||
3. Offering tools for [hyperparameter tuning](https://www.ultralytics.com/glossary/hyperparameter-tuning)
|
||||
4. Allowing for collaborative analysis of model performance
|
||||
5. Facilitating easy sharing of model artifacts and results
|
||||
|
||||
These features help researchers and developers iterate faster and make data-driven decisions to improve their YOLO26 models.
|
||||
Reference in New Issue
Block a user