Kornia 0.6 — High Level Computer Vision for AI
Kornia is a modern set of Computer Vision libraries built on top of PyTorch that implements curated low level Computer Vision algorithms for the areas of robotics, camera localization and AR.
In this release we have focused on providing more high level APIs for the most demanded tasks in our community:
We provide the ImageRegistrator API from which you can leverage to align automatically two images by a process of direct optimisation using the PyTorch Autograd differentiability.
We added kornia.tracking module, which now contains HomographyTracker. You can use it to perform planar object tracking, which is useful for creating augmented reality apps. We have created an example of such application here:
Image stitching is the process of combining multiple images with overlapping fields of view to produce a segmented panorama. Here, we provide ImageStitcher to easily stitch a number of images.
You know what is common between examples of image stitching, image matching and homography tracking in kornia? They use our freshly-introduced RANSAC module for robust estimation of geometric transformations like homography and epipolar geometry. While it is not (yet!) as fancy as the latest OpenCV RANSAC, it does run on GPU in parallel mode and is good enough for many practical applications.
Kornia leverages differentiable and GPU image data augmentation through the module kornia.augmentation by implementing the functionality to be easily used with torch.nn.Sequential and other advanced containers such as AugmentationSequential, ImageSequential, PatchSequential and VideoSequential.
One can use the kornia apply for simple augmentations as follows:
Or use the differentiability to optimise the augmentations parameters as follows:
Additionally, this new release comes with a wide set of new functionalities and state-of-the-art models, operators and loss functions. Below you can find a triaged set of what you will find in the release.
- Experimental Training API with ImageClassification and SemanticSegmentation trainer classes including coding template examples that can be used as a reference to train your models.
- Video Deblurring features via DeFMO,
- Vision transformer and MobileVIT implemented in terms of generic operator to be used across different tasks or applications.
- Hausdorff distance loss for morphological erosion optimization.
- Homography metrics to evaluate or optimise.
- Connected components for classical image segmentation.
- Stereo camera API working with a horizontal stereo camera setup.
- Camera Calibration module, including image undistortion and PnP algorithms.
- Canny edge detection algorithm.
- CLAHE differentiable to equalize image using local optimization.
- Resize with antialias implementation.
- Integration with VPU via OpenCV-Luxonis programmable cameras [here].
- Added support for YUV420 and YUV422 color conversions.
- Implement separable_filter2d to boost x10 filtering operations.
- Improved our morphology package
We recommend to read carefully our release notes for 0.5.x [here]
Our current focus is on the following:
- Optimize and speed-up existing algorithms like Homography tracker, RANSAC, connected components and many others.
- Adding new features in terms of high-level algorithms, which can be plugged in user applications off-the-shelf.
- Expand the functionality of existing image matching, tracking and structure-from-motion modules.
- Work towards a Camera API.
Join the community
- Join our social network communities with 1.8k+ members:
- Twitter: we share the recent research and news for out mainstream community.
- Slack: come to us and chat with our engineers and mentors to get support and resolve your questions.
- LibreCV: its our Open Source and Machine Learning community forum. Come and have fun !
- Subscribe to our YouTube channel to get the latest video demos.
Kornia AI is a non-profit organization project without large funding in the background. If you use kornia, consider giving us support with resources, contributions and donations.
You can use our OpenCollective link for financial support: https://opencollective.com/kornia
If you need our help for adopting kornia in your commercial product, or implementing some special feature, courseware and more advanced tiers reach us at firstname.lastname@example.org