July 28, 2014

Image Detection in iOS 8

By

Prior to iOS 8, Core Image included facial recognition, allowing applications to easily detect facial features such as mouth and eye positions. This is covered in the Introduction to Core Image article in our Developer Center. In iOS 8, Apple has added additional image recognizers for detecting rectangles as well as QR codes.

CIDetectors

All of the image detectors from Core Image are available in the CIDetector class. This makes the API for doing image recognition the same regardless of which type of detector you are using. You can create an image detector via factory methods on CIDetector as listed below:

  • CIDetector.CreateRectangleDetector
  • CIDetector.CreateQRDetector
  • CIDetector.CreateFaceDetector

These methods take CIContext and CIDetectorOptions instances respectively. A CIContext orchestrates everything that happens when working with Core Image, such as applying filters and performing image detection.

CIDetectorOptions, as the name implies, allows various options to be set. For example you could set the accuracy the detector should use.

Rectangle Detector

With the CIContext and CIDetectorOptions, you can create a CIDetector, as the following code does to create a rectangle detector:

var detector = CIDetector.CreateRectangleDetector (context, options);

Any image features recognized by the detector are returned in an array of CIFeature objects when calling FeaturesInImage for a given CIImage:

var ciImage = CIImage.FromCGImage (imageIn.CGImage);
var features = detector.FeaturesInImage (ciImage);

Each feature has a Bounds property containing a rectangle that encloses the particular feature. This can be used to perform imaging operations on the detected feature. For example, say you want to highlight a feature on an image when it is recognized. You could use the Bounds to create an overlay over the original image:

var overlay = CIImage.ImageWithColor (CIColor.FromRgba (1.0f, 0.0f, 0.0f, 0.7f));
overlay = overlay.ImageByCroppingToRect (features [0].Bounds);
var ciImageWithOverlay = overlay.CreateByCompositingOverImage (ciImage);

This results in a semi-transparent overlay as shown below:

rectangle image detector

QR Code Detector

Core Image previously included the ability to generate QR codes. Now it allows QR code detection as well.

Using a QR code detector is similar to a rectangle detector. Simply create the detector, this time by calling CreateQRDetector, and proceed in a similar fashion to retrieve a CIFeature array.

var detector = CIDetector.CreateQRDetector (context, options);

The following screenshot shows a QR code being detected:

qr code detector

One thing to be aware of, as of iOS 8 beta 4, the QR code detector only works on a device. Calling CreateQRDetector in the simulator returns null.

The addition of new detectors in Core Image, with its high-performance, easy to use API, allows for new scenarios that previously required third party imaging libraries. Combined with the extensive filtering support of Core Image, it’s very easy to create imaging applications in iOS.

The code from this post is available here.

Discuss this blog post in the Xamarin Forums

TwitterFacebookGoogle+LinkedInEmail