Object Detection in Android with Fritz AI
Add object detection to your mobile app using Fritz AI
In this piece, we shall focus on detecting objects in an image using Fritz AI .Fritz AI pre-trained models allow you to add object detection to your Android and iOS applications with just a few lines of code. The model will provide bounding boxes for each object detected. The models are trained on COCO, a large-scale object detection dataset. The model runs on the device, is fast, and doesn’t require any internet connection for inferencing. The model can also run on live video with a fast frame rate.
Getting Started
The first step is to create an Android project. Once you do, note the application ID. Next, log into your Fritz account and register the application. This will set up communication between your application and Fritz AI.
Click on the next button and name your application. While you are there, don’t forget to enter your application ID. You can find the ID in your app’s build.gradle file.
With that out of the way, you can now install the Fritz AI SDK. In your root-level Gradle file (build.gradle) include the Maven repository for Fritz.
Now add the dependencies for the SDK in app/build.gradle. We add the Fritz Core, Objection Detection, and Vision dependencies. Including the Objection Detection model in your application will make your application larger in size. Now that you have changed the Gradle files, ensure that you sync that with your project. That will download all the necessary dependencies.
Before you close that file add renderscript support to improve image processing performance. Also, specify aaptOptions to prevent compression of TFLite models.
Now register the FritzCustomModelService in the AndroidManifest.
The next step is to initialize the SDK by calling Fritz.configure() with your API Key.
With that in place, click next to verify that your application is able to communicate with Fritz.
Use Fritz Pre-trained Models
The model we shall use will draw a bounding box around the detected object and give us the confidence level.
The App Elements
The application is made up of a button that will choose an image and the image view.
Need help with your mobile machine learning project? Subscribe to the Fritz AI Newsletter for resources and expertise that will help you turn your idea into a production app.
Obtaining the User Permissions
We can obtain the image by overriding the onClick method of the button and attaching a click listener. We also obtain permission from the user in order to select an image.
Once we obtain the necessary permission, we get the image via the pickImage method. The method uses an Intent to get the image.
Next, override the method used to request permission.
Obtaining the Image
At this point, we can now obtain the image and create a Bitmap. The object detection model requires a Bitmap image.
Create a FritzVisionImage from an image
The next step is to create a FritzVisionImage from the Bitmap image.
Create an ObjectDetectionOnDeviceModel
Since we are using an on-device model we can obtain the predictor immediately.
Create an Object Detection Predictor
Next, define the predictor that we will use for making predictions.
Detect different objects in the image
Using this predictor, we can now run the predictions.
Displaying the result
The next step is to obtain the objects. If there are objects we obtain them and overlay the bounding box on the imageView.
Conclusion
Hopefully, this piece has shown you how easy it is to incorporate object detection capabilities in your Android application using Fritz AI. Check out the full source code in the repo below.
Editor’s Note: Heartbeat is a contributor-driven online publication and community dedicated to exploring the emerging intersection of mobile app development and machine learning. We’re committed to supporting and inspiring developers and engineers from all walks of life.
Editorially independent, Heartbeat is sponsored and published by Fritz AI, the machine learning platform that helps developers teach devices to see, hear, sense, and think. We pay our contributors, and we don’t sell ads.
If you’d like to contribute, head on over to our call for contributors. You can also sign up to receive our weekly newsletters (Deep Learning Weekly and the Fritz AI Newsletter), join us on Slack, and follow Fritz AI on Twitter for all the latest in mobile machine learning.
