With so many things on technology, frameworks, and technical stuff, I found something interesting today. Thinking about working on machine learning, I thought why not detect things with machine learning, it would be interesting if it will be in some web tech stuff. I encountered a machine learning library which is tensorflow.js. I studied it and learn to have some applications. I come to know that it can be used for object detection. Let us see how it can be used.
Set up
After studying about its setup, I come to know that it can be set up in two ways:
Using script tags.
Installation from NPM and using a build tool like Parcel.
The complete guide is given in Tensorflow.js which I found good to start with. I found it easy and good to use script tags and looked at the code to add the library in the project.
The script lines of code that one can use for implementation and use of tensorflow.js in the project is
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@2.0.0/dist/tf.min.js"></script>
Program
Later for the object detection, I have used a pre-trained model called MobileNet and copied some lines of code. I come to know that Tensorflow.js model takes browser-based image elements (<img>, <video>, <canvas> elements, for example) and in output, it returns an array of most likely predicted elements with their probabilities that the predicted object is same as the one shown in the image. The main part of the code taken from MobileNet:
// Load the model.
mobilenet.load().then(model => {
// Classify the image.
model.classify(img).then(predictions => {
console.log('Predictions: ');
console.log(predictions);
});
});
Testing
Later, I tried this code with an image of a cat and it gave me correct result. It was a Persian cat in the top result with the probability of 0.9148746728897095.
Then I thought, why not to try this on a video with webcam. Later I tried by showing a video of a snail. From a file of 31.8 KB and 1000 objects, it traversed and get the result. I was lucky to detect one of the 1000 objects that this model included. (entry 113 from 0 to 999).
Additional feature
I tried the live camera input of a recorded video, via getUserMedia. At first, it was not working and giving me the error: “Requested texture size [0x0] is invalid”. But later on, searching the internet, I found the solution that is to simply set the height and width attributes of the video element. I pointed the webcam at a video and it was shown on my screen. In a short time, it produced the result. Thankfully, it worked fine.
I added buttons to start and stop the webcam along with the space for the result.
The key piece of code is small and straightforward — Tensorflow and MobileNet helped a lot in completing this project:
One thing to note is that I have tried this on Chrome and not on any other browser with the webcam.
Thank you for reading! I hope you liked it.
From
Akhil Soni
You can connect with me on