In many homes, companies or hard-to-access industrial places, there are still consumption meters, electrical boxes, and many other measurement methods with a
mechanical counter that do not provide a direct, standardized interface for reading the consumption or any kind of index, with a computer.
So we need specialized time and employees to read, register and follow these measurements. As you can see this amounts to time and money…
One of our projects aims to fix this, and using a cheap embedded computer, the Raspberry Pi 2, and a few open-source libraries, we managed to make a dent in OCR(optic character recognition).
The system is in a incipient faze, and we could take it further with some form of cyclic redundancy check and a database of already trained data on different types of electrical boxes fonts.
We will try to expand an explain the system on the functional block diagram that follows:
For testing purposes and speed of developing we built our software with the ability of reading images from a file, not only from the camera interface.
We limited our image size to 640×480, so we wont need too much hard disk space or resources while processing. A higher resolution should not enhance the accuracy of the character recognition.
Once an image is loaded by the software we can start the image processing step, where we first remove colour, fuzzy or blurry edges and find the contours we are interested in.
Using Python, a very Raspberry Pi friendly scripting language, we are able to import the OpenCV interface. There are libraries for Java, C and C++ depending on your preferences and needs.
OpenCV comes with a plethora of methods and algorithms that can be used and developed further. Some of the techniques applied during our project are:
– GaussianBlur, used to reduce image noise and reduce detail;
– Canny, detection of character edges with low error rate, and only full connected edges;
– FindContours, where we only extract the contours with the right size for our case;
Once the number areas are extracted we can train the system on the data we have; note that a bigger training data sample results in a more accurate model, and less errors.
Before implementing machine learning, we tried using a different open source technology, Tesseract OCR. But this system would only work where the numbers are very clear and belong to a font that has already been introduced and learned by Tesseract. Luminosity is a big factor and only a slight change would alter the way OpenCV extracts shapes from an image.
We use a quite simple pattern recognition algorithm, k-Nearest Neighbors, used extensively in data mining and machine learning.
At this stage the user responds with the appropriate character from the keyboard, in case of a misshaped or error, we press “Space” and the computer discards that piece of data proposed, as incorrect. Here are a couple of samples from our example: as a “7” or as a “3”.
Training data is currently saved to a file locally, but we can have these samples uploaded to a database, so in the case of a new installation the system would check for already known fonts, if the luminosity is favorable that is.
During my testing of the raw system, the software had a 90% accuracy, and this can be very well improved through error checking and averaging the training data. Impressive is the low effort for hardware. In addition to the Raspberry Pi only the additional camera module was used, of course this can be replaced by a simple USB webcam.
Here are the steps taken by us, from a pristine out of the box raspberry Pi 2 to a ready to use OpenCV installation for Python:
#initial setup using raspi-config, here we are using the command line raspi-config interface
#expand the filesystem and remember to enable the camera interface(this is for the Raspberry Pi camera module, for a USB cam you dont need to do this);
>sudo reboot now
#update the system
>sudo apt-get update
>sudo apt-get upgrade
#building tools for the Raspbian linux
>sudo apt-get install build-essential git cmake pkg-config
#image input-output packages
>sudo apt-get install libjpeg-dev libtiff5-dev libjasper-dev libpng12-dev
#GTK development library – graphical interface library
>sudo apt-get install libgtk2.0-dev
#Python 2.7 and Python 3 header files
>sudo apt-get install python2.7-dev python3-dev
#get and install pip
>sudo python get-pip.py
#install numpy a fundamental package for scientific computing with Python
>pip install numpy
#grab the OpenCV version 3.0.0 and the contributor directory from the official repository and unzip it
>wget -O opencv.zip https://github.com/Itseez/opencv/archive/3.0.0.zip
>wget -O opencv_contrib.zip https://github.com/Itseez/opencv_contrib/archive/3.0.0.zip
#Change to the newly unzipped opencv folder and start the building process with GNU make
>cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=/usr/local \
-D INSTALL_C_EXAMPLES=ON \
-D INSTALL_PYTHON_EXAMPLES=ON \
-D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib-3.0.0/modules \
-D BUILD_EXAMPLES=ON ..
#this compiles opencv for Raspberry; if there are no errors go ahead and install
#links all libraries and installation files
>sudo make install
Now we have a newly Raspberry Pi, with a Python development environment and OpenCV version 3 installed.
You can test the OpenCV version at the command line, with:
>>> import cv2