Java opencv Tesseract OCR “code” region

I'm trying to automate someone's manual conversion of code to digital code

Then I started reading about OCR So I installed Tesseract OCR and tried some images It doesn't even detect anything close to the code

After reading some questions about stack overflow, I thought that the image needs some preprocessing, such as tilting the image to a horizontal image, which can be completed through OpenCV

Now my question is:

>In the case of the above image, which preprocessing method or other method should be used? > Second, can I rely on output? Will it work like the image above?

I hope someone can help me!

Solution

I decided to capture only the whole card, not the code By capturing the whole card, I can convert it into a simple perspective, and then I can easily get the "code" area

I also learned a lot Especially about speed This feature is slow on high resolution images The size is 3264 x 1836 and may take up to 10 seconds

What I did was to speed up the input matrix by resizing it to 1 / 4 This increases its speed by 4 ^ 2 times and gives me the least loss of accuracy The next step is to scale the quadrilateral we find back to normal size In this way, we can use the original source to convert the quadrilateral into a normal perspective

The code I created to detect the largest area is largely based on the code I found on stack overflow Unfortunately, they didn't work as I expected, so I combined more code fragments and made a lot of changes That's what I got:

private static double angle(Point p1,Point p2,Point p0 ) {
        double dx1 = p1.x - p0.x;
        double dy1 = p1.y - p0.y;
        double dx2 = p2.x - p0.x;
        double dy2 = p2.y - p0.y;
        return (dx1 * dx2 + dy1 * dy2) / Math.sqrt((dx1 * dx1 + dy1 * dy1) * (dx2 * dx2 + dy2 * dy2) + 1e-10);
    }



    private static MatOfPoint find(Mat src) throws Exception {
        Mat blurred = src.clone();
        Imgproc.medianBlur(src,blurred,9);

        Mat gray0 = new Mat(blurred.size(),CvType.CV_8U),gray = new Mat();

        List<MatOfPoint> contours = new ArrayList<>();

        List<Mat> blurredChannel = new ArrayList<>();
        blurredChannel.add(blurred);
        List<Mat> gray0Channel = new ArrayList<>();
        gray0Channel.add(gray0);

        MatOfPoint2f approxCurve;

        double maxArea = 0;
        int maxId = -1;

        for (int c = 0; c < 3; c++) {
            int ch[] = {c,0};
            Core.mixChannels(blurredChannel,gray0Channel,new MatOfInt(ch));

            int thresholdLevel = 1;
            for (int t = 0; t < thresholdLevel; t++) {
                if (t == 0) {
                    Imgproc.Canny(gray0,gray,10,20,3,true); // true ?
                    Imgproc.dilate(gray,new Mat(),new Point(-1,-1),1); // 1 ?
                } else {
                    Imgproc.adaptiveThreshold(gray0,thresholdLevel,Imgproc.ADAPTIVE_THRESH_GAUSSIAN_C,Imgproc.THRESH_BINARY,(src.width() + src.height()) / 200,t);
                }

                Imgproc.findContours(gray,contours,Imgproc.RETR_LIST,Imgproc.CHAIN_APPROX_SIMPLE);

                for (MatOfPoint contour : contours) {
                    MatOfPoint2f temp = new MatOfPoint2f(contour.toArray());

                    double area = Imgproc.contourArea(contour);
                    approxCurve = new MatOfPoint2f();
                    Imgproc.approxPolyDP(temp,approxCurve,Imgproc.arcLength(temp,true) * 0.02,true);

                    if (approxCurve.total() == 4 && area >= maxArea) {
                        double maxCosine = 0;

                        List<Point> curves = approxCurve.toList();
                        for (int j = 2; j < 5; j++)
                        {

                            double cosine = Math.abs(angle(curves.get(j % 4),curves.get(j - 2),curves.get(j - 1)));
                            maxCosine = Math.max(maxCosine,cosine);
                        }

                        if (maxCosine < 0.3) {
                            maxArea = area;
                            maxId = contours.indexOf(contour);
                            //contours.set(maxId,getHull(contour));
                        }
                    }
                }
            }
        }

        if (maxId >= 0) {
            return contours.get(maxId);
            //Imgproc.drawContours(src,maxId,new Scalar(255,.8),8);
        }
        return null;
    }

You can call it like this:

MathOfPoint contour = find(src);

View the answer of this quadrilateral detection from the outline and convert it to a simple perspective: Java opencv deskewing a contour