Convert image in Mat to BufferedImage
I have this source code which can open image for OpenCv in Mat then do some operation then bufferedImage should convert it to image so it can be displayed in JFrame. But I can not figure out how to pass arguments.
public class objectDetection < public void getMatImg()< Mat imageInMat = Imgcodecs.imread(getClass().getResource("lena.png").getPath()); >/* * * OpenCV code */ public static BufferedImage bufferedImage(Mat m) < int type = BufferedImage.TYPE_BYTE_GRAY; if (m.channels() >1) < type = BufferedImage.TYPE_3BYTE_BGR; >BufferedImage image = new BufferedImage(m.cols(), m.rows(), type); m.get(0, 0, ((DataBufferByte)image.getRaster().getDataBuffer()).getData()); return image; > 1 Answer 1
Here’s some code we used a while ago for 2.4.8 or .9. Let me know if it works for you.
I think we may have had some problems with it when we tried to pass in an existing BufferedImage, but it worked fine if you pass in null for the BufferedImage.
/** * Converts/writes a Mat into a BufferedImage. * * @param matrix Mat of type CV_8UC3 or CV_8UC1 * @return BufferedImage of type TYPE_3BYTE_BGR or TYPE_BYTE_GRAY */ public static BufferedImage matToBufferedImage(Mat matrix, BufferedImage bimg) < if ( matrix != null ) < int cols = matrix.cols(); int rows = matrix.rows(); int elemSize = (int)matrix.elemSize(); byte[] data = new byte[cols * rows * elemSize]; int type; matrix.get(0, 0, data); switch (matrix.channels()) < case 1: type = BufferedImage.TYPE_BYTE_GRAY; break; case 3: type = BufferedImage.TYPE_3BYTE_BGR; // bgr to rgb byte b; for(int i=0; ibreak; default: return null; > // Reuse existing BufferedImage if possible if (bimg == null || bimg.getWidth() != cols || bimg.getHeight() != rows || bimg.getType() != type) < bimg = new BufferedImage(cols, rows, type); >bimg.getRaster().setDataElements(0, 0, cols, rows, data); > else < // mat was null bimg = null; >return bimg; > How to convert OpenCV Mat object to BufferedImage object using Java?
If you try to read an image using the OpenCV imread() method it returns a Mat object. If you want to display the contents of the resultant Mat object using an AWT/Swings window You need to convert the Mat object to an object of the class java.awt.image.BufferedImage. To do so, you need to follow the steps given below −
- Encode the Mat to MatOfByte − First of all, you need to convert the matrix to the matrix of a byte. You can do it using the method imencode() of the class Imgcodecs. This method accepts a String parameter(specifying the image format), a Mat object (representing the image), a MatOfByte object.
- Convert the MatOfByte object to byte array − Convert the MatOfByte object into a byte array using the method toArray().
- Instantiate ByteArrayInputStream − Instantiate the ByteArrayInputStream class by passing the byte array created in the previous step to one of its constructors.
- Creating BufferedImage object − Pass the Input Stream object created in the previous step to the read() method of the ImageIO class. This will return a BufferedImage object.
Example
import java.awt.image.BufferedImage; import java.io.ByteArrayInputStream; import java.io.IOException; import java.io.InputStream; import javax.imageio.ImageIO; import org.opencv.core.Core; import org.opencv.core.Mat; import org.opencv.core.MatOfByte; import org.opencv.imgcodecs.Imgcodecs; public class Mat2BufferedImage < public static BufferedImage Mat2BufferedImage(Mat mat) throws IOException< //Encoding the image MatOfByte matOfByte = new MatOfByte(); Imgcodecs.imencode(".jpg", mat, matOfByte); //Storing the encoded Mat in a byte array byte[] byteArray = matOfByte.toArray(); //Preparing the Buffered Image InputStream in = new ByteArrayInputStream(byteArray); BufferedImage bufImage = ImageIO.read(in); return bufImage; >public static void main(String args[]) throws Exception < //Loading the OpenCV core library System.loadLibrary( Core.NATIVE_LIBRARY_NAME ); //Reading the Image from the file String file = "C:/EXAMPLES/OpenCV/sample.jpg"; Mat image = Imgcodecs.imread(file); BufferedImage obj = Mat2BufferedImage(image); System.out.println(obj); >> Output
BufferedImage@579bb367: type = 5 ColorModel: #pixelBits = 24 numComponents = 3 color space = java.awt.color.ICC_ColorSpace@1de0aca6 transparency = 1 has alpha = false isAlphaPre = false ByteInterleavedRaster: width = 500 height = 360 #numDataElements 3 dataOff[0] = 2
Cell0907
There has been couple of comments on the posts around the Mat and BufferedImage classes (see here). So, I took a step back to understand them better. These names refer to the array of pixels in the image in OpenCV (Mat) or in Java (BufferedImage) and the question comes on how to go from one to the other efficiently. The main reason I needed that was because to display the image processed in OpenCV in Java I had to transform it to BufferedImage first (there is no OpenCV imshow available).
Here is all what you wanted to know about Mat. but it doesn’t talk much about what you put inside (the images itself), so for an easier ride (hopefully), keep reading: 🙂
You can read one element with get(x,y), with x/y the position of the element or a group of elements with get(x,y,byte[]) with x,y=0,0 (for the full array; does it indicate the origin?). The result will get stored in the argument array byte[]. You can change one element with put, in reverse way as get. See example here.
On the other side, BufferedImage is a java subclass that describes an image with an accessible buffer of image data. A BufferedImage is comprised of a ColorModel and a Raster of image data. The 2nd is what holds the image pixels. See how to access them here.
To answer how to read/set the pixels on those structures, we will try to answer the title of this post, i.e., how to go from Mat (the openCV resulting image) to BufferedImage (to display), and do it in the most efficient way. Therefore, we proceed to benchmark few methods (see full code here). Hint: if you want to skip all the reading, jump to Method 4 🙂
Method 1 Go through a jpg. Although looks the cleanest code (actually suggested by one commenter), another commenter thought that this would take the highest computation effort, and basically trigger this whole post :).
public boolean MatToBufferedImage(Mat matrix) < long startTime = System.nanoTime(); MatOfByte mb=new MatOfByte(); Highgui.imencode(".jpg", matrix, mb); try < image = ImageIO.read(new ByteArrayInputStream(mb.toArray())); >catch (IOException e) < // TODO Auto-generated catch block e.printStackTrace(); return false; // Error >long endTime = System.nanoTime(); System.out.println(String.format("Elapsed time: %.2f ms", (float)(endTime - startTime)/1000000)); return true; // Successful > Detected 2 faces
Elapsed time: 25.94 ms
Detected 1 faces
Elapsed time: 27.56 ms
Detected 2 faces
Elapsed time: 27.37 ms
Detected 1 faces
Elapsed time: 26.96 ms
Detected 1 faces
Elapsed time: 35.70 ms
Detected 1 faces
Elapsed time: 27.32 ms
Method 2 Extract the data from Mat into an array, flip the blue and the red channels/columns and store in BufferedImage.
public boolean MatToBufferedImage(Mat matrix) < long startTime = System.nanoTime(); int cols = matrix.cols(); int rows = matrix.rows(); int elemSize = (int)matrix.elemSize(); byte[] data = new byte[cols * rows * elemSize]; int type; matrix.get(0, 0, data); switch (matrix.channels()) < case 1: type = BufferedImage.TYPE_BYTE_GRAY; break; case 3: type = BufferedImage.TYPE_3BYTE_BGR; // bgr to rgb byte b; for(int i=0; ibreak; default: return false; // Error > image = new BufferedImage(cols, rows, type); image.getRaster().setDataElements(0, 0, cols, rows, data); long endTime = System.nanoTime(); System.out.println(String.format("Elapsed time: %.2f ms", (float)(endTime - startTime)/1000000)); return true; // Successful > Detected 2 faces
Elapsed time: 2.27 ms
Detected 2 faces
Elapsed time: 2.81 ms
Detected 2 faces
Elapsed time: 2.25 ms
Detected 2 faces
Elapsed time: 2.75 ms
Detected 2 faces
Elapsed time: 2.22 ms
Substantial (10x!!) improvement, as the anonymous commenter had anticipated.
Method 3 We do the color conversion in OpenCV (see here for color conversions within OpenCV) and then save to BufferedImage:
public boolean MatToBufferedImage(Mat matrix) < long startTime = System.nanoTime(); Imgproc.cvtColor(matrix, matrix, Imgproc.COLOR_BGR2RGB); int cols = matrix.cols(); int rows = matrix.rows(); int elemSize = (int)matrix.elemSize(); byte[] data = new byte[cols * rows * elemSize]; matrix.get(0, 0, data); image = new BufferedImage(cols, rows, BufferedImage.TYPE_3BYTE_BGR); image.getRaster().setDataElements(0, 0, cols, rows, data); long endTime = System.nanoTime(); System.out.println(String.format("Elapsed time: %.2f ms", (float)(endTime - startTime)/1000000)); return true; // Successful >Detected 2 faces
Elapsed time: 2.19 ms
Detected 2 faces
Elapsed time: 12.68 ms
Detected 3 faces
Elapsed time: 2.04 ms
Detected 2 faces
Elapsed time: 2.91 ms
Detected 3 faces
Elapsed time: 2.05 ms
Detected 2 faces
Elapsed time: 2.84 ms
Maybe slightly faster than doing it by hand but. Notice also the long 12ms case above. Nevertheless, I observed that in the other algorithms too, so, I feel that is because the processor gets distracted with some other function.
Method 4 Finally, we get to what is the most efficient method. It basically goes straight from the BGR to the BufferedImage.
public boolean MatToBufferedImage(Mat matBGR) < long startTime = System.nanoTime(); int width = matBGR.width(), height = matBGR.height(), channels = matBGR.channels() ; byte[] sourcePixels = new byte[width * height * channels]; matBGR.get(0, 0, sourcePixels); // create new image and get reference to backing data image = new BufferedImage(width, height, BufferedImage.TYPE_3BYTE_BGR); final byte[] targetPixels = ((DataBufferByte) image.getRaster().getDataBuffer()).getData(); System.arraycopy(sourcePixels, 0, targetPixels, 0, sourcePixels.length); long endTime = System.nanoTime(); System.out.println(String.format("Elapsed time: %.2f ms", (float)(endTime - startTime)/1000000)); return true; >In this 4th method suggested by the anonymous commenter we get the best. Even up to 50x improvement respect to the method 1:
Detected 2 faces
Elapsed time: 0.51 ms
Detected 2 faces
Elapsed time: 1.22 ms
Detected 1 faces
Elapsed time: 0.47 ms
Detected 1 faces
Elapsed time: 1.32 ms
Detected 1 faces
Elapsed time: 0.48 ms
Detected 1 faces
Elapsed time: 1.77 ms
I still need to understand why this other method does not require to flip R and B, but hey, it works. So, thank you, sir! 🙂
Cheers!!
PS.: This other post talks about how to get the picture in BufferedImage into an array and does a nice work benchmarking two methods, either using getRGB or using ((DataBufferByte) bufferedImage.getRaster().getDataBuffer()).getData();
PS2.: More theory on BufferedImages.
PS3.: By the way, the method may be nicer if it was returning the image instead of accessing it as a variable of the mother class, but anyhow.
Convert OpenCV Mat object to BufferedImage
I am trying to create a helper function using OpenCV Java API that would process an input image and return the output byte array. The input image is a jpg file saved in the computer. The input and output image are displayed in the Java UI using Swing.
System.loadLibrary(Core.NATIVE_LIBRARY_NAME); // Load image from file Mat rgba = Highgui.imread(filePath); Imgproc.cvtColor(rgba, rgba, Imgproc.COLOR_RGB2GRAY, 0); // Convert back to byte[] and return byte[] return_buff = new byte[(int) (rgba.total() * rgba.channels())]; rgba.get(0, 0, return_buff); return return_buff; When the return_buff is returned and converted to BufferedImage I get NULL back. When I comment out the Imgproc.cvtColor function, the return_buff is properly converted to a BufferedImage that I can display. It seems like the Imgproc.cvtColor is returning a Mat object that I couldn’t display in Java. Here’s my code to convert from byte[] to BufferedImage:
InputStream in = new ByteArrayInputStream(inputByteArray); BufferedImage outputImage = ImageIO.read(in); 2 Answers 2
ImageIO.read(. ) (and the javax.imageio package in general) is for reading/writing images from/to file formats. What you have is an array containing «raw» pixels. It’s impossible for ImageIO to determine file format from this byte array. Because of this, it will return null .
Instead, you should create a BufferedImage from the bytes directly. I don’t know OpenCV that well, but I’m assuming that the result of Imgproc.cvtColor(rgba, rgba, Imgproc.COLOR_RGB2GRAY, 0) will be an image in grayscale (8 bits/sample, 1 sample/pixel). This is the same format as BufferedImage.TYPE_BYTE_GRAY . If this assumption is correct, you should be able to do:
// Read image to Mat as before Mat rgba = . ; Imgproc.cvtColor(rgba, rgba, Imgproc.COLOR_RGB2GRAY, 0); // Create an empty image in matching format BufferedImage gray = new BufferedImage(rgba.width(), rgba.height(), BufferedImage.TYPE_BYTE_GRAY); // Get the BufferedImage's backing array and copy the pixels directly into it byte[] data = ((DataBufferByte) gray.getRaster().getDataBuffer()).getData(); rgba.get(0, 0, data); Doing it this way, saves you one large byte array allocation and one byte array copy as a bonus. 🙂