Java load http file

How to download a file via HTTP GET and HTTP POST in Java without using any external libraries

Apart from uploading a file to a HTTP server endpoint, another common task for a Java HTTP client is to download a file from a HTTP server. Even though there are many Java external libraries to help us do so, using the facilities in the Java standard runtime installation is not difficult. Furthermore, we will be able to keep our Java application leaner if we can download files without additional dependencies.

In case you need a reference, this is how to download a file via HTTP GET and HTTP POST in Java without using any external libraries.

Downloading a file from a HTTP server endpoint via HTTP GET

Generally, downloading a file from a HTTP server endpoint via HTTP GET consists of the following steps:

• Construct the HTTP GET request to send to the HTTP server.
• Send the HTTP request and receive the HTTP Response from the HTTP server.
• Save the contents of the file from HTTP Response to a local file.

Java codes to download a file from a HTTP server endpoint via HTTP GET

Given these points, let us modify the codes from how to send a HTTP GET request in Java to save the contents of the file from a HTTP Response to a local file:

import java.io.IOException; import java.io.FileOutputStream; import java.net.HttpURLConnection; import java.net.URL; import java.nio.channels.Channels; import java.nio.channels.FileChannel; import java.nio.channels.ReadableByteChannel; public class HttpGetDownloadFileExample < public static void main(String[] args) < ReadableByteChannel readableChannelForHttpResponseBody = null; FileChannel fileChannelForDownloadedFile = null; try < // Define server endpoint URL robotsUrl = new URL("http://www.techcoil.com/robots.txt"); HttpURLConnection urlConnection = (HttpURLConnection) robotsUrl.openConnection(); // Get a readable channel from url connection readableChannelForHttpResponseBody = Channels.newChannel(urlConnection.getInputStream()); // Create the file channel to save file FileOutputStream fosForDownloadedFile = new FileOutputStream("robots.txt"); fileChannelForDownloadedFile = fosForDownloadedFile.getChannel(); // Save the body of the HTTP response to local file fileChannelForDownloadedFile.transferFrom(readableChannelForHttpResponseBody, 0, Long.MAX_VALUE); >catch (IOException ioException) < System.out.println("IOException occurred while contacting server."); >finally < if (readableChannelForHttpResponseBody != null) < try < readableChannelForHttpResponseBody.close(); >catch (IOException ioe) < System.out.println("Error while closing response body channel"); >> if (fileChannelForDownloadedFile != null) < try < fileChannelForDownloadedFile.close(); >catch (IOException ioe) < System.out.println("Error while closing file channel for downloaded file"); >> > > >

As shown above, we had created a class with a main method. When the main method is executed, our Java program does several things.

Firstly, we had declared an instance of ReadableByteChannel and an instance of FileChannel to contain null values so as to facilitate closure in the finally block.

After we had done so, we define a try block for running the file download logic that may throw checked exceptions.

Within the try block, we first use an instance of URL to get an instance of HttpURLConnection that points to a HTTP server endpoint for the file that we wish to download.

Once we have done so, we get an instance of ReadableByteChannel that is mapped to the input stream of urlConnection . In addition, we set it to the readableChannelForHttpResponseBody variable for manipulation later.

After we had an instance of ReadableByteChannel , we then proceed to get a FileChannel that will point to a local file path where we want to save the downloaded file. In this case, we want to save the downloaded file as robots.txt in the same directory where we run our Java program.

Finally, we use fileChannelForDownloadedFile.transferFrom method on readableChannelForHttpResponseBody to save the contents of the HTTP response body to the local file.

Downloading a file from a HTTP server endpoint via HTTP POST

There can be cases where the client need to supply some information to the HTTP server in order to download a file. In such a situation, HTTP POST is a more appropriate HTTP method to use for downloading the file.

As with HTTP GET, downloading of a file from the HTTP server via HTTP POST consists of the following steps:

• Construct the HTTP POST request to send to the HTTP server.
• Send the HTTP request and receive the HTTP Response from the HTTP server.
• Save the contents of the file from HTTP Response to a local file.

Java codes to download a file from a HTTP server endpoint via HTTP POST

Given these points, let’s modify the Java code from how to send a HTTP POST request in Java to save the contents of the file from a HTTP Response to a local file:

import java.io.BufferedWriter; import java.io.IOException; import java.io.FileOutputStream; import java.io.OutputStreamWriter; import java.net.HttpURLConnection; import java.net.URL; import java.nio.channels.Channels; import java.nio.channels.FileChannel; import java.nio.channels.ReadableByteChannel; public class HTTPPostDownloadFileExample < public static void main(String[] args) throws IOException < // Define the server endpoint to send the HTTP request to URL serverUrl = new URL("https://www.techcoil.com/process/proof-of-concepts/userNameAndLuckyNumberTextFileGeneration"); HttpURLConnection urlConnection = (HttpURLConnection)serverUrl.openConnection(); // Indicate that we want to write to the HTTP request body urlConnection.setDoOutput(true); urlConnection.setRequestMethod("POST"); // Writing the post data to the HTTP request body BufferedWriter httpRequestBodyWriter = new BufferedWriter(new OutputStreamWriter(urlConnection.getOutputStream())); httpRequestBodyWriter.write("visitorName=Johnny+Jacobs&luckyNumber=1234"); httpRequestBodyWriter.close(); // Get a readable channel from url connection ReadableByteChannel readableChannelForHttpResponseBody = Channels.newChannel(urlConnection.getInputStream()); // Create the file channel to save file FileOutputStream fosForDownloadedFile = new FileOutputStream("luckyNumber.txt"); FileChannel fileChannelForDownloadedFile = fosForDownloadedFile.getChannel(); // Save the contents of HTTP response to local file fileChannelForDownloadedFile.transferFrom(readableChannelForHttpResponseBody, 0, Long.MAX_VALUE); >>

As shown above, we had created a class with a main method which will propagate any occurrences of IOException back to the caller. When the main method is executed, our Java program does several things.

First, it creates an instance of HttpURLConnection , urlConnection , that maps to the HTTP server endpoint at https://www.techcoil.com/process/proof-of-concepts/userNameAndLuckyNumberTextFileGeneration . Given that, we then configure urlConnection to allow us to write to the HTTP request body via urlConnection.setDoOutput(true). After that, we set the HTTP method of the HTTP request as POST via urlConnection.setRequestMethod(«POST») .

Next, we use an instance of BufferedWriter to write some POST variables to the HTTP request body.

Once we had written the POST variables, we then get an instance of ReadableByteChannel from the input stream of urlConnection . Given that instance, we will then be able to read from the body of the HTTP response.

After we had an instance of ReadableByteChannel , we then proceed to get an instance of FileChannel that will point to a local file path where we want to save the downloaded file. In this case, we will save the downloaded file as luckyNumber.txt in the same directory where we run our Java program.

Finally, we use fileChannelForDownloadedFile.transferFrom on readableChannelForHttpResponseBody to save the contents of the HTTP response body to the local file.

About Clivant

Clivant a.k.a Chai Heng enjoys composing software and building systems to serve people. He owns techcoil.com and hopes that whatever he had written and built so far had benefited people. All views expressed belongs to him and are not representative of the company that he works/worked for.

Источник

Java download file

Often it is required to download a file directly from a URL and save to a directory on local system such as downloading a file from a remote repository. This requires reading a file from url and writing it to a local file.
This article will share different methods to download a file from URL in java. Methods defined here are applicable to all types of file such as a pdf file, an exe file, a txt file or a zip file etc.

1. Create a connection to the given file url.
2. Get input stream from the connection. This stream can be used to read file contents.
3. Create an output stream to the file to be downloaded.
4. Read the contents from the input stream and write to the output stream.

Java code to download file from URL with this method is given below.

import java.net.URL; import java.net.URLConnection; import java.io.FileOutputStream; import java.io.IOException; import java.io.InputStream; import java.io.OutputStream; public class FileDownloader < public static void main(String[] args) < OutputStream os = null; InputStream is = null; String fileUrl = "http://200.34.21.23:8080/app/file.txt"; String outputPath = "E:\\downloads\\downloaded.txt"; try < // create a url object URL url = new URL(fileUrl); // connection to the file URLConnection connection = url.openConnection(); // get input stream to the file is = connection.getInputStream(); // get output stream to download file os = new FileOutputStream(outputPath); final byte[] b = new byte[2048]; int length; // read from input stream and write to output stream while ((length = is.read(b)) != -1) < os.write(b, 0, length); >> catch (IOException e) < e.printStackTrace(); >finally < // close streams if (os != null) os.close(); if (is != null) is.close(); >> >

Remember that the output and input paths should end with the file name else there will be an error.

All the methods in this post read a file at a remote URL. If you want to read a file at local system, then refer this post.

1. Create an input stream to the file to be downloaded.
2. Create a new channel that will read data from this input stream.
3. Create an output stream that will write file contents after reading it from the channel created in Step 2.
4. Get the channel from this output stream and write the contents from channel created in Step 2.

You will understand the algorithm better after looking at the below code example.

import java.io.FileOutputStream; import java.io.IOException; import java.io.InputStream; import java.net.URL; import java.nio.channels.Channels; import java.nio.channels.ReadableByteChannel; import java.io.File; public class FileDownloader < public static void main(String[] args) < try < String fileUrl = "http://200.34.21.23:8080/app/file.pdf"; String outputPath = "E:\\downloads\\downloaded.pdf"; URL url = new URL(fileUrl); // create an input stream to the file InputStream inputStream = url.openStream(); // create a channel with this input stream ReadableByteChannel channel = Channels.newChannel( url.openStream()); // create an output stream FileOutputStream fos = new FileOutputStream( new File(outputPath)); // get output channel, read from input channel and write to it fos.getChannel().transferFrom(channel, 0, Long.MAX_VALUE); // close resources fos.close(); channel.close(); >catch(IOException e) < e.printStackTrace(); >> >

Note that transferFrom() method of a channel takes 3 arguments.
1. input file channel,
2. position at which it will start reading the file, 0 here means the beginning of file, and
3. number of bytes that will be transferred at one time. This value is set to a very large value( Long.MAX_VALUE ) for higher efficiency.

Learn different methods of writing to a file here .

Method 3 : Using Apache Commons IO Library
Apache Commons IO Library has a org.apache.commons.io.FileUtils class which contains a method copyURLToFile() .

This method takes two arguments:
1. java.net.URL object pointing to the source file, and
2. java.io.File object which points to an output file path.

Remember that both paths should contain the name of file at the end and output path should be a location on local system at which the file will be downloaded.
copyURLToFile() reads the file from remote location and copies it to the local machine. Example,

import org.apache.commons.io.FileUtils; public class FileDownloader < public static void main(String[] args) < String fileUrl = "http://200.34.21.23:8080/app/file.zip"; String outputPath = "E:\\downloads\\downloaded.zip"; FileUtils.copyURLToFile(new URL(fileUrl), new File(outputPath)); >>

You can add Apache Commons IO dependency to your project as per the build tool.

// Gradle
compile group: ‘org.apache.commons’, name: ‘commons-io’, version: ‘1.3.2’

Hope the article was useful in explaining different ways to download a file from URL in java.
Do not forget to click the clap below.

Источник

Java HttpURLConnection to download file from an HTTP URL

In this post, I will guide you how to write Java code to download files from web server programmatically.

You know, in Java, we can use the classes URL and HttpURLConnection in the package java.net to programmatically download a file from a given URL by following these steps:

• • Create a URL object for a given URL. The URL can be either:
  1. • • A direct link which contains the real file name at the end, for example:

  http://jdbc.postgresql.org/download/postgresql-9.2-1002.jdbc4.jar

  http://myserver.com/download?id=1234

  • • Open connection on the URL object – which would return an HttpURLConnection object if the URL is an HTTP URL.
    • Open the input stream of the opened connection.
    • Create an output stream to save file to disk.
    • Repeatedly read array of bytes from the input stream and write them to the output stream, until the input stream is empty.
    • Close the input stream, the output stream and the connection.

    package net.codejava.networking; import java.io.File; import java.io.FileOutputStream; import java.io.IOException; import java.io.InputStream; import java.net.HttpURLConnection; import java.net.URL; /** * A utility that downloads a file from a URL. * @author www.codejava.net * */ public class HttpDownloadUtility < private static final int BUFFER_SIZE = 4096; /** * Downloads a file from a URL * @param fileURL HTTP URL of the file to be downloaded * @param saveDir path of the directory to save the file * @throws IOException */ public static void downloadFile(String fileURL, String saveDir) throws IOException < URL url = new URL(fileURL); HttpURLConnection httpConn = (HttpURLConnection) url.openConnection(); int responseCode = httpConn.getResponseCode(); // always check HTTP response code first if (responseCode == HttpURLConnection.HTTP_OK) < String fileName = ""; String disposition = httpConn.getHeaderField("Content-Disposition"); String contentType = httpConn.getContentType(); int contentLength = httpConn.getContentLength(); if (disposition != null) < // extracts file name from header field int index = disposition.indexOf("filename="); if (index >0) < fileName = disposition.substring(index + 10, disposition.length() - 1); >> else < // extracts file name from URL fileName = fileURL.substring(fileURL.lastIndexOf("/") + 1, fileURL.length()); >System.out.println("Content-Type = " + contentType); System.out.println("Content-Disposition = " + disposition); System.out.println("Content-Length = " + contentLength); System.out.println("fileName = " + fileName); // opens input stream from the HTTP connection InputStream inputStream = httpConn.getInputStream(); String saveFilePath = saveDir + File.separator + fileName; // opens an output stream to save into file FileOutputStream outputStream = new FileOutputStream(saveFilePath); int bytesRead = -1; byte[] buffer = new byte[BUFFER_SIZE]; while ((bytesRead = inputStream.read(buffer)) != -1) < outputStream.write(buffer, 0, bytesRead); >outputStream.close(); inputStream.close(); System.out.println("File downloaded"); > else < System.out.println("No file to download. Server replied HTTP code: " + responseCode); >httpConn.disconnect(); > >

    Note that in the static method downloadFile() , we have to check HTTP response code from the server to make sure the URL is available (HTTP status 200). Then we extract the file name either from the HTTP header Content-Disposition (in case the URL is an indirect link), or from the URL itself (in case the URL is the direct link). We also print out some debugging information like Content-Type, Content-Disposition, Content-Length and file name.

    And here is a test program which employs the utility class above:

    package net.codejava.networking; import java.io.IOException; public class HttpDownloader < public static void main(String[] args) < String fileURL = "http://jdbc.postgresql.org/download/postgresql-9.2-1002.jdbc4.jar"; String saveDir = "E:/Download"; try < HttpDownloadUtility.downloadFile(fileURL, saveDir); >catch (IOException ex) < ex.printStackTrace(); >> >

    This program downloads the file postgresql-9.2-1002.jdbc4.jar and saves it into the directory E:/Download . It would produce the following output:

    Content-Type = application/java-archive Content-Disposition = null Content-Length = 579785 fileName = postgresql-9.2-1002.jdbc4.jar File downloaded

    Watch this video tutorial to see how to build, run and test this program:

    Related Java File Download Tutorials:

    Other Java network tutorials:

    About the Author:

    

    Nam Ha Minh is certified Java programmer (SCJP and SCWCD). He started programming with Java in the time of Java 1.4 and has been falling in love with Java since then. Make friend with him on Facebook and watch his Java videos you YouTube.

    Источник