CURL or client URL is a multifunctional utility that allows you to communicate with servers using HTTP, HTTPS, FTP, FTPS, SCP, SSH and many other protocols. The main application users have found is to use the utility to make requests to web servers. However, the functionality of the utility, thanks to the libcurl library, leaves a large scope for possibilities.
How to install curl?
Installation is standard, as for any Linux package of a similar system. It is necessary to update the repository first, if you use apt-manager, then type the command:
apt install curl -y
For an RPM-like package manager, use the appropriate commands:
dnf install curl -y
Having updated the repositories and installed the utility, let's turn to the tooltip and divide the help into categories or protocols of operation:
This list provides a complete list of protocols that can be used to communicate with servers. If you replace the word category in the previous command with one of the given items, a detailed guide on how to use each of the options will open:
The command above is as an example of searching for commands to work over the http protocol.
Basic examples of work
Let's look at the basic examples for working with the utility, but before that let's define the syntax used:
The main command for calling the curl utility, its main options, and the URL itself, which is accessed.
How do I download a file using Curl?
By default, when accessing the site curl forms a GET request, which asks the server for the headers and the file itself. To save the file, write the command:
Note that to download from a site that may have a self-signed certificate, you must additionally use the -k option.
After downloading the file will be saved in the directory /tmp/ under the name articles.pdf, for which the -o option is responsible. The other -l option means that if the site responds with a 301, 302, etc. redirect code, curl will automatically go to the correct site and download the file. The file can also be left with the original name and downloaded to the current directory:
However, if you want to output the file directly to the terminal, leave a dash instead of the file name:
Such a solution will allow you to view the contents of a file faster with its headers and meta description, which may be necessary to check the contents of the file. For example, pdf for the presence of embedded scripts.
All steps in the tutorial can be performed on powerful cloud servers. Serverspace provide isolated VPS / VDS servers for common and virtualize usage.
It will take some time to deploy server capacity. After that you can connect in any of the convenient ways. Let's return to explore utility.
How do I view headings using Curl?
Before downloading this or that file or caching, it is necessary to collect information about the page/file that will be accessed. For example, a pdf file is hosted on the site and it is necessary to find out its name, size and data about the last modification of the file. To do this, let's write the following command:
The MIME content-type field categorises the file as pdf. The next field, last-modified, contains the last modification date and the content-length field contains the size of the document body.
Continued interrupted download
Suppose we started downloading a file and then accidentally cancelled it. If the Accept-Ranges: bytes line is present in the server response headers, we can use the function to continue the cancelled download. To do this, the -c parameter must be added to the usual list of options when downloading -Lko, which will allow us to continue the download:
The file will then be saved to the directory where the user is currently located.
How to change User-Agent or other header?
While working with a web server, you may need to implement custom headers. For this purpose, curl has the -H option, which will allow you to add a header with a value:
There is a separate option -A for similar customisation of the User-Agent field. It must be specified with a new agent value, for example, the command may look like this:
The use of CURL facilitates various network operations and gives the user flexibility and control when interacting with servers. Thanks to its capabilities, users can efficiently perform a variety of tasks related to data exchange.