10 Examples of curl Command in UNIX and Linux

The curl is one of the essential commands to send HTTP requests from UNIX and Linux operating System. curl command is part of the cURL package and it's not just useful to send HTTP request but also allows you to transfer file using FTP and send mail using SMTP. The cURL utility supports many protocols including DICT, FILE, FTP, FTPS, Gopher, HTTP, HTTPS, IMAP, IMAP, LDAPS, LDAP, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMB, SMTP, SMTP, Telnet, and TFTP. It also supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, HTTP/2, cookies, user+password authentication (Basic, Plain, Digest, CRAM-MD5, NTLM, Negotiate and Kerberos), file transfer resume, proxy tunneling and more. It's particularly useful if you are working with web services.

You can use the curl command to call web services right from UNIX command prompt, receive a response, check whether your server is healthy or not. You can write testing and monitoring scripts using curl command. It's similar to wget command, which is also used for HTTP requests but it's much more powerful and supports many protocols.

The curl command is by default available in many UNIX installation but if it's not you can always download the latest version from http://curl.haxx.se/download.html, the latest version of the cURL package is curl 7.43.0, released on 17th June 2015. cURL has different packages available for different UNIX flavor e.g. Linux, Debian, Fedora, ArchLinux, AIX, FreeBSD, Suse, Redhat, Ubuntu and even for Mac OS X and Apple iOS.



10 Example of curl command in Linux

Here are some of the useful examples of curl command in Linux. You can use this command to test your REST API from the Linux command line. You can also check if your web application is up and down by using curl command in a script and then running it from crontab or a scheduling application like Autosys.

1) How to send HTTP request from UNIX
You can use curl or wget to send HTTP request from UNIX, as shown below:

$ curl http://google.com

By default, curl uses GET method to send HTTP request. You can also use query parameters while connecting to web services using HTTP as shown below:

$ curl http://api.openweathermap.org/data/2.5/weather?id=2172797


Don't forget to include URL inside single quotes if you include more than one query parameters. Why? because multiple query parameters are separated using & which has special meaning in the shell, to run the command in the background, by including it inside single quotes we use it literally, as shown below:

$ curl 'http://api.openweathermap.org/data/2.5/weather?lat=35&lon=139'



2) How to provide timeout for HTTP request in UNIX
Another interesting example of curl command is by using -m option. You can use curl -m option to provide a timeout for HTTP request. If server will not return any response within specified time period then curl will exit, as shown below

$ curl -m 3 'http://api.openweathermap.org/data/2.5/weather?lat=35&lon=139'

will wait for 3 seconds before timing out. BTW, -m is used for a couple of things in curl e.g. for the maximum number of redirects and maximum file size to download.

Alternatively, you can use more verbose --max-time, --max-redirs and --max-filesize options. Worth noting is maximum time allowed for transfer is in seconds, which means -m 5 means 5 seconds. You can use this option to check whether your website or web service is responsive or not.

How to use curl Command in UNIX and Linux


3) How to send HTTP POST request from Linux
You can send any type of HTTP request by using curl -X command in Linux. It allows you to specify HTTP methods other than GET, for example, following command will send HTTP POST request from linux command line:

$ curl -X POST http://api.openweathermap.org

Since GET is the default method, you don't need to specify it but if you want to send other HTTP methods e.g. PUT or POST, you can use this option. Btw, what is the use of POST request without sending any data? Let's see our next example, which allows you to send data to the server using curl command.


4) How to send data using HTTP POST in UNIX
You can also send POST request using curl -d option. -d is nothing but for data, which is sent as POST body. The data is expected to be URL-encoded.

$ curl -d 'lat=35&lon=139' http://api.openweathermap.org/data/2.5/weather

This will send query parameters as a POST request. You can also use -d option multiple times to specify different data pieces e.g.

$ curl -d lat=35 -d lon=139 http://api.openweathermap.org/data/2.5/weather

This will eventually be combined as -d 'lat=35&lon=139'. The -d option can also be used to submit HTML form data to the server. You can see The Linux command line:  A complete Introduction for more details on sending data using curl command.

How to send HTTP post request from Linux using curl



5) How to send a file via HTTP POST in Linux
If your data is large, you can keep them inside a file and tell curl the file name. It will transfer all that data using HTTP post request to the server, here is an example of sending a file using curl:

$ curl -d @requestData.txt http://api.openweathermap.org/data/2.5/weather

The @ sign tells curl that whatever follows is a file name. BTW, the content of file must be URL encoded. You can also use --data instead of -d if you prefer the most verbose option.


6) How to send username password for authentication using curl
You can use curl -u option to pass username and password for authentication. The -u option is a shortcut of --user option which specifies username and password for server authentication. If you using twitter, here is how you can update your status using curl command right from UNIX:

$ curl -u username:password -d status='curl is great' 
http://twitter.com/statuses/update.xml


If you don't wish your password to be saved in the shell history, you can omit password part and curl will prompt for password when it tries to authenticate on the server. As I told earlier, -d instructs curl to use HTTP POST method to send a request and "status=.." will be sent as part of the request body. You can further read Practical guide to Linux command and shell editors to learn more about it.

The curl Command example in UNIX and Linux



7) How to specify HTTP header using curl command in Linux
You can specify HTTP header by using curl -H option. Since with web service you often deal with either JSON or XML rather than HTML, it makes sense to specify content-type as either "application/json" or "application/xml". You can do this using curl as shown in the following example:

$ curl -H "Accept: application/json" 
'http://api.openweathermap.org/data/2.5/weather?lat=35&lon=139'

You can also use headers while sending data to server using HTTP post e.g.

$ curl -X PUT \
-H 'Content-Type: application/json' \
-d '{"id":"101", "description":"baby soap"}'
http://localhost:8080/item/add

One side tip, you can split your UNIX command into multiple lines by using \ (backslash). It makes your command more readable, especially if it's getting long.


8) How to view HTTP response header in Linux
You can use the curl -i command to view response header in UNIX. Since API and WebService provider is increasingly using HTTP header to provide information about caching, content-type, authorization etc, it's very useful to see HTTP header using curl as shown below:

$ curl -i http://api.openweathermap.org/data/2.5/weather?zip=94040,us

This will return all response headers from HTTP response as shown below:

10 Examples of curl Command in UNIX and Linux


That's all about how to use curl command to send HTTP request and receive a response. We have seen several examples of curl command ranging from sending simple HTTP request to timeout, with query parameters and different types of HTTP request e.g. POST, PUT, and DELETE.

It's actually a must know tool if you are working with REST web services which return JSON or XML output. You can test your web services right from UNIX box, can build scripts to test and monitor them as well. As a Java developer, I use curl a lot when I work with web services based application.

Other Linux command examples, you may like to explore
10 examples of find command in Linux (examples)
10 examples of grep command in Linux (see)
10 examples of sed command in UNIX (see)
10 examples of chmod command in Linux (see)
10 examples of xargs command in Linux (see)
10 Essential Networking commands in Linux (see)
10 examples of lsof command in Linux (see)
10 Examples of tar command in Linux (see)

Thanks for reading this tutorial, if you like then please share with friends and colleagues. If you have any other interesting curl examples, which you use daily or find useful, feel free to share with us too., 


No comments :

Post a Comment