Thursday, October 29, 2015

How to send HTTP request from UNIX or Linux? Use curl or wget command

You can use either curl or wget command to send HTTP request from UNIX or Linux operating system. Both commands allow you to send GET and POST request, which means you can also call REST web services.  I have a Java web application, which runs on Linux and exposes WebServices. I was writing a UNIX script to download In some data from that web service when I hit by the question, how do I make an HTTP call from UNIX? What is the UNIX command should I use? If you are also facing the same question, then you have come to right place. Basically, you can use two UNIX commands to make the HTTP request, wget, and curl. You can use curl and wget to send both GET and POST HTTP request. My use case was rather simple, I just need to make a GET call to web service to download the data, but these two commands are very powerful and provide many different options to interact with the web using HTTP right from the shell.

Between wget and curl (also known as cURL), curl (I call it "karl") is my favorite. Why? I think because I have used it more often than wget :-). You can make your choices if you have good knowledge of Linux command line and that's why I suggest every Java developer to read The Linux Command Line: A Complete Reference, the best way to become efficient in Linux. Now let's see how to send HTTP request from UNIX using  the curl command.

How to send HTTP request from UNIX curl vs wget command

UNIX command to send HTTP GET request

Here is one example of calling web service from Linux shell by sending HTTP GET request using cURL command:

$ curl,uk&appid=bd82977b86bf27fb59a04b61b657fb6f
{"coord":{"lon":-0.13,"lat":51.51},"weather":[{"id":804,"main":"Clouds","description":"overcast clouds","icon":"04n"}],"base":"stations","main":{"temp":282,"pressure":1022,"humidity":87,"temp_min":277.15,"temp_max":285.15},"visibility":10000,"wind":{"speed":1.5},"clouds":{"all":90},"dt":1445577409,"sys":{"type":1,"id":5093,"message":0.0201,"country":"GB","sunrise":1445582275,"sunset":1445619056},"id":2643743,"name":"London","cod":200}

You can also specify timeout using -m option as shown below:

$ curl -m 2,uk&appid=bd82977b86bf27fb59a04b61b657fb6f

This request will timeout in 2 seconds if it doesn't receive any response.

You can also use wget to send HTTP request and download the data. Only difference between curl and wget is that curl will print output in console and wget will store it in the file e.g.

$ wget http://localhost:8080/index.html

will download the content of index.html and store into a file with the same name.

Here is how you can use curl to download wget command in UNIX:

How to send HTTP request from UNIX? Use CURL command

UNIX command to send HTTP POST request

You can also use curl to send HTTP post request. All you need to do is use the --data option to specify the data you want to POST to web service e.g.

$ curl --data "param1=value1&param2=value2" http://locahost:8080/weather

if you want to send a data from file to web service, you can also use the following command:

$ curl -X POST -d @filename http://locahost:8080/weather

Similarly, if you want to upload a file you can do so by executing the following command:

$ curl --form "fileupload=@filename.txt" http://locahost:8080/weather

That's all about how to send HTTP request from UNIX and Linux. It's simple, just remember curl and wget command. you can explore their option by using man wget and man curl. Also, remember the difference between curl and wget, former prints the output in the console while later store the response in the same file as requested. I personally like curl because it's easier to use but if you like wget stick with it.

Further Learning
Linux Command Line Interface (CLI) Fundamentals
Linux System Administration Fundamentals
How Linux Works: What Every SuperUser Should Know

Related UNIX Command Tutorials
If you are Java developer often working in Linux or UNIX environment then you will also find following tutorials useful:
  • 10 examples of find command in UNIX (examples)
  • 10 examples of grep command in UNIX (examples)
  • 10 examples of date command in Linux (examples)
  • How to get IP address from hostname and vice-versa in Linux (command)
  • 10 examples of xargs command in Linux (examples)
  • 10 examples of tar command in UNIX (examples)
  • 10 examples of Vim in UNIX (examples)
  • How to create, update and delete soft link in UNIX (command)
  • How to delete empty files and directory in UNIX (solution)
  • How to make directory tree in one command? (example)
  • How to how long argument of a process in Solaris (command)
  • UNIX command to find out how long a process is running? (answer)
  • UNIX command to find the size of file and directory? (command)
  • 5 examples of sort command in Linux (examples)
  • 5 examples of kill command in Linux (examples)
  • 10 examples of chmod command in UNIX (examples)
  • 10 tips to work fast in UNIX? (tips)

Thanks for reading this article so far. If you like this article then please share with your friends and colleagues. If you have any questions or feedback then please drop a note.


Anonymous said...

Great article. In the curl request you should use quotes around the URL and the data being sent curl ",uk&appid=bd82977b86bf27fb59a04b61b657fb6f" otherwise curl seems to thing everything after the & is another request. Super weird, but fixes a lot of script problems. Thanks for the article!

consolechars said...

You can use wget and curl but some times in production enviroment you dont have curl wget etc :) . You can use /dev/tcp build kernel function
exec 15<>/dev/tcp/
echo -e "GET / HTTP/1.1\n\n" >&15
cat <&15

more :

Post a Comment