654

How can I do an HTTP GET from a Un*x shell script on a stock OS X system? (installing third-party software is not an option, for this has to run on a lot of different systems which I don't have control on).

For example if I start the Mercurial server locally doing a hg serve:

... $ hg serve

And then, from a Linux that has the wget command I do a wget:

... $  wget http://127.0.0.1:8000
--2010-12-31 22:18:25--  http://127.0.0.1:8000/
Connecting to 127.0.0.1:8000... connected.
HTTP request sent, awaiting response... 200 Script output follows
Length: unspecified [text/html]
Saving to: `index.html

And on the terminal in which I launched the "hg serve" command, I can indeed see that an HTTP GET made its way:

127.0.0.1 - - [30/Dec/2010 22:18:17] "GET / HTTP/1.0" 200 -

So on Linux one way to do an HTTP GET from a shell script is to use wget (if that command is installed of course).

What other ways are there to do the equivalent of a wget? I'm looking, in particular, for something that would work on stock OS X installs.

Improve this question
1

10 Answers 10

Reset to default
922

The following native command will work:

curl http://127.0.0.1:8000 -o outfile

Note that curl does not follow redirects by default. To tell it to do so, add -L to the argument list.

Improve this answer
9
  • 27
    Also, wget is available via both MacPorts and Fink.
    – Brian Clapper
    Commented Dec 31, 2010 at 20:26
  • 47
    I came here from a Google search on how to get wget on Mac, so despite the OP's requirement to avoid installation of 3rd party software, I'll provide a link to a quick guide I found on how to install wget without using MacPorts for future reference.
    – Eirik H
    Commented Mar 20, 2013 at 8:28
  • 45
    It's also available on Homebrew.
    – Daniel
    Commented Oct 10, 2014 at 0:42
  • 4
    Doesn't seem to follow forwards.
    – nroose
    Commented Jan 2, 2017 at 2:12
  • 35
    For downloading files you can use -O or --remote-name flag to auto rename downloaded file. e.g. curl -O http://somehost.org/file.zip
    – nimcap
    Commented Feb 22, 2018 at 10:49
604

brew install wget

Homebrew is a package manager for OSX analogous to yum, apt-get, choco, emerge, etc. Be aware that you will also need to install Xcode and the Command Line Tools. Virtually anyone who uses the command line in OSX will want to install these things anyway.

If you can't or don't want to use homebrew, you could also:

Install wget manually:

curl -# "http://ftp.gnu.org/gnu/wget/wget-1.17.1.tar.xz" -o "wget.tar.xz"
tar xf wget.tar.xz
cd wget-1.17.1
./configure --with-ssl=openssl -with-libssl-prefix=/usr/local/ssl && make -j8 && make install

Or, use a bash alias:

function _wget() { curl "${1}" -o $(basename "${1}") ; };
alias wget='_wget'
Improve this answer
7
  • 12
    You might take a moment to explain homebrew, but it's important that there be a newbie-visible wget answer here since the only other one was deleted by it's owner, and curl is an alternative rather than a literal equivalent.
    – Chris Stratton
    Commented Jun 13, 2013 at 18:42
  • 4
    Thanks, this was helpful to me as someone with brew already installed.
    – enderland
    Commented Jul 13, 2013 at 18:48
  • 4
    @Michaelangelo And that is not your job to vandalize others' posts. You should not be editing other answers to make your own points - that's inappropriate. In fact, the actions are being discussed on Meta: meta.stackoverflow.com/questions/315892/…
    – Zizouz212
    Commented Feb 1, 2016 at 20:07
  • 2
    @EricHartford Down-voting doesn't solve the problem. It provided one alternative, without listing the other option of how to manually install wget.
    – Michaelangel007
    Commented Feb 1, 2016 at 20:38
  • 1
    I think you have a good point. And I will edit the answer to incorporate your ideas.
    – Eric Hartford
    Commented Feb 1, 2016 at 20:46
114

Curl has a mode that is almost equivalent to the default wget.

curl -O <url>

This works just like

wget <url>

And, if you like, you can add this to your .bashrc:

alias wget='curl -O'

It's not 100% compatible, but it works for the most common wget usage (IMO)

Improve this answer
4
  • 3
    how do u do recursive with this?
    – Jasper
    Commented Nov 7, 2015 at 14:52
  • 1
    You can use bash to add this into a loop like this: for i in `seq 1 <n>` do curl -O <url>;done; where <n> is the number of times you want to iterate and <url> is the url to pull.
    – Blairg23
    Commented Dec 15, 2015 at 23:32
  • 2
    Sometimes you'll need to add the -L flag to follow location redirects. You can use curl -OL <url> to do that.
    – Gustavo Straube
    Commented Jan 27, 2016 at 12:48
  • 2
    -O also only applies to the next argument, so to download multiple URLs you have to use something like curl -O "$url1" -O "$url2" or printf %s\\n "$url1" "$url2"|xargs -n1 curl -O.
    – nisetama
    Commented Apr 19, 2016 at 0:47
45

1) on your mac type

nano /usr/bin/wget

2) paste the following in

#!/bin/bash
curl -L $1 -o $2

3) close then make it executable

chmod 777 /usr/bin/wget

That's it.

Improve this answer
6
  • 1
    Better than an alias.
    – M K
    Commented Feb 20, 2014 at 9:29
  • 4
    Almost correct. I believe that step one should be vim /usr/bin/wget though ;) haha just kidding. thanks for the answer -- this never really occurred to me and for some reason I don't feel like installing brew/fink/whatever, so kudos for the easy portable answer.
    – Kasapo
    Commented Apr 2, 2015 at 18:47
  • 5
    -L is important for handling http 301 responses. wget handles them by default.
    – Donn Lee
    Commented May 22, 2018 at 17:28
  • "curl -L resource.url/tar.tar.gz -O tar.tar.gz" worked fine, thanks for this sole workable solution among others in this thread for my use case.
    – Oleksii Kyslytsyn
    Commented Jul 8, 2018 at 12:12
  • sudo nano /usr/bin/wget
    – user1575941
    Commented Dec 21, 2018 at 21:59
20

Use curl;

curl http://127.0.0.1:8000 -o index.html
Improve this answer
17

Here's the Mac OS X equivalent of Linux's wget.

For Linux, for instance Ubuntu on an AWS instance, use:

wget http://example.com/textfile.txt

On a Mac, i.e. for local development, use this:

curl http://example.com/textfile.txt -o textfile.txt

The -o parameter is required on a Mac for output into a file instead of on screen. Specify a different target name for renaming the downloaded file.

Use capital -O for renaming with wget. Lowercase -o will specify output file for transfer log.

Improve this answer
6

Instead of going with equivalent, you can try "brew install wget" and use wget.

You need to have brew installed in your mac.

Improve this answer
1
  • 5
    Not gonna downvote, but this is a dupe of Eric's answered Jun 13 '13.
    – MarkHu
    Commented Sep 14, 2018 at 2:30
5

You can either build wget on the mac machine or use MacPorts to install it directly.

sudo port install wget

This would work like a charm, also you can update to the latest version as soon as it's available. Port is much more stable than brew, although has a lot less number of formula and ports.

You can install MacPorts from https://www.macports.org/install.php you can download the .pkg file and install it.

Improve this answer
1
  • and how do you install port?
    – Roy Lovejoy
    Commented Jan 9, 2017 at 22:21
4

You could use curl instead. It is installed by default into /usr/bin.

Improve this answer
2

wget Precompiled Mac Binary

For those looking for a quick wget install on Mac, check out Quentin Stafford-Fraser's precompiled binary here, which has been around for over a decade:

https://statusq.org/archives/2008/07/30/1954/

MD5 for 2008 wget.zip: 24a35d499704eecedd09e0dd52175582
MD5 for 2005 wget.zip: c7b48ec3ff929d9bd28ddb87e1a76ffb

No make/install/port/brew/curl junk. Just download, install, and run. Works with Mac OS X 10.3-10.12+.

Improve this answer
1

Not the answer you're looking for? Browse other questions tagged or ask your own question.