HTTPie vs cURL (Part 3)

Over the last decades cURL was the king in CLI HTTP client area. It has many features which is enough to test APIs. However, panta rei and world is changing constantly. Even is such scope where one tool has such a great adoption ratio new challengers occur. One of them is HTTPie. This is quite new tool which getting popular and popular. At a glance it's quite interesting choice. Let look deeply and compare both. It's time to fight. HTTPie vs cURL.

Authentication

At the beginning of the part let's head to authentication which is commonly first call during the session.

httpie

HTTPie has -a flag which allows to start authentication. For instance, to start the 'Basic' schema you need to send such command

$ http -a username:password localhost/basic-auth/username/password
HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Length: 51
Content-Type: application/json
Date: Mon, 01 Feb 2021 20:37:26 GMT
Server: gunicorn/19.9.0

{
    "authenticated": true,
    "user": "username"
}

Similarly, Digest schema looks almost the same you need to add -A digest to the command

$ http -A digest -a username:password localhost/digest-auth/httpie/username/password
HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Length: 51
Content-Type: application/json
Date: Mon, 01 Feb 2021 20:39:22 GMT
Server: gunicorn/19.9.0
Set-Cookie: fake=fake_value; Path=/
Set-Cookie: stale_after=never; Path=/

{
    "authenticated": true,
    "user": "username"
}

Putting password in the plain text in the terminal is always not a good idea that's why prompt facility comes into play. If you omit the password tin the command HTTPie will ask you to provide it.

$ http -a username localhost/basic-auth/username/password
http: password for username@localhost: 
HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Length: 51
Content-Type: application/json
Date: Mon, 01 Feb 2021 20:42:19 GMT
Server: gunicorn/19.9.0

{
    "authenticated": true,
    "user": "username"
}

Additionally, in the HTTPie ecosystem there is a possibility to extend functionality by writing the plugins. Most of the plugins created by community covers various mechanism of authentications which is great. Please continue the reading I'm sharing more details in the one of the next paragraphs.

cURL

Similarly, cURL has flag -u to enter the authentication framework where default is 'Basic' schema.

$ curl -u username:password localhost/basic-auth/username/password
{
  "authenticated": true, 
  "user": "username"
}

Likewise, to activate Digest schemat you need to add only --digest flag.

$ curl --digest -u username:password localhost/digest-auth/httpie/username/password
{
  "authenticated": true, 
  "user": "username"
}

Asking fot password via prompt is easy as it should be.

$ curl -u username localhost/basic-auth/username/password
Enter host password for user 'username':
{
  "authenticated": true, 
  "user": "username"
}

However I've noticed lack of suport to modern authentication framework like OAuth, JWT etc. Of course you can do it using raw cURL command (many of calls) but having plugins in cases of HTTPie which support it is much better.

Ignore certificate check

This is common problem in many internal services with self-signed certificates. How to fetch data from that endpoints? Ignore certificate to the rescue!

httpie

At the beginning I'm trying to fetch web page behind the self-signet certificate

$ https https://self-signed.badssl.com/

https: error: SSLError: HTTPSConnectionPool(host='self-signed.badssl.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)) while doing a GET request to URL: https://self-signed.badssl.com/

As we can see error occurred. To prevent that behaviour you need to add --verify=no to the command:

$ https --verify=no https://self-signed.badssl.com/
HTTP/1.1 200 OK
Cache-Control: no-store
Connection: keep-alive
Content-Encoding: gzip
Content-Type: text/html
Date: Sun, 16 May 2021 08:10:00 GMT
ETag: W/"60357389-1f6"
Last-Modified: Tue, 23 Feb 2021 21:28:41 GMT
Server: nginx/1.10.3 (Ubuntu)
Transfer-Encoding: chunked

<!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <link rel="shortcut icon" href="/icons/favicon-red.ico"/>
  <link rel="apple-touch-icon" href="/icons/icon-red.png"/>
  <title>self-signed.badssl.com</title>
  <link rel="stylesheet" href="/style.css">
  <style>body { background: red; }</style>
</head>
<body>
<div id="content">
  <h1 style="font-size: 12vw;">
    self-signed.<br>badssl.com
  </h1>
</div>

</body>
</html>

cURL

Similarly, let's to the same with cURL.

$ curl https://self-signed.badssl.com/
curl: (60) SSL certificate problem: self signed certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

Message error is different but we know what is going on. How to overcome this? Just one flag: -k

$ curl -k https://self-signed.badssl.com/
<!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <link rel="shortcut icon" href="/icons/favicon-red.ico"/>
  <link rel="apple-touch-icon" href="/icons/icon-red.png"/>
  <title>self-signed.badssl.com</title>
  <link rel="stylesheet" href="/style.css">
  <style>body { background: red; }</style>
</head>
<body>
<div id="content">
  <h1 style="font-size: 12vw;">
    self-signed.<br>badssl.com
  </h1>
</div>

</body>
</html>

Simple isn't it?

Timing breakdown in HTTPie and cURL

Personally I used it once but I believe it can be useful for some sort activities. However much better approach is to use dedicated tool for that like: Lighthouse

httpie

I've haven't found such functionality here.

cURL

Very long one-liner but does the job

$ curl -L --output /dev/null --silent --show-error --write-out 'lookup:        %{time_namelookup}nconnect:       %{time_connect}nappconnect:    %{time_appconnect}npretransfer:   %{time_pretransfer}nredirect:      %{time_redirect}nstarttransfer: %{time_starttransfer}ntotal:         %{time_total}n' google.com
lookup:        0.028137
connect:       0.053649
appconnect:    0.000000
pretransfer:   0.053828
redirect:      0.073857
starttransfer: 0.204022
total:         0.204437

Configuration file (HTTPie vs cURL)

Commonly in the config file you can adjust your environment, set some defaults parameters to make your life easier. Consequently, you no longer need to write these parameters over and over again.

httpie

Location is here: ~/.config/httpie/config.json

{     "__meta__": {         "about": "HTTPie configuration file",         "help": "https://github.com/jkbrzt/httpie#config",         "httpie": "0.9.9"     },

 

    "default_options": [         "--verbose",         "--traceback",         "--auth-type=edgegrid",         "--print=Hhb",         "--timeout=300",         "--style=autumn",         "--session-read-only","~/.httpie-custom-headers.json",=         "-adefault:"     ] }



$ cat ~/httpie-custom-headers.json
{
    "headers": {
        "Pragma": "foo"
     }
}

As you can see you can set here mamy elements like: headers, verbosity, printing elements etc. It's quite good.

cURL

Location is here: ~/.curlrc

Example

# change referrer URL
-e "https://www.google.com"

# change user agent 
-A "CUSTOM USER AGENT"

# some headers
-H  "Accept: text/html"
-H  "Accept-Language: en-US,en;"

Really useful.

Cookies

httpie

It sends cookies as regular headers so please go to headers paragraph.

cURL

Quite simple. Please have a look at schema:

curl -c "PARAM1=value1" URL

URL shortcuts

This is small part of the ecosystem 🙂

httpie

$ http :/get
HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Length: 254
Content-Type: application/json
Date: Mon, 01 Feb 2021 21:42:21 GMT
Server: gunicorn/19.9.0

{
    "args": {},
    "headers": {
        "Accept": "*/*",
        "Accept-Encoding": "gzip, deflate",
        "Connection": "keep-alive",
        "Host": "localhost",
        "User-Agent": "HTTPie/2.3.0"
    },
    "origin": "172.17.0.1",
    "url": "http://localhost/get"
}

cURL

For this particular case I haven't find any cURL equivalent.

Work with jq

jq is simple JSON CLI parser. I personally used it many times.

httpie

It works well with jq but you have to force HTTPie to print only body of the response to the stdout. It can be set in the configuration file or via parameter --print=b

$ http --print=b :/get | jq .origin
"172.17.0.1"

cURL

cURL works well too with jq. Just remember to switch to silent mode -s

$ curl -s localhost/get | jq .origin
"172.17.0.1"

Timeout

Many times timeout play key role especially when automation is being used.

HTTPie

In this case we simply need to add flag --timeout with value. Schema is below:

$ http --timeout=2.5

As example let's try to fetch web from private network

$ http --timeout=2 http://10.10.10.11

http: error: Request timed out (2.0s).

And another example - downloading 100MB of data

$ http --download --timeout=1 https://speed.hetzner.de/100MB.bin
HTTP/1.1 200 OK
Accept-Ranges: bytes
Connection: keep-alive
Content-Length: 104857600
Content-Type: application/octet-stream
Date: Mon, 03 May 2021 14:50:15 GMT
ETag: "5253f0fd-6400000"
Last-Modified: Tue, 08 Oct 2013 11:48:13 GMT
Server: nginx
Strict-Transport-Security: max-age=15768000; includeSubDomains

Downloading 100.00 MB to "100MB.bin"
Done. 100.00 MB in 4.05588s (24.66 MB/s)

As you can see --timeout doesn't work with --download mode.

cURL

Similarly to httpie curl has timeout flag which only limits the connection phase.

Schema:

--connect-timeout <seconds>

And example

$ curl --connect-timeout 2 http://10.10.10.11
curl: (28) Connection timed out after 2001 milliseconds

curl has as advantage here due to another flag --max-time which works well with downloading.

Schema:

-m, --max-time <seconds>

And example

$ curl --max-time 1 https://speed.hetzner.de/100MB.bin -o file.bin
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  6  100M    6 6383k    0     0  6390k      0  0:00:16 --:--:--  0:00:16 6390k
curl: (28) Operation timed out after 1000 milliseconds with 6536896 out of 104857600 bytes received

Other HTTPie vs cURL

httpie

These are two areas which are specific for httpie which can be game changer in this clash

• Sessions
• Plugins

Sessions is a way to add persistence to the subsequent performances. Basically when you need to add some headers, cookies in every HTTP calls one by one. This is saving time feature. You know that annoying filling then you had to type the same set of strings in the long one-liner. cURL has -c and -b options but only to play with cookies not headers.

Examples

$ http "http://httpbin.org/bearer" "Authorization: nbmnb"
HTTP/1.1 401 UNAUTHORIZED
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Length: 0
Content-Type: text/html; charset=utf-8
Date: Thu, 06 May 2021 19:50:07 GMT
Server: gunicorn/19.9.0
WWW-Authenticate: Bearer

$ http  --session=/tmp/session.json "http://httpbin.org/bearer" "Authorization: nbmnb"
HTTP/1.1 401 UNAUTHORIZED
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Length: 0
Content-Type: text/html; charset=utf-8
Date: Thu, 06 May 2021 19:50:45 GMT
Server: gunicorn/19.9.0
WWW-Authenticate: Bearer

$ cat /tmp/session.json 
{
    "__meta__": {
        "about": "HTTPie session file",
        "help": "https://httpie.org/doc#sessions",
        "httpie": "2.3.0"
    },
    "auth": {
        "password": null,
        "type": null,
        "username": null
    },
    "cookies": {},
    "headers": {
        "Authorization": "nbmnb"
    }
}

$ http  --session=/tmp/session.json "http://httpbin.org/bearer"
HTTP/1.1 401 UNAUTHORIZED
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Length: 0
Content-Type: text/html; charset=utf-8
Date: Thu, 06 May 2021 19:51:15 GMT
Server: gunicorn/19.9.0
WWW-Authenticate: Bearer

Plugins provide additional mechanisms to the core. However currently are limited to authentication area only. List of plugins is here

Example from here

# One off call
http --auth-type=ntlm --auth='domain\username:password' example.org

# Create session
$ http --session=logged-in --auth-type=ntlm --auth='domain\username:password' example.org

# Re-use auth
$ http --session=logged-in POST example.org hello=world

cURL

curl is much bigger tool than httpie is and that's why list of unique features is longer then for httpie. Let's list some of them.

• mail
• compressed
• FTP
• TLS
• Kerberos
• DNS
• HTTP2
• keepalive
• limit-rate
• parallel
• SOCKS
• speed limit

As you can see many of these features are out of scope of HTTP world. For some of us this can be useful cause using one tool we can do many different thongs but maybe it's better to concentrate on small chunk ot the pizza? I leave it unanswered.

Examples

# ftp

$ curl ftp://ftp.nvg.org/pub/
drwxrwxr-x   2 42       21           4096 Dec 19  2000 IPv6
drwxrwxr-x   3 42       21           4096 Dec 20  2000 NetBSD
drwxr-xr-x  15 1868     16           4096 Apr  7  2017 bbc
drwxr-xr-x   3 1027     0            4096 Dec 13  1995 cezton
drwxrwsr-x  18 607      32133        4096 Apr 28 20:23 cpc

[...]
drwxr-xr-x   2 42       21           4096 Dec 19  2000 unix
drwxrwxr-x   9 42       21           4096 Nov  8  2009 vms
drwxrwxr-x   2 0        0            4096 Oct 16  2000 warlock
drwxr-xr-x   3 0        0            4096 Feb 10  1999 windows

Documentation

https://httpie.io/docs

https://curl.se/docs/manpage.html

Summary of HTTPie vs cURL

Both tools are really good. As the http light command line client are comparable. httpie younger has some nice features which are really usefully nowadays especiallu to talk using json with API endpoints. On the other hand curl is more than just light http commad line client it's swiss army fork.

Generally speaking if i wouldn't known curl I would switch to httpie cause it covers 99% of my activities. but I', kind a old dog and I know curl pretty well so i stay with it (whit annoying contect -type json ) but I'll check from time httpie and its development especially plugins

Parts

Please continue reading

Part 1

Part 2