In the previous post, I talked about how to securely pass secrets to command line programs, and in this post I’m going to use cURL as an example. The two most common secrets you’ll want to pass to curl are API tokens and Basic authentication (old-school username and password).

Passing API Tokens

Almost every example I’ve seen that passes API tokens to curl uses environment variables to set the Authorization: Bearer header:

> curl -H "Authorization: Bearer $API_TOKEN" https://api.example.com/resource

To fully demonstrate this, I am going to run a mock API server in Python. Without any credentials, it fails to authenticate:

> curl http://127.0.0.1:8000/resource
{"errors": [{"reason": "Invalid Token"}]}

A valid token is required to access it:

> curl -H "Authorization: Bearer token123" http://127.0.0.1:8000/resource
{"msg": "success"}

As mentioned in the last post, even putting the token in a file (or password manager) to populate the environment variable is unsafe:

> cat api_token.txt
token123
> export API_TOKEN=$(cat api_token.txt)
> curl -H "Authorization: Bearer $API_TOKEN" http://127.0.0.1:8000/resource
{"msg": "success"}

But curl -H does have the ability to read from a file, even though it’s a bit obscure. The man page mentions special @file syntax to read from a file:

-H, --header <header/@file>

…

This option can take an argument in @filename style, which then adds a header for each line in the input file. Using @- makes curl read the header file from stdin.

One issue with this is that the file must contain the header prefix text, too. For example, to use this for an API token the file would need to be something like this:

> cat api_token_header.txt
Authorization: Bearer token123
> curl -H @api_token_header.txt http://127.0.0.1:8000/resource
{"msg": "success"}

The problem here is we don’t really want to store the full header and token in a file. We want to read the token from a password manager. But password managers will typically only return the secret itself, and we need to add the Authorization: Bearer prefix text.

Process substitution again comes to the rescue. It’s a little ugly, but here’s how it works:

> curl -H @<(echo "Authorization: Bearer $(cat api_token.txt)") \
    http://127.0.0.1:8000/resource
{"msg": "success"}

The key here is the echo command, which appends the token to the header text:

> echo "Authorization: Bearer $(cat api_token.txt)"
Authorization: Bearer token123

Combining this with <(...) process substitution securely passes the token via a pipe. The token is not in the environment and it is not passed on the command line. If you tried to view the ps arguments, all you would see is the temporary pipe file:

> pgrep -fl curl
34138 curl -H @/dev/fd/16 http://127.0.0.1:8000/resource

It’s also important to point out that echo is a built-in shell command. If /bin/echo were used, then the token would again be (very briefly) exposed to ps.

To show a real-world example with a real password manager, here’s how this could look with op, the 1Password CLI:

> export TOKEN_REF="op://Infrastructure/Mock API/token"
> curl -H @<(echo "Authorization: Bearer $(op read "$TOKEN_REF")") \
    http://127.0.0.1:8000/resource
{"msg": "success"}

You don’t have to put the 1Password token reference into an environment variable, but it is completely safe to do so. The only information that leaks into the environment is the reference, not the secret itself.

Passing Basic Authentication Credentials

This mock API can also be accessed using a username and password via curl -u/--user:

> curl -u user:pass123 http://127.0.0.1:8000/resource
{"msg": "success"}

Unfortunately, the -u option has no direct ability to read from a file, but cURL variables can be used with the curl --variable option (a feature in cURL 8.3.0+) to achieve the same goal:

--variable <[%]name=text/@file>

Set a variable with "name=content" or "name@file" (where "file" can be stdin if set to a single dash ("-")).

Variables can be used by adding --expand- in front of any existing option. Combining variables with process substitution can again be used to read the password from a file:

> curl --variable pw@<(cat password.txt) \
     --expand-user 'user:{{pw:trim}}' \
     http://127.0.0.1:8000/resource

There are three new concepts going on here:

  1. The --variable option sets a variable named pw to the contents of password.txt via process substitution.

  2. The --expand-user option is the same as -u/--user but with variable expansion enabled, in this case the pw variable via {{...}}.

  3. The :trim removes the trailing newline from the expanded variable.

And, finally, here’s an example using op to read from a password manager:

> export PASSWORD_REF="op://Infrastructure/Mock API/password"
> op read "$PASSWORD_REF"
pass123
> curl --variable pw@<(op read "$PASSWORD_REF") \
     --expand-user 'user:{{pw:trim}}' \
     http://127.0.0.1:8000/resource
{"msg": "success"}

That’s somewhat convoluted, but it is secure. The password never leaks out into ps or the environment:

> pgrep -fl curl
55299 curl --variable pw@/dev/fd/16 --expand-user user:{{pw:trim}} http://127.0.0.1:8000/resource

And that wraps up how to pass your API token and password securely to cURL!