Home - dejanu/linux GitHub Wiki

CURL specify multiple url in the same command vs WGET when you want to download files

-i include headers

-I = -X HEAD, but might not be implemented

# specify http method
--request GET or -X GET

-s silent
-v verbose

curl -o myfile save output to myfile
curl -O save the file as its existing name
 
-H set header
-X set http verb
-d set data (body)

BASIC auth

curl http://name:[email protected]/full/path/to/file

#specify user and password separately
curl -u name:passwd http://machine.domain/full/path/to/file

GET request and specify headers

# JSON header
curl -i -H "Accept: application/json" -H "Content-Type: application/json" http://hostname/resource`    

# XML header

curl -H "Accept: application/xml" -H "Content-Type: application/xml" -X GET http://hostname/resource`

POST:

Request parameters: curl --data "param1=value1&param2=value2" http://hostname/resource

For file upload: curl --form "[email protected]" http://hostname/resource
curl -T "{$(echo *.txt | tr ' ' ',')}" ftp://XXX/ -user YYY

RESTful HTTP Post:

curl -X POST -d @filename http://hostname/resource
curl -i -X POST -H "Content-Type: application/json" -d '{"name":"your_parameter"}' 127.0.0.1:5000/data/name .

curl sends POST requests with the default content type of application/x-www-form-urlencoded. If you want to send a json request, you will have to specify the correct content type header:

curl -vX POST http://server/api/v1/places.json -d @testplace.json \ --header "Content-Type: application/json" .

PUT

curl -i -X PUT httpbin.org/put -H "Content-type:application/json" -d '{"hello":"world"}' For logging into a site (auth):

curl -d "username=admin&password=admin&submit=Login" --dump-header headers http://localhost/Login
curl -L -b headers http://localhost/

**Save the content to a file ** curl -o website https://domain.com

Common Options

-#, --progress-bar Make curl display a simple progress bar instead of the more informational standard meter.

-b, --cookie <name=data> Supply cookie with request. If no =, then specifies the cookie file to use (see -c).

-c, --cookie-jar <file name> File to save response cookies to.

-d, --data <data> Send specified data in POST request. Details provided below.

-f, --fail Fail silently (don't output HTML error form if returned).

-F, --form <name=content> Submit form data.

-H, --header <header> Headers to supply with request.

-i, --include Include HTTP headers in the output.

-I, --head Fetch headers only.

-k, --insecure Allow insecure connections to succeed.

-L, --location Follow redirects.

-o, --output <file> Write output to . Can use --create-dirs in conjunction with this to create any directories specified in the -o path.

-O, --remote-name Write output to file named like the remote file (only writes to current directory).

-s, --silent Silent (quiet) mode. Use with -S to force it to show errors.

-v, --verbose Provide more information (useful for debugging).

-w, --write-out <format> Make curl display information on stdout after a completed transfer. See man page for more details on available variables. Convenient way to force curl to append a newline to output: -w "\n" (can add to ~/.curlrc).

-X, --request The request method to use.

POST

When sending data via a POST or PUT request, two common formats (specified via the Content-Type header) are:

  • application/json
  • application/x-www-form-urlencoded

Many APIs will accept both formats, so if you're using curl at the command line, it can be a bit easier to use the form urlencoded format instead of json because

  • the json format requires a bunch of extra quoting
  • curl will send form urlencoded by default, so for json the Content-Type header must be explicitly set

This gist provides examples for using both formats, including how to use sample data files in either format with your curl requests.

curl usage

For sending data with POST and PUT requests, these are common curl options:

  • request type

    • -X POST
    • -X PUT

  • content type header

  • -H "Content-Type: application/x-www-form-urlencoded"

  • -H "Content-Type: application/json"

  • data

    • form urlencoded: -d "param1=value1&param2=value2" or -d @data.txt
    • json: -d '{"key1":"value1", "key2":"value2"}' or -d @data.json

ssh into VM:

  1. add etho0 into /etc/network/interfaces file:
    auto eth0
    allow-hotplug eth0
    iface eth0 inet dhcp

  2. in the /etc/init.d/ dir you have all the scripts used when the system boots:
    /etc/init/d/networking {start|stop|reload|restart}

  3. The DNS settings are in /etc/resolv.conf

  4. OpenSSH server reads a configuration file when it is started. Usually this file is /etc/ssh/sshd_config:
    PermitRootLogin yes

WGET

header = user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36
header = accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3
  • Use link for file : $ wget -O wgetresponse -i wgetlink
⚠️ **GitHub.com Fallback** ⚠️