1. find if a server is listening on a specific port
  2. nc -zv 127.0.0.1 80

    source
  3. obtain new dynamic IP following reconfiguration of DHCP pool in router
  4. sudo ifconfig
    // I see that the wired interface is named enp0s31f6
    sudo ifconfig enp0s31f6 down
    sudo ifconfig enp0s31f6 up

  5. turn off / on wireless on Ubuntu 16.04
  6. source

    nmcli radio wifi off
    nmcli radio wifi on
    nmcli radio help
    nmcli radio wifi help
    

  7. find IP address of Internet host or domain / subdomain
  8. There's lots of ways with subtle differences:

    Some notes on the various subtleties involved:

    The dig program uses the OS resolver libraries and, as such, should be preferred over nslookup.

    The dig program is also not using the local /etc/hosts file whereas the getent hosts incantation and the host program both do. Depending on your use case you may therefore wish to use dig over getent or host (e.g. when you are trying to troubleshoot what an external client might resolve up to).

  9. how to use HTTP Basic Authentication with either wget or curl
  10. Using wget:

    wget -q -d -S -O - --auth-no-challenge --http-user=USERNAME --http-password=PASSWORD 'https://example.com/some/path?a=1&b=2'
    Using curl:
    curl -v --user USERNAME:PASSWORD 'https://example.com/some/path?a=1&b=2'

    As an added bonus, the verbose output on both incantation reveals (among other things):

  11. how to send an HTTP POST request directly to a server socket and read the response
  12. I was experimenting with a multipart POST failure and so I wrote the following script to directly send a POST request to some socket and read back the response from the same socket:

    #!/usr/bin/env bash
    if [[ ($# -ne 1) ]]
    then
        echo "usage is $0 "
    else
        FILE=$1
        exec 3<>/dev/tcp/some.server.edu/80 && cat $FILE >&3 && cat <&3
    fi

    E.g. in my case the request that was failing was:

    POST /csccli/getProperties?outputFormat=vot HTTP/1.1
    Content-Type: multipart/form-data
    Content-Length: 332
    Host: some.server.cfa.harvard.edu
    Connection: Keep-Alive
    User-Agent: Apache-HttpClient/4.5.2 (Java/1.7.0_95)
    Accept-Encoding: gzip,deflate
    
    ---~.-~.-~.-~.-~.-~.-~.-~.-~.
    Content-Disposition: form-data; name="query"
    Content-Type: text/plain; charset=ISO-8859-1
    Content-Transfer-Encoding: 8bit
    
    SELECT COUNT(*) FROM master_source
    
    ---~.-~.-~.-~.-~.-~.-~.-~.-~.--

    The above POST request was failing because the server couldn't find the query parameter which seemed weird to me (at that time) as the request clearly includes such a parameter. I eventually discovered that the failure was due to the fact that Content-Type header also has to indicate the boundary in case of multipart requests.

  13. how to find saved wifi password
  14. Sources: here and here.
    $ ls -a /etc/NetworkManager/system-connections/
    $ sudo cat WIFI_SSID_Name | grep psk
  15. switch vs. bridge
  16. source

    A switch is not a proper technical term (the IEEE 802.1D standard does not even mention the word "switch"). So a switch is just a marketing name for a multiport bridge. A bridge, in turn, is a layer 2 (data link) device that splits collision domains but not multicast domains. A hub is level 1 device that doesn't split collision domains (and therefore neither does it split broadcast domains) and knows nothing about MAC addresses. A hub only sees bits as it resides at the physical layer and simply repeats them to all its ports. A bridge is more intelligent in that it only floods initially, but over time, learns the MAC addresses of the computers connected to its ports and will eventually forward frames (the name for Layer 2 packets) only to the right ports.

  17. how to see what IP address a hostname or domain is resolved to via the hosts file
  18. Use getent hosts. E.g. on my machine:

    $ tail -1 /etc/hosts
    127.0.0.8 foo
    $ getent hosts foo
    127.0.0.8       foo

  19. How to sent a POST on https using curl (with basic Authorization headers too)
  20. Notes:
    curl -X POST -k -v -H 'Authorization:Basic Y3hjbG9naW4tZ2VuZXJpYy1jbGllbnQtYXBwLnVzZXJuYW1lOmN4Y2xvZ2luLWdlbmVyaWMtY2xpZW50LWFwcC5wd2Q=' https://cdatest.cfa.harvard.edu/cxclogin-dbdal-rest/jax-rs/userdb/serviceTicket/aauvc4d1dl4g3ovz3ejt0l5h0wv88iiri409bx5/authuser?useMode=normal
          
  21. How to setup a VPN using openconnect on Linux
  22. How to setup a VPN using vpnc on Linux
  23. how to select wireless network from the command line
    nmcli c
    sudo nmcli c up uuid <paste uuid here>
  24. how to scp a file through an intermediate host (a.k.a. jump host)
  25. how to open a shell through two successive SSH hops where SSH public key authentication is possible only on the first hop
  26. show local listening ports and which processes listen on them:
  27. sudo lsof -P -i -n
    More recently, I did the following to find the port an emacs daemon listens to:
    ps aux | grep -i emacs | grep -v grep
          
    … and then (after having discovered the port number from the previous command):
    $ lsof -Pan -p 32605 -i
    COMMAND   PID       USER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
    emacs   32605 mperdikeas    5u  IPv4 5351989      0t0  TCP 127.0.0.1:58545 (LISTEN)
          

    Yet more recently, I used the netstat program which has the option -l for listening sockets and can further narrow them down by type:

    -lt
    listening TCP ports
    -lu
    listening UDP ports
    -lx
    listening UNIX ports
    … together with the -p (--program) option one can get the PID and the name of the program listening on that port.

    For example, to find the process ids of all Emacs daemons listening on UNIX sockets I did a:

    netstat -lxp | grep -i emacs
            

    The ss tool can also provide the same information as the last incantation with:

    ss -lpx | grep -i emacs

  28. recursively copy directory to remote machine while maintaining symlinks
  29. When recursively copying with scp, symlinks are copied rather than being preserved, and so rsync has to be used instead.

    The following incantation has worked for me:

    rsync -avz -e ssh esac-rawdar/ user@172.17.12.55:/home/user/ESA/RAWDAR
            
    … and recursively copied folder "esac-rawdar" to the remote machine while preserving symlinks.

    CAVEAT:when the directories to copy over the network include cryptographic file containers like TrueCrypt it may be the case that the container (for privacy reasons) does not update the modification timestamp (this is configurable in TrueCrypt) and since the total size of the container also doesn't change, then the heuristic algorithm that scp uses may fail to copy the file.
    In those cases the --ignore-times (or -I) or --checksum flags are necessary. See this SO answer and my comment to it.

  30. how to enable / disable wireless from the command line
  31. source

    Ubuntu 14.04:

    nmcli nm wifi off
    nmcli nm wifi on
            

    Ubuntu 16.04:

    nmcli r wifi off
    nmcli r wifi on
            

  32. do a double (two spans) SSH tunnel between two machines (with a third machine sitting in the middle)
  33. bind to a port, continuously, and print on standard out whatever gets sent to that port
  34. Use the Netcat tool (abbreviation: nc) with the -k option as in (e.g. to bind to port 4444):
    nc -k -l 4444
    NB: In my Ubuntu system I've tried the above with the nc from the netcat-openbsd package (not the alternative nc is available in the netcat-traditional package).
  35. see the redicts of a web page
  36. Use the curl utility with the -I and -L options (to show the headers and to follow the redirects, respectively) as follows:
    curl -I -L www.google.com
  37. use a single outgoing IP address
  38. When behind a corporate NAT/firewall the packets you sent out (e.g. from you browser) may seem to emerge from two different IP addresses which could result in your being evicted. In those cases, find a remote system you have access to and open an ssh tunnel to it:
    ssh -L 2095:localhost:2095 root@trusted.remote.system.com
    You can then point your browser to the address: localhost:2095

    (the above instructions work provided your browser was redirected to port 2095 of the remote host to begin with). This will stabilize the IP address the remote system sees your packets coming from.

  39. virtual Ethernet interface
  40. A virtual IP address is created as simply as:
    ifconfig eth0:1 172.31.129.121
    and is only visible on the same machine and (?) LAN segment. The above incantation produces an eth1 interface with the indicated address. Local servers can then bind on that interface. You can also ping it from a remote machine and watch the MAC address enter that remote machine's arp cache. However it is not currently possible to assign a MAC address to that virtual interface that's different from the real one.
  41. How to view multicast groups
  42. Below are some incantations that work (I haven't correlated amongst them):
    netstat -g
    /sbin/ip maddress list
    ifconfig eth0 | grep MULTICAST
          
  43. Multicast groups
  44. Joining and leaving multicast groups can be done totally at the application layer when one opens a socket. Accordingly for software that has this concept (like JBoss AS7 in 'high availability'/clustering mode), you provide the multicast address at the command line, you don't need to set anything administratively in the routing tables or elsewhere.

    NB:There's no such concept as 'joining a multicast group'. The concept is 'joining a multicast group on a particular interface'.

  45. arp
  46. To see the arp tables (I believe there is some caching and therefore also expiry involved) of your local machine do a:
    sudo arp -a
    This will show known IP and MAC addresses for the hosts you've send packets to. Surprisingly (but perhaps not so since everything is routed and the arp tables only cache values for the LAN segment, i.e. the 1st hop towards the router), you'll see very few hosts there. This is the output on my machine:
    $ sudo arp -a
    kerio_control.neuropublic.gr (172.31.128.252) at 00:0f:20:32:c5:49 [ether] on eth0
    iasonaslb1.neuropublic.gr (172.31.128.247) at 00:0f:20:32:c5:49 [ether] on eth0
    iasonas.neuropublic.gr (172.31.128.249) at 00:0c:29:48:58:7c [ether] on eth0
    ? (172.31.128.140) at 02:bf:ac:1f:80:8e [ether] on eth0
            
  47. wireless problems in Ubuntu 12.04 LTS Precise
  48. Original article here. Gist is to try one of these:
  49. 5 Best Linux/Ubuntu compatible USB WiFi cards
    1. AirLink101 AWLL6075 Wireless N Mini USB Adapter
    2. Medialink – Wireless N USB Adapter – 802.11n
    3. ASUS (USB-N13) Wireless-N USB Adapter
    4. Panda Mini Wifi (b/g/n) 150Mbps Card
    5. TP-Link TL-WN722N 150Mbps High Gain Wireless USB Adapter
  50. use nc for HTTP troubleshooting
  51. First of all use nc.traditional:
    sudo apt-get install netcat-traditional
            
    Then spawn the nc.traditional binary in a loop (that's ok since the socket opens and dies with every request according to the stateless character of the HTTP protocol):
    while true ; do nc.traditional -l -p 8085 ; done
            
    With the above command we've opened a manual HTTP server listening at port 8085. One can now point the browser to: localhost:8085/foo and we'll see the request on the nc.traditional output. What's more, we'll be able to paste a response and see it in the browser. Moreover, after the exchange the nc.traditional binary won't die but (due to the while true) be available to service the next request coming from the browser. This allows us to test the cookie setting behaviour and parameters. E.g. (see the Netscape spec and this SO discussion for more) we can use this setting to test the cookie behavior when the cookie path attribute is used:
    1. launch nc.traditional as described above
    2. point the browser to localhost:8085/foo
    3. the following HTTP request appears at nc.traditional's screen
    4. GET /foo HTTP/1.1
      Host: localhost:8085
      User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:15.0) Gecko/20100101 conkeror/1.0pre (Debian-1.0~~pre+git1206072207-~nightly1)
      Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
      Accept-Language: en-us,en;q=0.5
      Accept-Encoding: gzip, deflate
      Connection: keep-alive
      Cookie: JSESSIONID=891d66f2-29fa-4568-9228-c917c5bf1317
      
      
                
    5. paste the following response to nc.traditional's screen:
    6. HTTP/1.1 200 OK
      Server: Apache-Coyote/1.1
      X-Powered-By: JSF/2.0
      Set-Cookie: JSESSIONID=891d66f2-29fa-4568-9228-c917c5bf1317; Path=/; HttpOnly
      Content-Type: text/html;charset=UTF-8
      Content-Length: 16
      Date: Tue, 04 Sep 2012 12:22:46 GMT
      
                  boo
      
                
      (take care with the Content-Length field)
    7. setting the path to / means that the cookie will be present in any subsequent request, e.g. http://localhost/booboo
    8. when the cookie path is not set to / the cookie will be present only in tail matching requests.
  52. copy a tarball with permissions between two different machines (no scp)
  53. sender:
    tar cpf - git-jboss/ | nc 172.31.129.12 8877
    receiver:
    nc -l -p 8877 | tar xpvf -
  54. make sure a service is listening at a given port
  55. Use nc with the -w option as in (for a timeout of 5 seconds):
    nc -w 5 94.66.59.15 80
        
  56. How to mount a smbfs file system with read/write permissions
  57. First, create the mount-point (/media/athina-prj) in the below example.
    Then, obtain your userid and groupid (using id) and then do a:
    sudo mount -t smbfs -o username=SEMANTIX\\mperdikeas,password=mperdikeas,dir_mode=0777,file_mode=0777,uid=mperdikeas,gid=mperdikeas //192.168.0.2/prj /media/athina-prj
        
  58. How to take a Linux box off the network
  59. log-in to a console and do:

    sudo ifconfig 

    (to find the link you wish to bring down, usually "eth0")

    sudo ifconfig eth0 down
    
    do work that has to be executed offline, and then:

    sudo ifconfig eth0 up
    
  60. SSH public key authentication