Skip to content

Linux Networking and Security Fundamentals-Day1

Hello there! Here you will find the lab material for day 1 of the Linux Networking & Security Fundamentals webinar. Enjoy!

Note

My main IP network is either: 10.0.2.0/24 or 10.0.42.0/16, depending on the lab being used for the day.

I run a Debian server and client, Fedora server and client, Ubuntu server, and some mystery Linux distributions. (Really, it's whatever I have available at the time!)

See this link for more information on how to set up your lab to follow along.

Disclaimer & Warning Link Here


Part I: Networking 101

In Part I of the webinar we start with some basic computer networking including a review of a typical local area network setup, basic TCP/IP, and a primer of the OSI model. This portion of the webinar is designed to act as a foundation for the rest of the course.

TCP/IP

TCP/IP is a suite of protocols used by computers to communicate with each other. It includes the protocols TCP, and IP, but also many others, such as ICMP, ARP, HTTP, FTP, POP, SMTP, DNS, DHCP, and so on. IP addresses are used by each computer that runs TCP/IP to identify them, and to facilitate the communication of packets of data.

A typical example of an IP network is: 192.168.1.0. The network portion of the address is 192.168.1. The host portion is 0. It's the host portion that differentiates between systems on the same network. For example, one computer might have the IP address 192.168.1.73. Another might have 192.168.1.142. And the gateway might use 192.168.1.1. It's that last octet, that last number that sets each system apart. Other IP networks can have a larger host portion. For example, in the 172.17.0.0 network, the network portion is 172.17, and the host portion is 0.0. So, a system on the 172.17 network might have an address such as 172.17.51.3. "51.3" would be the host portion of the IP address.

Lab 1-1

Use the ip a command

  • In the console (or terminal) type the ip a command and view the results. Here's an example on a Debian server:
root@deb51:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 52:54:00:f0:2b:b4 brd ff:ff:ff:ff:ff:ff
    inet 10.0.2.51/24 brd 10.0.2.255 scope global enp1s0
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:fef0:2bb4/64 scope link 
       valid_lft forever preferred_lft forever
3: enp7s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 52:54:00:0a:2a:c9 brd ff:ff:ff:ff:ff:ff

This shows two interfaces:

  1. lo, which is the local loopback, something that is found by default on every system that runs TCP/IP. "inet" shows the IPv4 address, which for lo is 127.0.0.1.
  2. enp1s0, which is the main network interface, used to access other systems and the Internet. The IPv4 address is 10.0.2.51.

Info

For help with the ip command type ip --help, and for more in-depth information, see the manual page man ip.

The OSI Reference Model

The Open Systems Interconnection (OSI) reference model is used to define how data is transmitted and received between systems. It defines how protocols work, and how programs will use those protocols. It is made up of seven layers, listed from layer 7 down to layer 1 below:

  • Layer 7: Application
  • Layer 6: Presentation
  • Layer 5: Session
  • Layer 4: Transport
  • Layer 3: Network
  • Layer 2: Data Link
  • Layer 1: Physical

Different protocols work on different layers. For example, in Lab 1-1 we used the ip a command. This displayed the IP address of the system (10.0.2.51). IP addresses exist on layer 3 of the OSI model: the network layer. Other protocols work on other layers. For instance, TCP works on layer 4 (transport), and HTTP works on layer 7 (application). When you have different protocols working on different layers, they are considered to be stacked upon each other. This leads to terms such as "OSI stack", "TCP/IP stack", or simply "network stack". The OSI model can be very helpful when designing programs, designing networks, and troubleshooting network connections, as well as doing packet analysis.

There are no other labs in this section. See the reference links below for more information about TCP/IP and the OSI model.

TCP/IP Reference Links


Part II: Networking Services & Configuration Files

The core networking services that are used by many Linux distributions include the networking, networkd, and NetworkManager services. These are the underlying services for other networking services such as HTTP, DNS, DHCP, and so on. Normally, a system will only use one of the three at any given time. The following three subsections include labs to work with each of the three.

The networking service

The networking service is mainly used by Debian servers. Therefore, you will need a Debian server (or some other system using the networking service) to accomplish the labs in this section.

Lab 2-1

Use common commands associated with the networking service

  • Verify if the networking service is running:

    systemctl status networking.service

  • Enable/disable the service:

    systemctl enable networking and systemctl disable networking

  • Start/stop the service:

    systemctl start networking and systemctl stop networking

  • Enable and start the service with one command:

    systemctl --now enable networking

  • Disable and stop the service with one command:

    systemctl --now disable networking

When you are finished, enable and start the networking service so that we can use the system normally.

Info

For more information about the systemctl command, type systemctl --help and man systemctl

Lab 2-2

Configure networking and down and up the interface

  • Analyze the networking configuration:

    ip a

  • View and modify the network configuration file.

    It is located at: /etc/networking/interfaces

Example

Here's an example of the interfaces file in Debian. You can use Debian's built-in text editor: Nano, but I prefer Vim. To install Vim, type apt install vim.

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
allow-hotplug enp1s0
iface enp1s0 inet static
    address 10.0.2.51/24
    gateway 10.0.2.1

If you configured your system to obtain an IP address during the installation of Debian, then it would show "dhcp" instead of "static", and there would be no address or gateway entries. But you should configure your server to use a static IP address based on your IP network as shown in the example. If you already have a static IP address, try changing the address to something different. Then, save and quit out of the file. If you are using Vim, save and quit by typing :wq and pressing enter.

Info

For more information about Vim, see this link.

  • Deactivate and reactivate the network interface:

    ifdown <interface> and ifup <interface>

    Here's a real example, performing both actions with a single command. In this case, the network interface is enp1s0 but your interface name will most likely be different.

    ifdown enp1s0 && ifup enp1s0

  • Test the connection:

    ping example.com or use whatever test site you wish.

Lab 2-3

Verify the DNS configuration

Most likely, the DNS configuration is correct. But we should check it anyway, and then test it.

  • Analyze the DNS configuration:

    cat /etc/resolv.conf

    This should show an IP address of a DNS server. For example, our Debian server above shows the DNS server IP as 10.0.2.1. That is the IP address of the built in DNS forwarder in the virtualization system (it is the same as the gateway address). In some cases you might have to modify the address to something else - for example, the IP address of the router on your network, or some other device's address on the LAN. But if you are using a virtualization system (VirtualBox, Vmware, Hyper-V, KVM, etc...) then the DNS forwarder should be the first IP on your computer's IP network. So for example, the Debian server above is on the 10.0.2.0 network. It has an IP of 10.0.2.51 and uses the virtualization system's IP of 10.0.2.1 for DNS and the gateway.

  • Test if DNS is resolving properly:

    ping prowse.tech

    This should translate the name prowse.tech to its current IP address. For example:

    root@deb51:~# ping prowse.tech
    PING prowse.tech (67.205.11.189) 56(84) bytes of data.
    64 bytes from apache2-igloo.allatou.dreamhost.com (67.205.11.189): icmp_seq=1 ttl=49 time=21.9 ms
    64 bytes from apache2-igloo.allatou.dreamhost.com (67.205.11.189): icmp_seq=2 ttl=49 time=19.8 ms
    64 bytes from apache2-igloo.allatou.dreamhost.com (67.205.11.189): icmp_seq=3 ttl=49 time=20.2 ms
    64 bytes from apache2-igloo.allatou.dreamhost.com (67.205.11.189): icmp_seq=4 ttl=49 time=20.9 ms
    ^C
    --- prowse.tech ping statistics ---
    4 packets transmitted, 4 received, 0% packet loss, time 7ms
    rtt min/avg/max/mdev = 19.770/20.698/21.947/0.824 ms
    

    In the example, prowse.tech is being translated (or resolved) to the IP address 67.205.11.189. Also, we are receiving replies. This is what you want so that you can connect properly to the Internet.

Info

For more information on Debian networking (for Debian servers with no GUI) then see this link: https://wiki.debian.org/NetworkConfiguration

systemd-networkd

The systemd-networkd service (or simply networkd) is used by Ubuntu server and in some other special situations. For best results, use an Ubuntu server to accomplish the following labs.

Lab 2-4

Start and stop the networkd service

  • Check the service first:

    systemctl status systemd-networkd

  • Stop the service

    systemctl stop systemd-networkd

  • Start the service

    systemctl start systemd-networkd

Lab 2-5

Analyze and modify the netplan configuration

  • Open the netplan configuration:

    For example: vim /etc/netplan/00-installer-config.yaml

Example

Here's an example of a netplan .yaml file in Ubuntu server:

# This is the network config written by 'subiquity'

network:       
  version: 2
  renderer: networkd
  ethernets:
    enp1s0:
      addresses: [10.0.2.53/24]
      gateway4: 10.0.2.1
      nameservers: 
        search: [example.com]
        addresses: [10.0.2.1]
The network interface name is enp1s0. The IP address of the server is 10.0.2.53. The gateway and DNS server IPs are one and the same: 10.0.2.1. Though you don't have to, I am using square brackets for some of the IP addresses. By doing so, you can easily add IP addresses, by comma-separating them. For example: [10.0.2.53/24,10.0.2.158/24]

  • Modify and apply the netplan configuration:

    If your configuration shows DHCP, then modify it to a static address, similar to the configuration above, but based on your IP network. Otherwise, change the IP address to something different. Then, save and quit out of vim.

    You can have netplan check your configuration by typing netplan try.

    If there are no error messages, proceed by saving the configuration: netplan apply

    Then test it with ping example.com

Lab 2-6

Analyze with networkctl

  • Use the networkctl command to find the available network interfaces. Here's an example of an Ubuntu Server's interfaces:
root@ubuntu-server:~# networkctl
IDX LINK   TYPE     OPERATIONAL SETUP     
  1 lo     loopback carrier     unmanaged 
  2 enp1s0 ether    routable    configured

2 links listed.

Here you can see that the system has the local loopback (like any Linux system by default), and the enp1s0 Ethernet network interface.

  • Use the networkctl status command to see the status of networkd-based interfaces. Here's an example of the results of that command.
root@ubuntu-server:~# networkctl status
           State: routable                         
         Address: 10.0.2.53 on enp1s0              
                  fe80::5054:ff:febc:b314 on enp1s0
         Gateway: 10.0.2.1 on enp1s0               
             DNS: 10.0.2.1                         
  Search Domains: example.com                      

Mar 30 14:41:36 ubuntu-server systemd[1]: Starting Network Service...
Mar 30 14:41:36 ubuntu-server systemd-networkd[682]: Enumeration completed
Mar 30 14:41:36 ubuntu-server systemd[1]: Started Network Service.
Mar 30 14:41:36 ubuntu-server systemd-networkd[682]: enp1s0: IPv6 successfully enabled
Mar 30 14:41:36 ubuntu-server systemd-networkd[682]: enp1s0: Link UP
Mar 30 14:41:36 ubuntu-server systemd[1]: Starting Wait for Network to be Configured...
Mar 30 14:41:36 ubuntu-server systemd-networkd[682]: enp1s0: Gained carrier
Mar 30 14:41:37 ubuntu-server systemd-networkd[682]: enp1s0: Gained IPv6LL
Mar 30 14:41:37 ubuntu-server systemd[1]: Finished Wait for Network to be Configured.

As you can see, the state of the connection is "routable" which means that this system can connect out to other systems and beyond to other networks (if available). The command also shows the IP address of the enp1s0 network interface, as well as the DNS and gateway IP addresses. The command also displays the status of the systemd-networkd service, which is started.

Info

Fore more information, see the following links:

Netplan Configurations: https://netplan.io/examples/

networkd on Debian: https://wiki.debian.org/SystemdNetworkd

NetworkManager

The NetworkManager service is probably the most commonly used networking service in Linux. It is the default on RHEL/Fedora/CentOS, and is used by default on Debian (as a client), Ubuntu Desktop, and many more Linux distributions.

Lab 2-7

Working with NetworkManager

  • View the status of the service:

    systemctl status NetworkManager

  • Turn off the service:

    systemctl stop NetworkManager

  • Test the network connection:

    ping example.com

  • Enable and start the service with one command:

    systemctl --now enable NetworkManager

  • Test the network connection again to make sure it works.

Lab 2-8

Analyze the different NetworkManager-related tools

There are at least six ways to modify the networking configuration on Linux systems that use NetworkManager. Use a Fedora Workstation or CentOS Workstation to view each of these. (Most of them are also available on other distros such as Debian and Ubuntu clients.)

  • Settings - Right-click the desktop and select Settings. Then, locate the Network option. Finally, click the gear in the Wired section. It should look similar to the image below.
Figure 2-1: Networking Settings on a Fedora Client

View all of the tabs on your system. In the figure we start with the Details tab. It shows that our IPv4 address is 10.0.2.55. You can make modifications to the IP settings in the IPv4 and IPv6 tabs.

  • nm-connection-editor - Open a terminal and simply type nm-connection-editor. This will display a graphical tool that looks quite similar to Settings>Network.

  • nmtui - Open a terminal and type nmtui. This is the NetworkManager tab-based user interface. It is a menu-driven system that is text only - it can run on a system with or without a desktop environment.

  • nmcli - Open a terminal and type nmcli. This will show you the networking configuration. The nmcli tool can also be used to modify the configuration either as single commands or within the nmcli shell. This is a popular tool because of its depth, and the fact that you can use it on servers and clients because it runs in the command line only. There will be a segment dedicated to nmcli later in the webinar.

  • Configuration files - Depending on the Linux distribution, the main networking configuration file(s) can be in different locations.

    • RHEL/CentOS location: /etc/sysconfig/network-scripts
    • Many other distros: /etc/NetworkManager/system-connections

Example on CentOS 8.2

Here's an example of a network configuration file on a CentOS system. I accessed it by typing:

vim /etc/sysconfig/network-scripts/ifcfg-ens3

TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=ens3
UUID=539c56e8-ca22-4b5d-8ecd-cb244d1c24bd
DEVICE=ens3
ONBOOT=yes
IPADDR=172.21.0.222
PREFIX=16
GATEWAY=172.21.0.1
DNS1=172.21.0.1
IPV6_PRIVACY=no
ZONE=public

You can see that all of the IP information is listed here. This particular system is on the 172.21.0.0 network and uses the IP address 172.21.0.222. The prefix is 16. That is another name for the netmask (or subnet mask). So the IP address could also be shown as 172.21.0.222/16. If for some reason the nmcli command doesn't work for you, the configuration file is a good backup plan!


Example on a Debian Client

Here's an example of a network configuration file on a Debian client system (meaning one with a desktop environment). I accessed it by typing:

cd /etc/NetworkManager/system-connections

and opening the configuration file: vim 'Wired connection 1.nmconnection

[connection]
id=Wired connection 1
uuid=89321b95-d3b5-307d-a9f2-ff441c3f61ba
type=ethernet
autoconnect-priority=-999
permissions=
timestamp=1615917585

[ethernet]
mac-address=52:54:00:6D:FB:AC
mac-address-blacklist=

[ipv4]
address1=10.0.2.52/24,10.0.2.1
dns=10.0.2.1;
dns-search=
method=manual

[ipv6]
addr-gen-mode=stable-privacy
dns-search=
method=auto

Here we find that the information is displayed a bit differently. Under the ipv4 section we see the IP address of the system "10.0.2.52/24" followed by the gateway address "10.0.2.1", with the DNS server listed afterward.

Note

Changes to a network configuration file on a system that uses NetworkManager require either an nmcli connection reload or a system reboot to take effect.

Lab 2-9

Working with the Cockpit Service

Cockpit is a program that can be run on a server allowing admins to analyze and configure the server from a remote system via a web browser. It can be installed on most Linux distributions but is included in Fedora/RHEL/CentOS by default (though it might not be enabled and started). A Fedora server and Fedora workstation are recommended for this lab.

  • Enable and start the Cockpit service - Type the following command to enable and start cockpit. systemctl --now enable cockpit.socket

  • Verify that it is running systemctl status cockpit.socket

  • Connect from a remote system - Configure the server from a Linux workstation with a desktop environment. Open a browser and connect to https://ip_address:9090. Where ip_address is the IP address of your server that you enabled cockpit on. For example, type:

    https://10.0.2.54:9090

    That should connect to the server. You may have to accept the security risk to continue, and then you will need to login to the server using the same credentials you have been using so far. After that, click "Networking" on the left-hand side. That wil display all of the networking information for the system, similar to Figure 2-2.

Figure 2-2: Browser-based usage of Cockpit controlling a Fedora Server

Lab 2-10

Working with journalctl

Linux keeps a "journal" or a list of events that have happened on the system. You can query this journal with the journalctl command.

  • Type journalctl and use the arrow keys or the pageup and pagedown keys to see some of the logs. There is a lot of information, so we need to filter it. Press 'q' to escape and continue on.

  • Filter by the unit or service. Issue the following command:

    journalctl -u NetworkManager

    This displays the journal entries related to the NetworkManager service. -u allows us to select the service (or unit) that we need to learn more about. But there can still be a lot of information so we can use the tail option to show the last 10 lines.

  • Using tail: Issue the following command to see the last 10 lines of the journal as they pertain to the NetworkManager service.

    journalctl -u NetworkManager | tail -10

  • Modify the command further so that the information is sorted into columns and is outputted to less.

    journalctl -u NetworkManager | tail -10 | column -t | less

    The results might look similar to this:

    Mar  30  18:37:41  fed-server  NetworkManager[672]:  <info>  [1617154661.2648]  device    (enp7s0):  state        change:      config       ->          ip-config    (reason  'none',   sys-iface-state:  'managed')       
    Mar  30  18:37:41  fed-server  NetworkManager[672]:  <info>  [1617154661.2744]  device    (enp7s0):  state        change:      ip-config    ->          ip-check     (reason  'none',   sys-iface-state:  'managed')       
    Mar  30  18:37:41  fed-server  NetworkManager[672]:  <info>  [1617154661.2801]  device    (enp7s0):  state        change:      ip-check     ->          secondaries  (reason  'none',   sys-iface-state:  'managed')       
    Mar  30  18:37:41  fed-server  NetworkManager[672]:  <info>  [1617154661.2811]  device    (enp7s0):  state        change:      secondaries  ->          activated    (reason  'none',   sys-iface-state:  'managed')       
    Mar  30  18:37:41  fed-server  NetworkManager[672]:  <info>  [1617154661.2940]  device    (enp7s0):  Activation:  successful,  device       activated.                                                                     
    Mar  30  18:37:41  fed-server  NetworkManager[672]:  <info>  [1617154661.2963]  manager:  startup    complete                                                                                                              
    Mar  30  18:37:43  fed-server  NetworkManager[672]:  <info>  [1617154663.6130]  dhcp6     (enp7s0):  activation:  beginning    transaction  (timeout    in           45       seconds)                                     
    Mar  30  18:37:43  fed-server  NetworkManager[672]:  <info>  [1617154663.6143]  policy:   set        'Wired       connection   1'           (enp7s0)    as           default  for       IPv6              routing     and  DNS
    Mar  30  18:38:28  fed-server  NetworkManager[672]:  <warn>  [1617154708.4711]  dhcp6     (enp7s0):  request      timed        out                                                                                         
    Mar  30  18:38:28  fed-server  NetworkManager[672]:  <info>  [1617154708.4716]  dhcp6     (enp7s0):  state        changed      unknown      ->          timeout        
    

    For more information on the journalctl command, type journalctl --help, and for more in-depth information, see the manual page: man journalctl

Info

For more information on NetworkManager, see the following links:

Summary

The three main networking services that we covered here are: networking, networkd, and NetworkManager. Each has its place and each has pros and cons compared to the others.

You will normally find the networking service on a Debian server, the networkd service on an Ubuntu server, and the NetworkManager service on a Red Hat Enterprise Linux server (or Fedora/CentOS).

The commands you use for each of the services will vary. The networking service will work well with commands such as ip a, ifup, ifdown, and modifying configuration files with VIM. networkd will work well with commands such as ip a, networkctl, and in the case of Ubuntu, netplan. The NetworkManager service is all about nmcli and the myriad of other tools available to you.

Study the key points of these services and their respective Linux distributions and the configuration files and commands used within each.


Part III: Network Commands & Configurations

There are dozens of commands that deal with Linux networking. While it is difficult to know them all (and their options), there are a few basic ones that just about every tech should know. We'll cover those in this section.

ip and ping

The ip and ping commands are two of the most basic analysis tools in Linux. They can be helpful when obtaining information, troubleshooting, and making network connections. For these labs you can use any Linux distribution.

Note

Remember: If you need more information about any command, to use the help file (for example ip --help, or simply ip -h) or the manual page (for example man ip). Use the help and manual pages for any command that you need more information about.

Lab 3-1

Analyzing with the ip command

  • Find out Ethernet-based information. Type ip link

    Below are results of the ip link command run on a Debian client:

    root@deb52:~# ip link
    1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
        link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
        link/ether 52:54:00:6d:fb:ac brd ff:ff:ff:ff:ff:ff
    3: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000
        link/ether 52:54:00:69:db:db brd ff:ff:ff:ff:ff:ff
    4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN mode DEFAULT group default qlen 1000
        link/ether 52:54:00:69:db:db brd ff:ff:ff:ff:ff:ff
    

    From this list we can see the ethernet-based information of each network connection on the system. For example, the 2nd connection enp1s0 has a MAC address of 52:54:00:6d:fb:ac. The MAC address is normally 6 octets and is colon-separated. This address is burned into the PROM chip of the network interface, or (in the case of virtual machines) it is assigned to a virtual network interface. We also see that enp1s0 is "UP", meaning that it is activated and ready to transmit and receive data.

  • Discover IPv4 and IPv6 addresses. Type ip a

    Below are results of the ip a command run on a Debian client:

    root@deb52:~# ip a
    1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
        link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
        inet 127.0.0.1/8 scope host lo
           valid_lft forever preferred_lft forever
        inet6 ::1/128 scope host 
           valid_lft forever preferred_lft forever
    2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
        link/ether 52:54:00:6d:fb:ac brd ff:ff:ff:ff:ff:ff
        inet 10.0.2.52/24 brd 10.0.2.255 scope global noprefixroute enp1s0
           valid_lft forever preferred_lft forever
        inet6 fe80::36b4:f533:4478:39c8/64 scope link noprefixroute 
           valid_lft forever preferred_lft forever
    3: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
        link/ether 52:54:00:69:db:db brd ff:ff:ff:ff:ff:ff
        inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
           valid_lft forever preferred_lft forever
    4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN group default qlen 1000
        link/ether 52:54:00:69:db:db brd ff:ff:ff:ff:ff:ff
    

    Now, in addition to the inforamtion shown with the ip link command, we can see the associated IPv4 and IPv6 addresses. For example, the 2nd interface (*enp1s0) hsa the IPv4 address 10.0.2.52/24. However, this command shows a lot of information. And for a system with a lot of network connections, it can quickly become unmanageable. So, we can sort and filter the information as we see fit.

  • Sort and filter using the ip command

    Example 1: ip -br a

    root@deb52:~# ip -br a
    lo               UNKNOWN        127.0.0.1/8 ::1/128 
    enp1s0           UP             10.0.2.52/24 fe80::36b4:f533:4478:39c8/64 
    virbr0           DOWN           192.168.122.1/24 
    virbr0-nic       DOWN        
    

    With this command we get a nice table of information telling us the state of the interface and the IP addresses. I use it often!

    Example 2: ip a | grep inet | sort -n

    root@deb52:~# ip a | grep inet | sort -n
        inet 10.0.2.52/24 brd 10.0.2.255 scope global noprefixroute enp1s0
        inet 127.0.0.1/8 scope host lo
        inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
        inet6 ::1/128 scope host 
        inet6 fe80::36b4:f533:4478:39c8/64 scope link noprefixroute 
    

    With this command we filter for the term "inet" and sort the results numerically. You could also add a "column -t" to make a nice table out of it.

    Example 3: ip --oneline addr | column -t

    root@deb52:~# ip --oneline addr | column -t
    1:  lo      inet   127.0.0.1/8                   scope  host             lo\            valid_lft  forever        preferred_lft  forever
    1:  lo      inet6  ::1/128                       scope  host             \              valid_lft  forever        preferred_lft  forever
    2:  enp1s0  inet   10.0.2.52/24                  brd    10.0.2.255       scope          global     noprefixroute  enp1s0\        valid_lft      forever        preferred_lft  forever
    2:  enp1s0  inet6  fe80::36b4:f533:4478:39c8/64  scope  link             noprefixroute  \          valid_lft      forever        preferred_lft  forever
    3:  virbr0  inet   192.168.122.1/24              brd    192.168.122.255  scope          global     virbr0\        valid_lft      forever        preferred_lft  forever
    

    With this command we show each IPv4 and IPv6 interface as one line of information.

    You can imagine the possibilities when it comes to a command like ip. Search for, and practice with different options to make your work more efficient, and easier on the eyes.

Lab 3-2

Adding and removing IP addresses with the ip command

  • To add an IP address, use the following syntax: ip a add <ip_address> dev <interface>. So for example:

    ip a add 10.0.2.152/24 dev enp1s0

    This would add the IP address 10.0.2.152 to the interface named enp1s0. However, this is meant for temporary purposes. If you need a persistent change then another tool is recommended, for example, nmcli.

  • To remove an IP address, simple change add to delete. For example:

    ip a delete 10.0.2.152/24 dev enp1s0

Lab 3-3

Working with the ip route command

  • To view the default gateway, type ip route show or ip route or simply ip r. Example:

    root@deb52:~# ip r
    default via 10.0.2.1 dev enp1s0 proto static metric 100 
    10.0.2.0/24 dev enp1s0 proto kernel scope link src 10.0.2.52 metric 100 
    192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1 linkdown 
    

    In the example, the default gateway is shown on the second line as 10.0.2.1. Any networks that this system has access to are also shown. In the example we have 10.0.2.0 and 192.168.122.0. To make columns, simply add the column -t option: ip r | column -t. (This is similar to the older route -n command if using the net-tools package.) If you have a lot of routes, you can filter for the default gateway easily by typing: ip r | grep default.

  • Remove and put back the default route (or gateway).

    In the following example, we use the command ip r delete default to remove the default route, then the ip r command to check our work, then to put it back: ip r add default via 10.0.2.1

    root@deb52:~# ip r delete default
    root@deb52:~# ip r
    10.0.2.0/24 dev enp1s0 proto kernel scope link src 10.0.2.52 metric 100 
    192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1 linkdown 
    root@deb52:~# ip r add default via 10.0.2.1
    root@deb52:~# ip r
    default via 10.0.2.1 dev enp1s0 
    10.0.2.0/24 dev enp1s0 proto kernel scope link src 10.0.2.52 metric 100 
    192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1 linkdown 
    
  • Add and remove a route.

    To add a new route, use the following syntax: ip r add <ip_network> dev <interface>. View the new route with the ip r command.

    To remove that route, change "add" to "delete". Example below:

    root@deb51:~# ip r add 172.21.0.0/16 dev enp1s0
    root@deb51:~# ip r
    default via 10.0.2.1 dev enp1s0 onlink 
    10.0.2.0/24 dev enp1s0 proto kernel scope link src 10.0.2.51 
    172.21.0.0/16 dev enp1s0 scope link 
    root@deb51:~# ip r delete 172.21.0.0/16 dev enp1s0
    root@deb51:~# ip r
    default via 10.0.2.1 dev enp1s0 onlink 
    10.0.2.0/24 dev enp1s0 proto kernel scope link src 10.0.2.51 
    

Lab 3-4

Working with the ping command

Ping is the most primal testing command. It can show you if your TCP/IP stack is working properly locally. It can tell you if you have access to the default gateway. It can tell you if other systems are "up", or at least, if they are accessible from your location. Here are some examples:

Example: Pinging the localhost on IPv4

root@deb52:~# ping -4 127.0.0.1
PING 127.0.0.1 (127.0.0.1) 56(84) bytes of data.
64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.056 ms
64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.069 ms
64 bytes from 127.0.0.1: icmp_seq=3 ttl=64 time=0.067 ms
^C
--- 127.0.0.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 52ms
rtt min/avg/max/mdev = 0.056/0.064/0.069/0.005 ms

With this command we ping the local system on its IPv4 loopback address: 127.0.0.1. The system received three 64 byte replies before I cancelled the operation by pressing Ctrl + c. You could also ping the localhost by name (for example deb52) or via IPv6 which would be ::1

Example: Pinging the gateway address

root@deb52:~# ping 10.0.2.1
PING 10.0.2.1 (10.0.2.1) 56(84) bytes of data.
64 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.126 ms
64 bytes from 10.0.2.1: icmp_seq=2 ttl=64 time=0.236 ms
64 bytes from 10.0.2.1: icmp_seq=3 ttl=64 time=0.226 ms
^C
--- 10.0.2.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 54ms
rtt min/avg/max/mdev = 0.126/0.196/0.236/0.049 ms

In this example, we pinged the gateway device which is 10.0.2.1. Notice the difference in the reply time. In the first example, it averaged 0.064 milliseconds (ms), and in the second example, it averaged 0.196 ms - substantially longer (but still quick). That's the difference between pinging the local system (which generated no network traffic) and pinging another system (even though the gateway is just a virtual device). If I was to ping my actual physical gateway, the reply times would average around 0.5 ms.

Example: Pinging a host on the Internet

root@deb52:~# ping prowse.tech
PING prowse.tech (67.205.11.189) 56(84) bytes of data.
64 bytes from apache2-igloo.allatou.dreamhost.com (67.205.11.189): icmp_seq=1 ttl=49 time=30.7 ms
64 bytes from apache2-igloo.allatou.dreamhost.com (67.205.11.189): icmp_seq=2 ttl=49 time=20.8 ms
64 bytes from apache2-igloo.allatou.dreamhost.com (67.205.11.189): icmp_seq=3 ttl=49 time=20.4 ms
^C
--- prowse.tech ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 6ms
rtt min/avg/max/mdev = 20.406/23.959/30.717/4.780 ms

In this example, we pinged "prowse.tech". Because the ping was against a name, that name had to be resolved to its IP address (67.205.11.189). Also note that the reply times are much higher, averaging 23.9 ms per reply. Ping rates of 20 - 100 ms are common when connecting to websites.

Example: Modifying the ping packet size and amount of replies

root@deb52:~# ping -c 5 -s 1024 10.0.2.1
PING 10.0.2.1 (10.0.2.1) 1024(1052) bytes of data.
1032 bytes from 10.0.2.1: icmp_seq=1 ttl=64 time=0.136 ms
1032 bytes from 10.0.2.1: icmp_seq=2 ttl=64 time=0.226 ms
1032 bytes from 10.0.2.1: icmp_seq=3 ttl=64 time=0.219 ms
1032 bytes from 10.0.2.1: icmp_seq=4 ttl=64 time=0.210 ms
1032 bytes from 10.0.2.1: icmp_seq=5 ttl=64 time=0.232 ms

--- 10.0.2.1 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 102ms
rtt min/avg/max/mdev = 0.136/0.204/0.232/0.038 ms

This time we added two options. -c (count) specifies the exact amount of ping requests (and hopefully, replies) - in this case I selected 5. You can see that I didn't have to press Ctrl + c to break out of the ping process. -s (packetsize) allows you to change the ICMP echo - I changed it from the default 64 bytes to 1024 bytes. The replies add 8 bytes of supervisory information, bringing it to 1032 bytes per reply packet.

This type of ping modification can be helpful when testing servers and attempting to simulate more traffic, or conducting longer tests with a specific amount of pings. The maximum packet size on most versions of Linux is 65507 bytes, which creates fragmented information, and could be blocked by a security device, so the maximum recommended testing amount is around 1400 bytes. (This keeps it within the realm of normal 1500 byte IP packets).

Hostname and DNS

We've done plenty up to this point with IP addresses. But most systems (and people) communicate by name. So a technician needs to know how to modify hostnames and configure DNS. Remember that the Domain Name System (DNS) is in charge of resolving domain names (and host names) to their respective IP addresses - and vice-versa.

Lab 3-5

Working with hostnamectl

  • View system information with the hostnamectl command

    Type hostnamectl to see the hostname and other information about the system. Example:

    root@deb52:~# hostnamectl
      Static hostname: deb52
            Icon name: computer-vm
              Chassis: vm
             Location: workplace2
           Machine ID: 42f779dc9123405c90f8fa73b6c83f7c
              Boot ID: 35b9c976794a4bb38ba7a4e81511152a
       Virtualization: kvm
     Operating System: Debian GNU/Linux 10 (buster)
               Kernel: Linux 4.19.0-16-amd64
         Architecture: x86-64
    

    Here we see the name of the computer is deb52. We get a whole lot of other information as well, including the operating system type and version, the version of the Linux kernel, and the architecture of the computer.

  • Use the hostnamectl command to change the hostname.

    • First, change the name by typing hostnamectl set-hostname newname
    • View the new name by entering hostnamectl
    • Close the terminal and open a new one to see the new name in the prompt.
    • Change the name back to the original.
    • Close the terminal and open a new one again.
    • Verify the name is back to the original. (You can also use the older hostname command to see the hostname only.)

    Example: (the four dashes represents closing the terminal and reopening it)

    root@deb52:~# hostnamectl set-hostname newname
    root@deb52:~# hostnamectl
       Static hostname: newname
             Icon name: computer-vm
               Chassis: vm
              Location: workplace2
            Machine ID: 42f779dc9123405c90f8fa73b6c83f7c
               Boot ID: 35b9c976794a4bb38ba7a4e81511152a
        Virtualization: kvm
      Operating System: Debian GNU/Linux 10 (buster)
                Kernel: Linux 4.19.0-16-amd64
          Architecture: x86-64
    ----
    root@newname:~# hostnamectl set-hostname deb52
    ----
    root@deb52:~# hostname
    deb52
    

    Note

    A fun program that shows similar information to hostnamectl is called neofetch. Install it by name with your distro's installer - for example apt install neofetch. Then run the program by simply typing neofetch.

Lab 3-6

Viewing the DNS configuration

You can view the DNS configuration in different ways depending on the distribution and the networking service used.

  • View the DNS server setting with resolv.conf:

    Type cat /etc/resolv.conf . The results will look similar to the example below.

    root@deb52:~# cat /etc/resolv.conf
    # Generated by NetworkManager
    nameserver 10.0.2.1
    

    In this example I used a Debian client. That system uses NetworkManager, which generates the DNS (or nameserver) information and places it in the resolv.conf file. To view/modify the DNS server IP address on a system that runs NetworkManager you would use the nmcli command.

  • View the DNS server setting with the nmcli command:

    To view the DNS server setting with nmcli, simply type nmcli and look toward the end of the results for "DNS configuration. Here's an example:

    DNS configuration:
            servers: 10.0.2.1
            interface: enp1s0
    

    To modify the DNS coniguration you could use syntax similar to this:

    nmcli connection modify enp1s0 ipv4.dns 10.0.2.3

    That would change the DNS server setting from 10.0.2.1 to 10.0.2.3. We'll cover nmcli more in an upcoming section.

  • View the DNS server setting with resolved.conf

    If a system uses the systemd-networkd network configuration, then the settings are stored in /etc/systemd/resolved.conf. For example:

    [Resolve]
    DNS=10.0.2.1
    FallbackDNS=10.0.2.2
    #Domains=
    #LLMNR=yes
    #MulticastDNS=yes
    #DNSSEC=allow-downgrade
    #DNSOverTLS=no
    #Cache=yes
    #DNSStubListener=yes
    #ReadEtcHosts=yes
    

    In this example, we have a primary DNS server (10.0.2.1) and a fallback, or secondary, DNS server (10.0.2.2). resolved.conf is less commonly used. Though Ubuntu server uses systemd-networkd, it relies on netplan as the front-end for the configuration. For example:

    # This is the network config written by 'subiquity'
    
    network:       
      version: 2
      renderer: networkd
      ethernets:
        enp1s0:
          addresses: [10.0.2.53/24]
          gateway4: 10.0.2.1
          nameservers: 
            search: [example.com]
            addresses: [10.0.2.1]
    

    Here, the DNS servers are known as "nameservers" and there is one listed at 10.0.2.1.

    Note

    If you were working at a Debian server with a default configuration, you would simply use resolv.conf, and would modify DNS there. This is because it uses the networking service, so networkd and NetworkManager do not apply.

nmcli

nmcli is the NetworkManager command-line interface. This is a very commonly used command line tool for analyzing, modifying, and troubleshooting network connections. It is used by Fedora/RHEL/CentOS, Debian (as a client) Ubuntu Desktop, and many more Linux distributions. It can be used as a single-command tool or within an interactive interface. We'll start with the nmcli command in the command line. Keep in mind - this command is boss!

Lab 3-7

View and analyze the network configuration with nmcli.

  • Basic usage of nmcli.

    Type nmcli to see your network configuration. Example results on a Debian client are below. Results on other systems running NetworkManager should be very similar.

    Example of the nmcli command on a Debian client

    enp1s0: connected to Wired connection 1
    "Red Hat Virtio"
    ethernet (virtio_net), 52:54:00:6D:FB:AC, hw, mtu 1500
    ip4 default
    inet4 10.0.2.52/24
    route4 10.0.2.0/24
    route4 0.0.0.0/0
    inet6 fe80::36b4:f533:4478:39c8/64
    route6 fe80::/64
    
    virbr0: connected to virbr0
    "virbr0"
    bridge, 52:54:00:69:DB:DB, sw, mtu 1500
    inet4 192.168.122.1/24
    route4 192.168.122.0/24
    
    lo: unmanaged
    "lo"
    loopback (unknown), 00:00:00:00:00:00, sw, mtu 65536
    
    virbr0-nic: unmanaged
    "virbr0-nic"
    tun, 52:54:00:69:DB:DB, sw, mtu 1500
    
    DNS configuration:
    servers: 10.0.2.1
    interface: enp1s0
    
    Use "nmcli device show" to get complete information about known devices and
    "nmcli connection show" to get an overview on active connection profiles.
    
    Consult nmcli(1) and nmcli-examples(5) manual pages for complete usage details.
    

    At the beginning of the results we see "enp1s0: connected to "Wired connection 1". enp1s0 is the Linux hardware-based name for the network interface. But NetworkManager gives these devices its own names - in this case, Wired connection 1. That is the name that we need to use when configuring the network interface with the nmcli command. Lets show how to add and remove static and DHCP-based IP addresses.

  • View the NetworkManager connections

    Type nmcli connection show. Here's an example:

    [sysadmin@smauggy ~]$ nmcli connection show
    NAME                UUID                                  TYPE      DEVICE 
    Wired connection 1  58f4b9c3-638d-31e4-bdbf-010a3b56bf47  ethernet  enp3s0 
    EMF-5B              257a1403-e9fd-4a05-bb5c-e91f7baf5274  wifi      wlp2s0 
    virbr0              12a61b1b-f6c6-42a1-b62c-7777cfb94763  bridge    virbr0 
    

    This example was taken from an actual laptop with wired and wireless connections. Under the "TYPE" column you can see there is an "ethernet" device named enp3s0. That is the wired connection, and so NetworkManager calls is "Wired connection 1". Under "TYPE" you will also see a "wifi" device named wlp2s0. In this case, NetworkManager refers to it by the name "EMF-5B" (which is actually the name of the wireless network it is connect too - a bit of a security issue, but one which is fixable).

    Note

    You can abbreviate (or truncate) nmcli commands a lot. For example, nmcli connection show can be reduced to nmcli con show, or just nmcli c show. In fact, the "show" portion isn't even necessary. So you could just type nmcli c and be done with it! You'll get the same results.

Lab 3-8

Working with nmcli in the command line

  • Add an IP address with nmcli. Example:

    nmcli connection modify "Wired connection 1" ipv4.method manual ipv4.address 10.0.2.152/24 ipv4.gateway 10.0.2.1 ipv4.dns 10.0.2.1
    

    In this example, we specify that the address to be added will be static, then we add the IP address 10.0.2.152/24, and then we add the gateway and DNS IP addresses. You can add multiple IPs if you needed to in this manner, just by comma separating them:

    nmcli connection modify "Wired connection 1" ipv4.method manual ipv4.address 10.0.2.152/24,10.0.2.153/24
    

    Note

    You can abbreviate here too, for example: nmcli con mod or even nmcli c m instead of nmcli connection modify.

    Combine this with tab completion. For example, for the network interface type "W and press the tab key. That will auto-complete the name of the interface, which in this case is "Wired connection 1". Combine auto-complete with abbreviations and it can be a real time-saver. Wonderful!

  • Down and up the interface

    Once you have made your changes, you need to deactivate and reactivate the network interface for the changes to take effect. This is known as "down" and "up" the interface. To do this type the following two commands: nmcli connection down "Wired connection 1" and nmcli connection up "Wired connection 1"

    At that point you should see the new IP addresses listed when you run the nmcli command. The following example shows a snippet of the nmcli results.

    enp1s0: connected to Wired connection 1
    "Red Hat Virtio"
    ethernet (virtio_net), 52:54:00:6D:FB:AC, hw, mtu 1500
    ip4 default
    inet4 10.0.2.152/24
    inet4 10.0.2.153/24
    route4 10.0.2.0/24
    

    You can see the two IP addresses that were added previously.

  • Set the interface to obtain an IP address from a DHCP server.

    Type nmcli c m "Wired connection 1" ipv4.method auto

    By selecting "auto" we set the interface to obtain all TCP/IP information from a DHCP server (if one is available) including it's IP address, netmask, gateway address, and DNS server IP address. Down and up the interface for the changes to take effect. You should see something similar to the example snippet below:

    enp1s0: connected to Wired connection 1
    "Red Hat Virtio"
    ethernet (virtio_net), 52:54:00:6D:FB:AC, hw, mtu 1500
    ip4 default
    inet4 10.0.2.152/24
    inet4 10.0.2.153/24
    inet4 10.0.2.139/24
    route4 10.0.2.0/24
    

    In this case, the system obtained an IP address from a DHCP server on my virtual network. It received the address 10.0.2.139. Now we have two static IP addresses and one dynamic IP address!

  • Remove one of the static IP addresses.

    Type nmcli c m "Wired connection 1" -ipv4.address 10.0.2.153/24 Note the - dash before ipv4, and type in an appropriate IP address based on your configuration. Down and up the interface and you should see the results with the nmcli command.

Lab 3-9

Working with the nmcli interactive shell

  • Access the nmcli shell.

    You can access the nmcli interactive shell for any one of your network interfaces. Once there, you can run multiple commands, and save them all at once. To access the nmcli shell for an interface enter the following:

    nmcli connection edit "Wired connection 1"

    Note that it says "edit" this time. That is the option that opens a shell. Remember to change the interface name based on your system. You should see something similar to the following:

    root@deb52:~# nmcli connection edit "Wired connection 1" 
    
    ===| nmcli interactive connection editor |===
    
    Editing existing '802-3-ethernet' connection: 'Wired connection 1'
    
    Type 'help' or '?' for available commands.
    Type 'print' to show all the connection properties.
    Type 'describe [<setting>.<prop>]' for detailed property description.
    
    You may edit the following settings: connection, 802-3-ethernet (ethernet), 802-1x, dcb, sriov, ethtool, match, ipv4, ipv6, tc, proxy
    nmcli> 
    

    Now, we can enter commands into the shell.

  • Remove the other static IP address.

    In the shell, type remove ipv4.address 10.0.2.152/24. This will remove the static IP address.

  • Set a new static IP address.

    Type set ipv4.address 10.0.2.52/24. This will set the original IP address that the system had before. If it asks, type "yes" to set the IP address to manual.

    Note

    You can also abbreviate here. Instead of "remove" type "r", and instead of "set", type "s". Every character counts!

  • Save the configuration, activate it, and quit.

    To save the configuration, simply type save. Then type activate to enable it. Finally, type quit to exit out of the shell. At this point, our IP configuration should be back to what it was when we started Lab 3-7.

Note

For more information about nmcli use the nmcli -h and man nmcli commands.

Also, see the following link:

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/networking_guide/sec-configuring_ip_networking_with_nmcli

Summary

In this section we covered a good amount of commands. When it comes to knowing commands, its all about practice, and researching the help and man files. The more you practice, the more fluent you become with the command structure.

Tip

But remember this! You can't know all commands and all options for each command. It just won't happen. It's more important to know how to search for the information you seek!

We worked with the ip command which allows us to analyze Ethernet information, TCP/IP information, routing information, and more. We also worked with ping. Some admins don't like to use the tool, but because it is such an easy and primal tool, and because it just works, it becomes almost inescapable. Ping can be used for basic testing, troubleshooting, and to get a basic baseline of a system.

Then we moved into hostnames and DNS. You can change the hostname with the hostnamectl command or by accessing the /etc/hostname file. DNS can be modified within /etc/resolv.conf, or (if using networkd) /etc/resolved.conf, or (if using NetworkManager) with the nmcli command (among other places).

The nmcli command is extremely powerful and well-versed. You can analyze systems, change TCP/IP information, work with it as a single command or in an interactive shell. One look at the MAN page shows you the depth of the command. That's one of those commands that requires a lot of practice to master - everyday.

So that wraps up this part. Remember to practice the commands and options, but also, to know how to search for the commands and options that you need to get your work done quickly and efficiently.


Part IV: Connecting between Computers with the Command Line

As often mentioned, sysadmins live in the command line. It's all about controlling systems remotely and moving files via the command line. There are plenty of tools available to accomplish these tasks. In this section we cover SSH, SCP, rsync, and SFTP.

SSH

The Secure Shell (or SSH) is used to remotely control another system from the command line. It involves a 4-step process.

  1. client initiates a connection to an SSH “server”
  2. Server sends out a public key
  3. The two handshake on parameters and open a secure channel
  4. the user at the client logs in to the server

There are a variety of SSH tools including PuTTY, WinSCP, CyberDuck, and so on. However, OpenSSH is built-in to most Linux Distributions and can be installed or updated as necessary. That’s what we will focus on during this webinar.

Lab 4-1

Using SSH

In this lab we'll verify whether SSH is running as a server, and connect from one machine to another using SSH.

  • Verify if SSH is available on the target computer.

    Use the command ssh -V to see if SSH is installed. The following example shows that OpenSSH version 7.9 is installed on a Debian client:

    user@deb52:~$ ssh -V
    OpenSSH_7.9p1 Debian-10+deb10u2, OpenSSL 1.1.1d  10 Sep 2019
    

    If it is not installed, you can install it, for example:

    apt install openssh-server

    Replace "apt" with whatever package manager your distribution uses.

  • Check if the OpenSSH server service is running.

    In most instances, OpenSSH uses two services: ssh (for the client) and sshd (for the server). We are interested in the server service on the target computer. To see if the server service is running, type the command:

    systemctl status sshd

    You should see something similar to the following:

    user@deb52:~$ systemctl status sshd
    ● ssh.service - OpenBSD Secure Shell server
       Loaded: loaded (/lib/systemd/system/ssh.service; enabled; vendor preset: enabled)
       Active: active (running) since Thu 2021-04-01 15:03:17 EDT; 9h ago
         Docs: man:sshd(8)
           man:sshd_config(5)
      Process: 513 ExecStartPre=/usr/sbin/sshd -t (code=exited, status=0/SUCCESS)
     Main PID: 542 (sshd)
        Tasks: 1 (limit: 4685)
       Memory: 6.6M
       CGroup: /system.slice/ssh.service
           └─542 /usr/sbin/sshd -D
    

    If it is not active and running, you can enable and start it by typing:

    systemctl --now enable sshd

    Note

    Make sure the service is active and running on the target computer, or you won't be able to "SSH" into that system to take control of it.

    You can also check the ports of the target system with the following command:

    ss-tulnw

    This will show results similar to the following:

    user@deb52:~$ ss -tulnw
    Netid    State     Recv-Q    Send-Q        Local Address:Port          Peer Address:Port    
    icmp6    UNCONN    0         0                         *:58                       *:*       
    udp      UNCONN    0         0             192.168.122.1:53                 0.0.0.0:*       
    udp      UNCONN    0         0            0.0.0.0%virbr0:67                 0.0.0.0:*       
    udp      UNCONN    0         0                   0.0.0.0:49470              0.0.0.0:*       
    udp      UNCONN    0         0                   0.0.0.0:5353               0.0.0.0:*       
    udp      UNCONN    0         0                      [::]:37078                 [::]:*       
    udp      UNCONN    0         0                      [::]:5353                  [::]:*       
    tcp      LISTEN    0         80                127.0.0.1:3306               0.0.0.0:*       
    tcp      LISTEN    0         32            192.168.122.1:53                 0.0.0.0:*       
    tcp      LISTEN    0         9                   0.0.0.0:21                 0.0.0.0:*       
    tcp      LISTEN    0         128                 0.0.0.0:22                 0.0.0.0:*       
    tcp      LISTEN    0         128                       *:80                       *:*       
    tcp      LISTEN    0         9                      [::]:21                    [::]:*       
    tcp      LISTEN    0         128                    [::]:22                    [::]:*   
    

    You can see on the 11th line that port 22 is being listened on. That is the default SSH inbound port.

    At this point, the OpenSSH server should be ready to accept connections.

  • Connect to the system running OpenSSH as a server from a separate computer.

    Now we can connect to our "server" using the SSH command from a client computer. In this lab I use a Fedora system to connect to the Debian system that is acting as the OpenSSH server. The standard command looks like this:

    ssh user@10.0.2.52

    ssh is the command. user is the account on the target system that you will use to control it. 10.0.2.52 is the target IP address that we are connecting to. That is the typical syntax when working with SSH. Of course, your usernames and IPs may be different.

    Note

    If you are using VirtualBox for your virtual machines, the typical SSH syntax probably won't work. You either need to set up some sort of routing, or configure port forwarding in VirtualBox (preferred). When you connect in this manner, the typical syntax looks similar to this:

    ssh user@127.0.0.1 -p 2222

    For more information on VirtualBox port forwarding, see this link.

    At this point, we should be able to control the target computer, based on the user account's permissions. The figure below shows a Fedora system (named fed-client) that has SSH'd into the Debian system (named deb52). After logging in with SSH as the user account, I then logged in as root by typing su - and the password for the root account on the Debian system.

    Figure 4-1: Connecting from a Fedora system to a Debian system with SSH

    Warning

    In the field it is recommended to avoid the use of the root account whenever possible. The use of root access in these labs is for educational purposes only.

    When finished, we can log out of the SSH session by typing exit or by pressing Ctrl + D on the keyboard.

SCP

SCP (or Secure Copy) allows a person to copy one file or entire directories to separate systems. It works as an extension of SSH. But it doesn’t log into systems, and there is no shell involved (unlike SFTP for example). It is primarily a one-time use command.

Lab 4-2

Using SCP

In this lab we'll use SCP to copy files from a local host to a remote host, and vice-versa. The general syntax for this would be:

`scp <source> <destination>`
  • Copy a file from the local host to the remote host.

    This is known as a "push" of data.

    The source will be a file (debian.iso) on our local system, which is a Debian client (deb52 at 10.0.2.52). The destination will be the remote computer, which is a Debian server (deb51 at 10.0.2.51).

    Note

    Any large file will do. In this lab I'm using a 300 MB Debian ISO image that I renamed to debian.iso. You can get the various Debian images at the following link: https://www.debian.org/distrib/netinst. You could also grab this particular image using the wget command:

    wget https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-10.9.0-amd64-netinst.iso

    Here's an example using SCP to copy from the local system to a remote system:

    scp debian.iso user@10.0.2.51:/home/user

    So the source is "debian.iso"

    And the desintation is "user@10.0.2.51:/home/user"

    Here is what we see on the screen when we issue the command from the Debian client.

    user@deb52:~/Downloads$ scp debian.iso user@10.0.2.51:/home/user
    user@10.0.2.51's password: 
    debian.iso                          100%  336MB 188.4MB/s   00:01   
    

    Let's break down what is happenening here. First, we issue the scp command. Second, we specify the file to be copied. The file is debian.iso, and its located in the Downloads directory. Next, we state the user account and IP address that we want to connect to. Remember that SCP works off of SSH, so the login process works in the same manner. In this case, we are connecting as "user" on the IP address 10.0.2.51. (Note that you should make sure that the "user" account exists on the remote system.) After that, we state the path on the destination computer where we want to copy the file. Always use a colon (:) after the IP address and before the path, then give a valid Linux path. In this case, we copied the file to the /home/user directory. When we press enter, it asks for credentials (based on SSH) - in this case the password of the user account on the remote system. Once we type the correct password, the file transfer is initiated. It copied the debian.iso file (336 MB in size) in one second to the destination. That's it!

    Note

    Remember to know exactly what the source and the destination are. Also, make sure that you know the user account and password of the remote machine. Finally, make sure that the user account in question has the appropriate permissions to copy files.

  • Copy a file from the remote host to the local host.

    This time we'll copy a file, but in reverse. We'll do this at the Debian client once again, but now we'll grab the data from the server. This is known as a "pull" - when the data comes from the remote system and arrives at the local system that we are working at.

    scp user@10.0.2.51:/home/user/debian.iso /home/user/Downloads/debian2.iso

    Here it is in action:

    user@deb52:~/Downloads$ scp user@10.0.2.51:/home/user/debian.iso /home/user/Downloads/debian2.iso
    user@10.0.2.51's password: 
    debian.iso                              100%  336MB 172.6MB/s   00:01    
    user@deb52:~/Downloads$ ls
    debian2.iso  debian.iso
    

    This time, the file is stored at the Debian server (which is 10.0.2.51). I simply took the same debian.iso file and copied it back. But I renamed it during the copy to "debian2.iso" so that it wouldn't overwrite the file we already had. Afterward, you can see the ls command displays both .iso files within the Downloads directory. Of course, you could copy it without a file name to whatever location you want, or change the name as you see fit. At that point, the basic rules of Linux copying take effect.

    Note

    You can also copy entire directories. Just use the -r option after scp. It works in the same manner as the cp command locally. You can in fact use SCP locally as well, but for the most part sysadmins use it for remote copying of files as that is what it is mainly designed for.

rsync

rsync is a fast, versatile, remote (and local) file-copying tool (to quote the MAN page!). While it can be used for local copying of files, it is more commonly used for remote copying. One of the great features of rsync is that it can act as an archive for your data and, after an initial file copy, will incrementally copy files as they are modified or added to a directory. This, among other things, makes it a formidable tool.

rsync should usually work over SSH by default. If you have rsync installed on both systems, and SSH installed on both systems (and you probably do), and the SSH server is running on the remote system, and you know the password or have the key, then rsync will work over SSH automatically. That is generally the use case for administrators.

Note

One case where rsync is not installed by default is with Debian Buster (running as a server). However, a quick apt install rsync will remedy the situation.

Lab 4-3

Working with rsync

  • Copy a file from a local system to a remote system.

    Example:

    rsync -a debian.iso user@10.0.2.51:/home/user

    As you can see, this is very similar to SCP. There is a source and a destination, and in this case we are "pushing" the file to a remote system (our Debian server). The only difference is the options that rsync uses (and there are a lot of them). I have added the archive option (-a) after rsync. This is a very commonly used option (in fact, I use it consistently).

    Here are the results:

    user@deb52:~/Downloads$ rsync -a debian.iso user@10.0.2.51:/home/user
    user@10.0.2.51's password: 
    user@deb52:~/Downloads$ 
    

    It worked, but we don't see any results. By default, rsync goes by the "no news is good news" rule. So it won't tell us what is happening unless we request that information. Of course, if there was an error (no such directory, no permissions, etc...) then the program would tell us that there was a failure. Let's take it to the next level, and show how to display the current progress.

  • Copy a group of files in a directory to a remote system.

    In this procedure we'll create two directories, and populate them with empty files. Then, we'll copy the both directories (and all the housed files) over to a remote location. Along the way we'll show how to display the progress of the file copy, and show how to use custom SSH ports.

    Note

    Make sure you are working in a test directory with no data.

    For this lab I will be working within one directory (called Downloads) in a relative manner - meaning I won't be changing directories.

    • First, create two test directories named "test1" and "test2"

      mkdir {test1,test2}

    • Next, create 10 files within the test1 directory.

      touch ./test1/file{1..10}

    • Now, verify that the files are inside of the test1 directory.

      ls ./test1

    • Use rsync to copy the files to the test2 directory.

      rsync ./test1/* ./test2

    At this point, we should have two sets of identical files in the test1 and test2 directories. we can easily check this by using the tree command from our current location. The results look like this:

    user@deb52:~/Downloads$ tree
    .
    ├── debian2.iso
    ├── debian.iso
    ├── test1
    │   ├── file1
    │   ├── file10
    │   ├── file2
    │   ├── file3
    │   ├── file4
    │   ├── file5
    │   ├── file6
    │   ├── file7
    │   ├── file8
    │   └── file9
    └── test2
        ├── file1
        ├── file10
        ├── file2
        ├── file3
        ├── file4
        ├── file5
        ├── file6
        ├── file7
        ├── file8
        └── file9
    
    2 directories, 22 files
    

    So you can see that we are currently located in the Downloads directory. Inside that we have the test1 and test2 directories, each with 10 files.

    Note

    If you don't have access to the tree command, simply install it. For example, Debian doesn't include it by default, so apt install tree.

    Now we can finally copy the contents to the remote system! This next command is a little more advanced.

    rsync -avP -e "ssh -p 22" {test1,test2} user@10.0.2.51:/home/user

    Here are the results at the local system where we run the command:

    Example

    user@deb52:~/Downloads$ rsync -avP -e "ssh -p 22" {test1,test2} user@10.0.2.51:/home/user
    user@10.0.2.51's password: 
    sending incremental file list
    test1/
    test1/file1
              0 100%    0.00kB/s    0:00:00 (xfr#1, to-chk=19/22)
    test1/file10
              0 100%    0.00kB/s    0:00:00 (xfr#2, to-chk=18/22)
    test1/file2
              0 100%    0.00kB/s    0:00:00 (xfr#3, to-chk=17/22)
    test1/file3
              0 100%    0.00kB/s    0:00:00 (xfr#4, to-chk=16/22)
    test1/file4
              0 100%    0.00kB/s    0:00:00 (xfr#5, to-chk=15/22)
    test1/file5
              0 100%    0.00kB/s    0:00:00 (xfr#6, to-chk=14/22)
    test1/file6
              0 100%    0.00kB/s    0:00:00 (xfr#7, to-chk=13/22)
    test1/file7
              0 100%    0.00kB/s    0:00:00 (xfr#8, to-chk=12/22)
    test1/file8
              0 100%    0.00kB/s    0:00:00 (xfr#9, to-chk=11/22)
    test1/file9
              0 100%    0.00kB/s    0:00:00 (xfr#10, to-chk=10/22)
    test2/
    test2/file1
              0 100%    0.00kB/s    0:00:00 (xfr#11, to-chk=9/22)
    test2/file10
              0 100%    0.00kB/s    0:00:00 (xfr#12, to-chk=8/22)
    test2/file2
              0 100%    0.00kB/s    0:00:00 (xfr#13, to-chk=7/22)
    test2/file3
              0 100%    0.00kB/s    0:00:00 (xfr#14, to-chk=6/22)
    test2/file4
              0 100%    0.00kB/s    0:00:00 (xfr#15, to-chk=5/22)
    test2/file5
              0 100%    0.00kB/s    0:00:00 (xfr#16, to-chk=4/22)
    test2/file6
              0 100%    0.00kB/s    0:00:00 (xfr#17, to-chk=3/22)
    test2/file7
              0 100%    0.00kB/s    0:00:00 (xfr#18, to-chk=2/22)
    test2/file8
              0 100%    0.00kB/s    0:00:00 (xfr#19, to-chk=1/22)
    test2/file9
              0 100%    0.00kB/s    0:00:00 (xfr#20, to-chk=0/22)
    
    sent 1,165 bytes  received 404 bytes  448.29 bytes/sec
    total size is 0  speedup is 0.00
    

    Everything copied to the server. But what exactly happened here? Well, let's review the command we issued:

    rsync -avP -e "ssh -p 22" {test1,test2} user@10.0.2.51:/home/user

    The -a option is for archiving which we used before. But we also added -v for verbose, meaning we get a detailed response of what rsync is doing. -P is for progress. This gives us the status in real time (super-handy). Next, we see ssh -p 22 which is telling us which shell to use (SSH), and the port to run it on (22). Because SSH is already running, and because we are using port 22 by default, we don't have to include this option. However, in many scenarios, SSH will be configured to use a different port (such as 2222) - in that case you would need to specify it: ssh -p 2222. We'll discuss this concept more in Day 2. Finally, we copied both directories {test1,test2} by simply comman separating them and encapsulating them in curly braces. You don't need the entire path or even relative paths when using rsync.

    Note

    You could also install the progress program (apt install progress) and run the watch progress command to view the progress of processes including large data transfers with rsync or SCP, DD processes, and anything else that runs behind the scenes.

    Subsequent transfers using the same command will work in an incremental fashion. That means that only new files or files that have been changed will be copied over. rsync processes each file during the initial and subsequent file copy processes. This makes for quick periodic (or incremental) backups.

    Note

    Automate with rsync! Use it within a cron job or other automated system to periodically copy files from one location to another. But be sure to test it periodically too! Make sure your file copies are running properly!

SFTP

Secure File Transfer Protocol (SFTP) is widely used for remote file uploads and retrieval. Like SCP, SFTP piggybacks on SSH. But where SCP is used for single file transfers, SFTP can be used to move multiple files and directories - multiple times within an SFTP session.

SFTP can resume file transfers and has more functionality than SCP, plus when you log in, you are placed in the SFTP shell, where you have many commands at your disposal. You can manipulate the remote system's directories and files as well as the local system's files and directories when you are logged into an SFTP shell.

Note

There are plenty of other SFTP packages available. For this lab we will simply use SFTP, which works as part of OpenSSH.

Lab 4-4

Using SFTP

In this lab, we'll connect to an SFTP server, view the contents, retrieve a file, and place a file on the server as well. In the example I'll be showing a Fedora system (10.0.2.55) connecting to a Debian system (10.0.2.52), but any Linux distribution will work in the same manner as long as you have OpenSSH installed.

  • Connect to the OpenSSH server (Debian client for this lab)

    sftp user@10.0.2.52

    Login with your password.

    Change to the Downloads directory: cd Downloads. Then type ls to see its contents. (If you are not using the Downloads directory, access whatever directory you are storing files in.)

    You should see similar results to this:

    [user@fed-client ~]$ sftp user@10.0.2.52
    user@10.0.2.52's password: 
    Connected to 10.0.2.52.
    sftp> cd Downloads/
    sftp> ls
    debian.iso   debian2.iso  test1        test2        
    sftp> 
    

    Here we can see the debian .iso files and the test1 and test2 directories that we worked with previously.

  • Retrieve a file from the SFTP server.

    Retrieving a file is easy. Type get <filename>. That's it. So for example:

    get debian.iso

    That will grab the debian.iso file from the remote system, and place it in the last directory we were working in. In this scenario I happened to be working in the Desktop directory. To show that the file indeed downloaded to our local system, we can run the lls command, which is local list directory contents. This is all shown in the example below:

    sftp> get debian.iso
    Fetching /home/user/Downloads/debian.iso to debian.iso
    /home/user/Downloads/debian.iso                                                                       100%  336MB 210.8MB/s   00:01    
    sftp> lls
    debian.iso
    sftp> 
    

    Note

    Many commands can be run locally while in an SFTP session. Just place an 'l' before the command. For example, cd becomes lcd, mkdir becomes lmkdir, and pwd becomes lpwd.

  • Place a file on the remote system.

    Pushing files to the remote system is easy also. Just use the put command. For example:

    put debian.iso

    That will put the file into the current remote working directory. That file is actually already there, so we can change the remote directory if need be. In the example below, I create a new directory, change to it, and then put the file in that new directory.

    sftp> lls
    debian.iso
    sftp> mkdir isos
    sftp> cd isos
    sftp> put debian.iso
    Uploading debian.iso to /home/user/Downloads/isos/debian.iso
    debian.iso                                                                                            100%  336MB 252.8MB/s   00:01    
    sftp> 
    

    While in an SFTP session, press the ? key to access a basic help system and list of commands that you can run.

    That's about it. When finished you can exit the session by typing bye or quit or by pressing Ctrl + d.

Summary

That wraps up the section on SSH, SCP, rsync, and SFTP. They are powerful tools, and each one has its own purpose. Practice with them to master their usage.

We worked with the Secure Shell (SSH) to remotely control systems from the command line. We also worked with Secure Copy (SCP) to easily copy files from one system to another over an SSH connection. Then we used an alternative to SCP called rsync which not only copies files, but can archive them, and work in an incremental fashion. Finally, we used Secure FTP (SFTP) to move files back and forth between a local system and a remote system. SFTP, unlike SCP and rsync, works within an interactive shell which allows us to perform many actions instead of just one.

With SCP, rsync, and SFTP, transactions are either push (from the local system to the remote system) or pull (from the remote system to the local system). The command structure is always command source destination regardless of whether you are performing a push or a pull.

Remember to practice these commands as much as possible to become familiar with them. This skill is necessary for most IT technicians.

Reminder: To learn more about any one of these commands, simply type <command> -h or man <command> - where <command> is either ssh, scp, rsync, or sftp.


And that does it for the Day 1 Lab Guide. Phew! Until the next round!