Kategoriak: All - processes - configuration - directories - scripts

arabera Dustin Cerda 5 years ago

1346

CCTC-Linux

The document provides insights into managing and verifying processes and system configurations in a Linux environment. It discusses how to identify unusual processes by examining the directories from which they are executed, using commands like `pwdx`, `ls`, and `ps` to gather information about process IDs and their working directories.

CCTC-Linux

Modes

Startup

File System Structure

Connection / Listening Ports

Find user/process making connection or listening on port:

netstat -lntp

List listen TCP, don't resolve port numbers, show program.

netstat -antp

List all TCP, don't resolve port numbers, show program.

netstat -lnp

Listening ports, don't resolve port numbers, show program.

netstat -anp

List all, don't resolve port numbers, show programs.

ss

Displays stats for PACKET, TCP, UDP, DCCP, RAW, and Unix domain sockets

Validity of Processes /proc


-pwdx- gets current working directory of process

ls -l /proc/{PID}/cwd

ps auxwwwe

ps -ef-get PID with:

ls -l /proc/PID/exe-find the path and binary from where it was invoked.

Aptitude/APT

Installs programs from repository

Boot Configuration Files

Busybox-init

Popular in embedded devices,

https://busybox.net/FAQ.html

https://busybox.net/about.html



SystemD



Features
Targets & Units

Sys V Init

Runlevels


6-Reboot
5-Xll (GUI)

Start the system normally with appropriate display manager (with GUI), same as runlevel 3 + display manager.

4-Unused or experimental

For special purposes.

3-Full multiuser mode, with networking

Starts the system normally.

2-Multiuser, without networking

Does not configure network interfaces and does not export network services.

1-Single user mode

Mode for administrative task.

0-halt

Shuts down the system.

Sys Changes after Mod of Boot Config File
Openrc

Extension of Sys V Init, adds parallel service startup, and dependency based boot.

Cron-Recurring Jobs


https://crontab.guru/


Minute, Hour, Day of the Month, Month of the Year, Month of the Week.

Linux Key Files

SETUID/SETGID

Why Does it Mater

When bash is invoked as an interactive login shell, or as a non-interactive shell with the --login option, it first reads and executes commands from the file /etc/profile, if that file exists. After reading that file, it looks for ~/.bash_profile, ~/.bash_login, and ~/.profile, in that order, and reads and executes commands from the first one that exists and is readable

When an interactive shell that is not a login shell is started, bash reads and executes commands from /etc/bash.bashrc and ~/.bashrc



Persistence

Also applies to any aliases, functions, scripts, etc. running from these file locations.

man bash | grep -EA48 "^INVOCACTION"

clarifies the interactive vs. non-interactive shell env. files

~/.profile

File applies only to login shells.

~/.bashrc

files applies only to BASH non-login shells

/etc/profile

File applies only to Bourne and BASH compatible shells; SETS $PATH VAR:echo $PATH | tr':''\n'

/etc/bash.bashrc

file applies only to interactive BASH shell

Kernel Comparison

Linux File Structure

Shell Differences

CCTC-Linux

Shell

File System

df -t, filesystem disk usage.


find / -maxdepth 1 -type d, filesystem information (main)




File Permissions

Sticky

When set on a directory only the owner of the file can delete or rename files in that directory. (used for directory only)



Uppercase "T" indicates the execute bit IS NOT set, lowercase "t" execute bit is set.

Set Group Identification (SGID)

Program set with SGID runs under the security context of the group of the program.

Uppercase "S" indicates the execute bit IS NOT set, lowercase "s" execute bit IS set.

Set User Identification (SUID)

Program with SUID runs under the security context of the user (owner) of the program



Uppercase "S" indicates the execute bit IS NOT set, lowercase "s" execute bit IS set.

Filesystem Ownership

ls -la-Show owner:group

chown-Change owner

chown {owner}:{group}{file}-Usage

chgrp {group}{file}-Usage


getent gets global config files


getent -help, get entries from Name Service Switch libraries

getent group, show all the groups

getent password

getent {group, sudo), show sudo group output only

File System Timestamps

EXT4

crtime: creation time

EXT3

  1. mtime: Modified time - time file content was last modified
  2. atime: Access time - time the file was last accessed (persistent for 24hrs)
  3. ctime: Change time - time of inode record change (file attribute changes: size, location, type, etc.)

What is Inode

  1. A Unix file is "stored" in two different parts of the disk - the data blocks and the inodes
  2. The data blocks contain the "contents" of the file. The information about the file is stored elsewhere - in the inode.
  3. The inode contains the following pieces of information:
  4. Owner ID
  5. Group ID
  6. Size of file
  7. Number of hard links to the file
  8. Time last accessed
  9. Time last modified
  10. Time inode last modified


ls -i: list the inode of a file


An iode is a data structure on a filesystem on Linux and other Unix-like operating systems that stores all the information about a file except its name and its actual data.


http://www.linfo.org/inode.html

Mouting Filesystem

mount -t {device} {dir}

unmount {device}

File System Types

File System Information (main)


find / -maxdepth -type d

sysfs

Pseudo file system providing information about kernel, hardware, device drivers.


Designed to add structure to the old procfs systems

Pseudo file system

tmpfs

Appears as a mounted volume, but is actually stored in volatile memory


/var/run on older distributions, /run on newer.

Temporary file storage

ext3/4

Hard Drive partitioning format

Dynamic File Systems
/dev

Device directory that is dynamically populated by udev (FKA devfs)

/tmp

Information about processes, connections, and some hardware.

Pseudo File Systems
/sys

Information about the system (hardware)

/proc

Information about processes, connections and some hardware.

Bash Order of Evaluation
  1. Shell reads commands from a file (script) or directly from the user's terminal.
  2. Initial job of the parsing engine is lexical analysis: to separate the stream of characters into words and apply meaning to the results. The word (token) is the base unit which the parser operates. Words are sequences of characters separated by metacharacters, which include simple separators like spaces and tabs, or characters that are special to the shell language, like semicolons and ampersands. Alias Expansion is performed at this step.
  3. Shell parses (analyzes and substitutes) the tokens into simple (cat file) and compound commands (if..then..fi).
  4. Bash performs various shell expansions, breaking the expanded tokens into list of filenames and commands and arguments.
  5. Brace expanision-echo {1...3}
  6. Tilde expansion is used to expand to several specific pathnames.
  7. home directories: echo ~
  8. current working directory: echo ~+
  9. previous working directory: echo~-
  10. Parameter and variable expansion
  11. word="car" echo $word echo ${word}s
  12. command substitution
  13. 'command' $(command)
  14. thedate=$(date)
  15. Arithmetic expansion
  16. a='expr $z + 3'
  17. a=$(($z+3))
  18. Work splitting
  19. IFS-Internal Field Separator <SPACE><TAB<>NEWLINE>
  20. After (Parameter, command, and arithmetic) expanision bash scans the results of IFSs.
  21. The IFS variable holds characters that bash sees as a word boundaries.
  22. Filename expansion
  23. echo "hello" > Middleton
  24. cat Midd*
  25. Redirection is performed if necessary, redirection operators and there operands are removed from the argument list.
  26. Commands are xecuted.
  27. Shell functions
  28. Builtin commands
  29. Hash table
  30. Path Variable
  31. Optionally the shell waits for the command to complete and collects its exit status.
CMD

Path

Hash

Builtin

Prep

Expansion

Alias

Redirection

Commands to Know

File Information

locate -e find

only if it exists

locate -i find

Returns EVERY file name containing "find" in the filename; case insensitive.

Compressed Files

zgrep

bzgrep

Search possibly compressed files for a regular expression.

Information Gathering / Help

{command} -h

{command} --help

whereis

if in $PATH

View binary path AND manpage path

-h / --help

Displays the most used switches/options for the chosen command

man

Displays the manual for a command, automatically paged using more:


Example: man -s 7 {command}

man -k {command}

manpath | tr':' '\in'


apropos-Each manual page has a short description, searches for keywords. (Non-Built in commands)

Boolean Logic

Semi-Colon ;

Executes next command unconditionally / regardless of the xit status

Example: ping -c4 127.0.0.1; echo "All done"


Logical/Operator &&

Executes next command only if the previous command succeeded / exit status of 0

Example: cd /tmp is && echo "it works"

cd /fakedire && ls && echo "It works"

NOT

Reverses input state

XOR

One of the inputs is true, but not both

OR

One of the inputs is true

AND

Both inputs are true

put a process in the foreground

put a process in the background


To put in the background hit CTRL-Z

%1

an alternate way of listing your own processes

kill

send a signal to noe or more processess to stop it


Kill 1 or %1 (saved job)


ps

list the processes running on the system

History

echo $HISTFILE

/home/username/.bash_history


echo $HISTFILESIZE

2000 (example)


echo $HISTSIZE

1000 (example)

$HISTFILESIZE

Determines the amount of lines saved to ~/.bash_history upon session exit.

$HISTSIZE

$HISTFILE

alias

unalias

type

type {Command}, evaluates bash order of evaluation to determine the first instance of the command


type -a{command}, shows all instances of command in the order of bash evaluation

hash

hash, displays bash's hash table (Recently executed commands)


hash -r, clear bash's hash table

which

which {nc,netcat}, evaluates the PATH in order and displays the first instance of the command if it exists


which -a{command}, evaluates the PATH in order and displays all instances of the command if it exists.

netcat

>ls /bin | grep netcat


>file /bin/netca*

Pipes
Named Pipe / FIFO
  1. Created with mkfifo command or mknod p command
  2. Exit on filesystem with a name
  3. Can be accessed by unrelated processes.
  4. Bi-directional


(FIFO, First In First Out)


Creates Named Pipes.


A named pipe has a file name on your file system and can be accessed by independent processes that were not spawned by the same parent process.

Unnamed Pipe: |
  1. Uni-directional
  2. Exists in RAM
  3. Opened at time of creation
Standard Streams
>> or 1>> / 2>&1

-to append rather than overwrite when redirecting.


-when redirecting standard error into standard output.

STDERR: Standard error

Output of a programs error handler, file descriptor 2


Represented as 2> when redirecting standard error to a file

STDOUT: Standard Output

Output of a program, file descriptor 1


Represented as 1> or > when redirecting standard output to a file

STDIN: Standard Input
Bash Mode

Input into a program, file descriptor 0

Represented as 0< or < when redirecting standard input from a file

Runs in the background.

Reads commands from user input.

Shell Types

Determining Shell Type (Shell options)


shopt login_shell

Shell Modes

Interactive

Interactive: reads commands from user input on a tty ($PS1 prompt variable is set)

Non-Interactive

Shells running scripts, can run in background.


Can't read Standard Input (STDIN)

Non-Login

Shell invoked from another shell


Started by a program without a login

Login

echo $0

Displays current shell.

Networking

SAMBA
File Transfer Methods

tftp

ftp

scp

Telnet



File transfer via telnet

uuencode/base64

uudecode/base64 -d

Program/protocol allows remote console connectivity; clear text protocol, uses port 23 by default.

Plain text

Basic Authentication

Gather Network Information
Enumerate Active Connections

watch

telnet

nc

nmap

route

Display routing table

arp

Display ARP table

ss & netstat

Display network conneciton info

ip & ifconfig

Display/configure network interface info

Network Super Servers
xinetd
  1. bind= <IP Address>: listens only on network interface for the service.
  2. only_from=<IP Addresses|Network>: accept connections only from IP addresses.
  3. no_access=<IP Addresses|network>: Deny connections only from IP addresses.
  4. access_times=hour:min-hour:min: sets time which users may access the server.
  5. banner=/usr/local/etc/deny_banner
inetd
Basic Network Services
smbd (samba)

Server Message Block Server Daemon

named (bind)

Dynamic Naming Service Server Daemon

dnsmasq, nscd

Name service cache Daemon

nfsd
iptables, nftables, ufw

Network filtering protocol service

snmpd
postfix, sendmail

Mail Server Daemon

sshd
nginx

High-performace HTTP server, reverse proxy, IMAP/POP3 proxy server

httpd (apache)

HTTP Daemon

ntpd

Network Time Protocol Daemon

Basic Network Commands
nslookup

Query internet name servers interactively.

dig

Interrogates DNS name servers.

host

DNS lookup utility.

netstat

Print network connection, routing tables, interface statistics, masquerade connections, and multicast memberships.

Socket

Network socket is an internal endpoint for sending or receiving data at a single node in a computer network.


Typical network socket format: (IP:Port): 1.2.3.4:25

Raw Socket

No layer checking done, it is up to the application using the raw socket ot interpret the data. RAW sockets are used as packet capture/sniffer programs as it captures "raw" data from the network interface card and passes it directly to the application.

Non-Raw Socket

Protocol stack processes its respective layer, performing address, checksum validation, removes its respective header and trailer and passes up the content to its immediate upper layer.

DNS
/etc/nsswitch.conf

Determines order of precedence.


Determines default order of precedence for DNS resolution.

/etc/resolv.conf

Name Server settings.


File that manages nameserver information.

/etc/hosts

List of hosts and associated IP's.


Central file that controls resolver setup (host.conf). Resides in /etc, tells resolver which services to use and in what order.

Logging & Auditing

Logs

find /var/log -maxdepth 1 -type f


Logs are managed by syslog daemon

systemctl status rsyslog.service

grep rsyslog /etc/rsyslog.conf

Logs About Logins

System Calls

Auditd

ausearch-search for events

aureport-creating reports

-f files

a-attributes-attempt to change attributes

x-execute-attempt ot execute file

r-read-attempt to read from file

w-write-attempt to write to file

auditctl for config auditing

-p what ot watch

-k keyword

-W remove rule

-w watch

-l list rules

Journald

Journald doesn't use Syslog faciliteis by default (can be enabled in conf). Journald is designed to work with PIDs, Process Names, and Service IDs. It does use severtiy codes.

Journalctl

journalctl --since "1 hour ago" time ranges

journalctl -b {boot msg log#} boot message

journalctl --list boots boot message blocks

journalctl --SINCE "2017-06-26 23:00:00" --UNTIL "2017-06-26 23:20:00"

journalctl --vacuum-size=20m retain only the past 20 MB

journalctl --vacuum-time=5d retain only the past five days

journalctl UID=0 messages for UID 0

journalctl -o verbose -k -p 3 verbose kernel (-k) info(60 priority msgs

journalctl -n 50 --since "1 hour ago" - last 50 messages from 1 hour ago

journalctl -b -1 -p "crit" -by priority

journalctl -u dhcpcd entries associated with serivce unit

journalctl --help

Severity Codes

7. Debug: Debug-level messages

6 info: Information messages

5 notice: Normal but significant condition

4 warn, warning: Warning conditions

3 err, error: Error conditions

2 crit: Critical conditions

1 alert: Action must be taken immediately

0 emerg, panic: Emergency: system is unsable

Facility Codes
Types of Logging Systems

Logging Domain Sockets

ls -la /dev/log-This is a domain socket type


ls -la /proc/kmsg-Kernel ring buffer


dmesg-Read from kernel ring buffer, non-persisted (Look here for buffer overflow info)

syslog-ng

rsyslog

Local Log Management

Logrotate is aprogram that can manage your local log files


Use cron to run logrotate


The main configuration file contains configuration settings for all logs is /etc/logrotate.conf


Individual configuration files are kept in the /etc/logrotate.d directory

Authentication/Authorization

A&A logs are almost always system logs; there are very few cases where this is not the case.


A&A logs differ from system logs, as system logs only deal with the kernel, and auth logs deal with both the kernel and the service attempting to authenticate.

Application

Applications can choose to handle their own logging, or to use the syslog or journald logging applications to handle their logs.


In most cases applicatoins will default to /var/log for log storage, unless they have a it specifically specified in the config file to use another resource.

Linux Boot Process

Startup Process & Such
Process States

Additional Characters


< :high-priority (not nice to other users)

N :low-priority (nice to other users)

L :has pages locked into memory (for real-time and custom IO)

s :is a session leader

l :is multi-threaded

+ :is in the foreground process group

Z-defunct (zombie) process, terminated but not reaped by its parent.

X-dead (should never be seen)

W-paging

T-stopped, either by a job control signal or because it is being traced

S-interruptible sleep

R-running or runnable

D-uninterpretable sleep

Zombies
Orphans


Display total number of PID's


cat /proc/sys/kernel/pid_max


Job Control

at

Reads a series of commands and executes them at a later time

&

kill %JID

^z

bg

fg

jobs

Daemons


Works on most systems:


Non-systemd / SysVinit systems

systemd

Enumeration Commands

PS

ps axfo pid, ppid, comm

ps auxf

forest view (BSD style)

-o "%cpu comm"

o=format

-aux

Resource: CPU and Memory usage process stats

-elf --forest

l=long list


--forest=ASCII forest



-ef

e=every process


f=file

User Space

Services start at runlevel started

Programs have own virtual memory

Kernel Space

Represented in ps list with []

Single memory space

Partition Hard Disk Types
Logical Volume Management (LVM)

Linux utilizes LVM physical volumes to create pools of storage known as volume groups.


volume groups can be expanded without repartitoning the underlying disk.

GUID Partition Table (GPT) disks

Up to 128 partitions

Partition sizes over 2TB

The gdisk utility is used to manage GPT partitions

-Logical Volume Management (LVM)

GPT Layout

Hard disks utilizing a MBR can have a maximum of 4 primary partitions and a max size of 2 TB each.


Extended partitions overcame the 4 primary issue by moving addressing for additional partiions into the extended partition itself rather than in the MBR table.


The fdisk utility is used to manage MBR disk partitions.

MBR Layout

Process
Runlevel
Init

-/sbin/init


  1. Parent process of all user space programs, always PID of 1
  2. 3 main initialization processes.


********************Initialization Process 1*****************

  1. System V init:
  2. Based on run levels
  3. 2 Primary components

/etc/inittab file

-determines the initial runlevel for the system to boot into.

-runlevel determined by initdefault option

-application run for applicable runlevels

less/etc/inittab


bootscripts

-/etc/init.d/rc

-Script that runs Start and Kill scripts out of applicable rc folder

o file /etc/init.d/rc

o less /etc/init.d/rc


*********************Initialization Process 2*******************

System D

-Initiated in 2010 to create a service manager for Linux

-Includes device management (udevd) and logging (journald)

-/sbin/init symbolically linked to /lib/systemd/systemd

o runs the /etc/systemd/default.target which is a symbolically link to desired initial traget in /lib/systemd/system

o target creates a dependency tree which calls other target scripts

o multiple systemd programs available to handle

-uses the terms targets and units

o target is essentially equivalent to runlevel

o unit is essentially equivalent to daemon (service)


*******************Initialization Process 3*********************


-upstart

o Created for Ubuntu desktops (Ubuntu 15.04+ are systemd)

o Event driven, starts jobs based on events

o Configuration files for jobs kept in /etc/init/

o Continuous monitoring of the system after startup

Kernel

Stage 0 (process 0)

  1. Establishes memory Management, detects CPU type, page tables.
  2. Mount initrd and unpacks initramfs from it.
  3. Initial RAM Disk (initrd) is used by kernel as temp root file system until kernel is booted and the root file system is mounted. Contains necessary drivers complied inside, which helps to access the hard drive partitions, and other hardware.
  4. Mounts root file system as specified in the "root=" in grub.conf (grub 2.X) menu list (grub 1.X).
  5. Kernel executes the /sbin/init program by default.
  6. Mounts initramfs as initial root file system
  7. initramfs loads drivers, mounts actual root file system as passed in root= by bootloader
  8. initramfs runs /sbins/init
GRUB

GRUB-Grand Unified Bootloader (Loads Kernel into RAM)


  1. Can choose kernel images during bootup.
  2. Displays a splash screen, waits for a few seconds.
  3. Has the knowledge of filesystems.
  4. Configuration file is /boot/grub/grub.conf

2.x

Much different than GRUB 1.x

Config file is grub.cfg

Install command-gub-install

Configuration command: grub-mkconfig

Configuration options: /etc/default/grub

Legacy

Menu drive boot menu.

Can boot different OS's depending on default setting or choose an OS to load

Controlled via a config file (location depends on distro, /boot/grub.conf or /boot/grub/grub.cfg or /boot/grub/menu.lst

MBR
  1. Located in 1st sector of bootable disk: /dev/hda or /dev/sda
  2. Less than 512 bytes in size
  3. Primary boot loader 1st 446 bytes
  4. Partition table nes 64 bytes
  5. MBR validation check last 2 bytes
  6. Contains GRUB (LILO for old sytems)
  7. Loads and executes GRUB boot loader.
BIOS
  1. Performs some system integrity checks
  2. Searches, loads, and executes boot loader program
  3. Loads and executes MBR

RegEx

Regex Characters
+

One or more of preceding character set or character.

?

May or may not appear {applies to preceding character set or character}.

*

Any characters/wildcard.

.

Any single character one time.

Regular Expressions

https://cs.lmu.edu/~ray/notes/regex/



RegEx -find / locate

locate -regex "{some regex}"

find. -regextype sed -regex "{some regex}"

locate -regex

Example- locate -regex "{some regex}"

find --help

Example- find .-regextype sed -regex "{some regex}"

egrep / grep -E

Extended grep -Both commands accept the full set of regular expression capabilities.

man grep

man grep | grep -EA3 "(\-E,|\-P,)"

grep --h

Shows different grep regex uses.

Main topic

Bash Commands

Internal Vairables
BASH_REMATCH

Captures regex matches

echo $PWD

Present working directory

echo EUID

Effective UserID, Id that shell is currently running May be changed using su or sudo, 0 indicates root

echo $UID

User ID, 0 for Root, ID comes from password file.

echo $BASHPID

PID of current BASH shell

echo $BASH

Path to the BASH binary

Variables
env

Show global variables.

set

Show local scope variables.

Loop Types
WHILE
FOR
IF