Linux

DevOps Linux System 01

• Azure's resource group is equivalent to GCP's project: a container for project-related resources (based on business logic)
• RedHat vs CentOS vs Debian vs Ubuntu
  1. Ubuntu: comes with a GUI
  2. Debian: more stable than Ubuntu, smaller size, rock solid as a base OS
  3. RedHat: the gold enterprise edition, quite expensive
  4. CentOS: the open-source counterpart of RedHat
• apt acts as an index connecting to different version repositories; apt update refreshes the package index
• ~/.jenkins can archive user files separately; JENKINS_HOME defaults to /var/lib/jenkins
• keep the same number of CPUs as executors (total number of concurrent jobs)
• SCM: source code management — pulls code from version control

DevOps Linux file struct 02

• /bin, /sbin, /usr/bin, /usr/sbin: The default directories for system executables — for example, ls lives at /bin/ls.
• /bin and /usr/bin are for general users (non-root), while /sbin and /usr/sbin are for root.
• /var: Stores logs and other data that programs generate at runtime. Log files go under /var/log; default mail storage is also here.
• /var contains data that changes during normal system operation. Its subdirectories grow and shrink frequently. Content that used to live in /usr was moved here to keep /usr relatively stable. This directory is system-specific and not shared over the network.
• /etc contains system configuration files, including many network configuration files.

DevOps Linux sudo vs su 03

• Difference between su and su -: su switches the user identity to root but keeps the current shell environment (the original user's PATH); su - switches both the identity and the shell environment to root's. Only su - avoids PATH errors. After su, pwd still shows the original user's working directory; after su -, it switches to root's home directory.
• sudo vs su: sudo = super user do; su = switch user
  1. su switches to root by default
  2. su - -c temporarily borrows root privileges to run a single command, then returns to the current user
  3. sudo su - == sudo -i: switches to root as a login shell. sudo su - requires the current user's password; su - requires the root password.
  4. Whether a user can use sudo depends on whether root has granted authorization in /etc/sudoers. Only root can edit that file with visudo.
• [root@localhost ~] means:
  1. Username: root
  2. Hostname: localhost
  3. Current path: ~ (the current user's home directory)
  4. Privilege indicator: # for root, $ for regular users
• Difference between ~ and / in [root@localhost ~]:
  1. ~ is root's home directory (equivalent to C:\Users\Administrator on Windows); / is the Linux root partition (equivalent to C:\).
  2. Under / there are /root (root's home) and /home (home directories for other users, typically /home/username)

DevOps Linux Cron Job

• set -x enables verbose logging (prints each command before executing); set +x disables it. It's an alternative to scattering echo statements throughout shell scripts.

• echo $?: checks whether the last command succeeded. Returns 0 on success, non-zero on failure.

• Searching files and content

  1. grep "search_string" filename — find lines in a file matching a string
  2. grep -e "regex" filename — find lines matching a regular expression
  3. grep -i "search_string" filename — case-insensitive search
  4. find / -name "*.log" | xargs grep "ERROR" — find all .log files from the root directory and search them for "ERROR"

• Pipes

  1. |: combine multiple commands when one cannot complete the task alone
  2. | xargs: use when the piped output is a list of filenames

• Scheduled tasks (cron)

  1. crontab -e — edit scheduled tasks
  2. crontab -l — list scheduled tasks

• Copy a file from local machine to a remote server (securely)

  1. scp -i ~/.ssh/key.pem <local_file> ec2XX@<public_IP>:<remote_path>

     e.g. scp -i ~/.ssh/my-key.pem newfile ec2@53.33.33.33:/home/ec2-user/newfile To copy from remote to local, reverse the arguments. FileZilla can also be used for transfers.

• Set file owner: sudo chown <username>:<group> <fileName>

• Watch a live log file: tail -f filename (-f follows the file as it grows)

• PID: process identification number — a unique ID automatically assigned to each process when it is created by the OS

• An executable file launched in the OS creates a running copy of the application called a process

• less is a handy command for reading file contents page by page

#AustraliaIT #DevOps #JobHunting #Programming

https://crontab.guru/

DevOps Regex Cheatsheet

Regular expressions are one of those things you learn and immediately forget — here's a quick reference:

• ^: matches the start of a string
• $: matches the end of a string
• *: 0 or more occurrences
• +: 1 or more occurrences
• ?: 0 or 1 occurrence
• x{n}: exactly n occurrences of x
• x{n,}: n or more occurrences of x
• x{n,m}: between n and m occurrences of x
• (): treat as a single group
• []: character set — matches any one character inside
• \d: a single digit
• \w: a single word character (letter, number, or underscore)
• \s: a single whitespace character (space, tab, or line break)
• \1: back-reference to the content captured by the first group, e.g. /(x)(y)\1/ matches (x)(y)(x)

\D, \W, \S are the negations of the above (non-digit, non-word, non-whitespace)

• Flags (placed after the closing / of a pattern)
  1. g: global search — returns all matches, not just the first
  2. m: multiline matching
  3. i: case-insensitive matching

DevOps Mac && Linux Environment Variables

• sh generally refers to the Linux shell, but on Mac sh behaves differently (generally considered more compatible)

• bash is an extension of sh; on Mac both sh and bash refer to bash

• Environment configuration load order:

  1. /etc/profile — loaded first, applies to all users, lowest priority (not recommended to modify)
  2. /etc/paths — global environment variable configuration file
  3. ~/.bash_profile → ~/.bash_login → ~/.profile → ~/.bashrc — loaded after system files; if bash_profile is found, the others are skipped. Order may differ on Linux.
  4. ~/.zshrc — used when oh-my-zsh is installed; the system reads ~/.zshrc on zsh startup and skips ~/.bash_profile. Add source ~/.bash_profile at the end of .zshrc to also load bash configuration.

• source ~/.bash_profile — reload the configuration file to apply environment variable changes immediately

• export ZSH=$HOME/.oh-my-zsh — export an environment variable; view it with echo $ZSH

• PATH=$PATH:$HOME/bin — append to the existing PATH using : as a separator; the shell searches each directory in order when looking for executables

• Environment variables set directly in the terminal (without adding to a configuration file) only apply to the current session and are lost on restart

• alias ll='ls -lt' — define a command shortcut; run alias alone to list all defined aliases

• env — display all environment variables

• echo $PATH — display the value of a specific environment variable

• Linux command chaining operators

  1. & — run a task in the background
  2. && — run the next command only if the previous one succeeded
  3. | — pipe the output of one command as the input to the next
  4. || — run the next command only if the previous one failed

• Linux IO streams: stdin, stdout, stderr — equivalent to Python's input(), print(), and exceptions

• Trim whitespace from both ends: sed -e 's/[[:space:]]*$//' | sed -e 's/^[[:space:]]*//'