Consider a simple problem in bioinformatics

Download the human TP53 mRNA transcript, variant 1:

NCBI accession: NM_000546.6

Problem of scale and reproducibility

This works well for one or a few sequences. But it sounds a bit cumbersome to document and reproduce, doesn’t it?

And what if we need to download dozens or hundreds of accessions?

This is where the CLI (command-line interface) comes in.

Let’s write a reproducible Bash workflow

First, we create and navigate to our project folder:

cd ~/projects/                  # Path varies by user and machine
mkdir -p tp53_example/data      # Create the project folder and data folder
cd tp53_example

Then, we can download the FASTA file using NCBI’s EFetch utility:

curl -o data/tp53_mrna.fasta \
"https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=nuccore&id=NM_000546.6&rettype=fasta&retmode=text"

With the file downloaded, we can inspect its contents:

head -n 2 data/tp53_mrna.fasta      # Display the first two lines
tail -n 4 data/tp53_mrna.fasta      # Display the last four lines
wc -l data/tp53_mrna.fasta          # Count the number of lines
less data/tp53_mrna.fasta           # Inspect the file interactively
nano data/tp53_mrna.fasta           # Open the file in a text editor

All these commands can be saved in a script file, such as scripts/tp53_download.sh, and then run with:

bash scripts/tp53_download.sh

Alright, so why Linux? Why not Windows or MacOS?

Bioinformatics workflows are heavily built around Unix/Linux command-line tools.

Commands such as head, cat, mv, cp, grep, awk, curl.

What should you be able to do after 2 hours?

GUI or CLI?

A GUI is useful.

The command line becomes especially useful when work involves:

What exactly are we installing?

Linux is not one operating system.

A Linux distribution (distro) combines:

• Linux kernel
• system utilities
• software packages
• package manager
• default configuration

Ubuntu: the one you will see in this course

Ubuntu is my primary Linux distro.

For this course, Ubuntu will be the default reference environment for demonstrations.

• beginner-friendly
• large software ecosystem
• extensive documentation
• common on personal computers and servers
• straightforward package management with apt and Snaps

Ubuntu is based on Debian

Both Ubuntu and Debian use the .deb package format and the apt package-management system.

Example:

sudo apt update
sudo apt install tree

Ubuntu is not a reskin of Debian

Ubuntu is based on Debian — but it’s its own distribution.

Ubuntu also adds its own technologies, such as Snap:

sudo snap install tldr   # Very useful utility, will be covered in a later section

How can I run Linux?

Knowing which distribution you want is only half the question.

You also need to decide how Linux will run on your computer:

Method 1: Virtual Machine

A virtual machine runs another operating system inside your current operating system.

Examples of a virtual machine:

• VirtualBox
• VMware
• Hyper-V
• GNOME Boxes

VM: Low risk, but with significant overhead

Method 2: Dual-Boot

A dual-boot system has two operating systems installed on the same computer.

Method 3: WSL2 (Windows Subsystem for Linux 2)

WSL2 uses a lightweight virtual-machine architecture with a real Linux kernel. It allows Linux environments to run alongside Windows, accessible through Windows Terminal. The command-line experience is similar to working with an HPC cluster via SSH.

WSL2 vs traditional virtual machines

Compared with a traditional virtual machine, WSL2 has fast startup and very low overhead. It provides close integration between Windows and Linux, allowing you to use Linux tools alongside your normal Windows applications.

Windows drives are accessible from Linux under paths such as /mnt/c/, while Linux files can also be accessed from Windows through \\wsl.localhost.

Installing WSL2

1. On Windows 10/11, open PowerShell as Administrator and run:

wsl --install

2. Restart Windows when prompted.
3. Open Ubuntu from the Start menu.
4. Create your Linux username and password.
5. Update Ubuntu:

sudo apt update && sudo apt upgrade

/ = filesystem root

Windows commonly uses drive letters like C: and D:

Linux presents one filesystem tree rooted at /. We often think of it as a tree with branches and leaves.

/
├── home/
│   ├── ht2905/
│   └── ace/
├── etc/
├── usr/
├── var/
└── tmp/

Where is my home?

The current user’s home directory is usually written as: ~

cd ~
pwd

If the username is ht2905:

/home/ht2905

Therefore ~ is a shortcut for /home/ht2905.

The exact path varies by machine.

Four symbols worth memorizing

Absolute or relative?

Absolute path

Starts at /.

/home/ht2905/project/data/sample.fastq.gz

Relative path

Starts from where you are now.

data/sample.fastq.gz

Same file, different address

You are here:

pwd
/home/ht2905/project

Then:

data/sample.fastq.gz

and:

/home/ht2905/project/data/sample.fastq.gz

refer to the same file.

Three things beginners often mix up

Terminal → gives you a place to type

↓

Shell (e.g., Bash, Zsh) → interprets what you type

↓

Command / program → performs an operation

↓

Operating system + filesystem

Where am I? Use: pwd

Print working directory

pwd
# Result: /home/projects/

What is here? Use: ls

List all files and folders under a specified directory

ls

ls -l      # detailed listing
ls -a      # include hidden files
ls -h      # human-readable sizes with long format
ls -lah    # combine common options

Useful flags for ls

ls -l      # detailed listing
ls -a      # include hidden files
ls -h      # human-readable sizes with long format
ls -lah    # combine common options

Compare these outputs:

Where should I go? Use: cd

cd: change directory.

cd data/
cd ..
cd results/
cd ~
cd /

A reliable beginner loop:

pwd
ls
cd <where>

Which user am I? Use: whoami

Print the current username. Self-explanatory.

whoami
# Result: ht2905 – My username

This matters especially on shared systems where:

• several people use the same server;
• permissions depend on the account;
• file ownership affects what you can change.

Command anatomy

command [options] [arguments]

Example:

ls -lah data/

ls         # command
-l -a -h   # options / flags
data/      #argument

Need a built-in manual? Try man

Simply type:

man <command>   # Display the built-in documentation pages – aka "man pages"
man ls

Think man is too verbose? Use tldr

man provides detailed documentation, but sometimes you just want a few practical examples.

For that, you can use tldr. tldr is really useful, but it isn’t shipped with Ubuntu by default.

To install tldr on Ubuntu:

sudo snap install tldr

After that, just type:

tldr <command>
tldr ls

Compare this output with that of man:

less vs. man

After that, just type:

tldr <command>
tldr ls

Compare this output with that of man:

Our Teaching Sandbox

Google Colab

%%bash

pwd
ls -lah

Or:

!pwd

• %%bash is used to mark a cell as a Bash script. It is not a command itself.
• ! is used to mark a line as a shell command.

How do I navigate deliberately?

Think of navigation as three questions:

1. Where am I? → pwd

2. What is here? → ls

3. Where do I want to go? → cd

Tab completion is your friend

Instead of:

cd very_long_project_directory_name

type:

cd ver<TAB>

Bash can complete filenames and directory names.

If the completion is ambiguous, press Tab twice to see matches.

alias ll='ls -lahF --color=auto'
alias tree='tree -C'    # Colorized tree view (requires tree package)

Organize your project!

A predictable project is easier to understand and automate:

cancer_rnaseq/
├── raw_data/      # original data
├── reference/     # genome / annotation
├── results/       # generated outputs
├── scripts/       # code
└── logs/          # logs / reports

Create a directory with mkdir

mkdir results

Nested directories:

mkdir -p project/results/qc
-p    # create parent directories as needed

Check:

ls -lah project/results/

Create an empty file with touch

Update a file’s timestamp, or create it if it does not exist

touch notes.txt

Then:

ls -l notes.txt
cat notes.txt # Outputs nothing

Copy a file with cp

cp sample.fastq.gz backup.fastq.gz

Copy into a directory:

cp sample.fastq.gz results/

Copy a directory recursively:

cp -r raw_data/ raw_data_backup/

Move files and folders around with mv

Move:

mv sample.fastq.gz raw_data/

Rename:

mv sample.fastq.gz sample_R1.fastq.gz

Remove files and folders with rm

rm notes.txt

Ask first:

rm -i notes.txt

Recursive removal: be careful!

rm notes.txt

Ask first:

rm -i notes.txt

Recursive removal:

rm -r old_results/
-r  # remove a directory and everything inside it

Summary: The Mental Model

mkdir  # create directory
touch  # create file / update timestamp
cp     #  duplicate
mv     # move or rename
rm     # remove

Before you analyze the data… inspect it.

Common text-based formats include:

• FASTA
• FASTQ
• SAM
• GTF / GFF
• BED
• TSV / CSV
• logs
• configuration files

Which tool should I use?

• Small file? → cat
• Extract the beginning? → head
• Extract the end? → tail
• Large file? → less

cat: print a file

cat notes.txt

Good for small files.

Not a good idea for:

30 GB FASTQ

head: extract the beginning

head sample.fastq

Exactly 20 lines:

head -n 20 sample.fastq

tail: extract the end

tail sample.fastq

Exactly 20 lines:

tail -n 20 sample.fastq

wc: Count lines, words, and bytes

A simplified FASTQ record:

@READ_001
ACGTTGCA...
+
FFFFFFFF...

1 FASTQ record = 4 lines

So:

wc -l sample.fastq

gives the total number of lines.

For a well-formed FASTQ file:

records = lines / 4

Example: 4,000,000 lines → 1,000,000 reads

less: Inspect without dumping everything

less large_file.txt

Useful keys:

Space / PageDown  → forward
b / PageUp        → backward
q                 → quit

What makes a file “hidden”?

Names beginning with . are normally hidden from plain ls.

ls
ls -a  # Show hidden files

Examples:

.git/
.gitignore
.bashrc

Can I see the file size?

Compare:

ls -l sample.fastq

with:

ls -lh sample.fastq

Human-readable sizes might look like:

12K
850M
7.4G

wget: Download from a URL

wget <URL>

Example:

wget https://example.org/sample.fastq.gz

Then verify:

ls -lh sample.fastq.gz

wget vs curl

wget

wget <URL>

Commonly used as a downloader.

curl

curl <URL>

General-purpose data transfer tool.

Save using the remote filename:

curl -O <URL>

Display or save?

This:

curl https://example.org/sample.txt

usually prints the response.

This:

curl -O https://example.org/sample.txt

saves the file.

Bonus: rsync, where does it fit?

Typical server transfer:

rsync sample_R1.fastq.gz \
      user@server:/data/project/raw_data/

Conceptually:

Laptop / Colab
      │
    rsync
      │
      ▼
Linux server / HPC

Useful for:

• repeated transfers;
• large datasets;
• synchronization;
• selected file metadata preservation.

Why can two users get different results?

Linux tracks:

owner
group
others

A long listing might begin:

-rwxr-x---

Read it as three blocks:

owner | group | others

Three permissions

Example:

-rwxr-x---

means:

owner  → rwx
group  → r-x
others → ---

Files and directories are different

For a file:

r → read contents
w → modify contents
x → execute as a program/script

For a directory:

r → list names
w → create/delete entries
x → traverse/access entries

Need to make a script executable? Use chmod

chmod +x hello.sh

Inspect:

ls -l hello.sh

Symbolic modes also exist:

chmod u+x hello.sh
chmod g-w data.txt

Permissions or ownership?

chmod → permissions
chown → ownership

Basic ownership syntax:

chown user:group file

In managed servers, ownership is often controlled by administrators.

Why does sequencing data end in .gz?

Instead of:

sample_R1.fastq

you often see:

sample_R1.fastq.gz

gzip compression reduces storage and transfer costs.

Compression changes one file

Compress:

gzip sample.fastq

Usually becomes:

sample.fastq.gz

Decompress:

gunzip sample.fastq.gz

Check:

ls -lh

Archiving bundles many files: tar

Create:

tar -cf results.tar results/

Extract:

tar -xf results.tar

Compressed archive:

tar -czf results.tar.gz results/

Extract:

tar -xzf results.tar.gz

How do I decode tar.gz?

tar = archive
 z   = gzip
 c   = create
 x   = extract
 f   = file

What if VS Code isn’t available? Use nano

nano notes.txt

Essential shortcuts:

Ctrl + O  → save
Ctrl + X  → exit
Ctrl + W  → search

Or an even more powerful alternative: vim

vim notes.txt

Vim is a powerful modal text editor. It can be extremely efficient, provided you can remember its many commands.

Unlike nano:

Normal mode  → navigate and run commands
Insert mode  → type text

The basic workflow:

i       → start typing
Esc     → return to Normal mode
:w      → save
:q      → quit
:wq     → save + quit

Want to learn vim?

vimtutor

A built-in interactive tutorial that teaches Vim’s core commands.

Navigation

pwd
ls
ls -l
ls -a
ls -h
ls -lah
cd DIR
cd ..
cd ~
cd /
whoami

Help & Shell Utilities

man COMMAND
tldr COMMAND

alias
alias ll='ls -lahF --color=auto'
alias tree='tree -C'

tree

Files & Directories

mkdir DIR
mkdir -p A/B/C

touch FILE

cp SRC DEST
cp SRC DIR/
cp -r DIR DEST

mv SRC DEST
mv OLD NEW

rm FILE
rm -i FILE
rm -r DIR

Inspecting Text Files

cat FILE

head FILE
head -n 20 FILE

tail FILE
tail -n 20 FILE

less FILE

wc FILE
wc -l FILE

Useful less keys:

Space / PageDown  → forward
b / PageUp        → backward
q                 → quit

Downloading & Transferring

wget URL

curl URL
curl -O URL

rsync SRC user@server:/path/

Permissions & Ownership

chmod +x FILE
chmod u+x FILE
chmod g-w FILE

chown user:group FILE

ls -l FILE

Compression & Archiving

gzip FILE
gunzip FILE.gz

tar -cf archive.tar DIR/
tar -xf archive.tar

tar -czf archive.tar.gz DIR/
tar -xzf archive.tar.gz

Remember:

tar    → archive
gzip   → compress
tar.gz → archive + gzip compression

Software Installation on Ubuntu

sudo apt update
sudo apt install PACKAGE

sudo snap install PACKAGE

Examples:

sudo apt install tree
sudo snap install tldr

Text Editors

nano FILE

vim FILE
vimtutor

Essential nano shortcuts:

Ctrl + O  → save
Ctrl + X  → exit
Ctrl + W  → search

Essential Vim commands taught today:

i        → insert mode
Esc      → Normal mode

:w       → save
:q       → quit
:wq      → save + quit