Milestone 8 Putting the Sec in DevSecOps - Oliver-Mustoe/Oliver-Mustoe-Tech-Journal GitHub Wiki

This page journals content related to NET/SEC/SYS-480 milestone 8.

Table of contents

  1. 480-utils updates
  2. 8.1 Splunk Enterprise Installation
  3. 8.2 Splunk Forwarder installation
  4. Milestone 8 reflection
  5. Created files
  6. Sources for Milestone 8

VM Inventory

480-utils updates

First I deployed a new Ubuntu VM named "splunk" (VM in vCenter shown below):

Deploy-Clone -LinkedClone -VMName ubuntu.22.04.1.base -CloneVMName splunk -defaultJSON ./480.json

Then in my 480-utils function, I added the following function to allow the editing of deployed VMs:

function Edit-VMs ([string]$defaultJSON="",[string]$VM="",[int]$CPU=0,[int]$Memory=0) {
    try{
        # Find the path of the json file
        if ($defaultJSON -eq "") {
            $defaultJSON = Read-Host -Prompt "Please enter the path for the default JSON config"
            $conf = Get-480Config -config_path $defaultJSON
        }
        else {
            $conf = Get-480Config -config_path $defaultJSON
        }

        # Connect to vcenter server
        480Connect -server $conf.vcenter_server

        # See if user has selected a VM, if not...
        if ($VM -eq ""){
            # Get all VMs and display them
            $vms = Get-VM
            $index=1
            foreach($vm in $vms){
                Write-Host [$index] $vm
                $index+=1
            }
            while($true){
                # Choose a VM
                [int]$pick_index = Read-Host "Which index number [x] do you wish to pick?"
                if($pick_index -lt $index -and $pick_index -gt 0) {
                    $selected_vm=$vms[$pick_index - 1]
                    break
                }
                else {
                    Write-Host "[ERROR: Please select an inbound index]" -ForegroundColor Red
                }
            }
        }
        else {
            $selected_vm=Get-VM -Name $VM
        }
        Write-Host "You selected",$selected_vm.Name

        # Gather the relevant information and display to screen
        $VmName=$selected_vm.Name
        $NumCpu=$selected_vm.NumCpu
        $RamCount=$selected_vm.MemoryGB

        Write-Host "Information for $VmName :
[CPU] $NumCpu
[RAM] $RamCount (GB)
        "

        # See if user has selected a cpu or memory amount, if not
        if ($Cpu -eq 0 -and $Memory -eq 0) {

            # Prompt user for what they want to change, switch on it
            $UserChange = (Read-Host -Prompt "Would you like to change $VmName's [C]PU or [R]AM or [E]xit (C/R/E)").ToLower()

            switch ($UserChange) {
                # For CPU/Memory, prompt the user for the new settings and set it
                "c" {
                    $NewCpu = Read-Host -Prompt "Please enter in the new CPU amount"

                    $selected_vm | set-VM -NumCpu $NewCpu
                }
                "r"{
                    $NewRam = Read-Host -Prompt "Please enter in the new RAM amount in GB"

                    $selected_vm | set-VM -MemoryGB $NewRam
                }
                # ELSE give the exit
                "e"{
                    exit
                }
                Default {
                    Write-Host "NOTHING HAS OCCURED"
                }
            }
        }
       else {
            # If CPU and Memory are explicit, set them to what the user asks
            if($Cpu){
                $selected_vm | set-VM -NumCpu $Cpu
            }
            if($Memory){
                $selected_vm | set-VM -MemoryGB $Memory
            }

        }
    }
    catch{
        StandardError -err $_
        break
    }
}

And increased my splunk VMs CPU and Memory like the following (deployment and applied changes shown below):

Edit-VMs -defaultJSON 480.json -VM splunk -CPU 4 -Memory 4

I then powered splunk on and moved it into the BLUE1 folder. I waited until splunk was full powered on and grabbed the IP with the command below:

get-VMIP -VMName splunk -defaultJSON ./480.json

Afterwards I shutdown the VM and made a snapshot in the splunk Actions dropdown > Snapshots > Take Snapshot > named "BEFORE ANSIBLE":

8.1 Splunk Enterprise Installation

With this setup, I made a Splunk account, created a directory in ansible/files named splunk, and downloaded the .deb file for Splunk enterprise onto xubuntu-wan to be copied to the splunk VM:

wget -O files/splunk/splunk-9.0.4-de405f4a7979-linux-2.6-amd64.deb "https://download.splunk.com/products/splunk/releases/9.0.4/linux/splunk-9.0.4-de405f4a7979-linux-2.6-amd64.deb"

I also downloaded the universal forwarder as well to be copied to one of the BLUE1 machines:

wget -O files/splunk/splunkforwarder-9.0.4-de405f4a7979-linux-2.6-amd64.deb "https://download.splunk.com/products/universalforwarder/releases/9.0.4/linux/splunkforwarder-9.0.4-de405f4a7979-linux-2.6-amd64.deb"

As well I downloaded the needed Add-on from https://splunkbase.splunk.com/app/833:

and moved it to my ansible/files/splunk directory:

cp -r ~/Downloads/splunk-add-on-for-unix-and-linux_880.tgz files/splunk/

Then I created a Ansible script/Inventory (splunk-enterprise-setup.yaml) to do the following:

  1. Install Splunk onto the Splunk VM with the following parameters (see here for the created files list):
    1. Hostname: splunk
    2. IP: 10.0.5.200/24
    3. Create a splunk service user
    4. Add-Ons: Splunk Add-on for Unix and Linux
    5. Added indexes: 1x called 480
    6. Receiver on the default port (9997)

Below is a run of the script (command/result):

ansible-playbook -i inventories/splunk-inventory.yaml splunk-enterprise-setup.yaml --ask-pass -K

Result:

8.2 Splunk Forwarder installation

I chose to use ubuntu-1 (10.0.5.30/24) as my Splunk forwarder, where in its logs would be forwarded to my splunk VM in the "480" index. First, using the same process as splunk, I created a new snapshot for the ubuntu-1-1 VM labeled "BEFORE FORWARDER":

I then created an Ansible script (splunk-forwarder-setup.yaml)/Updated my splunk-inventory for the following requirements (see here for the created files list):

  1. Install the Splunk universal forwarder on ubuntu-1
  2. Have the configuration on ubuntu-1 sends logs to the 480 index

Below is a run of the script (command/result):

ansible-playbook -i inventories/splunk-inventory.yaml splunk-forwarder-setup.yaml

Result seen in a Splunk search:

Milestone 8 reflection

The steps for creating the Splunk enterprise and forwarder was a very mixed process, so I decided to include the reflection for both steps in one reflection. The majority of the problems I was having with milestone were minor, annoying things. I didn't know about the user seed config to begin with, and I couldn't really find anything on the internet except that the instructor video mentioned a user seed file. After that, it caught my eye in an article I was reading and I was able to find that it was exactly what I needed to not manually enter in a password into the Splunk install. Making a designated service user was also a little confusing overall, but it was mostly a permissions headache which are solved through perseverance more than anything. The Add-on and the index were the last piece of the puzzle. The addon was confusing to install (the docs didn't help much) but it was a simple copy past. After installation, setting up the inputs.conf was annoying, but I got there in the end through trial and error. The index also had scarce documentation associated with it in terms of automation, but I was able to find a good guide for Ansible after a bit of Googling. Setting up the receiver was quite easy, as the documentation for Splunk clearly listed a config file to edit. The rest of the problems I had were simple misspellings/forgetting to cross my t's and dot my i's. Overall this milestone was a very good introduction to the Splunk installation and becoming more comfortable with Ansible.

Created files

  1. splunk-enterprise-setup.yaml

    1. An ansible playbook to fully provision a ubuntu VM from linked state to being a splunk enterprise server (first part of script shown):

  2. splunk-forwarder-setup.yaml

    1. An ansible playbook to provision a configured ubuntu VM with a splunk forwarder (first part of script shown):

  3. splunk-inventory.yaml

    1. An ansible inventory of both the enterprise and forwarder machines as well as needed variables:

  4. user-seed.j2

    1. A templated Jinja file that provides the user info needed by splunk on initial user setup:

  5. unix_inputs.conf.j2

    1. A templated Jinja file to enable the *nix addon for splunks inputs (first part of script shown):

  6. inputs.conf

    1. A config file to setup a reciever on a enterprise splunk instance for the TCP port 9997 (first part of script shown):

  7. forwarder_outputs.conf.j2

    1. A templated Jinja file that sets up a forwarders output group/the enterprise address and port it should be forwarding to:

Sources for Milestone 8

⚠️ **GitHub.com Fallback** ⚠️