Research VM Instance Farm

Several bare metal hosts are being made available to faculty and research groups to run self managed VM instances based on Vagrant and Virtual Box. This is being made available for users who need to run services that ICS Computing Support cannot readily provide. Anything that can be run by a user on their own machine can be run on the vagrant VM instance.

Server Details

Description Information
ClusterArchimedes Cluster
Count 24
Model HP Proliant SL390 G7
Processor 2 x Intel Xeon X5680 6 cores @ 3.33 Ghz (24-total cores double-threaded)
RAM 96 GB
HDD 240 GB SSD
RAID No
Redundant Power Yes

Purpose of Farm

Some examples of the usage for the farm:

  • Web server running applications ICS does not currently have.
  • ICS course needed to run an application for students to access.
  • Tool to assist faculty with creating of Canvas quizzes easier for online teaching from Azure
  • Deploy an API for a website
  • Set up a public-facing reverse proxy server

NOTE: This is not meant to be for an intensive compute VM but we can take requests case by case so email helpdesk anyway so we can discuss.

Host Assignment Details

  • Each bare metal host will be assigned to one vagrant instance initially.
  • As we run out of unassigned bare metal hosts, we cycle through the hosts.
    • We will look at the system usage and add additional VM as system resources allow.

What You Get

  • An assigned bare-metal host to run vagrant instance
  • An assigned static DHPC IP address based on a mac address, which is provided by Computing Support
  • CNAMES as requested
  • Ports opened at campus firewall as requested
  • Run the OS of your choice
  • Self-managed machine

Request a Research VM instance

If you are interested in a VM instance, please email helpdesk@ics.uci.edu with the following information:

  • Purpose
  • Number of vagrant machines needed
  • Valid users or group (so we can allow proper login rights)
  • Period of time that the instance will be needed
    • We want to know this so we can free up resources when it is no longer needed.
  • CNAMES
  • Open ports
  • Any other resource requirements

Getting Started

NOTE: It is recommended that a group account be used to set up and run the VM so that any member of the group can start/stop/manage the VM.

Vagrant File

This line would need be added to the vagrant file to ensure the VM gets the correct assigned IP address. The mac would change from the one in the example to the assigned one.

config.vm.network "public_network",
  use_dhcp_assigned_default_route: true,
  mac: "abcd1234efgh"

Vagrant instance without Mac Address

  • A vagrant instance on the assigned host can be run without an assigned mac address.
    • Random DHCP address on the 128.195.13.0 subnet is assigned.
    • IP will not be opened on the firewall.
  • Useful just to test install different vagrant instances.
  • When the instance is ready, assign it the mac address before starting up.

Vagrant instance with static IP address

  • If you wish to use a static IP, here are the details:
    • Subnet Mask: 255.255.255.0 or /24
    • Gateway: 128.195.13.1
    • DNS servers: 128.195.25.66, 128.195.1.66, 128.200.1.201

Routing Table

Even though the IP is assigned, we need to change the default gateway so the VM passes traffic through the 128.195.13.1 gateway instead of the internal network gateway.

When you type the command 'route -n' it should look like this:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.0.2.2        0.0.0.0         UG    100    0        0 eth0
0.0.0.0         128.195.13.1    0.0.0.0         UG    101    0        0 eth1
10.0.2.0        *               255.255.255.0   U     0      0        0 eth0
128.195.13.0    *               255.255.255.0   U     0      0        0 eth1

If not, it will not respond to ping on a different subnet. Because there is only default gateway specified, the metric doesn't matter. We want to remove the first line of the routing table above.

OS File change (Preferred Method)

To make the routing table permanent, you can change the network files for the OS.

CentOS

Edit the /etc/sysconfig/network-scripts/ifcfg-eth0 file. Change the 'DEFROUTE' variable from YES to NO.

DEFROUTE=no

Delete the default route for 10.0.2.2 first.

ip route del default via 10.0.2.2 dev eth0

Restart NetworkManager:

systemctl restart NetworkManager

Double check 10.0.2.2 default route is no longer there after NetworkManager is restarted by running “route -n”:

To do this via vagrant inline script upon start up, add the following:

config.vm.provision "shell", run: "always", inline: <<-SHELL
  sudo sed -i s/DEFROUTE=yes/DEFROUTE=no/g /etc/sysconfig/network-scripts/ifcfg-eth0
  sudo systemctl restart NetworkManager
SHELL

The script method only works if the DEFROUTE variable exists in the file. This is what the ifcfg-eth0 file looks like after kickstart:

# Generated by parse-kickstart
TYPE=Ethernet
DEVICE=eth0
ONBOOT=yes
BOOTPROTO=dhcp
DHCP_HOSTNAME=centos8.localdomain
IPV6_AUTOCONF=no
IPV6_DEFROUTE=no
IPV6_PEERDNS=no
IPV6_PEERROUTES=no
IPV6FORWARDING=no
IPV6_AUTOTUNNEL=no
PROXY_METHOD=none
BROWSER_ONLY=no
DEFROUTE=no
IPV4_FAILURE_FATAL=no
IPV6INIT=no
NAME="System eth0"
Ubuntu 18.04 and higher

Ubuntu 18.04 uses netplan to set the network interface.

Add the following to /etc/netplan/50-cloud-init.yaml:

dhcp4-overrides:
    use-routes: false

The file should look something like this:

# This file is generated from information provided by the datasource.  Changes
# to it will not persist across an instance reboot.  To disable cloud-init's
# network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
# network: {config: disabled}
network:
    ethernets:
        enp0s3:
            dhcp4: true
            match:
                macaddress: 02:1d:48:b0:d5:b9
            set-name: enp0s3
            dhcp4-overrides:
                use-routes: false
    version: 2

After setting the file, run:

sudo netplan apply

The default route for 10.0.2.2 should be gone and there should be only one default route.

Ubuntu 16.04

When setting the public network in Vagrantfile, it seemed to automatically edited the /etc/network/interfaces for ubuntu/trusty64:

#VAGRANT-BEGIN
# The contents below are automatically generated by Vagrant. Do not modify.
auto eth1
iface eth1 inet dhcp
    # We need to disable eth0, see GH-2648
    post-up ip route del default dev eth0 || true
    post-up dhclient $IFACE
    pre-down ip route add default dev eth0
#VAGRANT-END

For /etc/network/interfaces, it should look something like this:

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet dhcp

auto eth1
iface eth1 inet dhcp
  post-up ip route del default dev eth0 || true
  post-up dhclient $IFACE
  pre-down ip route add default dev eth0

Modify as needed and restart network service on it and check the routing table.

Inline shell script during vagrant up

NOTE: For documentation purposes only. Do not use this method. When CentOS renews the DHCP lease, a Network Manager script runs and causes the default you remove in the inline script to come back.

This method runs a shell script during the start up to modify the routing table. Just add the following to the VagrantFile. Change “eth0” to whatever is listed as the Iface for 10.0.2.2 when looking at the routing table.

NOTE: Spacing matters so when you cut and paste it, make sure there are spaces but no tabs.

config.vm.provision "shell",
  run: "always",
  inline: "ip route del default via 10.0.2.2 dev eth0"

or

config.vm.provision "shell",
  run: "always",
  inline: "eval `route -n | awk '{ if ($8 ==\"eth0\" && $2 != \"0.0.0.0\") print \"route del default gw \" $2; }'`"

IMPORTANT - While in the vagrant VM, if the network services is restarted, the default gateway that was removed will come back because the the 2 inline script examples only delete the route upon boot. Also, while the VM is up, the default gateway seems to come back after 24 hours so this method of changing the routing is not preferred.

Security

Vagrant machines are inherently insecure. Please read about the base box and default configuration:

https://www.vagrantup.com/docs/boxes/base

We will flesh out more recommendations on security setup soon.

SSH Login

There are various SSH settings you can set:

https://www.vagrantup.com/docs/vagrantfile/ssh_settings

A couple of simple lines you can add to the VagrantFile is:

config.ssh.username = 'my_username'
config.ssh.password = 'my_password'

It is recommended that a private key be used instead via the config.ssh.private_key_path instead though. Hope to get this working and have an example at some point.

Maintenance

  • The user needs to keep the OS on the vagrant machine patched.
  • If it has security vulnerabilities, the user needs to patch it in a timely manner or else it will be shut off.

Reclaiming Vagrant VM

  • Please email helpdesk@ics.uci.edu when the vagrant instance is no longer needed.
  • After one full year, we will check around July of each to see if any vagrant instances have been abandoned.

Troubleshooting

IP address assigned more than once

Be careful that you only run one instance of the vagrant instance with the assigned mac address. If a running vagrant process is not stopped and a second one is started up with the same assigned mac address, the DHCP server will assign both instances the same IP address. This can cause confusion. To see if this is happening, when you login to the bare metal host and ping the VM, you will see (DUP!) in the responses.

services/research_vm_instance_farm.txt · Last modified: 2022/05/09 11:22 by dutran
CC Attribution-Noncommercial-Share Alike 4.0 International
Driven by DokuWiki Recent changes RSS feed Valid CSS Valid XHTML 1.0