CoreOS is a lightweight Linux distribution that integrates a platform for distributed environments. It makes Docker containers first class and adds some great features such as service discovery with etcd and cluster management with fleet. This post won’t go into too much detail on the benefits of CoreOS, so I recommend you head to the CoreOS site to read more. Instead we will be talking about how to get CoreOS running on VMWare ESXi.

ESXi provides a good stack for running multiple VMs on bare metal hardware. The first step to get CoreOS running is to download the latest beta VMWare image:

curl -LO http://beta.release.core-os.net/amd64-usr/current/coreos_production_vmware_insecure.zip
unzip coreos_production_vmware_insecure.zip
cd coreos_production_vmware_insecure

Next, grab the OVF Tool from VMWare’s website. This will convert the VMWare data into a format that is suitable for importing into ESXi. If you are using Mac OS X then the path to ovftool will be at:

/Applications/VMware\ OVF\ Tool/ovftool

Run ovftool to convert CoreOS to an ovf file:

ovftool coreos_production_vmware_insecure.vmx coreos.insecure.ovf

You will now have a file called coreos.insecure.ovf that we will use to create the VM on ESXi with.

Create VM in ESXi

Open up vSphere Client and connect to your ESXi instance.

  • Naviate to File..Deploy OVF Template.
  • Click on Browse… and locate the path to the .ovf file that you created.
  • Follow the remaining steps to finish creating the VM.

Once the VM has been created, you might want to tweak the settings, RAM, vCPU, etc,.

cloud-config

Before we boot up the instance, we are going to create a config-drive containing our cloud-init configuration. config-drive is a method from OpenStack for providing user_data to an instance.

CoreOS only supports the user_data file in the config-drive.

On your local machine, do the following:

mkdir -p config-drive/openstack/latest

In the latest directory create a file named user_data, this is where we configure cloud-init. See this page for full documentation on CoreOS’s cloud-config capabilities.

user_data

#cloud-config
hostname: my-hostname
ssh_authorized_keys:
    - ssh-rsa YOUR_KEY_HERE
write_files:
    - path: /etc/systemd/network/static.network
      permissions: 0644
      content: |
        [Match]
        Name=ens33

        [Network]
        Address=10.0.1.5/24
        Gateway=10.0.1.1
        DNS=8.8.8.8
        DNS=8.8.4.4
    - path: /etc/iptables.rules
      permissions: 0644
      content: |
        *filter
        :INPUT DROP [0:0]
        :FORWARD ACCEPT [0:0]
        :OUTPUT ACCEPT [76:7696]
        -A INPUT -p tcp -m conntrack --ctstate NEW -m multiport --dports 22 -j ACCEPT
        -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
        -A INPUT -i lo -j ACCEPT
        -A INPUT -p icmp --icmp-type 8 -m state --state NEW,RELATED,ESTABLISHED -j ACCEPT
        -A INPUT -p icmp --icmp-type echo-reply -j ACCEPT
        COMMIT
coreos:
    units:
        - name: systemd-networkd.service
          command: start
        - name: iptables.service
          command: start
          content: |
            [Unit]
            Description=iptables
            Author=Me
            After=systemd-networkd.service

            [Service]
            Type=oneshot
            ExecStart=/usr/sbin/iptables-restore /etc/iptables.rules
            ExecReload=/usr/sbin/iptables-restore /etc/iptables.rules
            ExecStop=/usr/sbin/iptables-restore /etc/iptables.rules

            [Install]
            WantedBy=multi-user.target

The above cloud-config does the following:

  • Sets the hostname to my-hostname
  • Sets the SSH public key for the core user
  • Sets a static IP for the machine to 10.0.1.5
  • Adds a systemd unit for managing iptables with a couple basic rules (SSH, ICMP, etc,.)

After customizing the user_data for your VM, we can now create an ISO that we will attach to the VM.

The following command creates an ISO named configdrive.iso from the config-drive path that we created above.

mkisofs -R -V config-2 -o configdrive.iso config-drive

Finishing up

You will need to transfer the ISO file you created to your ESXi instance. You can scp it, for example.

Once the ISO is on the ESXi Server go to your VM settings and attach the ISO file to your VM.

Finally, boot the VM and you’ll see the console show up with your hostname. You can now ssh in to your CoreOS machine as the core user:

ssh core@ipaddress