Unix as a Cloud
bsd_day(2011), Bratislava
Cloud Computing
Utility ⇒ Ideology
User Perspective
Computer ⇒ Acess
Server ⇒ Service
Juggernaut Pioneers
- Google, Facebook & other $1B+ startups
- rigidly centralized
- hugely inefficient
Juggernaut Unix Users
- relatively minuscule regard for sharing
- massive multi-level self-repeating reinvention
- the industry of wasted effort & lost opportunity
Unix in 2011
- not fit for simple tasks anymore
- e-mail, web, file storage, etc
- any innovative cloud product
A Typical Disruption
- stretches platforms
- leverages massive resources
- is inherently open-minded
A Case Study
- a task for 100-1000+ machines
- need for a little more performance than theoretically possible
- complete reliance on open-source ecology
Heterogeny
- owned (4 DCs), rented (5 DCs), virtual
- boot from HDD, SSD, PXE/NFS, USB flash
- varying access, expected uptime and security
Heterogeneity
- primary roles: storage, processing, web, DB, memcache, sphinx
- hardware: from small virtual nodes to 72TB machines
- network: private/open, controlled/provided
Cloud Ideology
- multi-machine, multi-location abstractions
- failure anticipation and remediation
- deep automation
Unix as a Cloud
- lack of abstraction beyond a single host
- virtually zero hardware/kernel failure tolerance
- antique userland with manual transmission
Storage
- reliable storage, processing & streaming in one
- UFS2 per hard drive
- sha256-named objects, mapped to hard drives
Processing
- custom DB-backed dispatcher
- fault-tolerant race resolution
- limits and error handling
Configuration
- / on NFS ⇒ / in git
- role-aware/multi-role config files
- LDAP ⇒ passwd(5) in git
The Problem
- big clouds have very busy engineers working on them
- very little sharing (Hadoop, OpenStack)
- seminal opportunity for BSD
The Solution
- advocate sharing on the OS level (instead of middleware)
- start making BSD more convenient for large users
- embrace cloud ideology as a course of action
questions?
we're funding! (and hiring)