Channel: distributed systems

Crash-only software: More than meets the eye [LWN.net]

“only way to stop is to crash. only way to start is to recover”

devops distributed systems

count-min sketch

probabalistic approaches to stream calculations

distributed systems

Stanford Seminar - Joe Armstrong of Ericsson

Another good Joe Armstrong talk.

  • systems that they originally designed purely for fault-tolerance did better on scalability benchmarks than systems designed for scalability. Led to the observation that they are related and instead of a tradeoff (“you can have either scalability or fault-tolerance”), instead you have “you can have both or neither”
  • according to Alan Kay, messaging was really the key idea of OOP, as it truly decouples objects (original idea in Smalltalk/Squeak, but lost in XEROX PARC work)
  • error handling must be external because the machine the error happened on might be dead. Design fundamentally is to not have to program differently for everything running on one machine vs multiple machines.
  • Erlang terms are designed to be serialized/deserialized without parsing (just very low level marshalling). Part of the reason that databases like Riak can be so fast.

distributed systems erlang

Consensus Filters

Algorithms for approximate consensus. Nodes can asymptotically all converge on (roughly) the same value with very small amount of inter-node communication.

distributed systems