3 ms·
the entire point was that it's not mapreduce, it's a stream processing system. try http://wiki.s4.io/Manual/S4Overview#toc1 http://wiki.s4.io/Manual/S4Overview#
by rektide 16y ago
the entire point was that it's not mapreduce, it's a stream processing system. try http://wiki.s4.io/Manual/S4Overview#toc1 http://wiki.s4.io/Manual/S4Overview#toc1 for a "what is S4" intro.
The architecture resembles the Actors model, providing semantics of encapsulation and location transparency, thus allowing applications to be massively concurrent while exposing a simple programming interface to application developers.
and
S4 is a general-purpose, distributed, scalable, partially fault-tolerant, pluggable platform that allows programmers to easily develop applications for processing continuous unbounded streams of data.
go stream processing. also see Flume for related recent works.
- DanielRibeiro 16y agoSeems very much like Percolator from google: http://www.infoq.com/news/2010/10/google-percolator http://www.infoq.com/news/2010/10/google-percolator
- schumihan 16y agoI don't think so. Percolator does not support stream operators, and it supports transaction semantic. Streaming process engine like s4 will never handle the race conditions brought by multiple writers.
- atarashi 16y agoIndeed, it's not really MapReduce (as in Hadoop or Google's MapReduce), but Yahoo Labs did call it "Real-Time MapReduce." http://labs.yahoo.com/event/99 http://labs.yahoo.com/event/99