asavinov

52 ms·

61.
by asavinov 8y ago
It is based on a quite general (and novel) technology so it can be applied to many different data processing tasks like data integration, data migration, extract-transform-load (ETL), big data processing or stream processing. But I would li
62.
by asavinov 8y ago
Bistro Streams [1] is similar to Kafka Streams in how it is supposed to be used - it is implemented as a library which can be part of an application (including IoT analytics) or run at the edge, for example, on gateways or devices. However,
63.
Show HN: Bistro Streams – A light-weight column-oriented stream analytics server(github.com)
26 points by asavinov 8y ago|5 comments
64.
by asavinov 8y ago
The question whether we need distributed stream processing can hardly be answered unambiguously. In general, everything depends on what features we actually need: scalability, fault tolerance, high availability, performance (latency or thro
65.
by asavinov 8y ago
> it seems like it's almost a hybrid between a message queue and a database? If Kafka supported data processing (queries or whatever) then it would be much closer to databases. Also, databases are normally aware of the structure
66.
by asavinov 8y ago
Formally, null can be defined as empty tuple NULL=⧼⧽ as opposed to normal values which are non-empty tuples like ⧼5.0⧽. It is analogous to having empty set ∅={} and non-empty sets {5.0, 6.0, ...}. TUPLES SETS NULL=⧼⧽ ∅
67.
by asavinov 8y ago
Sad to see it go. I had professional interest to Fieldbook because I had been developing a similar "spreadsheet"-like application intended for radically changing the way table table is processed - http://conceptoriented
68.
by asavinov 8y ago
> Giving meaning to ... Ingesting such amounts of data is a challenge indeed. But problems will become much more complicated if it is necessary to perform complex analysis during data ingestion. Such analysis (not simply event pre-proces
69.
by asavinov 9y ago
I have also been a similar to tool for automatically extracting (informative) features. What I am trying to understand is whether it is possible to store the learned feature definitions for later use. For example, I want to learn interest
70.
by asavinov 9y ago
Bistro Streams is an open source light-weight stream analytics engine: https://github.com/asavinov/bistro It is a general-purpose and highly configurable data processing engine which can be applied to many workloads an
71.
by asavinov 9y ago
Functional Programming and Category Theory: http://nikgrozev.com/2016/03/14/functional-programming-and-c...
72.
by asavinov 9y ago
> Bistro creates just a new column in an existing table. What is new is that this column can use data from other tables - not necessarily this one. And eventually Bistro gets rid of joins and group-bys which are not difficult to understa
73.
by asavinov 9y ago
Honestly, this looks like a query and analysis system that could be built on top of a column store so you don't need to deal with the storage part of the system yet. (I'm a big KBD+ fan). I like this idea because it allows for r
74.
by asavinov 9y ago
You are right - it is an MVP, and the goal is to choose a direction. In fact, I am still not sure in which direction to go: * JavaScript data processing framework for in-browser data processing * Python framework like pandas * Big data proc
75.
by asavinov 9y ago
Source code for this web-app and previous projects: * UI (Angular 2): https://github.com/asavinov/sc-web * REST server: https://github.com/asavinov/sc-rest * Core engine: https://github
76.
by asavinov 9y ago
Bistro is not about a physical model and columnar (physical) representation although it relies on it. It is about logical level of representation and processing. There are at least three major questions when we want to introduce a new logic
77.
by asavinov 9y ago
> What is the use case? This is what I am trying to understand myself :) Previously I have implemented this approach as a web-app (self-service tool for working with tables where users can define columns as formulas - similar to spreadsh
78.
by asavinov 9y ago
At the physical (storage) level it is in-memory and organized as a column store, that is, internally it stores a list of tables (with no data) and a list of columns (each being a Java array). It is definitely interesting and important to im
79.
Show HN: Bistro – A light-weight column-oriented data processing engine(github.com)
95 points by asavinov 9y ago|29 comments
80.
Unsolved Problems in AI(medium.com)
4 points by asavinov 9y ago|0 comments
81.
by asavinov 9y ago
I finished Data Commander (MVP) - http://conceptoriented.com - A web-app for column-oriented data transformations. After getting some feedback I decided to position this technology differently. Instead of exposing the functional
82.
by asavinov 9y ago
especially if it is about programming languages which evolve for many years
83.
by asavinov 9y ago
I like this idea of representing inner (dot, scalar) product and outer product as one construct with two constituents which behaves like a complex number. This analogy with geometry makes complex numbers more realistic.
84.
by asavinov 9y ago
An index for a column-store and row-store is (conceptually) the same. Why is different is 1. How data is represented. A database is a collection of column objects. For example, it is easy to create a column or delete a column. 2. How data i
85.
by asavinov 9y ago
Here are some other posts explaining the nature of capsule networks, their goals and how they work: - https://medium.com/@pechyonkin/understanding-hintons-capsule... - https://hackernoon.com/what-is-a-c
86.
by asavinov 9y ago
> ... while paying much less attention to the vocal minority's complaints (generics, package system, etc). It is much more difficult not to add a feature than to add it, and the ability to reject even important feature requests is v
87.
by asavinov 9y ago
There is a startup which installs their servers in private homes and other buildings where they are used for heating. They essentially connect the market of server computing and heat. [1] https://cloudandheat.com [2] https:&#x2
88.
by asavinov 9y ago
I would say that it is a two-directional dependency: language constructs <--> design patterns Programmers experiment with and accumulates various design patterns using existing languages. Then the most useful of them are implem
89.
by asavinov 9y ago
> A focus on programming languages in my opinion, masks the real issues we face. Indeed, major problems of programming languages can hardly be solved in the area of programming languages itself as it is being done now. I would say that o
90.
by asavinov 9y ago
> The idea of "programming paradigms" is not a useful way to talk about programming language design. The term "paradigm" is too general indeed and therefore can be quite ambiguous in many situations because it covers

More ›