4 ms·

One of my pet ideas is a new programmer's editor that uses proportionally as much compute resources as Emacs did in the late-80s. An amount of computing that wo

by tlb 12y ago

One of my pet ideas is a new programmer's editor that uses proportionally as much compute resources as Emacs did in the late-80s. An amount of computing that would show up as a major line item on your department's timesharing bill. 1000 cores costs around $50/hr, so if it boosted my productivity measurably it'd be worth it. Searching all public github code for strings matching around my cursor seems like the sort of thing these 1000 cores should be doing.

tdicola 12y ago
Interesting idea, would definitely make you increase the efficiency of your time at the editor if it costs you $50/hour! But you don't need 1000 cores to fire off a bunch of search requests. Network calls are generally IO bound so even one or two cores can handle lots of requests.
- andrewstuart2 12y ago
  > all public github code In RAM.
  - dbarlett 12y ago
    Some napkin math - as of late 2013, GitHub's main Elasticsearch cluster had ~128 shards of ~120 GB [1]. Obviously they have more data now, and it's not clear whether the number of shards includes replicas, but at current EC2 on-demand prices[2]: RAM: 63 r3.8xlarge @ $2.80/hr = $176.40/hr SSD: 5 i2.4xlarge @ $3.41/hr = $ 17.05/hr HDD: 1 hs1.8xlarge @ $4.60/hr = $ 4.60/hr [1] http://www.elasticsearch.org/case-study/github/ http://www.elasticsearch.org/case-study/github/ [2] http://aws.amazon.com/ec2/pricing/ http://aws.amazon.com/ec2/pricing/
    - yohanatan 12y ago
      A hybrid multi-layer cache system would likely be more cost effective and still perform nearly as well as dedicated RAM (depending on how the data is structured/sharded/queried).
    - X-Istence 12y ago
      We don't even need to keep ALL of it in memory, we would just need to keep everything for the current language we are developing in. Maybe one or two others. Say I am working on a Angular/Pyramid application, all I would need is JavaScript and Python loaded.
ThisIBereave 12y ago
It should be noted that public github repos are _not_ public domain and not even necessarily open source unless the author specifies a license.
- timr 12y ago
  It's part of the GitHub terms of service: "By setting your repositories to be viewed publicly, you agree to allow others to view and fork your repositories."
  - rtb 12y ago
    Permission to "view and fork" is not the same as permission to copy and paste into a proprietary software product (i.e. this may not be legit for use by "enterprise" developers)
yohanatan 12y ago
I have a similar idea where the cores are put to work optimizing the produced binaries (profile-guided optimization, genetic algorithms, SAT solvers/theorem proving, etc). You could of course also do static analysis and all forms of testing (generative, property-based, unit, integration, performance, etc.).
herdrick 12y ago
http://www.engadget.com/2014/11/09/darpa-pliny-coding/ http://www.engadget.com/2014/11/09/darpa-pliny-coding/

by tlb 12y ago

tdicola 12y ago

Interesting idea, would definitely make you increase the efficiency of your time at the editor if it costs you $50/hour! But you don't need 1000 cores to fire off a bunch of search requests. Network calls are generally IO bound so even one or two cores can handle lots of requests.

andrewstuart2 12y ago
> all public github code In RAM.
- dbarlett 12y ago
  Some napkin math - as of late 2013, GitHub's main Elasticsearch cluster had ~128 shards of ~120 GB [1]. Obviously they have more data now, and it's not clear whether the number of shards includes replicas, but at current EC2 on-demand prices[2]: RAM: 63 r3.8xlarge @ $2.80/hr = $176.40/hr SSD: 5 i2.4xlarge @ $3.41/hr = $ 17.05/hr HDD: 1 hs1.8xlarge @ $4.60/hr = $ 4.60/hr [1] http://www.elasticsearch.org/case-study/github/ http://www.elasticsearch.org/case-study/github/ [2] http://aws.amazon.com/ec2/pricing/ http://aws.amazon.com/ec2/pricing/
  - yohanatan 12y ago
    A hybrid multi-layer cache system would likely be more cost effective and still perform nearly as well as dedicated RAM (depending on how the data is structured/sharded/queried).
  - X-Istence 12y ago
    We don't even need to keep ALL of it in memory, we would just need to keep everything for the current language we are developing in. Maybe one or two others. Say I am working on a Angular/Pyramid application, all I would need is JavaScript and Python loaded.

ThisIBereave 12y ago

It should be noted that public github repos are _not_ public domain and not even necessarily open source unless the author specifies a license.

timr 12y ago
It's part of the GitHub terms of service: "By setting your repositories to be viewed publicly, you agree to allow others to view and fork your repositories."
- rtb 12y ago
  Permission to "view and fork" is not the same as permission to copy and paste into a proprietary software product (i.e. this may not be legit for use by "enterprise" developers)

yohanatan 12y ago

I have a similar idea where the cores are put to work optimizing the produced binaries (profile-guided optimization, genetic algorithms, SAT solvers/theorem proving, etc). You could of course also do static analysis and all forms of testing (generative, property-based, unit, integration, performance, etc.).

herdrick 12y ago

http://www.engadget.com/2014/11/09/darpa-pliny-coding/ http://www.engadget.com/2014/11/09/darpa-pliny-coding/