5 ms·

Not really. They could be trying to build a company-wide data lake. Some files in the lake will be very small, others will be very large. Because it's in HDFS i

Not really. They could be trying to build a company-wide data lake. Some files in the lake will be very small, others will be very large. Because it's in HDFS it won't matter if the very small ones suddenly become very large. End users groan because people groan every time anything changes and they have to learn something new.

ghostly_s 9y ago
If users were previously empowered to do a task themselves, and now need to request another team do that task on their behalf, that's a very valid reason to be groaning IMO.
- wil421 9y ago
  You have to understand that enterprise class companies don’t want Access databases under people desks. It might be valid reason for the sales or ordering guy who loses control to complain. For the IT professionals at the company it allows them to create a scalable solution that can be used at an enterprise level.
  - maxxxxx 9y ago
    And suddenly every addition that could be done in 5 minutes now takes months or gets rejected by the devs.
    - jordanb 9y ago
      Hadoop isn't Oracle. If you need to write a file to the data lake, you write it. If you need to read something out you do that too. There's no schema, it's actually a file system with actual files in it, like your shared drive. Except that it's distributed, can handle arbitrarily large files and lets you perform queries on those files instead of downloading all 2TB to access a single row.
klodolph 9y ago
I mean, end users also groan because the user experience for spreadsheets is far better than writing map reduce jobs. One is a skill that a large percentage of BS graduates have, another is a skill that a small percentage of CS graduates have.
- jordanb 9y ago
  You don't need to write map reduce jobs to access data in Hadoop. In fact, you can do it from excel: https://support.office.com/en-us/article/Connect-to-a-Hadoop-Distributed-File-System-HDFS-Power-Query-8feeb417-e430-4c5b-b136-64b043a00fc2 https://support.office.com/en-us/article/Connect-to-a-Hadoop...

ghostly_s 9y ago

If users were previously empowered to do a task themselves, and now need to request another team do that task on their behalf, that's a very valid reason to be groaning IMO.

wil421 9y ago
You have to understand that enterprise class companies don’t want Access databases under people desks. It might be valid reason for the sales or ordering guy who loses control to complain. For the IT professionals at the company it allows them to create a scalable solution that can be used at an enterprise level.
- maxxxxx 9y ago
  And suddenly every addition that could be done in 5 minutes now takes months or gets rejected by the devs.
  - jordanb 9y ago
    Hadoop isn't Oracle. If you need to write a file to the data lake, you write it. If you need to read something out you do that too. There's no schema, it's actually a file system with actual files in it, like your shared drive. Except that it's distributed, can handle arbitrarily large files and lets you perform queries on those files instead of downloading all 2TB to access a single row.

klodolph 9y ago

I mean, end users also groan because the user experience for spreadsheets is far better than writing map reduce jobs. One is a skill that a large percentage of BS graduates have, another is a skill that a small percentage of CS graduates have.

jordanb 9y ago
You don't need to write map reduce jobs to access data in Hadoop. In fact, you can do it from excel: https://support.office.com/en-us/article/Connect-to-a-Hadoop-Distributed-File-System-HDFS-Power-Query-8feeb417-e430-4c5b-b136-64b043a00fc2 https://support.office.com/en-us/article/Connect-to-a-Hadoop...