3 ms·

> make a ZFS or BCacheFS pool with 20-30% redundancy bits and just go wild with CLI programs Lol. Data management is about safety, auditablity, access control,

> make a ZFS or BCacheFS pool with 20-30% redundancy bits and just go wild with CLI programs Lol. Data management is about safety, auditablity, access control, knowledge sharing and who bunch of other stuff. I would've immediately shown you the door as someone who i cannot trust data with.

zaphar 2y ago
What about his answer prevents any of that? As stated the question didn't require any of what you outline here. ZFS will probably do a better job of protecting your data than almost any other filesystem out there so it's not a bad foundation to start with if you want to protect data. Your entire post reeks of "I'm smarter than you" smugness while at the same time revealing no useful information or approaches. Near as I can tell no one should trust you with any data.
- apwell23 2y ago
  > Your entire post reeks of "I'm smarter than you" unlike "blows my mind" ? > As stated the question didn't require any of what you outline here. Right. OP mentioned it was "tricky question" . What makes it tricky is that all those attributes are implicitly assumed. I wouldn't interview at google and tell them my "stack" is "load it on your laptop". I would never say that in an interview even if I think that's the right "stack" .
  - zaphar 2y ago
    "blows my mind" is similar in tone yes. But I wasn't replying to the OP. Further the OP actually goes into some detail about how he would approach the problem. You do not. You are assuming you know what the OP meant by tricky question. And your assumption contradicts the rest of the OP's post regarding what he considered good answers to the question and why.
    - apwell23 2y ago
      [dead]
    - pdimitar 2y ago
      Honest question: was "blows my mind" so offensive? Thought it was quite obvious I meant that "it blows my mind people don't try the simpler stuff first, especially having in mind that it works for much bigger percentage than cloud providers would have you believe"? I guess it wasn't but even if so, it would be legitimately baffling how people manage to project so much negativity in three words that are slightly tongue-in-cheek casual comment on the state of affairs in an area whose value is not always clear (in my observations, only after you start having 20+ data sources it starts to pay off to have dedicated data team; I've been in teams only 3-4 devs and we still managed to have 15-ish data dashboards for the executives without too much cursing). An anecdote, surely, but what isn't?
      - zaphar 2y ago
        I generally don't find that sort of thing offensive when combined with useful alternative approaches like your post provided. However the phrase does come with a connotation that you are surprised by a lack of knowledge or skill in others. That can be taken as smug or elitist by someone in the wrong frame of mind.
        pdimitar 2y ago
        Thank you, that's helpful.
pdimitar 2y ago
I already qualified my statement quite well by stating my background but if it makes you feel better then sure, show me the door. :) I was never a data scientist, just a guy who helped whenever it was necessary.
- apwell23 2y ago
  > I already qualified my statement quite well by stating my background No. You qualified it with "blows my mind" . Why would it 'blow your mind' if you don't have any data background.
  - zaphar 2y ago
    He didn't say he didn't have any data background. He's clearly worked with data on several occasions as needed.
  - pdimitar 2y ago
    Are you trolling? Did you miss the part where I said I worked with data but wouldn't say I'm a professional data scientist? This negative cherry picking does not do your image any favors.
koverstreet 2y ago
this is how you know when someone takes themself too seriously buddy, you're just rolling off buzzwords and lording it over other people
- apwell23 2y ago
  buddy you suffer from NIH syndrome upset that no one wants your 'hacks'.
photonthug 2y ago
> Lol. Data management is about safety, auditablity, access control, knowledge sharing and who bunch of other stuff. I would've immediately shown you the door as someone who i cannot trust data with. No need to act smug and superior, especially since nothing about OP's plan here actually precludes having all the nice things you mentioned, or even having them inside $your_favorite_enterprise_environment. You risk coming across as a person who feels threatened by simple solutions, perhaps someone who wants to spend $500k in vendor subscriptions every year for simple and/or imaginary problems... exactly the type of thing TFA talks about. But I'll ask the question.. why do you think safety, auditablity, access control, and knowledge sharing are incompatible with CLI tools and a specific choice of file system? What's your preferred alternative? Are you sticking with that alternative regardless of how often the work load runs, how often it changes, and whether the data fits in memory or requires a cluster?
- apwell23 2y ago
  > No need to act smug and superior I responded with the same tone that gp responded with. "blows my mind" ( that people can be so stupid) .
  - photonthug 2y ago
    Another comment mentions this classic meme: > Consulting service: you bring your big data problems to me, I say "your data set fits in RAM", you pay me $10,000 for saving you $500,000. A lot of industry work really does fall into this category, and it's not controversial to say that going the wrong way on this thing is mind-blowing. More than not being controversial, it's not confrontational, because his comment was essentially re: the industry, whereas your comment is directed at a person. Drive by sniping where it's obvious you don't even care to debate the tech itself might get you a few "sick burn, bro" back-slaps from certain crowds, or the FUD approach might get traction with some in management, but overall it's not worth it. You don't sound smart or even professional, just nervous and afraid of every approach that you're not already intimately familiar with.
    - apwell23 2y ago
apwell23 2y ago
Edit: for above comment. My comment wasn't directed at parent. I was trying to be smart and write an inverse of original comment. Opposite scenario Where I as an interviewer was looking for a proper 'data stack' and interviewee responded with a bespoke solution. "not understanding the scale of "real" big data was a no-go in my eyes when hiring." i was trying to point out that you can never know where the interviewer is coming from. Unless i know interviewer personally i would bias towards playing it safe and go with 'enterpisey stack'

zaphar 2y ago

What about his answer prevents any of that? As stated the question didn't require any of what you outline here. ZFS will probably do a better job of protecting your data than almost any other filesystem out there so it's not a bad foundation to start with if you want to protect data. Your entire post reeks of "I'm smarter than you" smugness while at the same time revealing no useful information or approaches. Near as I can tell no one should trust you with any data.

apwell23 2y ago
> Your entire post reeks of "I'm smarter than you" unlike "blows my mind" ? > As stated the question didn't require any of what you outline here. Right. OP mentioned it was "tricky question" . What makes it tricky is that all those attributes are implicitly assumed. I wouldn't interview at google and tell them my "stack" is "load it on your laptop". I would never say that in an interview even if I think that's the right "stack" .
- zaphar 2y ago
  "blows my mind" is similar in tone yes. But I wasn't replying to the OP. Further the OP actually goes into some detail about how he would approach the problem. You do not. You are assuming you know what the OP meant by tricky question. And your assumption contradicts the rest of the OP's post regarding what he considered good answers to the question and why.
  - apwell23 2y ago
    [dead]
  - pdimitar 2y ago
    Honest question: was "blows my mind" so offensive? Thought it was quite obvious I meant that "it blows my mind people don't try the simpler stuff first, especially having in mind that it works for much bigger percentage than cloud providers would have you believe"? I guess it wasn't but even if so, it would be legitimately baffling how people manage to project so much negativity in three words that are slightly tongue-in-cheek casual comment on the state of affairs in an area whose value is not always clear (in my observations, only after you start having 20+ data sources it starts to pay off to have dedicated data team; I've been in teams only 3-4 devs and we still managed to have 15-ish data dashboards for the executives without too much cursing). An anecdote, surely, but what isn't?
    - zaphar 2y ago
      I generally don't find that sort of thing offensive when combined with useful alternative approaches like your post provided. However the phrase does come with a connotation that you are surprised by a lack of knowledge or skill in others. That can be taken as smug or elitist by someone in the wrong frame of mind.
      - pdimitar 2y ago
        Thank you, that's helpful.

pdimitar 2y ago

I already qualified my statement quite well by stating my background but if it makes you feel better then sure, show me the door. :) I was never a data scientist, just a guy who helped whenever it was necessary.

apwell23 2y ago
> I already qualified my statement quite well by stating my background No. You qualified it with "blows my mind" . Why would it 'blow your mind' if you don't have any data background.
- zaphar 2y ago
  He didn't say he didn't have any data background. He's clearly worked with data on several occasions as needed.
- pdimitar 2y ago
  Are you trolling? Did you miss the part where I said I worked with data but wouldn't say I'm a professional data scientist? This negative cherry picking does not do your image any favors.

koverstreet 2y ago

this is how you know when someone takes themself too seriously buddy, you're just rolling off buzzwords and lording it over other people

apwell23 2y ago
buddy you suffer from NIH syndrome upset that no one wants your 'hacks'.

photonthug 2y ago

> Lol. Data management is about safety, auditablity, access control, knowledge sharing and who bunch of other stuff. I would've immediately shown you the door as someone who i cannot trust data with. No need to act smug and superior, especially since nothing about OP's plan here actually precludes having all the nice things you mentioned, or even having them inside $your_favorite_enterprise_environment. You risk coming across as a person who feels threatened by simple solutions, perhaps someone who wants to spend $500k in vendor subscriptions every year for simple and/or imaginary problems... exactly the type of thing TFA talks about. But I'll ask the question.. why do you think safety, auditablity, access control, and knowledge sharing are incompatible with CLI tools and a specific choice of file system? What's your preferred alternative? Are you sticking with that alternative regardless of how often the work load runs, how often it changes, and whether the data fits in memory or requires a cluster?

apwell23 2y ago
> No need to act smug and superior I responded with the same tone that gp responded with. "blows my mind" ( that people can be so stupid) .
- photonthug 2y ago
  Another comment mentions this classic meme: > Consulting service: you bring your big data problems to me, I say "your data set fits in RAM", you pay me $10,000 for saving you $500,000. A lot of industry work really does fall into this category, and it's not controversial to say that going the wrong way on this thing is mind-blowing. More than not being controversial, it's not confrontational, because his comment was essentially re: the industry, whereas your comment is directed at a person. Drive by sniping where it's obvious you don't even care to debate the tech itself might get you a few "sick burn, bro" back-slaps from certain crowds, or the FUD approach might get traction with some in management, but overall it's not worth it. You don't sound smart or even professional, just nervous and afraid of every approach that you're not already intimately familiar with.
  - apwell23 2y ago

apwell23 2y ago

Edit: for above comment. My comment wasn't directed at parent. I was trying to be smart and write an inverse of original comment. Opposite scenario Where I as an interviewer was looking for a proper 'data stack' and interviewee responded with a bespoke solution. "not understanding the scale of "real" big data was a no-go in my eyes when hiring." i was trying to point out that you can never know where the interviewer is coming from. Unless i know interviewer personally i would bias towards playing it safe and go with 'enterpisey stack'