5 ms·

> Know complexity and scalability: Every time I had to code something, I was asked about the time and space complexity as well as whether I can simply make it m

• maxxxxx 9y ago
  "was being hit like 200 times a month." It's the same when people have 10GB of data and think they need big data scientists.
  • dvt 9y ago
    So true. At my old job, higher-ups insisted we use Spark on a data-set my iPhone could handle. Blows my mind to this day. On a personal note, I loathe having to do interviews (and I also dislike interviewing people), but if none of my projects take off, I'll have to head back into the maw of that soul-crushing corporate machine all over again.
    • maxxxxx 9y ago
      Same in my compnay. We have a dataset of 1 Terabyte that needs to be queried once a week. IT has talked management into installing a Hadoop cluster for this and we have to pay an analytics team. I told my boss that we can probably run this on a SQL database on a USB drive attached to his laptop without problems.
      • tqkxzugoaupvwqr 9y ago
        Suggest to your boss to do a test. If your off-the-shelf solution performs equally well, axe the cluster and analytics team. Maybe go to him with an estimated cost comparison.
        • maxxxxx 9y ago
          We already have done tests for in-house testing but nobody wants to stick his neck out and tell IT to go away. They have convinced upper management that we are doing "big data" and upper management likes the sound of this. It's the good old problem that when you do something that's "cool" at the moment you get more credibility than using tried and true tech like SQL databases. I'll probably also jump on the Hadoop bandwagon because it's good for the resume and more exposure inside the company. Total waste of money and we lose a lot of flexibility but who gives a f..k these days. Gotta go with the flow.
          • yeukhon 9y ago
            > We already have done tests for in-house testing but nobody wants to stick his neck out and tell IT to go away. So true, but in my experience, once you spread the words, other business owners will jump in like an infectious disease and ask for a ticket to use your system. Now your USB won't work anymore. So people "higher-up" want to bet on "fail big" than "fail short", very counter-intuitive, but that's exactly how some enterprise IT spend their budget.
          • murukesh_s 9y ago
            I would advice you to jump into Hadoop and also spark. It's a bloody mean world out there who wouldn't care what you have done or how you have done but just wanted to know if you can configure the most popular tool in the market.. you may explain them you didn't get a chance etc, but most of the time interviewers are like your current management.
    • alfiedotwtf 9y ago
      > At my old job, higher-ups insisted we use Spark... Saw the same thing, but with Kubernetes. They spent so much making the platform infinitely scalable, that they ran out of budget to build the actual product that was supposed to sit on top of that infrastructure (╯°□°）╯︵ ┻━┻
      • dvt 9y ago
        What's really screwed up is that several people that made that god-awful decision have now moved on to different companies (with pay raises I'm sure). So not only did they make an objectively terrible decision, but they also left the mess to whomever it ended up falling in the lap of, and not only that, but they got rewarded for it, too! Corporate software engineering is insanity.
        • maxxxxx 9y ago
          Always jump on the cool stuff even if it makes no sense for the business. In most places as SW engineer you are not rewarded for creating business value or saving money. You are being rewarded for knowing technologies.
          • mrep 9y ago
            For lower positions/companies, I would probably agree. If you want to make a lot of stable liquid money (400,000$+ for principal engineers at big companies and 7 figures for the top individual contributors), then you need to join a big company and deliver high value results consistently to get those offers/promotions (probably a little luck too).
            • lovich 9y ago
              True, but if you're not one of those principle engineers already then you likely have no ability influence what tech choices are being made and you should just learn the new tech so you can jump ship more easily in the future
    • itronitron 9y ago
      I hope they also set up a Raspberry Pi cluster on which to run Spark.
• yeukhon 9y ago
  This is “grill you on everything” kind of interview. Open discussion on system design is IMO a really cruical round. I am not saying the candidates have to have years of experience in scaling - I doubt most applicants have. FWIW, the application I built for my CS department when I was an undergraduate was getting at least 10-20 times of that amount at peak every week (probably hundreds per night) as assignments were due. We were fine on a single server running three vagrants - one for main backend, two for grading assignments in a sandbox. Shipping MVP is really crucial for a startup. Don’t build a mega-internet scale setup. Yes you can start using AWS and takes advantage of the basics scaling features, but there you go - talk about how to leverage those services to make scaling less a hurtle in the long round. That’s what I want to hear as an interviewer. I want to hear about trade-offs and ramp-up. Picking the grading application I worked on, as the candidate I would want to know a high-level view of the stack, and try to identify and address shortcomings with the interviewers together. System design round should feel like a collaboration, whiteboard “pair-sketching” (analogus to pair-programming”. If that was not your experience, sorry. Anyway, point being when there is a take-off, I want to be sure my coworkers and I have good intuitions and ready-to-use knowledge to keep our product afloat. More importantly, we want to start thinking about scaling, availability, fault tolerance and all the big words now. We don’t need a manager walk by and ask “so what should we do today team?” Everyone should be proactive. That should be the point of a system design round. Much of that round shoukd be spent on questioning and then answering back and forth. The interviewer SHOULD be talking too as much as the candidate; it should not feel like giving a talk or giving an oral read of your PhD thesis. If most of your interviewers onsite behave like an exam proctor rather than a peer, consider the culture a bit unfit for yourself (take offer now and do another hunt is up to you). This is IMO the best “culture fit” test both sides can use. Don’t judge on the elegancy of the solution (big plus for clever soltion), but reward points for how pleasant it is to be in a room working together.
  • lovich 9y ago
    That would be wonderful is interviews were actually done with the idea of finding out if a prospective employee was proactive or not, but most interviews are just the interviewers going off a checklist of questions they think they need to ask because they are effectively in a cargo cult. Take the questions on scalability for instance. I personally don't have any experience with scaling on requests, my highest demand was an application with <5000 users for the entire client base. I want the experience so in my last job hunt I apply to a number of jobs that I know get a lot of traffic. I tell the recruiter the chance to get experience in that skill set is why I am applying. The recruiter tells me the desire to learn is all they want. I get to the interviews and have the same talk with the engineers and the same response. They then ask me system design questions and I answer as best as I can along with bringing up approaches I've heard of but don't have experience with. Interview ends and the recruiter tells me they are passing because I don't have the experience in scaling they want. I had this happen with multiple companies. Not wanting an employee without that experience is perfectly fine, but why waste everyone's time when an interviewee tells you from the get go that they don't have one of your must haves? When resumes are passed over for implicit signals like which school you're from, unusual names, spelling errors, etc, why go through all of it? It's because most companies and most people interviewing have no idea what they want or what they are doing and are just copying approaches they've seen before. It's the same problem that gets you interviews where they want you to reverse red black trees on a whiteboard but the actual job is just implementing CRUD api's, or building jobs that run an algorithm that is basic algebra once a day. At this point a "grill you on everything" interview for any position that's not director level or someone getting paid double the market, because they actually need you to know everything, is just a red flag that the company has no idea what's going on and you'll have better luck making the interviewers like you as a person than actually learning all those skills
• Agathos 9y ago
  After weeks of stamping out CRUD features to order, conducting an interview might be their only chance to hang out and talk about high performance computing for once.
  • amorphid 9y ago
    Or maybe it takes almost an hour to service each of their 200 requests/month, and they're desperate to get their response into the sub 30 minute range :)
• icedchai 9y ago
  Maybe they're planning for growth? Many startups are quite optimistic about their future prospects. Reality is often quite different.