4 ms·

This is super! Best of luck ! BILLING PROFILE: bp7cc3b60e5ce167e99b1ce77

by dwaiba 2y ago

oblivus_cs 2y ago
Hello dwaiba, the necessary credits have successfully been allocated to your account. We’d love to hear your feedback—share your thoughts and you can earn another $15 in credits!
- dwaiba 2y ago
  Thanks. Looks good but do the GPUs like "1x H100 80GB NVLINK" support Infiniband?. Are these HGX Modules? (Since standard HGX comes in pairs of 4 or 8). Also, when will H200 be available? Great Stuff !
  - oblivuslimited 2y ago
    Thanks for the feedback! NVLink configurations are available only in 8x, it’s not possible to deploy 1x. These are PCIe modules connected via NVLink bridges, and they are not HGX. We do plan to expand our fleet with B200s, but H200s are not currently on the roadmap for on-demand. We’ve just activated the rest of your credits—thank you again for your feedback!
    - dwaiba 2y ago
      1. I tried with L40 for inference in MON1 with BentoML [@bentoml.service(resources={"gpu": 1}) class MyService: def __init__(self): import torch self.model = torch.load('model.pth').to('cuda:0')] and 2. RTX A4000 for training using pytorch training script [ device = "cuda" if torch.cuda.is_available() else "cpu" device] in OSL1 with sshkeys for root. Old CUDA 12.2 is provided with Mainline Ubuntu LTS 22.04 LTS as Latest Ubuntu. It is unstable (mostly) with driver upgrades as Device drops out though works intermittently !!! Start Scripts for pyenv and CUDA: #!/bin/bash DEBIAN_FRONTEND=noninteractive apt-get -y install wget zip unzip git jq DEBIAN_FRONTEND=noninteractive apt-get install -y python3-pip # Required for pyenv make DEBIAN_FRONTEND=noninteractive apt-get install -y build-essential zlib1g-dev libffi-dev libssl-dev libbz2-dev libreadline-dev libsqlite3-dev liblzma-dev libncurses-dev tk-dev DEBIAN_FRONTEND=noninteractive apt install -y --reinstall gcc-12 ln -s -f /usr/bin/gcc-12 /usr/bin/gcc # Remove existing nvidia drivers and install cuda 12.6.1 DEBIAN_FRONTEND=noninteractive apt-get -y remove --purge *nvidia* DEBIAN_FRONTEND=noninteractive apt-get -y remove --purge *cuda* DEBIAN_FRONTEND=noninteractive apt-get -y remove --purge *nvrtc* DEBIAN_FRONTEND=noninteractive apt-get -y autoremove --purge wget https://developer.download.nvidia.com/compute/cuda/12.6.1/local_installers/cuda_12.6.1_560.35.03_linux.run https://developer.download.nvidia.com/compute/cuda/12.6.1/lo... && sh cuda_12.6.1_560.35.03_linux.run --silent --override DEBIAN_FRONTEND=noninteractive apt-get -y update DEBIAN_FRONTEND=noninteractive apt-get -y upgrade DEBIAN_FRONTEND=noninteractive reboot
      - oblivuslimited 2y ago
        Thank you for the feedback! We’ve actually just updated the default Ubuntu OS to 550.90.12 and CUDA to 12.4 today. Would you mind checking it again?

This is super! Best of luck ! BILLING PROFILE: bp7cc3b60e5ce167e99b1ce77

by dwaiba 2y ago

This is super! Best of luck ! BILLING PROFILE: bp7cc3b60e5ce167e99b1ce77

Hello dwaiba, the necessary credits have successfully been allocated to your account. We’d love to hear your feedback—share your thoughts and you can earn another $15 in credits!

dwaiba 2y ago
Thanks. Looks good but do the GPUs like "1x H100 80GB NVLINK" support Infiniband?. Are these HGX Modules? (Since standard HGX comes in pairs of 4 or 8). Also, when will H200 be available? Great Stuff !
- oblivuslimited 2y ago
  Thanks for the feedback! NVLink configurations are available only in 8x, it’s not possible to deploy 1x. These are PCIe modules connected via NVLink bridges, and they are not HGX. We do plan to expand our fleet with B200s, but H200s are not currently on the roadmap for on-demand. We’ve just activated the rest of your credits—thank you again for your feedback!
  - dwaiba 2y ago
    1. I tried with L40 for inference in MON1 with BentoML [@bentoml.service(resources={"gpu": 1}) class MyService: def __init__(self): import torch self.model = torch.load('model.pth').to('cuda:0')] and 2. RTX A4000 for training using pytorch training script [ device = "cuda" if torch.cuda.is_available() else "cpu" device] in OSL1 with sshkeys for root. Old CUDA 12.2 is provided with Mainline Ubuntu LTS 22.04 LTS as Latest Ubuntu. It is unstable (mostly) with driver upgrades as Device drops out though works intermittently !!! Start Scripts for pyenv and CUDA: #!/bin/bash DEBIAN_FRONTEND=noninteractive apt-get -y install wget zip unzip git jq DEBIAN_FRONTEND=noninteractive apt-get install -y python3-pip # Required for pyenv make DEBIAN_FRONTEND=noninteractive apt-get install -y build-essential zlib1g-dev libffi-dev libssl-dev libbz2-dev libreadline-dev libsqlite3-dev liblzma-dev libncurses-dev tk-dev DEBIAN_FRONTEND=noninteractive apt install -y --reinstall gcc-12 ln -s -f /usr/bin/gcc-12 /usr/bin/gcc # Remove existing nvidia drivers and install cuda 12.6.1 DEBIAN_FRONTEND=noninteractive apt-get -y remove --purge *nvidia* DEBIAN_FRONTEND=noninteractive apt-get -y remove --purge *cuda* DEBIAN_FRONTEND=noninteractive apt-get -y remove --purge *nvrtc* DEBIAN_FRONTEND=noninteractive apt-get -y autoremove --purge wget https://developer.download.nvidia.com/compute/cuda/12.6.1/local_installers/cuda_12.6.1_560.35.03_linux.run https://developer.download.nvidia.com/compute/cuda/12.6.1/lo... && sh cuda_12.6.1_560.35.03_linux.run --silent --override DEBIAN_FRONTEND=noninteractive apt-get -y update DEBIAN_FRONTEND=noninteractive apt-get -y upgrade DEBIAN_FRONTEND=noninteractive reboot
    - oblivuslimited 2y ago
      Thank you for the feedback! We’ve actually just updated the default Ubuntu OS to 550.90.12 and CUDA to 12.4 today. Would you mind checking it again?