3 ms·

> 10GB is hardly enough. Once you import Pandas you’re on the limit. You can forget Pandas and scipy at the same Lambda. This sounds way off to me. 10 GB to in

> 10GB is hardly enough. Once you import Pandas you’re on the limit. You can forget Pandas and scipy at the same Lambda. This sounds way off to me. 10 GB to install a Python library?

sgarland 3y ago
I also thought this seemed extremely odd. Here is an unoptimized example, built on an M1 Mac: $ cat <<EOF > Dockerfile FROM python:3.12-slim-bookworm RUN apt-get update && apt-get install -y python3-pip RUN pip install pandas LABEL "name"="python-pandas" ENTRYPOINT ["python3"] EOF $ docker image ls -f 'label'='name'='python-pandas' REPOSITORY TAG IMAGE ID CREATED SIZE sgarland/python-pandas latest 89e31f6eb83d 9 minutes ago 764MB A more optimized version: $ cat <<EOF > Dockerfile FROM python:3.12-slim-bookworm RUN apt-get update && \ apt-get install -y --no-install-recommends python3-pip && \ pip install pandas && \ apt-get purge -y --autoremove python3-pip && \ rm -rf /var/lib/apt/lists/* LABEL "name"="python-pandas" ENTRYPOINT ["python3"] $ docker image ls -f 'label'='name'='python-pandas' REPOSITORY TAG IMAGE ID CREATED SIZE sgarland/python-pandas smaller 102308842b88 4 seconds ago 342MB sgarland/python-pandas latest 89e31f6eb83d 27 minutes ago 764MB Even adding in scipy didn't crack 500 MB: $ docker image ls -f 'label'='name'='python-pandas' REPOSITORY TAG IMAGE ID CREATED SIZE sgarland/python-pandas scipy 808535284f03 3 minutes ago 497MB sgarland/python-pandas smaller 102308842b88 9 minutes ago 342MB sgarland/python-pandas latest 89e31f6eb83d 36 minutes ago 764MB I'm not sure how they managed 10 GB. Here's the non-slim version, with no optimizations (this is much larger because `python3-pip` has the system default Python interpreter as as dependency, so this installs Python3.11 into the image): $ cat <<EOF > Dockerfile FROM python:3.12-bookworm RUN apt-get update RUN apt-get install -y python3-pip RUN pip install pandas scipy LABEL "name"="python-pandas" ENTRYPOINT ["python3"] EOF $ docker image ls -f 'label'='name'='python-pandas' REPOSITORY TAG IMAGE ID CREATED SIZE sgarland/python-pandas bigger f8f98e9a241c 8 seconds ago 1.44GB sgarland/python-pandas scipy 808535284f03 3 minutes ago 497MB sgarland/python-pandas smaller 102308842b88 9 minutes ago 342MB sgarland/python-pandas latest 89e31f6eb83d 36 minutes ago 764MB

> 10GB is hardly enough. Once you import Pandas you’re on the limit. You can forget Pandas and scipy at the same Lambda. This sounds way off to me. 10 GB to install a Python library?

sgarland 3y ago

I also thought this seemed extremely odd. Here is an unoptimized example, built on an M1 Mac: $ cat <<EOF > Dockerfile FROM python:3.12-slim-bookworm RUN apt-get update && apt-get install -y python3-pip RUN pip install pandas LABEL "name"="python-pandas" ENTRYPOINT ["python3"] EOF $ docker image ls -f 'label'='name'='python-pandas' REPOSITORY TAG IMAGE ID CREATED SIZE sgarland/python-pandas latest 89e31f6eb83d 9 minutes ago 764MB A more optimized version: $ cat <<EOF > Dockerfile FROM python:3.12-slim-bookworm RUN apt-get update && \ apt-get install -y --no-install-recommends python3-pip && \ pip install pandas && \ apt-get purge -y --autoremove python3-pip && \ rm -rf /var/lib/apt/lists/* LABEL "name"="python-pandas" ENTRYPOINT ["python3"] $ docker image ls -f 'label'='name'='python-pandas' REPOSITORY TAG IMAGE ID CREATED SIZE sgarland/python-pandas smaller 102308842b88 4 seconds ago 342MB sgarland/python-pandas latest 89e31f6eb83d 27 minutes ago 764MB Even adding in scipy didn't crack 500 MB: $ docker image ls -f 'label'='name'='python-pandas' REPOSITORY TAG IMAGE ID CREATED SIZE sgarland/python-pandas scipy 808535284f03 3 minutes ago 497MB sgarland/python-pandas smaller 102308842b88 9 minutes ago 342MB sgarland/python-pandas latest 89e31f6eb83d 36 minutes ago 764MB I'm not sure how they managed 10 GB. Here's the non-slim version, with no optimizations (this is much larger because `python3-pip` has the system default Python interpreter as as dependency, so this installs Python3.11 into the image): $ cat <<EOF > Dockerfile FROM python:3.12-bookworm RUN apt-get update RUN apt-get install -y python3-pip RUN pip install pandas scipy LABEL "name"="python-pandas" ENTRYPOINT ["python3"] EOF $ docker image ls -f 'label'='name'='python-pandas' REPOSITORY TAG IMAGE ID CREATED SIZE sgarland/python-pandas bigger f8f98e9a241c 8 seconds ago 1.44GB sgarland/python-pandas scipy 808535284f03 3 minutes ago 497MB sgarland/python-pandas smaller 102308842b88 9 minutes ago 342MB sgarland/python-pandas latest 89e31f6eb83d 36 minutes ago 764MB