5 ms·

Wouldn't "Serverless OCR" mean something like running tesseract locally on your computer, rather than creating an AI framework and running it on a server?

cachius 8mo ago
Serverless means spinning compute resources up on demand in the cloud vs. running a server permanently.
- dsr_ 8mo ago
  ~99.995% of the computing resources used on this are from somebody else's servers, running the LLM model.
- locknitpicker 8mo ago
  > Serverless means spinning compute resources up on demand in the cloud vs. running a server permanently. Not quite. Serverless means you can run a server permanently, but you need pay someone else to manage the infrastructure for you.
  - turtlebits 8mo ago
    Close. It means there's no persistent infra charges and you're charged on use. You dont run anything permanently.
    - dvfjsdhgfv 8mo ago
      It still doesn't capture the concept because, say, both AWS Lambda and EC2 can be run just for 5 minutes and only one of them is called serverless.
      - Stefan-H 8mo ago
        Unless the engineer takes steps to spin down EC2 infrastructure after execution, it is absolutely persistent compute that you're billed for whether you are doing actual processing or not. Whereas lambda and other services are billed only for execution time.
  - jwiz 8mo ago
    Depends if you mean "server" as in piece of metal (or vm), or as in "a daemon"
  - Stefan-H 8mo ago
    You might be conflating "cloud" with serverless. Serverless is where developers can focus on code, with little care of the infrastructure it runs on, and is pay-as-you-go.
    - locknitpicker 8mo ago
      > You might be conflating "cloud" with serverless. Serverless is where developers can focus on code, with little care of the infrastructure it runs on, and is pay-as-you-go. That's not what serverless means at all. Most function-as-a-service offerings require developers to bother about infrastructure aspects, such as runtimes and even underlying OS. They just don't bother about managing it. They deploy their code on their choice of infrastructure, and go on with their lives.
      - Stefan-H 8mo ago
        A runtime is notably NOT infrastructure, had you said instruction set you might have landed closer to making a compelling argument, but the whole point is that AWS (and other providers) abstract away the underlying infrastructure and allow the developers to as I said, have "little care of the infrastructure it runs on". There is often advanced networking that CAN be configured, as well as other infrastructure components developers can choose to configure.
normie3000 8mo ago
Thanks for noting this - for a moment I was excited.
- mrweasel 8mo ago
  When people mentions the number of lines of code, I've started to become suspicious. More often than not it's X number of lines, calling a massive library loading a large model, either locally or remote. We're just waiting for spinning up your entire company infrastructure in two lines of code, and then just being presented a Terraform shell script wrapper. I do agree with the use of serverless though. I feel like we agree long ago that serverless just means that you're not spinning up a physical or virtual server, but simply ask some cloud infrastructure to run your code, without having to care about how it's run.
  - goodmythical 8mo ago
    >implement RSA with this one simple line of python!
  - locknitpicker 8mo ago
    > When people mentions the number of lines of code, I've started to become suspicious. Low LoC count is a telltale sign that the project adds little to no value. It's a claim that the project integrates third party services and/or modules, and does a little plumbing to tie things together.
- xml 8mo ago
  You can still be excited! Recently, GLM-OCR was released, which is a relatively small OCR model (2.5 GB unquantized) that can run on CPU with good quality. I've been using it to digitize various hand-written notes and all my shopping receipts this week. https://github.com/zai-org/GLM-OCR https://github.com/zai-org/GLM-OCR (Shameless plug: I also maintain a simplified version of GLM-OCR without dependency on the transformers library, which makes it much easier to install: https://github.com/99991/Simple-GLM-OCR/ https://github.com/99991/Simple-GLM-OCR/)
spockz 8mo ago
Running it locally would typically be called “client(-)side”. But this caught me for a bit as well. :-)
- ahartmetz 8mo ago
  That's the beauty of such stupid terms. I use carless transportation (taxis).
  - wolfi1 8mo ago
    taxis are cars, aren't they?
    - BenjiWiebe 8mo ago
      Precisely. And serverless uses servers.
esafak 8mo ago
No, that would be "Running OCR locally..." 'Serverless' has become a term of art: https://en.wikipedia.org/wiki/Serverless_computing https://en.wikipedia.org/wiki/Serverless_computing
- dvfjsdhgfv 8mo ago
  It's good they note explicitly: > Serverless is a misnomer
mindslight 8mo ago
Yep. That fraudulent term finally got me this time. Totally serverless except for that remote 3rd party server. Sigh.

cachius 8mo ago

Serverless means spinning compute resources up on demand in the cloud vs. running a server permanently.

dsr_ 8mo ago
~99.995% of the computing resources used on this are from somebody else's servers, running the LLM model.
locknitpicker 8mo ago
> Serverless means spinning compute resources up on demand in the cloud vs. running a server permanently. Not quite. Serverless means you can run a server permanently, but you need pay someone else to manage the infrastructure for you.
- turtlebits 8mo ago
  Close. It means there's no persistent infra charges and you're charged on use. You dont run anything permanently.
  - dvfjsdhgfv 8mo ago
    It still doesn't capture the concept because, say, both AWS Lambda and EC2 can be run just for 5 minutes and only one of them is called serverless.
    - Stefan-H 8mo ago
      Unless the engineer takes steps to spin down EC2 infrastructure after execution, it is absolutely persistent compute that you're billed for whether you are doing actual processing or not. Whereas lambda and other services are billed only for execution time.
- jwiz 8mo ago
  Depends if you mean "server" as in piece of metal (or vm), or as in "a daemon"
- Stefan-H 8mo ago
  You might be conflating "cloud" with serverless. Serverless is where developers can focus on code, with little care of the infrastructure it runs on, and is pay-as-you-go.
  - locknitpicker 8mo ago
    > You might be conflating "cloud" with serverless. Serverless is where developers can focus on code, with little care of the infrastructure it runs on, and is pay-as-you-go. That's not what serverless means at all. Most function-as-a-service offerings require developers to bother about infrastructure aspects, such as runtimes and even underlying OS. They just don't bother about managing it. They deploy their code on their choice of infrastructure, and go on with their lives.
    - Stefan-H 8mo ago
      A runtime is notably NOT infrastructure, had you said instruction set you might have landed closer to making a compelling argument, but the whole point is that AWS (and other providers) abstract away the underlying infrastructure and allow the developers to as I said, have "little care of the infrastructure it runs on". There is often advanced networking that CAN be configured, as well as other infrastructure components developers can choose to configure.

normie3000 8mo ago

Thanks for noting this - for a moment I was excited.

mrweasel 8mo ago
When people mentions the number of lines of code, I've started to become suspicious. More often than not it's X number of lines, calling a massive library loading a large model, either locally or remote. We're just waiting for spinning up your entire company infrastructure in two lines of code, and then just being presented a Terraform shell script wrapper. I do agree with the use of serverless though. I feel like we agree long ago that serverless just means that you're not spinning up a physical or virtual server, but simply ask some cloud infrastructure to run your code, without having to care about how it's run.
- goodmythical 8mo ago
  >implement RSA with this one simple line of python!
- locknitpicker 8mo ago
  > When people mentions the number of lines of code, I've started to become suspicious. Low LoC count is a telltale sign that the project adds little to no value. It's a claim that the project integrates third party services and/or modules, and does a little plumbing to tie things together.
xml 8mo ago
You can still be excited! Recently, GLM-OCR was released, which is a relatively small OCR model (2.5 GB unquantized) that can run on CPU with good quality. I've been using it to digitize various hand-written notes and all my shopping receipts this week. https://github.com/zai-org/GLM-OCR https://github.com/zai-org/GLM-OCR (Shameless plug: I also maintain a simplified version of GLM-OCR without dependency on the transformers library, which makes it much easier to install: https://github.com/99991/Simple-GLM-OCR/ https://github.com/99991/Simple-GLM-OCR/)

spockz 8mo ago

Running it locally would typically be called “client(-)side”. But this caught me for a bit as well. :-)

ahartmetz 8mo ago
That's the beauty of such stupid terms. I use carless transportation (taxis).
- wolfi1 8mo ago
  taxis are cars, aren't they?
  - BenjiWiebe 8mo ago
    Precisely. And serverless uses servers.

esafak 8mo ago

No, that would be "Running OCR locally..." 'Serverless' has become a term of art: https://en.wikipedia.org/wiki/Serverless_computing https://en.wikipedia.org/wiki/Serverless_computing

dvfjsdhgfv 8mo ago
It's good they note explicitly: > Serverless is a misnomer

mindslight 8mo ago

Yep. That fraudulent term finally got me this time. Totally serverless except for that remote 3rd party server. Sigh.