14 ms·

Storage on Vercel

267 points by drecoe 3y ago

swyx 3y ago
Vercel is the second company to “eat the cloud from outside in”. (https://www.swyx.io/cloudflare-go https://www.swyx.io/cloudflare-go) its now my defacto playbook for building lasting bottom up disruptive cloud companies. start by giving away an extremely good free tier (cloudflare - cdn, vercel - nextjs+hosting) then add build time compute, run time compute, readonly kv store, and now full read write storage. (this is part of an overall cloud progression that i've been studying for a couple years https://twitter.com/swyx/status/1417136897894326277 https://twitter.com/swyx/status/1417136897894326277) many other platforms try to start off offering everything under the sun, claiming to care about the holistic end to end experience. vercel bet correctly on one wedge, built it up over years, and only now is expanding into storage. well earned.
- youngtaff 3y ago
  Cloudflare own and run the services they’re offering whereas it looks like Vercal storage is just other people’s services rebranded
  - asadm 3y ago
    Users don't care about that.
    - proxy9 3y ago
      I think they care. Vercel is VC money. Not even close to profitability. Anyone building anything but a toy needs to take into consideration that Vercel might not be here in 2 years from now
    - waboremo 3y ago
      Generally, that is very true. But when the bills start hitting, users absolutely do care and will go out of their way to avoid it.
      - asadm 3y ago
        Only when it reaches ridiculous enough number. Even as an individual who can easily do any cloud/k8s/lambdas/firecracker, I am paying Vercel a few hundred for my side projects.
        waboremo 3y ago
        True, but the ridiculous number might be lower than you expect. Vercel in particular has got a lot of feedback about their pricing!
    - the_mitsuhiko 3y ago
      … for as long as there is no cheaper competition.
    - mrkurt 3y ago
      They do, actually. We don't say anything about running our own infrastructure. And every time someone finds out, they're like "why didn't you lead with that?" I'm surprised how much people care about what's under the covers.
      - eropple 3y ago
        I think that in many ways is because Fly's approach is sufficiently high-minded (in other words, aiming most comfortably at relatively difficult problems) that it's probably pretty automatic to assume that it's built on an existing hyperscaler. Mentioning that you run your own infrastructure adds encouragement that you know what you're doing. Given the general state of PaaS offerings, this is a differentiator.
- ghc 3y ago
  Classic VC-backed B2B play. The trick, however, is finding a "wedge" people actually want on its own. Unfortunately that seems to be easier said than done.
  - swyx 3y ago
    well a good start is betting on the most popular framework in the most popular language and building the production ready stuff that is missing that the framework authors constitutionally cannot compete with you on :) there are better wedges financially speaking but theres pretty much no bigger bottom up developer wedge than this one
    - mattwad 3y ago
      They built the framework too, or was it a purchase originally?
      - dabeeeenster 3y ago
        Guillermo the Vercel CEO made Next.js
      - doodlesdev 3y ago
        Vercel (then called Zeit) developed NextJS themselves. I believe swyx meant React when he said "framework", even though one can argue that's technically incorrect it's been commonplace for some time to call it a framework.
  - cccybernetic 3y ago
    There’s a lot of magic with Vercel, but I think the true “wedge” was clear to many people: DX. The trick was really the execution, which they nailed. They realized early that there was a whole class of developers out there, many of them quite talented, who couldn’t stand SSH’ing into Linux boxes, configuring htaccess files, installing SSL certificates, etc. Many of these developers also worked at big companies, and held sway. They distilled the entire deployment process into a few clicks and boom, you have a real, functioning, fast website. Then they added templates, good marketing, and doubled down on React. The rest is history. It’s a well executed vision and I give them a lot of props for that.
- waboremo 3y ago
  I believe this is a core factor as to why Supabase hasn't joined Cloudflare and Vercel, they don't have that "outside" aspect. By all other counts, they are well established to do so and who knows maybe they do have something in the works to tie in their offerings together from the "outside".
  - swyx 3y ago
    (am supabase investor) fortunately supabase's value prop is so straight forward it doesnt quite need to do the "outside-in" maneuver. just make postgres into a developer friendly platform. huge TAM in itself. another investor i asked on why they put $millions into supabase simply said "RDS has more than $1b ARR"
    - waboremo 3y ago
      Great point about being direct, and the priority on DX seems to be a semi-moat in itself, worked for Vercel and Supabase so far!
- vultour 3y ago
  Will they run my workloads without requiring my attention for decades? I'd love to rewrite and move some old PHP sites from my webhost, but that one has been hosting them without requiring my attention for probably close to 15 years. I'm a little skeptical these fancy cloud runtimes will even exist a few years from now.
  - diroussel 3y ago
    Also nodejs is deprecating versions at an impressive rate. Todays LTS will be out of date soon. AWS and thus Vercel have to follow along. So you can’t run a node based serverless app for more than a year or two before you either put the work in to upgrade or just cross your fingers that the forced upgrade won’t break your app.
- c-hendricks 3y ago
  > https://twitter.com/swyx/status/1417136897894326277 https://twitter.com/swyx/status/1417136897894326277 Companies are leasing out the core aspects of computing (storage, networking, CPU, GPU, auth), nothing really insightful here. Just your average day in capitalism. > many other platforms try to start off offering everything under the sun Do they? Or do they also start small, then build up over the years until they have everything under the sun, rinsing and repeating the process.
- throwingrocks 3y ago
  > now my defacto playbook for building lasting bottom up disruptive cloud companies What companies have you built (or played a role in building) using this "playbook" that fit your description?
scsteps 3y ago
Very nice. I really wanted this to happen.
MattIPv4 3y ago
Cloudflare's R2 costs $0.36 per million read operations (after the 10 million you get for free) [1]. Vercel is wrapping R2 and is charging $2 per million reads [2]. They're also charging $0.15/GB for egress (after the 1GB you get free), when R2 charges nothing, and the storage cost is doubled from $0.015/GB on R2 to $0.03/GB on Vercel. That's quite the cost increase for the DX improvement. [1] https://developers.cloudflare.com/r2/pricing/ https://developers.cloudflare.com/r2/pricing/ [2] https://vercel.com/docs/storage/vercel-blob/usage-and-pricing https://vercel.com/docs/storage/vercel-blob/usage-and-pricin...
- doublepg23 3y ago
  Hijacking this comment to ask if anyone has had luck integrating Cloudflare R2 with Pleroma. I’ve not had any luck getting it to work though I’m also not well versed on the terminology.
- leerob 3y ago
  This isn't finalized yet, apologies for the confusion. We'll be updating the pricing for Blob shortly (it's in private beta and invite only). The pricing for KV and Postgres is up to date.
  - dexterdog 3y ago
    I already lean away from S3/Azureblob/etc because their egress prices are terrible. .15/GB is borderline criminal.
    - Demonto 3y ago
      150$ for a TB traffic?! Wtf
      - mrcwinn 3y ago
        What’s happening right now is a story we’ve seen plenty of times before. Overhead cost of convenience mixed with vendor lock-in, all while boasting about open source. I’d love to create tools that were convenient and had fair pricing. The challenge is that you’re trying to grow into a market with worse acquisition economics. Tough to win if winning is the goal. If anyone has a solution reach out. XD
MattIPv4 3y ago
Pricing for them: Blob: https://vercel.com/docs/storage/vercel-blob/usage-and-pricing https://vercel.com/docs/storage/vercel-blob/usage-and-pricin... KV: https://vercel.com/docs/storage/vercel-kv/usage-and-pricing https://vercel.com/docs/storage/vercel-kv/usage-and-pricing Postgres: https://vercel.com/docs/storage/vercel-postgres/usage-and-pricing https://vercel.com/docs/storage/vercel-postgres/usage-and-pr...
- leerob 3y ago
  KV/Postgres are accurate, that Blob page sneaked in. Blob pricing should be figured out shortly, we'll update that page (it's in private beta, invite only currently).
  - dbbk 3y ago
    So take it down?
revskill 3y ago
Cloud to me is overpriced and not transparent. Example, with Upstash redis: i deployed a fly.io connected to upstash. I don't use that service for a month (no query at all), but it still counted in billing. Wait, is upstash real serverless ? I have to pay for NOT using it ?
- epberry 3y ago
  Usage-based pricing in the cloud is a blessing and a curse. To take best advantage of it you need a third party tool. At least, this is the thesis behind my current employer (https://vantage.sh https://vantage.sh).
- mrkurt 3y ago
  Upstash on Fly.io is not serverless. You provision the RAM + connection counts you want, and then pay for that capacity. This is not what all our customers want, but it is much cheaper to run this way for most fullstack frameworks. It's not ideal for a Redis you don't use at all, though. We're considering a serverless / metered billing option for this. But what we have is much closer to spinning up a permanent Redis. :)
  - revskill 3y ago
    The dashboard on fly.io didn't say it's not serverless. SO it's not transparent. Look like a scam to me. For now, i had to say goodbye for most of cloud services. Good old VPS for now.
    - mrkurt 3y ago
      Ok that's a silly thing to say. Nowhere on our site do we say we offer serverless anything. When you provision a Redis, we offer you a selection of plans, you pick the one you want, and off you go. I don't understand why you'd assume that's serverless.
      - revskill 3y ago
        Because Upstash is serverless redis.
simonw 3y ago
Will this be available for Python, too? The example code is all JavaScript, but presumably this uses APIs under the hood that could be called by other languages. I've been a happy user of Python on Vercel for years, but I often feel like languages other than JavaScript are very poorly represented, both in the community and the documentation. My notes on running Python on Vercel: https://til.simonwillison.net/zeit-now/python-asgi-on-now-v2 https://til.simonwillison.net/zeit-now/python-asgi-on-now-v2
- cramforce 3y ago
  Yes! Any postgres and redis client will just work. And the blob store has a REST API.
  - simonw 3y ago
    Based on this code example: import { sql } from '@vercel/postgres'; const { rows } = await sql` INSERT INTO products (name) VALUES (${formData.get('name')}) `; Presumably authentication is handled transparently? I really like that - reminds me of Deno's new KV cloud stuff too. Is that done with environment variables? I'd want a way to tap into that from Python code as well.
    - cramforce 3y ago
      Yeah, if you go into the dashboard it gives you a bunch of options for connecting to the DB including the names of the automatically generated environment variables. And that includes POSTGRES_URL which most tools default to.
bambam24 3y ago
[dead]
lxe 3y ago
A very natural step for Vercel. Not having a prescribed statefulness/storage solution has been a pain since they've pivoted into strictly serverless.
bluebeel 3y ago
the margin that vercel makes on these products that you can easily integrate with nextjs (or anything else that runs on their infra) is crazy. All of them offer easy integration with the edge or lambas functions... I don't see how vercel justifies this price outside of DX
- sroussey 3y ago
  I mean, you can say that for serverless in general. Heck, you can say that for AWS etc as well.
- cccybernetic 3y ago
  Consumers have always been willing to pay a premium for simplicity & convenience. No difference here: devs will pay for DX.
  - bluebeel 3y ago
    I understand that bu Vercel KV is almost 2x more expensive than Upstash. Even the vercel/kv package is just a wrapper for upstash/redis - I mean what is the benefit of using Vercel KV instead of Upstash? What DX is improved? Upstash offers a nice UI and easy integration with edge functions and lambas functions. It's the same with neon database and cloudflare r2. The only plus value I see is the centralised dashboard. Paying 2x for a single dashboard?
    - ethomson 3y ago
      Product Manager for Vercel's storage products here. Today we announced three new storage products in beta; they're based on infrastructure that are provided by partners. There's a lot that goes in to pricing products -- and our products are distinct from our partners' products. We have different roadmaps and will introduce different features as we continue development. So I don't love to make too many comparisons between apples and oranges. But I suspect that you might be comparing Upstash's per-command pricing for _regional_ requests ($0.20 per 100k) to Vercel KV's? In fact, Vercel KV is multi-region, so the more apt comparison is Upstash's pricing for _global_ requests ($0.40 per 100k).
wereHamster 3y ago
The postgres code snippet is a text-book example of how to enable SQL injection attacks. Great performance from the Vercel marketing team.
- teaearlgraycold 3y ago
  I don't think you understand what the code is doing. That's a string templating function, not a raw templated string. `${formData.get('name')}` will get passed as a parameter in a var-arg, not stuck straight into the string.
  - wereHamster 3y ago
    Right. The code converts all expressions into $N-parameters. However Postgres does not support parameters in all locations, eg. this would fail: import { tableName } from "../shared" sql`SELECT * FROM ${tableName} WHERE id=${formData.get('id')}` The @vercel/postgres package needs a big disclaimer that it works very differently from node-postgres and what is and is not allowed.
    - jitl 3y ago
      That would result in an error then, not an injection
    - porsager 3y ago
      They've looked at Postgres.js (https://github.com/porsager/postgres https://github.com/porsager/postgres) before — wouldn't mind if they enabled those other cases in the same way. Also, node-postgres is ripe for sql injections because you just concatenate strings yourself when using it...
      - jmull 3y ago
        > node-postgres is ripe for sql injections because you just concatenate strings yourself when using it... Why the heck would you do that? I guess for any library that accepts SQL, someone might concatenate strings rather than use the parameter handling.
teaearlgraycold 3y ago
I looked into serverless postgres offerings a few months ago and came to realize that while it is possible, it doesn't appear that we have figured out how to develop with it as well as we have with traditional postgres. ORMs are fairly incompatible with serverless postgres, and schema migrations are difficult to handle because you have an unknown number of asynchronously updated copies of your code spread around the world. Definitely something you can account for with good planning, but probably not worth the headache to the small and medium-sized teams that would want to sign up for Vercel.
- theturtletalks 3y ago
  I had the same experience. I have a Next.js app and Postgres DB I host on a server-ful PaaS. I hosted the same app on Vercel with Neon DB and the app was significantly slower. The first time Neon is connected, it will be slow so I was prepared for that. Consequent reloads and using the app normally was slow too. Maybe integrating Neon directly with Vercel will fix these issues. I just don’t see the value prop to use Vercel for SaaS apps. Using a running server for those works better, cheaper, and faster. For consumer-facing storefronts, Vercel is a no-brainer though.
  - maccaw 3y ago
    Have you two seen this? https://github.com/neondatabase/serverless https://github.com/neondatabase/serverless
    - theturtletalks 3y ago
      I used Neon’s cloud DB service directly
  - aledalgrande 3y ago
    That's one thing I don't understand about all this serverless + serverless stuff. Aren't we adding network calls (into different subnets) and cold starts everywhere? That sounds terrible from a performance point of view.
- 3y ago
babyshake 3y ago
This is more or less like Railway, but hosted by Vercel?
l5870uoo9y 3y ago
Essentially added a "Storage" tab and configuration management in Vercel for selected integrations. And then charge a premium for that: - Vercel Postgres pricing: https://vercel.com/docs/storage/vercel-postgres/usage-and-pricing https://vercel.com/docs/storage/vercel-postgres/usage-and-pr... - Neon Postgres pricing: https://neon.tech/docs/introduction/billing https://neon.tech/docs/introduction/billing You can properly still buy Postgres directly from Neon or choose another database provider: https://vercel.com/integrations#databases https://vercel.com/integrations#databases
crooked-v 3y ago
This is cool and whatever, but after the work my company has put in to actively move things away from Vercel I'm going to assume it's hugely overpriced by default compared to even other similar SaaS services.
- 015a 3y ago
  Its extremely expensive, and I'm aware of two companies migrating away from NextJS itself (neither were ever hosting with Vercel, though). I've only talked in depth with one of them, but the reason is complexity: they want "a react framework" and every update to NextJS substantially raises the complexity bar, introducing a large number of features that they don't want or need. There's also been at least one incident of a NextJS bug which only affected non-Vercel deployments; and that incident came up in this discussion as evidence for their concern that NextJS doesn't have a future outside of Vercel (the company and the platform).
  - riogordo2go 3y ago
    This is also my concern. If the happy path is Serverless/Edge (on Vercel) with React Server Components I'm in for a big rewrite. I'm holding off any new investments in the Next.js ecosystem until the dust settles or a more community focused and stable alternative arises.
    - niyazpk 3y ago
      What are the leading alternatives? I know about RemixJS, but not too up to date on others and how each of these compare against each other.
      - ovao 3y ago
        Remix is probably the most notable, but there’s Astro as well: https://astro.build/ https://astro.build/ My impression on Remix is fairly solid, but it doesn’t have quite the “just write and deploy” appeal that Vercel’s done quite a good job with, and they’re somewhat trailing behind Next.js in terms of tracking newer React features (which, in practical terms, doesn’t matter much, since even beta Next.js features tend to ship in a very alpha state of quality). That said, if you’re willing to sweat a bit on getting something deployed, Remix delivers a pretty good experience overall.
billiam 3y ago
Is the (moderate) improvement in Dx and writing one check worth the increased costs? In the case of blob storage, aren't most Vercel customers paying Cloudflare already?
frankfrank13 3y ago
cloud was a mistake
- grrdotcloud 3y ago
  I want to disagree but thanks be to the lack of agreed definitions I will agree.
  - frankfrank13 3y ago
    i was being intentionally stupid, but based on this pricing I'm forced to think that Vercel et al. are just praying on junior/inexperienced devs. This is like a 4x premium (maybe more) over AWS, and that is already a huge premium.
    - deleted 3y ago
      [deleted]
franky47 3y ago
> 'use server'; > Next.js Server Actions (to be announced Thursday) Looking forward to have to re-learn the Next.js API, again! /s
torartc 3y ago
Feels like the pricing is very high.
luckluckduck 3y ago
I really wish this had been Fly or Supabase. I’ve used Neon and it is much less performant and often times completely non responsive. My company already uses Cloudflare so the rest isn’t a big change for us.
- gigasmurf2077 3y ago
  Can you elaborate on your issues with Neon? Interested to know
  - nikita 3y ago
    Please let us know what Neon performance issue you are having. We have been steadily working on performance and it keeps getting better. Our next ongoing investment is cold starts.
- pier25 3y ago
  You mean the KV? Fly had a Redis service but they shut it down. Although tbh it's trivial to add Redis to a Fly app with Docker.
jmull 3y ago
> import { sql } from '@vercel/postgres'; Careful here. You'd be building a vercel-specific API into the middle of your data access stack. That might be fine... e.g., if it has a limited/specific use or lifetime, so that you know you won't get stuck. But otherwise, use a standard, portable client. The docs say you can use any postgres client, so you should be able to choose whatever you prefer.
jamesmcintyre 3y ago
I'm wondering why vercel chose upstash vs cloudflare kv store? I've worked on very large deployments of cloudflare workers + kv store and the performance is amazing (and pricing). It would seem to me that upstash would add hops/latency vs cloudflare kv store?
- gsanderson 3y ago
  I would guess for people who want Redis? As I recall Cloudflare KV is more limited and wouldn't be a suitable drop-in replacement.
- kbumsik 3y ago
  Doesn't Cloudflare KV have to be used with Cloudflare Workers?
  - uceumice 3y ago
    I believe, the implication was, that as Vercel's Edge is built on top of Cloudflare Workers, it could be more natural to integrate with Cloudflare's own KV offering.
chrismsimpson 3y ago
Serverless everywhere yields apps with massive latency & loading times and if timeouts aren’t well tuned (read: set unreasonably high) you often get dropped connections on api calls. I am currently migrating an app off vercel to raw compute (EC2) and the difference is night and day.
- dahwolf 3y ago
  I'm also noting a generic pattern of intermittent response times in pretty much anything serverless. When you take a traditional tech stack, say LAMP on a dedicated server, performance is very constant overall. You even develop a type of muscle memory for it and adapt a click pace/flow when you use an application like that intensely. No so much with cloud-native serverless. A page may respond fast and then 10 times slower the next time. There's just so many moving parts (virtualization, edge, cache, cold/warm compute, out of network dependencies, etc) that it feels random to a user.
- fireant 3y ago
  It's funny that on one hand people are preaching about Time To First Byte and SSR and what not, and on the other people from CF/Vercel and others are massively investing into serverless which almost by definition has to have drastically degraded latency compared to normal apps.
pier25 3y ago
> Simultaneously, as the world moves away from monolithic architectures to composable ones It's been the opposite for me. Back around 2015-2016 I was very excited about cloud functions, serverless, etc, but these past years I've gone back to mostly running VMs with regular persistent apps. Complexity has gone down considerably and there's zero lock in. With Docker I have full control over the platform and can run these apps pretty much anywhere I want and how I see fit. With Fly it's trivial to get scale to zero. I wrote a little tutorial here recently: https://community.fly.io/t/implementing-scale-to-zero-is-super-easy/12415 https://community.fly.io/t/implementing-scale-to-zero-is-sup... I still use some serverless stuff for very specific use cases, like enhancing a static site with a bit of backend logic, but definitely not as the main solution to run the bulk of my backend apps.
- dham 3y ago
  I'm right with you. This stuff was exciting like ~5 years ago. I did several projects with Firebase functions as well as Lambda. Randomly on a few of my projects, the functions don't work anymore. Although I haven't touched it in like 4 years. Either way, I'm back on the monolith train and probably won't look back for a while.
  - pier25 3y ago
    > Randomly on a few of my projects, the functions don't work anymore Yeah that happened to me like a year or two ago. Google changed something on the platform running my functions and my app stopped working without any notice.
- Alifatisk 3y ago
  This, feels to me like the opposite shift is happening. People moving away from microservices and going back to the monolithic arch because they realized they don’t need the scale and complexity FAANG is working on.
  - yieldcrv 3y ago
    there's tons of free storage too I know people are allergic to the word blockchain, but you can store variables on them for nearly free (its a one-time cost), have unlimited free read traffic, and have your users pay to update your state for you your frontends just read that and update dynamically cloud platforms aren't able to compete with that, and you can just do system design that is applicable to this infrastructure while scrapping every idea that doesn't work - just like in the before times
brundolf 3y ago
Nice. This was a major reason I've never hosted anything on Vercel: I didn't want to juggle multiple providers. Glad to know it's an option now
proxy9 3y ago
Gimme a effin break. Man people are increasingly lazy. We get excited about a Neondb wrapper now? Really?
moomoo11 3y ago
Am I the only Luddite who Deploys Go app containers. Pick a cloud. Use whatever they offer. Deploys static JS to cdn Calls it a day. I really do not understand these tools. Who is using them? I’ve built and worked on apps used by millions of people. Really don’t understand where the benefit is or if people are just over complicating basic things to extract money from chump developers?
- creakingstairs 3y ago
  Yes their DX is technically “over complicating basic things to extract money from chump developers” but it lets users just write code and automate away everything else with couple of click. It limiting your options can be a good thing, people can just ship instead of being stuck in analysis paralysis forever. I also found it quite nice entry point for newbie developers. I’ve been donating my time to help out new developers, and learning deployment (it isn’t ”basic” for complete newbies) on top of everything can be daunting. Vercel lets them deploy for free and effortlessly so that they can show off what they’ve learned. That being said, I personally wouldn’t touch Vercel for any serious work either.
  - pjmlp 3y ago
    Problem is that how things go, many SaaS products are now making partnerships with Vercel for their webhooks and hosting of frontends and middleware configurations, with SDKs that are Vercel ready and DIY for anyone else.
- overbytecode 3y ago
  Do you deploy just the JS to a CDN or a whole website? I wanted to do something similar, throw a static website on a CDN and use a Go API as its backend. But unsure how to get around CORS in a reliable way.
  - fireant 3y ago
    You setup CORS on your backend, it does not matter where or how you frontend works. IMO biggest problem is that if your website is public facing, then just having an SPA is bad for crawlers and not great for UX.
le_sigh_88 3y ago
I really can't stand this company and it's business model. It preys on inexperienced engineers, locks them into an ecosystem of SaaS products that abstracts away important concepts they should learn, and charges an absolutely absurd premium while doing it. No serious company is ever going to use these criminally expensive software, so it's quite clear what their target market is. Unfortunately, they have such a strong hold of the Twitter and Youtube space with their army of cringey influencers. This company does nothing but a disservice to the next generation of web developers. Really a shame.
- mhoad 3y ago
  This managed to encapsulate something that has been in the tip of my tongue ever since I first encountered them. That company is just a sea of red flags to me.
  - dbbk 3y ago
    The Apple iPhone reveal styled announcement videos are also just absurd. Today, we are reinventing the web!
- rykuno 3y ago
  > It preys on inexperienced engineers For a weekend/side project that needs db and storage, why would I spend days correctly setting up, securing, and provisioning all my infra when I could just pay a little extra and never worry about it? When creating a new project I wear all hats. Project management, UX/UI designing, Frontend, Backend, Infra, User testing, security. Having to maintain multiple projects that all bring income in as a solo developer wearing all the hats is very time consuming, this is an absolute godsend and I'll pay for the DX happily.
  - ncphillips 3y ago
    I definitely agree with this But also I just started a side project with Rails. I got a CI/CD production environment with GitHub Actions, Postgres, Redis, sidekiq, and storage on Fly.io in an hour or two. The thing that makes me more comfortable here is that none of my application code references Fly.io at all. If the project needs to grow beyond Fly I just change some environment variables and migrate the data.

267 points by drecoe 3y ago

swyx 3y ago

Vercel is the second company to “eat the cloud from outside in”. (https://www.swyx.io/cloudflare-go https://www.swyx.io/cloudflare-go) its now my defacto playbook for building lasting bottom up disruptive cloud companies. start by giving away an extremely good free tier (cloudflare - cdn, vercel - nextjs+hosting) then add build time compute, run time compute, readonly kv store, and now full read write storage. (this is part of an overall cloud progression that i've been studying for a couple years https://twitter.com/swyx/status/1417136897894326277 https://twitter.com/swyx/status/1417136897894326277) many other platforms try to start off offering everything under the sun, claiming to care about the holistic end to end experience. vercel bet correctly on one wedge, built it up over years, and only now is expanding into storage. well earned.

youngtaff 3y ago
Cloudflare own and run the services they’re offering whereas it looks like Vercal storage is just other people’s services rebranded
- asadm 3y ago
  Users don't care about that.
  - proxy9 3y ago
    I think they care. Vercel is VC money. Not even close to profitability. Anyone building anything but a toy needs to take into consideration that Vercel might not be here in 2 years from now
  - waboremo 3y ago
    Generally, that is very true. But when the bills start hitting, users absolutely do care and will go out of their way to avoid it.
    - asadm 3y ago
      Only when it reaches ridiculous enough number. Even as an individual who can easily do any cloud/k8s/lambdas/firecracker, I am paying Vercel a few hundred for my side projects.
      - waboremo 3y ago
        True, but the ridiculous number might be lower than you expect. Vercel in particular has got a lot of feedback about their pricing!
  - the_mitsuhiko 3y ago
    … for as long as there is no cheaper competition.
  - mrkurt 3y ago
    They do, actually. We don't say anything about running our own infrastructure. And every time someone finds out, they're like "why didn't you lead with that?" I'm surprised how much people care about what's under the covers.
    - eropple 3y ago
      I think that in many ways is because Fly's approach is sufficiently high-minded (in other words, aiming most comfortably at relatively difficult problems) that it's probably pretty automatic to assume that it's built on an existing hyperscaler. Mentioning that you run your own infrastructure adds encouragement that you know what you're doing. Given the general state of PaaS offerings, this is a differentiator.
ghc 3y ago
Classic VC-backed B2B play. The trick, however, is finding a "wedge" people actually want on its own. Unfortunately that seems to be easier said than done.
- swyx 3y ago
  well a good start is betting on the most popular framework in the most popular language and building the production ready stuff that is missing that the framework authors constitutionally cannot compete with you on :) there are better wedges financially speaking but theres pretty much no bigger bottom up developer wedge than this one
  - mattwad 3y ago
    They built the framework too, or was it a purchase originally?
    - dabeeeenster 3y ago
      Guillermo the Vercel CEO made Next.js
    - doodlesdev 3y ago
      Vercel (then called Zeit) developed NextJS themselves. I believe swyx meant React when he said "framework", even though one can argue that's technically incorrect it's been commonplace for some time to call it a framework.
- cccybernetic 3y ago
  There’s a lot of magic with Vercel, but I think the true “wedge” was clear to many people: DX. The trick was really the execution, which they nailed. They realized early that there was a whole class of developers out there, many of them quite talented, who couldn’t stand SSH’ing into Linux boxes, configuring htaccess files, installing SSL certificates, etc. Many of these developers also worked at big companies, and held sway. They distilled the entire deployment process into a few clicks and boom, you have a real, functioning, fast website. Then they added templates, good marketing, and doubled down on React. The rest is history. It’s a well executed vision and I give them a lot of props for that.
waboremo 3y ago
I believe this is a core factor as to why Supabase hasn't joined Cloudflare and Vercel, they don't have that "outside" aspect. By all other counts, they are well established to do so and who knows maybe they do have something in the works to tie in their offerings together from the "outside".
- swyx 3y ago
  (am supabase investor) fortunately supabase's value prop is so straight forward it doesnt quite need to do the "outside-in" maneuver. just make postgres into a developer friendly platform. huge TAM in itself. another investor i asked on why they put $millions into supabase simply said "RDS has more than $1b ARR"
  - waboremo 3y ago
    Great point about being direct, and the priority on DX seems to be a semi-moat in itself, worked for Vercel and Supabase so far!
vultour 3y ago
Will they run my workloads without requiring my attention for decades? I'd love to rewrite and move some old PHP sites from my webhost, but that one has been hosting them without requiring my attention for probably close to 15 years. I'm a little skeptical these fancy cloud runtimes will even exist a few years from now.
- diroussel 3y ago
  Also nodejs is deprecating versions at an impressive rate. Todays LTS will be out of date soon. AWS and thus Vercel have to follow along. So you can’t run a node based serverless app for more than a year or two before you either put the work in to upgrade or just cross your fingers that the forced upgrade won’t break your app.
c-hendricks 3y ago
> https://twitter.com/swyx/status/1417136897894326277 https://twitter.com/swyx/status/1417136897894326277 Companies are leasing out the core aspects of computing (storage, networking, CPU, GPU, auth), nothing really insightful here. Just your average day in capitalism. > many other platforms try to start off offering everything under the sun Do they? Or do they also start small, then build up over the years until they have everything under the sun, rinsing and repeating the process.
throwingrocks 3y ago
> now my defacto playbook for building lasting bottom up disruptive cloud companies What companies have you built (or played a role in building) using this "playbook" that fit your description?

scsteps 3y ago

Very nice. I really wanted this to happen.

MattIPv4 3y ago

Cloudflare's R2 costs $0.36 per million read operations (after the 10 million you get for free) [1]. Vercel is wrapping R2 and is charging $2 per million reads [2]. They're also charging $0.15/GB for egress (after the 1GB you get free), when R2 charges nothing, and the storage cost is doubled from $0.015/GB on R2 to $0.03/GB on Vercel. That's quite the cost increase for the DX improvement. [1] https://developers.cloudflare.com/r2/pricing/ https://developers.cloudflare.com/r2/pricing/ [2] https://vercel.com/docs/storage/vercel-blob/usage-and-pricing https://vercel.com/docs/storage/vercel-blob/usage-and-pricin...

doublepg23 3y ago
Hijacking this comment to ask if anyone has had luck integrating Cloudflare R2 with Pleroma. I’ve not had any luck getting it to work though I’m also not well versed on the terminology.
leerob 3y ago
This isn't finalized yet, apologies for the confusion. We'll be updating the pricing for Blob shortly (it's in private beta and invite only). The pricing for KV and Postgres is up to date.
- dexterdog 3y ago
  I already lean away from S3/Azureblob/etc because their egress prices are terrible. .15/GB is borderline criminal.
  - Demonto 3y ago
    150$ for a TB traffic?! Wtf
    - mrcwinn 3y ago
      What’s happening right now is a story we’ve seen plenty of times before. Overhead cost of convenience mixed with vendor lock-in, all while boasting about open source. I’d love to create tools that were convenient and had fair pricing. The challenge is that you’re trying to grow into a market with worse acquisition economics. Tough to win if winning is the goal. If anyone has a solution reach out. XD

MattIPv4 3y ago

Pricing for them: Blob: https://vercel.com/docs/storage/vercel-blob/usage-and-pricing https://vercel.com/docs/storage/vercel-blob/usage-and-pricin... KV: https://vercel.com/docs/storage/vercel-kv/usage-and-pricing https://vercel.com/docs/storage/vercel-kv/usage-and-pricing Postgres: https://vercel.com/docs/storage/vercel-postgres/usage-and-pricing https://vercel.com/docs/storage/vercel-postgres/usage-and-pr...

leerob 3y ago
KV/Postgres are accurate, that Blob page sneaked in. Blob pricing should be figured out shortly, we'll update that page (it's in private beta, invite only currently).
- dbbk 3y ago
  So take it down?

revskill 3y ago

Cloud to me is overpriced and not transparent. Example, with Upstash redis: i deployed a fly.io connected to upstash. I don't use that service for a month (no query at all), but it still counted in billing. Wait, is upstash real serverless ? I have to pay for NOT using it ?

epberry 3y ago
Usage-based pricing in the cloud is a blessing and a curse. To take best advantage of it you need a third party tool. At least, this is the thesis behind my current employer (https://vantage.sh https://vantage.sh).
mrkurt 3y ago
Upstash on Fly.io is not serverless. You provision the RAM + connection counts you want, and then pay for that capacity. This is not what all our customers want, but it is much cheaper to run this way for most fullstack frameworks. It's not ideal for a Redis you don't use at all, though. We're considering a serverless / metered billing option for this. But what we have is much closer to spinning up a permanent Redis. :)
- revskill 3y ago
  The dashboard on fly.io didn't say it's not serverless. SO it's not transparent. Look like a scam to me. For now, i had to say goodbye for most of cloud services. Good old VPS for now.
  - mrkurt 3y ago
    Ok that's a silly thing to say. Nowhere on our site do we say we offer serverless anything. When you provision a Redis, we offer you a selection of plans, you pick the one you want, and off you go. I don't understand why you'd assume that's serverless.
    - revskill 3y ago
      Because Upstash is serverless redis.

simonw 3y ago

Will this be available for Python, too? The example code is all JavaScript, but presumably this uses APIs under the hood that could be called by other languages. I've been a happy user of Python on Vercel for years, but I often feel like languages other than JavaScript are very poorly represented, both in the community and the documentation. My notes on running Python on Vercel: https://til.simonwillison.net/zeit-now/python-asgi-on-now-v2 https://til.simonwillison.net/zeit-now/python-asgi-on-now-v2

cramforce 3y ago
Yes! Any postgres and redis client will just work. And the blob store has a REST API.
- simonw 3y ago
  Based on this code example: import { sql } from '@vercel/postgres'; const { rows } = await sql` INSERT INTO products (name) VALUES (${formData.get('name')}) `; Presumably authentication is handled transparently? I really like that - reminds me of Deno's new KV cloud stuff too. Is that done with environment variables? I'd want a way to tap into that from Python code as well.
  - cramforce 3y ago
    Yeah, if you go into the dashboard it gives you a bunch of options for connecting to the DB including the names of the automatically generated environment variables. And that includes POSTGRES_URL which most tools default to.

bambam24 3y ago

[dead]

lxe 3y ago

A very natural step for Vercel. Not having a prescribed statefulness/storage solution has been a pain since they've pivoted into strictly serverless.

bluebeel 3y ago

the margin that vercel makes on these products that you can easily integrate with nextjs (or anything else that runs on their infra) is crazy. All of them offer easy integration with the edge or lambas functions... I don't see how vercel justifies this price outside of DX

sroussey 3y ago
I mean, you can say that for serverless in general. Heck, you can say that for AWS etc as well.
cccybernetic 3y ago
Consumers have always been willing to pay a premium for simplicity & convenience. No difference here: devs will pay for DX.
- bluebeel 3y ago
  I understand that bu Vercel KV is almost 2x more expensive than Upstash. Even the vercel/kv package is just a wrapper for upstash/redis - I mean what is the benefit of using Vercel KV instead of Upstash? What DX is improved? Upstash offers a nice UI and easy integration with edge functions and lambas functions. It's the same with neon database and cloudflare r2. The only plus value I see is the centralised dashboard. Paying 2x for a single dashboard?
  - ethomson 3y ago
    Product Manager for Vercel's storage products here. Today we announced three new storage products in beta; they're based on infrastructure that are provided by partners. There's a lot that goes in to pricing products -- and our products are distinct from our partners' products. We have different roadmaps and will introduce different features as we continue development. So I don't love to make too many comparisons between apples and oranges. But I suspect that you might be comparing Upstash's per-command pricing for _regional_ requests ($0.20 per 100k) to Vercel KV's? In fact, Vercel KV is multi-region, so the more apt comparison is Upstash's pricing for _global_ requests ($0.40 per 100k).

wereHamster 3y ago

The postgres code snippet is a text-book example of how to enable SQL injection attacks. Great performance from the Vercel marketing team.

teaearlgraycold 3y ago
I don't think you understand what the code is doing. That's a string templating function, not a raw templated string. `${formData.get('name')}` will get passed as a parameter in a var-arg, not stuck straight into the string.
- wereHamster 3y ago
  Right. The code converts all expressions into $N-parameters. However Postgres does not support parameters in all locations, eg. this would fail: import { tableName } from "../shared" sql`SELECT * FROM ${tableName} WHERE id=${formData.get('id')}` The @vercel/postgres package needs a big disclaimer that it works very differently from node-postgres and what is and is not allowed.
  - jitl 3y ago
    That would result in an error then, not an injection
  - porsager 3y ago
    They've looked at Postgres.js (https://github.com/porsager/postgres https://github.com/porsager/postgres) before — wouldn't mind if they enabled those other cases in the same way. Also, node-postgres is ripe for sql injections because you just concatenate strings yourself when using it...
    - jmull 3y ago
      > node-postgres is ripe for sql injections because you just concatenate strings yourself when using it... Why the heck would you do that? I guess for any library that accepts SQL, someone might concatenate strings rather than use the parameter handling.

teaearlgraycold 3y ago

I looked into serverless postgres offerings a few months ago and came to realize that while it is possible, it doesn't appear that we have figured out how to develop with it as well as we have with traditional postgres. ORMs are fairly incompatible with serverless postgres, and schema migrations are difficult to handle because you have an unknown number of asynchronously updated copies of your code spread around the world. Definitely something you can account for with good planning, but probably not worth the headache to the small and medium-sized teams that would want to sign up for Vercel.

theturtletalks 3y ago
I had the same experience. I have a Next.js app and Postgres DB I host on a server-ful PaaS. I hosted the same app on Vercel with Neon DB and the app was significantly slower. The first time Neon is connected, it will be slow so I was prepared for that. Consequent reloads and using the app normally was slow too. Maybe integrating Neon directly with Vercel will fix these issues. I just don’t see the value prop to use Vercel for SaaS apps. Using a running server for those works better, cheaper, and faster. For consumer-facing storefronts, Vercel is a no-brainer though.
- maccaw 3y ago
  Have you two seen this? https://github.com/neondatabase/serverless https://github.com/neondatabase/serverless
  - theturtletalks 3y ago
    I used Neon’s cloud DB service directly
- aledalgrande 3y ago
  That's one thing I don't understand about all this serverless + serverless stuff. Aren't we adding network calls (into different subnets) and cold starts everywhere? That sounds terrible from a performance point of view.
3y ago

babyshake 3y ago

This is more or less like Railway, but hosted by Vercel?

l5870uoo9y 3y ago

Essentially added a "Storage" tab and configuration management in Vercel for selected integrations. And then charge a premium for that: - Vercel Postgres pricing: https://vercel.com/docs/storage/vercel-postgres/usage-and-pricing https://vercel.com/docs/storage/vercel-postgres/usage-and-pr... - Neon Postgres pricing: https://neon.tech/docs/introduction/billing https://neon.tech/docs/introduction/billing You can properly still buy Postgres directly from Neon or choose another database provider: https://vercel.com/integrations#databases https://vercel.com/integrations#databases

crooked-v 3y ago

This is cool and whatever, but after the work my company has put in to actively move things away from Vercel I'm going to assume it's hugely overpriced by default compared to even other similar SaaS services.

015a 3y ago
Its extremely expensive, and I'm aware of two companies migrating away from NextJS itself (neither were ever hosting with Vercel, though). I've only talked in depth with one of them, but the reason is complexity: they want "a react framework" and every update to NextJS substantially raises the complexity bar, introducing a large number of features that they don't want or need. There's also been at least one incident of a NextJS bug which only affected non-Vercel deployments; and that incident came up in this discussion as evidence for their concern that NextJS doesn't have a future outside of Vercel (the company and the platform).
- riogordo2go 3y ago
  This is also my concern. If the happy path is Serverless/Edge (on Vercel) with React Server Components I'm in for a big rewrite. I'm holding off any new investments in the Next.js ecosystem until the dust settles or a more community focused and stable alternative arises.
  - niyazpk 3y ago
    What are the leading alternatives? I know about RemixJS, but not too up to date on others and how each of these compare against each other.
    - ovao 3y ago
      Remix is probably the most notable, but there’s Astro as well: https://astro.build/ https://astro.build/ My impression on Remix is fairly solid, but it doesn’t have quite the “just write and deploy” appeal that Vercel’s done quite a good job with, and they’re somewhat trailing behind Next.js in terms of tracking newer React features (which, in practical terms, doesn’t matter much, since even beta Next.js features tend to ship in a very alpha state of quality). That said, if you’re willing to sweat a bit on getting something deployed, Remix delivers a pretty good experience overall.

billiam 3y ago

Is the (moderate) improvement in Dx and writing one check worth the increased costs? In the case of blob storage, aren't most Vercel customers paying Cloudflare already?

frankfrank13 3y ago

cloud was a mistake

grrdotcloud 3y ago
I want to disagree but thanks be to the lack of agreed definitions I will agree.
- frankfrank13 3y ago
  i was being intentionally stupid, but based on this pricing I'm forced to think that Vercel et al. are just praying on junior/inexperienced devs. This is like a 4x premium (maybe more) over AWS, and that is already a huge premium.
  - deleted 3y ago
    [deleted]

franky47 3y ago

> 'use server'; > Next.js Server Actions (to be announced Thursday) Looking forward to have to re-learn the Next.js API, again! /s

torartc 3y ago

Feels like the pricing is very high.

luckluckduck 3y ago

I really wish this had been Fly or Supabase. I’ve used Neon and it is much less performant and often times completely non responsive. My company already uses Cloudflare so the rest isn’t a big change for us.

gigasmurf2077 3y ago
Can you elaborate on your issues with Neon? Interested to know
- nikita 3y ago
  Please let us know what Neon performance issue you are having. We have been steadily working on performance and it keeps getting better. Our next ongoing investment is cold starts.
pier25 3y ago
You mean the KV? Fly had a Redis service but they shut it down. Although tbh it's trivial to add Redis to a Fly app with Docker.

jmull 3y ago

> import { sql } from '@vercel/postgres'; Careful here. You'd be building a vercel-specific API into the middle of your data access stack. That might be fine... e.g., if it has a limited/specific use or lifetime, so that you know you won't get stuck. But otherwise, use a standard, portable client. The docs say you can use any postgres client, so you should be able to choose whatever you prefer.

jamesmcintyre 3y ago

I'm wondering why vercel chose upstash vs cloudflare kv store? I've worked on very large deployments of cloudflare workers + kv store and the performance is amazing (and pricing). It would seem to me that upstash would add hops/latency vs cloudflare kv store?

gsanderson 3y ago
I would guess for people who want Redis? As I recall Cloudflare KV is more limited and wouldn't be a suitable drop-in replacement.
kbumsik 3y ago
Doesn't Cloudflare KV have to be used with Cloudflare Workers?
- uceumice 3y ago
  I believe, the implication was, that as Vercel's Edge is built on top of Cloudflare Workers, it could be more natural to integrate with Cloudflare's own KV offering.

chrismsimpson 3y ago

Serverless everywhere yields apps with massive latency & loading times and if timeouts aren’t well tuned (read: set unreasonably high) you often get dropped connections on api calls. I am currently migrating an app off vercel to raw compute (EC2) and the difference is night and day.

dahwolf 3y ago
I'm also noting a generic pattern of intermittent response times in pretty much anything serverless. When you take a traditional tech stack, say LAMP on a dedicated server, performance is very constant overall. You even develop a type of muscle memory for it and adapt a click pace/flow when you use an application like that intensely. No so much with cloud-native serverless. A page may respond fast and then 10 times slower the next time. There's just so many moving parts (virtualization, edge, cache, cold/warm compute, out of network dependencies, etc) that it feels random to a user.
fireant 3y ago
It's funny that on one hand people are preaching about Time To First Byte and SSR and what not, and on the other people from CF/Vercel and others are massively investing into serverless which almost by definition has to have drastically degraded latency compared to normal apps.

pier25 3y ago

> Simultaneously, as the world moves away from monolithic architectures to composable ones It's been the opposite for me. Back around 2015-2016 I was very excited about cloud functions, serverless, etc, but these past years I've gone back to mostly running VMs with regular persistent apps. Complexity has gone down considerably and there's zero lock in. With Docker I have full control over the platform and can run these apps pretty much anywhere I want and how I see fit. With Fly it's trivial to get scale to zero. I wrote a little tutorial here recently: https://community.fly.io/t/implementing-scale-to-zero-is-super-easy/12415 https://community.fly.io/t/implementing-scale-to-zero-is-sup... I still use some serverless stuff for very specific use cases, like enhancing a static site with a bit of backend logic, but definitely not as the main solution to run the bulk of my backend apps.

dham 3y ago
I'm right with you. This stuff was exciting like ~5 years ago. I did several projects with Firebase functions as well as Lambda. Randomly on a few of my projects, the functions don't work anymore. Although I haven't touched it in like 4 years. Either way, I'm back on the monolith train and probably won't look back for a while.
- pier25 3y ago
  > Randomly on a few of my projects, the functions don't work anymore Yeah that happened to me like a year or two ago. Google changed something on the platform running my functions and my app stopped working without any notice.
Alifatisk 3y ago
This, feels to me like the opposite shift is happening. People moving away from microservices and going back to the monolithic arch because they realized they don’t need the scale and complexity FAANG is working on.
- yieldcrv 3y ago
  there's tons of free storage too I know people are allergic to the word blockchain, but you can store variables on them for nearly free (its a one-time cost), have unlimited free read traffic, and have your users pay to update your state for you your frontends just read that and update dynamically cloud platforms aren't able to compete with that, and you can just do system design that is applicable to this infrastructure while scrapping every idea that doesn't work - just like in the before times

brundolf 3y ago

Nice. This was a major reason I've never hosted anything on Vercel: I didn't want to juggle multiple providers. Glad to know it's an option now

proxy9 3y ago

Gimme a effin break. Man people are increasingly lazy. We get excited about a Neondb wrapper now? Really?

moomoo11 3y ago

Am I the only Luddite who Deploys Go app containers. Pick a cloud. Use whatever they offer. Deploys static JS to cdn Calls it a day. I really do not understand these tools. Who is using them? I’ve built and worked on apps used by millions of people. Really don’t understand where the benefit is or if people are just over complicating basic things to extract money from chump developers?

creakingstairs 3y ago
Yes their DX is technically “over complicating basic things to extract money from chump developers” but it lets users just write code and automate away everything else with couple of click. It limiting your options can be a good thing, people can just ship instead of being stuck in analysis paralysis forever. I also found it quite nice entry point for newbie developers. I’ve been donating my time to help out new developers, and learning deployment (it isn’t ”basic” for complete newbies) on top of everything can be daunting. Vercel lets them deploy for free and effortlessly so that they can show off what they’ve learned. That being said, I personally wouldn’t touch Vercel for any serious work either.
- pjmlp 3y ago
  Problem is that how things go, many SaaS products are now making partnerships with Vercel for their webhooks and hosting of frontends and middleware configurations, with SDKs that are Vercel ready and DIY for anyone else.
overbytecode 3y ago
Do you deploy just the JS to a CDN or a whole website? I wanted to do something similar, throw a static website on a CDN and use a Go API as its backend. But unsure how to get around CORS in a reliable way.
- fireant 3y ago
  You setup CORS on your backend, it does not matter where or how you frontend works. IMO biggest problem is that if your website is public facing, then just having an SPA is bad for crawlers and not great for UX.

le_sigh_88 3y ago

I really can't stand this company and it's business model. It preys on inexperienced engineers, locks them into an ecosystem of SaaS products that abstracts away important concepts they should learn, and charges an absolutely absurd premium while doing it. No serious company is ever going to use these criminally expensive software, so it's quite clear what their target market is. Unfortunately, they have such a strong hold of the Twitter and Youtube space with their army of cringey influencers. This company does nothing but a disservice to the next generation of web developers. Really a shame.

mhoad 3y ago
This managed to encapsulate something that has been in the tip of my tongue ever since I first encountered them. That company is just a sea of red flags to me.
- dbbk 3y ago
  The Apple iPhone reveal styled announcement videos are also just absurd. Today, we are reinventing the web!
rykuno 3y ago
> It preys on inexperienced engineers For a weekend/side project that needs db and storage, why would I spend days correctly setting up, securing, and provisioning all my infra when I could just pay a little extra and never worry about it? When creating a new project I wear all hats. Project management, UX/UI designing, Frontend, Backend, Infra, User testing, security. Having to maintain multiple projects that all bring income in as a solo developer wearing all the hats is very time consuming, this is an absolute godsend and I'll pay for the DX happily.
- ncphillips 3y ago
  I definitely agree with this But also I just started a side project with Rails. I got a CI/CD production environment with GitHub Actions, Postgres, Redis, sidekiq, and storage on Fly.io in an hour or two. The thing that makes me more comfortable here is that none of my application code references Fly.io at all. If the project needs to grow beyond Fly I just change some environment variables and migrate the data.