6 ms·
Best practices for TypeScript monorepo
- moojd 4y agoIf you're willing to go whole hog into pnpm, rush.js is really nice. You get a single unified node_modules and lock file for your entire repo with individual node_modules for each app/package projected with symlinks and local packages automatically symlinked together. We also have it set up so that when we push to our develop branch it builds, tests and bumps a prerelease version for only changed packages and dependents.
- yodon 4y agoRushJS looks amazing, thanks for mentioning it.
- seibelj 4y agoMy company has a large typescript monorepo with many (~100) devs all committing to it daily. We use rush and pnpm (and eslint and some other stuff). Very solid repository. No complaints. Pleasant to work in
- andrew_ 4y agoRush is great. But you're absolutely right. It is whole-hog and very prescriptive. I personally prefer being closer to the bare metal and using pnpm straight up.
- moojd 4y agoAbsolutely. I just want to echo this for anyone in this thread considering using rush. Straying from the way rush wants to do things will bring pain. It is definitely not a good fit for every project but embracing it has solved a ton of problems for us (having tons of duplicated node_modules directories bringing intellij to it's knees, keeping versions in sync, path resolution, publishing, deployment)
- johnfn 4y agoThis seems on topic and it's something I've wondered for a while. Does anyone have any good strategies for quickly running TypeScript against an entire repo? I want to be able to quickly validate that a change that I made didn't introduce any errors. The problem is that our repo is a few hundred thousand lines, and doing `yarn run tsc` takes a long time. I currently use VSCode, which does good at incremental feedback in the current file, but there's still a blind spot where I'm not sure if I've introduced issues that affect files that I'm not currently editing.
- qudat 4y agoThis is the best I could find: https://stackoverflow.com/questions/44676944/how-to-compile-a-specific-file-with-tsc-using-the-paths-compiler-option/60950355#60950355 https://stackoverflow.com/questions/44676944/how-to-compile-...
- wdfx 4y agoBreak your package up into multiple smaller packages in a monorepo. Use tsc incremental and something like nx which can cache and only rebuild changed packages.
- Normal_gaussian 4y agoTypescript in watch mode in a background process (with compilation output on or off) is the default recommendation here. (and what I do, alongside prettier formats and some macros). That is effectively what happens in VSCode, via the lsp, so I'm guessing something is being lazy and its a bug with the integration.
- WorldMaker 4y agoMy understanding is that VSCode's use of the LSP does a lot of "horizon shortening" to keep things performant in a large workspace. If the workspace is small enough it does seem to check every file all the time, as the workspace grows it starts to prioritize open editors and gets lazier about checking other files, and I've even seen situations where it prioritizes the current editors (visible tabs/splits) entirely over even the rest of open editors. I don't think it's necessarily "a bug", but it's certainly seems an intentional laziness to prioritize the editing experience over the compile experience. Which is a long winded thing to say that it's still often useful in VS Code to keep a full tsc --watch process around even if you think the LSP's horizon is wide enough to catch everything because tsc --watch will always have different performance priorities to VSCode's editing experience.
- andrew_ 4y ago> With Yarn Workspaces, installing packages across workspaces becomes faster and lighter. Additionally, it prevents package duplication across workspaces and also makes it possible to create links between directories that depend on each other, ensuring that all directories are consistent in the monorepo. Nice that they included a footnote about pnpm. Makes me want to write a counter article about using pnpm instead. pnpm is faster than yarn 1 (variable with yarn 2, depending on use) and workspaces are just easier. Lest we forget hoisting and dependency dedupe, which is far and away superior with pnpm. My pnpm + TS setup is as follows: - /shared/ directory at root that contains tsconfig.base.json, tsconfig.eslint.json, tsconfig.json (which is only symlinked to, has relative settings for the directories it's linked into) - tsconfig.json at the root, extending shared/tsconfig.base.json, including all things that an editor might care about - /packages/ (or /services/) at the root - /packages/{package}/tsconfig.json -> symlinked to -> /shared/tsconfig.json This setup allows various editors to validate code in any directory that might have TS (and JS!), editor plugins that use ESLint to have a TS config reference, and allows deployment and/or build processes to operate on individual entities in packages|services without having to build the world. Note that this isn't ideal in monorepos where you want to build the world. In monorepos where I have packages and services, for example, and the services are dependent on the packages build built, I leverage pnpm's recursive script ability with filtering to build some of the world, but not all. (Happy to answer questions on pnpm monorepos)
- cercatrova 4y agoYarn 3 has a pnpm style linker. It works pretty well, on par with pnpm itself.
- iddan 4y agoYup, we are using it in FlyCode and it works great.
- interstice 4y agoI love the idea of working with pnpm and I currently use yarn workspaces, but last I heard pnpm has issues with next/nuxt, associated ecosystem packages and deployment (Vercel etc). Are there ways around these problems?
- abuob 4y agoI'm dabbling in the monorepo-tool-space for quite a while already and somehow managed to never hear about Turborepo, seems very interesting! Just a few years back, the monorepo-tooling-landscape left much to be desired, there were a lot of opinionated 'zero-config' tools out there that always seemed to fall apart the moment you strayed from their happy path. I even went so far to create my own tool (https://github.com/abuob/yanice https://github.com/abuob/yanice), in parts because it was fun and taught me a lot and in parts because I simply didn't find something fitting our usecase. It's cool that the tooling in this area is getting better and better, monorepos solve a lot of very annoying enterprise problems but require solid tooling to make it work, even when way smaller than google-scale.
- deleted 4y ago[deleted]
- msoad 4y agoThis is result of almost thousands of hours trying to fix a huge TypeScript monorepo DO NOT HAVE CYCLIC DEPENDENCIES BETWEEN PROJECTS This will prevent all sort of caching systems (including tsc's own --incremental flag) to work.
- niedzielski 4y agoDo you use any tooling to detect this? I had this issue some years back and it wasn't obvious. My builds just took longer and longer.
- no_wizard 4y agoDependency Cruiser works great, can even render visuals: https://github.com/sverweij/dependency-cruiser https://github.com/sverweij/dependency-cruiser NX[0] also has logic for handling this issue [0]: https://nx.dev/ https://nx.dev/
- pc2g4d 4y agoI've been using DPDM: https://github.com/acrazing/dpdm https://github.com/acrazing/dpdm These are my package.json scripts for detecting cycles: "cycles": "run-p cycles:client cycles:server", "cycles:client": "dpdm client/src/\*/\*.ts --exit-code circular:1", "cycles:server": "dpdm server/src/server.ts --exit-code circular:1", The exit code is set like that so that the CI will fail on cycles, since we run this in our CI build. I also hear people use ESLint, probably this: https://github.com/import-js/eslint-plugin-import/blob/main/docs/rules/no-cycle.md https://github.com/import-js/eslint-plugin-import/blob/main/... The fact that the compiler itself doesn't error on cycle imports, and that the errors caused by those imports are so opaque, seems like an oversight to me.
- Fudgel 4y agoThere's an eslint plugin to detect circular dependencies: https://github.com/import-js/eslint-plugin-import/blob/main/docs/rules/no-cycle.md https://github.com/import-js/eslint-plugin-import/blob/main/...
- ly3xqhl8g9 4y ago
- tyleo 4y ago> Use Absolute Paths for Importing Code Blocks I've always felt this one should be built into the TypeScript compiler. Most of my projects are set up to share some utilities and its annoying to integrate babel or other tools just to fix the paths. I've previously used [ttypescript](https://github.com/cevek/ttypescript https://github.com/cevek/ttypescript) with the [typescript-transform-paths](https://www.npmjs.com/package/typescript-transform-paths https://www.npmjs.com/package/typescript-transform-paths) plugin. Gosh, it would be enough if TypeScript just natively supported plugins in the tsconfig (which is what ttypescript provides).
- ntr-- 4y agotypescript DOES support this out of the box. https://www.typescriptlang.org/tsconfig#baseUrl https://www.typescriptlang.org/tsconfig#baseUrl
- tyleo 4y agoThis has not worked in my experience. It allows the IDE to understand the paths but the generated js doesn’t work… maybe I did something wrong?
- madeofpalk 4y ago> Use Absolute Paths for Importing Code Blocks import { FormType } from '../../../../types/form'; import { DateType } from '../../../../types/date'; // This is considered bad practice Disagree on this one. Beauty of typescript is giving you auto-import autocompletion in your editor, and not having to worry about relative paths any more. I don't think I've manually written an import statement in years.
- latchkey 4y agoI think this can be generalized into "use the method that works best with automated tooling".
- madeofpalk 4y agoI think that's very fair, and great advice. We have a strong preference against eslint rules that don't have an auto-fix option.
- Jochim 4y agoPersonally I'm a fan of using relative paths for anything in the same directory or a sub-directory of the current file and absolute paths for everything else. I feel like it keeps things clean while also providing a way to distinguish between closely related modules and imports from further away in your application.
- throwaway284534 4y agoI’ve also fallen back into favoring relative paths and doing my best to keep nested paths to a minimum. This also makes publishing on Deno’s module listing possible too!
- bsimpson 4y agoUsing relative imports that ../ out of your package directory confuses TypeScript about which files are in-scope (and how to structure the resulting files). You can end up with weird shit like my-library/dist/my-library/index.js. (I'm sure you can configure your way out of this. That's a puzzle I haven't solved and don't intend to invest time into.) Yarn Berry has a node_modules mode now, which makes it behave like Yarn Classic and NPM. It also allows you to import sibling projects in a monorepo: "peerDependencies": { "my-library": "workspace:*", which means you can import { thing } from 'my-library' and have it work as expected, even when my-library is a sibling project in your monorepo.
- WatchDog 4y agoThe state of multi module js/ts projects, is such a disappointment when you are used to working in the JVM ecosystem. The tooling sucks, configuration is overly redundant, it’s fragile and when it works it’s super slow.
- bsimpson 4y agoI really like https://github.com/google/wireit/ https://github.com/google/wireit/ for this purpose! My monorepo looks like this: project-1/packages/core project-1/packages/preact project-2/packages/core project-2/packages/demos etc. where a bunch of related projects live top-level in a repo. Each project has a packages folder that includes the core implementation, as well as demos, framework-specific adaptors, etc. In each package's package.json, I have a series of commands (convert the TS to JS, make a bundle, deploy to Firebase, etc.). Each command can depend on another, either in the same project or anywhere else in the file hierarchy. This provides two benefits: 1. Iterating across packages is faster, because I don't have to worry about making sure each package rebuilds in the right order if I make a change in a library. 2. Filesystem concerns are separated: rollup only needs to worry about bundling, and it only needs to bundle web-facing projects. The only tool my libraries need is tsc. (Before wireit, using TypeScript and Rollup together was a pain in the ass because you'd have to fiddle with picking the right TS plugin and configuring it. This was also often the long pole on doing a Rollup version upgrade. Decoupling the two makes Rollup way simpler/easier/nicer to use, which makes wireit awesome even if you don't have multiple packages.) It's also a good replacement for lerna. Each of my packages has a publish command that runs `gitpkg publish`. My root package.json has a publish command that depends on all the packages' publish commands. Thus, I can run `yarn run publish` in the root of my monorepo and trust that all of my packages have been published to our git host appropriately. (gitpkg lets you turn a git host into a private registry, so you can share modules without setting up an NPM registry.) Here's a snippet from one of my package.jsons. They basically all look like this. (start is complicated because of https://github.com/google/wireit/issues/33 https://github.com/google/wireit/issues/33. When that's resolved, it will be as simple as the others.) "scripts": { "start": "yarn run -TB concurrently \"yarn run build:tsc --watch\" \"rollup --config ./rollup.config.js --watch\" \"dhost site --bind-all\"", "build": "yarn run -TB wireit", "build:tsc": "yarn run -TB wireit", "build:bundle": "yarn run -TB wireit", "deploy": "yarn run -TB wireit" }, "wireit": { "build": { "dependencies": [ "build:tsc", "build:bundle" ] }, "build:tsc": { "command": "tsc --build --pretty", "clean": "if-file-deleted", "files": [ "src/**/*.{ts,tsx}", "tsconfig.json", "../../tsconfig.common.json" ], "output": [ "dist/**", ".tsbuildinfo" ], "packageLocks": [ "yarn.lock" ], "dependencies": [ "../core:build", "../preact:build" ] }, "build:bundle": { "command": "rollup --config ./rollup.config.js", "clean": true, "files": [ "dist/**/*.js", "rollup.config.js" ], "output": [ "site/mount.js", ".tsbuildinfo" ], "packageLocks": [ "yarn.lock" ], "dependencies": [ "build:tsc" ] }, "deploy": { "command": "firebase deploy", "dependencies": [ "build:bundle" ] } },
- brunojppb 4y agoGreat article! I am also using turborepo and it has been fantastic! Being able to skip builds from packages that have already been built either in CI or in other developers machines is really amazing. If you are not using Vercel caching, I’ve built my own open-source turbo cache backend[1] that can be self-hosted [1] https://github.com/brunojppb/turbo-racer https://github.com/brunojppb/turbo-racer