14 ms·

Microsoft Office migration from Source Depot to Git

366 points by dshacker 1y ago

ksynwa 1y ago
Not doubting it but I don't understand how a shallow clone of OneNote would be 200GB.
- paulddraper 1y ago
  Must have videos or binaries.
  - LtWorf 1y ago
    They probably vendor every single .dll it uses.
    - skrebbel 1y ago
      that's a lot of .dll files!
- dshacker 1y ago
  Shallow clone of all of office, not onenote.
  - ksynwa 1y ago
    Oh alright. Thanks.
mrheosuper 1y ago
[flagged]
- sherdil2022 1y ago
  If it were that simple, would 100s of engineers spend so much time and effort? They did what they have to and spent the time and energy to maintain some semblance of commit and change history.
  - firesteelrain 1y ago
    GP has a valid point. We had a Git repo managed in BitBucket that was gigantic because it contained binary files and the team didn’t know about LFS and storing them in an external tool like Artifactory. So checkouts took forever and even with shallow clones it took forever. With a CI/CD system running constantly and tests needing constant full coverage and hundreds of developers well it eats into developers time. We can’t just prune all the branches well because of compliance rules. So we ended up removing all the binary artifacts before cloning into a new repo then making the old repo as read only. Microsoft seemed to want to mirror everything rather than keep source depot alive. We had another case where we had a subversion system that went out of security compliance that we simply ported to our git systems and abandoned it. So my guess is they wanted everything to look the same and not just importing the code.
  - hulitu 1y ago
    > If it were that simple, would 100s of engineers spend so much time and effort? Taking into acvount that they rounded corners in Office, I would say, yes.
- senkora 1y ago
  > Microsoft had to collaborate with GitHub to invent the Virtual File System for Git (VFS for Git) just to make this migration possible. Without VFS, a fresh clone of the Office repository (a shallow git clone would take 200 GB of disk space) would take days and consume hundreds of gigabytes.
  - deleted 1y ago
    [deleted]
  - speed_spread 1y ago
    Having had yesterday the dubious pleasure of using MS Word for the first time in a decade, I can safely affirm that they could have have just piped the whole Office repo to the Windows equivalent of /dev/null and nothing of value would have been lost.
    - bigbuppo 1y ago
      The worst part about Word is that it has been feature complete since Office 97, except they've made the UI worse each and every version since then. I wish I could get excited about a new version of Office or WordPerfect, but neither Microsoft or Corel has figured out how to innovate in the past three decades. And no, slapping """AI""" in there isn't the solution. There are so many possibities but they just sort of do nothing with it now that they make a few billion a month on Microsoft 365 subscriptions.
  - golergka 1y ago
    It takes less than an hour on my third world apartment wifi to download Call of Duty Modern Warfare remake which is over 200 gygabytes. Since we're not talking about remote work here, I think Microsoft offices and servers (probably on local network) might have managed similar bandwidth back then.
    - TowerTall 1y ago
      There is a lot more to it than that. Check out "The largest Git repo on the planet" by Brian Harry who was in charge of the git migration and Azure DevOps (Microsoft's pendant to GitHub) https://devblogs.microsoft.com/bharry/the-largest-git-repo-on-the-planet/ https://devblogs.microsoft.com/bharry/the-largest-git-repo-o...
- kirb 1y ago
  Don’t do this on a repository with 35+ years of history! That’s all valuable information you want to keep.
  - speed_spread 1y ago
    Anything before Office 2003 you can delete. Anything after Office 2003 you can also delete. There, saved you a few terabytes.
- samtheprogram 1y ago
  This isn’t Reddit. It’s a lame drive-by joke. The system is working.
smitty1e 1y ago
> We spent months debugging line ending handling "Gosh, that sounds like a right mother," said Unix.
pm90 1y ago
Its oddly fascinating that Microsoft has managed to survive for so long with ancient/bad tools for software engineering. Almost like “life finds a way” but for software dev. From the outside it seems like they are doing better now after embracing OSS/generic dev tools.
- com2kid 1y ago
  At one point source depot was Toincredibly advanced, and there are still features that it had that git doesn't. Directory mapping being a stand out feature! Being able to only pull down certain directories from a depot and also remap where they are locally, and even have the same file be in multiple places. Makes sharing dependencies across multiple projects really easy, and a lot of complicated tooling around "monorepos" wouldn't need to exist if git supported directory mapping. (You can get 80% of the way there with symlinks but in my experience they eventually break in git when too many different platforms making commits) Also at one point I maintained an obscenely advanced test tool at MS, it pounded through millions of test cases across a slew of CPU architectures, intermingling emulators and physical machines that were connected to dev boxes hosting test code over a network controlled USB switch. (See: https://meanderingthoughts.hashnode.dev/how-microsoft-tested-compilers-circa-2006 https://meanderingthoughts.hashnode.dev/how-microsoft-tested... for more details!) Microsoft had some of the first code coverage tools for C/C++, spun out of a project from Microsoft Research. Their debuggers are still some of the best in the world. NodeJS debugging in 2025 is dog shit compared to C# debugging in 2005.
  - bsder 1y ago
    > git supported directory mapping. Is this a "git" failure or a "Linux filesystems suck" failure? It seems like "Linux fileystems" are starting to creak under several directions (Nix needing binary patching, atomic desktops having poor deduplication, containers being unable to do smart things with home directories or too many overlays). Would Linux simply sucking it up and adopting ZFS solve this or am I missing something?
    - MobiusHorizons 1y ago
azhenley 1y ago
I spent nearly a week of my Microsoft internship in 2016 adding support for Source Depot to the automated code reviewer that I was building (https://austinhenley.com/blog/featurestheywanted.html https://austinhenley.com/blog/featurestheywanted.html) despite having no idea what Source Depot was! Quite a few devs were still using it even then. I wonder if everything has been migrated to git yet.
- sciencesama 1y ago
  Naah still a lot of stuff works on sd !! Those sd commands and setting up sd gives me chills !!
- hacker_homie 1y ago
  Most of the day to day is in git, now.
- PretzelPirate 1y ago
  I miss CodeFlow everyday. It was such a great tool to use.
  - Plasmoid2000ad 1y ago
    CodeFlow lives on and is still held in high regard. It's even made it's way to support the github repos, not just git. https://chromewebstore.google.com/detail/codeflow/aphnoipocoffpdafmiidfmaiadhilelm https://chromewebstore.google.com/detail/codeflow/aphnoipoco... Still buried as internal only though.
3eb7988a1663 1y ago
We communicated the same information through multiple channels: weekly emails, Teams, wiki docs, team presentations, and office hours. The rule: if something was important, people heard it at least 3 times through different mediums. If only this were standard. Last week I received the only notification that a bunch of internal systems were being deleted in two weeks. No scream test, no archiving, just straight deletion. Sucks to be you if you missed the email for any reason.
- MBCook 1y ago
  No kidding. The amount of things that change in important environments without anyone telling people outside their teams in some organizations can be maddening.
- dshacker 1y ago
  Even with this, there were many surprised people. I'm still amazed at all of the people that can ignore everything and just open their IDE and code (and maybe never see teams or email)
  - pvdebbe 1y ago
    In my previous company it came to me as a surprise to learn from a third party that our office had moved lol.
  - sofixa 1y ago
    Alternatively, communications fatigue. How many emails does the average employee get with nonsense that doesn't apply to them? Oh cool, we have a new VP. Oh cool, that department had a charity drive. Oh cool, system I've never heard of is getting replaced by a new one, favourite of this guy I've never heard of. Add in the various spam (be it attacks or just random vendors trying to sell something). At some point, people start to zone out and barely skim, if that, most of their work emails. Same with work chats, which are also more prone to people sharing random memes or photos from their picnic last week or their latest lego set.
    - 1y ago
90s_dev 1y ago
I actually remember using Perforce back in like 2010 or something. And I can't remember why or for which client or employer. I just remember it was stupid.
- dboreham 1y ago
  And expensive.
- broodbucket 1y ago
  There's still a lot of Perforce around. I've thankfully managed to avoid it but I have plenty of friends in the industry who still have to use it.
  - HideousKojima 1y ago
    Perforce is still widely used in the game industry
- gmueckl 1y ago
  Perforce is convoluted and confusing, but I don't think it's really fair to call it stupid. It is still virtually unmatched in a couple of areas.
  - 90s_dev 1y ago
    I wasn't being fair, I was being mean. Perforce is stupid and ugly.
  - bananaboy 1y ago
    I would say it's no more convoluted and confusing than git. I used Perforce professionally for quite a few years in gamedev, and found that a bit confusing at first. Then I was self-employed and used git, and coming to git from Perforce I found it very confusing at first. But then I grew to love it. Now I'm back to working for a big gamedev company and we use Perforce and I feel very proficient in both.
- bob1029 1y ago
90s_dev 1y ago
In about 2010, I briefly had a contract with a security firm with one dev, and there was no source control, and everything written was in low quality PHP. I quit after a week.
- golergka 1y ago
  What kind of security services did they provide? Breaches?
  - layer8 1y ago
    Job security for the dev, probably.
- dshacker 1y ago
  php_final_final_v2.zip shipped to production. A classic. I had a similar experience with https://www.ioncube.com/ https://www.ioncube.com/ php encryption. Everything encrypted and no source control.
israrkhan 1y ago
We did migrate from Perforce to Git for a fairly large repositories, and I can relate to some of the issues. Luckily we did not had to invent VFS, although git-lfs was useful for large files.
carlual 1y ago
> Authenticity mattered more than production value. Thanks for sharing this authentic story! As an ex-MSFT in a relatively small product line that only started switching to Git from SourceDepot in 2015, right before I left, I can truly empathize with how incredible a job you guys have done!
- dshacker 1y ago
  Yeah, it was a whole journey. I can't believe it happened. Thanks for your comment.
  - carlual 1y ago
    Thank you! Btw, it reminds me of the book "Showstopper" about the journey of releasing Windows NT; highly recommended!
    - tux1968 1y ago
      Thanks for the recommendation! I was just about to reread "Soul Of A New Machine", but will try Showstopper instead, since it sounds to be the same genre.
      - zem 1y ago
        tangentially, if you like that genre one of my favourite books in it is "where wizards stay up late", about the development of the internet.
  - hacker_homie 1y ago
    I spent a lot of time coaching people out of source depot, it was touch and go there for a while. It was worth it though thank you for Your effort.
MBCook 1y ago
Could someone explain the ideas of forward integration and reverse integration in Source Depot? I’d never heard of Source Depot before today.
- israrkhan 1y ago
  source depot is (was?) essentially a fork of perforce.
  - MBCook 1y ago
    The article mentioned something along those lines, but I’ve never used it either. I’ve only ever really used CVS, SVN, and Git.
    - int_19h 1y ago
      Perforce is broadly similar to SVN in semantics, and the same branching logic applies to both. Basically if you have the notion of long-lived main branch and feature branches (and possibly an hierarchy in between, e.g. product- or component-specific branches), you need to flow code between them in an organized way. Forward/reverse integration simply describes the direction in which this is done - FI for main -> feature, RI for feature -> main.
- dshacker 1y ago
  RI/FI is similar to having long-lived branches in Git. Imagine you have a "develop-word" branch in git. The admins for that branch would merge all of the changes of their code to "main" and from "main" to their long lived branches. It was a little bit different than long-lived git branches as they also had a file filter (my private branch only had onenote code and it was the "onenote" branch)
  - mikepurvis 1y ago
    I've long wanted a hosted Git service that would help me maintain long lived fork branches. I know there's some necessary manual work that is occasionally required to integrate patches, but the existing tooling that I'm familiar with for this kind of thing is overly focused on Debian packaging (quilt, git-buildpackage) and has horrifyingly poor ergonomics. I'd love a system that would essentially be a source control of my patches, while also allowing a first class view of the upstream source + patches applied, giving me clear controls to see exactly when in the upstream history the breakages were introduced, so that I'm less locking in precise upstream versions that can accept the patches, and more actively engaging with ranges of upstream commits/tags. I can't imagine how such a thing would actually be commercially useful, but darned if would be an obvious fit for AI to automatically examine the upstream and patch history and propose migrations.
BobbyTables2 1y ago
I’d like to know when Microsoft internally migrated away from Visual SourceSafe… They should have recalled it to avoid continued public use…
- dshacker 1y ago
  I didn't even know Microsoft SourceSafe existed.
  - masklinn 1y ago
    Lucky you. Definitely one of the worst tools I’ve had the displeasure of working with. Made worse by people building on top of it for some insane reason.
    - moron4hire 1y ago
      It was at least a little better than CVS, but with SVN available at the same time, never understood the mentality of the offices that I worked at using Source Safe instead of SVN.
      - masklinn 1y ago
        > It was at least a little better than CVS Highly debatable. CVS has a horrendous UI, but didn’t have a tendency to corrupt itself at the drop of a hat and didn’t require locking files to edit them by default (and then require a repository admin to come in and unlock files when a colleague went on holidays with files checked out). Also didn’t require shared write access to an SMB share (one of the reasons it corrupted itself so regularly).
        moron4hire 1y ago
        Uhh, CVS definitely regularly corrupted itself. I nearly lost my senior research thesis in undergrad because of it. The only thing that saved me was understanding professors and the fact that I could drive back home to my parents' house and get back to my review in 30 minutes, where I had a good copy of my code I could put on an Iomega Zip disk from my desktop instead of the corrupted copy we couldn't pull from CVS in the CS lab.
RyJones 1y ago
I was on the team that migrated Microsoft from XNS to TCP/IP - it was way less involved, but similar lessons learned. Migrating from MSMAIL -> Exchange, though - that was rough
- aaronbrethorst 1y ago
  Is that what inspired the "Exchange: The Most Feared and Loathed Team in Microsoft" license plate frames? I'm probably getting a bit of the wording wrong. It's been nearly 20 years since I saw one.
  - RyJones 1y ago
    Probably. A lot of people really loved MSMAIL; not so much Exchange. I have more long, boring stories about projects there, but that’s for another day
    - canucker2016 1y ago
      And sometimes they loved MSMAIL for the weirdest reasons... MSMAIL was designed for Win3.x. Apps didn't have multiple threads. The MSMAIL client app that everyone used would create the email to be sent and store the email file on the system. An invisible app, the Mail Pump, would check for email to be sent and received during idle time (N.B. Other apps could create/send emails via APIs, so you couldn't have the email processing logic in only the MSMAIL client app). So the user could hit the Send button and the email would be moved to the Outbox to be sent. The mail pump wouldn't get a chance to process the outgoing email for a few seconds, so during that small window, if the user decided that they had been too quick to reply, they could retract that outgoing email. Career-limited move averted. Exchange used a client-server architecture for email. Email client would save the email in the outbox and the server would notice the email almost instantly and send it on its way before the user blinked in most cases. A few users complained that Exchange, in essence, was too fast. They couldn't retract a misguided email reply, even if they had reflexes as quick as the Flash.
      - 1y ago
palmotea 1y ago
What's the connection (if any) between "Source Depot" and TFSVC?
- tamlin 1y ago
  Source Depot was based on Perforce. Microsoft bought a license for the Perforce source code and made changes to work at Microsoft scale (Windows, Office). TFS was developed in the Studio team. It was designed to work on Microsoft scale and some teams moved over to it (SQL server). It was also available as a fairly decent product (leagues better than SourceSafe).
- nfg 1y ago
  None that I know of, Source Depot is derived from Perforce.
hulitu 1y ago
> Microsoft Office migration from Source Depot to Git Will they get an annoing window, in the midle of the migration, telling them that Office must be updated now, or the world will end ?
lillecarl 1y ago
This article makes out thousands of engineers that are good enough to qualify at Microsoft and work on Office but haven't used git yet? That sounds a bit overplayed tbh, if you haven't used git you must live under a rock. You can't use Source Depot at home. Overall good story though
- dshacker 1y ago
  You’d be surprised at the amount of people at Microsoft that their entire career have been at Microsoft (pre-git-creation) that never used Git. Git is relatively new (2005) but source control systems are not.
  - shakna 1y ago
    That's still two decades. Git is so popular Microsoft bought one of the major forges 7 years ago. To have never touched it in the last decade? You've got a gap in your CV.
    - stockerta 1y ago
      Not everyone wants to code for hobby, so if their work not uses git then they too will not use it.
      - dkdbejwi383 1y ago
        Not everyone _can_ code as a hobby. Some of us are old and have families and other commitments
        shakna 1y ago
        That's when you can only hope that your workplace is one that trains - so the investment isn't one sided.
      - qingcharles 1y ago
        Agreed. In my professional career, the vast majority of devs I've worked with never wrote a single line of code outside of the office.
AdamN 1y ago
I feel like we're well into the longtail now. Are there other SCM systems or is it the end of history for source control and git is the one and done solution?
- masklinn 1y ago
  Mercurial still has some life to it (excluding Meta’s fork of it), jj is slowly gaining, fossil exists. And afaik P4 still does good business, because DVCS in general and git in particular remain pretty poor at dealing with large binary assets so it’s really not great for e.g. large gamedev. Unity actually purchased PlasticSCM a few years back, and has it as part of their cloud offering. Google uses its own VCS called Piper which they developed when they outgrew P4.
  - zem 1y ago
    google also has a mercurial interface to piper
- linkpuff 1y ago
  There are some other solutions (like jujutsu, which while using git as storage medium, has some differences in the handling of commits). But I do believe we reached a critical point where git is the one stop shop for all the source control needs despite it's flaws/complexity.
- dgellow 1y ago
  Perforce is used in game dev, animation, etc. git is pretty poor at dealing with lots of really large assets
  - qiine 1y ago
    why is this still the case ?
    - rwmj 1y ago
      I've been checking in large (10s to 100s MBs) tarballs into one git repo that I use for managing a website archive for a few years, and it can be made to work but it's very painful. I think there are three main issues: 1. Since it's a distributed VCS, everyone must have a whole copy of the entire repo. But that means anyone cloning the repo or pulling significant commits is going to end up downloading vast amounts of binaries. If you can directly copy the .git dir to the other machine first instead of using git's normal cloning mechanism then it's not as bad, but you're still fundamentally copying everything: $ du -sh .git 55G .git 2. git doesn't "know" that something is a binary (although it seems to in some circumstances), so some common operations try to search them or operate on them in other ways as if they were text. (I just ran git log -S on that repo and git ran out of memory and crashed, on a machine with 64GB of RAM). 3. The cure for this (git lfs) is worse than the disease. LFS is so bad/strange that I stopped using it and went back to putting the tarballs in git.
2d8a875f-39a2-4 1y ago
Always nice to read a new retelling of this old story. TFA throws some shade at how "a single get of the office repo took some hours" then elides the fact that such an operation was practically impossible to do on git at all without creating a new file system (VFS). Perforce let users check out just the parts of a repo that they needed, so I assume most SD users did that instead of getting every app in the Office suite every time. VFS basically closes that gap on git ("VFS for Git only downloads objects as they are needed"). Perforce/SD were great for the time and for the centralised VCS use case, but the world has moved on I guess.
- daemin 1y ago
  Some companies have developed their own technology like VFS for use with Perforce, so you can check out the entire suite of applications but only pull the files when you try to access them in a specific way. This is a lot more important in game development where massive source binary assets are stored along side text files. It uses the same technology that's built into Windows that the remote drive programs (probably) use. Personally I kind of still want some sort of server based VCS which can store your entire companies set of source without needing to keep the entire history locally when you check out something. But unfortunately git is still good enough to use on an ad-hoc basis between machines for me that I don't feel the need to set up a central server and CI/CD pipeline yet. Also being able to stash, stage hunks, and interactively rebase commits are features that I like and work well with the way I work.
  - sixothree 1y ago
    Doesn’t SVN let you check out and commit any folder or file at any depth of a project you choose? Maybe not the checkouts and commit, but that log history for a single subtree is something I miss from the SVN tooling.
    - gilbertbw 1y ago
      Can you not achieve the log history on a subtree with `git log my/subfolder/`? Tools like TortoiseGit let you right click on a folder and view the log of changes to it.
0points 1y ago
Having used vss in the 90s myself, it surprised me it wasn't even mentioned. VSS (Visual SourceSafe) being Microsoft's own source versioning protocol, unlike Source Depot which was licensed from Perforce.
- tamlin 1y ago
  Yes, I used VSS as a solo developer in the 90s. It was a revelation at the time. I met other VCS systems at grad school (RCS, CVS). I started a job at MSFT in 2004 and I recall someone explaining that VSS was unsafe and prone to corruption. No idea if that was true, or just lore, but it wasn't an option for work anyway.
  - mmastrac 1y ago
    We used to call it Visual Source Unsafe because it was corrupting repos all the time.
    - skipkey 1y ago
      As I recall, one problem was you got silent corruption if you ran out of disk space during certain operations, and there were things that took significantly more disk space while in flight than when finished, so you wouldn’t even know. When I was at Microsoft, Source Depot was the nicer of the two version control systems I had to use. The other, Source Library Manager, was much worse.
    - meepmorp 1y ago
      iirc, we called it visual source shred kinda nice to know it wasn't just our experience
  - sumtechguy 1y ago
    The integration with sourcesafe and all of the tools was pretty cool back then. Nothing else really had that level of integration at the time. However, VSS was seriously flakey. It would corrupt randomly for no real reason. Daily backups were always being restored in my workplace. Then they picked PVCS. At least it didnt corrupt itself. I think VSS was fine if you used it on a local machine. If you put it on a network drive things would just flake out. It also got progressively worse as newer versions came out. Nice GUI, very straight forward to teach someone how to use it (checkout file, change, check in like a book), random corruptions about sums up VSS. That checkin/out model seems simpler for people to grasp. The virtual/branch systems most of the other ones use is kind of a mental block for many until they grok it.
ThinkBeat 1y ago
What were the biggest hurdles? Where did Git fall short? How did you structure the repo(s)? Where there many artifacts that went into integration with GitLFS?
airstrike 1y ago
> Today, as I type these words, I work at Snowflake. Snowflake has around ~2,000 engineers. When I was in Office, Office alone was around ~4,000 engineers. I'm sorry, what?! 4,000 engineers doing what, exactly? Excel turns 40 this year and has changed very little in those four decades. I can't imagine you need 4,000 engineers just to keep it backwards compatible. In the meantime we've seen entire companies built with a ragtag team of hungry devs.
- golf1052 1y ago
  The author mentions the list of products that were impacted with this migration > Word, Excel, Powerpoint, Sway, Publisher, Access, Project, OneNote, Shared Services (OSI), Shared UX, + everything in the web
zoobab 1y ago
[flagged]
- ekianjo 1y ago
  You mean like the whole office suite?
throwaway889900 1y ago
Thank goodness I don't have to use IBM's Rational Team Concert anymore. Even just thinking about it makes me shudder.
- mosdl 1y ago
  It was a great tool for losing changes!
deleted 1y ago
[deleted]
danielodievich 1y ago
I want to thank dev leads who trained this green-behind-the-ears engineer on mysteries of Source Depot. Once I understood it, it was quite illuminating. I am glad we only had a dependency on WinCE and IE, and so the clone only took 20 minutes instead of days. I don't remember your names but I remember your willingness to step up and help and onboard new person so they could start being productive. I pay this attitude forward with new hires here in my team no matter where I go.
b0a04gl 1y ago
[dead]
bariumbitmap 1y ago
> In the early 2000s, Microsoft faced a dilemma. Windows was growing enormously complex, with millions of lines of code that needed versioning. Git? Didn’t exist. SVN? Barely crawling out of CVS’s shadow. I wonder if Microsoft ever considered using BitKeeper, a commercial product that began development in 1998 and had its public release in 2000. Maybe centralized systems like Perforce were the norm and a DVCS like BitKeeper was considered strange or unproven?
- wslh 1y ago
  There was SourceSafe (VSS) around that time and TFVC afterwards.
  - bariumbitmap 1y ago
    Neither SourceSafe nor TFVC were distributed version control systems (DVCS) so I'm not sure what you mean.
jeffbee 1y ago
One thing I find annoying about these Perforce hate stories: yes it's awkward to branch in Perforce. It is also the case that there is no need to ever create a branch for feature development when you use Perforce. It's like complaining that it is hard to grate cheese with a trumpet. That just isn't applicable.
- dshacker 1y ago
  I mean yeah, in theory. I just found it really hard to work on multiple changes at a time vs git branches. Back in SD i used to have 2 or 3 machines + multiple VMs to be able to work on multiple things at a time. So I wouldn't say no need. In git I can do work, submit, switch, do work submit switch.
nhatminh2019 1y ago
[flagged]
t1234s 1y ago
With a product like this that spans many decades would the source repo contain all of these versions and the changes over time. For instance word 97 - 2000 - 2003 - 2007, etc..
- DrammBA 1y ago
  I would hope they forked the repo for each new version, to keep the same core but being free to refactor huge parts without affecting previous versions.

366 points by dshacker 1y ago

ksynwa 1y ago

Not doubting it but I don't understand how a shallow clone of OneNote would be 200GB.

paulddraper 1y ago
Must have videos or binaries.
- LtWorf 1y ago
  They probably vendor every single .dll it uses.
  - skrebbel 1y ago
    that's a lot of .dll files!
dshacker 1y ago
Shallow clone of all of office, not onenote.
- ksynwa 1y ago
  Oh alright. Thanks.

mrheosuper 1y ago

[flagged]

sherdil2022 1y ago
If it were that simple, would 100s of engineers spend so much time and effort? They did what they have to and spent the time and energy to maintain some semblance of commit and change history.
- firesteelrain 1y ago
  GP has a valid point. We had a Git repo managed in BitBucket that was gigantic because it contained binary files and the team didn’t know about LFS and storing them in an external tool like Artifactory. So checkouts took forever and even with shallow clones it took forever. With a CI/CD system running constantly and tests needing constant full coverage and hundreds of developers well it eats into developers time. We can’t just prune all the branches well because of compliance rules. So we ended up removing all the binary artifacts before cloning into a new repo then making the old repo as read only. Microsoft seemed to want to mirror everything rather than keep source depot alive. We had another case where we had a subversion system that went out of security compliance that we simply ported to our git systems and abandoned it. So my guess is they wanted everything to look the same and not just importing the code.
- hulitu 1y ago
  > If it were that simple, would 100s of engineers spend so much time and effort? Taking into acvount that they rounded corners in Office, I would say, yes.
senkora 1y ago
> Microsoft had to collaborate with GitHub to invent the Virtual File System for Git (VFS for Git) just to make this migration possible. Without VFS, a fresh clone of the Office repository (a shallow git clone would take 200 GB of disk space) would take days and consume hundreds of gigabytes.
- deleted 1y ago
  [deleted]
- speed_spread 1y ago
  Having had yesterday the dubious pleasure of using MS Word for the first time in a decade, I can safely affirm that they could have have just piped the whole Office repo to the Windows equivalent of /dev/null and nothing of value would have been lost.
  - bigbuppo 1y ago
    The worst part about Word is that it has been feature complete since Office 97, except they've made the UI worse each and every version since then. I wish I could get excited about a new version of Office or WordPerfect, but neither Microsoft or Corel has figured out how to innovate in the past three decades. And no, slapping """AI""" in there isn't the solution. There are so many possibities but they just sort of do nothing with it now that they make a few billion a month on Microsoft 365 subscriptions.
- golergka 1y ago
  It takes less than an hour on my third world apartment wifi to download Call of Duty Modern Warfare remake which is over 200 gygabytes. Since we're not talking about remote work here, I think Microsoft offices and servers (probably on local network) might have managed similar bandwidth back then.
  - TowerTall 1y ago
    There is a lot more to it than that. Check out "The largest Git repo on the planet" by Brian Harry who was in charge of the git migration and Azure DevOps (Microsoft's pendant to GitHub) https://devblogs.microsoft.com/bharry/the-largest-git-repo-on-the-planet/ https://devblogs.microsoft.com/bharry/the-largest-git-repo-o...
kirb 1y ago
Don’t do this on a repository with 35+ years of history! That’s all valuable information you want to keep.
- speed_spread 1y ago
  Anything before Office 2003 you can delete. Anything after Office 2003 you can also delete. There, saved you a few terabytes.
samtheprogram 1y ago
This isn’t Reddit. It’s a lame drive-by joke. The system is working.

smitty1e 1y ago

> We spent months debugging line ending handling "Gosh, that sounds like a right mother," said Unix.

pm90 1y ago

Its oddly fascinating that Microsoft has managed to survive for so long with ancient/bad tools for software engineering. Almost like “life finds a way” but for software dev. From the outside it seems like they are doing better now after embracing OSS/generic dev tools.

com2kid 1y ago
At one point source depot was Toincredibly advanced, and there are still features that it had that git doesn't. Directory mapping being a stand out feature! Being able to only pull down certain directories from a depot and also remap where they are locally, and even have the same file be in multiple places. Makes sharing dependencies across multiple projects really easy, and a lot of complicated tooling around "monorepos" wouldn't need to exist if git supported directory mapping. (You can get 80% of the way there with symlinks but in my experience they eventually break in git when too many different platforms making commits) Also at one point I maintained an obscenely advanced test tool at MS, it pounded through millions of test cases across a slew of CPU architectures, intermingling emulators and physical machines that were connected to dev boxes hosting test code over a network controlled USB switch. (See: https://meanderingthoughts.hashnode.dev/how-microsoft-tested-compilers-circa-2006 https://meanderingthoughts.hashnode.dev/how-microsoft-tested... for more details!) Microsoft had some of the first code coverage tools for C/C++, spun out of a project from Microsoft Research. Their debuggers are still some of the best in the world. NodeJS debugging in 2025 is dog shit compared to C# debugging in 2005.
- bsder 1y ago
  > git supported directory mapping. Is this a "git" failure or a "Linux filesystems suck" failure? It seems like "Linux fileystems" are starting to creak under several directions (Nix needing binary patching, atomic desktops having poor deduplication, containers being unable to do smart things with home directories or too many overlays). Would Linux simply sucking it up and adopting ZFS solve this or am I missing something?
  - MobiusHorizons 1y ago

azhenley 1y ago

I spent nearly a week of my Microsoft internship in 2016 adding support for Source Depot to the automated code reviewer that I was building (https://austinhenley.com/blog/featurestheywanted.html https://austinhenley.com/blog/featurestheywanted.html) despite having no idea what Source Depot was! Quite a few devs were still using it even then. I wonder if everything has been migrated to git yet.

sciencesama 1y ago
Naah still a lot of stuff works on sd !! Those sd commands and setting up sd gives me chills !!
hacker_homie 1y ago
Most of the day to day is in git, now.
PretzelPirate 1y ago
I miss CodeFlow everyday. It was such a great tool to use.
- Plasmoid2000ad 1y ago
  CodeFlow lives on and is still held in high regard. It's even made it's way to support the github repos, not just git. https://chromewebstore.google.com/detail/codeflow/aphnoipocoffpdafmiidfmaiadhilelm https://chromewebstore.google.com/detail/codeflow/aphnoipoco... Still buried as internal only though.

3eb7988a1663 1y ago

We communicated the same information through multiple channels: weekly emails, Teams, wiki docs, team presentations, and office hours. The rule: if something was important, people heard it at least 3 times through different mediums. If only this were standard. Last week I received the only notification that a bunch of internal systems were being deleted in two weeks. No scream test, no archiving, just straight deletion. Sucks to be you if you missed the email for any reason.

MBCook 1y ago
No kidding. The amount of things that change in important environments without anyone telling people outside their teams in some organizations can be maddening.
dshacker 1y ago
Even with this, there were many surprised people. I'm still amazed at all of the people that can ignore everything and just open their IDE and code (and maybe never see teams or email)
- pvdebbe 1y ago
  In my previous company it came to me as a surprise to learn from a third party that our office had moved lol.
- sofixa 1y ago
  Alternatively, communications fatigue. How many emails does the average employee get with nonsense that doesn't apply to them? Oh cool, we have a new VP. Oh cool, that department had a charity drive. Oh cool, system I've never heard of is getting replaced by a new one, favourite of this guy I've never heard of. Add in the various spam (be it attacks or just random vendors trying to sell something). At some point, people start to zone out and barely skim, if that, most of their work emails. Same with work chats, which are also more prone to people sharing random memes or photos from their picnic last week or their latest lego set.
  - 1y ago

90s_dev 1y ago

I actually remember using Perforce back in like 2010 or something. And I can't remember why or for which client or employer. I just remember it was stupid.

dboreham 1y ago
And expensive.
broodbucket 1y ago
There's still a lot of Perforce around. I've thankfully managed to avoid it but I have plenty of friends in the industry who still have to use it.
- HideousKojima 1y ago
  Perforce is still widely used in the game industry
gmueckl 1y ago
Perforce is convoluted and confusing, but I don't think it's really fair to call it stupid. It is still virtually unmatched in a couple of areas.
- 90s_dev 1y ago
  I wasn't being fair, I was being mean. Perforce is stupid and ugly.
- bananaboy 1y ago
  I would say it's no more convoluted and confusing than git. I used Perforce professionally for quite a few years in gamedev, and found that a bit confusing at first. Then I was self-employed and used git, and coming to git from Perforce I found it very confusing at first. But then I grew to love it. Now I'm back to working for a big gamedev company and we use Perforce and I feel very proficient in both.
bob1029 1y ago

90s_dev 1y ago

In about 2010, I briefly had a contract with a security firm with one dev, and there was no source control, and everything written was in low quality PHP. I quit after a week.

golergka 1y ago
What kind of security services did they provide? Breaches?
- layer8 1y ago
  Job security for the dev, probably.
dshacker 1y ago
php_final_final_v2.zip shipped to production. A classic. I had a similar experience with https://www.ioncube.com/ https://www.ioncube.com/ php encryption. Everything encrypted and no source control.

israrkhan 1y ago

We did migrate from Perforce to Git for a fairly large repositories, and I can relate to some of the issues. Luckily we did not had to invent VFS, although git-lfs was useful for large files.

carlual 1y ago

> Authenticity mattered more than production value. Thanks for sharing this authentic story! As an ex-MSFT in a relatively small product line that only started switching to Git from SourceDepot in 2015, right before I left, I can truly empathize with how incredible a job you guys have done!

dshacker 1y ago
Yeah, it was a whole journey. I can't believe it happened. Thanks for your comment.
- carlual 1y ago
  Thank you! Btw, it reminds me of the book "Showstopper" about the journey of releasing Windows NT; highly recommended!
  - tux1968 1y ago
    Thanks for the recommendation! I was just about to reread "Soul Of A New Machine", but will try Showstopper instead, since it sounds to be the same genre.
    - zem 1y ago
      tangentially, if you like that genre one of my favourite books in it is "where wizards stay up late", about the development of the internet.
- hacker_homie 1y ago
  I spent a lot of time coaching people out of source depot, it was touch and go there for a while. It was worth it though thank you for Your effort.

MBCook 1y ago

Could someone explain the ideas of forward integration and reverse integration in Source Depot? I’d never heard of Source Depot before today.

israrkhan 1y ago
source depot is (was?) essentially a fork of perforce.
- MBCook 1y ago
  The article mentioned something along those lines, but I’ve never used it either. I’ve only ever really used CVS, SVN, and Git.
  - int_19h 1y ago
    Perforce is broadly similar to SVN in semantics, and the same branching logic applies to both. Basically if you have the notion of long-lived main branch and feature branches (and possibly an hierarchy in between, e.g. product- or component-specific branches), you need to flow code between them in an organized way. Forward/reverse integration simply describes the direction in which this is done - FI for main -> feature, RI for feature -> main.
dshacker 1y ago
RI/FI is similar to having long-lived branches in Git. Imagine you have a "develop-word" branch in git. The admins for that branch would merge all of the changes of their code to "main" and from "main" to their long lived branches. It was a little bit different than long-lived git branches as they also had a file filter (my private branch only had onenote code and it was the "onenote" branch)
- mikepurvis 1y ago
  I've long wanted a hosted Git service that would help me maintain long lived fork branches. I know there's some necessary manual work that is occasionally required to integrate patches, but the existing tooling that I'm familiar with for this kind of thing is overly focused on Debian packaging (quilt, git-buildpackage) and has horrifyingly poor ergonomics. I'd love a system that would essentially be a source control of my patches, while also allowing a first class view of the upstream source + patches applied, giving me clear controls to see exactly when in the upstream history the breakages were introduced, so that I'm less locking in precise upstream versions that can accept the patches, and more actively engaging with ranges of upstream commits/tags. I can't imagine how such a thing would actually be commercially useful, but darned if would be an obvious fit for AI to automatically examine the upstream and patch history and propose migrations.

BobbyTables2 1y ago

I’d like to know when Microsoft internally migrated away from Visual SourceSafe… They should have recalled it to avoid continued public use…

dshacker 1y ago
I didn't even know Microsoft SourceSafe existed.
- masklinn 1y ago
  Lucky you. Definitely one of the worst tools I’ve had the displeasure of working with. Made worse by people building on top of it for some insane reason.
  - moron4hire 1y ago
    It was at least a little better than CVS, but with SVN available at the same time, never understood the mentality of the offices that I worked at using Source Safe instead of SVN.
    - masklinn 1y ago
      > It was at least a little better than CVS Highly debatable. CVS has a horrendous UI, but didn’t have a tendency to corrupt itself at the drop of a hat and didn’t require locking files to edit them by default (and then require a repository admin to come in and unlock files when a colleague went on holidays with files checked out). Also didn’t require shared write access to an SMB share (one of the reasons it corrupted itself so regularly).
      - moron4hire 1y ago
        Uhh, CVS definitely regularly corrupted itself. I nearly lost my senior research thesis in undergrad because of it. The only thing that saved me was understanding professors and the fact that I could drive back home to my parents' house and get back to my review in 30 minutes, where I had a good copy of my code I could put on an Iomega Zip disk from my desktop instead of the corrupted copy we couldn't pull from CVS in the CS lab.

RyJones 1y ago

I was on the team that migrated Microsoft from XNS to TCP/IP - it was way less involved, but similar lessons learned. Migrating from MSMAIL -> Exchange, though - that was rough

aaronbrethorst 1y ago
Is that what inspired the "Exchange: The Most Feared and Loathed Team in Microsoft" license plate frames? I'm probably getting a bit of the wording wrong. It's been nearly 20 years since I saw one.
- RyJones 1y ago
  Probably. A lot of people really loved MSMAIL; not so much Exchange. I have more long, boring stories about projects there, but that’s for another day
  - canucker2016 1y ago
    And sometimes they loved MSMAIL for the weirdest reasons... MSMAIL was designed for Win3.x. Apps didn't have multiple threads. The MSMAIL client app that everyone used would create the email to be sent and store the email file on the system. An invisible app, the Mail Pump, would check for email to be sent and received during idle time (N.B. Other apps could create/send emails via APIs, so you couldn't have the email processing logic in only the MSMAIL client app). So the user could hit the Send button and the email would be moved to the Outbox to be sent. The mail pump wouldn't get a chance to process the outgoing email for a few seconds, so during that small window, if the user decided that they had been too quick to reply, they could retract that outgoing email. Career-limited move averted. Exchange used a client-server architecture for email. Email client would save the email in the outbox and the server would notice the email almost instantly and send it on its way before the user blinked in most cases. A few users complained that Exchange, in essence, was too fast. They couldn't retract a misguided email reply, even if they had reflexes as quick as the Flash.
    - 1y ago

palmotea 1y ago

What's the connection (if any) between "Source Depot" and TFSVC?

tamlin 1y ago
Source Depot was based on Perforce. Microsoft bought a license for the Perforce source code and made changes to work at Microsoft scale (Windows, Office). TFS was developed in the Studio team. It was designed to work on Microsoft scale and some teams moved over to it (SQL server). It was also available as a fairly decent product (leagues better than SourceSafe).
nfg 1y ago
None that I know of, Source Depot is derived from Perforce.

hulitu 1y ago

> Microsoft Office migration from Source Depot to Git Will they get an annoing window, in the midle of the migration, telling them that Office must be updated now, or the world will end ?

lillecarl 1y ago

This article makes out thousands of engineers that are good enough to qualify at Microsoft and work on Office but haven't used git yet? That sounds a bit overplayed tbh, if you haven't used git you must live under a rock. You can't use Source Depot at home. Overall good story though

dshacker 1y ago
You’d be surprised at the amount of people at Microsoft that their entire career have been at Microsoft (pre-git-creation) that never used Git. Git is relatively new (2005) but source control systems are not.
- shakna 1y ago
  That's still two decades. Git is so popular Microsoft bought one of the major forges 7 years ago. To have never touched it in the last decade? You've got a gap in your CV.
  - stockerta 1y ago
    Not everyone wants to code for hobby, so if their work not uses git then they too will not use it.
    - dkdbejwi383 1y ago
      Not everyone _can_ code as a hobby. Some of us are old and have families and other commitments
      - shakna 1y ago
        That's when you can only hope that your workplace is one that trains - so the investment isn't one sided.
    - qingcharles 1y ago
      Agreed. In my professional career, the vast majority of devs I've worked with never wrote a single line of code outside of the office.

AdamN 1y ago

I feel like we're well into the longtail now. Are there other SCM systems or is it the end of history for source control and git is the one and done solution?

masklinn 1y ago
Mercurial still has some life to it (excluding Meta’s fork of it), jj is slowly gaining, fossil exists. And afaik P4 still does good business, because DVCS in general and git in particular remain pretty poor at dealing with large binary assets so it’s really not great for e.g. large gamedev. Unity actually purchased PlasticSCM a few years back, and has it as part of their cloud offering. Google uses its own VCS called Piper which they developed when they outgrew P4.
- zem 1y ago
  google also has a mercurial interface to piper
linkpuff 1y ago
There are some other solutions (like jujutsu, which while using git as storage medium, has some differences in the handling of commits). But I do believe we reached a critical point where git is the one stop shop for all the source control needs despite it's flaws/complexity.
dgellow 1y ago
Perforce is used in game dev, animation, etc. git is pretty poor at dealing with lots of really large assets
- qiine 1y ago
  why is this still the case ?
  - rwmj 1y ago
    I've been checking in large (10s to 100s MBs) tarballs into one git repo that I use for managing a website archive for a few years, and it can be made to work but it's very painful. I think there are three main issues: 1. Since it's a distributed VCS, everyone must have a whole copy of the entire repo. But that means anyone cloning the repo or pulling significant commits is going to end up downloading vast amounts of binaries. If you can directly copy the .git dir to the other machine first instead of using git's normal cloning mechanism then it's not as bad, but you're still fundamentally copying everything: $ du -sh .git 55G .git 2. git doesn't "know" that something is a binary (although it seems to in some circumstances), so some common operations try to search them or operate on them in other ways as if they were text. (I just ran git log -S on that repo and git ran out of memory and crashed, on a machine with 64GB of RAM). 3. The cure for this (git lfs) is worse than the disease. LFS is so bad/strange that I stopped using it and went back to putting the tarballs in git.

2d8a875f-39a2-4 1y ago

Always nice to read a new retelling of this old story. TFA throws some shade at how "a single get of the office repo took some hours" then elides the fact that such an operation was practically impossible to do on git at all without creating a new file system (VFS). Perforce let users check out just the parts of a repo that they needed, so I assume most SD users did that instead of getting every app in the Office suite every time. VFS basically closes that gap on git ("VFS for Git only downloads objects as they are needed"). Perforce/SD were great for the time and for the centralised VCS use case, but the world has moved on I guess.

daemin 1y ago
Some companies have developed their own technology like VFS for use with Perforce, so you can check out the entire suite of applications but only pull the files when you try to access them in a specific way. This is a lot more important in game development where massive source binary assets are stored along side text files. It uses the same technology that's built into Windows that the remote drive programs (probably) use. Personally I kind of still want some sort of server based VCS which can store your entire companies set of source without needing to keep the entire history locally when you check out something. But unfortunately git is still good enough to use on an ad-hoc basis between machines for me that I don't feel the need to set up a central server and CI/CD pipeline yet. Also being able to stash, stage hunks, and interactively rebase commits are features that I like and work well with the way I work.
- sixothree 1y ago
  Doesn’t SVN let you check out and commit any folder or file at any depth of a project you choose? Maybe not the checkouts and commit, but that log history for a single subtree is something I miss from the SVN tooling.
  - gilbertbw 1y ago
    Can you not achieve the log history on a subtree with `git log my/subfolder/`? Tools like TortoiseGit let you right click on a folder and view the log of changes to it.

0points 1y ago

Having used vss in the 90s myself, it surprised me it wasn't even mentioned. VSS (Visual SourceSafe) being Microsoft's own source versioning protocol, unlike Source Depot which was licensed from Perforce.

tamlin 1y ago
Yes, I used VSS as a solo developer in the 90s. It was a revelation at the time. I met other VCS systems at grad school (RCS, CVS). I started a job at MSFT in 2004 and I recall someone explaining that VSS was unsafe and prone to corruption. No idea if that was true, or just lore, but it wasn't an option for work anyway.
- mmastrac 1y ago
  We used to call it Visual Source Unsafe because it was corrupting repos all the time.
  - skipkey 1y ago
    As I recall, one problem was you got silent corruption if you ran out of disk space during certain operations, and there were things that took significantly more disk space while in flight than when finished, so you wouldn’t even know. When I was at Microsoft, Source Depot was the nicer of the two version control systems I had to use. The other, Source Library Manager, was much worse.
  - meepmorp 1y ago
    iirc, we called it visual source shred kinda nice to know it wasn't just our experience
- sumtechguy 1y ago
  The integration with sourcesafe and all of the tools was pretty cool back then. Nothing else really had that level of integration at the time. However, VSS was seriously flakey. It would corrupt randomly for no real reason. Daily backups were always being restored in my workplace. Then they picked PVCS. At least it didnt corrupt itself. I think VSS was fine if you used it on a local machine. If you put it on a network drive things would just flake out. It also got progressively worse as newer versions came out. Nice GUI, very straight forward to teach someone how to use it (checkout file, change, check in like a book), random corruptions about sums up VSS. That checkin/out model seems simpler for people to grasp. The virtual/branch systems most of the other ones use is kind of a mental block for many until they grok it.

ThinkBeat 1y ago

What were the biggest hurdles? Where did Git fall short? How did you structure the repo(s)? Where there many artifacts that went into integration with GitLFS?

airstrike 1y ago

> Today, as I type these words, I work at Snowflake. Snowflake has around ~2,000 engineers. When I was in Office, Office alone was around ~4,000 engineers. I'm sorry, what?! 4,000 engineers doing what, exactly? Excel turns 40 this year and has changed very little in those four decades. I can't imagine you need 4,000 engineers just to keep it backwards compatible. In the meantime we've seen entire companies built with a ragtag team of hungry devs.

golf1052 1y ago
The author mentions the list of products that were impacted with this migration > Word, Excel, Powerpoint, Sway, Publisher, Access, Project, OneNote, Shared Services (OSI), Shared UX, + everything in the web

zoobab 1y ago

[flagged]

ekianjo 1y ago
You mean like the whole office suite?

throwaway889900 1y ago

Thank goodness I don't have to use IBM's Rational Team Concert anymore. Even just thinking about it makes me shudder.

mosdl 1y ago
It was a great tool for losing changes!

deleted 1y ago

[deleted]

danielodievich 1y ago

I want to thank dev leads who trained this green-behind-the-ears engineer on mysteries of Source Depot. Once I understood it, it was quite illuminating. I am glad we only had a dependency on WinCE and IE, and so the clone only took 20 minutes instead of days. I don't remember your names but I remember your willingness to step up and help and onboard new person so they could start being productive. I pay this attitude forward with new hires here in my team no matter where I go.

b0a04gl 1y ago

[dead]

bariumbitmap 1y ago

> In the early 2000s, Microsoft faced a dilemma. Windows was growing enormously complex, with millions of lines of code that needed versioning. Git? Didn’t exist. SVN? Barely crawling out of CVS’s shadow. I wonder if Microsoft ever considered using BitKeeper, a commercial product that began development in 1998 and had its public release in 2000. Maybe centralized systems like Perforce were the norm and a DVCS like BitKeeper was considered strange or unproven?

wslh 1y ago
There was SourceSafe (VSS) around that time and TFVC afterwards.
- bariumbitmap 1y ago
  Neither SourceSafe nor TFVC were distributed version control systems (DVCS) so I'm not sure what you mean.

jeffbee 1y ago

One thing I find annoying about these Perforce hate stories: yes it's awkward to branch in Perforce. It is also the case that there is no need to ever create a branch for feature development when you use Perforce. It's like complaining that it is hard to grate cheese with a trumpet. That just isn't applicable.

dshacker 1y ago
I mean yeah, in theory. I just found it really hard to work on multiple changes at a time vs git branches. Back in SD i used to have 2 or 3 machines + multiple VMs to be able to work on multiple things at a time. So I wouldn't say no need. In git I can do work, submit, switch, do work submit switch.

nhatminh2019 1y ago

[flagged]

t1234s 1y ago

With a product like this that spans many decades would the source repo contain all of these versions and the changes over time. For instance word 97 - 2000 - 2003 - 2007, etc..

DrammBA 1y ago
I would hope they forked the repo for each new version, to keep the same core but being free to refactor huge parts without affecting previous versions.