6 ms·

Well, the team is small, we're up to three whole devs now. Since 2014 we've been doing a lot of work o help bring our VT implementation up to par (to help enabl

by zadjii 8y ago

Well, the team is small, we're up to three whole devs now. Since 2014 we've been doing a lot of work o help bring our VT implementation up to par (to help enable WSL), as well as implementing new features such as 256/RGB color support, improved accessibility features, and we're even releasing a pty-like API for Windows. Not only that, but since the team was founded we discovered that conhost was not the only console implementation in Windows, though it's the only one available on Desktop. There was a pretty massive undertaking to unify all the separate console implementations on different SKUs under one application, so that all of them would be improved at the same time. It didn't have a terrible lot of impact on the everyday developer, but it was important to help internal engineering efforts.

gmueckl 8y ago
Thank you for explaining! Please don't take my initial post as criticism of your work. There is just this visible disconnect between approximately 4 years of work and comparatively simple features that have been shown off so far. So there must be a story there. I was not aware of Windows having had different console implementations in the past. Can you elaborate on that point? How did that come to pass? Also,I am curious about one other thing: the Windows Console is notorious for having extremely bad perfomance when displaying lots of text. Last time I checked I could only get around 100 lines of text per second on Windows while Linux terminal emulations can show more than 10 times that amount. This may seem petty, but some programs are slowed down by this in practice. Can we expect improvements in that area as well?
- lozenge 8y ago
  As I understand it Windows 3 ran in DOS, Windows 98 had DOS integrated, Windows NT had it emulated and ME was somewhere in between. It wouldn't surprise me if parts from all four have survived til today for compatibility.
  - chx 8y ago
    ME wasn't between the least. It was DOS based. There is no inbetween. It's either IO.SYS or NTOSKRNL.EXE. It made using the real mode MS DOS even worse by shipping a special MS DOS which skipped CONFIG.SYS and AUTOEXEC.BAT, hidden the files and removed DOS from the boot menu. To fix this, check http://retrosystemsrevival.blogspot.com/2018/06/dos-for-windows-me.html http://retrosystemsrevival.blogspot.com/2018/06/dos-for-wind... this -- quite new blog post, just last month, for something this old.
    - bitcrazed 8y ago
      Actually, no - Windows ME did not depend on MS-DOS. For all the bad rap it got (some of which was deserved), ME bypassed real-mode MS-DOS (and its config files) at bootup, and loaded its own protected-mode drivers, etc. in IO.SYS. It also loaded the main registry hive only once, and parallelized PnP resolution, significantly improving boot-up time. It also incorporated Windows 2000's networking stack, and added support for UPnP. In many ways, ME was a stepping stone, getting some users on less-capable hardware onto Windows 2000 class OS features, without requiring a complete hardware upgrade.
      - chx 8y ago
        Erm, nope. IFSMGR.VXD depends on IFSHLP.SYS still. Everyone's favorite error message: the Microsoft Installable File System Manager cannot find the helper driver. Please ensure that IFSHLP.SYS has been installed. This means Windows ME needs a real mode DOS helper for certain file system operations. I can't remember whether the INT 21h function 55h (Create PSP) was still called as frequently as in earlier Windows and handled in DOS but I betcha.
        bitcrazed 8y ago
        IFSHLP.SYS was originally created during the implementation of OS/2 1.2, and later released in Windows 3.11 running on 32-bit 386 Enhanced Mode. Its job was to provide 32-bit file access (bypassing 16-bit DOS file IO mechanisms), and ensuring nothing else on the system could intercept INT 21h calls. ME didn't depend on MS-DOS (though it did provide support for running MS-DOS apps).
  - TazeTSchnitzel 8y ago
    The 9x series and ME use DOS during the boot process but not once booted, except for legacy disk access (if there's no 32-bit Win9x disk driver). Microsoft went to huge pains to make Win9x act as if it was an application on top of DOS for compatibility reasons, but it very much isn't, it's a full-blown OS. This is almost true of Windows 3.x too or Windows/286 and Windows/386.
  - JdeBP 8y ago
    These should help you and others. They cover some of the errors in the above, as well as the change to the console implementation in Windows NT 6.1. * https://superuser.com/a/319187/38062 https://superuser.com/a/319187/38062 * http://jdebp.info./FGA/a-command-interpreter-is-not-a-console.html http://jdebp.info./FGA/a-command-interpreter-is-not-a-consol... * http://jdebp.info./FGA/tui-console-and-terminal-paradigms.html http://jdebp.info./FGA/tui-console-and-terminal-paradigms.ht...
- JdeBP 8y ago
  Linux terminal emulators are by no means uniform about this. Scrolling speed, and whether to not re-render for every character, are points of contention amongst the XTerm, Alacritty, and VTE people.
  - cryptonector 8y ago
    I just set the scrollback history to zero and run tmux. Scrollback speed problem solved.
- zadjii 8y ago
  Oh I certainly don't take it as such. It can be extra frustrating as both a developer and a user of the console to want to be able to improve everything, but know that there is only so much time to get features done, and rarely do usability features rise in priority above supporting other teams. > I was not aware of Windows having had different console implementations in the past. Can you elaborate on that point? How did that come to pass? It's not really something most developers outside the company would have even been aware of honestly. At different points in time, there have been various different teams across Microsoft who had been working on bringing up new pieces of hardware (Xbox, HoloLens, phones, etc) and they found that the existing console didn't work on those minimal Windows OS's. At the time, there was no Console team at all, so whoever was working on that device just implemented their own console that would work on that prototypical device. That prototypical console got re-used and extended for other devices as they were developed, then abandoned as the platforms matured. Then, in 2014, when the Console team was formed, we found out that not only were we responsible for the desktop console, but these other implementations as well. > Can we expect improvements in that area as well? Stay tuned :)
Karliss 8y ago
Pty-like API sounds great. I assume you are referring to what's discussed here https://github.com/Microsoft/console/issues/57 https://github.com/Microsoft/console/issues/57 .
- zadjii 8y ago
  Yep that's it!
komali2 8y ago
Are you able to speak at all to the labor assignment? Microsoft is big, why only put 3 engineers on this project?
- threeseed 8y ago
  This is bizarre to me. You simply can not attract developers to your platform without a first rate console experience. Why even bother with Powershell, Linux Subsystem etc without getting this crucial component working perfectly ?
  - izacus 8y ago
    Windows is the biggest, most popular development platform in the world. Are you not even slightly aware of the bias and bubble you live in?!
    - Rjevski 8y ago
      Which languages? Maybe for desktop app development (but that itself is quite a niche market now). I have yet to see a web developer use Windows. All I see are Macs and the occasional Linux.
      - oblio 8y ago
        Location is everything. Outside of the US Macs aren't as popular and enterprise or in-house web development rarely happens on Macs or God-forbid!* Linux. * I'm kidding, but talk to your average IT department head about Linux desktops :)
      - erikbye 8y ago
        Have you heard of C#? ASP.NET? A few of those devs use Windows...
        Rjevski 8y ago
        Are those still used for new projects?
slededit 8y ago
He can't say it publicly but the code behind the console is crap. Not just from a modern perspective looking back, the original devs hated it too. The book Show Stopper! Has some colorful words about it. And since it's so core to the system and deeply intertwined you can't just go in and start arbitrarily changing things. Everyone has been too scared to touch it for the last 20 years. This is painstaking careful work. I don't think throwing more people at it would really help too much. My hat is off to you and your team for finally fixing this.
- zadjii 8y ago
  Oh no, I can say that quite publicly. When I started on the team in 2015, that code was some nightmarish shit. It was hardly componentized, you'd have 70 different places in the code all calling out to GDI to do rendering at seemingly random points in the app's lifecycle. It was painful to get work done in the codebase. After a LOT of refactoring it's finally in a state that's really quite managable, and much more modernized.
  - slededit 8y ago
    I hope you updated all that to D2D. I was always deeply offended you forced office to move with Universal but had the gall to link GDI in and keep using it for your own stuff :).
    - zadjii 8y ago
      That part comes next ;) Getting all of the rendering calls isolated to one component was a pretty big task in itself, and just isolating and de-duplicating them improved our rendering perf by something like 30%
      - westajay 8y ago
        Your efforts are noticeable and appreciated! It was always a mystery to me why the Windows console was stuck in time from the NT days, and your work sends a signal to all those doing sysadmin and programming work that Microsoft is supporting these use cases in Windows.
russellbeattie 8y ago
I don't use Windows mostly because the console experience is so bad. I'm sure there are many others like me. Your superiors should realize they have lost generations of developers because of this and dedicate the resources needed to fix it. How they haven't realized this yet, astounds me.
- atr2 8y ago
  Use mintty (or wsltty for WSL). It's slow but it's good. The Windows console really is rubbish, even after all these updates; don't bother with it.
  - erikbye 8y ago
    Mintty + tmux. It's how I survive Windows. Also: https://github.com/mintty/wsltty https://github.com/mintty/wsltty
- bitcrazed 8y ago
  As a matter of interest, have you tried Windows 10's Console?
erikbye 8y ago
Could you fix proper emitting of tab escape codes so completion in REPLs, etc., work properly?
icc97 8y ago
Please ignore the parent you replied to. The changes you've put into the console are so fundamentally important - and brilliant. Being able to snap a window and have the code flow properly and fill the window completely is great. In the same way that the Linux desktop has fast caught up with the Windows desktop, the Windows terminal is fast catching up with the Linux terminal. The stuff your changing most people will never notice, but you've done more for the console in 3 years than was done in the previous 20 as far as I can tell. It's getting very close to where I can actually use Vim full time at a DOS prompt - it's actually waiting for Vim to catch up with the 24-bit colour changes that you made [0]. Plus there's the nicety of bold/italic fonts but I'd imagine that might not occur for a while. [0]: https://github.com/vim/vim/issues/1270 https://github.com/vim/vim/issues/1270
mschaef 8y ago
Thanks for your work on this... this is going to make a big difference for lots of folks. (Myself included.)

by zadjii 8y ago

gmueckl 8y ago

Thank you for explaining! Please don't take my initial post as criticism of your work. There is just this visible disconnect between approximately 4 years of work and comparatively simple features that have been shown off so far. So there must be a story there. I was not aware of Windows having had different console implementations in the past. Can you elaborate on that point? How did that come to pass? Also,I am curious about one other thing: the Windows Console is notorious for having extremely bad perfomance when displaying lots of text. Last time I checked I could only get around 100 lines of text per second on Windows while Linux terminal emulations can show more than 10 times that amount. This may seem petty, but some programs are slowed down by this in practice. Can we expect improvements in that area as well?

lozenge 8y ago
As I understand it Windows 3 ran in DOS, Windows 98 had DOS integrated, Windows NT had it emulated and ME was somewhere in between. It wouldn't surprise me if parts from all four have survived til today for compatibility.
- chx 8y ago
  ME wasn't between the least. It was DOS based. There is no inbetween. It's either IO.SYS or NTOSKRNL.EXE. It made using the real mode MS DOS even worse by shipping a special MS DOS which skipped CONFIG.SYS and AUTOEXEC.BAT, hidden the files and removed DOS from the boot menu. To fix this, check http://retrosystemsrevival.blogspot.com/2018/06/dos-for-windows-me.html http://retrosystemsrevival.blogspot.com/2018/06/dos-for-wind... this -- quite new blog post, just last month, for something this old.
  - bitcrazed 8y ago
    Actually, no - Windows ME did not depend on MS-DOS. For all the bad rap it got (some of which was deserved), ME bypassed real-mode MS-DOS (and its config files) at bootup, and loaded its own protected-mode drivers, etc. in IO.SYS. It also loaded the main registry hive only once, and parallelized PnP resolution, significantly improving boot-up time. It also incorporated Windows 2000's networking stack, and added support for UPnP. In many ways, ME was a stepping stone, getting some users on less-capable hardware onto Windows 2000 class OS features, without requiring a complete hardware upgrade.
    - chx 8y ago
      Erm, nope. IFSMGR.VXD depends on IFSHLP.SYS still. Everyone's favorite error message: the Microsoft Installable File System Manager cannot find the helper driver. Please ensure that IFSHLP.SYS has been installed. This means Windows ME needs a real mode DOS helper for certain file system operations. I can't remember whether the INT 21h function 55h (Create PSP) was still called as frequently as in earlier Windows and handled in DOS but I betcha.
      - bitcrazed 8y ago
        IFSHLP.SYS was originally created during the implementation of OS/2 1.2, and later released in Windows 3.11 running on 32-bit 386 Enhanced Mode. Its job was to provide 32-bit file access (bypassing 16-bit DOS file IO mechanisms), and ensuring nothing else on the system could intercept INT 21h calls. ME didn't depend on MS-DOS (though it did provide support for running MS-DOS apps).
- TazeTSchnitzel 8y ago
  The 9x series and ME use DOS during the boot process but not once booted, except for legacy disk access (if there's no 32-bit Win9x disk driver). Microsoft went to huge pains to make Win9x act as if it was an application on top of DOS for compatibility reasons, but it very much isn't, it's a full-blown OS. This is almost true of Windows 3.x too or Windows/286 and Windows/386.
- JdeBP 8y ago
  These should help you and others. They cover some of the errors in the above, as well as the change to the console implementation in Windows NT 6.1. * https://superuser.com/a/319187/38062 https://superuser.com/a/319187/38062 * http://jdebp.info./FGA/a-command-interpreter-is-not-a-console.html http://jdebp.info./FGA/a-command-interpreter-is-not-a-consol... * http://jdebp.info./FGA/tui-console-and-terminal-paradigms.html http://jdebp.info./FGA/tui-console-and-terminal-paradigms.ht...
JdeBP 8y ago
Linux terminal emulators are by no means uniform about this. Scrolling speed, and whether to not re-render for every character, are points of contention amongst the XTerm, Alacritty, and VTE people.
- cryptonector 8y ago
  I just set the scrollback history to zero and run tmux. Scrollback speed problem solved.
zadjii 8y ago
Oh I certainly don't take it as such. It can be extra frustrating as both a developer and a user of the console to want to be able to improve everything, but know that there is only so much time to get features done, and rarely do usability features rise in priority above supporting other teams. > I was not aware of Windows having had different console implementations in the past. Can you elaborate on that point? How did that come to pass? It's not really something most developers outside the company would have even been aware of honestly. At different points in time, there have been various different teams across Microsoft who had been working on bringing up new pieces of hardware (Xbox, HoloLens, phones, etc) and they found that the existing console didn't work on those minimal Windows OS's. At the time, there was no Console team at all, so whoever was working on that device just implemented their own console that would work on that prototypical device. That prototypical console got re-used and extended for other devices as they were developed, then abandoned as the platforms matured. Then, in 2014, when the Console team was formed, we found out that not only were we responsible for the desktop console, but these other implementations as well. > Can we expect improvements in that area as well? Stay tuned :)

Karliss 8y ago

Pty-like API sounds great. I assume you are referring to what's discussed here https://github.com/Microsoft/console/issues/57 https://github.com/Microsoft/console/issues/57 .

zadjii 8y ago
Yep that's it!

komali2 8y ago

Are you able to speak at all to the labor assignment? Microsoft is big, why only put 3 engineers on this project?

threeseed 8y ago
This is bizarre to me. You simply can not attract developers to your platform without a first rate console experience. Why even bother with Powershell, Linux Subsystem etc without getting this crucial component working perfectly ?
- izacus 8y ago
  Windows is the biggest, most popular development platform in the world. Are you not even slightly aware of the bias and bubble you live in?!
  - Rjevski 8y ago
    Which languages? Maybe for desktop app development (but that itself is quite a niche market now). I have yet to see a web developer use Windows. All I see are Macs and the occasional Linux.
    - oblio 8y ago
      Location is everything. Outside of the US Macs aren't as popular and enterprise or in-house web development rarely happens on Macs or God-forbid!* Linux. * I'm kidding, but talk to your average IT department head about Linux desktops :)
    - erikbye 8y ago
      Have you heard of C#? ASP.NET? A few of those devs use Windows...
      - Rjevski 8y ago
        Are those still used for new projects?

slededit 8y ago

He can't say it publicly but the code behind the console is crap. Not just from a modern perspective looking back, the original devs hated it too. The book Show Stopper! Has some colorful words about it. And since it's so core to the system and deeply intertwined you can't just go in and start arbitrarily changing things. Everyone has been too scared to touch it for the last 20 years. This is painstaking careful work. I don't think throwing more people at it would really help too much. My hat is off to you and your team for finally fixing this.

zadjii 8y ago
Oh no, I can say that quite publicly. When I started on the team in 2015, that code was some nightmarish shit. It was hardly componentized, you'd have 70 different places in the code all calling out to GDI to do rendering at seemingly random points in the app's lifecycle. It was painful to get work done in the codebase. After a LOT of refactoring it's finally in a state that's really quite managable, and much more modernized.
- slededit 8y ago
  I hope you updated all that to D2D. I was always deeply offended you forced office to move with Universal but had the gall to link GDI in and keep using it for your own stuff :).
  - zadjii 8y ago
    That part comes next ;) Getting all of the rendering calls isolated to one component was a pretty big task in itself, and just isolating and de-duplicating them improved our rendering perf by something like 30%
    - westajay 8y ago
      Your efforts are noticeable and appreciated! It was always a mystery to me why the Windows console was stuck in time from the NT days, and your work sends a signal to all those doing sysadmin and programming work that Microsoft is supporting these use cases in Windows.

russellbeattie 8y ago

I don't use Windows mostly because the console experience is so bad. I'm sure there are many others like me. Your superiors should realize they have lost generations of developers because of this and dedicate the resources needed to fix it. How they haven't realized this yet, astounds me.

atr2 8y ago
Use mintty (or wsltty for WSL). It's slow but it's good. The Windows console really is rubbish, even after all these updates; don't bother with it.
- erikbye 8y ago
  Mintty + tmux. It's how I survive Windows. Also: https://github.com/mintty/wsltty https://github.com/mintty/wsltty
bitcrazed 8y ago
As a matter of interest, have you tried Windows 10's Console?

erikbye 8y ago

Could you fix proper emitting of tab escape codes so completion in REPLs, etc., work properly?

icc97 8y ago

Please ignore the parent you replied to. The changes you've put into the console are so fundamentally important - and brilliant. Being able to snap a window and have the code flow properly and fill the window completely is great. In the same way that the Linux desktop has fast caught up with the Windows desktop, the Windows terminal is fast catching up with the Linux terminal. The stuff your changing most people will never notice, but you've done more for the console in 3 years than was done in the previous 20 as far as I can tell. It's getting very close to where I can actually use Vim full time at a DOS prompt - it's actually waiting for Vim to catch up with the 24-bit colour changes that you made [0]. Plus there's the nicety of bold/italic fonts but I'd imagine that might not occur for a while. [0]: https://github.com/vim/vim/issues/1270 https://github.com/vim/vim/issues/1270

mschaef 8y ago

Thanks for your work on this... this is going to make a big difference for lots of folks. (Myself included.)